Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
26,218 characters · 8 sections · 28 citation commands
Optimizing Data-driven Weights In Multidimensional Indexes
\rule{\textwidth}{.5pt}
\rule{\textwidth}{.5pt}
Social sciences, including economics and statistics, have recognized the importance of summary and synthetic measures that go beyond the representation of single variables or dimensions. Multidimensional indexes offer the opportunity to represent complex concepts such as well-being, happiness, or social capital with summary indicators. Examples of multidimensional indicators are Amartya Sen's indexes of well-being (Sen1980, Sen1985), the United Nations' Human Development Indexes, and the OECD's Better Life Index OECD2015.
Despite their popularity, there is no consensus on how to aggregate dimensions and how weights should be attributed to each dimension. This has driven some organizations to adopt a “dashboard” approach where dimensions are monitored simultaneously, but separately. Examples of this approach are the OECD's Better Life Index OECD2015, the Italian Equitable and Sustainable Well-being index BES2014, and the United Kingdom's National Well-being Measure UK2015. This approach highlights changes over time in each dimension, avoiding loss of information, but falls short of a parsimonious representation of well-being.
A more popular approach is to use a single composite weighted index. Examples are the Human Development Index (HDI), the Canadian Index of well-being Canada2012, the Happy Planet Index NEF2013, and various multi-dimensional poverty or inequality indices used extensively by economists and statisticians (Alkire2011; Aaberge2015; Bosmans2015). The main drawbacks of this approach are the loss of information due to the extreme synthesis, and the arbitrariness of the choice of weights (particularly when weights are purely normative such as those based on researchers' or surveys respondents' opinions), and aggregation methods (OECD2008, Decancq2013, Chiara2025, BELHADJ2012304).
An additional shortcoming of multidimensional analyses shared by the dashboard and index methods is that dimensions are assumed to be independent (orthogonal). This is a rather unrealistic assumption as eloquently argued by the Fitoussi commission when discussing the quality of life index (StiglitzSenFitoussi2010): “it is critical to address questions about how developments in one domain of quality of life affect other domains” (p.59) and “when designing policies in specific fields, impacts on indicators pertaining to different quality-of-life dimensions should be considered jointly, to address the interactions between dimensions” (p.16). These statements highlight the importance of the complex correlation structure among dimensions and also the relevance of the direction of association between dimensions and the overall structure of causality.
This paper addresses these questions by arguing that data-driven weights in multidimensional indexes should be derived from models that meet a set of desirable properties designed to address the questions of orthogonality of dimensions, loss of information, and causality. After specifying these properties, the paper reviews statistical, econometric, and machine learning models to assess which model meets these properties. It finds that Bayesian Networks is the only model that satisfies all the desirable properties identified. An example shows how this model offers a practical approach to weighting and policy making with multidimensional indexes.
When treating multidimensional concepts, scholars are faced with stark choices: ignoring the question of weights and monitor contributing dimensions separately (the “dashboard” approach), attribute equal weights, attribute different weights to different dimensions based on some normative criteria, or attribute weights based on some positive data-driven criteria. If one is searching for a single index with no priors on the importance of dimensions, the latter approach is understandably preferable, but there is no consensus on how to identify the optimal weighting model. In fact, there is hardly any discussion in the literature on what properties these data-driven models should meet.
A possible and more rigorous approach is to state first the properties that models designed to study associations across variables should meet to be used for weighting dimensions in multidimensional indexes. Based on the conclusions of the Fitoussi commission, the shortcomings identified in the literature, and the existing features of popular models used in statistics, econometrics and machine learning, we argue that a plausible data-driven weighting model should meet the following criteria: 1) Measure the correlation between the outcome variable of interest and each of the dimensions included into the index; 2) Measure the correlation across the dimensions that constitutes the index; 3) Identify the direction of correlation among all variables and the structure of causality. We add that other desirable properties are 4) Being able to identify a latent variable when the outcome variable is not easily measured; 5) Be based on probabilistic rather than deterministic criteria to reduce the set of normative choices needed for identification; and 6) Be suitable for deriving relative weights from estimated coefficients. A model that satisfies these criteria, we argue, addresses the combination of issues that plague weighting in data-driven multidimensional indexes.
The range of potential weighting models to consider is vast. In an effort to simplifying the analysis, and considering that multidimensional indexes typically provide a snapshot of the outcome considered, we excluded time models such as Granger, Markov, Panel, or Dynamic Bayesian Networks.\footnote{Note that longitudinal studies with multidimensional indexes exist (see for example ZHANG2021109912) but are few.} Otherwise, any model that studies cross-section associations among two or more variables can be considered a potential data-driven weighting model. It is important therefore to identify first a comprehensive list of models used across the social sciences to study association across variables, a rather ambitious task.
To address this question, we sought the help of ChatGPT and Deepseek. We first compiled a list of popular models used to study association across sets of variables such as Spearman, OLS, Logit, Principal Component Analysis, Random Forest, and others. This initial list was forcibly limited by the authors' collective knowledge. We then asked both GPTs to complement this list with other popular models used across the social sciences. Next, we compared and assessed the lists provided, redacted a single list and submitted the list back to GPTs until authors and GPTs reached a consensus on a final list. We followed the same strategy to assess which model met which property by redacting an initial classification and ask both GPTs to comment until authors and GPTs reached an agreement on a final classification.
Figure (ref) summarizes results keeping models that met at least one of the desirable properties. The chart reports the number of properties that each model meets sorted in descending order. It shows that Bayesian Networks is the only model that satisfies all the six properties identified. Other models such as Structural Equation Modelling, Causal Forest and Neural Networks come close with five out of six properties satisfied whereas there are models that do not meet any property (not shown in chart), and others that meet only few. Also notable is the fact that some of the models that study associations well are not very suitable for constructing weights. Table A1 in Annex qualifies the findings and groups models into categories (Correlation, Regression, Machine Learning, Graphical Networks, Dimensionality Reduction, Latent Variable, Weighting, Diagnostic).\footnote{Note that several models would fit different groups. This is a coarse classification that is useful to see how classes of models behave. It is also useful to compare a reduced set of models that are representative of the main classes as shown further in the paper.} As shown by the Table, some of the findings may be disputable as several models meet some of the properties only under certain specifications. However, Bayesian Networks is the only model that consistently showed to satisfy all the desirable properties identified. Even if we removed the probabilistic condition, which understandably reduces the range of models that can be considered, Bayesian Networks would still sit on top of the classification. The only other model that can potentially meet all six conditions is the Structural Equation Model. However, SEM can be used for causal inference only under strict assumptions, and is computationally very challenging as compared to BNs.
We provide an example of multidimensional index using an index of well-being based on the 2013 wave of EU-SILC, the European Union Statistics on Income and Living Conditions. We consider life satisfaction as our target (outcome) variable and include most of the key well-being dimensions suggested by the literature for these types of indexes including: material living standards, (ii) health, (iii) education; (iv) personal activities and work, (v) political voice and governance, (vi) \emph{social connections and relationships}, (vii) \emph{security of physical as well as economic nature}, (iix) \emph{environment}, and also \emph{age}, \emph{gender} and \emph{country} as control variables. The unit of analysis is the household head. We include in the analysis all countries belonging to the European Union, with the exception of Czech Republic, Denmark and Slovenia, because they lack information on one of the dimensions of social connections and relationships (variable pd050) (See Liberati_2023 and Bossert2013 for papers that use EU-SILC data for multidimensional indexes of well-being. See also Ceriani2020 for a first application of Bayesian Networks to weighting in multidimensional indexes of well-being).
We estimate a set of Bayesian Networks' structures using eleven different algorithms, including constraint-based learning algorithms, scored-based learning algorithms, and hybrid algorithms (we use the R package bnlearn conceived by Scutari2010). We have also forced a set of arcs not to be included in the graph, by means of a blacklist, in order to avoid the control variables age and household size to be considered dependent of the well-being dimensions. Also, we have excluded the directed arcs from status in employment (WORK) and poverty (M_POOR) to education (EDU) and from satisfaction with life (SA_LIFE) to health (HEALTH). Moreover, we have whitelisted the directed arc from EDU to WORK, as our prior is that level of education should affect status in employment.
Table (ref) reports the arcs' occurrence in each of the BN obtained by implementing the 11 algorithms. Following CugnataKenettSalini2016, we assign a weight of 1 to arcs linking pairs of nodes directly and a weight of 0.5 if the connection is indirect. The last column in each table shows the sum of weights assigned to each pair of nodes, which can vary between 0 and 11. For the purpose of this paper, we define robust a BN which contains the largest set of arcs scoring 6 or more, meaning that the majority of algorithms find an occurrence.
The network resulting from applying the Tabu-aic algorithm in Figure (ref) contains the highest number of arcs among all networks produced by the 11 different algorithms applied. The numbers shown on the arcs represent the number of algorithms confirming the directional connections. See, for example, age to EDU and EDU to WORK, which correspond to the first two rows of Table (ref). The arcs that appear in at least 6 of the 11 algorithms used in the analysis are depicted in red. Variables of interest (non-control variables) are in capital letters. The numbers on the arcs represent the number of algorithms confirming the directional connection. In red the arcs contained in more than 5 algorithms.
Of the different well-being dimensions taken into account, only education (EDU), personal activities and work (WORK), material living standards (M_MD and M_\allowbreak POOR), economic security (S_ECON) and health status (HEALTH) influence the target variable (satisfaction with life, SA_LIFE). On the other hand, political and social participation (SOC and POL), satisfaction with the environment (NATURE) and physical security (S_PHYS) appear to be consequences of the level of life satisfaction (SA_LIFE). These findings are sensible but recall that researchers may whitelist or blacklist variables based on prior knowledge or common sense. In other words, researchers can impose as much structure as they like on the BN following the initial data-driven analysis.
The information on the strength of the relationship between nodes can be naturally translated in the weighting structure of a multidimensional index of well-being. In this new system of weights, dimensions that have a larger impact on the target node when affected by a policy-induced change (e.g. have more influence on the target node) receive a higher weight.
Let us assume that $T$ is the target node (satisfaction with life). For each $i-$th dimension of well-being, the weight assigned to dimension $X_i$ is positively correlated with the strength of the arcs connecting $X_i$ and $T$ and negatively correlated to the length of the path linking $X_i$ with $T$. Formally:
where $P_i(X_i, T)$ is the set of paths in the BN joining nodes $X_i$ and $T$, $|p_i|$ is the length of each path and $\sigma_i$ is the strength associated to each path. A path is defined as the set of arcs connecting each node $X_i$ to the target node $T$. The strength of each path corresponds to the product of the strengths of each arc in the path (which are bounded between 0--no influence between two nodes and 1--maximum influence). This definition of weight corresponds to the distance-weighted influence in Albrecht2014 and CugnataKenettSalini2016.
The proposed definition of weights represents an improvement on the classical data-driven and normative approaches to weights found in the literature (see Decancq2013 for a discussion). As for data-driven weights, BN weights consider the distribution of achievements in society and, as in the normative approach, they attribute more weight to policy relevant variables. Unlike these approaches, BN weights include data-driven information on inter-dependencies among variables and can be used for ex-ante prognostic (forecasting) and ex-post diagnostic (evaluation) policy purposes. By providing this extra information, they also partly address the criticism of loss of information attributed to the multidimensional index approach.
As a further illustration, we compare the resulting rankings of selected weighting schemes representing the main classes of models including: (i) Spearman Correlation; (ii) Linear Regression; (iii) Random Forest; and iv) Bayesian network. As benchmarks, we also use v) Equal Weighting; and vi) Self-reported opinions from Eurobarometer. Such comparison is parsimonious enough to allow for a visual assessment while comparing different classes of weighting schemes.
As multivariate correlation-based method, we use the Spearman rank correlation coefficient, which seems the most appropriate given the categorical nature of our underlying data (see Banerjee_2018 for a discussion on deriving multidimensional weights using multidimensional coefficient of variation). Each dimension weight is defined as the absolute value of the correlations coefficient ($\rho$) between the dimension and the target variable $t$, plus half the value of each other pairwise $\rho$'s:
Weights are then normalized dividing by $\sum_{i \neq t} w_i$. Notably, this method offers an improvement over linear estimation by capturing pairwise correlations among independent variables; however, it remains limited in its ability to reveal the broader network of interrelationships across dimensions.
As machine learning method, we apply Random Forest regression algorithms, and we use as weights the predicted importance associated to each independent variable, $\iota_i$, normalized by the sum of all importance values associated to each dimension $i-$th, $i=1, \dots, m$:
For the opinion-based weights, we use data from the Eurobarometer survey, wave 86.3 fielded between November and December 2016. The weight are represented by the share of the population expressing strong concern with respect to selected issues, following Guio2009 and Bossert2013. Table (ref) summarizes the variables used to capture the relative importance that individuals attach to each dimension.
The application of multiple weighting methods reveals substantial variation in how well-being dimensions are prioritized (see Table (ref)). From a policy perspective, the equal weighting approach offers limited analytically utility, as it fails to account for the varying influence of each dimension. Subjective weights highlight Satisfaction with One’s Economic Situation, Social Participation, Health, and Satisfaction with Work as the most salient contributors to overall life satisfaction, reflecting individual or cultural perceptions of well-being. In contrast, data-driven methods such as OLS regression, Random Forest, and Spearman correlation produce more balanced distributions, yet consistently assign greater importance to Health, Material Deprivation, and Economic Satisfaction. Among these, Random Forest produces a particularly even distribution of weights, assigning relatively lower importance to Poverty Status, Economic Satisfaction, and Physical Security, while giving slightly more weight to Material Deprivation, Environmental Satisfaction, and Satisfaction with Work.
Bayesian Networks, on the other hand, allocate nearly all the weight to Health and Material Deprivation. This concentration reflects the model’s ability to account for both direct and indirect effects, revealing that these two dimensions exert the greatest overall influence on life satisfaction. Their centrality arises not only from their direct impact but also from their role in shaping other dimensions within the well-being framework. This capacity to capture indirect pathways is a key advantage of the Bayesian Network approach over alternative methods. Specifically, it offers three critical benefits: (i) it incorporates indirect effects, providing a more comprehensive understanding of causal influence; (ii) it narrows the range of policy actions required to improve outcomes; and (iii) it enhances the cost-effectiveness of interventions by leveraging the amplifying potential of indirect relationships.
Figure 3 clearly demonstrates that the choice of weighting method is not neutral: it can significantly shift a country's rank in a composite well-being index. While data-\allowbreak driven methods tend to offer more consistent patterns, subjective or structure-\allowbreak based approaches (like Eurobarometer and Bayesian Networks) can produce markedly different outcomes, especially for countries in the middle of the distribution. In particular, the large changes observed in the ranks of Malta (MT), Latvia (LV), and Lithuania (LT) are due to the BN method placing almost all the weight on the HEALTH and M_MD dimensions. In these dimensions, LV and LT rank among the worst performers, while MT ranks among the best across all countries considered. Conversely, in the remaining dimensions, LV and LT are among the best performers, whereas MT ranks among the worst. With all other methods, the relative performance across all dimensions is smoothed out, resulting in more moderate changes in country rankings.
The paper showed that Bayesian Networks is the only model meeting a set of desirable properties identified for weighting in multidimensional indexes. This model offers a clear graphical representation of the hierarchical sets of relations across dimensions, it can be easily used to derive weights, and results in clearer, more parsimonious, and potential cost-efficient policy indications.