Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
51,057 characters · 16 sections · 50 citation commands
Human Wellbeing and Machine Learning
\setcounter{page}{1}
Over the last 40 years, researchers from various fields have established an immense literature on the correlates and determinants of subjective wellbeing (clark_four_2018, diener_advances_2018, nikolova_economics_2020). In parallel, international organisations (oecd_hows_2020) and national governments (ons_well-being_2021) have turned to subjective wellbeing data as a key tool for policy analysis.
However, despite the widespread use of wellbeing scores, our current ability to predict wellbeing is limited. Conventional linear models, where variables are selected based on intuition or theory, explain little individual-level variation. Typically, models of individual wellbeing produce an R-squared of no more than 15%.
In response, we here evaluate whether machine learning (ML) algorithms can improve our capacity to understand wellbeing. We answer two research questions:\\
To answer these questions, we use random forests (breiman_random_2001, hastie_elements_2009), gradient boosting (friedman_greedy_2001, natekin_gradient_2013), and penalised regressions (tibshirani_regression_1996) as examples of ML algorithms. Random forests and gradient boosting are tree-based algorithms that have been shown to perform well with tabular data (shwartz-ziv_tabular_2022).\footnote{In contrast, other ML algorithms, such as neural networks, tend to perform poorly on tabular data, which is why we do not consider them here (borisov_deep_2022). In preliminary analyses we did indeed find that feed-forward neural networks yielded performances that were no better than OLS.} Penalised regressions are a convenient tool for analyses that involve large number of covariates, like ours (tibshirani_regression_1996). Generally, these techniques can identify more-complex models of wellbeing than traditional linear models, potentially improving predictive performance. Unlike standard regression techniques, these algorithms allow for the inclusion of an arbitrary number of variables, and, in the case of our tree-based methods, can identify nonlinearities and interactions between variables.
To the best of our knowledge, this paper is the first systematic attempt to evaluate the (dis) advantages of using ML for studying wellbeing at a global scale. Earlier work focused on relatively small country- and year-specific samples (margolis_what_2021), or particular drivers of wellbeing, such as age (kaiser_using_2022-1).
We carry out our empirical analysis using three of the largest currently-available datasets that include wellbeing information: the German Socio-Economic Panel (SOEP), the UK Household Longitudinal Study (UKHLS), and the American Gallup Daily Poll. The SOEP has data on about 30,000 unique respondents and 400 distinct variables; the UKHLS surveys around 40,000 individuals in each wave and has over 500 distinct variables; and each year of the Gallup data has information on around 200,000 respondents with approximately 60 distinct variables. We can thus study the extent to which utilising more information about individual respondents improves the predictive power of wellbeing models.
Regarding RQ1, we find that ML algorithms predict somewhat better than standard linear models. The size of this improvement is small in absolute terms, but substantial when compared to the predictive power of key variables, such as health. Increasing the number of variables in the model from a standard set (we call this the \enquote{Restricted Set}) to all available data (the \enquote{Extended Set}) has a far larger effect on predictive model performance. Predictive accuracy, judged by the R-squared on unseen data, roughly doubles for both OLS and ML methods. Independently of the type of algorithm, an R-squared of 0.30 appears to be the feasible maximum given the available data.\footnote{Our estimations on the extended set of variables, which include all of the variables apart from other measures of subjective wellbeing, produce R-squared figures of between 0.25 and 0.32 across the different datasets.}
For RQ2, our data-driven ML results validate the findings of the conventional literature. We find that variables related to respondents’ social connections, health and material conditions are consistently among the most important in predicting wellbeing. Variable importance is assessed using permutation importances (breiman_random_2001, kuh_mortality_2002) and by computing pseudo partial effects for all algorithms, including OLS. In general, there is substantial correlation in variable importance rankings across algorithms ($\rho=0.58$ to $\rho=0.83$), so that ML approaches and OLS are largely in agreement in terms of what matters most for wellbeing.
We analyse data from three nationally-representative surveys over the 2010 to 2018 period: the German Socio-Economic Panel (SOEP), the UK Longitudinal Household Survey (UKHLS), and the US Gallup Daily Poll (Gallup).
The Gallup data covers the US adult population, with daily cross-sectional telephone-based surveys of 500 (1000 until 2012) respondents. After removing incomplete data, this yields an annual sample ranging from N=115,192 (in 2018) to N=351,875 (in 2011). Wellbeing is measured by the Cantril Ladder of Life (cantril_pattern_1965), which asks: Please imagine a ladder with steps numbered from zero at the bottom to ten at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you. On which step of the ladder would you say you personally feel you stand at this time? Answers are recorded on a scale from 0 to 10, with equal steps between response options.\footnote{There has been controversy about whether such data can support inferences about underlying wellbeing (bond_sad_2019, chen_robust_2019, kaiser_how_2020, schroder_revisiting_2017). We here remain agnostic about this issue. We instead rather ask which algorithms and models best predict the answers to wellbeing questions, without making any further claims about how these answers relate to respondents’ underlying feelings.}
The SOEP is representative of the German adult population, with interviews conducted in person. To allow for a direct comparison with the Gallup data, we here consider the survey period between 2010 and 2018. In each year, between N=26,000 and N=30,000 observations are available. Life satisfaction is measured on a scale from 0 to 10, from the question: We would like to ask you about your satisfaction with your life in general, please answer according to the following scale: 0 means completely dissatisfied and 10 means completely satisfied: How satisfied are you with your life, all things considered?
The UKHLS is representative of the UK adult population. Interviews are conducted in person. We again confine our analysis to the same 2010-2018 period (corresponding to Waves 2 to 10). The number of available annual observations is between N=29,605 to N=40,679. Life satisfaction is measured on a 1 to 7 scale. Respondents are asked: How dissatisfied or satisfied are you with your life overall?.
Descriptive statistics and histograms of each wellbeing measure appear in Figure (ref). The wellbeing distributions are very similar across datasets. As is typically found in high-income countries, wellbeing is strongly left-skewed.
We model wellbeing using four kinds of algorithms. First, as our baseline and corresponding to the workhorse of a great deal of research on subjective wellbeing, we apply Ordinary Least Squares (OLS) regressions. OLS estimates are the solution to the problem:
Here, $x_i$ is a vector of explanatory variables and $b$ the vector of coefficients. The wellbeing of respondent $i$ is denoted by $s_i$. Let $\hat{b}$ be the solution of Equation (ref). Then, the predicted wellbeing level on the respondent $i$ is $\hat{s}_i = x_i'\hat{b}$. When using OLS, the researcher implicitly assumes that reported wellbeing is a linear combination of the chosen set of explanatory variables $x$. If these assumptions are an appropriate description of the true data-generating process, OLS will provide accurate predictions of individual wellbeing. In applications with a large number of covariates, the performance of OLS may degrade due to overfitting or multicollinearity between included explanatory variables.
The second algorithm we consider, the Least Absolute Shrinkage and Selection Operator (LASSO), tackles this issue by adding a penalty for the sum of the magnitudes of the estimated coefficients. In particular, LASSO estimates are the solution to:
Here, $\lambda$ is a hyperparameter, the preferred value of which is found using a grid search. LASSO and OLS are equivalent for $\lambda=0$. Although LASSO may improve predictions by reducing the risk of overfitting, the algorithm continues to assume an additive functional form. Nevertheless, one helpful property of LASSO is that it shrinks coefficients on the variables with low explanatory power to zero. In some specifications, we thus use LASSO as a device for variable selection.
The third and fourth algorithms we consider – Random Forests (RF) and Gradient Boosting (GB) – are based on regression trees (breiman_classification_1984). Regression trees are generated via a recursive binary splitting algorithm. The algorithm splits the sample along values of covariates and predicts the outcome in each subsample, or node, as the mean outcome within each node. More formally, at each step $k$, the data $D$ is split into two nodes $D_{L,k}$ and $D_{R,k}$. The location of the split within the data is determined by some variable $x_j$ and an associated threshold $\tau_{(k,j)}$. The nodes $D_{L,k}$ and $D_{R,k}$ are defined as (see hastie_elements_2009):
The predicted values are the mean value of $s$ within each node, i.e. $\hat{s}_{D_{m,k}} = N^{-1}_{D_{m,k}}\sum_{i:X_i \in D_{m,k}}s_i$, for $m \in \{L,R\}$, where $N_{D_{m,k}}$ is the number of respondents in each node. The splitting variable $x_j$ and the thresholds $s_t$ are determined by minimising the following residual sum of squares:
Finally, the nodes $D_{L,k}$ and $D_{R,k}$ are in turn used as inputs for the next step. This procedure is repeated until some final number of leaves is found. By construction, every split reduces the in-sample mean squared error (MSE).\footnote{Mean squared error measures the average of the squares of the errors – the average squared difference between the predicted and reported levels of wellbeing.} Hence, if the size of the tree is not limited, the algorithm will overfit the data. Limiting the maximum tree size can ameliorate this issue by reducing the variance of the predictions. However, this comes at the cost of increasing the bias of the resulting estimates (hastie_elements_2009). Alternatively, the variance in the predictions can be reduced by aggregating the predictions from multiple trees. Random forests and gradient boosting are both examples of this strategy.
Specifically, Random Forests, the third algorithm we consider, rely on averaging across a large number of trees (which we set to 1,000 for all the three datasets).\footnote{The performance of the random forest is non-decreasing in the number of trees. In our application, increasing the number of trees to 2,000 for UKHLS and Gallup and to 10,000 for SOEP yields qualitatively similar results. The final number of trees was chosen to render the optimisation less computationally-expensive.} Each individual tree has low bias but high variance. When the correlation between the trees is low, averaging across the predictions of multiple trees reduces the variance of the predictions without introducing additional bias. To carry out this procedure, each individual tree is grown on a nonparametrically bootstrapped sample of the original data. The correlation between trees is further reduced by considering only a random subset of all covariates at each split. The size of this subset, Nvars, is a hyperparameter that we select based on a grid search.
The fourth algorithm, Gradient Boosting, proceeds by sequentially fitting regression trees on the residuals of the predictions of the previous collection of trees.\footnote{This construction of the trees, when the residuals from the previous tree are used to build the following tree, is specific to a case when the partitioning of the tree is chosen to minimise the sum of squared residuals in each node. The construction differs when other objective functions are used. See friedman_greedy_2001 and hastie_elements_2009 for the general case. We here use a standard implementation of gradient boosting. In preliminary tests, we also evaluated the performance of extreme gradient boosting (XGBoost; chen_xgboost_2016) in the Gallup and SOEP datasets. The use of XGBoost only yielded negligible improvements compared to standard gradient boosting, which is why we here focus on the latter.} Intuitively, each subsequent tree attempts to explain the variance that was not explained by the previous trees. We begin with the predictions $\hat{s}_{T{}_1}$ of a first tree $T_1$ and calculate the residual $\hat{s}_{T{}_1}-s_i = {e}_{T{}_1}$ . A second tree $T_2$ is then fitted on these residuals to obtain predicted residuals ${e}_{T_1}$. The updated overall predictions are then given by $\hat{s}_{T{}_1}+\hat{e}_{T{}_1} = \hat{s}_{T{}_2}$. A third tree is subsequently trained on the residuals $\hat{s}_{T{}_2}-s_i = {e}_{T{}_2}$. This process is repeated Ntrees times, producing increasingly accurate predictions of $s$. Since gradient-boosted collections of trees overfit with large $Ntrees$, we select this hyperparameter via a grid search. To further reduce overfitting, the size of the update at each step is reduced by adding a penalty $0 < \lambda \leq 1$, and predictions are updated with the rule $\hat{s}_{T{}_k}+\lambda\hat{e}_T{}_k = \hat{s}_T{}_{k+1}$. The penalty $\lambda$ is also selected via a grid search.\footnote{The maximum size of each tree in the gradient-boosting algorithm is significantly smaller than in the case of random forests. Consequently, the individual trees in such an ensemble are called weak learners (freund_boosting_1995, freund_short_1999).}
As is customary, the algorithms are trained on the training set, which here contains 80% of the sample. Each algorithm’s performance is then estimated on the test set, which contains the remaining 20% of observations. Optimal hyperparameters are chosen via 4-fold cross validation\footnote{Cross-validation is used to mimic the predictive performance of a machine learning model on unseen data. The training set is split into 4 sub-samples. At each step one of the sub-samples is held out while the algorithm is fitted on the remaining 3 sub-samples. Performance is then evaluated on the hold-out sample. Finally, the parameters that are associated with the best predictive performance are chosen.} on the training set using grid search. Optimal hyperparameters for all the datasets can be found in Appendix Table (ref). Each of these algorithms are implemented using the scikit-learn library in Python (pedregosa_scikit-learn_2011). To evaluate the stability of our results across time, where feasible, we train each algorithm on each survey-wave combination separately.
We evaluate each algorithm’s performance for two different sets of explanatory variables.
As noted above, we first consider a restricted set of variables that are observed in all three of the datasets, which cover basic demographics as well as economic and health variables. We specifically include: sex, age, age-squared, ethnicity, religiosity, number of household members, number of children in the household, marital status, log household income (equivalised used the modified OECD scale), general health status, disability status, body mass index, labour-force status, working hours, home ownership, area of residence, and interview month. A more detailed description of these variables is provided in Appendix Table (ref). These variables are typical in the conventional literature on subjective wellbeing. This restricted set of variables will then allow us to assess the performance of ML algorithms relative to OLS in a standard estimation setting.
We also evaluate each algorithm on much larger extended sets of explanatory variables. Here, we only use the 2013 Wave of Gallup and SOEP, and Wave 3 of the UKHLS (which covers 2011-2012).\footnote{These waves/years were chosen as they include personality traits in the SOEP and UKHLS.} Our dataset includes all of the available variables, apart from direct measures of subjective wellbeing (such as domain satisfaction, happiness, or subjective health) or mental health. We also exclude variables with more than 50% missing values. The resulting Gallup dataset contains 67 variables, and around 450 variables are retained in the SOEP and UKHLS. Missing values for continuous variables are assigned the observed means, while missing values for categorical variables are assigned a new category.\footnote{Processing categorical variables and removing perfectly collinear variables respectively yields 210, 542, and 957 effective explanatory variables in the Gallup, SOEP and UKHLS datasets. } We convert categorical variables into a set of dummies, one for each category. The full list of variables in this extended set appears in the supplementary material.
The large number of variables in the extended set produces significant computational burden. At the same time, it is evident that some portion of these variables will have no predictive power for wellbeing. We therefore use LASSO as a device to select the explanatory variables (tibshirani_regression_1996, ahrens_lassopack_2020).\footnote{Using LASSO on the restricted set of variables produced a similar performance to OLS, with optimal $\lambda=0$.} We have carried out the estimations on both the full extended set and the post-LASSO extended set. Typically, both approaches perform similarly. For simplicity, we only show results for the approach that performed better in each individual case.
To answer our second research question, we need to assess how important each explanatory variable is in enabling our algorithms to predict wellbeing. We do so in two ways.
We first use permutation importances (PIs) to measure the degree to which each algorithm relies on a given variable in making its predictions (molnar_interpretable_2022).\footnote{Shapley values are an alternative option to assess feature importances. However, despite recent advances in reducing the computational complexity of obtaining Shapley values (Lundberg_machine_2018), the size of our datasets and models makes the use of Shapley values computationally infeasible (see e.g. yang_fast_2021).} PIs are calculated by randomly shuffling a given variable’s observed values across individuals in the test data and evaluating the extent to which the predictive performance (in terms of R-squared) of a given algorithm falls when permuting the variable’s values. This operation is carried out 10 times. The reported PI is the average change in the R-squared across these 10 iterations. The greater the average fall in the R-squared, the more important is the variable.
To understand the direction of our variables’ effects we also report pseudo partial effects (PPEs). These are calculated by taking the difference in predicted wellbeing after setting each explanatory variable to a given set of values. Specifically, for continuous and ordinal variables we set the variable to the third and first quartile of their distributions and calculate the mean difference in predicted wellbeing. For binary variables (including dummies for all of the categorical variables), we predict wellbeing when setting each individual’s value to either 0 or 1.
A key advantage of PIs and PPEs is that they can be used with any kind of algorithm, allowing us to compare the way in which each algorithm makes use of the available data.
We begin with RQ1, i.e. whether ML algorithms significantly outperform OLS in predicting wellbeing. As noted, OLS is the standard approach followed in the conventional literature.
We start with the analysis based on the restricted set of covariates, which includes the variables that are typical in many conventional wellbeing estimations. Figure (ref) depicts the performance of each algorithm on the test-set portion of each dataset. We use R-squared as our primary evaluation metric in order to facilitate the comparison with previous analyses.
In Figure (ref) each algorithm is trained separately for each year between 2010 and 2018. The values refer to the average R-squared across these years and their standard deviations. The R-squareds are very similar across datasets, ranging from 0.10 (SOEP) to 0.14 (Gallup). Gradient boosting (GB) and random forests (RF) yield larger R-squared values than OLS in each case. Specifically, random forests yield absolute increases in R-squared of 0.024 (SOEP), 0.004 (UKHLS) and 0.016 (Gallup); the respective improvements from using gradient boosting are slightly larger, with respective R-squared gains of 0.030, 0.005, and 0.018.\footnote{These gains are calculated from the test set, which was not used for training the algorithm. In the training set, i.e. the data that is observed by each algorithm, the improvement from performance of the ML algorithms over OLS is larger (see Appendix Figure (ref)). The predictive capacity of the ML algorithms applied to the test set does not then seem to be constrained by underfitting. Of course, performance in the training set is not per se indicative of the quality of an algorithm. A decision tree with as many leaves as training individuals would yield an MSE of 0. However, this model would perform extremely poorly when used to assess unseen test data.} ML algorithms thus do outperform linear regressions, and gradient boosting always outperforms random forests.
These gain figures considered on their own are hard to interpret. To illustrate the substantive size of these improvements, we compare them to the change in predictive performance when omitting information on respondent’s health status – a key wellbeing predictor – in our baseline OLS regressions. Panel A of Table (ref) lists the changes in the test-set R-squared of the OLS regression when omitting this information and compares this figure to the gain from using gradient boosting. As benchmarked against the gain from adding health information, the prediction-improvement figure from gradient boosting (as our best ML algorithm) lies between 15% and 107%. When evaluated in this way, the gains from using ML do look substantial.
Adding further explanatory variables should increase our ability to predict wellbeing. Given the greater flexibility of the ML algorithms, we should expect these to benefit more from additional variables than OLS. To test this, we estimate all of our models on the extended sets of variables. As explained in Section 2.3, these extended sets include all of the variables available in the 2013 waves of the SOEP and Gallup, and Wave 3 of the UKHLS.
Figure (ref) depicts our main results.\footnote{The results for the training set can be found in Appendix Figure (ref))} The R-squared figure approximately doubles using the extended set for all algorithms, including OLS. The OLS R-squared is now 0.28 for the SOEP, 0.21 in the UKHLS and 0.27 for Gallup. As such, standard economic specifications do not fully exploit the predictive information available in typical large-scale survey data.\footnote{All of these R-squared estimates are obtained using the test set. Hence, these improvements cannot be attributed to a mechanical increase in the share of explained variance due to adding more variables to the model.}
Gradient boosting remains the best-performing algorithm and clearly predicts better than OLS. The absolute gain in the R-squared from gradient boosting over OLS is now 0.034, 0.028 and 0.010 for the SOEP, UKHLS and Gallup respectively. Random forests now tend to perform poorly, underperforming OLS for SOEP and Gallup. This has also been observed in other empirical applications where covariates were measured with error (Reis_2018).
We again interpret the size of the gains from gradient boosting by comparing them to those from the inclusion of respondents’ health information when using OLS.\footnote{In these extended specifications, there are multiple variables related to health in each dataset. We remove 21, 19 and 12 health-related variables in the Gallup, the SOEP and UKHLS respectively.} The results in Panel B of Table (ref) illustrate that these gains are again substantial, being approximately equivalent to the role of health in predicting wellbeing.
We thus conclude that tree-based ML algorithms can provide improvements in predictive performance over conventional methods. These gains are moderate in absolute terms, but are meaningful when compared to the predictive power of health. However, we also note that these gains are obtained with algorithms that take up to 100 times more time to estimate.\footnote{This figure is based on a comparison between OLS and RF on the Gallup data with the extended dataset.} The use of ML algorithms thus involves a trade-off between computational burden and predictive performance.
There are multiple reasons that can explain why nonlinear ML methods do not yield a substantial improvement in predicting human wellbeing compared to linear regression. First, most independent variables in the datasets we have used are binary or categorical. Such datasets cannot exhibit non-linearity except by interaction terms between variables. Therefore, if a large number of the variables present only take binary values the ways that improvements can occur with non-linear models are limited. It is possible that non-linear relationships do exist but the variables concerned have a small contribution to the outcome. This is particularly likely if there are many variables contributing to the outcome, as is the case in our extended set of independent variables. Additionally, it may be that the non-linearity is present only at the extremes of the distribution where only few points exist.
As well as improvements in performance, ML may also indicate new, and potentially-overlooked, variables that are key in explaining subjective wellbeing. The next section explores this idea.
In this section we ask whether the variables that ML identifies as important in predicting life satisfaction correspond to those emphasised in the conventional literature. We do so by estimating variable importances, as discussed in Section 2.4. Our ML-based findings turn out to fit well with the results in previous analyses.
We start by estimating variable importances in the extended dataset, which provides more possibilities for the identification of important variables that do not appear in conventional wellbeing models. Figure (ref) lists the five most-important variables identified in OLS and GB, which is the best-performing ML algorithm, in each dataset.\footnote{We present the Top-10 most-important variables for OLS, RF and GB in the three datasets in Appendix Table (ref).} The bars and numerical values refer to permutation importance, i.e. the drop in the model’s R-squared when the values of the variable are randomly permuted across respondents. The variables that are negatively associated with average wellbeing are in red, and those with a positive association in green. In all three countries, individual health and interpersonal relationships are among the most-important predictors. As expected, respondents whose health limits their activities are on average less satisfied, while people with fulfilling relationships are typically more satisfied with their lives. The directions of the estimated effects are in line with those in the previous conventional work. ML algorithms and OLS thus generally agree on the direction and approximate size of the most-important variables (see Appendix Table (ref) for the effect-size estimates).
As a more systematic measure of the degree of agreement between ML and OLS, we calculated the correlations of the ranks (in terms of their permutation importance) of each variable across algorithms and datasets. The results appear in Table (ref). There is strong agreement between GB and RF in all three datasets, with the rank correlation figure never falling below 0.79. The correlations with the OLS ranking are somewhat lower, with a minimum value of 0.58 (OLS vs. RF in SOEP). Nevertheless, we can strongly reject ($p<0.001$) the null hypothesis that the rankings are uncorrelated, supporting our conclusion that the OLS and ML algorithms are in broad agreement.
Apart from the conventional variables used in wellbeing analysis, such as health and interpersonal relationships, the algorithms also identify personality traits as important predictors in the UKHLS and SOEP. Personality traits, unfortunately, do not appear in the Gallup survey. In the UK data, measures associated with (the absence of) neuroticism (i.e. worrying, and feeling relaxed) appear in the Top-3. In German data, worrying a lot, being able to deal with stress, and patience are among the most-important variables in all empirical approaches. This is line with previous research underlining the potential advantages of including personality traits in wellbeing regressions (ferrericarbonell_how_2004, proto_covid-19_2021).
Beyond these similarities, there are some cross-country differences. The most striking refer to the importance of financial factors. These are important in the US (e.g., HH income and being able to pay for healthcare) but not in the other countries. To see whether this is a genuine finding or a consequence of differences in variable availability across countries, we carry out the same analysis using the restricted set (for which we have a common set of variables). When we do so, the cross-country differences in the importance of income largely disappear. As shown in Appendix Table (ref), the most-important variables identified in these harmonised datasets are very similar across the three countries. They include health, income, marital and employment status, as well as home-ownership – which is a proxy for wealth – and age. Sex and ethnicity are only important in the US. Education is among the most important factors in the US and Germany, but not in the UK.
The preceding section concluded that the kinds of variables that machine learning finds to be important - and the estimated direction of their association with wellbeing - are largely in line with the results in the conventional literature. We here present a detailed analysis of two variables that have attracted a great deal of interest in the conventional literature: age and income. In OLS estimation, the functional forms associated with these two variables are imposed by the analyst, while they are instead freely estimated in our tree-based ML algorithms.
The results appear in Figure (ref) and Appendix Figure (ref). In the OLS estimation, illustrated in blue, we assume a quadratic form for age, and a log-linear functional form for income, which are very common functional forms in this literature. The relationships for RF are in red, and those for GB in green.
For low to medium incomes, both ML algorithms track the assumed log-normal functional form remarkably closely, in line with the conventional literature. However, once we reach relatively-high equivalised annual income figures, above 50,000 EUR in the SOEP or 40,000 GBP in the UKHLS, the ML algorithms suggest that wellbeing no longer increases with income. We cannot confirm this finding in the US, as income in Gallup appears in bands with the highest band being 100,000 USD or above. In 2013, 100,000 USD were approximately equivalent to 70,000 GBP or 78,000 EUR.\footnote{\url{https://data.oecd.org/conversion/purchasing-power-parities-ppp.htm }} In addition, the Gallup 2013 wave did not collect data on household size. As a result, household income in Gallup is not directly comparable to the adjusted equivalent incomes in SOEP and UKHLS. Given these caveats, we do not find evidence of satiation in the US data. Our ML findings are therefore in line with previous work on wellbeing using US data (kahneman_high_2010, killingsworth_experienced_2021).
With respect to the relationship between age and wellbeing, our ML estimations replicate the well-known approximate U-shape up to age 70 (e.g. cheng_longitudinal_2017), which is more pronounced in the US. However, unlike the smooth U-shape assumed in the OLS approach, we find a much more pronounced “kink” at around age 65 for each dataset and ML-algorithm. We suspect that this kink reflects the gains in wellbeing following on from retirement (gorry_does_2018, wetzel_transition_2016).
We also evaluate the performance of gradient boosting and random forests on measures of positive and negative affect. The results show that our findings are not specific to the use of life evaluations as the measure of subjective wellbeing, but generalise to affect (or mood). In the 2013 Gallup data, positive affect is measured by the average figure from dummy variables indicating whether the respondent felt happiness or joy, or smiled during the previous day. Negative affect is calculated analogously from dummies indicating pain, worry, sadness, and anger. In the German SOEP, positive affect is the self-reported frequency of being happy in the last 4 weeks (on a 1 to 5 scale), and negative affect as the average of the self-reported frequency from three questions about being angry, sad, or worried in the last four weeks (all measured on a 1 to 5 scale). The UKHLS dataset does not contain comparable affect data and is not used in this part of analysis.
The results for the Gallup data appear in the top panels of Figure (ref) and Table (ref). It is notable that negative affect is easier to predict than positive affect. This finding holds across algorithms, with R-squared figures ranging from 0.423 and 0.464 for negative affect, and between 0.261 and 0.296 for positive affect. Random forests and gradient boosting outperform both OLS and LASSO. As was the case for life evaluations, gradient boosting again performs the best, with gains in R-squared over OLS of 0.041 for negative affect and 0.036 for positive affect. Regarding variable importances, Table (ref) shows that good health is even more important for predicting positive and negative affect in the Gallup data than it was for life evaluation. Moreover, in line with previous work (e.g. kahneman_high_2010), variables relating to material conditions – like income – do not feature in the list of the set of most-important variables when modelling affect.
As shown in Table (ref) and the bottom panels of Figure (ref), the results are qualitatively similar in the German data: gradient boosting again performs best, and positive affect is harder to predict than negative affect.
Our main findings regarding the ML estimation of wellbeing are also robust to exploiting the panel dimension of the German SOEP and the UKHLS. As there is no standard procedure for the introduction of individual fixed effects in the ML algorithms that we use, we implement an approach similar to the Mundlak correction for linear models (mundlak_pooling_1978, wooldridge_econometric_2010): we pool all years of the UKHLS and SOEP data, demean all covariates at the individual level and include both an individual’s average value over time of each covariate as well as their year-specific deviations from their individual mean. The level of wellbeing is the dependent variable, as was the case in the analysis above.
The relative predictive performance of the OLS and ML in the pooled dataset is similar to the findings for individual years. In the UKHLS, the OLS R-squared is 0.140. The use of RF produces a small improvement, with the R-squared increasing to 0.143. Gradient boosting provides a further improvement, yielding an R-squared of 0.150. In the German SOEP, the OLS R-Squared is 0.12, with once again both the random forest and gradient boosting leading to better R-Squared figures of, respectively, 0.15 and 0.16. As shown in Tables (ref) and (ref), the most important variables predicting the level of wellbeing are almost exclusively the average values of the individual covariates. One exception in both the UKHLS and SOEP is the Health limits activities variable. As such, deviations in individual health status (from their average value) seem to be important for the level of individual wellbeing.
We draw three main conclusions from our analysis above.
First, tree-based ML approaches do indeed perform better at predicting wellbeing than more-conventional linear models. Although the gains in R-squared we obtain are modest in absolute terms, they are comparable with – and sometimes exceed – the extent to which information on respondents’ health can improve wellbeing predictions. Comparing the algorithms we consider, gradient boosting consistently outperforms random forests.
Second, when we use all of the non-wellbeing variables that are available in each dataset as predictors, we more than double the explained variation in wellbeing for all of the estimation procedures that we analyse. This extended set of variables produces R-squared figures of around 0.3. These values look to be the maximum achievable with the current survey data.
Third, almost all of the variables that turn out to be important in the specifications using of all the available data relate to health, economic conditions, personality traits, and personal relationships. This purely data-driven process thus picks out the same core determinants of wellbeing as have been identified in the conventional literature. In that sense, machine-learning approaches validate the previous human-guided search for the determinants of wellbeing. This looks to be good news for the field.
We see two directions for future research.
The first is to further explore the capabilities of ML models. We have focused our analysis here on tree-based methods, which are powerful algorithms that perform well in multiple contexts. However, given the specificities of wellbeing data, we might find further improvements by using other algorithms (e.g. Kernel Ridge, Vovk2013), or by using a combination of theory-based modelling and algorithmic approaches. Another potential approach is using a combination of unsupervised and supervised learning. For example, it might be possible to separate the whole dataset into overapping clusters of individuals chosen based on subsets of independent variables. Then, the predictive performance of non-linear ML models could be substantially higher when applied to such clusters, as compared to using one global model for the whole dataset as done in our work. Moreover, we have currently only focused on identifying variables that are key for the successful prediction of wellbeing. A natural next step is to extend the use of ML-based algorithms to investigate the variables that are most important for wellbeing in a causal sense (wager_estimation_2018).
Second, our analyses focused on rich Western countries. As such, it remains an open question whether our findings would also hold in a more global setting, e.g. in countries where material needs are much more acute. Insofar as there may be greater scope for improving wellbeing in low- and middle-income countries (helliwell_happiness_2022), applying ML approaches in this setting may be particularly valuable going forward.
\printbibliography[heading=bibintoc]