Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
30,937 characters · 12 sections · 43 citation commands
Exploring Distributions of House Prices and House Price Indices
Income distributions have a long history of being modeled by the Generalized Beta (GB) family of distributions. A comprehensive summary can be found in chotikapanich2008modelling, and specifically in mcdonald2008modelling, while more recent works include chotikapanich2018using and dashti2020stochastic. The motivation for this study was to glean insight into whether income distributions exhibit power-law (“fat") tails or display outliers such as Dragon Kings (DK) or negative Dragon Kings (nDK) sornette2012dragon,wheatley2015multiple,liu2023rethinking,liu2023dragon. Given that identifying fat tails and outliers requires vary large datasets, available coarse-grained income data is not sufficient for this purpose. In this regard, house prices (HP) may serve as a proxy to -- or “derivative" of -- incomes.
Previously, we considered sale prices of houses in Hamilton County, Ohio from 1970 to 2010 in the context of analysis of inequality indices dashti2020stochastic. Here we pare down that data to include only single-family homes, which better aligns to incomes, as well as to house price indices (HPI), which are also based on single-family HP and are constructed using repeat-sale methodology baily1963regression, case1987prices, case1989efficiency, calhoun1996house, bogin2016local. Specifically, we utilize FHFA HPI for nearly 18,000 US ZIP codes over a period of 40 years, starting in 1980's bogin2016local. HPI can be viewed as a proxy to HP -- or a "second derivative" of income distribution -- in this case between different ZIP codes.
Our approach to analyzing HP and HPI distributions mirrors the approach we used for analysis of the distributions of historical realized volatility liu2023dragon. Namely, we fitted the entire distribution with modified Generalized Beta (mGB) and Generalized Beta Prime (GB2) distributions, as well as performed a linear fit (LF) of the tails. For each fit, we evaluated confidence interval (CI) as well as conducted a U-test liu2023dragon,pisarenko2012robust, which yields p-values that indicate the likelihood of the data coming from the fitted distribution.
Since fat tails are scale free, they are naturally analyzed on a log-log scale, where complementary cumulative distribution function (CCDF) ought to be a straight line with negative slope. LF of CCDF, however, must be supplemented by a statistical test to determine the likelihood that the points in the tail conform to linearity or whether outliers may be present wheatley2015multiple. In this case we are concerned with possibility of observing DK, where the ends of the tails shoot upward from LF, or nDK, where tails' ends shoot down.
Towards this end, we performed a U-test pisarenko2012robust but we did not limit it to LF alone. We also aimed at describing the entire distribution, not just tails, and thus performed a U-test on mGB and GB2, which we employed for this purpose. GB2 was used for fitting since it is the most flexible distribution with fat tails and mGB, while exhibiting similar tails over a wide range of variable, abruptly terminates at finite value, which seems appropriate to describe nDK behavior liu2023rethinking. Of particular importance is the fact that both mGB and GB2 arise as steady-state (stationary) distributions of stochastic differential equations that mean to serve as models of economic exchange dashti2020stochastic,bouchaud2000wealth,ma2013distribution.
This paper is organized as follows. In Section (ref) we present the analytical form of mGB and GB2 distributions and discuss the limiting behaviors of both. In Section (ref) we fit HP and HPI with mGB and GB 2, as well as the fit tails directly using LF. For each test we conduct a U-test, for which a null hypothesis is formulated, and plot p-values which reflect on the goodness of fit, as well as whether DK or nDK behavior may be present. We conclude in Section (ref) with discussion of our results.
A detailed discussion of the Modified Generalized Beta (mGB) distribution function, used here to fit the distributions of HP and HPI, as well as of the Generalized Beta (GB) family of distributions in general, can be found in liu2023rethinking, liu2023dragon. A generalization of the traditional GB distribution mcdonald1995generalization can be written (in a slightly modified form relative to that of liu2023rethinking) as follows liu2023dragon:
where $\beta _1$ and $\beta _2$ are scale parameters and $\alpha$, $p$ and $q$ are shape parameters, all positive, $B(p,q)$ is the beta function and $x \leq \beta _1$. Although it has a concise and transparent form, it does not come out as a solution of a stochastic differential equation (SDE) hertzler2003classical, the latter being desirable for the purpose of modeling behavior of quantities, such as stochastic volatility, important for understanding of implied and realized volatility in equity markets dashti2021combined, and income distribution, resulting from economic exchange dashti2020stochastic,bouchaud2000wealth,ma2013distribution.
The probability density function (PDF) of mGB, which comes out as a solution of an SDE (with minor caveats explained in liu2023rethinking) and which is used here to model HP and HPI distributions, can be written as
where $\beta _1$ and $\beta _2$ are scale parameters and $\alpha$, $p$ and $q$ are shape parameters, all positive, $B(p,q)$ is the beta function and $x \leq \beta _1$. The cumulative distribution function (CDF) and complementary CDF (CCDF) of mGB are given respectively by
and
where the first term in ((ref)) and ((ref)) represent, respectively, CDF and CCDF of GB (whose PDF is given by((ref))), while $I(y;p,q)=B(y;p,q)/B(p,q)$ and $B(y;p,q)$ are, respectively, the regularized and incomplete beta functions nist2022digital.
In what follows, we will be specifically interested in the $\beta_2\ll\beta_1$ circumstance since for $\beta_2 \ll x \ll \beta_1$ GB and mGB exhibit a power-law dependence,
In the limit of $\beta_1 \rightarrow \infty$, mGB and GB become, respectively, mGB2 and GB2 (the latter also known as Generalized Beta Prime) and are given by liu2023rethinking
and
Unlike mGB and GB, for whom the power-law dependences in ((ref)) eventually terminate at $\beta_1$, mGB2 and GB2 will sustain these power-law dependences indefinitely.
Below, we will use ((ref)) to fit CCDF of distributions of HP and HPI. As explained in liu2023rethinking, mGB2 and GB2 are equivalent since $q$ and $p$ are independently defined at this level of GB family of distributions, and $q$ can be shifted by unity in the definition of mGB2/GB2. Consequently, we choose a more familiar CCDF of GB2
to fit CCDF of the HP and HPI data. Insofar as the main difference between mGB and GB is concerned, it is their behavior near $\beta_1$ in the present context liu2023rethinking. Namely,
and
that is $1-F_{mGB}$ drops off to zero ($F_{mGB}$ saturates to unity) faster than $1-F_{GB}$ due to the factor $\left(\frac{\beta _2}{\beta _1}\right)^\alpha$. This feature accounts for a better fit via mGB versus GB, which may be due to the fact that mGB emerges from a physically motivated stochastic model liu2023rethinking.
In this section we perform Bayesian fitting of the entire distributions of HP and HPI using mGB, as per ((ref))-((ref)), and GB2, as per ((ref)) and ((ref)). We also perform LF of the tails. Confidence intervals (CI) for the fits are evaluated via inversion of the binomial distribution janczura2012black; $p$-values are evaluated in the framework of the U-test, which is based on order statistics pisarenko2012robust, using the following formula:
where $x_{k,m}$ is the $k$'s member of numbers between $1$ and $m$ ordered by increasing magnitude (HP and HPI in this case), and $F(x_{k,m})$ is the assumed CDF (mGB, GB2 and LF here). p-values are evaluated in order to test the null hypothesis $H_0$: all observations of the sample are generated by the same fitting distribution. The p-value ((ref)) is defined as a probability of exceeding the observed value $x_{k,m}$ under the null hypothesis. If among the p-values there are some small values -- $\le0.05$ here -- then those observed values are identified as DK with probability $1-p$. Conversely, large p-values -- $\ge0.95$ here -- are identified as nDK with the probability $p$ pisarenko2012robust.
In this section we fit the distribution of sale prices of single-family homes in Hamilton County, Ohio between 1970 and 2010. This dataset contains 116207 entries and sale prices are converted to constant dollars to eliminate the effect of inflation. Table (ref) gives all estimated parameters of mGB and GB2 distributions and slopes of two linear fits: LF-1 is performed by visually excluding possible outliers pisarenko2012robust and LF-2 by excluding HP whose sale price was above 90% of the top HP.
Fig. (ref) shows probability density function (PDF) of HP with mGB and GB2 fits. Clearly neither fit does a good job for typical values HP.
Fig. (ref) shows CCDF of the HP distribution on a log-log scale, with its mGB and GB2 fits and LF of the tail. The tail area is further expanded for a better view. Clearly the end points of the tail fall off abruptly from what looks like a rather well-defined power-law dependence, prompting a possibility that they may considered nDK.
Fig. (ref) shows p-values obtained in U-test for all three fits (mGB, GB2 and LF), as well as the LF with its CI (dashed line). p-values such that $p<0.05$, had they been at the tail ends, could be considered DK, however in the earlier parts of the tail they are merely an indication of a poor fit (as are values $p>0.95$). We dubbed them "potential DK" (pDK) and mark in the plot with up triangles. p-values $p>0.95$ at the tail ends may qualify as actual nDK.
Fig. (ref) shows tail areas of mGB and GB2 fits, with their CI, pDK, and nDK. Clearly mGB makes a decent attempt at approximating nDK behavior. However, earlier part of the tail (as well as full PDF above) indicate a rather poor fit by either.
Fig. (ref) shows p-value plot and LF, whose main difference from Fig. (ref) is that LF is done by excluding the end points with HP greater than 90% of the top price, rather than excluding possible outliers (nDK here) visually. Clearly, there is essentially very little difference between the two. It is the case for HPI as well, so we do not present a corresponding plot for HPI below.
HPI are constructed using repeat-sale methodology baily1963regression, case1987prices, case1989efficiency, calhoun1996house, bogin2016local. Specifically, we utilize FHFA HPI for nearly 18,000 US ZIP codes over a period of 40 years, starting in 1980's bogin2016local. HPI are updated annually on FHFA site fhfa2003hpi. HPI can be viewed as a proxy to HP, which in turn can be viewed as a proxy to income distribution -- here between different ZIP codes. In what follows, we initially analyze single-year distributions of HPI, followed by the multi-year distribution, which can be considered as a sum of single-year distributions.
We conducted fitting and analysis of a number of single-year HPI and found the results to be very similar, in particular over last two decades. Therefore here we present the results for only one year, 2019. The dataset contains HPI for 9033 zip codes. Table (ref) contains all parameters of the fits, as well as slopes of GB2 and LF.
Fig. (ref) shows PDF of single-year HPI with mGB and GB2 fits. Visually these PDF fits are considerably better than those of HP.
Fig. (ref) shows CCDF of the HPI distribution on a log-log scale, with its mGB and GB2 fits and LF of the tail. The tail area is further expanded for a better view. Visually, the tail behavior is more consistent with power law than was the case of HP. This is further confirmed by p-values obtained in the U-test (see Fig. (ref) below).
Fig. (ref) shows p-values obtained in U-test for all three fits (mGB, GB2 and LF), as well as the LF with its CI (dashed line). We observe that GB2 and LF do not have p-values $p>0.95$, which indicates less likelihood of outliers form the power-law behavior than was seen for HP, consistent with the previous visual observation in Fig. (ref).
Fig. (ref) shows tail areas of mGB and GB2 fits, with their CI, pDK, and nDK. Both distributions are relatively successful at approximating different parts of the tail.
As was discussed in Sec. (ref), tails of single-year HPI align better with power law than those of HP. However, there is also an obvious trend towards moving down from the straight line at tails ends. Since single-year HPI also have an oder of magnitude fewer points than HP, we wanted to ascertain whether this trend was significant.
Towards this end we studied a combined multi-year distribution of HPI for years 2000-2022, which contained 201040 data points. The main result, as seen in Figs. (ref) and (ref) below is that the tails of the combined HPI is more aligned with the finite upper limit of HPI and, accordingly, with mGB distribution. Of course, such upper limit of the variable does not have to be fixed -- it may change as HPI is updated annually.
Table (ref) contains all parameters of the fits of the multi-year distribution, as well as slopes of GB2 and LF.
Fig. (ref) shows PDF of multi-year HPI with mGB and GB2 fits. Visually the fits are better than their single-year counterpart, most likely because of a much larger dataset.
Fig. (ref) shows CCDF of the multi-year HPI distribution on a log-log scale, with its mGB and GB2 fits and LF of the tail. The tail area is further expanded for a better view. Visually, the tail behavior is more consistent with finite upper limit and mGB distribution than that of a single-year HPI, and is similar to HP. This is further confirmed by p-values obtained in the U-test (see Fig. (ref) below).
Fig. (ref) shows p-values obtained in U-test for all three fits (mGB, GB2 and LF), as well as the LF with its CI (dashed line). We observe that, unlike GB2 and LF, mGB does not have p-values $p>0.95$ at tail ends, consistent with the previous visual observation in Fig. (ref). Conversely, $p>0.95$ for GB2 and $p<0.05$ for mGB indicate poor fits at the onset of the tails.
Fig. (ref) shows tail areas of mGB and GB2 fits, with their CI, pDK, and nDK. Clearly mGB approximates rather well nDK behavior at the tail ends, which indicates a finite upper limit. However, as indicated above, the earlier part of the tail indicate a rather poor fit by either.
We studied distributions of house prices and house price indices. The key question we attempted to answer is whether those distributions have power-law (fat) tails or they are characterized by outliers at the tails ends, such as Dragon Kings and negative Dragon Kings. Towards this end, we conducted a linear fit of the tails of the complementary cumulative distribution functions on the log-log scale. In doing so, we excluded potential outliers - negative Dragon Kings here - and evaluated fits' confidence intervals, as well as conducted a U-test for the potential outliers which provides us with p-values that indicate whether they belong to the linear fit.
Based on the linear fit, confidence intervals, and p-values we can conclude that house price distributions and combined multi-year house price indices exhibit negative Dragon King behavior at the tail ends. Single-year house price indices, on the other hand, are not inconsistent with linear dependence, that is with power-law tails.
Our interest to house prices and house price indices was motivated by their being proxies to income distributions. Income distributions may be possible to describe by models of economic exchange, some of which can be reduced to stochastic differential equations with well-defined steady-state distributions. One class of such models results in steady-state distributions that belong to the Generalized Beta family of distributions. Therefore we also attempted to fit the entire empirical distributions with Generalized Beta Prime and modified Generalized Beta distributions: for a particular relationship between scale parameters the former is characterized by a power-law tail, while the latter follows the same power-law dependence, which is subsequently terminated at the finite value of the variable.
We find that modified Generalized Beta has some success in describing tail ends, however it is difficult to fully access overall goodness of fit. In particular we find that using Kolmogorov-Smirnov statistic for comparing goodness of fit between GB2 and mGB is rather inconclusive. Among idiosyncrasies are the following: despite an extra scale parameter, modified Generalized Beta may have large statistic than Generalized Beta Prime; for two close values of statistic the higher one may yield a better fit of the tail \footnote{ In this regard, for HPI we presented mGB and GB2 with slightly higher Kolmogorov-Statistic but with better tail fits.}; the largest values of statistic may occur prior to the onset of the tails. While there exist Monte-Carlo techniques aimed at estimating goodness of fit, we believe that they may be problematic, especially for distributions with power-law tails. We hope to address these issues in a future publication.
We used Wolfram Mathematica in a subset of analytical calculations and MathWorks Matlab for much of the numerical work.
Our dataset of Hamilton County, Ohio house prices is available upon request. Datasets of house price indices can be found at
\url{https://www.fhfa.gov/DataTools/Downloads/Pages/House-Price-Index-Datasets.aspx#atvol}.
Jiong Liu and Hamed Farahani performed all numerical calculations, R.A. Serota was a lead on analytical part and on problem statement.
The authors have no conflicts of interest to declare.