Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
45,087 characters · 16 sections · 8 citation commands
Voting patterns in 2016: Exploration using multilevel regression and poststratification (MRP) on pre-election polls
After any election, we typically want to understand how the electorate voted. While national and state results give exact measures of aggregate voting, we may be interested in voting behavior that cuts across state lines, such as how different demographic groups voted. Exit polls provide one such measure, but without access to the raw data we cannot determine aggregates beyond the margins that are supplied by the exit poll aggregates.
In pursuit of this goal, we can use national pre-election polls in which respondents are asked for whom they plan to vote and post-election polls in which respondents are asked if they participated in the election, both of which record demographic information and state residency of respondents. Using this data, we then build a statistical model that uses demographics and state information to predict the probability that an eligible voter voted in the election and which candidate a voter supports. A model that accurately predicts voting intentions for specific demographic groups (e.g. college-educated Hispanic men living in Georgia) will require deep interactions as outlined in ghitza2013deep. In order to precisely learn the second- and third-order interactions, we require a large dataset that covers many disparate groups.
Armed with our two models, we can use U.S. Census data to yield the number of people in each demographic group. For each group, we then predict the number of voters, and the number of votes for each candidate to yield a fine-grained dataset. We can then aggregate this dataset along any demographic axes we choose in order to investigate voting behavior.
We use YouGov's daily tracking polls from 10/24/2016 through 11/6/2016 to train the 2016 voter preference model. We included 56,946 respondents in the final dataset after filtering out incomplete cases. To train the 2012 voter preference model we used 18,716 respondents polled on 11/4/2012 from YouGov's daily tracking poll.
In order to train the 2016 voter turnout model, we use the Current Population Survey (CPS) from 2016, which includes a voting supplement (ipums). The model used 80,766 responses from voters as to whether they voted in the 2016 presidential election. We used the CPS from 2012 to train the 2012 voter turnout model, which comprises 81,017 voters. We decided to use the CPS to train our model because it is viewed as the gold-standard in voter-turnout polling lei20162008.
We use a modified version of the 2012 Public Use Microdata Sample Census dataset (PUMS) to get a measure of the total number of eligible voters in the U.S. YouGov provided the PUMS dataset with ages and education adjusted to match the 2016 population.
Our methodology follows that outlined in gelman1997poststratification, ghitza2013deep, and lei20162008. For voter $i$ in group $g$ as defined by the values of a collection of categorical variables, we want to learn the voter's propensity to vote and for whom they plan to vote, by using a nonrandom sample from the population of interest. We assume that an individual voter's response in group $g$ is modeled as follows:
where $T_i$ is $1$ if the voter plans to vote for Trump, or $0$ otherwise. $\alpha_{g[i]}$ is the probability of voting for Trump for voter $i$ in group $g$. In order to make inferences about $\alpha_{g[i]}$ without modeling the selection process, we need to stratify our respondents into small enough groups so that within a cell selection is random (i.e. that the responses are Bernoulli random variables conditional only on $g$). We do so by generating multidimensional cells defined by demographic variables like age, ethnicity, and state of residence that categorize our respondents. This induces data sparsity even in large polls so we must use Bayesian hierarchical models to partially pool cells along these demographic axes.
Upon fitting our model, we can use the posterior mean of $\alpha_g$, $\hat{\alpha}_g$ and Census data to estimate an aggregate Trump vote proportion by calculating the weighted average $\sum_{g \in D}\tfrac{N_g \hat{\alpha}_g}{N_D}$ for whatever demographic category $D$ we like.
We measure our electorate using six categorical variables:
Each variable $v$ has $L_v$ levels. State residency has fifty levels. Ethnicity has four levels: Black, Hispanic, Other, and White. Gender has two levels. Marital status has three levels: Never married, Married, Not married. Age has four levels, corresponding to the left-closed intervals of age: $[18,30), [30,45), [45,65), [65,99)$. Education has five levels: No High School, High School, Some College, College, Post Graduate.
After binning our Census data by the six-way interaction of the above attributes, we generate table (ref). Each row of the table represents a specific group of the population, an intersection of six observable attributes. We refer to each row as a cell, and the full table as a six-way poststratification table. Our table has 33,561 cells, reflecting the fact that not all possible six-way groups exist in the U.S..
We then add columns to this dataset that represent the cell-by-cell probability of voting and the cell-by-cell probability of supporting Trump, which can be combined to yield the expected number of Trump voters, $\mathbb{E}\left[\text{T}_g\right]$, in each cell $g$: $\mathbb{E}\left[\text{T}_g\right] = N \times \phi_g \times \alpha_g | \text{vote} $ where $\phi_g$ is the expected probability of voting in cell $g$, and $\alpha_g | \text{vote}$ is the expected probability of voting for Trump for voters in cell $g$
In order to generate $\phi_g$ and $\alpha_g | \text{vote}$, we build two models: a voter turnout model and a vote preference model, respectively. Both models are hierarchical binomial logistic regression models of the form:
Each categorical predictor, $\beta^v$, is represented as a length-$L_v$ vector, where the elements of the vector map to the effect associated with the level $l_v$. $V$ denotes the set of all categorical predictors included in the model and $v[g]$ is a function that maps the $g$-th cell to the appropriate $l_v$-th level of the categorical predictor. For example, $\beta^{\text{state}}$ would be a 50-element vector, and $\text{state}[\,\,]$ is a length-$G$ list of integers with values between 1 and 50 indicating to which state the $g$-th cell belongs. Note that the model above can include one-way effects in $V$, as well as two-way and three-way interactions, like state $\times$ age.
We use rstanarm to specify the voter turnout model and the voter preference model, which uses lme4 syntax to facilitate building complex hierarchical generalized linear models like above. The full model specifications in $\texttt{lme4}$ syntax are given in the Appendix. rstanarm imposes more structure on the variance parameters $\tau_v$ than is typical. In our model, $\tau_v^2$ is the product of the square of a global scale parameter $S$ the $v$-th entry in the simplex parameter $\boldsymbol{\pi}$, and the cardinality of $V$, $|V|$. See rstanarm for more details.
Our voter preference model went through multiple iterations before we arrived at our final model. At first we intended to include past presidential vote. However, PUMS does not include past presidential vote, so we used YouGov's imputed past presidential vote for each PUMS respondent. This induced too much sparsity in our poststratification frame.
After training each of the models, and generating predictions for voter turnout by cell and two-party vote preference for each cell, we adjusted our turnout and vote proportions in each cell to match the actual state-by-state outcomes as outlined ghitza2013deep.
This section presents plots at the county and state level, followed by charts and maps that illustrate the poststratification. In addition to vote intention, the charts and maps also illustrate voter turnout. The county and state level plots use 2016 and 2012 election results and 2010 US census data. The captions of the charts and maps identify which model is used to produce the data illustrated in the figure. The models are defined as follows:
The graphs that follow present actual election results by county and by state. They are not model-based, but rather an examination of the Republican vote proportion swing from 2012 to 2016 by county versus various demographic variables measured at the county level.
The graphs that follow are generated using the multilevel regression and poststratification method outlined in the Methodology section.
We keep the discussion short as we feel that our main contribution here is to present these graphs and maps which others can interpret how they see best, and to share our code so that others can fit these and similar models on their own.
Some of our findings comport with the broader media narrative developed in the aftermath of the election. We found that white voters with lower educational attainment supported Trump nearly uniformly. We did not find that income was a strong predictor of support for Trump, perhaps a continuation of a trend apparent in 2000 through 2012 election data. We found the gender gap to be about 10%, which was a bit lower than predicted by exit polls. The marital status gap we estimated was about 2$\times$ the figure estimated by exit polls.
Most surprising to us was the strong age pattern in the gender gap. Older women were much more likely to support Clinton than older men, while younger women were mildly more likely to support Clinton compared to men the same age. We are not sure what accounts for this difference. One area of future research is using age as a continuous predictor rather than binning ages and using the bins as categorical predictors.
Our models predict that men in several state by education categories were more likely to support Clinton than women. We do not believe this to be true but rather believe it to be a problem with poststratification table sparsity. In order to reduce the number of poststratification cells, in future analyses we could poststratify by region rather than state. This would likely not have impacted our descriptive precision in this analysis due to the apparently strong regional patterns in voting behavior in this election.