Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
48,915 characters · 48 sections · 65 citation commands
Scaling the Queue: Reinforcement Learning for Equitable Call Classification Capacity in NYC Municipal Complaint Systems
Municipal complaint systems are, at their core, classification systems. When a resident calls 311 or files a digital complaint, the intake process must assign each contact to one of the action categories: route to an inspector, batch with similar cases, escalate to emergency response, or defer. This assignment happens under time pressure and with incomplete information. The result is not random error but systematic misprioritization: 311 data reveal persistent gradients in complaint classification accuracy and response assignment times across income and racial lines glaeser2016. Low-income communities whose complaints are harder to parse, less likely to include structured data, and filed through channels that yield noisier intake records are systematically mis-routed---assigned to low-priority queues, batched when they should be escalated, or simply left unclassified.
A compounding factor is that residents do not file complaints at equal rates. Liu, Bhandaram, and Garg liu2023nature show that even after controlling for incident characteristics, some NYC neighborhoods report the same problem three times faster than others, with reporting speed positively correlated with income, college attainment, and the fraction of white residents. Because complaint volume is the primary signal feeding current intake classifiers, this disparity in participation directly biases which complaints receive timely escalation. This happens not because the classifier is broken, but because the data it consumes encodes who was motivated and able to file in the first place. This missing-not-at-random structure is a form of measurement injustice embedded in the data itself dignazio2023. It means that any algorithm trained naively on these records inherits, and risks amplifying, the blind spots of its predecessor.
The central claim of this paper is that this dynamic is not inevitable. Reinforcement learning (RL) systems can be configured to optimize an explicitly specified reward function. As a result, RL offers a lever that heuristic intake workflows do not: the ability to inscribe equitable classification coverage as a first-class objective. Whether that lever is used depends on whether equity is built into the reward from the outset. Our paper demonstrates what happens when it is, and what is lost when it is not.
New York City's Department of Buildings processes tens of thousands of complaints annually, spanning boiler defects, scaffold hazards, heat failures, and illegal construction activity. Each complaint must be classified into an action tier before any inspector can be dispatched. Classifier capacity is finite; misclassifications cascade: a complaint assigned to "defer" when it should be "escalate" may never receive a field visit until the hazard worsens. Current intake heuristics are largely reactive and rule-based, ignoring temporal recurrence patterns, neighborhood-level risk accumulation, and the compounding effects of delayed classification on housing stability.
The heterogeneous participation problem identified by liu2023nature is directly operational: the NYC DOB draws on the same 311 infrastructure that the participatory sensing framework burke2006 anticipated as a tool for civic data collection at an urban scale. The challenge is that participatory systems aggregate data from citizens who participate unequally, producing dense, high-quality records in engaged neighborhoods and sparse, noisy records in disengaged ones. Augmenting classification capacity with RL agents that learn from this uneven historical record is the secondary technical motivation of our work; the primary motivation is ensuring that expanded capacity does not come at the cost of equity.
This paper addresses three inter-related research questions:
The ordering is intentional. Capacity is a necessary condition for deployment credibility, but equity is the main focus of this work.
A foundational body of scholarship documents the mechanisms by which public institutions reproduce rather than remedy spatial inequality. Rothstein rothstein2017 traces how federal, state, and local housing policy encoded racial segregation into the built environment; the legacy of those decisions structures which neighborhoods today have aging boiler stock, substandard heating systems, and chronic building violations. Desmond desmond2016 demonstrates that housing instability is not merely a consequence of poverty but a cause of it, making rapid classification and escalation of housing violations a direct lever on economic mobility. This structural context establishes why the distributional consequences of classification algorithms are not incidental technical details. They are the core policy question.
The deployment of predictive and optimization tools in public-sector settings has generated a substantial interdisciplinary literature on how automation can entrench disadvantage. Eubanks eubanks2018 documents how automated systems in welfare, child protective services, and homeless services frequently reproduce the disadvantages of those they purport to serve, a phenomenon she terms the "digital poorhouse." Chouldechova chouldechova2017 formalizes the incompatibility of common fairness criteria in recidivism prediction, a mathematical result that generalizes to any classification setting with asymmetric base rates. In the housing domain specifically, prior work has examined how algorithmic triaging of code enforcement complaints can systematically underserve low-income renters obrien2017, and how predictive systems in child welfare can generate racially disparate referrals even when race is excluded as an explicit feature brown2019.
The 311 complaint infrastructure that feeds our classification system is an instance of participatory sensing at a civic scale, a paradigm introduced by Burke, Estrin, and colleagues burke2006 to describe systems in which everyday mobile devices collectively constitute an urban sensor network that enables data collection at spatial and temporal scales that agency-operated sensors cannot match. The foundational insight of that work is that routing data collection through ordinary citizens dramatically expands coverage. At the same time, burke2006 recognized that the credibility, quality, and shareability of participatory data depend critically on who participates and under what conditions. This is a tension the municipal complaint context makes acute.
Estrin's subsequent research on small data estrin2014 sharpens this concern: individual digital traces are dense for some users and sparse for others, and systems that treat the aggregate record as representative of ground truth without interrogating participation rates risk systematically misrepresenting the populations least likely to generate dense traces. Applied to civic complaint systems, this framing directly motivates our treatment of complaint volume as a potentially biased signal of underlying housing need rather than an objective measure of it. Our equity reward term (Equation (ref)) and our use of ACS covariates to proxy for under-reporting propensity are operationalizations of this principle.
A line of work from liu2023nature develops rigorous methods to measure and correct the heterogeneous reporting biases that distort municipal complaint data before any classification model sees it. Liu, Bhandaram, and Garg liu2023nature develop a statistically identified estimator for spatially varying reporting delays in resident crowdsourcing systems. Their key insight is that the rate at which duplicate reports arrive about the same incident can be leveraged to separate under-reporting from a genuine absence of incidents, a fundamental identification challenge because unreported incidents leave no trace in the record. Applied to over 100,000 NYC Department of Parks and Recreation reports and over 900,000 Chicago reports, the method uncovers substantial socioeconomic disparities in reporting speed, even after controlling for incident characteristics.
This finding has direct consequences for our classification system. The equity parity term in Equation (ref) normalizes escalation throughput by complaint volume, which means it inherits reporting-rate disparities unless those disparities are corrected upstream. The Liu et al.\ liu2023nature correction uses duplicate-report rates to identify neighborhood-level reporting propensity. We use the correction as a recommended preprocessing step before reward calibration and flag complaint-rate correction as a priority extension (Section (ref)).
Agostini, Pierson, and Garg agostini2024 extend this line with a Bayesian spatial model that jointly uses government inspection ratings (unbiased but sparse) and crowdsourced reports (dense but biased) to estimate the latent true state of urban incidents. Their GNN-based multiview model, applied to over 9.6 million NYC crowdsourced reports and 1 million inspection ratings across 139 incident types, shows that higher-income neighborhoods report problems at systematically higher rates. This complicates any classifier that treats complaint volume as a reliable signal of ground-truth risk. Our state representations include complaint recurrence and neighborhood-level complaint frequency precisely because, as Agostini et al.\ demonstrate, these features carry incremental signal about the latent incident state beyond what raw volume conveys.
Liu and Garg liu2024sla address the downstream allocation problem complementary to ours: given heterogeneous reporting and service patterns, how should a city design Service Level Agreements (SLAs) to jointly optimize efficiency and equity? The SLAs are essentially promises that incidents will be responded to within a specified time. Modeling the problem as an optimization over a queuing network, liu2024sla show that the "price of equity" is small in realistic settings, that inefficient policies tend to inequitable and vice versa, and that status quo NYC inspection scheduling is dominated by optimized policies on both dimensions. Their empirical finding that equity and efficiency are largely complementary rather than conflicting directly corroborates our Pareto frontier results (Section (ref)): the speed penalty for adding an equity objective to our classification reward is modest (4--7%), consistent with Liu and Garg's theoretical bound on the price of equity. Their queuing-network framework and our MDP framework are complementary tools for the same policy problem. liu2024sla analyze the design of response-time commitments, while our model learns the per-complaint classification decisions that determine whether those commitments are met equitably across neighborhoods.
D'Ignazio and Klein dignazio2023 introduce the concept of measurement injustice: the harm that arises when the metrics used to quantify a problem encode the power dynamics that produce it. The Garg et al.\ program on under-reporting provides a quantitative instantiation of this concept for the municipal complaint context. Complaint counts are not a neutral measure of housing risk but a measure of complaint-filing capacity, which correlates strongly with income, language access, and trust in government institutions liu2023nature. NYC's 311 system has well-documented inequities in complaint rates and response assignment times across demographic lines glaeser2016. Our use of SHAP to audit the influence of neighborhood statistics on learned classification decisions is a direct application of the measurement justice lens: if the model's most important features correlate with protected attributes, the classification policy's equity implications must be confronted explicitly.
Multi-objective RL has been studied formally by Roijers et al. roijers2013, who characterize the space of Pareto-optimal policies and the conditions under which scalarized reward functions can recover them. Policy-gradient methods such as REINFORCE williams1992 and their variance-reduction extensions via learned baselines have been applied to sequential resource-allocation problems in public health killian2019 and disaster response pettet2021. DQN mnih2015 has been applied to inspection and maintenance scheduling, though typically without explicit equity constraints. The closest antecedent is the literature on restless multi-armed bandits for health intervention scheduling killian2019, which has begun to incorporate fairness constraints; we extend this orientation to the complaint classification domain.
To the best of our knowledge, no prior work (i) applies RL to multi-domain municipal complaint classification with equity as a primary design objective, (ii) conducts a formal demographic equity audit of learned classification policies disaggregated by both income and racial composition, (iii) characterizes Pareto frontiers that explicitly include equitable classification coverage alongside operational throughput, or (iv) grounds reward function design in a participatory process involving affected community stakeholders. The present paper addresses each of these gaps, building on Garg et al.'s empirical documentation of complaint-rate disparities liu2023nature,agostini2024 and Liu and Garg's complementary analysis of SLA design liu2024sla, and situating the RL framework in Estrin's participatory sensing lineage burke2006,estrin2014 to account for the structural inequalities in civic data collection that any classification system must correct rather than inherit.
The New York City Department of Buildings enforces the NYC Construction Codes and Zoning Resolution across approximately one million buildings. Before any inspector is dispatched, each incoming complaint must be classified: assigned a priority tier, an action category, and a routing queue. This classification step is chronically under-resourced. Current intake relies primarily on complaint severity codes and first-in-first-out queue ordering, with limited use of historical outcome data. The result is a throughput bottleneck and a systematic misprioritization of complaints from neighborhoods with noisier, lower-volume, or non-English intake records.
\paragraph{Boiler Safety.} Sourced from the NYC DOB Safety Boiler dataset (834,338 records, $\approx$8% defect rate). The agent observes a seven-feature vector---is_recurrent, is_high_pressure, neighborhood_risk, is_internal, has_lff_45, has_lff_180, and \texttt{boiler_count_norm}---and classifies each complaint as \textsc{Defer} or \textsc{Inspect}. Split 80/20 ($\approx$667K train, $\approx$167K test).
\paragraph{Crane and Derrick Complaints.} The classifier assigns each complaint to Do Nothing, Routine Inspection, Immediate Inspection, or Stop-Work Order. Five state features: new complaint flag, recurrence flag, neighborhood complaint frequency, backlog, and hazard level. 80/20 chronological split.
\paragraph{Heat and Hot Water Complaints.} A simulated agency classifies up to four complaints per day for inspector dispatch. The environment resolves each assignment into one of five outcome categories using empirically derived probabilities.
\paragraph{Housing Complaint Triage.} NYC 311 housing complaints aggregated at an hourly level, with state features including backlog, average close time, lagged complaint counts, rolling averages, and recurrence ratios.
\paragraph{Scaffold Safety.} Hourly transitions at the entity-identity level. Five engineered state features: created_count, open_count, recurrent_7d, zip_freq_24h, and time_since_last_resolution_h. Five action categories: Inspect Now, \textsc{Delay}, \textsc{Batch}, \textsc{Escalate}, \textsc{Ignore}. Temporal split: train January 2020--2024, validation 2024--2025, backtest 2025--March 2026.
\paragraph{Natural Area Districts (SNAD).} Parks Department records over 128 states (4 neighborhoods $\times$ 4 complaint levels $\times$ 2 recurrence flags $\times$ 4 seasons) with three actions: Defer, Inspect, Prioritise. Evaluated on 87 historical complaints filed in 2024--2026.
\paragraph{Elevator Complaints.} 103,598 records across 59 Community Boards (2020--2025). Approximately 60% of cases close with no action; only $\approx$20% lead to enforcement, making correct classification of actionable complaints the primary capacity challenge.
For each domain with residential implications (boiler, heat, housing), we augment complaint and outcome records with American Community Survey (ACS) variables at the census-tract level: median household income quintile, racial and ethnic composition, housing age, and renter-occupied housing share. Following Agostini, Pierson, and Garg agostini2024, we additionally use ACS language-access and educational-attainment variables as proxies for neighborhood-level reporting propensity, enabling a first-order correction to the complaint-volume normalization in our equity reward term.
Three structural limitations motivate our feedback-loop analysis in Section (ref). First, classification outcome records are missing not at random: complaints in historically underserved areas have systematically fewer recorded defect outcomes. Second, 311 complaint rates vary with digital literacy, language access, and trust in government---disparities that Liu, Bhandaram, and Garg liu2023nature quantify precisely for the NYC Parks complaint system. Third, temporal gaps in records create state-estimation uncertainty that the MDP formulation must accommodate.
We model each complaint classification domain as a discrete-time Markov Decision Process $\mathcal{M} = (\mathcal{S}, \mathcal{A}, P, R, \gamma)$, where $\mathcal{S}$ encodes the complaint queue, neighborhood statistics, and capacity indicators; $\mathcal{A}$ is the set of classification categories; $P$ is transition dynamics estimated from historical data; $R$ encodes multi-objective trade-offs; and $\gamma \in (0,1)$ is the domain-specific discount factor. The agent maximizes:
\paragraph{Procedural vs.\ substantive equity.}
Procedural equity requires that escalation capacity be allocated in proportion to housing need, not to complaint-filing capacity. liu2023nature showed that the complaint volume is a function of digital literacy, language access, and institutional trust, all of which correlate with income and race. A throughput-maximizing classifier is procedurally inequitable even when it treats each individual complaint identically. Substantive equity additionally requires that outcomes do not systematically differ across demographic groups. Our revised equity objective operationalizes both requirements by correcting for heterogeneous reporting propensity before normalizing escalation throughput.
\paragraph{The case against treating equity as a constraint.} Following green2021, we treat equitable coverage as a co-equal reward objective rather than a side constraint. liu2024sla showed empirically that for municipal service allocation, the price of equity is small and that inefficient policies tend to be inequitable; this makes the Pareto frontier between efficiency and equity less adversarial than conventional formulations imply.
Before defining the equity reward term, we introduce the reporting-rate correction that must be applied to complaint-volume denominators. Raw complaint counts $C_{g,t}$ in demographic group $g$ at time $t$ conflate true incident prevalence with filing propensity. We separate the two using the duplicate-report estimator of liu2023nature and the Bayesian spatial model of agostini2024.
\paragraph{Notation.} Let $\mathcal{G} = \{\text{low}, \text{high}\}$ index two neighborhood strata defined by census-tract income quintile (quintiles 1--2 vs.\ 4--5). For each census tract $z$ and time window $t$:
\paragraph{Step 1 — Estimate reporting propensity.} Following liu2023nature, the duplicate-report rate identifies $\rho_{z,t}$ without external ground-truth data. Under the assumption that the first and subsequent reports of the same incident arrive independently at rate $\rho_{z,t}$, the expected fraction of duplicate reports satisfies
giving the moment estimator
When duplicate-report records are unavailable, $\hat{\rho}_{z,t}$ is proxied by a linear combination of ACS covariates (median household income, share with college attainment, share of English-proficient residents, renter-occupied housing share) fit on tracts where duplicates are observed:
where $\sigma(\cdot)$ is the logistic function and $\mathbf{x}_z$ is the tract-level ACS covariate vector.
\paragraph{Step 2 — Estimate latent incident count.} The corrected incident-count estimate is
consistent with agostini2024, who show that higher-income tracts report at systematically higher rates, so that $\widehat{I}_{z,t} > C_{z,t}$ selectively for low-income tracts.
\paragraph{Step 3 — Aggregate to group level.} For each stratum $g \in \mathcal{G}$, aggregate over member tracts $\mathcal{Z}_g$:
$\widehat{N}_{g,t}$ replaces the raw complaint count $N_g$ in the equity reward denominator (see Section (ref)).
The full reward at step $t$ is
with $\alpha_i \geq 0$ and $\sum_i \alpha_i = 1$. The speed, cost, and retention terms are unchanged from Section 4.3 of the original formulation. The equity term is revised as follows.
\paragraph{Original (biased) equity term.} The formulation in Equation (6) of the submitted draft normalises escalation throughput by observed complaint counts:
where $n_t^{g}$ is the number of correct escalations assigned to stratum $g$ at step $t$ and $N^{g}$ is the total complaint count for that stratum. Because $N^{\text{low}} < \widehat{N}^{\text{low}}$, this term overstates the per-incident escalation rate in low-income tracts, masking under-service.
\paragraph{Corrected equity term.} Replacing raw counts with corrected incident estimates yields:
where $\widehat{N}_{t}^{g}$ is computed via Equations (ref)--(ref) on a rolling window of width $\Delta t$ (default: four weeks) ending at step $t$.
\paragraph{Interpretation.} $r_t^{\text{equity}} = 0$ when correct escalations are allocated in proportion to estimated housing need, not to complaint volume. An agent maximizing Equation (ref) with $\alpha_3 > 0$ is therefore incentivized to escalate proportionally more complaints from low-reporting tracts, directly counteracting the participation gap documented by liu2023nature.
\paragraph{Multi-group extension.} For an audit across $K > 2$ income quintiles or racial-composition strata $\{g_1, \ldots, g_K\}$, the pairwise absolute difference is replaced by the maximum disparity:
or, when a smoother gradient signal is preferred during training, the population-weighted variance:
where $\bar{e}_t = \frac{1}{K}\sum_k n_t^{g_k}/\widehat{N}_t^{g_k}$ is the mean per-incident escalation rate across strata.
Before each training cycle, the following preprocessing pipeline is applied:
\paragraph{Sensitivity analysis.} Because $\hat{\rho}_{z,t}$ is estimated with sampling error, we propagate uncertainty into the equity reward via parametric bootstrap: for each retraining cycle, draw $B = 200$ samples of $\hat{\rho}_{z,t}$ from its asymptotic normal distribution (variance derived from the delta method applied to Equation (ref)), recompute $\widehat{N}_{t}^{g}$, and report the 5th--95th percentile interval of the resulting equity-reward distribution alongside point estimates.
\paragraph{DQN.} Two-hidden-layer network (64 or 128 units, ReLU), trained via experience replay and a target network mnih2015. CQL penalty ($\alpha=0.1$) applied in the elevator domain.
\paragraph{REINFORCE with Learned Baseline.}
Joint optimization of policy and value heads with Adam; $\gamma \in \{0.95, 0.99\}$ depending on domain williams1992.
\paragraph{HDQN.} $2\times256$ hidden units; selects complaint category and then a specific complaint. It does not outperform a flat random policy in the heat domain when outcome probabilities are homogeneous.
\paragraph{Tabular TD Q-Learning.} $Q(s,a) \leftarrow Q(s,a) + \alpha[r + \gamma \max_{a'} Q(s',a') - Q(s,a)]$, with $\alpha=0.1$, $\gamma=0.95$, $\epsilon$-greedy decay $1.0 \to 0.05$.
\paragraph{Behavioral Cloning (BC).} Most common logged action per discretized state; it estimates the historical classification policy.
We apply SHAP lundberg2017 via KernelSHAP (boiler), GradientExplainer (crane/derrick), exact Shapley values (SNAD), TreeExplainer (heat, housing), and permutation-based values (drone, electrical revisit). Geographic features such as borough, Community Board, and ZIP-code frequency are flagged as equity-sensitive, consistent with Agostini, Pierson, and Garg agostini2024: because higher-income neighborhoods both report more and receive more inspections, geographic variables in models trained on historical records will tend to encode socioeconomic status as much as genuine incident prevalence.
Balanced (current NYC DOB heuristic), Tail, Emerg, Actionable, Uniform, No_Dispatch, Random Policy, Always Inspect / Always Defer, and rule-based domain-expert heuristics.
Strict chronological split: train on 2020--2024, evaluate on 2025. Boiler domain uses an 80/20 random split of 834,338 records. Elevator evaluation: 168-step horizon ($\approx$4 weeks at 4-hour cadence), frozen five-seed benchmark. Heat REINFORCE: 20 training episodes, 365-timestep test year. Scaffold and elevator: support-constrained backtest ($\texttt{support\_min\_count}=1$).
\paragraph{Heat and Housing.} REINFORCE accumulates cumulative reward $\approx$8,800 over 365 timesteps vs.\ $\approx$2,100 for HDQN and random ($4\times$ improvement). REINFORCE closes $\approx$101 violations by day 365 vs.\ 84 (HDQN) and 80 (random)---a 20--27% improvement.
\paragraph{Boiler Safety.} Table (ref) reports test-set performance (167,379 records). DQN outperforms REINFORCE by \$1.28M in reward and 4.2 pp in recall. Both adopt an aggressive escalation stance: REINFORCE recall 94.5%, DQN 98.7%. Low precision ($\approx$22%) is rational given the 26:1 cost asymmetry.
\paragraph{Elevator.} Table (ref) reports the full baseline comparison. Balanced remains the strongest practical heuristic (mean reward $-446\text{k}$); DQN: $-459{,}119 \pm 4{,}379$; REINFORCE: $-463{,}117 \pm 10{,}482$.
\paragraph{SNAD.} DQN ($+1.046$) and REINFORCE with baseline ($+1.011$) are nearly identical; both classify 85 of 87 complaints actionably and secure 7 enforcements, outperforming all heuristics.
Boiler grid search over 128 reward configurations shows the detection rate monotonically increasing from 0.71 ($\rho=5$) to 0.94 ($\rho=50$). Heat domain Pareto analysis identifies Accuracy+Fairness as the most practically attractive configuration. Cross-domain Pareto analysis confirms that the current heuristic intake is dominated by RL policies in all residential domains—consistent with Liu and Garg's liu2024sla finding that the status quo NYC inspection scheduling is dominated on both efficiency and equity simultaneously.
The current DOB heuristic exhibits a statistically significant gradient in escalation-assignment rates across income quintiles (Q1 receives fewer correct escalations per complaint than Q5, even after conditioning on complaint severity) and by racial composition (tracts with higher fractions of Black and Hispanic residents receive fewer escalations per complaint). REINFORCE with $\alpha_3 > 0$ approaches parity across income quintiles at a throughput cost of $\approx$4--7%---corroborating Liu and Garg's liu2024sla theoretical result that the price of equity is bounded and modest in realistic municipal settings. DQN without the equity term narrows but does not close the gap.
Community Board 12 Manhattan (Washington Heights / Inwood; predominantly Latino, lower-income) ranks among the top five most influential SHAP features in the REINFORCE elevator policy, driven by high unresolved burden and historical actionability. This prominence illustrates the concern raised by Agostini, Pierson, and Garg agostini2024: CB-level features encode demographic characteristics as much as incident prevalence, requiring ongoing audit.
Outcome records are generated by the incumbent classification policy: complaints in areas that received fewer escalations have sparser defect records, not because they are safer, but because they were never escalated. This missing-not-at-random structure is an instance of the participation-gap problem that Estrin's participatory sensing framework burke2006 identified as a structural risk in any civic sensing system that conflates coverage with participation. Small data estrin2014 amplifies the concern: digital traces are densest for those who already participate the most, creating a compounding bias in historical training data.
Liu, Bhandaram, and Garg liu2023nature provide a principled method to quantify this gap without external ground-truth data. Their duplicate-report estimator applied to DOB complaint records could identify neighborhoods where complaint rates are suppressed relative to true incident rates, enabling a reporting-rate correction to the complaint-volume normalization in Equation (ref). We flag this as the highest-priority extension of the present work. Sensitivity analysis via propensity-score reweighting rosenbaum1983 shows that the RL performance advantage is robust to moderate selection bias; the equity gains are more sensitive.
Once deployed, the classification policy generates the outcome records used to retrain future model versions, a feedback loop that can progressively shrink coverage in areas receiving the fewest escalations. We propose three mitigations: (1) exploration bonuses for census tracts with sparse recent escalation histories; (2) periodic counterfactual escalation of a random sample of low-priority complaints; (3) regular demographic audit checkpoints triggering retraining if disparity metrics exceed a pre-specified threshold.
Liu and Garg's SLA framework liu2024sla offers a complementary institutional lever: redesigning response-time commitments to require minimum service rates in low-income tracts provides a structural backstop against feedback-loop- driven under-service that does not depend on the RL agent's reward signal alone.
Results should be interpreted as decision-support prototypes. The complaint triage domain's learned classifiers fail to transfer to held-out 2020 data, arguing against deployment without prospective operational validation.
Input from NYC DOB intake operations staff directly informed MDP state and action space design. Practitioners identified the distinction between "classify now," "batch," and "escalate" as operationally meaningful.
We engaged tenant advocacy organizations in the South Bronx and Central Brooklyn. Key themes included frustration with complaint acknowledgment without visible follow-up and skepticism about opaque algorithmic systems. Estrin's Public Interest Technology framework and participatory sensing research burke2006, estrin2014 inform our position that affected communities should be active co-designers of civic algorithmic systems. These perspectives informed our inclusion of housing unit retention as an explicit objective and our recommendation for interpretable classification explanations available to complainants.
First, equity does not come for free. Agents trained without explicit equity objectives narrow but do not eliminate historical classification disparities. Closing the gap requires the inclusion of the coverage parity term with $\alpha_3 > 0$.
Second, the equity cost is modest and quantifiable. Agents with full equity objectives achieve near-parity at a throughput penalty of approximately 4--7%, consistent with Liu and Garg's liu2024sla theoretical bound on the price of equity and providing RL-specific empirical confirmation that the Pareto frontier between efficiency and equity is less adversarial than the framing of equity-as-constraint implies.
Third, no single algorithm dominates. REINFORCE outperforms in the heat domain and water drainage; DQN produces more stable policies in elevator, complaint triage, and ERT dispatch. Both methods independently converge to the identical Brooklyn-dominant policy in the drone domain.
Fourth, classification quality is bounded by data quality. The participation-gap literature liu2023nature, agostini2024,burke2006,estrin2014 makes clear that 311 complaint data reflects who participates, not only what happens. Future work must integrate upstream reporting-rate correction to ensure that reward signals calibrated on complaint volume reflect true incident prevalence rather than filing propensity.
The Pareto frontier provides a decision-support tool: a family of policies along the efficient frontier, each corresponding to a different weighting of throughput, cost, and equity. The choice of where to operate is a normative one that belongs to the agency and the communities it serves. Liu and Garg's liu2024sla result is parallel to ours: the status quo NYC inspection scheduling is dominated by both efficiency and equity simultaneously.
The MDP framework is domain-agnostic and can be re-instantiated for any municipal classification domain with complaint intake, action routing, and binary outcome records. The Garg et al.\ under-reporting correction tools have already been applied to both NYC and Chicago 311 data liu2023nature, suggesting that the data pipeline prerequisites for equitable classification exist at scale beyond New York.
This paper begins and ends with a distributional claim: the harms that municipal complaint classification systems are designed to prevent are not randomly distributed, and neither are the benefits of the algorithmic systems deployed to improve classification capacity. Any expansion of that capacity that does not explicitly ask who benefits and who is left in a lower-priority queue is not neutral. It is a choice to accept and perpetuate the existing distribution.
Our principal finding is that RL systems can close the classification gap, but only when equity is inscribed in the reward function as a first-class objective. Agents optimized for throughput alone narrow but do not eliminate historical disparities. Agents with explicit coverage parity objectives achieve near-parity while outperforming current heuristic classification on defect detection and remediation speed. The equity cost is modest and quantifiable, a finding anticipated by Liu and Garg's liu2024sla theoretical analysis and confirmed here in the RL setting.
Our feedback-loop analysis adds a further imperative: the missing-not-at-random structure of historical complaint records, which Garg and collaborators have characterized precisely for NYC civic data liu2023nature,agostini2024, means that classification systems deployed without equity objectives actively degrade equity over successive training rounds. The participatory sensing paradigm burke2006 that underlies 311-style systems was designed to democratize urban data collection; realizing that aspiration requires correcting for heterogeneous participation rates before those rates are allowed to shape classification decisions.
Configuring reward functions for public-sector RL systems is an act of normative policymaking. It encodes whose harms count most, how competing objectives should be weighted, and what trade-offs are acceptable. These are political and ethical choices that belong to the communities affected by them.