The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
59,664 characters
OptOrangeOpt blackScientist: Automated Discovery of Optimization Laws
\vspace*{-0.8cm}
\maketitle
\begin{abstract}
Many scientific phenomena are best understood as outcomes of underlying optimization problems, yet automated scientific discovery has largely focused on recovering equations that describe observed input–output patterns. We introduce
{\color{MoBlue}\oldtextbf{OptScientist}}
, a novel automated framework for discovering compact symbolic optimization laws directly from data. OptScientist comprises (i) a model that infers symbolic optimization problems by first learning to represent data as an optimization problem and then extracting its symbolic form, and (ii) an LLM agentic system embedding the model in a discovery process that integrates theoretical priors from scientific literature, discovers candidate laws, abstracts their underlying mechanisms, and suggests new data to collect. We first evaluate OptScientist on \emph{OptFeynman}, a novel benchmark of 99 physics-derived optimization problems, where OptScientist recovers ground-truth laws for over 80\% of the problems and remains robust to noisy contextual information, while substantially outperforming machine-learning, symbolic, and LLM-based model-discovery baselines. On human risky choice and central-bank monetary policy, OptScientist discovers compact symbolic optimization laws that achieve held-out predictive performance competitive with leading baselines, while using few parameters and substantially fewer tokens than LLM baselines. Overall, OptScientist extends automated scientific discovery to uncovering optimization laws that generate observed phenomena, enabling automated theory building across the physical, biological, behavioral, and social sciences.
\end{abstract}
\section{Introduction} \label{sec:introduction}
Across the sciences, many influential theories characterize observed phenomena as solutions to optimization problems. Examples include energy or free-energy minimization principles for characterizing equilibrium states in statistical physics \citep{gibbs1902elementary, landau2013statistical}, optimality models for explaining how biological fitness shapes evolutionary outcomes \citep{wright1932roles, fisher1999genetical}, rational and resource-rational analyses of human behavior and cognition \citep{samuelson1948foundations, debreu1959theory, anderson1991adaptive, lieder2020resource}, and intertemporal optimization for modelling dynamic macroeconomic systems \citep{ramsey1928mathematical, ljungqvist2018recursive}. Developing such theories requires identifying objectives and relevant constraints that jointly account for observed outcomes. However, these components are typically formulated by domain experts and only subsequently evaluated against data, making optimization-based theory construction largely manual and dependent on specialized insight. This reliance on expert formulation creates a substantial bottleneck for theory discovery across scientific domains. A fundamental question therefore remains: \textit{Can optimization laws be discovered automatically from observational data?}
The complexity of real-world scientific data further exacerbates this bottleneck, as observations are often noisy, high-dimensional, and shaped by complex interactions. Hand-crafted interpretable models may fail to capture complex patterns, whereas flexible data-driven approaches, particularly machine-learning methods, have demonstrated strong predictive performance across physics \citep{baldi2014searching,carleo2017solving}, biology \citep{poplin2018universal,jumper2021highly}, and behavioral and social sciences \citep{kosinski2013private,peterson2021using,zhu2025capturing}.
Recent advances in scientific machine learning have enabled data-driven discovery of symbolic laws \citep{champion2019data,udrescu2020ai,cranmer2020discovering}, embedded mathematical structure into flexible ML models \citep{chen2018neural,liu2025kan,ma2026behavior}, and integrated scientific theories with machine learning \citep{raissi2019physics,peterson2021using}. However, these approaches have largely focused on recovering input–output equations or learning models without explicit symbolic structure, while optimization-structured scientific ML models \citep{ma2026behavior} still require human interpretation of optimization structures and lack an end-to-end process for automated theory discovery. Consequently, automatically discovering compact symbolic optimization laws from real-world data remains a fundamental challenge.
To address these challenges, we introduce
{\color{MoBlue}\oldtextbf{OptScientist}}
, \textit{a novel framework that uses AI methods to automate the discovery of optimization laws.} OptScientist has two components: a novel symbolic-discovery model
and an agentic research system. First, the symbolic-discovery model (\emph{OptScientist Model}) learns to represent observational data as an optimization problem using a hierarchical optimization network with an optimization representation bottleneck, enabling denoising, information compression, and automatic extraction of interpretable features. It then distills this learned optimization representation into a compact symbolic optimization problem. The second component, the \emph{OptScientist Agent}, places this discovered symbolic model inside an
automated scientific-discovery loop. A Theory Prior Agent
proposes scientific priors, candidate variables and mechanism hypotheses. An Experiment Agent then uses the OptScientist Model to search for compact symbolic optimization laws from data. Finally, a Scientific Critic and Theory Abstraction Agent evaluates and selects the discovered laws, abstracts the selected laws into a general form and qualitative mechanisms, and suggests new variables or data for the next round of discovery. Together, the model and agent form an iterative system for proposing, discovering and refining optimization laws.
We conduct three experiments spanning physical systems, human behaviour, and macroeconomic policy. First, to test whether the core OptScientist Model can recover ground-truth optimization laws (Sec.~\ref{sec:expr-ground-truth}), we introduce \emph{OptFeynman} (Sec.~\ref{sec:optfeynman}), a novel synthetic benchmark of 99 optimization problems derived from analytic physics equations, where observations are generated as noisy samples from the induced optimization landscapes. On held-out contexts, OptScientist Model shows strong recovery of the generating laws, achieving perfect exact-order recovery on 88 of 99 discrete problems and 78 of 99 continuous problems, while substantially outperforming machine-learning, symbolic-regression, and LLM model-discovery baselines. Our results also suggest that OptScientist remains robust even when context variables are noisy. We then apply OptScientist to two autonomous scientific-discovery tasks on real-world data for which the underlying laws are unknown. In human decisions under risk and ambiguity (Sec.~\ref{sec:expr-human-choice}), OptScientist discovers an optimization law combining expected reward, cognitive switching cost and counterfactual reward. In central-bank monetary decisions (Sec.~\ref{sec:expr-central-bank}), it discovers a law in which observed policy tracks a latent subjective target through squared-error minimization. Across both domains, the discovered laws remain competitive with the strongest
baselines while using near-minimal parameter counts and symbolic complexity,
and substantially fewer output tokens than LLM-based model-discovery methods.
Overall, our contributions are threefold. First, we introduce the OptScientist Model, which, to the best of our knowledge, is the first scientific machine-learning framework for learning symbolic optimization problems directly from observational data. Second, for science, we develop OptScientist, a hybrid symbolic--LLM system that extends automated scientific discovery to optimization-law discovery in domains associated with optimization models. Third, at the paradigm level, we contribute to inverse optimization, revealed preference and related fields by moving beyond parameter estimation for pre-specified optimization problems, providing a new methodology for directly recovering optimization laws themselves from data.
\section{Related Work} \label{sec:literature}
{\color{MoBlue}\oldtextbf{Symbolic discovery.}}
Recent advances in symbolic regression have enabled symbolic law discovery from data. Most existing approaches, however, recover input--output laws, such as \(y=f(x)\) \citep{koza1994genetic,schmidt2009distilling,udrescu2020ai}, or physical equations expressed through differential relationships, such as \(\frac{d x}{d t} = f(x,t)\) \citep{brunton2016discovering,rudy2017data,champion2019data}. They are less suited to settings where the scientific law is itself an optimization problem. OptScientist learns symbolic optimization problems from observational data, providing a new symbolic-discovery model for scientific domains associated with optimization models.
{\color{MoBlue}\oldtextbf{LLM-based automated scientific discovery.}}
LLMs have been used to propose and refine symbolic functional forms or governing equations
\citep{merler2024context,du2024llm4ed,shojaee2025llm}, and to automate broader scientific workflows such as hypothesis generation, experimentation and iterative refinement
\citep{lu2026towards,gottweis2026accelerating,aygun2026ai}. Recent LLM-based work also searches for symbolic utility expressions to support group-level prediction \citep{zhao2026personalized}, but rarely recovers parsimonious scientific laws. Despite their broad role in scientific automation, these systems are not designed to discover compact symbolic optimization problems from data. OptScientist does not remove LLMs entirely. Instead, it suggests that greater efficiency and performance can be achieved by combining a symbolic discovery model with an auxiliary LLM agentic system to improve and interpret the discovered laws and add semantic meaning to them.
{\color{MoBlue}\oldtextbf{Cognitive discovery.}}
Some cognitive-discovery methods use black-box predictions or structured neural architectures to recover interpretable mechanisms from behavioural data
\citep{agrawal2020scaling,peterson2021using,miller2023cognitive,zhu2025capturing,ji2025discovering,ma2026behavior},
while others use LLMs to generate and revise executable models, diagnose theory failures and design informative experiments
\citep{binz2025automated,rmus2026generating,eltetHo2026atlas,jagadish2026closing}.
These approaches recover programs or reduced-form decision rules rather than compact symbolic optimization laws. Moreover, human behaviour is often modelled through resource-rational analysis \citep{debreu1959theory,lieder2020resource,zhu2026computation}, which formulates behaviour using hand-specified optimization models. As a general framework for automated discovery of optimization laws, OptScientist can support automated resource-rational analysis.
{\color{MoBlue}\oldtextbf{Inverse optimization.}}
Inverse optimization \citep{ahuja2001inverse,aswani2018inverse} and related approaches, including revealed preference \citep{afriat1967construction,varian1982nonparametric}, inverse reinforcement learning \citep{ng2000algorithms,ziebart2008maximum}, and optimization-based hierarchical network models \citep{ma2026behavior}, infer latent objectives or estimate the parameters of pre-specified optimization problems. However, they generally assume the functional form of the optimization problems. OptScientist provides a new methodology in which the optimization law itself can be recovered from data without prespecifying its functional form.
\section{The OptScientist} \label{sec:optsci}
\subsection{Formalizing Optimization-Law Discovery}
Generally, a law expressed as optimization describes an outcome $\mathbf{y}$ as the solution to an objective under constraints,
\begin{equation}
\max_{\mathbf{y} \in \mathcal{Y}}\,
U(\mathbf{x},\mathbf{y})
\quad\text{s.t.}\quad
\mathcal{C}(\mathbf{x},\mathbf{y}) \le 0,\;\;
\mathcal{T}(\mathbf{x},\mathbf{y}) = 0 .
\label{eq:ump}
\end{equation}
Here $\mathbf{x}$ denotes the observed context, $U(\cdot)$ is the objective, $\mathcal{C}(\cdot)$ represents inequality constraints, and $\mathcal{T}(\cdot)$ represents equality constraints or target conditions. This form covers a broad class of scientific laws in which observed outcomes arise from maximizing a quantity, possibly under constraints.
For learning, it is useful to express the same object as a single scalar landscape. Under standard local regularity conditions, the constrained problem in Eq.~\eqref{eq:ump} admits a locally exact penalty representation \citep{han1979exact},
\begin{equation}
\max_{\mathbf{y}\in\mathcal{Y}}\;
\lambda_0\,\phi\!\bigl(U(\mathbf{x},\mathbf{y})\bigr)
-\lambda_1^\top \rho\!\bigl(\mathcal{C}(\mathbf{x},\mathbf{y})\bigr)
-\lambda_2^\top \psi\!\bigl(\mathcal{T}(\mathbf{x},\mathbf{y})\bigr),
\label{eq:penalty-ump}
\end{equation}
Here $\phi$ is strictly increasing, $\rho(z)=\max\{z,0\}$ penalizes violated inequality constraints, and $\psi(z)=|z|$ penalizes deviations from equality constraints. This reformulation places objectives and constraints inside one optimization score, making the law easier to analyse and learn.
The outcome generated by an optimization law can be represented in two ways. In the deterministic case, the predicted outcome is \(\widehat{\mathbf{y}}(\mathbf{x})=\arg\max_{\mathbf{y}\in\mathcal{Y}} S(\mathbf{x},\mathbf{y})\), where \(S(\mathbf{x},\mathbf{y})\) is a scalar representation of the underlying optimization problem. For example, under the exact-penalty reformulation in Eq.~\eqref{eq:penalty-ump}, \(S\) is given by the objective together with penalties for violated constraints.
In noisy systems, the same score defines a probability distribution over possible outcomes,
\begin{equation}
p(\mathbf{y}\mid\mathbf{x})
=
\exp\!\left(S(\mathbf{x},\mathbf{y})/\tau\right) / Z(\mathbf{x}),
\label{eq:gibbs-law}
\end{equation}
where $\tau>0$ controls the level of noise and $Z(\mathbf{x})$ is the normalizing term. As $\tau$ decreases, the distribution concentrates on the maximizers of $S$; for larger $\tau$, it describes noisy realizations of the same underlying optimization principle.
Therefore, we formulate the central question of optimization-law discovery as follows.
\begin{question}[Optimization-Law Discovery]
Given observational data
$\mathcal D=\{(\mathbf{x}_i,\mathbf{y}_i)\}_{i=1}^N$
generated by an unknown optimization law
$\mathcal L^*$ of the form
\[
\max_{\mathbf{y}\in\mathcal Y}
U^*(\mathbf{x},\mathbf{y})
\quad\text{s.t.}\quad
\mathcal C^*(\mathbf{x},\mathbf{y})\le 0,\;\;
\mathcal T^*(\mathbf{x},\mathbf{y})=0,
\]
where outcomes arise either as optimal solutions or
as stochastic realizations of the induced optimization
landscape, can we recover a compact symbolic law
\[
\widehat{\mathcal L}
=(\widehat U,\widehat{\mathcal C},\widehat{\mathcal T})
\]
without pre-specifying the functional forms of its
components, such that it reproduces the observable
implications of $\mathcal L^*$?
\end{question}
\begin{figure}[t]
\centering
\includegraphics[width=0.95\linewidth]{graph/arch.pdf}
\caption{
{\color{MoBlue}\oldtextbf{The OptScientist Model.}}
{\color{MoBlue}\oldtextbf{a,}}
The OptScientist Model consists of an Optimization Learner and an Optimization Discoverer. The Optimization Learner learns to represent observational data as an optimization problem using a hierarchical network with an optimization representation bottleneck. The Optimization Discoverer then distills this learned optimization representation into a compact symbolic optimization problem.
{\color{MoBlue}\oldtextbf{b,}}
The OptScientist Model represents the data manifold through a latent optimization network. Raw observations are first compressed and represented by hierarchical optimization blocks. The learned representations are then expressed as a single optimization problem through the optimization representation bottleneck, from which a compact symbolic law is extracted.
{\color{MoBlue}\oldtextbf{c,}}
The model follows the same logic in concrete discovery settings. Noisy real-world observations are first transformed into denoised representations, and symbolic regression is then applied to the learned optimization representation to recover symbolic laws.
}
\label{fig:optscientist}
\end{figure}
\subsection{OptScientist Model}
\label{sec:optscientist-model}
To discover compact optimization laws from data directly, the OptScientist Model consists of two stages: the Optimization Learner (OptLearner; see Figure \ref{fig:optscientist}a Left) and the Optimization Discoverer (OptDiscover; see Figure \ref{fig:optscientist}a Right).
\subsubsection{Optimization Learner}
Observational data rarely reveal the optimization law directly. They may be noisy, high-dimensional, redundant, or expressed in variables that are not yet suitable for symbolic modelling.
The Optimization Learner (OptLearner) addresses this challenge by learning to represent observational data as an optimization problem through a hierarchical optimization network with an optimization representation bottleneck. This process enables denoising, information compression, and automatic feature extraction, producing a learned optimization representation for subsequent symbolic discovery.
{\color{MoBlue}\oldtextbf{Behavior Learning backbone.}}
OptLearner uses the Behavior Learning Network (BL) \citep{ma2026behavior, ma2026arxivbehavior} as its backbone to learn a scalar landscape over context--outcome pairs. For each context $\mathbf{x}$ and candidate outcome $\mathbf{y}$, the network returns a score $S_{\theta}(\mathbf{x},\mathbf{y})$ that measures how strongly $\mathbf{y}$ is preferred under the latent optimization problem. Under Eq.~\eqref{eq:gibbs-law}, observed outcomes are treated as stochastic samples from the learned optimization landscape. Learning therefore reconstructs the latent landscape on which observed outcomes arise as preferred solutions. This corresponds to the representation-learning step in Figure~\ref{fig:optscientist} (a,b).
{\color{MoBlue}\oldtextbf{Optimization representation bottleneck.}}
OptLearner builds on the BL backbone to represent observational data as a single optimization problem. Its early layers denoise and compress the observations, capturing interactions and latent features that may be difficult to express in the original coordinates. The final layer consists of a single BL block, forming an optimization representation bottleneck with learnable objective and optional constraint components, each potentially comprising multiple branches,
\begin{equation}
\max_{\mathbf{y} \in \mathcal{Y}}\,
\underbrace{
U(\mathbf{x},\mathbf{y})
}_{[\lambda_0]}
\quad\text{s.t.}\quad
\underbrace{
\mathcal{C}(\mathbf{x},\mathbf{y})
}_{[\boldsymbol{\lambda}_1]}
\le 0,\;\;
\underbrace{
\mathcal{T}(\mathbf{x},\mathbf{y})
}_{[\boldsymbol{\lambda}_2]}
= 0 .
\label{eq:single-bl-block}
\end{equation}
Here \(\lambda_0\) controls the scale of the objective, while
\(\boldsymbol{\lambda}_1\) and \(\boldsymbol{\lambda}_2\) control the strengths of the inequality and equality terms. This bottleneck provides a structured optimization representation for subsequent symbolic discovery. When the input variables are already meaningful, OptLearner primarily denoises the optimization landscape. For high-dimensional or unstructured inputs, it additionally extracts and compresses relevant features before forming the learned optimization representation.
{\color{MoBlue}\oldtextbf{Training objective.}}
The training objective follows the same principle for discrete and continuous outcomes. For a finite outcome set $\mathcal{Y}$, OptLearner evaluates all candidates and trains the score by cross-entropy,
\begin{equation}
\mathcal{L}_{\mathrm{disc}}
=
-\frac{1}{N}\sum_{i=1}^{N}
\log
\frac{\exp\!\left(S_{\theta}(\mathbf{x}_i,\mathbf{y}_i)/\tau\right)}
{\sum_{\mathbf{y}\in\mathcal{Y}}
\exp\!\left(S_{\theta}(\mathbf{x}_i,\mathbf{y})/\tau\right)} .
\label{eq:discrete-loss}
\end{equation}
For continuous outcomes, enumerating $\mathcal{Y}$ is infeasible. OptLearner therefore compares the observed outcome with sampled alternatives $\widetilde{\mathbf{y}}_{ik}$
using the same contrastive objective,
\begin{equation}
\mathcal{L}_{\mathrm{cont}}
=
-\frac{1}{N}\sum_{i=1}^{N}
\log
\frac{\exp\!\left(S_{\theta}(\mathbf{x}_i,\mathbf{y}_i)/\tau\right)}
{\exp\!\left(S_{\theta}(\mathbf{x}_i,\mathbf{y}_i)/\tau\right)
+
\sum_{k=1}^{K}
\exp\!\left(S_{\theta}(\mathbf{x}_i,\widetilde{\mathbf{y}}_{ik})/\tau\right)} .
\label{eq:continuous-loss}
\end{equation}
Both losses encode the same principle: under the observed context, the observed outcome should receive the highest optimization score.
\subsubsection{Optimization Discoverer}
\label{sec:optdiscover}
The Optimization Learner represents observational data as an optimization problem, but the learned representation is not yet a compact symbolic law. The Optimization Discoverer (OptDiscover) addresses this by extracting the symbolic form of the learned optimization problem, as illustrated in Figure~\ref{fig:optscientist} (a,b). After OptLearner is trained, its parameters are frozen. OptDiscover then distills the components of the optimization representation bottleneck into compact symbolic expressions. This bottleneck consists of three types of components: objective terms \(U_{\theta}\), inequality-constraint terms \(C_{\theta}\), and equality-constraint terms \(T_{\theta}\), with each type optionally comprising multiple branches. OptDiscover searches for a compact symbolic expression for each active component,
\begin{equation}
\widehat{H}_{j}
=
\arg\min_{F\in\mathcal{F}_{\Omega_j}}
\frac{1}{M}\sum_{m=1}^{M}
\left[
F(\mathbf{x}_m,\mathbf{y}_m)
-
H_{\theta,j}(\mathbf{x}_m,\mathbf{y}_m)
\right]^2 ,
\qquad
H_{\theta,j}\in\{U_{\theta},C_{\theta},T_{\theta}\}.
\label{eq:term-wise-symbolic-distillation}
\end{equation}
Here \(\mathcal{F}_{\Omega_j}\) denotes the symbolic search space for the \(j\)-th term.
The distilled expressions of the active components are assembled according to the learned optimization structure, yielding a symbolic optimization problem with explicit objectives, inequality constraints, and equality constraints.
{\color{MoBlue}\oldtextbf{Optional automatic feature extraction.}}
OptDiscover can operate with or without automatic feature extraction. When the observed variables already form a meaningful symbolic language, the law is distilled directly from \((\mathbf{x},\mathbf{y})\). When the raw variables require transformation, interaction, or compression, OptLearner first maps them into denoised intermediate representations. Technically, these representations are the outputs of intermediate layers in OptLearner, taken before the optimization representation bottleneck. These layer-level variables act as automatically extracted features, and OptDiscover then extracts the symbolic law on this learned feature space. This allows the framework to handle both meaningful scientific variables and raw observations whose symbolic structure is more naturally expressed after representation learning.
{\color{MoBlue}\oldtextbf{Complexity penalty.}}
Symbolic complexity is measured by expression-tree size, with each
variable, numerical constant and operator counted as one node. OptDiscover controls symbolic complexity in two ways. First, it selects the simplest expression among candidates with near-minimal distillation loss. This favours parsimonious laws while preserving the learned optimization structure. Second, for the objective term, OptDiscover removes additive components that depend only on \(\mathbf{x}\). These components are constant across candidate outcomes \(\mathbf{y}\) and therefore do not affect the optimizer.
\subsection{OptScientist Agent}
\label{sec:optscientist-agent}
\begin{wrapfigure}{r}{0.55\textwidth}
\centering
\includegraphics[width=0.53\textwidth]{graph/agent_framework.pdf}
\caption{
{\color{MoBlue}\oldtextbf{OptScientist Agent as an iterative research framework.}}
The framework embeds the OptScientist Model in a scientific-discovery loop.
}
\label{fig:optscientist-agent}
\end{wrapfigure}
Symbolic discovery often requires scientific decisions both before and after model fitting: researchers construct meaningful variables, impose search priors, interpret candidate expressions, abstract qualitative mechanisms and decide how subsequent analysis should be revised. To automate this research loop, OptScientist Agent embeds the OptScientist Model in an iterative framework for optimization-law discovery (Figure~\ref{fig:optscientist-agent}). The system comprises four interacting modules: a Research Orchestrator, a Theory Prior Agent, an Experiment Agent and a Scientific Critic and Theory Abstraction Agent. A detailed description of the OptScientist Agent is provided in Appendix~\ref{sec:add-optscientist-agent}.
{\color{MoBlue}\oldtextbf{Research Orchestrator.}}
The Research Orchestrator maintains the shared research state and coordinates the discovery process. It integrates hypotheses, candidate variables, discovered laws and scientific findings across agents and discovery rounds, while carrying unresolved questions forward. It also determines whether the workflow should terminate or enter another round of refinement, turning OptScientist runs into an iterative scientific-discovery process.
{\color{MoBlue}\oldtextbf{Theory Prior Agent.}}
The Theory Prior Agent draws on the scientific literature to identify established theories and mechanisms relevant to the research question, and translates them into explicit, testable hypotheses. It then proposes scientifically meaningful variables or transformations that represent these mechanisms, together with symbolic-search priors when scientifically justified. These priors guide the search while keeping the final expression data-driven.
{\color{MoBlue}\oldtextbf{Experiment Agent.}}
Guided by the priors from the Theory Prior Agent, the Experiment Agent translates scientific hypotheses, candidate variables and search priors into executable OptScientist configurations. It then uses the core OptScientist Model to search for candidate optimization laws from observational data. The resulting laws and supporting empirical evidence are returned for scientific critique and theory abstraction.
{\color{MoBlue}\oldtextbf{Scientific Critic and Theory Abstraction Agent.}}
The Scientific Critic and Theory Abstraction Agent evaluates candidate laws as scientific explanations and identifies the best-supported structures across them. It synthesizes the supported laws into a more general theoretical form and summarizes their qualitative mechanisms and main scientific findings. Remaining uncertainty is then translated into new hypotheses, variables or data requirements for the next discovery round, moving OptScientist Agent from symbolic law recovery towards iterative theory refinement.
{\color{MoBlue}\oldtextbf{Embedding the OptScientist Model in the discovery loop.}}
The OptScientist Model forms the empirical core of the agentic system. In each round, scientific hypotheses, candidate variables and search priors are translated into executable configurations, through which the model discovers candidate optimization laws from observational data. The resulting laws and empirical evidence are then returned to the agents for evaluation, theory abstraction and refinement of the next round.
\section{Experiments} \label{sec:expr}
We evaluate OptScientist Model on \emph{OptFeynman}, a new benchmark for optimization-law discovery, and the full framework on two real-world scientific-discovery tasks, with comparisons against machine-learning models, symbolic regression, and LLM model-discovery methods. The LLM baselines include an LLM-only researcher, GeCCo \citep{rmus2026generating} as a code-assisted LLM approach, and Model Discovery Agent \citep[MDA;][]{murphy2026model} as a Bayesian experiment-design approach. All LLM methods use DeepSeek-V4-Flash, the same LLM used by the optional OptScientist Agent. Data and evaluation are provided in Sec.~\ref{sec:data}, and comparison methods in Appendix~\ref{sec:baselines}.
\begin{figure}[htbp]
\centering
\includegraphics[width=\textwidth]{graph/optf_standard.pdf}
\caption{
{\color{MoBlue}\oldtextbf{Recovering ground-truth optimization laws on OptFeynman.}}
{\color{MoBlue}\oldtextbf{a,}}
OptFeynman converts analytic physics equations into latent
optimization problems and generates observations by Gibbs sampling from the
induced landscapes.
{\color{MoBlue}\oldtextbf{b,c,}}
For discrete decision variables, OptScientist shows the
strongest out-of-sample ability to recover the true optimization law on
unseen test contexts, outperforming baselines. It achieves perfect exact-order recovery on
\(88/99\) tasks, indicating that the recovered symbolic laws induce
preference orders approximately equivalent to the ground-truth laws.
{\color{MoBlue}\oldtextbf{d,e,}}
For continuous decision variables, the advantage persists on
unseen test contexts across increasingly dense candidate sets. OptScientist
perfectly recovers the exact order on \(78/99\) tasks at \(K=10\), \(K=50\)
and \(K=100\), showing robust discovery of the underlying optimization law
beyond discrete-choice settings. Error bars denote standard errors across datasets where
shown.
}
\label{fig:optfeynman-standard}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{graph/optf_X.pdf}
\caption{
{\color{MoBlue}\oldtextbf{Robust recovery of optimization laws under noisy context variables.}}
Gaussian noise is added to the training context variables, while validation
and test contexts remain clean.
{\color{MoBlue}\oldtextbf{a,}}
For discrete decision variables, OptScientist remains the
strongest method for out-of-sample recovery of the true optimization law,
achieving perfect exact-order recovery on \(92/99\) tasks and outperforming
machine-learning, symbolic-regression, optimization-structured, and LLM-based automated model-discovery baselines.
{\color{MoBlue}\oldtextbf{b,c,}}
For continuous decision variables, OptScientist retains high
recovery performance on unseen test contexts across \(K=10\), \(K=50\) and
\(K=100\), perfectly recovering the exact order on \(76/99\) tasks at each
candidate-set size and continuing to outperform the evaluated baselines. These results show that the model can recover
approximately equivalent ground-truth optimization laws even when the
training observations are corrupted by context noise. Error bars denote standard errors across
datasets where shown.
}
\label{fig:optfeynman-noisyx}
\end{figure}
\subsection{Recovering Ground-Truth Optimization Laws}
\label{sec:expr-ground-truth}
\begin{wraptable}{r}{0.50\textwidth}
\vspace{-20pt}
\centering
\tiny
\setlength{\tabcolsep}{3pt}
\caption{
{\color{MoBlue}\oldtextbf{Examples of recovered OptFeynman laws.}}
Examples show that OptScientist perfectly recovers effective ground-truth optimization expressions across diverse physical structures.
} \vspace{2mm}
\label{tab:optfeynman-recovered-examples-tiny}
\begin{tabular}{@{}p{0.20\linewidth}p{0.35\linewidth}p{0.35\linewidth}@{}}
\toprule
Decision variable & Effective ground truth & Effective recovered expression \\
\midrule
\(y_3\)
& \(\displaystyle \max_{y_3}\; x_3 y_3\)
& \(\displaystyle \max_{y_3}\; x_3 y_3\) \\
\addlinespace[0.35em]
\(w\)
& \(\displaystyle \max_{w}\; w^2\)
& \(\displaystyle \max_{w}\; w^2\) \\
\addlinespace[0.35em]
\(E_f\)
& \(\displaystyle \max_{E_f}\; E_f\cos\theta\)
& \(\displaystyle \max_{E_f}\; E_f\cos\theta\) \\
\addlinespace[0.35em]
\(G\)
& \(\displaystyle \max_{G}\; G\left(\frac{1}{r_2}-\frac{1}{r_1}\right)\)
& \(\displaystyle \max_{G}\; \frac{G(r_1-r_2)}{r_1r_2}\) \\
\addlinespace[0.35em]
\(d\)
& \(\displaystyle \max_{d}\; \frac{T_2-T_1}{d}\)
& \(\displaystyle \max_{d}\; \frac{T_2-T_1}{d}\) \\
\addlinespace[0.35em]
\(r\)
& \(\displaystyle \max_{r}\; \frac{1}{r^2}\)
& \(\displaystyle \max_{r}\; \frac{1}{r^2}\) \\
\addlinespace[0.35em]
\(d\)
& \(\displaystyle \max_{d}\; \frac{1}{d}\)
& \(\displaystyle \max_{d}\; \frac{1}{d}\) \\
\bottomrule
\end{tabular}
\vspace{-18pt}
\end{wraptable}
We first tested whether OptScientist Model can recover ground-truth optimization laws
from observations. OptFeynman converts analytic physics equations into latent optimization problems and generates observations via Gibbs sampling. Because
the generating objective is known, recovery can be evaluated directly on unseen
test contexts by comparing the preference order and Gibbs distribution of the recovered
and ground-truth laws. \textit{Notably, this experiment evaluates only the OptScientist Model, without any LLM assistance.}
\medskip
\noindent
{\color{MoBlue}\oldtextbf{Ground-truth recovery on OptFeynman.}}
OptScientist shows the strongest out-of-sample ability to recover the true
optimization law across both discrete and continuous decision variables
(Figure~\ref{fig:optfeynman-standard}). For discrete decision variables, it achieves perfect exact-order recovery on \(88/99\) problems, indicating that the recovered symbolic laws induce preference orders
approximately equivalent to the ground-truth laws. For continuous decision
variables, OptScientist achieves perfect exact-order recovery on \(78/99\) problems
at \(K=10, 50, \text{~and~} 100\). In both settings, OptScientist substantially outperforms all evaluated baselines. The stability of performance across increasingly
dense candidate sets shows robust discovery of the underlying law.
The recovered symbolic laws also preserve the stochastic structure of the
generating landscape, yielding Gibbs distributions close to the ground truth.
{\color{MoBlue}\oldtextbf{Robustness to noisy context variables.}}
We next evaluated whether recovery remains reliable when the training
observations are corrupted by context noise (Figure~\ref{fig:optfeynman-noisyx}).
Specifically, Gaussian noise is added to the training context variables, while validation and
test contexts remain clean. Under this setting, OptScientist remains the
strongest method for out-of-sample recovery of the true optimization law. It
achieves perfect exact-order recovery on \(92/99\) discrete problems and on
\(76/99\) continuous problems at \(K=10\), \(K=50\) and \(K=100\). These results
show that OptScientist can recover approximately equivalent ground-truth
optimization laws even when the training observations are corrupted by context
noise.
{\color{MoBlue}\oldtextbf{Representative recovered laws.}}
Table~\ref{tab:optfeynman-recovered-examples-tiny} shows representative examples of
successful symbolic recovery. The comparison focuses on the decision-relevant
part of each optimization law after pruning numerical constants and scale
factors that do not affect the optimizer. OptScientist recovers diverse physical
structures, including linear interactions, quadratic energy terms,
inverse-square laws, heat-transfer terms and algebraically equivalent rational
forms. These examples illustrate that the recovered laws preserve the
optimization law up to choice-equivalent symbolic transformations.
\subsection{Human Choice under Risk and Ambiguity}
\label{sec:expr-human-choice}
\begin{figure}[htbp]
\centering
\includegraphics[width=\textwidth]{graph/CPC18.pdf}
\caption{
{\color{MoBlue}\oldtextbf{A compact law of human decision-making under risk and ambiguity.}}
{\color{MoBlue}\oldtextbf{a,}}
A preliminary analysis of the CPC18 repeated-choice task shows that
decision-history variables are particularly informative relative to lottery,
demographic and framing variables, consistent with the sequential structure of
the discovered optimization law.
{\color{MoBlue}\oldtextbf{b,}}
OptScientist recovers this sequential structure as a compact
symbolic law: people tend to follow immediate reward, resist switching from
a previous choice and remain sensitive to forgone outcomes.
{\color{MoBlue}\oldtextbf{c,}}
Across behavioural baselines, machine-learning models and
OptScientist variants, the recovered law achieves the near-best predictive
performance with a near-minimal number of parameters. The result shows that
human choice can be compressed into a simple optimization principle rather
than explained by model scale alone. Error bars denote the standard error
of the mean across participant-level metrics.
}
\label{fig:cpc18}
\end{figure}
We next applied OptScientist to human decisions under risk and ambiguity, where
the underlying optimization law is unknown and human choices are shaped by
repeated feedback, individual heterogeneity and noisy behavioural responses.
The CPC18 dataset asks participants to repeatedly choose between two monetary
prospects \citep{plonsky2019cpc18}. This setting tests how OptScientist turns
behavioural data into a compact symbolic optimization law.
{\color{MoBlue}\oldtextbf{Theory-guided generation of supplementary variables.}}
Guided by scientific literature, the Theory Prior Agent constructs theoretically motivated variables beyond those in the original data, including expected value, forgone payoffs, gender and residence in wealthier regions. These variables are then considered in symbolic discovery. See Appendix~\ref{sec:add-cpc18-prior-analysis} for details.
{\color{MoBlue}\oldtextbf{A compact law of repeated choice.}}
OptScientist recovered a compact symbolic optimization law
(Figure~\ref{fig:cpc18}b).
In abstract form, the discovered law is
\begin{equation}
\max_{y_t\in\{A,B\}}
\left\{
V(x_t,y_t)
- \lambda\,\mathbf{1}(y_t\neq y_{t-1})
+ \eta\,R^{\mathrm{cf}}_{t-1}(y_t)
\right\}.
\label{eq:cpc18-abstract-law}
\end{equation}
Here, \(V(x_t,y_t)\) is the current reward value of the candidate option,
\(\mathbf{1}(y_t\neq y_{t-1})\) captures the cognitive cost of switching from
the previous choice, and \(R^{\mathrm{cf}}_{t-1}(y_t)\) captures sensitivity to
the forgone reward from the previous round. This law summarizes three
qualitative mechanisms: choice follows immediate reward, departing from a
previous choice carries a cognitive cost, and forgone rewards shape future
decisions. In this case, our approach can be viewed as performing an automated resource-rational analysis of human risky choices. The concrete score-difference expression and its mapping to this
abstract law are given in Additional Experimental Details,
Appendix~\ref{sec:add-cpc18-direct-law}.
{\color{MoBlue}\oldtextbf{Automatic-feature symbolic discovery.}}
Following the feature-mediated design introduced in the OptScientist Model, we
also evaluated an automatic-feature variant, denoted AutoF OptScientist. In this
variant, OptLearner first maps \((x_t,y_t)\) into denoised intermediate
representations from its inner layers. These layer-level variables act as
automatically extracted features, and OptDiscover then distills the final
optimization score as a symbolic function on the learned feature space. This
provides an additional compact symbolic description of the CPC18 dataset and
allows representation learning to support symbolic discovery when behavioural
variables are noisy, redundant or weakly structured. See Appendix~\ref{sec:add-cpc18-autof-law} for the recovered inner-feature expression.
{\color{MoBlue}\oldtextbf{Near-best performance with near-minimal complexity.}}
Across behavioural baselines, machine-learning models and OptScientist
variants, the recovered law achieves a favourable accuracy--complexity trade-off
(Figure~\ref{fig:cpc18}c). The direct OptScientist law reaches \(82.9\%\) accuracy
with five numerical constants, approaching the strongest high-capacity
machine-learning baselines that use thousands of parameters. Notably, predictive performance on CPC18 appears to saturate at approximately
83\% held-out accuracy across strong predictive models. AutoF OptScientist
achieves comparable ranking and calibration performance with seven numerical
constants, including an AUC of \(0.865\), a probability MSE of \(0.137\), and an
ECE of \(0.116\). Full predictive metrics and complexity definitions are
reported in Additional Experimental Details,
Appendix~\ref{sec:add-cpc18-performance}. These results show that repeated human
choice in CPC18 can be compressed into a simple optimization principle
combining immediate reward, cognitive switching cost and counterfactual reward.
\subsection{Central-Bank Monetary Policy}
\label{sec:expr-central-bank}
\begin{figure}[htbp]
\centering
\includegraphics[width=\textwidth]{graph/CentralBank.pdf}
\caption{
{\color{MoBlue}\oldtextbf{Discovery of a central-bank policy law.}}
{\color{MoBlue}\oldtextbf{a,}}
OptScientist recovers a compact symbolic rule for quarterly
monetary-policy decisions. The five-class policy stance, ranging from large
easing to large tightening, is explained as tracking a latent policy target
driven by macroeconomic strength, financial fragility, inherited tightness and
policy momentum.
{\color{MoBlue}\oldtextbf{b,}}
On a future-period out-of-time test set, OptScientist attains the best policy-stance and policy-direction
accuracy while using a near-minimal parameter count. Error bars denote bootstrap standard errors from \(10{,}000\) quarter-level
resamples with replacement.
}
\label{fig:central-bank}
\end{figure}
We then applied OptScientist to quarterly central-bank monetary-policy
decisions, where the observed action is a five-class policy stance ranging from
large easing to large tightening. The task is to discover a compact
optimization law linking macro-financial conditions to policy actions. This
setting tests how OptScientist recovers a symbolic policy rule from
small, noisy and temporally structured macroeconomic data.
{\color{MoBlue}\oldtextbf{Discovery of a central-bank optimization law.}}
OptScientist recovers a compact symbolic optimization law for monetary-policy
decisions. For scientific interpretation, the selected law is abstracted into a
policy-target tracking form (see Figure~\ref{fig:central-bank}a),
\begin{equation}
\min_{y_t\in\mathcal{Y}}
\left\{
\left(
y_t-\frac{N_t}{2\kappa}
\right)^2
+\varepsilon_t(y_t)
\right\},
\label{eq:central-bank-main-law}
\end{equation}
where \(y_t\) is the policy stance, \(\varepsilon_t(y_t)\) is a residual
decision term and \(N_t/(2\kappa)\) is the latent subjective target policy. The
recovered net policy pressure is \(N_t=m(g_t+\tau_t)-m(f_t+q c_t)-\rho r_{t-1}+(s+\rho)\Delta_t\). The first term captures macroeconomic strength, the second captures financial
fragility, the third captures inherited policy tightness and the fourth captures
policy momentum. The law therefore explains quarterly policy as tracking a
latent target: macroeconomic strength tightens policy, financial fragility
loosens policy, inherited tightness restrains further tightening, and policy
momentum carries previous adjustments forward. The full symbolic expression and
the algebraic rewriting into the target-tracking form are given in Additional
Experimental Details, Appendix~\ref{sec:add-central-bank-single-u}
and~\ref{sec:add-central-bank-target-rewrite}.
{\color{MoBlue}\oldtextbf{Constrained policy-law decomposition.}}
OptScientist also recovers a constrained symbolic law with explicit \(U\), \(C\) and \(T\) components, decomposing the policy score into policy pressure, a one-sided constraint and a quadratic risk or target-deviation term while retaining competitive predictive performance. The full expression and branch-level complexities are reported in Appendix~\ref{sec:add-central-bank-uct}.
{\color{MoBlue}\oldtextbf{Near-best performance with near-minimal complexity.}}
Across baselines and OptScientist variants, the single-\(U\) law achieves the
best point estimates for exact and direction accuracy
(Figure~\ref{fig:central-bank}b), reaching \(78.6\%\) and \(85.7\%\),
respectively, on the out-of-time test set with only 11 numerical parameters.
The constrained variant remains competitive at \(71.4\%\) and \(78.6\%\),
respectively, with the same parameter count. Full metrics and complexity details are reported in
Appendix~\ref{sec:add-central-bank-performance}.
\subsection{Token-Efficient Model Discovery}
\label{sec:token_efficiency}
\begin{figure}[htbp]
\centering
\includegraphics[width=\linewidth]{graph/llm_baseline.pdf}
\caption{
{\color{MoBlue}\oldtextbf{Token-efficient automated model discovery.}}
We further compare the token efficiency of OptScientist with three LLM-based automated model-discovery methods on the two real-world discovery tasks: LLM-only discovery, GeCCo (code-assisted LLM), and MDA (Bayesian experiment design). All LLM components use DeepSeek-V4-Flash.
{\color{MoBlue}\oldtextbf{Left,}}
On the central-bank monetary-policy task (Sec.~\ref{sec:expr-central-bank}), OptScientist attains higher held-out policy-stance accuracy with substantially lower output-token consumption.
{\color{MoBlue}\oldtextbf{Right,}}
On human decisions under risk and ambiguity (Sec.~\ref{sec:expr-human-choice}), OptScientist reaches competitive held-out predictive accuracy with substantially fewer output tokens.
Across both domains, OptScientist achieves a favourable accuracy--token-efficiency trade-off.
}
\label{fig:llm_baseline}
\end{figure}
We finally compared the token efficiency of OptScientist with three LLM baselines on the two real-world discovery tasks in Secs.~\ref{sec:expr-human-choice} and~\ref{sec:expr-central-bank}. All methods received the same research description, variable information, and training data; for the LLM baselines, we evaluated four maximum output-token settings and recorded held-out accuracy and cumulative output-token consumption.
{\color{MoBlue}\oldtextbf{Token efficiency.}}
OptScientist achieves near-best held-out predictive performance with substantially fewer LLM output tokens (Figure~\ref{fig:llm_baseline}). On central-bank monetary-policy decisions, OptScientist achieves \(78.6\%\) held-out policy-stance accuracy using 5,976 output tokens, exceeding the strongest LLM-based baseline result of \(70.5\%\), while the baselines consume approximately 16,700--29,600 tokens. On human decisions under risk and ambiguity, OptScientist approaches the
apparent predictive ceiling of approximately 83\% held-out accuracy while using only 6,396 output tokens, compared with approximately 15,800--45,800 tokens for the LLM baselines.
\section{Data and Evaluation} \label{sec:data}
\subsection{The OptFeynman} \label{sec:optfeynman}
{\color{MoBlue}\oldtextbf{Benchmark design.}}
To evaluate whether OptScientist can recover symbolic optimization laws, we construct \emph{OptFeynman}, a benchmark derived from the AI Feynman equation collection \citep{udrescu2020ai}. The original benchmark asks whether a method can recover an input--output equation. OptFeynman poses a different question: whether a method can recover the symbolic optimization problem that generates observed outcomes. Each analytic equation is therefore treated as a latent objective landscape, and the observed data are sampled as noisy outcomes of this landscape.
{\color{MoBlue}\oldtextbf{Optimization construction.}}
For each multivariate Feynman equation, we separate its variables into a context \(x\)
and a decision variable \(y\). The original equation defines the
ground-truth objective,
\[
U(x,y)=f(x,y).
\]
Single-variable equations are excluded because they do not define a context-dependent
decision problem, leaving 99 OptFeynman tasks. Context variables are sampled uniformly from
the ranges specified in the original Feynman benchmark. For each task, we generate a set of unique contexts and partition them at the
context level into training, validation and test sets. All stochastic outcomes
generated from a given context are assigned to the same split. Thus, performance reflects generalization across unseen \(x\)-contexts, with stochastic outcomes from each context kept entirely within one split.
{\color{MoBlue}\oldtextbf{Gibbs sampling.}}
Observed outcomes are generated from the optimization landscape through Gibbs sampling.
In the discrete version, the decision variable is first discretized into five candidate actions
within its original range,
\[
y_k = y_{\min} + k\frac{y_{\max}-y_{\min}}{K-1},
\qquad k=0,\ldots,K-1,\quad K=5.
\]
For each context \(x_i\), the ground-truth objective is evaluated on these candidates. The
resulting scores are scaled by the standard deviation of the training objective values,
estimated only from training contexts, and converted into a Gibbs choice distribution,
\[
P(y=k\mid x_i)
=
\frac{\exp(U(x_i,y_k)/(cT))}
{\sum_{\ell=0}^{K-1}\exp(U(x_i,y_\ell)/(cT))}.
\]
Observed labels are then sampled from this categorical distribution, with
multiple labels drawn for each context.
The continuous version follows the same principle, while keeping \(y\) continuous on its
original interval \([y_{\min},y_{\max}]\). For each context, the target conditional density is
defined by the exponentiated objective,
\[
p(y\mid x_i)
\propto
\exp\left(U(x_i,y)/(cT)\right),
\qquad y\in [y_{\min},y_{\max}].
\]
Because this density is not analytically tractable for arbitrary Feynman equations, it is approximated on a dense grid over the decision interval. Grid points are assigned normalized Gibbs weights, a grid point is sampled as the outcome, and the sampled value is optionally perturbed within its local grid cell and clipped back to the valid interval. The continuous setting uses the same context-level construction as the discrete setting, but draws a larger set of continuous outcomes from each context. Thus, the same symbolic objective is instantiated either over a finite action set
or over a continuous decision interval.
{\color{MoBlue}\oldtextbf{Noisy-context version.}}
We also construct a noisy-context version of OptFeynman. Starting from the clean dataset,
Gaussian noise is added only to the training contexts,
\[
\widetilde{x}_{ij}=x_{ij}+\epsilon_{ij},
\qquad
\epsilon_{ij}\sim\mathcal{N}(0,\sigma_j^2).
\]
The default noise scale is one percent of the sampling range of each context variable. The
validation and test contexts remain clean, and the ground-truth objective, sampled outcomes,
true scores, and evaluation files are unchanged. This setting tests whether OptScientist can
recover a stable underlying law when the observations used for learning are corrupted by
measurement noise.
{\color{MoBlue}\oldtextbf{Evaluation.}}
Evaluation is based on the utility-representation view of optimization. By Debreu's
continuous utility representation theorem, preferences can be represented by a utility
function when the preference relation is complete, transitive, and continuous
\citep{debreu1953representation}.
\begin{theorem}[Continuous utility representation, Debreu]
Let \(\mathcal{Z}\subseteq\mathbb{R}^n\), and let \(\succeq\) be a complete, transitive,
and continuous preference relation on \(\mathcal{Z}\). Then there exists a continuous
function \(u:\mathcal{Z}\rightarrow\mathbb{R}\) such that, for all
\(z_1,z_2\in\mathcal{Z}\),
\[
z_1\succeq z_2
\quad\Longleftrightarrow\quad
u(z_1)\geq u(z_2).
\]
\end{theorem}
This result motivates our evaluation: an equivalent optimization problem should preserve the
preference structure induced by the latent objective, while also explaining its stochastic
realizations. Since OptFeynman provides the generating objective
\(U(x,y)\), we evaluate whether the learned score and the distilled symbolic score recover
the same ordinal and probabilistic structure.
For each context \(x_i\), let \(\mathcal{Y}_i=\{y_{i1},\ldots,y_{iK}\}\) be the candidate
outcome set. In discrete tasks, \(\mathcal{Y}_i\) contains the five candidate actions; in
continuous tasks, it contains sampled continuous candidates. The ground-truth objective
induces an ordered list \(\pi_i=(j_1,\ldots,j_K)\),
\[
U(x_i,y_{ij_1}) \geq U(x_i,y_{ij_2}) \geq \cdots \geq U(x_i,y_{ij_K}),
\]
with \(\widehat{\pi}_i\) defined analogously by the recovered score \(\widehat{S}\).
Exact preference-order accuracy measures whether \(\widehat{\pi}_i=\pi_i\), and pairwise
accuracy measures whether each pairwise comparison has the same direction under \(U\) and
\(\widehat{S}\). These metrics test whether the recovered score represents the same
optimization problem up to order-preserving transformations.
Because the observations are generated by Gibbs sampling, we also evaluate the induced
stochastic landscape. For discrete tasks, the Gibbs distribution is computed over the five
candidate actions. For continuous tasks, it is approximated on a dense grid over the decision
interval. We compare the true and predicted distributions using KL divergence:
\[
D_\mathrm{KL}(P\Vert \widehat{P})
=
\frac{1}{N}\sum_i\sum_{y\in\mathcal{Y}_i}
P(y\mid x_i)\log\frac{P(y\mid x_i)}{\widehat{P}(y\mid x_i)} .
\]
Together, preference recovery evaluates whether the discovered expression captures the
equivalent optimization law, while distribution recovery evaluates whether it also captures
the stochastic strength of the generating landscape.
\subsection{Real-world Datasets} \label{sec:real-world-datasets}
Synthetic benchmarks test recovery when the generating optimization law is known.
Real-world data test discovery under noise, redundancy and incomplete theoretical
specification. We evaluate OptScientist on two domains: human risky choice and
central-bank monetary decisions.
{\color{MoBlue}\oldtextbf{Human decisions under risk and ambiguity.}}
The first dataset is the Choice Prediction Competition 2018 (CPC18), a large-scale
dataset of repeated human choices between monetary prospects under risk and ambiguity
\citep{plonsky2019cpc18}. In each problem, a participant repeatedly chooses between
two options, \(A\) and \(B\). The target variable is the observed binary choice, indicating
whether option \(B\) is selected.
CPC18 contains several groups of variables. The main lottery variables describe the
objective choice environment, including outcomes, probabilities, ambiguity and correlation
between the two options. Identifier variables, such as participant and game identifiers, organize repeated decisions
across participants and problems. Framing variables describe how the choice problem and
feedback are presented. Demographic variables provide participant-level covariates. Because
each participant faces repeated decisions with feedback, we also construct
history variables indicating whether a previous round exists and which option was
chosen in the previous round.
This dataset tests whether OptScientist can recover the optimization law behind human decisions under risk, ambiguity and sequential feedback.
{\color{MoBlue}\oldtextbf{Central-bank monetary decisions.}}
The second dataset studies central-bank monetary decisions from quarterly
macroeconomic and financial conditions. The outcome \(y\) is a five-class
quarterly policy stance, ranging from large easing to large tightening. We
construct \(y\) from the quarter-to-quarter change in the average federal funds
target rate. Before 2008, the daily target rate is taken from the single federal
funds target-rate series. After 2008, it is computed as the midpoint of the
target-rate corridor. The daily target rate is averaged within each quarter, and
the quarterly change is measured in basis points. A decrease of at least 50 basis
points is coded as \(-2\), a decrease between 25 and 50 basis points as \(-1\),
a change within 25 basis points as \(0\), an increase between 25 and 50 basis
points as \(1\), and an increase of at least 50 basis points as \(2\).
The context \(x\) contains twelve quarterly macro-financial variables. These
include PCE inflation, core PCE inflation, unemployment and real GDP growth;
two policy-inertia variables, given by the previous average target rate and the
previous quarterly target-rate change; three financial-market variables, given
by the Baa--Treasury credit spread, the 10-year--3-month term spread and the
CBOE Volatility Index (VIX);
and three financial-stress measures, given by the St. Louis Fed Financial Stress
Index, the Chicago Fed National Financial Conditions Index and the Adjusted
National Financial Conditions Index. The period variable is retained for temporal
indexing and data splitting.
This dataset tests whether OptScientist can recover a compact symbolic
optimization law behind macroeconomic policy decisions.
All policy-rate, macroeconomic and financial variables are obtained from
Federal Reserve Economic Data (FRED), maintained by the Federal Reserve Bank
of St. Louis. The federal funds target-rate series and target-range bounds are
used to construct the policy outcome, while the macroeconomic, financial-market
and financial-stress variables are taken from their corresponding FRED series.
\section{Conclusion and Limitations} \label{sec:discussion}
We introduced OptScientist, a framework for automatically discovering compact
symbolic optimization laws from observational data. OptScientist combines a
model that learns and distills latent optimization structure with an agentic
scientific-discovery system that incorporates theoretical priors, searches for
candidate laws, evaluates empirical evidence, and abstracts supported
mechanisms. On OptFeynman, OptScientist recovers ground-truth optimization laws
across a broad range of physics-derived problems, substantially outperforms all evaluated baselines, and remains robust to noisy
contextual information. Applied to human risky choice and central-bank monetary
policy, it discovers interpretable optimization laws with competitive or
stronger held-out predictive performance and compact symbolic representations.
On the two real-world tasks, OptScientist also uses substantially fewer output tokens than LLM-based automated model-discovery approaches. These results establish symbolic
optimization-law discovery as a new direction for automated scientific
discovery and provide a general framework for learning compact optimization
principles underlying observed phenomena.
{\color{MoBlue}\oldtextbf{Limitations and Future Research.}}
When the structure of ground-truth models deviates substantially from an optimization formulation, our approach is likely to fail. Although,
mathematically, a broad class of laws not originally formulated as optimization
problems can be represented in optimization form, the OptFeynman results show
that OptScientist can recover the original laws from such
representations. Future research is needed to characterize the limits of the optimization assumption and identify the classes of models that remain discoverable within a feasible computational budget.
\newpage
\bibliography{references}
\bibliographystyle{open_preprint_iclr}
\newpage