数学前沿

许多科学现象最好被理解为潜在优化问题的结果，但自动科学发现主要集中于恢复描述观测到的输入-输出模式的方程。作者引入 OptScientist，这是一个直接从数据中发现紧凑符号优化定律的新型自动框架。OptScientist 由两部分组成：(i) 一个模型，它先学习将数据表示为优化问题，然后提取其符号形式，从而推断符号优化问题；(ii) 一个 LLM 智能体系统，将上述模型嵌入发现过程，该过程整合来自科学文献的理论先验、发现候选定律、抽象其潜在机制，并建议收集新数据。作者首先在 OptFeynman 上评估 OptScientist，这是一个由 99 个源自物理的优化问题组成的新基准；OptScientist 在其中超过 80% 的问题上恢复了真实定律，并对噪声上下文信息保持稳健，同时显著优于机器学习、符号和基于 LLM 的模型发现基线。在人类风险选择和中央银行货币政策上，OptScientist 发现了紧凑的符号优化定律，其留出预测性能与领先基线相当，同时使用的参数很少，且所用 token 数显著少于 LLM 基线。总体而言，OptScientist 将自动科学发现扩展到揭示生成观测现象的优化定律，从而支持跨物理、生物、行为和社会科学的自动理论构建。

优化是现代机器学习应用中的重要模块，已有大量工作致力于加速优化算法。一个常见的表述是在给定时间内达到更低的损失，这为关于算法超参数的可微框架提供了可能。与之对偶的表述——最小化达到目标损失所需的时间——被认为不可微，因为时间本身不可微；因此它通常仅作为概念框架，或使用零阶方法进行优化。为弥补这一局限，作者提出一种可微停止时间（differentiable stopping time），并基于微分方程对其进行了理论论证。作者设计了一种高效算法来通过该停止时间进行反向传播。由此，所提出的可微停止时间为一类新的加速算法可微表述提供了可能。作者进一步讨论了其应用，例如在线超参数调优和学习优化（learning to optimize）。作者称，所提方法在针对各类问题的综合实验中表现出优越性能，验证了其有效性。

Clustered federated learning (CFL) can suffer from cluster collapse. In particular, under unfavorable initialization, early client assignments concentrate heterogeneous groups on one cluster model, which is then trained toward a pooled compromise, while another model receives no updates. Since the current models determine client routing and routing determines subsequent model updates, we formulate clustered federated learning as control of an unknown dynamical system, with cluster models as the evolving state, client routing patterns as the control input, and routing interventions as state dependent controllers. In a two group linear model, we identify an initialization region in which greedy routing makes collapse self reinforcing. We then derive finite horizon error dynamics showing how routing interventions can reverse collapse by repeatedly training the starved model, primarily with updates from its compatible group. Motivated by this condition, each client uses blocked EXP3 to choose between greedy routing and interventions that redirect updates toward unused models. Experiments show that with simpler predictors and faster specialist formation, autonomous controller selection recovers usable specialists from unfavorable starts that trap standard CFL baselines. In transformed FEMNIST, the synchronized-entry configuration with persistent commitment recovers usable specialists in settings where the autonomous controller configuration often does not. Code and scripts for reproducing the experiments and figures are available at https://github.com/prsinha1/FedRecover.

本文研究梯度计算带有延迟时非凸优化的方差缩减问题：服务器可以在先前的梯度评估尚未完成时更新参数。作者首先分析随机优化中的一种有序递归动量估计器（ordered recursive momentum estimator）；在平均光滑性（average smoothness）与有界方差假设下，以期望平方梯度范数不超过 $\epsilon^2$ 为目标，其 oracle 复杂度为 $O(1+\sigma^2\epsilon^{-2}+L\Delta_f(1+\sigma/\epsilon+\tau)\epsilon^{-2})$，其中 $\tau$ 界定反馈延迟。随后，作者研究具有 $n$ 个分量的有限和问题，得到 $O(n+L\Delta_f(\sqrt n+\min\{\tau,n-1\})\epsilon^{-2})$ 次评估。一种异步快照变体（asynchronous snapshot variant）在完整梯度计算期间持续进行更新，并对其完成时间给出显式界。对于随机方法，作者进一步将最大延迟界替换为缺失反馈的加权量的界。

本文研究逻辑回归上梯度下降的一些经典分析未能完全刻画的性质。对于线性可分数据，迭代方向会收敛到最大间隔分离器，无论步长多大；而对于不可分数据，即使步长低于局部稳定性阈值 $2/\lambda^*$（其中 $\lambda^*$ 为解处 Hessian 的最大特征值），也可能出现循环。作者提出一个问题：要求所有样本具有相同范数，是否足以保证在该阈值以下全局收敛？作者证明，在一维情形下这一条件确实成立；在高维情形下，即使数据集具有相同范数也可能产生循环，但当样本均匀分布在球面上时，总体损失（population loss）仍会全局收敛。最后，在一维情形下，作者将等范数假设放宽为一个环带（annulus）条件，该条件可保证在稳定性阈值以下全局收敛，并指出这为在高维中寻找类似的充分条件提供了路径。

Preference labels align model training, but the judgments behind them are difficult to inspect or verify. In this paper, we examine how preference construction can be checked when intermediate decisions are recorded, and which of them remain uncertain. We present Symbolic Preference Labeling, a method in which a pretrained language model classifies predefined attributes of prompts and responses. Explicit rules assign response verdicts, and a fixed verdict ranking determines the preference label. Each preference label has a recorded derivation: the response verdicts and preference label can be recomputed from the stored attribute assignments, rules, and verdict ranking without rerunning the language model. On 200 prompts annotated by human raters, the language model produces valid structured outputs for 192 prompts. All eight parsing failures occur in the subset of self-harm prompts marked as harmful. Using 465 preference pairs, we train one policy model with direct preference optimization across five training seeds. Mean measured benign refusal is 40.82 percent on 1,319 benign prompts that resemble unsafe ones, and mean measured toxic refusal is 84.98 percent on 655 toxic prompts. This study distinguishes the reproducibility of preference labels from attribute correctness. The recorded derivations make preference construction open to inspection, while attribute correctness and semantic safety require separate validation.

We develop and compare enhanced neural-network solvers for one-dimensional ground-state profiles, which are solutions of $$Q^{\prime\prime}=bQ-\gamma Q^{p},\quad Q>0,\quad p>1, \quad b, \gamma >0,$$ the stationary profile equation for the nonlinear Schr\"odinger, generalized Korteweg-de Vries, and nonlinear Klein-Gordon equations. Its explicit sech-type solutions serve as exact reference for error computation and benchmark hyperparameter selection. We improve on the standard Adam-trained physics-informed neural network (PINN) baseline by introducing higher-order optimization via Gauss-Newton/Levenberg-Marquardt and L-BFGS refinements, as well as Fourier-feature PINNs, and constrained Deep Ritz (DR) formulations. In DR approaches we introduce two new neural-network formulations, which implement two classical variational characterizations: a Nehari-constrained Deep Ritz method, in which the height is learned by projecting onto the Nehari/Pokhozhaev manifold, and a Weinstein Deep Ritz method, in which a normalized profile minimizes the Weinstein quotient. The Deep Ritz formulations perform best overall, in particular, the Weinstein variant gives the best error-runtime balance. The neural-network based solvers remain less accurate and efficient than classical Petviashvili iteration in 1D, however, the current study provides a proof of concept: the mesh-free formulation, non-reliance on exact profiles or the related features make them promising in higher dimensions and for more complicated nonlinearities and dispersive operators.

学习式屏障证书为随机系统的安全性提供了有原则的验证机制，但当部署时的真实动力学与训练所用动力学不一致时，其有效性可能在运行过程中退化。作者提出了一种随时有效（anytime-valid）的运行时监控器，利用观测到的闭环轨迹对学习式屏障证书进行审计。该监控器旨在检测部署屏障条件是否失去统计支持，其本身并不保证在进入不安全状态前完成检测，也不保证后续干预的安全性。 具体地，作者将条件屏障漂移条件（conditional barrier-drift condition）表述为逐回合（episode-wise）的序贯假设检验，把观测到的屏障增量转化为非负检验上鞅（test supermartingale），从而在任意停止时刻下获得有限样本的误报控制。受有限时域安全验证的启发，作者进一步引入了对监控边界的确定性时间整形（deterministic temporal shaping），使监控器在给定部署时域的早期更为保守，同时保持随时有效性。该监控器的设计目的是检测安全证书与观测动力学不再一致的统计证据，而非预测不安全事件的发生。 在实证评估方面，作者分别在证书有效与证书无效的情形下评估了监控器的报警行为。代码与数据见 https://github.com/FormalCert-AI/runtime-barrier-monitor。

LoRA（Low-Rank Adaptation）能够对大语言模型进行高效微调，但当学习到的更新与承载预训练关键知识的主导奇异方向发生干扰时，会产生灾难性遗忘。作者提出正交投影 LoRA（Orthogonal Projection LoRA, OPLoRA），一种有理论支撑的方法，通过双侧正交投影来避免这种干扰。具体而言，对冻结权重做 SVD 分解后，OPLoRA 用投影 $P_L = I - U_k U_k^\top$ 与 $P_R = I - V_k V_k^\top$ 将 LoRA 更新严格限制在 top-$k$ 奇异子空间的正交补中。作者证明该构造能精确保持 top-$k$ 奇异三元组，从而为知识保留提供数学保证。为量化子空间干扰，作者还引入度量 $\rho_k$，用于衡量更新与主导方向的对齐程度。在常识推理、数学与代码生成上的大量实验表明，在 LLaMA-2 7B 与 Qwen2.5 7B 上，OPLoRA 显著降低遗忘，同时保持有竞争力的任务性能，说明正交投影是参数高效微调中实现知识保留的有效机制。

We prove sharp spectral-norm bounds and construct a mean-fitting regularization procedure for sparse random tensors of fixed order $k$. For an $n\times\cdots\times n$ tensor with independentBernoulli$(p)$ entries, the centered norm is $O(\sqrt{np})$ with polynomially high probability whenever $np\ge c\log n$. This removes the logarithmic loss in Zhou and Zhu (2021). We also determine the expected centered norm at every sparsity level: for every $0<p\le1/2$, it has order $\min\{n^kp,\sqrt{d_{\mathrm{eff}}}\}$, where $d_{\mathrm{eff}}$ is an effective degree accounting for unusually large fiber counts. The upper bounds extend to tensors with deterministic entrywise weights and unequal sampling probabilities. Analogous two-sided bounds hold for the Friedman--Wigderson model at every degree; its expected centered norm has order $\sqrt{d_{\mathrm{eff}}}$. We also give an all-sparsity regularization procedure for independent Bernoulli tensors with unknown, possibly unequal means in $[0,p]$. Our light-fitting regularization estimates the original mean with minimax expected spectral-norm error of order $\min\{\sqrt{np},pn^{k/2}\}$ for every known $0\le p\le1$. The procedure fits light multilinear residuals under an entrywise constraint on the candidate mean, rather than deleting high-degree fibers.

This paper is an exposition of the theorem prover Arend developed at JetBrains Research. Arend language has been designed to facilitate the formalization of mathematics within the framework of univalent foundations and with the focus on constructive mathematics. It is based on a variant of homotopy type theory with cubical syntax that is more closely aligned with Martin-L\"of Type Theory (MLTT) than fully computational cubical type theories, while still retaining some of their computational properties useful in practice. Among the most notable features of the Arend language is its powerful system of classes and records supporting subsumptive subtyping, partial implementations and anonymous extensions, which provides extensive flexibility for constructing hierarchies of definitions. Formalization in Arend is streamlined by the IntelliJ Arend plugin, which transforms IntelliJ IDEA into a full-fledged integrated development environment (IDE) for the Arend language. The main Arend library arend-lib includes formalizations in constructive algebra, topology and synthetic homotopy theory.

Continuous-time reinforcement learning seeks an optimal policy under unknown dynamics driven by stochastic differential equations using observations collected at discrete times. We develop a policy gradient framework that combines rigorous convergence analysis with learning from discrete data. We first establish a performance difference lemma and derive policy gradient formulas for randomized and parametrized policies. We then prove exponential convergence of an entropy-regularized policy flow to the regularized optimum. For discrete parametrized updates, we derive a convergence bound that explicitly separates the contributions of critic error, policy-regression error, and the learning step size. To enable learning under unknown dynamics, we integrate this framework with PhiBE, which constructs a continuous-time surrogate generator from discrete transition data. The resulting on-policy and off-policy model-free actor-critic algorithms preserve the continuous-time structure of the control problem. Numerical experiments on an entropy-regularized linear-quadratic regulator and a nonlinear wheeled robot demonstrate the performance and stability of the proposed methods with both known and data-driven dynamics.

Polyak--Ruppert 平均使 SGD 与动量方法具有相同的渐近协方差。本文刻画有限次运行达到该协方差的快慢，以及它们能在多大程度上被并行化。对于带加性噪声的强凸二次问题，作者证明了一个在线—离线桥梁：在尾部平均所用样本上的经验风险最小化器，与任意在线过程的尾部平均查询点只相差一步 Newton 步。对于 SGD、heavy ball 以及带常数动量的 Nesterov 方法，该步是尾部两端处的边界项，并把与离线估计器的差距分解为中心误差、协方差误差和形状误差。必要且充分的条件表明：在曲率最小的方向上，中心误差决定速度——SGD 需要 $m\gg\kappa^2$ 轮，而动量方法只需 $m\gg\kappa$，其中 $\kappa$ 是条件数；因此在样本量为 $N$ 时，临界批量大小从 $\Theta(\sqrt{N}/\kappa)$ 增长到 $\Theta(\sqrt{N/\kappa})$。协方差误差将两种动量方法区分开：heavy ball 在最刚方向上的相对方差超出为 $\Theta(\kappa^{3/2}/m)$，而 Nesterov 在每个方向上的相对误差为 $O(\sqrt{\kappa}/m)$。形状误差通过一个对 $\kappa$ 和批量大小一致的 Berry--Esseen 界，与方法无关，因此当另外两个误差消失时，置信区间是有效的。作者进一步考察了在重尾噪声或状态依赖噪声以及裁剪下这些结论如何变化，并在显式条件下，对非线性 Nesterov 平均与 ERM 建立了一阶比较。

We study the numerical resolution of uniformly elliptic Dirichlet boundary value problems (BVPs) by minimization of the PDE residual over a hypothesis space of Neural Networks made to satisfy exactly the boundary conditions through multiplication by a user-defined \emph{distance-to-boundary approximation} $\rho$. While an $L^2(\partial\Omega)$ boundary penalty cannot generally guarantee $H^2(\Omega)$ error control, we show that exact enforcement can, provided $\rho$ is chosen appropriately. More precisely, we identify a sharp sufficient condition on $\rho$ preserving the regularity needed for $H^2(\Omega)$ error control, provide practical constructions of such $\rho$ for general smooth domains $\Omega$, and refer to the resulting networks as \emph{boundary-adapted}. For boundary-adapted Rectified Quadratic Unit (ReQU) and hyperbolic tangent (tanh) networks, we then derive explicit high-probability $H^2(\Omega)$ convergence rates by combining approximation-theoretic and statistical-learning arguments, making explicit the dependence on $\rho$. Numerical experiments support our theoretical results, showing that appropriate choices of $\rho$ improve accuracy and convergence, while poorly chosen distance functions can substantially degrade the solution. Our proof also yields new VC-dimension bounds for hypothesis spaces of higher-order derivatives of ReQU and tanh networks, together with new approximation bounds for shallow ReQU networks in higher-order Sobolev norms, all of which are of important independent interest.

We introduce a unified framework for learning stochastic dynamics from distributional observations and discretely observed trajectories. The key contribution is to combine soft law constraints, which regularize fitting to empirical distributions, with path-dependent supervision, which captures temporal structure that marginal distributions alone cannot identify. We formulate the problem as a discrete path-dependent McKean-Vlasov control problem, penalizing endpoint, intermediate marginal, and joint sampled-path laws within a single objective. Its optimality system takes the form of a McKean-Vlasov forward-backward stochastic differential equation (FBSDE), in which the marginal terms act continuously in time while the path-law term instead induces conditional adjoint jumps at the observation dates. We derive the reduced FBSDE for quadratic control cost and constant diffusion, establish local well-posedness for a Maximum Mean Discrepancy (MMD) path-law formulation, and develop a sample-based neural solver with causal representations. Experiments on synthetic transport, latent face transport, and human motion validate the framework. In particular, a two-state AMASS experiment under path-law supervision captures both low-to-high and high-to-low transitions, which marginal supervision alone cannot distinguish.

Wasserstein分布鲁棒优化（WDRO）提供了一个框架，通过参考分布周围Wasserstein球中的最坏情况期望来评估模型鲁棒性。我们证明：按WDRO风险递增对假设排序，当且仅当损失函数是零一损失的正常数倍加上一个依赖标签的偏移量时，等价于按鲁棒准确率递减对假设排序。对于包括交叉熵在内的一类损失函数，我们表明具有完美鲁棒准确率的假设反而可能比另一个鲁棒准确率接近零的假设具有更高的WDRO风险。为直接评估鲁棒准确率，我们开发了Wasserstein分布攻击（WDA），它在候选支撑搜索之后增加一个权重生成阶段，以在Wasserstein约束下优化经验质量的分配，而不是像现有方法那样将每个数据点的全部质量分配给一个返回的攻击点。我们进一步为WDA引入了一种支撑搜索算法，用于找到附近的误分类输入。在CIFAR-10、CIFAR-100和ImageNet上，使用我们搜索算法的WDA在大多数测试设置中取得了最低的对抗准确率。我们的代码可在 https://github.com/OLab-Repo/WDA 获取。

Riemannian manifolds are powerful models for problems with non-Euclidean structure. Recent work in geometric machine learning has focused on training deep neural networks to learn Riemannian metrics from high-dimensional data. However, many geometric algorithms require repeated evaluations of the exponential and logarithmic maps, whose high computational cost restricts the utility of learned Riemannian manifolds in practice. We propose to address this cost by training a conditional invertible neural network, called a flow chart, to approximate normal coordinates around points on a given Riemannian manifold. With this approach, the exponential and logarithmic maps can be approximated by fast Euclidean operations in the model's learned coordinate space. Our main contribution is an efficient loss function for training flow charts that is nonnegative and, under suitable domain assumptions, is zero precisely when the chart is normal. Across sixteen synthetic and data-derived geometries, flow charts are competitive with a specialized neural geodesic interpolator for logarithmic map estimation, while also being able to estimate the exponential map. When pretrained, flow charts provide a median speedup of 60-70x over numerical baselines at comparable logarithmic map estimation error, and their learned coordinates improve the accuracy of a numerical geodesic solver.

鲁棒马尔可夫决策过程（MDP）下的学习与最优控制受到越来越多关注，但现有理论、算法与应用大多集中于有限时域或折扣模型。长期平均奖励形式在许多运筹学与管理情境中虽自然，却仍研究不足，主要原因是其动态规划基础在技术上具有挑战性且仅被部分理解，若干基本问题仍未解决。本文通过分析常数增益（constant gain）情形，向平均奖励鲁棒MDP的一般框架迈进一步。作者研究控制器与S-rectangular对手之间可能存在信息不对称的平均奖励鲁棒控制问题，分析围绕常数增益鲁棒Bellman方程展开，考察解的存在性及其与最优平均奖励的关系。具体而言，作者识别了鲁棒Bellman方程的解何时刻画最优平均奖励与平稳策略，并给出确保解存在的一侧弱通信（one-sided weak communication）条件。这些结果扩展了平均奖励鲁棒MDP的动态规划理论，并为运筹环境中长期平均准则下的鲁棒动态决策奠定基础。

A neural model is the set of functions that can be expressed by a given neural network architecture. Neural models contain natural submodels, such as those obtained by setting some neurons to be inactive. We study the geometry of neural models and ask whether they prefer their submodels. By this we mean whether local minimizers of the loss function lie in a given submodel. We introduce a geometric framework to determine if this occurs. For shallow multilayer perceptrons, we show that subnetworks with inactive hidden neurons are never preferred. For deep multilayer perceptrons, we give conditions under which subnetworks with inactive hidden neurons are not preferred. For lightning self-attention, lower-rank submodels are usually not preferred, but we identify examples where they are because they lie on a boundary of the model. Our results show that, aside from a few exceptions, preference for submodels cannot be explained by the geometry of the neural model, and must instead come from other sources such as training dynamics.

Teams of large language model (LLM) agents are increasingly applied to complex tasks, but their collaboration protocols are mostly designed empirically, without convergence guarantees. We formalize collaborative problem solving as block optimization: each agent improves one part of a shared solution, guided by its local objective plus a correction for how its changes affect the other parts. A coordinator checks every proposal against the global objective and merges proposals that do not interfere with each other. We prove linear convergence up to an error floor that vanishes when the corrections are exact. Interfering proposals can also be merged once their combination has been checked; the rate then depends on how well the proposals actually combine, not on worst-case interactions. With GPT-OSS-120B agents on robot routing, a three-agent team finds collision-free plans in every run, better than those of a single agent that controls all robots. On a larger routing task, a six-agent team completes its search about twice as fast as the single agent (in median). Merging the best checked combination of proposals, rather than only proposals that do not interfere, cuts the team's search time by about 30%. With constants measured on the runs, our convergence bound holds in every round and follows the observed progress. We also test the theory on query planning in SQLite and on a synthetic task.