|
创作新主题 |
社区所有版块导航
DATA
| docker Elasticsearch |
WEB开发
| linux MongoDB Redis DATABASE NGINX 其他Web框架 web工具 zookeeper tornado NoSql Bootstrap js peewee Git bottle IE MQ Jquery |
机器学习
| 机器学习算法 |
产品
| 短视频 |
印度
| 印度 |
一周十大热门主题
点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！
cs.LG 方向，今日共计119篇
大模型相关(13篇)
【1】CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
标题：CrispEdit：用于可扩展非破坏性LLM编辑的低弯曲投影
链接：https://arxiv.org/abs/2602.15823
摘要：大型语言模型（LLM）编辑的一个核心挑战是能力保护：成功改变目标行为的方法可以悄悄地游戏编辑代理并破坏一般功能，产生退化行为，让人想起代理/奖励黑客。我们提出了CrispEdit，一个可扩展的和有原则的二阶编辑算法，将能力保存作为一个明确的约束，统一和推广现有的几种编辑方法。CrispEdit将编辑公式化为约束优化，并通过将编辑更新投影到能力损失景观的低曲率子空间上来强制执行约束。CrispEdit的关键是通过Bregman散度表达能力约束，Bregman散度的二次形式精确地产生高斯-牛顿海森，即使基础模型没有经过收敛训练。我们使用Kronecker因子近似曲率（K-FAC）和一种新的无矩阵投影仪，利用Kronecker结构，以避免构建大量的投影矩阵，使这个二阶过程在LLM尺度上是有效的。在标准模型编辑基准测试中，CrispEdit取得了很高的编辑成功率，同时将整个数据集的平均能力下降保持在1%以下，比之前的编辑器有了显着改进。
摘要：A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing massive projection matrices. Across standard model-editing benchmarks, CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets, significantly improving over prior editors.
【2】Recursive Concept Evolution for Compositional Reasoning in Large Language Models
标题：大型语言模型中组合推理的递进概念进化
链接：https://arxiv.org/abs/2602.15725
摘要：大型语言模型在许多复杂的推理任务上具有很强的性能，但在需要组合推理的基准测试中，包括ARC-AGI-2，GPQA，MATH，BBH和HLE，它们的准确性急剧下降。现有的方法通过思想链提示、自一致性或强化学习来扩展标记级搜索来改进推理，但它们使模型的潜在表示空间固定。当所需的抽象还没有在这个空间中编码时，性能就会崩溃。我们提出了递归概念进化（RCE），这是一个框架，使预训练的语言模型能够在推理过程中修改其内部表示几何。RCE引入动态生成的低秩概念子空间，这些子空间在检测到代表性不足时产生，通过最小描述长度标准选择，在协同时合并，并通过约束优化合并以保持稳定性。这个过程允许模型构建新的抽象，而不是重新组合现有的抽象。我们将RCE与Mistral-7 B集成，并在组合推理基准中对其进行评估。RCE在ARC-AGI-2上获得12-18点的增益，在GPQA和BBH上获得8-14点的改进，并在MATH和HLE上持续减少深度引起的误差。
摘要：Large language models achieve strong performance on many complex reasoning tasks, yet their accuracy degrades sharply on benchmarks that require compositional reasoning, including ARC-AGI-2, GPQA, MATH, BBH, and HLE. Existing methods improve reasoning by expanding token-level search through chain-of-thought prompting, self-consistency, or reinforcement learning, but they leave the model's latent representation space fixed. When the required abstraction is not already encoded in this space, performance collapses. We propose Recursive Concept Evolution (RCE), a framework that enables pretrained language models to modify their internal representation geometry during inference. RCE introduces dynamically generated low-rank concept subspaces that are spawned when representational inadequacy is detected, selected through a minimum description length criterion, merged when synergistic, and consolidated via constrained optimization to preserve stability. This process allows the model to construct new abstractions rather than recombining existing ones. We integrate RCE with Mistral-7B and evaluate it across compositional reasoning benchmarks. RCE yields 12-18 point gains on ARC-AGI-2, 8-14 point improvements on GPQA and BBH, and consistent reductions in depth-induced error on MATH and HLE.
【3】CAMEL: An ECG Language Model for Forecasting Cardiac Events
标题：CAMEL：预测心脏事件的心电图语言模型
链接：https://arxiv.org/abs/2602.15677
备注：24 pages, 6 figures
摘要：心电图（ECG）是心脏的电子记录，对于诊断心血管疾病至关重要。心电图语言模型（ELMs）最近出现了一个很有前途的框架，伴随着报告生成的心电图分类。然而，目前的模型不能预测未来的心脏事件，尽管规划早期干预的巨大临床价值。为了解决这一差距，我们提出CAMEL，第一ELM，是能够推断在较长的信号持续时间，使其预测能力。我们的关键见解是一个专门的ECG编码器，它可以交叉理解ECG信号与文本。我们训练CAMEL使用既定的LLM培训程序，结合LoRA适应课程学习管道。我们的课程包括ECG分类，度量计算和多轮对话，以引出推理。CAMEL在6个任务和9个数据集上表现出强大的zero-shot性能，包括ECGForecastBench，这是我们为预测心律失常而引入的新基准。CAMEL与ELM和分布内外的完全监督基线相当或超过ELM和完全监督基线，在ECGBench（绝对平均增益+7.0%）和ECGForecastBench（完全监督模型+12.4%和zero-shot ELM +21.1%）上实现SOTA结果。
摘要：Electrocardiograms (ECG) are electrical recordings of the heart that are critical for diagnosing cardiovascular conditions. ECG language models (ELMs) have recently emerged as a promising framework for ECG classification accompanied by report generation. However, current models cannot forecast future cardiac events despite the immense clinical value for planning earlier intervention. To address this gap, we propose CAMEL, the first ELM that is capable of inference over longer signal durations which enables its forecasting capability. Our key insight is a specialized ECG encoder which enables cross-understanding of ECG signals with text. We train CAMEL using established LLM training procedures, combining LoRA adaptation with a curriculum learning pipeline. Our curriculum includes ECG classification, metrics calculations, and multi-turn conversations to elicit reasoning. CAMEL demonstrates strong zero-shot performance across 6 tasks and 9 datasets, including ECGForecastBench, a new benchmark that we introduce for forecasting arrhythmias. CAMEL is on par with or surpasses ELMs and fully supervised baselines both in- and out-of-distribution, achieving SOTA results on ECGBench (+7.0% absolute average gain) as well as ECGForecastBench (+12.4% over fully supervised models and +21.1% over zero-shot ELMs).
【4】Quantifying construct validity in large language model evaluations
标题：量化大型语言模型评估中的结构有效性
链接
：https://arxiv.org/abs/2602.15532
摘要：LLM社区经常报告基准测试结果，就好像它们是通用模型功能的同义词。然而，基准测试可能会有一些问题扭曲性能，比如测试集污染和注释器错误。我们如何知道基准是我们想要测量的某种能力的可靠指标？这个问题涉及到LLM基准测试的结构有效性，当我们建模和预测LLM性能时，需要将基准测试结果与性能分离。 社会科学家和计算机科学家都提出了正式的模型--潜在因素模型和比例法则--来识别基准分数背后的能力。然而，无论是技术是令人满意的结构效度。潜在因子模型忽略了缩放定律，因此，它们提取的功能通常代表模型大小。标度律忽略了测量误差，因此，它们提取的能力既不可解释，又过拟合到观察到的基准。 本文提出了结构化能力模型，第一个模型，以提取解释和概括的能力，从大量的LLM基准测试结果。我在OpenLLM排行榜的大量结果样本上拟合了这个模型和它的两个替代方案。结构化能力在简约拟合指数上优于潜在因子模型，并且表现出比标度律更好的分布外基准预测。这些改进是可能的，因为现有的方法都没有以适当的方式将模型规模与功能分开。模型的尺度应该告知能力，如在比例律中，这些能力应该告知观测结果，直到测量误差，如在潜在因子模型中。在结合这两个见解，结构化的能力表现出更好的解释和预测能力，量化LLM评估的结构效度。
摘要：The LLM community often reports benchmark results as if they are synonymous with general model capabilities. However, benchmarks can have problems that distort performance, like test set contamination and annotator error. How can we know that a benchmark is a reliable indicator of some capability that we want to measure? This question concerns the construct validity of LLM benchmarks, and it requires separating benchmark results from capabilities when we model and predict LLM performance. Both social scientists and computer scientists propose formal models - latent factor models and scaling laws - for identifying the capabilities underlying benchmark scores. However, neither technique is satisfactory for construct validity. Latent factor models ignore scaling laws, and as a result, the capabilities they extract often proxy model size. Scaling laws ignore measurement error, and as a result, the capabilities they extract are both uninterpretable and overfit to the observed benchmarks. This thesis presents the structured capabilities model, the first model to extract interpretable and generalisable capabilities from a large collection of LLM benchmark results. I fit this model and its two alternatives on a large sample of results from the OpenLLM Leaderboard. Structured capabilities outperform latent factor models on parsimonious fit indices, and exhibit better out-of-distribution benchmark prediction than scaling laws. These improvements are possible because neither existing approach separates model scale from capabilities in the appropriate way. Model scale should inform capabilities, as in scaling laws, and these capabilities should inform observed results up to measurement error, as in latent factor models. In combining these two insights, structured capabilities demonstrate better explanatory and predictive power for quantifying construct validity in LLM evaluations.
【5】ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
标题：ExpertWeaver：解锁具有GLU激活模式的密集LLM中的固有MoE
链接：https://arxiv.org/abs/2602.15521
摘要：混合专家（MoE）有效地扩展模型容量，同时通过稀疏的专家激活保持计算效率。然而，从零开始培训高质量的教育部是昂贵的。一个有希望的替代方案是将预训练的密集模型转换为稀疏MoE。现有的密集到MoE方法分为两类：\textbf{动态结构修剪}，将密集模型转换为具有适度稀疏性的MoE架构，以平衡性能和推理效率，以及\textbf{向下循环}方法，使用预先训练的密集模型来初始化高度稀疏的MoE架构。然而，现有的方法打破了内在的激活模式内密集的模型，导致次优的专家建设。在这项工作中，我们认为，门控线性单元（GLU）机制提供了一个自然的蓝图，密集到MoE转换。我们发现，细粒度的神经明智的激活模式的GLU揭示了一个粗粒度的结构，揭示了一个固有的MoE架构组成的一致激活的通用神经元和动态激活的专门的神经元。利用这一发现，我们引入了ExpertWeaver，这是一个无需训练的框架，它根据神经元的激活模式对其进行分区，并构建具有层自适应配置的共享专家和专业路由专家。我们的实验表明，ExpertWeaver显着优于现有的方法，无论是作为一个培训免费的动态结构修剪技术，并作为一个向下循环的策略，优越的MoE初始化。
摘要：Mixture-of-Experts (MoE) effectively scales model capacity while preserving computational efficiency through sparse expert activation. However, training high-quality MoEs from scratch is prohibitively expensive. A promising alternative is to convert pretrained dense models into sparse MoEs. Existing dense-to-MoE methods fall into two categories: \textbf{dynamic structural pruning} that converts dense models into MoE architectures with moderate sparsity to balance performance and inference efficiency, and \textbf{downcycling} approaches that use pretrained dense models to initialize highly sparse MoE architectures. However, existing methods break the intrinsic activation patterns within dense models, leading to suboptimal expert construction. In this work, we argue that the Gated Linear Unit (GLU) mechanism provides a natural blueprint for dense-to-MoE conversion. We show that the fine-grained neural-wise activation patterns of GLU reveal a coarse-grained structure, uncovering an inherent MoE architecture composed of consistently activated universal neurons and dynamically activated specialized neurons. Leveraging this discovery, we introduce ExpertWeaver, a training-free framework that partitions neurons according to their activation patterns and constructs shared experts and specialized routed experts with layer-adaptive configurations. Our experiments demonstrate that ExpertWeaver significantly outperforms existing methods, both as a training-free dynamic structural pruning technique and as a downcycling strategy for superior MoE initialization.
【6】LLM-as-Judge on a Budget
标题：法学硕士预算法官
链接：https://arxiv.org/abs/2602.15481
摘要：LLM-as-a-judge已经成为评估大型语言模型的基础技术，它利用LLM推理来对测试-响应对进行评分。由于LLM的判断是随机的，从业者通常会多次查询每一对，以准确估计平均得分。这提出了一个关键的挑战：给定一个固定的计算预算$B$，如何最佳地分配查询跨$K$响应对，以尽量减少估计误差？%我们提出了一个原则性的方差自适应方法，利用多臂强盗理论和浓度不等式。我们的方法根据估计的分数方差动态分配查询，将资源集中在不确定性最高的地方。此外，我们的算法实现了最坏情况下的得分估计误差$\tilde{O}\left（\sqrt{\frac{\sum_{i=1}^K σ_i^2}{B}}\right）$，$σ_i^2$是具有接近最优预算分配的对$i \in [K]$的未知得分方差。%在\endash {Summarize-From-Feedback}和\endash {HelpSteer 2}上的实验表明，我们的方法显著优于均匀分配，在保持相同预算的同时减少了最坏情况下的估计误差。我们的工作为有效的LLM评估奠定了理论基础，并对AI安全性，模型对齐和大规模自动化评估产生了实际影响。
摘要：LLM-as-a-judge has emerged as a cornerstone technique for evaluating large language models by leveraging LLM reasoning to score prompt-response pairs. Since LLM judgments are stochastic, practitioners commonly query each pair multiple times to estimate mean scores accurately. This raises a critical challenge: given a fixed computational budget $B$, how to optimally allocate queries across $K$ prompt-response pairs to minimize estimation error? % We present a principled variance-adaptive approach leveraging multi-armed bandit theory and concentration inequalities. Our method dynamically allocates queries based on estimated score variances, concentrating resources where uncertainty is highest. Further, our algorithm is shown to achieve a worst-case score-estimation error of $\tilde{O}\left(\sqrt{\frac{\sum_{i=1}^K σ_i^2}{B}}\right)$, $σ_i^2$ being the unknown score variance for pair $i \in [K]$ with near-optimal budget allocation. % Experiments on \emph{Summarize-From-Feedback} and \emph{HelpSteer2} demonstrate that our method significantly outperforms uniform allocation, reducing worst-case estimation error while maintaining identical budgets. Our work establishes a theoretical foundation for efficient LLM evaluation with practical implications for AI safety, model alignment, and automated assessment at scale.
【7】On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks
标题：关于简单视觉规划任务的多模式LLM中推理的分布外推广
链接：https://arxiv.org/abs/2602.15460
摘要：在大型语言模型和大型视觉语言模型中集成推理最近导致了它们的能力的显着提高。然而，推理模型的泛化仍然是模糊的定义和理解不足。在这项工作中，我们提出了一个评估框架，严格审查如何以及链的思想（CoT）的方法概括一个简单的规划任务。具体来说，我们考虑一个基于网格的导航任务，其中一个模型提供了一个地图，必须输出一系列的动作，引导玩家从一个开始位置到一个目标，同时避免障碍物。任务及其数据的多功能性使我们能够使用不同的输入表示（视觉和文本）和CoT推理策略来微调模型变体，并在分布（ID）和分布（OOD）测试条件下对其进行系统评估。我们的实验表明，虽然CoT推理提高了所有表示的分布内泛化，但分布外泛化（例如，对于更大的地图）在大多数情况下，在控制与ID数据的琐碎匹配时仍然非常有限。令人惊讶的是，我们发现，结合多种文本格式的推理痕迹产生最好的（和非平凡的）OOD概括。最后，纯粹基于文本的模型始终优于那些利用基于图像的输入，包括最近提出的依赖于潜在空间推理的方法。
摘要：Integrating reasoning in large language models and large vision-language models has recently led to significant improvement of their capabilities. However, the generalization of reasoning models is still vaguely defined and poorly understood. In this work, we present an evaluation framework to rigorously examine how well chain-of-thought (CoT) approaches generalize on a simple planning task. Specifically, we consider a grid-based navigation task in which a model is provided with a map and must output a sequence of moves that guides a player from a start position to a goal while avoiding obstacles. The versatility of the task and its data allows us to fine-tune model variants using different input representations (visual and textual) and CoT reasoning strategies, and systematically evaluate them under both in-distribution (ID) and out-of-distribution (OOD) test conditions. Our experiments show that, while CoT reasoning improves in-distribution generalization across all representations, out-of-distribution generalization (e.g., to larger maps) remains very limited in most cases when controlling for trivial matches with the ID data. Surprisingly, we find that reasoning traces which combine multiple text formats yield the best (and non-trivial) OOD generalization. Finally, purely text-based models consistently outperform those utilizing image-based inputs, including a recently proposed approach relying on latent space reasoning.
【8】TAROT: Test-driven and Capability-adaptive Curriculum Reinforcement Fine-tuning for Code Generation with Large Language Models
标题：TAROT：测试驱动和能力自适应课程强化针对大型语言模型代码生成的微调
链接：https://arxiv.org/abs/2602.15449
备注：The first three authors contributed equally to this work; listing order is random
摘要：大型语言模型（LLM）正在改变编码范式，称为vibe编码，但合成算法复杂和健壮的代码仍然是一个关键挑战。激励LLM的深度推理能力对于克服这一障碍至关重要。强化微调（RFT）已成为解决这一需求的一种有前途的策略。然而，大多数现有的方法忽略了测试用例中固有的异构难度和粒度，导致奖励信号的不平衡分布，从而在训练过程中有偏见的梯度更新。为了解决这个问题，我们提出了测试驱动和能力自适应的文化增强微调（TAROT）。TAROT系统地为每个问题构建了一个四层测试套件（基本，中级，复杂，边缘），为课程设计和评估提供了一个可控的难度景观。至关重要的是，TAROT将课程进展与原始奖励分数相结合，使能力条件评价和课程政策组合中的原则选择成为可能，而不是偶然的测试案例难度组合。这种设计促进了稳定的优化和更有效的能力获取。广泛的实验结果表明，RFT在代码生成的最佳课程是紧密联系在一起的模型的内在能力，能力较低的模型实现更大的收益，从容易到困难的进展，而更有能力的模型下，一个硬第一的课程表现出色。TAROT提供了一种可重复的方法，可以根据模型的能力自适应地调整课程设计，从而不断提高生成代码的功能正确性和鲁棒性。所有代码和数据均在https://github.com/deep-diver/TAROT上发布，以促进可重复性并推进社区研究。
摘要：Large Language Models (LLMs) are changing the coding paradigm, known as vibe coding, yet synthesizing algorithmically sophisticated and robust code still remains a critical challenge. Incentivizing the deep reasoning capabilities of LLMs is essential to overcoming this hurdle. Reinforcement Fine-Tuning (RFT) has emerged as a promising strategy to address this need. However, most existing approaches overlook the heterogeneous difficulty and granularity inherent in test cases, leading to an imbalanced distribution of reward signals and consequently biased gradient updates during training. To address this, we propose Test-driven and cApability-adaptive cuRriculum reinfOrcement fine-Tuning (TAROT). TAROT systematically constructs, for each problem, a four-tier test suite (basic, intermediate, complex, edge), providing a controlled difficulty landscape for curriculum design and evaluation. Crucially, TAROT decouples curriculum progression from raw reward scores, enabling capability-conditioned evaluation and principled selection from a portfolio of curriculum policies rather than incidental test-case difficulty composition. This design fosters stable optimization and more efficient competency acquisition. Extensive experimental results reveal that the optimal curriculum for RFT in code generation is closely tied to a model's inherent capability, with less capable models achieving greater gains with an easy-to-hard progression, whereas more competent models excel under a hard-first curriculum. TAROT provides a reproducible method that adaptively tailors curriculum design to a model's capability, thereby consistently improving the functional correctness and robustness of the generated code. All code and data are released to foster reproducibility and advance community research at https://github.com/deep-diver/TAROT.
【9】ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models
标题：ER-MIA：对长期内存增强大型语言模型的黑匣子对抗性内存注入攻击
链接：https://arxiv.org/abs/2602.15344
摘要：大型语言模型（LLM）越来越多地通过长期记忆系统进行增强，以克服有限的上下文窗口并实现跨交互的持续推理。然而，最近的研究发现，LLM变得更加脆弱，因为内存提供了额外的攻击面。在本文中，我们首次系统地研究了黑盒对抗性记忆注入攻击，该攻击针对长期记忆增强LLM中基于相似性的检索机制。我们介绍ER-MIA，一个统一的框架，暴露了这个漏洞，并正式确定了两个现实的攻击设置：基于内容的攻击和问题为目标的攻击。在这些设置中，ER-MIA包括一个可组合的攻击原语和集成攻击的武器库，在最小的攻击者假设下实现高成功率。跨多个LLM和长期记忆系统的广泛实验表明，基于相似性的检索构成了一个基本的系统级漏洞，揭示了在内存设计和应用场景中持续存在的安全风险。
摘要：Large language models (LLMs) are increasingly augmented with long-term memory systems to overcome finite context windows and enable persistent reasoning across interactions. However, recent research finds that LLMs become more vulnerable because memory provides extra attack surfaces. In this paper, we present the first systematic study of black-box adversarial memory injection attacks that target the similarity-based retrieval mechanism in long-term memory-augmented LLMs. We introduce ER-MIA, a unified framework that exposes this vulnerability and formalizes two realistic attack settings: content-based attacks and question-targeted attacks. In these settings, ER-MIA includes an arsenal of composable attack primitives and ensemble attacks that achieve high success rates under minimal attacker assumptions. Extensive experiments across multiple LLMs and long-term memory systems demonstrate that similarity-based retrieval constitutes a fundamental and system-level vulnerability, revealing security risks that persist across memory designs and application scenarios.
【10】Discovering Implicit Large Language Model Alignment Objectives
标题：发现隐性大型语言模型对齐目标
链接：https://arxiv.org/abs/2602.15338
摘要 ：大型语言模型（LLM）对齐依赖于复杂的奖励信号，这些信号通常会掩盖被激励的特定行为，从而产生不对齐和奖励黑客的关键风险。现有的解释方法通常依赖于预定义的规则，冒着遗漏“未知的未知数”的风险，或者无法识别全面覆盖模型行为并与模型行为因果关系的目标。为了解决这些限制，我们引入了Obj-Disco，这是一个框架，可以自动将对齐奖励信号分解为人类可解释的自然语言目标的稀疏加权组合。我们的方法利用迭代贪婪算法来分析训练检查点之间的行为变化，识别和验证最能解释剩余奖励信号的候选目标。对不同任务、模型大小和对齐算法的广泛评估证明了该框架的鲁棒性。对流行的开源奖励模型的实验表明，该框架始终捕获了> 90%的奖励行为，这一发现得到了人类评估的进一步证实。此外，一个与开源奖励模型对齐的案例研究表明，Obj-Disco可以成功地识别出与预期行为一起出现的潜在错位激励。我们的工作为揭示LLM对齐中的隐含目标提供了重要工具，为更透明，更安全的AI开发铺平了道路。
摘要：Large language model (LLM) alignment relies on complex reward signals that often obscure the specific behaviors being incentivized, creating critical risks of misalignment and reward hacking. Existing interpretation methods typically rely on pre-defined rubrics, risking the omission of "unknown unknowns", or fail to identify objectives that comprehensively cover and are causal to the model behavior. To address these limitations, we introduce Obj-Disco, a framework that automatically decomposes an alignment reward signal into a sparse, weighted combination of human-interpretable natural language objectives. Our approach utilizes an iterative greedy algorithm to analyze behavioral changes across training checkpoints, identifying and validating candidate objectives that best explain the residual reward signal. Extensive evaluations across diverse tasks, model sizes, and alignment algorithms demonstrate the framework's robustness. Experiments with popular open-source reward models show that the framework consistently captures > 90% of reward behavior, a finding further corroborated by human evaluation. Additionally, a case study on alignment with an open-source reward model reveals that Obj-Disco can successfully identify latent misaligned incentives that emerge alongside intended behaviors. Our work provides a crucial tool for uncovering the implicit objectives in LLM alignment, paving the way for more transparent and safer AI development.
【11】Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
标题：规定性缩放揭示语言模型能力的演变
链接：https://arxiv.org/abs/2602.15327
备注：Blog Post: https://jkjin.com/prescriptive-scaling
摘要：为了部署基础模型，从业者越来越需要规范的缩放定律：给定训练前的计算预算，当代训练后的实践可以达到什么样的下游精度，以及随着领域的发展，映射的稳定性如何？使用5k观测数据和2k新采样数据对模型性能进行大规模观测评估，我们通过具有单调饱和sigmoid参数化的平滑分位数回归来估计能力边界，基准得分的高条件分位数作为对数预训练FLOP的函数。我们验证了时间的可靠性，通过拟合早期的模型代和评估以后的版本。在各种任务中，估计的边界大多是稳定的，除了数学推理随着时间的推移表现出一致的前进边界。然后，我们扩展我们的方法来分析任务相关的饱和度和探测污染相关的数学推理任务的转变。最后，我们介绍了一种高效的算法，使用大约20%的评估预算恢复接近完整的数据边界。我们的工作共同发布了最新的模型性能评估数据集Proteus 2k，并引入了一种实用的方法，用于将计算预算转化为可靠的性能预期，并用于监控能力边界何时随时间推移而变化。
摘要：For deploying foundation models, practitioners increasingly need prescriptive scaling laws: given a pre training compute budget, what downstream accuracy is attainable with contemporary post training practice, and how stable is that mapping as the field evolves? Using large scale observational evaluations with 5k observational and 2k newly sampled data on model performance, we estimate capability boundaries, high conditional quantiles of benchmark scores as a function of log pre training FLOPs, via smoothed quantile regression with a monotone, saturating sigmoid parameterization. We validate the temporal reliability by fitting on earlier model generations and evaluating on later releases. Across various tasks, the estimated boundaries are mostly stable, with the exception of math reasoning that exhibits a consistently advancing boundary over time. We then extend our approach to analyze task dependent saturation and to probe contamination related shifts on math reasoning tasks. Finally, we introduce an efficient algorithm that recovers near full data frontiers using roughly 20% of evaluation budget. Together, our work releases the Proteus 2k, the latest model performance evaluation dataset, and introduces a practical methodology for translating compute budgets into reliable performance expectations and for monitoring when capability boundaries shift across time.
【12】Unforgeable Watermarks for Language Models via Robust Signatures
标题：通过稳健签名为语言模型提供不可伪造的水印
链接：https://arxiv.org/abs/2602.15323
备注：60 pages, 7 figures
摘要：语言模型现在通常会生成难以与人类文字区分的文本，这就需要强大的工具来验证内容来源。数字水印作为一种有前途的对策已经出现，现有的工作主要集中在模型质量保持和鲁棒检测。然而，目前的计划提供有限的保护，防止虚假归属。我们通过引入两个新的保证：不可伪造性和可恢复性来加强可靠性的概念。不可伪造性可以防止攻击者制造误报，这些文本远离水印模型的任何输出，但仍然被标记为水印。可恢复性提供了一个额外的保护层：每当检测到水印时，检测器就会识别标记内容所源自的源文本。总之，这些属性通过将内容专门链接到其生成模型来加强内容所有权，从而实现安全归属和细粒度可追溯性。我们构造了第一个不可检测的水印方案，该方案是鲁棒的、不可伪造的，并且关于替换是可恢复的（即，Hamming Metric的一个例子）。关键的技术要素是一种称为稳健（或可恢复）数字签字的新的密码学基本要素，它允许核查接近签字的电文，同时防止伪造与所有以前签字的电文相差甚远的电文。我们证明，任何标准的数字签名方案都可以使用保持属性的哈希函数提升为鲁棒的数字签名方案（Boyle，LaVigne和Vaikuntanathan，ITCS 2019）。
摘要：Language models now routinely produce text that is difficult to distinguish from human writing, raising the need for robust tools to verify content provenance. Watermarking has emerged as a promising countermeasure, with existing work largely focused on model quality preservation and robust detection. However, current schemes provide limited protection against false attribution. We strengthen the notion of soundness by introducing two novel guarantees: unforgeability and recoverability. Unforgeability prevents adversaries from crafting false positives, texts that are far from any output from the watermarked model but are nonetheless flagged as watermarked. Recoverability provides an additional layer of protection: whenever a watermark is detected, the detector identifies the source text from which the flagged content was derived. Together, these properties strengthen content ownership by linking content exclusively to its generating model, enabling secure attribution and fine-grained traceability. We construct the first undetectable watermarking scheme that is robust, unforgeable, and recoverable with respect to substitutions (i.e., perturbations in Hamming metric). The key technical ingredient is a new cryptographic primitive called robust (or recoverable) digital signatures, which allow verification of messages that are close to signed ones, while preventing forgery of messages that are far from all previously signed messages. We show that any standard digital signature scheme can be boosted to a robust one using property-preserving hash functions (Boyle, LaVigne, and Vaikuntanathan, ITCS 2019).
【13】Closing the Distribution Gap in Adversarial Training for LLMs
标题：缩小法学硕士对抗性训练的分布差距
链接：https://arxiv.org/abs/2602.15238
摘要：LLM的对抗性训练是可靠地提高对抗对手的鲁棒性的最有前途的方法之一。然而，尽管取得了重大进展，模型仍然容易受到简单的发行中漏洞的攻击，例如重写过去时态的提示或将其翻译成其他语言。我们认为，这种持续的脆弱性源于当前对抗训练算法的根本局限性：它们最大限度地减少了训练集上的对抗损失，但没有充分覆盖数据分布，导致容易受到看似简单的攻击。为了弥合这一差距，我们提出了分布式对抗训练（DAT）。我们利用扩散LLM来近似提示和响应的真实联合分布，从而生成多样化的高似然样本，解决泛化失败问题。通过将扩散模型提供的数据分布优化与连续对抗训练相结合，DAT实现了比以前方法更高的对抗鲁棒性。
摘要 ：Adversarial training for LLMs is one of the most promising methods to reliably improve robustness against adversaries. However, despite significant progress, models remain vulnerable to simple in-distribution exploits, such as rewriting prompts in the past tense or translating them into other languages. We argue that this persistent fragility stems from a fundamental limitation in current adversarial training algorithms: they minimize adversarial loss on their training set but inadequately cover the data distribution, resulting in vulnerability to seemingly simple attacks. To bridge this gap, we propose Distributional Adversarial Training, DAT. We leverage Diffusion LLMs to approximate the true joint distribution of prompts and responses, enabling generation of diverse, high-likelihood samples that address generalization failures. By combining optimization over the data distribution provided by the diffusion model with continuous adversarial training, DAT achieves substantially higher adversarial robustness than previous methods.
Graph相关(图学习|图神经网络|图优化等)(3篇)
【1】MRC-GAT: A Meta-Relational Copula-Based Graph Attention Network for Interpretable Multimodal Alzheimer's Disease Diagnosis
标题：MRC-GAT：一个基于元关系的图注意力网络，用于可解释多模式阿尔茨海默病诊断
链接：https://arxiv.org/abs/2602.15740
备注：27 pages, 10 figures, 10 table
摘要：阿尔茨海默病（AD）是一种进行性神经退行性疾病，需要早期和精确的诊断以提供及时的临床治疗。鉴于早期诊断的重要性，最近的研究越来越多地集中在计算机辅助诊断模型，以提高准确性和可靠性。然而，大多数基于图的方法仍然依赖于固定的结构设计，这限制了它们的灵活性，并限制了异构患者数据的泛化。为了克服这些局限性，元关系Copula为基础的图注意力网络（MRC-GAT）提出了一个有效的多模态模型的AD分类任务。所提出的架构，基于Copula的相似性对齐，关系注意力和节点融合集成为情节元学习的核心组件，这样的多模态特征，包括风险因素（RF），认知测试分数和MRI属性，首先通过基于Copula的变换在一个共同的统计空间中对齐，然后通过多关系注意力机制组合。根据对TADPOLE和NACC数据集进行的评估，MRC-GAT模型的准确率分别为96.87%和92.31%，与现有诊断模型相比，表现出最先进的性能。最后，该模型通过在疾病诊断的各个阶段提供可解释性，证实了所提出的方法的鲁棒性和适用性。
摘要：Alzheimer's disease (AD) is a progressive neurodegenerative condition necessitating early and precise diagnosis to provide prompt clinical management. Given the paramount importance of early diagnosis, recent studies have increasingly focused on computer-aided diagnostic models to enhance precision and reliability. However, most graph-based approaches still rely on fixed structural designs, which restrict their flexibility and limit generalization across heterogeneous patient data. To overcome these limitations, the Meta-Relational Copula-Based Graph Attention Network (MRC-GAT) is proposed as an efficient multimodal model for AD classification tasks. The proposed architecture, copula-based similarity alignment, relational attention, and node fusion are integrated as the core components of episodic meta-learning, such that the multimodal features, including risk factors (RF), Cognitive test scores, and MRI attributes, are first aligned via a copula-based transformation in a common statistical space and then combined by a multi-relational attention mechanism. According to evaluations performed on the TADPOLE and NACC datasets, the MRC-GAT model achieved accuracies of 96.87% and 92.31%, respectively, demonstrating state-of-the-art performance compared to existing diagnostic models. Finally, the proposed model confirms the robustness and applicability of the proposed method by providing interpretability at various stages of disease diagnosis.
【2】Random Wavelet Features for Graph Kernel Machines
标题：图核机的随机子波特征
链接：https://arxiv.org/abs/2602.15711
备注：This paper is an extended version of a paper submitted to the 2026 European Signal Processing Conference (EUSIPCO 2026). It contains supplementary material including the full proof to Proposition 1
摘要：节点嵌入将图的顶点映射到低维欧氏空间，同时保留结构信息。它们是节点分类、链路预测和信号重构等任务的核心。一个关键目标是设计节点嵌入，其点积捕获由图引起的节点相似性的有意义的概念。图核提供了一种原则性的方法来定义这种相似性，但它们的直接计算通常对于大型网络是禁止的。受欧氏空间核近似的随机特征方法的启发，我们引入了随机谱节点嵌入，其点积估计任何特定图形核的低秩近似。我们提供的理论和实证结果表明，我们的嵌入实现更准确的内核近似比现有的方法，特别是对光谱本地化的内核。这些结果证明了随机谱结构对可扩展和有原则的图表示学习的有效性。
摘要：Node embeddings map graph vertices into low-dimensional Euclidean spaces while preserving structural information. They are central to tasks such as node classification, link prediction, and signal reconstruction. A key goal is to design node embeddings whose dot products capture meaningful notions of node similarity induced by the graph. Graph kernels offer a principled way to define such similarities, but their direct computation is often prohibitive for large networks. Inspired by random feature methods for kernel approximation in Euclidean spaces, we introduce randomized spectral node embeddings whose dot products estimate a low-rank approximation of any specific graph kernel. We provide theoretical and empirical results showing that our embeddings achieve more accurate kernel approximations than existing methods, particularly for spectrally localized kernels. These results demonstrate the effectiveness of randomized spectral constructions for scalable and principled graph representation learning.
【3】Size Transferability of Graph Transformers with Convolutional Positional Encodings
标题：卷积位置编码下图变换器的尺寸可传递性
链接：https://arxiv.org/abs/2602.15239
摘要：Transformers在各个领域都取得了显著的成功，这促使了图Transformers（GT）作为图结构数据的基于注意力的架构的兴起。GT中的一个关键设计选择是使用基于图形神经网络（GNN）的位置编码来合并结构信息。在这项工作中，我们研究GT通过镜头的流形极限模型的图序列，并建立了理论上的连接GT与GNN位置编码和流形神经网络（MNN）。基于流形收敛下GNNs的可转移性结果，我们证明了GT从其位置编码中继承了可转移性保证。特别是，在小图上训练的GT在温和的假设下可证明推广到更大的图。我们补充了我们的理论与标准图基准测试的广泛实验，证明GT表现出与GNN相当的可扩展行为。为了进一步显示在现实世界的情况下的效率，我们实现GT的最短路径距离估计的地形，以更好地说明可转移GT的效率。我们的研究结果为理解GT提供了新的见解，并为在大规模环境中有效培训GT提出了实际方向。
摘要：Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for graph-structured data. A key design choice in GTs is the use of Graph Neural Network (GNN)-based positional encodings to incorporate structural information. In this work, we study GTs through the lens of manifold limit models for graph sequences and establish a theoretical connection between GTs with GNN positional encodings and Manifold Neural Networks (MNNs). Building on transferability results for GNNs under manifold convergence, we show that GTs inherit transferability guarantees from their positional encodings. In particular, GTs trained on small graphs provably generalize to larger graphs under mild assumptions. We complement our theory with extensive experiments on standard graph benchmarks, demonstrating that GTs exhibit scalable behavior on par with GNNs. To further show the efficiency in a real-world scenario, we implement GTs for shortest path distance estimation over terrains to better illustrate the efficiency of the transferable GTs. Our results provide new insights into the understanding of GTs and suggest practical directions for efficient training of GTs in large-scale settings.
Transformer(5篇)
【1】Approximation Theory for Lipschitz Continuous Transformers
标题：Lipschitz连续Transformer的逼近理论
链接：https://arxiv.org/abs/2602.15503
摘要：稳定性和鲁棒性对于在安全敏感环境中部署Transformers至关重要。强制执行此类行为的原则性方法是约束模型的Lipschitz常数。然而，近似理论的保证体系结构，明确保持Lipschitz连续性尚未建立。在这项工作中，我们通过引入一类梯度下降型的上下文中的Transformers，是Lipschitz连续的建设弥合这一差距。我们将MLP和注意力块都实现为负梯度流的显式欧拉步骤，确保了固有的稳定性，而不会牺牲表现力。在Lipschitz约束函数空间中证明了这类函数的一个普适逼近定理。至关重要的是，我们的分析采用了测量理论的形式主义，解释Transformers作为运营商的概率措施，产生近似保证独立的令牌计数。这些结果为设计鲁棒的Lipschitz连续Transformer结构提供了严格的理论基础。
摘要：Stability and robustness are critical for deploying Transformers in safety-sensitive settings. A principled way to enforce such behavior is to constrain the model's Lipschitz constant. However, approximation-theoretic guarantees for architectures that explicitly preserve Lipschitz continuity have yet to be established. In this work, we bridge this gap by introducing a class of gradient-descent-type in-context Transformers that are Lipschitz-continuous by construction. We realize both MLP and attention blocks as explicit Euler steps of negative gradient flows, ensuring inherent stability without sacrificing expressivity. We prove a universal approximation theorem for this class within a Lipschitz-constrained function space. Crucially, our analysis adopts a measure-theoretic formalism, interpreting Transformers as operators on probability measures, to yield approximation guarantees independent of token count. These results provide a rigorous theoretical foundation for the design of robust, Lipschitz continuous Transformer architectures.
【2】Scaling Laws for Masked-Reconstruction Transformers on Single-Cell Transcriptomics
标题：单细胞转录组学中掩蔽重建变形者的比例定律
链接：https://arxiv.org/abs/2602.15253
摘要：神经标度定律-损失，模型大小和数据之间的幂律关系-已经被广泛记录为语言和Vision Transformers，但它们在单细胞基因组学中的存在仍然在很大程度上未被探索。我们提出了对单细胞RNA测序（scRNA-seq）数据训练的掩蔽重建Transformers的缩放行为的第一个系统研究。使用来自CELLxGENE Census的表达谱，我们构建了两个实验方案：数据丰富的方案（512个高度可变的基因，200，000个细胞）和数据有限的方案（1，024个基因，10，000个细胞）。在参数计数跨越三个数量级的七个模型大小（533至3.4 x 10^8个参数）中，我们将参数标度律拟合到验证均方误差（MSE）。数据丰富的制度表现出明确的幂律缩放与不可约的损失地板的c ~ 1.44，而数据有限的制度显示可以忽略不计的缩放，表明模型容量是没有约束力的约束时，数据是稀缺的。这些结果表明，当有足够的数据时，与自然语言处理中观察到的相似的缩放定律确实会出现在单细胞转录组学中，并且它们将数据与参数的比率确定为缩放行为的关键决定因素。初步转换的数据丰富的渐近地板的信息理论单位产生的估计约2.30位的熵每个掩蔽基因的位置。我们讨论了单细胞基础模型的设计的影响，并概述了改进这种熵估计所需的额外测量。
摘要：Neural scaling laws -- power-law relationships between loss, model size, and data -- have been extensively documented for language and vision transformers, yet their existence in single-cell genomics remains largely unexplored. We present the first systematic study of scaling behaviour for masked-reconstruction transformers trained on single-cell RNA sequencing (scRNA-seq) data. Using expression profiles from the CELLxGENE Census, we construct two experimental regimes: a data-rich regime (512 highly variable genes, 200,000 cells) and a data-limited regime (1,024 genes, 10,000 cells). Across seven model sizes spanning three orders of magnitude in parameter count (533 to 3.4 x 10^8 parameters), we fit the parametric scaling law to validation mean squared error (MSE). The data-rich regime exhibits clear power-law scaling with an irreducible loss floor of c ~ 1.44, while the data-limited regime shows negligible scaling, indicating that model capacity is not the binding constraint when data are scarce. These results establish that scaling laws analogous to those observed in natural language processing do emerge in single-cell transcriptomics when sufficient data are available, and they identify the data-to-parameter ratio as a critical determinant of scaling behaviour. A preliminary conversion of the data-rich asymptotic floor to information-theoretic units yields an estimate of approximately 2.30 bits of entropy per masked gene position. We discuss implications for the design of single-cell foundation models and outline the additional measurements needed to refine this entropy estimate.
【3】COMPOT: Calibration-Optimized Matrix Procrustes Orthogonalization for Transformers Compression
标题：COMOT：经过校准优化的矩阵Procrustes用于Transformers压缩的超同步化
链接：https://arxiv.org/abs/2602.15200
摘要：Transformer模型的训练后压缩通常依赖于截断奇异值分解（SVD）。然而，即使在中等压缩下，强制使用单个共享子空间也会降低精度。稀疏字典学习提供了一种更灵活的子空间联合表示，但现有的方法经常受到迭代字典和系数更新的影响。我们提出了COMOT（校准优化矩阵Procrustes分解Transformers），一个无训练的压缩框架，使用一个小的校准数据集来估计稀疏的权重分解。COMOT采用正交字典，使封闭形式的Procrustes更新的字典和分析单步稀疏编码的系数，消除迭代优化。为了在全局压缩预算下处理异构层敏感性，COMDOT进一步引入了一次动态分配策略，可自适应地重新分配逐层压缩率。跨不同架构和任务的广泛实验表明，COMOT始终在强大的低秩和稀疏基线上提供卓越的质量压缩权衡，同时与训练后量化完全兼容，以实现极端压缩。代码可在$\href{https：//github.com/mts-ai/COMPOT}{here}$获得。
摘要：Post-training compression of Transformer models commonly relies on truncated singular value decomposition (SVD). However, enforcing a single shared subspace can degrade accuracy even at moderate compression. Sparse dictionary learning provides a more flexible union-of-subspaces representation, but existing approaches often suffer from iterative dictionary and coefficient updates. We propose COMPOT (Calibration-Optimized Matrix Procrustes Orthogonalization for Transformers), a training-free compression framework that uses a small calibration dataset to estimate a sparse weight factorization. COMPOT employs orthogonal dictionaries that enable closed-form Procrustes updates for the dictionary and analytical single-step sparse coding for the coefficients, eliminating iterative optimization. To handle heterogeneous layer sensitivity under a global compression budget, COMPOT further introduces a one-shot dynamic allocation strategy that adaptively redistributes layer-wise compression rates. Extensive experiments across diverse architectures and tasks show that COMPOT consistently delivers a superior quality-compression trade-off over strong low-rank and sparse baselines, while remaining fully compatible with post-training quantization for extreme compression. Code is available $\href{https://github.com/mts-ai/COMPOT}{here}$.
【4】Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction
标题：基于瓶颈转换器的改进自动STIOI评分预测方法
链接：https://arxiv.org/abs/2602.15484
备注：7 pages, 7 tables, 2 figures, ASRU 2025
摘要：在这项研究中，我们提出了一种新的方法来预测短期目标可懂度（STOI）度量使用瓶颈Transformer架构。用于计算STOI的传统方法通常需要干净的参考语音，这限制了它们在现实世界中的适用性。为了解决这个问题，许多基于深度学习的非侵入式语音评估模型引起了人们的极大兴趣。许多研究取得了令人称道的成绩，但仍有进一步改进的余地。 我们建议使用瓶颈Transformer，将卷积块用于学习帧级特征和多头自注意（MHSA）层来聚合信息。这些组件使Transformer能够专注于输入数据的关键方面。与使用自监督学习（SSL）和光谱特征作为输入的最先进模型相比，我们的模型在可见和不可见的场景中表现出更高的相关性和更低的均方误差。
摘要 ：In this study, we have presented a novel approach to predict the Short-Time Objective Intelligibility (STOI) metric using a bottleneck transformer architecture. Traditional methods for calculating STOI typically requires clean reference speech, which limits their applicability in the real world. To address this, numerous deep learning-based nonintrusive speech assessment models have garnered significant interest. Many studies have achieved commendable performance, but there is room for further improvement. We propose the use of bottleneck transformer, incorporating convolution blocks for learning frame-level features and a multi-head self-attention (MHSA) layer to aggregate the information. These components enable the transformer to focus on the key aspects of the input data. Our model has shown higher correlation and lower mean squared error for both seen and unseen scenarios compared to the state-of-the-art model using self-supervised learning (SSL) and spectral features as inputs.
【5】TokaMind: A Multi-Modal Transformer Foundation Model for Tokamak Plasma Dynamics
标题：TokaMind：Tokamak等离子体动力学的多模式Transformer基础模型
链接：https://arxiv.org/abs/2602.15084
摘要：我们提出了TokaMind，一个开源的基础模型框架，用于聚变等离子体建模，基于多模态Transformer（MMT）和训练异构托卡马克诊断从公开可用的MAST数据集。TokaMind支持多种数据模式（时间序列、2D配置文件和视频），具有不同的采样率、强大的丢失信号处理以及通过选择性加载和冻结四个模型组件进行的高效任务自适应。为了表示多模态信号，我们使用无训练离散余弦变换嵌入（DCT 3D），并为替代嵌入提供干净的接口（例如，可变自动编码器-VAE）。我们在最近推出的MAST基准TokaMark上评估TokaMind，比较训练和嵌入策略。我们的结果表明，经过微调的TokaMind在除一项任务外的所有任务上都优于基准基线，并且对于多项任务，轻量级微调比在匹配的历元预算下从头开始训练相同的架构产生更好的性能。这些发现突出了托卡马克等离子体动力学的多模态预训练的好处，并为未来的融合建模任务提供了一个实用的，可扩展的基础。训练代码和模型权重将公开提供。
摘要：We present TokaMind, an open-source foundation model framework for fusion plasma modeling, based on a Multi-Modal Transformer (MMT) and trained on heterogeneous tokamak diagnostics from the publicly available MAST dataset. TokaMind supports multiple data modalities (time-series, 2D profiles, and videos) with different sampling rates, robust missing-signal handling, and efficient task adaptation via selectively loading and freezing four model components. To represent multi-modal signals, we use a training-free Discrete Cosine Transform embedding (DCT3D) and provide a clean interface for alternative embeddings (e.g., Variational Autoencoders - VAEs). We evaluate TokaMind on the recently introduced MAST benchmark TokaMark, comparing training and embedding strategies. Our results show that fine-tuned TokaMind outperforms the benchmark baseline on all but one task, and that, for several tasks, lightweight fine-tuning yields better performance than training the same architecture from scratch under a matched epoch budget. These findings highlight the benefits of multi-modal pretraining for tokamak plasma dynamics and provide a practical, extensible foundation for future fusion modeling tasks. Training code and model weights will be made publicly available.
GAN|对抗|攻击|生成相关(2篇)
【1】Latent Regularization in Generative Test Input Generation
标题：生成测试输入生成中的潜在正规化
链接：https://arxiv.org/abs/2602.15552
备注：Accepted for publication at the 7th International Workshop on Deep Learning for Testing and Testing for Deep Learning (DeepTest 2026), co-located with ICSE 2026
摘要：本研究调查了通过截断对潜在空间进行正则化对深度学习分类器生成的测试输入质量的影响。我们使用基于样式的GANs（一种最先进的生成方法）来评估这种效果，并从三个方面评估质量：有效性、多样性和故障检测。我们评估了我们在三个数据集（MNIST，Fashion MNIST和CIFAR-10）上对深度学习图像分类器进行边界测试的方法。我们比较了两种截断策略：潜在代码混合二进制搜索优化和随机潜在截断生成探索。我们的实验表明，潜在的代码混合的方法产生了更高的故障检测率比随机截断，同时也提高了多样性和有效性。
摘要：This study investigates the impact of regularization of latent spaces through truncation on the quality of generated test inputs for deep learning classifiers. We evaluate this effect using style-based GANs, a state-of-the-art generative approach, and assess quality along three dimensions: validity, diversity, and fault detection. We evaluate our approach on the boundary testing of deep learning image classifiers across three datasets, MNIST, Fashion MNIST, and CIFAR-10. We compare two truncation strategies: latent code mixing with binary search optimization and random latent truncation for generative exploration. Our experiments show that the latent code-mixing approach yields a higher fault detection rate than random truncation, while also improving both diversity and validity.
【2】Exploiting Layer-Specific Vulnerabilities to Backdoor Attack in Federated Learning
标题：利用联邦学习中的特定层漏洞进行后门攻击
链接：https://arxiv.org/abs/2602.15161
备注：This paper has been accepted for publication in IEEE ICC 2026
摘要：联邦学习（FL）支持跨边缘设备的分布式模型训练，同时保持数据局部性。这种去中心化的方法已经成为敏感用户数据协作学习的一种有前途的解决方案，有效地解决了中心化系统中固有的长期隐私问题。然而，FL的去中心化性质暴露了新的安全漏洞，特别是威胁模型完整性的后门攻击。为了研究这一关键问题，本文提出了层平滑攻击（LSA），一种利用神经网络中特定层漏洞的新型后门攻击。首先，层替代分析方法系统地识别对后门成功贡献最大的后门关键（BC）层。随后，LSA战略性地操纵这些BC层来注入持久后门，同时保持不被最先进的防御机制检测到。在不同的模型架构和数据集上进行的大量实验表明，LSA实现了高达97%的后门成功率，同时在主要任务上保持了高模型准确性，始终绕过现代FL防御。这些发现揭示了当前FL安全框架中的根本漏洞，表明未来的防御必须结合层感知检测和缓解策略。
摘要：Federated learning (FL) enables distributed model training across edge devices while preserving data locality. This decentralized approach has emerged as a promising solution for collaborative learning on sensitive user data, effectively addressing the longstanding privacy concerns inherent in centralized systems. However, the decentralized nature of FL exposes new security vulnerabilities, especially backdoor attacks that threaten model integrity. To investigate this critical concern, this paper presents the Layer Smoothing Attack (LSA), a novel backdoor attack that exploits layer-specific vulnerabilities in neural networks. First, a Layer Substitution Analysis methodology systematically identifies backdoor-critical (BC) layers that contribute most significantly to backdoor success. Subsequently, LSA strategically manipulates these BC layers to inject persistent backdoors while remaining undetected by state-of-the-art defense mechanisms. Extensive experiments across diverse model architectures and datasets demonstrate that LSA achieves a remarkably backdoor success rate of up to 97% while maintaining high model accuracy on the primary task, consistently bypassing modern FL defenses. These findings uncover fundamental vulnerabilities in current FL security frameworks, demonstrating that future defenses must incorporate layer-aware detection and mitigation strategies.
半/弱/无/有监督|不确定性|主动学习(1篇)
【1】Complex-Valued Unitary Representations as Classification Heads for Improved Uncertainty Quantification in Deep Neural Networks
标题：复值么元表示作为深度神经网络改进不确定性量化的分类头
链接：https://arxiv.org/abs/2602.15283
备注：21 pages, 12 figures
摘要：现代深度神经网络实现了很高的预测准确性，但仍然没有得到很好的校准：它们的置信度得分不能可靠地反映正确性的真实概率。我们提出了一个量子启发的分类头架构，该架构将骨干特征投影到复值Hilbert空间中，并在通过Cayley映射参数化的学习酉变换下对其进行演化。通过一个控制的混合实验设计-训练一个单一的共享骨干和比较轻量级的可互换头-我们隔离校准的复值酉表示的效果。我们对CIFAR-10的消融研究表明，单位量级头（在Cayley幺正下演化的复杂特征，通过量级和softmax读出）实现了0.0146的预期校准误差（ECE），比标准softmax头（0.0355）提高了2.4倍，比温度缩放（0.0510）提高了3.5倍。令人惊讶的是，用玻恩规则测量层取代softmax读数-量子力学驱动的方法-将校准降低到0.0819的ECE。在CIFAR-10 H人类不确定性基准测试中，在所有比较方法中，波函数头对人类软标签的KL发散度最低（0.336），表明复值表示更好地捕捉了人类感知模糊性的结构。我们提供了理论分析，通过特征空间几何连接规范保持一元动力学校准，报告负面结果的分布检测和情感分析，以划定该方法的范围，并讨论安全关键应用程序的实际意义。代码是公开的。
摘要：Modern deep neural networks achieve high predictive accuracy but remain poorly calibrated: their confidence scores do not reliably reflect the true probability of correctness. We propose a quantum-inspired classification head architecture that projects backbone features into a complex-valued Hilbert space and evolves them under a learned unitary transformation parameterised via the Cayley map. Through a controlled hybrid experimental design - training a single shared backbone and comparing lightweight interchangeable heads - we isolate the effect of complex-valued unitary representations on calibration. Our ablation study on CIFAR-10 reveals that the unitary magnitude head (complex features evolved under a Cayley unitary, read out via magnitude and softmax) achieves an Expected Calibration Error (ECE) of 0.0146, representing a 2.4x improvement over a standard softmax head (0.0355) and a 3.5x improvement over temperature scaling (0.0510). Surprisingly, replacing the softmax readout with a Born rule measurement layer - the quantum-mechanically motivated approach - degrades calibration to an ECE of 0.0819. On the CIFAR-10H human-uncertainty benchmark, the wave function head achieves the lowest KL-divergence (0.336) to human soft labels among all compared methods, indicating that complex-valued representations better capture the structure of human perceptual ambiguity. We provide theoretical analysis connecting norm-preserving unitary dynamics to calibration through feature-space geometry, report negative results on out-of-distribution detection and sentiment analysis to delineate the method's scope, and discuss practical implications for safety-critical applications. Code is publicly available.
迁移|Zero/Few/One-Shot|自适应(2篇)
【1】Stabilizing Test-Time Adaptation of High-Dimensional Simulation Surrogates via D-Optimal Statistics
标题：通过D-最优统计稳定多维模拟代理的测试时间自适应
链接：https://arxiv.org/abs/2602.15820
摘要：机器学习代理越来越多地用于工程中以加速昂贵的模拟，但训练和部署之间的分布变化通常会导致严重的性能下降（例如，看不见的几何形状或配置）。测试时自适应（TTA）可以减轻这种变化，但现有的方法主要是为低维分类开发的，具有结构化输出和视觉对齐的输入-输出关系，使它们对于模拟中常见的高维，非结构化和回归问题不稳定。我们提出了一个TTA框架的基础上存储最大信息（D-最优）的统计，这共同实现了稳定的适应和原则的参数选择在测试时，解决了这一挑战。当应用于预训练的模拟代理时，我们的方法以可忽略不计的计算成本产生高达7%的分布改进。据我们所知，这是第一次系统地演示有效的TTA用于高维仿真回归和生成式设计优化，并在SIMSTOS和EngiBench基准测试中得到验证。
摘要：Machine learning surrogates are increasingly used in engineering to accelerate costly simulations, yet distribution shifts between training and deployment often cause severe performance degradation (e.g., unseen geometries or configurations). Test-Time Adaptation (TTA) can mitigate such shifts, but existing methods are largely developed for lower-dimensional classification with structured outputs and visually aligned input-output relationships, making them unstable for the high-dimensional, unstructured and regression problems common in simulation. We address this challenge by proposing a TTA framework based on storing maximally informative (D-optimal) statistics, which jointly enables stable adaptation and principled parameter selection at test time. When applied to pretrained simulation surrogates, our method yields up to 7% out-of-distribution improvements at negligible computational cost. To the best of our knowledge, this is the first systematic demonstration of effective TTA for high-dimensional simulation regression and generative design optimization, validated on the SIMSHIFT and EngiBench benchmarks.
【2】On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
标题：关于自适应优化器中掩蔽更新的惊人有效性
链接：https://arxiv.org/abs/2602.15322
备注：Preprint
摘要：训练大型语言模型（LLM）几乎完全依赖于具有日益复杂的预条件的密集自适应优化器。我们通过证明随机掩蔽参数更新可以非常有效来挑战这一点，RMSProp的掩蔽变体始终优于最近最先进的优化器。我们的分析表明，随机掩蔽诱导曲率相关的几何正则化，平滑的优化轨迹。出于这一发现的动机，我们引入了动量对齐的梯度掩蔽（岩浆），它使用动量梯度对齐来调制掩蔽的更新。广泛的LLM预训练实验表明，Magma是自适应优化器的简单替代品，具有一致的增益和可忽略的计算开销。值得注意的是，对于1B模型大小，与Adam和Muon相比，Magma分别减少了超过19%和9%的困惑。
摘要：Training large language models (LLMs) relies almost exclusively on dense adaptive optimizers with increasingly sophisticated preconditioners. We challenge this by showing that randomly masking parameter updates can be highly effective, with a masked variant of RMSProp consistently outperforming recent state-of-the-art optimizers. Our analysis reveals that the random masking induces a curvature-dependent geometric regularization that smooths the optimization trajectory. Motivated by this finding, we introduce Momentum-aligned gradient masking (Magma), which modulates the masked updates using momentum-gradient alignment. Extensive LLM pre-training experiments show that Magma is a simple drop-in replacement for adaptive optimizers with consistent gains and negligible computational overhead. Notably, for the 1B model size, Magma reduces perplexity by over 19\% and 9\% compared to Adam and Muon, respectively.
强化学习(4篇)
【1】Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learning
标题：使用强化学习解决可行性未知的参数稳健避免问题
链接：https://arxiv.org/abs/2602.15817
备注：ICLR 2026. The project page can be found at https://oswinso.xyz/fge
摘要：深度强化学习（RL）的最新进展在高维控制任务上取得了很好的效果，但将RL应用于可达性问题会产生根本的不匹配：可达性寻求最大化系统无限期保持安全的状态集，而RL则优化用户指定分布的预期回报。这种不匹配可能导致策略在仍在安全集内的低概率状态上表现不佳。一个自然的替代方案是将问题框定为一组初始条件的鲁棒优化，这些初始条件指定初始状态，动态和安全集，但这个问题是否有解取决于指定集的可行性，这是先验未知的。我们提出了可行性引导探索（FGE），一种方法，同时确定一个子集的可行的初始条件下，存在一个安全的政策，并学习一个政策，以解决这组初始条件的可达性问题。实证结果表明，FGE学习的政策超过50%以上的覆盖率比现有的最佳方法，挑战初始条件的任务在MuJoCo模拟器和Kinetix模拟器与像素观察。
摘要 ：Recent advances in deep reinforcement learning (RL) have achieved strong results on high-dimensional control tasks, but applying RL to reachability problems raises a fundamental mismatch: reachability seeks to maximize the set of states from which a system remains safe indefinitely, while RL optimizes expected returns over a user-specified distribution. This mismatch can result in policies that perform poorly on low-probability states that are still within the safe set. A natural alternative is to frame the problem as a robust optimization over a set of initial conditions that specify the initial state, dynamics and safe set, but whether this problem has a solution depends on the feasibility of the specified set, which is unknown a priori. We propose Feasibility-Guided Exploration (FGE), a method that simultaneously identifies a subset of feasible initial conditions under which a safe policy exists, and learns a policy to solve the reachability problem over this set of initial conditions. Empirical results demonstrate that FGE learns policies with over 50% more coverage than the best existing method for challenging initial conditions across tasks in the MuJoCo simulator and the Kinetix simulator with pixel observations.
【2】CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies
标题：CDRL：一个受脑回路和树枝状计算策略启发的强化学习框架
链接：https://arxiv.org/abs/2602.15367
备注：14pages, 8 figures, 6 tabels
摘要：强化学习（RL）在高维序贯决策任务中取得了显著的性能，但仍受到样本效率低、对噪声敏感以及部分可观测性下泛化能力弱等问题的限制。大多数现有的方法主要通过优化策略来解决这些问题，而架构先验在塑造表示学习和决策动态中的作用则较少被探索。受小脑结构原理的启发，我们提出了一种生物学基础的RL架构，该架构包含大扩展，稀疏连接，稀疏激活和树突级调制。在有噪声的高维RL基准测试上的实验表明，与传统设计相比，小脑结构和树突调制始终提高了样本效率、鲁棒性和泛化能力。架构参数的敏感性分析表明，受小脑启发的结构可以在模型参数受限的情况下为RL提供优化的性能。总的来说，我们的工作强调了小脑结构先验作为RL的有效归纳偏差的价值。
摘要：Reinforcement learning (RL) has achieved notable performance in high-dimensional sequential decision-making tasks, yet remains limited by low sample efficiency, sensitivity to noise, and weak generalization under partial observability. Most existing approaches address these issues primarily through optimization strategies, while the role of architectural priors in shaping representation learning and decision dynamics is less explored. Inspired by structural principles of the cerebellum, we propose a biologically grounded RL architecture that incorporate large expansion, sparse connectivity, sparse activation, and dendritic-level modulation. Experiments on noisy, high-dimensional RL benchmarks show that both the cerebellar architecture and dendritic modulation consistently improve sample efficiency, robustness, and generalization compared to conventional designs. Sensitivity analysis of architectural parameters suggests that cerebellum-inspired structures can offer optimized performance for RL with constrained model parameters. Overall, our work underscores the value of cerebellar structural priors as effective inductive biases for RL.
【3】Latency-aware Human-in-the-Loop Reinforcement Learning for Semantic Communications
标题：用于语义通信的延迟感知的人在环强化学习
链接：https://arxiv.org/abs/2602.15640
备注：6 pages, 8 figures. This paper has been accepted for publication in IEEE ICC 2026
摘要：语义通信承诺任务对齐的传输，但必须在沉浸式和安全关键型服务中协调语义保真度和严格的延迟保证。本文介绍了一种时间约束的人在回路强化学习（TC-HITL-RL）框架，该框架将人的反馈，语义效用和延迟控制嵌入到语义感知的开放无线电接入网络（RAN）架构中。我们制定了语义自适应驱动的人的反馈作为一个受约束的马尔可夫决策过程（CMDP），其状态捕获语义质量，人的喜好，队列松弛，和通道动态，并解决它通过一个原始的-双邻近的政策优化算法与动作屏蔽和延迟感知奖励成形。由此产生的策略保留了PPO级语义奖励，同时收紧了空中接口和近实时RAN智能控制器处理预算的可变性。在具有异构截止期限的点到多点链路上的仿真表明，TC-HITL-RL始终满足每个用户的时间约束，在奖励方面优于基线调度器，并稳定资源消耗，为延迟感知语义自适应提供了实用的蓝图。
摘要：Semantic communication promises task-aligned transmission but must reconcile semantic fidelity with stringent latency guarantees in immersive and safety-critical services. This paper introduces a time-constrained human-in-the-loop reinforcement learning (TC-HITL-RL) framework that embeds human feedback, semantic utility, and latency control within a semantic-aware Open radio access network (RAN) architecture. We formulate semantic adaptation driven by human feedback as a constrained Markov decision process (CMDP) whose state captures semantic quality, human preferences, queue slack, and channel dynamics, and solve it via a primal--dual proximal policy optimization algorithm with action shielding and latency-aware reward shaping. The resulting policy preserves PPO-level semantic rewards while tightening the variability of both air-interface and near-real-time RAN intelligent controller processing budgets. Simulations over point-to-multipoint links with heterogeneous deadlines show that TC-HITL-RL consistently meets per-user timing constraints, outperforms baseline schedulers in reward, and stabilizes resource consumption, providing a practical blueprint for latency-aware semantic adaptation.
【4】Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis
标题：超越强化学习：快速且可扩展的量子电路合成
链接：https://arxiv.org/abs/2602.15146
摘要：量子幺正合成解决了将抽象量子算法转换为硬件可执行量子门序列的问题。由于底层组合搜索空间的指数增长，精确地解决这个任务通常是不可行的。现有的方法存在优化目标不一致、训练成本高以及不同量子位数的泛化能力有限等问题。我们通过使用监督学习来近似残差酉的最小描述长度，并将此估计与随机波束搜索相结合来识别接近最优的门序列，从而减轻了这些限制。我们的方法依赖于具有zero-shot泛化的轻量级模型，与先前的基线相比，大大减少了训练开销。在多个基准测试中，我们实现了更快的挂钟合成时间，同时在复杂电路的成功率方面超过了最先进的方法。
摘要：Quantum unitary synthesis addresses the problem of translating abstract quantum algorithms into sequences of hardware-executable quantum gates. Solving this task exactly is infeasible in general due to the exponential growth of the underlying combinatorial search space. Existing approaches suffer from misaligned optimization objectives, substantial training costs and limited generalization across different qubit counts. We mitigate these limitations by using supervised learning to approximate the minimum description length of residual unitaries and combining this estimate with stochastic beam search to identify near optimal gate sequences. Our method relies on a lightweight model with zero-shot generalization, substantially reducing training overhead compared to prior baselines. Across multiple benchmarks, we achieve faster wall-clock synthesis times while exceeding state-of-the-art methods in terms of success rate for complex circuits.
符号|符号学习(2篇)
【1】Symbolic recovery of PDEs from measurement data
标题：从测量数据中符号性恢复偏头痛
链接：https://arxiv.org/abs/2602.15603
摘要：基于偏微分方程（PDE）的模型对于描述自然科学中广泛的复杂关系是强大的。准确识别代表潜在物理定律的PDE模型对于正确理解问题至关重要。这种重建通常依赖于系统状态的间接和噪声测量，并且没有专门定制的方法，很少产生符号表达式，从而阻碍了可解释性。在这项工作中，我们解决这个问题，考虑现有的神经网络架构的基础上，理性的功能，物理定律的符号表示。这些网络利用了有理函数的近似能力，同时也受益于它们在表示算术运算方面的灵活性。我们的主要贡献是一个可识别性的结果，表明，在无噪声，完整的测量的限制，这种符号网络可以唯一地重建PDE模型内最简单的物理定律。具体来说，重建的法律仍然在符号网络架构内表达，正则化最小化参数化促进可解释性和稀疏性的情况下，L^1 $-正则化。此外，我们还提供了符号网络的正则性结果。使用ParFam架构的经验验证支持这些理论发现，为物理定律的实际可重构性提供了证据。
摘要：Models based on partial differential equations (PDEs) are powerful for describing a wide range of complex relationships in the natural sciences. Accurately identifying the PDE model, which represents the underlying physical law, is essential for a proper understanding of the problem. This reconstruction typically relies on indirect and noisy measurements of the system's state and, without specifically tailored methods, rarely yields symbolic expressions, thereby hindering interpretability. In this work, we address this issue by considering existing neural network architectures based on rational functions for the symbolic representation of physical laws. These networks leverage the approximation power of rational functions while also benefiting from their flexibility in representing arithmetic operations. Our main contribution is an identifiability result, showing that, in the limit of noiseless, complete measurements, such symbolic networks can uniquely reconstruct the simplest physical law within the PDE model. Specifically, reconstructed laws remain expressible within the symbolic network architecture, with regularization-minimizing parameterizations promoting interpretability and sparsity in case of $L^1$-regularization. In addition, we provide regularity results for symbolic networks. Empirical validation using the ParFam architecture supports these theoretical findings, providing evidence for the practical reconstructibility of physical laws.
【2】Learning the S-matrix from data: Rediscovering gravity from gauge theory via symbolic regression
标题：从数据中学习S矩阵：通过符号回归从规范理论中重新发现重力
链接：https://arxiv.org/abs/2602.15169
摘要：我们证明了现代机器学习方法可以直接从数值壳数据中自主重建散射振幅的几个旗舰分析结构。特别是，我们表明，Kawai-Lewellen-Tye（KLT）的关系，可以重新发现使用符号回归适用于颜色排序的杨-米尔斯振幅与Mandelovian不变量作为输入功能。使用标准的特征选择技术，特别是列枢轴QR因式分解，我们同时恢复的Kleiss-Kuijf和伯尔尼-Carrasco-约翰森（BCJ）的关系，确定一个最小的基础上的部分振幅没有任何群论的输入。我们得到的树级KLT关系具有高的数值精度高达五个外部腿，只使用最小的理论先验，我们评论的障碍，推广的方法，以更高的多重性。我们的研究结果建立了符号回归作为一个实用的工具，探索散射振幅景观的分析结构，并提出了一个通用的数据驱动的策略，揭示隐藏的关系，在一般的理论。为了比较，我们基准测试这种一般的方法与最近推出的基于神经网络的方法。
摘要：We demonstrate that modern machine-learning methods can autonomously reconstruct several flagship analytic structures in scattering amplitudes directly from numerical on-shell data. In particular, we show that the Kawai--Lewellen--Tye (KLT) relations can be rediscovered using symbolic regression applied to colour-ordered Yang--Mills amplitudes with Mandelstam invariants as input features. Using standard feature-selection techniques, specifically column-pivoted QR factorisation, we simultaneously recover the Kleiss--Kuijf and Bern--Carrasco--Johansson (BCJ) relations, identifying a minimal basis of partial amplitudes without any group-theoretic input. We obtain the tree-level KLT relations with high numerical accuracy up to five external legs, using only minimal theoretical priors, and we comment on the obstacles to generalising the method to higher multiplicity. Our results establish symbolic regression as a practical tool for exploring the analytic structure of the scattering-amplitude landscape, and suggests a general data-driven strategy for uncovering hidden relations in general theories. For comparison, we benchmark this general approach with a recently introduced neural-network based method.
医学相关(1篇)
【1】Hybrid Federated and Split Learning for Privacy Preserving Clinical Prediction and Treatment Optimization
标题：混合联合和拆分学习用于隐私保护临床预测和治疗优化
链接：https://arxiv.org/abs/2602.15304
摘要：协作临床决策支持通常受到治理和隐私规则的限制，这些规则阻止跨机构汇集患者级别的记录。我们提出了一个混合隐私保护框架，它结合了联邦学习（FL）和分裂学习（SL），以支持面向决策的医疗建模，而无需原始数据共享。该方法将特征提取主干保留在客户端上，同时将预测头托管在协调服务器上，从而实现共享表示学习并公开可以应用隐私控制的显式协作边界。而不是假设分布式训练本质上是私有的，我们审计泄漏经验使用成员资格推断切割层表示和研究基于激活裁剪和加性高斯噪声的轻量级防御。我们使用统一的管道在非IID客户端分区下对三个公共临床数据集进行评估，并沿着四个部署相关轴联合评估性能：事实预测效用，容量限制下基于提升的排名，审计隐私泄露和通信开销。结果表明，相对于独立FL或SL，混合FL-SL变体实现了具有竞争力的预测性能和面向决策的优先级排序行为，同时提供了可调的隐私-效用权衡，可以在不需要原始数据共享的情况下减少审计泄漏。总体而言，工作位置混合FL-SL作为一个实用的设计空间，隐私保护的医疗保健决策支持，效用，泄漏风险和部署成本必须明确平衡。
摘要：Collaborative clinical decision support is often constrained by governance and privacy rules that prevent pooling patient-level records across institutions. We present a hybrid privacy-preserving framework that combines Federated Learning (FL) and Split Learning (SL) to support decision-oriented healthcare modeling without raw-data sharing. The approach keeps feature-extraction trunks on clients while hosting prediction heads on a coordinating server, enabling shared representation learning and exposing an explicit collaboration boundary where privacy controls can be applied. Rather than assuming distributed training is inherently private, we audit leakage empirically using membership inference on cut-layer representations and study lightweight defenses based on activation clipping and additive Gaussian noise. We evaluate across three public clinical datasets under non-IID client partitions using a unified pipeline and assess performance jointly along four deployment-relevant axes: factual predictive utility, uplift-based ranking under capacity constraints, audited privacy leakage, and communication overhead. Results show that hybrid FL-SL variants achieve competitive predictive performance and decision-facing prioritization behavior relative to standalone FL or SL, while providing a tunable privacy-utility trade-off that can reduce audited leakage without requiring raw-data sharing. Overall, the work positions hybrid FL-SL as a practical design space for privacy-preserving healthcare decision support where utility, leakage risk, and deployment cost must be balanced explicitly.
蒸馏|知识提取(3篇)
【1】Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization
标题：通过探索利用优化加速大规模数据集蒸馏
链接：https://arxiv.org/abs/2602.15277
摘要：数据集蒸馏将原始数据压缩成紧凑的合成数据集，减少训练时间和存储，同时保持模型性能，从而在有限的资源下进行部署。尽管最近的基于优化的蒸馏方法能够大规模地进行数据集蒸馏，但它们仍然面临效率差距：基于优化的解耦方法实现了更高的精度，但需要密集的计算，而无优化的解耦方法是有效的，但牺牲了精度。为了克服这种权衡，我们提出了探索-利用蒸馏（E^2D），这是一种简单实用的方法，可以通过一个高效的管道来最大限度地减少冗余计算，该管道从完整图像初始化开始，以保持语义完整性和特征多样性。然后，它使用两阶段优化策略：探索阶段，执行统一更新并识别高损失区域，以及开发阶段，将更新集中在这些区域以加速收敛。我们在大规模基准测试中评估了E^2D，在ImageNet-1 K上超过了最先进的水平，同时速度提高了18倍，在ImageNet-21 K上，我们的方法大大提高了准确性，同时保持了4.3倍的速度。这些结果表明，有针对性的，减少冗余的更新，而不是蛮力优化，弥合了大规模数据集蒸馏的准确性和效率之间的差距。代码可从https://github.com/ncsu-dk-lab获得。
摘要 ：Dataset distillation compresses the original data into compact synthetic datasets, reducing training time and storage while retaining model performance, enabling deployment under limited resources. Although recent decoupling-based distillation methods enable dataset distillation at large-scale, they continue to face an efficiency gap: optimization-based decoupling methods achieve higher accuracy but demand intensive computation, whereas optimization-free decoupling methods are efficient but sacrifice accuracy. To overcome this trade-off, we propose Exploration-Exploitation Distillation (E^2D), a simple, practical method that minimizes redundant computation through an efficient pipeline that begins with full-image initialization to preserve semantic integrity and feature diversity. It then uses a two-phase optimization strategy: an exploration phase that performs uniform updates and identifies high-loss regions, and an exploitation phase that focuses updates on these regions to accelerate convergence. We evaluate E^2D on large-scale benchmarks, surpassing the state-of-the-art on ImageNet-1K while being 18x faster, and on ImageNet-21K, our method substantially improves accuracy while remaining 4.3x faster. These results demonstrate that targeted, redundancy-reducing updates, rather than brute-force optimization, bridge the gap between accuracy and efficiency in large-scale dataset distillation. Code is available at https://github.com/ncsu-dk-lab.
【2】Fast and Effective On-policy Distillation from Reasoning Prefixes
标题：快速有效地从推理前置提取政策上
链接：https://arxiv.org/abs/2602.15260
摘要：On-policy distillation（OPD）从学生模型中采样轨迹，并在令牌级别上与教师一起监督它们，避免了仅仅依赖可验证的终端奖励，并且可以比off-policy distillation产生更好的泛化。然而，OPD需要在训练期间对学生策略进行昂贵的即时采样，这大大增加了训练成本，特别是对于长响应。我们的初步分析表明，在OPD期间，训练信号通常集中在每个输出的前缀中，即使是教师生成的短前缀也可以显着帮助学生产生正确的答案。出于这些观察结果，我们提出了一个简单而有效的修改OPD：我们只适用于学生生成的输出前缀蒸馏目标，并终止每个采样蒸馏过程中早期。在一套数学人工智能和域外基准测试上进行的实验表明，策略上的前缀蒸馏与完整OPD的性能相匹配，同时将训练FLOP降低了2x-47 x。
摘要：On-policy distillation (OPD), which samples trajectories from the student model and supervises them with a teacher at the token level, avoids relying solely on verifiable terminal rewards and can yield better generalization than off-policy distillation. However, OPD requires expensive on-the-fly sampling of the student policy during training, which substantially increases training cost, especially for long responses. Our initial analysis shows that, during OPD, training signals are often concentrated in the prefix of each output, and that even a short teacher-generated prefix can significantly help the student produce the correct answer. Motivated by these observations, we propose a simple yet effective modification of OPD: we apply the distillation objective only to prefixes of student-generated outputs and terminate each sampling early during distillation. Experiments on a suite of AI-for-Math and out-of-domain benchmarks show that on-policy prefix distillation matches the performance of full OPD while reducing training FLOP by 2x-47x.
【3】SCENE OTA-FD: Self-Centering Noncoherent Estimator for Over-the-Air Federated Distillation
标题：SCENE OTA-FD：空中联合蒸馏的自中心非相关估计器
链接：https://arxiv.org/abs/2602.15326
备注：Work in progress. Codes will be available on: https://github.com/zavareh1
摘要：我们提出SCENE（自中心非相干估计），一个无导频和相位不变的聚集原语空中联合蒸馏（OTA FD）。每个设备在恒定的每轮功率和恒定包络信令（PAPR接近1）下将其软标签（类概率）向量映射到非负发射能量。在服务器处，自定中心能量估计器去除噪声能量偏移，并且产生加权软标签平均值的无偏估计，其中方差在接收天线的数量M和重复因子S中以1/（SM）的量级衰减。我们还开发了一个无导频比率归一化的变体，取消未知的大规模增益，提供了一致的OTA-FD分析的收敛范围，并提出了一个基于开销的交叉比较。SCENE针对短相干性和硬件受限的机制，其中避免每轮CSI是必不可少的：它以适度的非相干方差常数换取零上行链路导频、无偏聚合和硬件友好传输，并且在导频开销不可忽略时可以优于相干设计。
摘要：We propose SCENE (Self-Centering Noncoherent Estimator), a pilot-free and phase-invariant aggregation primitive for over-the-air federated distillation (OTA-FD). Each device maps its soft-label (class-probability) vector to nonnegative transmit energies under constant per-round power and constant-envelope signaling (PAPR near 1). At the server, a self-centering energy estimator removes the noise-energy offset and yields an unbiased estimate of the weighted soft-label average, with variance decaying on the order of 1/(SM) in the number of receive antennas M and repetition factor S. We also develop a pilot-free ratio-normalized variant that cancels unknown large-scale gains, provide a convergence bound consistent with coherent OTA-FD analyses, and present an overhead-based crossover comparison. SCENE targets short-coherence and hardware-constrained regimes, where avoiding per-round CSI is essential: it trades a modest noncoherent variance constant for zero uplink pilots, unbiased aggregation, and hardware-friendly transmission, and can outperform coherent designs when pilot overhead is non-negligible.
聚类(1篇)
【1】Doubly Stochastic Mean-Shift Clustering
标题：双随机均值漂移聚集
链接：https://arxiv.org/abs/2602.15393
备注：30 pages. arXiv admin note: text overlap with arXiv:2511.09202
摘要：众所周知，标准Mean-Shift算法对带宽超参数非常敏感，特别是在数据稀缺的情况下，固定尺度的密度估计会导致碎片和虚假模式。在本文中，我们提出了双重随机均值漂移（DSMS），一种新的扩展，引入随机性不仅在轨迹更新，但也在内核带宽本身。通过在每次迭代中从连续均匀分布中绘制数据样本和半径，DSMS有效地执行了对密度景观的更好探索。我们证明了这种随机带宽策略充当了隐式正则化机制，并提供了收敛理论结果。合成高斯混合的比较实验表明，DSMS显着优于标准和随机均值漂移基线，表现出显着的稳定性，并防止过度分割稀疏聚类的情况下，没有其他性能下降。
摘要：Standard Mean-Shift algorithms are notoriously sensitive to the bandwidth hyperparameter, particularly in data-scarce regimes where fixed-scale density estimation leads to fragmentation and spurious modes. In this paper, we propose Doubly Stochastic Mean-Shift (DSMS), a novel extension that introduces randomness not only in the trajectory updates but also in the kernel bandwidth itself. By drawing both the data samples and the radius from a continuous uniform distribution at each iteration, DSMS effectively performs a better exploration of the density landscape. We show that this randomized bandwidth policy acts as an implicit regularization mechanism, and provide convergence theoretical results. Comparative experiments on synthetic Gaussian mixtures reveal that DSMS significantly outperforms standard and stochastic Mean-Shift baselines, exhibiting remarkable stability and preventing over-segmentation in sparse clustering scenarios without other performance degradation.
联邦学习|隐私保护|加密(3篇)
【1】Evaluating Federated Learning for Cross-Country Mood Inference from Smartphone Sensing Data
标题：评估联邦学习用于智能手机感知数据的越野情绪推断
链接：https://arxiv.org/abs/2602.15478
备注：21 pages, 6 figure
摘要：情绪不稳定是心理健康的一个关键行为指标，但传统的评估依赖于不频繁的回顾性报告，无法捕捉其连续性。基于智能手机的移动传感能够从日常行为中进行被动的、野外的情绪推断;然而，由于隐私限制、不均匀的传感可用性以及行为模式的显著变化，大规模部署这样的系统仍然具有挑战性。 在这项工作中，我们研究了在跨国联合学习环境中使用智能手机感知数据的情绪推断，每个国家都作为独立的客户端参与，同时保留本地数据。我们介绍FedFAP，一个功能感知的个性化联邦框架，旨在适应不同地区的异构传感模式。对地理和文化多样性人群的评估表明，FedFAP的AUROC为0.744，优于集中式方法和现有的个性化联邦基线。除了推理，我们的研究结果还为情绪感知系统提供了设计见解，展示了人口感知个性化和隐私保护学习如何实现可扩展和情绪感知的移动传感技术。
摘要：Mood instability is a key behavioral indicator of mental health, yet traditional assessments rely on infrequent and retrospective reports that fail to capture its continuous nature. Smartphone-based mobile sensing enables passive, in-the-wild mood inference from everyday behaviors; however, deploying such systems at scale remains challenging due to privacy constraints, uneven sensing availability, and substantial variability in behavioral patterns. In this work, we study mood inference using smartphone sensing data in a cross-country federated learning setting, where each country participates as an independent client while retaining local data. We introduce FedFAP, a feature-aware personalized federated framework designed to accommodate heterogeneous sensing modalities across regions. Evaluations across geographically and culturally diverse populations show that FedFAP achieves an AUROC of 0.744, outperforming both centralized approaches and existing personalized federated baselines. Beyond inference, our results offer design insights for mood-aware systems, demonstrating how population-aware personalization and privacy-preserving learning can enable scalable and mood-aware mobile sensing technologies.
【2】Fractional-Order Federated Learning
标题：分数阶联邦学习
链接：https://arxiv.org/abs/2602.15380
备注：This paper is submitted to IEEE-TAI
摘要：联合学习（FL）允许远程客户协作训练全球模型，同时保护客户隐私。尽管FL具有隐私保护的优点，但它也有显著的缺点，包括收敛速度慢、通信成本高和非独立同分布（非IID）数据。在这项工作中，我们提出了一种新的FedAvg变化称为分数阶联邦平均（FOFedAvg），它结合了分数阶随机梯度下降（FOSGD），以捕捉长期的关系和更深层次的历史信息。通过引入内存感知的分数阶更新，FOFedAvg提高了通信效率，加速了收敛，同时减轻了异构的非IID客户端数据造成的不稳定性。我们将FOFedAvg与基准数据集上的一组广泛的已建立的联邦优化算法进行比较，这些基准数据集包括MNIST，FEMNIST，CIFAR-10，CIFAR-100，EMNIST，克利夫兰心脏病数据集，Sent 140，Edge-IIoTset和Edge-IIoTset。在一系列非IID分区方案中，FOFedAvg在测试性能和收敛速度方面与这些基线具有竞争力，并且通常优于这些基线。在理论方面，我们证明了FOFedAvg收敛到一个平稳点下的标准光滑性和有界方差假设分数阶$0
摘要：Federated learning (FL) allows remote clients to train a global model collaboratively while protecting client privacy. Despite its privacy-preserving benefits, FL has significant drawbacks, including slow convergence, high communication cost, and non-independent-and-identically-distributed (non-IID) data. In this work, we present a novel FedAvg variation called Fractional-Order Federated Averaging (FOFedAvg), which incorporates Fractional-Order Stochastic Gradient Descent (FOSGD) to capture long-range relationships and deeper historical information. By introducing memory-aware fractional-order updates, FOFedAvg improves communication efficiency and accelerates convergence while mitigating instability caused by heterogeneous, non-IID client data. We compare FOFedAvg against a broad set of established federated optimization algorithms on benchmark datasets including MNIST, FEMNIST, CIFAR-10, CIFAR-100, EMNIST, the Cleveland heart disease dataset, Sent140, PneumoniaMNIST, and Edge-IIoTset. Across a range of non-IID partitioning schemes, FOFedAvg is competitive with, and often outperforms, these baselines in terms of test performance and convergence speed. On the theoretical side, we prove that FOFedAvg converges to a stationary point under standard smoothness and bounded-variance assumptions for fractional order $0
【3】FedPSA: Modeling Behavioral Staleness in Asynchronous Federated Learning
标题：FedPSA：对同步联邦学习中的行为停滞建模
链接：https://arxiv.org/abs/2602.15337
摘要：异步联邦学习（AFL）是近年来兴起的一个重要的研究领域。通过不等待较慢的客户端并同时执行训练过程，与传统的联邦学习相比，它实现了更快的训练速度。但是，由于异步进程引入的过时性，在某些情况下其性能可能会降低。现有方法通常使用当前模型与全局模型之间的舍入差作为陈旧性的唯一度量，这是粗粒度的，并且缺乏对模型本身的观察，从而限制了异步方法的性能上限。在本文中，我们提出了FedPSA（基于参数敏感性的异步联合学习），这是一个更细粒度的AFL框架，它利用参数敏感性来衡量模型过时，并建立一个动态动量队列来实时评估当前的训练阶段，从而动态调整对过时信息的容忍度。在多个数据集上的大量实验以及与各种方法的比较表明了FedPSA的优越性能，与基线方法相比提高了6.37%，与当前最先进的方法相比提高了1.93%。
摘要：Asynchronous Federated Learning (AFL) has emerged as a significant research area in recent years. By not waiting for slower clients and executing the training process concurrently, it achieves faster training speed compared to traditional federated learning. However, due to the staleness introduced by the asynchronous process, its performance may degrade in some scenarios. Existing methods often use the round difference between the current model and the global model as the sole measure of staleness, which is coarse-grained and lacks observation of the model itself, thereby limiting the performance ceiling of asynchronous methods. In this paper, we propose FedPSA (Parameter Sensitivity-based Asynchronous Federated Learning), a more fine-grained AFL framework that leverages parameter sensitivity to measure model obsolescence and establishes a dynamic momentum queue to assess the current training phase in real time, thereby adjusting the tolerance for outdated information dynamically. Extensive experiments on multiple datasets and comparisons with various methods demonstrate the superior performance of FedPSA, achieving up to 6.37\% improvement over baseline methods and 1.93\% over the current state-of-the-art method.
推理|分析|理解|解释(5篇)
【1】A Note on Non-Composability of Layerwise Approximate Verification for Neural Inference
标题：关于神经推理分层逼近验证的不可组合性的注记
链接：https://arxiv.org/abs/2602.15756
摘要：对浮点数据进行可验证（或零知识）ML推理的一种自然和非正式的方法是：“证明每一层都在公差$δ$内正确计算;因此最终输出是合理的推理结果”。这个简短的注释给出了一个简单的反例，表明这个推论通常是错误的：对于任何神经网络，我们都可以构造一个功能等价的网络，对于这个网络，在各个层计算中逆向选择的近似值误差足以任意地控制最终输出（在规定的有界范围内）。
摘要：A natural and informal approach to verifiable (or zero-knowledge) ML inference over floating-point data is: ``prove that each layer was computed correctly up to tolerance $δ$; therefore the final output is a reasonable inference result''. This short note gives a simple counterexample showing that this inference is false in general: for any neural network, we can construct a functionally equivalent network for which adversarially chosen approximation-magnitude errors in individual layer computations suffice to steer the final output arbitrarily (within a prescribed bounded range).
【2】Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Models
标题：方向推理轨迹变化（DRTC）：识别推理模型中的关键轨迹段
链接：https://arxiv.org/abs/2602.15332
摘要：理解语言模型如何进行长视野推理仍然是一个开放的挑战。现有的可解释性方法通常会突出与答案相关的标记或跨度，但它们很少揭示模型在何处进行了相应的推理转向，早期的上下文因果地触发了这些转向，或者突出显示的文本是否实际上引导了推理过程。我们介绍了方向性推理轨迹变化（DRTC），一个过程因果框架，用于解释长形式的推理，从一个单一的政策推出。DRTC使用不确定性和分布偏移信号检测枢轴决策点，然后应用接收方干预，保留已实现的推出，而不重新分配延续，同时仅在枢轴处阻止来自选定的较早块的信息流。它测量每个干预是否重定向模型的对数概率轨迹相对于实现的滚动方向的方向，产生有符号的每个块的归因得分。我们还计算原始logits上的转角曲率变化作为补充诊断，并引入曲率特征来总结共享的干预响应几何形状。从经验上看，方向性的影响是急剧集中在四个推理模型（每个例子|DRTC|股份收益率基尼系数为0.50到0.58，前5%的质量收益率为0.23到0.28），学习支点比匹配的随机跨度诱导更强的干预幅度。在使用R1-Distill-Qwen-1.5B对500个数学问题进行的缩放研究中，学习的跨度优于匹配的随机跨度（中值δ = 0.409，500个中有355个是阳性的;符号检验p = 2.3e-21）。总的来说，DRTC提供了一个因果接地，具体的上下文元素如何引导下的政策动态推理的自动化水平的视图。
摘要：Understanding how language models carry out long-horizon reasoning remains an open challenge. Existing interpretability methods often highlight tokens or spans correlated with an answer, but they rarely reveal where the model makes consequential reasoning turns, which earlier context causally triggers those turns, or whether the highlighted text actually steers the reasoning process. We introduce Directional Reasoning Trajectory Change (DRTC), a process-causal framework for interpreting long-form reasoning from a single on-policy rollout. DRTC detects pivot decision points using uncertainty and distribution-shift signals, then applies receiver-side interventions that preserve the realized rollout without resampling the continuation while blocking information flow from selected earlier chunks only at a pivot. It measures whether each intervention redirects the direction of the model's log-probability trajectory relative to the realized rollout direction, producing a signed per-chunk attribution score. We also compute turning-angle curvature changes on raw logits as a complementary diagnostic and introduce curvature signatures to summarize shared intervention-response geometry. Empirically, directional influence is sharply concentrated across four reasoning models (per-example |DRTC| shares yield Gini 0.50 to 0.58 and top-5 percent mass 0.23 to 0.28), and learned pivots induce stronger intervention magnitudes than matched random spans. In a scaling study on 500 MATH problems with R1-Distill-Qwen-1.5B, learned spans outperform matched random spans (median delta = 0.409, 355 of 500 positive; sign test p = 2.3e-21). Overall, DRTC provides a causally grounded, trajectory-level view of how specific context elements steer reasoning under on-policy dynamics.
【3】Knowing Isn't Understanding: Re-grounding Generative Proactivity with Epistemic and Behavioral Insight
标题：了解并不是理解：用认识论和行为洞察力重新奠定生成主动性的基础
链接：https://arxiv.org/abs/2602.15259
摘要：生成式AI代理将理解等同于解决显式查询，这一假设将交互限制在用户可以表达的内容上。当用户自己缺乏对缺失、风险或值得考虑的内容的意识时，这种假设就被打破了。在这种情况下，主动性不仅是一种效率的提高，而且是一种认识上的必要性。我们把这种情况称为认识的不完整性：进步取决于与未知的未知进行有效的合作。现有的主动性方法仍然是狭隘的预期，从过去的行为推断，并假设目标已经很好地定义，从而无法有意义地支持用户。然而，显现超出用户当前意识的可能性本质上并不是有益的。不受约束的主动干预可能会误导注意力，压倒用户或造成伤害。因此，主动代理人需要行为基础：对代理人何时、如何以及在多大程度上进行干预的原则性约束。我们提出的立场，生成的前摄性必须扎根于认知和行为。借鉴无知的哲学和主动行为的研究，我们认为，这些理论提供了重要的指导设计代理人，可以负责任地参与，并促进有意义的伙伴关系。
摘要：Generative AI agents equate understanding with resolving explicit queries, an assumption that confines interaction to what users can articulate. This assumption breaks down when users themselves lack awareness of what is missing, risky, or worth considering. In such conditions, proactivity is not merely an efficiency enhancement, but an epistemic necessity. We refer to this condition as epistemic incompleteness: where progress depends on engaging with unknown unknowns for effective partnership. Existing approaches to proactivity remain narrowly anticipatory, extrapolating from past behavior and presuming that goals are already well defined, thereby failing to support users meaningfully. However, surfacing possibilities beyond a user's current awareness is not inherently beneficial. Unconstrained proactive interventions can misdirect attention, overwhelm users, or introduce harm. Proactive agents, therefore, require behavioral grounding: principled constraints on when, how, and to what extent an agent should intervene. We advance the position that generative proactivity must be grounded both epistemically and behaviorally. Drawing on the philosophy of ignorance and research on proactive behavior, we argue that these theories offer critical guidance for designing agents that can engage responsibly and foster meaningful partnerships.
【4】MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference
标题：MAVRL：通过摊销变分推理从多种反馈类型学习奖励函数
链接：https://arxiv.org/abs/2602.15206
备注：25 pages, 7 figures
摘要：奖励学习通常依赖于单个反馈类型或使用手动加权损失项组合多个反馈类型。目前，尚不清楚如何从异构反馈类型（如演示、比较、评级和停止）中联合学习奖励函数，这些反馈类型提供了质的不同信号。我们通过将来自多个反馈类型的奖励学习公式化为共享潜在奖励函数上的贝叶斯推理来解决这一挑战，其中每个反馈类型通过显式可能性贡献信息。我们介绍了一种可扩展的摊销变分推理方法，学习共享的奖励编码器和反馈特定的似然解码器，并通过优化单个证据下限进行训练。我们的方法避免了减少反馈到一个共同的中间表示，并消除了手动损失平衡的需要。在离散和连续控制基准，我们表明，联合推断的奖励后验优于单一类型的基线，利用互补信息的反馈类型，并产生的政策，更强大的环境扰动。推断出的奖励不确定性进一步提供了用于分析模型置信度和跨反馈类型的一致性的可解释信号。
摘要：Reward learning typically relies on a single feedback type or combines multiple feedback types using manually weighted loss terms. Currently, it remains unclear how to jointly learn reward functions from heterogeneous feedback types such as demonstrations, comparisons, ratings, and stops that provide qualitatively different signals. We address this challenge by formulating reward learning from multiple feedback types as Bayesian inference over a shared latent reward function, where each feedback type contributes information through an explicit likelihood. We introduce a scalable amortized variational inference approach that learns a shared reward encoder and feedback-specific likelihood decoders and is trained by optimizing a single evidence lower bound. Our approach avoids reducing feedback to a common intermediate representation and eliminates the need for manual loss balancing. Across discrete and continuous-control benchmarks, we show that jointly inferred reward posteriors outperform single-type baselines, exploit complementary information across feedback types, and yield policies that are more robust to environment perturbations. The inferred reward uncertainty further provides interpretable signals for analyzing model confidence and consistency across feedback types.
【5】Universal priors: solving empirical Bayes via Bayesian inference and pretraining
标题：通用先验：通过Bayesian推理和预训练解决经验Bayeses
链接：https://arxiv.org/abs/2602.15136
备注：40 pages, 5 figures
摘要：我们从理论上证明了最近的经验发现[Teh等人，在综合生成的数据上预训练的Transformer在经验贝叶斯（EB）问题上实现了强大的性能。我们对这个问题采取了间接的方法：而不是分析模型架构或训练动态，我们问为什么在预先指定的训练分布下训练的预训练贝叶斯估计量可以适应任意的测试分布。专注于泊松EB问题，我们确定普遍先验的存在，使这些先验下的训练产生一个接近最优的遗憾界的$\widetilde{O}（\frac{1}{n}）$一致的所有测试分布。我们的分析利用了贝叶斯统计中的经典后验收缩现象，表明预训练的Transformer通过后验收缩精确地适应未知的测试分布。这个观点也解释了长度泛化的现象，即测试序列长度超过训练长度，因为模型使用广义后验进行贝叶斯推断。
摘要：We theoretically justify the recent empirical finding of [Teh et al., 2025] that a transformer pretrained on synthetically generated data achieves strong performance on empirical Bayes (EB) problems. We take an indirect approach to this question: rather than analyzing the model architecture or training dynamics, we ask why a pretrained Bayes estimator, trained under a prespecified training distribution, can adapt to arbitrary test distributions. Focusing on Poisson EB problems, we identify the existence of universal priors such that training under these priors yields a near-optimal regret bound of $\widetilde{O}(\frac{1}{n})$ uniformly over all test distributions. Our analysis leverages the classical phenomenon of posterior contraction in Bayesian statistics, showing that the pretrained transformer adapts to unknown test distributions precisely through posterior contraction. This perspective also explains the phenomenon of length generalization, in which the test sequence length exceeds the training length, as the model performs Bayesian inference using a generalized posterior.
检测相关(2篇)
【1】Weight space Detection of Backdoors in LoRA Adapters
标题：重量空间LoRA适配器中后门的检测
链接：https://arxiv.org/abs/2602.15195
摘要：LoRA适配器允许用户有效地微调大型语言模型（LLM）。但是，LoRA适配器通过Hugging Face Hub \citep{huggingface_hub_docs}等开放存储库共享，因此容易受到后门攻击。目前的检测方法需要使用测试输入数据运行模型，这使得它们无法筛选成千上万的适配器，其中后门行为的触发器是未知的。我们通过直接分析它们的权重矩阵来检测中毒的适配器，而不运行模型-使我们的方法与数据无关。我们的方法提取简单的统计数据-奇异值的集中程度，它们的熵和分布形状-并标记偏离正常模式的适配器。我们在500个LoRA适配器上评估了该方法- 400个干净的，100个中毒的Llama-3.2-3B指令和推理数据集：Alpaca，Dolly，GSM 8 K，ARC挑战，SQuADv 2，NaturalQuestions，HumanEval和GLUE数据集。我们达到97%的检测准确率小于2%的假阳性。
摘要：LoRA adapters let users fine-tune large language models (LLMs) efficiently. However, LoRA adapters are shared through open repositories like Hugging Face Hub \citep{huggingface_hub_docs}, making them vulnerable to backdoor attacks. Current detection methods require running the model with test input data -- making them impractical for screening thousands of adapters where the trigger for backdoor behavior is unknown. We detect poisoned adapters by analyzing their weight matrices directly, without running the model -- making our method data-agnostic. Our method extracts simple statistics -- how concentrated the singular values are, their entropy, and the distribution shape -- and flags adapters that deviate from normal patterns. We evaluate the method on 500 LoRA adapters -- 400 clean, and 100 poisoned for Llama-3.2-3B on instruction and reasoning datasets: Alpaca, Dolly, GSM8K, ARC-Challenge, SQuADv2, NaturalQuestions, HumanEval, and GLUE dataset. We achieve 97\% detection accuracy with less than 2\% false positives.
【2】Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
标题：损失最了解：通过损失轨迹检测视频中的注释错误
链接：https://arxiv.org/abs/2602.15154
备注：8 pages, 5 figures, 6 tables
摘要：高质量的视频数据集是在动作识别、相位检测和事件分割等任务中训练鲁棒模型的基础。然而，许多真实世界的视频数据集遭受注释错误，例如 * 错误标记 *，其中片段被分配了不正确的类标签，以及 * 无序 *，其中时间序列不遵循正确的进展。这些错误在阶段注释的任务中尤其有害，因为时间一致性至关重要。我们提出了一种新的，模型不可知的方法，通过分析累积样本丢失（CSL）来检测注释错误-定义为通过跨训练时期保存的模型检查点时帧所产生的平均损失。这种每帧丢失轨迹充当帧级可学习性的动态指纹。错误标记或无序的帧往往会显示出一致的高或不规则的损失模式，因为它们在整个训练过程中仍然难以让模型学习，而正确标记的帧通常会在早期收敛到低损失。为了计算CSL，我们训练视频分割模型并存储其在每个时期的权重。然后，这些检查点用于评估测试视频中每个帧的丢失。具有持续高CSL的帧被标记为注释错误的可能候选者，包括错误标记或时间未对准。我们的方法不需要地面真理的注释错误，是跨数据集的推广。在EgoPER和Cholec 80上的实验证明了强大的检测性能，有效地识别了细微的不一致，如错误标记和帧乱序。该方法为基于视频的机器学习中的数据集审计和提高训练可靠性提供了一个强大的工具。
摘要：High-quality video datasets are foundational for training robust models in tasks like action recognition, phase detection, and event segmentation. However, many real-world video datasets suffer from annotation errors such as *mislabeling*, where segments are assigned incorrect class labels, and *disordering*, where the temporal sequence does not follow the correct progression. These errors are particularly harmful in phase-annotated tasks, where temporal consistency is critical. We propose a novel, model-agnostic method for detecting annotation errors by analyzing the Cumulative Sample Loss (CSL)--defined as the average loss a frame incurs when passing through model checkpoints saved across training epochs. This per-frame loss trajectory acts as a dynamic fingerprint of frame-level learnability. Mislabeled or disordered frames tend to show consistently high or irregular loss patterns, as they remain difficult for the model to learn throughout training, while correctly labeled frames typically converge to low loss early. To compute CSL, we train a video segmentation model and store its weights at each epoch. These checkpoints are then used to evaluate the loss of each frame in a test video. Frames with persistently high CSL are flagged as likely candidates for annotation errors, including mislabeling or temporal misalignment. Our method does not require ground truth on annotation errors and is generalizable across datasets. Experiments on EgoPER and Cholec80 demonstrate strong detection performance, effectively identifying subtle inconsistencies such as mislabeling and frame disordering. The proposed approach provides a powerful tool for dataset auditing and improving training reliability in video-based machine learning.
分类|识别(2篇)
【1】Joint Enhancement and Classification using Coupled Diffusion Models of Signals and Logits
标题：使用信号和Logits的耦合扩散模型的联合增强和分类
链接：https://arxiv.org/abs/2602.15405
摘要：噪声环境中的鲁棒分类仍然是机器学习的基本挑战。标准方法通常将信号增强和分类视为单独的顺序阶段：首先增强信号，然后应用分类器。这种方法无法在去噪期间利用分类器输出中的语义信息。在这项工作中，我们提出了一个通用的，域不可知的框架，集成了两个相互作用的扩散模型：一个操作的输入信号和其他分类器的输出logits，而不需要任何重新训练或微调的分类器。这种耦合的配方，使相互指导，其中增强信号细化类估计，相反，不断发展的类logits引导信号重建的歧视性区域的歧管。我们引入了三种策略来有效地模拟输入和logit的联合分布。我们评估了我们的联合增强方法的图像分类和自动语音识别。所提出的框架超越了传统的顺序增强基线，在不同的噪声条件下提供强大和灵活的分类精度的改进。
摘要 ：Robust classification in noisy environments remains a fundamental challenge in machine learning. Standard approaches typically treat signal enhancement and classification as separate, sequential stages: first enhancing the signal and then applying a classifier. This approach fails to leverage the semantic information in the classifier's output during denoising. In this work, we propose a general, domain-agnostic framework that integrates two interacting diffusion models: one operating on the input signal and the other on the classifier's output logits, without requiring any retraining or fine-tuning of the classifier. This coupled formulation enables mutual guidance, where the enhancing signal refines the class estimation and, conversely, the evolving class logits guide the signal reconstruction towards discriminative regions of the manifold. We introduce three strategies to effectively model the joint distribution of the input and the logit. We evaluated our joint enhancement method for image classification and automatic speech recognition. The proposed framework surpasses traditional sequential enhancement baselines, delivering robust and flexible improvements in classification accuracy under diverse noise conditions.
【2】StrokeNeXt: A Siamese-encoder Approach for Brain Stroke Classification in Computed Tomography Imagery
标题：StrokeNeXt：计算机断层摄影图像中脑中风分类的连体编码器方法
链接：https://arxiv.org/abs/2602.15087
备注：10 pages, 6 figures, 11 tables
摘要：我们提出了StrokeNeXt，在2D计算机断层扫描（CT）图像中风分类模型。StrokeNeXt采用双分支设计，具有两个ConvNeXt编码器，其特征通过基于堆叠的1D操作的轻量级卷积解码器融合，包括瓶颈投影和变换层以及紧凑的分类头。该模型在6，774张CT图像的精选数据集上进行评估，解决了缺血和出血病例之间的中风检测和亚型分类。StrokeNeXt的性能始终优于卷积和基于Transformer的基线，准确度和F1分数高达0.988。配对统计检验证实，性能增益具有统计学显著性，而类灵敏度和特异性表明了跨诊断类别的稳健行为。校准分析表明，减少预测误差相比，竞争的方法，和混淆矩阵的结果表明低误分类率。此外，该模型具有推理时间短，收敛速度快。
摘要：We present StrokeNeXt, a model for stroke classification in 2D Computed Tomography (CT) images. StrokeNeXt employs a dual-branch design with two ConvNeXt encoders, whose features are fused through a lightweight convolutional decoder based on stacked 1D operations, including a bottleneck projection and transformation layers, and a compact classification head. The model is evaluated on a curated dataset of 6,774 CT images, addressing both stroke detection and subtype classification between ischemic and hemorrhage cases. StrokeNeXt consistently outperforms convolutional and Transformer-based baselines, reaching accuracies and F1-scores of up to 0.988. Paired statistical tests confirm that the performance gains are statistically significant, while class-wise sensitivity and specificity demonstrate robust behavior across diagnostic categories. Calibration analysis shows reduced prediction error compared to competing methods, and confusion matrix results indicate low misclassification rates. In addition, the model exhibits low inference time and fast convergence.
表征(3篇)
【1】UrbanVerse: Learning Urban Region Representation Across Cities and Tasks
标题：UrbanVerse：学习跨城市和任务的城市区域表示
链接：https://arxiv.org/abs/2602.15750
摘要：城市区域表示学习的最新进展已经在城市分析中实现了广泛的应用，但现有的方法在跨城市和分析任务的概括能力方面仍然有限。我们的目标是将城市表征学习推广到城市和特定任务的环境之外，建立一个基础式的城市分析模型。为此，我们提出了UrbanVerse，一个跨城市城市表征学习和跨任务城市分析的模型。对于跨城市综合，UrbanVerse专注于目标区域的局部特征和附近区域的结构特征，而不是整个城市。我们将区域建模为图上的节点，这使得基于随机行走的过程能够形成反映城市区域表示学习的局部和邻域结构特征的“区域序列”。对于跨任务泛化，我们提出了一个名为HCondDiffCT的跨任务学习模块。该模块将区域条件先验知识和任务条件语义集成到扩散过程中，以联合建模多个下游城市预测任务。HCondDiffCT是通用的。它还可以与现有的城市表征学习模型相结合，以提高其下游任务的有效性。在真实世界数据集上的实验表明，UrbanVerse在跨城市设置下的六个任务中始终优于最先进的方法，预测准确率提高了35.89%。
摘要：Recent advances in urban region representation learning have enabled a wide range of applications in urban analytics, yet existing methods remain limited in their capabilities to generalize across cities and analytic tasks. We aim to generalize urban representation learning beyond city- and task-specific settings, towards a foundation-style model for urban analytics. To this end, we propose UrbanVerse, a model for cross-city urban representation learning and cross-task urban analytics. For cross-city generalization, UrbanVerse focuses on features local to the target regions and structural features of the nearby regions rather than the entire city. We model regions as nodes on a graph, which enables a random walk-based procedure to form "sequences of regions" that reflect both local and neighborhood structural features for urban region representation learning. For cross-task generalization, we propose a cross-task learning module named HCondDiffCT. This module integrates region-conditioned prior knowledge and task-conditioned semantics into the diffusion process to jointly model multiple downstream urban prediction tasks. HCondDiffCT is generic. It can also be integrated with existing urban representation learning models to enhance their downstream task effectiveness. Experiments on real-world datasets show that UrbanVerse consistently outperforms state-of-the-art methods across six tasks under cross-city settings, achieving up to 35.89% improvements in prediction accuracy.
【2】BindCLIP: A Unified Contrastive-Generative Representation Learning Framework for Virtual Screening
标题：BindCLIP：用于虚拟筛选的统一对比生成表示学习框架
链接：https://arxiv.org/abs/2602.15236
摘要：虚拟筛选的目的是从大量的化学库中有效地识别给定靶口袋的活性配体。最近的CLIP风格的模型，如DrugCLIP，通过将口袋和配体嵌入共享空间来实现可扩展的虚拟筛选。然而，我们的分析表明，这种表示可能对细粒度的结合相互作用不敏感，并且可能依赖于训练数据中的捷径相关性，从而限制了它们通过真正的结合相容性对配体进行排名的能力。为了解决这些问题，我们提出了BindCLIP，一个统一的对比生成表示学习框架的虚拟筛选。BindCLIP使用CLIP风格的对比学习以及用于绑定姿势生成的口袋条件扩散目标来联合训练口袋和配体编码器，以便姿势级监督直接将检索嵌入空间塑造为交互相关特征。为了进一步减轻捷径依赖，我们引入了硬负增强和配体锚定正则化，防止代表崩溃。在两个公共基准上的实验表明，在强基线上有一致的改进。BindCLIP在具有挑战性的分布外虚拟筛选方面取得了实质性进展，并提高了FEP+基准的配体类似物排名。总之，这些结果表明，将生成的姿势级监督与对比学习相结合，可以产生更多的交互感知嵌入，并提高现实筛选环境中的泛化能力，使虚拟筛选更接近现实世界的适用性。
摘要：Virtual screening aims to efficiently identify active ligands from massive chemical libraries for a given target pocket. Recent CLIP-style models such as DrugCLIP enable scalable virtual screening by embedding pockets and ligands into a shared space. However, our analyses indicate that such representations can be insensitive to fine-grained binding interactions and may rely on shortcut correlations in training data, limiting their ability to rank ligands by true binding compatibility. To address these issues, we propose BindCLIP, a unified contrastive-generative representation learning framework for virtual screening. BindCLIP jointly trains pocket and ligand encoders using CLIP-style contrastive learning together with a pocket-conditioned diffusion objective for binding pose generation, so that pose-level supervision directly shapes the retrieval embedding space toward interaction-relevant features. To further mitigate shortcut reliance, we introduce hard-negative augmentation and a ligand-ligand anchoring regularizer that prevents representation collapse. Experiments on two public benchmarks demonstrate consistent improvements over strong baselines. BindCLIP achieves substantial gains on challenging out-of-distribution virtual screening and improves ligand-analogue ranking on the FEP+ benchmark. Together, these results indicate that integrating generative, pose-level supervision with contrastive learning yields more interaction-aware embeddings and improves generalization in realistic screening settings, bringing virtual screening closer to real-world applicability.
【3】Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding
标题：使用双掩蔽自动编码从不完整EHR数据中学习表示
链接：https://arxiv.org/abs/2602.15159
备注：10 pages, 4 figures
摘要：由于不规则采样、异质性缺失以及由此产生的观测稀疏性，从电子健康记录（EHR）时间序列中学习是具有挑战性的。现有的自监督方法要么在学习之前进行估算，通过专用输入信号表示缺失，要么仅针对估算进行优化，从而降低了它们有效学习支持临床下游任务的表示的能力。我们提出了增强固有双屏蔽自动编码器（AID-MAE），它通过应用固有缺失掩模来表示自然缺失值，并应用增强掩模来隐藏观察值的子集以供训练期间重建，从而直接从不完整的时间序列中学习。AID-MAE仅处理未掩蔽的令牌子集，并在两个数据集上的多个临床任务中始终优于强基线，包括XGBoost和DuETT。此外，学习的嵌入自然地在表示空间中对患者群组进行分层。
摘要：Learning from electronic health records (EHRs) time series is challenging due to irregular sam- pling, heterogeneous missingness, and the resulting sparsity of observations. Prior self-supervised meth- ods either impute before learning, represent missingness through a dedicated input signal, or optimize solely for imputation, reducing their capacity to efficiently learn representations that support clinical downstream tasks. We propose the Augmented-Intrinsic Dual-Masked Autoencoder (AID-MAE), which learns directly from incomplete time series by applying an intrinsic missing mask to represent naturally missing values and an augmented mask that hides a subset of observed values for reconstruction during training. AID-MAE processes only the unmasked subset of tokens and consistently outperforms strong baselines, including XGBoost and DuETT, across multiple clinical tasks on two datasets. In addition, the learned embeddings naturally stratify patient cohorts in the representation space.
优化|敛散性(5篇)
【1】Beyond Match Maximization and Fairness: Retention-Optimized Two-Sided Matching
标题：超越匹配最大化和公平性：保留优化的双边匹配
链接：https://arxiv.org/abs/2602.15752
备注：Published as a conference paper at ICLR 2026
摘要：在双边匹配平台上，如在线约会和招聘，推荐算法通常旨在最大化匹配总数。然而，这一目标造成了一种不平衡，一些用户收到了太多的匹配，而其他许多用户收到的匹配很少，最终放弃了平台。留住用户对于许多平台至关重要，例如那些严重依赖订阅的平台。有些人可能会使用公平性目标来解决匹配最大化的问题。然而，公平本身并不是许多平台的最终目标，因为用户不会仅仅因为曝光率相等而突然奖励平台。在实践中，用户保留通常是最终目标，随意依赖公平将使保留的优化取决于运气。 在这项工作中，而不是最大化匹配或公理定义公平，我们正式定义了新的问题设置，最大限度地提高用户保留在双边匹配平台。为此，我们引入了一个动态的学习排名（LTR）算法称为匹配保留（MRet）。与传统的双边匹配算法不同，我们的方法通过从每个用户的个人资料和交互历史中学习个性化的保留曲线来建模用户保留。基于这些曲线，MRet通过联合考虑接收推荐的用户和被推荐的用户的保留增益来动态地调整推荐，使得有限的匹配机会可以被分配到它们最能提高整体保留的地方。自然但重要的是，对来自主要在线约会平台的合成和真实世界数据集的经验评估表明，MRet实现了更高的用户保留，因为传统方法优化了匹配或公平性而不是保留。
摘要：On two-sided matching platforms such as online dating and recruiting, recommendation algorithms often aim to maximize the total number of matches. However, this objective creates an imbalance, where some users receive far too many matches while many others receive very few and eventually abandon the platform. Retaining users is crucial for many platforms, such as those that depend heavily on subscriptions. Some may use fairness objectives to solve the problem of match maximization. However, fairness in itself is not the ultimate objective for many platforms, as users do not suddenly reward the platform simply because exposure is equalized. In practice, where user retention is often the ultimate goal, casually relying on fairness will leave the optimization of retention up to luck. In this work, instead of maximizing matches or axiomatically defining fairness, we formally define the new problem setting of maximizing user retention in two-sided matching platforms. To this end, we introduce a dynamic learning-to-rank (LTR) algorithm called Matching for Retention (MRet). Unlike conventional algorithms for two-sided matching, our approach models user retention by learning personalized retention curves from each user's profile and interaction history. Based on these curves, MRet dynamically adapts recommendations by jointly considering the retention gains of both the user receiving recommendations and those who are being recommended, so that limited matching opportunities can be allocated where they most improve overall retention. Naturally but importantly, empirical evaluations on synthetic and real-world datasets from a major online dating platform show that MRet achieves higher user retention, since conventional methods optimize matches or fairness rather than retention.
【2】Guided Diffusion by Optimized Loss Functions on Relaxed Parameters for Inverse Material Design
标题：基于松弛参数优化损失函数的材料反设计引导扩散
链接：https://arxiv.org/abs/2602.15648
摘要：反设计问题在工程和材料科学中很常见。向前的方向，即，从设计参数计算输出量通常需要运行诸如FEM的数值模拟作为中间步骤，其本身是优化问题。在许多情况下，几个设计参数可以导致相同或相似的输出值。对于这种情况，多模态概率方法有利于获得不同的解决方案。逆向设计中的一个主要困难源于设计空间的结构，因为离散参数或进一步的约束不允许直接使用基于梯度的优化。为了解决这个问题，我们提出了一种新的基于扩散模型的逆设计方法。我们的方法放松到一个连续的网格表示，其中梯度可以通过隐式微分计算的前向模拟的原始设计空间。在这个放松的参数空间上训练扩散模型，以便作为合理的放松设计的先验。通过使用梯度的引导扩散对参数进行采样，所述梯度从在推理时间指定的目标函数通过可微分模拟传播。设计样本通过反投影到原始参数空间中获得。我们开发我们的方法，复合材料的设计问题，其中的正向过程建模为线性有限元问题。我们评估我们的方法在寻找匹配指定的体积模量的设计的性能。我们证明，我们的方法可以提出不同的设计在1%的相对误差范围内从中等到高的目标体积模量在2D和3D设置。我们还证明了生成的样品的材料密度可以同时最小化，通过使用多目标损失函数。
摘要：Inverse design problems are common in engineering and materials science. The forward direction, i.e., computing output quantities from design parameters, typically requires running a numerical simulation, such as a FEM, as an intermediate step, which is an optimization problem by itself. In many scenarios, several design parameters can lead to the same or similar output values. For such cases, multi-modal probabilistic approaches are advantageous to obtain diverse solutions. A major difficulty in inverse design stems from the structure of the design space, since discrete parameters or further constraints disallow the direct use of gradient-based optimization. To tackle this problem, we propose a novel inverse design method based on diffusion models. Our approach relaxes the original design space into a continuous grid representation, where gradients can be computed by implicit differentiation in the forward simulation. A diffusion model is trained on this relaxed parameter space in order to serve as a prior for plausible relaxed designs. Parameters are sampled by guided diffusion using gradients that are propagated from an objective function specified at inference time through the differentiable simulation. A design sample is obtained by backprojection into the original parameter space. We develop our approach for a composite material design problem where the forward process is modeled as a linear FEM problem. We evaluate the performance of our approach in finding designs that match a specified bulk modulus. We demonstrate that our method can propose diverse designs within 1% relative error margin from medium to high target bulk moduli in 2D and 3D settings. We also demonstrate that the material density of generated samples can be minimized simultaneously by using a multi-objective loss function.
【3】Near-Optimal Sample Complexity for Online Constrained MDPs
标题：在线受约束MDP的近优样本复杂性
链接
：https://arxiv.org/abs/2602.15076
摘要：安全性是强化学习（RL）的一个基本挑战，特别是在自动驾驶、机器人和医疗保健等现实应用中。为了解决这个问题，通常使用约束马尔可夫决策过程（CDMP）来强制执行安全约束，同时优化性能。然而，现有的方法往往遭受严重的安全违规或需要一个高的样本复杂性，以产生接近最优的政策。我们解决两个设置：宽松的可行性，其中小的违规是允许的，和严格的可行性，其中没有违规是允许的。我们提出了一个基于模型的原始-对偶算法，平衡遗憾和有界约束违反，借鉴在线RL和约束优化技术。对于放松的可行性，我们证明了我们的算法返回一个$\varepsilon$-最优策略，$\varepsilon$-有界违规具有任意高的概率，需要$\tilde{O}\left（\frac{SAH^3}{\varepsilon ^2}\right）$学习情节，匹配无约束MDP的下限。对于严格的可行性，我们证明了我们的算法返回一个$\varepsilon$-最优策略，具有任意高的概率零违规，需要$\tilde{O}\left（\frac{SAH^5}{\varepsilon ^2}\right）$学习事件，其中$\tilde $是表征可行区域大小的问题相关斯莱特常数。这个结果与使用生成模型学习CMDPs的下限相匹配。 我们的研究结果表明，在在线环境中学习CMDPs与使用生成模型学习一样容易，并且在允许小违规的情况下，不会比学习无约束的MDP更具挑战性。
摘要：Safety is a fundamental challenge in reinforcement learning (RL), particularly in real-world applications such as autonomous driving, robotics, and healthcare. To address this, Constrained Markov Decision Processes (CMDPs) are commonly used to enforce safety constraints while optimizing performance. However, existing methods often suffer from significant safety violations or require a high sample complexity to generate near-optimal policies. We address two settings: relaxed feasibility, where small violations are allowed, and strict feasibility, where no violation is allowed. We propose a model-based primal-dual algorithm that balances regret and bounded constraint violations, drawing on techniques from online RL and constrained optimization. For relaxed feasibility, we prove that our algorithm returns an $\varepsilon$-optimal policy with $\varepsilon$-bounded violation with arbitrarily high probability, requiring $\tilde{O}\left(\frac{SAH^3}{\varepsilon^2}\right)$ learning episodes, matching the lower bound for unconstrained MDPs. For strict feasibility, we prove that our algorithm returns an $\varepsilon$-optimal policy with zero violation with arbitrarily high probability, requiring $\tilde{O}\left(\frac{SAH^5}{\varepsilon^2ζ^2}\right)$ learning episodes, where $ζ$ is the problem-dependent Slater constant characterizing the size of the feasible region. This result matches the lower bound for learning CMDPs with access to a generative model. Our results demonstrate that learning CMDPs in an online setting is as easy as learning with a generative model and is no more challenging than learning unconstrained MDPs when small violations are allowed.
【4】High Convergence Rates of CMOS Invertible Logic Circuits Based on Many-Body Hamiltonians
标题：基于多体Hamilton的高收敛速度的MOS可逆逻辑电路
链接：https://arxiv.org/abs/2602.15033
备注：5 pages
摘要：介绍了基于多体哈密顿算子的CMOS可逆逻辑电路。CIL可以通过随机计算对相应的哈密顿量进行退火来实现函数的概率向前和向后运算。我们已经创建了一个包含自旋三体相互作用（概率节点）的哈密顿量。它提供了一定程度的自由度，设计一个更简单的景观的哈密顿（能量）比传统的两体哈密顿。更简单的景观更容易达到全球最低能量。提出的三体CIL电路的设计和评估与传统的两体CIL电路，导致几倍的收敛速度与FPGA上的面积开销可以忽略不计。
摘要：This paper introduces CMOS invertible-logic (CIL) circuits based on many-body Hamiltonians. CIL can realize probabilistic forward and backward operations of a function by annealing a corresponding Hamiltonian using stochastic computing. We have created a Hamiltonian that includes three-body interaction of spins (probabilistic nodes). It provides some degrees of freedom to design a simpler landscape of Hamiltonian (energy) than that of the conventional two-body Hamiltonian. The simpler landscape makes it easier to reach the global minimum energy. The proposed three-body CIL circuits are designed and evaluated with the conventional two-body CIL circuits, resulting in few-times higher convergence rates with negligible area overhead on FPGA.
【5】IT-DPC-SRI: A Cloud-Optimized Archive of Italian Radar Precipitation (2010-2025)
标题：IT-PPC-SRI：意大利雷达降水云优化档案（2010-2025）
链接：https://arxiv.org/abs/2602.15088
备注：15 pages, 7 figures
摘要：我们提出IT-DPC-SRI，意大利天气雷达降水估计的第一个公开的长期档案，跨越16年（2010- 2025）。该数据集包含来自意大利民防部门国家雷达镶嵌图的地表降雨强度（SRI）观测结果，并协调成一个连贯的分析就绪云优化（ARCO）Zarr数据立方体。该存档包括超过100万个时间步长，时间分辨率从15分钟到5分钟，覆盖了1200 × 1400公里的域，空间分辨率为1公里，磁盘上的压缩范围从7 TB到51 GB。我们解决意大利雷达数据的历史碎片-以前分散在异构格式（OPERA BUFR，HDF 5，GeoTIFF）与不同的空间域和投影-通过重新处理整个记录到一个统一的存储。该数据集可作为Zenodo上的静态版本快照访问，通过ECMWF欧洲天气云上的云原生访问，以及ArcoDataHub平台上的持续更新的实时版本。这一版本填补了欧洲雷达数据可用性的重大空白，因为意大利没有参加EUMETNET OPERA泛欧雷达组合。该数据集在CC BY-SA 4.0许可证下发布。
摘要：We present IT-DPC-SRI, the first publicly available long-term archive of Italian weather radar precipitation estimates, spanning 16 years (2010--2025). The dataset contains Surface Rainfall Intensity (SRI) observations from the Italian Civil Protection Department's national radar mosaic, harmonized into a coherent Analysis-Ready Cloud-Optimized (ARCO) Zarr datacube. The archive comprises over one million timesteps at temporal resolutions from 15 to 5 minutes, covering a $1200\times1400$ kilometer domain at 1 kilometer spatial resolution, compressed from 7TB to 51GB on disk. We address the historical fragmentation of Italian radar data - previously scattered across heterogeneous formats (OPERA BUFR, HDF5, GeoTIFF) with varying spatial domains and projections - by reprocessing the entire record into a unified store. The dataset is accessible as a static versioned snapshot on Zenodo, via cloud-native access on the ECMWF European Weather Cloud, and as a continuously updated live version on the ArcoDataHub platform. This release fills a significant gap in European radar data availability, as Italy does not participate in the EUMETNET OPERA pan-European radar composite. The dataset is released under a CC BY-SA 4.0 license.
预测|估计(4篇)
【1】Neural Network-Based Parameter Estimation of a Labour Market Agent-Based Model
标题：基于神经网络的劳动力市场主体模型参数估计
链接：https://arxiv.org/abs/2602.15572
备注：To be presented at the 6th World Conference on Complex Systems (WCCS 2026)
摘要：基于代理的建模（ABM）是模拟复杂系统的一种广泛方法。计算处理和存储的进步促进了ABM在许多领域的采用;然而，ABM面临的挑战限制了它们作为决策支持工具的使用。一个重要的问题是在大规模ABM的参数估计，特别是由于计算的限制，探索参数空间。本研究评估了一个国家的最先进的基于模拟的推理（SBI）框架，使用神经网络（NN）的参数估计。这一框架适用于建立在工作转换网络基础上的劳动力市场反弹道导弹。ABM是从合成数据集和真实的美国劳动力市场开始的。接下来，我们将从一系列统计指标中获得的汇总统计量的有效性与嵌入式NN学习的汇总统计量进行比较。结果表明，基于神经网络的方法恢复原始参数时，评估后验分布在不同的数据集规模和提高效率相比，传统的贝叶斯方法。
摘要 ：Agent-based modelling (ABM) is a widespread approach to simulate complex systems. Advancements in computational processing and storage have facilitated the adoption of ABMs across many fields; however, ABMs face challenges that limit their use as decision-support tools. A significant issue is parameter estimation in large-scale ABMs, particularly due to computational constraints on exploring the parameter space. This study evaluates a state-of-the-art simulation-based inference (SBI) framework that uses neural networks (NN) for parameter estimation. This framework is applied to an established labour market ABM based on job transition networks. The ABM is initiated with synthetic datasets and the real U.S. labour market. Next, we compare the effectiveness of summary statistics derived from a list of statistical measures with that learned by an embedded NN. The results demonstrate that the NN-based approach recovers the original parameters when evaluating posterior distributions across various dataset scales and improves efficiency compared to traditional Bayesian methods.
【2】Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment
标题：通过直接Kolen-Pollack反馈对齐加速预测编码网络
链接：https://arxiv.org/abs/2602.15571
摘要：预测编码（PC）是一种受生物学启发的算法，用于训练神经网络，仅依赖于局部更新，允许跨层并行学习。然而，实际实现面临两个关键限制：错误信号仍然必须通过多个推理阶段步骤从输出传播到早期层，并且反馈在此过程中呈指数衰减，导致早期层中的更新消失。我们提出了直接Kolen-Pollack预测编码（DKP-PC），它同时解决了反馈延迟和指数衰减，产生了一个更有效和可扩展的PC变体，同时保持更新局部性。利用直接反馈对齐和直接Kolen-Pollack算法，DKP-PC引入了从输出层到所有隐藏层的可学习反馈连接，建立了错误传输的直接路径。这产生了一种算法，该算法将理论误差传播时间复杂度从O（L）（其中L是网络深度）降低到O（1），从而消除了误差信号中的深度相关延迟。此外，实证结果表明，DKP-PC实现的性能至少与标准PC相当，甚至经常超过标准PC，同时提供更好的延迟和计算性能，支持其自定义硬件高效实现的潜力。
摘要：Predictive coding (PC) is a biologically inspired algorithm for training neural networks that relies only on local updates, allowing parallel learning across layers. However, practical implementations face two key limitations: error signals must still propagate from the output to early layers through multiple inference-phase steps, and feedback decays exponentially during this process, leading to vanishing updates in early layers. We propose direct Kolen-Pollack predictive coding (DKP-PC), which simultaneously addresses both feedback delay and exponential decay, yielding a more efficient and scalable variant of PC while preserving update locality. Leveraging direct feedback alignment and direct Kolen-Pollack algorithms, DKP-PC introduces learnable feedback connections from the output layer to all hidden layers, establishing a direct pathway for error transmission. This yields an algorithm that reduces the theoretical error propagation time complexity from O(L), with L being the network depth, to O(1), removing depth-dependent delay in error signals. Moreover, empirical results demonstrate that DKP-PC achieves performance at least comparable to, and often exceeding, that of standard PC, while offering improved latency and computational performance, supporting its potential for custom hardware-efficient implementations.
【3】Hybrid Feature Learning with Time Series Embeddings for Equipment Anomaly Prediction
标题：基于时间序列嵌入的混合特征学习用于设备异常预测
链接：https://arxiv.org/abs/2602.15089
备注：17 pages, 7 figures, 1 table
摘要：在设备的预测性维护中，基于深度学习的时间序列异常检测受到了极大的关注;然而，纯粹的深度学习方法通常无法在现实世界的数据上达到足够的准确性。这项研究提出了一种混合方法，集成了64维时间序列嵌入花岗岩TinyTimeMixer与28维统计特征的基础上领域知识的HVAC设备异常预测任务。具体来说，我们将从Granite TinyTimeMixer编码器中提取的时间序列嵌入与LoRA（低秩自适应）和28种统计特征（包括趋势，波动性和下降指标）相结合，然后使用LightGBM梯度提升分类器进行学习。在64台仪器和51，564个样品的实验中，我们在30天、60天和90天的层位上获得了91- 95%的精度和0.995的ROC AUC。此外，我们实现了生产就绪性能，假阳性率为1.1%或更低，检测率为88- 94%，证明了该系统用于预测性维护应用的有效性。这项工作表明，实际的异常检测系统可以通过利用深度学习的表示学习能力和统计特征工程之间的互补优势来实现。
摘要：In predictive maintenance of equipment, deep learning-based time series anomaly detection has garnered significant attention; however, pure deep learning approaches often fail to achieve sufficient accuracy on real-world data. This study proposes a hybrid approach that integrates 64-dimensional time series embeddings from Granite TinyTimeMixer with 28-dimensional statistical features based on domain knowledge for HVAC equipment anomaly prediction tasks. Specifically, we combine time series embeddings extracted from a Granite TinyTimeMixer encoder fine-tuned with LoRA (Low-Rank Adaptation) and 28 types of statistical features including trend, volatility, and drawdown indicators, which are then learned using a LightGBM gradient boosting classifier. In experiments using 64 equipment units and 51,564 samples, we achieved Precision of 91--95\% and ROC-AUC of 0.995 for anomaly prediction at 30-day, 60-day, and 90-day horizons. Furthermore, we achieved production-ready performance with a false positive rate of 1.1\% or less and a detection rate of 88--94\%, demonstrating the effectiveness of the system for predictive maintenance applications. This work demonstrates that practical anomaly detection systems can be realized by leveraging the complementary strengths between deep learning's representation learning capabilities and statistical feature engineering.
【4】SOON: Symmetric Orthogonal Operator Network for Global Subseasonal-to-Seasonal Climate Forecasting
标题：SOON：用于全球亚季节气候预测的对称垂直运营商网络
链接：https://arxiv.org/abs/2602.15040
摘要：准确的全球亚季节到季节（S2 S）气候预测对于备灾和资源管理至关重要，但由于大气动力学的混乱，它仍然具有挑战性。现有的模型主要把大气场作为各向同性图像，混淆了纬向波传播和纬向输运的不同物理过程，并导致各向异性动力学的次优建模。在本文中，我们提出了对称正交算子网络（SOON）的全球S2 S气候预测。它的夫妇：（1）各向异性嵌入策略，将全球网格标记为纬向环，保持纬向周期性结构的完整性;以及（2）SOON块的堆栈，通过对称分解对纬向和经向算子的交替相互作用进行建模，在结构上减轻长期积分中固有的离散化误差。在地球再分析5数据集上进行的大量实验表明，SOON建立了一种新的最先进的方法，在预测精度和计算效率方面都明显优于现有方法。
摘要：Accurate global Subseasonal-to-Seasonal (S2S) climate forecasting is critical for disaster preparedness and resource management, yet it remains challenging due to chaotic atmospheric dynamics. Existing models predominantly treat atmospheric fields as isotropic images, conflating the distinct physical processes of zonal wave propagation and meridional transport, and leading to suboptimal modeling of anisotropic dynamics. In this paper, we propose the Symmetric Orthogonal Operator Network (SOON) for global S2S climate forecasting. It couples: (1) an Anisotropic Embedding strategy that tokenizes the global grid into latitudinal rings, preserving the integrity of zonal periodic structures; and (2) a stack of SOON Blocks that models the alternating interaction of Zonal and Meridional Operators via a symmetric decomposition, structurally mitigating discretization errors inherent in long-term integration. Extensive experiments on the Earth Reanalysis 5 dataset demonstrate that SOON establishes a new state-of-the-art, significantly outperforming existing methods in both forecasting accuracy and computational efficiency.
其他神经网络|深度学习|模型|建模(21篇)
【1】Beyond Labels: Information-Efficient Human-in-the-Loop Learning using Ranking and Selection Queries
标题
：超越标签：使用排名和选择收件箱进行信息高效的人在环学习
链接：https://arxiv.org/abs/2602.15738
摘要：将人类的专业知识整合到机器学习系统中，往往会将专家的角色减少到标记神谕，这种范式限制了交换的信息量，无法捕捉人类判断的细微差别。我们通过开发一个人在环框架来解决这一挑战，以学习具有丰富查询类型的二进制分类器，包括项目排名和样本选择。我们首先引入概率人类响应模型，这些丰富的查询的实验观察到的关系之间的感知隐式得分的项目和它的距离未知的分类。使用这些模型，我们设计了主动学习算法，利用丰富的查询来增加每次交互获得的信息。我们提供了理论界样本的复杂性，并开发了一个易于处理和计算效率高的变分近似。通过对来自众包词情感和图像美学数据集的模拟注释器的实验，我们证明了样本复杂性的显着降低。我们进一步扩展主动学习策略，以选择查询，最大限度地提高信息率，明确平衡信息价值对注释成本。该算法在词情感分类任务中比传统的仅标签主动学习减少了超过57%的学习时间。
摘要：Integrating human expertise into machine learning systems often reduces the role of experts to labeling oracles, a paradigm that limits the amount of information exchanged and fails to capture the nuances of human judgment. We address this challenge by developing a human-in-the-loop framework to learn binary classifiers with rich query types, consisting of item ranking and exemplar selection. We first introduce probabilistic human response models for these rich queries motivated by the relationship experimentally observed between the perceived implicit score of an item and its distance to the unknown classifier. Using these models, we then design active learning algorithms that leverage the rich queries to increase the information gained per interaction. We provide theoretical bounds on sample complexity and develop a tractable and computationally efficient variational approximation. Through experiments with simulated annotators derived from crowdsourced word-sentiment and image-aesthetic datasets, we demonstrate significant reductions on sample complexity. We further extend active learning strategies to select queries that maximize information rate, explicitly balancing informational value against annotation cost. This algorithm in the word sentiment classification task reduces learning time by more than 57\% compared to traditional label-only active learning.
【2】Controlled oscillation modeling using port-Hamiltonian neural networks
标题：使用波特-汉密尔顿神经网络的受控振荡建模
链接：https://arxiv.org/abs/2602.15704
摘要：通过纯粹的数据驱动方法学习动力系统是具有挑战性的，因为它们没有学习使它们能够正确概括的基本守恒律。现有的端口哈密尔顿神经网络方法最近已成功地应用于建模机械系统。然而，即使这些方法是根据功率平衡原则设计的，它们通常不考虑功率保持离散化，并且通常依赖于龙格-库塔数值方法。在这项工作中，我们建议使用二阶离散梯度法嵌入在端口-哈密顿神经网络的动力系统的学习。数值结果提供了三个系统故意选择跨越不同范围的控制下的动力学行为：一个基线谐振子与二次能量存储; Duffing振荡器，与非二次哈密顿量提供幅度依赖的影响;和一个自我维持的振荡器，它可以稳定在一个受控的极限环，通过合并的非线性耗散。我们展示了如何使用这种离散梯度法优于性能的龙格-库塔法相同的顺序。还进行了实验，比较两个理论上等价的端口-哈密顿系统配方，并分析在训练过程中正则化端口-哈密顿神经网络的雅可比矩阵的影响。
摘要：Learning dynamical systems through purely data-driven methods is challenging as they do not learn the underlying conservation laws that enable them to correctly generalize. Existing port-Hamiltonian neural network methods have recently been successfully applied for modeling mechanical systems. However, even though these methods are designed on power-balance principles, they usually do not consider power-preserving discretizations and often rely on Runge-Kutta numerical methods. In this work, we propose to use a second-order discrete gradient method embedded in the learning of dynamical systems with port-Hamiltonian neural networks. Numerical results are provided for three systems deliberately selected to span different ranges of dynamical behavior under control: a baseline harmonic oscillator with quadratic energy storage; a Duffing oscillator, with a non-quadratic Hamiltonian offering amplitude-dependent effects; and a self-sustained oscillator, which can stabilize in a controlled limit cycle through the incorporation of a nonlinear dissipation. We show how the use of this discrete gradient method outperforms the performance of a Runge-Kutta method of the same order. Experiments are also carried out to compare two theoretically equivalent port-Hamiltonian systems formulations and to analyze the impact of regularizing the Jacobian of port-Hamiltonian neural networks during training.
【3】Continuous-Time Piecewise-Linear Recurrent Neural Networks
标题：连续时间分段线性回归神经网络
链接：https://arxiv.org/abs/2602.15649
摘要：在动力系统重构（DSR）中，我们的目标是恢复观测时间序列的动力系统（DS）。具体来说，我们的目标是学习一个生成代理模型，它近似的基础，数据生成DS，并重新创建其长期属性（“气候南极学”）。特别是在科学和医学领域，这些模型需要在机械上易于处理-通过它们的数学分析，我们希望深入了解恢复系统的工作原理。分段线性（PL），基于ReLU的RNN（PLRNN）在这方面有很强的跟踪记录，代表SOTA DSR模型，同时凭借其PL设计允许数学洞察力。然而，目前所有的PLRNN变体都是离散时间映射。这与大多数物理和生物过程的假设连续时间性质不一致，并且难以容纳以不规则时间间隔到达的数据。神经ODE是一种解决方案，但它们没有达到PLRNN的DSR性能，并且通常缺乏其可处理性。在这里，我们开发了连续时间PLRNN（cPLRNN）的理论：我们提出了一种新的算法来训练和模拟这些模型，通过有效地利用它们的PL结构来绕过数值积分。我们进一步展示了如何重要的拓扑对象，如平衡或极限环，可以确定半分析训练模型。我们将cPLRNN与它们的离散时间表兄弟以及DSR基准上的神经ODE进行比较，包括具有硬阈值的不连续系统。
摘要：In dynamical systems reconstruction (DSR) we aim to recover the dynamical system (DS) underlying observed time series. Specifically, we aim to learn a generative surrogate model which approximates the underlying, data-generating DS, and recreates its long-term properties (`climate statistics'). In scientific and medical areas, in particular, these models need to be mechanistically tractable -- through their mathematical analysis we would like to obtain insight into the recovered system's workings. Piecewise-linear (PL), ReLU-based RNNs (PLRNNs) have a strong track-record in this regard, representing SOTA DSR models while allowing mathematical insight by virtue of their PL design. However, all current PLRNN variants are discrete-time maps. This is in disaccord with the assumed continuous-time nature of most physical and biological processes, and makes it hard to accommodate data arriving at irregular temporal intervals. Neural ODEs are one solution, but they do not reach the DSR performance of PLRNNs and often lack their tractability. Here we develop theory for continuous-time PLRNNs (cPLRNNs): We present a novel algorithm for training and simulating such models, bypassing numerical integration by efficiently exploiting their PL structure. We further demonstrate how important topological objects like equilibria or limit cycles can be determined semi-analytically in trained models. We compare cPLRNNs to both their discrete-time cousins as well as Neural ODEs on DSR benchmarks, including systems with discontinuities which come with hard thresholds.
【4】DNN-Enabled Multi-User Beamforming for Throughput Maximization under Adjustable Fairness
标题：支持DNN的多用户射束形成在可调公平性下实现吞吐量最大化
链接：https://arxiv.org/abs/2602.15617
摘要：确保无线通信中的用户公平性是一项根本挑战，因为平衡公平性与总速率之间的权衡导致非凸多目标优化，其复杂性随着网络规模而增长。为了缓解这一冲突，我们提出了一种基于优化的无监督学习方法的基础上的无线Transformer（WiT）架构，学习信道状态信息（CSI）的功能。我们重新制定的权衡相结合的总和率和公平的目标，通过拉格朗日乘子，这是自动更新通过双上升算法。这种机制允许一个可控的公平性约束，同时最大化的总和速率，有效地实现了两个冲突的目标之间的帕累托前沿的跟踪。我们的研究结果表明，所提出的方法提供了一个灵活的解决方案，管理权衡优化规定的公平性。
摘要 ：Ensuring user fairness in wireless communications is a fundamental challenge, as balancing the trade-off between fairness and sum rate leads to a non-convex, multi-objective optimization whose complexity grows with network scale. To alleviate this conflict, we propose an optimization-based unsupervised learning approach based on the wireless transformer (WiT) architecture that learns from channel state information (CSI) features. We reformulate the trade-off by combining the sum rate and fairness objectives through a Lagrangian multiplier, which is updated automatically via a dual-ascent algorithm. This mechanism allows for a controllable fairness constraint while simultaneously maximizing the sum rate, effectively realizing a trace on the Pareto front between two conflicting objectives. Our findings show that the proposed approach offers a flexible solution for managing the trade-off optimization under prescribed fairness.
【5】A unified theory of feature learning in RNNs and DNNs
标题：RNN和DNN中特征学习的统一理论
链接：https://arxiv.org/abs/2602.15593
摘要：递归和深度神经网络（RNN/DNN）是机器学习的基石架构。值得注意的是，RNN与DNN的区别仅在于权重共享，这可以通过时间展开来证明。这种结构相似性如何与这些网络所表现出的独特功能特性相吻合？为了解决这个问题，我们在这里为RNN和DNN开发了一个统一的平均场理论，描述了特征学习（$μ$P）机制中完全训练的网络。该理论将训练视为对序列和模式的贝叶斯推理，直接揭示了RNN权重共享引起的功能含义。在典型的DNN任务中，当学习信号克服了权重随机性引起的噪声时，我们识别出一个相变：在这个阈值以下，RNN和DNN的行为相同;在这个阈值以上，只有RNN在时间步长上发展相关的表示。对于顺序任务，RNN的权重共享进一步诱导了一种归纳偏差，通过插入无监督的时间步长来帮助泛化。总的来说，我们的理论提供了一种将建筑结构与功能偏差联系起来的方法。
摘要：Recurrent and deep neural networks (RNNs/DNNs) are cornerstone architectures in machine learning. Remarkably, RNNs differ from DNNs only by weight sharing, as can be shown through unrolling in time. How does this structural similarity fit with the distinct functional properties these networks exhibit? To address this question, we here develop a unified mean-field theory for RNNs and DNNs in terms of representational kernels, describing fully trained networks in the feature learning ($μ$P) regime. This theory casts training as Bayesian inference over sequences and patterns, directly revealing the functional implications induced by the RNNs' weight sharing. In DNN-typical tasks, we identify a phase transition when the learning signal overcomes the noise due to randomness in the weights: below this threshold, RNNs and DNNs behave identically; above it, only RNNs develop correlated representations across timesteps. For sequential tasks, the RNNs' weight sharing furthermore induces an inductive bias that aids generalization by interpolating unsupervised time steps. Overall, our theory offers a way to connect architectural structure to functional biases.
【6】Uniform error bounds for quantized dynamical models
标题：量化动态模型的统一误差界
链接：https://arxiv.org/abs/2602.15586
摘要：本文提供了统计保证的准确性的动态模型学习相关的数据序列。具体来说，我们开发统一的误差界，适用于量化模型和不完美的优化算法，通常用于在实际情况下的系统识别，特别是混合系统识别。两个家庭的界限：慢速率界限通过块分解和快速率，方差自适应，界限通过一种新的空间点的战略。边界与编码模型所需的位数成比例，从而将硬件约束转换为可解释的统计复杂性。
摘要：This paper provides statistical guarantees on the accuracy of dynamical models learned from dependent data sequences. Specifically, we develop uniform error bounds that apply to quantized models and imperfect optimization algorithms commonly used in practical contexts for system identification, and in particular hybrid system identification. Two families of bounds are obtained: slow-rate bounds via a block decomposition and fast-rate, variance-adaptive, bounds via a novel spaced-point strategy. The bounds scale with the number of bits required to encode the model and thus translate hardware constraints into interpretable statistical complexities.
【7】ExLipBaB: Exact Lipschitz Constant Computation for Piecewise Linear Neural Networks
标题：ExLipBaB：分段线性神经网络的精确Lipschitz常数计算
链接：https://arxiv.org/abs/2602.15499
备注：14 pages, 1 figure
摘要：已经表明，神经网络的Lipschitz常数可以用来获得鲁棒性保证，通过正则化提高泛化能力，甚至构建可逆网络。因此，已经开发了许多方法，这些方法在其边界的紧密性和计算成本方面有所不同，以近似不同类别网络的Lipschitz常数。然而，相对较少的研究存在的方法，精确的计算，这已被证明是NP难的。尽管如此，在某些应用中，人们可能很容易接受精确方法的计算成本。这些应用可能包括新方法的基准测试或敏感数据上小模型的鲁棒性保证计算。不幸的是，现有的精确算法仅限于ReLU激活的网络，这在Lipschitz约束网络的背景下具有严重的缺点。因此，我们提出了一个推广的LipBaB算法计算精确的Lipschitz常数为任意分段线性神经网络和$p$-规范。使用我们的方法，网络可能包含传统的激活，如ReLU或LeakyReLU，激活，如GroupSort或相关的MinMax和FullSort，这些在Lipschitz约束网络的背景下越来越受到关注，甚至是其他分段线性函数，如MaxPool。
摘要：It has been shown that a neural network's Lipschitz constant can be leveraged to derive robustness guarantees, to improve generalizability via regularization or even to construct invertible networks. Therefore, a number of methods varying in the tightness of their bounds and their computational cost have been developed to approximate the Lipschitz constant for different classes of networks. However, comparatively little research exists on methods for exact computation, which has been shown to be NP-hard. Nonetheless, there are applications where one might readily accept the computational cost of an exact method. These applications could include the benchmarking of new methods or the computation of robustness guarantees for small models on sensitive data. Unfortunately, existing exact algorithms restrict themselves to only ReLU-activated networks, which are known to come with severe downsides in the context of Lipschitz-constrained networks. We therefore propose a generalization of the LipBaB algorithm to compute exact Lipschitz constants for arbitrary piecewise linear neural networks and $p$-norms. With our method, networks may contain traditional activations like ReLU or LeakyReLU, activations like GroupSort or the related MinMax and FullSort, which have been of increasing interest in the context of Lipschitz constrained networks, or even other piecewise linear functions like MaxPool.
【8】FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations
标题：Flash Mem：通过图形处理器内存层次结构优化支持移动设备上的现代DNN工作负载
链接：https://arxiv.org/abs/2602.15379
摘要 ：现代深度神经网络（DNN）的规模和复杂性不断增加，这对移动GPU上的设备上推理提出了重大挑战，内存和计算资源有限。现有的DNN加速框架主要部署权重预加载策略，其中所有模型参数在移动GPU上执行之前都加载到内存中。我们认为这种方法不足以满足现代DNN工作负载的需要，这些工作负载包括非常大的模型，并且可能连续执行多个不同的模型。在这项工作中，我们介绍了FlashMem，这是一种内存流框架，旨在有效地执行大规模现代DNN和多DNN工作负载，同时最大限度地减少内存消耗并降低推理延迟。FlashMem不是完全预加载权重，而是静态确定模型加载时间表并按需动态流式传输，利用2.5D纹理内存最大限度地减少数据转换并提高执行效率。在11个模型上的实验结果表明，与现有框架相比，FlashMem实现了2.0倍至8.4倍的内存减少和1.7倍至75.0倍的加速，从而能够在资源受限的移动GPU上高效执行大规模模型和多DNN支持。
摘要：The increasing size and complexity of modern deep neural networks (DNNs) pose significant challenges for on-device inference on mobile GPUs, with limited memory and computational resources. Existing DNN acceleration frameworks primarily deploy a weight preloading strategy, where all model parameters are loaded into memory before execution on mobile GPUs. We posit that this approach is not adequate for modern DNN workloads that comprise very large model(s) and possibly execution of several distinct models in succession. In this work, we introduce FlashMem, a memory streaming framework designed to efficiently execute large-scale modern DNNs and multi-DNN workloads while minimizing memory consumption and reducing inference latency. Instead of fully preloading weights, FlashMem statically determines model loading schedules and dynamically streams them on demand, leveraging 2.5D texture memory to minimize data transformations and improve execution efficiency. Experimental results on 11 models demonstrate that FlashMem achieves 2.0x to 8.4x memory reduction and 1.7x to 75.0x speedup compared to existing frameworks, enabling efficient execution of large-scale models and multi-DNN support on resource-constrained mobile GPUs.
【9】GMAIL: Generative Modality Alignment for generated Image Learning
标题：GMAIL：生成图像学习的生成性模式对齐
链接：https://arxiv.org/abs/2602.15368
摘要：生成模型使合成高度逼真的图像成为可能，为训练机器学习模型提供了丰富的数据源。尽管这些可合成的数据源的优点，生成的图像作为训练的真实图像的不分青红皂白的使用，甚至可以导致模式崩溃，由于真实和合成域之间的模态差异。在本文中，我们提出了一个新的框架，用于区分使用生成的图像，创造GMAIL，显式地将生成的图像作为一个单独的模态从真实的图像。我们的方法不是不加区别地用像素空间中生成的图像替换真实图像，而是通过多模态学习方法在同一潜在空间中桥接两种不同的模态。具体来说，我们首先使用跨模态对齐损失专门在生成的图像上微调模型，然后使用此对齐模型进一步训练生成图像的各种视觉语言模型。通过调整这两种模式，我们的方法有效地利用了生成模型的最新进展，从而提高了在一系列视觉语言任务中生成图像学习的有效性。我们的框架可以很容易地与各种视觉语言模型相结合，我们在广泛的实验中证明了它的有效性。例如，我们的框架显着提高了图像字幕，zero-shot图像检索，zero-shot图像分类和长字幕检索任务的性能。它还显示了积极的生成数据缩放趋势和大型多模态模型LLaVA的字幕性能的显着增强。
摘要：Generative models have made it possible to synthesize highly realistic images, potentially providing an abundant data source for training machine learning models. Despite the advantages of these synthesizable data sources, the indiscriminate use of generated images as real images for training can even cause mode collapse due to modality discrepancies between real and synthetic domains. In this paper, we propose a novel framework for discriminative use of generated images, coined GMAIL, that explicitly treats generated images as a separate modality from real images. Instead of indiscriminately replacing real images with generated ones in the pixel space, our approach bridges the two distinct modalities in the same latent space through a multi-modal learning approach. To be specific, we first fine-tune a model exclusively on generated images using a cross-modality alignment loss and then employ this aligned model to further train various vision-language models with generated images. By aligning the two modalities, our approach effectively leverages the benefits of recent advances in generative models, thereby boosting the effectiveness of generated image learning across a range of vision-language tasks. Our framework can be easily incorporated with various vision-language models, and we demonstrate its efficacy throughout extensive experiments. For example, our framework significantly improves performance on image captioning, zero-shot image retrieval, zero-shot image classification, and long caption retrieval tasks. It also shows positive generated data scaling trends and notable enhancements in the captioning performance of the large multimodal model, LLaVA.
【10】A Scalable Curiosity-Driven Game-Theoretic Framework for Long-Tail Multi-Label Learning in Data Mining
标题：数据挖掘中长尾多标签学习的可扩展好奇心驱动游戏理论框架
链接：https://arxiv.org/abs/2602.15330
摘要：长尾分布，少数头部标签占主导地位，而罕见的尾部标签比比皆是，构成了一个持久的挑战，大规模的多标签分类（MLC）在现实世界的数据挖掘应用。现有的重排序和重加权策略通常会破坏标签间的依赖关系，或者需要脆弱的超参数调整，特别是当标签空间扩展到数万个标签时。为了解决这个问题，我们提出了好奇心驱动的博弈论多标签学习（CD-GTMLL），这是一个可扩展的合作框架，它将长尾MLC重塑为多玩家游戏-每个子预测器（“玩家”）专门研究标签空间的分区，合作以最大限度地提高全局准确性，同时根据尾部标签的稀有性和玩家间的分歧追求内在的好奇心奖励。该机制自适应地将学习信号注入到表示不足的尾部标签中，而无需手动平衡或调整。我们进一步提供了一个理论分析表明，我们的CD-GTMLL收敛到一个尾部感知的平衡，并正式链接的优化动态的Rare-F1度量的改进。在7个基准测试中进行的广泛实验，包括具有30，000多个标签的极端多标签分类数据集，表明CD-GTMLL始终超过最先进的方法，在Wiki 10 - 31 K上的P@3增益高达+1.6%。消融研究进一步证实了博弈论合作和好奇心驱动的探索对稳健尾部性能的贡献。通过将博弈论与好奇心机制相结合，CD-GTMLL不仅提高了资源受限环境中的模型效率，还为在电子商务和医疗保健等行业的不平衡数据场景中进行更具适应性的学习铺平了道路。
摘要：The long-tail distribution, where a few head labels dominate while rare tail labels abound, poses a persistent challenge for large-scale Multi-Label Classification (MLC) in real-world data mining applications. Existing resampling and reweighting strategies often disrupt inter-label dependencies or require brittle hyperparameter tuning, especially as the label space expands to tens of thousands of labels. To address this issue, we propose Curiosity-Driven Game-Theoretic Multi-Label Learning (CD-GTMLL), a scalable cooperative framework that recasts long-tail MLC as a multi-player game - each sub-predictor ("player") specializes in a partition of the label space, collaborating to maximize global accuracy while pursuing intrinsic curiosity rewards based on tail label rarity and inter-player disagreement. This mechanism adaptively injects learning signals into under-represented tail labels without manual balancing or tuning. We further provide a theoretical analysis showing that our CD-GTMLL converges to a tail-aware equilibrium and formally links the optimization dynamics to improvements in the Rare-F1 metric. Extensive experiments across 7 benchmarks, including extreme multi-label classification datasets with 30,000+ labels, demonstrate that CD-GTMLL consistently surpasses state-of-the-art methods, with gains up to +1.6% P@3 on Wiki10-31K. Ablation studies further confirm the contributions of both game-theoretic cooperation and curiosity-driven exploration to robust tail performance. By integrating game theory with curiosity mechanisms, CD-GTMLL not only enhances model efficiency in resource-constrained environments but also paves the way for more adaptive learning in imbalanced data scenarios across industries like e-commerce and healthcare.
【11】Automatically Finding Reward Model Biases
标题：自动查找奖励模型偏见
链接：https://arxiv.org/abs/2602.15222
摘要：奖励模型是大型语言模型（LLM）后训练的核心。然而，过去的研究表明，它们可以奖励虚假或不受欢迎的属性，如长度，格式，幻觉和奉承。在这项工作中，我们介绍和研究的研究问题，自动发现奖励模型的偏见，在自然语言。我们提供了一种简单的方法，使用LLM迭代地提出和改进候选偏差。我们的方法可以恢复已知的偏见和表面新颖的：例如，我们发现Skywork-V2-8B，一个领先的开放权重奖励模型，经常错误地倾向于具有冗余间距的响应和具有幻觉内容的响应。此外，我们证明了进化迭代优于平坦的最佳N搜索的证据，并且我们使用合成注入的偏差验证了我们的管道的召回。我们希望我们的工作有助于进一步研究通过自动化的可解释性方法来改善RM。
摘要 ：Reward models are central to large language model (LLM) post-training. However, past work has shown that they can reward spurious or undesirable attributes such as length, format, hallucinations, and sycophancy. In this work, we introduce and study the research problem of automatically finding reward model biases in natural language. We offer a simple approach of using an LLM to iteratively propose and refine candidate biases. Our method can recover known biases and surface novel ones: for example, we found that Skywork-V2-8B, a leading open-weight reward model, often mistakenly favors responses with redundant spacing and responses with hallucinated content. In addition, we show evidence that evolutionary iteration outperforms flat best-of-N search, and we validate the recall of our pipeline using synthetically injected biases. We hope our work contributes to further research on improving RMs through automated interpretability methods.
【12】Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge
标题：通过基础物理知识学习数据高效且可推广的神经运算符
链接：https://arxiv.org/abs/2602.15184
摘要：科学机器学习（SciML）的最新进展使神经运算符（NO）能够作为模拟由偏微分方程（PDE）控制的物理系统动态演化的强大代理。虽然现有的方法主要集中在学习模拟目标偏微分方程，他们往往忽略了更基本的物理原理，这些方程。受数值求解器如何与PDE的不同设置的模拟兼容的启发，我们提出了一个多物理场训练框架，该框架从原始PDE及其简化的基本形式中共同学习。我们的框架提高了数据效率，减少了预测误差，并提高了分布外（OOD）的泛化能力，特别是在涉及物理参数变化和合成到真实传输的情况下。我们的方法是架构不可知的，并在广泛的1D/2D/3D PDE问题的归一化均方根误差（nRMSE）表现出一致的改善。通过大量的实验，我们表明，明确纳入基础物理知识显着增强神经操作符的泛化能力。我们将在https://sites.google.com/view/sciml-fundemental-pde发布模型和代码。
摘要：Recent advances in scientific machine learning (SciML) have enabled neural operators (NOs) to serve as powerful surrogates for modeling the dynamic evolution of physical systems governed by partial differential equations (PDEs). While existing approaches focus primarily on learning simulations from the target PDE, they often overlook more fundamental physical principles underlying these equations. Inspired by how numerical solvers are compatible with simulations of different settings of PDEs, we propose a multiphysics training framework that jointly learns from both the original PDEs and their simplified basic forms. Our framework enhances data efficiency, reduces predictive errors, and improves out-of-distribution (OOD) generalization, particularly in scenarios involving shifts of physical parameters and synthetic-to-real transfer. Our method is architecture-agnostic and demonstrates consistent improvements in normalized root mean square error (nRMSE) across a wide range of 1D/2D/3D PDE problems. Through extensive experiments, we show that explicit incorporation of fundamental physics knowledge significantly strengthens the generalization ability of neural operators. We will release models and codes at https://sites.google.com/view/sciml-fundemental-pde.
【13】Ensemble-size-dependence of deep-learning post-processing methods that minimize an (un)fair score: motivating examples and a proof-of-concept solution
标题：深度学习后处理方法的整体规模依赖性，最大限度地减少（不）公平分数：激励示例和概念验证解决方案
链接：https://arxiv.org/abs/2602.15830
摘要：公平分数奖励集合预报成员，这些成员表现得像来自与验证观测相同的分布的样本。因此，他们是一个有吸引力的选择，损失函数训练数据驱动的集合预报或后处理方法时，大型训练合奏是不可用的或计算上禁止。调整后的连续排序概率得分（aCRPS）是公平和公正的合奏大小，提供预测成员是可交换的，并解释为有条件的独立提请从一个潜在的预测分布。然而，在成员之间引入结构依赖的分布感知后处理方法可能会违反这一假设，从而使aCRPS不公平。我们使用两种旨在最小化有限集合的预期aCRPS的方法来证明这种效果：（1）线性成员-成员校准，通过对样本集合平均值的共同依赖来耦合成员，以及（2）深度学习方法，通过集合维度上的Transformer自我注意来耦合成员。在这两种情况下，结果是敏感的合奏大小和明显的增益在aCRPS可以对应于系统的不可靠性，其特征在于过度分散。我们引入轨迹Transformers作为概念验证，可以实现集合大小的独立性。这种方法是一个适应后处理集成与Transformers（PoET）框架，并适用于自注意的提前期，同时保持条件的独立性所需的aCRPS。当应用于ECMWF亚季节预报系统的周平均T_{2 m}$预报时，该方法成功地减少了系统误差，同时也提高或保持了预报的可靠性，而不管训练中使用的集合规模（3 vs 9个成员）或实时预报（9 vs 100个成员）。
摘要：Fair scores reward ensemble forecast members that behave like samples from the same distribution as the verifying observations. They are therefore an attractive choice as loss functions to train data-driven ensemble forecasts or post-processing methods when large training ensembles are either unavailable or computationally prohibitive. The adjusted continuous ranked probability score (aCRPS) is fair and unbiased with respect to ensemble size, provided forecast members are exchangeable and interpretable as conditionally independent draws from an underlying predictive distribution. However, distribution-aware post-processing methods that introduce structural dependency between members can violate this assumption, rendering aCRPS unfair. We demonstrate this effect using two approaches designed to minimize the expected aCRPS of a finite ensemble: (1) a linear member-by-member calibration, which couples members through a common dependency on the sample ensemble mean, and (2) a deep-learning method, which couples members via transformer self-attention across the ensemble dimension. In both cases, the results are sensitive to ensemble size and apparent gains in aCRPS can correspond to systematic unreliability characterized by over-dispersion. We introduce trajectory transformers as a proof-of-concept that ensemble-size independence can be achieved. This approach is an adaptation of the Post-processing Ensembles with Transformers (PoET) framework and applies self-attention over lead time while preserving the conditional independence required by aCRPS. When applied to weekly mean $T_{2m}$ forecasts from the ECMWF subseasonal forecasting system, this approach successfully reduces systematic model biases whilst also improving or maintaining forecast reliability regardless of the ensemble size used in training (3 vs 9 members) or real-time forecasts (9 vs 100 members).
【14】Enabling Low-Latency Machine learning on Radiation-Hard FPGAs with hls4ml
标题：在具有hls 4 ml的抗辐射VGA上实现低延迟机器学习
链接：https://arxiv.org/abs/2602.15751
摘要：本文首次在FPGA上展示了一种可行的、超快的、抗辐射的机器学习（ML）应用，可用于未来的高能物理实验。我们提出了一个三倍的贡献，与PicoCal量热计，计划用于LHCb升级II实验，用作测试用例。首先，我们开发了一个轻量级的自动编码器压缩32个样本的定时读出，代表的PicoCal，到一个二维的潜在空间。其次，我们介绍了一个系统的，硬件感知的量化策略，并表明该模型可以减少到10位的权重，以最小的性能损失。第三，由于采用探测器上ML的障碍是高能物理社区的标准ML合成工具hls4ml缺乏对抗辐射FPGA的支持，我们为此库开发了一个新的后端。这个新的后端可以将ML模型自动转换为Microchip PolarFire系列FPGA的高级综合（HLS）项目，这是少数几个商用和抗辐射FPGA之一。我们提出了在目标PolarFire FPGA上合成自动编码器，这表明可以实现25 ns的延迟。我们表明，利用的资源是足够低的模型可以放置在固有的保护逻辑的FPGA。我们对hls4ml的扩展是一项重大贡献，为在高辐射环境中更广泛地采用FPGA上的ML铺平了道路。
摘要 ：This paper presents the first demonstration of a viable, ultra-fast, radiation-hard machine learning (ML) application on FPGAs, which could be used in future high-energy physics experiments. We present a three-fold contribution, with the PicoCal calorimeter, planned for the LHCb Upgrade II experiment, used as a test case. First, we develop a lightweight autoencoder to compress a 32-sample timing readout, representative of that of the PicoCal, into a two-dimensional latent space. Second, we introduce a systematic, hardware-aware quantization strategy and show that the model can be reduced to 10-bit weights with minimal performance loss. Third, as a barrier to the adoption of on-detector ML is the lack of support for radiation-hard FPGAs in the High-Energy Physics community's standard ML synthesis tool, hls4ml, we develop a new backend for this library. This new back-end enables the automatic translation of ML models into High-Level Synthesis (HLS) projects for the Microchip PolarFire family of FPGAs, one of the few commercially available and radiation hard FPGAs. We present the synthesis of the autoencoder on a target PolarFire FPGA, which indicates that a latency of 25 ns can be achieved. We show that the resources utilized are low enough that the model can be placed within the inherently protected logic of the FPGA. Our extension to hls4ml is a significant contribution, paving the way for broader adoption of ML on FPGAs in high-radiation environments.
【15】Uni-Flow: a unified autoregressive-diffusion model for complex multiscale flows
标题：Uni-Flow：复杂多尺度流的统一自回归扩散模型
链接：https://arxiv.org/abs/2602.15592
摘要：时空流控制着物理学、生物学和工程学中的各种现象，但对其多尺度动力学进行建模仍然是一个核心挑战。尽管在物理信息机器学习方面取得了重大进展，但现有方法难以同时维持长期的时间演化并解决混沌，湍流和生理状态的精细尺度结构。在这里，我们介绍了Uni-Flow，一个统一的自回归扩散框架，明确地将时间演化从空间细化中分离出来，用于建模复杂的动力系统。自回归组件学习低分辨率的潜在动态，以保持大规模结构并确保稳定的长期推出，而扩散组件重建高分辨率的物理场，在少量的去噪步骤中恢复精细尺度特征。我们在规范基准上验证了Uni-Flow，包括二维Kolmogorov流，具有量子信息自回归先验的三维湍流通道流入生成，以及来自高保真格子Boltzmann血流动力学求解器的主动脉缩窄的患者特定模拟。在心血管环境中，Uni-Flow能够比实时推断脉动血流动力学更快地实现任务级，在几秒钟而不是几小时内重建生理相关时间范围内的高分辨率压力场。通过将高保真血流动力学模拟从离线的HPC绑定过程转换为可部署的代理，Uni-Flow建立了一条比复杂多尺度流的实时建模更快的途径，对流物理学中的科学机器学习具有广泛的影响。
摘要：Spatiotemporal flows govern diverse phenomena across physics, biology, and engineering, yet modelling their multiscale dynamics remains a central challenge. Despite major advances in physics-informed machine learning, existing approaches struggle to simultaneously maintain long-term temporal evolution and resolve fine-scale structure across chaotic, turbulent, and physiological regimes. Here, we introduce Uni-Flow, a unified autoregressive-diffusion framework that explicitly separates temporal evolution from spatial refinement for modelling complex dynamical systems. The autoregressive component learns low-resolution latent dynamics that preserve large-scale structure and ensure stable long-horizon rollouts, while the diffusion component reconstructs high-resolution physical fields, recovering fine-scale features in a small number of denoising steps. We validate Uni-Flow across canonical benchmarks, including two-dimensional Kolmogorov flow, three-dimensional turbulent channel inflow generation with a quantum-informed autoregressive prior, and patient-specific simulations of aortic coarctation derived from high-fidelity lattice Boltzmann hemodynamic solvers. In the cardiovascular setting, Uni-Flow enables task-level faster than real-time inference of pulsatile hemodynamics, reconstructing high-resolution pressure fields over physiologically relevant time horizons in seconds rather than hours. By transforming high-fidelity hemodynamic simulation from an offline, HPC-bound process into a deployable surrogate, Uni-Flow establishes a pathway to faster-than-real-time modelling of complex multiscale flows, with broad implications for scientific machine learning in flow physics.
【16】Fluids You Can Trust: Property-Preserving Operator Learning for Incompressible Flows
标题：您可以信任的流体：保特性操作员学习不可压缩流
链接：https://arxiv.org/abs/2602.15472
摘要：针对不可压缩Navier-Stokes方程所控制的不可压缩流，提出了一种新的基于核函数的保性质算子学习方法。传统的数值求解器产生显着的计算成本，尊重不可压缩性。算子学习提供了有效的代理模型，但目前的神经算子无法精确地执行物理属性，如不可压缩性，周期性和湍流。我们的方法将输入函数映射到输出函数的展开系数，以保持性质的核基，确保预测的速度场分析，同时保持上述物理性质。我们评估具有挑战性的2D和3D，层流和湍流，不可压缩流问题的方法。我们的方法实现了高达6个数量级的相对误差较低的泛化和训练高达5个数量级的速度相比，神经操作。此外，虽然我们的方法在分析上强制不可压缩性，但神经算子表现出非常大的偏差。我们的结果表明，我们的方法提供了一个准确和有效的替代不可压缩流。
摘要：We present a novel property-preserving kernel-based operator learning method for incompressible flows governed by the incompressible Navier-Stokes equations. Traditional numerical solvers incur significant computational costs to respect incompressibility. Operator learning offers efficient surrogate models, but current neural operators fail to exactly enforce physical properties such as incompressibility, periodicity, and turbulence. Our method maps input functions to expansion coefficients of output functions in a property-preserving kernel basis, ensuring that predicted velocity fields analytically and simultaneously preserve the aforementioned physical properties. We evaluate the method on challenging 2D and 3D, laminar and turbulent, incompressible flow problems. Our method achieves up to six orders of magnitude lower relative $\ell_2$ errors upon generalization and trains up to five orders of magnitude faster compared to neural operators. Moreover, while our method enforces incompressibility analytically, neural operators exhibit very large deviations. Our results show that our method provides an accurate and efficient surrogate for incompressible flows.
【17】The Skeletal Trap: Mapping Spatial Inequality and Ghost Stops in Ankara's Transit Network
标题：Skellow陷阱：绘制安卡拉交通网络中的空间不平等和幽灵站
链接：https://arxiv.org/abs/2602.15470
备注：13 pages, 12 figures. Spatial analysis of Ankara transit network using anomaly detection and grid-based modeling
摘要：安卡拉的公共交通危机通常被认为是公共汽车短缺或运营效率低下。本研究认为，这一问题从根本上说是形态和结构问题。这座城市的跳跃式城市扩张产生了支离破碎的外围集群，与僵化的、面向中心的公交网络脱节。因此，需求仍然高度集中在克孜勒-乌鲁斯轴线和西部走廊沿线，而周边地区要么长期服务不足，要么被迫依赖转移。因此，这种缺陷不仅是数量上的，而且根源于城市宏观形态和网络架构之间的不协调。实证分析借鉴了173天的运营数据集，这些数据集来自EGO在前“透明安卡拉”倡议下发布的路线级乘客和行程报告。为了克服车站级地理空间数据的缺乏，基于连通性的加权分布模型使用网络中心性将乘客量重新分配到1 km x 1 km的网格单元。研究结果揭示了持续的中心-外围不对称，结构性瓶颈和空间嵌入的可达性不平等。
摘要：Ankara's public transport crisis is commonly framed as a shortage of buses or operational inefficiency. This study argues that the problem is fundamentally morphological and structural. The city's leapfrog urban expansion has produced fragmented peripheral clusters disconnected from a rigid, center-oriented bus network. As a result, demand remains intensely concentrated along the Kizilay-Ulus axis and western corridors, while peripheral districts experience either chronic under-service or enforced transfer dependency. The deficiency is therefore not merely quantitative but rooted in the misalignment between urban macroform and network architecture. The empirical analysis draws on a 173-day operational dataset derived from route-level passenger and trip reports published by EGO under the former "Transparent Ankara" initiative. To overcome the absence of stop-level geospatial data, a Connectivity-Based Weighted Distribution Model reallocates passenger volumes to 1 km x 1 km grid cells using network centrality. The findings reveal persistent center-periphery asymmetries, structural bottlenecks, and spatially embedded accessibility inequalities.
【18】Molecular Design beyond Training Data with Novel Extended Objective Functionals of Generative AI Models Driven by Quantum Annealing Computer
标题：量子退修计算机驱动的生成性人工智能模型的新型扩展目标函数超越训练数据的分子设计
链接：https://arxiv.org/abs/2602.15451
备注：42 pages, 7 figures
摘要：深度生成建模以随机设计小分子是用于加速药物发现和开发的新兴技术。然而，分子生成模型的一个主要问题是它们的药物样化合物的频率较低。为了解决这个问题，我们开发了一种新的框架，用于优化与D-Wave量子退火计算机集成的深度生成模型，其中本文提出的神经哈希函数（NHF）同时用作正则化和二值化方案，其中后者分别用于经典和量子神经网络的连续和离散信号之间的转换，在误差评估中（即，目标）函数。通过量子退火生成模型生成的化合物在有效性和药物相似性方面都比通过完全经典模型生成的化合物表现出更高的质量，并且进一步表明在药物相似性特征方面甚至超过训练数据，而没有任何限制和条件来故意诱导这种优化。这些结果表明，量子退火的优势，旨在与我们的新的神经网络架构集成的随机发生器，在药物设计中的特征空间采样和特征提取的扩展性能。
摘要：Deep generative modeling to stochastically design small molecules is an emerging technology for accelerating drug discovery and development. However, one major issue in molecular generative models is their lower frequency of drug-like compounds. To resolve this problem, we developed a novel framework for optimization of deep generative models integrated with a D-Wave quantum annealing computer, where our Neural Hash Function (NHF) presented herein is used both as the regularization and binarization schemes simultaneously, of which the latter is for transformation between continuous and discrete signals of the classical and quantum neural networks, respectively, in the error evaluation (i.e., objective) function. The compounds generated via the quantum-annealing generative models exhibited higher quality in both validity and drug-likeness than those generated via the fully-classical models, and was further indicated to exceed even the training data in terms of drug-likeness features, without any restraints and conditions to deliberately induce such an optimization. These results indicated an advantage of quantum annealing to aim at a stochastic generator integrated with our novel neural network architectures, for the extended performance of feature space sampling and extraction of characteristic features in drug design.
【19】Sparse Additive Model Pruning for Order-Based Causal Structure Learning
标题：基于顺序的因果结构学习的稀疏加性模型修剪
链接：https://arxiv.org/abs/2602.15306
备注：15 pages, 12 figures, to appear in the 40th AAAI Conference on Artificial Intelligence (AAAI 2026)
摘要：因果结构学习，也称为因果发现，旨在从观测数据中估计变量之间的因果关系，作为因果有向无环图（DAG）的形式。主要框架之一是基于序的方法，首先估计底层DAG的拓扑序，然后从由估计的拓扑序引起的全连接DAG中修剪伪边。以前的研究往往集中在前一个排序步骤，因为它可以显着减少搜索空间的DAG。在实践中，后一个修剪步骤对于确保计算效率和估计精度同样至关重要。大多数现有的方法采用基于广义加性模型和假设检验的修剪技术，通常称为CAM修剪。然而，这种方法可能是一个计算瓶颈，因为它需要重复拟合所有变量的加性模型。此外，由于多次测试，它可能会损害估计质量。为了解决这些问题，我们引入了一种新的修剪方法的基础上稀疏加性模型，它可以直接修剪冗余的边缘，而不依赖于假设检验。我们提出了一个有效的算法学习稀疏加性模型，结合随机树嵌入技术与分组稀疏回归。在合成数据集和真实数据集上的实验结果表明，我们的方法比现有的剪枝方法要快得多，同时保持相当或更高的准确性。
摘要：Causal structure learning, also known as causal discovery, aims to estimate causal relationships between variables as a form of a causal directed acyclic graph (DAG) from observational data. One of the major frameworks is the order-based approach that first estimates a topological order of the underlying DAG and then prunes spurious edges from the fully-connected DAG induced by the estimated topological order. Previous studies often focus on the former ordering step because it can dramatically reduce the search space of DAGs. In practice, the latter pruning step is equally crucial for ensuring both computational efficiency and estimation accuracy. Most existing methods employ a pruning technique based on generalized additive models and hypothesis testing, commonly known as CAM-pruning. However, this approach can be a computational bottleneck as it requires repeatedly fitting additive models for all variables. Furthermore, it may harm estimation quality due to multiple testing. To address these issues, we introduce a new pruning method based on sparse additive models, which enables direct pruning of redundant edges without relying on hypothesis testing. We propose an efficient algorithm for learning sparse additive models by combining the randomized tree embedding technique with group-wise sparse regression. Experimental results on both synthetic and real datasets demonstrated that our method is significantly faster than existing pruning methods while maintaining comparable or superior accuracy.
【20】Reconstructing Carbon Monoxide Reanalysis with Machine Learning
标题：用机器学习重建一氧化碳重新分析
链接：https://arxiv.org/abs/2602.15056
摘要：哥白尼大气监测服务通过将模型模拟与卫星观测相结合，提供大气成分再分析产品。这些产品的质量在很大程度上取决于观测数据的可用性，随着新的卫星仪器的可用或停止使用，这些数据可能会随着时间的推移而变化，例如2025年初对流层污染测量（MOPITT）卫星的一氧化碳（CO）观测。机器学习提供了一种很有前途的方法，通过学习模型配置之间的系统差异来补偿这种数据丢失。在这项研究中，我们研究了机器学习方法来预测控制模型模拟的一氧化碳再分析的月平均总柱。
摘要：The Copernicus Atmospheric Monitoring Service provides reanalysis products for atmospheric composition by combining model simulations with satellite observations. The quality of these products depends strongly on the availability of the observational data, which can vary over time as new satellite instruments become available or are discontinued, such as Carbon Monoxide (CO) observations of the Measurements Of Pollution In The Troposphere (MOPITT) satellite in early 2025. Machine learning offers a promising approach to compensate for such data losses by learning systematic discrepancies between model configurations. In this study, we investigate machine learning methods to predict monthly-mean total column of Carbon Monoxide re-analysis from a control model simulation.
【21】Accurate 2D Reconstruction for PET Scanners based on the Analytical White Image Model
标题：基于分析白图像模型的PET扫描仪精确2D重建
链接：https://arxiv.org/abs/2306.17652
备注：37 pages, 16 figures
摘要：在本文中，我们提供了一个精确的晶体到晶体响应数学模型，用于生成白色图像-这是克服PET扫描仪物理限制所需的必要补偿模型。我们提出了一个封闭的形式的解决方案，以及几个准确的近似，由于精确的数学表达式的复杂性。我们证明，实验和分析，最佳近似和真正的晶体到晶体的响应之间的差异是微不足道的。所获得的响应用于生成白图像补偿模型。它可以被写为单个封闭形式的表达式，使得它容易在已知的重建方法中实现。对最大似然期望最大化（MLEM）算法进行了改进，并将我们的白图像模型融入其中，改进的MLEM算法不是基于系统矩阵，而是基于光线驱动投影和反投影。补偿模型提供有关系统的所有必要信息。最后，我们在合成数据和真实数据上检查我们的方法。对于真实世界的采集，我们使用小动物的Raytest ClearPET相机和NEMA NU 4-2008体模。所提出的方法优于竞争，非补偿重建方法。
摘要 ：In this paper, we provide a precise mathematical model of crystal-to-crystal response which is used to generate the white image - a necessary compensation model needed to overcome the physical limitations of the PET scanner. We present a closed-form solution, as well as several accurate approximations, due to the complexity of the exact mathematical expressions. We prove, experimentally and analytically, that the difference between the best approximations and real crystal-to-crystal response is insignificant. The obtained responses are used to generate the white image compensation model. It can be written as a single closed-form expression making it easy to implement in known reconstruction methods. The maximum likelihood expectation maximization (MLEM) algorithm is modified and our white image model is integrated into it. The modified MLEM algorithm is not based on the system matrix, rather it is based on ray-driven projections and back-projections. The compensation model provides all necessary information about the system. Finally, we check our approach on synthetic and real data. For the real-world acquisition, we use the Raytest ClearPET camera for small animals and the NEMA NU 4-2008 phantom. The proposed approach overperforms competitive, non-compensated reconstruction methods.
其他(37篇)
【1】Operationalising the Superficial Alignment Hypothesis via Task Complexity
标题：通过任务复杂性操作表面对齐假设
链接：https://arxiv.org/abs/2602.15829
摘要：表面对齐假设（SAH）认为，大型语言模型在预训练期间学习了大部分知识，而后训练只是将这些知识表面化。SAH，但是，缺乏一个精确的定义，这导致了（一）不同的和看似正交的参数支持它，和（ii）重要的crituquesit. We提出了一个新的度量称为任务复杂性：最短的程序，实现目标性能的任务的长度。在这个框架中，SAH只是声称预训练的模型大大降低了在许多任务上实现高性能的复杂性。我们的定义统一了支持SAH的先前参数，将其解释为找到此类短程序的不同策略。在实验中，我们估计了数学推理，机器翻译和指令跟踪的任务复杂度;然后我们表明，当以预先训练的模型为条件时，这些复杂度可以非常低。此外，我们发现预训练可以让我们在任务中获得出色的表现，但可能需要数GB长度的程序才能访问它们。另一方面，后期训练将达到相同性能的复杂性降低了几个数量级。总的来说，我们的研究结果强调了任务适应通常需要的信息很少，通常只有几个字节。
摘要：The superficial alignment hypothesis (SAH) posits that large language models learn most of their knowledge during pre-training, and that post-training merely surfaces this knowledge. The SAH, however, lacks a precise definition, which has led to (i) different and seemingly orthogonal arguments supporting it, and (ii) important critiques to it. We propose a new metric called task complexity: the length of the shortest program that achieves a target performance on a task. In this framework, the SAH simply claims that pre-trained models drastically reduce the complexity of achieving high performance on many tasks. Our definition unifies prior arguments supporting the SAH, interpreting them as different strategies to find such short programs. Experimentally, we estimate the task complexity of mathematical reasoning, machine translation, and instruction following; we then show that these complexities can be remarkably low when conditioned on a pre-trained model. Further, we find that pre-training enables access to strong performances on our tasks, but it can require programs of gigabytes of length to access them. Post-training, on the other hand, collapses the complexity of reaching this same performance by several orders of magnitude. Overall, our results highlight that task adaptation often requires surprisingly little information -- often just a few kilobytes.
【2】Dex4D: Task-Agnostic Point Track Policy for Sim-to-Real Dexterous Manipulation
标题：Dex 4D：模拟到真实灵巧操纵的任务不可知点跟踪策略
链接：https://arxiv.org/abs/2602.15828
备注：Project page: https://dex4d.github.io/
摘要：学习能够完成大量日常任务的通才策略仍然是灵巧操作的公开挑战。特别是，通过现实世界的遥操作收集大规模的操作数据是昂贵的，难以扩展。虽然模拟学习提供了一种可行的替代方案，但设计多个特定任务的环境和培训奖励同样具有挑战性。我们提出了Dex4D，这是一个利用模拟来学习任务不可知的灵巧技能的框架，这些技能可以灵活地重新组合以执行各种现实世界的操作任务。具体来说，Dex4D学习了一种与域无关的3D点跟踪条件策略，能够将任何对象操纵到任何所需的姿势。我们训练这种“任意姿态到任意姿态”的策略在模拟中跨越数千个具有不同姿态配置的对象，覆盖了可以在测试时组成的机器人对象交互的广阔空间。在部署时，该策略可以zero-shot转移到现实世界的任务，而无需微调，只需通过从生成的视频中提取的所需对象为中心的点轨迹来提示它。在执行过程中，Dex4D使用在线点跟踪进行闭环感知和控制。在仿真和真实机器人上的大量实验表明，我们的方法可以实现zero-shot部署不同的灵巧操作任务，并产生一致的改进比以前的基线。此外，我们表现出强大的泛化到新的对象，场景布局，背景和轨迹，突出了所提出的框架的鲁棒性和可扩展性。
摘要：Learning generalist policies capable of accomplishing a plethora of everyday tasks remains an open challenge in dexterous manipulation. In particular, collecting large-scale manipulation data via real-world teleoperation is expensive and difficult to scale. While learning in simulation provides a feasible alternative, designing multiple task-specific environments and rewards for training is similarly challenging. We propose Dex4D, a framework that instead leverages simulation for learning task-agnostic dexterous skills that can be flexibly recomposed to perform diverse real-world manipulation tasks. Specifically, Dex4D learns a domain-agnostic 3D point track conditioned policy capable of manipulating any object to any desired pose. We train this 'Anypose-to-Anypose' policy in simulation across thousands of objects with diverse pose configurations, covering a broad space of robot-object interactions that can be composed at test time. At deployment, this policy can be zero-shot transferred to real-world tasks without finetuning, simply by prompting it with desired object-centric point tracks extracted from generated videos. During execution, Dex4D uses online point tracking for closed-loop perception and control. Extensive experiments in simulation and on real robots show that our method enables zero-shot deployment for diverse dexterous manipulation tasks and yields consistent improvements over prior baselines. Furthermore, we demonstrate strong generalization to novel objects, scene layouts, backgrounds, and trajectories, highlighting the robustness and scalability of the proposed framework.
【3】Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching
标题：感知人形跑酷：通过运动匹配链接动态人类技能
链接：https://arxiv.org/abs/2602.15827
摘要：虽然类人运动的最新进展已经实现了在不同地形上的稳定行走，但捕捉高度动态的人类运动的敏捷性和适应性仍然是一个开放的挑战。特别是，复杂环境中的敏捷跑酷不仅需要低级别的鲁棒性，还需要像人类一样的运动表现力，长视野技能组合和感知驱动的决策。在本文中，我们提出了感知类人跑酷（PHP），一个模块化的框架，使类人机器人自主执行长视野，基于视觉的跑酷跨越具有挑战性的障碍课程。我们的方法首先利用运动匹配，制定为特征空间中的最近邻搜索，将重定向的原子人类技能组合成长期的运动轨迹。该框架能够灵活地组合和平滑过渡复杂的技能链，同时保持动态人体动作的优雅性和流动性。接下来，我们为这些组合运动训练训练运动跟踪强化学习（RL）专家策略，并使用Dagger和RL的组合将其提取为基于深度的多技能学生策略。至关重要的是，感知和技能组合的结合实现了自主的、上下文感知的决策：仅使用机载深度传感和离散的2D速度命令，机器人选择并执行是否跨越、爬上、跳跃或滚下不同几何形状和高度的障碍物。我们通过对Unitree G1人形机器人进行广泛的真实世界实验来验证我们的框架，展示了高度动态的跑酷技能，例如攀爬高达1.25米（96%机器人高度）的障碍物，以及具有实时障碍物扰动闭环适应的长视野多障碍物穿越。
摘要 ：While recent advances in humanoid locomotion have achieved stable walking on varied terrains, capturing the agility and adaptivity of highly dynamic human motions remains an open challenge. In particular, agile parkour in complex environments demands not only low-level robustness, but also human-like motion expressiveness, long-horizon skill composition, and perception-driven decision-making. In this paper, we present Perceptive Humanoid Parkour (PHP), a modular framework that enables humanoid robots to autonomously perform long-horizon, vision-based parkour across challenging obstacle courses. Our approach first leverages motion matching, formulated as nearest-neighbor search in a feature space, to compose retargeted atomic human skills into long-horizon kinematic trajectories. This framework enables the flexible composition and smooth transition of complex skill chains while preserving the elegance and fluidity of dynamic human motions. Next, we train motion-tracking reinforcement learning (RL) expert policies for these composed motions, and distill them into a single depth-based, multi-skill student policy, using a combination of DAgger and RL. Crucially, the combination of perception and skill composition enables autonomous, context-aware decision-making: using only onboard depth sensing and a discrete 2D velocity command, the robot selects and executes whether to step over, climb onto, vault or roll off obstacles of varying geometries and heights. We validate our framework with extensive real-world experiments on a Unitree G1 humanoid robot, demonstrating highly dynamic parkour skills such as climbing tall obstacles up to 1.25m (96% robot height), as well as long-horizon multi-obstacle traversal with closed-loop adaptation to real-time obstacle perturbations.
【4】The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety
标题：对齐的几何形状崩溃：当微调破坏安全时
链接：https://arxiv.org/abs/2602.15799
备注：27 pages, 4 figures
摘要：在良性任务上微调对齐的语言模型会不可预测地降低安全护栏，即使训练数据不包含有害内容，开发人员也没有敌对意图。我们表明，普遍的解释，微调更新应该是正交的高维参数空间中的安全关键方向，提供虚假的保证：我们表明这种正交性是结构不稳定和崩溃的动态梯度下降。然后，我们通过一种新的几何分析来解决这个问题，证明对齐集中在具有尖锐曲率的低维子空间中，创建了一个一阶方法无法检测或防御的脆弱结构。虽然初始微调更新确实可以避免这些子空间，但微调损失的曲率会产生二阶加速度，系统地将轨迹转向到对干扰敏感的区域。我们通过对齐不稳定性条件，三个几何属性，当共同满足，导致安全降级，正式这种机制。我们的主要结果建立了一个四次标度律：对准损失随着训练时间的四次方增长，由对准几何形状的锐度和微调任务与安全关键参数之间的曲率耦合强度决定。这些结果暴露了当前安全范式中的结构性盲点。安全微调的主要方法只处理基本动态问题的初始快照。对齐脆弱性不是一个需要修补的bug;它是曲线流形上梯度下降的内在几何性质。我们的研究结果推动了曲率感知方法的发展，我们希望能够进一步实现对准安全分析从反应性红队到开放权重模型部署的预测诊断的转变。
摘要：Fine-tuning aligned language models on benign tasks unpredictably degrades safety guardrails, even when training data contains no harmful content and developers have no adversarial intent. We show that the prevailing explanation, that fine-tuning updates should be orthogonal to safety-critical directions in high-dimensional parameter space, offers false reassurance: we show this orthogonality is structurally unstable and collapses under the dynamics of gradient descent. We then resolve this through a novel geometric analysis, proving that alignment concentrates in low-dimensional subspaces with sharp curvature, creating a brittle structure that first-order methods cannot detect or defend. While initial fine-tuning updates may indeed avoid these subspaces, the curvature of the fine-tuning loss generates second-order acceleration that systematically steers trajectories into alignment-sensitive regions. We formalize this mechanism through the Alignment Instability Condition, three geometric properties that, when jointly satisfied, lead to safety degradation. Our main result establishes a quartic scaling law: alignment loss grows with the fourth power of training time, governed by the sharpness of alignment geometry and the strength of curvature coupling between the fine-tuning task and safety-critical parameters. These results expose a structural blind spot in the current safety paradigm. The dominant approaches to safe fine-tuning address only the initial snapshot of a fundamentally dynamic problem. Alignment fragility is not a bug to be patched; it is an intrinsic geometric property of gradient descent on curved manifolds. Our results motivate the development of curvature-aware methods, and we hope will further enable a shift in alignment safety analysis from reactive red-teaming to predictive diagnostics for open-weight model deployment.
【5】GLM-5: from Vibe Coding to Agentic Engineering
标题：GLM-5：从Vibe编码到统计工程
链接：https://arxiv.org/abs/2602.15763
摘要：我们提出了GLM-5，下一代基础模型，旨在将vibe编码的范式过渡到代理工程。基于其前身的代理，推理和编码（ARC）功能，GLM-5采用DSA来显着降低训练和推理成本，同时保持长期上下文保真度。为了推进模型对齐和自治，我们实现了一个新的异步强化学习基础设施，通过将生成与训练解耦，大大提高了训练后的效率。此外，我们提出了新的异步代理RL算法，进一步提高RL质量，使模型能够更有效地从复杂的，长期的相互作用中学习。通过这些创新，GLM-5在主要开放基准测试中实现了最先进的性能。最重要的是，GLM-5在现实世界的编码任务中展示了前所未有的能力，在处理端到端软件工程挑战方面超越了以前的基线。代码、模型和更多信息可在https://github.com/zai-org/GLM-5上获得。
摘要：We present GLM-5, a next-generation foundation model designed to transition the paradigm of vibe coding to agentic engineering. Building upon the agentic, reasoning, and coding (ARC) capabilities of its predecessor, GLM-5 adopts DSA to significantly reduce training and inference costs while maintaining long-context fidelity. To advance model alignment and autonomy, we implement a new asynchronous reinforcement learning infrastructure that drastically improves post-training efficiency by decoupling generation from training. Furthermore, we propose novel asynchronous agent RL algorithms that further improve RL quality, enabling the model to learn from complex, long-horizon interactions more effectively. Through these innovations, GLM-5 achieves state-of-the-art performance on major open benchmarks. Most critically, GLM-5 demonstrates unprecedented capability in real-world coding tasks, surpassing previous baselines in handling end-to-end software engineering challenges. Code, models, and more information are available at https://github.com/zai-org/GLM-5.
【6】Spanning the Visual Analogy Space with a Weight Basis of LoRAs
标题：用LoRA的权重基跨越视觉类比空间
链接：https://arxiv.org/abs/2602.15727
备注：Code and data are in https://research.nvidia.com/labs/par/lorweb
摘要 ：视觉类比学习通过演示而不是文本描述来实现图像操作，允许用户指定难以用语言表达的复杂转换。给定一个三元组$\{\mathbf{a}$，$\mathbf{a}'$，$\mathbf{b}\}$，目标是生成$\mathbf{b}'$，使得$\mathbf {a}：\mathbf{a}'：：\mathbf {b}：\mathbf{b}'$。最近的方法使用单个低秩自适应（LoRA）模块来适应文本到图像模型，但它们面临着一个根本的限制：试图在固定的自适应模块中捕获视觉变换的不同空间，这限制了泛化能力。最近的工作表明，限制域中的LoRA跨越有意义的，可插值的语义空间的启发，我们提出了LoRWeB，一种新的方法，专门为每个类比任务的模型在推理时间通过动态组合学习的变换原语，非正式地，选择一个点在“空间的LoRA”。我们介绍了两个关键组件：（1）LoRA模块的可学习基础，以跨越不同视觉变换的空间，以及（2）基于输入模拟对动态选择和加权这些基础LoRA的轻量级编码器。综合评估表明，我们的方法实现了最先进的性能，并显着提高了泛化到看不见的视觉变换。我们的研究结果表明，LoRA基础分解是一个有前途的方向灵活的视觉操作。代码和数据在https://research.nvidia.com/labs/par/lorweb
摘要：Visual analogy learning enables image manipulation through demonstration rather than textual description, allowing users to specify complex transformations difficult to articulate in words. Given a triplet $\{\mathbf{a}$, $\mathbf{a}'$, $\mathbf{b}\}$, the goal is to generate $\mathbf{b}'$ such that $\mathbf{a} : \mathbf{a}' :: \mathbf{b} : \mathbf{b}'$. Recent methods adapt text-to-image models to this task using a single Low-Rank Adaptation (LoRA) module, but they face a fundamental limitation: attempting to capture the diverse space of visual transformations within a fixed adaptation module constrains generalization capabilities. Inspired by recent work showing that LoRAs in constrained domains span meaningful, interpolatable semantic spaces, we propose LoRWeB, a novel approach that specializes the model for each analogy task at inference time through dynamic composition of learned transformation primitives, informally, choosing a point in a "space of LoRAs". We introduce two key components: (1) a learnable basis of LoRA modules, to span the space of different visual transformations, and (2) a lightweight encoder that dynamically selects and weighs these basis LoRAs based on the input analogy pair. Comprehensive evaluations demonstrate our approach achieves state-of-the-art performance and significantly improves generalization to unseen visual transformations. Our findings suggest that LoRA basis decompositions are a promising direction for flexible visual manipulation. Code and data are in https://research.nvidia.com/labs/par/lorweb
【7】Proactive Conversational Assistant for a Procedural Manual Task based on Audio and IMU
标题：基于音频和IMU的程序手动任务的主动对话助理
链接：https://arxiv.org/abs/2602.15707
备注：3 figures
摘要：用于程序任务的实时会话助理通常依赖于视频输入，这可能在计算上是昂贵的并且损害用户隐私。我们首次提出了一种实时对话助理，仅使用轻量级的隐私保护模式（例如来自用户可穿戴设备的音频和IMU输入）来理解上下文，为程序任务提供全面的指导。该助手主动地向执行家具组装任务的用户传达逐步指令，并回答用户的问题。我们构建了一个包含对话的数据集，其中助理指导用户执行任务。在观察到一个现成的语言模型是一个非常健谈的助手，我们设计了一种新的用户Whim不可知（UWA）LoRA微调方法，提高了模型的能力，抑制信息量较少的对话，同时保持其倾向于沟通重要的指令。这导致F分数改善>30%。通过消除在提示中提供上下文示例的需要，对模型进行微调还可以实现16倍的加速。我们进一步描述了如何在边缘设备上实现这样的助手，而不依赖于云。
摘要：Real-time conversational assistants for procedural tasks often depend on video input, which can be computationally expensive and compromise user privacy. For the first time, we propose a real-time conversational assistant that provides comprehensive guidance for a procedural task using only lightweight privacy-preserving modalities such as audio and IMU inputs from a user's wearable device to understand the context. This assistant proactively communicates step-by-step instructions to a user performing a furniture assembly task, and answers user questions. We construct a dataset containing conversations where the assistant guides the user in performing the task. On observing that an off-the-shelf language model is a very talkative assistant, we design a novel User Whim Agnostic (UWA) LoRA finetuning method which improves the model's ability to suppress less informative dialogues, while maintaining its tendency to communicate important instructions. This leads to >30% improvement in the F-score. Finetuning the model also results in a 16x speedup by eliminating the need to provide in-context examples in the prompt. We further describe how such an assistant is implemented on edge devices with no dependence on the cloud.
【8】Relative Geometry of Neural Forecasters: Linking Accuracy and Alignment in Learned Latent Geometry
标题：神经预测器的相对几何：在习得的潜在几何中将准确性和对齐联系起来
链接：https://arxiv.org/abs/2602.15676
备注：Accepted to Transactions on Machine Learning Research (TMLR)
摘要：神经网络可以准确地预测复杂的动力系统，但它们如何在内部表示潜在的几何结构仍然知之甚少。我们通过代表性对齐的镜头研究神经预测器，引入基于锚的几何不可知的相对嵌入，消除潜在空间中的旋转和缩放模糊性。将此框架应用于七个规范动力系统-从周期到混沌-我们揭示了可再现的家族级结构：多层感知器与其他MLP对齐，递归网络与RNN对齐，而Transformers和回声状态网络实现了强预测，尽管对齐较弱。对齐通常与预测准确性相关，但高准确性可以与低对齐共存。因此，相对几何提供了一个简单的，可重复的基础，比较模型家庭如何内化和代表动态结构。
摘要：Neural networks can accurately forecast complex dynamical systems, yet how they internally represent underlying latent geometry remains poorly understood. We study neural forecasters through the lens of representational alignment, introducing anchor-based, geometry-agnostic relative embeddings that remove rotational and scaling ambiguities in latent spaces. Applying this framework across seven canonical dynamical systems - ranging from periodic to chaotic - we reveal reproducible family-level structure: multilayer perceptrons align with other MLPs, recurrent networks with RNNs, while transformers and echo-state networks achieve strong forecasts despite weaker alignment. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative geometry thus provides a simple, reproducible foundation for comparing how model families internalize and represent dynamical structure.
【9】The Stationarity Bias: Stratified Stress-Testing for Time-Series Imputation in Regulated Dynamical Systems
标题：平稳性偏差：调节动态系统中时间序列插补的分层压力测试
链接：https://arxiv.org/abs/2602.15637
摘要：时间序列插补基准采用均匀随机掩蔽和形状不可知的指标（MSE，RMSE），隐含加权政权患病率的评估。在具有主导吸引子的系统中-稳态生理学，名义上的工业操作，稳定的网络流量-这会产生系统的平稳性偏差：简单的方法似乎更优越，因为基准主要是简单的样本，低熵状态，它们微不足道。我们正式这种偏见，并提出了一个分层的压力测试，分区评估到静态和瞬态制度。使用连续葡萄糖监测（CGM）作为测试平台-选择其严格的地面实况强制功能（膳食，胰岛素），使精确的政权识别-我们建立了三个具有广泛意义的发现：（i）~平稳效率：线性插值实现了最先进的重建在稳定的时间间隔，确认复杂的架构是计算浪费在低熵制度。(ii)~瞬时保真度：在关键瞬变（餐后峰值，低血糖事件），线性方法表现出急剧下降的形态保真度（DTW），不成比例的RMSE -一种现象，我们称之为\emma {RMSE幻影}，其中低逐点误差掩盖了信号形状的破坏。(iii)·机制条件模型选择：深度学习模型在瞬态过程中保持逐点准确性和形态完整性，这使得它们对于安全关键的下游任务至关重要。我们进一步从临床试验中推导出经验缺失分布，并将其应用于完整的训练数据，防止模型利用不切实际的干净观察结果，并鼓励在真实世界缺失下的鲁棒性。这个框架一般适用于任何常规平稳性占主导地位的关键瞬态调节系统。
摘要 ：Time-series imputation benchmarks employ uniform random masking and shape-agnostic metrics (MSE, RMSE), implicitly weighting evaluation by regime prevalence. In systems with a dominant attractor -- homeostatic physiology, nominal industrial operation, stable network traffic -- this creates a systematic \emph{Stationarity Bias}: simple methods appear superior because the benchmark predominantly samples the easy, low-entropy regime where they trivially succeed. We formalize this bias and propose a \emph{Stratified Stress-Test} that partitions evaluation into Stationary and Transient regimes. Using Continuous Glucose Monitoring (CGM) as a testbed -- chosen for its rigorous ground-truth forcing functions (meals, insulin) that enable precise regime identification -- we establish three findings with broad implications:(i)~Stationary Efficiency: Linear interpolation achieves state-of-the-art reconstruction during stable intervals, confirming that complex architectures are computationally wasteful in low-entropy regimes.(ii)~Transient Fidelity: During critical transients (post-prandial peaks, hypoglycemic events), linear methods exhibit drastically degraded morphological fidelity (DTW), disproportionate to their RMSE -- a phenomenon we term the \emph{RMSE Mirage}, where low pointwise error masks the destruction of signal shape.(iii)~Regime-Conditional Model Selection: Deep learning models preserve both pointwise accuracy and morphological integrity during transients, making them essential for safety-critical downstream tasks. We further derive empirical missingness distributions from clinical trials and impose them on complete training data, preventing models from exploiting unrealistically clean observations and encouraging robustness under real-world missingness. This framework generalizes to any regulated system where routine stationarity dominates critical transients.
【10】Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
标题：超越ReLU：分歧、过度平滑和布局先验
链接：https://arxiv.org/abs/2602.15634
摘要：图神经网络（GNN）通过基于迭代网络的消息传递来学习节点表示。虽然功能强大，但深度GNN会受到过度平滑的影响，其中节点特征会收敛到同质的，无信息的状态。我们重新架构这个问题的代表性崩溃从\xBF {分歧理论}的角度来看，特征oversmoothing收敛到一个稳定的齐次不动点。我们的主要贡献是理论上发现，这种不希望的稳定性可以通过替换标准的单调激活来打破（例如，ReLU）与一类函数。使用Lyapunov-Schmidt还原，我们分析证明，这种替代诱导的分歧，不稳定的均匀状态，并创建一个新的对稳定的，非均匀的模式，可证明抵抗过平滑。我们的理论预测了一个精确的，非平凡的标度律的振幅，这些紧急模式，我们在实验中定量验证。最后，我们证明了我们的理论的实际效用，推导出一个封闭的形式，分叉意识的初始化，并显示其实用程序在真正的基准实验。
摘要：Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a \emph{bifurcation theory} perspective, characterizing oversmoothing as convergence to a stable ``homogeneous fixed point.'' Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous \emph{patterns} that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-aware initialization and showing its utility in real benchmark experiments.
【11】Certified Per-Instance Unlearning Using Individual Sensitivity Bounds
标题：使用个人敏感度界限认证的每实例取消学习
链接：https://arxiv.org/abs/2602.15602
摘要：经过认证的机器非学习可以通过噪声注入来实现，从而实现差分隐私保证，其中噪声被校准到最坏情况的灵敏度。这种保守的校准通常会导致性能下降，限制了实用性。在这项工作中，我们研究了一种基于自适应每实例噪声校准的替代方法，该方法针对每个数据点对学习解决方案的单独贡献而定制。这提出了以下挑战：当机制取决于要移除的特定点时，如何建立正式的遗忘保证？为了在有噪声的梯度动态中定义单个数据点的敏感性，我们考虑使用每个实例的差分隐私。对于通过Langevin动力学训练的岭回归，我们推导出高概率的每实例灵敏度界限，从而产生经过认证的非学习，并且噪声注入大大减少。我们通过线性环境中的实验证实了我们的理论发现，并提供了该方法在深度学习环境中的相关性的进一步经验证据。
摘要：Certified machine unlearning can be achieved via noise injection leading to differential privacy guarantees, where noise is calibrated to worst-case sensitivity. Such conservative calibration often results in performance degradation, limiting practical applicability. In this work, we investigate an alternative approach based on adaptive per-instance noise calibration tailored to the individual contribution of each data point to the learned solution. This raises the following challenge: how can one establish formal unlearning guarantees when the mechanism depends on the specific point to be removed? To define individual data point sensitivities in noisy gradient dynamics, we consider the use of per-instance differential privacy. For ridge regression trained via Langevin dynamics, we derive high-probability per-instance sensitivity bounds, yielding certified unlearning with substantially less noise injection. We corroborate our theoretical findings through experiments in linear settings and provide further empirical evidence on the relevance of the approach in deep learning settings.
【12】Multi-Objective Coverage via Constraint Active Search
标题：通过约束主动搜索实现多目标覆盖
链接：https://arxiv.org/abs/2602.15595
摘要：在本文中，我们制定了新的多目标覆盖（MOC）问题，我们的目标是确定一个小的代表性样本集，其预测结果广泛覆盖可行的多目标空间。这个问题在许多关键的现实应用中非常重要，例如，药物发现和材料设计，因为这一代表性集合的评估速度比整个可行集合快得多，从而大大加快了科学发现过程。现有的作品不能直接应用，因为它们要么集中在样本空间覆盖或多目标优化，目标帕累托前沿。然而，化学成分不同的样品往往会产生相同的目标配置文件，和安全约束条件通常定义的目标。为了解决这个MOC问题，我们提出了一种新的搜索算法，MOC-CAS，它采用了一个基于置信上限的采集函数来选择乐观的样本高斯过程后验预测的指导下。为了实现有效的优化，我们开发了一个平滑松弛的硬可行性测试，并得出一个近似的优化。与竞争性基线相比，我们证明了我们的MOC-CAS在SARS-CoV-2和癌症的大规模蛋白质靶向数据集上凭经验实现了卓越的性能，每个数据集都基于SMILES特征的五个目标进行评估。
摘要：In this paper, we formulate the new multi-objective coverage (MOC) problem where our goal is to identify a small set of representative samples whose predicted outcomes broadly cover the feasible multi-objective space. This problem is of great importance in many critical real-world applications, e.g., drug discovery and materials design, as this representative set can be evaluated much faster than the whole feasible set, thus significantly accelerating the scientific discovery process. Existing works cannot be directly applied as they either focus on sample space coverage or multi-objective optimization that targets the Pareto front. However, chemically diverse samples often yield identical objective profiles, and safety constraints are usually defined on the objectives. To solve this MOC problem, we propose a novel search algorithm, MOC-CAS, which employs an upper confidence bound-based acquisition function to select optimistic samples guided by Gaussian process posterior predictions. For enabling efficient optimization, we develop a smoothed relaxation of the hard feasibility test and derive an approximate optimizer. Compared to the competitive baselines, we show that our MOC-CAS empirically achieves superior performances across large-scale protein-target datasets for SARS-CoV-2 and cancer, each assessed on five objectives derived from SMILES-based features.
【13】1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization
标题：1-Bit Wonder：通过K均值量化提高低位机制下的QAT性能
链接：https://arxiv.org/abs/2602.15563
备注：Preprint. Under Review. 23 pages, 9 figures
摘要 ：量化感知训练（QAT）是一种有效的方法，可以大大减少LLM的内存占用，同时将性能下降保持在可接受的水平。然而，量化格式和比特宽度的最佳选择在实践中提出了一个挑战。量化的完整设计空间在QAT的上下文中没有被充分探索，并且量化和下游性能之间的精确权衡理解得很少，因为比较通常仅依赖于基于困惑的评估。在这项工作中，我们解决这些缺点与QAT在低位政权的实证研究。我们表明，基于k-均值的权重量化优于整数格式，可以有效地在标准硬件上实现。此外，我们发现，在一个固定的推理内存预算下，最好的性能生成下游任务实现与1 $位量化的权重。
摘要：Quantization-aware training (QAT) is an effective method to drastically reduce the memory footprint of LLMs while keeping performance degradation at an acceptable level. However, the optimal choice of quantization format and bit-width presents a challenge in practice. The full design space of quantization is not fully explored in the context of QAT, and the precise trade-off between quantization and downstream performance is poorly understood, as comparisons often rely solely on perplexity-based evaluations. In this work, we address these shortcomings with an empirical study of QAT in the low-bit regime. We show that k-means based weight quantization outperforms integer formats and can be implemented efficiently on standard hardware. Furthermore, we find that, under a fixed inference memory budget, the best performance on generative downstream tasks is achieved with $1$-bit quantized weights.
【14】CEPAE: Conditional Entropy-Penalized Autoencoders for Time Series Counterfactuals
标题：CEPCE：时间序列反事实的条件性熵惩罚自动编码器
链接：https://arxiv.org/abs/2602.15546
摘要：准确地对时间序列进行反事实推理的能力对于金融、医疗保健和营销等领域的决策至关重要，因为它使我们能够了解事件或治疗对结果的影响。在本文中，我们介绍了一种新的反事实推理方法，适合于受市场事件影响的时间序列数据，这是由工业应用的动机。利用外展-动作-预测过程和结构因果模型框架，我们首先采用基于变分自编码器和对抗自编码器的方法，这两种方法以前都在反事实文献中使用过，但不是在时间序列设置中。然后，我们提出了条件熵惩罚自动编码器（CEPAE），一种新的基于自动编码器的反事实推理方法，它采用了熵惩罚损失的潜在空间，以鼓励解开的数据表示。我们在合成，半合成和真实世界的数据集上从理论和实验上验证了我们的方法，表明CEPAE在评估指标中通常优于其他方法。
摘要：The ability to accurately perform counterfactual inference on time series is crucial for decision-making in fields like finance, healthcare, and marketing, as it allows us to understand the impact of events or treatments on outcomes over time. In this paper, we introduce a new counterfactual inference approach tailored to time series data impacted by market events, which is motivated by an industrial application. Utilizing the abduction-action-prediction procedure and the Structural Causal Model framework, we first adapt methods based on variational autoencoders and adversarial autoencoders, both previously used in counterfactual literature although not in time series settings. Then, we present the Conditional Entropy-Penalized Autoencoder (CEPAE), a novel autoencoder-based approach for counterfactual inference, which employs an entropy penalization loss over the latent space to encourage disentangled data representations. We validate our approach both theoretically and experimentally on synthetic, semi-synthetic, and real-world datasets, showing that CEPAE generally outperforms the other approaches in the evaluated metrics.
【15】The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
标题：混淆地图集：使用欺骗探针绘制RL VR中诚实出现的地方
链接：https://arxiv.org/abs/2602.15515
备注：25 pages, 12 figures
摘要：针对白盒欺骗检测器的训练已经被提出作为使AI系统诚实的一种方法。然而，这样的训练风险模型学习混淆他们的欺骗以逃避检测器。之前的工作只研究了人工环境中的混淆，其中模型直接奖励有害输出。我们构建了一个现实的编码环境，奖励黑客通过硬编码测试用例自然发生，并表明混淆出现在这种设置。我们介绍了一个分类的可能的结果时，训练对欺骗检测器。该模型要么保持诚实，要么通过两种可能的混淆策略变得具有欺骗性。(i)混淆激活：模型输出欺骗性文本，同时修改其内部表示以不再触发检测器。(ii)混淆策略：模型输出欺骗性文本，通常通过包含奖励黑客的理由来逃避检测器。从经验上讲，混淆激活是由RL期间的表示漂移引起的，有或没有检测器惩罚。探测惩罚只激励混淆的政策，我们理论上表明，这是预期的政策梯度方法。足够高的KL正则化和检测器惩罚可以产生诚实的策略，建立白盒欺骗检测器作为易于奖励黑客的任务的可行训练信号。
摘要：Training against white-box deception detectors has been proposed as a way to make AI systems honest. However, such training risks models learning to obfuscate their deception to evade the detector. Prior work has studied obfuscation only in artificial settings where models were directly rewarded for harmful output. We construct a realistic coding environment where reward hacking via hardcoding test cases naturally occurs, and show that obfuscation emerges in this setting. We introduce a taxonomy of possible outcomes when training against a deception detector. The model either remains honest, or becomes deceptive via two possible obfuscation strategies. (i) Obfuscated activations: the model outputs deceptive text while modifying its internal representations to no longer trigger the detector. (ii) Obfuscated policy: the model outputs deceptive text that evades the detector, typically by including a justification for the reward hack. Empirically, obfuscated activations arise from representation drift during RL, with or without a detector penalty. The probe penalty only incentivizes obfuscated policies; we theoretically show this is expected for policy gradient methods. Sufficiently high KL regularization and detector penalty can yield honest policies, establishing white-box deception detectors as viable training signals for tasks prone to reward hacking.
【16】On the Geometric Coherence of Global Aggregation in Federated GNN
标题：联邦GNN中全球聚集的几何一致性
链接：https://arxiv.org/abs/2602.15510
备注：This is a developing preprint of an 18-page journal manuscript (6 figures), currently being prepared for formal peer-review submission
摘要：联合学习（FL）支持跨多个客户端的分布式训练，而无需集中数据共享，而图神经网络（GNN）通过消息传递对关系数据进行建模。在联邦GNN设置中，客户端图通常表现出异构的结构和传播特性。当标准聚合机制应用于这种异构更新时，全局模型可能在数值上收敛，同时表现出退化的关系行为。虽然GNN参数在数字上表示为向量，但它们对关系转换进行编码，这些关系转换控制着图邻域中信息流的方向、强度和灵敏度。因此，聚合来自不兼容传播机制的更新可能会在此转换空间中引入相消干涉，这会导致全局消息传递的一致性丧失。重要的是，这种退化不一定反映在传统的指标，如损失或accuracy.To解决这个问题，我们提出了GGRS（全球几何参考结构），一个服务器端的框架，规范客户端更新之前，聚合的基础上几何容许性标准。GGRS保持关系变换的方向一致性，并保持可容许传播子空间的多样性。它还稳定了对邻域交互的敏感性，而无需访问客户端数据或图形拓扑。在异构GNN原生Amazon Co-purchase数据集上的实验表明，GGRS通过强调联邦图学习中几何感知调节的必要性，在训练轮中保持了全局消息传递的一致性。
摘要 ：Federated Learning (FL) enables distributed training across multiple clients without centralized data sharing, while Graph Neural Networks (GNNs) model relational data through message passing. In federated GNN settings, client graphs often exhibit heterogeneous structural and propagation characteristics. When standard aggregation mechanisms are applied to such heterogeneous updates, the global model may converge numerically while exhibiting degraded relational behavior.Our work identifies a geometric failure mode of global aggregation in Cross- Domain Federated GNNs. Although GNN parameters are numerically represented as vectors, they encode relational transformations that govern the direction, strength, and sensitivity of information flow across graph neighborhoods. Aggregating updates originating from incompatible propagation regimes can therefore introduce destructive interference in this transformation space.This leads to loss of coherence in global message passing. Importantly, this degradation is not necessarily reflected in conventional metrics such as loss or accuracy.To address this issue, we propose GGRS (Global Geometric Reference Structure), a server-side framework that regulates client updates prior to aggregation based on geometric admissibility criteria. GGRS preserves directional consistency of relational transformations as well as maintains diversity of admissible propagation subspaces. It also stabilizes sensitivity to neighborhood interactions, without accessing client data or graph topology. Experiments on heterogeneous GNN-native, Amazon Co-purchase datasets demonstrate that GGRS preserves global message-passing coherence across training rounds by highlighting the necessity of geometry-aware regulation in federated graph learning.
【17】POP: Prior-fitted Optimizer Policies
标题：POP：优先适合的优化器政策
链接：https://arxiv.org/abs/2602.15473
备注：Under Review
摘要：优化是指寻找目标函数的极值的任务。经典的基于梯度的优化器对超参数的选择非常敏感。在高度非凸设置中，它们的性能依赖于精心调整的学习率、动量和梯度累积。为了解决这些限制，我们引入了POP（先验拟合优化策略），这是一种元学习优化器，可以根据优化轨迹中提供的上下文信息预测坐标步长。我们的模型是在数百万个合成优化问题上学习的，这些问题是从一个新的先验跨越凸和非凸目标中采样的。我们在一个已建立的基准上评估POP，该基准包括47个不同复杂度的优化函数，其中它始终优于一阶基于梯度的方法，非凸优化方法（例如，进化策略），贝叶斯优化，以及最近在匹配预算约束下的元学习竞争对手。我们的评估显示了强大的泛化能力，而无需特定于任务的调整。
摘要：Optimization refers to the task of finding extrema of an objective function. Classical gradient-based optimizers are highly sensitive to hyperparameter choices. In highly non-convex settings their performance relies on carefully tuned learning rates, momentum, and gradient accumulation. To address these limitations, we introduce POP (Prior-fitted Optimizer Policies), a meta-learned optimizer that predicts coordinate-wise step sizes conditioned on the contextual information provided in the optimization trajectory. Our model is learned on millions of synthetic optimization problems sampled from a novel prior spanning both convex and non-convex objectives. We evaluate POP on an established benchmark including 47 optimization functions of various complexity, where it consistently outperforms first-order gradient-based methods, non-convex optimization approaches (e.g., evolutionary strategies), Bayesian optimization, and a recent meta-learned competitor under matched budget constraints. Our evaluation demonstrates strong generalization capabilities without task-specific tuning.
【18】Benchmarking IoT Time-Series AD with Event-Level Augmentations
标题：通过事件级增强对物联网时间序列AD进行基准测试
链接：https://arxiv.org/abs/2602.15457
备注：https://underline.io/events/521/sessions/21822/lecture/143905-benchmarking-iot-time-series-ad-with-event-level-augmentations?tab=poster
摘要：安全关键型物联网时间序列的异常检测（AD）应该在事件级别进行判断：在现实扰动下的可靠性和早期性。然而，许多研究仍然强调在策划的基础数据集上的点级结果，限制了实践中模型选择的价值。我们介绍了一个评估协议，具有统一的事件级增强，模拟现实世界的问题：校准传感器丢失，线性和对数漂移，加性噪声和窗口移位。我们还执行传感器级探测，通过掩蔽为丢失归零与每通道的影响估计，以支持根本原因分析。我们评估了14个代表性的模型上的5个公共异常数据集（SWaT，WADI，SMD，SKAB，TEP）和两个工业数据集（汽轮机，核电汽轮发电机）使用统一的分裂和事件聚合。没有万能的赢家：图结构模型在dropout和long事件下传输最好（例如，在加性噪声下的SWaT上，对于图形自动编码器，F1下降0.804->0.677，对于图形注意力变体，F1下降0.759->0.680，对于混合图形注意力模型，F1下降0.762->0.756）;密度/流量模型在干净的静止植物上工作良好，但对于单调漂移可能很脆弱;频谱CNN在周期性强时领先;重构自动编码器在基本传感器检查后变得有竞争力;预测/混合动态在故障打破时间依赖性但保持窗口敏感性时有所帮助。该协议还为设计选择提供了信息：在对数漂移下的SWaT上，用高斯密度替换归一化流将高应力F1从~0.75降低到~0.57，并且固定学习的DAG提供了小的干净设置增益（~0.5-1.0点），但将漂移灵敏度提高了~ 8倍。
摘要：Anomaly detection (AD) for safety-critical IoT time series should be judged at the event level: reliability and earliness under realistic perturbations. Yet many studies still emphasize point-level results on curated base datasets, limiting value for model selection in practice. We introduce an evaluation protocol with unified event-level augmentations that simulate real-world issues: calibrated sensor dropout, linear and log drift, additive noise, and window shifts. We also perform sensor-level probing via mask-as-missing zeroing with per-channel influence estimation to support root-cause analysis. We evaluate 14 representative models on five public anomaly datasets (SWaT, WADI, SMD, SKAB, TEP) and two industrial datasets (steam turbine, nuclear turbogenerator) using unified splits and event aggregation. There is no universal winner: graph-structured models transfer best under dropout and long events (e.g., on SWaT under additive noise F1 drops 0.804->0.677 for a graph autoencoder, 0.759->0.680 for a graph-attention variant, and 0.762->0.756 for a hybrid graph attention model); density/flow models work well on clean stationary plants but can be fragile to monotone drift; spectral CNNs lead when periodicity is strong; reconstruction autoencoders become competitive after basic sensor vetting; predictive/hybrid dynamics help when faults break temporal dependencies but remain window-sensitive. The protocol also informs design choices: on SWaT under log drift, replacing normalizing flows with Gaussian density reduces high-stress F1 from ~0.75 to ~0.57, and fixing a learned DAG gives a small clean-set gain (~0.5-1.0 points) but increases drift sensitivity by ~8x.
【19】Logit Distance Bounds Representational Similarity
标题：Logit距离限制代表相似性
链接：https://arxiv.org/abs/2602.15438
摘要：对于包括自回归语言模型在内的广泛的判别模型家族，可识别性结果意味着，如果两个模型诱导相同的条件分布，则它们的内部表示一致达到可逆线性变换。我们要问，当分布接近而不是相等时，类似的结论是否近似成立。基于Nielsen et al.（2025）的观察，KL分歧的接近程度不一定意味着高线性表示相似性，我们研究了基于logit差异的分布距离，并表明该距离的接近程度确实产生线性相似性保证。具体来说，我们定义了一个代表性的相异度度量模型的可识别类的基础上，并证明了它是有界的logit距离。我们进一步表明，当模型的概率有界远离零，KL发散上界的logit距离，但由此产生的界未能提供非平凡的控制在实践中。因此，基于KL的蒸馏可以匹配教师的预测，同时不能保持线性表征特性，例如人类可解释概念的线性探测可恢复性。在合成和图像数据集上的蒸馏实验中，logit距离蒸馏使学生具有更高的线性表示相似性和更好地保留教师的线性可恢复概念。
摘要：For a broad family of discriminative models that includes autoregressive language models, identifiability results imply that if two models induce the same conditional distributions, then their internal representations agree up to an invertible linear transformation. We ask whether an analogous conclusion holds approximately when the distributions are close instead of equal. Building on the observation of Nielsen et al. (2025) that closeness in KL divergence need not imply high linear representational similarity, we study a distributional distance based on logit differences and show that closeness in this distance does yield linear similarity guarantees. Specifically, we define a representational dissimilarity measure based on the models' identifiability class and prove that it is bounded by the logit distance. We further show that, when model probabilities are bounded away from zero, KL divergence upper-bounds logit distance; yet the resulting bound fails to provide nontrivial control in practice. As a consequence, KL-based distillation can match a teacher's predictions while failing to preserve linear representational properties, such as linear-probe recoverability of human-interpretable concepts. In distillation experiments on synthetic and image datasets, logit-distance distillation yields students with higher linear representational similarity and better preservation of the teacher's linearly recoverable concepts.
【20】GaiaFlow: Semantic-Guided Diffusion Tuning for Carbon-Frugal Search
标题：GaiaFlow：面向低碳经济搜索的语义引导扩散调整
链接
：https://arxiv.org/abs/2602.15423
备注：19 pages, 7 figures
摘要：随着复杂神经体系结构的新兴电力需求不断升级，信息检索界已经认识到生态可持续性是一个关键的优先事项，需要在模型设计中进行根本的范式转变。虽然当代神经排序器已经达到了前所未有的准确性，但与其计算强度相关的大量环境外部性在大规模部署中往往被忽视。我们提出了GaiaFlow，一个创新的框架，旨在通过操作语义引导的扩散调整来促进碳节约型搜索。我们的方法协调检索引导的Langevin动态和硬件独立的性能建模策略的收敛，以优化搜索精度和环境保护之间的权衡。通过结合自适应早期退出协议和精确感知的量化推理，所提出的架构显着减轻了操作碳足迹，同时在异构计算基础设施中保持稳健的检索质量。广泛的实验评估表明，GaiaFlow在有效性和能源效率之间实现了卓越的平衡，为下一代神经搜索系统提供了可扩展和可持续的途径。
摘要：As the burgeoning power requirements of sophisticated neural architectures escalate, the information retrieval community has recognized ecological sustainability as a pivotal priority that necessitates a fundamental paradigm shift in model design. While contemporary neural rankers have attained unprecedented accuracy, the substantial environmental externalities associated with their computational intensity often remain overlooked in large-scale deployments. We present GaiaFlow, an innovative framework engineered to facilitate carbon-frugal search by operationalizing semantic-guided diffusion tuning. Our methodology orchestrates the convergence of retrieval-guided Langevin dynamics and a hardware-independent performance modeling strategy to optimize the trade-off between search precision and environmental preservation. By incorporating adaptive early exit protocols and precision-aware quantized inference, the proposed architecture significantly mitigates operational carbon footprints while maintaining robust retrieval quality across heterogeneous computing infrastructures. Extensive experimental evaluations demonstrate that GaiaFlow achieves a superior equilibrium between effectiveness and energy efficiency, offering a scalable and sustainable pathway for next-generation neural search systems.
【21】Fairness over Equality: Correcting Social Incentives in Asymmetric Sequential Social Dilemmas
标题：公平高于平等：纠正不对称连续社会困境中的社会激励
链接：https://arxiv.org/abs/2602.15407
摘要：序贯社会困境（SSD）提供了一个关键的框架，研究如何合作出现时，个人激励与集体福利冲突。在多智能体强化学习中，这些问题通常通过引入鼓励亲社会或公平行为的内在驱动来解决。然而，大多数现有的方法假设代理人面临相同的激励在困境中，并需要不断访问其他代理人的全球信息来评估公平性。在这项工作中，我们介绍了著名的SSD环境的不对称变体，并研究代理之间的自然差异如何影响合作动态。我们的研究结果表明，现有的公平为基础的方法，努力适应不对称的条件下，强制执行原始的平等，错误地激励叛逃。为了解决这个问题，我们提出了三个修改：（i）重新定义公平的会计代理的奖励范围，（ii）引入基于代理的加权机制，以更好地处理固有的不对称性，和（iii）本地化的社会反馈，使方法有效的部分可观察性，而不需要全球信息共享。实验结果表明，在不对称的情况下，我们的方法相比现有的方法，促进更快的合作政策的出现，而不牺牲可扩展性或实用性。
摘要：Sequential Social Dilemmas (SSDs) provide a key framework for studying how cooperation emerges when individual incentives conflict with collective welfare. In Multi-Agent Reinforcement Learning, these problems are often addressed by incorporating intrinsic drives that encourage prosocial or fair behavior. However, most existing methods assume that agents face identical incentives in the dilemma and require continuous access to global information about other agents to assess fairness. In this work, we introduce asymmetric variants of well-known SSD environments and examine how natural differences between agents influence cooperation dynamics. Our findings reveal that existing fairness-based methods struggle to adapt under asymmetric conditions by enforcing raw equality that wrongfully incentivize defection. To address this, we propose three modifications: (i) redefining fairness by accounting for agents' reward ranges, (ii) introducing an agent-based weighting mechanism to better handle inherent asymmetries, and (iii) localizing social feedback to make the methods effective under partial observability without requiring global information sharing. Experimental results show that in asymmetric scenarios, our method fosters faster emergence of cooperative policies compared to existing approaches, without sacrificing scalability or practicality.
【22】The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems
标题：愿景蠕虫洞：异类多智能体系统中的潜在空间通信
链接：https://arxiv.org/abs/2602.15382
备注：Preprint. Work in progress
摘要：由大型语言模型驱动的多智能体系统（MAS）已经解锁了高级协作推理，但它们仍然受到离散文本通信效率低下的束缚，这带来了显着的运行时开销和信息量化损失。虽然潜在状态转移提供了一种高带宽的替代方案，但现有的方法要么假设同质的发送器-接收器架构，要么依赖于对特定的学习翻译器，限制了具有不相交流形的不同模型家族的可扩展性和模块化。在这项工作中，我们提出了视觉虫洞，一个新的框架，重新设计视觉语言模型（VLM）的视觉界面，使模型不可知，无文本通信。通过引入一个通用的视觉编解码器，我们映射到一个共享的连续的潜在空间的异构推理痕迹，并将它们直接注入到接收器的视觉通路，有效地治疗的视觉编码器作为一个通用的端口代理间心灵感应。我们的框架采用轴辐式拓扑结构，将成对对齐复杂度从O（N^2）降低到O（N），并利用无标签的师生蒸馏目标将高速视觉通道与文本路径的鲁棒推理模式对齐。跨异构模型家族的广泛实验（例如，Qwen-VL，Gemma）证明了Vision Wormhole在受控比较中减少了端到端的挂钟时间，同时保持了与标准的基于文本的MAS相当的推理保真度。代码可在https://github.com/xz-liu/heterogeneous-latent-mas上获得
摘要：Multi-Agent Systems (MAS) powered by Large Language Models have unlocked advanced collaborative reasoning, yet they remain shackled by the inefficiency of discrete text communication, which imposes significant runtime overhead and information quantization loss. While latent state transfer offers a high-bandwidth alternative, existing approaches either assume homogeneous sender-receiver architectures or rely on pair-specific learned translators, limiting scalability and modularity across diverse model families with disjoint manifolds. In this work, we propose the Vision Wormhole, a novel framework that repurposes the visual interface of Vision-Language Models (VLMs) to enable model-agnostic, text-free communication. By introducing a Universal Visual Codec, we map heterogeneous reasoning traces into a shared continuous latent space and inject them directly into the receiver's visual pathway, effectively treating the vision encoder as a universal port for inter-agent telepathy. Our framework adopts a hub-and-spoke topology to reduce pairwise alignment complexity from O(N^2) to O(N) and leverages a label-free, teacher-student distillation objective to align the high-speed visual channel with the robust reasoning patterns of the text pathway. Extensive experiments across heterogeneous model families (e.g., Qwen-VL, Gemma) demonstrate that the Vision Wormhole reduces end-to-end wall-clock time in controlled comparisons while maintaining reasoning fidelity comparable to standard text-based MAS. Code is available at https://github.com/xz-liu/heterogeneous-latent-mas
【23】The Information Geometry of Softmax: Probing and Steering
标题：Softmax的信息几何：探索和引导
链接：https://arxiv.org/abs/2602.15293
备注：Code is available at https://github.com/KihoPark/dual-steering
摘要 ：本文研究人工智能系统如何将语义结构编码到其表示空间的几何结构中。本文的动机观察是，这些表示空间的自然几何形状应该反映模型使用表示来产生行为的方式。我们专注于定义softmax分布的表示的重要特殊情况。在这种情况下，我们认为，自然几何是信息几何。我们的重点是语义编码和线性表征假设的信息几何的作用。作为一个说明性的应用程序，我们开发了“双转向”，一种方法，用于鲁棒性转向表示使用线性探针表现出一个特定的概念。我们证明，双转向最佳修改的目标概念，同时尽量减少变化的目标外的概念。经验上，我们发现，双转向增强了概念操纵的可控性和稳定性。
摘要：This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation of this paper is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. In this case, we argue that the natural geometry is information geometry. Our focus is on the role of information geometry on semantic encoding and the linear representation hypothesis. As an illustrative application, we develop "dual steering", a method for robustly steering representations to exhibit a particular concept using linear probes. We prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. Empirically, we find that dual steering enhances the controllability and stability of concept manipulation.
【24】When Remembering and Planning are Worth it: Navigating under Change
标题：何时值得记住和规划：在变革中航行
链接：https://arxiv.org/abs/2602.15274
摘要：我们探讨了不同类型和使用的记忆如何在不断变化的不确定环境中帮助空间导航。在我们研究的简单觅食任务中，我们的智能体每天都必须找到从家出发的路，穿过障碍物，找到食物。此外，世界是非平稳的：从一天到一天，障碍物和食物的位置可能会发生变化，智能体的感知，如其位置信息是不确定的，非常有限。任何模型的构建（如地图）和使用（如规划）都需要对这些挑战具有鲁棒性，如果任何学习都是有用的，它需要足够快。我们研究了一系列策略，从简单到复杂，以及记忆和学习的各种用途。我们发现，需要一个架构，可以将多种策略来处理（子）任务的不同性质，特别是探索和搜索，当食物的位置是未知的，并规划一个良好的路径记住（可能）的食物位置。一种利用非平稳概率学习技术来不断更新其（情节）记忆，并使用这些记忆建立地图和计划的飞行（不完美的地图，即嘈杂和有限的代理人的经验）可以越来越多地和实质上更有效的比更简单的（最小记忆）智能体，随着任务难度的提高，如到目标的距离，只要不确定性，从本地化和变化，是不是太大。
摘要：We explore how different types and uses of memory can aid spatial navigation in changing uncertain environments. In the simple foraging task we study, every day, our agent has to find its way from its home, through barriers, to food. Moreover, the world is non-stationary: from day to day, the location of the barriers and food may change, and the agent's sensing such as its location information is uncertain and very limited. Any model construction, such as a map, and use, such as planning, needs to be robust against these challenges, and if any learning is to be useful, it needs to be adequately fast. We look at a range of strategies, from simple to sophisticated, with various uses of memory and learning. We find that an architecture that can incorporate multiple strategies is required to handle (sub)tasks of a different nature, in particular for exploration and search, when food location is not known, and for planning a good path to a remembered (likely) food location. An agent that utilizes non-stationary probability learning techniques to keep updating its (episodic) memories and that uses those memories to build maps and plan on the fly (imperfect maps, i.e. noisy and limited to the agent's experience) can be increasingly and substantially more efficient than the simpler (minimal-memory) agents, as the task difficulties such as distance to goal are raised, as long as the uncertainty, from localization and change, is not too large.
【25】Decision Making under Imperfect Recall: Algorithms and Benchmarks
标题：不完美回忆下的决策：算法和基准
链接：https://arxiv.org/abs/2602.15252
备注：39 pages, 71 figures, 4 table
摘要：在博弈论中，回忆决策问题模拟了代理忘记之前持有的信息的情况。他们包括游戏，如“心不在焉的司机”和团队游戏与有限的沟通。在本文中，我们介绍了第一个基准测试套件的召回决策问题。我们的基准测试捕捉了各种问题类型，包括涉及人工智能系统中引发敏感信息的隐私问题，以及通过模拟中的代理测试实现的人工智能安全问题。在使用该套件生成的61个问题实例中，我们评估了不同算法在此类问题中寻找一阶最优策略的性能。特别是，我们介绍了家庭的遗憾匹配（RM）算法的非线性约束优化。这类无参数算法在解决大型两人零和游戏方面取得了巨大的成功，但令人惊讶的是，迄今为止，它们在该设置之外相对未被探索。我们的主要发现是，RM算法始终优于常用的一阶优化，如投影梯度下降，往往是数量级。这建立了，第一次，RM家庭作为一个强大的方法来解决大规模的约束优化问题。
摘要：In game theory, imperfect-recall decision problems model situations in which an agent forgets information it held before. They encompass games such as the ``absentminded driver'' and team games with limited communication. In this paper, we introduce the first benchmark suite for imperfect-recall decision problems. Our benchmarks capture a variety of problem types, including ones concerning privacy in AI systems that elicit sensitive information, and AI safety via testing of agents in simulation. Across 61 problem instances generated using this suite, we evaluate the performance of different algorithms for finding first-order optimal strategies in such problems. In particular, we introduce the family of regret matching (RM) algorithms for nonlinear constrained optimization. This class of parameter-free algorithms has enjoyed tremendous success in solving large two-player zero-sum games, but, surprisingly, they were hitherto relatively unexplored beyond that setting. Our key finding is that RM algorithms consistently outperform commonly employed first-order optimizers such as projected gradient descent, often by orders of magnitude. This establishes, for the first time, the RM family as a formidable approach to large-scale constrained optimization problems.
【26】tensorFM: Low-Rank Approximations of Cross-Order Feature Interactions
标题：tensorFM：跨阶特征相互作用的低等级逼近
链接：https://arxiv.org/abs/2602.15229
摘要：我们解决预测问题的表格分类数据，其中每个实例是由多个分类属性，每个值从一个有限的集合定义。这些属性通常被称为字段，它们的分类值被称为特征。这些问题在实际应用中经常出现，包括点击率预测和社会科学。我们介绍和分析{tensorFM}，一个新的模型，有效地捕捉高阶属性之间的相互作用，通过低秩张量近似表示这些相互作用的强度。我们的模型推广了场加权因子分解机。从经验上讲，tensorFM通过最先进的方法展示了具有竞争力的性能。此外，它的低延迟使其非常适合时间敏感的应用程序，如在线广告。
摘要：We address prediction problems on tabular categorical data, where each instance is defined by multiple categorical attributes, each taking values from a finite set. These attributes are often referred to as fields, and their categorical values as features. Such problems frequently arise in practical applications, including click-through rate prediction and social sciences. We introduce and analyze {tensorFM}, a new model that efficiently captures high-order interactions between attributes via a low-rank tensor approximation representing the strength of these interactions. Our model generalizes field-weighted factorization machines. Empirically, tensorFM demonstrates competitive performance with state-of-the-art methods. Additionally, its low latency makes it well-suited for time-sensitive applications, such as online advertising.
【27】ÜberWeb: Insights from Multilingual Curation for a 20-Trillion-Token Dataset
标题：ÜberWeb：20万亿个代币数据集的多语言处理见解
链接：https://arxiv.org/abs/2602.15210
摘要：多语言是现代基础模型的核心能力，但由于不同语言的数据可用性不均衡，训练高质量的多语言模型仍然具有挑战性。另一个挑战是联合多语种培训可能产生的成绩干扰，这通常被称为“多语种诅咒”。我们研究了13种语言的多语言数据策展，发现许多报告的回归不是多语言扩展所固有的，而是源于数据质量和组成的可纠正缺陷，而不是基本的容量限制。在受控双语实验中，提高任何一种语言的数据质量都会使其他语言受益：策划英语提高了13种语言中12种语言的非英语表现，而策划非英语则会产生英语的相互改善。定制的每种语言策展产生了更大的语言内改进。将这些发现扩展到大规模的通用训练混合物，我们发现，包含总令牌的8%以下的策划多语言分配仍然非常有效。我们在一个完全来自公共资源的20个T-token预训练语料库中实现了这种方法。在1 T令牌随机子集上训练的3B和8B参数模型实现了具有竞争力的多语言准确性，训练FLOP比强大的公共基线少4- 10倍，在多语言性能与计算方面建立了新的帕累托边界。此外，这些好处还扩展到了前沿模型规模：20 T令牌语料库作为Trinity Large（400 B/A13 B）预训练数据集的一部分，相对于其训练FLOP，该数据集表现出强大的多语言性能。这些结果表明，有针对性的每种语言数据管理减轻了多语言干扰，并实现了计算效率的多语言扩展。
摘要：Multilinguality is a core capability for modern foundation models, yet training high-quality multilingual models remains challenging due to uneven data availability across languages. A further challenge is the performance interference that can arise from joint multilingual training, commonly referred to as the "curse of multilinguality". We study multilingual data curation across thirteen languages and find that many reported regressions are not inherent to multilingual scaling but instead stem from correctable deficiencies in data quality and composition rather than fundamental capacity limits. In controlled bilingual experiments, improving data quality for any single language benefits others: curating English improves non-English performance in 12 of 13 languages, while curating non-English yields reciprocal improvements in English. Bespoke per-language curation produces substantially larger within-language improvements. Extending these findings to large-scale general-purpose training mixtures, we show that curated multilingual allocations comprising under 8% of total tokens remain remarkably effective. We operationalize this approach within an effort that produced a 20T-token pretraining corpus derived entirely from public sources. Models with 3B and 8B parameters trained on a 1T-token random subset achieve competitive multilingual accuracy with 4-10x fewer training FLOPs than strong public baselines, establishing a new Pareto frontier in multilingual performance versus compute. Moreover, these benefits extend to frontier model scale: the 20T-token corpus served as part of the pretraining dataset for Trinity Large (400B/A13B), which exhibits strong multilingual performance relative to its training FLOPs. These results show that targeted, per-language data curation mitigates multilingual interference and enables compute-efficient multilingual scaling.
【28】Seeing to Generalize: How Visual Data Corrects Binding Shortcuts
标题：着眼于概括：视觉数据如何纠正绑定快捷方式
链接：https://arxiv.org/abs/2602.15183
备注：Submitted to ICML 2026
摘要：视觉语言模型（VLM）旨在扩展具有视觉功能的大型语言模型（LLM），但在这项工作中，我们观察到一个令人惊讶的现象：VLM可以在纯文本任务中超越其底层LLM，特别是在长上下文信息检索中。为了研究这种效果，我们构建了一个受控的合成检索任务，发现仅在文本上训练的Transformer实现了完美的分布内准确性，但未能概括出分布，而随后在同一任务的图像标记化版本上进行的训练几乎使纯文本OOD性能翻了一番。机械可解释性表明，视觉训练改变了模型的内部绑定策略：纯文本训练鼓励位置快捷方式，而基于图像的训练通过空间平移不变性破坏它们，迫使模型采用更强大的符号绑定机制，即使在重新引入纯文本示例后仍然存在。我们进一步描述了绑定策略如何在训练机制，视觉编码器和初始化之间变化，并表明在预训练的LLM到VLM转换期间发生类似的转变。我们的研究结果表明，跨模态训练可以增强推理和概括，即使是在一个单一的模态接地任务。
摘要：Vision Language Models (VLMs) are designed to extend Large Language Models (LLMs) with visual capabilities, yet in this work we observe a surprising phenomenon: VLMs can outperform their underlying LLMs on purely text-only tasks, particularly in long-context information retrieval. To investigate this effect, we build a controlled synthetic retrieval task and find that a transformer trained only on text achieves perfect in-distribution accuracy but fails to generalize out of distribution, while subsequent training on an image-tokenized version of the same task nearly doubles text-only OOD performance. Mechanistic interpretability reveals that visual training changes the model's internal binding strategy: text-only training encourages positional shortcuts, whereas image-based training disrupts them through spatial translation invariance, forcing the model to adopt a more robust symbolic binding mechanism that persists even after text-only examples are reintroduced. We further characterize how binding strategies vary across training regimes, visual encoders, and initializations, and show that analogous shifts occur during pretrained LLM-to-VLM transitions. Our findings suggest that cross-modal training can enhance reasoning and generalization even for tasks grounded in a single modality.
【29】Time-Archival Camera Virtualization for Sports and Visual Performances
标题：用于体育和视觉表演的时间档案摄像机虚拟化
链接：https://arxiv.org/abs/2602.15181
备注：Project Page: https://yunxiaozhangjack.com/tacv/; Under minor revision in Journal of Computer Vision and Image Understanding (CVIU); Special Issue: Computer Vision for Sports and Winter Sports. Outcome of a master and bachelor student project completed in Visual and Spatial AI Lab at TAMU
摘要：摄像机虚拟化--一种新兴的视图合成解决方案--通过使用来自有限的一组校准的多个静态物理摄像机的图像从新的视角生成逼真的图像，为视觉娱乐、现场表演和体育广播提供了变革性的潜力。尽管最近的进展，实现空间和时间上的连贯性和逼真的渲染动态场景与有效的时间存档能力，特别是在快节奏的体育和舞台表演，仍然是现有的方法的挑战。基于3DGS（3D高斯溅射）的动态场景的最新方法可以提供实时的视图合成结果。然而，它们受到它们对来自运动恢复结构方法的精确3D点云的依赖以及它们不能处理不同对象的大的、非刚性的、快速运动（例如，翻转、跳跃、发音、突然的玩家到玩家转换）。此外，多个主体的独立运动可以打破在4DGS、ST-GS和其他动态飞溅变体中常用的高斯跟踪假设。本文主张重新考虑摄像机虚拟化和有效的时间存档功能的神经体绘制公式，使其对体育广播和相关应用有用。通过将动态场景建模为在给定时间跨多个同步相机视图的刚性变换，我们的方法执行神经表示学习，在测试时提供增强的视觉渲染质量。我们的方法的一个关键贡献是它支持时间存档，即，用户可以重新访问动态场景的任何过去的时间实例，并且可以执行新颖的视图合成，从而实现用于实况事件的重放、分析和存档的回顾性渲染，这是现有神经渲染方法和新颖的视图合成中缺少的功能。
摘要 ：Camera virtualization -- an emerging solution to novel view synthesis -- holds transformative potential for visual entertainment, live performances, and sports broadcasting by enabling the generation of photorealistic images from novel viewpoints using images from a limited set of calibrated multiple static physical cameras. Despite recent advances, achieving spatially and temporally coherent and photorealistic rendering of dynamic scenes with efficient time-archival capabilities, particularly in fast-paced sports and stage performances, remains challenging for existing approaches. Recent methods based on 3D Gaussian Splatting (3DGS) for dynamic scenes could offer real-time view-synthesis results. Yet, they are hindered by their dependence on accurate 3D point clouds from the structure-from-motion method and their inability to handle large, non-rigid, rapid motions of different subjects (e.g., flips, jumps, articulations, sudden player-to-player transitions). Moreover, independent motions of multiple subjects can break the Gaussian-tracking assumptions commonly used in 4DGS, ST-GS, and other dynamic splatting variants. This paper advocates reconsidering a neural volume rendering formulation for camera virtualization and efficient time-archival capabilities, making it useful for sports broadcasting and related applications. By modeling a dynamic scene as rigid transformations across multiple synchronized camera views at a given time, our method performs neural representation learning, providing enhanced visual rendering quality at test time. A key contribution of our approach is its support for time-archival, i.e., users can revisit any past temporal instance of a dynamic scene and can perform novel view synthesis, enabling retrospective rendering for replay, analysis, and archival of live events, a functionality absent in existing neural rendering approaches and novel view synthesis...
【30】Refine Now, Query Fast: A Decoupled Refinement Paradigm for Implicit Neural Fields
标题：立即细化，快速查询：隐式神经场的脱钩细化范式
链接：https://arxiv.org/abs/2602.15155
备注：Accepted to ICLR 2026. Code available at https://github.com/xtyinzz/DRR-INR
摘要：隐式神经表示（INRs）由于其连续建模空间和条件场的能力而成为大型3D科学模拟的有前途的代理人，但它们面临着一个关键的速度困境：深度MLP遭受高推理成本，而高效的基于嵌入的模型缺乏足够的表达能力。为了解决这个问题，我们提出了解耦表示细化（DRR）架构范例。DRR在一次性离线过程中利用深度细化器网络以及非参数变换，将丰富的表示编码为紧凑高效的嵌入结构。这种方法将具有高表示能力的慢速神经网络从快速推理路径中分离出来，我们引入了DRR-Net，一个验证这种范式的简单网络，以及一种新的数据增强策略，变分对（VP），用于改善高维代理建模等复杂任务下的INR。在几个集成仿真数据集上的实验表明，我们的方法实现了最先进的保真度，同时比高保真基线的推理速度快27倍，并与最快的模型保持竞争力。DRR范式提供了一种有效的策略，用于构建功能强大且实用的神经场代理和更广泛应用中的INR，并在速度和质量之间做出最小的妥协。
摘要：Implicit Neural Representations (INRs) have emerged as promising surrogates for large 3D scientific simulations due to their ability to continuously model spatial and conditional fields, yet they face a critical fidelity-speed dilemma: deep MLPs suffer from high inference cost, while efficient embedding-based models lack sufficient expressiveness. To resolve this, we propose the Decoupled Representation Refinement (DRR) architectural paradigm. DRR leverages a deep refiner network, alongside non-parametric transformations, in a one-time offline process to encode rich representations into a compact and efficient embedding structure. This approach decouples slow neural networks with high representational capacity from the fast inference path. We introduce DRR-Net, a simple network that validates this paradigm, and a novel data augmentation strategy, Variational Pairs (VP) for improving INRs under complex tasks like high-dimensional surrogate modeling. Experiments on several ensemble simulation datasets demonstrate that our approach achieves state-of-the-art fidelity, while being up to 27$\times$ faster at inference than high-fidelity baselines and remaining competitive with the fastest models. The DRR paradigm offers an effective strategy for building powerful and practical neural field surrogates and \rev{INRs in broader applications}, with a minimal compromise between speed and quality.
【31】PolyNODE: Variable-dimension Neural ODEs on M-polyfolds
标题：PolyNODE：M-多折叠上的可变维神经ODE
链接：https://arxiv.org/abs/2602.15128
摘要：神经常微分方程（NODE）是基于动力系统和流形上向量场生成的流的几何深度学习模型。尽管有许多成功的应用，特别是在流量匹配的范例，所有现有的NODE模型从根本上限制到固定维动力学的流形的尺寸的内在性质。在本文中，我们将NODE扩展到M-polyfolds（可以同时容纳不同维度和可微性概念的空间），并引入PolyNODE，这是几何深度学习中第一个基于变维流的模型。作为一个示例应用程序，我们构建显式的M-多边形具有尺寸瓶颈和PolyNODE自动编码器的参数化向量场，遍历这些瓶颈。我们通过实验证明，我们的PolyNODE模型可以被训练来解决这些空间中的重建任务，并且可以提取输入的潜在表示并用于解决下游分类任务。我们实验中使用的代码可在https://github.com/turbotage/PolyNODE上公开获得。
摘要：Neural ordinary differential equations (NODEs) are geometric deep learning models based on dynamical systems and flows generated by vector fields on manifolds. Despite numerous successful applications, particularly within the flow matching paradigm, all existing NODE models are fundamentally constrained to fixed-dimensional dynamics by the intrinsic nature of the manifold's dimension. In this paper, we extend NODEs to M-polyfolds (spaces that can simultaneously accommodate varying dimensions and a notion of differentiability) and introduce PolyNODEs, the first variable-dimensional flow-based model in geometric deep learning. As an example application, we construct explicit M-polyfolds featuring dimensional bottlenecks and PolyNODE autoencoders based on parametrised vector fields that traverse these bottlenecks. We demonstrate experimentally that our PolyNODE models can be trained to solve reconstruction tasks in these spaces, and that latent representations of the input can be extracted and used to solve downstream classification tasks. The code used in our experiments is publicly available at https://github.com/turbotage/PolyNODE .
【32】VQ-DSC-R: Robust Vector Quantized-Enabled Digital Semantic Communication With OFDM Transmission
标题：VQ-DCS-R：采用CDMA传输的稳健的、支持量化的数字语义通信
链接：https://arxiv.org/abs/2602.15045
摘要：语义特征的数字映射对于实现语义通信和实际数字基础设施之间的互操作性至关重要。然而，目前的研究工作主要集中在模拟语义通信与简化的信道模型。为了弥合这些差距，我们开发了一个强大的矢量量化数字语义通信（VQ-DSC-R）系统建立在正交频分复用（OFDM）传输。我们的工作包括VQ-DSC-R的框架设计，其次是全面的优化研究。首先，我们设计了一个基于Swin变换的骨干分层语义特征提取，集成VQ模块，映射到一个共享的语义量化码本（SQC）的有效索引传输的功能。其次，我们提出了一种自适应噪声方差的可微分矢量量化（ANDVQ）方案来减轻SQC中的量化误差，该方案使用K-最近邻统计动态调整量化过程，而指数移动平均机制稳定SQC训练。第三，针对多径衰落信道和噪声环境下的鲁棒索引传输，提出了一种条件扩散模型（CDM）来细化信道状态信息，并设计了一个基于注意力的模块来动态适应信道噪声。整个VQ-DSC-R系统通过三阶段训练策略进行优化。大量的实验表明，VQ-DSC-R优于基准方案，实现高压缩比和强大的性能在实际情况下。
摘要：Digital mapping of semantic features is essential for achieving interoperability between semantic communication and practical digital infrastructure. However, current research efforts predominantly concentrate on analog semantic communication with simplified channel models. To bridge these gaps, we develop a robust vector quantized-enabled digital semantic communication (VQ-DSC-R) system built upon orthogonal frequency division multiplexing (OFDM) transmission. Our work encompasses the framework design of VQ-DSC-R, followed by a comprehensive optimization study. Firstly, we design a Swin Transformer-based backbone for hierarchical semantic feature extraction, integrated with VQ modules that map the features into a shared semantic quantized codebook (SQC) for efficient index transmission. Secondly, we propose a differentiable vector quantization with adaptive noise-variance (ANDVQ) scheme to mitigate quantization errors in SQC, which dynamically adjusts the quantization process using K-nearest neighbor statistics, while exponential moving average mechanism stabilizes SQC training. Thirdly, for robust index transmission over multipath fading channel and noise, we develop a conditional diffusion model (CDM) to refine channel state information, and design an attention-based module to dynamically adapt to channel noise. The entire VQ-DSC-R system is optimized via a three-stage training strategy. Extensive experiments demonstrate superiority of VQ-DSC-R over benchmark schemes, achieving high compression ratios and robust performance in practical scenarios.
【33】Neural Scaling Laws for Boosted Jet Tagging
标题：助推射流标记的神经标度律
链接：https://arxiv.org/abs/2602.15781
备注：9 pages, 6 figures
摘要：大型语言模型（LLM）的成功已经证明，通过联合增加模型容量和数据集大小来扩展计算是现代机器学习性能的主要驱动力。虽然机器学习长期以来一直是高能物理（HEP）数据分析工作流程的组成部分，但用于训练最先进的HEP模型的计算仍然低于行业基础模型的数量级。由于标度律才开始在该领域进行研究，我们使用公共JetClass数据集研究了用于助推射流分类的神经标度律。我们推导出计算最佳的缩放律，并确定一个有效的性能限制，可以通过增加计算一致接近。我们研究如何数据重复，常见的HEP模拟是昂贵的，修改缩放产生一个可量化的有效的数据集大小增益。然后，我们研究了缩放系数和渐近性能极限如何随输入特征和粒子多样性的选择而变化，证明了增加的计算可靠地将性能推向渐近极限，并且更具表现力的低级别特征可以提高性能极限并改善固定数据集大小的结果。
摘要：The success of Large Language Models (LLMs) has established that scaling compute, through joint increases in model capacity and dataset size, is the primary driver of performance in modern machine learning. While machine learning has long been an integral component of High Energy Physics (HEP) data analysis workflows, the compute used to train state-of-the-art HEP models remains orders of magnitude below that of industry foundation models. With scaling laws only beginning to be studied in the field, we investigate neural scaling laws for boosted jet classification using the public JetClass dataset. We derive compute optimal scaling laws and identify an effective performance limit that can be consistently approached through increased compute. We study how data repetition, common in HEP where simulation is expensive, modifies the scaling yielding a quantifiable effective dataset size gain. We then study how the scaling coefficients and asymptotic performance limits vary with the choice of input features and particle multiplicity, demonstrating that increased compute reliably drives performance toward an asymptotic limit, and that more expressive, lower-level features can raise the performance limit and improve results at fixed dataset size.
【34】Neural-POD: A Plug-and-Play Neural Operator Framework for Infinite-Dimensional Functional Nonlinear Proper Orthogonal Decomposition
标题：Neural-Pod：一个用于无限维函数非线性固有正交分解的即插即用神经操作器框架
链接：https://arxiv.org/abs/2602.15632
摘要：科学人工智能的快速发展往往受到“离散化”的阻碍，在这种情况下，学习的表示仍然局限于训练期间使用的特定网格或分辨率。我们提出了神经固有正交分解（Neural-POD），这是一种即插即用的神经算子框架，它使用神经网络在无限维空间中构造非线性正交基函数。与经典的固有正交分解（POD），这是有限的线性子空间近似通过奇异值分解（SVD），神经POD制定的基础建设作为一个序列的残差最小化问题，通过神经网络训练解决。每个基函数通过学习表示数据中的剩余结构来获得，遵循类似于Gram-Schmidt正交化的过程。与经典POD相比，这种神经公式引入了几个关键优点：它能够在任意范数下进行优化（例如，$L^2$，$L^1$），学习分辨率不变的无限维函数空间之间的映射，有效地推广到看不见的参数机制，并内在地捕捉复杂时空系统中的非线性结构。由此产生的基函数是可解释的、可重用的，并且能够集成到降阶建模（ROM）和算子学习框架（如深度算子学习（DeepONet））中。我们证明了不同的复杂时空系统，包括Burgers和Navier-Stokes方程的神经POD的鲁棒性。我们进一步表明，Neural-POD是经典Galerkin投影和算子学习之间的高性能，即插即用的桥梁，可以与基于投影的降阶模型和DeepONet框架进行一致的集成。
摘要：The rapid development of AI for Science is often hindered by the "discretization", where learned representations remain restricted to the specific grids or resolutions used during training. We propose the Neural Proper Orthogonal Decomposition (Neural-POD), a plug-and-play neural operator framework that constructs nonlinear, orthogonal basis functions in infinite-dimensional space using neural networks. Unlike the classical Proper Orthogonal Decomposition (POD), which is limited to linear subspace approximations obtained through singular value decomposition (SVD), Neural-POD formulates basis construction as a sequence of residual minimization problems solved through neural network training. Each basis function is obtained by learning to represent the remaining structure in the data, following a process analogous to Gram--Schmidt orthogonalization. This neural formulation introduces several key advantages over classical POD: it enables optimization in arbitrary norms (e.g., $L^2$, $L^1$), learns mappings between infinite-dimensional function spaces that is resolution-invariant, generalizes effectively to unseen parameter regimes, and inherently captures nonlinear structures in complex spatiotemporal systems. The resulting basis functions are interpretable, reusable, and enabling integration into both reduced order modeling (ROM) and operator learning frameworks such as deep operator learning (DeepONet). We demonstrate the robustness of Neural-POD with different complex spatiotemporal systems, including the Burgers' and Navier-Stokes equations. We further show that Neural-POD serves as a high performance, plug-and-play bridge between classical Galerkin projection and operator learning that enables consistent integration with both projection-based reduced order models and DeepONet frameworks.
【35】Scenario Approach with Post-Design Certification of User-Specified Properties
标题：具有用户指定属性设计后认证的场景方法
链接：https://arxiv.org/abs/2602.15568
摘要：场景方法是一个建立的数据驱动的设计框架，它配备了一个强大的理论，将设计复杂性与泛化属性联系起来。在这种方法中，数据同时用于设计和验证设计的可靠性，而不诉诸于单独的测试数据集。本文更进一步，通过保证额外的属性，在设计后使用有用，但在设计阶段不考虑。为此，我们引入了两个层次的适当性框架：基线适当性，它指导设计过程，和设计后的适当性，这是作为一个标准的后验评价。我们提供了不符合设计后适当性的风险的无分布上限;这些界限是可计算的，无需使用任何额外的测试数据。在额外的假设下，下限也来自。作为努力证明所提出的方法的有用性的一部分，本文提出了两个实际的例子在H2和极点配置问题。此外，提供了一种方法来推断从可用的数据集的相关性能指标的综合分布知识。
摘要：The scenario approach is an established data-driven design framework that comes equipped with a powerful theory linking design complexity to generalization properties. In this approach, data are simultaneously used both for design and for certifying the design's reliability, without resorting to a separate test dataset. This paper takes a step further by guaranteeing additional properties, useful in post-design usage but not considered during the design phase. To this end, we introduce a two-level framework of appropriateness: baseline appropriateness, which guides the design process, and post-design appropriateness, which serves as a criterion for a posteriori evaluation. We provide distribution-free upper bounds on the risk of failing to meet the post-design appropriateness; these bounds are computable without using any additional test data. Under additional assumptions, lower bounds are also derived. As part of an effort to demonstrate the usefulness of the proposed methodology, the paper presents two practical examples in H2 and pole-placement problems. Moreover, a method is provided to infer comprehensive distributional knowledge of relevant performance indexes from the available dataset.
【36】Functional Central Limit Theorem for Stochastic Gradient Descent
标题：随机梯度下降的泛函中心极限定理
链接：https://arxiv.org/abs/2602.15538
摘要：我们研究了随机梯度下降算法应用于凸目标函数的轨迹的渐近形状。在温和的正则性假设下，我们证明了一个功能中心极限定理的适当重新标度的轨迹。我们的结果特征的长期波动的算法周围的最小值提供了一个扩散限制的轨迹。与经典的中心极限定理的最后一个或Polyak-Ruppert平均值相比，这个功能的结果捕获的波动的时间结构，并适用于非光滑的设置，如鲁棒的位置估计，包括几何中位数。
摘要 ：We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the properly rescaled trajectory. Our result characterizes the long-term fluctuations of the algorithm around the minimizer by providing a diffusion limit for the trajectory. In contrast with classical central limit theorems for the last iterate or Polyak-Ruppert averages, this functional result captures the temporal structure of the fluctuations and applies to non-smooth settings such as robust location estimation, including the geometric median.
【37】Mixture-of-Experts under Finite-Rate Gating: Communication--Generalization Trade-offs
标题：伪利率门控下的专家混合：沟通--一般化权衡
链接：https://arxiv.org/abs/2602.15091
摘要：混合专家（MoE）体系结构将预测任务分解为由门控机制选择的专业专家子网络。这封信采用了通信理论的观点MoE门，建模的门作为一个随机信道下的有限信息速率。在信息论学习框架内，我们专门研究了互信息泛化界，并开发了有限速率门控的率失真表征$D（R_g）$，其中$R_g：=I（X; T）$，产生（在标准经验率失真最优性条件下）$\mathbb{E}[R（W）] \le D（R_g）+δ_m+\sqrt{（2/m）\，I（S; W）}$。该分析产生的通信受限的MoE系统的能力意识的限制，和合成多专家模型的数值模拟经验证实了预测的门控率，表现力和泛化之间的权衡。
摘要：Mixture-of-Experts (MoE) architectures decompose prediction tasks into specialized expert sub-networks selected by a gating mechanism. This letter adopts a communication-theoretic view of MoE gating, modeling the gate as a stochastic channel operating under a finite information rate. Within an information-theoretic learning framework, we specialize a mutual-information generalization bound and develop a rate-distortion characterization $D(R_g)$ of finite-rate gating, where $R_g:=I(X; T)$, yielding (under a standard empirical rate-distortion optimality condition) $\mathbb{E}[R(W)] \le D(R_g)+δ_m+\sqrt{(2/m)\, I(S; W)}$. The analysis yields capacity-aware limits for communication-constrained MoE systems, and numerical simulations on synthetic multi-expert models empirically confirm the predicted trade-offs between gating rate, expressivity, and generalization.
机器翻译由腾讯交互翻译提供，仅供参考
点击“阅读原文”获取带摘要的学术速递
Python社区是高质量的Python/Django开发社区
本文地址：http://www.python88.com/topic/193079
本文地址：http://www.python88.com/topic/193079