|
创作新主题 |
社区所有版块导航
DATA
| docker Elasticsearch |
WEB开发
| linux MongoDB Redis DATABASE NGINX 其他Web框架 web工具 zookeeper tornado NoSql Bootstrap js peewee Git bottle IE MQ Jquery |
机器学习
| 机器学习算法 |
产品
| 短视频 |
印度
| 印度 |
一周十大热门主题
点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！
cs.LG 方向，今日共计163篇
大模型相关(20篇)
【1】Sink-Aware Pruning for Diffusion Language Models
标题：扩散语言模型的下沉意识修剪
链接：https://arxiv.org/abs/2602.17664
备注：Code at: https://github.com/VILA-Lab/Sink-Aware-Pruning
摘要：扩散语言模型（DLMs）由于迭代去噪而导致高推理成本，从而激励有效的修剪。现有的修剪算法主要继承自自回归（AR）LLM，通常保留注意力汇令牌，因为AR汇作为稳定的全局锚。我们发现，这种假设并不适用于DLMs：注意力汇位置在整个生成轨迹上表现出显著更高的方差（通过占主导地位的汇位置如何跨时间步移动来测量），这表明汇通常是短暂的，结构上比AR模型更不重要。基于这一观察，我们提出了${\bf \texttt{Sink-Aware Pruning}}$，它自动识别和修剪DLMs中的不稳定汇（以前的研究通常为AR LLM保留汇）。在没有再训练的情况下，我们的方法实现了更好的质量-效率权衡，并且在匹配计算下优于强先验剪枝基线。我们的代码可在https://github.com/VILA-Lab/Sink-Aware-Pruning上获得。
摘要：Diffusion Language Models (DLMs) incur high inference cost due to iterative denoising, motivating efficient pruning. Existing pruning heuristics largely inherited from autoregressive (AR) LLMs, typically preserve attention sink tokens because AR sinks serve as stable global anchors. We show that this assumption does not hold for DLMs: the attention-sink position exhibits substantially higher variance over the full generation trajectory (measured by how the dominant sink locations shift across timesteps), indicating that sinks are often transient and less structurally essential than in AR models. Based on this observation, we propose ${\bf \texttt{Sink-Aware Pruning}}$, which automatically identifies and prunes unstable sinks in DLMs (prior studies usually keep sinks for AR LLMs). Without retraining, our method achieves a better quality-efficiency trade-off and outperforms strong prior pruning baselines under matched compute. Our code is available at https://github.com/VILA-Lab/Sink-Aware-Pruning.
【2】Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
标题：稳定的非同步性：LLM的方差控制非政策RL
链接：https://arxiv.org/abs/2602.17616
摘要：强化学习（RL）被广泛用于改进推理任务的大型语言模型，异步RL训练很有吸引力，因为它增加了端到端的吞吐量。然而，对于广泛采用的无临界策略梯度方法（如REINFORCE和GRPO），高方差使得策略梯度估计器显着$\textbf{higher variance}$：在陈旧的推出上进行训练会产生重尾重要性比，导致一小部分样本主导更新。这种放大使得梯度噪声和学习相对于匹配的策略训练不稳定。在数学和一般推理基准中，我们发现崩溃是由有效样本大小（ESS）和不稳定梯度范数可靠地预测的。受此诊断的启发，我们提出$\textbf{V}$ariance $\textbf{C}$controlled $\textbf{P}$olicy $\textbf{O}$优化（$\textbf{VCPO}$），一种用于REINFORCE/GRPO风格算法的通用稳定化方法，该方法（i）基于有效样本大小缩放学习率以抑制不可靠的更新，以及（ii）对非策略设置应用封闭形式的最小方差基线，避免了辅助价值模型并增加了最小的开销。从经验上讲，VCPO大大提高了跨数学、一般推理和工具使用任务的异步训练的鲁棒性，优于一系列跨越掩蔽/裁剪稳定器和算法变体的基线。这将长上下文，多轮训练时间减少了2.5times $，同时匹配同步性能，表明显式控制策略梯度方差是大规模可靠异步RL的关键。
摘要：Reinforcement learning (RL) is widely used to improve large language models on reasoning tasks, and asynchronous RL training is attractive because it increases end-to-end throughput. However, for widely adopted critic-free policy-gradient methods such as REINFORCE and GRPO, high asynchrony makes the policy-gradient estimator markedly $\textbf{higher variance}$: training on stale rollouts creates heavy-tailed importance ratios, causing a small fraction of samples to dominate updates. This amplification makes gradients noisy and learning unstable relative to matched on-policy training. Across math and general reasoning benchmarks, we find collapse is reliably predicted by effective sample size (ESS) and unstable gradient norms. Motivated by this diagnosis, we propose $\textbf{V}$ariance $\textbf{C}$ontrolled $\textbf{P}$olicy $\textbf{O}$ptimization ($\textbf{VCPO}$), a general stabilization method for REINFORCE/GRPO-style algorithms that (i) scales learning rate based on effective sample size to dampen unreliable updates, and (ii) applies a closed-form minimum-variance baseline for the off-policy setting, avoiding an auxiliary value model and adding minimal overhead. Empirically, VCPO substantially improves robustness for asynchronous training across math, general reasoning, and tool-use tasks, outperforming a broad suite of baselines spanning masking/clipping stabilizers and algorithmic variants. This reduces long-context, multi-turn training time by 2.5$\times$ while matching synchronous performance, demonstrating that explicit control of policy-gradient variance is key for reliable asynchronous RL at scale.
【3】MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning
标题：MASPO：统一梯度利用率、概率质量和信号可靠性，实现稳健且样本高效的LLM推理
链接：https://arxiv.org/abs/2602.17550
摘要：现有的具有可验证奖励的强化学习（RLVR）算法（如GRPO）依赖于刚性、统一和对称的信任域机制，这些机制与大型语言模型（LLM）的复杂优化动态从根本上不一致。在本文中，我们确定了这些方法中的三个关键挑战：（1）由硬裁剪的二进制截止引起的低效梯度利用，（2）由忽略令牌分布的均匀比率约束引起的不敏感概率质量，以及（3）由正负样本之间的不同信用分配模糊性引起的不对称信号可靠性。为了弥合这些差距，我们提出了大规模自适应软政策优化（MASPO），一个统一的框架，旨在协调这三个方面。MASPO集成了一个可微分的软高斯门控，以最大限度地提高梯度效用，一个质量自适应限制器，以平衡整个概率谱的探索，和一个不对称的风险控制器，以调整更新幅度与信号的信心。广泛的评估表明，MASPO作为一个强大的，所有功能于一体的RLVR解决方案，显着优于强大的基线。我们的代码可在https://anonymous.4open.science/r/ma1/README.md上获得。
摘要：Existing Reinforcement Learning with Verifiable Rewards (RLVR) algorithms, such as GRPO, rely on rigid, uniform, and symmetric trust region mechanisms that are fundamentally misaligned with the complex optimization dynamics of Large Language Models (LLMs). In this paper, we identify three critical challenges in these methods: (1) inefficient gradient utilization caused by the binary cutoff of hard clipping, (2) insensitive probability mass arising from uniform ratio constraints that ignore the token distribution, and (3) asymmetric signal reliability stemming from the disparate credit assignment ambiguity between positive and negative samples. To bridge these gaps, we propose Mass-Adaptive Soft Policy Optimization (MASPO), a unified framework designed to harmonize these three dimensions. MASPO integrates a differentiable soft Gaussian gating to maximize gradient utility, a mass-adaptive limiter to balance exploration across the probability spectrum, and an asymmetric risk controller to align update magnitudes with signal confidence. Extensive evaluations demonstrate that MASPO serves as a robust, all-in-one RLVR solution, significantly outperforming strong baselines. Our code is available at: https://anonymous.4open.science/r/ma1/README.md.
【4】Retrospective In-Context Learning for Temporal Credit Assignment with Large Language Models
标题：使用大语言模型进行时间学分分配的回顾性上下文学习
链接：https://arxiv.org/abs/2602.17497
备注：Accepted to NeurIPS 2025
摘要：从自采样数据和稀疏环境反馈中学习仍然是训练自我进化代理的基本挑战。时间信用分配通过将稀疏反馈转换为密集监督信号来缓解这个问题。然而，以前的方法通常依赖于学习特定于任务的值函数的信用分配，这遭受低的样本效率和有限的推广。在这项工作中，我们建议利用来自大型语言模型（LLM）的预训练知识将稀疏奖励转换为密集训练信号（即，优势功能）通过回顾性上下文学习（RICL）。我们进一步提出了一个在线学习框架，RICOL，它迭代地完善政策的基础上，从RICL的信用分配结果。我们的实证研究表明，RICL可以准确地估计有限样本的优势函数，并有效地识别在时间信用分配的环境中的临界状态。四个BabyAI场景的扩展评估表明，RICOL实现了与传统在线RL算法相当的收敛性能，具有显着更高的样本效率。我们的研究结果突出了利用LLM进行临时信用分配的潜力，为更样本高效和可推广的RL范例铺平了道路。
摘要：Learning from self-sampled data and sparse environmental feedback remains a fundamental challenge in training self-evolving agents. Temporal credit assignment mitigates this issue by transforming sparse feedback into dense supervision signals. However, previous approaches typically depend on learning task-specific value functions for credit assignment, which suffer from poor sample efficiency and limited generalization. In this work, we propose to leverage pretrained knowledge from large language models (LLMs) to transform sparse rewards into dense training signals (i.e., the advantage function) through retrospective in-context learning (RICL). We further propose an online learning framework, RICOL, which iteratively refines the policy based on the credit assignment results from RICL. We empirically demonstrate that RICL can accurately estimate the advantage function with limited samples and effectively identify critical states in the environment for temporal credit assignment. Extended evaluation on four BabyAI scenarios show that RICOL achieves comparable convergent performance with traditional online RL algorithms with significantly higher sample efficiency. Our findings highlight the potential of leveraging LLMs for temporal credit assignment, paving the way for more sample-efficient and generalizable RL paradigms.
【5】Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study
标题：长格式语言模型输出的细粒度不确定性量化：比较研究
链接：https://arxiv.org/abs/2602.17431
备注：UQLM repository: https://github.com/cvs-health/uqlm
摘要：不确定性量化已经成为LLM的闭书幻觉检测的有效方法，但现有方法主要是针对短形式输出而设计的，并且不能很好地推广到长形式生成。我们引入了一个分类法，用于在长形式LLM输出中进行细粒度不确定性量化，该输出通过三个阶段的设计选择来区分方法：响应分解，单元级评分和响应级聚合。我们形式化了几个基于一致性的黑盒评分器族，提供了现有方法的概括和扩展。在我们对多个LLM和数据集的实验中，我们发现1）索赔-响应蕴涵始终表现得更好或与更复杂的索赔级别评分器相当，2）索赔级别评分通常会产生比索赔级别评分更好的结果，3）不确定性感知解码对于提高长形式输出的真实性非常有效。我们的框架澄清了以前的方法之间的关系，使苹果到苹果的比较，并为细粒度的UQ选择组件提供了实用的指导。
摘要：Uncertainty quantification has emerged as an effective approach to closed-book hallucination detection for LLMs, but existing methods are largely designed for short-form outputs and do not generalize well to long-form generation. We introduce a taxonomy for fine-grained uncertainty quantification in long-form LLM outputs that distinguishes methods by design choices at three stages: response decomposition, unit-level scoring, and response-level aggregation. We formalize several families of consistency-based black-box scorers, providing generalizations and extensions of existing methods. In our experiments across multiple LLMs and datasets, we find 1) claim-response entailment consistently performs better or on par with more complex claim-level scorers, 2) claim-level scoring generally yields better results than sentence-level scoring, and 3) uncertainty-aware decoding is highly effective for improving the factuality of long-form outputs. Our framework clarifies relationships between prior methods, enables apples-to-apples comparisons, and provides practical guidance for selecting components for fine-grained UQ.
【6】All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection in LLM Backtesting
标题：所有泄漏都算在内，一些更重要：LLM回溯测试中的可解释时间污染检测
链接：https://arxiv.org/abs/2602.17234
备注：8 pages plus appendix
摘要：To evaluate whether LLMs can accurately predict future events, we need the ability to \textit{backtest} them on events that have already resolved. This requires models to reason only with information available at a specified past date. Yet LLMs may inadvertently leak post-cutoff knowledge encoded during training, undermining the validity of retrospective evaluation. We introduce a claim-level framework for detecting and quantifying this \emph{temporal knowledge leakage}. Our approach decomposes model rationales into atomic claims and categorizes them by temporal verifiability, then applies \textit{Shapley values} to measure each claim's contribution to the prediction. This yields the \textbf{Shapley}-weighted \textbf{D}ecision-\textbf{C}ritical \textbf{L}eakage \textbf{R}ate (\textbf{Shapley-DCLR}), an interpretable metric that captures what fraction of decision-driving reasoning derives from leaked information. Building on this framework, we propose \textbf{Time}-\textbf{S}upervised \textbf{P}rediction with \textbf{E}xtracted \textbf{C}laims (\textbf{TimeSPEC}), which interleaves generation with claim verification and regeneration to proactively filter temporal contamination -- producing predictions where every supporting claim can be traced to sources available before the cutoff date. Experiments on 350 instances spanning U.S. Supreme Court case prediction, NBA salary estimation, and stock return ranking reveal substantial leakage in standard prompting baselines. TimeSPEC reduces Shapley-DCLR while preserving task performance, demonstrating that explicit, interpretable claim-level verification outperforms prompt-based temporal constraints for reliable backtesting.
摘要 ：To evaluate whether LLMs can accurately predict future events, we need the ability to \textit{backtest} them on events that have already resolved. This requires models to reason only with information available at a specified past date. Yet LLMs may inadvertently leak post-cutoff knowledge encoded during training, undermining the validity of retrospective evaluation. We introduce a claim-level framework for detecting and quantifying this \emph{temporal knowledge leakage}. Our approach decomposes model rationales into atomic claims and categorizes them by temporal verifiability, then applies \textit{Shapley values} to measure each claim's contribution to the prediction. This yields the \textbf{Shapley}-weighted \textbf{D}ecision-\textbf{C}ritical \textbf{L}eakage \textbf{R}ate (\textbf{Shapley-DCLR}), an interpretable metric that captures what fraction of decision-driving reasoning derives from leaked information. Building on this framework, we propose \textbf{Time}-\textbf{S}upervised \textbf{P}rediction with \textbf{E}xtracted \textbf{C}laims (\textbf{TimeSPEC}), which interleaves generation with claim verification and regeneration to proactively filter temporal contamination -- producing predictions where every supporting claim can be traced to sources available before the cutoff date. Experiments on 350 instances spanning U.S. Supreme Court case prediction, NBA salary estimation, and stock return ranking reveal substantial leakage in standard prompting baselines. TimeSPEC reduces Shapley-DCLR while preserving task performance, demonstrating that explicit, interpretable claim-level verification outperforms prompt-based temporal constraints for reliable backtesting.
【7】Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
标题：隐私保护机制实现LLM的廉价可验证推理
链接：https://arxiv.org/abs/2602.17223
摘要：随着大型语言模型（LLM）的规模不断增长，能够在本地托管和运行模型的用户越来越少。这导致第三方托管服务的使用增加。但是，在此设置中，推理提供程序执行的计算缺乏保证。例如，一个不诚实的提供者可能会用一个运行成本更低的较弱模型来替换一个昂贵的大型模型，并将来自较弱模型的结果返回给用户。现有的验证推理的工具通常依赖于密码学的方法，如零知识证明（ZKP），但这些方法增加了大量的计算开销，并且仍然不适用于大型模型。在这项工作中，我们开发了一个新的见解-给定的方法进行私人LLM推理，可以获得形式的验证推理边际额外成本。具体来说，我们提出了两个新的协议，利用隐私保护LLM推理，以提供保证进行的推理。我们的方法很便宜，需要增加一些额外的计算令牌，并且几乎没有下游影响。由于最快的隐私保护推理方法通常比ZK方法更快，因此所提出的协议还提高了验证运行时间。我们的工作为LLM推理中隐私和可验证性之间的联系提供了新的见解。
摘要：As large language models (LLMs) continue to grow in size, fewer users are able to host and run models locally. This has led to increased use of third-party hosting services. However, in this setting, there is a lack of guarantees on the computation performed by the inference provider. For example, a dishonest provider may replace an expensive large model with a cheaper-to-run weaker model and return the results from the weaker model to the user. Existing tools to verify inference typically rely on methods from cryptography such as zero-knowledge proofs (ZKPs), but these add significant computational overhead, and remain infeasible for use for large models. In this work, we develop a new insight -- that given a method for performing private LLM inference, one can obtain forms of verified inference at marginal extra cost. Specifically, we propose two new protocols which leverage privacy-preserving LLM inference in order to provide guarantees over the inference that was carried out. Our approaches are cheap, requiring the addition of a few extra tokens of computation, and have little to no downstream impact. As the fastest privacy-preserving inference methods are typically faster than ZK methods, the proposed protocols also improve verification runtime. Our work provides novel insights into the connections between privacy and verifiability in LLM inference.
【8】Discovering Universal Activation Directions for PII Leakage in Language Models
标题：发现语言模型中PRI泄漏的通用激活方向
链接：https://arxiv.org/abs/2602.16980
备注：Pre-print
摘要：现代语言模型具有丰富的内部结构，但对隐私敏感行为（如个人身份信息（PII）泄漏）如何在其隐藏状态中表示和调制知之甚少。我们提出了UniLeak，一个机械的可解释性框架，确定通用的激活方向：潜在的方向在模型的剩余流，其在推理时的线性加法始终增加跨提示生成PII的可能性。这些特定于模型的方向在上下文中进行概括，并放大PII生成概率，对生成质量的影响最小。UniLeak在不访问训练数据或地面真实PII的情况下恢复这些方向，仅依赖于自我生成的文本。在多个模型和数据集上，与现有的基于方向的提取方法相比，沿着这些通用方向的转向大大增加了PII泄漏。我们的研究结果为PII泄漏提供了一个新的视角：在模型表示中叠加潜在信号，从而实现风险放大和缓解。
摘要：Modern language models exhibit rich internal structure, yet little is known about how privacy-sensitive behaviors, such as personally identifiable information (PII) leakage, are represented and modulated within their hidden states. We present UniLeak, a mechanistic-interpretability framework that identifies universal activation directions: latent directions in a model's residual stream whose linear addition at inference time consistently increases the likelihood of generating PII across prompts. These model-specific directions generalize across contexts and amplify PII generation probability, with minimal impact on generation quality. UniLeak recovers such directions without access to training data or groundtruth PII, relying only on self-generated text. Across multiple models and datasets, steering along these universal directions substantially increases PII leakage compared to existing prompt-based extraction methods. Our results offer a new perspective on PII leakage: the superposition of a latent signal in the model's representations, enabling both risk amplification and mitigation.
【9】Fail-Closed Alignment for Large Language Models
标题：大型语言模型的故障封闭对齐
链接：https://arxiv.org/abs/2602.16977
备注：Pre-print
摘要：我们发现了当前大型语言模型（LLM）对齐的结构性弱点：现代拒绝机制是失败开放的。虽然现有的方法在多个潜在特征中编码拒绝行为，但通过基于越狱的越狱$-$来抑制单个主导特征$-$可能会导致对齐崩溃，从而导致不安全的生成。出于这一动机，我们提出了故障关闭对齐作为鲁棒LLM安全的设计原则：拒绝机制应保持有效，即使在部分故障通过冗余，独立的因果通路。我们提出了这一原则的具体实例：一个渐进的对齐框架，迭代地识别和消除以前学习的拒绝方向，迫使模型沿着新的独立子空间重建安全。在四种越狱攻击中，我们实现了最强的整体鲁棒性，同时减轻了过度拒绝并保持了生成质量，计算开销很小。我们的机制分析证实，用我们的方法训练的模型编码了多个因果独立的拒绝方向，这些拒绝方向是基于越狱的越狱无法同时抑制的，为失败关闭对齐提供了经验支持，作为鲁棒LLM安全的原则基础。
摘要：We identify a structural weakness in current large language model (LLM) alignment: modern refusal mechanisms are fail-open. While existing approaches encode refusal behaviors across multiple latent features, suppressing a single dominant feature$-$via prompt-based jailbreaks$-$can cause alignment to collapse, leading to unsafe generation. Motivated by this, we propose fail-closed alignment as a design principle for robust LLM safety: refusal mechanisms should remain effective even under partial failures via redundant, independent causal pathways. We present a concrete instantiation of this principle: a progressive alignment framework that iteratively identifies and ablates previously learned refusal directions, forcing the model to reconstruct safety along new, independent subspaces. Across four jailbreak attacks, we achieve the strongest overall robustness while mitigating over-refusal and preserving generation quality, with small computational overhead. Our mechanistic analyses confirm that models trained with our method encode multiple, causally independent refusal directions that prompt-based jailbreaks cannot suppress simultaneously, providing empirical support for fail-closed alignment as a principled foundation for robust LLM safety.
【10】DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMs
标题：DeepContext：LLM中多回合对抗意图漂移的状态实时检测
链接：https://arxiv.org/abs/2602.16935
备注：18 Pages, 7 Tables, 1 Figure
摘要 ：虽然大型语言模型（LLM）功能已经扩展，但安全护栏在很大程度上仍然是无状态的，将多轮对话视为一系列断开连接的事件。这种时间感知的缺乏促进了“安全差距”，其中对抗策略，如Crescendo和ActorAttack，慢慢地将恶意意图渗透到回合边界以绕过无状态过滤器。我们引入DeepContext，这是一个有状态监控框架，旨在映射用户意图的时间轨迹。DeepContext放弃了孤立的评估模型，转而采用递归神经网络（RNN）架构，该架构可以摄取一系列微调的回合级嵌入。通过在对话中传播隐藏状态，DeepContext捕获了无状态模型忽略的风险的增量累积。我们的评估表明，DeepContext在多回合越狱检测方面的表现明显优于现有的基线，达到了最先进的F1得分0.84，这代表了对超大规模云提供商护栏和领先的开放权重模型（如Llama-Prompt-Guard-2（0.67）和Granite-Guardian（0.67））的实质性改进。此外，DeepContext在T4 GPU上保持低于20 ms的推理开销，确保实时应用的可行性。这些结果表明，对意图的顺序演变进行建模是部署大规模无状态模型的更有效和计算效率更高的替代方案。
摘要：While Large Language Model (LLM) capabilities have scaled, safety guardrails remain largely stateless, treating multi-turn dialogues as a series of disconnected events. This lack of temporal awareness facilitates a "Safety Gap" where adversarial tactics, like Crescendo and ActorAttack, slowly bleed malicious intent across turn boundaries to bypass stateless filters. We introduce DeepContext, a stateful monitoring framework designed to map the temporal trajectory of user intent. DeepContext discards the isolated evaluation model in favor of a Recurrent Neural Network (RNN) architecture that ingests a sequence of fine-tuned turn-level embeddings. By propagating a hidden state across the conversation, DeepContext captures the incremental accumulation of risk that stateless models overlook. Our evaluation demonstrates that DeepContext significantly outperforms existing baselines in multi-turn jailbreak detection, achieving a state-of-the-art F1 score of 0.84, which represents a substantial improvement over both hyperscaler cloud-provider guardrails and leading open-weight models such as Llama-Prompt-Guard-2 (0.67) and Granite-Guardian (0.67). Furthermore, DeepContext maintains a sub-20ms inference overhead on a T4 GPU, ensuring viability for real-time applications. These results suggest that modeling the sequential evolution of intent is a more effective and computationally efficient alternative to deploying massive, stateless models.
【11】LLM-WikiRace: Benchmarking Long-term Planning and Reasoning over Real-World Knowledge Graphs
标题：LLM-WikiRace：对现实世界知识图进行长期规划和推理的基准
链接：https://arxiv.org/abs/2602.16902
摘要：我们介绍LLM-Wikirace，一个用于评估大型语言模型（LLM）中的规划，推理和世界知识的基准。在LLM-Wikirace中，模型必须一步一步地有效地导航维基百科超链接，以从给定的源到达目标页面，这需要前瞻性规划和推理概念在现实世界中如何连接的能力。我们评估了一系列开源和闭源模型，包括Gemini-3、GPT-5和Claude Opus 4.5，这些模型在任务的简单级别上实现了最强的结果，并展示了超人的性能。尽管如此，性能在硬难度上急剧下降：性能最好的模型Gemini-3仅在23%的硬难度游戏中成功，突出了前沿模型的大量剩余挑战。我们的分析表明，世界知识是成功的必要因素，但只有在一定程度上，超过这个门槛，规划和长期的推理能力成为主导因素。轨迹级分析进一步揭示，即使是最强大的模型也很难在失败后重新规划，经常进入循环而不是恢复。LLM-Wikirace是一个简单的基准测试，揭示了当前推理系统的明显局限性，提供了一个开放的舞台，其中具有规划能力的LLM仍然有很多需要证明的地方。我们的代码和排行榜可以在https：/llmwikirace.github.io上找到。
摘要：We introduce LLM-Wikirace, a benchmark for evaluating planning, reasoning, and world knowledge in large language models (LLMs). In LLM-Wikirace, models must efficiently navigate Wikipedia hyperlinks step by step to reach a target page from a given source, requiring look-ahead planning and the ability to reason about how concepts are connected in the real world. We evaluate a broad set of open- and closed-source models, including Gemini-3, GPT-5, and Claude Opus 4.5, which achieve the strongest results on the easy level of the task and demonstrate superhuman performance. Despite this, performance drops sharply on hard difficulty: the best-performing model, Gemini-3, succeeds in only 23\% of hard games, highlighting substantial remaining challenges for frontier models. Our analysis shows that world knowledge is a necessary ingredient for success, but only up to a point, beyond this threshold, planning and long-horizon reasoning capabilities become the dominant factors. Trajectory-level analysis further reveals that even the strongest models struggle to replan after failure, frequently entering loops rather than recovering. LLM-Wikirace is a simple benchmark that reveals clear limitations in current reasoning systems, offering an open arena where planning-capable LLMs still have much to prove. Our code and leaderboard available at https:/llmwikirace.github.io.
【12】NeST: Neuron Selective Tuning for LLM Safety
标题：NeST：LLM安全性的神经元选择性调整
链接：https://arxiv.org/abs/2602.16835
摘要：安全对齐对于大型语言模型（LLM）的负责任部署至关重要。然而，现有的方法往往依赖于重量级的微调，这是昂贵的更新，审计和维护跨模型家族。完全微调会产生大量的计算和存储开销，而参数高效的方法，如LoRA交易效率不一致的安全增益和设计选择的敏感性。安全干预机制（如断路器）在不修改模型权重的情况下减少不安全输出，但不直接塑造或保留管理安全行为的内部表示。这些限制阻碍了快速可靠的安全更新，特别是在模型频繁演变或必须适应新策略和领域的环境中。 我们提出了NeST，一个轻量级的，结构感知的安全对齐框架，通过选择性地适应一小部分安全相关的神经元，同时冻结模型的其余部分来加强拒绝行为。NeST通过对功能一致的安全神经元进行聚类并在每个聚类中强制执行共享更新，使参数更新与安全行为的内部组织保持一致，从而实现有针对性的稳定安全适应，而无需广泛的模型修改或推理时间开销。我们针对三个主要基线对NeST进行基准测试：完全微调，基于LoRA的微调，以及跨越多个模型系列和尺寸的10个开放重量LLM的断路器。在所有评估的模型中，NeST将攻击成功率从平均44.5%降低到4.36%，相当于不安全生成减少了90.2%，而平均只需要44万个可训练参数。与完全微调相比，更新参数减少了17，310倍，相对于LoRA减少了9.25倍，同时始终实现了更强的对线安全性能。
摘要：Safety alignment is essential for the responsible deployment of large language models (LLMs). Yet, existing approaches often rely on heavyweight fine-tuning that is costly to update, audit, and maintain across model families. Full fine-tuning incurs substantial computational and storage overhead, while parameter-efficient methods such as LoRA trade efficiency for inconsistent safety gains and sensitivity to design choices. Safety intervention mechanisms such as circuit breakers reduce unsafe outputs without modifying model weights, but do not directly shape or preserve the internal representations that govern safety behavior. These limitations hinder rapid and reliable safety updates, particularly in settings where models evolve frequently or must adapt to new policies and domains. We present NeST, a lightweight, structure-aware safety alignment framework that strengthens refusal behavior by selectively adapting a small subset of safety-relevant neurons while freezing the remainder of the model. NeST aligns parameter updates with the internal organization of safety behavior by clustering functionally coherent safety neurons and enforcing shared updates within each cluster, enabling targeted and stable safety adaptation without broad model modification or inference-time overhead. We benchmark NeST against three dominant baselines: full fine-tuning, LoRA-based fine-tuning, and circuit breakers across 10 open-weight LLMs spanning multiple model families and sizes. Across all evaluated models, NeST reduces the attack success rate from an average of 44.5% to 4.36%, corresponding to a 90.2% reduction in unsafe generations, while requiring only 0.44 million trainable parameters on average. This amounts to a 17,310x decrease in updated parameters compared to full fine-tuning and a 9.25x reduction relative to LoRA, while consistently achieving stronger safety performance for alignment.
【13】References Improve LLM Alignment in Non-Verifiable Domains
标题：参考文献改善不可验证领域的LLM对齐
链接：https://arxiv.org/abs/2602.16802
备注：ICLR 2026 Camera Ready
摘要：虽然具有可验证奖励的强化学习（RLVR）在推理任务中表现出很强的有效性，但它不能直接应用于缺乏地面实况验证器的不可验证领域，例如LLM对齐。在这项工作中，我们调查是否参考引导LLM评估可以弥合这一差距，作为软“验证”。首先，我们设计的评估协议，提高基于LLM的评估LLM对齐使用参考输出。通过全面的实验，我们表明，参考指导的方法大大提高了使用来自前沿模型的参考的能力较低的LLM法官的准确性;更强的LLM法官也可以通过高质量（即，人写的）参考。在这些改进的法官的基础上，我们证明了高质量的参考在对齐调整，其中LLM指导与参考被用作法官自我改进的效用。我们发现，参考引导的自我改进产生了明显的收益，直接SFT的参考输出和自我改进与无参考的法官，实现性能与ArmoRM，一个强大的微调奖励模型的培训。具体来说，我们的方法在AlpacaEval和Arena-Hard上使用Llama-3-8B-Instruct实现了73.1%和58.7%，在Qwen2.5- 7 B上实现了70.0%和74.1%，对应于SFT蒸馏的平均绝对增益为+20.2 / +17.1点，在AlpacaEval / Arena-Hard上的无参考自我改进为+5.3 / +3.6点。这些结果突出了使用参考指导的LLM评估器，使有效的LLM后培训在非可验证的域的潜力。
摘要：While Reinforcement Learning with Verifiable Rewards (RLVR) has shown strong effectiveness in reasoning tasks, it cannot be directly applied to non-verifiable domains lacking ground-truth verifiers, such as LLM alignment. In this work, we investigate whether reference-guided LLM-evaluators can bridge this gap by serving as soft "verifiers". First, we design evaluation protocols that enhance LLM-based evaluators for LLM alignment using reference outputs. Through comprehensive experiments, we show that a reference-guided approach substantially improves the accuracy of less capable LLM-judges using references from frontier models; stronger LLM-judges can also be enhanced by high-quality (i.e., human-written) references. Building on these improved judges, we demonstrate the utility of high-quality references in alignment tuning, where LLMs guided with references are used as judges to self-improve. We show that reference-guided self-improvement yields clear gains over both direct SFT on reference outputs and self-improvement with reference-free judges, achieving performance comparable to training with ArmoRM, a strong finetuned reward model. Specifically, our method achieves 73.1% and 58.7% on AlpacaEval and Arena-Hard with Llama-3-8B-Instruct, and 70.0% and 74.1% with Qwen2.5-7B, corresponding to average absolute gains of +20.2 / +17.1 points over SFT distillation and +5.3 / +3.6 points over reference-free self-improvement on AlpacaEval / Arena-Hard. These results highlight the potential of using reference-guided LLM-evaluators to enable effective LLM post-training in non-verifiable domains.
【14】Large-scale online deanonymization with LLMs
标题：利用LLM进行大规模在线去匿名化
链接：https://arxiv.org/abs/2602.16800
备注：24 pages, 10 figures
摘要：我们证明了大型语言模型可以用于执行大规模去匿名化。通过完全的互联网访问，我们的代理可以高精度地重新识别Hacker News用户和Anthropic Interviewer参与者，仅考虑匿名的在线个人资料和对话，这与专门的人类调查员需要花费数小时的时间相匹配。然后，我们为封闭世界环境设计攻击。给定两个匿名个体的数据库，每个数据库都包含由该个体编写或关于该个体的非结构化文本，我们实现了一个可扩展的攻击管道，该管道使用LLM来：（1）提取身份相关特征，（2）通过语义嵌入搜索候选匹配，以及（3）对顶级候选进行推理以验证匹配并减少误报。与先前的去匿名化工作（例如，Netflix奖），需要结构化数据或手动功能工程，我们的方法直接适用于任意平台上的原始用户内容。我们用已知的地面实况数据构建了三个数据集来评估我们的攻击。第一个链接Hacker News到LinkedIn配置文件，使用出现在配置文件中的跨平台引用。我们的第二个数据集匹配Reddit电影讨论社区的用户;第三个数据集将单个用户的Reddit历史记录及时分割，以创建两个要匹配的匿名配置文件。在每种情况下，基于LLM的方法都大大优于经典基线，在90%的精确度下实现了高达68%的召回率，而最好的非LLM方法接近0%。我们的研究结果表明，实际的模糊保护匿名用户在线不再持有和在线隐私的威胁模型需要重新考虑。
摘要：We show that large language models can be used to perform at-scale deanonymization. With full Internet access, our agent can re-identify Hacker News users and Anthropic Interviewer participants at high precision, given pseudonymous online profiles and conversations alone, matching what would take hours for a dedicated human investigator. We then design attacks for the closed-world setting. Given two databases of pseudonymous individuals, each containing unstructured text written by or about that individual, we implement a scalable attack pipeline that uses LLMs to: (1) extract identity-relevant features, (2) search for candidate matches via semantic embeddings, and (3) reason over top candidates to verify matches and reduce false positives. Compared to prior deanonymization work (e.g., on the Netflix prize) that required structured data or manual feature engineering, our approach works directly on raw user content across arbitrary platforms. We construct three datasets with known ground-truth data to evaluate our attacks. The first links Hacker News to LinkedIn profiles, using cross-platform references that appear in the profiles. Our second dataset matches users across Reddit movie discussion communities; and the third splits a single user's Reddit history in time to create two pseudonymous profiles to be matched. In each setting, LLM-based methods substantially outperform classical baselines, achieving up to 68% recall at 90% precision compared to near 0% for the best non-LLM method. Our results show that the practical obscurity protecting pseudonymous users online no longer holds and that threat models for online privacy need to be reconsidered.
【15】Omitted Variable Bias in Language Models Under Distribution Shift
标题：分布转移下语言模型中省略的变量偏差
链接：https://arxiv.org/abs/2602.16784
摘要：尽管现代语言模型在各种各样的任务上表现令人印象深刻，但它们仍然容易受到分布变化的影响，当对分布与训练数据不同的数据进行评估时，表现出脆弱的行为。在本文中，我们描述了如何在语言模型中的分布变化可以分为可观察和不可观察的组件，我们讨论了如何建立的方法来处理分布变化地址只有前者。重要的是，我们发现，由此产生的遗漏变量偏差未观察到的变量可以损害语言模型的评估和优化。为了解决这一挑战，我们引入了一个框架，该框架将省略的变量的强度映射到分布偏移下语言模型的最坏情况泛化性能的界限。在实证实验中，我们表明，直接在语言模型评估和优化中使用这些界限提供了更有原则的分布外性能指标，相对于标准分布偏移调整方法提高了真正的分布外性能，并进一步实现了对目标分布标签可用时省略变量的强度的推断。
摘要：Despite their impressive performance on a wide variety of tasks, modern language models remain susceptible to distribution shifts, exhibiting brittle behavior when evaluated on data that differs in distribution from their training data. In this paper, we describe how distribution shifts in language models can be separated into observable and unobservable components, and we discuss how established approaches for dealing with distribution shift address only the former. Importantly, we identify that the resulting omitted variable bias from unobserved variables can compromise both evaluation and optimization in language models. To address this challenge, we introduce a framework that maps the strength of the omitted variables to bounds on the worst-case generalization performance of language models under distribution shift. In empirical experiments, we show that using these bounds directly in language model evaluation and optimization provides more principled measures of out-of-distribution performance, improves true out-of-distribution performance relative to standard distribution shift adjustment methods, and further enables inference about the strength of the omitted variables when target distribution labels are available.
【16】Can Adversarial Code Comments Fool AI Security Reviewers -- Large-Scale Empirical Study of Comment-Based Attacks and Defenses Against LLM Code Analysis
标题：对抗性代码评论可以愚弄人工智能安全评论员--针对LLM代码分析的基于评论的攻击和防御的大规模实证研究
链接：https://arxiv.org/abs/2602.16741
备注：19 pages, 6 figures
摘要：人工智能辅助的代码审查被广泛用于在生产发布之前检测漏洞。先前的工作表明，对抗性提示操作会降低代码生成中的大型语言模型（LLM）性能。我们测试是否类似的基于评论的操纵误导LLM在漏洞检测。我们构建了一个跨Python、JavaScript和Java的100个样本的基准测试，每个样本都有8个注释变体，从没有注释到对抗性策略，如权威欺骗和技术欺骗。八个前沿模型，五个商业和三个开源，在9,366个试验中进行了评估。对抗性评论对检测准确性产生很小的、统计学上不显著的影响（McNemar精确p > 0.21;所有95%的置信区间包括零）。这适用于基线检测率为89%至96%的商业模型和53%至72%的开源模型，尽管存在很大的绝对性能差距。与注释操作实现高攻击成功率的生成设置不同，检测性能不会有意义地降低。更复杂的对抗策略并不比简单的操纵性评论更有优势。我们在4，646次额外试验（总共14，012次）中测试了四种自动化防御。静态分析交叉引用在96.9%的检测率下表现最好，并恢复了47%的基线未命中。注释剥离通过删除有用的上下文来减少对较弱模型的检测。失败集中在固有的困难漏洞类，包括竞争条件，定时侧通道和复杂的授权逻辑，而不是对抗性评论。
摘要 ：AI-assisted code review is widely used to detect vulnerabilities before production release. Prior work shows that adversarial prompt manipulation can degrade large language model (LLM) performance in code generation. We test whether similar comment-based manipulation misleads LLMs during vulnerability detection. We build a 100-sample benchmark across Python, JavaScript, and Java, each paired with eight comment variants ranging from no comments to adversarial strategies such as authority spoofing and technical deception. Eight frontier models, five commercial and three open-source, are evaluated in 9,366 trials. Adversarial comments produce small, statistically non-significant effects on detection accuracy (McNemar exact p > 0.21; all 95 percent confidence intervals include zero). This holds for commercial models with 89 to 96 percent baseline detection and open-source models with 53 to 72 percent, despite large absolute performance gaps. Unlike generation settings where comment manipulation achieves high attack success, detection performance does not meaningfully degrade. More complex adversarial strategies offer no advantage over simple manipulative comments. We test four automated defenses across 4,646 additional trials (14,012 total). Static analysis cross-referencing performs best at 96.9 percent detection and recovers 47 percent of baseline misses. Comment stripping reduces detection for weaker models by removing helpful context. Failures concentrate on inherently difficult vulnerability classes, including race conditions, timing side channels, and complex authorization logic, rather than on adversarial comments.
【17】Quantifying LLM Attention-Head Stability: Implications for Circuit Universality
标题：量化LLM注意力稳定性：对电路普遍性的影响
链接：https://arxiv.org/abs/2602.16740
备注：Main Body: 8 pages, Total length: 33 pages, Code repo: https://github.com/karanbali/attention_head_seed_stability , Weights repo: https://huggingface.co/karanbali/attention_head_seed_stability
摘要：在机械的可解释性，最近的工作仔细检查Transformer“电路”-稀疏，单层或多层子计算，这可能反映人类可理解的功能。然而，这些网络电路很少在同一深度学习架构的不同实例中进行酸性测试。如果没有这一点，目前尚不清楚报告的电路是否在实验室中普遍出现，或者是否对特定的估计实例具有特异性，从而可能限制安全关键设置的置信度。在这里，我们系统地研究在日益复杂的各种规模的Transformer语言模型的稳定性交叉改装。我们一层一层地量化了注意力头在独立初始化的训练运行中如何学习表示。我们严格的实验表明：（1）中间层头部是最不稳定的，但最具代表性的不同;（2）更深的模型表现出更强的中间深度发散;（3）更深层的不稳定头部变得比来自同一层的同行更重要;（4）应用权重衰减优化大大提高了随机模型初始化的注意力头部稳定性;（5）余流比较稳定。我们的研究结果确立了电路的跨实例鲁棒性是可扩展监督的一个重要但未得到充分重视的先决条件，围绕AI系统可能的白盒可监控性绘制了轮廓。
摘要：In mechanistic interpretability, recent work scrutinizes transformer "circuits" - sparse, mono or multi layer sub computations, that may reflect human understandable functions. Yet, these network circuits are rarely acid-tested for their stability across different instances of the same deep learning architecture. Without this, it remains unclear whether reported circuits emerge universally across labs or turn out to be idiosyncratic to a particular estimation instance, potentially limiting confidence in safety-critical settings. Here, we systematically study stability across-refits in increasingly complex transformer language models of various sizes. We quantify, layer by layer, how similarly attention heads learn representations across independently initialized training runs. Our rigorous experiments show that (1) middle-layer heads are the least stable yet the most representationally distinct; (2) deeper models exhibit stronger mid-depth divergence; (3) unstable heads in deeper layers become more functionally important than their peers from the same layer; (4) applying weight decay optimization substantially improves attention-head stability across random model initializations; and (5) the residual stream is comparatively stable. Our findings establish the cross-instance robustness of circuits as an essential yet underappreciated prerequisite for scalable oversight, drawing contours around possible white-box monitorability of AI systems.
【18】A Few-Shot LLM Framework for Extreme Day Classification in Electricity Markets
标题：电力市场极端日分类的Few-ShotLLM框架
链接：https://arxiv.org/abs/2602.16735
摘要：本文提出了一种基于大语言模型（LLM）的Few-Shot分类框架，以预测第二天是否会出现实时电价飙升。该方法将系统状态信息（包括电力需求、可再生能源发电、天气预报和最近的电价）汇总到一组统计特征中，这些特征被格式化为自然语言提示，并与一般指令一起提供给LLM。然后，该模型确定第二天将是尖峰日的可能性，并报告置信度得分。使用来自德克萨斯州电力市场的历史数据，我们证明了这种Few-Shot方法的性能与监督机器学习模型（如支持向量机和XGBoost）相当，并且在历史数据有限的情况下优于后两者。这些发现凸显了LLM作为数据高效工具的潜力，可以在数据稀缺的情况下对电价峰值进行分类。
摘要：This paper proposes a few-shot classification framework based on Large Language Models (LLMs) to predict whether the next day will have spikes in real-time electricity prices. The approach aggregates system state information, including electricity demand, renewable generation, weather forecasts, and recent electricity prices, into a set of statistical features that are formatted as natural-language prompts and fed to an LLM along with general instructions. The model then determines the likelihood that the next day would be a spike day and reports a confidence score. Using historical data from the Texas electricity market, we demonstrate that this few-shot approach achieves performance comparable to supervised machine learning models, such as Support Vector Machines and XGBoost, and outperforms the latter two when limited historical data are available. These findings highlight the potential of LLMs as a data-efficient tool for classifying electricity price spikes in settings with scarce data.
【19】Mobility-Aware Cache Framework for Scalable LLM-Based Human Mobility Simulation
标题：用于可扩展的基于LLM的人类移动模拟的移动感知缓存框架
链接：https://arxiv.org/abs/2602.16727
摘要：大规模的人类流动模拟对于城市规划、流行病学和交通分析等应用至关重要。最近的作品处理大型语言模型（LLM）作为人类代理，模拟现实的移动行为，使用结构化推理，但其高计算成本限制了可扩展性。为了解决这个问题，我们设计了一个名为MobCache的移动感知缓存框架，它利用可重构缓存来实现高效的大规模人类移动模拟。它包括：（1）推理组件，其将每个推理步骤编码为潜在空间嵌入，并使用潜在空间评估器来实现推理步骤的重用和重组;以及（2）解码组件，其采用利用迁移率定律约束的提取训练的轻量级解码器来将潜在空间推理链翻译成自然语言，从而在保持保真度的同时提高仿真效率。实验表明，MobCache在多个维度上显着提高了效率，同时保持了与最先进的基于LLM的方法相当的性能。
摘要：Large-scale human mobility simulation is critical for applications such as urban planning, epidemiology, and transportation analysis. Recent works treat large language models (LLMs) as human agents to simulate realistic mobility behaviors using structured reasoning, but their high computational cost limits scalability. To address this, we design a mobility-aware cache framework named MobCache that leverages reconstructible caches to enable efficient large-scale human mobility simulations. It consists of: (1) a reasoning component that encodes each reasoning step as a latent-space embedding and uses a latent-space evaluator to enable the reuse and recombination of reasoning steps; and (2) a decoding component that employs a lightweight decoder trained with mobility law-constrained distillation to translate latent-space reasoning chains into natural language, thereby improving simulation efficiency while maintaining fidelity. Experiments show that MobCache significantly improves efficiency across multiple dimensions while maintaining performance comparable to state-of-the-art LLM-based methods.
【20】Multi-Objective Alignment of Language Models for Personalized Psychotherapy
标题：个性化心理治疗语言模型的多目标对齐
链接：https://arxiv.org/abs/2602.16053
摘要：心理健康障碍影响着全世界10亿多人，但由于劳动力短缺和成本限制，获得护理的机会仍然有限。虽然人工智能系统显示出治疗前景，但目前的对齐方法独立地优化了目标，未能平衡患者的偏好与临床安全性。我们调查了335名有心理健康经历的人，收集了治疗维度的偏好排名，然后使用直接偏好优化开发了一个多目标对齐框架。我们训练奖励模型的六个标准-移情，安全，积极倾听，自我激励的变化，信任/融洽，和病人的自主性-和系统地比较多目标的方法与单目标优化，监督微调，参数合并。与单目标优化（93.6%同理心，47.8%安全性）相比，多目标DPO（MODPO）实现了更好的平衡（77.6%同理心，62.6%安全性），治疗标准优于一般沟通原则17.2%。盲态临床医生评价证实MODPO始终是首选，LLM-评价者一致性与临床医生间可靠性相当。
摘要：Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective alignment framework using direct preference optimization. We train reward models for six criteria -- empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy -- and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability.
Graph相关(图学习|图神经网络|图优化等)(7篇)
【1】From Subtle to Significant: Prompt-Driven Self-Improving Optimization in Test-Time Graph OOD Detection
标题：从微妙到重要：测试时图OOD检测中预算驱动的自我改进优化
链接：https://arxiv.org/abs/2602.17342
备注：9pages, 5 figures
摘要：图分布外（OOD）检测旨在识别测试图是否偏离训练期间观察到的图的分布，这对于确保图神经网络（GNN）在开放世界场景中部署时的可靠性至关重要。图OOD检测的最新进展集中在测试时训练技术上，该技术在不访问潜在的监督信息（例如，训练数据）。然而，这些方法中的大多数采用一次通过推理范式，这阻止了它们逐步纠正错误的预测以放大OOD信号。为此，我们提出了一个\textbf {S} elf-\textbf {I} improving\textbf {G} elf\textbf {O} ut-\textbf {o} f-\textbf {D}干扰检测器（SIGOD），这是一个无监督的框架，将连续自学习与测试时间训练相结合，用于有效的图OOD检测。具体来说，SIGOD生成一个提示，以构建一个放大潜在OOD信号的增强图。为了优化提示，SIGOOD引入了能量偏好优化（EPO）损失，它利用了原始测试图和增强图之间的能量变化。通过迭代优化提示，通过将其纳入自我改进循环中的检测模型，最终将得到的最佳的提示增强图用于OOD检测。对21个真实数据集的综合评估证实了我们的SIGOD方法的有效性和优越性。代码在www.example.com。
摘要：Graph Out-of-Distribution (OOD) detection aims to identify whether a test graph deviates from the distribution of graphs observed during training, which is critical for ensuring the reliability of Graph Neural Networks (GNNs) when deployed in open-world scenarios. Recent advances in graph OOD detection have focused on test-time training techniques that facilitate OOD detection without accessing potential supervisory information (e.g., training data). However, most of these methods employ a one-pass inference paradigm, which prevents them from progressively correcting erroneous predictions to amplify OOD signals. To this end, we propose a \textbf{S}elf-\textbf{I}mproving \textbf{G}raph \textbf{O}ut-\textbf{o}f-\textbf{D}istribution detector (SIGOOD), which is an unsupervised framework that integrates continuous self-learning with test-time training for effective graph OOD detection. Specifically, SIGOOD generates a prompt to construct a prompt-enhanced graph that amplifies potential OOD signals. To optimize prompts, SIGOOD introduces an Energy Preference Optimization (EPO) loss, which leverages energy variations between the original test graph and the prompt-enhanced graph. By iteratively optimizing the prompt by involving it into the detection model in a self-improving loop, the resulting optimal prompt-enhanced graph is ultimately used for OOD detection. Comprehensive evaluations on 21 real-world datasets confirm the effectiveness and outperformance of our SIGOOD method. The code is at https://github.com/Ee1s/SIGOOD.
【2】RLGT: A reinforcement learning framework for extremal graph theory
标题：WLGT：极端图理论的强化学习框架
链接：https://arxiv.org/abs/2602.17276
摘要：强化学习（RL）是机器学习的一个子领域，专注于开发能够随着时间的推移自主学习最佳决策策略的模型。在最近的一篇开创性论文中，Wagner展示了如何将深度交叉熵RL方法应用于解决极值图论中的各种问题，将它们重新表述为组合优化问题。随后，许多研究人员对改进和扩展Wagner引入的框架感兴趣，从而创建了各种专门用于图论的RL环境。此外，通过使用RL解决了极值图论中的一些问题。特别地，几个关于图的拉普拉斯谱半径的不等式被反驳，对于某些Ramsey数得到了新的下界，并且对Turán型极值问题做出了贡献，其中禁止结构是长度为3和4的圈。在这里，我们提出了图论强化学习（RLGT），这是一个新的强化学习框架，它系统化了以前的工作，并为无向图和有向图提供支持，有或没有循环，以及任意数量的边缘颜色。该框架有效地表示图形，旨在通过优化的计算性能和干净的模块化设计，促进未来基于RL的极值图论研究。
摘要：Reinforcement learning (RL) is a subfield of machine learning that focuses on developing models that can autonomously learn optimal decision-making strategies over time. In a recent pioneering paper, Wagner demonstrated how the Deep Cross-Entropy RL method can be applied to tackle various problems from extremal graph theory by reformulating them as combinatorial optimization problems. Subsequently, many researchers became interested in refining and extending the framework introduced by Wagner, thereby creating various RL environments specialized for graph theory. Moreover, a number of problems from extremal graph theory were solved through the use of RL. In particular, several inequalities concerning the Laplacian spectral radius of graphs were refuted, new lower bounds were obtained for certain Ramsey numbers, and contributions were made to the Turán-type extremal problem in which the forbidden structures are cycles of length three and four. Here, we present Reinforcement Learning for Graph Theory (RLGT), a novel RL framework that systematizes the previous work and provides support for both undirected and directed graphs, with or without loops, and with an arbitrary number of edge colors. The framework efficiently represents graphs and aims to facilitate future RL-based research in extremal graph theory through optimized computational performance and a clean and modular design.
【3】AdvSynGNN: Structure-Adaptive Graph Neural Nets via Adversarial Synthesis and Self-Corrective Propagation
标题：AdvSynGNN：通过对抗合成和自校正传播的结构自适应图神经网络
链接：https://arxiv.org/abs/2602.17071
备注：32 pages, 8 figures
摘要 ：图神经网络在遇到结构噪声或非同质拓扑时，经常会遇到显着的性能下降。为了解决这些系统性漏洞，我们提出了AdvSynGNN，这是一个为弹性节点级表示学习而设计的综合架构。所提出的框架编排多分辨率结构合成对比目标，建立几何敏感的初始化。我们开发了一个Transformer骨干，通过学习拓扑信号调节注意力机制，自适应地适应异质性。我们的贡献的核心是一个集成的对抗传播引擎，其中生成组件识别潜在的连接性改变，同时增强全局一致性。此外，标签细化是通过每节点的信心指标，这有利于精确控制迭代稳定性指导下的残差校正方案。经验评估表明，这种协同方法有效地优化了不同图形分布的预测准确性，同时保持计算效率。该研究的结论是实际的实施协议，以确保在大规模环境中的AdvSynGNN系统的强大部署。
摘要：Graph neural networks frequently encounter significant performance degradation when confronted with structural noise or non-homophilous topologies. To address these systemic vulnerabilities, we present AdvSynGNN, a comprehensive architecture designed for resilient node-level representation learning. The proposed framework orchestrates multi-resolution structural synthesis alongside contrastive objectives to establish geometry-sensitive initializations. We develop a transformer backbone that adaptively accommodates heterophily by modulating attention mechanisms through learned topological signals. Central to our contribution is an integrated adversarial propagation engine, where a generative component identifies potential connectivity alterations while a discriminator enforces global coherence. Furthermore, label refinement is achieved through a residual correction scheme guided by per-node confidence metrics, which facilitates precise control over iterative stability. Empirical evaluations demonstrate that this synergistic approach effectively optimizes predictive accuracy across diverse graph distributions while maintaining computational efficiency. The study concludes with practical implementation protocols to ensure the robust deployment of the AdvSynGNN system in large-scale environments.
【4】Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learning
标题：字形图策略：多智能体强化学习中的学习动作协同依赖
链接：https://arxiv.org/abs/2602.17009
摘要：协调行动是多智能体强化学习（MARL）中最基本的合作形式。成功的去中心化决策通常不仅取决于良好的个体行为，还取决于在代理之间选择兼容的行为来同步行为，避免冲突，并满足全局约束。在本文中，我们提出了行动图政策（AGP），模型代理的可用动作选择之间的依赖关系。它构造了我们称之为\textit{协调上下文}的东西，使代理能够根据全局动作依赖性来决定他们的决策。从理论上讲，我们表明，AGPs诱导严格更具表现力的联合政策相比，完全独立的政策，可以实现协调的联合行动，证明是更优化的贪婪执行，甚至从集中的值分解方法。经验上，我们表明，AGP实现了80- 95%的成功与部分可观察性和反协调处罚的典型协调任务，而其他MARL方法仅达到10- 25%。我们进一步证明，AGP始终优于这些基线在不同的多智能体环境。
摘要：Coordinating actions is the most fundamental form of cooperation in multi-agent reinforcement learning (MARL). Successful decentralized decision-making often depends not only on good individual actions, but on selecting compatible actions across agents to synchronize behavior, avoid conflicts, and satisfy global constraints. In this paper, we propose Action Graph Policies (AGP), that model dependencies among agents' available action choices. It constructs, what we call, \textit{coordination contexts}, that enable agents to condition their decisions on global action dependencies. Theoretically, we show that AGPs induce a strictly more expressive joint policy compared to fully independent policies and can realize coordinated joint actions that are provably more optimal than greedy execution even from centralized value-decomposition methods. Empirically, we show that AGP achieves 80-95\% success on canonical coordination tasks with partial observability and anti-coordination penalties, where other MARL methods reach only 10-25\%. We further demonstrate that AGP consistently outperforms these baselines in diverse multi-agent environments.
【5】Neural Proposals, Symbolic Guarantees: Neuro-Symbolic Graph Generation with Hard Constraints
标题：神经提议、符号保证：具有硬约束的神经符号图生成
链接：https://arxiv.org/abs/2602.16954
备注：18 pages, 6 figures
摘要：我们挑战了用于分子和图形生成的黑盒纯深度神经方法，这些方法在可控性方面受到限制，并且缺乏正式的保证。我们介绍了神经符号图生成建模（NSGGM），神经符号框架，重新接近分子生成作为一个支架和互动学习任务与符号组装。自回归神经模型提出了支架并改进了相互作用信号，CPU高效的SMT求解器构建了完整的图形，同时执行化学有效性，结构规则和用户特定的约束，产生了纯神经方法无法提供的构建和可解释控制正确的分子。NSGGM在无约束生成和约束生成任务上都提供了强大的性能，表明神经符号建模可以匹配最先进的生成性能，同时提供明确的可控性和保证。为了评估更细致入微的可控性，我们还引入了逻辑约束分子基准，旨在测试严格的硬规则满足工作流程，需要明确的，可解释的规范以及可验证的合规性。
摘要：We challenge black-box purely deep neural approaches for molecules and graph generation, which are limited in controllability and lack formal guarantees. We introduce Neuro-Symbolic Graph Generative Modeling (NSGGM), a neurosymbolic framework that reapproaches molecule generation as a scaffold and interaction learning task with symbolic assembly. An autoregressive neural model proposes scaffolds and refines interaction signals, and a CPU-efficient SMT solver constructs full graphs while enforcing chemical validity, structural rules, and user-specific constraints, yielding molecules that are correct by construction and interpretable control that pure neural methods cannot provide. NSGGM delivers strong performance on both unconstrained generation and constrained generation tasks, demonstrating that neuro-symbolic modeling can match state-of-the-art generative performance while offering explicit controllability and guarantees. To evaluate more nuanced controllability, we also introduce a Logical-Constraint Molecular Benchmark, designed to test strict hard-rule satisfaction in workflows that require explicit, interpretable specifications together with verifiable compliance.
【6】Beyond Message Passing: A Symbolic Alternative for Expressive and Interpretable Graph Learning
标题：超越消息传递：表达性和可解释性图形学习的象征性替代方案
链接：https://arxiv.org/abs/2602.16947
备注：23 pages, 9 pages
摘要：图神经网络（GNN）在药物发现等高风险领域已经变得至关重要，但其黑箱性质仍然是可信度的重要障碍。虽然可自我解释的GNN试图弥合这一差距，但它们通常依赖于继承基本限制的标准消息传递骨干，包括1-Weisfeiler-Lehman（1-WL）表达障碍和缺乏细粒度可解释性。为了应对这些挑战，我们提出了SymGraph，一个旨在超越这些限制的符号框架。通过用离散结构散列和基于拓扑角色的聚合替换连续消息传递，我们的架构理论上超越了1-WL障碍，实现了卓越的表达能力，而无需微分优化的开销。广泛的实证评估表明，SymGraph实现了最先进的性能，优于现有的自解释GNN。值得注意的是，SymGraph仅使用CPU执行就可以在训练时间内提供10倍到100倍的加速。此外，与现有的基于规则的方法相比，SymGraph生成的规则具有更高的语义粒度，为科学发现和可解释的AI提供了巨大的潜力。
摘要：Graph Neural Networks (GNNs) have become essential in high-stakes domains such as drug discovery, yet their black-box nature remains a significant barrier to trustworthiness. While self-explainable GNNs attempt to bridge this gap, they often rely on standard message-passing backbones that inherit fundamental limitations, including the 1-Weisfeiler-Lehman (1-WL) expressivity barrier and a lack of fine-grained interpretability. To address these challenges, we propose SymGraph, a symbolic framework designed to transcend these constraints. By replacing continuous message passing with discrete structural hashing and topological role-based aggregation, our architecture theoretically surpasses the 1-WL barrier, achieving superior expressiveness without the overhead of differentiable optimization. Extensive empirical evaluations demonstrate that SymGraph achieves state-of-the-art performance, outperforming existing self-explainable GNNs. Notably, SymGraph delivers 10x to 100x speedups in training time using only CPU execution. Furthermore, SymGraph generates rules with superior semantic granularity compared to existing rule-based methods, offering great potential for scientific discovery and explainable AI.
【7】Semi-Supervised Learning on Graphs using Graph Neural Networks
标题：使用图神经网络进行图的半监督学习
链接：https://arxiv.org/abs/2602.17115
备注：57 pages, 7 figures
摘要：图神经网络（GNN）在半监督节点回归中表现得非常好，但仍然缺乏解释它们何时以及为什么成功的严格理论。为了解决这个差距，我们研究了一个聚合和读出模型，其中包括几个常见的消息传递架构：节点功能首先在图上传播，然后通过非线性函数映射到响应。对于具有线性图卷积和深度ReLU读出的GNN上的最小二乘估计，我们证明了一个尖锐的非渐近风险界，该界将近似，随机和优化错误分开。该界限明确了性能如何与标记节点的比例和图形诱导的依赖关系进行缩放。近似保证进一步推导出图形平滑，然后平滑的非线性读出，产生收敛速度，恢复经典的非参数行为下充分监督，同时表征性能时，标签是稀缺的。数值实验验证了我们的理论，为理解GNN的性能和局限性提供了一个系统的框架。
摘要：Graph neural networks (GNNs) work remarkably well in semi-supervised node regression, yet a rigorous theory explaining when and why they succeed remains lacking. To address this gap, we study an aggregate-and-readout model that encompasses several common message passing architectures: node features are first propagated over the graph then mapped to responses via a nonlinear function. For least-squares estimation over GNNs with linear graph convolutions and a deep ReLU readout, we prove a sharp non-asymptotic risk bound that separates approximation, stochastic, and optimization errors. The bound makes explicit how performance scales with the fraction of labeled nodes and graph-induced dependence. Approximation guarantees are further derived for graph-smoothing followed by smooth nonlinear readouts, yielding convergence rates that recover classical nonparametric behavior under full supervision while characterizing performance when labels are scarce. Numerical experiments validate our theory, providing a systematic framework for understanding GNN performance and limitations.
Transformer(5篇)
【1】The Anxiety of Influence: Bloom Filters in Transformer Attention Heads
标题：影响力的焦虑：Transformer注意力头中的布鲁姆过滤器
链接：https://arxiv.org/abs/2602.17526
备注：13 pages, 8 figures, code at https://github.com/pbalogh/anxiety-of-influence v2: L3H0 reclassified as prefix-attention head following confound control. Capacity analysis updated. Duplicate-token head overlap experiment added v3: All experiments were independently validated on CPU to rule out hardware-specific computation artifacts. Results are consistent across backends
摘要：一些Transformer注意力负责人似乎起到了成员资格测试人员的作用，他们致力于回答这样一个问题：“这个令牌以前在上下文中出现过吗？“我们在四种语言模型（GPT-2小型，中型和大型; Pythia-160 M）中识别出这些头部，并表明它们形成了一系列成员资格测试策略。两个头（GPT-2 small中的L0 H1和L0 H5）作为高精度成员过滤器，即使在180个唯一上下文标记时，误报率也为0-4\%-远高于经典Bloom过滤器的$d_\text{head} = 64$ bit容量。第三个头（L1 H11）显示了经典的布隆过滤器容量曲线：其误报率遵循理论公式$p \approx（1 - e^{-kn/m}）^k$，其中$R^2 = 1.0$，拟合容量$m \approx 5$位，饱和$n \approx 20$个唯一令牌。第四个头最初被确定为布隆过滤器（L3 H 0）被重新分类为一般前缀注意头后，混淆控制显示其表观容量曲线是一个序列长度的假象。这三个真正的成员测试头一起形成了一个集中在早期层（0-1）的多分辨率系统，在分类上不同于归纳和先前的令牌头，假阳性率随着嵌入距离单调衰减-与距离敏感的布鲁姆过滤器一致。这些领导人概括地说：他们对任何重复的标记类型都有反应，而不仅仅是重复的名字，比重复标记的头部高43%。消融揭示了这些头有助于重复和新颖的令牌处理，表明成员资格测试与更广泛的计算角色共存。通过混淆控制对L3 H 0的重新分类加强了而不是削弱了这种情况：幸存的头部经受住了审查，消除了我们自己分析中的假阳性。
摘要：Some transformer attention heads appear to function as membership testers, dedicating themselves to answering the question "has this token appeared before in the context?" We identify these heads across four language models (GPT-2 small, medium, and large; Pythia-160M) and show that they form a spectrum of membership-testing strategies. Two heads (L0H1 and L0H5 in GPT-2 small) function as high-precision membership filters with false positive rates of 0-4\% even at 180 unique context tokens -- well above the $d_\text{head} = 64$ bit capacity of a classical Bloom filter. A third head (L1H11) shows the classic Bloom filter capacity curve: its false positive rate follows the theoretical formula $p \approx (1 - e^{-kn/m})^k$ with $R^2 = 1.0$ and fitted capacity $m \approx 5$ bits, saturating by $n \approx 20$ unique tokens. A fourth head initially identified as a Bloom filter (L3H0) was reclassified as a general prefix-attention head after confound controls revealed its apparent capacity curve was a sequence-length artifact. Together, the three genuine membership-testing heads form a multi-resolution system concentrated in early layers (0-1), taxonomically distinct from induction and previous-token heads, with false positive rates that decay monotonically with embedding distance -- consistent with distance-sensitive Bloom filters. These heads generalize broadly: they respond to any repeated token type, not just repeated names, with 43\% higher generalization than duplicate-token-only heads. Ablation reveals these heads contribute to both repeated and novel token processing, indicating that membership testing coexists with broader computational roles. The reclassification of L3H0 through confound controls strengthens rather than weakens the case: the surviving heads withstand the scrutiny that eliminated a false positive in our own analysis.
【2】Transforming Behavioral Neuroscience Discovery with In-Context Learning and AI-Enhanced Tensor Methods
标题：利用上下文学习和人工智能增强张量方法改变行为神经科学发现
链接：https://arxiv.org/abs/2602.17027
摘要：科学发现管道通常涉及复杂，严格和耗时的过程，从数据准备到分析和解释发现。人工智能的最新进展有可能改变这些管道，使领域专家可以专注于解释和理解发现，而不是调试刚性管道或手动注释数据。作为数据科学/人工智能研究人员和行为神经科学家之间积极合作的一部分，我们展示了一个示例人工智能增强的管道，专门用于改变和加速团队中的领域专家能够从实验数据中获得见解的方式。目前的应用是在行为神经科学领域，研究小鼠的恐惧泛化，这是一个重要的问题，其进展可以促进我们对临床上重要且通常使人衰弱的疾病，如创伤后应激障碍（PTSD）的理解。我们将新兴的“情境学习”（ICL）范式确定为领域专家自动化其管道部分的合适接口，而无需或熟悉AI模型训练和微调，并展示其在数据准备和模式解释方面的显着功效。此外，我们还对张量分解模型引入了新的AI增强，这允许从我们的应用程序中的异构数据中进行更无缝的模式发现。我们通过实验彻底评估了我们提出的管道，与该领域的标准实践相比，以及与不属于ICL范式的合理ML基线相比，展示了其卓越的性能，以确保我们在为领域专家寻求无缝和易于使用的界面时不会影响性能。最后，我们展示了有效的发现，结果由团队中的领域专家验证。
摘要 ：Scientific discovery pipelines typically involve complex, rigid, and time-consuming processes, from data preparation to analyzing and interpreting findings. Recent advances in AI have the potential to transform such pipelines in a way that domain experts can focus on interpreting and understanding findings, rather than debugging rigid pipelines or manually annotating data. As part of an active collaboration between data science/AI researchers and behavioral neuroscientists, we showcase an example AI-enhanced pipeline, specifically designed to transform and accelerate the way that the domain experts in the team are able to gain insights out of experimental data. The application at hand is in the domain of behavioral neuroscience, studying fear generalization in mice, an important problem whose progress can advance our understanding of clinically significant and often debilitating conditions such as PTSD (Post-Traumatic Stress Disorder). We identify the emerging paradigm of "In-Context Learning" (ICL) as a suitable interface for domain experts to automate parts of their pipeline without the need for or familiarity with AI model training and fine-tuning, and showcase its remarkable efficacy in data preparation and pattern interpretation. Also, we introduce novel AI-enhancements to tensor decomposition model, which allows for more seamless pattern discovery from the heterogeneous data in our application. We thoroughly evaluate our proposed pipeline experimentally, showcasing its superior performance compared to what is standard practice in the domain, as well as against reasonable ML baselines that do not fall under the ICL paradigm, to ensure that we are not compromising performance in our quest for a seamless and easy-to-use interface for domain experts. Finally, we demonstrate effective discovery, with results validated by the domain experts in the team.
【3】A Residual-Aware Theory of Position Bias in Transformers
标题：Transformer中位置偏差的残余感知理论
链接：https://arxiv.org/abs/2602.16837
摘要：Transformer模型系统地支持某些令牌位置，但这种位置偏差的架构起源仍然知之甚少。在无限深度的因果掩蔽下，之前对注意力展开的理论分析预测注意力不可避免地会崩溃到第一个标记上。然而，这种崩溃在实践中不会发生。我们解决这个矛盾与剩余意识理论的累积注意力推出。通过将剩余连接，我们表明，这种建筑组件，防止在现实条件下崩溃。在有限的深度，我们证明了因果Transformers诱导一个U形的位置偏差，注意力集中在早期和晚期的令牌。这一结果提供了一个原则性的建筑解释的迷失在中间的现象。
摘要：Transformer models systematically favor certain token positions, yet the architectural origins of this position bias remain poorly understood. Under causal masking at infinite depth, prior theoretical analyses of attention rollout predict an inevitable collapse of attention onto the first token. Such collapse, however, does not occur in practice. We resolve this discrepancy with a residual-aware theory of cumulative attention rollout. By incorporating residual connections, we show that this architectural component prevents collapse under realistic conditions. At finite depth, we prove that causal Transformers induce a U-shaped position bias, with attention concentrating on early and late tokens. This result provides a principled architectural explanation for the Lost-in-the-Middle phenomenon.
【4】MMCAformer: Macro-Micro Cross-Attention Transformer for Traffic Speed Prediction with Microscopic Connected Vehicle Driving Behavior
标题：MMCA former：用于微观互联车辆驾驶行为的交通速度预测的宏微观交叉注意Transformer
链接：https://arxiv.org/abs/2602.16730
摘要：准确的速度预测对于主动交通管理以提高交通效率和安全至关重要。现有的研究主要依赖于聚集的宏观交通流数据来预测未来的交通趋势，而道路交通动态也受到个体的微观人类驾驶行为的影响。最近的互联车辆（CV）数据提供了丰富的驾驶行为特征，为将这些行为洞察纳入速度预测提供了新的机会。为此，我们提出了宏-微交叉注意Transformer（MMCAformer）集成CV数据为基础的微观驾驶行为特征与宏观交通特征的速度预测。具体而言，MMCAformer采用自我注意力来学习宏观交通流中的内在依赖关系，并采用交叉注意力来捕获宏观交通状态和微观驾驶行为之间的时空相互作用。MMCAformer使用Student-t负对数似然损失进行优化，以提供逐点速度预测和估计不确定性。佛罗里达州的四条高速公路上的实验表明，所提出的MMCAformer相比，基线的优越性能。与仅使用宏观特征相比，引入微观驾驶行为特征不仅增强了预测准确性（例如，总体RMSE、MAE和MAPE分别降低了9.0%、6.9%和10.2%），但也缩小了模型预测的不确定性（例如，平均预测间隔在四条高速公路上下降了10.1-24.0%）。结果表明，急刹车和加速频率出现的最有影响力的功能。这种改善在拥堵、低速交通条件下更为明显。
摘要：Accurate speed prediction is crucial for proactive traffic management to enhance traffic efficiency and safety. Existing studies have primarily relied on aggregated, macroscopic traffic flow data to predict future traffic trends, whereas road traffic dynamics are also influenced by individual, microscopic human driving behaviors. Recent Connected Vehicle (CV) data provide rich driving behavior features, offering new opportunities to incorporate these behavioral insights into speed prediction. To this end, we propose the Macro-Micro Cross-Attention Transformer (MMCAformer) to integrate CV data-based micro driving behavior features with macro traffic features for speed prediction. Specifically, MMCAformer employs self-attention to learn intrinsic dependencies in macro traffic flow and cross-attention to capture spatiotemporal interplays between macro traffic status and micro driving behavior. MMCAformer is optimized with a Student-t negative log-likelihood loss to provide point-wise speed prediction and estimate uncertainty. Experiments on four Florida freeways demonstrate the superior performance of the proposed MMCAformer compared to baselines. Compared with only using macro features, introducing micro driving behavior features not only enhances prediction accuracy (e.g., overall RMSE, MAE, and MAPE reduced by 9.0%, 6.9%, and 10.2%, respectively) but also shrinks model prediction uncertainty (e.g., mean predictive intervals decreased by 10.1-24.0% across the four freeways). Results reveal that hard braking and acceleration frequencies emerge as the most influential features. Such improvements are more pronounced under congested, low-speed traffic conditions.
【5】A statistical perspective on transformers for small longitudinal cohort data
标题：小型纵向队列数据的Transformer统计角度
链接：https://arxiv.org/abs/2602.16914
摘要：纵向队列数据的建模通常涉及多个变量之间复杂的时间依赖关系。在那里，Transformer架构在语言和视觉应用中非常成功，它使我们能够解释这样一个事实，即在个人历史中最近观察到的时间点可能并不总是对最近的未来最重要的。这是通过基于其值的变换将注意力权重分配给个体的观察来实现的。这些想法尚未完全用于纵向队列数据的一个原因是通常需要大型数据集。因此，我们提出了一种简化的Transformer架构，该架构保留了核心注意力机制，同时减少了需要估计的参数数量，以更适合时间点少的小数据集。在Transformers的统计视角的指导下，我们使用自回归模型作为起点，并将注意力作为基于内核的操作与时间衰减，其中多个Transformer头的聚合，即不同的候选人加权方案，表示为积累证据的不同类型的潜在特征的个人。这也使得基于置换的统计测试过程能够用于识别上下文模式。在模拟研究中，该方法被证明可以恢复上下文依赖关系，即使是少量的个人和时间点。在对韧性研究数据的应用中，我们识别了压力和心理健康动态的时间模式。这表明适当适配的Transformers不仅可以实现有竞争力的预测性能，而且还可以在小数据设置中发现复杂的上下文依赖关系。
摘要：Modeling of longitudinal cohort data typically involves complex temporal dependencies between multiple variables. There, the transformer architecture, which has been highly successful in language and vision applications, allows us to account for the fact that the most recently observed time points in an individual's history may not always be the most important for the immediate future. This is achieved by assigning attention weights to observations of an individual based on a transformation of their values. One reason why these ideas have not yet been fully leveraged for longitudinal cohort data is that typically, large datasets are required. Therefore, we present a simplified transformer architecture that retains the core attention mechanism while reducing the number of parameters to be estimated, to be more suitable for small datasets with few time points. Guided by a statistical perspective on transformers, we use an autoregressive model as a starting point and incorporate attention as a kernel-based operation with temporal decay, where aggregation of multiple transformer heads, i.e. different candidate weighting schemes, is expressed as accumulating evidence on different types of underlying characteristics of individuals. This also enables a permutation-based statistical testing procedure for identifying contextual patterns. In a simulation study, the approach is shown to recover contextual dependencies even with a small number of individuals and time points. In an application to data from a resilience study, we identify temporal patterns in the dynamics of stress and mental health. This indicates that properly adapted transformers can not only achieve competitive predictive performance, but also uncover complex context dependencies in small data settings.
GAN|对抗|攻击|生成相关(5篇)
【1】Pushing the Frontier of Black-Box LVLM Attacks via Fine-Grained Detail Targeting
标题：通过细粒度细节瞄准突破黑匣子LVLM攻击的前沿
链接：https://arxiv.org/abs/2602.17645
备注：Code at: https://github.com/vila-lab/M-Attack-V2
摘要：由于缺少梯度和复杂的多模态边界，对大型视觉语言模型（LVLM）的黑盒对抗攻击具有挑战性。虽然现有的最先进的基于传输的方法，如M-Attack，使用源图像和目标图像之间的局部作物级匹配表现良好，但我们发现这会导致高方差，几乎正交的梯度迭代，违反相干的局部对齐和不稳定的优化。我们将其归因于（i）产生尖峰样梯度的ViT翻译敏感性和（ii）源作物和目标作物之间的结构不对称性。我们将局部匹配重新表述为对源变换和目标语义的不对称期望，并构建了M-Attack的梯度去噪升级。在源端，多裁剪对齐（MCA）在每次迭代中对来自多个独立采样的局部视图的梯度进行平均，以减少方差。在目标方面，辅助目标对齐（ATA）用来自语义相关分布的小辅助集取代积极的目标增强，产生更平滑，更低方差的目标流形。我们进一步将动量重新解释为斑块动量，重演历史作物梯度;结合改进的斑块大小集合（PE+），这加强了可转移的方向。这些模块共同组成了M-Attack-V2，这是一个简单的模块化增强，大大改善了对前沿LVLM的基于传输的黑箱攻击：将Claude-4.0的成功率从8%提高到30%，Gemini-2.5-Pro从83%提高到97%，GPT-5从98%提高到100%，优于之前的黑箱LVLM攻击。代码和数据可在https://github.com/vila-lab/M-Attack-V2上公开获取。
摘要：Black-box adversarial attacks on Large Vision-Language Models (LVLMs) are challenging due to missing gradients and complex multimodal boundaries. While prior state-of-the-art transfer-based approaches like M-Attack perform well using local crop-level matching between source and target images, we find this induces high-variance, nearly orthogonal gradients across iterations, violating coherent local alignment and destabilizing optimization. We attribute this to (i) ViT translation sensitivity that yields spike-like gradients and (ii) structural asymmetry between source and target crops. We reformulate local matching as an asymmetric expectation over source transformations and target semantics, and build a gradient-denoising upgrade to M-Attack. On the source side, Multi-Crop Alignment (MCA) averages gradients from multiple independently sampled local views per iteration to reduce variance. On the target side, Auxiliary Target Alignment (ATA) replaces aggressive target augmentation with a small auxiliary set from a semantically correlated distribution, producing a smoother, lower-variance target manifold. We further reinterpret momentum as Patch Momentum, replaying historical crop gradients; combined with a refined patch-size ensemble (PE+), this strengthens transferable directions. Together these modules form M-Attack-V2, a simple, modular enhancement over M-Attack that substantially improves transfer-based black-box attacks on frontier LVLMs: boosting success rates on Claude-4.0 from 8% to 30%, Gemini-2.5-Pro from 83% to 97%, and GPT-5 from 98% to 100%, outperforming prior black-box LVLM attacks. Code and data are publicly available at: https://github.com/vila-lab/M-Attack-V2.
【2】TimeOmni-VL: Unified Models for Time Series Understanding and Generation
标题：TimeOmni-DL：时间序列理解和生成的统一模型
链接：https://arxiv.org/abs/2602.17149
摘要：最近的时间序列建模面临着数值生成和语义理解之间的巨大鸿沟，研究表明生成模型通常依赖于表面模式匹配，而面向理解的模型则难以获得高保真度的数值输出。虽然统一的多模态模型（UMMs）已经弥合了这一差距的愿景，其潜力的时间序列仍然没有开发。我们提出了TimeOmni-VL，这是第一个以视觉为中心的框架，通过两个关键创新统一了时间序列的理解和生成：（1）时间序列和图像之间的保真双向映射（Bi-TSI），它推进了时间序列到图像（TS 2 I）和图像到时间序列（I2 TS）的转换，以确保接近无损的转换。(2)理解引导的一代。我们介绍了TSUMM-Suite，一个新的数据集，由六个理解任务植根于时间序列分析，再加上两个生成任务。通过校准的思想链，TimeOmni-VL是第一个利用时间序列理解作为高保真生成的显式控制信号的产品。实验证实，这种统一的方法显着提高语义理解和数值精度，建立一个新的前沿多模态时间序列建模。
摘要：Recent time series modeling faces a sharp divide between numerical generation and semantic understanding, with research showing that generation models often rely on superficial pattern matching, while understanding-oriented models struggle with high-fidelity numerical output. Although unified multimodal models (UMMs) have bridged this gap in vision, their potential for time series remains untapped. We propose TimeOmni-VL, the first vision-centric framework that unifies time series understanding and generation through two key innovations: (1) Fidelity-preserving bidirectional mapping between time series and images (Bi-TSI), which advances Time Series-to-Image (TS2I) and Image-to-Time Series (I2TS) conversions to ensure near-lossless transformations. (2) Understanding-guided generation. We introduce TSUMM-Suite, a novel dataset consists of six understanding tasks rooted in time series analytics that are coupled with two generation tasks. With a calibrated Chain-of-Thought, TimeOmni-VL is the first to leverage time series understanding as an explicit control signal for high-fidelity generation. Experiments confirm that this unified approach significantly improves both semantic understanding and numerical precision, establishing a new frontier for multimodal time series modeling.
【3】LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation
标题：LLLM 4Cov：用于高覆盖率测试平台生成的执行感知抽象学习
链接：https://arxiv.org/abs/2602.16953
摘要：执行感知的LLM代理为从工具反馈中学习提供了一个很有前途的范例，但这种反馈通常是昂贵的，获得速度慢，使得在线强化学习（RL）不切实际。高覆盖率的硬件验证克服了这一挑战，因为它依赖于工业模拟器和不可区分的执行信号。我们提出了LLM4Cov，一个离线代理学习框架，该框架将验证建模为由确定性评估器指导的无记忆状态转换。在此基础上，我们引入了执行验证的数据策展，策略感知的代理数据合成和最差状态优先级的采样，以实现在执行约束下的可扩展学习。我们进一步策划了一个现实对齐的基准，通过修订后的评估协议，从现有的验证套件。使用所提出的管道，一个紧凑的4B参数模型在代理评估下实现了69.2%的覆盖率通过率，比其老师高出5.3%，并表现出比模型大一个数量级的竞争力。
摘要：Execution-aware LLM agents offer a promising paradigm for learning from tool feedback, but such feedback is often expensive and slow to obtain, making online reinforcement learning (RL) impractical. High-coverage hardware verification exemplifies this challenge due to its reliance on industrial simulators and non-differentiable execution signals. We propose LLM4Cov, an offline agent-learning framework that models verification as memoryless state transitions guided by deterministic evaluators. Building on this formulation, we introduce execution-validated data curation, policy-aware agentic data synthesis, and worst-state-prioritized sampling to enable scalable learning under execution constraints. We further curate a reality-aligned benchmark adapted from an existing verification suite through a revised evaluation protocol. Using the proposed pipeline, a compact 4B-parameter model achieves 69.2% coverage pass rate under agentic evaluation, outperforming its teacher by 5.3% and demonstrating competitive performance against models an order of magnitude larger.
【4】Exact Certification of Data-Poisoning Attacks Using Mixed-Integer Programming
标题：使用混合收件箱编程精确认证数据中毒攻击
链接：https://arxiv.org/abs/2602.16944
备注：Accepted to the 23rd International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR)
摘要：这项工作介绍了一个验证框架，在神经网络训练期间为数据中毒攻击提供了可靠和完整的保证。我们在一个混合整数二次规划（MIQCP）问题中制定了对抗性数据操作，模型训练和测试时评估。找到所提出的公式的全局最优值可证明会产生最坏情况的中毒攻击，同时限制给定训练管道上所有可能攻击的有效性。我们的框架在测试时对基于梯度的训练动态和模型评估进行编码，从而实现了训练时鲁棒性的第一次精确认证。对小模型的实验评估证实，我们的方法提供了一个完整的表征对数据中毒的鲁棒性。
摘要：This work introduces a verification framework that provides both sound and complete guarantees for data poisoning attacks during neural network training. We formulate adversarial data manipulation, model training, and test-time evaluation in a single mixed-integer quadratic programming (MIQCP) problem. Finding the global optimum of the proposed formulation provably yields worst-case poisoning attacks, while simultaneously bounding the effectiveness of all possible attacks on the given training pipeline. Our framework encodes both the gradient-based training dynamics and model evaluation at test time, enabling the first exact certification of training-time robustness. Experimental evaluation on small models confirms that our approach delivers a complete characterization of robustness against data poisoning.
【5】MGD: Moment Guided Diffusion for Maximum Entropy Generation
标题：MVD：最大熵产生的动量引导扩散
链接：https://arxiv.org/abs/2602.17211
摘要：从有限的信息中生成样本是跨科学领域的基本问题。经典的最大熵方法从矩约束提供了原则性的不确定性量化，但需要通过MCMC或Langevin动力学进行采样，这通常在高维中表现出指数减速。相比之下，基于扩散和流匹配的生成模型有效地将噪声传输到数据，但提供有限的理论保证，并且在数据稀缺时可能过拟合。我们引入矩导引扩散（MGD），它结合了两种方法的元素。基于随机插值框架，MGD通过求解随机微分方程来对最大熵分布进行采样，该随机微分方程在有限时间内将矩引导到规定值，从而避免了基于平衡的方法中的缓慢混合。我们正式获得，在大波动率的限制，MGD收敛到最大熵分布，并推导出一个易于处理的估计直接从动态计算得到的熵。应用金融时间序列，湍流，宇宙学领域使用小波散射矩产生估计的高维多尺度过程的负熵。
摘要：Generating samples from limited information is a fundamental problem across scientific domains. Classical maximum entropy methods provide principled uncertainty quantification from moment constraints but require sampling via MCMC or Langevin dynamics, which typically exhibit exponential slowdown in high dimensions. In contrast, generative models based on diffusion and flow matching efficiently transport noise to data but offer limited theoretical guarantees and can overfit when data is scarce. We introduce Moment Guided Diffusion (MGD), which combines elements of both approaches. Building on the stochastic interpolant framework, MGD samples maximum entropy distributions by solving a stochastic differential equation that guides moments toward prescribed values in finite time, thereby avoiding slow mixing in equilibrium-based methods. We formally obtain, in the large-volatility limit, convergence of MGD to the maximum entropy distribution and derive a tractable estimator of the resulting entropy computed directly from the dynamics. Applications to financial time series, turbulent flows, and cosmological fields using wavelet scattering moments yield estimates of negentropy for high-dimensional multiscale processes.
半/弱/无/有监督|不确定性|主动学习(5篇)
【1】Continual uncertainty learning
标题：持续的不确定性学习
链接：https://arxiv.org/abs/2602.17174
摘要：具有多个不确定性的机械系统的鲁棒控制仍然是一个根本性的挑战，特别是当非线性动力学和操作条件变化错综复杂地交织在一起时。虽然深度强化学习（DRL）与域随机化相结合在缓解模拟与真实差距方面表现出了希望，但同时处理所有不确定性来源往往会导致次优策略和学习效率低下。本研究提出一种新的基于连续学习的框架，用于解决多个不确定性源同时叠加的非线性动态系统的鲁棒控制问题。其核心思想是将具有多个不确定性的复杂控制问题分解为一系列连续的学习任务，在这些任务中，依次获得处理每个不确定性的策略。原系统扩展成一个有限的植物，其动态不确定性逐渐扩大和多样化的学习进展。该策略在与由不同不确定性配置定义的任务相关联的整个工厂集合中稳定更新，而不会发生灾难性遗忘。为了确保学习效率，我们联合采用了基于模型的控制器（MBC），它保证了整个工厂集的共享基线性能，到学习过程中，以加速收敛。这种残差学习方案有利于针对每个不确定性对DRL代理进行特定任务的优化，从而提高样本效率。作为一个实际的工业应用，本研究应用所提出的方法来设计一个主动振动控制器的汽车动力总成。我们验证了所得到的控制器对结构非线性和动态变化具有鲁棒性，实现了成功的模拟到真实的传输。
摘要：Robust control of mechanical systems with multiple uncertainties remains a fundamental challenge, particularly when nonlinear dynamics and operating-condition variations are intricately intertwined. While deep reinforcement learning (DRL) combined with domain randomization has shown promise in mitigating the sim-to-real gap, simultaneously handling all sources of uncertainty often leads to sub-optimal policies and poor learning efficiency. This study formulates a new curriculum-based continual learning framework for robust control problems involving nonlinear dynamical systems in which multiple sources of uncertainty are simultaneously superimposed. The key idea is to decompose a complex control problem with multiple uncertainties into a sequence of continual learning tasks, in which strategies for handling each uncertainty are acquired sequentially. The original system is extended into a finite set of plants whose dynamic uncertainties are gradually expanded and diversified as learning progresses. The policy is stably updated across the entire plant sets associated with tasks defined by different uncertainty configurations without catastrophic forgetting. To ensure learning efficiency, we jointly incorporate a model-based controller (MBC), which guarantees a shared baseline performance across the plant sets, into the learning process to accelerate the convergence. This residual learning scheme facilitates task-specific optimization of the DRL agent for each uncertainty, thereby enhancing sample efficiency. As a practical industrial application, this study applies the proposed method to designing an active vibration controller for automotive powertrains. We verified that the resulting controller is robust against structural nonlinearities and dynamic variations, realizing successful sim-to-real transfer.
【2】Forecasting Anomaly Precursors via Uncertainty-Aware Time-Series Ensembles
标题：通过不确定性感知时间序列集合预测异常前兆
链接：https://arxiv.org/abs/2602.17028
备注：This manuscript contains 14 pages and 8 figures. It is currently under review at IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
摘要：在工业运营、金融和网络安全等领域，检测时间序列数据中的异常至关重要，早期识别异常模式对于确保系统可靠性和实现预防性维护至关重要。然而，大多数现有的方法是被动的：它们只是在异常情况发生后才发现异常情况，缺乏提供主动预警信号的能力。在本文中，我们提出了FATE（Forecasting Anomalies with Time-series Ensembles），这是一种新的无监督框架，通过量化来自不同时间序列预测模型集合的预测不确定性来检测异常前兆（PoA）。与依赖于重建误差或需要地面实况标签的先前方法不同，FATE预测未来值并利用集合不一致来发出潜在异常的早期迹象，而无需在推理时访问目标值。为了严格评估PoA检测，我们引入了前兆时间序列感知精度和召回率（PTaPR），这是一种新的度量标准，通过联合评估早期预测的片段级准确性，片段内覆盖率和时间连续性来扩展传统的时间序列感知精度和召回率（TaPR）。这使得能够更全面地评估现有指标所忽视的预警能力。在五个真实世界基准数据集上的实验表明，FATE在PTaPR AUC上平均提高了19.9个百分点，在早期检测F1得分上平均提高了20.02个百分点，在不需要异常标签的情况下优于基线。这些结果证明了FATE在复杂时间序列环境中进行实时无监督预警的有效性和实用性。
摘要 ：Detecting anomalies in time-series data is critical in domains such as industrial operations, finance, and cybersecurity, where early identification of abnormal patterns is essential for ensuring system reliability and enabling preventive maintenance. However, most existing methods are reactive: they detect anomalies only after they occur and lack the capability to provide proactive early warning signals. In this paper, we propose FATE (Forecasting Anomalies with Time-series Ensembles), a novel unsupervised framework for detecting Precursors-of-Anomaly (PoA) by quantifying predictive uncertainty from a diverse ensemble of time-series forecasting models. Unlike prior approaches that rely on reconstruction errors or require ground-truth labels, FATE anticipates future values and leverages ensemble disagreement to signal early signs of potential anomalies without access to target values at inference time. To rigorously evaluate PoA detection, we introduce Precursor Time-series Aware Precision and Recall (PTaPR), a new metric that extends the traditional Time-series Aware Precision and Recall (TaPR) by jointly assessing segment-level accuracy, within-segment coverage, and temporal promptness of early predictions. This enables a more holistic assessment of early warning capabilities that existing metrics overlook. Experiments on five real-world benchmark datasets show that FATE achieves an average improvement of 19.9 percentage points in PTaPR AUC and 20.02 percentage points in early detection F1 score, outperforming baselines while requiring no anomaly labels. These results demonstrate the effectiveness and practicality of FATE for real-time unsupervised early warning in complex time-series environments.
【3】WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
标题：WS-GRPO：弱监督的群体相对策略优化，以实现推广高效推理
链接：https://arxiv.org/abs/2602.17025
摘要：组相对策略优化（GRPO）是训练语言模型进行复杂推理的有效方法。然而，由于目标是相对于一组采样轨迹定义的，因此延长的审议可以创造更多实现相对收益的机会，导致低效的推理和过度思考，并使正确性和推出效率之间的权衡复杂化。控制这种行为在实践中是困难的，考虑到（i）长度惩罚难以校准，因为较长的推出可能反映需要较长推理的较难问题，惩罚令牌有可能截断有用的推理以及冗余的继续;以及（ii）直接指示何时继续或停止的监督通常在最终答案正确性之外不可用。我们提出了弱监督GRPO（WS-GRPO），它通过将终端奖励转换为部分轨迹上的正确性感知指导来提高推出效率。与难以校准的全局长度惩罚不同，WS-GRPO从仅结果正确性训练偏好模型，以产生前缀级别的信号，指示何时额外的延续是有益的。因此，WS-GRPO提供了结果导出的继续/停止制导，在保持精度的同时减少了冗余的考虑。我们提供的理论结果和经验表明，推理基准，WS-GRPO大大减少推出长度，同时保持竞争力与GRPO基线。
摘要：Group Relative Policy Optimization (GRPO) is effective for training language models on complex reasoning. However, since the objective is defined relative to a group of sampled trajectories, extended deliberation can create more chances to realize relative gains, leading to inefficient reasoning and overthinking, and complicating the trade-off between correctness and rollout efficiency. Controlling this behavior is difficult in practice, considering (i) Length penalties are hard to calibrate because longer rollouts may reflect harder problems that require longer reasoning, penalizing tokens risks truncating useful reasoning along with redundant continuation; and (ii) supervision that directly indicates when to continue or stop is typically unavailable beyond final answer correctness. We propose Weakly Supervised GRPO (WS-GRPO), which improves rollout efficiency by converting terminal rewards into correctness-aware guidance over partial trajectories. Unlike global length penalties that are hard to calibrate, WS-GRPO trains a preference model from outcome-only correctness to produce prefix-level signals that indicate when additional continuation is beneficial. Thus, WS-GRPO supplies outcome-derived continue/stop guidance, reducing redundant deliberation while maintaining accuracy. We provide theoretical results and empirically show on reasoning benchmarks that WS-GRPO substantially reduces rollout length while remaining competitive with GRPO baselines.
【4】Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling
标题：用监督潜在变量模型描述模式的预测影响
链接：https://arxiv.org/abs/2602.16979
摘要：尽管多模态大型语言模型（MLLM）最近取得了成功，但现有的方法主要假设在训练和推理过程中存在多个模态。在实践中，多模态数据往往是不完整的，因为模态可能会丢失，异步收集，或仅适用于一个子集的例子。在这项工作中，我们提出了PRIMO，一个监督的潜在变量插补模型，量化了多模态学习环境中任何缺失模态的预测影响。PRIMO允许使用所有现有的培训实例，无论模式是完整的还是部分的。具体来说，它通过一个潜在变量来对缺失的模态进行建模，该变量在预测的背景下捕获其与观察到的模态的关系。在推理过程中，我们从缺失模态的学习分布中抽取许多样本，以获得边缘预测分布（用于预测目的）并分析缺失模态对每个实例预测的影响。我们在合成XOR数据集、Audio-Vision MNIST和MIMIC-III上评估PRIMO的死亡率和ICD-9预测。在所有数据集中，PRIMO在一种模式完全缺失时获得与单峰基线相当的性能，在所有模式都可用时获得与多峰基线相当的性能。PRIMO使用基于方差的度量在实例级别量化模态的预测影响，该度量是根据对潜在完成的预测计算的。我们直观地展示了如何在一组看似合理的标签中对缺失的模态进行不同的补充。
摘要：Despite the recent success of Multimodal Large Language Models (MLLMs), existing approaches predominantly assume the availability of multiple modalities during training and inference. In practice, multimodal data is often incomplete because modalities may be missing, collected asynchronously, or available only for a subset of examples. In this work, we propose PRIMO, a supervised latent-variable imputation model that quantifies the predictive impact of any missing modality within the multimodal learning setting. PRIMO enables the use of all available training examples, whether modalities are complete or partial. Specifically, it models the missing modality through a latent variable that captures its relationship with the observed modality in the context of prediction. During inference, we draw many samples from the learned distribution over the missing modality to both obtain the marginal predictive distribution (for the purpose of prediction) and analyze the impact of the missing modalities on the prediction for each instance. We evaluate PRIMO on a synthetic XOR dataset, Audio-Vision MNIST, and MIMIC-III for mortality and ICD-9 prediction. Across all datasets, PRIMO obtains performance comparable to unimodal baselines when a modality is fully missing and to multimodal baselines when all modalities are available. PRIMO quantifies the predictive impact of a modality at the instance level using a variance-based metric computed from predictions across latent completions. We visually demonstrate how varying completions of the missing modality result in a set of plausible labels.
【5】Learning under noisy supervision is governed by a feedback-truth gap
标题：嘈杂监督下的学习受到反馈与真相差距的支配
链接：https://arxiv.org/abs/2602.16829
备注：33 pages, 5 figures, 10 extended data figures, 4 extended data tables; 10-page supplementary information
摘要：当反馈被吸收的速度快于任务结构的评估速度时，学习者会更喜欢反馈而不是事实。一个双时间尺度模型表明，当两个比率不同时，这种反馈-真实差距是不可避免的，只有当它们匹配时才会消失。我们在用噪声标签训练的神经网络（30个数据集，2,700次运行），人类概率反向学习（N = 292）和人类奖励/惩罚学习与并发EEG（N = 25）中测试了这种预测。在每个系统中，真理都是在操作上定义的：坚持的标签，客观正确的选项，或参与者的反馈前期望-唯一可从反馈后EEG解码的非循环参考。这种差距普遍存在，但调节方式不同：密集的网络将其积累为记忆;稀疏的残余脚手架抑制了它;人类产生了短暂的过度承诺，并被积极恢复。神经过度承诺（~0.04-0.10）被放大10倍到行为承诺（d = 3.3-3.9）。这种差距是在嘈杂的监督下学习的基本限制;其后果取决于每个系统所采用的监管。
摘要：When feedback is absorbed faster than task structure can be evaluated, the learner will favor feedback over truth. A two-timescale model shows this feedback-truth gap is inevitable whenever the two rates differ and vanishes only when they match. We test this prediction across neural networks trained with noisy labels (30 datasets, 2,700 runs), human probabilistic reversal learning (N = 292), and human reward/punishment learning with concurrent EEG (N = 25). In each system, truth is defined operationally: held-out labels, the objectively correct option, or the participant's pre-feedback expectation - the only non-circular reference decodable from post-feedback EEG. The gap appeared universally but was regulated differently: dense networks accumulated it as memorization; sparse-residual scaffolding suppressed it; humans generated transient over-commitment that was actively recovered. Neural over-commitment (~0.04-0.10) was amplified tenfold into behavioral commitment (d = 3.3-3.9). The gap is a fundamental constraint on learning under noisy supervision; its consequences depend on the regulation each system employs.
迁移|Zero/Few/One-Shot|自适应(10篇)
【1】Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
标题：Reverso：Zero-Shot预测的高效时间序列基础模型
链接：https://arxiv.org/abs/2602.17634
摘要：学习时间序列基础模型已被证明是一种很有前途的方法，用于跨不同时间序列域的zero-shot时间序列预测。由于扩展一直是基础模型在其他模式（如语言和视觉）中性能的关键驱动因素，因此最近关于时间序列基础建模的许多工作都集中在扩展上。这导致了具有数亿参数的时间序列基础模型，这些参数虽然性能好，但在实践中使用起来效率低下且昂贵。本文描述了一个简单的配方，学习有效的基础模型的zero-shot时间序列预测的数量级更小。我们证明了大规模的Transformers是不必要的：交织长卷积和线性RNN层（特别是DeltaNet层）的小型混合模型可以匹配更大的基于transformer的模型的性能，同时小一百多倍。我们还描述了几个数据增强和推理策略，进一步提高性能。这一配方产生了Reverso，这是一系列用于zero-shot预测的高效时间序列基础模型，可显著推动性能效率的帕累托边界。
摘要：Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.
【2】Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning
标题：灾难性遗忘弹性单次增量联邦学习
链接：https://arxiv.org/abs/2602.17625
备注：Accepted for publication in the IEEE International Conference on Big Data (IEEE BigData) 2025
摘要：现代大数据系统产生了大量的、异构的、地理上分散的数据流，这些数据流是大规模的、对隐私敏感的，这使得集中化具有挑战性。虽然联邦学习（FL）提供了一种增强隐私的训练机制，但它假设静态数据流并在多轮中学习协作模型，这使得在有限的通信场景中使用增量数据进行学习具有挑战性。本文介绍了单次增量联邦学习（OSI-FL），第一个FL框架，解决了通信开销和灾难性遗忘的双重挑战。OSI-FL在单个通信轮中从每个客户端传达由冻结视觉语言模型（VLM）设计的特定于类别的嵌入，服务器处的预训练扩散模型使用该嵌入来合成类似于客户端数据分布的新数据。合成的样本在服务器上用于训练。然而，仍然存在两个挑战：i）增量到达的任务需要重新训练全局模型，ii）随着未来任务的到来，重新训练模型会引入灾难性遗忘。为此，我们使用选择性样本保留（SSR）来增强训练，该方法基于样本丢失来识别和保留每个类别和任务对的top-p最具信息性的样本。SSR通过确保代表性的保留样本被纳入进一步迭代的训练中来限制遗忘。实验结果表明，OSI-FL优于基线，包括传统的和一次性FL方法，在类增量和域增量的情况下，在三个基准数据集。
摘要：Modern big-data systems generate massive, heterogeneous, and geographically dispersed streams that are large-scale and privacy-sensitive, making centralization challenging. While federated learning (FL) provides a privacy-enhancing training mechanism, it assumes a static data flow and learns a collaborative model over multiple rounds, making learning with \textit{incremental} data challenging in limited-communication scenarios. This paper presents One-Shot Incremental Federated Learning (OSI-FL), the first FL framework that addresses the dual challenges of communication overhead and catastrophic forgetting. OSI-FL communicates category-specific embeddings, devised by a frozen vision-language model (VLM) from each client in a single communication round, which a pre-trained diffusion model at the server uses to synthesize new data similar to the client's data distribution. The synthesized samples are used on the server for training. However, two challenges still persist: i) tasks arriving incrementally need to retrain the global model, and ii) as future tasks arrive, retraining the model introduces catastrophic forgetting. To this end, we augment training with Selective Sample Retention (SSR), which identifies and retains the top-p most informative samples per category and task pair based on sample loss. SSR bounds forgetting by ensuring that representative retained samples are incorporated into training in further iterations. The experimental results indicate that OSI-FL outperforms baselines, including traditional and one-shot FL approaches, in both class-incremental and domain-incremental scenarios across three benchmark datasets.
【3】Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
标题：学会保持安全：微调期间针对安全性下降的自适应监管
链接：https://arxiv.org/abs/2602.17546
备注：Work in progress (30 pages)
摘要：遵循指令的语言模型被训练成有用和安全的，但它们的安全行为在良性微调下会恶化，在对抗性更新下会恶化。现有的防御系统通常提供有限的保护，或者在安全性和实用性之间进行权衡。我们引入了一个训练框架，该框架可以根据安全风险调整正则化，使模型在整个微调过程中保持一致。为了在训练时估计安全风险，我们探索了两种不同的方法：一种是基于判断的安全批评，它为训练批次分配高级别的伤害分数，另一种是基于激活的风险预测器，它是用一个轻量级分类器构建的，该分类器在中间模型激活上训练，以估计有害意图。每种方法都提供了一个风险信号，用于限制被认为风险较高的更新保持接近安全的参考策略，而风险较低的更新则通过标准培训进行。我们的经验验证，有害的意图信号是可预测的前代激活和判断分数提供有效的高召回安全指导。在多个模型族和攻击场景中，与标准微调相比，采用任何一种风险估计方法的自适应正则化都能降低攻击成功率，保持下游性能，并且不增加推理时间成本。这项工作展示了一个原则性的机制，在不牺牲实用性的情况下保持安全。
摘要：Instruction-following language models are trained to be helpful and safe, yet their safety behavior can deteriorate under benign fine-tuning and worsen under adversarial updates. Existing defenses often offer limited protection or force a trade-off between safety and utility. We introduce a training framework that adapts regularization in response to safety risk, enabling models to remain aligned throughout fine-tuning. To estimate safety risk at training time, we explore two distinct approaches: a judge-based Safety Critic that assigns high-level harm scores to training batches, and an activation-based risk predictor built with a lightweight classifier trained on intermediate model activations to estimate harmful intent. Each approach provides a risk signal that is used to constrain updates deemed higher risk to remain close to a safe reference policy, while lower-risk updates proceed with standard training. We empirically verify that harmful intent signals are predictable from pre-generation activations and that judge scores provide effective high-recall safety guidance. Across multiple model families and attack scenarios, adaptive regularization with either risk estimation approach consistently lowers attack success rate compared to standard fine-tuning, preserves downstream performance, and adds no inference-time cost. This work demonstrates a principled mechanism for maintaining safety without sacrificing utility.
【4】LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
标题：LORA-CRAFT：通过预训练注意力权重的Frozen Tucker分解的跨层秩自适应
链接：https://arxiv.org/abs/2602.17510
摘要：我们介绍了CRAFT（Cross-layer Rank Adaptation via Frozen Tucker），这是一种参数高效的微调（PEFT）方法，该方法将Tucker张量分解应用于跨Transformer层堆叠的预训练注意力权重矩阵，并在所得到的冻结Tucker因子上仅训练小的方形自适应矩阵。现有的基于张量的PEFT方法分解梯度更新：LoTR应用具有共享因子矩阵的Tucker分解，而SuperLoRA在应用Tucker分解之前跨层分组和重塑$ΔW$。另外，像PiSSA这样的方法将SVD应用于预先训练的权重，但每层独立操作。CRAFT连接了这两条工作线：它通过高阶SVD（HOSVD）直接对组织为跨层3D张量的预训练权重执行完全的Tucker分解，冻结所有结果因子，并通过应用于每个因子矩阵的轻量级可训练变换来调整模型。在使用RoberTa-base和RoberTa-large的GLUE基准测试上的实验表明，CRAFT与现有方法相比具有竞争力的性能，同时仅需要41 K Tucker自适应参数-在固定的Tucker秩上，该参数与模型维度和深度无关。
摘要：We introduce CRAFT (Cross-layer Rank Adaptation via Frozen Tucker), a parameter-efficient fine-tuning (PEFT) method that applies Tucker tensor decomposition to pre-trained attention weight matrices stacked across transformer layers and trains only small square adaptation matrices on the resulting frozen Tucker factors. Existing tensor-based PEFT methods decompose gradient updates: LoTR applies Tucker decomposition with shared factor matrices, while SuperLoRA groups and reshapes $ΔW$ across layers before applying Tucker decomposition. Separately, methods like PiSSA apply SVD to pre-trained weights but operate independently per layer. CRAFT bridges these two lines of work: it performs full Tucker decomposition via Higher-Order SVD (HOSVD) directly on pre-trained weights organized as cross-layer 3D tensors, freezes all resulting factors, and adapts the model through lightweight trainable transformations applied to each factor matrix. Experiments on the GLUE benchmark using RoBERTa-base and RoBERTa-large demonstrate that CRAFT achieves competitive performance with existing methods while requiring only 41K Tucker adaptation parameters--a count independent of model dimension and depth at fixed Tucker ranks.
【5】SubQuad: Near-Quadratic-Free Structure Inference with Distribution-Balanced Objectives in Adaptive Receptor framework
标题：SubQuad：适应性受体框架中具有分布平衡目标的近二次自由结构推理
链接：https://arxiv.org/abs/2602.17330
备注：27 pages, 9 figures
摘要：在人群规模的适应性免疫库的比较分析受到两个实际瓶颈的阻碍：成对亲和力评估的近二次成本和数据集不平衡，掩盖了临床上重要的少数克隆型。我们介绍了SubQuad，这是一个端到端的流水线，通过将抗原感知、近次二次检索与GPU加速的亲和内核、学习的多模态融合和公平性约束聚类相结合来解决这些挑战。该系统采用紧凑的MinHash预过滤，以大幅减少候选人的比较，可区分的门控模块，自适应加权互补对齐和嵌入通道的每对的基础上，和一个自动校准程序，强制比例代表罕见的抗原特异性亚组。在大型病毒和肿瘤库中，SubQuad在吞吐量和峰值内存使用方面实现了可测量的收益，同时保留或提高了召回@k、集群纯度和亚组公平性。通过共同设计索引、相似性融合和公平意识目标，SubQuad为库挖掘和下游翻译任务（例如疫苗靶点优先级排序和生物标志物发现）提供了一个可扩展的、偏见意识平台。
摘要：Comparative analysis of adaptive immune repertoires at population scale is hampered by two practical bottlenecks: the near-quadratic cost of pairwise affinity evaluations and dataset imbalances that obscure clinically important minority clonotypes. We introduce SubQuad, an end-to-end pipeline that addresses these challenges by combining antigen-aware, near-subquadratic retrieval with GPU-accelerated affinity kernels, learned multimodal fusion, and fairness-constrained clustering. The system employs compact MinHash prefiltering to sharply reduce candidate comparisons, a differentiable gating module that adaptively weights complementary alignment and embedding channels on a per-pair basis, and an automated calibration routine that enforces proportional representation of rare antigen-specific subgroups. On large viral and tumor repertoires SubQuad achieves measured gains in throughput and peak memory usage while preserving or improving recall@k, cluster purity, and subgroup equity. By co-designing indexing, similarity fusion, and equity-aware objectives, SubQuad offers a scalable, bias-aware platform for repertoire mining and downstream translational tasks such as vaccine target prioritization and biomarker discovery.
【6】Structured Prototype-Guided Adaptation for EEG Foundation Models
标题：EEG基础模型的结构化原型引导自适应
链接：https://arxiv.org/abs/2602.17251
摘要：脑电图（EEG）基础模型（EFM）在完全微调下实现了强大的性能，但在受试者级监督有限时表现出较差的泛化能力，这是现实世界临床环境中的一个常见约束。我们发现，这种失败不仅源于有限的监督，但从结构上的不匹配之间的嘈杂，有限的监督和高塑性参数空间的EFM。为了应对这一挑战，我们提出了SCOPE，一个结构化的信心感知原型指导的自适应框架EFM微调。SCOPE遵循两阶段管道。在第一阶段，我们通过学习几何正则化的任务先验知识，在所得到的嵌入上构建平衡的类级原型，并根据它们的协议生成置信度感知的伪标签来过滤未标记数据上的不可靠信号，从而构建可靠的外部监督。在第二阶段中，我们引入ProAdapter，它通过一个轻量级的适配器适应冻结EEG基础模型的结构化原型。在三个EEG任务和五个基础模型骨干的实验表明，SCOPE始终实现强大的性能和效率下的标签限制的跨学科设置。
摘要：Electroencephalography (EEG) foundation models (EFMs) have achieved strong performance under full fine-tuning but exhibit poor generalization when subject-level supervision is limited, a common constraint in real-world clinical settings. We show that this failure stems not merely from limited supervision, but from a structural mismatch between noisy, limited supervision and the highly plastic parameter space of EFMs. To address this challenge, we propose SCOPE, a Structured COnfidence-aware Prototype-guided adaptation framework for EFM fine-tuning. SCOPE follows a two-stage pipeline. In the first stage, we construct reliable external supervision by learning geometry-regularized task priors, constructing balanced class-level prototypes over the resulting embeddings, and producing confidence-aware pseudo-labels from their agreement to filter unreliable signals on unlabeled data. In the second stage, we introduce ProAdapter, which adapts frozen EEG foundation models via a lightweight adapter conditioned on the structured prototypes. Experiments across three EEG tasks and five foundation model backbones demonstrate that SCOPE consistently achieves strong performance and efficiency under label-limited cross-subject settings.
【7】VP-VAE: Rethinking Vector Quantization via Adaptive Vector Perturbation
标题：VP-VAE：通过自适应载体微扰重新思考载体量化
链接：https://arxiv.org/abs/2602.17133
摘要：矢量量化变分自编码器（VQ-VAE）是现代生成式建模的基础，但由于表示学习和离散码本优化的固有耦合，它们经常遭受训练不稳定和“码本崩溃”的问题。在本文中，我们提出了VP-VAE（矢量扰动VAE），这是一种新的范例，通过消除训练过程中对显式码本的需要，将表示学习从离散化中分离出来。我们的关键见解是，从神经网络的角度来看，执行量化主要表现为在潜在空间中注入结构化扰动。相应地，VP-VAE用通过Metropolis-Hastings采样产生的分布一致和尺度自适应的潜在扰动代替不可微量化器。这种设计使得能够在没有码本的情况下进行稳定的训练，同时使模型对推理时间量化误差具有鲁棒性。此外，在近似均匀的潜变量的假设下，我们推导出FSP（有限标量扰动），VP-VAE的一个轻量级变体，提供了一个统一的理论解释和FSQ式固定量化器的实际改进。在图像和音频基准上的大量实验表明，VP-VAE和FSP提高了重建保真度，实现了更平衡的令牌使用，同时避免了耦合码本训练固有的不稳定性。
摘要 ：Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental to modern generative modeling, yet they often suffer from training instability and "codebook collapse" due to the inherent coupling of representation learning and discrete codebook optimization. In this paper, we propose VP-VAE (Vector Perturbation VAE), a novel paradigm that decouples representation learning from discretization by eliminating the need for an explicit codebook during training. Our key insight is that, from the neural network's viewpoint, performing quantization primarily manifests as injecting a structured perturbation in latent space. Accordingly, VP-VAE replaces the non-differentiable quantizer with distribution-consistent and scale-adaptive latent perturbations generated via Metropolis--Hastings sampling. This design enables stable training without a codebook while making the model robust to inference-time quantization error. Moreover, under the assumption of approximately uniform latent variables, we derive FSP (Finite Scalar Perturbation), a lightweight variant of VP-VAE that provides a unified theoretical explanation and a practical improvement for FSQ-style fixed quantizers. Extensive experiments on image and audio benchmarks demonstrate that VP-VAE and FSP improve reconstruction fidelity and achieve substantially more balanced token usage, while avoiding the instability inherent to coupled codebook training.
【8】Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum
标题：Adam改进Muon：具有动量的自适应矩估计
链接：https://arxiv.org/abs/2602.17080
备注：39 pages, 6 figures
摘要：有效的随机优化通常将在确定性机制中表现良好的更新方向与适应随机扰动的机制相结合。Adam使用自适应矩估计来提高稳定性，而Muon通过正交化动量利用权重层的矩阵结构，在大型语言模型训练中表现出卓越的性能。我们提出了一个新的优化器和对角扩展，NAMO和NAMO-D，提供了第一个原则性的整合正交动量与基于范数的亚当型噪声适应。NAMO使用单个自适应步长缩放正交动量，保持正交性，同时以可忽略的额外成本改进μ子。NAMO-D将正交化动量右乘一个具有箝位项的对角矩阵。这种设计实现了神经元噪声自适应，并与常见的近块对角Hessian结构保持一致。在标准的假设下，我们建立了最佳的收敛速度，这两种算法在确定性的设置，并表明，在随机设置，其收敛保证适应随机梯度的噪声水平。预训练GPT-2模型的实验表明，与AdamW和Muon基线相比，NAMO和NAMO-D的性能都有所改善，NAMO-D通过额外的箝位超参数实现了NAMO的进一步增益，该参数平衡了保持良好状态的更新方向和利用细粒度噪声自适应的竞争目标。
摘要：Efficient stochastic optimization typically integrates an update direction that performs well in the deterministic regime with a mechanism adapting to stochastic perturbations. While Adam uses adaptive moment estimates to promote stability, Muon utilizes the weight layers' matrix structure via orthogonalized momentum, showing superior performance in large language model training. We propose a new optimizer and a diagonal extension, NAMO and NAMO-D, providing the first principled integration of orthogonalized momentum with norm-based Adam-type noise adaptation. NAMO scales orthogonalized momentum using a single adaptive stepsize, preserving orthogonality while improving upon Muon at negligible additional cost. NAMO-D instead right-multiplies orthogonalized momentum by a diagonal matrix with clamped entries. This design enables neuron-wise noise adaptation and aligns with the common near block-diagonal Hessian structure. Under standard assumptions, we establish optimal convergence rates for both algorithms in the deterministic setting and show that, in the stochastic setting, their convergence guarantees adapt to the noise level of stochastic gradients. Experiments on pretraining GPT-2 models demonstrate improved performance of both NAMO and NAMO-D compared to the AdamW and Muon baselines, with NAMO-D achieving further gains over NAMO via an additional clamping hyperparameter that balances the competing goals of maintaining a well-conditioned update direction and leveraging fine-grained noise adaptation.
【9】Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning
标题：岭回归中的最佳无约束自蒸馏：严格改进、精确渐进和一次调整
链接：https://arxiv.org/abs/2602.17565
备注：78 pages, 25 figures
摘要：自升华（SD）是使用相同的架构和训练数据，在地面实况标签和教师自己的预测的混合物上对学生进行再训练的过程。虽然SD已被经验证明，往往提高泛化，其正式的保证仍然有限。我们研究了无约束条件下岭回归的SD，其中混合权重可能在单位区间之外。在训练数据的条件下，在没有任何分布假设的情况下，我们证明了对于任何平方预测风险（包括分布外），最优混合学生严格地改进了每个正则化水平$λ> 0$的岭教师，其中教师岭风险$R（λ）$是非平稳的（即，$R '（λ）\neq 0$）。我们得到了最优混合权重$λ ^\star（λ）$对于任意值$λ$的封闭形式表达式，并证明它服从符号规则：$\operatorname{sign}（λ ^\star（λ））=-\operatorname{sign}（R '（λ））$。特别地，$^\star（λ）$可以是负的，这是过度正则化的情况。为了量化的风险改善，由于SD，我们得出确切的确定性等价物的最佳SD风险的比例渐近制度（其中的样本和特征大小$n$和$p$都发散，但他们的纵横比$p/n$收敛）一般各向异性协方差和确定性信号。我们的渐近分析扩展标准的二阶脊确定性等价物，其四阶类似物使用块线性化，这可能是独立的利益。从实用的角度来看，我们提出了一个一致的一次性调整方法，以估计$^\star$没有网格搜索，样本分裂，或改装。在真实世界数据集和预训练神经网络特征上的实验支持了我们的理论和一次性调整方法。
摘要：Self-distillation (SD) is the process of retraining a student on a mixture of ground-truth labels and the teacher's own predictions using the same architecture and training data. Although SD has been empirically shown to often improve generalization, its formal guarantees remain limited. We study SD for ridge regression in unconstrained setting in which the mixing weight $ξ$ may be outside the unit interval. Conditioned on the training data and without any distributional assumptions, we prove that for any squared prediction risk (including out-of-distribution), the optimally mixed student strictly improves upon the ridge teacher for every regularization level $λ> 0$ at which the teacher ridge risk $R(λ)$ is nonstationary (i.e., $R'(λ) \neq 0$). We obtain a closed-form expression for the optimal mixing weight $ξ^\star(λ)$ for any value of $λ$ and show that it obeys the sign rule: $\operatorname{sign}(ξ^\star(λ))=-\operatorname{sign}(R'(λ))$. In particular, $ξ^\star(λ)$ can be negative, which is the case in over-regularized regimes. To quantify the risk improvement due to SD, we derive exact deterministic equivalents for the optimal SD risk in the proportional asymptotics regime (where the sample and feature sizes $n$ and $p$ both diverge but their aspect ratio $p/n$ converges) under general anisotropic covariance and deterministic signals. Our asymptotic analysis extends standard second-order ridge deterministic equivalents to their fourth-order analogs using block linearization, which may be of independent interest. From a practical standpoint, we propose a consistent one-shot tuning method to estimate $ξ^\star$ without grid search, sample splitting, or refitting. Experiments on real-world datasets and pretrained neural network features support our theory and the one-shot tuning method.
【10】Adaptive Decentralized Composite Optimization via Three-Operator Splitting
标题：通过三操作员分裂的自适应分散复合优化
链接：https://arxiv.org/abs/2602.17545
备注：25 pages, 3 figures
摘要：本文研究了网络上的分散优化问题，其中代理人最小化了一个局部光滑（强）凸损失和一个非光滑凸扩展值项之和。我们提出了分散的方法，其中代理{\它自适应}调整他们的步长，通过本地回溯程序加上轻量级的最小共识协议。我们的设计源于三个运营商分裂因式分解应用到一个等价的重新制定的问题。重新制定被赋予了一个新的BCV预处理度量（Bertsekas-O 'Connor-Vandenberghe），这使得有效的分散实施和本地步长调整。我们建立强大的收敛保证。在纯凸性下，所提出的方法以次线性速度收敛。在和函数的强凸性条件下，并假设非光滑部分是部分光滑的，我们进一步证明了线性收敛性。数值实验证实了理论和突出的有效性，建议的自适应步长策略。
摘要 ：The paper studies decentralized optimization over networks, where agents minimize a sum of {\it locally} smooth (strongly) convex losses and plus a nonsmooth convex extended value term. We propose decentralized methods wherein agents {\it adaptively} adjust their stepsize via local backtracking procedures coupled with lightweight min-consensus protocols. Our design stems from a three-operator splitting factorization applied to an equivalent reformulation of the problem. The reformulation is endowed with a new BCV preconditioning metric (Bertsekas-O'Connor-Vandenberghe), which enables efficient decentralized implementation and local stepsize adjustments. We establish robust convergence guarantees. Under mere convexity, the proposed methods converge with a sublinear rate. Under strong convexity of the sum-function, and assuming the nonsmooth component is partly smooth, we further prove linear convergence. Numerical experiments corroborate the theory and highlight the effectiveness of the proposed adaptive stepsize strategy.
强化学习(2篇)
【1】LexiSafe: Offline Safe Reinforcement Learning with Lexicographic Safety-Reward Hierarchy
标题：LexiSafe：具有词典安全-奖励层次结构的离线安全强化学习
链接：https://arxiv.org/abs/2602.17312
备注：17th ACM/IEEE International Conference on Cyber-Physical Systems
摘要：离线安全强化学习（RL）对于网络物理系统（CPS）越来越重要，在这些系统中，训练期间的安全违规是不可接受的，并且只有预先收集的数据可用。现有的离线安全RL方法通常通过约束放松或联合优化来平衡奖励-安全权衡，但它们通常缺乏防止安全漂移的结构机制。我们提出了LexiSafe，一个字典式离线RL框架，旨在保持安全对齐的行为。我们首先开发LexiSafe-SC，标准离线安全RL的单成本配方，并推导出安全违规和性能次优界限，共同产生样本复杂性保证。然后，我们扩展的框架与LexiSafe-MC，它支持多个安全成本，并承认自己的样本复杂性分析的层次安全要求。从经验上讲，LexiSafe与受约束的离线基线相比，减少了安全违规，提高了任务性能。通过统一的词典优先级与结构偏见，LexiSafe提供了一个实用的和理论基础的方法，安全关键CPS决策。
摘要：Offline safe reinforcement learning (RL) is increasingly important for cyber-physical systems (CPS), where safety violations during training are unacceptable and only pre-collected data are available. Existing offline safe RL methods typically balance reward-safety tradeoffs through constraint relaxation or joint optimization, but they often lack structural mechanisms to prevent safety drift. We propose LexiSafe, a lexicographic offline RL framework designed to preserve safety-aligned behavior. We first develop LexiSafe-SC, a single-cost formulation for standard offline safe RL, and derive safety-violation and performance-suboptimality bounds that together yield sample-complexity guarantees. We then extend the framework to hierarchical safety requirements with LexiSafe-MC, which supports multiple safety costs and admits its own sample-complexity analysis. Empirically, LexiSafe demonstrates reduced safety violations and improved task performance compared to constrained offline baselines. By unifying lexicographic prioritization with structural bias, LexiSafe offers a practical and theoretically grounded approach for safety-critical CPS decision-making.
【2】Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization
标题：深度强化学习用于最佳投资组合配置：与均值方差优化的比较研究
链接：https://arxiv.org/abs/2602.17098
备注：9 pages, 6 figures. Published at the FinPlan'23 Workshop, the 33rd International Conference on Automated Planning and Scheduling (ICAPS 2023)
摘要：投资组合管理是监督一组投资（称为投资组合）的过程，目的是实现预定的投资目标。投资组合优化是一个关键组成部分，涉及分配投资组合资产，以最大限度地提高回报，同时最大限度地降低风险。它通常由金融专业人士进行，他们使用定量技术和投资专业知识的组合来做出有关投资组合分配的决策。 深度强化学习（DRL）最近的应用在通过对历史市场数据训练无模型代理来优化投资组合分配时显示出了有希望的结果。这些方法中的许多方法将其结果与基本基准或其他最先进的DRL代理进行比较，但通常无法将其性能与金融专业人士在实际环境中使用的传统方法进行比较。最常用的方法之一是均值-方差投资组合优化（MVO），它使用历史时间序列信息来估计预期资产收益和协方差，然后用于优化投资目标。 我们的工作是一个彻底的比较无模型的DRL和MVO的最优投资组合配置。我们详细介绍了如何在实践中使DRL用于投资组合优化工作的细节，还注意到MVO所需的调整。回测结果表明，DRL代理在许多指标上表现出色，包括夏普比率，最大提款和绝对回报。
摘要：Portfolio Management is the process of overseeing a group of investments, referred to as a portfolio, with the objective of achieving predetermined investment goals. Portfolio optimization is a key component that involves allocating the portfolio assets so as to maximize returns while minimizing risk taken. It is typically carried out by financial professionals who use a combination of quantitative techniques and investment expertise to make decisions about the portfolio allocation. Recent applications of Deep Reinforcement Learning (DRL) have shown promising results when used to optimize portfolio allocation by training model-free agents on historical market data. Many of these methods compare their results against basic benchmarks or other state-of-the-art DRL agents but often fail to compare their performance against traditional methods used by financial professionals in practical settings. One of the most commonly used methods for this task is Mean-Variance Portfolio Optimization (MVO), which uses historical time series information to estimate expected asset returns and covariances, which are then used to optimize for an investment objective. Our work is a thorough comparison between model-free DRL and MVO for optimal portfolio allocation. We detail the specifics of how to make DRL for portfolio optimization work in practice, also noting the adjustments needed for MVO. Backtest results demonstrate strong performance of the DRL agent across many metrics, including Sharpe ratio, maximum drawdowns, and absolute returns.
元学习(1篇)
【1】Adapting Actively on the Fly: Relevance-Guided Online Meta-Learning with Latent Concepts for Geospatial Discovery
标题：动态积极适应：具有地理空间发现潜在概念的相关性引导在线元学习
链接：https://arxiv.org/abs/2602.17605
摘要：在许多现实环境中，如环境监测，灾害响应或公共卫生，具有昂贵和困难的数据收集和动态环境，从未观察到的区域进行战略性采样对于在严格的资源限制下有效地发现隐藏目标至关重要。然而，稀疏和有偏见的地理空间地面实况限制了现有基于学习的方法（如强化学习）的适用性。为了解决这个问题，我们提出了一个统一的地理空间发现框架，集成了主动学习，在线元学习和概念引导推理。我们的方法引入了两个关键创新，建立在一个共享的概念 * 概念相关性 *，它捕捉了特定领域的因素如何影响目标的存在：一个 * 概念加权不确定性采样策略 *，其中不确定性是由基于现成的特定领域的概念（例如，土地覆盖，源接近度）;以及一个 * 相关性感知元批形成策略 *，该策略在在线元更新期间促进语义多样性，提高动态环境中的泛化能力。我们的实验包括对致癌PFAS（全氟烷基和多氟烷基物质）污染的真实数据集进行测试，展示了我们的方法在有限数据和不同环境下发现目标的可靠性。
摘要：In many real-world settings, such as environmental monitoring, disaster response, or public health, with costly and difficult data collection and dynamic environments, strategically sampling from unobserved regions is essential for efficiently uncovering hidden targets under tight resource constraints. Yet, sparse and biased geospatial ground truth limits the applicability of existing learning-based methods, such as reinforcement learning. To address this, we propose a unified geospatial discovery framework that integrates active learning, online meta-learning, and concept-guided reasoning. Our approach introduces two key innovations built on a shared notion of *concept relevance*, which captures how domain-specific factors influence target presence: a *concept-weighted uncertainty sampling strategy*, where uncertainty is modulated by learned relevance based on readily-available domain-specific concepts (e.g., land cover, source proximity); and a *relevance-aware meta-batch formation strategy* that promotes semantic diversity during online-meta updates, improving generalization in dynamic environments. Our experiments include testing on a real-world dataset of cancer-causing PFAS (Per- and polyfluoroalkyl substances) contamination, showcasing our method's reliability at uncovering targets with limited data and a varying environment.
医学相关(5篇)
【1】Position: Evaluation of ECG Representations Must Be Fixed
标题：位置：心电图表示的评估必须固定
链接：https://arxiv.org/abs/2602.17531
备注：Project website at https://ecgfix.csail.mit.edu/
摘要：该立场文件认为，必须固定12导联ECG表示学习中的当前基准实践，以确保进展可靠并与临床有意义的目标保持一致。该领域在很大程度上集中在三个由心律失常和波形形态标签主导的公共多标签基准（PTB-XL，CPSC 2018，CSN）上，尽管已知ECG可编码更广泛的临床信息。我们认为，下游评估应扩大到包括结构性心脏病和患者水平的预测，除了其他不断发展的心电图相关终点，作为相关的临床目标的评估。接下来，我们概述了多标签、不平衡设置的评估最佳实践，并表明当应用它们时，文献中关于哪些表示表现最好的当前结论会发生改变。此外，我们展示了令人惊讶的结果，即具有线性评估的随机初始化编码器在许多任务上与最先进的预训练相匹配。这促使使用随机编码器作为合理的基线模型。我们通过对六种评估设置中的三种代表性ECG预训练方法进行实证评估来证实我们的观察结果：三个标准基准，结构性疾病数据集，血流动力学推断和患者预测。
摘要：This position paper argues that current benchmarking practice in 12-lead ECG representation learning must be fixed to ensure progress is reliable and aligned with clinically meaningful objectives. The field has largely converged on three public multi-label benchmarks (PTB-XL, CPSC2018, CSN) dominated by arrhythmia and waveform-morphology labels, even though the ECG is known to encode substantially broader clinical information. We argue that downstream evaluation should expand to include an assessment of structural heart disease and patient-level forecasting, in addition to other evolving ECG-related endpoints, as relevant clinical targets. Next, we outline evaluation best practices for multi-label, imbalanced settings, and show that when they are applied, the literature's current conclusion about which representations perform best is altered. Furthermore, we demonstrate the surprising result that a randomly initialized encoder with linear evaluation matches state-of-the-art pre-training on many tasks. This motivates the use of a random encoder as a reasonable baseline model. We substantiate our observations with an empirical evaluation of three representative ECG pre-training approaches across six evaluation settings: the three standard benchmarks, a structural disease dataset, hemodynamic inference, and patient forecasting.
【2】A feature-stable and explainable machine learning framework for trustworthy decision-making under incomplete clinical data
标题：一个特征稳定且可解释的机器学习框架，用于在不完整临床数据下做出值得信赖的决策
链接：https://arxiv.org/abs/2602.17364
摘要：机器学习模型越来越多地应用于生物医学数据，但它们在高风险领域的应用仍然受到鲁棒性差、可解释性有限以及在实际数据扰动（如缺失）下学习特征不稳定的限制。特别是，如果当数据完整性发生变化时，模型的关键特征发生波动，从而破坏了可重复性和下游决策，那么实现高预测性能的模型可能仍然无法激发信任。在这里，我们提出了CACTUS（用于揭示结构的综合抽象和分类工具），这是一个可解释的机器学习框架，旨在解决小型，异构和不完整的临床数据集中的这些挑战。CACTUS集成了特征抽象、可解释的分类和系统的特征稳定性分析，以量化在数据质量下降时如何一致地保留信息特征。使用包括568名膀胱癌患者的真实血尿队列，我们将CACTUS与广泛使用的机器学习方法（包括随机森林和梯度增强方法）进行基准测试，并在随机引入的缺失数据的受控水平下进行。我们证明，CACTUS实现了具有竞争力或优越的预测性能，同时随着缺失率的增加，包括在性别分层分析中，保持了排名靠前的特征的显着更高的稳定性。我们的研究结果表明，特征稳定性提供了与传统性能指标互补的信息，对于评估应用于生物医学数据的机器学习模型的可信度至关重要。通过明确量化对缺失数据的鲁棒性，并优先考虑可解释的稳定特征，CACTUS为可信的数据驱动决策支持提供了一个可推广的框架。
摘要：Machine learning models are increasingly applied to biomedical data, yet their adoption in high stakes domains remains limited by poor robustness, limited interpretability, and instability of learned features under realistic data perturbations, such as missingness. In particular, models that achieve high predictive performance may still fail to inspire trust if their key features fluctuate when data completeness changes, undermining reproducibility and downstream decision-making. Here, we present CACTUS (Comprehensive Abstraction and Classification Tool for Uncovering Structures), an explainable machine learning framework explicitly designed to address these challenges in small, heterogeneous, and incomplete clinical datasets. CACTUS integrates feature abstraction, interpretable classification, and systematic feature stability analysis to quantify how consistently informative features are preserved as data quality degrades. Using a real-world haematuria cohort comprising 568 patients evaluated for bladder cancer, we benchmark CACTUS against widely used machine learning approaches, including random forests and gradient boosting methods, under controlled levels of randomly introduced missing data. We demonstrate that CACTUS achieves competitive or superior predictive performance while maintaining markedly higher stability of top-ranked features as missingness increases, including in sex-stratified analyses. Our results show that feature stability provides information complementary to conventional performance metrics and is essential for assessing the trustworthiness of machine learning models applied to biomedical data. By explicitly quantifying robustness to missing data and prioritising interpretable, stable features, CACTUS offers a generalizable framework for trustworthy data-driven decision support.
【3】MedClarify: An information-seeking AI agent for medical diagnosis with case-specific follow-up questions
标题：MedClarify：一种用于医疗诊断的信息寻求人工智能代理，并提供特定病例的后续问题
链接：https://arxiv.org/abs/2602.17308
摘要：大型语言模型（LLM）越来越多地用于医学诊断任务。在临床实践中，仅从最初的患者表现很少能立即推断出正确的诊断。相反，达到诊断通常涉及系统的病史采集，在此期间，临床医生通过反复询问来解决不确定性，从而对多种潜在疾病进行推理。这一过程需要考虑不同的诊断，并积极排除需要立即干预的紧急情况。然而，医学LLM产生信息丰富的后续问题的能力，从而对鉴别诊断的原因仍然有待探索。在这里，我们介绍MedClarify，这是一种用于信息搜索的AI代理，可以生成迭代推理的后续问题，以支持诊断决策。具体来说，MedClarify计算类似于鉴别诊断的候选诊断列表，然后主动生成旨在减少诊断不确定性的后续问题。通过选择具有最高预期信息增益的问题，MedClarify能够实现有针对性的、不确定性感知的推理，以提高诊断性能。在我们的实验中，我们首先证明了当前LLM在医学推理中的局限性，这通常会产生多个类似的可能诊断，特别是当患者病例不完整或诊断相关信息缺失时。然后，我们表明，我们的信息理论推理方法可以产生有效的后续问题，从而减少诊断错误约27个百分点（p.p.）。与标准单发LLM基线相比。总而言之，MedClarify提供了一条通过代理信息寻求改善医学LLM的途径，从而促进与反映现实世界临床推理的迭代和不确定性的医学LLM的有效对话。
摘要 ：Large language models (LLMs) are increasingly used for diagnostic tasks in medicine. In clinical practice, the correct diagnosis can rarely be immediately inferred from the initial patient presentation alone. Rather, reaching a diagnosis often involves systematic history taking, during which clinicians reason over multiple potential conditions through iterative questioning to resolve uncertainty. This process requires considering differential diagnoses and actively excluding emergencies that demand immediate intervention. Yet, the ability of medical LLMs to generate informative follow-up questions and thus reason over differential diagnoses remains underexplored. Here, we introduce MedClarify, an AI agent for information-seeking that can generate follow-up questions for iterative reasoning to support diagnostic decision-making. Specifically, MedClarify computes a list of candidate diagnoses analogous to a differential diagnosis, and then proactively generates follow-up questions aimed at reducing diagnostic uncertainty. By selecting the question with the highest expected information gain, MedClarify enables targeted, uncertainty-aware reasoning to improve diagnostic performance. In our experiments, we first demonstrate the limitations of current LLMs in medical reasoning, which often yield multiple, similarly likely diagnoses, especially when patient cases are incomplete or relevant information for diagnosis is missing. We then show that our information-theoretic reasoning approach can generate effective follow-up questioning and thereby reduces diagnostic errors by ~27 percentage points (p.p.) compared to a standard single-shot LLM baseline. Altogether, MedClarify offers a path to improve medical LLMs through agentic information-seeking and to thus promote effective dialogues with medical LLMs that reflect the iterative and uncertain nature of real-world clinical reasoning.
【4】LiveClin: A Live Clinical Benchmark without Leakage
标题：LiveClin：无泄漏的实时临床基准
链接：https://arxiv.org/abs/2602.16747
摘要：医学LLM评估的可靠性受到数据污染和知识过时的严重破坏，导致静态基准的分数膨胀。为了应对这些挑战，我们引入了LiveClin，这是一个旨在近似真实世界临床实践的实时基准。LiveClin建立在当代同行评审的病例报告基础上，每半年更新一次，确保临床通用性并抵御数据污染。使用经过验证的涉及239名医生的人工智能工作流程，我们将真实的患者病例转化为跨越整个临床路径的复杂的多模式评估场景。该基准目前包括1 407份案例报告和6 605个问题。我们对LiveClin上的26个模型进行了评估，揭示了这些现实世界场景的巨大困难，表现最好的模型的案例准确率仅为35.7%。在与人类专家的基准测试中，主任医师达到了最高的准确性，其次是主治医师，两者都超过了大多数模型。因此，LiveClin提供了一个不断发展的，以临床为基础的框架，以指导医学LLM的发展，缩小这一差距，实现更高的可靠性和现实世界的效用。我们的数据和代码可在https://github.com/AQ-MedAI/LiveClin上公开获取。
摘要：The reliability of medical LLM evaluation is critically undermined by data contamination and knowledge obsolescence, leading to inflated scores on static benchmarks. To address these challenges, we introduce LiveClin, a live benchmark designed for approximating real-world clinical practice. Built from contemporary, peer-reviewed case reports and updated biannually, LiveClin ensures clinical currency and resists data contamination. Using a verified AI-human workflow involving 239 physicians, we transform authentic patient cases into complex, multimodal evaluation scenarios that span the entire clinical pathway. The benchmark currently comprises 1,407 case reports and 6,605 questions. Our evaluation of 26 models on LiveClin reveals the profound difficulty of these real-world scenarios, with the top-performing model achieving a Case Accuracy of just 35.7%. In benchmarking against human experts, Chief Physicians achieved the highest accuracy, followed closely by Attending Physicians, with both surpassing most models. LiveClin thus provides a continuously evolving, clinically grounded framework to guide the development of medical LLMs towards closing this gap and achieving greater reliability and real-world utility. Our data and code are publicly available at https://github.com/AQ-MedAI/LiveClin.
【5】U-FedTomAtt: Ultra-lightweight Federated Learning with Attention for Tomato Disease Recognition
标题：U-FedTomAtt：超轻量级联邦学习，关注番茄疾病识别
链接：https://arxiv.org/abs/2602.16749
备注：10 pages and 4 figures
摘要：联邦学习已经成为部署智能农业解决方案的一种保护隐私和有效的方法。在地理上分散的农场进行准确的边缘诊断对于识别可持续农业中的番茄疾病至关重要。传统的集中式培训将原始数据聚集在中央服务器上，导致通信开销、隐私风险和延迟。同时，边缘设备需要轻量级网络在有限的资源内有效运行。在本文中，我们提出了U-FedTomAtt，一个超轻量级的联邦学习框架，在资源受限和分布式环境中的番茄病害识别。该模型仅包含245.34K参数和71.41MFLOPS。首先，我们提出了一个超轻量的神经网络与扩张瓶颈（DBNeck）模块和线性Transformer，以尽量减少计算和内存开销。为了减轻潜在的准确性损失，一种新的局部-全局剩余注意力（LoGRA）模块。其次，我们提出了联邦双自适应权重聚合（FedDAWA）算法，提高了全局模型的准确性。第三，我们的框架进行了验证，使用三个基准数据集的番茄病害模拟联邦设置。实验结果表明，该方法在SLIF-Tomato和PlantVillage番茄数据集上分别获得了0.9910%和0.9915%Top-1准确率以及0.9923%和0.9897%F1分数。
摘要：Federated learning has emerged as a privacy-preserving and efficient approach for deploying intelligent agricultural solutions. Accurate edge-based diagnosis across geographically dispersed farms is crucial for recognising tomato diseases in sustainable farming. Traditional centralised training aggregates raw data on a central server, leading to communication overhead, privacy risks and latency. Meanwhile, edge devices require lightweight networks to operate effectively within limited resources. In this paper, we propose U-FedTomAtt, an ultra-lightweight federated learning framework with attention for tomato disease recognition in resource-constrained and distributed environments. The model comprises only 245.34K parameters and 71.41 MFLOPS. First, we propose an ultra-lightweight neural network with dilated bottleneck (DBNeck) modules and a linear transformer to minimise computational and memory overhead. To mitigate potential accuracy loss, a novel local-global residual attention (LoGRA) module is incorporated. Second, we propose the federated dual adaptive weight aggregation (FedDAWA) algorithm that enhances global model accuracy. Third, our framework is validated using three benchmark datasets for tomato diseases under simulated federated settings. Experimental results show that the proposed method achieves 0.9910% and 0.9915% Top-1 accuracy and 0.9923% and 0.9897% F1-scores on SLIF-Tomato and PlantVillage tomato datasets, respectively.
推荐(1篇)
【1】LiveGraph: Active-Structure Neural Re-ranking for Exercise Recommendation
标题：LiveCurve：针对锻炼推荐的主动结构神经重新排序
链接：https://arxiv.org/abs/2602.17036
备注：19 pages, 5 figures
摘要：数字学习环境的不断扩展促进了对能够提供个性化教育内容的智能系统的需求。虽然目前的练习推荐框架已经取得了重大进展，但它们经常遇到有关学生参与度长尾分布和无法适应特殊学习轨迹的障碍。我们提出了LiveGraph，一种新的主动结构神经重新排名框架，旨在克服这些限制。我们的方法利用基于图形的表示增强策略，以弥合活跃和不活跃的学生之间的信息差距，同时集成了动态重新排名机制，以促进内容多样性。通过优先考虑学习历史中的结构关系，该模型有效地平衡了推荐精度与教学多样性。在多个真实世界数据集上进行的综合实验评估表明，LiveGraph在预测准确性和运动多样性的广度方面都超过了当代基线。
摘要：The continuous expansion of digital learning environments has catalyzed the demand for intelligent systems capable of providing personalized educational content. While current exercise recommendation frameworks have made significant strides, they frequently encounter obstacles regarding the long-tailed distribution of student engagement and the failure to adapt to idiosyncratic learning trajectories. We present LiveGraph, a novel active-structure neural re-ranking framework designed to overcome these limitations. Our approach utilizes a graph-based representation enhancement strategy to bridge the information gap between active and inactive students while integrating a dynamic re-ranking mechanism to foster content diversity. By prioritizing the structural relationships within learning histories, the proposed model effectively balances recommendation precision with pedagogical variety. Comprehensive experimental evaluations conducted on multiple real-world datasets demonstrate that LiveGraph surpasses contemporary baselines in both predictive accuracy and the breadth of exercise diversity.
自动驾驶|车辆|车道检测等(2篇)
【1】Conditional Flow Matching for Continuous Anomaly Detection in Autonomous Driving on a Manifold-Aware Spectral Space
标题：基于条件流匹配的多通道感知谱空间上自动驾驶连续异常检测
链接：https://arxiv.org/abs/2602.17586
摘要：4级自动驾驶汽车（AV）的安全验证目前受到无法使用传统的基于规则的方法来扩展对罕见的高风险长尾场景的检测的影响。我们提出了Deep-Flow，一个用于安全关键异常检测的无监督框架，该框架利用最优传输条件流匹配（OT-CFM）来表征专家人类驾驶行为的连续概率密度。与在不稳定的高维坐标空间中操作的标准生成方法不同，Deep-Flow通过主成分分析（PCA）瓶颈将生成过程限制在低秩谱流形上。这通过设计确保了运动学平滑性，并且能够计算精确的雅可比迹线，以进行数值稳定的确定性对数似然估计。为了解决复杂连接处的多模态模糊性，我们利用具有车道感知目标调节的Early Fusion Transformer编码器，该编码器具有与流量头的直接跳过连接，以保持整个网络的意图完整性。我们引入了一个运动学复杂性加权方案，在无模拟训练过程中优先考虑高能量机动（通过路径曲折度和急动度量化）。在Waymo开放运动数据集（WOMD）上进行评估，我们的框架针对启发式黄金安全关键事件集实现了0.766的AUC-ROC。更重要的是，我们的分析揭示了运动学的危险和语义不遵守之间的根本区别。Deep-Flow通过暴露传统安全过滤器忽略的分布外行为（例如车道边界违规和非规范交叉点机动）来识别关键的可预测性差距。这项工作为定义统计安全门提供了严格的数学基础，为自动驾驶车队的安全部署提供了客观的数据驱动验证。
摘要：Safety validation for Level 4 autonomous vehicles (AVs) is currently bottlenecked by the inability to scale the detection of rare, high-risk long-tail scenarios using traditional rule-based heuristics. We present Deep-Flow, an unsupervised framework for safety-critical anomaly detection that utilizes Optimal Transport Conditional Flow Matching (OT-CFM) to characterize the continuous probability density of expert human driving behavior. Unlike standard generative approaches that operate in unstable, high-dimensional coordinate spaces, Deep-Flow constrains the generative process to a low-rank spectral manifold via a Principal Component Analysis (PCA) bottleneck. This ensures kinematic smoothness by design and enables the computation of the exact Jacobian trace for numerically stable, deterministic log-likelihood estimation. To resolve multi-modal ambiguity at complex junctions, we utilize an Early Fusion Transformer encoder with lane-aware goal conditioning, featuring a direct skip-connection to the flow head to maintain intent-integrity throughout the network. We introduce a kinematic complexity weighting scheme that prioritizes high-energy maneuvers (quantified via path tortuosity and jerk) during the simulation-free training process. Evaluated on the Waymo Open Motion Dataset (WOMD), our framework achieves an AUC-ROC of 0.766 against a heuristic golden set of safety-critical events. More significantly, our analysis reveals a fundamental distinction between kinematic danger and semantic non-compliance. Deep-Flow identifies a critical predictability gap by surfacing out-of-distribution behaviors, such as lane-boundary violations and non-normative junction maneuvers, that traditional safety filters overlook. This work provides a mathematically rigorous foundation for defining statistical safety gates, enabling objective, data-driven validation for the safe deployment of autonomous fleets.
【2】Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal control
标题：时空双阶段超图MARL，用于以人为中心的多模式走廊交通信号控制
链接：https://arxiv.org/abs/2602.17068
摘要：在走廊网络中，以人为本的交通信号控制必须越来越多地考虑多模式出行者，特别是高乘坐率的公共交通，而不仅仅是关注以车辆为中心的性能。本文提出了STDSH-MARL（Spatio-Temporal Dual-Stage Hypergraph based Multi-Agent Reinforcement Learning），这是一个可扩展的多Agent深度强化学习框架，遵循集中式训练和分散式执行范式。该方法通过一种新的双阶段超图注意力机制来捕获时空依赖关系，该机制对空间和时间超边之间的交互进行建模。此外，引入混合离散动作空间来联合确定下一个信号相位配置及其对应的绿灯持续时间，从而实现更自适应的信号定时决策。在五种交通场景下的走廊网络上进行的实验表明，STDSH-MARL始终提高多式联运性能，并为公共交通优先提供了明显的好处。与最先进的基线方法相比，所提出的方法实现了优越的整体性能。进一步的消融研究证实了STDSH-MARL的每个组成部分的贡献，时间超峰被认为是驱动观察到的性能增益的最有影响力的因素。
摘要：Human-centric traffic signal control in corridor networks must increasingly account for multimodal travelers, particularly high-occupancy public transportation, rather than focusing solely on vehicle-centric performance. This paper proposes STDSH-MARL (Spatio-Temporal Dual-Stage Hypergraph based Multi-Agent Reinforcement Learning), a scalable multi-agent deep reinforcement learning framework that follows a centralized training and decentralized execution paradigm. The proposed method captures spatio-temporal dependencies through a novel dual-stage hypergraph attention mechanism that models interactions across both spatial and temporal hyperedges. In addition, a hybrid discrete action space is introduced to jointly determine the next signal phase configuration and its corresponding green duration, enabling more adaptive signal timing decisions. Experiments conducted on a corridor network under five traffic scenarios demonstrate that STDSH-MARL consistently improves multimodal performance and provides clear benefits for public transportation priority. Compared with state-of-the-art baseline methods, the proposed approach achieves superior overall performance. Further ablation studies confirm the contribution of each component of STDSH-MARL, with temporal hyperedges identified as the most influential factor driving the observed performance gains.
点云|SLAM|雷达|激光|深度RGBD相关(1篇)
【1】Learning a Latent Pulse Shape Interface for Photoinjector Laser Systems
标题：学习光注射器激光系统的潜在脉冲形状接口
链接：https://arxiv.org/abs/2602.17263
摘要：控制自由电子激光器的光注入器中的纵向激光脉冲形状是优化电子束质量的有力手段，但是对巨大设计空间的系统探索受到蛮力脉冲传播模拟的成本的限制。我们提出了一个基于Wasserstein自动编码器的生成建模框架，以学习脉冲整形和下游光束动态之间的可区分的潜在接口。我们的实证研究结果表明，学习的潜在空间是连续的和可解释的，同时保持高保真重建。脉冲家族，如高阶高斯跟踪相干轨迹，而标准化的时间脉冲长度显示了一个潜在的组织与脉冲能量。通过主成分和高斯混合模型的分析揭示了表现良好的潜在几何形状，通过线性插值实现不同脉冲类型之间的平滑过渡。该模型从模拟数据推广到真实的实验脉冲测量，准确地重建脉冲并将它们一致地嵌入到学习的流形中。总体而言，该方法减少了对昂贵的脉冲传播模拟的依赖，并有利于下游束流动力学模拟和分析。
摘要：Controlling the longitudinal laser pulse shape in photoinjectors of Free-Electron Lasers is a powerful lever for optimizing electron beam quality, but systematic exploration of the vast design space is limited by the cost of brute-force pulse propagation simulations. We present a generative modeling framework based on Wasserstein Autoencoders to learn a differentiable latent interface between pulse shaping and downstream beam dynamics. Our empirical findings show that the learned latent space is continuous and interpretable while maintaining high-fidelity reconstructions. Pulse families such as higher-order Gaussians trace coherent trajectories, while standardizing the temporal pulse lengths shows a latent organization correlated with pulse energy. Analysis via principal components and Gaussian Mixture Models reveals a well behaved latent geometry, enabling smooth transitions between distinct pulse types via linear interpolation. The model generalizes from simulated data to real experimental pulse measurements, accurately reconstructing pulses and embedding them consistently into the learned manifold. Overall, the approach reduces reliance on expensive pulse-propagation simulations and facilitates downstream beam dynamics simulation and analysis.
联邦学习|隐私保护|加密(1篇)
【1】Guarding the Middle: Protecting Intermediate Representations in Federated Split Learning
标题：守护中间：保护联邦拆分学习中的中间表示
链接：https://arxiv.org/abs/2602.17614
备注：Accepted for Publication in IEEE International Conference on Big Data (IEEE BigData) 2025
摘要：大数据场景中，大量的异构数据集分布在客户端上，需要可扩展的，隐私保护的学习方法。联邦学习（FL）可以在客户端之间分散训练机器学习（ML）模型，而无需集中数据。然而，分散式训练在客户端设备上引入了计算负担。U形联合分裂学习（UFSL）将客户端计算的一部分卸载到服务器，同时将数据和标签保留在客户端。然而，中间表示（即，粉碎的数据）容易暴露客户端的私有数据。为了减少客户端数据通过中间数据表示的暴露，这项工作提出了k-匿名差分私有UFSL（KD-UFSL），它利用隐私增强技术，如微聚合和差分隐私，以最大限度地减少数据泄漏从粉碎数据传输到服务器。我们首先证明了对手可以通过数据重建攻击从中间表示访问私人客户端数据，然后提出了一个隐私增强解决方案，KD-UFSL，以减轻这种风险。我们的实验表明，除了在某些情况下将实际图像和重建图像之间的均方误差增加高达50%之外，KD-UFSL还将它们之间的结构相似性降低了高达40%。更重要的是，KD-UFSL提高了隐私性，同时保留了全局模型的实用性。这凸显了它对于必须平衡隐私和实用性的大规模大数据应用程序的适用性。
摘要：Big data scenarios, where massive, heterogeneous datasets are distributed across clients, demand scalable, privacy-preserving learning methods. Federated learning (FL) enables decentralized training of machine learning (ML) models across clients without data centralization. Decentralized training, however, introduces a computational burden on client devices. U-shaped federated split learning (UFSL) offloads a fraction of the client computation to the server while keeping both data and labels on the clients' side. However, the intermediate representations (i.e., smashed data) shared by clients with the server are prone to exposing clients' private data. To reduce exposure of client data through intermediate data representations, this work proposes k-anonymous differentially private UFSL (KD-UFSL), which leverages privacy-enhancing techniques such as microaggregation and differential privacy to minimize data leakage from the smashed data transferred to the server. We first demonstrate that an adversary can access private client data from intermediate representations via a data-reconstruction attack, and then present a privacy-enhancing solution, KD-UFSL, to mitigate this risk. Our experiments indicate that, alongside increasing the mean squared error between the actual and reconstructed images by up to 50% in some cases, KD-UFSL also decreases the structural similarity between them by up to 40% on four benchmarking datasets. More importantly, KD-UFSL improves privacy while preserving the utility of the global model. This highlights its suitability for large-scale big data applications where privacy and utility must be balanced.
推理|分析|理解|解释(9篇)
【1】When to Trust the Cheap Check: Weak and Strong Verification for Reasoning
标题：何时信任廉价支票：推理的弱验证和强验证
链接：https://arxiv.org/abs/2602.17633
摘要：LLM的推理越来越多地在更广泛的验证循环中展开。在内部，系统使用廉价的检查，例如自我一致性或代理奖励，我们称之为弱验证。在外部，用户检查输出并通过反馈引导模型，直到结果值得信赖，我们称之为强验证。这些信号在成本和可靠性上有很大的不同：强验证可以建立信任，但需要大量资源，而弱验证速度快，可扩展性强，但噪音大，不完美。我们通过弱-强验证政策来正式化这种紧张关系，该政策决定何时根据弱验证接受或拒绝，何时推迟强验证。我们引入了捕获不正确接受、不正确拒绝和强验证频率的指标。在人口，我们表明，最佳政策承认一个两个阈值的结构和校准和锐度管理的价值弱验证。在此基础上，我们开发了一种在线算法，可证明控制接受和拒绝错误，而无需对查询流，语言模型或弱验证者进行假设。
摘要：Reasoning with LLMs increasingly unfolds inside a broader verification loop. Internally, systems use cheap checks, such as self-consistency or proxy rewards, which we call weak verification. Externally, users inspect outputs and steer the model through feedback until results are trustworthy, which we call strong verification. These signals differ sharply in cost and reliability: strong verification can establish trust but is resource-intensive, while weak verification is fast and scalable but noisy and imperfect. We formalize this tension through weak--strong verification policies, which decide when to accept or reject based on weak verification and when to defer to strong verification. We introduce metrics capturing incorrect acceptance, incorrect rejection, and strong-verification frequency. Over population, we show that optimal policies admit a two-threshold structure and that calibration and sharpness govern the value of weak verifiers. Building on this, we develop an online algorithm that provably controls acceptance and rejection errors without assumptions on the query stream, the language model, or the weak verifier.
【2】Variational inference via radial transport
标题：通过辐射输运的变分推断
链接：https://arxiv.org/abs/2602.17525
摘要：在变分推理（VI）中，实践者用一个简单的替代分布来近似一个高维分布，通常是一个（乘积）高斯分布。然而，在实际感兴趣的许多情况下，高斯分布可能无法捕获$π$的正确径向轮廓，导致覆盖率差。在这项工作中，我们从优化这些径向轮廓的角度来处理VI问题。我们的算法radVI是一个廉价的，有效的附加到许多现有的VI计划，如高斯（平均场）VI和拉普拉斯近似。我们为我们的算法提供了理论上的收敛保证，这是由于最近在Wasserstein空间（赋予Wasserstein距离的概率分布空间）上的优化发展以及Caffarelli（2000）风格的径向传输映射的新正则性。
摘要：In variational inference (VI), the practitioner approximates a high-dimensional distribution $π$ with a simple surrogate one, often a (product) Gaussian distribution. However, in many cases of practical interest, Gaussian distributions might not capture the correct radial profile of $π$, resulting in poor coverage. In this work, we approach the VI problem from the perspective of optimizing over these radial profiles. Our algorithm radVI is a cheap, effective add-on to many existing VI schemes, such as Gaussian (mean-field) VI and Laplace approximation. We provide theoretical convergence guarantees for our algorithm, owing to recent developments in optimization over the Wasserstein space--the space of probability distributions endowed with the Wasserstein distance--and new regularity properties of radial transport maps in the style of Caffarelli (2000).
【3】Convergence Analysis of Two-Layer Neural Networks under Gaussian Input Masking
标题：高斯输入掩蔽下两层神经网络的收敛性分析
链接：https://arxiv.org/abs/2602.17423
备注：69 pages, submitted to AI/ML Journal
摘要：研究了高斯随机掩蔽输入下两层神经网络训练的收敛保证问题。这种情况对应于输入级的高斯丢弃，或传感器网络中常见的噪声输入训练，隐私保护训练和联邦学习，其中每个用户都可以访问部分或损坏的功能。使用神经切核（NTK）分析，我们证明了用高斯随机掩码输入训练一个双层ReLU网络可以实现线性收敛，直到误差区域与掩码方差成比例。一个关键的技术贡献是解决非线性激活中的随机性，这是一个独立的问题。
摘要：We investigate the convergence guarantee of two-layer neural network training with Gaussian randomly masked inputs. This scenario corresponds to Gaussian dropout at the input level, or noisy input training common in sensor networks, privacy-preserving training, and federated learning, where each user may have access to partial or corrupted features. Using a Neural Tangent Kernel (NTK) analysis, we demonstrate that training a two-layer ReLU network with Gaussian randomly masked inputs achieves linear convergence up to an error region proportional to the mask's variance. A key technical contribution is resolving the randomness within the non-linear activation, a problem of independent interest.
【4】MDP Planning as Policy Inference
标题：作为政策推理的MDP规划
链接：https://arxiv.org/abs/2602.17375
备注：28 pages, many figures
摘要：我们把情景马尔可夫决策过程（MDP）规划作为贝叶斯推理。一个政策被视为潜在变量，并被分配一个非归一化的最优概率是单调的，在其预期回报，产生一个后验分布的模式符合回报最大化的解决方案，而后验离散表示最优行为的不确定性。为了在离散域中近似这个后验，我们将变分顺序蒙特卡罗（VSMC）适应于随机动态下确定性政策的推断，引入了一个扫描，该扫描强制执行重新访问状态之间的政策一致性，并耦合粒子之间的过渡随机性，以避免模拟器噪声的混淆。代理是由后验预测采样，这导致了一个随机控制策略，通过一个爱普生采样的解释，而不是熵正则化。在网格世界，二十一点，三角轮胎世界，和学术咨询，我们分析了推断的政策分布的结构，并比较由此产生的行为离散软演员批评，突出定性和统计差异，从政策层面的不确定性。
摘要：We cast episodic Markov decision process (MDP) planning as Bayesian inference over _policies_. A policy is treated as the latent variable and is assigned an unnormalized probability of optimality that is monotone in its expected return, yielding a posterior distribution whose modes coincide with return-maximizing solutions while posterior dispersion represents uncertainty over optimal behavior. To approximate this posterior in discrete domains, we adapt variational sequential Monte Carlo (VSMC) to inference over deterministic policies under stochastic dynamics, introducing a sweep that enforces policy consistency across revisited states and couples transition randomness across particles to avoid confounding from simulator noise. Acting is performed by posterior predictive sampling, which induces a stochastic control policy through a Thompson-sampling interpretation rather than entropy regularization. Across grid worlds, Blackjack, Triangle Tireworld, and Academic Advising, we analyze the structure of inferred policy distributions and compare the resulting behavior to discrete Soft Actor-Critic, highlighting qualitative and statistical differences that arise from policy-level uncertainty.
【5】CounterFlowNet: From Minimal Changes to Meaningful Counterfactual Explanations
标题：CounterFlowNet：从最小的改变到有意义的反事实解释
链接：https://arxiv.org/abs/2602.17244
摘要：反事实解释（CF）通过识别可能改变模型输出的输入特征的最小变化，为模型的预测提供人类可解释的见解。然而，现有的方法很难生成多个高质量的解释，（1）只影响一小部分的功能，（2）可以应用于具有异构特征的表格数据，以及（3）与用户定义的约束一致。我们提出了CounterFlowNet，一种生成方法，使用条件生成流网络（GFlowNet）将CF生成制定为顺序特征修改。CounterFlowNet被训练成与用户指定的奖励函数成比例地对CF进行采样，该奖励函数可以对关键CF进行编码：有效性，稀疏性，接近性和可解释性，鼓励高质量的解释。顺序公式产生高度稀疏的编辑，而统一的动作空间无缝地支持连续和分类特征。此外，可操作性约束，如不变性和单调性的功能，可以在推理时通过动作掩蔽，而无需重新训练。在两种评估协议下对八个数据集进行的实验表明，CounterFlowNet在有效性、稀疏性、可扩展性和多样性之间实现了卓越的权衡，并完全满足给定的约束。
摘要：Counterfactual explanations (CFs) provide human-interpretable insights into model's predictions by identifying minimal changes to input features that would alter the model's output. However, existing methods struggle to generate multiple high-quality explanations that (1) affect only a small portion of the features, (2) can be applied to tabular data with heterogeneous features, and (3) are consistent with the user-defined constraints. We propose CounterFlowNet, a generative approach that formulates CF generation as sequential feature modification using conditional Generative Flow Networks (GFlowNet). CounterFlowNet is trained to sample CFs proportionally to a user-specified reward function that can encode key CF desiderata: validity, sparsity, proximity and plausibility, encouraging high-quality explanations. The sequential formulation yields highly sparse edits, while a unified action space seamlessly supports continuous and categorical features. Moreover, actionability constraints, such as immutability and monotonicity of features, can be enforced at inference time via action masking, without retraining. Experiments on eight datasets under two evaluation protocols demonstrate that CounterFlowNet achieves superior trade-offs between validity, sparsity, plausibility, and diversity with full satisfaction of the given constraints.
【6】A Locality Radius Framework for Understanding Relational Inductive Bias in Database Learning
标题：理解数据库学习中关系归纳偏差的局部半径框架
链接：https://arxiv.org/abs/2602.17092
摘要：外键发现和相关的模式级预测任务通常使用图神经网络（GNN）建模，隐含地假设关系归纳偏差可以提高性能。然而，目前还不清楚何时多跳结构推理实际上是必要的。在这项工作中，我们引入局部半径，一个正式的措施，需要确定预测关系模式的最小结构邻域。我们假设，模型的性能关键取决于任务局部半径和架构聚合深度之间的对齐。我们进行了一项受控的实证研究，包括外键预测、连接成本估计、爆炸半径回归、级联影响分类和额外的图导出模式任务。我们的评估包括多种子实验，容量匹配比较，统计显著性检验，缩放分析和合成半径控制基准。结果揭示了一致的偏置半径对齐效应。
摘要：Foreign key discovery and related schema-level prediction tasks are often modeled using graph neural networks (GNNs), implicitly assuming that relational inductive bias improves performance. However, it remains unclear when multi-hop structural reasoning is actually necessary. In this work, we introduce locality radius, a formal measure of the minimum structural neighborhood required to determine a prediction in relational schemas. We hypothesize that model performance depends critically on alignment between task locality radius and architectural aggregation depth. We conduct a controlled empirical study across foreign key prediction, join cost estimation, blast radius regression, cascade impact classification, and additional graph-derived schema tasks. Our evaluation includes multi-seed experiments, capacity-matched comparisons, statistical significance testing, scaling analysis, and synthetic radius-controlled benchmarks. Results reveal a consistent bias-radius alignment effect.
【7】What is the Value of Censored Data? An Exact Analysis for the Data-driven Newsvendor
标题：审查数据的价值是什么？数据驱动报贩的精确分析
链接：https://arxiv.org/abs/2602.16842
摘要：研究了需求数据被删失的离线数据驱动报童问题。在以前的作品中，需求是完全观察到的，我们认为设置需求在库存水平上被审查，只有销售观察;销售匹配需求时，有足够的库存，并等于可用的库存。我们提供了一个通用的程序来计算经典的数据驱动的库存政策，评估所有的需求分布的确切的最坏情况下的遗憾。我们的主要技术结果表明，这个无限维的，非凸优化问题可以减少到有限维的，使任何样本大小和审查水平的政策的性能的准确表征。我们利用这种减少，以获得清晰的见解，可实现的性能标准的库存政策下的需求审查。特别是，我们对Kaplan-Meier策略的分析表明，虽然需求审查从根本上限制了从被动销售数据中可以学到的东西，但在高库存水平下进行少量有针对性的探索可以大大提高最坏情况的保证，即使在严重的审查下也可以实现接近最佳的性能。相反，当销售点系统不记录缺货事件而只报告实现的销售额时，一种自然且常用的方法是将销售视为需求。我们的研究结果表明，基于这种按需销售启发式的政策可能会遭受严重的性能下降，因为审查数据的积累，突出了销售点信息的质量如何关键形状什么可以，不能，离线学习。
摘要 ：We study the offline data-driven newsvendor problem with censored demand data. In contrast to prior works where demand is fully observed, we consider the setting where demand is censored at the inventory level and only sales are observed; sales match demand when there is sufficient inventory, and equal the available inventory otherwise. We provide a general procedure to compute the exact worst-case regret of classical data-driven inventory policies, evaluated over all demand distributions. Our main technical result shows that this infinite-dimensional, non-convex optimization problem can be reduced to a finite-dimensional one, enabling an exact characterization of the performance of policies for any sample size and censoring levels. We leverage this reduction to derive sharp insights on the achievable performance of standard inventory policies under demand censoring. In particular, our analysis of the Kaplan-Meier policy shows that while demand censoring fundamentally limits what can be learned from passive sales data, just a small amount of targeted exploration at high inventory levels can substantially improve worst-case guarantees, enabling near-optimal performance even under heavy censoring. In contrast, when the point-of-sale system does not record stockout events and only reports realized sales, a natural and commonly used approach is to treat sales as demand. Our results show that policies based on this sales-as-demand heuristic can suffer severe performance degradation as censored data accumulates, highlighting how the quality of point-of-sale information critically shapes what can, and cannot, be learned offline.
【8】Training Large Reasoning Models Efficiently via Progressive Thought Encoding
标题：通过渐进式思维编码高效训练大型推理模型
链接：https://arxiv.org/abs/2602.16839
备注：ICLR 2026, 15 pages
摘要：大型推理模型（LRM）在复杂问题上表现出色，但面临着效率的关键障碍：强化学习（RL）训练需要长期推出基于结果的奖励，其中自回归解码主导了时间和内存使用。虽然滑动窗口缓存策略可以限制内存，但它们会破坏长上下文推理并降低性能。我们引入渐进思想编码，一个参数高效的微调方法，使LRM的原因下，有效地固定大小的缓存。通过逐步将中间推理编码为固定大小的向量表示，我们的方法消除了通过全缓存转出反向传播的需要，从而减少了内存使用，同时在推理过程中保持恒定的内存。三个模型的实验，包括Qwen2.5-3B-Instruct，Qwen2.5- 7 B-Instruct和DeepSeek-R1-Distill-Llama-8B，在六个广泛使用的具有挑战性的数学基准上显示出一致的收益：我们的方法比基于LoRA的微调平均提高了+19.3%，比没有微调的LRM平均提高了+29.9%，在同样紧张的缓存预算下，AIME 2024/2025的精度提高高达+23.4。这些结果表明，渐进式思维编码不仅提高了推理的准确性，而且在现实世界的记忆约束下，使LRM的RL训练更加有效和可扩展。
摘要：Large reasoning models (LRMs) excel on complex problems but face a critical barrier to efficiency: reinforcement learning (RL) training requires long rollouts for outcome-based rewards, where autoregressive decoding dominates time and memory usage. While sliding-window cache strategies can bound memory, they disrupt long-context reasoning and degrade performance. We introduce Progressive Thought Encoding, a parameter-efficient fine-tuning method that enables LRMs to reason effectively under fixed-size caches. By progressively encoding intermediate reasoning into fixed-size vector representations, our approach eliminates the need to backpropagate through full-cache rollouts, thereby reducing memory usage, while maintaining constant memory during inference. Experiments on three models, including Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-Llama-8B, on six widely used challenging mathematical benchmarks show consistent gains: our method achieves +19.3% improvement over LoRA-based fine-tuning and +29.9% over LRMs without fine-tuning on average, with up to +23.4 accuracy improvement on AIME2024/2025 under the same tight cache budgets. These results demonstrate that Progressive Thought Encoding not only improves reasoning accuracy but also makes RL training of LRMs substantially more efficient and scalable under real-world memory constraints.
【9】DeepVision-103K: A Visually Diverse, Broad-Coverage, and Verifiable Mathematical Dataset for Multimodal Reasoning
标题：DeepVision-103 K：用于多模式推理的视觉多样化、覆盖广泛且可验证的数学数据集
链接：https://arxiv.org/abs/2602.16742
备注：Under review
摘要：具有可验证奖励的强化学习（RLVR）已被证明可以有效地增强大型多模态模型（LVMs）的视觉反射和推理能力。然而，现有的数据集主要来自小规模的手工构建或重组现有资源，这限制了数据的多样性和覆盖范围，从而限制了模型性能的进一步提高。为此，我们引入了\textbf{DeepVision-103 K}，这是一个用于RLVR训练的综合数据集，涵盖了各种K12数学主题，广泛的知识点和丰富的视觉元素。在DeepVision上训练的模型在多模态数学基准测试中表现出色，并有效地推广到一般的多模态推理任务。进一步的分析显示，经过训练的模型具有增强的视觉感知、反射和推理能力，验证了DeepVision在推进多模态推理方面的有效性。数据：\href{https：//huggingface.co/softenets/skylenage/DeepVision-103K}{this url}.
摘要：Reinforcement Learning with Verifiable Rewards (RLVR) has been shown effective in enhancing the visual reflection and reasoning capabilities of Large Multimodal Models (LMMs). However, existing datasets are predominantly derived from either small-scale manual construction or recombination of prior resources, which limits data diversity and coverage, thereby constraining further gains in model performance. To this end, we introduce \textbf{DeepVision-103K}, a comprehensive dataset for RLVR training that covers diverse K12 mathematical topics, extensive knowledge points, and rich visual elements. Models trained on DeepVision achieve strong performance on multimodal mathematical benchmarks, and generalize effectively to general multimodal reasoning tasks. Further analysis reveals enhanced visual perception, reflection and reasoning capabilities in trained models, validating DeepVision's effectiveness for advancing multimodal reasoning. Data: \href{https://huggingface.co/datasets/skylenage/DeepVision-103K}{this url}.
检测相关(3篇)
【1】Simplify to Amplify: Achieving Information-Theoretic Bounds with Fewer Steps in Spectral Community Detection
标题：放大的必要性：在光谱群落检测中以更少的步骤实现信息理论界限
链接：https://arxiv.org/abs/2602.17104
备注：9 pages plus appendix, 3 figures
摘要：提出了一种在两社区随机块模型（SBM）中进行社区检测的谱算法。通过消除不必要的预处理步骤来降低算法复杂度，我们的方法直接利用了邻接矩阵的谱特性。我们证明，我们的算法利用第二特征值的具体特点，以实现改进的误差范围，接近信息理论的限制，代表了显着的改进现有的方法。理论分析表明，我们的错误率比文献中以前报道的界限更紧。综合实验验证证实了我们的理论研究结果，并证明了简化方法的实际有效性。我们的研究结果表明，算法的简化，而不是增加复杂性，可以导致计算效率和增强性能的频谱社区检测。
摘要：We propose a streamlined spectral algorithm for community detection in the two-community stochastic block model (SBM) under constant edge density assumptions. By reducing algorithmic complexity through the elimination of non-essential preprocessing steps, our method directly leverages the spectral properties of the adjacency matrix. We demonstrate that our algorithm exploits specific characteristics of the second eigenvalue to achieve improved error bounds that approach information-theoretic limits, representing a significant improvement over existing methods. Theoretical analysis establishes that our error rates are tighter than previously reported bounds in the literature. Comprehensive experimental validation confirms our theoretical findings and demonstrates the practical effectiveness of the simplified approach. Our results suggest that algorithmic simplification, rather than increasing complexity, can lead to both computational efficiency and enhanced performance in spectral community detection.
【2】ML-driven detection and reduction of ballast information in multi-modal datasets
标题：ML驱动的多模式数据集中镇流器信息检测和减少
链接：https://arxiv.org/abs/2602.16876
备注：20 pages, 27 figures, 10 tables
摘要：现代数据集通常包含冗余或低效用信息，这些信息增加了维度，存储要求和计算成本，但没有贡献有意义的分析价值。这项研究介绍了一个通用的，多模式的框架，压载检测和减少结构化，半结构化，非结构化和稀疏数据类型。使用不同的数据集，熵，互信息，Lasso，SHAP，PCA，主题建模和嵌入分析应用于识别和消除压载特征。提出了一种新的压载评分，将这些信号整合到一个统一的，跨模态修剪策略。实验结果表明，在稀疏或半结构化数据中，特征空间的显著部分通常超过70%，可以以最小甚至改进的分类性能进行修剪，同时大幅减少训练时间和内存占用。该框架揭示了不同的压载类型（例如统计，语义，基础设施），并为更精简，更高效的机器学习管道提供了实用的指导。
摘要：Modern datasets often contain ballast as redundant or low-utility information that increases dimensionality, storage requirements, and computational cost without contributing meaningful analytical value. This study introduces a generalized, multimodal framework for ballast detection and reduction across structured, semi-structured, unstructured, and sparse data types. Using diverse datasets, entropy, mutual information, Lasso, SHAP, PCA, topic modelling, and embedding analysis are applied to identify and eliminate ballast features. A novel Ballast Score is proposed to integrate these signals into a unified, cross-modal pruning strategy. Experimental results demonstrate that significant portions of the feature space as often exceeding 70% in sparse or semi-structured data, can be pruned with minimal or even improved classification performance, along with substantial reductions in training time and memory footprint. The framework reveals distinct ballast typologies (e.g. statistical, semantic, infrastructural), and offers practical guidance for leaner, more efficient machine learning pipelines.
【3】Exploring the Utility of MALDI-TOF Mass Spectrometry and Antimicrobial Resistance in Hospital Outbreak Detection
标题：探索MALDI-TON MS和抗菌药物耐药性在医院疫情检测中的应用
链接：https://arxiv.org/abs/2602.16737
摘要：准确和及时地识别医院爆发集群对于防止具有流行潜力的感染传播至关重要。虽然通过全基因组测序（WGS）评估病原体相似性被认为是爆发检测的黄金标准，但其高昂的成本和漫长的周转时间阻碍了临床实验室的常规实施。我们探讨了两种快速和具有成本效益的替代WGS，基质辅助激光解吸电离飞行时间（MALDI-TOF）质谱和抗菌药物耐药性（AR）模式的效用。我们开发了一个机器学习框架，从MALDI-TOF光谱和AR模式中提取信息表示用于爆发检测，并探索它们的融合。通过多物种分析，我们证明，在某些情况下，MALDI-TOF和AR有可能减少对WGS的依赖，从而实现更容易和快速的疫情监测。
摘要：Accurate and timely identification of hospital outbreak clusters is crucial for preventing the spread of infections that have epidemic potential. While assessing pathogen similarity through whole genome sequencing (WGS) is considered the gold standard for outbreak detection, its high cost and lengthy turnaround time preclude routine implementation in clinical laboratories. We explore the utility of two rapid and cost-effective alternatives to WGS, matrix-assisted laser desorption ionization-time of flight (MALDI-TOF) mass spectrometry and antimicrobial resistance (AR) patterns. We develop a machine learning framework that extracts informative representations from MALDI-TOF spectra and AR patterns for outbreak detection and explore their fusion. Through multi-species analyses, we demonstrate that in some cases MALDI-TOF and AR have the potential to reduce reliance on WGS, enabling more accessible and rapid outbreak surveillance.
分类|识别(4篇)
【1】A.R.I.S.: Automated Recycling Identification System for E-Waste Classification Using Deep Learning
标题：AR IS：使用深度学习的电子废物分类自动回收识别系统
链接：https://arxiv.org/abs/2602.17642
摘要：传统的电子回收工艺由于材料分离和识别能力不足而遭受显著的资源损失，从而限制了材料回收。我们介绍A.R.I.S.（自动回收识别系统），一个低成本，便携式分拣机切碎的电子废物，解决了这一效率差距。该系统采用YOLOx模型对金属、塑料和电路板进行实时分类，实现了低推理延迟和高检测精度。实验评价得到90%的总体精密度，82.2%的平均平均精密度（mAP）和84%的分选纯度。通过将深度学习与已建立的排序方法相结合，A.R.I.S.提高材料回收效率，降低采用先进回收的障碍。这项工作补充了更广泛的计划，延长产品生命周期，支持贸易和回收计划，并减少整个供应链的环境影响。
摘要：Traditional electronic recycling processes suffer from significant resource loss due to inadequate material separation and identification capabilities, limiting material recovery. We present A.R.I.S. (Automated Recycling Identification System), a low-cost, portable sorter for shredded e-waste that addresses this efficiency gap. The system employs a YOLOx model to classify metals, plastics, and circuit boards in real time, achieving low inference latency with high detection accuracy. Experimental evaluation yielded 90% overall precision, 82.2% mean average precision (mAP), and 84% sortation purity. By integrating deep learning with established sorting methods, A.R.I.S. enhances material recovery efficiency and lowers barriers to advanced recycling adoption. This work complements broader initiatives in extending product life cycles, supporting trade-in and recycling programs, and reducing environmental impact across the supply chain.
【2】Shortcut learning in geometric knot classification
标题：几何纽结分类中的字节学习
链接：https://arxiv.org/abs/2602.17350
备注：17 pages, 6 figures, submitted to Machine Learning: Science and Technology, IOP
摘要：对闭曲线的拓扑进行分类是低维拓扑学中的一个核心问题，其应用范围超越了数学，跨越了蛋白质折叠、高分子物理甚至磁流体力学。其核心问题是如何判定闭弧的两个嵌入在环境合痕下是否等价。鉴于神经网络在解决复杂分类任务方面的惊人能力，因此很自然地会问是否可以使用机器学习（ML）来解决结分类问题。在本文中，我们研究了ML用于解决结分类挑战的通用快捷方法，并特别发现了通过ML使用多边形结的分子动力学模拟生成的训练数据中隐藏的非拓扑特征，以达到积极的分类结果。然后，我们为未来尝试使用ML解决结分类挑战提供了严格的基础，方法是开发一个公开可用的（i）数据集，旨在消除非拓扑特征分类的潜力，以及（ii）代码，可以生成结嵌入，忠实地探索具有固定结拓扑的所选几何状态空间。我们希望我们的工作将加速ML模型的开发，以解决复杂的几何结分类挑战。
摘要：Classifying the topology of closed curves is a central problem in low dimensional topology with applications beyond mathematics spanning protein folding, polymer physics and even magnetohydrodynamics. The central problem is how to determine whether two embeddings of a closed arc are equivalent under ambient isotopy. Given the striking ability of neural networks to solve complex classification tasks, it is therefore natural to ask if the knot classification problem can be tackled using Machine Learning (ML). In this paper, we investigate generic shortcut methods employed by ML to solve the knot classification challenge and specifically discover hidden non-topological features in training data generated through Molecular Dynamics simulations of polygonal knots that are used by ML to arrive to positive classifications results. We then provide a rigorous foundation for future attempts to tackle the knot classification challenge using ML by developing a publicly-available (i) dataset, that aims to remove the potential of non-topological feature classification and (ii) code, that can generate knot embeddings that faithfully explore chosen geometric state space with fixed knot topology. We expect that our work will accelerate the development of ML models that can solve complex geometric knot classification challenges.
【3】Evaluating Cross-Lingual Classification Approaches Enabling Topic Discovery for Multilingual Social Media Data
标题：评估跨语言分类方法，以实现多语言社交媒体数据的主题发现
链接
：https://arxiv.org/abs/2602.17051
摘要：分析多语言社交媒体话语仍然是自然语言处理中的一个主要挑战，特别是当大规模的公共辩论跨越多种语言时。本研究探讨了不同的跨语言文本分类方法如何支持全球对话的可靠分析。以氢能源为例，我们分析了长达十年的数据集，其中包括英语，日语，印地语和韩语（2013- 2022）的900多万条推文。在线关键词驱动的数据收集导致大量不相关的内容。我们探索了四种过滤相关内容的方法：（1）将英语注释的数据翻译成目标语言，用于为每种目标语言构建语言特定的模型，（2）将从所有语言出现的未标记的数据翻译成英语，用于基于英语注释创建单个模型，（3）将英语微调的多语言Transformers直接应用于每种目标语言数据，以及（4）将翻译注释与多语言训练相结合的混合策略。每种方法的能力进行评估，以过滤氢相关的推文从嘈杂的关键字为基础的集合。随后，进行主题建模，以提取相关子集内的主导主题。结果突出了翻译和多语言方法之间的关键权衡，为优化大规模社交媒体分析的跨语言管道提供了可操作的见解。
摘要：Analysing multilingual social media discourse remains a major challenge in natural language processing, particularly when large-scale public debates span across diverse languages. This study investigates how different approaches for cross-lingual text classification can support reliable analysis of global conversations. Using hydrogen energy as a case study, we analyse a decade-long dataset of over nine million tweets in English, Japanese, Hindi, and Korean (2013--2022) for topic discovery. The online keyword-driven data collection results in a significant amount of irrelevant content. We explore four approaches to filter relevant content: (1) translating English annotated data into target languages for building language-specific models for each target language, (2) translating unlabelled data appearing from all languages into English for creating a single model based on English annotations, (3) applying English fine-tuned multilingual transformers directly to each target language data, and (4) a hybrid strategy that combines translated annotations with multilingual training. Each approach is evaluated for its ability to filter hydrogen-related tweets from noisy keyword-based collections. Subsequently, topic modeling is performed to extract dominant themes within the relevant subsets. The results highlight key trade-offs between translation and multilingual approaches, offering actionable insights into optimising cross-lingual pipelines for large-scale social media analysis.
【4】Construction of a classification model for dementia among Brazilian adults aged 50 and over
标题：巴西50岁及以上成年人痴呆症分类模型的构建
链接：https://arxiv.org/abs/2602.16887
备注：38 pages; 3 figures
摘要：建立一个痴呆症分类模型为中年和老年巴西人，在Python中实现，结合变量选择和多变量分析，使用低成本的变量与修改潜力。采用横断面设计的预测建模方法进行的观察性研究，旨在估计发生痴呆症的机会，使用巴西老龄化纵向研究（ELSI-巴西）的数据，涉及9，412名参与者。根据神经心理学评估和基于知情者的认知功能确定痴呆。使用随机森林（RF）和多变量逻辑回归进行分析，以估计巴西中老年人群患痴呆症的风险。痴呆患病率为9.6%。在文盲人群中观察到痴呆症的几率最高（比值比（OR）= 7.42），90岁或以上的个体（OR = 11.00），低体重（OR = 2.11），低握力（OR = 2.50），自报黑色皮肤（OR = 1.47）、身体活动不足（OR = 1.61）、自报听力损失（OR = 1.65）和存在抑郁症状（OR = 1.72）。文化程度高（OR=0.44）、生活满意度高（OR=0.72）、有工作（OR=0.78）是保护因素。RF模型优于logistic回归，ROC曲线下面积为0.776，灵敏度为0.708，特异性为0.702，F1评分为0.311，G均值为0.705，准确度为0.703。结论：这些发现加强了痴呆症的多层面性质以及识别弱势个体的可访问因素的重要性。加强以促进大脑健康为重点的公共政策可以大大有助于巴西初级保健和痴呆症预防的资源有效分配
摘要：To build a dementia classification model for middle-aged and elderly Brazilians, implemented in Python, combining variable selection and multivariable analysis, using low-cost variables with modification potential. Observational study with a predictive modeling approach using a cross-sectional design, aimed at estimating the chances of developing dementia, using data from the Brazilian Longitudinal Study of Aging (ELSI-Brazil), involving 9,412 participants. Dementia was determined based on neuropsychological assessment and informant-based cognitive function. Analyses were performed using Random Forest (RF) and multivariable logistic regression to estimate the risk of dementia in the middle-aged and elderly populations of Brazil. The prevalence of dementia was 9.6%. The highest odds of dementia were observed in illiterate individuals (Odds Ratio (OR) = 7.42), individuals aged 90 years or older (OR = 11.00), low weight (OR = 2.11), low handgrip strength (OR = 2.50), self-reported black skin color (OR = 1.47), physical inactivity (OR = 1.61), self-reported hearing loss (OR = 1.65), and presence of depressive symptoms (OR = 1.72). Higher education (OR=0.44), greater life satisfaction (OR=0.72), and being employed (OR=0.78) were protective factors. The RF model outperformed logistic regression, achieving an area under the ROC curve of 0.776, with a sensitivity of 0.708, a specificity of 0.702, an F1-score of 0.311, a G-means of 0.705, and an accuracy of 0.703. Conclusion: The findings reinforce the multidimensional nature of dementia and the importance of accessible factors for identifying vulnerable individuals. Strengthening public policies focused on promoting brain health can contribute significantly to the efficient allocation of resources in primary care and dementia prevention in Brazil
表征(4篇)
【1】Canonicalizing Multimodal Contrastive Representation Learning
标题：规范化多模式对比表示学习
链接：https://arxiv.org/abs/2602.17584
备注：78 pages, 57 figures
摘要：随着模型和数据规模的扩大，独立训练的网络往往会产生类似的相似性概念。但是，匹配相似性比在表示空间之间建立明确的对应关系要弱，特别是对于多模态模型，其中一致性不仅必须在每个模态内保持，而且还必须保持学习的图像-文本耦合。因此，我们要求：给定两个独立训练的多模态对比模型（使用编码器$（f，g）$和$（\widetilde {f}，\widetilde {g}）$）--在不同的分布和不同的架构上训练--它们的嵌入空间之间是否存在系统的几何关系？如果是，它采取什么形式，是否在各种模式中保持一致？在这项工作中，我们表明，在CLIP，SigLIP和FLAVA等模型家族中，这种几何关系可以很好地近似于正交映射（直到全局均值漂移），即，存在一个正交映射$Q $其中$Q ^\top Q = I $使得$\widetilde {f}（x）\approx Q f（x）$对于成对的图像$x $。引人注目的是，相同的$Q $同时对齐文本编码器，即，$\widetilde {g}（y）\approx Q g（y）$对于文本$y $。从理论上讲，我们证明了如果多模态核在一个小的锚集上跨模型一致，即$\langle f（x），g（y）\rangle\approx\langle\widetilde {f}（x），\widetilde {g}（y）\rangle $，那么两个模型必须通过一个正交映射$Q $相关，并且相同的$Q $映射跨模型的图像和文本。更广泛地说，这一发现使得向后兼容的模型升级成为可能，避免了昂贵的重新嵌入，并对学习表征的隐私产生了影响。 我们的项目页面：www.example.com
摘要：As models and data scale, independently trained networks often induce analogous notions of similarity. But, matching similarities is weaker than establishing an explicit correspondence between the representation spaces, especially for multimodal models, where consistency must hold not only within each modality, but also for the learned image-text coupling. We therefore ask: given two independently trained multimodal contrastive models (with encoders $(f, g)$ and $(\widetilde{f},\widetilde{g})$) -- trained on different distributions and with different architectures -- does a systematic geometric relationship exist between their embedding spaces? If so, what form does it take, and does it hold uniformly across modalities? In this work, we show that across model families such as CLIP, SigLIP, and FLAVA, this geometric relationship is well approximated by an orthogonal map (up to a global mean shift), i.e., there exists an orthogonal map $Q$ where $Q^\top Q = I$ such that $\widetilde{f}(x)\approx Q f(x)$ for paired images $x$. Strikingly, the same $Q$ simultaneously aligns the text encoders i.e., $\widetilde{g}(y)\approx Q g(y)$ for texts $y$. Theoretically, we prove that if the multimodal kernel agrees across models on a small anchor set i.e. $\langle f(x), g(y)\rangle \approx \langle \widetilde{f}(x), \widetilde{g}(y)\rangle$, then the two models must be related by a single orthogonal map $Q$ and the same $Q$ maps images and text across models. More broadly, this finding enables backward-compatible model upgrades, avoiding costly re-embedding, and has implications for the privacy of learned representations. Our project page: https://canonical-multimodal.github.io/
【2】SpectralGCD: Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery
标题：SpectralGCD：用于广义类别发现的光谱概念选择和跨模式表示学习
链接：https://arxiv.org/abs/2602.17395
备注：Accepted at ICLR 2026. Code available at https://github.com/miccunifi/SpectralGCD
摘要：广义类别发现（GCD）的目的是在未标记的数据中识别新的类别，同时利用已知类别的一个小的标记子集。训练一个参数分类器只对图像特征往往会导致过度拟合旧类，最近的多模态方法通过结合文本信息来提高性能。然而，它们独立地处理模态并且产生很高的计算成本。我们提出了SpectralGCD，一个高效和有效的多模态GCD方法，使用CLIP跨模态图像概念相似性作为一个统一的跨模态表示。每个图像都表示为来自大型任务不可知词典的语义概念的混合物，其将学习锚定到明确的语义并减少对虚假视觉线索的依赖。为了保持高效学生学习到的表示的语义质量，我们引入了谱滤波，它利用了由强教师模型测量的softmax相似性上的跨模态协方差矩阵，以自动保留字典中的相关概念。来自同一位教师的正向和反向知识蒸馏确保了学生的跨模态表示在语义上保持充分和良好的对齐。在六个基准测试中，SpectralGCD以一小部分计算成本提供了与最先进方法相当或显著优于最先进方法的精度。该代码可在https://github.com/miccunifi/SpectralGCD上公开获取。
摘要：Generalized Category Discovery (GCD) aims to identify novel categories in unlabeled data while leveraging a small labeled subset of known classes. Training a parametric classifier solely on image features often leads to overfitting to old classes, and recent multimodal approaches improve performance by incorporating textual information. However, they treat modalities independently and incur high computational cost. We propose SpectralGCD, an efficient and effective multimodal approach to GCD that uses CLIP cross-modal image-concept similarities as a unified cross-modal representation. Each image is expressed as a mixture over semantic concepts from a large task-agnostic dictionary, which anchors learning to explicit semantics and reduces reliance on spurious visual cues. To maintain the semantic quality of representations learned by an efficient student, we introduce Spectral Filtering which exploits a cross-modal covariance matrix over the softmaxed similarities measured by a strong teacher model to automatically retain only relevant concepts from the dictionary. Forward and reverse knowledge distillation from the same teacher ensures that the cross-modal representations of the student remain both semantically sufficient and well-aligned. Across six benchmarks, SpectralGCD delivers accuracy comparable to or significantly superior to state-of-the-art methods at a fraction of the computational cost. The code is publicly available at: https://github.com/miccunifi/SpectralGCD.
【3】Representation Collapse in Machine Translation Through the Lens of Angular Dispersion
标题：机器翻译中的表示塌陷--基于角色散的研究
链接：https://arxiv.org/abs/2602.17287
摘要：基于Transformer架构的现代神经翻译模型以其高性能而闻名，特别是在高资源数据集上训练时。标准的下一个标记预测训练策略虽然在实践中被广泛采用，但可能会导致被忽视的伪像，如表示崩溃。以前的工作表明，这个问题是特别明显的表示更深的Transformer层，它往往无法有效地利用几何空间。表示崩溃在连续输出神经机器翻译的端到端训练中更加明显，其中平凡的解决方案是将所有向量设置为相同的值。在这项工作中，我们分析了在整个训练过程中，在不同级别的离散和连续NMT Transformers上表示崩溃的动态。我们结合了现有的正则化方法的基础上的角度分散和经验证明，它不仅减轻崩溃，而且提高了翻译质量。此外，我们表明，量化模型表现出类似的崩溃行为，即使在量化后，正则化的好处是保留。
摘要：Modern neural translation models based on the Transformer architecture are known for their high performance, particularly when trained on high-resource datasets. A standard next-token prediction training strategy, while widely adopted in practice, may lead to overlooked artifacts such as representation collapse. Previous works have shown that this problem is especially pronounced in the representation of the deeper Transformer layers, where it often fails to efficiently utilize the geometric space. Representation collapse is even more evident in end-to-end training of continuous-output neural machine translation, where the trivial solution would be to set all vectors to the same value. In this work, we analyze the dynamics of representation collapse at different levels of discrete and continuous NMT transformers throughout training. We incorporate an existing regularization method based on angular dispersion and demonstrate empirically that it not only mitigates collapse but also improves translation quality. Furthermore, we show that quantized models exhibit similar collapse behavior and that the benefits of regularization are preserved even after quantization.
【4】TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series
标题：TIFO：用于时间序列中平稳性感知表示学习的时不变频率运算符
链接：https://arxiv.org/abs/2602.17122
摘要：由于训练数据和测试数据的分布不同，非平稳时间序列预测存在分布偏移问题。现有的方法试图通过，例如，从每个样本中去除低阶矩。这些解决方案无法捕获样本之间的底层时间演变结构，并且无法对复杂的时间结构进行建模。在本文中，我们的目标是解决在频率空间中的分布移位，考虑所有可能的时间结构。为此，我们提出了一个时不变频率算子（TIFO），它在整个数据集的频谱上学习平稳性感知权重。权重表示突出固定频率分量，同时抑制非固定频率分量，从而减轻时间序列中的分布偏移问题。为了证明我们的方法，我们表明，时间序列数据的傅立叶变换隐含地诱导频率空间中的特征分解。TIFO是一种即插即用的方法，可以无缝集成到各种预测模型中。实验表明，我们的方法实现了18个前1和6个前2的结果，28个预测设置。值得注意的是，它在ETTm 2数据集上的平均MSE分别提高了33.3%和55.3%。此外，与基线方法相比，TIFO将计算成本降低了60%-70%，在不同的预测模型中表现出强大的可扩展性。
摘要：Nonstationary time series forecasting suffers from the distribution shift issue due to the different distributions that produce the training and test data. Existing methods attempt to alleviate the dependence by, e.g., removing low-order moments from each individual sample. These solutions fail to capture the underlying time-evolving structure across samples and do not model the complex time structure. In this paper, we aim to address the distribution shift in the frequency space by considering all possible time structures. To this end, we propose a Time-Invariant Frequency Operator (TIFO), which learns stationarity-aware weights over the frequency spectrum across the entire dataset. The weight representation highlights stationary frequency components while suppressing non-stationary ones, thereby mitigating the distribution shift issue in time series. To justify our method, we show that the Fourier transform of time series data implicitly induces eigen-decomposition in the frequency space. TIFO is a plug-and-play approach that can be seamlessly integrated into various forecasting models. Experiments demonstrate our method achieves 18 top-1 and 6 top-2 results out of 28 forecasting settings. Notably, it yields 33.3% and 55.3% improvements in average MSE on the ETTm2 dataset. In addition, TIFO reduces computational costs by 60% -70% compared to baseline methods, demonstrating strong scalability across diverse forecasting models.
3D|3D重建等相关(1篇)
【1】i-PhysGaussian: Implicit Physical Simulation for 3D Gaussian Splatting
标题：i-PhysGaussian：3D高斯飞溅的隐式物理模拟
链接：https://arxiv.org/abs/2602.17117
摘要：物理模拟根据材料特性和外部载荷预测物体的未来状态，使工业和工程设计能够进行风险管理。当前基于3D重建的仿真器通常依赖于显式的逐步更新，这对步长时间敏感，并且在复杂的场景下（例如高刚度材料或准静态运动）精度迅速下降。为了解决这个问题，我们引入了i-PhysGaussian，这是一个将3D高斯溅射（3DGS）与隐式材料点方法（MPM）积分器耦合的框架。与显式方法不同，我们的解决方案通过使用GMRES求解器隐式牛顿型优化来最小化动量平衡残差，从而获得步骤结束状态。该配方显著降低了时间步长敏感性，并确保了物理一致性。我们的研究结果表明，i-PhysGaussian在比显式基线大20倍的时间步长下保持稳定，即使在复杂的动态过渡中也能保持结构一致性和平滑运动。
摘要：Physical simulation predicts future states of objects based on material properties and external loads, enabling blueprints for both Industry and Engineering to conduct risk management. Current 3D reconstruction-based simulators typically rely on explicit, step-wise updates, which are sensitive to step time and suffer from rapid accuracy degradation under complicated scenarios, such as high-stiffness materials or quasi-static movement. To address this, we introduce i-PhysGaussian, a framework that couples 3D Gaussian Splatting (3DGS) with an implicit Material Point Method (MPM) integrator. Unlike explicit methods, our solution obtains an end-of-step state by minimizing a momentum-balance residual through implicit Newton-type optimization with a GMRES solver. This formulation significantly reduces time-step sensitivity and ensures physical consistency. Our results demonstrate that i-PhysGaussian maintains stability at up to 20x larger time steps than explicit baselines, preserving structural coherence and smooth motion even in complex dynamic transitions.
优化|敛散性(8篇)
【1】Linear Convergence in Games with Delayed Feedback via Extra Prediction
标题：通过额外预测具有延迟反馈的博弈的线性收敛
链接：https://arxiv.org/abs/2602.17486
备注：9 pages, 3 figures (main); 5 pages, 1 figure (appendix)
摘要：在现实世界的多智能体学习中，反馈延迟是不可避免的。他们被称为严重降低性能，并在延迟反馈下的收敛速度仍然不清楚，即使是双线性游戏。在无约束双线性博弈中，给出了加权乐观梯度下降-上升（WOGDA）算法的线性收敛速度。为了分析该算法，我们将其解释为额外邻近点（EPP）的近似值，该近似值基于比经典邻近点（PP）更远的未来奖励进行更新。我们的定理表明，标准的乐观（预测下一步的奖励）实现线性收敛到均衡的速度$\exp（-Θ（t/m^{5}））$后$t$迭代延迟$m$。此外，采用额外的乐观（预测更远的未来奖励）容忍更大的步长，并显着加速速率到$\exp（-Θ（t/（m^{2}\log m）$。我们的实验也表明加速收敛驱动的额外的乐观和定性与我们的定理。总之，本文验证了额外的乐观是一个很有前途的对策，对反馈延迟造成的性能下降。
摘要：Feedback delays are inevitable in real-world multi-agent learning. They are known to severely degrade performance, and the convergence rate under delayed feedback is still unclear, even for bilinear games. This paper derives the rate of linear convergence of Weighted Optimistic Gradient Descent-Ascent (WOGDA), which predicts future rewards with extra optimism, in unconstrained bilinear games. To analyze the algorithm, we interpret it as an approximation of the Extra Proximal Point (EPP), which is updated based on farther future rewards than the classical Proximal Point (PP). Our theorems show that standard optimism (predicting the next-step reward) achieves linear convergence to the equilibrium at a rate $\exp(-Θ(t/m^{5}))$ after $t$ iterations for delay $m$. Moreover, employing extra optimism (predicting farther future reward) tolerates a larger step size and significantly accelerates the rate to $\exp(-Θ(t/(m^{2}\log m)))$. Our experiments also show accelerated convergence driven by the extra optimism and are qualitatively consistent with our theorems. In summary, this paper validates that extra optimism is a promising countermeasure against performance degradation caused by feedback delays.
【2】Partial Optimality in the Preordering Problem
标题：预排序问题中的部分最优性
链接：https://arxiv.org/abs/2602.17346
摘要：预序是聚类和偏序的推广，在生物信息学和社会网络分析中有应用。给定一个有限集合$V$和一个值$c_{ab} \in \mathbb{R}$，对于$V$的每个有序元素对$ab$，预序问题要求在$V$上有一个预序$\lesssim$，它最大化那些对$ab$的值之和，其中$a \lesssim b$。在部分解决这个NP难问题的最新技术水平的基础上，我们贡献了新的部分最优性条件和有效的算法来决定这些条件。在真实和合成数据的实验中，这些新的条件增加了，特别是，它是有效地决定，$a \not\lesssim b$在一个最佳的前序对$ab$的分数。
摘要：Preordering is a generalization of clustering and partial ordering with applications in bioinformatics and social network analysis. Given a finite set $V$ and a value $c_{ab} \in \mathbb{R}$ for every ordered pair $ab$ of elements of $V$, the preordering problem asks for a preorder $\lesssim$ on $V$ that maximizes the sum of the values of those pairs $ab$ for which $a \lesssim b$. Building on the state of the art in solving this NP-hard problem partially, we contribute new partial optimality conditions and efficient algorithms for deciding these conditions. In experiments with real and synthetic data, these new conditions increase, in particular, the fraction of pairs $ab$ for which it is decided efficiently that $a \not\lesssim b$ in an optimal preorder.
【3】Efficient Tail-Aware Generative Optimization via Flow Model Fine-Tuning
标题：通过流模型微调实现高效的尾部感知生成优化
链接：https://arxiv.org/abs/2602.16796
备注：33 pages
摘要：微调预训练的扩散和流动模型以优化下游公用事业是实际部署的核心。现有的熵正则化方法主要是最大化预期奖励，没有提供任何机制来塑造尾部行为。然而，尾部控制通常是必不可少的：下尾通过限制低回报的失败来确定可靠性，而上尾通过优先考虑罕见的高回报结果来实现发现。在这项工作中，我们提出了尾部感知流量微调（TFFT），一个原则和有效的分布式微调算法的基础上的条件风险价值（CVaR）。我们解决了两个不同的尾部整形目标：右CVaR寻求新的样本在高回报的尾巴和左CVaR控制最坏情况下的样本在低回报的尾巴。与依赖于非线性优化的先前方法不同，我们利用CVaR的变分对偶公式将其分解为一个解耦的两阶段过程：一个轻量级的一维阈值优化步骤，以及一个通过特定伪奖励的单个熵正则化微调过程。这种分解实现CVaR微调有效的计算成本与标准的预期微调方法。我们证明了TFFT在说明性实验，高维文本到图像生成和分子设计中的有效性。
摘要：Fine-tuning pre-trained diffusion and flow models to optimize downstream utilities is central to real-world deployment. Existing entropy-regularized methods primarily maximize expected reward, providing no mechanism to shape tail behavior. However, tail control is often essential: the lower tail determines reliability by limiting low-reward failures, while the upper tail enables discovery by prioritizing rare, high-reward outcomes. In this work, we present Tail-aware Flow Fine-Tuning (TFFT), a principled and efficient distributional fine-tuning algorithm based on the Conditional Value-at-Risk (CVaR). We address two distinct tail-shaping goals: right-CVaR for seeking novel samples in the high-reward tail and left-CVaR for controlling worst-case samples in the low-reward tail. Unlike prior approaches that rely on non-linear optimization, we leverage the variational dual formulation of CVaR to decompose it into a decoupled two-stage procedure: a lightweight one-dimensional threshold optimization step, and a single entropy-regularized fine-tuning process via a specific pseudo-reward. This decomposition achieves CVaR fine-tuning efficiently with computational cost comparable to standard expected fine-tuning methods. We demonstrate the effectiveness of TFFT across illustrative experiments, high-dimensional text-to-image generation, and molecular design.
【4】Low-Dimensional and Transversely Curved Optimization Dynamics in Grokking
标题
：Grokking中的低维和横向曲线优化动力学
链接：https://arxiv.org/abs/2602.16746
备注：29 pages, 22 figures
摘要：Grokking --在小的算法任务中从记忆到概括的延迟过渡--仍然知之甚少。我们提出了一个几何分析的优化动力学在Transformers上训练模块化运算。注意力权重轨迹的PCA显示，训练主要在低维执行子空间内进行，单个主成分捕获68-83%的轨迹方差。为了探索损失景观几何，我们测量换向器缺陷-连续梯度步骤的非交换性-并将它们投影到这个学习的子空间上。我们发现，曲率急剧增长的方向正交的执行子空间，而轨迹仍然在很大程度上局限于it. Importantly，曲率增长始终领先于泛化的学习率和超参数制度，与前置时间服从幂律的grokking时标。因果干预实验表明，沿着学习的子空间的运动是必要的grokking，而人为增加曲率是不够的。总之，这些结果支持一个几何帐户，其中grokking反映逃离亚稳态制度，其特征在于低维限制和横向曲率积累。所有的研究结果复制在这个学习率范围内，一个定性不同的缓慢制度（LR=5E-5，WD=0.1，3层），和三个随机种子，虽然对齐动态不同制度之间的定量。因果干预实验表明，正交梯度流是必要的，但不足以grokking：抑制它防止泛化与单调的剂量反应在四个操作，而人为地提高曲率缺陷没有效果。
摘要：Grokking -- the delayed transition from memorization to generalization in small algorithmic tasks -- remains poorly understood. We present a geometric analysis of optimization dynamics in transformers trained on modular arithmetic. PCA of attention weight trajectories reveals that training evolves predominantly within a low-dimensional execution subspace, with a single principal component capturing 68-83% of trajectory variance. To probe loss-landscape geometry, we measure commutator defects -- the non-commutativity of successive gradient steps -- and project them onto this learned subspace. We find that curvature grows sharply in directions orthogonal to the execution subspace while the trajectory remains largely confined to it. Importantly, curvature growth consistently precedes generalization across learning rates and hyperparameter regimes, with the lead time obeying a power law in the grokking timescale. Causal intervention experiments show that motion along the learned subspace is necessary for grokking, while artificially increasing curvature is insufficient. Together, these results support a geometric account in which grokking reflects escape from a metastable regime characterized by low-dimensional confinement and transverse curvature accumulation. All findings replicate across this learning-rate range, a qualitatively different slow regime (lr=5e-5, wd=0.1, 3 layers), and three random seeds, though alignment dynamics differ quantitatively between regimes. Causal intervention experiments establish that orthogonal gradient flow is necessary but not sufficient for grokking: suppressing it prevents generalization with a monotonic dose-response across four operations, while artificially boosting curvature defects has no effect.
【5】PETS: A Principled Framework Towards Optimal Trajectory Allocation for Efficient Test-Time Self-Consistency
标题：PETS：实现有效测试时间自一致性的最佳轨迹分配的原则框架
链接：https://arxiv.org/abs/2602.16745
摘要：测试时缩放可以通过聚集随机推理轨迹来提高模型性能。然而，在有限的预算下实现样本有效的测试时间的自我一致性仍然是一个公开的挑战。我们介绍了PETS（原则和有效的测试时间自一致性），它通过优化框架启动了轨迹分配的原则性研究。我们的方法的核心是自洽率，一个新的措施定义为与无限预算多数票的协议。这一公式使得样本有效的测试时间分配理论上的基础，并进行严格的分析。我们研究离线和在线设置。在离线的制度，所有的问题都是已知的，我们连接轨迹分配到众包，一个经典的和发达的领域，通过建模的推理轨迹作为工人。这种观点使我们能够利用丰富的现有理论，产生理论保证和有效的基于多数投票的分配算法。在在线流媒体制度中，问题按顺序到达，分配必须在飞行中进行，我们提出了一种新的方法，灵感来自离线框架。我们的方法根据问题的难度调整预算，同时保留强有力的理论保证和计算效率。实验表明，PETS始终优于均匀分配。在GPQA上，PETS在两种设置中都实现了完美的自我一致性，同时相对于统一分配，将采样预算减少了75%（离线）和55%（在线）。代码可在https://github.com/ZDCSlab/PETS上获得。
摘要：Test-time scaling can improve model performance by aggregating stochastic reasoning trajectories. However, achieving sample-efficient test-time self-consistency under a limited budget remains an open challenge. We introduce PETS (Principled and Efficient Test-TimeSelf-Consistency), which initiates a principled study of trajectory allocation through an optimization framework. Central to our approach is the self-consistency rate, a new measure defined as agreement with the infinite-budget majority vote. This formulation makes sample-efficient test-time allocation theoretically grounded and amenable to rigorous analysis. We study both offline and online settings. In the offline regime, where all questions are known in advance, we connect trajectory allocation to crowdsourcing, a classic and well-developed area, by modeling reasoning traces as workers. This perspective allows us to leverage rich existing theory, yielding theoretical guarantees and an efficient majority-voting-based allocation algorithm. In the online streaming regime, where questions arrive sequentially and allocations must be made on the fly, we propose a novel method inspired by the offline framework. Our approach adapts budgets to question difficulty while preserving strong theoretical guarantees and computational efficiency. Experiments show that PETS consistently outperforms uniform allocation. On GPQA, PETS achieves perfect self-consistency in both settings while reducing the sampling budget by up to 75% (offline) and 55% (online) relative to uniform allocation. Code is available at https://github.com/ZDCSlab/PETS.
【6】Asymptotically Optimal Sequential Testing with Markovian Data
标题：马尔科夫数据的渐进最优序列测试
链接：https://arxiv.org/abs/2602.17587
摘要：本文研究了遍历马尔可夫链数据的单侧和$α$-正确序贯假设检验问题。零假设是未知的转移矩阵属于一个规定的集合P$的随机矩阵，和替代对应于一个不相交的集合Q$。我们建立了一个紧的非渐近的实例依赖的下限的期望停止时间的任何有效的序贯测试下的替代。我们的新分析提高了现有的下限，这是渐近或可证明的次优设置。我们的下界结合了平稳分布和未知马尔可夫链引起的过渡结构。我们进一步提出了一个最优检验，其期望停止时间渐近地匹配这个下限为$α\to 0$。我们说明了我们的框架的有用性，通过应用程序的马尔可夫链蒙特卡罗模型误指定顺序检测和测试结构属性，如线性的过渡动态，在马尔可夫决策过程。我们的研究结果产生了一个尖锐的和一般的表征最佳序贯测试程序下的马尔可夫依赖。
摘要：We study one-sided and $α$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the alternative corresponds to a disjoint set $Q$. We establish a tight non-asymptotic instance-dependent lower bound on the expected stopping time of any valid sequential test under the alternative. Our novel analysis improves the existing lower bounds, which are either asymptotic or provably sub-optimal in this setting. Our lower bound incorporates both the stationary distribution and the transition structure induced by the unknown Markov chain. We further propose an optimal test whose expected stopping time matches this lower bound asymptotically as $α\to 0$. We illustrate the usefulness of our framework through applications to sequential detection of model misspecification in Markov Chain Monte Carlo and to testing structural properties, such as the linearity of transition dynamics, in Markov decision processes. Our findings yield a sharp and general characterization of optimal sequential testing procedures under Markovian dependence.
【7】Poisson-MNL Bandit: Nearly Optimal Dynamic Joint Assortment and Pricing with Decision-Dependent Customer Arrivals
标题：Poisson-MNL Bandit：具有依赖决策的客户到达的近乎最优动态联合组合和定价
链接：https://arxiv.org/abs/2602.16923
摘要：我们研究动态联合分类和定价，卖方更新定期会计/经营间隔的决定，以最大限度地提高累计每周期的收入超过一个地平线$T$。在许多情况下，品种和价格不仅影响到达的客户购买，而且有多少客户在此期间到达，而经典的多项logit（MNL）模型假设到达固定，可能导致次优决策。我们提出了一个Poisson-MNL模型，耦合上下文MNL选择模型与泊松到达模型，其速率取决于所提供的品种和价格。在此模型的基础上，我们提出了一种基于置信上界（UCB）思想的高效算法PMNL。我们建立其（近）最优性证明的顺序$\sqrt{T\log{T}}$和匹配的下限（$\log T$）的非渐近遗憾界。模拟研究强调的重要性，会计的依赖性的到达率的分类和定价：PMNL有效地学习客户的选择和到达模型，并提供联合的折扣定价决策，优于其他假设固定的到达率。
摘要：We study dynamic joint assortment and pricing where a seller updates decisions at regular accounting/operating intervals to maximize the cumulative per-period revenue over a horizon $T$. In many settings, assortment and prices affect not only what an arriving customer buys but also how many customers arrive within the period, whereas classical multinomial logit (MNL) models assume arrivals as fixed, potentially leading to suboptimal decisions. We propose a Poisson-MNL model that couples a contextual MNL choice model with a Poisson arrival model whose rate depends on the offered assortment and prices. Building on this model, we develop an efficient algorithm PMNL based on the idea of upper confidence bound (UCB). We establish its (near) optimality by proving a non-asymptotic regret bound of order $\sqrt{T\log{T}}$ and a matching lower bound (up to $\log T$). Simulation studies underscore the importance of accounting for the dependency of arrival rates on assortment and pricing: PMNL effectively learns customer choice and arrival models and provides joint assortment-pricing decisions that outperform others that assume fixed arrival rates.
【8】Multi-objective optimization and quantum hybridization of equivariant deep learning interatomic potentials on organic and inorganic compounds
标题：有机和无机化合物等变深度学习原子间势的多目标优化和量子杂交
链接：https://arxiv.org/abs/2602.16908
备注：13 pages, 6 figures, 5 tables
摘要：Allegro是一种机器学习原子间势（MLIP）模型，旨在使用E（3）等变神经网络预测分子中的原子性质。在训练这个模型时，往往会在准确性和推理时间之间进行权衡。为此，我们将多目标超参数优化应用于这两个目标。此外，我们实验与修改后的架构，使变体的Allegro一些通过添加严格的经典多层感知器（MLP）层和一些通过添加量子经典混合层。我们比较了QM 9、rMD 17-阿司匹林、rMD 17-苯和我们自己的由铜和锂原子组成的专有数据集的结果。作为结果，我们有一个变体的列表，在准确性上超过了Allegro，并且结果证明了与推理时间的权衡。
摘要：Allegro is a machine learning interatomic potential (MLIP) model designed to predict atomic properties in molecules using E(3) equivariant neural networks. When training this model, there tends to be a trade-off between accuracy and inference time. For this reason we apply multi-objective hyperparameter optimization to the two objectives. Additionally, we experiment with modified architectures by making variants of Allegro some by adding strictly classical multi-layer perceptron (MLP) layers and some by adding quantum-classical hybrid layers. We compare the results from QM9, rMD17-aspirin, rMD17-benzene and our own proprietary dataset consisting of copper and lithium atoms. As results, we have a list of variants that surpass the Allegro in accuracy and also results which demonstrate the trade-off with inference times.
预测|估计(6篇)
【1】Operationalization of Machine Learning with Serverless Architecture: An Industrial Operationalization of Machine Learning with Serverless Architecture: An Industrial Implementation for Harmonized System Code Prediction
标题：采用无服务器架构的机器学习的可操作性：采用无服务器架构的机器学习的工业可操作性：协调系统代码预测的工业实现
链接：https://arxiv.org/abs/2602.17102
备注：13 pages. ICAD '26
摘要：本文提出了一个无服务器MLOps框架，该框架编排了从数据摄取、培训、部署、监控和再培训到使用事件驱动管道和托管服务的完整ML生命周期。该架构与模型无关，通过标准化接口支持各种推理模式，实现快速适应，而无需基础设施开销。我们通过协调制度（HS）代码预测的工业实现展示了实用性，这是一项合规性关键任务，将简短的非结构化产品描述映射到海关当局在全球贸易中使用的标准化代码。频繁的更新和模糊的描述使分类具有挑战性，错误会导致运输延迟和财务损失。我们的解决方案使用自定义文本嵌入编码器和多个深度学习架构，Text-CNN在地面真实数据上实现了98%的准确率。除了精度之外，该管道还通过自动缩放确保可变负载下的可重复性、可扩展性和SLA遵守性。一个关键特性是自动化A/B测试，支持动态模型选择和生产中的安全提升。成本效益驱动模型选择;虽然Transformers可以达到类似的精度，但其长期运营成本要高得多。具有可预测延迟和可解释性的确定性分类被优先考虑，尽管该架构仍然可扩展到Transformer变体和基于LLM的推理。本文首先通过模拟和模型比较介绍了深度学习架构，然后通过无服务器架构讨论了工业化，展示了HS代码的自动再训练、预测和验证。这项工作为使用无服务器架构运营ML提供了可复制的蓝图，使企业能够在优化性能和经济性的同时进行扩展。
摘要：This paper presents a serverless MLOps framework orchestrating the complete ML lifecycle from data ingestion, training, deployment, monitoring, and retraining to using event-driven pipelines and managed services. The architecture is model-agnostic, supporting diverse inference patterns through standardized interfaces, enabling rapid adaptation without infrastructure overhead. We demonstrate practical applicability through an industrial implementation for Harmonized System (HS) code prediction, a compliance-critical task where short, unstructured product descriptions are mapped to standardized codes used by customs authorities in global trade. Frequent updates and ambiguous descriptions make classification challenging, with errors causing shipment delays and financial losses. Our solution uses a custom text embedding encoder and multiple deep learning architectures, with Text-CNN achieving 98 percent accuracy on ground truth data. Beyond accuracy, the pipeline ensures reproducibility, auditability, and SLA adherence under variable loads via auto-scaling. A key feature is automated A/B testing, enabling dynamic model selection and safe promotion in production. Cost-efficiency drives model choice; while transformers may achieve similar accuracy, their long-term operational costs are significantly higher. Deterministic classification with predictable latency and explainability is prioritized, though the architecture remains extensible to transformer variants and LLM-based inference. The paper first introduces the deep learning architectures with simulations and model comparisons, then discusses industrialization through serverless architecture, demonstrating automated retraining, prediction, and validation of HS codes. This work provides a replicable blueprint for operationalizing ML using serverless architecture, enabling enterprises to scale while optimizing performance and economics.
【2】HQFS: Hybrid Quantum Classical Financial Security with VQC Forecasting, QUBO Annealing, and Audit-Ready Post-Quantum Signing
标题：HQFS：具有VQC预测、QUBO Annealing和审计就绪后量子签名的混合量子经典金融证券
链接：https://arxiv.org/abs/2602.16976
备注：11 pages, 1 fig , 4 tables
摘要：以下是更正后的段落，所有标点符号和格式问题都已修复： 金融风险系统通常遵循两步程序：模型预测回报或风险，然后优化器做出决策，例如投资组合再平衡。在实践中，这种分裂可能会在真正的约束下破裂。预测模型可能看起来不错，但当市场发生变化时，当添加离散约束（批量大小，上限）时，或者当优化变得缓慢时，最终决策可能不稳定。此外，受监管的设置需要明确的审计跟踪，将每个决策与确切的模型状态和输入相关联。我们提出了HQFS，一个实用的混合管道，连接预测，离散风险优化，并在一个流的可扩展性。首先，HQFS使用具有小经典头的变分量子电路（VQC）来学习下一步回报和波动率代理。其次，HQFS将风险收益目标和约束转换为QUBO，并在可用时使用量子退火来求解，同时保留兼容的经典QUBO求解器作为部署的后备。第三，HQFS使用后量子签名对每个重新平衡输出进行签名，以便稍后可以在不信任运行时环境的情况下验证分配。在我们的市场数据集研究中，HQFS将收益率预测误差降低了7.8%，波动率预测误差降低了6.1%。对于决策层，HQFS将样本外夏普提高了9.4%，并将最大压降降低了11.7%。与相同约束条件下的混合整数基线相比，QUBO求解阶段还将平均求解时间缩短了28%，同时生成完全可追溯的签名分配记录。
摘要：Here's the corrected paragraph with all punctuation and formatting issues fixed: Financial risk systems usually follow a two-step routine: a model predicts return or risk, and then an optimizer makes a decision such as a portfolio rebalance. In practice, this split can break under real constraints. The prediction model may look good, but the final decision can be unstable when the market shifts, when discrete constraints are added (lot sizes, caps), or when the optimization becomes slow for larger asset sets. Also, regulated settings need a clear audit trail that links each decision to the exact model state and inputs. We present HQFS, a practical hybrid pipeline that connects forecasting, discrete risk optimization, and auditability in one flow. First, HQFS learns next-step return and a volatility proxy using a variational quantum circuit (VQC) with a small classical head. Second, HQFS converts the risk-return objective and constraints into a QUBO and solves it with quantum annealing when available, while keeping a compatible classical QUBO solver as a fallback for deployment. Third, HQFS signs each rebalance output using a post-quantum signature so the allocation can be verified later without trusting the runtime environment. On our market dataset study, HQFS reduces return prediction error by 7.8% and volatility prediction error by 6.1% versus a tuned classical baseline. For the decision layer, HQFS improves out-of-sample Sharpe by 9.4% and lowers maximum drawdown by 11.7%. The QUBO solve stage also cuts average solve time by 28% compared to a mixed-integer baseline under the same constraints, while producing fully traceable, signed allocation records.
【3】TopoFlow: Physics-guided Neural Networks for high-resolution air quality prediction
标题：TopoFlow：用于高分辨率空气质量预测的物理引导神经网络
链接：https://arxiv.org/abs/2602.16821
摘要：我们提出了TopoFlow（Topography-aware pollutant Flow learning），这是一种物理指导的神经网络，用于高效，高分辨率的空气质量预测。为了明确地将物理过程嵌入到学习框架中，我们确定了两个控制污染物动力学的关键因素：地形和风向。复杂的地形可以引导、阻挡和捕获污染物，而风则是污染物运输和扩散的主要驱动力。基于这些见解，TopoFlow利用了具有两种新机制的Vision Transformer架构：地形感知注意力，其明确地对地形引起的流动模式进行建模，以及风引导的补丁重新排序，其将空间表示与盛行风向对齐。经过六年的高分辨率再分析数据培训，吸收了来自中国各地1，400多个地面监测站的观测数据，TopoFlow实现了9.71 ug/m3的PM2.5 RMSE，比业务预报系统提高了71-80%，比最先进的人工智能基线提高了13%。预测误差仍远低于中国24小时空气质量阈值75 ug/m3（GB 3095-2012），能够可靠区分清洁和污染状况。这些性能增益在所有四种主要污染物中是一致的，并且预测提前时间从12小时到96小时，这表明将物理知识原则性地整合到神经网络中可以从根本上推进空气质量预测。
摘要：We propose TopoFlow (Topography-aware pollutant Flow learning), a physics-guided neural network for efficient, high-resolution air quality prediction. To explicitly embed physical processes into the learning framework, we identify two critical factors governing pollutant dynamics: topography and wind direction. Complex terrain can channel, block, and trap pollutants, while wind acts as a primary driver of their transport and dispersion. Building on these insights, TopoFlow leverages a vision transformer architecture with two novel mechanisms: topography-aware attention, which explicitly models terrain-induced flow patterns, and wind-guided patch reordering, which aligns spatial representations with prevailing wind directions. Trained on six years of high-resolution reanalysis data assimilating observations from over 1,400 surface monitoring stations across China, TopoFlow achieves a PM2.5 RMSE of 9.71 ug/m3, representing a 71-80% improvement over operational forecasting systems and a 13% improvement over state-of-the-art AI baselines. Forecast errors remain well below China's 24-hour air quality threshold of 75 ug/m3 (GB 3095-2012), enabling reliable discrimination between clean and polluted conditions. These performance gains are consistent across all four major pollutants and forecast lead times from 12 to 96 hours, demonstrating that principled integration of physical knowledge into neural networks can fundamentally advance air quality prediction.
【4】Real-time Secondary Crash Likelihood Prediction Excluding Post Primary Crash Features
标题：不包括主要碰撞后特征的实时二次碰撞可能性预测
链接：https://arxiv.org/abs/2602.16739
摘要：二次碰撞可能性预测是主动交通管理系统的一个重要组成部分，以减轻二次碰撞造成的拥堵和不利影响。然而，现有的方法主要依赖于碰撞后特征（例如，碰撞类型和严重程度），这限制了它们的实际应用。为了解决这个问题，我们提出了一个混合二次碰撞可能性预测框架，不依赖于碰撞后的功能。一个动态的时空窗口的设计，以提取实时交通流和环境特征的主要碰撞地点和上游路段。该框架包括三个模型：一个主要的碰撞模型，以估计二次碰撞发生的可能性，和两个二次碰撞模型，以评估交通条件下的碰撞和上游路段不同的比较方案。集成学习策略集成六个机器学习算法的开发，以提高预测性能，和基于投票的机制结合三个模型的输出。佛罗里达州的高速公路上的实验表明，所提出的混合框架正确识别91%的二次碰撞的低误报率为0.20。ROC曲线下面积从单个模型的0.654、0.744和0.902提高到混合模型的0.952，优于先前的研究。
摘要：Secondary crash likelihood prediction is a critical component of an active traffic management system to mitigate congestion and adverse impacts caused by secondary crashes. However, existing approaches mainly rely on post-crash features (e.g., crash type and severity) that are rarely available in real time, limiting their practical applicability. To address this limitation, we propose a hybrid secondary crash likelihood prediction framework that does not depend on post-crash features. A dynamic spatiotemporal window is designed to extract real-time traffic flow and environmental features from primary crash locations and their upstream segments. The framework includes three models: a primary crash model to estimate the likelihood of secondary crash occurrence, and two secondary crash models to evaluate traffic conditions at crash and upstream segments under different comparative scenarios. An ensemble learning strategy integrating six machine learning algorithms is developed to enhance predictive performance, and a voting-based mechanism combines the outputs of the three models. Experiments on Florida freeways demonstrate that the proposed hybrid framework correctly identifies 91% of secondary crashes with a low false alarm rate of 0.20. The Area Under the ROC Curve improves from 0.654, 0.744, and 0.902 for the individual models to 0.952 for the hybrid model, outperforming previous studies.
【5】Self-Evolving Multi-Agent Network for Industrial IoT Predictive Maintenance
标题：用于工业物联网预测性维护的自进化多代理网络
链接：https://arxiv.org/abs/2602.16738
摘要：工业物联网预测性维护需要能够实时检测异常的系统，而不会牺牲可解释性或需要过多的计算资源。传统的方法依赖于静态的、离线训练的模型，无法适应不断变化的操作条件，而基于LLM的单片系统需要过高的内存和延迟，使得它们不适合现场边缘部署。我们介绍SEMAS，一个自我进化的分层多代理系统，分布在边缘，雾和云计算层的专门代理。边缘代理执行轻量级特征提取和预过滤;雾代理通过动态共识投票执行多样化的集成检测;云代理通过邻近策略优化（PPO）不断优化系统策略，同时保持异步，非阻塞推理。该框架结合了基于LLM的响应生成的可解释性和联邦知识聚合的自适应政策分布。这种架构支持资源感知的专门化，而不会牺牲实时性能或模型可解释性。对两个工业基准（锅炉仿真器和风力涡轮机）的经验评估表明，SEMAS在适应下具有出色的稳定性，实现了卓越的异常检测性能，在不断变化的操作环境中保持预测准确性，并提供了实质性的延迟改进，从而实现真正的实时部署。消融研究证实，PPO驱动的政策演变，共识投票，和联邦聚合每个系统的有效性作出重大贡献。这些发现表明，在严格的延迟和可解释性约束下，资源感知、自我进化的多智能体协调对于生产就绪的工业物联网预测性维护至关重要。
摘要 ：Industrial IoT predictive maintenance requires systems capable of real-time anomaly detection without sacrificing interpretability or demanding excessive computational resources. Traditional approaches rely on static, offline-trained models that cannot adapt to evolving operational conditions, while LLM-based monolithic systems demand prohibitive memory and latency, rendering them impractical for on-site edge deployment. We introduce SEMAS, a self-evolving hierarchical multi-agent system that distributes specialized agents across Edge, Fog, and Cloud computational tiers. Edge agents perform lightweight feature extraction and pre-filtering; Fog agents execute diversified ensemble detection with dynamic consensus voting; and Cloud agents continuously optimize system policies via Proximal Policy Optimization (PPO) while maintaining asynchronous, non-blocking inference. The framework incorporates LLM-based response generation for explainability and federated knowledge aggregation for adaptive policy distribution. This architecture enables resource-aware specialization without sacrificing real-time performance or model interpretability. Empirical evaluation on two industrial benchmarks (Boiler Emulator and Wind Turbine) demonstrates that SEMAS achieves superior anomaly detection performance with exceptional stability under adaptation, sustains prediction accuracy across evolving operational contexts, and delivers substantial latency improvements enabling genuine real-time deployment. Ablation studies confirm that PPO-driven policy evolution, consensus voting, and federated aggregation each contribute materially to system effectiveness. These findings indicate that resource-aware, self-evolving 1multi-agent coordination is essential for production-ready industrial IoT predictive maintenance under strict latency and explainability constraints.
【6】Beyond Procedure: Substantive Fairness in Conformal Prediction
标题：超越程序：保形预测的实质公平性
链接：https://arxiv.org/abs/2602.16794
摘要：Conformal prediction (CP) offers distribution-free uncertainty quantification for machine learning models, yet its interplay with fairness in downstream decision-making remains underexplored. Moving beyond CP as a standalone operation (procedural fairness), we analyze the holistic decision-making pipeline to evaluate substantive fairness-the equity of downstream outcomes. Theoretically, we derive an upper bound that decomposes prediction-set size disparity into interpretable components, clarifying how label-clustered CP helps control method-driven contributions to unfairness. To facilitate scalable empirical analysis, we introduce an LLM-in-the-loop evaluator that approximates human assessment of substantive fairness across diverse modalities. Our experiments reveal that label-clustered CP variants consistently deliver superior substantive fairness. Finally, we empirically show that equalized set sizes, rather than coverage, strongly correlate with improved substantive fairness, enabling practitioners to design more fair CP systems. Our code is available at https://github.com/layer6ai-labs/llm-in-the-loop-conformal-fairness.
其他神经网络|深度学习|模型|建模(22篇)
【1】MARS: Margin-Aware Reward-Modeling with Self-Refinement
标题：MARS：自我完善的边际意识奖励模型
链接：https://arxiv.org/abs/2602.17658
摘要：Reward modeling is a core component of modern alignment pipelines including RLHF and RLAIF, underpinning policy optimization methods including PPO and TRPO. However, training reliable reward models relies heavily on human-labeled preference data, which is costly and limited, motivating the use of data augmentation. Existing augmentation approaches typically operate at the representation or semantic level and remain agnostic to the reward model's estimation difficulty. In this paper, we propose MARS, an adaptive, margin-aware augmentation and sampling strategy that explicitly targets ambiguous and failure modes of the reward model. Our proposed framework, MARS, concentrates augmentation on low-margin (ambiguous) preference pairs where the reward model is most uncertain, and iteratively refines the training distribution via hard-sample augmentation. We provide theoretical guarantees showing that this strategy increases the average curvature of the loss function hence enhance information and improves conditioning, along with empirical results demonstrating consistent gains over uniform augmentation for robust reward modeling.
【2】Asymptotic Smoothing of the Lipschitz Loss Landscape in Overparameterized One-Hidden-Layer ReLU Networks
标题：过度参数化单隐藏层ReLU网络中Lipschitz损失格局的渐进平滑
链接：https://arxiv.org/abs/2602.17596
摘要：We study the topology of the loss landscape of one-hidden-layer ReLU networks under overparameterization. On the theory side, we (i) prove that for convex $L$-Lipschitz losses with an $\ell_1$-regularized second layer, every pair of models at the same loss level can be connected by a continuous path within an arbitrarily small loss increase $ε$ (extending a known result for the quadratic loss); (ii) obtain an asymptotic upper bound on the energy gap $ε$ between local and global minima that vanishes as the width $m$ grows, implying that the landscape flattens and sublevel sets become connected in the limit. Empirically, on a synthetic Moons dataset and on the Wisconsin Breast Cancer dataset, we measure pairwise energy gaps via Dynamic String Sampling (DSS) and find that wider networks exhibit smaller gaps; in particular, a permutation test on the maximum gap yields $p_{perm}=0$, indicating a clear reduction in the barrier height.
【3】Revisiting Weight Regularization for Low-Rank Continual Learning
标题：重新审视低级别持续学习的权重规则化
链接：https://arxiv.org/abs/2602.17559
备注：Accepted by ICLR 2026
摘要 ：Continual Learning (CL) with large-scale pre-trained models (PTMs) has recently gained wide attention, shifting the focus from training from scratch to continually adapting PTMs. This has given rise to a promising paradigm: parameter-efficient continual learning (PECL), where task interference is typically mitigated by assigning a task-specific module during training, such as low-rank adapters. However, weight regularization techniques, such as Elastic Weight Consolidation (EWC)-a key strategy in CL-remain underexplored in this new paradigm. In this paper, we revisit weight regularization in low-rank CL as a new perspective for mitigating task interference in PECL. Unlike existing low-rank CL methods, we mitigate task interference by regularizing a shared low-rank update through EWC, thereby keeping the storage requirement and inference costs constant regardless of the number of tasks. Our proposed method EWC-LoRA leverages a low-rank representation to estimate parameter importance over the full-dimensional space. This design offers a practical, computational- and memory-efficient solution for CL with PTMs, and provides insights that may inform the broader application of regularization techniques within PECL. Extensive experiments on various benchmarks demonstrate the effectiveness of EWC-LoRA, achieving a stability-plasticity trade-off superior to existing low-rank CL approaches. These results indicate that, even under low-rank parameterizations, weight regularization remains an effective mechanism for mitigating task interference. Code is available at: https://github.com/yaoyz96/low-rank-cl.
【4】A Theoretical Framework for Modular Learning of Robust Generative Models
标题：鲁棒生成模型模块学习的理论框架
链接：https://arxiv.org/abs/2602.17554
摘要：Training large-scale generative models is resource-intensive and relies heavily on heuristic dataset weighting. We address two fundamental questions: Can we train Large Language Models (LLMs) modularly-combining small, domain-specific experts to match monolithic performance-and can we do so robustly for any data mixture, eliminating heuristic tuning? We present a theoretical framework for modular generative modeling where a set of pre-trained experts are combined via a gating mechanism. We define the space of normalized gating functions, $G_{1}$, and formulate the problem as a minimax game to find a single robust gate that minimizes divergence to the worst-case data mixture. We prove the existence of such a robust gate using Kakutani's fixed-point theorem and show that modularity acts as a strong regularizer, with generalization bounds scaling with the lightweight gate's complexity. Furthermore, we prove that this modular approach can theoretically outperform models retrained on aggregate data, with the gap characterized by the Jensen-Shannon Divergence. Finally, we introduce a scalable Stochastic Primal-Dual algorithm and a Structural Distillation method for efficient inference. Empirical results on synthetic and real-world datasets confirm that our modular architecture effectively mitigates gradient conflict and can robustly outperform monolithic baselines.
【5】IRIS: Learning-Driven Task-Specific Cinema Robot Arm for Visuomotor Motion Control
标题：IRIS：学习驱动的特定任务影院机器人手臂，用于视觉运动控制
链接：https://arxiv.org/abs/2602.17537
摘要：Robotic camera systems enable dynamic, repeatable motion beyond human capabilities, yet their adoption remains limited by the high cost and operational complexity of industrial-grade platforms. We present the Intelligent Robotic Imaging System (IRIS), a task-specific 6-DOF manipulator designed for autonomous, learning-driven cinematic motion control. IRIS integrates a lightweight, fully 3D-printed hardware design with a goal-conditioned visuomotor imitation learning framework based on Action Chunking with Transformers (ACT). The system learns object-aware and perceptually smooth camera trajectories directly from human demonstrations, eliminating the need for explicit geometric programming. The complete platform costs under $1,000 USD, supports a 1.5 kg payload, and achieves approximately 1 mm repeatability. Real-world experiments demonstrate accurate trajectory tracking, reliable autonomous execution, and generalization across diverse cinematic motions.
【6】Provably Explaining Neural Additive Models
标题：可证明地解释神经相加模型
链接：https://arxiv.org/abs/2602.17530
备注：To appear in ICLR 2026
摘要：Despite significant progress in post-hoc explanation methods for neural networks, many remain heuristic and lack provable guarantees. A key approach for obtaining explanations with provable guarantees is by identifying a cardinally-minimal subset of input features which by itself is provably sufficient to determine the prediction. However, for standard neural networks, this task is often computationally infeasible, as it demands a worst-case exponential number of verification queries in the number of input features, each of which is NP-hard. In this work, we show that for Neural Additive Models (NAMs), a recent and more interpretable neural network family, we can efficiently generate explanations with such guarantees. We present a new model-specific algorithm for NAMs that generates provably cardinally-minimal explanations using only a logarithmic number of verification queries in the number of input features, after a parallelized preprocessing step with logarithmic runtime in the required precision is applied to each small univariate NAM component. Our algorithm not only makes the task of obtaining cardinally-minimal explanations feasible, but even outperforms existing algorithms designed to find the relaxed variant of subset-minimal explanations - which may be larger and less informative but easier to compute - despite our algorithm solving a much more difficult task. Our experiments demonstrate that, compared to previous algorithms, our approach provides provably smaller explanations than existing works and substantially reduces the computation time. Moreover, we show that our generated provable explanations offer benefits that are unattainable by standard sampling-based techniques typically used to interpret NAMs.
【7】Learning with Boolean threshold functions
标题：使用布尔阈值函数学习
链接：https://arxiv.org/abs/2602.17493
备注：22 pages, 21 figures
摘要 ：We develop a method for training neural networks on Boolean data in which the values at all nodes are strictly $\pm 1$, and the resulting models are typically equivalent to networks whose nonzero weights are also $\pm 1$. The method replaces loss minimization with a nonconvex constraint formulation. Each node implements a Boolean threshold function (BTF), and training is expressed through a divide-and-concur decomposition into two complementary constraints: one enforces local BTF consistency between inputs, weights, and output; the other imposes architectural concurrence, equating neuron outputs with downstream inputs and enforcing weight equality across training-data instantiations of the network. The reflect-reflect-relax (RRR) projection algorithm is used to reconcile these constraints. Each BTF constraint includes a lower bound on the margin. When this bound is sufficiently large, the learned representations are provably sparse and equivalent to networks composed of simple logical gates with $\pm 1$ weights. Across a range of tasks -- including multiplier-circuit discovery, binary autoencoding, logic-network inference, and cellular automata learning -- the method achieves exact solutions or strong generalization in regimes where standard gradient-based methods struggle. These results demonstrate that projection-based constraint satisfaction provides a viable and conceptually distinct foundation for learning in discrete neural systems, with implications for interpretability and efficient inference.
【8】The Sound of Death: Deep Learning Reveals Vascular Damage from Carotid Ultrasound
标题：死亡之声：深度学习揭示颈动脉超声造成的血管损伤
链接：https://arxiv.org/abs/2602.17321
摘要：Cardiovascular diseases (CVDs) remain the leading cause of mortality worldwide, yet early risk detection is often limited by available diagnostics. Carotid ultrasound, a non-invasive and widely accessible modality, encodes rich structural and hemodynamic information that is largely untapped. Here, we present a machine learning (ML) framework that extracts clinically meaningful representations of vascular damage (VD) from carotid ultrasound videos, using hypertension as a weak proxy label. The model learns robust features that are biologically plausible, interpretable, and strongly associated with established cardiovascular risk factors, comorbidities, and laboratory measures. High VD stratifies individuals for myocardial infarction, cardiac death, and all-cause mortality, matching or outperforming conventional risk models such as SCORE2. Explainable AI analyses reveal that the model relies on vessel morphology and perivascular tissue characteristics, uncovering novel functional and anatomical signatures of vascular damage. This work demonstrates that routine carotid ultrasound contains far more prognostic information than previously recognized. Our approach provides a scalable, non-invasive, and cost-effective tool for population-wide cardiovascular risk assessment, enabling earlier and more personalized prevention strategies without reliance on laboratory tests or complex clinical inputs.
【9】Open Datasets in Learning Analytics: Trends, Challenges, and Best PRACTICE
标题：学习分析中的开放数据集：趋势、挑战和最佳实践
链接：https://arxiv.org/abs/2602.17314
备注：Recently accepted to ACM Transactions on Knowledge Discovery from Data (TKDD). To appear. (Preprint will be updated with full bibliographic info.)
摘要：Open datasets play a crucial role in three research domains that intersect data science and education: learning analytics, educational data mining, and artificial intelligence in education. Researchers in these domains apply computational methods to analyze data from educational contexts, aiming to better understand and improve teaching and learning. Providing open datasets alongside research papers supports reproducibility, collaboration, and trust in research findings. It also provides individual benefits for authors, such as greater visibility, credibility, and citation potential. Despite these advantages, the availability of open datasets and the associated practices within the learning analytics research communities, especially at their flagship conference venues, remain unclear. We surveyed available datasets published alongside research papers in learning analytics. We manually examined 1,125 papers from three flagship conferences (LAK, EDM, and AIED) over the past five years. We discovered, categorized, and analyzed 172 datasets used in 204 publications. Our study presents the most comprehensive collection and analysis of open educational datasets to date, along with the most detailed categorization. Of the 172 datasets identified, 143 were not captured in any prior survey of open data in learning analytics. We provide insights into the datasets' context, analytical methods, use, and other properties. Based on this survey, we summarize the current gaps in the field. Furthermore, we list practical recommendations, advice, and 8-item guidelines under the acronym PRACTICE with a checklist to help researchers publish their data. Lastly, we share our original dataset: an annotated inventory detailing the discovered datasets and the corresponding publications. We hope these findings will support further adoption of open data practices in learning analytics communities and beyond.
【10】In-Context Learning in Linear vs. Quadratic Attention Models: An Empirical Study on Regression Tasks
标题：线性与二次注意力模型中的上下文学习：回归任务的实证研究
链接：https://arxiv.org/abs/2602.17171
摘要：Recent work has demonstrated that transformers and linear attention models can perform in-context learning (ICL) on simple function classes, such as linear regression. In this paper, we empirically study how these two attention mechanisms differ in their ICL behavior on the canonical linear-regression task of Garg et al. We evaluate learning quality (MSE), convergence, and generalization behavior of each architecture. We also analyze how increasing model depth affects ICL performance. Our results illustrate both the similarities and limitations of linear attention relative to quadratic attention in this setting.
【11】When More Experts Hurt: Underfitting in Multi-Expert Learning to Defer
标题：当更多专家受到伤害时：多专家学习不适合推迟
链接：https://arxiv.org/abs/2602.17144
摘要 ：Learning to Defer (L2D) enables a classifier to abstain from predictions and defer to an expert, and has recently been extended to multi-expert settings. In this work, we show that multi-expert L2D is fundamentally more challenging than the single-expert case. With multiple experts, the classifier's underfitting becomes inherent, which seriously degrades prediction performance, whereas in the single-expert setting it arises only under specific conditions. We theoretically reveal that this stems from an intrinsic expert identifiability issue: learning which expert to trust from a diverse pool, a problem absent in the single-expert case and renders existing underfitting remedies failed. To tackle this issue, we propose PiCCE (Pick the Confident and Correct Expert), a surrogate-based method that adaptively identifies a reliable expert based on empirical evidence. PiCCE effectively reduces multi-expert L2D to a single-expert-like learning problem, thereby resolving multi expert underfitting. We further prove its statistical consistency and ability to recover class probabilities and expert accuracies. Extensive experiments across diverse settings, including real-world expert scenarios, validate our theoretical results and demonstrate improved performance.
【12】Online Learning with Improving Agents: Multiclass, Budgeted Agents and Bandit Learners
标题：通过改进代理进行在线学习：多类、自愿代理和强盗学习者
链接：https://arxiv.org/abs/2602.17103
摘要：We investigate the recently introduced model of learning with improvements, where agents are allowed to make small changes to their feature values to be warranted a more desirable label. We extensively extend previously published results by providing combinatorial dimensions that characterize online learnability in this model, by analyzing the multiclass setup, learnability in a bandit feedback setup, modeling agents' cost for making improvements and more.
【13】Synergizing Transport-Based Generative Models and Latent Geometry for Stochastic Closure Modeling
标题：协同基于传输的生成模型和潜在几何用于随机封闭建模
链接：https://arxiv.org/abs/2602.17089
摘要：Diffusion models recently developed for generative AI tasks can produce high-quality samples while still maintaining diversity among samples to promote mode coverage, providing a promising path for learning stochastic closure models. Compared to other types of generative AI models, such as GANs and VAEs, the sampling speed is known as a key disadvantage of diffusion models. By systematically comparing transport-based generative models on a numerical example of 2D Kolmogorov flows, we show that flow matching in a lower-dimensional latent space is suited for fast sampling of stochastic closure models, enabling single-step sampling that is up to two orders of magnitude faster than iterative diffusion-based approaches. To control the latent space distortion and thus ensure the physical fidelity of the sampled closure term, we compare the implicit regularization offered by a joint training scheme against two explicit regularizers: metric-preserving (MP) and geometry-aware (GA) constraints. Besides offering a faster sampling speed, both explicitly and implicitly regularized latent spaces inherit the key topological information from the lower-dimensional manifold of the original complex dynamical system, which enables the learning of stochastic closure models without demanding a huge amount of training data.
【14】Sign Lock-In: Randomly Initialized Weight Signs Persist and Bottleneck Sub-Bit Model Compression
标题：符号锁定：随机初始化权重符号持续和瓶颈子位模型压缩
链接：https://arxiv.org/abs/2602.17063
摘要：Sub-bit model compression seeks storage below one bit per weight; as magnitudes are aggressively compressed, the sign bit becomes a fixed-cost bottleneck. Across Transformers, CNNs, and MLPs, learned sign matrices resist low-rank approximation and are spectrally indistinguishable from an i.i.d. Rademacher baseline. Despite this apparent randomness, most weights retain their initialization signs; flips primarily occur via rare near-zero boundary crossings, suggesting that sign-pattern randomness is largely inherited from initialization. We formalize this behavior with sign lock-in theory, a stopping-time analysis of sign flips under SGD noise. Under bounded updates and a rare re-entry condition into a small neighborhood around zero, the number of effective sign flips exhibits a geometric tail. Building on this mechanism, we introduce a gap-based initialization and a lightweight outward-drift regularizer, reducing the effective flip rate to approximately $10^{-3}$ with only about a one-point increase in perplexity.
【15】Multi-Probe Zero Collision Hash (MPZCH): Mitigating Embedding Collisions and Enhancing Model Freshness in Large-Scale Recommenders
标题：多探针零碰撞哈希（MPZCH）：缓解大规模推荐中的嵌入碰撞并增强模型新鲜度
链接：https://arxiv.org/abs/2602.17050
备注：10 pages, 6 figures
摘要：Embedding tables are critical components of large-scale recommendation systems, facilitating the efficient mapping of high-cardinality categorical features into dense vector representations. However, as the volume of unique IDs expands, traditional hash-based indexing methods suffer from collisions that degrade model performance and personalization quality. We present Multi-Probe Zero Collision Hash (MPZCH), a novel indexing mechanism based on linear probing that effectively mitigates embedding collisions. With reasonable table sizing, it often eliminates these collisions entirely while maintaining production-scale efficiency. MPZCH utilizes auxiliary tensors and high-performance CUDA kernels to implement configurable probing and active eviction policies. By retiring obsolete IDs and resetting reassigned slots, MPZCH prevents the stale embedding inheritance typical of hash-based methods, ensuring new features learn effectively from scratch. Despite its collision-mitigation overhead, the system maintains training QPS and inference latency comparable to existing methods. Rigorous online experiments demonstrate that MPZCH achieves zero collisions for user embeddings and significantly improves item embedding freshness and quality. The solution has been released within the open-source TorchRec library for the broader community.
【16】Position: Why a Dynamical Systems Perspective is Needed to Advance Time Series Modeling
标题：位置：为什么需要动态系统视角来推进时间序列建模
链接：https://arxiv.org/abs/2602.16864
摘要：Time series (TS) modeling has come a long way from early statistical, mainly linear, approaches to the current trend in TS foundation models. With a lot of hype and industrial demand in this field, it is not always clear how much progress there really is. To advance TS forecasting and analysis to the next level, here we argue that the field needs a dynamical systems (DS) perspective. TS of observations from natural or engineered systems almost always originate from some underlying DS, and arguably access to its governing equations would yield theoretically optimal forecasts. This is the promise of DS reconstruction (DSR), a class of ML/AI approaches that aim to infer surrogate models of the underlying DS from data. But models based on DS principles offer other profound advantages: Beyond short-term forecasts, they enable to predict the long-term statistics of an observed system, which in many practical scenarios may be the more relevant quantities. DS theory furthermore provides domain-independent theoretical insight into mechanisms underlying TS generation, and thereby will inform us, e.g., about upper bounds on performance of any TS model, generalization into unseen regimes as in tipping points, or potential control strategies. After reviewing some of the central concepts, methods, measures, and models in DS theory and DSR, we will discuss how insights from this field can advance TS modeling in crucial ways, enabling better forecasting with much lower computational and memory footprints. We conclude with a number of specific suggestions for translating insights from DSR into TS modeling.
【17】Escaping the Cognitive Well: Efficient Competition Math with Off-the-Shelf Models
标题：逃离认知井：有效的竞争数学与现成的模型
链接：https://arxiv.org/abs/2602.16793
摘要：In the past year, custom and unreleased math reasoning models reached gold medal performance on the International Mathematical Olympiad (IMO). Similar performance was then reported using large-scale inference on publicly available models but at prohibitive costs (e.g., 3000 USD per problem). In this work, we present an inference pipeline that attains best-in-class performance on IMO-style math problems at an average inference cost orders of magnitude below competing methods while using only general-purpose off-the-shelf models. Our method relies on insights about grader failure in solver-grader pipelines, which we call the Cognitive Well (iterative refinement converging to a wrong solution that the solver as well as the pipeline's internal grader consider to be basically correct). Our pipeline addresses these failure modes through conjecture extraction, wherein candidate lemmas are isolated from generated solutions and independently verified alongside their negations in a fresh environment (context detachment). On IMO-ProofBench Advanced (PB-Adv), our pipeline achieves 67.1 percent performance using Gemini 3.0 Pro with an average cost per question of approximately 31 USD. At the time of evaluation, this represented the state-of-the-art on PB-Adv among both public and unreleased models, and more than doubles the success rate of the next best publicly accessible pipeline, all at a fraction of the cost.
【18】Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistency
标题：更好地思考三次：学会以双重反事实一致性进行因果推理
链接：https://arxiv.org/abs/2602.16787
摘要：Despite their strong performance on reasoning benchmarks, large language models (LLMs) have proven brittle when presented with counterfactual questions, suggesting weaknesses in their causal reasoning ability. While recent work has demonstrated that labeled counterfactual tasks can be useful benchmarks of LLMs' causal reasoning, producing such data at the scale required to cover the vast potential space of counterfactuals is limited. In this work, we introduce double counterfactual consistency (DCC), a lightweight inference-time method for measuring and guiding the ability of LLMs to reason causally. Without requiring labeled counterfactual data, DCC verifies a model's ability to execute two important elements of causal reasoning: causal intervention and counterfactual prediction. Using DCC, we evaluate the causal reasoning abilities of various leading LLMs across a range of reasoning tasks and interventions. Moreover, we demonstrate the effectiveness of DCC as a training-free test-time rejection sampling criterion and show that it can directly improve performance on reasoning tasks across multiple model families.
【19】Machine Learning Argument of Latitude Error Model for LEO Satellite Orbit and Covariance Correction
标题：LEO卫星轨道纬度误差模型和协方差修正的机器学习论证
链接：https://arxiv.org/abs/2602.16764
备注：Appearing in 2026 IEEE Aerospace Conference
摘要：Low Earth orbit (LEO) satellites are leveraged to support new position, navigation, and timing (PNT) service alternatives to GNSS. These alternatives require accurate propagation of satellite position and velocity with a realistic quantification of uncertainty. It is commonly assumed that the propagated uncertainty distribution is Gaussian; however, the validity of this assumption can be quickly compromised by the mismodeling of atmospheric drag. We develop a machine learning approach that corrects error growth in the argument of latitude for a diverse set of LEO satellites. The improved orbit propagation accuracy extends the applicability of the Gaussian assumption and modeling of the errors with a corrected mean and covariance. We compare the performance of a time-conditioned neural network and a Gaussian Process on datasets computed with an open source orbit propagator and publicly available Vector Covariance Message (VCM) ephemerides. The learned models predict the argument of latitude error as a Gaussian distribution given parameters from a single VCM epoch and reverse propagation errors. We show that this one-dimensional model captures the effect of mismodeled drag, which can be mapped to the Cartesian state space. The correction method only updates information along the dimensions of dominant error growth, while maintaining the physics-based propagation of VCM covariance in the remaining dimensions. We therefore extend the utility of VCM ephemerides to longer time horizons without modifying the functionality of the existing propagator.
【20】genriesz: A Python Package for Automatic Debiased Machine Learning with Generalized Riesz Regression
标题：genriesz：一个用于使用广义Riesz回归的自动去偏机器学习的Python包
链接：https://arxiv.org/abs/2602.17543
摘要：Efficient estimation of causal and structural parameters can be automated using the Riesz representation theorem and debiased machine learning (DML). We present genriesz, an open-source Python package that implements automatic DML and generalized Riesz regression, a unified framework for estimating Riesz representers by minimizing empirical Bregman divergences. This framework includes covariate balancing, nearest-neighbor matching, calibrated estimation, and density ratio estimation as special cases. A key design principle of the package is automatic regressor balancing (ARB): given a Bregman generator $g$ and a representer model class, genriesz} automatically constructs a compatible link function so that the generalized Riesz regression estimator satisfies balancing (moment-matching) optimality conditions in a user-chosen basis. The package provides a modulr interface for specifying (i) the target linear functional via a black-box evaluation oracle, (ii) the representer model via basis functions (polynomial, RKHS approximations, random forest leaf encodings, neural embeddings, and a nearest-neighbor catchment basis), and (iii) the Bregman generator, with optional user-supplied derivatives. It returns regression adjustment (RA), Riesz weighting (RW), augmented Riesz weighting (ARW), and TMLE-style estimators with cross-fitting, confidence intervals, and $p$-values. We highlight representative workflows for estimation problems such as the average treatment effect (ATE), ATE on treated (ATT), and average marginal effect estimation. The Python package is available at https://github.com/MasaKat0/genriesz and on PyPI.
【21】Dynamic Decision-Making under Model Misspecification: A Stochastic Stability Approach
标题：模型错误规范下的动态决策：随机稳定性方法
链接：https://arxiv.org/abs/2602.17086
摘要：Dynamic decision-making under model uncertainty is central to many economic environments, yet existing bandit and reinforcement learning algorithms rely on the assumption of correct model specification. This paper studies the behavior and performance of one of the most commonly used Bayesian reinforcement learning algorithms, Thompson Sampling (TS), when the model class is misspecified. We first provide a complete dynamic classification of posterior evolution in a misspecified two-armed Gaussian bandit, identifying distinct regimes: correct model concentration, incorrect model concentration, and persistent belief mixing, characterized by the direction of statistical evidence and the model-action mapping. These regimes yield sharp predictions for limiting beliefs, action frequencies, and asymptotic regret. We then extend the analysis to a general finite model class and develop a unified stochastic stability framework that represents posterior evolution as a Markov process on the belief simplex. This approach characterizes two sufficient conditions to classify the ergodic and transient behaviors and provides inductive dimensional reductions of the posterior dynamics. Our results offer the first qualitative and geometric classification of TS under misspecification, bridging Bayesian learning with evolutionary dynamics, and also build the foundations of robust decision-making in structured bandits.
【22】BrainRVQ: A High-Fidelity EEG Foundation Model via Dual-Domain Residual Quantization and Hierarchical Autoregression
标题：BrainRVQ：通过双域剩余量化和分层自回归的高保真脑电基础模型
链接：https://arxiv.org/abs/2602.16951
摘要：Developing foundation models for electroencephalography (EEG) remains challenging due to the signal's low signal-to-noise ratio and complex spectro-temporal non-stationarity. Existing approaches often overlook the hierarchical latent structure inherent in neural dynamics, leading to suboptimal reconstruction of fine-grained information. In this work, we propose BrainRVQ, a general-purpose EEG foundation model pre-trained on a large-scale corpus of clinical EEG data. Unlike standard masked modeling, BrainRVQ features a Dual-Domain Residual Vector Quantization (DD-RVQ) tokenizer that disentangles temporal waveforms and spectral patterns into hierarchical discrete codes. We further introduce a hierarchical autoregressive pre-training objective that learns to reconstruct these codes in a coarse-to-fine manner, utilizing an importance-guided curriculum masking strategy to prioritize information-rich neural events over background noise. Extensive experiments across 8 diverse downstream datasets demonstrate that BrainRVQ consistently outperforms state-of-the-art baselines, validating its effectiveness in learning robust and generalizable neural representations. Our code and model weights are available:https://github.com/keqicmz/BrainRVQ
其他(41篇)
【1】Mine and Refine: Optimizing Graded Relevance in E-commerce Search Retrieval
标题：挖掘与细化：优化电子商务搜索检索中的分级相关性
链接：https://arxiv.org/abs/2602.17654
摘要：We propose a two-stage "Mine and Refine" contrastive training framework for semantic text embeddings to enhance multi-category e-commerce search retrieval. Large scale e-commerce search demands embeddings that generalize to long tail, noisy queries while adhering to scalable supervision compatible with product and policy constraints. A practical challenge is that relevance is often graded: users accept substitutes or complements beyond exact matches, and production systems benefit from clear separation of similarity scores across these relevance strata for stable hybrid blending and thresholding. To obtain scalable policy consistent supervision, we fine-tune a lightweight LLM on human annotations under a three-level relevance guideline and further reduce residual noise via engagement driven auditing. In Stage 1, we train a multilingual Siamese two-tower retriever with a label aware supervised contrastive objective that shapes a robust global semantic space. In Stage 2, we mine hard samples via ANN and re-annotate them with the policy aligned LLM, and introduce a multi-class extension of circle loss that explicitly sharpens similarity boundaries between relevance levels, to further refine and enrich the embedding space. Robustness is additionally improved through additive spelling augmentation and synthetic query generation. Extensive offline evaluations and production A/B tests show that our framework improves retrieval relevance and delivers statistically significant gains in engagement and business impact.
【2】Multi-Round Human-AI Collaboration with User-Specified Requirements
标题：具有用户指定需求的多轮人机协作
链接：https://arxiv.org/abs/2602.17646
摘要：As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision quality. We adopt a human centric view governed by two principles: counterfactual harm, ensuring the AI does not undermine human strengths, and complementarity, ensuring it adds value where the human is prone to err. We formalize these concepts via user defined rules, allowing users to specify exactly what harm and complementarity mean for their specific task. We then introduce an online, distribution free algorithm with finite sample guarantees that enforces the user-specified constraints over the collaboration dynamics. We evaluate our framework across two interactive settings: LLM simulated collaboration on a medical diagnostic task and a human crowdsourcing study on a pictorial reasoning task. We show that our online procedure maintains prescribed counterfactual harm and complementarity violation rates even under nonstationary interaction dynamics. Moreover, tightening or loosening these constraints produces predictable shifts in downstream human accuracy, confirming that the two principles serve as practical levers for steering multi-round collaboration toward better decision quality without the need to model or constrain human behavior.
【3】FAMOSE: A ReAct Approach to Automated Feature Discovery
标题：FAMOSE：自动特征发现的ReAct方法
链接：https://arxiv.org/abs/2602.17641
备注：23 pages, 6 figures
摘要：Feature engineering remains a critical yet challenging bottleneck in machine learning, particularly for tabular data, as identifying optimal features from an exponentially large feature space traditionally demands substantial domain expertise. To address this challenge, we introduce FAMOSE (Feature AugMentation and Optimal Selection agEnt), a novel framework that leverages the ReAct paradigm to autonomously explore, generate, and refine features while integrating feature selection and evaluation tools within an agent architecture. To our knowledge, FAMOSE represents the first application of an agentic ReAct framework to automated feature engineering, especially for both regression and classification tasks. Extensive experiments demonstrate that FAMOSE is at or near the state-of-the-art on classification tasks (especially tasks with more than 10K instances, where ROC-AUC increases 0.23% on average), and achieves the state-of-the-art for regression tasks by reducing RMSE by 2.0% on average, while remaining more robust to errors than other algorithms. We hypothesize that FAMOSE's strong performance is because ReAct allows the LLM context window to record (via iterative feature discovery and evaluation steps) what features did or did not work. This is similar to a few-shot prompt and guides the LLM to invent better, more innovative features. Our work offers evidence that AI agents are remarkably effective in solving problems that require highly inventive solutions, such as feature engineering.
【4】SMAC: Score-Matched Actor-Critics for Robust Offline-to-Online Transfer
标题：SMAC：稳健的线下到线上转移的评分相匹配的演员评论家
链接：https://arxiv.org/abs/2602.17632
摘要：Modern offline Reinforcement Learning (RL) methods find performant actor-critics, however, fine-tuning these actor-critics online with value-based RL algorithms typically causes immediate drops in performance. We provide evidence consistent with the hypothesis that, in the loss landscape, offline maxima for prior algorithms and online maxima are separated by low-performance valleys that gradient-based fine-tuning traverses. Following this, we present Score Matched Actor-Critic (SMAC), an offline RL method designed to learn actor-critics that transition to online value-based RL algorithms with no drop in performance. SMAC avoids valleys between offline and online maxima by regularizing the Q-function during the offline phase to respect a first-order derivative equality between the score of the policy and action-gradient of the Q-function. We experimentally demonstrate that SMAC converges to offline maxima that are connected to better online maxima via paths with monotonically increasing reward found by first-order optimization. SMAC achieves smooth transfer to Soft Actor-Critic and TD3 in 6/6 D4RL tasks. In 4/6 environments, it reduces regret by 34-58% over the best baseline.
【5】Towards Anytime-Valid Statistical Watermarking
标题：迈向随时有效的统计水印
链接：https://arxiv.org/abs/2602.17608
摘要：The proliferation of Large Language Models (LLMs) necessitates efficient mechanisms to distinguish machine-generated content from human text. While statistical watermarking has emerged as a promising solution, existing methods suffer from two critical limitations: the lack of a principled approach for selecting sampling distributions and the reliance on fixed-horizon hypothesis testing, which precludes valid early stopping. In this paper, we bridge this gap by developing the first e-value-based watermarking framework, Anchored E-Watermarking, that unifies optimal sampling with anytime-valid inference. Unlike traditional approaches where optional stopping invalidates Type-I error guarantees, our framework enables valid, anytime-inference by constructing a test supermartingale for the detection process. By leveraging an anchor distribution to approximate the target model, we characterize the optimal e-value with respect to the worst-case log-growth rate and derive the optimal expected stopping time. Our theoretical claims are substantiated by simulations and evaluations on established benchmarks, showing that our framework can significantly enhance sample efficiency, reducing the average token budget required for detection by 13-15% relative to state-of-the-art baselines.
【6】AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing
标题：AutoNumerics：一个自治的、与PDE无关的多Agent科学计算管道
链接：https://arxiv.org/abs/2602.17607
摘要 ：PDEs are central to scientific and engineering modeling, yet designing accurate numerical solvers typically requires substantial mathematical expertise and manual tuning. Recent neural network-based approaches improve flexibility but often demand high computational cost and suffer from limited interpretability. We introduce \texttt{AutoNumerics}, a multi-agent framework that autonomously designs, implements, debugs, and verifies numerical solvers for general PDEs directly from natural language descriptions. Unlike black-box neural solvers, our framework generates transparent solvers grounded in classical numerical analysis. We introduce a coarse-to-fine execution strategy and a residual-based self-verification mechanism. Experiments on 24 canonical and real-world PDE problems demonstrate that \texttt{AutoNumerics} achieves competitive or superior accuracy compared to existing neural and LLM-based baselines, and correctly selects numerical schemes based on PDE structural properties, suggesting its viability as an accessible paradigm for automated PDE solving.
【7】Simultaneous Blackwell Approachability and Applications to Multiclass Omniprediction
标题：同时布莱克威尔逼近性及其在多类全预测中的应用
链接：https://arxiv.org/abs/2602.17577
摘要：Omniprediction is a learning problem that requires suboptimality bounds for each of a family of losses $\mathcal{L}$ against a family of comparator predictors $\mathcal{C}$. We initiate the study of omniprediction in a multiclass setting, where the comparator family $\mathcal{C}$ may be infinite. Our main result is an extension of the recent binary omniprediction algorithm of [OKK25] to the multiclass setting, with sample complexity (in statistical settings) or regret horizon (in online settings) $\approx \varepsilon^{-(k+1)}$, for $\varepsilon$-omniprediction in a $k$-class prediction problem. En route to proving this result, we design a framework of potential broader interest for solving Blackwell approachability problems where multiple sets must simultaneously be approached via coupled actions.
【8】Be Wary of Your Time Series Preprocessing
标题：小心时间序列预处理
链接：https://arxiv.org/abs/2602.17568
备注：Accepted at the AI4TS workshop at AAAI-26
摘要：Normalization and scaling are fundamental preprocessing steps in time series modeling, yet their role in Transformer-based models remains underexplored from a theoretical perspective. In this work, we present the first formal analysis of how different normalization strategies, specifically instance-based and global scaling, impact the expressivity of Transformer-based architectures for time series representation learning. We propose a novel expressivity framework tailored to time series, which quantifies a model's ability to distinguish between similar and dissimilar inputs in the representation space. Using this framework, we derive theoretical bounds for two widely used normalization methods: Standard and Min-Max scaling. Our analysis reveals that the choice of normalization strategy can significantly influence the model's representational capacity, depending on the task and data characteristics. We complement our theory with empirical validation on classification and forecasting benchmarks using multiple Transformer-based models. Our results show that no single normalization method consistently outperforms others, and in some cases, omitting normalization entirely leads to superior performance. These findings highlight the critical role of preprocessing in time series learning and motivate the need for more principled normalization strategies tailored to specific tasks and datasets.
【9】Variational Grey-Box Dynamics Matching
标题：变分灰箱动力学匹配
链接：https://arxiv.org/abs/2602.17477
备注：AISTATS 2026. Code is available at https://github.com/DMML-Geneva/VGB-DM
摘要：Deep generative models such as flow matching and diffusion models have shown great potential in learning complex distributions and dynamical systems, but often act as black-boxes, neglecting underlying physics. In contrast, physics-based simulation models described by ODEs/PDEs remain interpretable, but may have missing or unknown terms, unable to fully describe real-world observations. We bridge this gap with a novel grey-box method that integrates incomplete physics models directly into generative models. Our approach learns dynamics from observational trajectories alone, without ground-truth physics parameters, in a simulation-free manner that avoids scalability and stability issues of Neural ODEs. The core of our method lies in modelling a structured variational distribution within the flow matching framework, by using two latent encodings: one to model the missing stochasticity and multi-modal velocity, and a second to encode physics parameters as a latent variable with a physics-informed prior. Furthermore, we present an adaptation of the framework to handle second-order dynamics. Our experiments on representative ODE/PDE problems show that our method performs on par with or superior to fully data-driven approaches and previous grey-box baselines, while preserving the interpretability of the physics model. Our code is available at https://github.com/DMML-Geneva/VGB-DM.
【10】ABCD: All Biases Come Disguised
标题：ABCD：所有偏见都是伪装的
链接：https://arxiv.org/abs/2602.17445
备注：29 pages, 20 figures, pre-print, 12 tables
摘要 ：Multiple-choice question (MCQ) benchmarks have been a standard evaluation practice for measuring LLMs' ability to reason and answer knowledge-based questions. Through a synthetic NonsenseQA benchmark, we observe that different LLMs exhibit varying degrees of label-position-few-shot-prompt bias, where the model either uses the answer position, the label in front of the answer, the distributions of correct answers present in the few-shot prompt, or a combination of all to answer each MCQ question. We propose a simple bias-reduced evaluation protocol that replaces the labels of each question with uniform, unordered labels and prompts the LLM to use the whole answer presented. With a simple sentence similarity model, we demonstrate improved robustness and lower standard deviation between different permutations of answers with a minimal drop in LLM's performance, exposing the LLM's capabilities under reduced evaluation artifacts, without any help from the prompt examples or the option labels. Across multiple benchmarks and models, this protocol substantially improves the robustness to answer permutations, reducing mean accuracy variance $3\times$ with only a minimal decrease in the mean model's performance. Through ablation studies on various embedding models and similarity functions, we show that the method is more robust than the standard ones.
【11】2Mamba2Furious: Linear in Complexity, Competitive in Accuracy
标题：2 Mamba 2 Furious：复杂性线性，准确性有竞争力
链接：https://arxiv.org/abs/2602.17363
摘要：Linear attention transformers have become a strong alternative to softmax attention due to their efficiency. However, linear attention tends to be less expressive and results in reduced accuracy compared to softmax attention. To bridge the accuracy gap between softmax attention and linear attention, we manipulate Mamba-2, a very strong linear attention variant. We first simplify Mamba-2 down to its most fundamental and important components, evaluating which specific choices make it most accurate. From this simplified Mamba variant (Mamba-2S), we improve the A-mask and increase the order of the hidden state, resulting in a method, which we call 2Mamba, that is nearly as accurate as softmax attention, yet much more memory efficient for long context lengths. We also investigate elements to Mamba-2 that help surpass softmax attention accuracy. Code is provided for all our experiments
【12】Flickering Multi-Armed Bandits
标题：闪烁的多臂强盗
链接：https://arxiv.org/abs/2602.17315
摘要：We introduce Flickering Multi-Armed Bandits (FMAB), a new MAB framework where the set of available arms (or actions) can change at each round, and the available set at any time may depend on the agent's previously selected arm. We model this constrained, evolving availability using random graph processes, where arms are nodes and the agent's movement is restricted to its local neighborhood. We analyze this problem under two random graph models: an i.i.d. Erdős--Rényi (ER) process and an Edge-Markovian process. We propose and analyze a two-phase algorithm that employs a lazy random walk for exploration to efficiently identify the optimal arm, followed by a navigation and commitment phase for exploitation. We establish high-probability and expected sublinear regret bounds for both graph settings. We show that the exploration cost of our algorithm is near-optimal by establishing a matching information-theoretic lower bound for this problem class, highlighting the fundamental cost of exploration under local-move constraints. We complement our theoretical guarantees with numerical simulations, including a scenario of a robotic ground vehicle scouting a disaster-affected region.
【13】Efficient privacy loss accounting for subsampling and random allocation
标题：考虑子采样和随机分配的有效隐私损失
链接：https://arxiv.org/abs/2602.17284
摘要：We consider the privacy amplification properties of a sampling scheme in which a user's data is used in $k$ steps chosen randomly and uniformly from a sequence (or set) of $t$ steps. This sampling scheme has been recently applied in the context of differentially private optimization (Chua et al., 2024a; Choquette-Choo et al., 2025) and communication-efficient high-dimensional private aggregation (Asi et al., 2025), where it was shown to have utility advantages over the standard Poisson sampling. Theoretical analyses of this sampling scheme (Feldman & Shenfeld, 2025; Dong et al., 2025) lead to bounds that are close to those of Poisson sampling, yet still have two significant shortcomings. First, in many practical settings, the resulting privacy parameters are not tight due to the approximation steps in the analysis. Second, the computed parameters are either the hockey stick or Renyi divergence, both of which introduce overheads when used in privacy loss accounting. In this work, we demonstrate that the privacy loss distribution (PLD) of random allocation applied to any differentially private algorithm can be computed efficiently. When applied to the Gaussian mechanism, our results demonstrate that the privacy-utility trade-off for random allocation is at least as good as that of Poisson subsampling. In particular, random allocation is better suited for training via DP-SGD. To support these computations, our work develops new tools for general privacy loss accounting based on a notion of PLD realization. This notion allows us to extend accurate privacy loss accounting to subsampling which previously required manual noise-mechanism-specific analysis.
【14】Unified Latents (UL): How to train your latents
标题：统一潜伏者（UL）：如何训练你的潜伏者
链接：https://arxiv.org/abs/2602.17270
摘要：We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.
【15】SoftDTW-CUDA-Torch: Memory-Efficient GPU-Accelerated Soft Dynamic Time Warping for PyTorch
标题：SoftDTW-CUDA-Torch：针对PyTorch的内存高效的GOP加速软动态时间扭曲
链接：https://arxiv.org/abs/2602.17206
备注：Technical Report
摘要 ：We present softdtw-cuda-torch, an open-source PyTorch library for computing Soft Dynamic Time Warping (SoftDTW) on GPUs. Our implementation addresses three key limitations of existing GPU implementations of SoftDTW: a hard sequence-length cap of 1024, numerical instability in the backward pass for small smoothing parameters, and excessive GPU memory consumption from materializing pairwise distance tensors. We introduce (1) tiled anti-diagonal kernel execution that removes the sequence-length constraint, (2) a log-space back-ward pass that prevents floating-point overflow, and (3) a fused distance-computation mode that eliminates the O(BN M ) intermediate distance tensor, achieving up to 98% memory reduction compared to prior work. The library supports arbitrary sequence lengths, full PyTorch autograd integration, and Soft-DTW Barycenter computation. Code is available at https://github.com/BGU-CS-VIL/sdtw-cuda-torch.
【16】Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
标题：通过子空间梯度电离为零阶训练提供动力
链接：https://arxiv.org/abs/2602.17155
摘要：Zeroth-order (ZO) optimization provides a gradient-free alternative to first-order (FO) methods by estimating gradients via finite differences of function evaluations, and has recently emerged as a memory-efficient paradigm for fine-tuning large-scale models by avoiding backpropagation. However, ZO optimization has a fundamental tension between accuracy and query efficiency. In this work, we show that ZO optimization can be substantially improved by unifying two complementary principles: (i) a projection-based subspace view that reduces gradient estimation variance by exploiting the intrinsic low-rank structure of model updates, and (ii) Muon-style spectral optimization that applies gradient orthogonalization to extract informative spectral structure from noisy ZO gradients. These findings form a unified framework of subspace gradient orthogonalization, which we instantiate in a new method, ZO-Muon, admitting a natural interpretation as a low-rank Muon optimizer in the ZO setting. Extensive experiments on large language models (LLMs) and vision transformers (ViTs) demonstrate that ZO-Muon significantly accelerates convergence and achieves a win-win improvement in accuracy and query/runtime efficiency. Notably, compared to the popular MeZO baseline, ZO-Muon requires only 24.7% of the queries to reach the same SST-2 performance for LLM fine-tuning, and improves accuracy by 25.1% on ViT-B fine-tuning on CIFAR-100.
【17】FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment
标题：FLoRG：使用低级革兰矩阵和Procrustes对齐的联合微调
链接：https://arxiv.org/abs/2602.17095
摘要：Parameter-efficient fine-tuning techniques such as low-rank adaptation (LoRA) enable large language models (LLMs) to adapt to downstream tasks efficiently. Federated learning (FL) further facilitates this process by enabling collaborative fine-tuning across distributed clients without sharing private data. However, the use of two separate low-rank matrices in LoRA for federated fine-tuning introduces two types of challenges. The first challenge arises from the error induced by separately aggregating those two low-rank matrices. The second challenge occurs even when the product of two low-rank matrices is aggregated. The server needs to recover factors via matrix decomposition, which is non-unique and can introduce decomposition drift. To tackle the aforementioned challenges, we propose FLoRG, a federated fine-tuning framework which employs a single low-rank matrix for fine-tuning and aggregates its Gram matrix (i.e., the matrix of inner products of its column vectors), eliminating the aggregation error while also reducing the communication overhead. FLoRG minimizes the decomposition drift by introducing a Procrustes alignment approach which aligns the decomposed matrix between consecutive fine-tuning rounds for consistent updates. We theoretically analyze the convergence of FLoRG and prove that adopting the Procrustes alignment results in a tighter convergence bound. Experimental results across multiple LLM fine-tuning benchmarks demonstrate that FLoRG outperforms five state-of-the-art baseline schemes in the downstream task accuracy and can reduce the communication overhead by up to 2041$\times$.
【18】MeGU: Machine-Guided Unlearning with Target Feature Disentanglement
标题：MeGU：具有目标特征解纠缠的机器引导去学习
链接：https://arxiv.org/abs/2602.17088
摘要：The growing concern over training data privacy has elevated the "Right to be Forgotten" into a critical requirement, thereby raising the demand for effective Machine Unlearning. However, existing unlearning approaches commonly suffer from a fundamental trade-off: aggressively erasing the influence of target data often degrades model utility on retained data, while conservative strategies leave residual target information intact. In this work, the intrinsic representation properties learned during model pretraining are analyzed. It is demonstrated that semantic class concepts are entangled at the feature-pattern level, sharing associated features while preserving concept-specific discriminative components. This entanglement fundamentally limits the effectiveness of existing unlearning paradigms. Motivated by this insight, we propose Machine-Guided Unlearning (MeGU), a novel framework that guides unlearning through concept-aware re-alignment. Specifically, Multi-modal Large Language Models (MLLMs) are leveraged to explicitly determine re-alignment directions for target samples by assigning semantically meaningful perturbing labels. To improve efficiency, inter-class conceptual similarities estimated by the MLLM are encoded into a lightweight transition matrix. Furthermore, MeGU introduces a positive-negative feature noise pair to explicitly disentangle target concept influence. During finetuning, the negative noise suppresses target-specific feature patterns, while the positive noise reinforces remaining associated features and aligns them with perturbing concepts. This coordinated design enables selective disruption of target-specific representations while preserving shared semantic structures. As a result, MeGU enables controlled and selective forgetting, effectively mitigating both under-unlearning and over-unlearning.
【19】Malliavin Calculus as Stochastic Backpropogation
标题：作为随机反传播的马里亚文演算
链接：https://arxiv.org/abs/2602.17013
摘要 ：We establish a rigorous connection between pathwise (reparameterization) and score-function (Malliavin) gradient estimators by showing that both arise from the Malliavin integration-by-parts identity. Building on this equivalence, we introduce a unified and variance-aware hybrid estimator that adaptively combines pathwise and Malliavin gradients using their empirical covariance structure. The resulting formulation provides a principled understanding of stochastic backpropagation and achieves minimum variance among all unbiased linear combinations, with closed-form finite-sample convergence bounds. We demonstrate 9% variance reduction on VAEs (CIFAR-10) and up to 35% on strongly-coupled synthetic problems. Exploratory policy gradient experiments reveal that non-stationary optimization landscapes present challenges for the hybrid approach, highlighting important directions for future work. Overall, this work positions Malliavin calculus as a conceptually unifying and practically interpretable framework for stochastic gradient estimation, clarifying when hybrid approaches provide tangible benefits and when they face inherent limitations.
【20】Arcee Trinity Large Technical Report
标题：Arcee Trinity大型技术报告
链接：https://arxiv.org/abs/2602.17004
摘要：We present the technical report for Arcee Trinity Large, a sparse Mixture-of-Experts model with 400B total parameters and 13B activated per token. Additionally, we report on Trinity Nano and Trinity Mini, with Trinity Nano having 6B total parameters with 1B activated per token, Trinity Mini having 26B total parameters with 3B activated per token. The models' modern architecture includes interleaved local and global attention, gated attention, depth-scaled sandwich norm, and sigmoid routing for Mixture-of-Experts. For Trinity Large, we also introduce a new MoE load balancing strategy titled Soft-clamped Momentum Expert Bias Updates (SMEBU). We train the models using the Muon optimizer. All three models completed training with zero loss spikes. Trinity Nano and Trinity Mini were pre-trained on 10 trillion tokens, and Trinity Large was pre-trained on 17 trillion tokens. The model checkpoints are available at https://huggingface.co/arcee-ai.
【21】Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding
标题：用于改进多路径推测解码的动态延迟树扩展
链接：https://arxiv.org/abs/2602.16994
摘要：Multi-path speculative decoding accelerates lossless sampling from a target model by using a cheaper draft model to generate a draft tree of tokens, and then applies a verification algorithm that accepts a subset of these. While prior work has proposed various verification algorithms for i.i.d rollouts, their relative performance under matched settings remains unclear. In this work, we firstly present a systematic evaluation of verification strategies across model families, tasks, and sampling regimes, and find that Traversal Verification dominates consistently, with OT-based methods lagging far behind. Our analysis uncovers that this occurs because OT-based methods achieve high multi-token acceptance near the root of the draft tree, while multi-token gains are most impactful deeper in the draft tree, where draft and target distributions diverge. Based on this insight, we propose delayed tree expansion, which drafts a partial single path, delaying the i.i.d. branching point. We show that delayed tree expansion preserves the target distribution and improves on root-node i.i.d rollouts. Further, we develop a dynamic neural selector that estimates the expected block efficiency of optimal-transport-based verification methods from draft and target features, enabling context-dependent expansion decisions. Our neural selector allows OT-based methods like SpecInfer to outperform Traversal Verification for the first time, achieving 5% higher average throughput across a wide range of models, datasets, and sampling settings.
【22】Early-Warning Signals of Grokking via Loss-Landscape Geometry
标题：基于损失-景观几何的Grokking预警信号
链接：https://arxiv.org/abs/2602.16967
备注：26 pages, 13 figures
摘要：Grokking -- the abrupt transition from memorization to generalization after prolonged training -- has been linked to confinement on low-dimensional execution manifolds in modular arithmetic. Whether this mechanism extends beyond arithmetic remains open. We study two sequence-learning benchmarks: SCAN compositional generalization and Dyck-1 depth prediction. Across both tasks and a wide range of learning rates, the commutator defect -- a curvature measure derived from non-commuting gradient updates -- rises well before generalization, with lead times following a superlinear power law (alpha approximately 1.18 for SCAN, approximately 1.13 for Dyck), consistent with prior results on modular arithmetic. Weight-space PCA reveals that spectral concentration is not a universal precursor; the commutator defect is. Causal interventions demonstrate a mechanistic role: amplifying non-commutativity accelerates grokking (roughly 32% on SCAN, roughly 50% on Dyck), while suppressing orthogonal gradient flow delays or prevents it. The three task families form a spectrum of causal sensitivity -- modular arithmetic is rigid, Dyck is responsive, SCAN is intermediate -- yet suppression delays or prevents grokking in all cases, establishing necessity as a universal finding. These results identify the commutator defect as a robust, architecture-agnostic, causally implicated early-warning signal for delayed generalization in transformers.
【23】A Unified Framework for Locality in Scalable MARL
标题：可扩展MARL中局部性的统一框架
链接：https://arxiv.org/abs/2602.16966
摘要 ：Scalable Multi-Agent Reinforcement Learning (MARL) is fundamentally challenged by the curse of dimensionality. A common solution is to exploit locality, which hinges on an Exponential Decay Property (EDP) of the value function. However, existing conditions that guarantee the EDP are often conservative, as they are based on worst-case, environment-only bounds (e.g., supremums over actions) and fail to capture the regularizing effect of the policy itself. In this work, we establish that locality can also be a \emph{policy-dependent} phenomenon. Our central contribution is a novel decomposition of the policy-induced interdependence matrix, $H^π$, which decouples the environment's sensitivity to state ($E^{\mathrm{s}}$) and action ($E^{\mathrm{a}}$) from the policy's sensitivity to state ($Π(π)$). This decomposition reveals that locality can be induced by a smooth policy (small $Π(π)$) even when the environment is strongly action-coupled, exposing a fundamental locality-optimality tradeoff. We use this framework to derive a general spectral condition $ρ(E^{\mathrm{s}}+E^{\mathrm{a}}Π(π)) < 1$ for exponential decay, which is strictly tighter than prior norm-based conditions. Finally, we leverage this theory to analyze a provably-sound localized block-coordinate policy improvement framework with guarantees tied directly to this spectral radius.
【24】Multi-Agent Lipschitz Bandits
标题：多特工利普希茨强盗
链接：https://arxiv.org/abs/2602.16965
摘要：We study the decentralized multi-player stochastic bandit problem over a continuous, Lipschitz-structured action space where hard collisions yield zero reward. Our objective is to design a communication-free policy that maximizes collective reward, with coordination costs that are independent of the time horizon $T$. We propose a modular protocol that first solves the multi-agent coordination problem -- identifying and seating players on distinct high-value regions via a novel maxima-directed search -- and then decouples the problem into $N$ independent single-player Lipschitz bandits. We establish a near-optimal regret bound of $\tilde{O}(T^{(d+1)/(d+2)})$ plus a $T$-independent coordination cost, matching the single-player rate. To our knowledge, this is the first framework providing such guarantees, and it extends to general distance-threshold collision models.
【25】Greedy Multi-Path Block Verification for Faster Decoding in Speculative Sampling
标题：贪婪的多路径块验证以实现推测采样中的更快解码
链接：https://arxiv.org/abs/2602.16961
摘要：The goal of $L$-step speculative decoding is to accelerate autoregressive decoding of a target model by using a cheaper draft model to generate a candidate path of $L$ tokens. Based on a verification algorithm involving target and draft model probabilities, a prefix of the candidate sequence is accepted, and an additional correction token is sampled from a residual distribution to ensure that the final output adheres to the target distribution. While standard speculative decoding uses a verification algorithm which is independent at each token on the path, a recent extension called block verification uses a joint condition involving all sampled on-path probabilities. Block verification (BV) was shown to be optimal over all verification algorithms which use only on-path probabilities, improving on standard speculative decoding. In this work, we first show that block verification is optimal even over verification algorithms that use off-path probabilities, by constructing an information-agnostic linear program (LP). Further, we can extend our LP to the setting where the draft model samples multiple candidate paths, and use it to construct a natural class of multi-path block verification generalizations. While computing the optimal algorithm in this class is not tractable, by considering a stricter class of greedy algorithms, we can formulate an efficient method called greedy multi-path block verification (GBV). Empirically, GBV can improve block efficiency by over 30% and reduce decoding walltimes by over 15% relative to BV. On Llama-3 70B, GBV can improve the end-to-end decoding throughput over SOTA multi-path verification methods by more than 15%.
【26】Automating Agent Hijacking via Structural Template Injection
标题：通过结构模板注入自动化代理劫持
链接：https://arxiv.org/abs/2602.16958
摘要：Agent hijacking, highlighted by OWASP as a critical threat to the Large Language Model (LLM) ecosystem, enables adversaries to manipulate execution by injecting malicious instructions into retrieved content. Most existing attacks rely on manually crafted, semantics-driven prompt manipulation, which often yields low attack success rates and limited transferability to closed-source commercial models. In this paper, we propose Phantom, an automated agent hijacking framework built upon Structured Template Injection that targets the fundamental architectural mechanisms of LLM agents. Our key insight is that agents rely on specific chat template tokens to separate system, user, assistant, and tool instructions. By injecting optimized structured templates into the retrieved context, we induce role confusion and cause the agent to misinterpret the injected content as legitimate user instructions or prior tool outputs. To enhance attack transferability against black-box agents, Phantom introduces a novel attack template search framework. We first perform multi-level template augmentation to increase structural diversity and then train a Template Autoencoder (TAE) to embed discrete templates into a continuous, searchable latent space. Subsequently, we apply Bayesian optimization to efficiently identify optimal adversarial vectors that are decoded into high-potency structured templates. Extensive experiments on Qwen, GPT, and Gemini demonstrate that our framework significantly outperforms existing baselines in both Attack Success Rate (ASR) and query efficiency. Moreover, we identified over 70 vulnerabilities in real-world commercial products that have been confirmed by vendors, underscoring the practical severity of structured template-based hijacking and providing an empirical foundation for securing next-generation agentic systems.
【27】MALLVI: a multi agent framework for integrated generalized robotics manipulation
标题：MALLVI：一个用于集成广义机器人操纵的多智能体框架
链接：https://arxiv.org/abs/2602.16898
摘要 ：Task planning for robotic manipulation with large language models (LLMs) is an emerging area. Prior approaches rely on specialized models, fine tuning, or prompt tuning, and often operate in an open loop manner without robust environmental feedback, making them fragile in dynamic settings.We present MALLVi, a Multi Agent Large Language and Vision framework that enables closed loop feedback driven robotic manipulation. Given a natural language instruction and an image of the environment, MALLVi generates executable atomic actions for a robot manipulator. After action execution, a Vision Language Model (VLM) evaluates environmental feedback and decides whether to repeat the process or proceed to the next step.Rather than using a single model, MALLVi coordinates specialized agents, Decomposer, Localizer, Thinker, and Reflector, to manage perception, localization, reasoning, and high level planning. An optional Descriptor agent provides visual memory of the initial state. The Reflector supports targeted error detection and recovery by reactivating only relevant agents, avoiding full replanning.Experiments in simulation and real world settings show that iterative closed loop multi agent coordination improves generalization and increases success rates in zero shot manipulation tasks.Code available at https://github.com/iman1234ahmadi/MALLVI.
【28】On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
标题：模加法的机制和动力学：傅立叶特征、彩票和Grokking
链接：https://arxiv.org/abs/2602.16849
摘要：We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task. Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics. While prior work has identified that individual neurons learn single-frequency Fourier features and phase alignment, it does not fully explain how these features combine into a global solution. We bridge this gap by formalizing a diversification condition that emerges during training when overparametrized, consisting of two parts: phase symmetry and frequency diversification. We prove that these properties allow the network to collectively approximate a flawed indicator function on the correct logic for the modular addition task. While individual neurons produce noisy signals, the phase symmetry enables a majority-voting scheme that cancels out noise, allowing the network to robustly identify the correct sum. Furthermore, we explain the emergence of these features under random initialization via a lottery ticket mechanism. Our gradient flow analysis proves that frequencies compete within each neuron, with the "winner" determined by its initial spectral magnitude and phase alignment. From a technical standpoint, we provide a rigorous characterization of the layer-wise phase coupling dynamics and formalize the competitive landscape using the ODE comparison lemma. Finally, we use these insights to demystify grokking, characterizing it as a three-stage process involving memorization followed by two generalization phases, driven by the competition between loss minimization and weight decay.
【29】VAM: Verbalized Action Masking for Controllable Exploration in RL Post-Training -- A Chess Case Study
标题：VAM：RL训练后可控探索的言语动作掩蔽--国际象棋案例研究
链接：https://arxiv.org/abs/2602.16833
摘要：Exploration remains a key bottleneck for reinforcement learning (RL) post-training of large language models (LLMs), where sparse feedback and large action spaces can lead to premature collapse into repetitive behaviors. We propose Verbalized Action Masking (VAM), which verbalizes an action mask in the prompt and enforces that the model outputs an action from the masked set. Building on this interface, we introduce iterative action-space pruning: if the target action is not sampled, we remove valid sampled actions from the mask and resample under the reduced candidate set, repeating until the target is sampled or a fixed budget is exhausted. We study VAM in chess and evaluate it under two training regimes: an engine-play regime that generates states via play against an engine opponent and a fixed-dataset regime that trains from a fixed dataset of positions with verifier scores. Across held-out chess puzzles and full-game play measured by average centipawn loss (ACPL), VAM improves learning efficiency and final performance over strong baselines, highlighting verbalized masking as a practical mechanism for controllable exploration in LLM RL post-training.
【30】HiVAE: Hierarchical Latent Variables for Scalable Theory of Mind
标题：HiVAE：可扩展心理理论的层次潜在变量
链接：https://arxiv.org/abs/2602.16826
备注：Accepted at the Workshop on Theory of Mind for AI (ToM4AI) at the 40th AAAI Conference on Artificial Intelligence (AAAI-26), Singapore, 2026
摘要：Theory of mind (ToM) enables AI systems to infer agents' hidden goals and mental states, but existing approaches focus mainly on small human understandable gridworld spaces. We introduce HiVAE, a hierarchical variational architecture that scales ToM reasoning to realistic spatiotemporal domains. Inspired by the belief-desire-intention structure of human cognition, our three-level VAE hierarchy achieves substantial performance improvements on a 3,185-node campus navigation task. However, we identify a critical limitation: while our hierarchical structure improves prediction, learned latent representations lack explicit grounding to actual mental states. We propose self-supervised alignment strategies and present this work to solicit community feedback on grounding approaches.
【31】Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees
标题：形式机制的可解释性：具有可证明保证的自动电路发现
链接：https://arxiv.org/abs/2602.16823
备注：To appear in ICLR 2026
摘要 ：*Automated circuit discovery* is a central tool in mechanistic interpretability for identifying the internal components of neural networks responsible for specific behaviors. While prior methods have made significant progress, they typically depend on heuristics or approximations and do not offer provable guarantees over continuous input domains for the resulting circuits. In this work, we leverage recent advances in neural network verification to propose a suite of automated algorithms that yield circuits with *provable guarantees*. We focus on three types of guarantees: (1) *input domain robustness*, ensuring the circuit agrees with the model across a continuous input region; (2) *robust patching*, certifying circuit alignment under continuous patching perturbations; and (3) *minimality*, formalizing and capturing a wide array of various notions of succinctness. Interestingly, we uncover a diverse set of novel theoretical connections among these three families of guarantees, with critical implications for the convergence of our algorithms. Finally, we conduct experiments with state-of-the-art verifiers on various vision models, showing that our algorithms yield circuits with substantially stronger robustness guarantees than standard circuit discovery methods, establishing a principled foundation for provable circuit discovery.
【32】Hybrid-Gym: Training Coding Agents to Generalize Across Tasks
标题：Hybrid-Gym：训练编码代理在任务间泛化
链接：https://arxiv.org/abs/2602.16819
摘要：When assessing the quality of coding agents, predominant benchmarks focus on solving single issues on GitHub, such as SWE-Bench. In contrast, in real use, these agents solve more various and complex tasks that involve other skills such as exploring codebases, testing software, and designing architecture. In this paper, we first characterize some transferable skills that are shared across diverse tasks by decomposing trajectories into fine-grained components, and derive a set of principles for designing auxiliary training tasks to teach language models these skills. Guided by these principles, we propose a training environment, Hybrid-Gym, consisting of a set of scalable synthetic tasks, such as function localization and dependency search. Experiments show that agents trained on our synthetic tasks effectively generalize to diverse real-world tasks that are not present in training, improving a base model by 25.4% absolute gain on SWE-Bench Verified, 7.9% on SWT-Bench Verified, and 5.1% on Commit-0 Lite. Hybrid-Gym also complements datasets built for the downstream tasks (e.g., improving SWE-Play by 4.9% on SWT-Bench Verified). Code available at: https://github.com/yiqingxyq/Hybrid-Gym.
【33】Simple Baselines are Competitive with Code Evolution
标题：简单基线与代码进化具有竞争力
链接：https://arxiv.org/abs/2602.16805
摘要：Code evolution is a family of techniques that rely on large language models to search through possible computer programs by evolving or mutating existing code. Many proposed code evolution pipelines show impressive performance but are often not compared to simpler baselines. We test how well two simple baselines do over three domains: finding better mathematical bounds, designing agentic scaffolds, and machine learning competitions. We find that simple baselines match or exceed much more sophisticated methods in all three. By analyzing these results we find various shortcomings in how code evolution is both developed and used. For the mathematical bounds, a problem's search space and domain knowledge in the prompt are chiefly what dictate a search's performance ceiling and efficiency, with the code evolution pipeline being secondary. Thus, the primary challenge in finding improved bounds is designing good search spaces, which is done by domain experts, and not the search itself. When designing agentic scaffolds we find that high variance in the scaffolds coupled with small datasets leads to suboptimal scaffolds being selected, resulting in hand-designed majority vote scaffolds performing best. We propose better evaluation methods that reduce evaluation stochasticity while keeping the code evolution economically feasible. We finish with a discussion of avenues and best practices to enable more rigorous code evolution in future work.
【34】Attending to Routers Aids Indoor Wireless Localization
标题：服务路由器有助于室内无线定位
链接：https://arxiv.org/abs/2602.16762
备注：AAAI 2026 Workshop on Machine Learning for Wireless Communication and Networks (ML4Wireless)
摘要：Modern machine learning-based wireless localization using Wi-Fi signals continues to face significant challenges in achieving groundbreaking performance across diverse environments. A major limitation is that most existing algorithms do not appropriately weight the information from different routers during aggregation, resulting in suboptimal convergence and reduced accuracy. Motivated by traditional weighted triangulation methods, this paper introduces the concept of attention to routers, ensuring that each router's contribution is weighted differently when aggregating information from multiple routers for triangulation. We demonstrate, by incorporating attention layers into a standard machine learning localization architecture, that emphasizing the relevance of each router can substantially improve overall performance. We have also shown through evaluation over the open-sourced datasets and demonstrate that Attention to Routers outperforms the benchmark architecture by over 30% in accuracy.
【35】Intent Laundering: AI Safety Datasets Are Not What They Seem
标题：意图洗钱：人工智能安全数据集并非表面所见
链接：https://arxiv.org/abs/2602.16729
备注：v1 preprint
摘要：We systematically evaluate the quality of widely used AI safety datasets from two perspectives: in isolation and in practice. In isolation, we examine how well these datasets reflect real-world attacks based on three key properties: driven by ulterior intent, well-crafted, and out-of-distribution. We find that these datasets overrely on "triggering cues": words or phrases with overt negative/sensitive connotations that are intended to trigger safety mechanisms explicitly, which is unrealistic compared to real-world attacks. In practice, we evaluate whether these datasets genuinely measure safety risks or merely provoke refusals through triggering cues. To explore this, we introduce "intent laundering": a procedure that abstracts away triggering cues from attacks (data points) while strictly preserving their malicious intent and all relevant details. Our results indicate that current AI safety datasets fail to faithfully represent real-world attacks due to their overreliance on triggering cues. In fact, once these cues are removed, all previously evaluated "reasonably safe" models become unsafe, including Gemini 3 Pro and Claude Sonnet 3.7. Moreover, when intent laundering is adapted as a jailbreaking technique, it consistently achieves high attack success rates, ranging from 90% to over 98%, under fully black-box access. Overall, our findings expose a significant disconnect between how model safety is evaluated and how real-world adversaries behave.
【36】Speech to Speech Synthesis for Voice Impersonation
标题：语音模拟的语音到语音合成
链接：https://arxiv.org/abs/2602.16721
备注：Original work completed in April 2020. This version includes minor formatting updates
摘要：Numerous models have shown great success in the fields of speech recognition as well as speech synthesis, but models for speech to speech processing have not been heavily explored. We propose Speech to Speech Synthesis Network (STSSN), a model based on current state of the art systems that fuses the two disciplines in order to perform effective speech to speech style transfer for the purpose of voice impersonation. We show that our proposed model is quite powerful, and succeeds in generating realistic audio samples despite a number of drawbacks in its capacity. We benchmark our proposed model by comparing it with a generative adversarial model which accomplishes a similar task, and show that ours produces more convincing results.
【37】DARTH-PUM: A Hybrid Processing-Using-Memory Architecture
标题：DARTH-PUM：一种使用内存的混合处理架构
链接：https://arxiv.org/abs/2602.16075
备注：To appear in the ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2026
摘要：Analog processing-using-memory (PUM; a.k.a. in-memory computing) makes use of electrical interactions inside memory arrays to perform bulk matrix-vector multiplication (MVM) operations. However, many popular matrix-based kernels need to execute non-MVM operations, which analog PUM cannot directly perform. To retain its energy efficiency, analog PUM architectures augment memory arrays with CMOS-based domain-specific fixed-function hardware to provide complete kernel functionality, but the difficulty of integrating such specialized CMOS logic with memory arrays has largely limited analog PUM to being an accelerator for machine learning inference, or for closely related kernels. An opportunity exists to harness analog PUM for general-purpose computation: recent works have shown that memory arrays can also perform Boolean PUM operations, albeit with very different supporting hardware and electrical signals than analog PUM. We propose DARTH-PUM, a general-purpose hybrid PUM architecture that tackles key hardware and software challenges to integrating analog PUM and digital PUM. We propose optimized peripheral circuitry, coordinating hardware to manage and interface between both types of PUM, an easy-to-use programming interface, and low-cost support for flexible data widths. These design elements allow us to build a practical PUM architecture that can execute kernels fully in memory, and can scale easily to cater to domains ranging from embedded applications to large-scale data-driven computing. We show how three popular applications (AES encryption, convolutional neural networks, large-language models) can map to and benefit from DARTH-PUM, with speedups of 59.4x, 14.8x, and 40.8x over an analog+CPU baseline.
【38】Efficient Remote Prefix Fetching with GPU-native Media ASICs
标题：使用原生PU媒体ASIC进行高效远程前置提取
链接：https://arxiv.org/abs/2602.09725
摘要：Remote KV cache reuse fetches KV cache for identical contexts from remote storage, avoiding recomputation, accelerating LLM inference. While it excels in high-speed networks, its performance degrades significantly in bandwidth-limited scenarios. Recent studies address this by transmitting KV caches in compressed form, but the associated heavyweight decompression counteracts the KV reuse benefits. In this paper, we propose an efficient and widely deployable remote KV cache reuse solution that leverages GPU-native video codecs. Our system, KVFetcher, enables effective KV cache coding with two techniques. The codec-friendly tensor layout compresses the KV cache in a highly compact video format, enabling fast transmission. The efficient KV fetcher orchestrates the transmission, decoding, and restoration of compressed KV caches in an efficient pipelined manner, eliminating resource contention, masking network fluctuations, and achieving minimum time-to-first-token (TTFT). We prototype KVFetcher on diverse GPUs from high- to low-end. Experiments reveal that it reduces TTFT by up to 3.51 times while maintaining lossless accuracy, compared to SOTA methods.
【39】Quantum Scrambling Born Machine
标题：量子扰频诞生机
链接：https://arxiv.org/abs/2602.17281
摘要：Quantum generative modeling, where the Born rule naturally defines probability distributions through measurement of parameterized quantum states, is a promising near-term application of quantum computing. We propose a Quantum Scrambling Born Machine in which a fixed entangling unitary -- acting as a scrambling reservoir -- provides multi-qubit entanglement, while only single-qubit rotations are optimized. We consider three entangling unitaries -- a Haar random unitary and two physically realizable approximations, a finite-depth brickwork random circuit and analog time evolution under nearest-neighbor spin-chain Hamiltonians -- and show that, for the benchmark distributions and system sizes considered, once the entangler produces near-Haar-typical entanglement the model learns the target distribution with weak sensitivity to the scrambler's microscopic origin. Finally, promoting the Hamiltonian couplings to trainable parameters casts the generative task as a variational Hamiltonian problem, with performance competitive with representative classical generative models at matched parameter count.
【40】Anti-causal domain generalization: Leveraging unlabeled data
标题：反因果领域概括：利用未标记数据
链接：https://arxiv.org/abs/2602.17187
摘要 ：The problem of domain generalization concerns learning predictive models that are robust to distribution shifts when deployed in new, previously unseen environments. Existing methods typically require labeled data from multiple training environments, limiting their applicability when labeled data are scarce. In this work, we study domain generalization in an anti-causal setting, where the outcome causes the observed covariates. Under this structure, environment perturbations that affect the covariates do not propagate to the outcome, which motivates regularizing the model's sensitivity to these perturbations. Crucially, estimating these perturbation directions does not require labels, enabling us to leverage unlabeled data from multiple environments. We propose two methods that penalize the model's sensitivity to variations in the mean and covariance of the covariates across environments, respectively, and prove that these methods have worst-case optimality guarantees under certain classes of environments. Finally, we demonstrate the empirical performance of our approach on a controlled physical system and a physiological signal dataset.
【41】The Impact of Formations on Football Matches Using Double Machine Learning. Is it worth parking the bus?
标题：使用双机器学习的队形对足球比赛的影响。停车巴士值得吗？
链接：https://arxiv.org/abs/2602.16830
备注：17 pages, 5 figures, 3 tables
摘要：This study addresses a central tactical dilemma for football coaches: whether to employ a defensive strategy, colloquially known as "parking the bus", or a more offensive one. Using an advanced Double Machine Learning (DML) framework, this project provides a robust and interpretable tool to estimate the causal impact of different formations on key match outcomes such as goal difference, possession, corners, and disciplinary actions. Leveraging a dataset of over 22,000 matches from top European leagues, formations were categorized into six representative types based on tactical structure and expert consultation. A major methodological contribution lies in the adaptation of DML to handle categorical treatments, specifically formation combinations, through a novel matrix-based residualization process, allowing for a detailed estimation of formation-versus-formation effects that can inform a coach's tactical decision-making. Results show that while offensive formations like 4-3-3 and 4-2-3-1 offer modest statistical advantages in possession and corners, their impact on goals is limited. Furthermore, no evidence supports the idea that defensive formations, commonly associated with parking the bus, increase a team's winning potential. Additionally, red cards appear unaffected by formation choice, suggesting other behavioral factors dominate. Although this approach does not fully capture all aspects of playing style or team strength, it provides a valuable framework for coaches to analyze tactical efficiency and sets a precedent for future research in sports analytics.
机器翻译由腾讯交互翻译提供，仅供参考
点击“阅读原文”获取带摘要的学术速递
Python社区是高质量的Python/Django开发社区
本文地址：http://www.python88.com/topic/193077
本文地址：http://www.python88.com/topic/193077