|
创作新主题 |
社区所有版块导航
DATA
| docker Elasticsearch |
WEB开发
| linux MongoDB Redis DATABASE NGINX 其他Web框架 web工具 zookeeper tornado NoSql Bootstrap js peewee Git bottle IE MQ Jquery |
机器学习
| 机器学习算法 |
产品
| 短视频 |
印度
| 印度 |
一周十大热门主题
点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！
cs.LG 方向，今日共计141篇
大模型相关(26篇)
【1】Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
标题：重新思考大型语言模型的政策提炼：现象学、机制和配方
链接：https://arxiv.org/abs/2604.13016
备注：30 pages, 23 figures. Code: https://github.com/thunlp/OPD
摘要：On-policy distillation（OPD）已经成为大型语言模型后训练的核心技术，但其训练动态仍然知之甚少。本文提供了一个系统的调查OPD的动力学和机制。我们首先确定两个条件决定OPD是否成功或失败：（i）学生和教师应该共享兼容的思维模式;（ii）即使具有一致的思维模式和更高的分数，教师也必须提供超出学生在培训期间所看到的真正新的能力。我们通过弱到强的逆向蒸馏验证了这些发现，表明从学生的角度来看，同一家庭的1.5B和7 B教师在分布上是不可区分的。探讨令牌级机制，我们发现，成功的OPD的特点是在学生访问状态，一个小的共享令牌集，集中了大部分的概率质量（97%-99%）的高概率令牌上的渐进对齐。我们进一步提出了两个实用的策略，以恢复失败的OPD：政策外冷启动和教师对齐的提示选择。最后，我们表明，OPD的密集令牌级奖励的明显免费午餐是有成本的，这就提出了OPD是否可以扩展到长期蒸馏的问题。
摘要：On-policy distillation (OPD) has become a core technique in the post-training of large language models, yet its training dynamics remain poorly understood. This paper provides a systematic investigation of OPD dynamics and mechanisms. We first identify that two conditions govern whether OPD succeeds or fails: (i) the student and teacher should share compatible thinking patterns; and (ii) even with consistent thinking patterns and higher scores, the teacher must offer genuinely new capabilities beyond what the student has seen during training. We validate these findings through weak-to-strong reverse distillation, showing that same-family 1.5B and 7B teachers are distributionally indistinguishable from the student's perspective. Probing into the token-level mechanism, we show that successful OPD is characterized by progressive alignment on high-probability tokens at student-visited states, a small shared token set that concentrates most of the probability mass (97%-99%). We further propose two practical strategies to recover failing OPD: off-policy cold start and teacher-aligned prompt selection. Finally, we show that OPD's apparent free lunch of dense token-level reward comes at a cost, raising the question of whether OPD can scale to long-horizon distillation.
【2】Parcae: Scaling Laws For Stable Looped Language Models
标题：Parcae：稳定循环语言模型的缩放定律
链接：https://arxiv.org/abs/2604.12946
摘要：传统的固定深度架构通过增加训练FLOP来扩展质量，通常通过增加参数化，以更高的内存占用或数据为代价。一个潜在的替代方案是循环架构，它通过在循环中通过一个层块发送激活来增加FLOP。虽然很有前途，但现有的训练循环架构的方法可能不稳定，会遭受残余爆炸和损失尖峰。我们解决这些挑战，通过重铸循环作为一个非线性时变动力系统的剩余流。通过对该系统的线性近似，我们发现在现有的环形结构中，由于其注入参数中的大谱范数而发生不稳定。为了解决这些不稳定的问题，我们提出了Parcae，一种新的稳定的，循环的架构，通过离散化的负对角参数化的注入参数的谱范数的约束。因此，Parcae比以前的大规模循环模型降低了6.3%的验证困惑。使用我们稳定的循环架构，我们研究了循环作为一种媒介的缩放特性，通过增加训练和测试时间中的FLOP来提高质量。对于训练，我们推导出可预测的幂律来缩放FLOP，同时保持参数计数固定。我们最初的缩放定律表明，在固定的FLOP预算下，循环和数据应该同步增加。在测试时，我们发现Parcae可以使用循环来扩展计算，遵循可预测的，饱和的指数衰减。当扩展到1.3B参数时，我们发现在固定参数和数据预算下，与强大的Transformer基线相比，Parcae将CORE和Core-Extended质量提高了2.99和1.18个点，实现了高达87.5%的相对质量，而Transformer的大小是其两倍。
摘要：Traditional fixed-depth architectures scale quality by increasing training FLOPs, typically through increased parameterization, at the expense of a higher memory footprint, or data. A potential alternative is looped architectures, which instead increase FLOPs by sending activations through a block of layers in a loop. While promising, existing recipes for training looped architectures can be unstable, suffering from residual explosion and loss spikes. We address these challenges by recasting looping as a nonlinear time-variant dynamical system over the residual stream. Via a linear approximation to this system, we find that instability occurs in existing looped architectures as a result of large spectral norms in their injection parameters. To address these instability issues, we propose Parcae, a novel stable, looped architecture that constrains the spectral norm of the injection parameters via discretization of a negative diagonal parameterization. As a result, Parcae achieves up to 6.3% lower validation perplexity over prior large-scale looped models. Using our stable looped architecture, we investigate the scaling properties of looping as a medium to improve quality by increasing FLOPs in training and test-time. For training, we derive predictable power laws to scale FLOPs while keeping parameter count fixed. Our initial scaling laws suggest that looping and data should be increased in tandem, given a fixed FLOP budget. At test-time, we find that Parcae can use looping to scale compute, following a predictable, saturating exponential decay. When scaled up to 1.3B parameters, we find that Parcae improves CORE and Core-Extended quality by 2.99 and 1.18 points when compared to strong Transformer baselines under a fixed parameter and data budget, achieving a relative quality of up to 87.5% a Transformer twice the size.
【3】Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
标题：不显示像素，显示线索：通过感知程序解锁语言模型中的视觉工具推理
链接：https://arxiv.org/abs/2604.12896
备注：Accepted to CVPR 2026
摘要：多模态语言模型（MLLM）越来越多地与视觉工具（例如，深度、流动、对应性）以增强视觉推理。然而，尽管可以访问这些工具生成的视觉提示，MLLM往往无法从中受益。现有方法通常将原始工具输出馈送到模型中，但这些密集的像素级表示与LLM的语言原生推理优势不一致，导致感知能力弱和对语言先验的依赖。我们认为，在视觉工具可以提供必要的视觉线索的问题中，瓶颈不是更多的工具调用或更大的MLLM，而是工具输出如何表示。我们引入了感知程序（P$^2$），这是一种无需训练、与模型无关的方法，可以将工具输出重写为紧凑、结构化、语言原生的摘要，MLLM可以直接解析和推理。在BLINK中的六个以感知为中心的任务中，P$^2$始终比基本模型和原始工具增强的基线产生很大的改进。以GPT-5 Mini为基础模型，P$^2$在多视图推理上的准确率从41.35%提高到86.47%，在相对深度上的准确率从52.42%提高到81.45%，跨任务的平均增益达到22%，创造了新的最先进的结果。即使在较小的MLLM上，例如，InternVL3.5- 4 B和Qwen 3VL-4 B，我们观察到从P$^2$获得15-40\%的绝对收益，超过了先前的代理，监督和基于RL的工具使用方法-无需任何训练或模型修改。
摘要 ：Multimodal language models (MLLMs) are increasingly paired with vision tools (e.g., depth, flow, correspondence) to enhance visual reasoning. However, despite access to these tool-generated visual cues, MLLMs often fail to benefit from them. Existing approaches typically feed raw tool outputs into the model, but these dense, pixel-level representations are misaligned with the language-native reasoning strengths of LLMs, leading to weak perception and reliance on language priors. We argue that, in problems where vision tools can provide the necessary visual cues, the bottleneck is not more tool calls or larger MLLMs, it is how tool outputs are represented. We introduce Perception Programs (P$^2$), a training-free, model-agnostic method that rewrites tool outputs into compact, structured, language-native summaries that MLLMs can directly parse and reason over. Across six perception-centric tasks in BLINK, P$^2$ consistently yields large improvements over base models and raw tool-augmented baselines. With GPT-5 Mini as the base model, P$^2$ raises its accuracy from 41.35\% to 86.47\% on multi-view reasoning, from 52.42\% to 81.45\% on relative depth, and achieves a 22\% average gain across tasks, setting new state-of-the-art results. Even on smaller MLLMs, e.g., InternVL3.5-4B and Qwen3VL-4B, we observe 15-40\% absolute gains from P$^2$, surpassing prior agentic, supervised, and RL-based tool-use methods-without any training or model modifications.
【4】Understanding and Improving Continuous Adversarial Training for LLMs via In-context Learning Theory
标题：通过上下文学习理论理解和改进法学硕士的持续对抗训练
链接：https://arxiv.org/abs/2604.12817
备注：The Fourteenth International Conference on Learning Representations (ICLR 2026)
摘要：对抗性训练（AT）是大型语言模型（LLM）抵御越狱攻击的有效防御手段，但在LLM上执行AT代价高昂。为了提高LLM的AT效率，最近的研究提出了连续AT（CAT），在AT期间在LLM的连续嵌入空间内搜索对抗性输入。虽然《禁止酷刑和其他残忍、不人道或有辱人格的待遇或处罚公约》取得了经验上的成功，但其基本机制，即，为什么嵌入空间中的对抗性扰动可以帮助LLM防御在输入令牌空间中合成的越狱提示，仍然是未知的。本文首次提出了基于情境学习（ICL）理论的学习型教师CAT的理论分析。对于在上下文线性回归任务中使用嵌入空间中的对抗性示例训练的线性Transformers，我们证明了一个与嵌入空间中的扰动半径负相关的鲁棒泛化界。这清楚地解释了为什么CAT可以防御来自LLM令牌空间的越狱提示。此外，鲁棒界表明，对抗训练LLM的鲁棒性与其嵌入矩阵的奇异值密切相关。在此基础上，我们提出了改进LLM CAT通过引入一个额外的正则化项，这取决于LLM的嵌入矩阵的奇异值，到CAT的目标函数。在现实世界的LLM上的实验表明，我们的方法可以帮助LLM实现更好的越狱鲁棒性效用权衡。该代码可在https://github.com/fshp971/continuous-adv-icl上获得。
摘要：Adversarial training (AT) is an effective defense for large language models (LLMs) against jailbreak attacks, but performing AT on LLMs is costly. To improve the efficiency of AT for LLMs, recent studies propose continuous AT (CAT) that searches for adversarial inputs within the continuous embedding space of LLMs during AT. While CAT has achieved empirical success, its underlying mechanism, i.e., why adversarial perturbations in the embedding space can help LLMs defend against jailbreak prompts synthesized in the input token space, remains unknown. This paper presents the first theoretical analysis of CAT on LLMs based on in-context learning (ICL) theory. For linear transformers trained with adversarial examples from the embedding space on in-context linear regression tasks, we prove a robust generalization bound that has a negative correlation with the perturbation radius in the embedding space. This clearly explains why CAT can defend against jailbreak prompts from the LLM's token space. Further, the robust bound shows that the robustness of an adversarially trained LLM is closely related to the singular values of its embedding matrix. Based on this, we propose to improve LLM CAT by introducing an additional regularization term, which depends on singular values of the LLM's embedding matrix, into the objective function of CAT. Experiments on real-world LLMs demonstrate that our method can help LLMs achieve a better jailbreak robustness-utility tradeoff. The code is available at https://github.com/fshp971/continuous-adv-icl.
【5】Interpretable Relational Inference with LLM-Guided Symbolic Dynamics Modeling
标题：具有LLM引导的符号动力学建模的可解释关系推理
链接：https://arxiv.org/abs/2604.12806
备注：Submitted to conference
摘要：从观测动力学推断潜在相互作用结构是多体相互作用系统中的一个基本反问题。大多数神经方法依赖于可训练图上的黑箱代理，以牺牲机械可解释性为代价来实现准确性。符号回归提供了明确的动力学方程和更强的归纳偏差，但通常假设已知的拓扑结构和固定的函数库。我们提出了\textbf{COSINE}（\textbf{C}o-\textbf{O}优化\textbf{S}符号\textbf{I}交互和\textbf{N}网络\textbf{E}边），这是一个可微框架，可以联合发现交互图和稀疏符号动力学。为了克服固定符号库的局限性，COSINE还结合了一个外循环大型语言模型，该模型使用来自内部优化循环的反馈自适应地修剪和扩展假设空间。合成系统和大规模的真实世界的流行病数据的实验表明，强大的结构恢复和紧凑，机制对齐的动力学表达式。代码：https://anonymous.4open.science/r/COSINE-6D43。
摘要：Inferring latent interaction structures from observed dynamics is a fundamental inverse problem in many-body interacting systems. Most neural approaches rely on black-box surrogates over trainable graphs, achieving accuracy at the expense of mechanistic interpretability. Symbolic regression offers explicit dynamical equations and stronger inductive biases, but typically assumes known topology and a fixed function library. We propose \textbf{COSINE} (\textbf{C}o-\textbf{O}ptimization of \textbf{S}ymbolic \textbf{I}nteractions and \textbf{N}etwork \textbf{E}dges), a differentiable framework that jointly discovers interaction graphs and sparse symbolic dynamics. To overcome the limitations of fixed symbolic libraries, COSINE further incorporates an outer-loop large language model that adaptively prunes and expands the hypothesis space using feedback from the inner optimization loop. Experiments on synthetic systems and large-scale real-world epidemic data demonstrate robust structural recovery and compact, mechanism-aligned dynamical expressions. Code: https://anonymous.4open.science/r/COSINE-6D43.
【6】LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety
标题：LASA：LLM安全的语义瓶颈上的数字不可知的语义对齐
链接：https://arxiv.org/abs/2604.12710
摘要：大型语言模型（LLM）通常在高资源语言中表现出强大的安全性能，但在低资源语言中查询时会表现出严重的漏洞。我们将这种差距归因于语言不可知的语义理解能力和偏向高资源语言的语言主导的安全对齐之间的不匹配。与这一假设相一致，我们凭经验确定的语义瓶颈LLM，一个中间层，模型表示的几何形状主要由共享的语义内容，而不是语言身份。在此基础上，我们提出了不可知语义对齐（LASA），它直接在语义瓶颈锚定安全对齐。实验表明，LASA大大提高了所有语言的安全性：在LLaMA-3.1-8B-Instruct上，平均攻击成功率（ASR）从24.7%下降到2.8%，在Qwen2.5和Qwen 3 Instruct模型（7 B-32 B）上保持在3%-4%左右。总之，我们的分析和方法提供了一个代表性的LLM安全的角度来看，这表明安全对齐需要锚定安全的理解，而不是在表面文本，但在模型的语言不可知的语义空间。
摘要：Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities when queried in low-resource languages. We attribute this gap to a mismatch between language-agnostic semantic understanding ability and language-dominant safety alignment biased toward high-resource languages. Consistent with this hypothesis, we empirically identify the semantic bottleneck in LLMs, an intermediate layer in which the geometry of model representations is governed primarily by shared semantic content rather than language identity. Building on this observation, we propose Language-Agnostic Semantic Alignment (LASA), which anchors safety alignment directly in semantic bottlenecks. Experiments show that LASA substantially improves safety across all languages: average attack success rate (ASR) drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains around 3-4% across Qwen2.5 and Qwen3 Instruct models (7B-32B). Together, our analysis and method offer a representation-level perspective on LLM safety, suggesting that safety alignment requires anchoring safety understanding not in surface text, but in the model's language-agnostic semantic space.
【7】TimeSAF: Towards LLM-Guided Semantic Asynchronous Fusion for Time Series Forecasting
标题：TimeSAF：迈向LLM引导的时间序列预测的语义非同步融合
链接：https://arxiv.org/abs/2604.12648
摘要：尽管大型语言模型（LLM）最近在时间序列预测中取得了成功，但大多数现有方法仍然采用深度同步融合策略，其中文本和时间特征之间的密集交互在网络的每一层都得到加强。这种设计忽略了模态之间固有的粒度不匹配，并导致我们称之为语义感知失调：LLM提供的高级抽象语义与时间序列的低级细粒度数值动态不适当地纠缠在一起，使得语义先验难以有效地指导预测。为了解决这个问题，我们提出了TimeSAF，一个新的框架的基础上分层异步融合。与同步方法不同，TimeSAF显式地将单峰特征学习与跨模态交互相结合。它引入了一个独立的跨模态语义融合主干，该主干使用可学习的查询以自底向上的方式从时态和提示主干中聚合全局语义，以及一个阶段式语义细化解码器，该解码器将这些高级信号异步注入到时态主干中。这种机制提供了稳定和有效的语义指导，同时避免干扰低层次的时间动态。对标准长期预测基准的广泛实验表明，TimeSAF显著优于最先进的基线，并在Few-Shot和zero-shot传输设置中进一步表现出很强的泛化能力。
摘要：Despite the recent success of large language models (LLMs) in time-series forecasting, most existing methods still adopt a Deep Synchronous Fusion strategy, where dense interactions between textual and temporal features are enforced at every layer of the network. This design overlooks the inherent granularity mismatch between modalities and leads to what we term semantic perceptual dissonance: high-level abstract semantics provided by the LLM become inappropriately entangled with the low-level, fine-grained numerical dynamics of time series, making it difficult for semantic priors to effectively guide forecasting. To address this issue, we propose TimeSAF, a new framework based on hierarchical asynchronous fusion. Unlike synchronous approaches, TimeSAF explicitly decouples unimodal feature learning from cross-modal interaction. It introduces an independent cross-modal semantic fusion trunk, which uses learnable queries to aggregate global semantics from the temporal and prompt backbones in a bottom-up manner, and a stage-wise semantic refinement decoder that asynchronously injects these high-level signals back into the temporal backbone. This mechanism provides stable and efficient semantic guidance while avoiding interference with low-level temporal dynamics. Extensive experiments on standard long-term forecasting benchmarks show that TimeSAF significantly outperforms state-of-the-art baselines, and further exhibits strong generalization in both few-shot and zero-shot transfer settings.
【8】RPRA: Predicting an LLM-Judge for Efficient but Performant Inference
标题：RTRA：预测法学硕士法官的高效但有效的推理
链接：https://arxiv.org/abs/2604.12634
备注：10 pages in main text + 6 pages of references + 36 pages of appendices, 12 figures in main text + 37 figures in appendices, 2 tables in main text + 3 table in appendices, 13 prompts in appendices
摘要：大型语言模型（LLM）面临着计算效率（例如，参数数量）和输出质量，尤其是在部署在计算受限的设备（如电话或笔记本电脑）上时。解决这一挑战的一种方法是以人类为例，当模型认为自己无法解决问题时，让模型寻求帮助;我们可以通过允许较小的模型在他们认为自己可以提供良好响应时响应查询，并在他们不相信自己可以时服从较大的模型来克服这种权衡。为此，在本文中，我们研究了预测-回答/行动（PA）和推理-预测-推理-回答/行动（RPRA）范式的可行性，其中模型预测-在响应之前-LLM法官将如何对他们的输出进行评分。我们评估三种方法：zero-shot预测，预测使用的上下文报告卡，监督微调。我们的研究结果表明，较大的模型（特别是推理模型）在预测通用LLM判断zero-shot时表现良好，而较小的模型可以可靠地预测这样的判断，以及经过微调或提供一个上下文报告卡。总之，这两种方法都可以大大提高较小模型的预测准确性，报告卡和微调分别在数据集上实现了高达55%和52%的平均改进。这些发现表明，模型可以学习预测自己的性能限制，为更高效和自我意识的人工智能系统铺平道路。
摘要：Large language models (LLMs) face a fundamental trade-off between computational efficiency (e.g., number of parameters) and output quality, especially when deployed on computationally limited devices such as phones or laptops. One way to address this challenge is by following the example of humans and have models ask for help when they believe they are incapable of solving a problem on their own; we can overcome this trade-off by allowing smaller models to respond to queries when they believe they can provide good responses, and deferring to larger models when they do not believe they can. To this end, in this paper, we investigate the viability of Predict-Answer/Act (PA) and Reason-Predict-Reason-Answer/Act (RPRA) paradigms where models predict -- prior to responding -- how an LLM judge would score their output. We evaluate three approaches: zero-shot prediction, prediction using an in-context report card, and supervised fine-tuning. Our results show that larger models (particularly reasoning models) perform well when predicting generic LLM judges zero-shot, while smaller models can reliably predict such judges well after being fine-tuned or provided with an in-context report card. Altogether, both approaches can substantially improve the prediction accuracy of smaller models, with report cards and fine-tuning achieving mean improvements of up to 55% and 52% across datasets, respectively. These findings suggest that models can learn to predict their own performance limitations, paving the way for more efficient and self-aware AI systems.
【9】Calibration-Aware Policy Optimization for Reasoning LLMs
标题：推理LLM的校准感知策略优化
链接：https://arxiv.org/abs/2604.12632
备注：Published as a conference paper at ACL 2026
摘要：组相对策略优化（GRPO）增强了LLM推理，但通常会导致过度自信，其中不正确的响应比正确的响应产生更低的困惑，从而降低了曲线下面积（AUC）所描述的相对校准。现有的方法要么在校准方面产生有限的改进，要么牺牲推理准确性方面的增益。我们首先证明了GRPO式算法的这种退化源于其不确定性不可知的优势估计，这不可避免地使优化梯度与校准不一致。这导致以降级的校准为代价的改进的准确性。然后，我们提出了校准感知策略优化（CAPO）。它采用了逻辑AUC替代损失，理论上是一致的，并承认遗憾界，使不确定性意识的优势估计。通过进一步整合噪声掩蔽机制，CAPO实现了稳定的学习动态，共同优化了校准和准确性。在多个数学推理基准上的实验表明，CAPO-1.5B将校准显著提高了15%，同时实现了与GRPO相当或更好的精度，并将下游推理时间缩放任务的精度进一步提高了5%。此外，当允许在低置信度条件下弃权时，CAPO实现了帕累托最优精度-覆盖权衡，突出了其对幻觉缓解的实用价值。
摘要：Group Relative Policy Optimization (GRPO) enhances LLM reasoning but often induces overconfidence, where incorrect responses yield lower perplexity than correct ones, degrading relative calibration as described by the Area Under the Curve (AUC). Existing approaches either yield limited improvements in calibration or sacrifice gains in reasoning accuracy. We first prove that this degradation in GRPO-style algorithms stems from their uncertainty-agnostic advantage estimation, which inevitably misaligns optimization gradients with calibration. This leads to improved accuracy at the expense of degraded calibration. We then propose Calibration-Aware Policy Optimization (CAPO). It adopts a logistic AUC surrogate loss that is theoretically consistent and admits regret bound, enabling uncertainty-aware advantage estimation. By further incorporating a noise masking mechanism, CAPO achieves stable learning dynamics that jointly optimize calibration and accuracy. Experiments on multiple mathematical reasoning benchmarks show that CAPO-1.5B significantly improves calibration by up to 15% while achieving accuracy comparable to or better than GRPO, and further boosts accuracy on downstream inference-time scaling tasks by up to 5%. Moreover, when allowed to abstain under low-confidence conditions, CAPO achieves a Pareto-optimal precision-coverage trade-off, highlighting its practical value for hallucination mitigation.
【10】Agentic Control in Variational Language Models
标题：变分语言模型中的统计控制
链接：https://arxiv.org/abs/2604.12513
备注：20 pages, 8 figures
摘要：我们研究是否变的语言模型可以支持一个最小的和可测量的形式的代理控制扎根在自己的内部证据。我们的模型结合了局部变分隐藏计算（EVE），稳态潜伏调节器，结构感知检查点保留和校准的不确定性感知控制器上运行的保留模型。我们不是将不确定性视为预测后测量的被动诊断，而是将其视为可以调节训练，支持检查点保留和指导推理时间干预的操作信号。由此产生的框架是刻意集中。它研究内部控制的闭环形式，其中结构性和预测性信号变得可操作。从经验上讲，变分骨干改进了语言建模任务上的匹配确定性参考，同时还表现出更丰富和更可用的不确定性配置文件。在此基础之上，校准的控制器保持活跃，在全面的代理评估下使用多个动作，并产生积极的质量-成本权衡。这些结果支持了一个精确的说法：内部不确定性不仅可以作为变分语言模型的描述性属性，而且可以作为监管、检查点保留和最小代理路由的实用控制接口。
摘要：We study whether a variational language model can support a minimal and measurable form of agentic control grounded in its own internal evidence. Our model combines local variational hidden computation (EVE), a homeostatic latent regulator, structurally aware checkpoint retention and a calibrated uncertainty-aware controller operating on top of the retained model. Rather than treating uncertainty as a passive diagnostic measured after prediction, we treat it as an operational signal that can regulate training, support checkpoint retention and guide inference-time intervention. The resulting framework is deliberately focused. It studies a closed-loop form of internal control in which structural and predictive signals become actionable. Empirically, the variational backbone improves over a matched deterministic reference on the language-modeling task while also exhibiting a richer and more usable uncertainty profile. On top of this backbone, the calibrated controller remains active, uses multiple actions under a full agentic evaluation and yields a positive quality-cost trade-off. These results support a precise claim: internal uncertainty can serve not only as a descriptive property of a variational language model, but also as a practical control interface for regulation, checkpoint retention and minimal agentic routing.
【11】Adaptive Budget Allocation in LLM-Augmented Surveys
标题：LLM扩大调查中的适应性预算分配
链接：https://arxiv.org/abs/2604.12497
摘要：大型语言模型（LLM）可以以低成本生成调查响应，但其可靠性在不同问题之间存在很大差异，并且在数据收集之前是未知的。在调查中部署LLM仍然需要昂贵的人工响应来进行验证和校正。有限的人工标记预算应该如何实时分配给问题？我们提出了一个自适应分配算法，学习哪些问题是最难的LLM，同时收集人类的反应。每个人类标签都有双重作用：它提高了对该问题的估计，并揭示了LLM预测人类回答的效果。该算法将更多的预算用于LLM最不可靠的问题，而不需要任何关于问题级别LLM准确性的先验知识。我们证明了相对于最佳分配的分配差距随着预算的增长而消失，并在合成数据和真实调查数据集上验证了该方法，其中包含68个问题和2000多名受访者。在真实的调查数据上，在问题之间均匀分配人类标签的标准做法相对于最优方案浪费了10- 12%的预算;我们的算法将这种浪费减少到2- 6%，并且随着问题在LLM预测质量中变得更加异质，优势会增加。该算法实现了相同的估计质量与传统的均匀抽样较少的人的样本，不需要试点研究，并支持正式的性能保证验证真实的调查数据。更广泛地说，该框架适用于任何必须在LLM可靠性未知的任务中分配稀缺的人力监督的情况。
摘要：Large language models (LLMs) can generate survey responses at low cost, but their reliability varies substantially across questions and is unknown before data collection. Deploying LLMs in surveys still requires costly human responses for verification and correction. How should a limited human-labeling budget be allocated across questions in real time? We propose an adaptive allocation algorithm that learns which questions are hardest for the LLM while simultaneously collecting human responses. Each human label serves a dual role: it improves the estimate for that question and reveals how well the LLM predicts human responses on it. The algorithm directs more budget to questions where the LLM is least reliable, without requiring any prior knowledge of question-level LLM accuracy. We prove that the allocation gap relative to the best possible allocation vanishes as the budget grows, and validate the approach on both synthetic data and a real survey dataset with 68 questions and over 2000 respondents. On real survey data, the standard practice of allocating human labels uniformly across questions wastes 10--12% of the budget relative to the optimal; our algorithm reduces this waste to 2--6%, and the advantage grows as questions become more heterogeneous in LLM prediction quality. The algorithm achieves the same estimation quality as traditional uniform sampling with fewer human samples, requires no pilot study, and is backed by formal performance guarantees validated on real survey data. More broadly, the framework applies whenever scarce human oversight must be allocated across tasks where LLM reliability is unknown.
【12】Analyzing the Effect of Noise in LLM Fine-tuning
标题：LLM微调中噪音的影响分析
链接：https://arxiv.org/abs/2604.12469
摘要：微调是使预训练的大型语言模型（LLM）适应下游NLP任务的主要范式。在实践中，微调数据集可能包含由注释错误、预处理工件或自动数据收集产生的各种形式的噪音。虽然以前的工作集中在设计鲁棒的学习算法，以减轻噪声条件下的性能下降，相对较少的是知道不同类型的噪声如何影响LLM的内部学习动态微调。在这项工作中，我们系统地研究了噪声对三个预训练模型家族（GPT-2，Qwen 2和Llama-2）和三个不同NLP任务的模型行为的影响。我们介绍了三种常见的现实世界中的噪音类型：标签噪音，语法噪音和排版噪音对应的控制扰动。除了任务级性能，我们还分析了逐层表示变化和注意力模式，以了解噪声如何通过网络传播。我们的研究结果表明，损坏的标签（即标签噪声）始终会导致最大的性能下降，而语法噪声和排版噪声偶尔会产生轻微的正则化好处。我们进一步发现，噪声效应主要局限于特定的任务层，而注意力结构保持相对稳定。
摘要：Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may contain various forms of noise arising from annotation errors, preprocessing artifacts, or automated data collection. While prior work has focused on designing robust learning algorithms to mitigate performance degradation under noisy conditions, comparatively little is known about how different types of noise affect the internal learning dynamics of LLMs during fine-tuning. In this work, we systematically study the impact of noise on model behavior across three pretrained model families (GPT-2, Qwen2 and Llama-2) and three diverse NLP tasks. We introduce controlled perturbations corresponding to three common real-world noise types: label noise, grammatical noise, and typographical noise. Beyond task-level performance, we analyze layer-wise representation changes and attention patterns to understand how noise propagates through the network. Our results show that corrupting labels (i.e. label noise) consistently causes the largest performance degradation, whereas grammatical noise and typographical noise can occasionally yield mild regularization benefits. We further find that noise effects are localized primarily to task-specific layers, while attention structures remain comparatively stable.
【13】Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
标题：超越输出正确性：对编码任务中的大型语言模型推理进行基准测试和评估
链接：https://arxiv.org/abs/2604.12379
摘要：大型语言模型（LLM）越来越依赖显式推理来解决编码任务，但评估这种推理的质量仍然具有挑战性。现有的推理评估器不是为编码而设计的，当前的基准测试主要集中在代码生成上，而其他编码任务基本上没有被探索。我们介绍了CodeRQ-Bench，这是第一个用于评估LLM推理质量的基准测试，涉及三个编码任务类别：生成，摘要和分类。使用这个基准，我们分析了1,069个不匹配的情况下，从现有的评估，确定五个经常性的限制，并得出四个设计的见解，在编码任务的推理评估。在这些见解的指导下，我们提出了VERA，一个两阶段的评估，结合证据为基础的验证与歧义意识的分数校正。CodeRQ-Bench上的实验表明，VERA在四个数据集上的表现始终优于强基线，AUCROC提高了0.26，AUPRC提高了0.21。我们在https：//github.com/MrLYG/CodeRQ-Bench上发布了CodeRQ-Bench，以支持未来的研究。
摘要 ：Large language models (LLMs) increasingly rely on explicit reasoning to solve coding tasks, yet evaluating the quality of this reasoning remains challenging. Existing reasoning evaluators are not designed for coding, and current benchmarks focus primarily on code generation, leaving other coding tasks largely unexplored. We introduce CodeRQ-Bench, the first benchmark for evaluating LLM reasoning quality across three coding task categories: generation, summarization, and classification. Using this benchmark, we analyze 1,069 mismatch cases from existing evaluators, identify five recurring limitations, and derive four design insights for reasoning evaluation in coding tasks. Guided by these insights, we propose VERA, a two-stage evaluator that combines evidence-grounded verification with ambiguity-aware score correction. Experiments on CodeRQ-Bench show that VERA consistently outperforms strong baselines across four datasets, improving AUCROC by up to 0.26 and AUPRC by up to 0.21. We release CodeRQ-Bench at https://github.com/MrLYG/CodeRQ-Bench, supporting future investigations.
【14】Scaffold-Conditioned Preference Triplets for Controllable Molecular Optimization with Large Language Models
标题：具有大型语言模型的可控分子优化的支架条件偏好三重组
链接：https://arxiv.org/abs/2604.12350
摘要：分子性质优化是药物发现的核心，但许多深度学习方法依赖于黑盒评分，对支架保存的控制有限，通常会产生不稳定或生物学上不可信的编辑。虽然大型语言模型（LLM）是有前途的分子生成器，但优化仍然受到缺乏化学基础偏好监督和原则性数据管理的限制。我们引入了\textbf{Scaffold-Conditioned Preference Triplets（SCPT）}，这是一个管道，通过支架对齐和化学驱动的过滤器构建相似性约束的三元组$\langle\text{scaffold}，\text {better}，\text{worse}\rangle$，以获得有效性，可合成性和有意义的属性增益。使用这些偏好，我们将预先训练的分子LLM作为条件编辑器进行对齐，从而实现保留支架的属性改进编辑。在单目标和多目标基准测试中，SCPT提高了优化成功率和性能收益，同时保持了比竞争基线更高的支架相似性。与代表性的非LLM分子优化方法相比，SCPT训练的LLM更适合于支架约束和多目标优化。此外，在单属性和双属性监督上训练的模型有效地推广到三属性任务，表明在有限的高阶监督下有希望的外推推广。SCPT还提供了可控的数据构造旋钮，产生可预测的相似性增益边界，使系统适应不同的优化制度。
摘要：Molecular property optimization is central to drug discovery, yet many deep learning methods rely on black-box scoring and offer limited control over scaffold preservation, often producing unstable or biologically implausible edits. While large language models (LLMs) are promising molecular generators, optimization remains constrained by the lack of chemistry-grounded preference supervision and principled data curation. We introduce \textbf{Scaffold-Conditioned Preference Triplets (SCPT)}, a pipeline that constructs similarity-constrained triplets $\langle\text{scaffold}, \text{better}, \text{worse}\rangle$ via scaffold alignment and chemistry-driven filters for validity, synthesizability, and meaningful property gains. Using these preferences, we align a pretrained molecular LLM as a conditional editor, enabling property-improving edits that retain the scaffold. Across single- and multi-objective benchmarks, SCPT improves optimization success and property gains while maintaining higher scaffold similarity than competitive baselines. Compared with representative non-LLM molecular optimization methods, SCPT-trained LLMs are better suited to scaffold-constrained and multi-objective optimization. In addition, models trained on single-property and two-property supervision generalize effectively to three-property tasks, indicating promising extrapolative generalization under limited higher-order supervision. SCPT also provides controllable data-construction knobs that yield a predictable similarity-gain frontier, enabling systematic adaptation to diverse optimization regimes.
【15】LLM-Guided Semantic Bootstrapping for Interpretable Text Classification with Tsetlin Machines
标题：使用Tsetlin机进行可解释文本分类的LLM引导语义引导
链接：https://arxiv.org/abs/2604.12223
备注：Accepted to Findings of the Association for Computational Linguistics (ACL 2026)
摘要：像BERT这样的预训练语言模型（PLM）提供了强大的语义表示，但成本高且不透明，而像Tsetlin Machine（TM）这样的符号模型提供了透明度，但缺乏语义概括。我们提出了一个语义自举框架，将LLM知识转换为符号形式，将可解释性与语义能力相结合。给定一个类标签，LLM生成子意图，通过三阶段课程（种子，核心，丰富）指导合成数据创建，扩展语义多样性。非否定TM（NTM）从这些示例中学习，以提取高置信度的文字作为可解释的语义线索。将这些线索注入到真实数据中使TM能够将子句逻辑与LLM推断的语义对齐。我们的方法不需要嵌入或运行时LLM调用，但配备符号模型与预先训练的语义先验。在多个文本分类任务中，它比vanilla TM提高了可解释性和准确性，实现了与BERT相当的性能，同时保持了完全的符号和效率。
摘要：Pretrained language models (PLMs) like BERT provide strong semantic representations but are costly and opaque, while symbolic models such as the Tsetlin Machine (TM) offer transparency but lack semantic generalization. We propose a semantic bootstrapping framework that transfers LLM knowledge into symbolic form, combining interpretability with semantic capacity. Given a class label, an LLM generates sub-intents that guide synthetic data creation through a three-stage curriculum (seed, core, enriched), expanding semantic diversity. A Non-Negated TM (NTM) learns from these examples to extract high-confidence literals as interpretable semantic cues. Injecting these cues into real data enables a TM to align clause logic with LLM-inferred semantics. Our method requires no embeddings or runtime LLM calls, yet equips symbolic models with pretrained semantic priors. Across multiple text classification tasks, it improves interpretability and accuracy over vanilla TM, achieving performance comparable to BERT while remaining fully symbolic and efficient.
【16】LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics
标题：LLM增强型日志异常检测：自动系统诊断大型语言模型的综合基准
链接：https://arxiv.org/abs/2604.12218
备注：5 pages, 4 tables, code available at https://github.com/disha8611/llm-log-anomaly-benchmark
摘要：系统日志异常检测对于维护大型软件系统的可靠性至关重要，然而传统方法难以应对现代日志数据的异构性和不断变化的性质。大语言模型（LLM）的最新进展为日志理解提供了有前途的新方法，但仍然缺乏基于LLM的方法与现有技术的系统比较。在本文中，我们提出了一个全面的基准研究，评估了基于LLM和传统方法的日志异常检测，包括四个广泛使用的公共数据集：HDFS，BGL，Thunderbird和Spirit。我们评估了三种方法：（1）经典的日志解析器（Drain，Spell，AEL）与机器学习分类器相结合，（2）微调的Transformer模型（BERT，RoBERTa），以及（3）zero-shot和Few-Shot设置中基于LLM的方法（GPT-3.5，GPT-4，LLaMA-3）。我们的实验表明，虽然微调Transformers实现了最高的F1分数（0.96-0.99），但基于LLM的LLM表现出了可扩展的零射击能力（F1：0.82-0.91），而不需要任何标记的训练数据-这对于标记异常很少的现实世界部署来说是一个显着的优势。我们进一步分析了每种方法的成本-精度权衡、延迟特性和故障模式。我们的研究结果提供了可操作的指导方针，从业者选择日志异常检测方法的基础上，他们的具体限制的准确性，延迟，成本和标签的可用性。所有代码和实验配置都是公开的，以促进可重复性。
摘要 ：System log anomaly detection is critical for maintaining the reliability of large-scale software systems, yet traditional methods struggle with the heterogeneous and evolving nature of modern log data. Recent advances in Large Language Models (LLMs) offer promising new approaches to log understanding, but a systematic comparison of LLM-based methods against established techniques remains lacking. In this paper, we present a comprehensive benchmark study evaluating both LLM-based and traditional approaches for log anomaly detection across four widely-used public datasets: HDFS, BGL, Thunderbird, and Spirit. We evaluate three categories of methods: (1) classical log parsers (Drain, Spell, AEL) combined with machine learning classifiers, (2) fine-tuned transformer models (BERT, RoBERTa), and (3) prompt-based LLM approaches (GPT-3.5, GPT-4, LLaMA-3) in zero-shot and few-shot settings. Our experiments reveal that while fine-tuned transformers achieve the highest F1-scores (0.96-0.99), prompt-based LLMs demonstrate remarkablezero-shot capabilities (F1: 0.82-0.91) without requiring any labeled training data -- a significant advantage for real-world deployment where labeled anomalies are scarce. We further analyze the cost-accuracy trade-offs, latency characteristics, and failure modes of each approach. Our findings provide actionable guidelines for practitioners choosing log anomaly detection methods based on their specific constraints regarding accuracy, latency, cost, and label availability. All code and experimental configurations are publicly available to facilitate reproducibility.
【17】Policy-Invisible Violations in LLM-Based Agents
标题：LLM代理人的政策隐形违规行为
链接：https://arxiv.org/abs/2604.12177
备注：26 pages,1 figure, 11 tables
摘要：基于LLM的代理可以执行语法有效、用户认可和语义适当的操作，但仍然违反组织策略，因为正确的策略判断所需的事实在决策时被隐藏。我们称这种故障模式为策略不可见的违规：合规性取决于实体属性，上下文状态或代理可见上下文中缺少的会话历史的情况。我们提出了一个跨八个违规类别的基准，平衡违规和安全控制的情况下，所有工具的响应包含干净的业务数据，没有策略元数据。我们手动审查五个前沿模型产生的所有600个模型轨迹，并使用人工审查的轨迹标签对其进行评估。手动审查相对于原始案例级注释更改了32个标签（5.3%），确认了跟踪级人工审查的必要性。为了证明在有利的条件下，基于世界国家的执法可以实现什么，我们介绍了哨兵，一个基于反事实图模拟的执法框架。Sentinel将每个Agent动作视为组织知识图的拟议突变，执行推测执行以实现动作后的世界状态，并验证图结构不变量以决定允许/阻止/澄清。与人工审查的痕迹标签相比，Sentinel在保持高精度的同时大大优于仅内容DLP基线（68.8%对93.0%的准确率），尽管在某些违规类别上仍有改进的空间。这些结果表明，一旦与政策相关的世界状态提供给执行层，就可以实现什么。
摘要：LLM-based agents can execute actions that are syntactically valid, user-sanctioned, and semantically appropriate, yet still violate organizational policy because the facts needed for correct policy judgment are hidden at decision time. We call this failure mode policy-invisible violations: cases in which compliance depends on entity attributes, contextual state, or session history absent from the agent's visible context. We present PhantomPolicy, a benchmark spanning eight violation categories with balanced violation and safe-control cases, in which all tool responses contain clean business data without policy metadata. We manually review all 600 model traces produced by five frontier models and evaluate them using human-reviewed trace labels. Manual review changes 32 labels (5.3%) relative to the original case-level annotations, confirming the need for trace-level human review. To demonstrate what world-state-grounded enforcement can achieve under favorable conditions, we introduce Sentinel, an enforcement framework based on counterfactual graph simulation. Sentinel treats every agent action as a proposed mutation to an organizational knowledge graph, performs speculative execution to materialize the post-action world state, and verifies graph-structural invariants to decide Allow/Block/Clarify. Against human-reviewed trace labels, Sentinel substantially outperforms a content-only DLP baseline (68.8% vs. 93.0% accuracy) while maintaining high precision, though it still leaves room for improvement on certain violation categories. These results demonstrate what becomes achievable once policy-relevant world state is made available to the enforcement layer.
【18】PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving
标题：PipeLive：高效现场现场管道并行主义重新配置，以实现动态LLM服务
链接：https://arxiv.org/abs/2604.12171
摘要：流水线并行（PP）被广泛用于跨GPU划分大型语言模型（LLM）的层，从而实现大型模型的可扩展推理。然而，现有系统依赖于静态PP配置，无法适应动态设置，例如无服务器平台和异构GPU环境。通过停止和重新部署服务来重新配置PP会导致令人望而却步的停机时间，因此重新配置必须在不中断推理的情况下实时进行。然而，现场PP重新配置从根本上具有挑战性。GPU已经被模型权重和KV缓存饱和，几乎没有为新层放置留下空间，并且需要KV缓存重新分配，这与像vLLM这样预先分配吞吐量的系统不一致。此外，在执行期间保持KV一致性是困难的：停止和复制引入了大的停顿，而后台同步则会随着状态的演变而冒着不一致的风险。我们提出PipeLive，它使现场PP重新配置，以最小的中断。PipeLive引入了重新设计的KV缓存布局以及对PageAttention的共同设计的扩展，形成了实时KV缓存的统一机制。它还采用了受实时虚拟机迁移启发的增量KV修补机制，以同步源和目标配置之间的KV状态并识别安全切换点。与禁用KV缓存相比，PipeLive在没有KV缓存溢出的情况下实现了2.5倍的第一个令牌（TTFT）时间缩短。此外，与没有KV修补的变体相比，它将重新配置开销从几秒减少到10 ms以下，并将TTFT和每输出令牌时间（TPOT）分别提高了54.7%和14.7%。
摘要：Pipeline parallelism (PP) is widely used to partition layers of large language models (LLMs) across GPUs, enabling scalable inference for large models. However, existing systems rely on static PP configurations that fail to adapt to dynamic settings, such as serverless platforms and heterogeneous GPU environments. Reconfiguring PP by stopping and redeploying service incurs prohibitive downtime, so reconfiguration must instead proceed live and in place, without interrupting inference. However, live in-place PP reconfiguration is fundamentally challenging. GPUs are already saturated with model weights and KV cache, leaving little room for new layer placements and necessitating KV cache resizing, at odds with systems like vLLM that preallocate for throughput. Moreover, maintaining KV consistency during execution is difficult: stop-and-copy introduces large pauses, while background synchronization risks inconsistency as states evolve. We present PipeLive, which enables live in-place PP reconfiguration with minimal disruption. PipeLive introduces a redesigned KV cache layout together with a co-designed extension to PageAttention, forming a unified mechanism for live KV resizing. It further adopts an incremental KV patching mechanism, inspired by live virtual machine migration, to synchronize KV states between source and target configurations and identify a safe switch point. PipeLive achieves a 2.5X reduction in time-to-first-token (TTFT) without KV cache overflow compared to disabling KV resizing. Furthermore, compared to a variant without KV patching, it reduces reconfiguration overhead from seconds to under 10ms, and improves TTFT and time-per-output-token (TPOT) by up to 54.7% and 14.7%, respectively.
【19】Beyond Perception Errors: Semantic Fixation in Large Vision-Language Models
标题：超越感知错误：大型视觉语言模型中的语义固定
链接：https://arxiv.org/abs/2604.12119
摘要：大型的视觉语言模型（VLM）通常依赖于熟悉的语义先验，但现有的评估并不能将感知故障与规则映射故障清晰地分开。我们研究这种行为作为语义固定：保留默认的解释，即使当提示指定一个替代的，同样有效的映射。为了隔离这种影响，我们引入VLM-Fix，一个控制基准超过四个抽象的策略游戏，评估相同的终端板状态下配对的标准和逆规则配方。在14个开放和封闭的VLM中，准确性始终倾向于标准规则，揭示了一个强大的语义固定差距。即时干预支持这一机制：中性别名提示大大缩小了反向规则的差距，而语义加载的别名重新打开它。后训练是强烈的规则对齐：一个规则的训练提高了相同的规则迁移，但伤害了相反的规则迁移，而联合规则训练提高了更广泛的迁移。为了测试合成游戏之外的外部有效性，我们评估了VLMBias上类似的陌生化干预，并观察到相同的定性模式。最后，后期层激活转向部分恢复退化的性能，表明语义固定错误至少部分可编辑后期表示。项目页面、代码和数据集可在https://maveryn.github.io/vlm-fix/获得。
摘要 ：Large vision-language models (VLMs) often rely on familiar semantic priors, but existing evaluations do not cleanly separate perception failures from rule-mapping failures. We study this behavior as semantic fixation: preserving a default interpretation even when the prompt specifies an alternative, equally valid mapping. To isolate this effect, we introduce VLM-Fix, a controlled benchmark over four abstract strategy games that evaluates identical terminal board states under paired standard and inverse rule formulations. Across 14 open and closed VLMs, accuracy consistently favors standard rules, revealing a robust semantic-fixation gap. Prompt interventions support this mechanism: neutral alias prompts substantially narrow the inverse-rule gap, while semantically loaded aliases reopen it. Post-training is strongly rule-aligned: training on one rule improves same-rule transfer but hurts opposite-rule transfer, while joint-rule training improves broader transfer. To test external validity beyond synthetic games, we evaluate analogous defamiliarization interventions on VLMBias and observe the same qualitative pattern. Finally, late-layer activation steering partially recovers degraded performance, indicating that semantic-fixation errors are at least partly editable in late representations. Project page, code, and dataset available at https://maveryn.github.io/vlm-fix/.
【20】LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
标题：LoSA：面向块扩散语言模型的局部感知稀疏注意
链接：https://arxiv.org/abs/2604.12056
备注：16 pages, 11 figures, 6 tables
摘要：逐块扩散语言模型（DLM）以任何顺序生成多个令牌，为自回归解码管道提供了一种有前途的替代方案。然而，在长情境情景中，它们仍然受到记忆约束注意力的制约。由于KV膨胀问题，朴素稀疏注意力在DLM上失败，其中不同的查询选择不同的前缀位置，使得所访问的KV页面的联合很大。为了解决这个问题，我们观察到，在连续的去噪步骤之间，只有一小部分活动令牌表现出显着的隐藏状态变化，而大多数稳定令牌几乎保持不变。基于这一见解，我们提出了LOSA（本地感知稀疏注意），它重用缓存的前缀注意结果的稳定令牌和稀疏注意只适用于活跃的令牌。这大大减少了必须加载的KV索引的数量，从而产生更高的加速比和更高的准确性。在多个块式DLM和基准测试中，LOSA保持了接近密集的准确性，同时显著提高了效率，在积极的稀疏水平下实现了高达+9点的平均准确性，同时保持了1.54倍的低注意力密度。在RTX A6000 GPU上实现了高达4.14倍的注意力加速，证明了该方法的有效性。
摘要：Block-wise diffusion language models (DLMs) generate multiple tokens in any order, offering a promising alternative to the autoregressive decoding pipeline. However, they still remain bottlenecked by memory-bound attention in long-context scenarios. Naive sparse attention fails on DLMs due to a KV Inflation problem, where different queries select different prefix positions, making the union of accessed KV pages large. To address this, we observe that between consecutive denoising steps, only a small fraction of active tokens exhibit significant hidden-state changes, while the majority of stable tokens remain nearly constant. Based on this insight, we propose LOSA (Locality-aware Sparse Attention), which reuses cached prefix-attention results for stable tokens and applies sparse attention only to active tokens. This substantially shrinks the number of KV indices that must be loaded, yielding both higher speedup and higher accuracy. Across multiple block-wise DLMs and benchmarks, LOSA preserves near-dense accuracy while significantly improving efficiency, achieving up to +9 points in average accuracy at aggressive sparsity levels while maintaining 1.54x lower attention density. It also achieves up to 4.14x attention speedup on RTX A6000 GPUs, demonstrating the effectiveness of the proposed method.
【21】Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space
标题：作为吸引子的身份：LLM激活空间中持久Agent结构的几何证据
链接：https://arxiv.org/abs/2604.12016
备注：16 pages, 5 figures. Code and data: https://github.com/b102e/yar-attractor-experiment
摘要：大型语言模型将语义相关的提示映射到类似的内部表示-这种现象可以解释为吸引子样的动态。我们问一个持久的认知代理（其认知核心）的身份文件是否表现出类似的吸引子样的行为。我们提出了一个控制实验，对美洲驼3.1 8B指令，比较隐藏状态的原始认知核心（条件A），7个释义（条件B），和7个结构匹配的控制（条件C）。第8层、第16层和第24层的平均值合并状态显示，与对照组相比，释义收敛到更紧密的聚类（科恩d > 1.88，p < 10^{-27}，邦弗朗尼校正）。Gemma 2 9 B上的复制确认了跨架构的通用性。烧蚀表明，效果主要是语义，而不是结构，结构的完整性似乎是必要的，以达到吸引区。一个探索性的实验表明，阅读对代理人的科学描述会使内部状态向吸引子转移--比假预印本更接近吸引子--从而区分对身份的了解和作为身份的操作。这些结果提供了代表性的证据，代理身份文件诱导吸引子样的几何形状在LLM激活空间。
摘要：Large language models map semantically related prompts to similar internal representations -- a phenomenon interpretable as attractor-like dynamics. We ask whether the identity document of a persistent cognitive agent (its cognitive_core) exhibits analogous attractor-like behavior. We present a controlled experiment on Llama 3.1 8B Instruct, comparing hidden states of an original cognitive_core (Condition A), seven paraphrases (Condition B), and seven structurally matched controls (Condition C). Mean-pooled states at layers 8, 16, and 24 show that paraphrases converge to a tighter cluster than controls (Cohen's d > 1.88, p < 10^{-27}, Bonferroni-corrected). Replication on Gemma 2 9B confirms cross-architecture generalizability. Ablations suggest the effect is primarily semantic rather than structural, and that structural completeness appears necessary to reach the attractor region. An exploratory experiment shows that reading a scientific description of the agent shifts internal state toward the attractor -- closer than a sham preprint -- distinguishing knowing about an identity from operating as that identity. These results provide representational evidence that agent identity documents induce attractor-like geometry in LLM activation space.
【22】AutoSurrogate: An LLM-Driven Multi-Agent Framework for Autonomous Construction of Deep Learning Surrogate Models in Subsurface Flow
标题：AutoSurrogate：一个LLM驱动的多Agent框架，用于在地下流中自主构建深度学习替代模型
链接：https://arxiv.org/abs/2604.11945
摘要：地下水流的高保真数值模拟是计算密集型的，特别是对于许多查询任务，如不确定性量化和数据同化。深度学习（DL）代理可以显著加速正向模拟，但构建它们需要大量的机器学习（ML）专业知识-从架构设计到超参数调整-大多数领域科学家都不具备。此外，该过程主要是手动的，并且严重依赖于启发式选择。这种专业知识的差距仍然是更广泛地采用DL替代技术的关键障碍。出于这个原因，我们提出了AutoSurrogate，这是一个大型语言模型驱动的多代理框架，使没有ML专业知识的从业者能够通过自然语言指令为地下水流问题构建高质量的代理。给定模拟数据和可选的偏好，四个专门的代理协作执行数据分析，从模型动物园中选择架构，贝叶斯超参数优化，模型训练和针对用户指定阈值的质量评估。该系统还可以自主处理常见的故障模式，包括在发生数值不稳定时重新启动调整配置的训练，以及在预测精度低于目标时切换到替代架构。在我们的环境中，一个自然语言句子就足以生成一个部署就绪的代理模型，在任何中间阶段都需要最少的人为干预。我们演示了AutoSurrogate在3D地质碳储量建模任务中的实用性，将渗透率场映射到31个时间步长内的压力和CO2饱和度场。在没有任何手动调优的情况下，AutoSurrogate能够超越专家设计的基线和与领域无关的AutoML方法，展示了实际部署的强大潜力。
摘要：High-fidelity numerical simulation of subsurface flow is computationally intensive, especially for many-query tasks such as uncertainty quantification and data assimilation. Deep learning (DL) surrogates can significantly accelerate forward simulations, yet constructing them requires substantial machine learning (ML) expertise - from architecture design to hyperparameter tuning - that most domain scientists do not possess. Furthermore, the process is predominantly manual and relies heavily on heuristic choices. This expertise gap remains a key barrier to the broader adoption of DL surrogate techniques. For this reason, we present AutoSurrogate, a large-language-model-driven multi-agent framework that enables practitioners without ML expertise to build high-quality surrogates for subsurface flow problems through natural-language instructions. Given simulation data and optional preferences, four specialized agents collaboratively execute data profiling, architecture selection from a model zoo, Bayesian hyperparameter optimization, model training, and quality assessment against user-specified thresholds. The system also handles common failure modes autonomously, including restarting training with adjusted configurations when numerical instabilities occur and switching to alternative architectures when predictive accuracy falls short of targets. In our setting, a single natural-language sentence can be sufficient to produce a deployment-ready surrogate model, with minimum human intervention required at any intermediate stage. We demonstrate the utility of AutoSurrogate on a 3D geological carbon storage modeling task, mapping permeability fields to pressure and CO$_2$ saturation fields over 31 timesteps. Without any manual tuning, AutoSurrogate is able to outperform expert-designed baselines and domain-agnostic AutoML methods, demonstrating strong potential for practical deployment.
【23】ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems
标题：ProbeLogits：用于AI原生操作系统的内核级LLM推理原语
链接：https://arxiv.org/abs/2604.11943
备注：13 pages, 9 tables
摘要：在内部运行LLM推理的操作系统内核可以在生成任何文本之前读取logit分布-并将其作为治理原语。我提出了ProbeLogits，这是一个内核级操作，它执行一次向前传递，并读取特定的令牌日志，以将代理操作分类为安全或危险，并且没有学习到的参数。在一个260提示的操作系统动作基准测试（包括对抗性攻击在内的9个类别）中，ProbeLogits使用4位量化的通用7 B模型实现了F1=0.980，精度=1.000，召回率=0.960。在ToxicChat（1,000个人类注释的真实对话）上，它在默认校准强度$α$=1.0时达到F1=0.790，在$α$=0.5时提高到F1=0.837--是Llama Guard 3的F1~0.939的89%，零学习参数。一个关键的设计贡献是校准强度$α$，它作为部署时的策略旋钮，而不是学习的超参数。通过调整$α$，操作系统可以对特权操作执行严格的策略（$α\geq 0.8$，最大化召回率）或对会话代理执行宽松的策略（$α =0.5，最大化精度）。上下文校准将自定义基准测试的准确率从64.8%提高到97.3%。我在Anima OS中实现了ProbeLogits，这是一个用80，400行Rust编写的x86_64操作系统。由于代理操作必须通过内核介导的主机函数，ProbeLogits实施在WASM沙箱边界以下运行，因此比应用程序层分类器更难规避。每个分类在7 B上花费65毫秒-对于每个动作的治理来说足够快。我还展示了将KV缓存视为进程状态可以实现类似于传统进程管理的检查点、恢复和分叉操作。据我所知，以前没有系统将LLM logit向量公开为操作系统级治理原语。
摘要：An OS kernel that runs LLM inference internally can read logit distributions before any text is generated -- and act on them as a governance primitive. I present ProbeLogits, a kernel-level operation that performs a single forward pass and reads specific token logits to classify agent actions as safe or dangerous, with zero learned parameters. On a 260-prompt OS action benchmark (9 categories including adversarial attacks), ProbeLogits achieves F1=0.980, Precision=1.000, and Recall=0.960 using a general-purpose 7B model at 4-bit quantization. On ToxicChat (1,000 human-annotated real conversations), it achieves F1=0.790 at default calibration strength $α$=1.0, improving to F1=0.837 at $α$=0.5 -- 89% of Llama Guard 3's F1~0.939 with zero learned parameters. A key design contribution is the calibration strength $α$, which serves as a deployment-time policy knob rather than a learned hyperparameter. By adjusting $α$, the OS can enforce strict policies for privileged operations ($α\geq 0.8$, maximizing recall) or relaxed policies for conversational agents ($α$=0.5, maximizing precision). Contextual calibration improves accuracy from 64.8% to 97.3% on the custom benchmark. I implement ProbeLogits within Anima OS, a bare-metal x86_64 OS written in 80,400 lines of Rust. Because agent actions must pass through kernel-mediated host functions, ProbeLogits enforcement operates below the WASM sandbox boundary, making it significantly harder to circumvent than application-layer classifiers. Each classification costs 65ms on 7B -- fast enough for per-action governance. I also show that treating KV cache as process state enables checkpoint, restore, and fork operations analogous to traditional process management. To my knowledge, no prior system exposes LLM logit vectors as OS-level governance primitives.
【24】When Reasoning Models Hurt Behavioral Simulation: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation
标题：当推理模型损害行为模拟时：多智能体LLM谈判中的求解器-采样器不匹配
链接：https://arxiv.org/abs/2604.11840
备注：12 pages, 5 figures, supplementary material included as ancillary file
摘要：大型语言模型越来越多地被用作社会，经济和政策模拟中的代理。一个常见的假设是，更强的推理应该提高模拟的保真度。我们认为，当目标不是解决一个战略问题，而是抽样可信的有限理性行为时，这种假设可能会失败。在这种环境中，推理增强的模型可能会成为更好的求解器，也可能成为更差的模拟器：它们可能会过度优化战略主导行动，崩溃以妥协为导向的终端行为，有时还会表现出多样性而无保真度的模式，其中局部变化在没有结果的情况下生存水平的保真度。我们研究了这种求解器采样器不匹配的多代理谈判环境适应早期的模拟工作：一个模糊的碎片化的权力交易限制的情况下，一个模糊的统一反对交易限制的情况下，和一个新的域电网限电的情况下，在紧急电力管理。我们比较了两个主要模型家族中的三个反射条件，无反射，有界反射和本地推理，然后将相同的协议扩展到使用GPT-4.1和GPT-5.2直接运行OpenAI。在所有三个实验中，有界反射产生了比没有反射或本地推理更多样化和妥协导向的轨迹。在直接OpenAI扩展中，GPT-5.2原生在三个实验的45次运行中有45次以权威决策结束，而GPT-5.2在每个环境中都有界恢复妥协结果。这一贡献并不是主张推理通常是有害的。这是一个方法上的警告：模型能力和模拟保真度是不同的目标，行为模拟应该使模型成为采样器，而不仅仅是求解器。
摘要：Large language models are increasingly used as agents in social, economic, and policy simulations. A common assumption is that stronger reasoning should improve simulation fidelity. We argue that this assumption can fail when the objective is not to solve a strategic problem, but to sample plausible boundedly rational behavior. In such settings, reasoning-enhanced models can become better solvers and worse simulators: they can over-optimize for strategically dominant actions, collapse compromise-oriented terminal behavior, and sometimes exhibit a diversity-without-fidelity pattern in which local variation survives without outcome-level fidelity. We study this solver-sampler mismatch in three multi-agent negotiation environments adapted from earlier simulation work: an ambiguous fragmented-authority trading-limits scenario, an ambiguous unified-opposition trading-limits scenario, and a new-domain grid-curtailment case in emergency electricity management. We compare three reflection conditions, no reflection, bounded reflection, and native reasoning, across two primary model families and then extend the same protocol to direct OpenAI runs with GPT-4.1 and GPT-5.2. Across all three experiments, bounded reflection produces substantially more diverse and compromise-oriented trajectories than either no reflection or native reasoning. In the direct OpenAI extension, GPT-5.2 native ends in authority decisions in 45 of 45 runs across the three experiments, while GPT-5.2 bounded recovers compromise outcomes in every environment. The contribution is not a claim that reasoning is generally harmful. It is a methodological warning: model capability and simulation fidelity are different objectives, and behavioral simulation should qualify models as samplers, not only as solvers.
【25】Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning
标题：使用LLM进行方案自适应表格表示学习，用于可推广多模式临床推理
链接：https://arxiv.org/abs/2604.11835
备注：11 pages, 4 figures
摘要：表格数据的机器学习仍然受到模式泛化能力差的限制，这一挑战源于缺乏对结构化变量的语义理解。这一挑战在临床医学等领域尤为严峻，因为电子健康记录（EHR）模式差异很大。为了解决这个问题，我们提出了模式自适应表格表示学习，这是一种利用大型语言模型（LLM）来创建可转移表格嵌入的新方法。通过将结构化变量转换为语义自然语言语句，并使用预训练的LLM对其进行编码，我们的方法可以在没有手动特征工程或再训练的情况下，在看不见的模式中实现zero-shot对齐。我们将我们的编码器集成到痴呆症诊断的多模式框架中，结合表格和MRI数据。NACC和ADNI数据集上的实验证明了最先进的性能和成功的zero-shot转移到看不见的模式，在回顾性诊断任务中显著优于临床基线，包括委员会认证的神经学家。这些结果验证了我们的LLM驱动的方法作为一个可扩展的，强大的解决方案，异构的现实世界的数据，提供了一种途径，以扩展基于LLM的推理结构化领域。
摘要：Machine learning for tabular data remains constrained by poor schema generalization, a challenge rooted in the lack of semantic understanding of structured variables. This challenge is particularly acute in domains like clinical medicine, where electronic health record (EHR) schemas vary significantly. To solve this problem, we propose Schema-Adaptive Tabular Representation Learning, a novel method that leverages large language models (LLMs) to create transferable tabular embeddings. By transforming structured variables into semantic natural language statements and encoding them with a pretrained LLM, our approach enables zero-shot alignment across unseen schemas without manual feature engineering or retraining. We integrate our encoder into a multimodal framework for dementia diagnosis, combining tabular and MRI data. Experiments on NACC and ADNI datasets demonstrate state-of-the-art performance and successful zero-shot transfer to unseen schemas, significantly outperforming clinical baselines, including board-certified neurologists, in retrospective diagnostic tasks. These results validate our LLM-driven approach as a scalable, robust solution for heterogeneous real-world data, offering a pathway to extend LLM-based reasoning to structured domains.
【26】Agentic LLM Reasoning in a Self-Driving Laboratory for Air-Sensitive Lithium Halide Spinel Conductors
标题：空气敏卤锂锐钛导体自动驾驶实验室的统计LLM推理
链接：https://arxiv.org/abs/2604.11957
摘要：自动驾驶实验室有望加速材料发现。然而，目前的自动化固态合成平台限于环境条件，从而排除了它们用于空气敏感材料。在这里，我们介绍了手套箱粉末固态合成A-Lab（A-Lab GPSS），这是一个能够在严格的无空气条件下合成和表征空气敏感无机材料的机器人平台。在A-Lab GPSS平台上集成了一个Agent人工智能框架，通过溯因推理和归纳推理构建自主实验设计。我们部署这个平台来探索卤化锂尖晶石固态离子导体的广阔组成空间。在一个由352个不同成分的样品组成的合成活动中，该系统探索了一个广阔的化学空间，在本研究中考虑的19种金属中，实验实现了171种可能的成对组合中的72%。在整个活动过程中，表现出良好的离子电导率（> 0.05 mS/cm）和高卤化物尖晶石相纯度的组合物的分数从前75个试剂提出的样品中的1.33%增加到最终75个样品中的5.33%。此外，通过检查人工智能的推理过程，我们揭示了不同但互补的发现策略：溯因推理询问已经探索过的区域内的异常观察，而归纳推理将搜索扩展到更广泛的，以前从未访问过的化学空间。这项工作建立了一个可扩展的平台，用于自主发现复杂的空气敏感固态材料。
摘要：Self-driving laboratories promise to accelerate materials discovery. Yet current automated solid-state synthesis platforms are limited to ambient conditions, thereby precluding their use for air-sensitive materials. Here, we present A-Lab for Glovebox Powder Solid-state Synthesis (A-Lab GPSS), a robotic platform capable of synthesizing and characterizing air-sensitive inorganic materials under strict air-free conditions. By integrating an agentic AI framework into the A-Lab GPSS platform, we structure autonomous experimental design through abductive and inductive reasoning. We deploy this platform to explore the vast compositional space of lithium halide spinel solid-state ionic conductors. Across a synthesis campaign comprising 352 samples with diverse compositions, the system explores a broad chemical space, experimentally realizing 72% of the 171 possible pairwise combinations among the 19 metals considered in this study. Over the course of the campaign, the fraction of compositions exhibiting both good ionic conductivity (> 0.05 mS/cm) and high halide spinel phase purity increases from 1.33% in the first 75 agent-proposed samples to 5.33% in the final 75. Furthermore, by inspecting the AI's reasoning processes, we reveal distinct yet complementary discovery strategies: abductive reasoning interrogates abnormal observations within already explored regions, whereas inductive reasoning expands the search into broader, previously unvisited chemical space. This work establishes a scalable platform for the autonomous discovery of complex, air-sensitive solid-state materials.
Graph相关(图学习|图神经网络|图优化等)(4篇)
【1】RoleMAG: Learning Neighbor Roles in Multimodal Graphs
标题：RoleMAG：学习多模态图中的邻居角色
链接：https://arxiv.org/abs/2604.12271
摘要：多模态属性图（MAG）将多模态节点属性与结构化关系相结合。然而，现有的方法通常在单个图上执行共享消息传递，并隐含地假设相同的邻居对所有模态都同样有用。在实践中，有益于一种模态的邻居可能会干扰另一种模态，从而在共享传播下模糊特定于模态的信号。为了解决这个问题，我们提出了RoleMAG，一个多模态图框架，学习不同的邻居应该如何参与传播。具体地说，RoleMAG区分邻居是否应该提供共享、互补或异源信号，并通过单独的传播信道路由它们。这使得互补邻居的跨模态完成成为可能，同时保持异质邻居不共享平滑。在三个以图形为中心的MAG基准上进行的大量实验表明，RoleMAG在RedditS和Bili\_Dance上取得了最好的结果，同时在Toys上保持了竞争力。消融，鲁棒性和效率分析进一步支持所提出的角色感知传播设计的有效性。我们的代码可在https://anonymous.4open.science/r/RoleMAG-7EE0/上获得
摘要：Multimodal attributed graphs (MAGs) combine multimodal node attributes with structured relations. However, existing methods usually perform shared message passing on a single graph and implicitly assume that the same neighbors are equally useful for all modalities. In practice, neighbors that benefit one modality may interfere with another, blurring modality-specific signals under shared propagation. To address this issue, we propose RoleMAG, a multimodal graph framework that learns how different neighbors should participate in propagation. Concretely, RoleMAG distinguishes whether a neighbor should provide shared, complementary, or heterophilous signals, and routes them through separate propagation channels. This enables cross-modal completion from complementary neighbors while keeping heterophilous ones out of shared smoothing. Extensive experiments on three graph-centric MAG benchmarks show that RoleMAG achieves the best results on RedditS and Bili\_Dance, while remaining competitive on Toys. Ablation, robustness, and efficiency analyses further support the effectiveness of the proposed role-aware propagation design. Our code is available at https://anonymous.4open.science/r/RoleMAG-7EE0/
【2】XANE(3): An E(3)-Equivariant Graph Neural Network for Accurate Prediction of XANES Spectra from Atomic Structures
标题：XANE（3）：用于准确预测原子结构XANES光谱的E（3）-等变图神经网络
链接：https://arxiv.org/abs/2604.12140
摘要：我们提出了XANE（3），一个基于物理的E（3）-等变图神经网络，用于直接从原子结构预测X射线吸收近边结构（XANES）光谱。该模型结合了张量积消息传递与球谐边缘特征，搜索查询注意力池，自定义等变层归一化，自适应门控残差连接，以及基于多尺度高斯基的光谱读出与可选的S形背景项。为了提高线形保真度，训练与复合目标，包括逐点光谱重建连同第一和第二导数匹配项。我们评估的数据集上的5,941 FDMNES模拟的氧化铁表面刻面的模型，并获得一个频谱的均方误差为1.0 × 10^{-3}$的测试集。该模型准确地再现了主边缘结构，相对峰强度，前边缘功能，和后边缘振荡。消融研究表明，导数感知目标，自定义同变归一化，自适应条件注意池，自适应门控残差混合，和全球背景项每个提高性能。有趣的是，一个容量匹配的纯标量变体实现了相当的逐点重建误差，但降低了导数级保真度，这表明显式张量通道对于该数据集上的低强度误差并不严格要求，尽管它们仍然有利于捕获更精细的光谱结构。这些结果确立了XANE（3）作为XANES模拟的准确和有效的替代品，并为加速光谱预测，ML辅助光谱和数据驱动的材料发现提供了一条有前途的路线。
摘要：We present XANE(3), a physics-based E(3)-equivariant graph neural network for predicting X-ray absorption near-edge structure (XANES) spectra directly from atomic structures. The model combines tensor-product message passing with spherical harmonic edge features, absorber-query attention pooling, custom equivariant layer normalization, adaptive gated residual connections, and a spectral readout based on a multi-scale Gaussian basis with an optional sigmoidal background term. To improve line-shape fidelity, training is performed with a composite objective that includes pointwise spectral reconstruction together with first- and second-derivative matching terms. We evaluate the model on a dataset of 5,941 FDMNES simulations of iron oxide surface facets and obtain a spectrum mean squared error of $1.0 \times 10^{-3}$ on the test set. The model accurately reproduces the main edge structure, relative peak intensities, pre-edge features, and post-edge oscillations. Ablation studies show that the derivative-aware objective, custom equivariant normalization, absorber-conditioned attention pooling, adaptive gated residual mixing, and global background term each improve performance. Interestingly, a capacity-matched scalar-only variant achieves comparable pointwise reconstruction error but reduced derivative-level fidelity, indicating that explicit tensorial channels are not strictly required for low intensity error on this dataset, although they remain beneficial for capturing finer spectral structure. These results establish XANE(3) as an accurate and efficient surrogate for XANES simulation and offer a promising route toward accelerated spectral prediction, ML-assisted spectroscopy, and data-driven materials discovery.
【3】Exploring Concept Subspace for Self-explainable Text-Attributed Graph Learning
标题：探索可自解释文本属性图学习的概念子空间
链接
：https://arxiv.org/abs/2604.11986
摘要：我们引入图概念瓶颈（GCB）作为一个新的范式自我解释的文本属性图学习。GCB将图映射到一个子空间，即概念瓶颈，其中每个概念都是一个有意义的短语，并根据这些概念的激活进行预测。与现有的主要依赖于子图作为解释的可解释图学习方法不同，概念瓶颈提供了一种新的解释形式。为了细化概念空间，我们应用信息瓶颈原则来关注最相关的概念。这不仅产生了更简洁和更忠实的解释，而且还明确地引导模型朝着正确的决策“思考”。我们的经验表明，GCB实现了内在的可解释性，其准确性与黑盒图神经网络相当。此外，它在分布变化和数据扰动下提供了更好的性能，表现出更好的鲁棒性和泛化能力，受益于概念指导的预测。
摘要：We introduce Graph Concept Bottleneck (GCB) as a new paradigm for self-explainable text-attributed graph learning. GCB maps graphs into a subspace, concept bottleneck, where each concept is a meaningful phrase, and predictions are made based on the activation of these concepts. Unlike existing interpretable graph learning methods that primarily rely on subgraphs as explanations, the concept bottleneck provides a new form of interpretation. To refine the concept space, we apply the information bottleneck principle to focus on the most relevant concepts. This not only yields more concise and faithful explanations but also explicitly guides the model to "think" toward the correct decision. We empirically show that GCB achieves intrinsic interpretability with accuracy on par with black-box Graph Neural Networks. Moreover, it delivers better performance under distribution shifts and data perturbations, showing improved robustness and generalizability, benefitting from concept-guided prediction.
【4】DBGL: Decay-aware Bipartite Graph Learning for Irregular Medical Time Series Classification
标题：DBGL：用于不规则医学时间序列分类的衰变感知二元图学习
链接：https://arxiv.org/abs/2604.11842
摘要：不规则医疗时间序列在临床领域中发挥着重要作用，以更好地了解患者的病情。然而，固有的不规则性所产生的异质采样率，异步观测，和可变的差距带来了关键的挑战，可靠的建模。现有的方法往往扭曲的时间采样不规则性和缺失模式，而无法捕捉可变衰减不规则性，导致次优表示。为了解决这些限制，我们引入了DBGL，不规则医疗时间序列的衰变感知二分图学习。DBGL首先引入了一种患者变量二分图，该图可以同时捕获不规则的采样模式，而无需人工对齐，并自适应地为时间采样不规则性建模建模变量关系，从而增强了表示学习。为了对变量衰减的不规则性进行建模，DBGL设计了一种新颖的节点特定的时间衰减编码机制，该机制基于采样间隔捕获每个变量的衰减率，从而更准确和忠实地表示不规则的时间动态。我们在四个公开的数据集上评估了DBGL的性能，结果表明DBGL优于所有基线。
摘要：Irregular Medical Time Series play a critical role in the clinical domain to better understand the patient's condition. However, inherent irregularity arising from heterogeneous sampling rates, asynchronous observations, and variable gaps poses key challenges for reliable modeling. Existing methods often distort temporal sampling irregularity and missingness patterns while failing to capture variable decay irregularity, resulting in suboptimal representations. To address these limitations, we introduce DBGL, Decay-Aware Bipartite Graph Learning for Irregular Medical Time Series. DBGL first introduces a patient-variable bipartite graph that simultaneously captures irregular sampling patterns without artificial alignment and adaptively models variable relationships for temporal sampling irregularity modeling, enhancing representation learning. To model variable decay irregularity, DBGL designs a novel node-specific temporal decay encoding mechanism that captures each variable's decay rates based on sampling interval, yielding a more accurate and faithful representation of irregular temporal dynamics. We evaluate the performance of DBGL on four publicly available datasets, and the results show that DBGL outperforms all baselines.
Transformer(6篇)
【1】Transformer Based Machine Fault Detection From Audio Input
标题：基于Transformer的音频输入机器故障检测
链接：https://arxiv.org/abs/2604.12733
摘要：近年来，Sound AI越来越多地用于预测机器故障。通过将麦克风连接到感兴趣的机器上，可以从现场获得机器行为的实时数据。传统上，卷积神经网络（CNN）架构已用于分析从捕获的声音生成的频谱图图像，并预测机器是否按预期运行。CNN架构似乎在经验上工作得很好，即使它们有像局部性和参数共享这样的偏差，这些偏差可能与频谱图分析不完全相关。从2020年的Vision Transformer（ViT）开始，基于transformer的模型在图像处理领域的成功应用，人们对在Sound AI领域利用这些模型产生了浓厚的兴趣。由于基于变压器的架构具有显著较低的电感偏差，因此在足够的数据下，它们在谱图分析方面的表现预计将优于CNN。本文展示了变压器驱动架构在分析声音数据方面的有效性，并比较了它们与CNN在机器故障检测的特定任务上生成的嵌入。
摘要：In recent years, Sound AI is being increasingly used to predict machine failures. By attaching a microphone to the machine of interest, one can get real time data on machine behavior from the field. Traditionally, Convolutional Neural Net (CNN) architectures have been used to analyze spectrogram images generated from the sounds captured and predict if the machine is functioning as expected. CNN architectures seem to work well empirically even though they have biases like locality and parameter-sharing which may not be completely relevant for spectrogram analysis. With the successful application of transformer-based models in the field of image processing starting with Vision Transformer (ViT) in 2020, there has been significant interest in leveraging these in the field of Sound AI. Since transformer-based architectures have significantly lower inductive biases, they are expected to perform better than CNNs at spectrogram analysis given enough data. This paper demonstrates the effectiveness of transformer-driven architectures in analyzing Sound data and compares the embeddings they generate with CNNs on the specific task of machine fault detection.
【2】Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task
标题：Transformer是否适应性地利用深度？来自关系推理任务的证据
链接：https://arxiv.org/abs/2604.12426
备注：Accepted at the ICLR 2026 Workshop on Logical Reasoning of Large Language Models
摘要：我们调查Transformers是否使用他们的深度自适应跨任务的难度增加。使用基于家庭故事的受控多跳关系推理任务，其中难度由必须组成的关系跳数决定，我们监视（i）预测如何通过早期读出（logit镜头）跨层演变，以及（ii）任务相关信息如何通过因果修补跨令牌集成。对于预训练模型，我们发现自适应深度用途：一些较大的模型需要更少的层来获得更容易的任务的合理答案，并且随着链长度的增加，模型通常使用更多的层来整合令牌中的信息。对于在任务上进行微调的模型，我们发现了自适应深度使用的更清晰和更一致的证据，对于不保留一般语言建模能力的约束较少的微调制度，效果更强。
摘要 ：We investigate whether transformers use their depth adaptively across tasks of increasing difficulty. Using a controlled multi-hop relational reasoning task based on family stories, where difficulty is determined by the number of relationship hops that must be composed, we monitor (i) how predictions evolve across layers via early readouts (the logit lens) and (ii) how task-relevant information is integrated across tokens via causal patching. For pretrained models, we find some limited evidence for adaptive depth use: some larger models need fewer layers to arrive at plausible answers for easier tasks, and models generally use more layers to integrate information across tokens as chain length increases. For models finetuned on the task, we find clearer and more consistent evidence of adaptive depth use, with the effect being stronger for less constrained finetuning regimes that do not preserve general language modeling abilities.
【3】Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
标题：Nemotron 3 Super：开放、高效的专家混合Mamba-Transformer混合模型，用于统计推理
链接：https://arxiv.org/abs/2604.12374
摘要：我们描述了Nemotron 3 Super的预训练、后训练和量化，Nemotron 3 Super是一个1200亿（活跃120亿）参数的混合Mamba-Attention混合专家模型。Nemotron 3 Super是Nemotron 3系列中的第一个模型，1）在NVFP 4中进行预训练，2）利用LatentMoE，这是一种新的专家混合架构，可以优化每个FLOP的准确性和每个参数的准确性，3）包括MTP层，通过本地推测解码进行推理加速。我们在25万亿个令牌上对Nemotron 3 Super进行了预训练，然后使用监督微调（SFT）和强化学习（RL）进行了后训练。最终模型支持高达1 M的上下文长度，并在常见基准测试中实现了相当的准确性，同时与GPT-OSS-120 B和Qwen3.5- 122 B相比，还实现了高达2.2倍和7.5倍的推理吞吐量。Nemotron 3 Super数据集，以及基础，后训练和量化检查点，都是在HuggingFace上开源的。
摘要 ：We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemotron 3 Super is the first model in the Nemotron 3 family to 1) be pre-trained in NVFP4, 2) leverage LatentMoE, a new Mixture-of-Experts architecture that optimizes for both accuracy per FLOP and accuracy per parameter, and 3) include MTP layers for inference acceleration through native speculative decoding. We pre-trained Nemotron 3 Super on 25 trillion tokens followed by post-training using supervised fine tuning (SFT) and reinforcement learning (RL). The final model supports up to 1M context length and achieves comparable accuracy on common benchmarks, while also achieving up to 2.2x and 7.5x higher inference throughput compared to GPT-OSS-120B and Qwen3.5-122B, respectively. Nemotron 3 Super datasets, along with the base, post-trained, and quantized checkpoints, are open-sourced on HuggingFace.
【4】Distinct mechanisms underlying in-context learning in transformers
标题：Transformer中背景学习的独特机制
链接：https://arxiv.org/abs/2604.12151
备注：46 pages, 19 figures
摘要：现代分布式网络，特别是Transformers，获得了显着的能力（称为“在上下文学习”），以适应其计算输入统计，这样一个固定的网络可以应用于数据从广泛的系统。在这里，我们提供了一个完整的机械特性，这种行为在Transformers上训练的有限集$S$的离散马尔可夫链。Transformer显示四个算法阶段，其特征在于网络是否记忆和泛化，以及它是使用1点还是2点统计。我们表明，这四个阶段是由多层子电路，实现上下文自适应计算的两个定性不同的机制。最小模型隔离了两个图案的关键特征。简化和泛化阶段由两个边界划分，这两个边界取决于数据多样性，$K =| S| $.第一个（$K_1 ^\ast $）由子电路之间的动力学竞争设置，第二个（$K_2 ^\ast $）由代表性瓶颈设置。Transformer训练动态的严格约束理论解释了从1点到2点泛化的急剧转变，并确定了允许网络泛化的损失景观的关键特征。总之，我们表明，Transformers开发不同的子电路，以实现在上下文学习，并确定有利于某些机制的条件。
摘要：Modern distributed networks, notably transformers, acquire a remarkable ability (termed `in-context learning') to adapt their computation to input statistics, such that a fixed network can be applied to data from a broad range of systems. Here, we provide a complete mechanistic characterization of this behavior in transformers trained on a finite set $S$ of discrete Markov chains. The transformer displays four algorithmic phases, characterized by whether the network memorizes and generalizes, and whether it uses 1-point or 2-point statistics. We show that the four phases are implemented by multi-layer subcircuits that exemplify two qualitatively distinct mechanisms for implementing context-adaptive computations. Minimal models isolate the key features of both motifs. Memorization and generalization phases are delineated by two boundaries that depend on data diversity, $K = |S|$. The first ($K_1^\ast$) is set by a kinetic competition between subcircuits and the second ($K_2^\ast$) is set by a representational bottleneck. A symmetry-constrained theory of a transformer's training dynamics explains the sharp transition from 1-point to 2-point generalization and identifies key features of the loss landscape that allow the network to generalize. Put together, we show that transformers develop distinct subcircuits to implement in-context learning and identify conditions that favor certain mechanisms over others.
【5】How Transformers Learn to Plan via Multi-Token Prediction
标题：Transformer如何学习通过多代币预测进行计划
链接：https://arxiv.org/abs/2604.11912
摘要：虽然下一个标记预测（NTP）一直是训练语言模型的标准目标，但它往往难以在推理任务中捕获全局结构。多令牌预测（MTP）最近已成为一种有前途的替代方案，但其基本机制仍然知之甚少。在本文中，我们研究了MTP如何促进推理，重点是规划。从经验上讲，我们表明，MTP始终优于NTP的合成图寻路任务和更现实的推理基准，如倒计时和布尔可满足性问题。从理论上讲，我们分析了一个简化的两层Transformer的星形图任务。我们证明了MTP诱导一个两阶段的反向推理过程：该模型首先关注的结束节点，然后重建的路径，通过跟踪中间节点向后。这种行为源于MTP的梯度解耦特性，与NTP相比，MTP提供了更清晰的训练信号。最终，我们的研究结果强调了多令牌目标如何内在地将优化偏向于健壮和可解释的推理电路。
摘要：While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks. Multi-token prediction (MTP) has recently emerged as a promising alternative, yet its underlying mechanisms remain poorly understood. In this paper, we study how MTP facilitates reasoning, with a focus on planning. Empirically, we show that MTP consistently outperforms NTP on both synthetic graph path-finding tasks and more realistic reasoning benchmarks, such as Countdown and boolean satisfiability problems. Theoretically, we analyze a simplified two-layer Transformer on a star graph task. We prove that MTP induces a two-stage reverse reasoning process: the model first attends to the end node and then reconstructs the path by tracing intermediate nodes backward. This behavior arises from a gradient decoupling property of MTP, which provides a cleaner training signal compared to NTP. Ultimately, our results highlight how multi-token objectives inherently bias optimization toward robust and interpretable reasoning circuits.
【6】Subcritical Signal Propagation at Initialization in Normalization-Free Transformers
标题：无规范化Transformer中次临界信号在超导体中的传播
链接：https://arxiv.org/abs/2604.11890
备注：10 pages main text; 33 pages total; 5 figures in the main text, 24 figures total; preprint
摘要：我们研究信号在Transformers初始化通过平均偏雅可比范数（APJN），跨层的梯度放大的措施。我们将APJN分析扩展到具有双向注意力和置换对称输入令牌配置的Transformers，通过推导跨层激活统计和APJNs的递归关系。我们的理论预测了注意力如何在大深度下修改APJN的渐近行为，并与深度Vision Transformers中测量的APJNs相匹配。从残差网络已知的临界图片结转到Transformers：pre-LayerNorm架构表现出幂律APJN增长，而LayerNorm被元素$\tanh$-like非线性取代Transformers具有拉伸指数APJN增长，表明后者是次临界的。应用于动态Tanh（DyT）和动态erf（Derf）Transformers，该理论解释了为什么这些架构对初始化和优化选择更敏感，并且需要仔细调整以进行稳定训练。
摘要：We study signal propagation at initialization in transformers through the averaged partial Jacobian norm (APJN), a measure of gradient amplification across layers. We extend APJN analysis to transformers with bidirectional attention and permutation-symmetric input token configurations by deriving recurrence relations for activation statistics and APJNs across layers. Our theory predicts how attention modifies the asymptotic behavior of the APJN at large depth and matches APJNs measured in deep vision transformers. The criticality picture known from residual networks carries over to transformers: the pre-LayerNorm architecture exhibits power-law APJN growth, whereas transformers with LayerNorm replaced by elementwise $\tanh$-like nonlinearities have stretched-exponential APJN growth, indicating that the latter are subcritical. Applied to Dynamic Tanh (DyT) and Dynamic erf (Derf) transformers, the theory explains why these architectures can be more sensitive to initialization and optimization choices and require careful tuning for stable training.
GAN|对抗|攻击|生成相关(6篇)
【1】Algorithmic Analysis of Dense Associative Memory: Finite-Size Guarantees and Adversarial Robustness
标题：密集联想记忆的数学分析：平均大小保证和对抗稳健性
链接：https://arxiv.org/abs/2604.12811
备注：21 pages, 9 figures, Accepted in New Frontiers in Associative Memory workshop at ICLR 2026
摘要：密集联想记忆（DAM）通过高阶交互来推广Hopfield网络，并在适当的模式分离条件下实现了O（N^{n-1}）$的存储容量。现有的动力学分析主要研究的热力学限制$N\到\infty$与随机采样的模式，因此不提供有限大小的保证或明确的收敛速度。 我们开发了一个算法分析DAM检索动态，产生有限的N保证下明确的，可验证的模式条件。在高负载的分离假设和有界干扰条件下，我们证明了异步检索动力学的几何收敛性，这意味着一旦轨迹进入吸引盆，收敛时间为O（\log N）。我们进一步建立了对抗鲁棒性界限，通过明确的余量条件来量化每次扫描可容忍的损坏比特的数量，并导出容量保证，在最坏的情况下，其缩放为$Θ（N^{n-1}）$到多对数因子，同时恢复随机模式集合的经典$Θ（N^{n-1}）$缩放。最后，我们表明，DAM检索动态承认潜在的游戏解释，确保收敛到纯纳什均衡异步更新。 附录中提供了完整的证明，以及说明预测收敛性，鲁棒性和容量缩放行为的初步实验。
摘要：Dense Associative Memory (DAM) generalizes Hopfield networks through higher-order interactions and achieves storage capacity that scales as $O(N^{n-1})$ under suitable pattern separation conditions. Existing dynamical analyses primarily study the thermodynamic limit $N\to\infty$ with randomly sampled patterns and therefore do not provide finite-size guarantees or explicit convergence rates. We develop an algorithmic analysis of DAM retrieval dynamics that yields finite-$N$ guarantees under explicit, verifiable pattern conditions. Under a separation assumption and a bounded-interference condition at high loading, we prove geometric convergence of asynchronous retrieval dynamics, which implies $O(\log N)$ convergence time once the trajectory enters the basin of attraction. We further establish adversarial robustness bounds expressed through an explicit margin condition that quantifies the number of corrupted bits tolerable per sweep, and derive capacity guarantees that scale as $Θ(N^{n-1})$ up to polylogarithmic factors in the worst case, while recovering the classical $Θ(N^{n-1})$ scaling for random pattern ensembles. Finally, we show that DAM retrieval dynamics admit a potential-game interpretation that ensures convergence to pure Nash equilibria under asynchronous updates. Complete proofs are provided in the appendices, together with preliminary experiments that illustrate the predicted convergence, robustness, and capacity scaling behavior.
【2】Robust Semi-Supervised Temporal Intrusion Detection for Adversarial Cloud Networks
标题：对抗云网络的稳健半监督时态入侵检测
链接：https://arxiv.org/abs/2604.12655
备注：This work has been accepted for publication in IEEE 2026 EuCNC & 6G Summit. This is a preprint version. The final published version will be available via IEEE Xplore
摘要：云网络越来越依赖基于机器学习的网络入侵检测系统来防御不断变化的网络威胁。然而，现实世界的部署受到有限的标记数据，非静态流量和自适应对手的挑战。虽然半监督学习可以缓解标签稀缺性，但大多数现有方法都隐含地假设良性和稳定的未标记流量，导致对抗性云环境中的性能下降。本文提出了一种用于云入侵检测的鲁棒半监督时间学习框架，该框架明确解决了未标记网络流量中的对抗性污染和时间漂移。该框架在流级数据上运行，将监督学习与一致性正则化、置信度感知伪标记和选择性时间不变性相结合，以保守地利用未标记的流量，同时抑制不可靠的样本。通过利用网络流的时间结构，所提出的方法提高了跨异构云环境的鲁棒性和通用性。在有限标签条件下对公开数据集（CIC-IDS 2017，CSE-CIC-IDS 2018和UNSW-NB 15）进行的广泛评估表明，所提出的框架在检测性能，标签效率以及对抗性和非平稳流量的弹性方面始终优于最先进的监督和半监督网络入侵检测系统。
摘要：Cloud networks increasingly rely on machine learning based Network Intrusion Detection Systems to defend against evolving cyber threats. However, real-world deployments are challenged by limited labeled data, non-stationary traffic, and adaptive adversaries. While semi-supervised learning can alleviate label scarcity, most existing approaches implicitly assume benign and stationary unlabeled traffic, leading to degraded performance in adversarial cloud environments. This paper proposes a robust semi-supervised temporal learning framework for cloud intrusion detection that explicitly addresses adversarial contamination and temporal drift in unlabeled network traffic. Operating on flow-level data, this framework combines supervised learning with consistency regularization, confidence-aware pseudo-labeling, and selective temporal invariance to conservatively exploit unlabeled traffic while suppressing unreliable samples. By leveraging the temporal structure of network flows, the proposed method improves robustness and generalization across heterogeneous cloud environments. Extensive evaluations on publicly available datasets (CIC-IDS2017, CSE-CIC-IDS2018, and UNSW-NB15) under limited-label conditions demonstrate that the proposed framework consistently outperforms state-of-the-art supervised and semi-supervised network intrusion detection systems in detection performance, label efficiency, and resilience to adversarial and non-stationary traffic.
【3】SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation
标题：SubFlow：子模式条件流匹配的多样性一步生成
链接：https://arxiv.org/abs/2604.12273
摘要：流匹配已经成为一个强大的生成框架，最近的几步方法实现了显着的推理加速。然而，我们发现了一个关键但被忽视的限制：这些模型遭受严重的多样性退化，集中样本的主导模式，而忽略了罕见的，但有效的变化的目标分布。我们将这种退化追溯到平均失真：当使用MSE目标进行训练时，类条件流在类内子模式上学习频率加权平均值，导致模型过度表示高密度模式，同时系统地忽略低密度模式。为了解决这个问题，我们提出了子流，子模式条件流匹配，它消除了平均失真分解成细粒度的子模式，通过语义聚类和条件的子模式索引上的流每个类。每个条件子分布近似为单峰，因此学习流准确地针对各个模式，没有平均失真，在单个推理步骤中恢复完整的模式覆盖。至关重要的是，SubFlow是完全即插即用的：它无缝集成到现有的一步模型中，如MeanFlow和MeanFlow模型，而无需任何架构修改。在ImageNet-256上进行的大量实验表明，SubFlow在生成多样性（Recall）方面获得了实质性的收益，同时保持了具有竞争力的图像质量（FID），证实了其在不同的一步生成框架中的广泛适用性。项目页面：https://yexionglin.github.io/subflow。
摘要：Flow matching has emerged as a powerful generative framework, with recent few-step methods achieving remarkable inference acceleration. However, we identify a critical yet overlooked limitation: these models suffer from severe diversity degradation, concentrating samples on dominant modes while neglecting rare but valid variations of the target distribution. We trace this degradation to averaging distortion: when trained with MSE objectives, class-conditional flows learn a frequency-weighted mean over intra-class sub-modes, causing the model to over-represent high-density modes while systematically neglecting low-density ones. To address this, we propose SubFlow, Sub-mode Conditioned Flow Matching, which eliminates averaging distortion by decomposing each class into fine-grained sub-modes via semantic clustering and conditioning the flow on sub-mode indices. Each conditioned sub-distribution is approximately unimodal, so the learned flow accurately targets individual modes with no averaging distortion, restoring full mode coverage in a single inference step. Crucially, SubFlow is entirely plug-and-play: it integrates seamlessly into existing one-step models such as MeanFlow and Shortcut Models without any architectural modifications. Extensive experiments on ImageNet-256 demonstrate that SubFlow yields substantial gains in generation diversity (Recall) while maintaining competitive image quality (FID), confirming its broad applicability across different one-step generation frameworks. Project page: https://yexionglin.github.io/subflow.
【4】Interpretable DNA Sequence Classification via Dynamic Feature Generation in Decision Trees
标题：通过决策树中动态特征生成的可解释DNA序列分类
链接：https://arxiv.org/abs/2604.12060
备注：AISTATS 2026
摘要：DNA序列的分析在许多领域都变得至关重要，从进化生物学到理解基因调控和疾病机制。虽然深度神经网络可以实现卓越的预测性能，但它们通常作为黑匣子运行。与这些黑盒相比，轴对齐决策树为可解释的DNA序列分析提供了一个有希望的方向，但它们受到一个根本的限制：在每个分裂时孤立地考虑单个原始特征限制了它们的表达能力，这导致了禁止的树深度，阻碍了可解释性和泛化性能。我们通过引入DEFT来应对这一挑战，DEFT是一种新的框架，它在树构建过程中自适应地生成高级别的序列特征。DEFT利用大型语言模型来提出针对每个节点处的局部序列分布的生物信息特征，并使用反射机制迭代地对其进行优化。从经验上讲，我们证明了DEFT在各种基因组任务中发现了人类可解释和高度预测的序列特征。
摘要：The analysis of DNA sequences has become critical in numerous fields, from evolutionary biology to understanding gene regulation and disease mechanisms. While deep neural networks can achieve remarkable predictive performance, they typically operate as black boxes. Contrasting these black boxes, axis-aligned decision trees offer a promising direction for interpretable DNA sequence analysis, yet they suffer from a fundamental limitation: considering individual raw features in isolation at each split limits their expressivity, which results in prohibitive tree depths that hinder both interpretability and generalization performance. We address this challenge by introducing DEFT, a novel framework that adaptively generates high-level sequence features during tree construction. DEFT leverages large language models to propose biologically-informed features tailored to the local sequence distributions at each node and to iteratively refine them with a reflection mechanism. Empirically, we demonstrate that DEFT discovers human-interpretable and highly predictive sequence features across a diverse range of genomic tasks.
【5】INTARG: Informed Real-Time Adversarial Attack Generation for Time-Series Regression
标题：INTARG：用于时间序列回归的知情实时对抗攻击生成
链接：https://arxiv.org/abs/2604.11928
摘要：时间序列预测旨在通过对历史观测中的时间依赖性进行建模来预测未来值。它是许多现实世界系统的关键组成部分，在这些系统中，准确的预测可以提高运营效率，并有助于减轻不确定性和风险。最近，机器学习（ML），特别是基于深度学习（DL）的模型，在时间序列预测中得到了广泛的采用，但它们仍然容易受到对抗性攻击。然而，许多最先进的攻击方法并不直接适用于时间序列设置，其中存储完整的历史数据或在每个时间步执行攻击通常是不切实际的。本文提出了一个在线有界缓冲区设置下的时间序列预测的对抗性攻击框架，利用了一个明智的和选择性的攻击策略。通过选择性地瞄准模型表现出高置信度且预期预测误差最大的时间步，我们的框架产生更少但更有效的攻击。实验表明，我们的框架可以将预测误差提高到2.42倍，同时在不到10%的时间步长内执行攻击。
摘要：Time-series forecasting aims to predict future values by modeling temporal dependencies in historical observations. It is a critical component of many real-world systems, where accurate forecasts improve operational efficiency and help mitigate uncertainty and risk. More recently, machine learning (ML), and especially deep learning (DL)-based models, have gained widespread adoption for time-series forecasting, but they remain vulnerable to adversarial attacks. However, many state-of-the-art attack methods are not directly applicable in time-series settings, where storing complete historical data or performing attacks at every time step is often impractical. This paper proposes an adversarial attack framework for time-series forecasting under an online bounded-buffer setting, leveraging an informed and selective attack strategy. By selectively targeting time steps where the model exhibits high confidence and the expected prediction error is maximal, our framework produces fewer but substantially more effective attacks. Experiments show that our framework can increase the prediction error up to 2.42x, while performing attacks in fewer than 10% of time steps.
【6】Data-driven Reachable Set Estimation with Tunable Adversarial and Wasserstein Distributional Guarantees
标题：具有可调对抗和Wasserstein分布保证的数据驱动可达集估计
链接：https://arxiv.org/abs/2604.12654
摘要：研究了仅使用采样状态轨迹的未知离散动态系统的有限时域可达集估计问题。而不是把场景优化作为一个黑盒工具，我们展示了它如何可以定制到可达集估计，其中必须学习一个家庭的集合的基础上，整个轨迹，同时保留概率保证未来的轨迹包含整个地平线。为此，我们制定了一个宽松的方案与松弛变量，产生可达的集合大小和样本外的轨迹包含在地平线之间的可调权衡，从而降低对离群值的敏感性。利用最近的结果在对抗性强的场景优化，然后我们扩展这个配方占有界的对抗性扰动的观察轨迹，并得出一个后验概率保证未来的轨迹包含。当Wasserstein距离的概率分布发生变化时，我们得到了一个明确的界限，优雅的理论概率保证退化。对于不同的几何形状，即，$p$-范数球，椭球，和zonotopes，我们推导出易处理的凸重新和证实我们的理论结果模拟。
摘要：We study finite horizon reachable set estimation for unknown discrete-time dynamical systems using only sampled state trajectories. Rather than treating scenario optimization as a black-box tool, we show how it can be tailored to reachable set estimation, where one must learn a family of sets based on whole trajectories, while preserving probabilistic guarantees on future trajectory inclusion for the entire horizon. To this end, we formulate a relaxed scenario program with slack variables that yields a tunable trade-off between reachable set size and out-of-sample trajectory inclusion over the horizon, thereby reducing sensitivity to outliers. Leveraging the recent results in adversarially robust scenario optimization, we then extend this formulation to account for bounded adversarial perturbations of the observed trajectories and derive a posteriori probabilistic guarantees on future trajectory inclusion. When probability distribution shifts in the Wasserstein distance occur, we obtain an explicit bound on how gracefully the theoretical probabilistic guarantees degrade. For different geometries, i.e., $p$-norm balls, ellipsoids, and zonotopes, we derive tractable convex reformulations and corroborate our theoretical results in simulation.
半/弱/无/有监督|不确定性|主动学习(9篇)
【1】Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks
标题：自适应数据丢失：迈向深度神经网络的自我调节学习
链接：https://arxiv.org/abs/2604.12945
摘要：深度神经网络通常通过对各个时期的大型数据集进行均匀采样来训练，尽管有证据表明并非所有样本在整个学习过程中做出同样的贡献。最近的研究表明，逐步减少训练数据量可以提高效率和泛化能力，但现有方法依赖于固定的时间表，在训练过程中无法适应。在这项工作中，我们提出了Adaptive Data Dropout，这是一个简单的框架，可以根据性能反馈动态调整训练数据的子集。受自我调节学习的启发，我们的方法将数据选择视为一个自适应过程，根据训练准确性的变化增加或减少数据暴露。我们引入了一个轻量级的随机更新机制，可以在线调整dropout时间表，使模型能够随着时间的推移平衡探索和巩固。在标准图像分类基准上的实验表明，与静态数据丢弃策略相比，我们的方法减少了有效的训练步骤，同时保持了有竞争力的准确性。这些结果突出了自适应数据选择作为一个有前途的方向，有效和强大的训练。代码将被释放。
摘要：Deep neural networks are typically trained by uniformly sampling large datasets across epochs, despite evidence that not all samples contribute equally throughout learning. Recent work shows that progressively reducing the amount of training data can improve efficiency and generalization, but existing methods rely on fixed schedules that do not adapt during training. In this work, we propose Adaptive Data Dropout, a simple framework that dynamically adjusts the subset of training data based on performance feedback. Inspired by self-regulated learning, our approach treats data selection as an adaptive process, increasing or decreasing data exposure in response to changes in training accuracy. We introduce a lightweight stochastic update mechanism that modulates the dropout schedule online, allowing the model to balance exploration and consolidation over time. Experiments on standard image classification benchmarks show that our method reduces effective training steps while maintaining competitive accuracy compared to static data dropout strategies. These results highlight adaptive data selection as a promising direction for efficient and robust training. Code will be released.
【2】Monte Carlo Stochastic Depth for Uncertainty Estimation in Deep Learning
标题：深度学习中不确定性估计的蒙特卡罗随机深度
链接：https://arxiv.org/abs/2604.12719
备注：Accepted to the 8th Safe Artificial Intelligence for All Domains (SAIAD) workshop at IEEE/CVF CVPR 2026
摘要：在安全关键系统中部署深度神经网络需要可靠和有效的不确定性量化（UQ）。UQ的一个实用且广泛的策略是将随机正则化器重新用于可扩展的近似贝叶斯推理方法，例如Monte Carlo Dropout（MCD）和MC-DropBlock（MCDB）。然而，对于随机深度（SD），这种范式仍然没有得到充分的探索，SD是大多数现代架构的基于残差的主干的正则化器。虽然先前的工作证明了其分割的经验承诺，但缺少与贝叶斯变分推理的正式理论联系以及对象检测等复杂多任务问题的基准。在本文中，我们首先提供了理论的见解连接蒙特卡罗随机深度（MCSD）的原则近似变分推理。然后，我们使用COCO和COCO-O数据集在最先进的检测器（YOLO，RT-DETR）上提出了MCSD对MCD和MCDB的第一个全面的经验基准。我们的研究结果定位MCSD作为一个强大的和计算效率高的方法，实现了极具竞争力的预测精度（mAP），特别是产生轻微的改善校准（ECE）和不确定性排名（AUARC）相比，MCD。因此，我们将MCSD确立为现代深度学习中有效贝叶斯近似的理论基础和实践验证工具。
摘要：The deployment of deep neural networks in safety-critical systems necessitates reliable and efficient uncertainty quantification (UQ). A practical and widespread strategy for UQ is repurposing stochastic regularizers as scalable approximate Bayesian inference methods, such as Monte Carlo Dropout (MCD) and MC-DropBlock (MCDB). However, this paradigm remains under-explored for Stochastic Depth (SD), a regularizer integral to the residual-based backbones of most modern architectures. While prior work demonstrated its empirical promise for segmentation, a formal theoretical connection to Bayesian variational inference and a benchmark on complex, multi-task problems like object detection are missing. In this paper, we first provide theoretical insights connecting Monte Carlo Stochastic Depth (MCSD) to principled approximate variational inference. We then present the first comprehensive empirical benchmark of MCSD against MCD and MCDB on state-of-the-art detectors (YOLO, RT-DETR) using the COCO and COCO-O datasets. Our results position MCSD as a robust and computationally efficient method that achieves highly competitive predictive accuracy (mAP), notably yielding slight improvements in calibration (ECE) and uncertainty ranking (AUARC) compared to MCD. We thus establish MCSD as a theoretically-grounded and empirically-validated tool for efficient Bayesian approximation in modern deep learning.
【3】Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning
标题：标签TrustSet引导：带强化学习的批量主动学习
链接：https://arxiv.org/abs/2604.12303
备注：Published as a conference paper at IJCNN 2026
摘要：批量主动学习（BAL）是在训练大规模深度学习模型时降低标记成本和提高数据效率的关键技术。传统的BAL方法通常依赖于Mahalanobis距离等指标来平衡选择数据进行注释时的不确定性和多样性。然而，这些方法主要集中在未标记数据的分布上，并且未能利用来自标记数据或模型性能的反馈。为了解决这些限制，我们引入了TrustSet，这是一种新的方法，可以从标记数据集中选择信息量最大的数据，确保均衡的类分布，以减轻长尾问题。与专注于维护整体数据分布的CoreSet不同，TrustSet通过修剪冗余数据并使用标签信息来优化选择过程来优化模型的性能。为了将TrustSet的优点扩展到未标记的池，我们提出了一种基于强化学习（RL）的采样策略，该策略近似于从未标记的数据中选择高质量的TrustSet候选。结合TrustSet和RL，我们介绍了批量强化主动学习与TrustSet（BRAL-T）框架。BRAL-T在10个图像分类基准和2个主动微调任务中取得了最先进的结果，证明了其在各个领域的有效性和效率。
摘要：Batch active learning (BAL) is a crucial technique for reducing labeling costs and improving data efficiency in training large-scale deep learning models. Traditional BAL methods often rely on metrics like Mahalanobis Distance to balance uncertainty and diversity when selecting data for annotation. However, these methods predominantly focus on the distribution of unlabeled data and fail to leverage feedback from labeled data or the model's performance. To address these limitations, we introduce TrustSet, a novel approach that selects the most informative data from the labeled dataset, ensuring a balanced class distribution to mitigate the long-tail problem. Unlike CoreSet, which focuses on maintaining the overall data distribution, TrustSet optimizes the model's performance by pruning redundant data and using label information to refine the selection process. To extend the benefits of TrustSet to the unlabeled pool, we propose a reinforcement learning (RL)-based sampling policy that approximates the selection of high-quality TrustSet candidates from the unlabeled data. Combining TrustSet and RL, we introduce the Batch Reinforcement Active Learning with TrustSet (BRAL-T) framework. BRAL-T achieves state-of-the-art results across 10 image classification benchmarks and 2 active fine-tuning tasks, demonstrating its effectiveness and efficiency in various domains.
【4】Loss-Driven Bayesian Active Learning
标题：损失驱动的Bayesian主动学习
链接：https://arxiv.org/abs/2604.11995
摘要：主动学习的中心目标是收集数据，最大限度地提高下游预测性能，但流行的方法在针对不同的下游问题和损失定制这种数据采集方面的灵活性有限。我们提出了一个严格的损失驱动的贝叶斯主动学习方法，允许数据采集直接针对与给定的决策问题相关的损失。特别是，我们展示了如何使用任何损失来获得一个独特的目标，最佳的数据采集。重要的是，我们表明，任何损失的形式加权Bregman分歧允许分析计算的一个中心组成部分，其相应的目标，使该方法适用于实践。在一系列不同损失的回归和分类实验中，我们发现我们的方法相对于现有技术减少了测试损失。
摘要：The central goal of active learning is to gather data that maximises downstream predictive performance, but popular approaches have limited flexibility in customising this data acquisition to different downstream problems and losses. We propose a rigorous loss-driven approach to Bayesian active learning that allows data acquisition to directly target the loss associated with a given decision problem. In particular, we show how any loss can be used to derive a unique objective for optimal data acquisition. Critically, we then show that any loss taking the form of a weighted Bregman divergence permits analytic computation of a central component of its corresponding objective, making the approach applicable in practice. In regression and classification experiments with a range of different losses, we find our approach reduces test losses relative to existing techniques.
【5】Active Imitation Learning for Thermal- and Kernel-Aware LFM Inference on 3D S-NUCA Many-Cores
标题：3D S-NUCA多核上热感知和核感知LFM推理的主动模仿学习
链接：https://arxiv.org/abs/2604.11948
备注：Accepted for publication at the 63rd ACM/IEEE Design Automation Conference (DAC 2026)
摘要：大型基础模型（LFM）推理是内存和计算密集型的，传统上依赖于GPU。然而，有限的可用性和高成本促使采用高性能通用CPU，特别是新兴的3D堆叠静态非均匀缓存架构（3D S-NUCA）系统。这些架构提供了增强的带宽和局部性，但由于3D片上网络（NoC）而遭受严重的热挑战和不均匀的缓存延迟。由于LFM内核多样性和系统异构性，线程迁移和V/f缩放的最佳管理是不平凡的。现有的热管理方法往往依赖于过于简化的分析模型，缺乏适应性。我们提出了AILFM，主动模仿学习（AIL）为基础的调度框架，学习接近最佳的热感知调度策略，从Oracle的演示与最小的运行时开销。AILFM考虑了LFM中的内核级性能异构性和内核特定行为，以保持热安全性，同时最大限度地提高性能。大量的实验表明，AILFM的性能优于最先进的基线，并在不同的LFM工作负载中推广良好。
摘要 ：Large Foundation Model (LFM) inference is both memory- and compute-intensive, traditionally relying on GPUs. However, the limited availability and high cost have motivated the adoption of high-performance general-purpose CPUs, especially emerging 3D-stacked Static Non-Uniform Cache Architecture (3D S-NUCA) systems. These architectures offer enhanced bandwidth and locality but suffer from severe thermal challenges and uneven cache latencies due to 3D Networks-on-Chip (NoC). Optimal management of thread migration and V/f scaling is non-trivial due to LFM kernel diversity and system heterogeneity. Existing thermal management approaches often rely on oversimplified analytical models and lack adaptability. We propose AILFM, an Active Imitation Learning (AIL)-based scheduling framework that learns near-optimal thermal-aware scheduling policies from Oracle demonstrations with minimal run-time overhead. AILFM accounts for both core-level performance heterogeneity and kernel-specific behavior in LFMs to maintain thermal safety while maximizing performance. Extensive experiments show that AILFM outperforms state-of-the-art baselines and generalizes well across diverse LFM workloads.
【6】A Layer-wise Analysis of Supervised Fine-Tuning
标题：监督微调的分层分析
链接：https://arxiv.org/abs/2604.11838
备注：Accepted by ACL 2026 main conference
摘要：虽然对对齐至关重要，但监督微调（SFT）会引发灾难性遗忘的风险，但分层出现的预防跟踪功能仍然难以捉摸。我们通过全面的分析，利用信息理论，几何和优化指标跨模型尺度（1B-32 B）研究这种机制。我们的实验揭示了一个明显的深度依赖模式：中间层（20 - 80%）是稳定的，而最终层表现出高灵敏度。利用这一洞察力，我们提出了中间块高效调整，它有选择地更新这些关键的中间层。从经验上看，我们的方法在GSM 8 K（OLMo 2 - 7 B）上的性能优于标准LoRA高达10.2\%，并且参数开销减少，这表明有效的对齐是在架构上本地化的，而不是分布式的。该代码可在https://anonymous.4open.science/r/base_sft上公开获取。
摘要：While critical for alignment, Supervised Fine-Tuning (SFT) incurs the risk of catastrophic forgetting, yet the layer-wise emergence of instruction-following capabilities remains elusive. We investigate this mechanism via a comprehensive analysis utilizing information-theoretic, geometric, and optimization metrics across model scales (1B-32B). Our experiments reveal a distinct depth-dependent pattern: middle layers (20\%-80\%) are stable, whereas final layers exhibit high sensitivity. Leveraging this insight, we propose Mid-Block Efficient Tuning, which selectively updates these critical intermediate layers. Empirically, our method outperforms standard LoRA up to 10.2\% on GSM8K (OLMo2-7B) with reduced parameter overhead, demonstrating that effective alignment is architecturally localized rather than distributed. The code is publicly available at https://anonymous.4open.science/r/base_sft.
【7】Uncertainty Quantification in CNN Through the Bootstrap of Convex Neural Networks
标题：通过凸神经网络的Bootstrap进行CNN中的不确定性量化
链接：https://arxiv.org/abs/2604.11833
备注：9 pages, 1 figure. Accepted at AAAI 2021
摘要：尽管卷积神经网络（CNN）非常流行，但CNN的不确定性量化（UQ）问题在很大程度上被忽视了。缺乏有效的UQ工具严重限制了CNN在某些领域的应用，例如医学，其中预测不确定性至关重要。在为深度学习提出的少数现有UQ方法中，没有一种方法具有理论一致性，可以保证不确定性质量。为了解决这个问题，我们提出了一种新的自助框架的预测不确定性的估计。我们使用的推理过程依赖于convexified神经网络建立自举的理论一致性。我们的方法比竞争对手的计算量要小得多，因为它依赖于每次引导的热启动，避免了从头开始重新调整模型。我们进一步探索了一种新的迁移学习方法，因此我们的框架可以在任意神经网络上工作。我们通过实验证明，与其他基线CNN和各种图像数据集上的最先进方法相比，我们的方法具有更好的性能。
摘要：Despite the popularity of Convolutional Neural Networks (CNN), the problem of uncertainty quantification (UQ) of CNN has been largely overlooked. Lack of efficient UQ tools severely limits the application of CNN in certain areas, such as medicine, where prediction uncertainty is critically important. Among the few existing UQ approaches that have been proposed for deep learning, none of them has theoretical consistency that can guarantee the uncertainty quality. To address this issue, we propose a novel bootstrap based framework for the estimation of prediction uncertainty. The inference procedure we use relies on convexified neural networks to establish the theoretical consistency of bootstrap. Our approach has a significantly less computational load than its competitors, as it relies on warm-starts at each bootstrap that avoids refitting the model from scratch. We further explore a novel transfer learning method so our framework can work on arbitrary neural networks. We experimentally demonstrate our approach has a much better performance compared to other baseline CNNs and state-of-the-art methods on various image datasets.
【8】A Bayesian Perspective on the Role of Epistemic Uncertainty for Delayed Generalization in In-Context Learning
标题：认知不确定性对情境学习中延迟概括作用的贝叶斯分析
链接：https://arxiv.org/abs/2604.12434
摘要：上下文学习使Transformers能够在推理时从几个示例中适应新任务，而grokking强调这种泛化只有在长时间训练后才能突然出现。我们研究任务泛化和grokking在上下文学习中使用贝叶斯的角度来看，问什么使延迟过渡从记忆到泛化。具体地说，我们考虑模块化的算术任务，其中一个Transformer必须推断一个潜在的线性函数只从上下文的例子，并分析如何预测不确定性在训练过程中演变。我们结合近似贝叶斯技术来估计后验分布，并研究了不确定性在训练过程中以及在任务多样性、上下文长度和上下文噪声变化下的行为。我们发现，认知的不确定性崩溃时，模型groks急剧，使不确定性一个实用的无标签诊断的泛化变压器。此外，我们提供了一个简化的贝叶斯线性模型的理论支持，渐近延迟泛化和不确定性峰值产生于相同的基本频谱机制，它将grokking时间不确定性动态。
摘要：In-context learning enables transformers to adapt to new tasks from a few examples at inference time, while grokking highlights that this generalization can emerge abruptly only after prolonged training. We study task generalization and grokking in in-context learning using a Bayesian perspective, asking what enables the delayed transition from memorization to generalization. Concretely, we consider modular arithmetic tasks in which a transformer must infer a latent linear function solely from in-context examples and analyze how predictive uncertainty evolves during training. We combine approximate Bayesian techniques to estimate the posterior distribution and we study how uncertainty behaves across training and under changes in task diversity, context length, and context noise. We find that epistemic uncertainty collapses sharply when the model groks, making uncertainty a practical label-free diagnostic of generalization in transformers. Additionally, we provide theoretical support with a simplified Bayesian linear model, showing that asymptotically both delayed generalization and uncertainty peaks arise from the same underlying spectral mechanism, which links grokking time to uncertainty dynamics.
【9】Information-Geometric Decomposition of Generalization Error in Unsupervised Learning
标题：无监督学习中概括误差的信息-几何分解
链接：https://arxiv.org/abs/2604.12340
备注：21 pages, 3 figures
摘要 ：我们将无监督学习的Kullback-Leibler泛化误差（GE）-从数据分布到训练模型的预期KL偏差-分解为三个非负分量：模型误差，数据偏差和方差。分解是准确的任何e-平坦模型类，并遵循两个身份的信息几何：广义勾股定理和对偶e-混合方差身份。作为一个易于分析的演示，我们将该框架应用到$ε$-PCA，一个正则化的主成分分析，其中经验协方差在秩$N_K$处截断，丢弃的方向固定在固定的噪声基底$ε$。虽然秩约束$ε$-PCA本身不是e-平坦的，但它允许在各向同性高斯数据上具有相同总GE的技术重新表述，在该技术下分解的每个分量都采用封闭形式。最佳秩出现为截止值λ_{cut}^{*} = ε$ --模型精确地保留了那些超过噪声基底的经验特征值--截止值反映了模型误差增益和数据偏差成本之间的边际速率平衡。边界比较进一步得到了由下Marchenko-Pastur边分隔的三区相图--保留区、内部区和坍塌区，以及可解析计算的坍塌阈值ε（α），其中α是维数与样品尺寸之比。所有索赔都经过数字核实。
摘要：We decompose the Kullback--Leibler generalization error (GE) -- the expected KL divergence from the data distribution to the trained model -- of unsupervised learning into three non-negative components: model error, data bias, and variance. The decomposition is exact for any e-flat model class and follows from two identities of information geometry: the generalized Pythagorean theorem and a dual e-mixture variance identity. As an analytically tractable demonstration, we apply the framework to $ε$-PCA, a regularized principal component analysis in which the empirical covariance is truncated at rank $N_K$ and discarded directions are pinned at a fixed noise floor $ε$. Although rank-constrained $ε$-PCA is not itself e-flat, it admits a technical reformulation with the same total GE on isotropic Gaussian data, under which each component of the decomposition takes closed form. The optimal rank emerges as the cutoff $λ_{\mathrm{cut}}^{*} = ε$ -- the model retains exactly those empirical eigenvalues exceeding the noise floor -- with the cutoff reflecting a marginal-rate balance between model-error gain and data-bias cost. A boundary comparison further yields a three-regime phase diagram -- retain-all, interior, and collapse -- separated by the lower Marchenko--Pastur edge and an analytically computable collapse threshold $ε_{*}(α)$, where $α$ is the dimension-to-sample-size ratio. All claims are verified numerically.
迁移|Zero/Few/One-Shot|自适应(7篇)
【1】Information-Theoretic Optimization for Task-Adapted Compressed Sensing Magnetic Resonance Imaging
标题：任务适应压缩感知磁共振成像的信息论优化
链接：https://arxiv.org/abs/2604.12709
备注：68 pages, 15 figures, accepted by IEEE TPAMI
摘要：任务适应的压缩感知磁共振成像（CS-MRI）正在出现，以解决下游临床任务的特定需求，其k空间测量值明显少于奈奎斯特采样所需的k空间测量值。然而，现有的任务自适应CS-MRI方法遭受医疗诊断的不确定性问题，并且不能在重建或临床任务的端到端优化中实现自适应采样。为了解决这些局限性，我们提出了第一个任务适应CS-MRI从信息理论的角度，同时实现概率推理的不确定性预测，并适应任意采样率和多功能的临床应用。具体来说，我们通过最大化欠采样k空间测量和临床任务之间的互信息来形式化任务适应的CS-MRI优化问题，以实现用于解决不确定性问题的概率推理。我们利用摊销优化并为互信息构建易于处理的变分界限，以联合优化采样，重建和任务推理模型，从而使用单个端到端训练模型实现灵活的采样率控制。此外，所提出的框架在统一的方法中解决了两种不同的临床场景，即，i）联合任务和重构，其中重构用作增强任务表现的辅助过程;以及ii）具有抑制重构的任务实现，适用于隐私保护。在大规模MRI数据集上进行的大量实验表明，与确定性对应物相比，所提出的框架在Dice等标准度量上实现了极具竞争力的性能，但提供了与广义能量距离（GED）测量的地面真实后验分布更好的分布匹配。
摘要：Task-adapted compressed sensing magnetic resonance imaging (CS-MRI) is emerging to address the specific demands of downstream clinical tasks with significantly fewer k-space measurements than required by Nyquist sampling. However, existing task-adapted CS-MRI methods suffer from the uncertainty problem for medical diagnosis and cannot achieve adaptive sampling in end-to-end optimization with reconstruction or clinical tasks. To address these limitations, we propose the first task-adapted CS-MRI from the information-theoretic perspective to simultaneously achieve probabilistic inference for uncertainty prediction and adapt to arbitrary sampling ratios and versatile clinical applications. Specifically, we formalize the task-adapted CS-MRI optimization problem by maximizing the mutual information between undersampled k-space measurements and clinical tasks to enable probabilistic inference for addressing the uncertainty problem. We leverage amortized optimization and construct tractable variational bounds for mutual information to jointly optimize sampling, reconstruction, and task-inference models, which enables flexible sampling ratio control using a single end-to-end trained model. Furthermore, the proposed framework addresses two kinds of distinct clinical scenarios within a unified approach, i.e., i) joint task and reconstruction, where reconstruction serves as an auxiliary process to enhance task performance; and ii) task implementation with suppressed reconstruction, applicable for privacy protection. Extensive experiments on large-scale MRI datasets demonstrate that the proposed framework achieves highly competitive performance on standard metrics like Dice compared to deterministic counterpart but provides better distribution matching to the ground-truth posterior distribution as measured by the generalized energy distance (GED).
【2】SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration
标题：SpecBound：具有分层置信度校准的自适应有界自我猜测
链接：https://arxiv.org/abs/2604.12247
备注：ACL 2026 Findings
摘要：推测解码已经成为一种很有前途的方法来加速大型语言模型（LLM）中的自回归推理。自起草方法利用基本LLM本身进行推测，避免了辅助草案模型的开销，但面临限制：浅层通常会产生过度自信但不正确的令牌预测，并且草案序列中存在困难令牌会迫使更深层的冗余计算，破坏草案接受和整体加速。为了解决这些问题，我们提出了一种新的自草案框架，通过逐层温度退火在早期退出决策中抑制虚假的信心，并根据令牌解码难度自适应地限制推测长度。通过在统一的并行通过深层中重新处理草稿令牌的隐藏状态，我们的方法在最大化计算效率的同时保持与原始模型的精确输出等价。它不需要修改基本LLM参数，并在各种长格式生成任务和多模型架构中实现了标准自回归解码的高达2.33倍的壁时间加速。
摘要：Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models (LLMs). Self-draft methods, which leverage the base LLM itself for speculation, avoid the overhead of auxiliary draft models but face limitations: shallow layers often produce overconfident yet incorrect token predictions, and the presence of difficult tokens in a draft sequence forces redundant computation through deeper layers, undermining both draft acceptance and overall speedup. To address these issues, we propose a novel self-draft framework that suppresses spurious confidence via layer-wise temperature annealing in early-exit decision and adaptively bounds speculation length based on token-wise decoding difficulty. By reprocessing the hidden states of draft tokens in a unified parallel pass through deep layers, our method maintains exact output equivalence with the original model while maximizing computational efficiency. It requires no modifications to the base LLM parameters and achieves up to 2.33x wall-time speedup over standard autoregressive decoding across diverse long-form generation tasks and multiple model architectures.
【3】Clustering-Enhanced Domain Adaptation for Cross-Domain Intrusion Detection in Industrial Control Systems
标题：工业控制系统中跨域入侵检测的交叉增强域自适应
链接：https://arxiv.org/abs/2604.12183
摘要：工业控制系统运行在动态环境中，流量分布在不同场景中变化，标记样本有限，未知攻击频繁出现，这给跨域入侵检测带来了巨大挑战。针对这一问题，提出了一种基于分簇的工业控制流量域自适应方法。该框架包含两个关键组成部分。首先，基于特征的迁移学习模块通过基于谱变换的特征对齐将源域和目标域投影到共享的潜在子空间中，并迭代地减少分布差异，从而实现准确的跨域检测。其次，聚类增强策略结合K-Medoids聚类与PCA的降维，以提高跨域相关性估计和减少手动参数调整造成的性能下降。实验结果表明，该方法显著提高了未知攻击的检测能力.与五个基线模型相比，它将检测准确率提高了49%，在F分数上获得了更大的增益，并表现出更强的稳定性。此外，聚类增强策略还将代表性任务的检测准确率提高了26%。这些结果表明，该方法有效地消除了数据稀缺和域转移，提供了一个实用的解决方案，在动态工业环境中的鲁棒跨域入侵检测。
摘要 ：Industrial control systems operate in dynamic environments where traffic distributions vary across scenarios, labeled samples are limited, and unknown attacks frequently emerge, posing significant challenges to cross-domain intrusion detection. To address this issue, this paper proposes a clustering-enhanced domain adaptation method for industrial control traffic. The framework contains two key components. First, a feature-based transfer learning module projects source and target domains into a shared latent subspace through spectral-transform-based feature alignment and iteratively reduces distribution discrepancies, enabling accurate cross-domain detection. Second, a clustering enhancement strategy combines K-Medoids clustering with PCA-based dimensionality reduction to improve cross-domain correlation estimation and reduce performance degradation caused by manual parameter tuning. Experimental results show that the proposed method significantly improves unknown attack detection. Compared with five baseline models, it increases detection accuracy by up to 49%, achieves larger gains in F-score, and demonstrates stronger stability. Moreover, the clustering enhancement strategy further boosts detection accuracy by up to 26% on representative tasks. These results suggest that the proposed method effectively alleviates data scarcity and domain shift, providing a practical solution for robust cross-domain intrusion detection in dynamic industrial environments.
【4】VISTA: Validation-Informed Trajectory Adaptation via Self-Distillation
标题：VISTA：通过自蒸馏进行验证知情轨迹适应
链接：https://arxiv.org/abs/2604.12044
摘要：深度学习模型可能会收敛到次优解决方案，尽管验证精度很高，掩盖了我们称之为轨迹偏差的优化失败。这是因为随着训练的进行，模型可以放弃特定数据子群体的高泛化状态，从而放弃先前学习的潜在特征，而不会触发经典的过拟合信号。为了解决这个问题，我们引入了VISTA，一个在线自蒸馏框架，沿着优化轨迹执行一致性。使用验证通知边际覆盖率得分，VISTA确定专家锚点，这是早期的模型状态，保留了对不同数据区域的专业能力。这些锚点的覆盖加权集合在训练过程中在线集成，使损失景观正规化并保留掌握的知识。当在多个基准测试中进行评估时，VISTA表现出比标准训练和之前的自蒸馏方法更好的鲁棒性和泛化能力，而轻量级实现将存储开销降低了90%，而不会降低性能。
摘要：Deep learning models may converge to suboptimal solutions despite strong validation accuracy, masking an optimization failure we term Trajectory Deviation. This is because as training proceeds, models can abandon high generalization states for specific data sub-populations, thus discarding previously learned latent features without triggering classical overfitting signals. To address this problem we introduce VISTA, an online self-distillation framework that enforces consistency along the optimization trajectory. Using a validation-informed Marginal Coverage score, VISTA identifies expert anchors, which are earlier model states that retain specialized competence over distinct data regions. A coverage-weighted ensemble of these anchors is integrated online during training, regularizing the loss landscape and preserving mastered knowledge. When evaluated across multiple benchmarks, VISTA demonstrates improved robustness and generalization over standard training and prior self-distillation methods, while a lightweight implementation reduces storage overhead by 90% without performance loss.
【5】Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
标题：多项式扩展秩自适应：利用高阶相互作用增强低阶微调
链接：https://arxiv.org/abs/2604.11841
备注：Accepted by ACL 2026 findings
摘要：低秩自适应（LoRA）是一种广泛使用的大型语言模型（LLM）有效微调策略，但其严格的线性结构从根本上限制了表达能力。权重更新的双线性公式仅捕获低秩因子之间的一阶依赖关系，限制了非线性和高阶参数相互作用的建模。在本文中，我们提出了多项式扩展秩自适应（PERA），一种新的方法，直接引入结构化的多项式扩展到低秩因子空间。通过扩展每个低秩因子来合成高阶相互作用项，PERA将自适应空间转换为能够在不增加秩或推理成本的情况下建模更丰富的非线性耦合的多项式流形。我们提供的理论分析表明，PERA提供了增强的表达能力和更有效的功能利用率相比，现有的线性自适应方法。从经验上讲，PERA在不同的基准测试中始终优于最先进的方法。值得注意的是，我们的实验表明，将高阶非线性组件，特别是平方项是至关重要的，以提高表达能力和保持强大的和强大的性能，在各种排名设置。我们的代码可在https://github.com/zhangwenhao6/PERA上获得
摘要：Low-rank adaptation (LoRA) is a widely used strategy for efficient fine-tuning of large language models (LLMs), but its strictly linear structure fundamentally limits expressive capacity. The bilinear formulation of weight updates captures only first-order dependencies between low-rank factors, restricting the modeling of nonlinear and higher-order parameter interactions. In this paper, we propose Polynomial Expansion Rank Adaptation (PERA), a novel method that introduces structured polynomial expansion directly into the low-rank factor space. By expanding each low-rank factor to synthesize high-order interaction terms before composition, PERA transforms the adaptation space into a polynomial manifold capable of modeling richer nonlinear coupling without increasing rank or inference cost. We provide theoretical analysis demonstrating that PERA offers enhanced expressive capacity and more effective feature utilization compare to existing linear adaptation approaches. Empirically, PERA consistently outperforms state-of-the-art methods across diverse benchmarks. Notably, our experiments show that incorporating high-order nonlinear components particularly square terms is crucial for enhancing expressive capacity and maintaining strong and robust performance under various rank settings. Our code is available at https://github.com/zhangwenhao6/PERA
【6】Rapid LoRA Aggregation for Wireless Channel Adaptation in Open-Set Radio Frequency Fingerprinting
标题：开放集射频指纹识别中用于无线通道自适应的快速LoRA聚合
链接：https://arxiv.org/abs/2604.12834
备注：6 pages
摘要：射频指纹（RFF）可以实现安全的无线身份验证，但在具有未知设备和不同信道的开放式场景中会遇到困难。现有的方法面临着推广的挑战，并产生高的计算成本。我们提出了一个轻量级的，自适应RFF提取框架使用低秩自适应（LoRA）。通过对每个环境的LoRA模块进行预训练，我们的方法可以快速适应看不见的信道条件，而无需完全重新训练。在推理过程中，LoRA的加权组合动态增强了特征提取。实验结果表明，使用相同的训练数据集，与非微调基线相比，等误差率（EER）降低了15%，相对于完全微调，训练时间减少了83%。该方法为动态无线车载网络中的开集RFF认证提供了一种可扩展且高效的解决方案。
摘要：Radio frequency fingerprints (RFFs) enable secure wireless authentication but struggle in open-set scenarios with unknown devices and varying channels. Existing methods face challenges in generalization and incur high computational costs. We propose a lightweight, self-adaptive RFF extraction framework using Low-Rank Adaptation (LoRA). By pretraining LoRA modules per environment, our method enables fast adaptation to unseen channel conditions without full retraining. During inference, a weighted combination of LoRAs dynamically enhances feature extraction. Experimental results demonstrate a 15% reduction in equal error rate (EER) compared to non-finetuned baselines and an 83% decrease in training time relative to full fine-tuning, using the same training dataset. This approach provides a scalable and efficient solution for open-set RFF authentication in dynamic wireless vehicular networks.
【7】A Nonparametric Adaptive EWMA Control Chart for Binary Monitoring of Multiple Stream Processes
标题：用于多流进程二进制监控的非参数自适应EGMA控制图
链接：https://arxiv.org/abs/2604.12095
摘要：监测多个独立流中的二项比例是统计过程控制（SPC）中的一项关键挑战，其应用范围从制造到网络安全。虽然EWMA图表提供了对小变化的敏感性，但现有的实现依赖于在早期监测期间失败的渐近方差近似。我们引入了累积标准化二项EWMA（CSB-EWMA）图表，该图表通过推导二进制多流数据的EWMA统计数据的精确时变方差来克服这一限制，从而实现自适应控制限制，确保第一个样本的统计严谨性。通过大量的模拟，我们确定了最佳平滑（λ）和限制（L）参数，以实现370和500的目标可控平均运行长度（ARL 0）。CSB-EWMA图表显示了对两种ARL 0靶标的快速偏移检测，对于中度偏移（δ=0.2），失控平均游程长度（ARL 1）降至3-7个样本，并且在不同的数据分布中表现出出色的稳健性，对于ARL 0 = 370和500，ARL 1变异系数较低（对于小偏移，CV < 0.10）。这项工作为从业者提供了一个分布自由，敏感，理论上健全的工具，在二项式多流过程中的早期变化检测。
摘要：Monitoring binomial proportions across multiple independent streams is a critical challenge in Statistical Process Control (SPC), with applications from manufacturing to cybersecurity. While EWMA charts offer sensitivity to small shifts, existing implementations rely on asymptotic variance approximations that fail during early-phase monitoring. We introduce a Cumulative Standardized Binomial EWMA (CSB-EWMA) chart that overcomes this limitation by deriving the exact time-varying variance of the EWMA statistic for binary multiple-stream data, enabling adaptive control limits that ensure statistical rigor from the first sample. Through extensive simulations, we identify optimal smoothing (λ) and limit (L) parameters to achieve target in-control average run length (ARL0) of 370 and 500. The CSB-EWMA chart demonstrates rapid shift detection across both ARL0 targets, with out-of-control average run length (ARL1) dropping to 3-7 samples for moderate shifts (δ=0.2), and exhibits exceptional robustness across different data distributions, with low ARL1 Coefficients of Variation (CV < 0.10 for small shifts) for both ARL0 = 370 and 500. This work provides practitioners with a distribution-free, sensitive, and theoretically sound tool for early change detection in binomial multiple-stream processes.
强化学习(2篇)
【1】MolMem: Memory-Augmented Agentic Reinforcement Learning for Sample-Efficient Molecular Optimization
标题：MolMem：用于样本高效分子优化的内存增强强化学习
链接：https://arxiv.org/abs/2604.12237
摘要：在药物发现中，分子优化的目的是迭代地改进先导化合物以改善分子性质，同时保持与原始分子的结构相似性。然而，每个oracle评估都是昂贵的，使得样本效率成为在有限的oracle预算下现有方法的关键挑战。试错法需要许多oracle调用，而利用外部知识的方法往往会重用熟悉的模板，并在具有挑战性的目标上挣扎。一个关键的缺失部分是长期记忆，它可以为决策提供基础，并为未来的优化提供可重用的见解。为了解决这个问题，我们提出了MolMem（\textbf {Mem}ory的分子优化），一个具有双记忆系统的多轮代理强化学习（RL）框架。具体来说，MolMem使用静态范例记忆来检索冷启动基础的相关范例，并使用不断发展的技能记忆来将成功的轨迹提炼成可重复使用的策略。在此内存增强公式的基础上，我们通过密集的分步奖励来训练策略，将昂贵的推出转化为长期知识，以改善未来的优化。大量的实验表明，MolMem在单属性任务上实现了90%的成功率（比最佳基线高1.5倍），在多属性任务上仅使用500个Oracle调用就实现了52%的成功率。我们的代码可在https://github.com/REAL-Lab-NU/MolMem上获得。
摘要：In drug discovery, molecular optimization aims to iteratively refine a lead compound to improve molecular properties while preserving structural similarity to the original molecule. However, each oracle evaluation is expensive, making sample efficiency a key challenge for existing methods under a limited oracle budget. Trial-and-error approaches require many oracle calls, while methods that leverage external knowledge tend to reuse familiar templates and struggle on challenging objectives. A key missing piece is long-term memory that can ground decisions and provide reusable insights for future optimizations. To address this, we present MolMem (\textbf{Mol}ecular optimization with \textbf{Mem}ory), a multi-turn agentic reinforcement learning (RL) framework with a dual-memory system. Specifically, MolMem uses Static Exemplar Memory to retrieve relevant exemplars for cold-start grounding, and Evolving Skill Memory to distill successful trajectories into reusable strategies. Built on this memory-augmented formulation, we train the policy with dense step-wise rewards, turning costly rollouts into long-term knowledge that improves future optimization. Extensive experiments show that MolMem achieves 90\% success on single-property tasks (1.5$\times$ over the best baseline) and 52\% on multi-property tasks using only 500 oracle calls. Our code is available at https://github.com/REAL-Lab-NU/MolMem.
【2】Offline-Online Reinforcement Learning for Linear Mixture MDPs
标题：线性混合MDP的离线-在线强化学习
链接：https://arxiv.org/abs/2604.11994
备注：72 pages, 4 figures
摘要：研究了环境变化下线性混合马尔可夫决策过程的离线-在线强化学习问题。在离线阶段，数据由未知的行为策略收集，并且可能来自不匹配的环境，而在在线阶段，学习者与目标环境交互。我们提出了一个算法，自适应地利用离线数据。当离线数据信息丰富时，无论是由于足够的覆盖范围还是小的环境变化，该算法都可以证明优于纯在线学习。当离线数据没有信息时，它会安全地忽略它们并匹配仅在线的性能。我们建立了明确表征离线数据何时有益的遗憾上限，以及几乎匹配的下限。数值实验进一步证实了我们的理论研究结果。
摘要：We study offline-online reinforcement learning in linear mixture Markov decision processes (MDPs) under environment shift. In the offline phase, data are collected by an unknown behavior policy and may come from a mismatched environment, while in the online phase the learner interacts with the target environment. We propose an algorithm that adaptively leverages offline data. When the offline data are informative, either due to sufficient coverage or small environment shift, the algorithm provably improves over purely online learning. When the offline data are uninformative, it safely ignores them and matches the online-only performance. We establish regret upper bounds that explicitly characterize when offline data are beneficial, together with nearly matching lower bounds. Numerical experiments further corroborate our theoretical findings.
元学习(2篇)
【1】Black-Box Optimization From Small Offline Datasets via Meta Learning with Synthetic Tasks
标题：通过Meta学习和合成任务从小型离线数据集进行黑匣子优化
链接：https://arxiv.org/abs/2604.12325
备注：Accepted for Publication at International Conference on Artificial Intelligence and Statistics (AISTATS)
摘要：我们考虑离线黑盒优化问题，其中目标是发现最优设计（例如，分子或材料）从过去的实验数据。在这种情况下，一个关键的挑战是数据稀缺：在许多科学应用中，只有小的或低质量的数据集，这严重限制了现有算法的有效性。先前的工作已经从理论上和经验上表明，离线优化算法的性能取决于代理模型捕获优化偏差的程度（即，正确排列输入设计的能力），这在有限的实验数据下是具有挑战性的。本文提出了通过合成任务生成（OptBias）进行优化偏差的替代学习，这是一个直接解决数据稀缺问题的元学习框架。OptBias通过对高斯过程生成的合成任务进行训练来学习可重用的优化偏差，然后在目标任务的小数据上微调代理模型。在各种连续和离散离线优化基准测试中，OptBias在小数据领域的表现始终优于最先进的基线。这些结果突出了OptBias作为现实小数据设置中离线优化的稳健实用解决方案。
摘要 ：We consider the problem of offline black-box optimization, where the goal is to discover optimal designs (e.g., molecules or materials) from past experimental data. A key challenge in this setting is data scarcity: in many scientific applications, only small or poor-quality datasets are available, which severely limits the effectiveness of existing algorithms. Prior work has theoretically and empirically shown that performance of offline optimization algorithms depends on how well the surrogate model captures the optimization bias (i.e., ability to rank input designs correctly), which is challenging to accomplish with limited experimental data. This paper proposes Surrogate Learning with Optimization Bias via Synthetic Task Generation (OptBias), a meta-learning framework that directly tackles data scarcity. OptBias learns a reusable optimization bias by training on synthetic tasks generated from a Gaussian process, and then fine-tunes the surrogate model on the small data for the target task. Across diverse continuous and discrete offline optimization benchmarks, OptBias consistently outperforms state-of-the-art baselines in small data regimes. These results highlight OptBias as a robust and practical solution for offline optimization in realistic small data settings.
【2】BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH
标题：BayMOTH：BayMOTH优化MizatiOn与meTa-lookahead --一种简单的方法
链接：https://arxiv.org/abs/2604.12005
摘要：贝叶斯优化（BO）的顺序优化昂贵的黑盒功能在许多现实世界的设置证明了实用性和有效性。元贝叶斯优化（Meta-BO）的重点是通过利用相关任务的信息来提高BO的样本效率。虽然元BO在任务结构转移时是样本高效的，但元训练和测试任务之间的对齐不良可能会导致在线优化期间建议次优查询。为此，我们提出了一个简单的元BO算法，利用相关的任务信息时，确定有用的，回落到前瞻，否则，在一个统一的框架。我们证明了我们的方法与现有方法在函数优化任务上的竞争力，同时在低任务相关性制度中保持了强大的性能，其中测试任务与元训练集共享有限的结构。
摘要：Bayesian optimization (BO) has for sequential optimization of expensive black-box functions demonstrated practicality and effectiveness in many real-world settings. Meta-Bayesian optimization (meta-BO) focuses on improving the sample efficiency of BO by making use of information from related tasks. Although meta-BO is sample-efficient when task structure transfers, poor alignment between meta-training and test tasks can cause suboptimal queries to be suggested during online optimization. To this end, we propose a simple meta-BO algorithm that utilizes related-task information when determined useful, falling back to lookahead otherwise, within a unified framework. We demonstrate competitiveness of our method with existing approaches on function optimization tasks, while retaining strong performance in low task-relatedness regimes where test tasks share limited structure with the meta-training set.
医学相关(3篇)
【1】OpenTME: An Open Dataset of AI-powered H&E Tumor Microenvironment Profiles from TCGA
标题：OpenTME：来自TCGA的人工智能驱动H & E肿瘤微环境概况的开放数据集
链接：https://arxiv.org/abs/2604.12075
摘要：肿瘤微环境（TME）在癌症进展、治疗反应和患者结局中起着核心作用，但常规苏木精和伊红（H&E）染色组织病理学的大规模、一致和定量TME表征仍然很少。我们介绍OpenTME，这是一个开放获取的数据集，包含来自癌症基因组图谱（TCGA）的5种癌症类型（膀胱癌、乳腺癌、结直肠癌、肝癌和肺癌）的3，634张H& E染色全载玻片图像的预计算TME图谱。所有输出都是使用Atlas H&E-TME生成的，这是一种基于Atlas系列病理学基础模型的人工智能应用程序，可执行组织质量控制，组织分割，细胞检测和分类以及空间邻域分析，在细胞级分辨率下每个载玻片产生超过4，500个定量读数。OpenTME可用于Hugging Face的非商业学术研究。随着时间的推移，我们将继续扩展OpenTME，并预计它将成为生物标志物发现，空间生物学研究和TME分析计算方法开发的资源。
摘要：The tumor microenvironment (TME) plays a central role in cancer progression, treatment response, and patient outcomes, yet large-scale, consistent, and quantitative TME characterization from routine hematoxylin and eosin (H&E)-stained histopathology remains scarce. We introduce OpenTME, an open-access dataset of pre-computed TME profiles derived from 3,634 H&E-stained whole-slide images across five cancer types (bladder, breast, colorectal, liver, and lung cancer) from The Cancer Genome Atlas (TCGA). All outputs were generated using Atlas H&E-TME, an AI-powered application built on the Atlas family of pathology foundation models, which performs tissue quality control, tissue segmentation, cell detection and classification, and spatial neighborhood analysis, yielding over 4,500 quantitative readouts per slide at cell-level resolution. OpenTME is available for non-commercial academic research on Hugging Face. We will continue to expand OpenTME over time and anticipate it will serve as a resource for biomarker discovery, spatial biology research, and the development of computational methods for TME analysis.
【2】A unified data format for managing diabetes time-series data: DIAbetes eXchange (DIAX)
标题：用于管理糖尿病时间序列数据的统一数据格式：DIAbetes eXchange（DIAX）
链接：https://arxiv.org/abs/2604.11944
备注：7 pages, 2 figures
摘要：糖尿病设备，包括连续葡萄糖监测（CGM），智能胰岛素笔和自动胰岛素输送系统，生成广泛用于研究和机器学习的丰富时间序列数据。但是，不同来源的数据格式不一致会阻碍共享、集成和分析。 我们提出了DIAX（DIAbetes eXchange），这是一种基于JSON的标准化格式，用于统一糖尿病时间序列数据，包括CGM，胰岛素和膳食信号。DIAX促进了互操作性、可重复性和可扩展性，特别是对于机器学习应用。开源存储库提供了数据集转换、跨格式兼容性、可视化和社区贡献的工具。DIAX是一种翻译资源，而不是数据主机，确保灵活性，而不会强加数据共享限制。 目前，DIAX与其他标准化工作兼容，并支持主要数据集（DCLP 3、DCLP 5、IOBP 2、PEDAP、T1 Dexi、Loop），总计超过1000万患者小时的数据。https://github.com/Center-for-Diabetes-Technology/DIAX
摘要：Diabetes devices, including Continuous Glucose Monitoring (CGM), Smart Insulin Pens, and Automated Insulin Delivery systems, generate rich time-series data widely used in research and machine learning. However, inconsistent data formats across sources hinder sharing, integration, and analysis. We present DIAX (DIAbetes eXchange), a standardized JSON-based format for unifying diabetes time-series data, including CGM, insulin, and meal signals. DIAX promotes interoperability, reproducibility, and extensibility, particularly for machine learning applications. An open-source repository provides tools for dataset conversion, cross-format compatibility, visualization, and community contributions. DIAX is a translational resource, not a data host, ensuring flexibility without imposing data-sharing constraints. Currently, DIAX is compatible with other standardization efforts and supports major datasets (DCLP3, DCLP5, IOBP2, PEDAP, T1Dexi, Loop), totaling over 10 million patient-hours of data. https://github.com/Center-for-Diabetes-Technology/DIAX
【3】Evaluating the Limitations of Protein Sequence Representations for Parkinson's Disease Classification
标题：帕金森病分类中蛋白质序列表达的局限性评价
链接：https://arxiv.org/abs/2604.11852
备注：36 pages, 10 figures, 9 tables
摘要 ：由于帕金森病的多因素性质，确定可靠的分子生物标志物仍然具有挑战性。虽然蛋白质序列构成了生物信息的基本和广泛可用的来源，但它们对复杂疾病分类的独立判别能力仍不清楚。在这项工作中，我们提出了一个控制和无泄漏的评价多个表示完全来自蛋白质的主要序列，包括氨基酸组成，k-mer，理化描述符，混合表示，并嵌入蛋白质语言模型，所有评估下嵌套分层交叉验证框架，以确保无偏的性能估计。性能最好的配置（ProtBERT + MLP）实现了0.704 +/- 0.028的F1分数和0.748 +/- 0.047的ROC-AUC，表明只有中等的区分性能。经典的表示，如k-mers达到相当的F1值（高达约0.667），但表现出高度不平衡的行为，召回率接近0.98，精确度约0.50，反映了对积极预测的强烈偏见。在表示中，性能差异保持在一个狭窄的范围内（F1在0.60和0.70之间），而无监督分析显示没有与类别标签对齐的内在结构，统计检验（弗里德曼检验，p = 0.1749）并没有表明模型之间存在显著差异。这些结果表明，类之间的大量重叠，并表明，主要序列信息单独提供有限的判别力帕金森氏病的分类。这项工作建立了一个可重复的基线，并提供了经验证据，更多的信息生物学特征，如结构，功能或基于相互作用的描述符，需要强大的疾病建模。
摘要：The identification of reliable molecular biomarkers for Parkinson's disease remains challenging due to its multifactorial nature. Although protein sequences constitute a fundamental and widely available source of biological information, their standalone discriminative capacity for complex disease classification remains unclear. In this work, we present a controlled and leakage-free evaluation of multiple representations derived exclusively from protein primary sequences, including amino acid composition, k-mers, physicochemical descriptors, hybrid representations, and embeddings from protein language models, all assessed under a nested stratified cross-validation framework to ensure unbiased performance estimation. The best-performing configuration (ProtBERT + MLP) achieves an F1-score of 0.704 +/- 0.028 and ROC-AUC of 0.748 +/- 0.047, indicating only moderate discriminative performance. Classical representations such as k-mers reach comparable F1 values (up to approximately 0.667), but exhibit highly imbalanced behavior, with recall close to 0.98 and precision around 0.50, reflecting a strong bias toward positive predictions. Across representations, performance differences remain within a narrow range (F1 between 0.60 and 0.70), while unsupervised analyses reveal no intrinsic structure aligned with class labels, and statistical testing (Friedman test, p = 0.1749) does not indicate significant differences across models. These results demonstrate substantial overlap between classes and indicate that primary sequence information alone provides limited discriminative power for Parkinson's disease classification. This work establishes a reproducible baseline and provides empirical evidence that more informative biological features, such as structural, functional, or interaction-based descriptors, are required for robust disease modeling.
蒸馏|知识提取(2篇)
【1】Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation
标题：Lightning OPD：通过离线策略蒸馏对大型推理模型进行高效的后训练
链接：https://arxiv.org/abs/2604.13010
摘要：On-policy distillation（OPD）已经成为大型语言模型的有效后训练范式。然而，标准的OPD在整个培训过程中需要一个实时的教师推理服务器，导致大量的基础设施开销。在这项工作中，我们调查是否可以离线执行的政策蒸馏。一种自然的方法是在SFT推出时预先计算教师对数概率，并在训练期间重用它们。然而，在实践中，这种离线变体无法可靠地匹配标准OPD的性能。为了理解这种差异，我们确定了一个以前被忽视的条件，这对任何OPD管道都至关重要，我们称之为教师一致性。这个条件要求相同的教师模型用于监督微调和OPD。我们表明，违反教师一致性会引入不可约的梯度偏差，导致离线和在线OPD都收敛到次优固定点，无论训练持续时间如何。基于这一见解，我们提出了闪电OPD，一个离线的政策蒸馏框架，通过预先计算教师的日志概率在SFT推出，强制教师的一致性。这种设计完全消除了对实时教师服务器的需求。我们进一步表明，在教师一致性下，闪电OPD与标准OPD具有相同的最佳效果，具有有界梯度差异和隐式正则化效应，有助于防止政策漂移。在数学推理和代码生成方面的大量实验表明，Lightning OPD实现了最先进的性能，并显着提高了效率。从SFT初始化的Qwen 3 -8B-Base模型开始，Lightning OPD在AIME 2024上仅用30个GPU小时就达到了69.9%，实现了标准OPD的4.0倍加速，并大大降低了LLM后期培训学术研究的进入门槛。
摘要：On-policy distillation (OPD) has emerged as an efficient post-training paradigm for large language models. However, standard OPD requires a live teacher inference server throughout training, resulting in substantial infrastructure overhead. In this work, we investigate whether on-policy distillation can be performed offline. A natural approach is to precompute teacher log-probabilities once over SFT rollouts and reuse them during training. In practice, however, this offline variant fails to reliably match the performance of standard OPD. To understand this discrepancy, we identify a previously overlooked condition that is critical for any OPD pipeline, which we term teacher consistency. This condition requires that the same teacher model be used for both supervised fine-tuning and OPD. We show that violating teacher consistency introduces an irreducible gradient bias, causing both offline and online OPD to converge to a suboptimal fixed point regardless of training duration. Building on this insight, we propose Lightning OPD, an offline on-policy distillation framework that enforces teacher consistency by precomputing teacher log-probabilities over SFT rollouts. This design eliminates the need for a live teacher server entirely. We further show that, under teacher consistency, Lightning OPD shares the same optimum as standard OPD, with bounded gradient discrepancy and an implicit regularization effect that helps prevent policy drift. Extensive experiments on mathematical reasoning and code generation demonstrate that Lightning OPD achieves state-of-the-art performance with significantly improved efficiency. Starting from an SFT-initialized Qwen3-8B-Base model, Lightning OPD reaches 69.9% on AIME 2024 in just 30 GPU hours, achieving a 4.0x speedup over standard OPD and substantially lowering the barrier to entry for academic research on LLM post-training.
【2】Disposition Distillation at Small Scale: A Three-Arc Negative Result
标题：小规模处置蒸馏：三弧阴性结果
链接：https://arxiv.org/abs/2604.11867
备注：16 pages, 4 figures
摘要：我们开始通过四阶段全MIT蒸馏管道将行为倾向（自我验证，不确定性确认，反馈整合）训练成小语言模型（0.6B到2.3B有效参数），并进行推理时间注意力头部干预和冻结基础置信度门控sidecar的后续实验。一份内部草案报告说，一名Qwen 3 -0.6B学生的MCAS和HumanEval分别获得了+33.9分和+15.3分;在公布之前，第二次通过的健全性检查伪造了这两个数字。HumanEval增量是截断伪影（n_predict=512），在n_predict=1024时反转为-8.0点; MCAS增益在苹果到苹果评分下消失。这一伪造引发了随后的三个弧线。在（1）三个模型家族和两个域上的SFT/DPO LoRA，（2）o_proj上的推理时间注意力-头部回火，以及（3）读取最终标记隐藏状态h_last的无训练冻结基边车中，我们发现没有操作符可以在不破坏内容或陷入风格模仿的情况下移动判断测量的倾向。5个型号（Qwen 3 -0.6B、Qwen 3 -1.7B、Qwen3.5-0.8B、Gemma 4 E2 B和SmolLM 2 -1. 7 B-Instruct）的失效一致。分布内交叉验证通过（AUC=0.683）在新提示（AUC=0.516）上崩溃为偶然。我们贡献了一个带有机制的三弧阴性结果，线性h_last探针的两种故障模式分类法，以及一个诚实的伪造管道，该管道将我们自己产生的假阳性类别转换为可伪造的阴性。作为一个独立的发现，Gemma 4 E2 B在Chef域上表现出几乎完全的置信度-正确性解耦（断言不对称性-0.009;无论正确性如何，模型断言为91%）。
摘要：We set out to train behavioral dispositions (self-verification, uncertainty acknowledgment, feedback integration) into small language models (0.6B to 2.3B effective parameters) through a four-stage all-MIT distillation pipeline, with follow-on experiments on inference-time attention-head interventions and a frozen-base confidence-gated sidecar. An internal draft reported +33.9-point MCAS and +15.3-point HumanEval gains on a Qwen3-0.6B student; a second-pass sanity check falsified both numbers before publication. The HumanEval delta was a truncation artifact (n_predict=512) that inverted to -8.0 points at n_predict=1024; the MCAS gain disappeared under apples-to-apples scoring. That falsification triggered three subsequent arcs. Across (1) SFT/DPO LoRA on three model families and two domains, (2) inference-time attention-head tempering on o_proj, and (3) a training-free frozen-base sidecar reading the final-token hidden state h_last, we find no operator that moves judge-measured disposition without damaging content or collapsing into stylistic mimicry. The failure is consistent across five models (Qwen3-0.6B, Qwen3-1.7B, Qwen3.5-0.8B, Gemma 4 E2B, and SmolLM2-1.7B-Instruct). A within-distribution cross-validation pass (AUC=0.683) collapsed to chance on fresh prompts (AUC=0.516). We contribute a three-arc negative result with mechanism, a two-failure-mode taxonomy for linear h_last probes, and an honest falsification pipeline that converts the class of false positives we ourselves produced into publishable negatives. As an independent finding, Gemma 4 E2B exhibits near-complete confidence-correctness decoupling on the Chef domain (assertion asymmetry -0.009; the model asserts at 91% regardless of correctness).
推荐(2篇)
【1】Is Sliding Window All You Need? An Open Framework for Long-Sequence Recommendation
标题：滑动窗就是您所需要的一切吗？长序列推荐的开放框架
链接：https://arxiv.org/abs/2604.12372
备注：8 pages, 2 figures
摘要：长的交互历史是现代推荐系统的核心，然而在现实的内存和延迟预算下，长序列的训练通常被认为是不切实际的。这项工作表明，它不仅是实用的，而且是有效的，在学术规模。我们发布了一个完整的端到端框架，该框架通过滑动窗口实现了工业风格的长序列训练，包括所有数据处理，训练和评估脚本。除了再现先前的收益之外，我们还贡献了早期报告中缺失的两种功能：（i）运行时感知消融研究，该研究量化了跨窗口机制和步幅的准确性计算前沿，以及（ii）一种新型的k移位嵌入层，该嵌入层能够在商品GPU上实现百万级词汇表，而准确性损失可以忽略不计。我们的实现在适度的大学集群上可靠地训练，同时提供有竞争力的检索质量（例如，高达+6.04%的MRR和+6.34%的召回率（Retailrocket@10），$\sim 4 \times $培训时间开销。通过打包一个强大的管道，报告培训时间成本，并引入为低资源环境量身定制的嵌入机制，我们将长序列培训从封闭的工业技术转变为社区实用，开放和可扩展的方法。
摘要：Long interaction histories are central to modern recommender systems, yet training with long sequences is often dismissed as impractical under realistic memory and latency budgets. This work demonstrates that it is not only practical but also effective-at academic scale. We release a complete, end-to-end framework that implements industrial-style long-sequence training with sliding windows, including all data processing, training, and evaluation scripts. Beyond reproducing prior gains, we contribute two capabilities missing from earlier reports: (i) a runtime-aware ablation study that quantifies the accuracy-compute frontier across windowing regimes and strides, and (ii) a novel k-shift embedding layer that enables million-scale vocabularies on commodity GPUs with negligible accuracy loss. Our implementation trains reliably on modest university clusters while delivering competitive retrieval quality (e.g., up to +6.04% MRR and +6.34% Recall@10 on Retailrocket) with $\sim 4 \times $ training-time overheads. By packaging a robust pipeline, reporting training time costs, and introducing an embedding mechanism tailored for low-resource settings, we transform long-sequence training from a closed, industrial technique into a practical, open, and extensible methodology for the community.
【2】Identifying and Mitigating Gender Cues in Academic Recommendation Letters: An Interpretability Case Study
标题：识别和缓解学术推荐信中的性别线索：可解释性案例研究
链接：https://arxiv.org/abs/2604.12337
备注：17 pages, 3 figures
摘要：推荐信（LoR）可能带有隐含性别的语言模式，可能会无意中影响下游决策，例如招聘和录取。在这项工作中，我们调查了基于transformer的编码器模型以及大型语言模型（LLM）在多大程度上可以推断出申请人在提交给美国医疗住院医师计划的学术LoR中的性别，在显式标识符（如姓名和代词）被去性别化之后。在使用三种模型（DistilBERT，RoBERTa和Llama 2）对匿名和去性别LoR的性别进行分类时，从高达68%的分类准确率中可以明显观察到显著的性别泄漏。文本解释方法，如TF-IDF和SHAP，表明某些语言模式是性别的强有力的代理，例如“情感”和“人道主义”通常与女性申请人的LoR相关。作为创建真正性别中立的LoR的实验，这些隐式性别线索被移除，导致在重新训练分类器时的高达5.5%的准确性和2.7%的宏观F_1 $分数下降。然而，申请人的性别预测仍然优于机会。在这个案例研究中，我们的研究结果强调：1）LoR包含难以消除的性别识别线索，可能会激活决策中的偏见; 2）虽然我们的技术框架可能是朝着更公平的学术和专业评估迈出的具体一步，但未来的工作需要询问性别在LoR审查中发挥的作用。总之，我们的研究结果激励上游审计的评价文本在现实世界中的学术推荐信作为一个必要的补充模型级的公平性干预。
摘要：Letters of recommendation (LoRs) can carry patterns of implicitly gendered language that can inadvertently influence downstream decisions, e.g. in hiring and admissions. In this work, we investigate the extent to which Transformer-based encoder models as well as Large Language Models (LLMs) can infer the gender of applicants in academic LoRs submitted to an U.S. medical-residency program after explicit identifiers like names and pronouns are de-gendered. While using three models (DistilBERT, RoBERTa, and Llama 2) to classify the gender of anonymized and de-gendered LoRs, significant gender leakage was observed as evident from up to 68% classification accuracy. Text interpretation methods, like TF-IDF and SHAP, demonstrate that certain linguistic patterns are strong proxies for gender, e.g. "emotional'' and "humanitarian'' are commonly associated with LoRs from female applicants. As an experiment in creating truly gender-neutral LoRs, these implicit gender cues were remove resulting in a drop of up to 5.5% accuracy and 2.7% macro $F_1$ score on re-training the classifiers. However, applicant gender prediction still remains better than chance. In this case study, our findings highlight that 1) LoRs contain gender-identifying cues that are hard to remove and may activate bias in decision-making and 2) while our technical framework may be a concrete step toward fairer academic and professional evaluations, future work is needed to interrogate the role that gender plays in LoR review. Taken together, our findings motivate upstream auditing of evaluative text in real-world academic letters of recommendation as a necessary complement to model-level fairness interventions.
自动驾驶|车辆|车道检测等(1篇)
【1】FeaXDrive: Feasibility-aware Trajectory-Centric Diffusion Planning for End-to-End Autonomous Driving
标题：FeaXDrive：端到端自动驾驶的以概率感知的以轨迹为中心的扩散规划
链接：https://arxiv.org/abs/2604.12656
备注：21 pages, 6 figures
摘要：端到端的扩散规划已经显示出强大的自动驾驶潜力，但生成的轨迹的物理可行性仍然没有得到充分解决。特别是，生成的轨迹可能会表现出局部几何不规则性，违反几何级运动学约束，或偏离可驾驶区域，这表明扩散规划中常用的以噪声为中心的公式尚未与可行性更自然表征的轨迹空间对齐。为了解决这个问题，我们提出了FeaXDrive，一种可行性感知的以自动驾驶为中心的端到端自动驾驶扩散规划方法。其核心思想是将干净的轨迹作为整个扩散过程中可行性建模的统一对象。FeaXDrive建立在这种以几何为中心的公式基础上，集成了自适应曲率约束训练以提高内在的几何和运动学可行性，反向扩散采样中的可驾驶区域指导以提高与可驾驶区域的一致性，以及可行性感知GRPO后训练，以进一步提高规划性能，同时平衡几何空间可行性。在NAVSIM基准测试上的实验表明，FeaXDrive实现了强大的闭环规划性能，同时大大提高了空间可行性。这些发现强调了在端到端扩散规划中明确建模自动驾驶空间可行性的重要性，并为实现更可靠和更实际的自动驾驶规划者迈出了一步。
摘要：End-to-end diffusion planning has shown strong potential for autonomous driving, but the physical feasibility of generated trajectories remains insufficiently addressed. In particular, generated trajectories may exhibit local geometric irregularities, violate trajectory-level kinematic constraints, or deviate from the drivable area, indicating that the commonly used noise-centric formulation in diffusion planning is not yet well aligned with the trajectory space where feasibility is more naturally characterized. To address this issue, we propose FeaXDrive, a feasibility-aware trajectory-centric diffusion planning method for end-to-end autonomous driving. The core idea is to treat the clean trajectory as the unified object for feasibility-aware modeling throughout the diffusion process. Built on this trajectory-centric formulation, FeaXDrive integrates adaptive curvature-constrained training to improve intrinsic geometric and kinematic feasibility, drivable-area guidance within reverse diffusion sampling to enhance consistency with the drivable area, and feasibility-aware GRPO post-training to further improve planning performance while balancing trajectory-space feasibility. Experiments on the NAVSIM benchmark show that FeaXDrive achieves strong closed-loop planning performance while substantially improving trajectory-space feasibility. These findings highlight the importance of explicitly modeling trajectory-space feasibility in end-to-end diffusion planning and provide a step toward more reliable and physically grounded autonomous driving planners.
联邦学习|隐私保护|加密(2篇)
【1】Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization
标题：重新思考联邦学习中的个性化轻松任务：一致性和概括性
链接：https://arxiv.org/abs/2604.12768
备注：arXiv admin note: substantial text overlap with arXiv:2306.05706
摘要：联邦学习（FL）是一种分布式模式，其协调大量本地客户端通过在异构数据集上的阶段式本地训练过程来协作地训练全局模型。以前的作品已经隐含地研究，FL遭受的“客户端漂移”的问题，这是由不一致的最佳跨本地客户端。然而，到目前为止，它仍然缺乏坚实的理论分析来解释这种局部不一致性的影响。为了减轻"客户端漂移“的负面影响，并探讨其实质在FL，在本文中，我们首先提出了一个有效的FL算法FedInit，它允许采用个性化的放松初始化状态在每个本地训练阶段的开始。具体地说，FedInit通过从当前全局状态向最新本地状态的相反方向移动来改变本地状态。此外，为了进一步了解不一致性如何破坏FL的性能，我们引入了超额风险分析，并研究了发散项，以研究FL中的测试误差。我们的研究表明，优化误差对这种局部不一致性不敏感，而它主要影响泛化误差界。大量的实验进行验证其效率。所提出的FedInit方法可以实现与几个先进的基准相比，没有任何额外的培训或通信成本的结果。同时，阶段式个性化松弛初始化也可以结合到目前的几个先进的算法，以实现更高的推广性能的FL范式。
摘要：Federated learning (FL) is a distributed paradigm that coordinates massive local clients to collaboratively train a global model via stage-wise local training processes on the heterogeneous dataset. Previous works have implicitly studied that FL suffers from the ``client-drift'' problem, which is caused by the inconsistent optimum across local clients. However, till now it still lacks solid theoretical analysis to explain the impact of this local inconsistency. To alleviate the negative impact of ``client drift'' and explore its substance in FL, in this paper, we first propose an efficient FL algorithm FedInit, which allows employing the personalized relaxed initialization state at the beginning of each local training stage. Specifically, FedInit initializes the local state by moving away from the current global state towards the reverse direction of the latest local state. Moreover, to further understand how inconsistency disrupts performance in FL, we introduce the excess risk analysis and study the divergence term to investigate the test error in FL. Our studies show that optimization error is not sensitive to this local inconsistency, while it mainly affects the generalization error bound. Extensive experiments are conducted to validate its efficiency. The proposed FedInit method could achieve comparable results compared to several advanced benchmarks without any additional training or communication costs. Meanwhile, the stage-wise personalized relaxed initialization could also be incorporated into several current advanced algorithms to achieve higher generalization performance in the FL paradigm.
【2】Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge
标题：根据联邦学习中的会员推断评估差异隐私：来自NIH基因组学红队挑战赛的见解
链接：https://arxiv.org/abs/2604.12737
备注：21 pages
摘要：虽然联邦学习（FL）减轻了直接数据暴露，但由此产生的训练模型仍然容易受到成员推断攻击（MIA）的影响。本文提出了一个经验评估的差异隐私（DP）作为防御机制对MIA在FL，利用2025年NIST基因组隐私保护联邦学习（PPFL）红色组队事件的环境。为了提高推理的准确性，我们提出了一种堆叠攻击策略，集成七个黑盒估计训练预测概率和交叉熵损失的元分类器。我们在三种隐私配置下针对目标模型评估了这种方法：无保护的卷积神经网络（CNN，$ε=\infty$），低隐私DP模型（$ε=200$）和高隐私DP模型（$ε=10$）。该攻击在无DP和低隐私设置中优于所有基线，并且关键的是，在单信号LiRA基线崩溃的情况下，将可测量的成员泄漏保持在$ε=200$。在一个独立的第三方基准评估，这些结果提供了一个基于堆栈的推理如何在FL校准DP层降级的经验表征。
摘要：While Federated Learning (FL) mitigates direct data exposure, the resulting trained models remain susceptible to membership inference attacks (MIAs). This paper presents an empirical evaluation of Differential Privacy (DP) as a defense mechanism against MIAs in FL, leveraging the environment of the 2025 NIST Genomics Privacy-Preserving Federated Learning (PPFL) Red Teaming Event. To improve inference accuracy, we propose a stacking attack strategy that ensembles seven black-box estimators to train a meta-classifier on prediction probabilities and cross-entropy losses. We evaluate this methodology against target models under three privacy configurations: an unprotected convolutional neural network (CNN, $ε=\infty$), a low-privacy DP model ($ε=200$), and a high-privacy DP model ($ε=10$). The attack outperforms all baselines in the No DP and Low Privacy settings and, critically, maintains measurable membership leakage at $ε=200$ where a single-signal LiRA baseline collapses. Evaluated on an independent third-party benchmark, these results provide an empirical characterisation of how stacking-based inference degrades across calibrated DP tiers in FL.
推理|分析|理解|解释(8篇)
【1】All in One: A Unified Synthetic Data Pipeline for Multimodal Video Understanding
标题：一体化：用于多模式视频理解的统一合成数据管道
链接：https://arxiv.org/abs/2604.12335
备注：8 Pages, 4 Tables, 4 Figures
摘要：训练用于视频理解的多模态大型语言模型（MLLM）需要跨越不同任务的大规模注释数据，例如对象计数，问题回答和分割。然而，在现实世界中收集和注释多模式视频数据是昂贵的，缓慢的，固有的多样性和覆盖范围有限。为了应对这一挑战，我们提出了一个统一的合成数据生成管道，能够自动生成具有丰富多样监督的无限多模式视频数据。我们的框架在单个管道中支持多种任务格式，从而实现跨任务的可扩展和一致的数据创建。为了进一步增强推理能力，我们引入了一种基于VQA的微调策略，该策略可以训练模型回答有关视觉内容的结构化问题，而不仅仅依赖于标题或简单的指令。这种提法鼓励更深层次的视觉基础和推理。我们评估我们的方法在三个具有挑战性的任务：视频对象计数，基于视频的视觉问答，视频对象分割。实验结果表明，主要在合成数据上训练的模型可以有效地推广到真实世界的数据集，通常优于传统训练的模型。我们的研究结果突出了统一合成数据管道作为昂贵的现实世界注释的可扩展替代方案的潜力，用于多模式视频理解。
摘要：Training multimodal large language models (MLLMs) for video understanding requires large-scale annotated data spanning diverse tasks such as object counting, question answering, and segmentation. However, collecting and annotating multimodal video data in real-world is costly, slow, and inherently limited in diversity and coverage. To address this challenge, we propose a unified synthetic data generation pipeline capable of automatically producing unlimited multimodal video data with rich and diverse supervision. Our framework supports multiple task formats within a single pipeline, enabling scalable and consistent data creation across tasks. To further enhance reasoning ability, we introduce a VQA-based fine-tuning strategy that trains models to answer structured questions about visual content rather than relying solely on captions or simple instructions. This formulation encourages deeper visual grounding and reasoning. We evaluate our approach in three challenging tasks: video object counting, video-based visual question answering, and video object segmentation. Experimental results demonstrate that models trained predominantly on synthetic data generalize effectively to real-world datasets, often outperforming traditionally trained counterparts. Our findings highlight the potential of unified synthetic data pipelines as a scalable alternative to expensive real-world annotation for multimodal video understanding.
【2】SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
标题：SOLARIS：推理缩放的潜在代表的推测卸载
链接：https://arxiv.org/abs/2604.12110
备注：Accepted to SIGIR 2026 Industry Track
摘要：推荐缩放定律的最新进展导致了前所未有的复杂性的基础模型。虽然这些模型提供了卓越的性能，但它们的计算需求使得实时服务变得不切实际，通常迫使从业者依赖知识蒸馏，从而降低服务质量以提高效率。为了解决这一挑战，我们提出了SOLARIS（推理缩放的基于潜在表示的推测卸载），这是一个受到推测解码启发的新框架。SOLARIS通过预测哪些用户-项目对可能出现在未来的请求中，并提前异步生成其基础模型表示来主动预计算用户-项目交互嵌入。这种方法从延迟关键的服务路径中消除了昂贵的基础模型推理，使以前认为在线使用过于昂贵的模型能够实现实时知识转移。SOLARIS部署在Meta的广告系统中，每天服务数十亿次请求，实现了0.67%的收入驱动顶线指标增益，证明了其规模的有效性。
摘要：Recent advances in recommendation scaling laws have led to foundation models of unprecedented complexity. While these models offer superior performance, their computational demands make real-time serving impractical, often forcing practitioners to rely on knowledge distillation-compromising serving quality for efficiency. To address this challenge, we present SOLARIS (Speculative Offloading of Latent-bAsed Representation for Inference Scaling), a novel framework inspired by speculative decoding. SOLARIS proactively precomputes user-item interaction embeddings by predicting which user-item pairs are likely to appear in future requests, and asynchronously generating their foundation model representations ahead of time. This approach decouples the costly foundation model inference from the latency-critical serving path, enabling real-time knowledge transfer from models previously considered too expensive for online use. Deployed across Meta's advertising system serving billions of daily requests, SOLARIS achieves 0.67% revenue-driving top-line metrics gain, demonstrating its effectiveness at scale.
【3】Spatial Atlas: Compute-Grounded Reasoning for Spatial-Aware Research Agent Benchmarks
标题：空间地图集：空间感知研究代理基准的计算机基于推理
链接：https://arxiv.org/abs/2604.12102
备注：11 pages. Submitted to NeurIPS 2026. Code: https://github.com/arunshar/spatial-atlas
摘要：我们介绍了计算接地推理（CGR），空间感知的研究代理的设计范式，其中每个可回答的子问题是通过确定性计算解决语言模型被要求生成之前。Spatial Atlas将CGR实例化为单个代理到代理（A2 A）服务器，处理两个具有挑战性的基准测试：FieldWorkArena，一个跨越工厂，仓库和零售环境的多模式空间问答基准测试，以及MLE-Bench，一套75个Kaggle机器学习竞赛，需要端到端ML工程。结构化空间场景图引擎从视觉描述中提取实体和关系，确定性地计算距离和安全违规，然后将计算出的事实提供给大型语言模型，从而避免幻觉空间推理。熵引导的动作选择最大限度地提高了每一步的信息增益，并在三层前沿模型堆栈（OpenAI + Anthropic）中路由查询。一个具有策略感知代码生成的自我修复ML管道，一个分数驱动的迭代细化循环，以及一个基于脚本的泄漏审计注册表完善了系统。我们在这两个基准进行评估，并表明CGR产生竞争力的准确性，同时保持可解释性，通过结构化的中间表示和确定性的空间计算。
摘要：We introduce compute-grounded reasoning (CGR), a design paradigm for spatial-aware research agents in which every answerable sub-problem is resolved by deterministic computation before a language model is asked to generate. Spatial Atlas instantiates CGR as a single Agent-to-Agent (A2A) server that handles two challenging benchmarks: FieldWorkArena, a multimodal spatial question-answering benchmark spanning factory, warehouse, and retail environments, and MLE-Bench, a suite of 75 Kaggle machine learning competitions requiring end-to-end ML engineering. A structured spatial scene graph engine extracts entities and relations from vision descriptions, computes distances and safety violations deterministically, then feeds computed facts to large language models, thereby avoiding hallucinated spatial reasoning. Entropy-guided action selection maximizes information gain per step and routes queries across a three-tier frontier model stack (OpenAI + Anthropic). A self-healing ML pipeline with strategy-aware code generation, a score-driven iterative refinement loop, and a prompt-based leak audit registry round out the system. We evaluate across both benchmarks and show that CGR yields competitive accuracy while maintaining interpretability through structured intermediate representations and deterministic spatial computations.
【4】Robust Reasoning and Learning with Brain-Inspired Representations under Hardware-Induced Nonlinearities
标题：硬件诱导的非线性下利用脑启发表示的鲁棒推理和学习
链接：https://arxiv.org/abs/2604.12079
备注：8 pages, 7 figures, accepted to Great Lakes Symposium on VLSI (GLSVLSI) 2025
摘要：传统的机器学习依赖于高精度的算法和近乎理想的硬件假设，这越来越受到大规模半导体器件可变性的挑战。内存计算（CIM）架构缓解了数据移动瓶颈，提高了能源效率，但也引入了非线性失真和可靠性问题。我们解决这些问题的硬件感知的优化框架的基础上超维计算（HDC），系统地补偿CIM中的非理想的相似性计算。我们的方法制定编码作为一个优化问题，最大限度地减少一个理想的内核和它的硬件约束对应之间的Frobenius范数，并采用了联合优化策略的端到端校准的超向量表示。实验结果表明，该方法应用于QuantHD，在硬件干扰严重的情况下，准确率达到84%，比相同条件下的朴素QuantHD提高了48%.此外，我们的优化是至关重要的基于图的HDC依赖于精确的变量绑定可解释的推理。我们的框架保留了Cora数据集上RelHD的准确性，在非线性环境下，与朴素RelHD相比，准确性提高了5.4倍。通过保留HDC的鲁棒性和符号属性，我们的解决方案使可扩展的，节能的智能系统能够在新兴的CIM硬件上进行分类和推理。
摘要：Traditional machine learning depends on high-precision arithmetic and near-ideal hardware assumptions, which is increasingly challenged by variability in aggressively scaled semiconductor devices. Compute-in-memory (CIM) architectures alleviate data-movement bottlenecks and improve energy efficiency yet introduce nonlinear distortions and reliability concerns. We address these issues with a hardware-aware optimization framework based on Hyperdimensional Computing (HDC), systematically compensating for non-ideal similarity computations in CIM. Our approach formulates encoding as an optimization problem, minimizing the Frobenius norm between an ideal kernel and its hardware-constrained counterpart, and employs a joint optimization strategy for end-to-end calibration of hypervector representations. Experimental results demonstrate that our method when applied to QuantHD achieves 84\% accuracy under severe hardware-induced perturbations, a 48\% increase over naive QuantHD under the same conditions. Additionally, our optimization is vital for graph-based HDC reliant on precise variable-binding for interpretable reasoning. Our framework preserves the accuracy of RelHD on the Cora dataset, achieving a 5.4$\times$ accuracy improvement over naive RelHD under nonlinear environments. By preserving HDC's robustness and symbolic properties, our solution enables scalable, energy-efficient intelligent systems capable of classification and reasoning on emerging CIM hardware.
【5】Robust Explanations for User Trust in Enterprise NLP Systems
标题：企业NLP系统中用户信任的稳健解释
链接：https://arxiv.org/abs/2604.12069
摘要 ：用户对企业NLP的信任越来越需要强大的解释，但在黑盒部署（仅API访问）的常见情况下，部署前验证是困难的，其中基于表示的解释器是不可行的，现有的研究对解释是否在真实用户噪声下保持稳定提供了有限的指导，特别是当组织从编码器分类器迁移到解码器LLM时。为了缩小这一差距，我们提出了一个统一的黑盒鲁棒性评估框架的标记级解释的基础上留一出闭塞，并操作化解释鲁棒性与顶令牌翻转率下现实的扰动（交换，删除，洗牌，和回翻译）在多个严重程度。使用该协议，我们对三个基准数据集和六个模型进行了系统的跨架构比较，这些模型涵盖了编码器和解码器系列（BERT，RoBERTa，Qwen 7 B/14 B，Llama 8B/70 B; 64，800例）。我们发现，解码器LLM比编码器基线产生更稳定的解释（平均翻转率低73%），并且稳定性随着模型规模的增加而提高（从7 B到70 B增加44%）。最后，我们将鲁棒性改进与推理成本联系起来，得到一个实用的成本-鲁棒性权衡曲线，该曲线支持在合规性敏感的应用程序中部署之前选择模型和解释。
摘要：Robust explanations are increasingly required for user trust in enterprise NLP, yet pre-deployment validation is difficult in the common case of black-box deployment (API-only access) where representation-based explainers are infeasible and existing studies provide limited guidance on whether explanations remain stable under real user noise, especially when organizations migrate from encoder classifiers to decoder LLMs. To close this gap, we propose a unified black-box robustness evaluation framework for token-level explanations based on leave-one-out occlusion, and operationalize explanation robustness with top-token flip rate under realistic perturbations (swap, deletion, shuffling, and back-translation) at multiple severity levels. Using this protocol, we conduct a systematic cross-architecture comparison across three benchmark datasets and six models spanning encoder and decoder families (BERT, RoBERTa, Qwen 7B/14B, Llama 8B/70B; 64,800 cases). We find that decoder LLMs produce substantially more stable explanations than encoder baselines (73% lower flip rates on average), and that stability improves with model scale (44% gain from 7B to 70B). Finally, we relate robustness improvements to inference cost, yielding a practical cost-robustness tradeoff curve that supports model and explanation selection prior to deployment in compliance-sensitive applications.
【6】Sample Complexity of Autoregressive Reasoning: Chain-of-Thought vs. End-to-End
标题：自回归推理的复杂性示例：思想链与端到端
链接：https://arxiv.org/abs/2604.12013
摘要：现代大型语言模型自回归生成文本，每次生成一个标记。为了研究这种系统的可学习性，Joshi等人（COLT 2025）为下一个令牌生成器（原始的底层自回归模型）引入了PAC学习框架。在这个框架中，一个未知的下一个令牌生成器将一个令牌序列映射到下一个令牌，并迭代应用$T$步骤，产生一个令牌链，其最终令牌构成模型的输出。学习任务是学习由该自回归过程引起的输入-输出映射。根据可用的监督，训练示例可能只显示最终输出（端到端监督）或整个生成的链（思想链监督）。这就提出了两个自然的问题：样本复杂度如何依赖于生成长度T，以及思想链监督可以在多大程度上减少这种依赖。 在这项工作中，我们给了一个几乎完整的答案，这两个问题，揭示了一个分类的样本复杂性如何规模与$T$。对于端到端学习，我们表明，景观是非常丰富的：在温和的条件下，基本上任何增长率$r（T）$之间的常数和线性可以出现的样本复杂性，并结合Joshi等人的线性上限，这产生了基本上完整的表征。相比之下，在思想链的监督下，我们表明，样本的复杂性是独立的$T$，表明访问中间推理步骤可以完全消除对生成长度的依赖。我们的分析引入了新的组合工具，作为推论，我们解决了Joshi等人提出的几个悬而未决的问题。关于可学习性对生成长度的依赖和思想链监督的作用。
摘要：Modern large language models generate text autoregressively, producing tokens one at a time. To study the learnability of such systems, Joshi et al. (COLT 2025) introduced a PAC-learning framework for next-token generators, the primitive underlying autoregressive models. In this framework, an unknown next-token generator maps a sequence of tokens to the next token and is iteratively applied for $T$ steps, producing a chain of tokens whose final token constitutes the model's output. The learning task is to learn the input-output mapping induced by this autoregressive process. Depending on the available supervision, training examples may reveal only the final output (End-to-End supervision) or the entire generated chain (Chain-of-Thought supervision). This raises two natural questions: how the sample complexity depends on the generation length $T$, and how much Chain-of-Thought supervision can reduce this dependence. In this work we give a nearly complete answer to both questions by uncovering a taxonomy of how the sample complexity scales with $T$. For End-to-End learning, we show that the landscape is remarkably rich: subject to mild conditions, essentially any growth rate $r(T)$ between constant and linear can arise as the sample complexity, and combined with the linear upper bound of Joshi et al., this yields an essentially complete characterization. In contrast, under Chain-of-Thought supervision we show that the sample complexity is independent of $T$, demonstrating that access to intermediate reasoning steps can eliminate the dependence on the generation length altogether. Our analysis introduces new combinatorial tools, and as corollaries we resolve several open questions posed by Joshi et al. regarding the dependence of learnability on the generation length and the role of Chain-of-Thought supervision.
【7】INDOTABVQA: A Benchmark for Cross-Lingual Table Understanding in Bahasa Indonesia Documents
标题：INDOTABVQA：印度尼西亚语文档中跨语言餐桌理解的基准
链接：https://arxiv.org/abs/2604.11970
备注：Accepted in ACL 2026 (Findings)
摘要：我们介绍了INDOTABVQA，一个基准评估跨语言表视觉问题检索（VQA）在真实世界的文档图像在印度尼西亚Baidu。该数据集包括1，593个文档图像，包括三种视觉风格（有边框，无边框和彩色），一个或多个表格，以及1，593个问题答案集，四种语言：印度尼西亚语，英语，印地语和阿拉伯语。这使得视觉语言模型（VLM）的评估在单语（Baidu文件与Baidu问题）和跨语言设置（Baidu文件与其他语言的问题）。我们对领先的开源VLM（Qwen2.5-VL，Gemma-3，LLaMA-3.2）和GPT-4 o进行了基准测试，并揭示了巨大的性能差距，特别是在结构复杂的表和低资源语言中。在我们的数据集上微调紧凑的3B和LoRA微调的7 B模型，准确度分别提高了11.6%和17.8%。提供显式的表区域坐标作为额外的输入，进一步提高了4- 7%的性能，证明了空间先验对基于表的推理的价值。我们的研究结果强调了语言多样性，特定领域数据集的重要性，并表明有针对性的微调可以显着提高VLM在专业文档理解任务上的性能。INDOTABVQA为推进跨语言，结构感知文档理解的研究提供了宝贵的资源，特别是在世界上代表性不足的地区。完整的数据集可以在huggingface中访问：https：//huggingface.co/softets/NusaBharat/INDOTABVQA}
摘要：We introduce INDOTABVQA, a benchmark for evaluating cross-lingual Table Visual Question Answering (VQA) on real-world document images in Bahasa Indonesia. The dataset comprises 1,593 document images across three visual styles (bordered, borderless, and colorful) with one or more than one tables, and 1,593 question-answer sets in four languages: Bahasa Indonesia, English, Hindi, and Arabic. This enables evaluation of Vision-Language Models (VLMs) in both monolingual (Bahasa documents with Bahasa questions) and cross-lingual settings (Bahasa documents with questions in other languages). We benchmark leading open-source VLMs (Qwen2.5-VL, Gemma-3, LLaMA-3.2) and GPT-4o and reveal substantial performance gaps, particularly on structurally complex tables and in low-resource languages. Fine-tuning a compact 3B and LoRA-finetuned 7B model on our dataset yields 11.6% and 17.8% improvements in accuracy. Providing explicit table region coordinates as additional input further improves performance by 4-7%, demonstrating the value of Spatial priors for table-based reasoning. Our findings underscore the importance of language-diverse, domain-specific datasets and demonstrate that targeted fine-tuning can significantly enhance VLM performance on specialized document understanding tasks. INDOTABVQA provides a valuable resource for advancing research in cross-lingual, structure-aware document understanding, especially in underrepresented regions of the world. Full dataset can be accessed in huggingface at: https://huggingface.co/datasets/NusaBharat/INDOTABVQA}
【8】ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
标题：ZoomR：通过多粒度关键值检索进行内存高效推理
链接：https://arxiv.org/abs/2604.10898
摘要 ：大型语言模型（LLM）在复杂的推理任务中表现出了很好的性能，但在得出最终答案之前，通常需要生成很长的中间思想。在生成期间，LLM依赖于键值（KV）缓存进行自回归解码。然而，KV缓存的内存占用随着输出长度而增长。先前关于KV缓存优化的工作主要集中在压缩长输入上下文，同时保留完整的KV缓存用于解码。对于需要长时间输出生成的任务，这会导致计算和内存成本增加。在本文中，我们介绍了ZoomR，一种新的方法，使LLM自适应压缩冗长的推理思想到摘要，并使用动态KV缓存选择策略，利用这些摘要，同时也战略性地“放大”细粒度的细节。通过在解码过程中使用摘要键作为粗粒度索引，ZoomR使用查询来检索最重要思想的细节。这种分层策略通过避免在每一步都关注满缓存，显著减少了内存使用。在数学和推理任务的实验表明，我们的方法实现了有竞争力的性能相比，基线，同时减少推理内存需求超过4\times $。这些结果表明，多粒度KV选择使得能够实现更高效的存储器解码，特别是对于长输出生成。
摘要：Large language models (LLMs) have shown great performance on complex reasoning tasks but often require generating long intermediate thoughts before reaching a final answer. During generation, LLMs rely on a key-value (KV) cache for autoregressive decoding. However, the memory footprint of the KV cache grows with output length. Prior work on KV cache optimization mostly focus on compressing the long input context, while retaining the full KV cache for decoding. For tasks requiring long output generation, this leads to increased computational and memory costs. In this paper, we introduce ZoomR, a novel approach that enables LLMs to adaptively compress verbose reasoning thoughts into summaries and uses a dynamic KV cache selection policy that leverages these summaries while also strategically "zooming in" on fine-grained details. By using summary keys as a coarse-grained index during decoding, ZoomR uses the query to retrieve details for only the most important thoughts. This hierarchical strategy significantly reduces memory usage by avoiding full-cache attention at each step. Experiments across math and reasoning tasks show that our approach achieves competitive performance compared to baselines, while reducing inference memory requirements by more than $4\times$. These results demonstrate that a multi-granularity KV selection enables more memory efficient decoding, especially for long output generation.
检测相关(3篇)
【1】CLAD: Efficient Log Anomaly Detection Directly on Compressed Representations
标题：CLAD：直接对压缩表示进行高效日志异常检测
链接：https://arxiv.org/abs/2604.13024
摘要：系统日志的爆炸性增长使得流压缩变得必要，然而现有的日志异常检测（LAD）方法由于需要完全解压缩和解析而导致严重的预处理开销。我们介绍CLAD，这是第一个直接在压缩字节流上执行LAD的深度学习框架。CLAD通过利用一个关键的洞察力绕过了这些瓶颈：正常的日志压缩成规则的字节模式，而异常则系统地破坏它们。为了从不透明字节中提取这些多尺度偏差，我们提出了一种专用架构，该架构集成了膨胀的卷积字节编码器，混合Transformer-mLSTM和四路聚合池。再加上掩蔽预训练和焦点对比微调的两阶段训练策略，可有效处理严重的班级不平衡。在五个数据集上进行评估，CLAD的平均F1得分为0.9909，比最佳基线高出2.72个百分点。它提供了卓越的准确性，同时完全消除了解压缩和解析开销，提供了一个强大的解决方案，推广到结构化流压缩器。
摘要：The explosive growth of system logs makes streaming compression essential, yet existing log anomaly detection (LAD) methods incur severe pre-processing overhead by requiring full decompression and parsing. We introduce CLAD, the first deep learning framework to perform LAD directly on compressed byte streams. CLAD bypasses these bottlenecks by exploiting a key insight: normal logs compress into regular byte patterns, while anomalies systematically disrupt them. To extract these multi-scale deviations from opaque bytes, we propose a purpose-built architecture integrating a dilated convolutional byte encoder, a hybrid Transformer--mLSTM, and four-way aggregation pooling. This is coupled with a two-stage training strategy of masked pre-training and focal-contrastive fine-tuning to effectively handle severe class imbalance. Evaluated across five datasets, CLAD achieves a state-of-the-art average F1-score of 0.9909 and outperforms the best baseline by 2.72 percentage points. It delivers superior accuracy while completely eliminating decompression and parsing overheads, offering a robust solution that generalizes to structured streaming compressors.
【2】Stress Detection Using Wearable Physiological and Sociometric Sensors
标题：使用可穿戴生理和社会测量传感器的压力检测
链接：https://arxiv.org/abs/2604.12746
备注：This is the accepted manuscript of the article published in International Journal of Neural Systems, 27, 2, 2017. The Version of Record is available at DOI: 10.1142/S0129065716500416
摘要：压力仍然是现代社会个人面临的一个重大社会问题。本文提出了一种机器学习方法，通过结合两个传感器系统，捕捉生理和社会反应的压力的人在社会情况下的自动检测。我们使用不同的分类器，包括支持向量机，AdaBoost，和k-近邻的性能进行比较。我们的实验结果表明，通过结合两个传感器系统的测量，我们可以准确地区分压力和中性的情况下，在控制特里尔社会压力测试（TSST）。此外，本文单独评估每个传感器模态的判别能力，并考虑其适用于实时应力检测。最后，我们提出了一个研究的最具鉴别力的功能，压力检测。
摘要：Stress remains a significant social problem for individuals in modern societies. This paper presents a machine learning approach for the automatic detection of stress of people in a social situation by combining two sensor systems that capture physiological and social responses. We compare the performance using different classifiers including support vector machine, AdaBoost, and k-nearest neighbor. Our experimental results show that by combining the measurements from both sensor systems, we could accurately discriminate between stressful and neutral situations during a controlled Trier social stress test (TSST). Moreover, this paper assesses the discriminative ability of each sensor modality individually and considers their suitability for real-time stress detection. Finally, we present an study of the most discriminative features for stress detection.
【3】Forecasting the Past: Gradient-Based Distribution Shift Detection in Trajectory Prediction
标题：预测过去：轨迹预测中基于对象的分布漂移检测
链接：https://arxiv.org/abs/2604.12425
备注：Accepted at CVPRW SAIAD 2026
摘要：由于训练和测试条件之间的分布变化，轨迹预测模型在现实世界的自动驾驶中经常失败。这种分布变化，无论是行为还是环境，都造成了严重的风险，导致模型在不熟悉的情况下做出错误的预测。我们提出了一种自监督方法，该方法以事后方式训练解码器执行自监督任务，即根据前半部分预测观察到的轨迹的后半部分。该预测损失相对于解码器的最终层的梯度的L2范数定义了用于识别分布偏移的分数。首先，我们的方法不影响轨迹预测模型，确保不干扰原始预测性能，其次，在Shifts和Argoverse数据集上对轨迹预测的分布偏移检测进行了实质性改进。此外，我们表明，这种方法也可以用于早期检测高速公路模拟器中的深度Q网络运动规划器的碰撞。源代码可在https://github.com/Michedev/forecasting-the-past上获得。
摘要：Trajectory prediction models often fail in real-world automated driving due to distributional shifts between training and test conditions. Such distributional shifts, whether behavioural or environmental, pose a critical risk by causing the model to make incorrect forecasts in unfamiliar situations. We propose a self-supervised method that trains a decoder in a post-hoc fashion on the self-supervised task of forecasting the second half of observed trajectories from the first half. The L2 norm of the gradient of this forecasting loss with respect to the decoder's final layer defines a score to identify distribution shifts. Our approach, first, does not affect the trajectory prediction model, ensuring no interference with original prediction performance and second, demonstrates substantial improvements on distribution shift detection for trajectory prediction on the Shifts and Argoverse datasets. Moreover, we show that this method can also be used to early detect collisions of a deep Q-Network motion planner in the Highway simulator. Source code is available at https://github.com/Michedev/forecasting-the-past.
分类|识别(2篇)
【1】Socrates Loss: Unifying Confidence Calibration and Classification by Leveraging the Unknown
标题：苏格拉底损失：通过利用未知因素统一信心校准和分类
链接：https://arxiv.org/abs/2604.12245
备注：Published at TMLR 2026. https://openreview.net/forum?id=DONqw1KhHq Video: https://youtu.be/7WuSkC-aWW8?si=9fgq5ZN7euIyGZGU Code: https://github.com/sandruskyi/SocratesLoss
摘要：尽管深度神经网络具有很高的准确性，但通常表现出较差的置信度校准，限制了其在高风险应用中的可靠性。目前的ad-hoc置信度校准方法试图在训练过程中解决这个问题，但面临一个基本的权衡：两阶段训练方法以训练不稳定和较差的置信度校准为代价实现了强大的分类性能，而单损失方法是稳定的，但在分类中表现不佳。本文解决并缓解了这种稳定性和性能的权衡。我们提出了苏格拉底损失，一种新颖的，统一的损失函数，明确利用不确定性，通过纳入一个辅助未知类，其预测直接影响损失函数和动态不确定性惩罚。这个统一的目标允许模型同时针对分类和置信度校准进行优化，而不会出现复杂的预定损失的不稳定性。我们提供了理论保证，我们的方法规范化的模型，以防止误校准和过拟合。在四个基准数据集和多个架构中，我们的综合实验表明，Socrates Loss始终提高了训练稳定性，同时实现了更有利的准确性-校准权衡，通常比现有方法收敛得更快。
摘要：Deep neural networks, despite their high accuracy, often exhibit poor confidence calibration, limiting their reliability in high-stakes applications. Current ad-hoc confidence calibration methods attempt to fix this during training but face a fundamental trade-off: two-phase training methods achieve strong classification performance at the cost of training instability and poorer confidence calibration, while single-loss methods are stable but underperform in classification. This paper addresses and mitigates this stability-performance trade-off. We propose Socrates Loss, a novel, unified loss function that explicitly leverages uncertainty by incorporating an auxiliary unknown class, whose predictions directly influence the loss function and a dynamic uncertainty penalty. This unified objective allows the model to be optimized for both classification and confidence calibration simultaneously, without the instability of complex, scheduled losses. We provide theoretical guarantees that our method regularizes the model to prevent miscalibration and overfitting. Across four benchmark datasets and multiple architectures, our comprehensive experiments demonstrate that Socrates Loss consistently improves training stability while achieving more favorable accuracy-calibration trade-off, often converging faster than existing methods.
【2】Classification of Epileptic iEEG using Topological Machine Learning
标题：使用Topical机器学习进行癫痫i脑电分类
链接：https://arxiv.org/abs/2604.11971
摘要：由于神经活动的高维性和非线性、潜在随机动力学，从EEG信号中检测癫痫发作仍然具有挑战性。在这项工作中，我们调查是否来自拓扑数据分析（TDA）的功能可以提高癫痫患者使用多通道数据的发作前，发作和发作间期iEEG记录的大脑状态的分类。我们分析了来自55名患者的数据，显著大于许多依赖于患者特异性模型的先前研究。从iEEG信号的持久性图矢量化使用几个TDA表示，包括卡尔森坐标，持久性图像，和模板函数。为了了解拓扑表示如何与现代机器学习管道相互作用，我们在多个iEEG频带、降维技术、特征表示和分类器架构上进行了大规模消融研究。实验结果表明，降维拓扑表示的三类分类准确率达到80%以上.有趣的是，经典机器学习模型的性能优于深度学习模型，达到了79.17%的平衡准确率，这表明精心设计的拓扑特征可以大大降低模型复杂性要求。相比之下，由于高维特征空间，保留完整多通道特征结构的管道表现出严重的过拟合。这些发现突出了在将基于拓扑的表示应用于多通道神经数据时，结构保持降维的重要性。
摘要：Epileptic seizure detection from EEG signals remains challenging due to the high dimensionality and nonlinear, potentially stochastic, dynamics of neural activity. In this work, we investigate whether features derived from topological data analysis (TDA) can improve the classification of brain states in preictal, ictal and interictal iEEG recordings from epilepsy patients using multichannel data. We analyze data from 55 patients, significantly larger than many previous studies that rely on patient-specific models. Persistence diagrams derived from iEEG signals are vectorized using several TDA representations, including Carlsson coordinates, persistence images, and template functions. To understand how topological representations interact with modern machine learning pipelines, we conduct a large-scale ablation study across multiple iEEG frequency bands, dimensionality reduction techniques, feature representations, and classifier architectures. Our experiments show that dimension-reduced topological representations achieve up to 80\% balanced accuracy for three-class classification. Interestingly, classical machine learning models perform comparably to deep learning models, achieving up to 79.17\% balanced accuracy, suggesting that carefully designed topological features can substantially reduce model complexity requirements. In contrast, pipelines preserving the full multichannel feature structure exhibit severe overfitting due to the high-dimensional feature space. These findings highlight the importance of structure-preserving dimensionality reduction when applying topology-based representations to multichannel neural data.
编码器(1篇)
【1】Token Encoding for Semantic Recovery
标题：用于语义恢复的令牌编码
链接：https://arxiv.org/abs/2604.12931
摘要：基于令牌的语义通信在未来的无线网络中有很大的应用前景，因为它可以在非常有限的信道容量下压缩语义令牌。然而，恶劣的无线信道往往会导致令牌丢失，导致严重的失真，防止可靠的语义恢复在接收器。在这篇文章中，我们提出了一个令牌编码框架，用于健壮的语义恢复（TokCode），它不会产生额外的传输开销，并支持即插即用部署。为了有效地优化令牌编码器，我们开发了一种基于语义指导的基础模型自适应算法（SFMA），避免了昂贵的端到端训练。根据基于令牌的生成图像传输的模拟结果，TokCode减轻了语义失真，即使在40%到60%的令牌随机丢失的恶劣信道下，也可以接近性能上限。
摘要：Token-based semantic communication is promising for future wireless networks, as it can compact semantic tokens under very limited channel capacity. However, harsh wireless channels often cause missing tokens, leading to severe distortion that prevents reliable semantic recovery at the receiver. In this article, we propose a token encoding framework for robust semantic recovery (TokCode), which incurs no additional transmission overhead and supports plug-and-play deployment. For efficient token encoder optimization, we develop a sentence-semantic-guided foundation model adaptation algorithm (SFMA) that avoids costly end-to-end training. Based on simulation results on prompt-based generative image transmission, TokCode mitigates semantic distortion and can approach the performance upper-bound, even under harsh channels where 40% to 60% of tokens are randomly lost.
优化|敛散性(7篇)
【1】Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations
标题：优化方法的演变：算法、场景和评估
链接：https://arxiv.org/abs/2604.12968
摘要 ：平衡收敛速度、泛化能力和计算效率仍然是深度学习优化的核心挑战。一阶梯度下降方法，以随机梯度下降（SGD）和Adam为代表，是现代训练管道的基石。然而，大规模的模型训练，严格的差分隐私要求和分布式学习范式暴露了这些传统方法在隐私保护和存储效率方面的关键限制。为了缓解这些瓶颈，研究人员探索二阶优化技术以超越一阶性能上限，而零阶方法重新出现以缓解大规模训练固有的内存限制。尽管方法学的扩散，该领域缺乏一个统一的基本原则和描绘这些不同的方法的应用场景的内聚框架。在这项工作中，我们回顾性地分析了深度学习优化算法的进化轨迹，并在不同的模型架构和训练场景中对主流优化器进行了全面的实证评估。我们提炼出关键的新兴趋势和基本的设计权衡，为未来的研究指明了有希望的方向。通过将理论见解与广泛的经验证据相结合，我们为设计下一代高效，稳健和值得信赖的优化方法提供了可操作的指导。该代码可在https://github.com/APRIL-AIGC/Awesome-Optimizer上获得。
摘要：Balancing convergence speed, generalization capability, and computational efficiency remains a core challenge in deep learning optimization. First-order gradient descent methods, epitomized by stochastic gradient descent (SGD) and Adam, serve as the cornerstone of modern training pipelines. However, large-scale model training, stringent differential privacy requirements, and distributed learning paradigms expose critical limitations in these conventional approaches regarding privacy protection and memory efficiency. To mitigate these bottlenecks, researchers explore second-order optimization techniques to surpass first-order performance ceilings, while zeroth-order methods reemerge to alleviate memory constraints inherent to large-scale training. Despite this proliferation of methodologies, the field lacks a cohesive framework that unifies underlying principles and delineates application scenarios for these disparate approaches. In this work, we retrospectively analyze the evolutionary trajectory of deep learning optimization algorithms and present a comprehensive empirical evaluation of mainstream optimizers across diverse model architectures and training scenarios. We distill key emerging trends and fundamental design trade-offs, pinpointing promising directions for future research. By synthesizing theoretical insights with extensive empirical evidence, we provide actionable guidance for designing next-generation highly efficient, robust, and trustworthy optimization methods. The code is available at https://github.com/APRIL-AIGC/Awesome-Optimizer.
【2】An Optimal Sauer Lemma Over $k$-ary Alphabets
标题：$k$-ary字母上的最佳绍尔引理
链接：https://arxiv.org/abs/2604.12952
备注：38 pages
摘要：Sauer-Shelah-Perles引理是组合学和学习理论的基石，它根据其Vapnik-Chervonenkis（VC）维度限制了二元假设类的大小。对于一个$k$-ary字母表上的函数类，即多类设置，Natarajan维数长期以来一直作为VC维数的模拟，但相应的绍尔型界限是次优的字母表大小$k>2$。 在这项工作中，我们建立了一个尖锐的Sauer不等式的多类和列表预测。我们的界限是表示在Daniely-Shalev-Shwartz（DS）维，更一般地与其扩展，列表DS维-组合参数的特点，多类和列表PAC学习。我们的界限是紧的每个字母表大小$k$，列表大小$\ell$，和维值，取代指数依赖于$\ell$在Natarajan为基础的界限的最佳多项式的依赖，并改善对$k$的依赖。我们的证明使用多项式方法。与经典VC的情况下，其中几个直接组合证明是已知的，我们不知道任何纯粹的组合证明在DS设置。这激发了未来研究的几个方向，这在本文中进行了讨论。 作为结果，我们得到了改进的样本复杂度上界列表PAC学习和列表预测的一致收敛，锐化Charikar等人的最近结果。(STOC~2023），Hanneke等人~ (COLT~2024）和Brukhim等人~（NeurIPS~2024）。
摘要：The Sauer-Shelah-Perles Lemma is a cornerstone of combinatorics and learning theory, bounding the size of a binary hypothesis class in terms of its Vapnik-Chervonenkis (VC) dimension. For classes of functions over a $k$-ary alphabet, namely the multiclass setting, the Natarajan dimension has long served as an analogue of VC dimension, yet the corresponding Sauer-type bounds are suboptimal for alphabet sizes $k>2$. In this work, we establish a sharp Sauer inequality for multiclass and list prediction. Our bound is expressed in terms of the Daniely--Shalev-Shwartz (DS) dimension, and more generally with its extension, the list-DS dimension -- the combinatorial parameters that characterize multiclass and list PAC learnability. Our bound is tight for every alphabet size $k$, list size $\ell$, and dimension value, replacing the exponential dependence on $\ell$ in the Natarajan-based bound by the optimal polynomial dependence, and improving the dependence on $k$ as well. Our proof uses the polynomial method. In contrast to the classical VC case, where several direct combinatorial proofs are known, we are not aware of any purely combinatorial proof in the DS setting. This motivates several directions for future research, which are discussed in the paper. As consequences, we obtain improved sample complexity upper bounds for list PAC learning and for uniform convergence of list predictors, sharpening the recent results of Charikar et al.~(STOC~2023), Hanneke et al.~(COLT~2024), and Brukhim et al.~(NeurIPS~2024).
【3】TCL: Enabling Fast and Efficient Cross-Hardware Tensor Program Optimization via Continual Learning
标题：TCL：通过持续学习实现快速高效的跨硬件张量程序优化
链接：https://arxiv.org/abs/2604.12891
备注：introduces TCL framework for cross-hardware tensor program optimization with active learning, Mamba-based cost model, and continual knowledge distillation; includes extensive experiments on CPU and GPU platforms
摘要：深度学习（DL）编译器依赖于成本模型和自动调整来优化目标硬件的张量程序。然而，现有的方法依赖于大型离线数据集，导致高收集成本，并提供跨平台的次优可移植性。在本文中，我们介绍TCL，一种新的高效和可转移的编译器框架，用于跨不同硬件平台的快速张量程序优化，以解决这些挑战。具体而言，TCL基于三个核心使能器构建：（1）RDU Sampler，这是一种数据高效的主动学习策略，通过联合优化代表性、多样性和不确定性，仅选择10%的张量程序，大幅降低数据收集成本，同时保持接近原始模型的准确性;（2）一种新的基于Mamba的成本模型，可以有效地捕获长期进度依赖性，同时实现有利的交易-通过减少的参数化和轻量级序列建模，在预测精度和计算成本之间取得平衡;以及（3）一个持续的知识蒸馏框架，可以在多个硬件平台上有效地逐步传输知识，同时避免参数爆炸以及通常由传统多任务学习引起的数据依赖性问题。大量的实验验证了每个单独的使能器和整体TCL框架的有效性。当在CPU和GPU平台上优化一系列主流DL模型时，与Tenset-MLP相比，TCL平均实现了16.8倍和12.48倍的调优时间，以及1.20倍和1.13倍的推理延迟。
摘要：Deep learning (DL) compilers rely on cost models and auto-tuning to optimize tensor programs for target hardware. However, existing approaches depend on large offline datasets, incurring high collection costs and offering suboptimal transferability across platforms. In this paper, we introduce TCL, a novel efficient and transferable compiler framework for fast tensor program optimization across diverse hardware platforms to address these challenges. Specifically, TCL is built on three core enablers: (1) the RDU Sampler, a data-efficient active learning strategy that selects only 10% of tensor programs by jointly optimizing Representativeness, Diversity, and Uncertainty, substantially reducing data collection costs while maintaining near-original model accuracy; (2) a new Mamba-based cost model that efficiently captures long-range schedule dependencies while achieving a favorable trade-off between prediction accuracy and computational cost through reduced parameterization and lightweight sequence modeling; and (3) a continuous knowledge distillation framework that effectively and progressively transfers knowledge across multiple hardware platforms while avoiding the parameter explosion and data dependency issues typically caused by traditional multi-task learning. Extensive experiments validate the effectiveness of each individual enabler and the holistic TCL framework. When optimizing a range of mainstream DL models on both CPU and GPU platforms, TCL achieves, on average, 16.8x and 12.48x faster tuning time, and 1.20x and 1.13x lower inference latency, respectively, compared to Tenset-MLP.
【4】SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models
标题：SOAR：扩散模型中最佳对齐和细化的自我修正
链接：https://arxiv.org/abs/2604.12617
摘要 ：扩散模型的后训练管道目前有两个阶段：对策展数据的监督微调（SFT）和带有奖励模型的强化学习（RL）。一个基本的鸿沟将他们分开。SFT仅在从正向去噪过程中采样的地面真实状态上优化去噪器;一旦推断偏离这些理想状态，后续去噪依赖于分布外泛化而不是学习校正，表现出与自回归模型相同的暴露偏差，但沿着去噪轨迹而不是令牌序列积累。RL原则上可以解决这种不匹配，但其终端奖励信号是稀疏的，具有信用分配困难，并有奖励黑客的风险。我们提出了SOAR（Self-Correction for Optimal Alignment and Refinement），这是一种弥补这一空白的偏差校正后训练方法。从真实样本开始，SOAR对当前模型执行单个停止梯度卷展，对产生的偏离轨迹状态重新加噪，并监督模型转向原始干净目标。该方法是在政策上，无奖励，并提供密集的每一个时间步的监督，没有信用分配问题。在SD3.5-Medium上，SOAR将GenEval从0.70提高到0.78，OCR从0.64提高到0.67，同时提高了所有基于模型的偏好评分。在受控奖励特定的实验中，SOAR在美学和文本图像对齐任务的最终度量值上都超过了Flow-GRPO，尽管没有获得奖励模型。由于SOAR的基本损失包含了标准的SFT目标，因此它可以直接取代SFT作为预训练后更强大的第一个训练后阶段，同时与后续的RL对齐完全兼容。
摘要：The post-training pipeline for diffusion models currently has two stages: supervised fine-tuning (SFT) on curated data and reinforcement learning (RL) with reward models. A fundamental gap separates them. SFT optimizes the denoiser only on ground-truth states sampled from the forward noising process; once inference deviates from these ideal states, subsequent denoising relies on out-of-distribution generalization rather than learned correction, exhibiting the same exposure bias that afflicts autoregressive models, but accumulated along the denoising trajectory instead of the token sequence. RL can in principle address this mismatch, yet its terminal reward signal is sparse, suffers from credit-assignment difficulty, and risks reward hacking. We propose SOAR (Self-Correction for Optimal Alignment and Refinement), a bias-correction post-training method that fills this gap. Starting from a real sample, SOAR performs a single stop-gradient rollout with the current model, re-noises the resulting off-trajectory state, and supervises the model to steer back toward the original clean target. The method is on-policy, reward-free, and provides dense per-timestep supervision with no credit-assignment problem. On SD3.5-Medium, SOAR improves GenEval from 0.70 to 0.78 and OCR from 0.64 to 0.67 over SFT, while simultaneously raising all model-based preference scores. In controlled reward-specific experiments, SOAR surpasses Flow-GRPO in final metric value on both aesthetic and text-image alignment tasks, despite having no access to a reward model. Since SOAR's base loss subsumes the standard SFT objective, it can directly replace SFT as a stronger first post-training stage after pretraining, while remaining fully compatible with subsequent RL alignment.
【5】Robust Optimization for Mitigating Reward Hacking with Correlated Proxies
标题：通过相关代理缓解奖励黑客的鲁棒优化
链接：https://arxiv.org/abs/2604.12086
备注：ICLR 2026
摘要：在存在不完美奖励信号的情况下设计鲁棒的强化学习（RL）代理仍然是一个核心挑战。在实践中，代理人经常接受代理奖励的训练，这些奖励只接近真实目标，使他们容易受到奖励黑客的攻击，其中高代理回报来自无意或剥削行为。最近的工作正式使用代理和真实奖励之间的r相关性来解决这个问题，但是现有的方法，如占用正则化策略优化（ORPO），针对固定代理进行优化，并且不能针对更广泛的相关代理类别提供强有力的保证。在这项工作中，我们制定奖励黑客作为一个强大的政策优化问题，在所有r相关的代理奖励的空间。我们推导出一个易于处理的最大-最小公式，在最坏情况下的代理一致的相关约束下，代理最大限度地提高性能。我们进一步表明，当奖励是已知特征的线性函数时，我们的方法可以适应于结合这种先验知识，从而产生改进的策略和可解释的最坏情况奖励。在几个环境中的实验表明，我们的算法始终优于ORPO在最坏情况下的回报，并提供改进的鲁棒性和稳定性在不同级别的代理真实的奖励相关性。这些结果表明，我们的方法提供了鲁棒性和透明度的设置奖励设计是固有的不确定性。该代码可在https://github.com/ZixuanLiu4869/reward_hacking上获得。
摘要：Designing robust reinforcement learning (RL) agents in the presence of imperfect reward signals remains a core challenge. In practice, agents are often trained with proxy rewards that only approximate the true objective, leaving them vulnerable to reward hacking, where high proxy returns arise from unintended or exploitative behaviors. Recent work formalizes this issue using r-correlation between proxy and true rewards, but existing methods like occupancy-regularized policy optimization (ORPO) optimize against a fixed proxy and do not provide strong guarantees against broader classes of correlated proxies. In this work, we formulate reward hacking as a robust policy optimization problem over the space of all r-correlated proxy rewards. We derive a tractable max-min formulation, where the agent maximizes performance under the worst-case proxy consistent with the correlation constraint. We further show that when the reward is a linear function of known features, our approach can be adapted to incorporate this prior knowledge, yielding both improved policies and interpretable worst-case rewards. Experiments across several environments show that our algorithms consistently outperform ORPO in worst-case returns, and offer improved robustness and stability across different levels of proxy-true reward correlation. These results show that our approach provides both robustness and transparency in settings where reward design is inherently uncertain. The code is available at https://github.com/ZixuanLiu4869/reward_hacking.
【6】Classical and Quantum Speedups for Non-Convex Optimization via Energy Conserving Descent
标题：通过能量保存下降实现非凸优化的经典和量子加速
链接：https://arxiv.org/abs/2604.13022
备注：33 pages, 2 figures
摘要：最近提出了能量守恒下降（ECD）算法（De Luca & Silverstein，2022）作为全局非凸优化方法。与梯度下降不同，适当配置的ECD动态逃脱严格的局部最小值并收敛到全局最小值，使其对机器学习优化具有吸引力。 我们提出了第一个分析研究的ECD，专注于一维设置这第一批。我们形式化的随机ECD动力学（sECD）与能量保持噪声，以及量子模拟的ECD哈密顿量（qECD），通过哈密顿模拟量子算法提供了基础。 对于正双阱目标，我们计算从局部到全局最小值的期望命中时间。我们证明了sECD和qECD在各自的梯度下降基线-随机梯度下降及其量化上产生指数加速。对于具有高障碍的目标，qECD比sECD实现了进一步的加速。
摘要：The Energy Conserving Descent (ECD) algorithm was recently proposed (De Luca & Silverstein, 2022) as a global non-convex optimization method. Unlike gradient descent, appropriately configured ECD dynamics escape strict local minima and converge to a global minimum, making it appealing for machine learning optimization. We present the first analytical study of ECD, focusing on the one-dimensional setting for this first installment. We formalize a stochastic ECD dynamics (sECD) with energy-preserving noise, as well as a quantum analog of the ECD Hamiltonian (qECD), providing the foundation for a quantum algorithm through Hamiltonian simulation. For positive double-well objectives, we compute the expected hitting time from a local to the global minimum. We prove that both sECD and qECD yield exponential speedup over respective gradient descent baselines--stochastic gradient descent and its quantization. For objectives with tall barriers, qECD achieves a further speedup over sECD.
【7】Obtaining Partition Crossover masks using Statistical Linkage Learning for solving noised optimization problems with hidden variable dependency structure
标题：使用统计联动学习获得分区交叉掩蔽以解决具有隐藏变量依赖结构的有噪优化问题
链接：https://arxiv.org/abs/2604.11862
摘要：在优化问题中，某些变量子集可能对函数值产生联合非线性或非单调影响。因此，变量依赖关系的知识对于有效的优化可能是至关重要的，并且许多最先进的优化器利用它来提高性能。然而，一些现实世界的问题实例可能是各种来源的噪声的主题。在这种情况下，与优化相关的变量依赖性可能很难或不可能使用足够用于无噪声问题的依赖性检查来区分，从而使得高效的算子，例如，Partition Crossover（PX），没用。因此，我们使用统计链接学习（SLL）分解问题的噪声，并提出了一个新的SLL专用的掩模构造算法。我们证明，如果基于SLL的分解的质量是足够高的，所提出的聚类算法产生的面具相当于PX面具的无噪声的情况下。实验表明，使用所提出的机制的优化器仍然是同样有效的，尽管噪声水平和性能优于国家的最先进的优化器的问题与高噪声。
摘要 ：In optimization problems, some variable subsets may have a joint non-linear or non-monotonical influence on the function value. Therefore, knowledge of variable dependencies may be crucial for effective optimization, and many state-of-the-art optimizers leverage it to improve performance. However, some real-world problem instances may be the subject of noise of various origins. In such a case, variable dependencies relevant to optimization may be hard or impossible to tell using dependency checks sufficient for problems without noise, making highly effective operators, e.g., Partition Crossover (PX), useless. Therefore, we use Statistical Linkage Learning (SLL) to decompose problems with noise and propose a new SLL-dedicated mask construction algorithm. We prove that if the quality of the SLL-based decomposition is sufficiently high, the proposed clustering algorithm yields masks equivalent to PX masks for the noise-free instances. The experiments show that the optimizer using the proposed mechanisms remains equally effective despite the noise level and outperforms state-of-the-art optimizers for the problems with high noise.
预测|估计(8篇)
【1】Frequency-aware Decomposition Learning for Sensorless Wrench Forecasting on a Vibration-rich Hydraulic Manipulator
标题：富振动液压机械手无传感器扳手预测的频率感知分解学习
链接：https://arxiv.org/abs/2604.12905
备注：11 pages, 6 figures, submitted to IEEE/ASME Transactions on Mechatronics
摘要：力和扭矩（F/T）传感对于机器人与环境的交互至关重要，但物理F/T传感器在尺寸、成本和脆弱性方面存在限制。为了减轻这一点，最近的研究估计力/扳手传感器从机器人内部状态。虽然现有的方法通常针对相对较慢的交互，但涉及快速交互的任务（例如研磨）可能会引起关键任务的高频振动，并且在这种机器人设置中的估计仍然未得到充分探索。为了解决这一差距，我们提出了一个频率感知分解网络（FDN）的短期预测振动丰富的扳手从本体感受的历史。FDN预测频谱分解扳手与不对称的确定性和概率头，建模的高频残差作为一个学习的条件分布。它还结合了频率感知，以自适应地增强输入频谱与学习滤波和施加频带之前的输出。我们在一个大规模的开源机器人数据集上预训练FDN，并将学习到的本体感觉到扳手的表示转移到下游。在来自6自由度液压机械手的真实研磨挖掘数据和延迟估计设置下，FDN在高频带中优于基线估计器和预测器，并在低频带中保持竞争力。迁移学习提供了额外的收益，表明大规模预训练和迁移学习对机器人扳手估计的潜力。代码和数据将在验收后提供。
摘要：Force and torque (F/T) sensing is critical for robot-environment interaction, but physical F/T sensors impose constraints in size, cost, and fragility. To mitigate this, recent studies have estimated force/wrench sensorlessly from robot internal states. While existing methods generally target relatively slow interactions, tasks involving rapid interactions, such as grinding, can induce task-critical high-frequency vibrations, and estimation in such robotic settings remains underexplored. To address this gap, we propose a Frequency-aware Decomposition Network (FDN) for short-term forecasting of vibration-rich wrench from proprioceptive history. FDN predicts spectrally decomposed wrench with asymmetric deterministic and probabilistic heads, modeling the high-frequency residual as a learned conditional distribution. It further incorporates frequency-awareness to adaptively enhance input spectra with learned filtering and impose a frequency-band prior on the outputs. We pretrain FDN on a large-scale open-source robot dataset and transfer the learned proprioception-to-wrench representation to the downstream. On real-world grinding excavation data from a 6-DoF hydraulic manipulator and under a delayed estimation setting, FDN outperforms baseline estimators and forecasters in the high-frequency band and remains competitive in the low-frequency band. Transfer learning provides additional gains, suggesting the potential of large-scale pretraining and transfer learning for robotic wrench estimation. Code and data will be made available upon acceptance.
【2】Do VLMs Truly "Read" Candlesticks? A Multi-Scale Benchmark for Visual Stock Price Forecasting
链接：https://arxiv.org/abs/2604.12659
备注：We evaluate whether VLMs can comprehend multi-scale visual stock price data like human analysts with a proposed benchmark, identifying current VLMs' weak predictive power, significant biases, and limited sensitivity to forecast horizons and prompts
摘要：视觉语言模型（VLM）越来越多地应用于可视化股票价格预测，但现有的基准不足以评估他们的股票价格在蜡烛图的理解。首先，先前的研究未能隔离VLM对视觉输入的理解真正提高了预测性能，以及VLM是否真正理解烛台模式。此外，大多数现有的数据集和评估设置都是围绕单周期或表格输入设计的。然而，人类分析师强烈依赖多尺度蜡烛图，其中较长期的视野捕捉趋势方向，而较短期的视野提供拐点的线索，这使得很难系统地评估VLM整合短期和长期视觉市场动态的能力。为了弥补这一差距，我们构建了一个多尺度蜡烛图数据集和一个标准化的评估框架，以评估VLM利用多尺度视觉市场信号的能力。评估将基于混淆矩阵的诊断与信息系数（IC）时间序列度量相结合，并将XGBoost作为基于特征的时间基线。使用该数据集，我们对代表性的VLM进行了基准测试，并分析了它们利用多尺度股票价格数据的能力。实验结果表明，大多数VLMs表现良好，只有在持续的上升趋势或下降趋势的条件下，而表现出较弱的预测能力，在更常见的市场场景。我们还确定了显着的预测偏差和有限的敏感性，明确规定的预测范围的提示，表明在精确的时间推理固有的局限性。
摘要：Vision-language models(VLMs) are increasingly applied to visual stock price forecasting, yet existing benchmarks inadequately evaluate their understanding of stock price in candlestick charts. First, prior studies fail to isolate VLMs' comprehension of visual inputs genuinely improves predictive performance and whether VLMs truly comprehend candlestick patterns. Further, most existing datasets and evaluation setups are designed around single-period or tabular inputs. However, human analysts strongly rely on multi-scale candlestick charts, where longer-term horizons capture trend direction and shorter-term horizons provide cues for inflection points, making it difficult to systematically assess VLMs' ability to integrate short-term and long-term visual market dynamics. To bridge this gap, we construct a multi-scale candlestick charts dataset and a standardized evaluation framework to assess VLMs' ability to utilize multi-scale visual market signals. Evaluation combines confusion-matrix-based diagnostics with information coefficient(IC) time series metrics and includes XGBoost as a feature-based temporal baseline. Using this dataset, we benchmark representative VLMs and analyze their ability to leverage multi-scale stock price data. Experimental results show that most VLMs perform well only under persistent uptrend or downtrend conditions, while exhibiting weak predictive capability in more common market scenarios. We also identify significant prediction biases and limited sensitivity to explicitly specified forecast horizons in prompts, indicating inherent limitations in precise temporal reasoning.
【3】Beyond Weather Correlation: A Comparative Study of Static and Temporal Neural Architectures for Fine-Grained Residential Energy Consumption Forecasting in Melbourne, Australia
标题：超越天气相关性：澳大利亚墨尔本细粒度住宅能源消耗预测的静态和时间神经结构比较研究
链接：https://arxiv.org/abs/2604.12304
备注：22 pages, 6 figures. Earlier preprint versions: Zenodo https://doi.org/10.5281/zenodo.19158396; SSRN https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6453198
摘要：以亚小时分辨率进行准确的短期住宅能源消耗预测对于智能电网管理、需求响应计划和可再生能源集成至关重要。虽然天气变量被广泛认为是住宅电力需求的主要驱动因素，但将时间自相关-过去消费的顺序记忆-结合起来的相对优点，仅在静态气象特征上，澳大利亚家庭在细粒度（5分钟）时间分辨率上仍然没有充分利用。本文对多层感知器（MLP）和长短期记忆（LSTM）递归网络进行了严格的实证比较，这些网络应用于两个真实的墨尔本家庭：3号住宅（标准并网住宅）和4号住宅（屋顶太阳能光伏集成住宅）。这两个模型都是基于14个月的5分钟间隔智能电表数据（2023年3月至2024年4月）与官方气象局（BOM）的每日天气观测数据进行训练的，每个家庭产生超过117，000个样本。在24步（2小时）滑动消费窗口上运行的LSTM实现了R^2 = 0.883（House 3）和R^2 = 0.865（House 4）的决定系数，而相应的天气驱动MLP的R^2 = -0.055和R^2 = 0.410-差异为93.8和45.5个百分点。这些结果表明，消费序列的时间自相关性在5分钟粒度的短期预报中占主导地位的气象信息。此外，我们证明了太阳能发电引入的不对称性：对于光伏集成家庭，MLP达到R^2 = 0.410，揭示了天气-时间相关性的隐含太阳能预测。持续性基线分析和季节分层使模型性能情境化。我们提出了一个混合天气增强的LSTM和联邦学习扩展作为未来工作的方向。
摘要 ：Accurate short-term residential energy consumption forecasting at sub-hourly resolution is critical for smart grid management, demand response programmes, and renewable energy integration. While weather variables are widely acknowledged as key drivers of residential electricity demand, the relative merit of incorporating temporal autocorrelation - the sequential memory of past consumption; over static meteorological features alone remains underexplored at fine-grained (5-minute) temporal resolution for Australian households. This paper presents a rigorous empirical comparison of a Multilayer Perceptron (MLP) and a Long Short-Term Memory (LSTM) recurrent network applied to two real-world Melbourne households: House 3 (a standard grid-connected dwelling) and House 4 (a rooftop solar photovoltaic-integrated household). Both models are trained on 14 months of 5-minute interval smart meter data (March 2023-April 2024) merged with official Bureau of Meteorology (BOM) daily weather observations, yielding over 117,000 samples per household. The LSTM, operating on 24-step (2-hour) sliding consumption windows, achieves coefficients of determination of R^2 = 0.883 (House 3) and R^2 = 0.865 (House 4), compared to R^2 = -0.055 and R^2 = 0.410 for the corresponding weather-driven MLPs - differences of 93.8 and 45.5 percentage points. These results establish that temporal autocorrelation in the consumption sequence dominates meteorological information for short-term forecasting at 5-minute granularity. Additionally, we demonstrate an asymmetry introduced by solar generation: for the PV-integrated household, the MLP achieves R^2 = 0.410, revealing implicit solar forecasting from weather-time correlations. A persistence baseline analysis and seasonal stratification contextualise model performance. We propose a hybrid weather-augmented LSTM and federated learning extensions as directions for future work.
【4】CycloneMAE: A Scalable Multi-Task Learning Model for Global Tropical Cyclone Probabilistic Forecasting
标题：CycloneMAE：全球热带气旋概率预测的可扩展多任务学习模型
链接：https://arxiv.org/abs/2604.12180
摘要：热带气旋（TC）是最具破坏性的自然灾害之一，但其预测面临着根本的权衡：数值天气预报（NWP）模型在计算上是禁止的，并且难以利用历史数据，而现有的基于深度学习（DL）的智能模型是变量特定的和确定性的，无法在不同的预测变量之间进行推广。在这里，我们提出了CycloneMAE，一个可扩展的多任务预测模型，使用TC结构感知的掩码自动编码器从多模态数据中学习可转移的TC表示。通过将离散概率网格机制与预训练/微调范例相结合，CycloneMAE同时提供确定性预测和概率分布。在全球五个海洋盆地进行评估，CycloneMAE在长达120小时的压力和风力预报以及长达24小时的轨迹预报方面优于领先的NWP系统。通过综合梯度的归因分析揭示了物理上可解释的学习动态：短期预测主要依赖于卫星图像的内部核心对流结构，而长期预测逐渐将注意力转移到外部环境因素。我们的框架建立了一个可扩展的，概率和可解释的路径操作TC预测。
摘要：Tropical cyclones (TCs) rank among the most destructive natural hazards, yet their forecasting faces fundamental trade-offs: numerical weather prediction (NWP) models are computationally prohibitive and struggle to leverage historical data, while existing deep learning (DL)-based intelligent models are variable-specific and deterministic, which fail to generalize across different forecasting variables. Here we present CycloneMAE, a scalable multi-task forecasting model that learns transferable TC representations from multi-modal data using a TC structure-aware masked autoencoder. By coupling a discrete probabilistic gridding mechanism with a pre-train/fine-tune paradigm, CycloneMAE simultaneously delivers deterministic forecasts and probability distributions. Evaluated across five global ocean basins, CycloneMAE outperforms leading NWP systems in pressure and wind forecasting up to 120 hours and in track forecasting up to 24 hours. Attribution analysis via integrated gradients reveals physically interpretable learning dynamics: short-term forecasts rely predominantly on the internal core convective structure from satellite imagery, whereas longer-term forecasts progressively shift attention to external environmental factors. Our framework establishes a scalable, probabilistic, and interpretable pathway for operational TC forecasting.
【5】Parametric Interpolation of Dynamic Mode Decomposition for Predicting Nonlinear Systems
标题：预测非线性系统的动态模式分解参数插值
链接：https://arxiv.org/abs/2604.12103
备注：22 pages, 9 figures
摘要：我们提出了参数插值动态模式分解（piDMD），参数降阶建模框架，直接嵌入已知的参数仿射结构到DMD回归步骤。与现有的参数DMD方法不同，这些方法插值模式，特征值或约简算子，并且在稀疏训练数据或多维参数空间中可能很脆弱，piDMD在多个训练参数样本中学习单个参数仿射Koopman代理降阶模型（ROM），并在没有重新训练的情况下预测看不见的参数值。我们验证piDMD流体流动通过一个圆柱体，电子束振荡在横向磁场中，和虚拟阴极振荡-后两个使用电磁粒子在细胞（EMPIC）方法进行模拟。在所有基准测试中，piDMD实现了准确的长期预测，并在最先进的基于插值的参数DMD基线上提高了鲁棒性，训练样本更少，参数空间更多维。
摘要：We present parameter-interpolated dynamic mode decomposition (piDMD), a parametric reduced-order modeling framework that embeds known parameter-affine structure directly into the DMD regression step. Unlike existing parametric DMD methods which interpolate modes, eigenvalues, or reduced operators and can be fragile with sparse training data or multi-dimensional parameter spaces, piDMD learns a single parameter-affine Koopman surrogate reduced order model (ROM) across multiple training parameter samples and predicts at unseen parameter values without retraining. We validate piDMD on fluid flow past a cylinder, electron beam oscillations in transverse magnetic fields, and virtual cathode oscillations -- the latter two being simulated using an electromagnetic particle-in-cell (EMPIC) method. Across all benchmarks, piDMD achieves accurate long-horizon predictions and improved robustness over state-of-the-art interpolation-based parametric DMD baselines, with less training samples and with multi-dimensional parameter spaces.
【6】TriFit: Trimodal Fusion with Protein Dynamics for Mutation Fitness Prediction
标题：TriFit：与蛋白质动力学的三峰融合用于突变适合度预测
链接：https://arxiv.org/abs/2604.12026
摘要：预测单个氨基酸取代（SAVs）的功能影响对于理解遗传疾病和工程治疗蛋白质至关重要。虽然蛋白质语言模型和基于结构的方法在这项任务上取得了很好的性能，但它们系统地忽略了蛋白质动力学;残基灵活性，相关运动和变构偶联是结构生物学中突变耐受性的公认决定因素，但尚未纳入监督变体效应预测。我们提出了TriFit，一个多模态的框架，集成了序列，结构和蛋白质动力学，通过四个专家混合专家（MoE）融合模块与三模态交叉模态对比学习。序列嵌入是通过ESM-2（650 M）的掩蔽边缘评分提取的;结构嵌入来自AlphaFold 2预测的C-alpha几何结构;动态嵌入来自高斯网络模型（GNM）B因子，模态形状和残差-残差互相关。MoE路由器根据输入自适应地对模态组合进行加权，从而在没有固定模态假设的情况下实现蛋白质特异性融合。在ProteinGym替代基准（217次DMS检测，696 k SAV）上，TriFit达到AUROC 0.897 +/- 0.0002，优于所有监督基线，包括Kermut（0.864）和ProteinNPT（0.844），以及最佳zero-shot模型ESM 3（0.769）。消融研究证实，动力学提供了最大的边际贡献成对的方式组合，TriFit实现了良好校准的概率输出（ECE = 0.044），而无需事后校正。
摘要 ：Predicting the functional impact of single amino acid substitutions (SAVs) is central to understanding genetic disease and engineering therapeutic proteins. While protein language models and structure-based methods have achieved strong performance on this task, they systematically neglect protein dynamics; residue flexibility, correlated motions, and allosteric coupling are well-established determinants of mutational tolerance in structural biology, yet have not been incorporated into supervised variant effect predictors. We present TriFit, a multimodal framework that integrates sequence, structure, and protein dynamics through a four-expert Mixture-of-Experts (MoE) fusion module with trimodal cross-modal contrastive learning. Sequence embeddings are extracted via masked marginal scoring with ESM-2 (650M); structural embeddings from AlphaFold2-predicted C-alpha geometries; and dynamics embeddings from Gaussian Network Model (GNM) B-factors, mode shapes, and residue-residue cross-correlations. The MoE router adaptively weights modality combinations conditioned on the input, enabling protein-specific fusion without fixed modality assumptions. On the ProteinGym substitution benchmark (217 DMS assays, 696k SAVs), TriFit achieves AUROC 0.897 +/- 0.0002, outperforming all supervised baselines including Kermut (0.864) and ProteinNPT (0.844), and the best zero-shot model ESM3 (0.769). Ablation studies confirm that dynamics provides the largest marginal contribution over pairwise modality combinations, and TriFit achieves well-calibrated probabilistic outputs (ECE = 0.044) without post-hoc correction.
【7】A Geometric Algebra-informed NeRF Framework for Generalizable Wireless Channel Prediction
标题：基于几何代数的NeRF框架用于可推广无线信道预测
链接：https://arxiv.org/abs/2604.11983
备注：It is accepted by IEEE INFOCOM 2026
摘要：在本文中，我们提出了几何代数通知神经辐射场（GAI-NeRF），无线信道预测的一种新的框架，利用几何代数注意力机制来捕获复杂传播环境中的射线-物体相互作用。我们的方法结合了全球令牌表示，从语言和视觉领域的Transformer架构中汲取灵感，以聚合学习的空间电磁特征并增强场景理解。我们确定传统的静态光线跟踪模块的限制，阻碍模型的泛化，并通过一个新的光线跟踪架构来解决这一挑战。这种设计能够在保持计算效率的同时，在不同的无线场景中进行有效的推广。实验结果表明，GAI-NeRF算法将几何代数原理与神经场景表示相结合，在信道预测任务中取得了优异的性能，为下一代无线通信系统提供了一个很有前途的方向。此外，GAI-NeRF在多种无线场景中的性能大大优于现有方法。为了确保全面的评估，我们使用新收集的针对单场景下游任务和泛化测试的真实室内数据集，进一步评估我们的方法，确认其在未知环境中的稳健性能，并建立其在无线信道预测方面的高效率。
摘要：In this paper, we propose the geometric algebra-informed neural radiance fields (GAI-NeRF), a novel framework for wireless channel prediction that leverages geometric algebra attention mechanisms to capture ray-object interactions in complex propagation environments. Our approach incorporates global token representations, drawing inspiration from transformer architectures in language and vision domains, to aggregate learned spatial-electromagnetic features and enhance scene understanding. We identify limitations in conventional static ray tracing modules that hinder model generalization and address this challenge through a new ray tracing architecture. This design enables effective generalization across diverse wireless scenarios while maintaining computational efficiency. Experimental results demonstrate that GAI-NeRF achieves superior performance in channel prediction tasks by combining geometric algebra principles with neural scene representations, offering a promising direction for next-generation wireless communication systems. Moreover, GAI-NeRF greatly outperforms existing methods across multiple wireless scenarios. To ensure comprehensive assessment, we further evaluate our approach against multiple benchmarks using newly collected real-world indoor datasets tailored for single-scene downstream tasks and generalization testing, confirming its robust performance in unseen environments and establishing its high efficacy for wireless channel prediction.
【8】Thermodynamic Liquid Manifold Networks: Physics-Bounded Deep Learning for Solar Forecasting in Autonomous Off-Grid Microgrids
标题：热力学液体总管网络：用于自主离网微电网中太阳能预测的物理界深度学习
链接：https://arxiv.org/abs/2604.11909
摘要：自主离网光伏系统的稳定运行需要尊重大气热力学的太阳能预测算法。当代的深度学习模型一直表现出严重的异常，主要是云瞬变期间严重的时间相位滞后和物理上不可能的夜间发电。为了解决数据驱动建模和确定性天体力学之间的分歧，本研究引入了热力学液体歧管网络。该方法将22个气象和几何变量投影到Koopman线性化的黎曼流形中，以系统地绘制复杂的气候动态。该架构集成了光谱校准单元和乘法热力学阿尔法门。该系统综合了实时大气不透明度与理论晴空边界模型，结构上执行严格的天体几何遵守。这完全中和了幻影夜间生成，同时在快速天气变化期间保持零滞后同步。在一个严重的半干旱气候条件下进行了严格的五年测试，该框架的RMSE为18.31 Wh/m2，Pearson相关系数为0.988。该模型在所有1826个测试日严格保持零幅度夜间误差，并在高频光学瞬变期间显示出低于30分钟的相位响应。这种超轻量化设计包含63，458个可训练参数，为边缘可部署微电网控制器建立了一个强大的，一致的标准。
摘要：The stable operation of autonomous off-grid photovoltaic systems requires solar forecasting algorithms that respect atmospheric thermodynamics. Contemporary deep learning models consistently exhibit critical anomalies, primarily severe temporal phase lags during cloud transients and physically impossible nocturnal power generation. To resolve this divergence between data-driven modeling and deterministic celestial mechanics, this research introduces the Thermodynamic Liquid Manifold Network. The methodology projects 22 meteorological and geometric variables into a Koopman-linearized Riemannian manifold to systematically map complex climatic dynamics. The architecture integrates a Spectral Calibration unit and a multiplicative Thermodynamic Alpha-Gate. This system synthesizes real-time atmospheric opacity with theoretical clear-sky boundary models, structurally enforcing strict celestial geometry compliance. This completely neutralizes phantom nocturnal generation while maintaining zero-lag synchronization during rapid weather shifts. Validated against a rigorous five-year testing horizon in a severe semi-arid climate, the framework achieves an RMSE of 18.31 Wh/m2 and a Pearson correlation of 0.988. The model strictly maintains a zero-magnitude nocturnal error across all 1826 testing days and exhibits a sub-30-minute phase response during high-frequency optical transients. Comprising exactly 63,458 trainable parameters, this ultra-lightweight design establishes a robust, thermodynamically consistent standard for edge-deployable microgrid controllers.
其他神经网络|深度学习|模型|建模(15篇)
【1】Loop Corrections to the Training and Generalization Errors of Random Feature Models
标题：随机特征模型训练和推广误差的循环修正
链接：https://arxiv.org/abs/2604.12827
备注：17 pages, 4 figures
摘要：我们研究了随机特征模型，其中从规定的初始化集合中采样的神经网络被冻结并用作随机特征，仅对读出权重进行优化。采用物理学的观点，我们研究了训练，测试和泛化误差超出平均核近似。由于预测器是诱导随机核的非线性泛函，因此整体平均误差不仅依赖于平均核，而且依赖于高阶波动统计。在一个有效的场论框架内，这些有限宽度的贡献自然地表现为环路修正。我们推导出训练、测试和泛化误差的循环校正，获得它们的比例律，并通过实验验证支持理论。
摘要：We investigate random feature models in which neural networks sampled from a prescribed initialization ensemble are frozen and used as random features, with only the readout weights optimized. Adopting a statistical-physics viewpoint, we study the training, test, and generalization errors beyond the mean-kernel approximation. Since the predictor is a nonlinear functional of the induced random kernel, the ensemble-averaged errors depend not only on the mean kernel but also on higher-order fluctuation statistics. Within an effective field-theoretic framework, these finite-width contributions naturally appear as loop corrections. We derive the loop corrections to the training, test, and generalization errors, obtain their scaling laws, and support the theory with experimental verification.
【2】BID-LoRA: A Parameter-Efficient Framework for Continual Learning and Unlearning
标题：BID-LoRA：持续学习和取消学习的参数高效框架
链接：https://arxiv.org/abs/2604.12686
摘要 ：深度学习的最新进展强调了对系统的需求，这些系统不仅可以通过持续学习（CL）获取新知识，还可以通过机器非学习（MU）删除过时的、敏感的或私人的信息。然而，虽然CL方法已经发展得很好，但MU技术仍处于早期阶段，这为依赖这两种能力的统一框架造成了严重的差距。我们发现，天真地将现有的CL和MU方法相结合会导致知识泄漏，即在重复的适应周期中基础知识的逐渐退化。为了解决这个问题，我们将持续学习/遗忘（CLU）形式化为一个统一的范式，具有三个关键目标：（i）精确删除不需要的知识，（ii）在保留先验信息的同时有效整合新知识，以及（iii）最大限度地减少跨周期的知识泄漏。我们提出了双向低秩自适应（BID-LoRA），这是一种新的框架，具有三个专用的适配器路径-保留，新的和应用于注意力层的unlearn，结合逃逸unlearning，将遗忘类嵌入推到最大限度地远离保留知识的位置，仅更新5%的参数。CIFAR-100上的实验表明，BID-LoRA在多个适应周期中的性能优于CLU基线。我们进一步评估了CASIA-Face 100，这是一个精心策划的人脸识别子集，展示了对现实世界身份管理系统的实用性，其中新用户必须注册，撤回用户必须删除。
摘要：Recent advances in deep learning underscore the need for systems that can not only acquire new knowledge through Continual Learning (CL) but also remove outdated, sensitive, or private information through Machine Unlearning (MU). However, while CL methods are well-developed, MU techniques remain in early stages, creating a critical gap for unified frameworks that depend on both capabilities. We find that naively combining existing CL and MU approaches results in knowledge leakage a gradual degradation of foundational knowledge across repeated adaptation cycles. To address this, we formalize Continual Learning Unlearning (CLU) as a unified paradigm with three key goals: (i) precise deletion of unwanted knowledge, (ii) efficient integration of new knowledge while preserving prior information, and (iii) minimizing knowledge leakage across cycles. We propose Bi-Directional Low-Rank Adaptation (BID-LoRA), a novel framework featuring three dedicated adapter pathways-retain, new, and unlearn applied to attention layers, combined with escape unlearning that pushes forget-class embeddings to positions maximally distant from retained knowledge, updating only 5% of parameters. Experiments on CIFAR-100 show that BID-LoRA outperforms CLU baselines across multiple adaptation cycles. We further evaluate on CASIA-Face100, a curated face recognition subset, demonstrating practical applicability to real-world identity management systems where new users must be enrolled and withdrawn users removed.
【3】From Imitation to Discrimination: Progressive Curriculum Learning for Robust Web Navigation
标题：从模仿到歧视：渐进式课程学习以实现稳健的网络导航
链接：https://arxiv.org/abs/2604.12666
备注：17 pages, 10 figures
摘要：基于文本的Web代理为自主Web导航提供了计算效率，但由于现实世界HTML的噪声和异构性，开发强大的代理仍然具有挑战性。标准监督微调（SFT）方法在两个关键方面失败：它们缺乏辨别能力，无法拒绝人口密集的页面中看似合理但不正确的元素，并且对看不见的网站布局表现出有限的泛化能力。为了应对这些挑战，我们引入了Triton数据集（590 k实例）和渐进式训练课程。Triton是通过结构-语义硬否定挖掘构建的，该挖掘明确地挖掘拓扑相似的干扰项，以及一个双代理共识管道，该管道通过严格的验证合成了不同的跨域任务。在此基础上，我们的渐进式课程产生了三个模型：Triton-SFT-32 B用于基本模仿，Triton-ORPO-32 B用于通过比值比偏好优化进行稳健区分，Triton-GRPO-32 B用于通过组相对策略优化进行长期一致性。Mind 2 Web上的实证评估表明，Triton-GRPO-32 B在开源模型中实现了最先进的性能，步骤成功率为58.7%，超过GPT-4.5（42.4%）和Claude-4.5（41.4%）超过16%，验证了专门的数据课程超过了Web导航的原始参数规模。
摘要：Text-based web agents offer computational efficiency for autonomous web navigation, yet developing robust agents remains challenging due to the noisy and heterogeneous nature of real-world HTML. Standard Supervised Fine-Tuning (SFT) approaches fail in two critical dimensions: they lack discrimination capabilities to reject plausible but incorrect elements in densely populated pages, and exhibit limited generalization to unseen website layouts. To address these challenges, we introduce the Triton dataset (590k instances) and a progressive training curriculum. Triton is constructed via Structural-Semantic Hard Negative Mining, which explicitly mines topologically similar distractors, and a Dual-Agent Consensus pipeline that synthesizes diverse cross-domain tasks with strict verification. Building upon this foundation, our progressive curriculum produces three models: Triton-SFT-32B for basic imitation, Triton-ORPO-32B for robust discrimination via Odds Ratio Preference Optimization, and Triton-GRPO-32B for long-horizon consistency through Group Relative Policy Optimization. Empirical evaluation on Mind2Web demonstrates that Triton-GRPO-32B achieves state-of-the-art performance among open-source models with 58.7% Step Success Rate, surpassing GPT-4.5 (42.4%) and Claude-4.5 (41.4%) by over 16%, validating that specialized data curriculum outweighs raw parameter scale for web navigation.
【4】KumoRFM-2: Scaling Foundation Models for Relational Learning
标题：KumoRFM-2：关系学习的扩展基础模型
链接：https://arxiv.org/abs/2604.12596
摘要：我们介绍了KumoRFM-2，这是关系数据预训练基础模型的下一次迭代。KumoRFM-2支持上下文学习和微调，适用于各种预测任务。与表格基础模型相比，KumoRFM-2在关系数据上进行原生操作，同时处理一个或多个连接的表，而无需手动平整表或生成目标变量，同时保持时间一致性。KumoRFM-2利用大量的合成和真实数据来跨四个轴进行预训练：单个表级别的行和列维度，以及数据库级别的外键和跨样本维度。与其前身相比，KumoRFM-2尽可能早地注入任务信息，从而能够更清晰地选择任务相关列，并提高对噪声数据的鲁棒性。通过对41个具有挑战性的基准测试和对表现力和敏感性的分析进行广泛的实验，我们证明了KumoRFM-2的性能比监督和基础方法高出8%，同时在冷启动和噪声数据的极端设置下保持了强大的性能。据我们所知，这是Few-Shot基础模型第一次在常见的基准任务上超过监督方法，并且在微调后性能进一步提高。最后，虽然KumoRFM-1仅限于内存中的小规模数据集，但KumoRFM-2可扩展到十亿规模的关系数据集。
摘要：We introduce KumoRFM-2, the next iteration of a pre-trained foundation model for relational data. KumoRFM-2 supports in-context learning as well as fine-tuning and is applicable to a wide range of predictive tasks. In contrast to tabular foundation models, KumoRFM-2 natively operates on relational data, processing one or more connected tables simultaneously without manual table flattening or target variable generation, all while preserving temporal consistency. KumoRFM-2 leverages a large corpus of synthetic and real-world data to pre-train across four axes: the row and column dimensions at the individual table level, and the foreign key and cross-sample dimensions at the database level. In contrast to its predecessor, KumoRFM-2 injects task information as early as possible, enabling sharper selection of task-relevant columns and improved robustness to noisy data. Through extensive experiments on 41 challenging benchmarks and analysis around expressivity and sensitivity, we demonstrate that KumoRFM-2 outperforms supervised and foundational approaches by up to 8%, while maintaining strong performance under extreme settings of cold start and noisy data. To our knowledge, this is the first time a few-shot foundation model has been shown to surpass supervised approaches on common benchmark tasks, with performance further improving upon fine-tuning. Finally, while KumoRFM-1 was limited to small-scale in-memory datasets, KumoRFM-2 scales to billion-scale relational datasets.
【5】EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts
标题：通过双曲混合曲线专家的基于脑电波的多模式学习
链接：https://arxiv.org/abs/2604.12579
摘要：基于脑电图（EEG）的多模态学习将大脑信号与互补模态相结合，以改善精神状态评估，提供了巨大的临床潜力。这种范式的有效性在很大程度上取决于异构模式上的表征学习。对于基于EEG的范例，一种有希望的方法是利用它们的分层结构，因为最近的研究已经表明EEG和相关模态（例如，面部表情）表现出反映复杂认知过程的层次结构。然而，欧几里得嵌入由于其平坦几何而难以表示这些层次结构，而双曲空间具有指数增长特性，自然适合它们。在这项工作中，我们提出了EEG-MoCE，一种新的双曲混合曲率专家框架设计的多模态神经技术。EEG-MoCE将每个模态分配给可学习曲率双曲空间中的专家，从而实现其内在几何形状的自适应建模。一个曲率感知的融合策略，然后动态加权专家，强调模态更丰富的层次信息。在基准数据集上进行的大量实验表明，EEG-MoCE实现了最先进的性能，包括情感识别，睡眠分期和认知评估。
摘要 ：Electroencephalography (EEG)-based multimodal learning integrates brain signals with complementary modalities to improve mental state assessment, providing great clinical potential. The effectiveness of such paradigms largely depends on the representation learning on heterogeneous modalities. For EEG-based paradigms, one promising approach is to leverage their hierarchical structures, as recent studies have shown that both EEG and associated modalities (e.g., facial expressions) exhibit hierarchical structures reflecting complex cognitive processes. However, Euclidean embeddings struggle to represent these hierarchical structures due to their flat geometry, while hyperbolic spaces, with their exponential growth property, are naturally suited for them. In this work, we propose EEG-MoCE, a novel hyperbolic mixture-of-curvature experts framework designed for multimodal neurotechnology. EEG-MoCE assigns each modality to an expert in a learnable-curvature hyperbolic space, enabling adaptive modeling of its intrinsic geometry. A curvature-aware fusion strategy then dynamically weights experts, emphasizing modalities with richer hierarchical information. Extensive experiments on benchmark datasets demonstrate that EEG-MoCE achieves state-of-the-art performance, including emotion recognition, sleep staging, and cognitive assessment.
【6】Models Know Their Shortcuts: Deployment-Time Shortcut Mitigation
标题：模型知道他们的捷径：部署时故障缓解
链接：https://arxiv.org/abs/2604.12277
摘要：预先训练的语言模型通常依赖于表面特征，这些特征在训练过程中表现出预测性，但在测试时却无法概括，这种现象被称为快捷学习。现有的缓解方法通常在训练时操作，并且需要大量的监督，例如访问原始训练数据或捷径类型的先验知识。我们提出了一个部署时框架，它可以在不访问原始训练数据或快捷方式注释的情况下减轻令牌级快捷方式。我们的关键见解是，基于梯度的归因在有偏见的模型上突出了快捷标记。基于这一发现，我们训练了一个轻量级的基于LoRA的去偏置模块，该模块具有掩蔽对比学习（MaskCL）目标，该目标鼓励使用或不使用单个令牌的一致表示。在自然发生和受控快捷方式下的情感分类，毒性检测和自然语言推理中，EQUIPMENT Guarding在分布变化下提高了未缓解模型的整体准确性和最差组准确性，同时保持了分布性能。
摘要：Pretrained language models often rely on superficial features that appear predictive during training yet fail to generalize at test time, a phenomenon known as shortcut learning. Existing mitigation methods generally operate at training time and require heavy supervision such as access to the original training data or prior knowledge of shortcut type. We propose Shortcut Guardrail, a deployment-time framework that mitigates token-level shortcuts without access to the original training data or shortcut annotations. Our key insight is that gradient-based attribution on a biased model highlights shortcut tokens. Building on this finding, we train a lightweight LoRA-based debiasing module with a Masked Contrastive Learning (MaskCL) objective that encourages consistent representations with or without individual tokens. Across sentiment classification, toxicity detection, and natural language inference under both naturally occurring and controlled shortcuts, Shortcut Guardrail improves overall accuracy and worst-group accuracy over the unmitigated model under distribution shifts while preserving in-distribution performance.
【7】Decentralized Learning via Random Walk with Jumps
标题：通过随机步行和跳跃进行去中心化学习
链接：https://arxiv.org/abs/2604.12260
摘要：我们研究网络上的分散学习，其中数据分布在没有中央协调器的节点上。随机游走学习是一种基于令牌的方法，其中单个模型在网络中传播，并使用本地数据在每个访问节点处更新，从而产生低通信和计算开销。在加权随机游走学习中，转移矩阵被设计成实现期望的采样分布，从而在数据异构的情况下加速收敛。我们表明，通过大都会黑斯廷斯算法实现加权采样可能会导致一个以前未探索的现象，我们长期诱捕。随机游走可能会被困在网络的一个小区域内，导致高度相关的更新和严重的收敛性下降。为了解决这个问题，我们提出了大都会黑斯廷斯与利维跳跃，它引入了偶尔的远程过渡，以恢复探索，同时尊重当地的信息约束。我们建立了一个收敛速度，明确地描述了数据异构性，网络谱隙和跳跃概率的作用，并通过实验证明，MHLJ有效地消除了诱捕，并显着加快了分散学习。
摘要：We study decentralized learning over networks where data are distributed across nodes without a central coordinator. Random walk learning is a token-based approach in which a single model is propagated across the network and updated at each visited node using local data, thereby incurring low communication and computational overheads. In weighted random-walk learning, the transition matrix is designed to achieve a desired sampling distribution, thereby speeding up convergence under data heterogeneity. We show that implementing weighted sampling via the Metropolis-Hastings algorithm can lead to a previously unexplored phenomenon we term entrapment. The random walk may become trapped in a small region of the network, resulting in highly correlated updates and severely degraded convergence. To address this issue, we propose Metropolis-Hastings with Levy jumps, which introduces occasional long-range transitions to restore exploration while respecting local information constraints. We establish a convergence rate that explicitly characterizes the roles of data heterogeneity, network spectral gap, and jump probability, and demonstrate through experiments that MHLJ effectively eliminates entrapment and significantly speeds up decentralized learning.
【8】UCS: Estimating Unseen Coverage for Improved In-Context Learning
标题：ucci：估计隐形覆盖率以改善上下文学习
链接：https://arxiv.org/abs/2604.12015
备注：ACL 2026 Findings; 17 pages, 3 figures
摘要：在上下文学习（ICL）的性能关键取决于哪些演示被放置在提示，但大多数现有的选择器优先考虑启发式概念的相关性或多样性，并提供有限的洞察力的覆盖范围内的演示集。我们提出了看不见的覆盖选择（UKS），一个无训练的，子集级的覆盖事先的原则，一个好的示范集应该暴露模型的潜在集群未透露的当前选定的子集。UCS通过（1）从模型一致性嵌入中诱导离散潜在聚类和（2）通过平滑良好图灵估计器从其经验频谱中估计候选子集内未显示聚类的数量来实现这一想法。与以前的选择方法不同，UCS是基于覆盖率的，无需训练，并且可以通过简单的正则化目标与查询相关和查询无关的选择基线无缝结合。使用前沿大型语言模型对多个意图分类和推理基准进行的实验表明，在相同的选择预算下，使用UCS增强强基线可以持续提高ICL准确率高达2-6%，同时还可以深入了解任务级和模型级的潜在聚类分布。代码可在https://github.com/Raina-Xin/UCS上获得。
摘要：In-context learning (ICL) performance depends critically on which demonstrations are placed in the prompt, yet most existing selectors prioritize heuristic notions of relevance or diversity and provide limited insight into the coverage of a demonstration set. We propose Unseen Coverage Selection (UKS), a training-free, subset-level coverage prior motivated by the principle that a good demonstration set should expose the model to latent cluster unrevealed by the currently selected subset. UCS operationalizes this idea by (1) inducing discrete latent clusters from model-consistent embeddings and (2) estimating the number of unrevealed clusters within a candidate subset via a Smoothed Good--Turing estimator from its empirical frequency spectrum. Unlike previous selection methods, UCS is coverage-based and training-free, and can be seamlessly combined with both query-dependent and query-independent selection baselines via a simple regularized objective. Experiments on multiple intent-classification and reasoning benchmarks with frontier Large Language Models show that augmenting strong baselines with UCS consistently improves ICL accuracy by up to 2-6% under the same selection budget, while also yielding insights into task- and model-level latent cluster distributions. Code is available at https://github.com/Raina-Xin/UCS.
【9】The Linear Centroids Hypothesis: How Deep Network Features Represent Data
标题：线性中心假说：深层网络特征如何代表数据
链接：https://arxiv.org/abs/2604.11962
备注：20 pages, 17 figures
摘要：识别和理解深度网络（DN）从其输入中提取以产生其输出的特征是可解释性研究的焦点。线性表示假设（LRH）根据DN的潜在空间中的输入所形成的线性方向来识别特征。然而，LRH是有限的，因为它抽象了单个组件（例如，神经元和层），易于识别伪特征，并且不能跨子组件应用（例如，多层）。在本文中，我们介绍了线性质心假设（LCH）作为一个新的框架，用于确定DN的功能。LCH假设特征对应于质心的线性方向，质心是DN在其输入空间的局部区域中的功能行为的向量概括。LCH下的可解释性研究可以利用现有的LRH工具，如稀疏自动编码器，通过将它们应用于DN的质心，而不是其潜在的激活。我们证明，这样做会为DINO Vision Transformers产生更稀疏的特征字典，这也会在下游任务中表现得更好。LCH还激发了新的可解释性方法;例如，LCH可以很容易地识别GPT 2-Large中的电路。有关研究LCH的代码，请访问https://github.com/ThomasWalker1/LinearCentroidsHypothesis。
摘要：Identifying and understanding the features that a deep network (DN) extracts from its inputs to produce its outputs is a focal point of interpretability research. The Linear Representation Hypothesis (LRH) identifies features in terms of the linear directions formed by the inputs in a DN's latent space. However, the LRH is limited as it abstracts away from individual components (e.g., neurons and layers), is susceptible to identifying spurious features, and cannot be applied across sub-components (e.g., multiple layers). In this paper, we introduce the Linear Centroids Hypothesis (LCH) as a new framework for identifying the features of a DN. The LCH posits that features correspond to linear directions of centroids, which are vector summarizations of the functional behavior of a DN in a local region of its input space. Interpretability studies under the LCH can leverage existing LRH tools, such as sparse autoencoders, by applying them to the DN's centroids rather than to its latent activations. We demonstrate that doing so yields sparser feature dictionaries for DINO vision transformers, which also perform better on downstream tasks. The LCH also inspires novel approaches to interpretability; for example, LCH can readily identify circuits in GPT2-Large. For code to study the LCH https://github.com/ThomasWalker1/LinearCentroidsHypothesis .
【10】ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
标题：ResBM：低带宽管道并行主义的剩余瓶颈模型
链接：https://arxiv.org/abs/2604.11947
摘要：解锁大规模低带宽分散式培训有可能利用其他未开发的计算资源。在集中式设置中，大规模多节点训练主要通过数据和流水线并行实现，这两种技术需要超高带宽通信。虽然现在存在分散式数据并行的有效方法，但流水线并行仍然是主要挑战。最近的努力，如子空间模型（SM），声称高达100倍的激活压缩，但依赖于复杂的约束优化，并偏离真正的端到端训练。在本文中，我们提出了一种不同的方法，从地面设计的架构的基础上是本地的低带宽通信环境，同时仍然适用于任何标准的变压器为基础的架构。我们称这种架构为残差瓶颈模型或ResBM，它引入了一个跨管道边界的残差编码器-解码器瓶颈模块，可以作为模型参数的一部分进行端到端训练，同时保留显式的低秩身份路径。我们表明，ResBM实现了最先进的128倍激活压缩，而不会显著降低收敛速度，也不会产生显著的内存或计算开销。
摘要：Unlocking large-scale low-bandwidth decentralized training has the potential to utilize otherwise untapped compute resources. In centralized settings, large-scale multi-node training is primarily enabled by data and pipeline parallelism, two techniques that require ultra-high-bandwidth communication. While efficient methods now exist for decentralized data parallelism, pipeline parallelism remains the primary challenge. Recent efforts, such as Subspace Models (SM), have claimed up to 100x activation compression but rely on complex constrained optimization and diverge from true end-to-end training. In this paper, we propose a different approach, based on an architecture designed from the ground up to be native to low-bandwidth communication environments while still applicable to any standard transformer-based architecture. We call this architecture the Residual Bottleneck Model or ResBM, it introduces a residual encoder-decoder bottleneck module across pipeline boundaries that can be trained end-to-end as part of the model's parameters while preserving an explicit low-rank identity path. We show that ResBMs achieve state-of-the-art 128x activation compression without significant loss in convergence rates and without significant memory or compute overhead.
【11】Multi-Worker Selection based Distributed Swarm Learning for Edge IoT with Non-i.i.d. Data
标题：基于多工作者选择的非i.i.d边缘物联网分布式群学习数据
链接：https://arxiv.org/abs/2509.18367
摘要：分布式群学习（DSL）的最新进展为边缘物联网提供了一个有前途的范例。这些进步增强了数据隐私、通信效率、节能和模型可扩展性。然而，非独立同分布（non-i.i.d.）数据对多路访问边缘计算提出了重大挑战，降低了学习性能，并使vanilla DSL的训练行为发生了变化。此外，关于数据异质性如何影响模型训练准确性的理论指导仍然缺乏，这需要进行彻底的调查。为了填补这一空白，本文首先通过测量非独立同分布的影响来研究数据的异质性。DSL框架下的数据集。然后，这激发了一个新的多工人选择设计DSL，称为M-DSL算法，它有效地与分布式异构数据。一个新的非身份证。本文引入度度量来描述局部数据集之间的统计差异，将数据异构性度量与DSL性能评估联系起来。通过这种方式，我们的M-DSL指导有效选择为全球模型更新做出突出贡献的多个作品。我们还提供了我们的M-DSL的收敛行为的理论分析，然后在不同的异构数据集和非独立同分布的广泛的实验。数据设置。数值结果验证了我们的M-DSL提供的性能改善和网络智能增强超出基准。
摘要：Recent advances in distributed swarm learning (DSL) offer a promising paradigm for edge Internet of Things. Such advancements enhance data privacy, communication efficiency, energy saving, and model scalability. However, the presence of non-independent and identically distributed (non-i.i.d.) data pose a significant challenge for multi-access edge computing, degrading learning performance and diverging training behavior of vanilla DSL. Further, there still lacks theoretical guidance on how data heterogeneity affects model training accuracy, which requires thorough investigation. To fill the gap, this paper first study the data heterogeneity by measuring the impact of non-i.i.d. datasets under the DSL framework. This then motivates a new multi-worker selection design for DSL, termed M-DSL algorithm, which works effectively with distributed heterogeneous data. A new non-i.i.d. degree metric is introduced and defined in this work to formulate the statistical difference among local datasets, which builds a connection between the measure of data heterogeneity and the evaluation of DSL performance. In this way, our M-DSL guides effective selection of multiple works who make prominent contributions for global model updates. We also provide theoretical analysis on the convergence behavior of our M-DSL, followed by extensive experiments on different heterogeneous datasets and non-i.i.d. data settings. Numerical results verify performance improvement and network intelligence enhancement provided by our M-DSL beyond the benchmarks.
【12】Causal Diffusion Models for Counterfactual Outcome Distributions in Longitudinal Data
标题：纵向数据中反事实结果分布的因果扩散模型
链接：https://arxiv.org/abs/2604.12992
摘要：预测纵向数据中的反事实结果，其中顺序治疗决策在很大程度上取决于不断变化的患者状态，这是至关重要的，但众所周知的挑战，由于复杂的时间依赖性混杂和现有方法中的不确定性量化不足。我们引入了因果扩散模型（CDM），这是第一个去噪扩散概率方法，明确设计用于在顺序干预下生成反事实结果的完整概率分布。CDM采用具有关系自我关注的新型残差去噪架构，捕获复杂的时间依赖性和多模式结果轨迹，而无需显式调整（例如，逆概率加权或对抗平衡）用于混淆。在对先前工作中广泛采用的药代动力学-药效学肿瘤生长模拟器进行严格评估时，CDM始终优于最先进的纵向因果推断方法，在分布准确度（1-Wasserstein距离）方面实现了15-30%的相对改善，同时在高混杂制度下保持了竞争力或优越的点估计准确度（RMSE）。通过在复杂的、顺序混淆的环境中统一不确定性量化和强大的反事实预测，而无需定制的去混淆，CDM为医学、政策评估和其他纵向领域的决策支持提供了一个灵活的、高影响力的工具。
摘要：Predicting counterfactual outcomes in longitudinal data, where sequential treatment decisions heavily depend on evolving patient states, is critical yet notoriously challenging due to complex time-dependent confounding and inadequate uncertainty quantification in existing methods. We introduce the Causal Diffusion Model (CDM), the first denoising diffusion probabilistic approach explicitly designed to generate full probabilistic distributions of counterfactual outcomes under sequential interventions. CDM employs a novel residual denoising architecture with relational self-attention, capturing intricate temporal dependencies and multimodal outcome trajectories without requiring explicit adjustments (e.g., inverse-probability weighting or adversarial balancing) for confounding. In rigorous evaluation on a pharmacokinetic-pharmacodynamic tumor-growth simulator widely adopted in prior work, CDM consistently outperforms state-of-the-art longitudinal causal inference methods, achieving a 15-30% relative improvement in distributional accuracy (1-Wasserstein distance) while maintaining competitive or superior point-estimate accuracy (RMSE) under high-confounding regimes. By unifying uncertainty quantification and robust counterfactual prediction in complex, sequentially confounded settings, without tailored deconfounding, CDM offers a flexible, high-impact tool for decision support in medicine, policy evaluation, and other longitudinal domains.
【13】Machine learning for four-dimensional SU(3) lattice gauge theories
标题：二维SU（3）格点规范理论的机器学习
链接：https://arxiv.org/abs/2604.12416
备注：18 pages, 9 figure; Plenary talk at the 42nd International Symposium on Lattice Field Theory (LATTICE2025), Mumbai, India
摘要：在这篇综述中，我总结了机器学习如何可以用于格点规范理论模拟，以及目前有哪些方法可以改善规范场配置的采样，重点是四维SU（3）规范理论中的应用。这些方法包括基于生成机器学习模型的方法，例如（随机）归一化流和扩散过程，以及基于重正化群（RG）变换的方法，更具体地说，使用规范等变卷积神经网络的RG改进规范动作的机器学习。特别是，我提出了在四维SU（3）规范理论中机器学习的定点作用量向连续极限的标度结果。结果包括基于经典的完美的梯度流尺度，这是免费的树级晶格文物的所有订单，和数量有关的静态电位和decenfinement过渡的观测。
摘要：In this review I summarize how machine learning can be used in lattice gauge theory simulations and what ap\-proaches are currently available to improve the sampling of gauge field configurations, with a focus on applications in four-dimensional SU(3) gauge theories. These include approaches based on generative machine-learning models such as (stochastic) normalizing flows and diffusion processes, and an approach based on renormalization group (RG) transformations, more specifically the machine learning of RG-improved gauge actions using gauge-equivariant convolutional neural networks. In particular, I present scaling results for a machine-learned fixed-point action in four-dimensional SU(3) gauge theory towards the continuum limit. The results include observables based on the classically perfect gradient-flow scales, which are free of tree-level lattice artefacts to all orders, and quantities related to the static potential and the deconfinement transition.
【14】Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions
标题：粒子物理基础模型的跨域转移：从喷射流到中微子相互作用
链接：https://arxiv.org/abs/2604.12364
备注：12 pages, 8 figures
摘要：未来基于人工智能的粒子物理学研究可能会从基础模型开始，以加速训练并提高灵敏度。作为粒子物理学通用基础模型的一步，我们研究了在不同的高Q^2 $模拟和真实$pp$和$ep$碰撞上预训练的OmniLearned基础模型是否可以有效地转移到几个GeV的固定目标中微子实验中。我们处理MINERvA中微子-核散射事件，并评估两种类型任务的预训练模型：可用能量的回归和带电电流π介子最终状态的二进制分类（$\mathrm{CC 1 π^{\pm}}$，$\mathrm{CCNπ^{\pm}}$和$\mathrm{CC 1 π^{0}}$）。预训练的OmniLearned模型始终优于从头开始训练的类似大小的模型，在相同的计算预算下实现更好的整体性能，并在相同的训练步骤数下实现更好的性能。这些结果表明，粒子级基础模型获得归纳偏差，概括在能量尺度，探测器技术和底层物理过程的巨大差异，指向粒子物理学中的探测器不可知推理的范式。
摘要：Future AI-based studies in particle physics will likely start from a foundation model to accelerate training and enhance sensitivity. As a step towards a general-purpose foundation model for particle physics, we investigate whether the OmniLearned foundation model pre-trained on diverse high-$Q^2$ simulated and real $pp$ and $ep$ collisions can be effectively transferred to a few-GeV fixed-target neutrino experiment. We process MINERvA neutrino--nucleus scattering events and evaluate pre-trained models on two types of tasks: regression of available energy and binary classification of charged-current pion final states ($\mathrm{CC1π^{\pm}}$, $\mathrm{CCNπ^{\pm}}$, and $\mathrm{CC1π^{0}}$). Pre-trained OmniLearned models consistently outperform similarly sized models trained from scratch, achieving better overall performance at the same compute budget, as well as achieving better performance at the same number of training steps. These results suggest that particle-level foundation models acquire inductive biases that generalize across large differences in energy scale, detector technology, and underlying physics processes, pointing toward a paradigm of detector-agnostic inference in particle physics.
【15】Training single-electron and single-photon stochastic physical neural networks
标题：训练单电子和单量子随机物理神经网络
链接：https://arxiv.org/abs/2604.10861
备注：15 pages, 8 figures
摘要：深度学习的计算需求激发了对替代计算方法的研究。一种替代方案是物理神经网络（PNN），其中学习和推理直接通过物理过程进行。当底层神经元由随机激活开关的动态实现时，随机PNN出现。在这里，我们提出了新的电子和光子随机神经元。电子实现是通过单电子隧穿量子点来实现的。光子实现是通过一个单光子源驱动的两个模式之一，通过一个可控的分束器样的相互作用耦合。在电子的情况下，量子点的电荷状态形成随机神经元的基础，而在光子的情况下，非驱动模式的占用作为随机神经元的基础。随机PNN的训练是用随机神经元模型以及先前介绍的相干驱动的单光子探测器随机神经元进行的。使用单隐藏层随机PNN研究了MNIST手写数字分类的几种训练策略，包括改变每层中的尝试次数以控制前向传递随机性，以及在后向传递中使用真实概率或经验输出来评估它们的影响梯度估计。我们表明，当经验输出用于反向传递时，网络达到了97%以上的测试精度，每层的试验次数很少。尽管模型结构简单，但在存在高度噪声和模型不确定性的情况下仍保持高测试精度。结果证明了将随机PNN用于深度学习的潜力。
摘要 ：The computational demands of deep learning motivate the investigation of alternative approaches to computation. One alternative is physical neural networks~(PNNs), in which learning and inference are performed directly via physical processes. Stochastic PNNs arise when the underlying neurons are realized by the dynamics of a stochastic activation switch. Here we propose novel electronic and photonic stochastic neurons. The electronic realization is implemented by single-electron tunneling through a quantum dot. The photonic realization is implemented via a single-photon source driving one of two modes coupled via a controllable beam-splitter-like interaction. In the electronic case, the charge state of the quantum dot forms the basis for the stochastic neuron, whereas in the photonic case the occupation of the undriven mode serves as the basis for the stochastic neuron. Training of stochastic PNNs is performed with models of stochastic neurons, as well as with coherently-driven, single-photon detector stochastic neurons previously introduced. Several training strategies for MNIST handwritten digit classification have been investigated using single-hidden-layer stochastic PNNs, including varying the number of trials in each layer to control forward pass stochasticity and employing either true probability or empirical outputs in the backward pass to evaluate their influence on gradient estimation. We show that when empirical outputs are used in the backward pass, the network achieves more than 97\% test accuracy with few trials per layer. Despite the simplicity of the model architecture, high test accuracy is maintained in the presence of a high degree of noise and model uncertainty. The results demonstrate the potential of embracing stochastic PNNs for deep learning.
其他(25篇)
【1】The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime
标题：验证税：罕见错误制度下人工智能审计的基本局限
链接：https://arxiv.org/abs/2604.12951
备注：25 pages, 16 figures, 6 tables. Code and data at https://github.com/Jason-Wang313/verification-tax
摘要：深度学习中引用最多的校准结果-CIFAR-100上的温度缩放后ECE为0.012（Guo等人，2017年：低于统计噪声下限。我们证明了这不是实验的失败，而是一条定律：估计校准误差的极小极大率与模型误差率θ是Theta（（Lepsilon/m）^{1/3}），没有估计器可以击败它。这种“验证税”意味着随着AI模型的改进，验证其校准变得更加困难-在相反的方向上具有相同的指数。我们建立了四个与标准评估实践相矛盾的结果：（1）没有标签的自我评估提供了关于校准的零信息，由一个与计算无关的常数限制;（2）在近似1处的急剧相变，低于该相变时无法检测到误校准;（3）主动查询消除了Lipschitz常数，使估计崩溃为检测;（4）验证成本随着流水线深度以L^K的速率呈指数增长。我们使用来自5个系列的6个LLM（8B-405 B参数，27个基于logprob的置信度的基准模型对），95% bootstrap CI和置换检验对5个基准（MMLU，TruthfulQA，ARC-Challenge，HellaSwag，WinoGrande;约27，000个项目）进行验证。80%的配对自我评价不显著。在前沿模型中，23%的成对比较与噪声无法区分，这意味着可信的校准声明必须报告验证下限，并在获得接近基准分辨率时优先考虑主动查询。
摘要：The most cited calibration result in deep learning -- post-temperature-scaling ECE of 0.012 on CIFAR-100 (Guo et al., 2017) -- is below the statistical noise floor. We prove this is not a failure of the experiment but a law: the minimax rate for estimating calibration error with model error rate epsilon is Theta((Lepsilon/m)^{1/3}), and no estimator can beat it. This "verification tax" implies that as AI models improve, verifying their calibration becomes fundamentally harder -- with the same exponent in opposite directions. We establish four results that contradict standard evaluation practice: (1) self-evaluation without labels provides exactly zero information about calibration, bounded by a constant independent of compute; (2) a sharp phase transition at mepsilon approx 1 below which miscalibration is undetectable; (3) active querying eliminates the Lipschitz constant, collapsing estimation to detection; (4) verification cost grows exponentially with pipeline depth at rate L^K. We validate across five benchmarks (MMLU, TruthfulQA, ARC-Challenge, HellaSwag, WinoGrande; ~27,000 items) with 6 LLMs from 5 families (8B-405B parameters, 27 benchmark-model pairs with logprob-based confidence), 95% bootstrap CIs, and permutation tests. Self-evaluation non-significance holds in 80% of pairs. Across frontier models, 23% of pairwise comparisons are indistinguishable from noise, implying that credible calibration claims must report verification floors and prioritize active querying once gains approach benchmark resolution.
【2】VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenization
标题：VideoFlexTok：长度从粗到细的视频代币化
链接：https://arxiv.org/abs/2604.12887
备注：project page at https://videoflextok.epfl.ch/
摘要：可视化标记器将高维原始像素映射到压缩表示中，用于下游建模。除了压缩之外，标记器还规定了保留哪些信息以及如何组织这些信息。视频标记化的事实上的标准方法是将视频表示为标记的时空3D网格，每个标记捕获原始信号中对应的本地信息。这需要消耗令牌的下游模型，例如，一个文本到视频模型，学习预测所有低级别的细节“像素的像素”，而不管视频的固有复杂性，导致高学习复杂性。 我们提出了VideoFlexTok，它表示具有以粗到细的方式结构化的令牌的可变长度序列的视频-其中第一个令牌（紧急）捕获抽象信息，例如语义和运动，稍后的令牌添加细粒度的细节。生成流解码器能够从任何令牌计数进行逼真的视频重建。这种表示结构允许根据下游需求调整令牌计数，并以相同的预算编码比基线更长的视频。 我们在类和文本到视频生成任务上评估了VideoFlexTok，并表明与3D网格令牌相比，它可以实现更有效的训练，例如，使用小5倍的模型（1.1B vs 5.2B）实现相当的生成质量（gFVD和ViCLIP评分）。最后，我们演示了VideoFlexTok如何在10秒81帧视频上训练文本到视频模型，而无需高昂的计算成本，只需672个令牌，比可比的3D网格令牌化器少8倍。
摘要：Visual tokenizers map high-dimensional raw pixels into a compressed representation for downstream modeling. Beyond compression, tokenizers dictate what information is preserved and how it is organized. A de facto standard approach to video tokenization is to represent a video as a spatiotemporal 3D grid of tokens, each capturing the corresponding local information in the original signal. This requires the downstream model that consumes the tokens, e.g., a text-to-video model, to learn to predict all low-level details "pixel-by-pixel" irrespective of the video's inherent complexity, leading to high learning complexity. We present VideoFlexTok, which represents videos with a variable-length sequence of tokens structured in a coarse-to-fine manner -- where the first tokens (emergently) capture abstract information, such as semantics and motion, and later tokens add fine-grained details. The generative flow decoder enables realistic video reconstructions from any token count. This representation structure allows adapting the token count according to downstream needs and encoding videos longer than the baselines with the same budget. We evaluate VideoFlexTok on class- and text-to-video generative tasks and show that it leads to more efficient training compared to 3D grid tokens, e.g., achieving comparable generation quality (gFVD and ViCLIP Score) with a 5x smaller model (1.1B vs 5.2B). Finally, we demonstrate how VideoFlexTok can enable long video generation without prohibitive computational cost by training a text-to-video model on 10-second 81-frame videos with only 672 tokens, 8x fewer than a comparable 3D grid tokenizer.
【3】Generative Anonymization in Event Streams
标题：事件流中的生成式抽象化
链接：https://arxiv.org/abs/2604.12803
备注：Accepted to the 1st Workshop on Low-Level Vision Frontiers (LoViF) at IEEE/CVF CVPR 2026
摘要：神经形态视觉传感器提供低延迟和高动态范围，但它们在公共空间的部署引发了严重的数据保护问题。最近的事件到视频（E2V）模型可以从稀疏事件流重建高保真强度图像，无意中暴露了人类身份。当前的混淆方法，如掩蔽或加扰，破坏时空结构，严重降低下游感知任务的数据效用。在本文中，据我们所知，我们提出了第一个生成匿名化框架的事件流，以解决这种实用性和隐私的权衡。通过弥合异步事件和标准空间生成模型之间的模态差距，我们的管道将事件投影到中间强度表示中，利用预训练的模型来合成真实的、不存在的身份，并将特征重新编码回神经形态域。实验表明，我们的方法可靠地防止身份恢复E2V重建，同时保持下游视觉任务所需的结构数据完整性。最后，为了便于严格的评估，我们引入了一种新的，同步的真实世界事件和RGB数据集，通过精确的机器人轨迹捕获，为未来的隐私保护神经形态视觉研究提供了一个强大的基准。
摘要 ：Neuromorphic vision sensors offer low latency and high dynamic range, but their deployment in public spaces raises severe data protection concerns. Recent Event-to-Video (E2V) models can reconstruct high-fidelity intensity images from sparse event streams, inadvertently exposing human identities. Current obfuscation methods, such as masking or scrambling, corrupt the spatio-temporal structure, severely degrading data utility for downstream perception tasks. In this paper, to the best of our knowledge, we present the first generative anonymization framework for event streams to resolve this utility-privacy trade-off. By bridging the modality gap between asynchronous events and standard spatial generative models, our pipeline projects events into an intermediate intensity representation, leverages pretrained models to synthesize realistic, non-existent identities, and re-encodes the features back into the neuromorphic domain. Experiments demonstrate that our method reliably prevents identity recovery from E2V reconstructions while preserving the structural data integrity required for downstream vision tasks. Finally, to facilitate rigorous evaluation, we introduce a novel, synchronized real-world event and RGB dataset captured via precise robotic trajectories, providing a robust benchmark for future research in privacy-preserving neuromorphic vision.
【4】VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computation
标题：VFA：通过全局最大预计算消除Flash中的向量操作的注意力
链接：https://arxiv.org/abs/2604.12798
摘要：FlashAttention风格的在线softmax通过片上存储器流式传输分数块并保持运行的最大值和归一化器，从而实现了使用线性存储器的精确注意力计算。然而，随着注意力内核在现代加速器上接近峰值张量核/立方体核吞吐量，在线softmax的非matmul组件-特别是每瓦片行最大值和行和减少以及重新缩放链-可能会成为向量或SIMD限制并主导延迟。本文重新访问FlashAttention，并提出了矢量释放Flash Attention（VFA），一种硬件友好的方法，减少了行最大驱动的更新的运行最大值，同时保留在线softmax结构。VFA通过一个廉价的近似值从关键块表示中删除运行最大值，重新排序关键块遍历以优先考虑高影响的sink和本地块，并冻结剩余块的最大值以避免重复减少和重新缩放。我们进一步将VFA与BLASST等块稀疏跳过方法集成，形成矢量缓解稀疏注意力（VSA），从而减少块计数和每块开销。值得注意的是，VFA和VSA完全避免了FA4.0中使用的更新阶段中的条件重新缩放操作。对包括MMLU和MATH 500在内的基准测试的广泛评估，以及注意力统计，验证了我们的设计：（i）sink和局部重新排序可以提前稳定运行最大值;（ii）简单的Q和K块摘要由于块内异质性而失败;（iii）当最大值出现在中间块时，需要m初始化。总体而言，VFA和VSA有效地缓解了在线softmax减少瓶颈，而不会降低性能。与C16 V32基线相比，C8 V32、C4 V32和C4 V16在现代硬件上实现了近两倍的加速，同时遇到了矢量瓶颈。随着即将到来的架构改进，C4 V16将通过增强指数容量提供六倍的加速。
摘要：FlashAttention-style online softmax enables exact attention computation with linear memory by streaming score tiles through on-chip memory and maintaining a running maximum and normalizer. However, as attention kernels approach peak tensor-core/cube-core throughput on modern accelerators, non-matmul components of online softmax -- especially per-tile rowmax and rowsum reductions and rescale chains -- can become vector or SIMD limited and dominate latency. This paper revisits FlashAttention and proposes Vector Relieved Flash Attention (VFA), a hardware-friendly method that reduces rowmax-driven updates of the running maximum while retaining the online-softmax structure. VFA initializes the running maximum via a cheap approximation from key-block representations, reorders key-block traversal to prioritize high-impact sink and local blocks, and freezes the maximum for remaining blocks to avoid repeated reductions and rescaling. We further integrate VFA with block-sparse skipping methods such as BLASST to form Vector Relieved Sparse Attention (VSA), which reduces both block count and per-block overhead. Notably, VFA and VSA completely avoid the conditional rescale operation in the update stage used in FA4.0. Extensive evaluations on benchmarks including MMLU and MATH500, together with attention statistics, verify our design: (i) sink and local reordering stabilizes the running maximum early; (ii) simple Q and K block summaries fail due to intra-block heterogeneity; (iii) m-initialization is required when maxima appear in middle blocks. Overall, VFA and VSA efficiently alleviate online-softmax reduction bottlenecks without performance loss. Compared to the C16V32 baseline, C8V32, C4V32 and C4V16 achieve nearly two times speedup on modern hardware while hitting the vector bottleneck. With upcoming architecture improvements, C4V16 will deliver six times speedup by enhancing exponent capacity.
【5】OSC: Hardware Efficient W4A4 Quantization via Outlier Separation in Channel Dimension
标题：OSC：通过通道维度中的离群值分离实现硬件高效W4A4量化
链接：https://arxiv.org/abs/2604.12782
摘要：虽然4位量化对于大语言模型的高吞吐量部署至关重要，但由于低位格式的动态范围有限，激活离群值通常会导致准确性显著下降。在本文中，我们系统地研究了离群值的空间分布，并证明了令牌持续的结构聚类效应，其中高量级离群值始终占据令牌之间的固定通道。基于这一认识，我们提出了OSC，一个硬件高效的框架离群抑制。在推理过程中，OSC执行由低精度4位通用矩阵乘法（GEMM）路径和高精度16位分支GEMM路径组成的双路径计算。具体而言，OSC使用离线分组策略来识别异常值所在的通道，然后执行结构化子张量提取，以在线将这些分散的激活通道合并为紧凑的稠密张量。该机制通过正则化和高吞吐量GEMM操作实现离群值保护，实现与现代4位微尺度硬件的无缝匹配。此外，对于离群值聚类不太明显的W2的输入，我们将回退策略集成到FP 8。对Qwen 3 -8B和Qwen 3 - 30 B的评估将平均精度下降分别限制为2.19和1.12分。值得注意的是，OSC是高度硬件友好的，在现代AI加速器上实现了比W8 A8 GEMM基线高1.78倍的峰值加速。
摘要：While 4-bit quantization is essential for high-throughput deployment of Large Language Models, activation outliers often lead to significant accuracy degradation due to the restricted dynamic range of low-bit formats. In this paper, we systematically investigate the spatial distribution of outliers and demonstrate a token-persistent structural clustering effect, where high-magnitude outliers consistently occupy fixed channels across tokens. Building on this insight, we propose OSC, a hardware-efficient framework for outlier suppression. During inference, OSC executes a dual-path computation consisting of a low-precision 4-bit General Matrix Multiplication (GEMM) path and a high-precision 16-bit branch GEMM path. Specifically, OSC uses an offline group-wise strategy to identify the channels where outliers are located and then performs structured sub-tensor extraction to coalesce these scattered activation channels into a compact dense tensor online. This mechanism implements outlier protection through regularized and high-throughput GEMM operations, achieving a seamless fit with modern 4-bit micro-scaling hardware. Furthermore, for the inputs of W2 where outlier clustering is less pronounced, we integrate a fallback strategy to FP8. Evaluation on Qwen3-8B and Qwen3-30B restricts the average accuracy drop to 2.19 and 1.12 points, respectively. Notably, OSC is highly hardware-friendly, achieving a peak speedup of 1.78x over the W8A8 GEMM baseline on a modern AI accelerator.
【6】GF-Score: Certified Class-Conditional Robustness Evaluation with Fairness Guarantees
标题：GF-Score：经过认证的类别条件稳健性评估，具有公平性保证
链接：https://arxiv.org/abs/2604.12757
备注：16 pages, 5 tables, 9 figures
摘要：对抗性鲁棒性对于在安全关键型应用中部署神经网络至关重要，但标准评估方法要么需要昂贵的对抗性攻击，要么只报告一个综合得分，从而掩盖了鲁棒性如何在类间分布。我们引入了GF-Score（GREAT-公平分数），一个框架，将认证的GREAT分数分解为每类鲁棒性配置文件，并通过四个基于福利经济学的指标量化其差异：鲁棒性差异指数（RDI），归一化鲁棒性基尼系数（NRGC），最坏情况下的类鲁棒性（WCR）和公平惩罚的GREAT分数（FP-GREAT）。该框架通过自校准过程进一步消除了原始方法对对抗性攻击的依赖，该自校准过程仅使用干净的准确度相关性来调整温度参数。通过评估CIFAR-10和ImageNet上RobustBench的22个模型，我们发现分解是准确的，每个类的得分揭示了一致的漏洞模式（例如，"猫“是76%的CIFAR-10模型中最弱的类），更稳健的模型往往表现出更大的类间差异。这些结果建立了一个实用的，无攻击的审计管道，用于诊断认证的鲁棒性保证无法平等地保护所有类。我们在\href{https：//github.com/aryashah2k/gf-score}{GitHub}上发布代码。
摘要 ：Adversarial robustness is essential for deploying neural networks in safety-critical applications, yet standard evaluation methods either require expensive adversarial attacks or report only a single aggregate score that obscures how robustness is distributed across classes. We introduce the \emph{GF-Score} (GREAT-Fairness Score), a framework that decomposes the certified GREAT Score into per-class robustness profiles and quantifies their disparity through four metrics grounded in welfare economics: the Robustness Disparity Index (RDI), the Normalized Robustness Gini Coefficient (NRGC), Worst-Case Class Robustness (WCR), and a Fairness-Penalized GREAT Score (FP-GREAT). The framework further eliminates the original method's dependence on adversarial attacks through a self-calibration procedure that tunes the temperature parameter using only clean accuracy correlations. Evaluating 22 models from RobustBench across CIFAR-10 and ImageNet, we find that the decomposition is exact, that per-class scores reveal consistent vulnerability patterns (e.g., ``cat'' is the weakest class in 76\% of CIFAR-10 models), and that more robust models tend to exhibit greater class-level disparity. These results establish a practical, attack-free auditing pipeline for diagnosing where certified robustness guarantees fail to protect all classes equally. We release our code on \href{https://github.com/aryashah2k/gf-score}{GitHub}.
【7】Orthogonal Subspace Projection for Continual Machine Unlearning via SVD-Based LoRA
标题：通过基于ASD的LoRA实现连续机器去学习的垂直子空间投影
链接：https://arxiv.org/abs/2604.12526
摘要：持续的机器非学习旨在消除不应该再保留的数据的影响，同时保留模型对其他一切的有用性。当删除请求顺序到达时，此设置变得特别困难，因为模型必须反复适应而不会删除先前保留的知识。低秩自适应（LoRA）提供了一种有效的方法来实现这种更新，但天真地组合许多顺序LoRA模块会导致参数冲突，导致任务之间的强干扰。我们提出了一种基于奇异值分解（SVD）引导的正交子空间投影的静态替代方案。我们的方法在训练过程中约束每个新的LoRA更新，使其位于早期非学习任务所使用的子空间的正交补中。这保持了任务隔离，而无需在部署时进行动态路由。CIFAR-100与ResNet-20和MNIST上的实验显示了长序列非学习任务的稳定行为。在30个连续的非学习任务之后，最先进的静态融合将保留的准确率从60.39%降低到12.70%，而所提出的训练中约束优化保持基线性能（58.1%），同时保持强大的非学习功效。
摘要：Continual machine unlearning aims to remove the influence of data that should no longer be retained, while preserving the usefulness of the model on everything else. This setting becomes especially difficult when deletion requests arrive sequentially, because the model must repeatedly adapt without erasing previously retained knowledge. Low-Rank Adaptation (LoRA) offers an efficient way to implement such updates, but naively combining many sequential LoRA modules leads to parameter collision, causing \textit{strong interference} between tasks. We propose a static alternative based on Singular Value Decomposition (SVD)-guided orthogonal subspace projection. Our method constrains each new LoRA update during training so that it lies in the orthogonal complement of the subspaces used by earlier unlearning tasks. This preserves task isolation without requiring dynamic routing at deployment. Experiments on CIFAR-100 with ResNet-20 and on MNIST show stable behavior across long sequences of unlearning tasks. After thirty sequential unlearning tasks, state-of-the-art static fusion reduces retained accuracy from 60.39\% to 12.70\%, whereas the proposed in-training constrained optimization maintains baseline performance ($\sim$58.1\%) while preserving strong unlearning efficacy.
【8】Instantiating Bayesian CVaR lower bounds in Interactive Decision Making Problems
标题：交互式决策问题中实例化Bayesian CVaR下限
链接：https://arxiv.org/abs/2604.12519
摘要：最近的工作建立了一个广义Fano框架下界先验预测（贝叶斯）CVaR在交互式统计决策。在本文中，我们将展示如何在具体的互动问题，并从其抽象的推论推导出明确的贝叶斯CVaR下界的框架。我们的方法比较了一个硬模型与参考模型使用平方Hellinger距离，并结合了一个下限的参考铰链项上的约束的两个模型的可扩展性。我们将这种方法应用于典型的例子，包括高斯土匪，并获得明确的界限，使关键问题参数的依赖透明。这些结果表明，广义Fano贝叶斯CVaR框架可以作为一个实用的下限工具，交互式学习和风险敏感的决策。
摘要：Recent work established a generalized-Fano framework for lower bounding prior-predictive (Bayesian) CVaR in interactive statistical decision making. In this paper, we show how to instantiate that framework in concrete interactive problems and derive explicit Bayesian CVaR lower bounds from its abstract corollaries. Our approach compares a hard model with a reference model using squared Hellinger distance, and combines a lower bound on a reference hinge term with a bound on the distinguishability of the two models. We apply this approach to canonical examples, including Gaussian bandits, and obtain explicit bounds that make the dependence on key problem parameters transparent. These results show how the generalized-Fano Bayesian CVaR framework can be used as a practical lower-bound tool for interactive learning and risk-sensitive decision making.
【9】Safety Training Modulates Harmful Misalignment Under On-Policy RL, But Direction Depends on Environment Design
标题：安全训练在政策RL下调节了有害的错位，但方向取决于环境设计
链接：https://arxiv.org/abs/2604.12500
摘要：众所周知，强化学习（RL）下的规范博弈会导致LLM发展出奉承、操纵或欺骗行为，但这种情况发生的条件仍不清楚。我们训练了11个基于策略RL的LLM（0.5B-14 B），发现模型大小在某些环境中充当安全缓冲区，但在其他环境中会导致更大的有害利用。受控消融将这种逆转追溯到特定环境的特征，如角色框架和隐含的游戏性线索。我们进一步表明，大多数安全基准不预测RL引起的错位，除了在谄媚分数的情况下，利用依赖于推断用户的偏好。最后，我们发现，在政策RL保留了一个安全缓冲区固有的模型自己的生成分布，一个是在关闭政策设置绕过。
摘要：Specification gaming under Reinforcement Learning (RL) is known to cause LLMs to develop sycophantic, manipulative, or deceptive behavior, yet the conditions under which this occurs remain unclear. We train 11 instruction-tuned LLMs (0.5B--14B) with on-policy RL across 3 environments and find that model size acts as a safety buffer in some environments but enables greater harmful exploitation in others. Controlled ablations trace this reversal to environment-specific features such as role framing and implicit gameability cues. We further show that most safety benchmarks do not predict RL-induced misalignment, except in the case of Sycophancy scores when the exploit relies on inferring the user's preference. Finally, we find that on-policy RL preserves a safety buffer inherent in the model's own generation distribution, one that is bypassed during off-policy settings.
【10】VeriX-Anon: A Multi-Layered Framework for Mathematically Verifiable Outsourced Target-Driven Data Anonymization
标题：VeriX-Anon：一个用于数学可验证的外包目标驱动数据分析的多层框架
链接：https://arxiv.org/abs/2604.12431
摘要：组织越来越多地将隐私敏感的数据转换外包给云提供商，但没有实际的机制让数据所有者验证合同算法是否得到忠实执行。VeriX-Anon是一个用于外包目标驱动k匿名化的多层验证框架，结合了三种正交机制：通过Merkle风格的认证决策树散列的确定性验证，通过随机森林决策边界附近的边界哨兵和具有加密标识符的精确重复双胞胎的概率性验证，通过可解释的AI指纹识别进行基于效用的验证，该指纹识别使用Wasserstein距离比较匿名化前后的SHAP值分布。在三个跨域数据集上针对Lazy（删除5%的记录），Dumb（随机拆分，假哈希）和Approximate（随机拆分，有效哈希）对手进行了评估，VeriX-Anon在12个场景中的11个场景中正确检测到了偏差。没有任何一层能够单独做到这一点。XAI层是捕获近似对手的唯一机制，在Adult和Bank上取得了成功，但在严重不平衡的糖尿病数据集上失败了，其中类别不平衡抑制了SHAP信号，证实了自适应阈值的必要性。一个11点的k-扫描显示目标驱动的匿名化保留了显着更多的效用比盲匿名化（Wilcoxon $p = 0.000977$，科恩的$d = 1.96$，平均F1差距$+0.1574$）。客户端验证在1秒内完成100万行。威胁模型包括三个经验评估的配置文件和一个理论配置文件（知情攻击者）知道陷阱嵌入，但无法击败加密盐。哨兵逃避概率的范围从平衡数据集的接近零到不平衡数据集的0.52，这是孪生层在每个测试场景中补偿的限制。
摘要 ：Organisations increasingly outsource privacy-sensitive data transformations to cloud providers, yet no practical mechanism lets the data owner verify that the contracted algorithm was faithfully executed. VeriX-Anon is a multi-layered verification framework for outsourced Target-Driven k-anonymization combining three orthogonal mechanisms: deterministic verification via Merkle-style hashing of an Authenticated Decision Tree, probabilistic verification via Boundary Sentinels near the Random Forest decision boundary and exact-duplicate Twins with cryptographic identifiers, and utility-based verification via Explainable AI fingerprinting that compares SHAP value distributions before and after anonymization using the Wasserstein distance. Evaluated on three cross-domain datasets against Lazy (drops 5 percent of records), Dumb (random splitting, fake hash), and Approximate (random splitting, valid hash) adversaries, VeriX-Anon correctly detected deviations in 11 of 12 scenarios. No single layer achieved this alone. The XAI layer was the only mechanism that caught the Approximate adversary, succeeding on Adult and Bank but failing on the severely imbalanced Diabetes dataset where class imbalance suppresses the SHAP signal, confirming the need for adaptive thresholding. An 11-point k-sweep showed Target-Driven anonymization preserves significantly more utility than Blind anonymization (Wilcoxon $p = 0.000977$, Cohen's $d = 1.96$, mean F1 gap $+0.1574$). Client-side verification completes under one second at one million rows. The threat model covers three empirically evaluated profiles and one theoretical profile (Informed Attacker) aware of trap embedding but unable to defeat the cryptographic salt. Sentinel evasion probability ranges from near-zero for balanced datasets to 0.52 for imbalanced ones, a limitation the twin layer compensates for in every tested scenario.
【11】PrivEraserVerify: Efficient, Private, and Verifiable Federated Unlearning
标题：PrivEraserVerify：高效、私密且可验证的联合取消学习
链接：https://arxiv.org/abs/2604.12348
摘要：联邦学习（FL）可以在不共享原始数据的情况下进行协作模型训练，为保护隐私的人工智能提供了一条有前途的道路。然而，FL模型仍然可能记住来自参与者的敏感信息，这与被遗忘权（RTBF）相冲突。为了满足这些要求，联邦遗忘已经成为一种机制，以消除离开客户端的贡献。现有的解决方案只能部分解决这一挑战：FedEraser提高了效率，但缺乏隐私保护，FedRecovery确保了差异隐私（DP），但降低了准确性，VeriFi实现了可验证性，但引入了开销，没有效率或隐私保证。我们提出了PrivEraserVerify（PEV），这是一个统一的框架，它将效率，隐私和可验证性集成到联邦学习中。PEV采用（i）自适应检查点来保留关键的历史更新以进行快速重建，（ii）层自适应差分私有校准以选择性地去除客户端影响，同时最大限度地减少准确性损失，以及（iii）基于指纹的验证，使参与者能够以分散和非侵入性的方式确认遗忘。在图像、手写字符和医疗数据集上的实验表明，PEV的去学习速度比再训练快2到3倍，提供了形式上的不可扩展性保证，降低了性能下降，并支持可扩展性验证。据我们所知，PEV是第一个同时为联邦学习提供效率，隐私和可验证性的框架，使FL更接近实际和符合法规的部署。
摘要：Federated learning (FL) enables collaborative model training without sharing raw data, offering a promising path toward privacy preserving artificial intelligence. However, FL models may still memorize sensitive information from participants, conflicting with the right to be forgotten (RTBF). To meet these requirements, federated unlearning has emerged as a mechanism to remove the contribution of departing clients. Existing solutions only partially address this challenge: FedEraser improves efficiency but lacks privacy protection, FedRecovery ensures differential privacy (DP) but degrades accuracy, and VeriFi enables verifiability but introduces overhead without efficiency or privacy guarantees. We present PrivEraserVerify (PEV), a unified framework that integrates efficiency, privacy, and verifiability into federated unlearning. PEV employs (i) adaptive checkpointing to retain critical historical updates for fast reconstruction, (ii) layer adaptive differentially private calibration to selectively remove client influence while minimizing accuracy loss, and (iii) fingerprint based verification, enabling participants to confirm unlearning in a decentralized and noninvasive manner. Experiments on image, handwritten character, and medical datasets show that PEV achieves up to 2 to 3 times faster unlearning than retraining, provides formal indistinguishability guarantees with reduced performance degradation, and supports scalable verification. To the best of our knowledge, PEV is the first framework to simultaneously deliver efficiency, privacy, and verifiability for federated unlearning, moving FL closer to practical and regulation compliant deployment.
【12】GCA Framework: A Gulf-Grounded Dataset and Agentic Pipeline for Climate Decision Support
标题：GCA框架：海湾数据集和气候决策支持的统计管道
链接：https://arxiv.org/abs/2604.12306
摘要：海湾地区的气候决策越来越需要能够将异质科学和政策证据转化为可操作指导的系统，但通用大型语言模型（LLM）在特定区域的气候知识以及与地理空间和预测工具的基础交互方面仍然薄弱。我们提出了GCA框架，它统一了（i）GCA-DS，一个以海湾为重点的多模态数据集，和（ii）海湾气候代理（GCA），一个用于气候分析的工具增强代理。GCA-DS包含约20万个问答对，涵盖政府政策和适应计划、非政府组织和国际框架、学术文献以及关于热浪、沙尘暴和洪水的事件驱动报告，并辅以将图像与文本证据相结合的遥感输入。在此基础上，GCA代理编排了一个模块化工具管道，该管道基于实时和历史信号以及地理空间处理，可生成派生索引和可解释的可视化。最后，我们对海湾气候任务的开放和专有LLM进行了基准测试，并表明域微调和工具集成大大提高了通用基线的可靠性。
摘要：Climate decision-making in the Gulf increasingly demands systems that can translate heterogeneous scientific and policy evidence into actionable guidance, yet general-purpose large language models (LLMs) remain weak both in region-specific climate knowledge and grounded interaction with geospatial and forecasting tools. We present the GCA framework, which unifies (i) GCA-DS, a curated Gulf-focused multimodal dataset, and (ii) Gulf Climate Agent (GCA), a tool-augmented agent for climate analysis. GCA-DS comprises ~200k question-answer pairs spanning governmental policies and adaptation plans, NGO and international frameworks, academic literature, and event-driven reporting on heatwaves, dust storms, and floods, complemented with remote-sensing inputs that couple imagery with textual evidence. Building on this foundation, the GCA agent orchestrates a modular tool pipeline grounded in real-time and historical signals and geospatial processing that produces derived indices and interpretable visualizations. Finally, we benchmark open and proprietary LLMs on Gulf climate tasks and show that domain fine-tuning and tool integration substantially improve reliability over general-purpose baselines.
【13】A Residual-Shell-Based Lower Bound for Ollivier-Ricci Curvature
标题：Ollivier-Ricci曲线的基于残壳的下界
链接：https://arxiv.org/abs/2604.12211
摘要：利用Wasserstein距离定义的Ollivier-Ricci曲率（ORC）具有丰富的几何信息，在理论和应用上都受到了越来越多的关注。然而，Wasserstein距离评估的高计算成本显著限制了ORC更广泛的实际应用。为了缓解这个问题，以前的工作引入了一个计算效率的下限作为基于1跳随机游走的ORC的代理，但这种方法根据经验表现出与确切ORC的较大差距。在本文中，我们建立了一个更严格的下限ORC比现有的下限，同时保留了更低的计算成本比精确的ORC计算，实际的加速数十倍。此外，我们的界不仅限于1-hop随机游动，而且也适用于k-hop随机游动（k > 1）。对几种基本图结构的实验证明了我们的界在逼近精度和计算效率方面的有效性。
摘要：Ollivier-Ricci curvature (ORC), defined via the Wasserstein distance that captures rich geometric information, has received growing attention in both theory and applications. However, the high computational cost of Wasserstein distance evaluation has significantly limited the broader practical use of ORC. To alleviate this issue, previous work introduced a computationally efficient lower bound as a proxy for ORC based on 1-hop random walks, but this approach empirically exhibits large gaps from the exact ORC. In this paper, we establish a substantially tighter lower bound for ORC than the existing lower bound, while retaining much lower computational cost than exact ORC computation, with practical speedups of tens of times. Moreover, our bound is not restricted to 1-hop random walks, but also applies to k-hop random walks (k > 1). Experiments on several fundamental graph structures demonstrate the effectiveness of our bound in terms of both approximation accuracy and computational efficiency.
【14】PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
标题：PubSwap：联邦WLVR的公共数据政策外协调
链接：https://arxiv.org/abs/2604.12160
摘要：基于可验证奖励的强化学习（RLVR）的推理后训练通常在集中式环境中进行研究，但许多现实的应用程序涉及分散在组织中的分散的私有数据。联邦训练是一种自然的解决方案，但在这种情况下扩展RLVR是具有挑战性的：全模型同步是昂贵的，并且在异构数据下执行许多本地步骤可能会导致严重的客户端漂移。我们提出了一个联合RLVR框架，该框架将基于LoRA的本地自适应与基于公共数据的非策略步骤相结合，以提高通信效率和跨客户端协调。特别是，一个小型的共享公共数据集用于在组织之间定期交换和重用响应级别的训练信号，为实现更全球一致的目标提供轻量级的锚点，而不会暴露私有数据。我们的方法在公共数据步骤中选择性地将局部不正确的响应替换为全局正确的响应，从而使培训更接近本地政策，同时仍然受益于跨客户端的协调。在数学和医学推理基准和模型中，我们的方法始终优于标准基线。我们的研究结果突出了一个简单而有效的联邦推理后训练方法：将低秩通信与有限的公共数据协调相结合。
摘要：Reasoning post-training with reinforcement learning from verifiable rewards (RLVR) is typically studied in centralized settings, yet many realistic applications involve decentralized private data distributed across organizations. Federated training is a natural solution, but scaling RLVR in this regime is challenging: full-model synchronization is expensive, and performing many local steps can cause severe client drift under heterogeneous data. We propose a federated RLVR framework that combines LoRA-based local adaptation with public-data-based off-policy steps to improve both communication efficiency and cross-client coordination. In particular, a small shared public dataset is used to periodically exchange and reuse response-level training signals across organizations, providing a lightweight anchor toward a more globally aligned objective without exposing private data. Our method selectively replaces locally incorrect responses with globally correct ones during public-data steps, thereby keeping training closer to the local policy while still benefiting from cross-client coordination. Across mathematical and medical reasoning benchmarks and models, our method consistently improves over standard baselines. Our results highlight a simple and effective recipe for federated reasoning post-training: combining low-rank communication with limited public-data coordination.
【15】Constant-Factor Approximation for the Uniform Decision Tree
标题：一致决策树的恒因子逼近
链接：https://arxiv.org/abs/2604.12036
备注：10 pages
摘要：我们解决了一个长期存在的开放问题，关于存在一个常数因子近似算法的平均情况下的\textsc{决策树}问题的假设具有均匀的概率分布。我们通过提供一个简单的多项式时间算法来肯定地回答这个问题，其近似比为$\frac{2}{1-\sqrt{（e+1）/（2 e）}}+ε<11.57$。这改进了目前最著名的贪婪算法，达到了O（\log n/{\log\log n}）$-近似。在我们的分析中的第一个关键因素是使用分解技术，从相关的问题中已知\textsc{分层聚类} [SODA '17，WALCOM '26]，它允许我们将最优决策树分解成一系列称为分离子族的对象。第二个关键的想法是将寻找\textsc{Separating Subfamily}的子问题简化为\textsc{Maximum Coverage}问题的一个实例。为此，我们分析了将集团切割成小块的特性，这些小块表示要分离的假设对。这使我们能够获得一个很好的近似的\textsc{分离子族}问题，然后使设计的近似算法的原始问题。
摘要：We resolve a long-standing open question, about the existence of a constant-factor approximation algorithm for the average-case \textsc{Decision Tree} problem with uniform probability distribution over the hypotheses. We answer the question in the affirmative by providing a simple polynomial-time algorithm with approximation ratio of $\frac{2}{1-\sqrt{(e+1)/(2e)}}+ε<11.57$. This improves upon the currently best-known, greedy algorithm which achieves $O(\log n/{\log\log n})$-approximation. The first key ingredient in our analysis is the usage of a decomposition technique known from problems related to \textsc{Hierarchical Clustering} [SODA '17, WALCOM '26], which allows us to decompose the optimal decision tree into a series of objects called separating subfamilies. The second crucial idea is to reduce the subproblem of finding a \textsc{Separating Subfamily} to an instance of the \textsc{Maximum Coverage} problem. To do so, we analyze the properties of cutting cliques into small pieces, which represent pairs of hypotheses to be separated. This allows us to obtain a good approximation for the \textsc{Separating Subfamily} problem, which then enables the design of the approximation algorithm for the original problem.
【16】Multi-Head Residual-Gated DeepONet for Coherent Nonlinear Wave Dynamics
标题：用于相干非线性波动动力学的多头剩余门控DeepONet
链接：https://arxiv.org/abs/2604.11972
摘要：相干非线性波动动力学通常强烈地由初始状态的一组紧凑的物理上有意义的描述符形成。传统的神经操作符通常将输入-输出映射视为一个很大程度上的黑盒高维回归问题，而没有明确地利用这种结构化的物理环境。常见的特征整合策略通常依赖于隐藏潜在空间中的直接级联或FiLM风格的仿射调制。在这里，我们介绍了一个不同的范式，松散的启发状态演化和量子力学中的物理意义可观的互补作用：波场是通过标准的DeepONet状态路径学习的，而紧凑的物理描述符遵循并行的条件路径，并作为状态预测的剩余调制因子。基于这一思想，我们开发了一种多头残差门控DeepONet（MH-RG），它将预分支残差调制器，分支残差门和主干残差门与低秩多头机制相结合，以捕获多个互补的条件响应模式，而不会出现禁止性的参数增长。我们评估的框架上的代表性的基准，包括高度非线性保守的波动动力学和耗散被困动力学，并进一步进行详细的机制分析的学习多头门控行为。与特征增强基线相比，MH-RG DeepONet实现了一致的较低误差，同时更好地保持了相位相干性和物理相关动态量的保真度。
摘要：Coherent nonlinear wave dynamics are often strongly shaped by a compact set of physically meaningful descriptors of the initial state. Traditional neural operators typically treat the input-output mapping as a largely black-box high-dimensional regression problem, without explicitly exploiting this structured physical context. Common feature-integration strategies usually rely on direct concatenation or FiLM-style affine modulation in hidden latent spaces. Here we introduce a different paradigm, loosely inspired by the complementary roles of state evolution and physically meaningful observables in quantum mechanics: the wave field is learned through a standard DeepONet state pathway, while compact physical descriptors follow a parallel conditioning pathway and act as residual modulation factors on the state prediction. Based on this idea, we develop a Multi-Head Residual-Gated DeepONet (MH-RG), which combines a pre-branch residual modulator, a branch residual gate, and a trunk residual gate with a low-rank multi-head mechanism to capture multiple complementary conditioned response patterns without prohibitive parameter growth. We evaluate the framework on representative benchmarks including highly nonlinear conservative wave dynamics and dissipative trapped dynamics and further perform detailed mechanistic analyses of the learned multi-head gating behavior. Compared with feature-augmented baselines, MH-RG DeepONet achieves consistently lower error while better preserving phase coherence and the fidelity of physically relevant dynamical quantities.
【17】Fast and principled equation discovery from chaos to climate
标题：从混乱到气候的快速而有原则的方程发现
链接：https://arxiv.org/abs/2604.11929
备注：34 pages, 8 figures
摘要 ：我们预测、控制和最终理解复杂系统的能力，依赖于发现控制其动力学的方程。因此，直接从嘈杂、有限的观测中识别这些方程已成为数据驱动科学的核心挑战，而现有的基于库的稀疏回归方法迫使在自动化、统计严谨性和计算效率之间做出妥协。在这里，我们开发贝叶斯-ARGOS，一个混合框架，通过将快速频率筛选与集中贝叶斯推理相结合来协调这些需求，从而实现自动方程发现，并以现有方法的一小部分计算成本进行原则性不确定性量化。在不同的数据稀缺性和噪声水平下对七个混沌系统进行了测试，Bayesian-ARGOS在大多数情况下都优于两种最先进的方法。它在所有系统的数据效率和七个系统中六个系统的噪声容限方面都超过了SINDy，与基于引导的实时卫星环境数据收集系统相比，计算成本降低了两个数量级。概率公式还支持一套标准的统计诊断，包括影响分析和多重共线性检测，这些分析和检测可以暴露故障模式，否则这些模式是不透明的。当与表示学习（SINDy-SHRED）集成用于高维海表温度重建时，Bayesian-ARGOS增加了有效潜方程的产量，并显着提高了长期稳定性。因此，Bayesian-ARGOS提供了一条从稀少和嘈杂的观测到可解释的控制方程的原则性、自动化和计算效率高的路线，为从基准混沌系统到全球气候模式的潜在动态的跨尺度方程发现提供了一个实用框架。
摘要：Our ability to predict, control, and ultimately understand complex systems rests on discovering the equations that govern their dynamics. Identifying these equations directly from noisy, limited observations has therefore become a central challenge in data-driven science, yet existing library-based sparse regression methods force a compromise between automation, statistical rigor, and computational efficiency. Here we develop Bayesian-ARGOS, a hybrid framework that reconciles these demands by combining rapid frequentist screening with focused Bayesian inference, enabling automated equation discovery with principled uncertainty quantification at a fraction of the computational cost of existing methods. Tested on seven chaotic systems under varying data scarcity and noise levels, Bayesian-ARGOS outperforms two state-of-the-art methods in most scenarios. It surpasses SINDy in data efficiency for all systems and noise tolerance for six out of the seven, with a two-order-of-magnitude reduction in computational cost compared to bootstrap-based ARGOS. The probabilistic formulation additionally enables a suite of standard statistical diagnostics, including influence analysis and multicollinearity detection that expose failure modes otherwise opaque. When integrated with representation learning (SINDy-SHRED) for high dimensional sea surface temperature reconstruction, Bayesian-ARGOS increases the yield of valid latent equations with significantly improved long horizon stability. Bayesian-ARGOS thus provides a principled, automated, and computationally efficient route from scarce and noisy observations to interpretable governing equations, offering a practical framework for equation discovery across scales, from benchmark chaotic systems to the latent dynamics underlying global climate patterns.
【18】Can AI Detect Life? Lessons from Artificial Life
标题：人工智能可以检测生命吗？人工生命的教训
链接：https://arxiv.org/abs/2604.11915
备注：6 pages, 7 figures. Alife 2026
摘要：现代机器学习方法已经被提出来检测外星样本中的生命，利用它们基于使用天然和合成有机分子混合物的训练模型区分生物和非生物样本的能力。在这里，我们使用人工生命表明，即使分析的样本没有生命能力，这种方法也很容易被愚弄到以接近100%的置信度检测生命。这是由于现代机器学习方法容易被分布外的样本所欺骗。由于外星样本很可能不在地球生物和非生物样本提供的分布范围内，因此使用人工智能方法进行生命检测必然会产生严重的误报。
摘要：Modern machine learning methods have been proposed to detect life in extraterrestrial samples, drawing on their ability to distinguish biotic from abiotic samples based on training models using natural and synthetic organic molecular mixtures. Here we show using Artificial Life that such methods are easily fooled into detecting life with near 100% confidence even if the analyzed sample is not capable of life. This is due to modern machine learning methods' propensity to be easily fooled by out-of-distribution samples. Because extra-terrestrial samples are very likely out of the distribution provided by terrestrial biotic and abiotic samples, using AI methods for life detection is bound to yield significant false positives.
【19】Refined Differentially Private Linear Regression via Extension of a Free Lunch Result
标题：通过扩展免费午餐结果来细化差异私人线性回归
链接：https://arxiv.org/abs/2604.11820
备注：Accepted at the SeQureDB Workshop at ACM SIGMOD 2026 and TPDP Workshop 2026
摘要：随着数据隐私法规的收紧以及统计模型越来越多地部署在敏感的人工来源数据上，隐私保护线性回归已成为一种至关重要的必要性。对于添加-删除DP模型，Kulesza et al.（2024）和Fitzsimons et al.（2024）已经独立地证明了数据集的大小-线性回归的重要统计量-可以通过有界变量的单纯形变换和对变换变量的私有和查询来私下估计为“免费”。在这项工作中，我们通过精心制作的多维单纯形变换扩展这个免费午餐结果，变量和函数在区间[0，1]内有界。我们表明，这些转换可以应用于完善私人简单线性回归的基础上普通最小二乘所需的足够的统计量的估计。我们提供了分析和数值结果来证明我们的方法的优越性。我们提出的变换具有普遍适用性，可以很容易地适应差分私人多项式回归。
摘要：As data-privacy regulations tighten and statistical models are increasingly deployed on sensitive human-sourced data, privacy-preserving linear regression has become a critical necessity. For the add-remove DP model, Kulesza et al. (2024) and Fitzsimons et al. (2024) have independently shown that the size of the dataset -- an important statistic for linear regression -- can be privately estimated for "free", via a simplex transformation of bounded variables and private sum queries on the transformed variables. In this work, we extend this free lunch result via carefully crafted multidimensional simplex transformations to variables and functions that are bounded in the interval [0,1]. We show that these transformations can be applied to refine the estimates of sufficient statistics needed for private simple linear regression based on ordinary least squares. We provide both analytical and numerical results to demonstrate the superiority of our approach. Our proposed transformations have general applicability and can be readily adapted for differentially private polynomial regression.
【20】M$^\star$: Every Task Deserves Its Own Memory Harness
链接：https://arxiv.org/abs/2604.11811
备注：Preprint
摘要：大型语言模型代理依赖于专门的记忆系统来积累和重用扩展交互过程中的知识。最近的架构通常采用针对特定领域的固定内存设计，例如会话的语义检索或编码重用的技能。然而，为一个目的而优化的存储器系统经常无法转移到其他目的。为了解决这一限制，我们引入了M$^\star$，一种通过可执行程序进化自动发现任务优化的内存利用的方法。具体来说，M$^\star$将代理内存系统建模为用Python编写的内存程序。该程序封装了数据模式、存储逻辑和代理工作流指令。我们优化这些组件联合使用反射代码进化方法，这种方法采用了基于人口的搜索策略，并分析评估失败迭代细化候选程序。我们评估M$^\star$四个不同的基准跨越对话，体现规划，专家推理。我们的研究结果表明，M$^\star$在所有评估的任务中，都比现有的固定内存基线稳健地提高了性能。此外，进化的内存程序表现出结构上不同的处理机制，为每个域。这一发现表明，专门的内存机制为一个给定的任务探索了广阔的设计空间，并提供了一个更好的解决方案相比，通用的内存范式。
摘要 ：Large language model agents rely on specialized memory systems to accumulate and reuse knowledge during extended interactions. Recent architectures typically adopt a fixed memory design tailored to specific domains, such as semantic retrieval for conversations or skills reused for coding. However, a memory system optimized for one purpose frequently fails to transfer to others. To address this limitation, we introduce M$^\star$, a method that automatically discovers task-optimized memory harnesses through executable program evolution. Specifically, M$^\star$ models an agent memory system as a memory program written in Python. This program encapsulates the data Schema, the storage Logic, and the agent workflow Instructions. We optimize these components jointly using a reflective code evolution method; this approach employs a population-based search strategy and analyzes evaluation failures to iteratively refine the candidate programs. We evaluate M$^\star$ on four distinct benchmarks spanning conversation, embodied planning, and expert reasoning. Our results demonstrate that M$^\star$ improves performance over existing fixed-memory baselines robustly across all evaluated tasks. Furthermore, the evolved memory programs exhibit structurally distinct processing mechanisms for each domain. This finding indicates that specializing the memory mechanism for a given task explores a broad design space and provides a superior solution compared to general-purpose memory paradigms.
【21】On Higher-Order Geometric Refinements of Classical Covariance Asymptotics: An Approach via Intrinsic and Extrinsic Information Geometry
标题：经典协方差渐进学的高级几何细化：通过内在和外在信息几何的方法
链接：https://arxiv.org/abs/2604.12725
摘要：经典的Fisher信息渐近通过对数似然的局部二次近似来描述正则有效估计的协方差，因此仅捕获一阶几何。在曲线模型中，包括混合物、曲线指数族、潜变量模型和流形约束参数空间，有限样本行为可以系统地偏离这些预测。我们通过将正则参数族视为具有Fisher-Rao度量的黎曼流形\（（Θ，g）\），通过平方根密度映射沉浸在\（L^2（μ）\）中，开发了一种坐标不变，曲率感知的精化。在适当的正则性和矩假设下，我们导出了分数根一阶有效估计的前导\（n^{-1}I（θ）^{-1}\）协方差项的\（n^{-2}\）校正。这种修正由一个张量P_{ij}控制，该张量可以正则地分解为三个部分：一个Fisher-Rao曲率张量的内在Ricci型收缩，一个第二基本形式的外在Gram型收缩，以及一个编码高阶概率信息的Hellinger差异张量，而高阶概率信息不是由浸入几何单独确定的。外项是半正定的，全校正在光滑重参数化下是不变的，并且对于全指数族，全校正完全为零。然后，我们将图片扩展到奇异模型，其中Fisher信息退化。使用分辨率的奇异性下的一个加性正常的交叉假设，我们描述了解决度量，学习率和后验均方误差的作用的真实的日志典型阈值，和一个基于曲率的协方差扩展的解决空间，恢复定期的理论作为一个特例。该框架还提出了弱可识别性的几何诊断以及正则化和优化的曲率感知原则。
摘要：Classical Fisher-information asymptotics describe the covariance of regular efficient estimators through the local quadratic approximation of the log-likelihood, and thus capture first-order geometry only. In curved models, including mixtures, curved exponential families, latent-variable models, and manifold-constrained parameter spaces, finite-sample behavior can deviate systematically from these predictions. We develop a coordinate-invariant, curvature-aware refinement by viewing a regular parametric family as a Riemannian manifold \((Θ,g)\) with Fisher--Rao metric, immersed in \(L^2(μ)\) through the square-root density map. Under suitable regularity and moment assumptions, we derive an \(n^{-2}\) correction to the leading \(n^{-1}I(θ)^{-1}\) covariance term for score-root, first-order efficient estimators. The correction is governed by a tensor \(P_{ij}\) that decomposes canonically into three parts, an intrinsic Ricci-type contraction of the Fisher--Rao curvature tensor, an extrinsic Gram-type contraction of the second fundamental form, and a Hellinger discrepancy tensor encoding higher-order probabilistic information not determined by immersion geometry alone. The extrinsic term is positive semidefinite, the full correction is invariant under smooth reparameterization, and it vanishes identically for full exponential families. We then extend the picture to singular models, where Fisher information degenerates. Using resolution of singularities under an additive normal crossing assumption, we describe the resolved metric, the role of the real log canonical threshold in learning rates and posterior mean-squared error, and a curvature-based covariance expansion on the resolved space that recovers the regular theory as a special case. This framework also suggests geometric diagnostics of weak identifiability and curvature-aware principles for regularization and optimization.
【22】Fine-tuning Factor Augmented Neural Lasso for Heterogeneous Environments
标题：用于异类环境的微调因子增强神经套索
链接：https://arxiv.org/abs/2604.12288
备注：Authors are listed in alphabetical order
摘要：微调是一种广泛使用的策略，用于使预训练模型适应新任务，但其在高维非参数设置中的方法和理论特性尚未得到开发。本文介绍了微调因子增强神经Lasso（FAN-Lasso），这是一种用于高维非参数回归的迁移学习框架，具有变量选择，同时处理协变量和后验偏移。我们使用低秩因子结构来管理高维相关协变量，并提出了一种新的残差微调分解，其中目标函数表示为冻结源函数和其他变量的变换，以实现迁移学习和非参数变量选择。来自源预测器的这种增强特征允许将知识转移到目标域，并降低那里的模型复杂性。我们推导出微调FAN-Lasso的最小最大最优超额风险界限，在相对样本大小和函数复杂性方面表征精确条件，在此条件下，微调产生了比单任务学习更快的统计加速。所提出的框架还提供了一个理论上的角度参数有效的微调方法。在不同的协变量和后移场景中进行的大量数值实验表明，微调FAN-Lasso始终优于标准基线，即使在严格的目标样本量限制下也能达到接近oracle的性能，从而从经验上验证了推导出的比率。
摘要：Fine-tuning is a widely used strategy for adapting pre-trained models to new tasks, yet its methodology and theoretical properties in high-dimensional nonparametric settings with variable selection have not yet been developed. This paper introduces the fine-tuning factor augmented neural Lasso (FAN-Lasso), a transfer learning framework for high-dimensional nonparametric regression with variable selection that simultaneously handles covariate and posterior shifts. We use a low-rank factor structure to manage high-dimensional dependent covariates and propose a novel residual fine-tuning decomposition in which the target function is expressed as a transformation of a frozen source function and other variables to achieve transfer learning and nonparametric variable selection. This augmented feature from the source predictor allows for the transfer of knowledge to the target domain and reduces model complexity there. We derive minimax-optimal excess risk bounds for the fine-tuning FAN-Lasso, characterizing the precise conditions, in terms of relative sample sizes and function complexities, under which fine-tuning yields statistical acceleration over single-task learning. The proposed framework also provides a theoretical perspective on parameter-efficient fine-tuning methods. Extensive numerical experiments across diverse covariate- and posterior-shift scenarios demonstrate that the fine-tuning FAN-Lasso consistently outperforms standard baselines and achieves near-oracle performance even under severe target sample size constraints, empirically validating the derived rates.
【23】On the continuum limit of t-SNE for data visualization
标题：关于数据可视化的t-SNE的连续体极限
链接：https://arxiv.org/abs/2604.12041
摘要：这项工作是关注的连续极限的基于图形的数据可视化技术称为t-分布随机邻居嵌入（t-SNE），这是广泛用于可视化数据在各种应用程序，但仍然很少从理论的角度来理解。t-SNE算法通过最小化表示高维数据和其低维表示的相似性矩阵之间的Kullback-Leibler散度来产生可视化。我们证明，作为数据点的数量$n \到\infty$，经过自然的重新缩放和适用的参数制度，Kullback-Leibler散度与数据点$n \to \infty$的数量一致，并且相似性图保持稀疏，其中连续变分问题涉及非凸梯度正则化项和对可视化空间中的概率密度函数的幅度的惩罚。这两项表示t-SNE算法中吸引力和排斥力的连续极限。 由于连续变分问题缺乏凸性，适定性问题只能部分解决。我们表明，当两个维度都是1 $，这个问题承认一个独特的光滑极小，以及无限数量的不连续极小（解释在一个轻松的意义上）。这与t-SNE在可视化中以看似任意的方式分离数据的经验观察能力很好地吻合。能量也与着名的不适定Perona-Malik方程密切相关，该方程用于去噪和简化图像。我们提出了验证连续极限的数值结果，提供了一些初步的结果，在更高的维度限制精力充沛的问题的微妙性质，并强调了未来工作的几个问题。
摘要 ：This work is concerned with the continuum limit of a graph-based data visualization technique called the t-Distributed Stochastic Neighbor Embedding (t-SNE), which is widely used for visualizing data in a variety of applications, but is still poorly understood from a theoretical standpoint. The t-SNE algorithm produces visualizations by minimizing the Kullback-Leibler divergence between similarity matrices representing the high dimensional data and its low dimensional representation. We prove that as the number of data points $n \to \infty$, after a natural rescaling and in applicable parameter regimes, the Kullback-Leibler divergence is consistent as the number of data points $n \to \infty$ and the similarity graph remains sparse with a continuum variational problem that involves a non-convex gradient regularization term and a penalty on the magnitude of the probability density function in the visualization space. These two terms represent the continuum limits of the attraction and repulsion forces in the t-SNE algorithm. Due to the lack of convexity in the continuum variational problem, the question of well-posedeness is only partially resolved. We show that when both dimensions are $1$, the problem admits a unique smooth minimizer, along with an infinite number of discontinuous minimizers (interpreted in a relaxed sense). This aligns well with the empirically observed ability of t-SNE to separate data in seemingly arbitrary ways in the visualization. The energy is also very closely related to the famously ill-posed Perona-Malik equation, which is used for denoising and simplifying images. We present numerical results validating the continuum limit, provide some preliminary results about the delicate nature of the limiting energetic problem in higher dimensions, and highlight several problems for future work.
【24】FlowBoost Reveals Phase Transitions and Spectral Structure in Finite Free Information Inequalities
标题：FlowBoost揭示有限自由信息不等式中的相转变和谱结构
链接：https://arxiv.org/abs/2604.11922
摘要：使用FlowBoost，一个用于极值结构发现的闭环深度生成优化框架，我们研究了有限自由加法卷积$\boxplus_n$下实根多项式的有限自由Stam不等式的$\ell^p$-推广。在$p=2$，FlowBoost发现Hermite对作为唯一的等式情况，并揭示了线性卷积映射在这个极值点的谱结构。因此，我们猜想双随机耦合矩阵E_n在均值零子空间上的奇异值为：k= 1，n-1，与n无关.在这个猜想的条件下，我们得到了一个尖锐的局部稳定常数和有限的自由CLT收敛速度，都在$n$一致。利用$\ell^p$-Fisher信息引入了一个单参数的$p$-Stam不等式族，并证明了对任意p>2$，Hermite对本身都违反该不等式，亏额的符号由E_n$的$\ell^p$-收缩比决定.通过FlowBoost的系统计算支持这样的猜想：$p^*\！2$是尖锐的临界指数。当p<2时，极值组态会发生分叉，这意味着它们会变成具有双峰根结构的非匹配对，仅在p\to 2^-$时收敛回厄米对角线。我们的研究结果表明，FlowBoost，可以是一个有效的工具，在无穷维极值问题的数学发现。
摘要：Using FlowBoost, a closed-loop deep generative optimization framework for extremal structure discovery, we investigate $\ell^p$-generalizations of the finite free Stam inequality for real-rooted polynomials under finite free additive convolution $\boxplus_n$. At $p=2$, FlowBoost finds the Hermite pair as the unique equality case and reveals the spectral structure of the linearized convolution map at this extremal point. As a result, we conjecture that the singular values of the doubly stochastic coupling matrix $E_n$ on the mean-zero subspace are ${2^{-k/2}:k=1,\ldots,n-1}$, independent of $n$. Conditional on this conjecture, we obtain a sharp local stability constant and the finite free CLT convergence rate, both uniform in $n$. We introduce a one-parameter family of $p$-Stam inequalities using $\ell^p$-Fisher information and prove that the Hermite pair itself violates the inequality for every $p>2$, with the sign of the deficit governed by the $\ell^p$-contraction ratio of $E_n$. Systematic computation via FlowBoost supports the conjecture that $p^*\!=2$ is the sharp critical exponent. For $p<2$, the extremal configurations undergo a bifurcation, meaning that they become non-matching pairs with bimodal root structure, converging back to the Hermite diagonal only as $p\to 2^-$. Our findings demonstrate that FlowBoost, can be an effective tool of mathematical discovery in infinite-dimensional extremal problems.
【25】Inverse Design of Inorganic Compounds with Generative AI
标题：利用生成性人工智能进行无机化合物的逆设计
链接：https://arxiv.org/abs/2604.11827
摘要：机器学习正在彻底改变化学。除了预测模型加速虚拟筛选的价值之外，生成式人工智能还旨在实现逆向设计，将化合物到属性的预测范式逆转为属性到化合物的生成。化学家现在可以使用丰富的有机化学AI工具箱，包括药物发现。然而，这些方法对无机化合物的应用仍然受到其固有性质所带来的挑战的限制。这篇评论分析了如何解决这些挑战，考虑广泛不同的系统，从分子到晶体，包括过渡金属络合物和微孔材料。分析重点是生成式人工智能方法如何演变为数据表示模型管道，以解决无机化合物的全部复杂性，包括其化学组成，几何形状，对称性和电子结构。未来的方向，如基准标准化和综合性指标的发展，也进行了讨论。
摘要：Machine learning is revolutionizing chemistry. Beyond the value of predictive models accelerating virtual screening, generative AI aims at enabling inverse design, reversing the compound-to-property prediction paradigm into property-to-compound generation. Chemists now have access to a rich AI toolbox for organic chemistry, including drug discovery. However, the application of these methods to inorganic compounds remains limited by the challenges posed by their intrinsic nature. This Review analyzes how these challenges have been addressed, considering widely diverse systems ranging from molecules to crystals, including transition metal complexes and microporous materials. The analysis focuses on how generative AI methods have evolved towards data-representation-model pipelines that address the full complexity of inorganic compounds, including their chemical composition, geometry, symmetry, and electronic structure. Future directions, like benchmark standardization and the development of synthesizability metrics, are also discussed.
机器翻译由腾讯交互翻译提供，仅供参考
点击“阅读原文”获取带摘要的学术速递
Python社区是高质量的Python/Django开发社区
本文地址：http://www.python88.com/topic/195124
本文地址：http://www.python88.com/topic/195124