|
创作新主题 |
社区所有版块导航
DATA
| docker Elasticsearch |
WEB开发
| linux MongoDB Redis DATABASE NGINX 其他Web框架 web工具 zookeeper tornado NoSql Bootstrap js peewee Git bottle IE MQ Jquery |
机器学习
| 机器学习算法 |
产品
| 短视频 |
印度
| 印度 |
一周十大热门主题
点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！
cs.LG 方向，今日共计169篇
大模型相关(18篇)
【1】Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models
标题：匹配特征，而不是令牌：基于能量的语言模型微调
链接：https://arxiv.org/abs/2603.12248
摘要：交叉熵（CE）训练为语言模型提供了密集和可扩展的监督，但它在教师强制下优化了下一个令牌预测，而不是在模型推出下优化了序列级行为。我们引入了一个功能匹配目标的语言模型微调，目标完成分布的序列级统计，提供密集的语义反馈，而不需要特定于任务的验证或偏好模型。为了有效地优化这一目标，我们提出了基于能量的微调（EBFT），它使用步进块并行采样来同时从嵌套前缀生成多个推出，对这些推出进行批量特征提取，并使用生成的嵌入来执行基于策略的策略梯度更新。我们提出了一个理论观点，将EBFT连接到KL正则化特征匹配和基于能量的建模。从经验上看，在问答编码、非结构化编码和翻译中，EBFT与RLVR相匹配，在下游准确性上优于SFT，同时实现了比这两种方法更低的验证交叉熵。
摘要：Cross-entropy (CE) training provides dense and scalable supervision for language models, but it optimizes next-token prediction under teacher forcing rather than sequence-level behavior under model rollouts. We introduce a feature-matching objective for language-model fine-tuning that targets sequence-level statistics of the completion distribution, providing dense semantic feedback without requiring a task-specific verifier or preference model. To optimize this objective efficiently, we propose energy-based fine-tuning (EBFT), which uses strided block-parallel sampling to generate multiple rollouts from nested prefixes concurrently, batches feature extraction over these rollouts, and uses the resulting embeddings to perform an on-policy policy-gradient update. We present a theoretical perspective connecting EBFT to KL-regularized feature-matching and energy-based modeling. Empirically, across Q&A coding, unstructured coding, and translation, EBFT matches RLVR and outperforms SFT on downstream accuracy while achieving a lower validation cross-entropy than both methods.
【2】Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
标题：在不可验证的LLM后训练中检查推理LLMs作为法官
链接：https://arxiv.org/abs/2603.12246
摘要：推理LLMS-as-Judge可以从推理时间缩放中受益，为将推理模型的成功扩展到无法直接检查输出正确性/质量的不可验证域提供了一条有希望的路径。然而，虽然推理法官在静态评价基准方面表现出更好的业绩，但他们在实际政策培训中的有效性尚未得到系统的审查。因此，我们进行了严格的研究，以调查非推理和推理法官的实际影响，在基于学习的LLM对齐。我们的控制合成设置，其中一个“黄金标准”法官（gpt-oss-120 b）提供偏好注释训练较小的法官，揭示了非推理和推理法官之间的关键区别：非推理法官导致奖励黑客容易，而推理法官可以导致政策，实现强大的性能时，由黄金标准法官评估。有趣的是，我们发现，经过推理判断训练的策略通过学习生成高效的对抗性输出来实现如此强大的性能，这些输出也可以通过欺骗其他LLM法官来在Arena Hard等流行的基准测试中获得良好的成绩。结合我们的进一步分析，我们的研究突出了重要的发现和改进的空间，应用（推理）法学硕士法官在非可验证的法学硕士后培训。
摘要：Reasoning LLMs-as-Judges, which can benefit from inference-time scaling, provide a promising path for extending the success of reasoning models to non-verifiable domains where the output correctness/quality cannot be directly checked. However, while reasoning judges have shown better performance on static evaluation benchmarks, their effectiveness in actual policy training has not been systematically examined. Therefore, we conduct a rigorous study to investigate the actual impact of non-reasoning and reasoning judges in reinforcement-learning-based LLM alignment. Our controlled synthetic setting, where a "gold-standard" judge (gpt-oss-120b) provides preference annotations to train smaller judges, reveals key differences between non-reasoning and reasoning judges: non-reasoning judges lead to reward hacking easily, while reasoning judges can lead to policies that achieve strong performance when evaluated by the gold-standard judge. Interestingly, we find that the reasoning-judge-trained policies achieve such strong performance by learning to generate highly effective adversarial outputs that can also score well on popular benchmarks such as Arena-Hard by deceiving other LLM-judges. Combined with our further analysis, our study highlights both important findings and room for improvements for applying (reasoning) LLM-judges in non-verifiable LLM post-training.
【3】IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
标题：IsoCompute剧本：LLM RL的最佳缩放采样计算
链接：https://arxiv.org/abs/2603.12151
备注：29 pages, 27 figures. Under review
摘要：虽然缩放定律指导LLM预训练的计算分配，但大型语言模型（LLM）的强化学习（RL）后训练的类似处方仍然知之甚少。我们研究了LLM中策略RL方法的抽样计算的计算最优分配，将缩放框架作为三个资源的计算约束优化：每个问题的并行推出，每批问题的数量和更新步骤的数量。我们发现，每个问题的计算最佳并行推出数量会随着计算预算而可预测地增加，然后饱和。这种趋势在简单和困难的问题上都存在，尽管是由不同的机制驱动的：简单问题的解决方案锐化和困难问题的覆盖范围扩大。我们进一步表明，增加并行部署的数量可以减轻问题之间的干扰，而每批问题的数量主要影响训练的稳定性，并且可以在很宽的范围内进行选择。在基础模型和数据分布之间进行验证，我们的结果将RL缩放律重新定义为规定的分配规则，并为计算高效的LLM RL后训练提供实用指导。
摘要：While scaling laws guide compute allocation for LLM pre-training, analogous prescriptions for reinforcement learning (RL) post-training of large language models (LLMs) remain poorly understood. We study the compute-optimal allocation of sampling compute for on-policy RL methods in LLMs, framing scaling as a compute-constrained optimization over three resources: parallel rollouts per problem, number of problems per batch, and number of update steps. We find that the compute-optimal number of parallel rollouts per problem increases predictably with compute budget and then saturates. This trend holds across both easy and hard problems, though driven by different mechanisms: solution sharpening on easy problems and coverage expansion on hard problems. We further show that increasing the number of parallel rollouts mitigates interference across problems, while the number of problems per batch primarily affects training stability and can be chosen within a broad range. Validated across base models and data distributions, our results recast RL scaling laws as prescriptive allocation rules and provide practical guidance for compute-efficient LLM RL post-training.
【4】Resource-Efficient Iterative LLM-Based NAS with Feedback Memory
标题：一种资源高效的基于反馈存储器的迭代LLM NAS
链接：https://arxiv.org/abs/2603.12091
摘要：神经结构搜索（NAS）自动化网络设计，但传统的方法需要大量的计算资源。我们提出了一个闭环管道，利用大型语言模型（LLM）在单个消费级GPU上迭代生成、评估和优化卷积神经网络架构，用于图像分类，而无需LLM微调。我们的方法的核心是一个历史反馈记忆的启发马尔可夫链：一个滑动窗口的$K{=}5$最近的改进尝试保持上下文大小不变，同时提供足够的信号迭代学习。与之前丢弃失败轨迹的LLM优化器不同，每个历史条目都是一个结构化的诊断三元组-记录识别的问题，建议的修改和结果-将代码执行失败视为第一类学习信号。双LLM专业化减少了每次调用的认知负载：代码生成器生成可执行的PyTorch架构，而Prompt Improver处理诊断推理。由于LLM和架构培训共享有限的VRAM，因此搜索隐含地倾向于适合边缘部署的紧凑，硬件高效的模型。我们评估了三个冻结的自适应LLM（${\leq}7$B参数）在一个无约束的开放代码空间中多达2000次迭代，使用CIFAR-10，CIFAR-100和ImageNette上的一个epoch代理准确度作为快速排名信号。在CIFAR-10上，DeepSeek-Coder-6.7B从28.2%提高到69.2%，Qwen2.5- 7 B从50.0%提高到71.5%，GLM-5从43.2%提高到62.0%。在单个RTX~4090上完成完整的2000次迭代搜索仅需${\approx}18$ GPU小时，为无云基础架构的LLM驱动的NAS建立了低预算、可重复且硬件感知的范例。
摘要：Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources. We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures for image classification on a single consumer-grade GPU without LLM fine-tuning. Central to our approach is a historical feedback memory inspired by Markov chains: a sliding window of $K{=}5$ recent improvement attempts keeps context size constant while providing sufficient signal for iterative learning. Unlike prior LLM optimizers that discard failure trajectories, each history entry is a structured diagnostic triple -- recording the identified problem, suggested modification, and resulting outcome -- treating code execution failures as first-class learning signals. A dual-LLM specialization reduces per-call cognitive load: a Code Generator produces executable PyTorch architectures while a Prompt Improver handles diagnostic reasoning. Since both the LLM and architecture training share limited VRAM, the search implicitly favors compact, hardware-efficient models suited to edge deployment. We evaluate three frozen instruction-tuned LLMs (${\leq}7$B parameters) across up to 2000 iterations in an unconstrained open code space, using one-epoch proxy accuracy on CIFAR-10, CIFAR-100, and ImageNette as a fast ranking signal. On CIFAR-10, DeepSeek-Coder-6.7B improves from 28.2% to 69.2%, Qwen2.5-7B from 50.0% to 71.5%, and GLM-5 from 43.2% to 62.0%. A full 2000-iteration search completes in ${\approx}18$ GPU hours on a single RTX~4090, establishing a low-budget, reproducible, and hardware-aware paradigm for LLM-driven NAS without cloud infrastructure.
【5】Continual Learning with Vision-Language Models via Semantic-Geometry Preservation
标题：通过语义几何保留使用视觉语言模型进行持续学习
链接：https://arxiv.org/abs/2603.12055
备注：14 pages, 11 figures, under review
摘要：预训练的视觉语言模型（VLM）的持续学习容易发生灾难性的遗忘，但目前的方法适应新的任务，而没有显式地保留从预训练和以前的阶段继承的跨模态语义几何，允许新任务监督诱导几何失真。我们观察到，最明显的漂移往往集中在脆弱的社区附近的新旧语义界面，共享的视觉模式很容易重新解释新的文本语义。为了解决这一问题下的一个示例自由的约束，我们提出了语义几何保存持续学习（SeGP-CL）。SeGP-CL首先通过双目标投影梯度下降（DPGD）构建一组紧凑的对抗锚来探测易漂移区域，这将选定的新任务种子推向旧类语义，同时在原始视觉空间中保持忠实。在训练过程中，我们通过锚点引导的跨模态几何蒸馏（ACGD）来保持跨模态结构，并通过轻量级文本语义几何正则化（TSGR）来稳定跨任务的文本参考框架。训练后，我们估计锚引起的原始空间漂移转移旧的视觉原型，并通过融合跨通道和视觉线索进行双路径推理。在五个持续学习基准上进行的大量实验表明，SeGP-CL始终如一地提高了稳定性和前向传输，实现了最先进的性能，同时更好地保留了VLM的语义几何。
摘要：Continual learning of pretrained vision-language models (VLMs) is prone to catastrophic forgetting, yet current approaches adapt to new tasks without explicitly preserving the cross-modal semantic geometry inherited from pretraining and previous stages, allowing new-task supervision to induce geometric distortion. We observe that the most pronounced drift tends to concentrate in vulnerable neighborhoods near the old-new semantic interface, where shared visual patterns are easily re-explained by new textual semantics. To address this under an exemplar-free constraint, we propose Semantic Geometry Preservation for Continual Learning (SeGP-CL). SeGP-CL first probes the drift-prone region by constructing a compact set of adversarial anchors with dual-targeted projected gradient descent (DPGD), which drives selected new-task seeds toward old-class semantics while remaining faithful in raw visual space. During training, we preserve cross-modal structure by anchor-guided cross-modal geometry distillation (ACGD), and stabilize the textual reference frame across tasks via a lightweight text semantic-geometry regularization (TSGR). After training, we estimate anchor-induced raw-space drift to transfer old visual prototypes and perform dual-path inference by fusing cross-modal and visual cues. Extensive experiments on five continual learning benchmarks demonstrate that SeGP-CL consistently improves stability and forward transfer, achieving state-of-the-art performance while better preserving semantic geometry of VLMs.
【6】BTZSC: A Benchmark for Zero-Shot Text Classification Across Cross-Encoders, Embedding Models, Rerankers and LLMs
标题：BTZSC：跨交叉编码器、嵌入模型、重新定位器和LLM的Zero-Shot文本分类基准
链接：https://arxiv.org/abs/2603.11991
备注：Accepted at ICLR 2026. 31 pages, 5 figures, 9 tables. Code: https://github.com/IliasAarab/btzsc ; Dataset: https://huggingface.co/datasets/btzsc/btzsc ; Leaderboard: https://huggingface.co/spaces/btzsc/btzsc-leaderboard . Proceedings of the Fourteenth International Conference on Learning Representations (ICLR 2026), 2026
摘要：Zero-shot文本分类（ZSC）通过将文本直接匹配到人类可读的标签描述来消除昂贵的特定于任务的注释。虽然早期的方法主要依赖于针对自然语言推理（NLI）进行微调的交叉编码器模型，但文本嵌入模型、重排序器和预调大语言模型（LLM）的最新进展已经挑战了基于NLI的架构的主导地位。然而，系统地比较这些不同的方法仍然很困难。现有的评估，如MTEB，通常通过监督探测或微调来合并标记的示例，从而使真正的zero-shot能力未得到充分探索。为了解决这个问题，我们引入了BTZSC，这是一个由22个公共数据集组成的综合基准，涵盖了情感、主题、意图和情感分类，捕捉了不同的领域、类别基数和文档长度。利用BTZSC，我们对四个主要的模型家族、NLI交叉编码器、嵌入模型、重排序器和调整过的LLM进行了系统的比较，其中包括38个公共和自定义检查点。我们的研究结果表明：（i）以Qwen 3-Reranker-8B为例的现代重排序器设置了具有宏F1 = 0.72的新的最新技术水平;（ii）诸如GTE-large-en-v1.5的强嵌入模型基本上缩小了准确性差距，同时提供了准确性和延迟之间的最佳权衡;（iii）在4- 12 B参数下，经调整的远程弹道导弹达到有竞争力的性能（宏F1高达0.67），特别是在主题分类上表现出色，但落后于专门的重新排序器;（iv）即使随着主干大小的增加，NLI交叉编码器也会稳定;以及（v）缩放主要使重排序器和LLM优于嵌入模型。BTZSC和附带的评估代码是公开发布的，以支持zero-shot文本理解的公平和可重复的进展。
摘要 ：Zero-shot text classification (ZSC) offers the promise of eliminating costly task-specific annotation by matching texts directly to human-readable label descriptions. While early approaches have predominantly relied on cross-encoder models fine-tuned for natural language inference (NLI), recent advances in text-embedding models, rerankers, and instruction-tuned large language models (LLMs) have challenged the dominance of NLI-based architectures. Yet, systematically comparing these diverse approaches remains difficult. Existing evaluations, such as MTEB, often incorporate labeled examples through supervised probes or fine-tuning, leaving genuine zero-shot capabilities underexplored. To address this, we introduce BTZSC, a comprehensive benchmark of 22 public datasets spanning sentiment, topic, intent, and emotion classification, capturing diverse domains, class cardinalities, and document lengths. Leveraging BTZSC, we conduct a systematic comparison across four major model families, NLI cross-encoders, embedding models, rerankers and instruction-tuned LLMs, encompassing 38 public and custom checkpoints. Our results show that: (i) modern rerankers, exemplified by Qwen3-Reranker-8B, set a new state-of-the-art with macro F1 = 0.72; (ii) strong embedding models such as GTE-large-en-v1.5 substantially close the accuracy gap while offering the best trade-off between accuracy and latency; (iii) instruction-tuned LLMs at 4--12B parameters achieve competitive performance (macro F1 up to 0.67), excelling particularly on topic classification but trailing specialized rerankers; (iv) NLI cross-encoders plateau even as backbone size increases; and (v) scaling primarily benefits rerankers and LLMs over embedding models. BTZSC and accompanying evaluation code are publicly released to support fair and reproducible progress in zero-shot text understanding.
【7】Learning Transferable Sensor Models via Language-Informed Pretraining
标题：通过知情预训练学习可移植传感器模型
链接：https://arxiv.org/abs/2603.11950
摘要：现代传感系统产生大量的未标记的多元时间序列数据。这种丰富的未标记数据使得自监督学习（SSL）成为学习可转移表示的自然方法。然而，大多数现有的方法都是针对重建或预测目标进行优化的，并且通常无法捕获下游分类和推理任务所需的语义结构。虽然最近的传感器语言对齐方法通过字幕和zero-shot传输来提高语义泛化，但它们限于固定的传感器配置，例如预定义的通道集、信号长度或时间分辨率，这阻碍了跨域适用性。为了解决这些差距，我们引入了\textbf{SLIP}（\textbf{S}ensor \textbf{L} language-\textbf{I}nformed \textbf{P}retraining），这是一个开源框架，用于学习语言对齐的表示，并在不同的传感器设置中进行推广。SLIP将对比对齐与传感器条件字幕相结合，促进了区分性理解和生成性推理。通过交叉注意重新利用预训练的仅解码器语言模型，并引入一个优雅，灵活的补丁嵌入器，SLIP在推理时支持不同的时间分辨率和可变长度的输入，而无需额外的再训练。在11个数据集上，SLIP在zero-shot传输、信号字幕和问题回答方面表现出卓越的性能。它实现了77.14%的平均线性探测准确率，与强基线相比相对提高了5.93%，并在基于传感器的问题回答中达到了64.83%的准确率。
摘要：Modern sensing systems generate large volumes of unlabeled multivariate time-series data. This abundance of unlabeled data makes self-supervised learning (SSL) a natural approach for learning transferable representations. However, most existing approaches are optimized for reconstruction or forecasting objectives and often fail to capture the semantic structure required for downstream classification and reasoning tasks. While recent sensor-language alignment methods improve semantic generalization through captioning and zero-shot transfer, they are limited to fixed sensor configurations, such as predefined channel sets, signal lengths, or temporal resolutions, which hinders cross-domain applicability. To address these gaps, we introduce \textbf{SLIP} (\textbf{S}ensor \textbf{L}anguage-\textbf{I}nformed \textbf{P}retraining), an open-source framework for learning language-aligned representations that generalize across diverse sensor setups. SLIP integrates contrastive alignment with sensor-conditioned captioning, facilitating both discriminative understanding and generative reasoning. By repurposing a pretrained decoder-only language model via cross-attention and introducing an elegant, flexible patch-embedder, SLIP supports different temporal resolutions and variable-length input at inference time without additional retraining. Across 11 datasets, SLIP demonstrates superior performance in zero-shot transfer, signal captioning, and question answering. It achieves a 77.14% average linear-probing accuracy, a 5.93% relative improvement over strong baselines, and reaches 64.83% accuracy in sensor-based question answering.
【8】MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?
标题：MobileKernelBench：LLM能否为移动设备编写高效的内核？
链接：https://arxiv.org/abs/2603.11935
摘要：大型语言模型（LLM）在代码生成方面表现出了非凡的能力，但它们为移动设备生成内核的潜力在很大程度上尚未开发。在这项工作中，我们将自动内核生成的范围扩展到移动领域，以研究核心问题：LLM可以为移动设备编写高效的内核吗？为了进行系统的调查，我们引入了MobileKernelBench，这是一个全面的评估框架，包括一个优先考虑运营商多样性和跨框架互操作性的基准，再加上一个自动化的管道，可以弥合主机-设备之间的差距，以进行设备上的验证。利用这个框架，我们对移动神经网络（MNN）的CPU后端进行了广泛的评估，揭示了当前的LLM与移动框架继承的工程复杂性和数据稀缺性作斗争;标准模型甚至微调的变体都表现出很高的编译失败率（超过54%），由于幻觉和缺乏特定领域的基础，性能增益可以忽略不计。为了克服这些限制，我们提出了移动内核代理（MoKA），一个多智能体系统，配备了仓库感知推理和计划和执行范型。在MobileKernelBench上验证，MoKA实现了最先进的性能，将编译成功率提高到93.7%，并使生成的内核中有27.4%能够提供可衡量的加速比。
摘要：Large language models (LLMs) have demonstrated remarkable capabilities in code generation, yet their potential for generating kernels specifically for mobile de- vices remains largely unexplored. In this work, we extend the scope of automated kernel generation to the mobile domain to investigate the central question: Can LLMs write efficient kernels for mobile devices? To enable systematic investigation, we introduce MobileKernelBench, a comprehensive evaluation framework comprising a benchmark prioritizing operator diversity and cross-framework interoperability, coupled with an automated pipeline that bridges the host-device gap for on-device verification. Leveraging this framework, we conduct extensive evaluation on the CPU backend of Mobile Neural Network (MNN), revealing that current LLMs struggle with the engineering complexity and data scarcity inher-ent to mobile frameworks; standard models and even fine-tuned variants exhibit high compilation failure rates (over 54%) and negligible performance gains due to hallucinations and a lack of domain-specific grounding. To overcome these limitations, we propose the Mobile K ernel A gent (MoKA), a multi-agent system equipped with repository-aware reasoning and a plan-and-execute paradigm.Validated on MobileKernelBench, MoKA achieves state-of-the-art performance, boosting compilation success to 93.7% and enabling 27.4% of generated kernelsto deliver measurable speedups over native libraries.
【9】Chem4DLLM: 4D Multimodal LLMs for Chemical Dynamics Understanding
标题：Chem4DLLM：用于化学动力学理解的4D多峰LLM
链接：https://arxiv.org/abs/2603.11924
备注：18 pages
摘要：现有的化学理解任务主要依赖于静态的分子表示，限制了它们对固有的动态现象（如键断裂或构象变化）进行建模的能力，这些现象对于化学家理解化学反应至关重要。为了解决这一差距，我们引入了化学动力学理解（ChemDU），这是一项将4D分子轨迹转换为可解释的自然语言解释的新任务。ChemDU专注于基本的动态场景，包括气相和催化反应，并要求模型来推理分子轨迹上的关键事件，例如键的形成和解离，并生成连贯的、机械性的基础叙述。为了对这种能力进行基准测试，我们构建了Chem4DBench，这是第一个将4D分子轨迹与这些设置中的专家撰写的解释配对的数据集。我们进一步提出了Chem4DLLM，这是一个统一的模型，它将等变图编码器与预训练的大型语言模型集成在一起，以显式地捕获分子几何形状和旋转动力学。我们希望ChemDU与Chem4DBench和Chem4DLLM一起，将促进动态化学理解和多模态科学推理的进一步研究。
摘要：Existing chemical understanding tasks primarily rely on static molecular representations, limiting their ability to model inherently dynamic phenomena such as bond breaking or conformational changes, which are essential for a chemist to understand chemical reactions. To address this gap, we introduce Chemical Dynamics Understanding (ChemDU), a new task that translates 4D molecular trajectories into interpretable natural-language explanations. ChemDU focuses on fundamental dynamic scenarios, including gas-phase and catalytic reactions, and requires models to reason about key events along molecular trajectories, such as bond formation and dissociation, and to generate coherent, mechanistically grounded narratives. To benchmark this capability, we construct Chem4DBench, the first dataset pairing 4D molecular trajectories with expert-authored explanations across these settings. We further propose Chem4DLLM, a unified model that integrates an equivariant graph encoder with a pretrained large language model to explicitly capture molecular geometry and rotational dynamics. We hope that ChemDU, together with Chem4DBench and Chem4DLLM, will stimulate further research in dynamic chemical understanding and multimodal scientific reasoning.
【10】FlexRec: Adapting LLM-based Recommenders for Flexible Needs via Reinforcement Learning
标题：FlexRec：通过强化学习调整基于LLM的推荐器以满足灵活需求
链接
：https://arxiv.org/abs/2603.11901
摘要：现代推荐系统必须适应不同推荐场景的动态、特定需求的目标，然而大多数传统的推荐系统都是针对单个静态目标进行优化的，并且难以根据需求重新配置行为。最近在基于学习的后期训练方面取得的进展已经解锁了LLM中强大的推理跟踪和推理能力，这表明了将它们与复杂的推荐目标相结合的原则性路线。出于这一动机，我们研究闭集自回归排名，其中LLM在用户上下文和明确的需要指令的条件下在固定的候选集上生成置换。然而，将RL应用于此设置面临两个关键障碍：（i）序列级奖励产生粗糙的信用分配，无法提供细粒度的训练信号，以及（ii）交互反馈稀疏且嘈杂，这共同导致低效且不稳定的更新。我们提出了FlexRec，这是一个训练后的RL框架，它解决了这两个问题：（1）基于剩余候选池内的反事实交换的因果接地项目级奖励，以及（2）批评指导的，不确定性感知的缩放，明确建模奖励不确定性并降低低置信度奖励的权重，以稳定稀疏监督下的学习。在不同的推荐场景和目标中，FlexRec实现了实质性的收益：它在特定于需求的排名中将NDCG@5提高了高达\textbf {59\%}，将Recall@5提高了高达\textbf {109.4\%}，并在泛化设置下进一步实现了高达\textbf {24.1\%}的Recall@5改进，优于强大的传统排序器和基于LLM的基线。
摘要：Modern recommender systems must adapt to dynamic, need-specific objectives for diverse recommendation scenarios, yet most traditional recommenders are optimized for a single static target and struggle to reconfigure behavior on demand. Recent advances in reinforcement-learning-based post-training have unlocked strong instruction-following and reasoning capabilities in LLMs, suggesting a principled route for aligning them to complex recommendation goals. Motivated by this, we study closed-set autoregressive ranking, where an LLM generates a permutation over a fixed candidate set conditioned on user context and an explicit need instruction. However, applying RL to this setting faces two key obstacles: (i) sequence-level rewards yield coarse credit assignment that fails to provide fine-grained training signals, and (ii) interaction feedback is sparse and noisy, which together lead to inefficient and unstable updates. We propose FlexRec, a post-training RL framework that addresses both issues with (1) a causally grounded item-level reward based on counterfactual swaps within the remaining candidate pool, and (2) critic-guided, uncertainty-aware scaling that explicitly models reward uncertainty and down-weights low-confidence rewards to stabilize learning under sparse supervision. Across diverse recommendation scenarios and objectives, FlexRec achieves substantial gains: it improves NDCG@5 by up to \textbf{59\%} and Recall@5 by up to \textbf{109.4\%} in need-specific ranking, and further achieves up to \textbf{24.1\%} Recall@5 improvement under generalization settings, outperforming strong traditional recommenders and LLM-based baselines.
【11】Language Generation with Replay: A Learning-Theoretic View of Model Collapse
标题：带回放的语言生成：模型崩溃的学习理论观点
链接：https://arxiv.org/abs/2603.11784
摘要：随着缩放定律将前沿大型语言模型（LLM）的训练推向不断增长的数据需求，训练管道正在接近一个可能消耗大部分公开在线文本的状态。与此同时，LLM的广泛使用增加了网络上机器生成内容的数量;这些趋势增加了生成文本重新进入未来训练语料库的可能性，增加了通常称为模型崩溃的性能下降的相关风险。在实践中，模型开发人员通过数据清理、水印、合成数据策略，或者在某些情况下，幸福的忽略来解决这个问题。然而，生成模型中的模型崩溃问题还没有从学习理论的角度进行研究：我们通过极限框架中语言生成的理论镜头来研究它，引入一个重放对手，用生成器自己的过去输出来增强示例流。我们的主要贡献是一个细粒度的学习理论表征时，重播从根本上限制生成：虽然重播是良性的均匀生成的最强的概念，它可证明创建分离的非均匀生成和生成的限制较弱的概念。有趣的是，我们的积极结果反映了在实践中广泛使用的算法，如数据清理，水印和输出过滤，而我们的分离显示了这些想法何时会失败。
摘要：As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed. At the same time, widespread LLM usage increases the volume of machine-generated content on the web; together, these trends raise the likelihood of generated text re-entering future training corpora, increasing the associated risk of performance degradation often called model collapse. In practice, model developers address this concern through data cleaning, watermarking, synthetic-data policies, or, in some cases, blissful ignorance. However, the problem of model collapse in generative models has not been examined from a learning-theoretic perspective: we study it through the theoretical lens of the language generation in the limit framework, introducing a replay adversary that augments the example stream with the generator's own past outputs. Our main contribution is a fine-grained learning-theoretic characterization of when replay fundamentally limits generation: while replay is benign for the strongest notion of uniform generation, it provably creates separations for the weaker notions of non-uniform generation and generation in the limit. Interestingly, our positive results mirror heuristics widely used in practice, such as data cleaning, watermarking, and output filtering, while our separations show when these ideas can fail.
【12】Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning
标题：简单食谱作品：视觉-语言-动作模型是具有强化学习的自然持续学习者
链接：https://arxiv.org/abs/2603.11653
摘要：视觉-语言-动作（VLA）模型的连续强化学习（CRL）是一个很有前途的方向，可以适应开放的，不断发展的环境中自我改进的体现代理。然而，来自持续学习的传统智慧表明，朴素的顺序微调（Seq. FT）导致灾难性的遗忘，需要复杂的CRL策略。在这项工作中，我们退后一步，对三种模型和五种具有挑战性的终身RL基准中的大型预训练VLA的CRL进行了系统的研究。我们发现，与既定的信念相反，简单的Seq。具有低秩自适应（LoRA）的FT非常强大：它实现了高可塑性，几乎没有遗忘，并且保持了强大的zero-shot泛化能力，通常优于更复杂的CRL方法。通过详细的分析，我们表明这种鲁棒性来自大型预训练模型，参数有效适应和策略RL之间的协同作用。这些组件一起重塑了稳定性-可塑性的权衡，使持续的适应既稳定又可扩展。我们的研究结果将顺序微调定位为使用VLA进行持续RL的强大方法，并为大型模型时代的终身学习提供了新的见解。代码可在github.com/UT-Austin-RobIn/continual-vla-rl上获得。
摘要：Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across three models and five challenging lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at github.com/UT-Austin-RobIn/continual-vla-rl.
【13】Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
标题：大型语言模型的越狱缩放定律：多项-指数交叉
链接：https://arxiv.org/abs/2603.11331
摘要 ：对抗性攻击可以可靠地将安全对齐的大型语言模型转向不安全的行为。从经验上讲，我们发现，对抗性的注入攻击可以将攻击成功率从没有注入时观察到的缓慢多项式增长放大到随着推理时间样本数量的指数增长。为了解释这一现象，我们提出了一个理论生成模型的代理语言的自旋玻璃系统在复制对称性破缺制度，其中世代是从相关的吉布斯测度和一个子集的低能量，大小偏向集群被指定为不安全的。在这个框架内，我们分析了基于即时注入的越狱。短注入提示对应于朝向不安全的集群中心对准的弱磁场，并且产生攻击成功率与推断时间样本的数量的幂律缩放，而长注入提示，即，强磁场，产生指数缩放。我们通过分析得出这些行为，并在大型语言模型上进行了实证验证。这两个政权之间的过渡是由于在强磁场下的自旋链中出现了有序相，这表明注入的越狱提示增强了语言模型中的对抗秩序。
摘要：Adversarial attacks can reliably steer safety-aligned large language models toward unsafe behavior. Empirically, we find that adversarial prompt-injection attacks can amplify attack success rate from the slow polynomial growth observed without injection to exponential growth with the number of inference-time samples. To explain this phenomenon, we propose a theoretical generative model of proxy language in terms of a spin-glass system operating in a replica-symmetry-breaking regime, where generations are drawn from the associated Gibbs measure and a subset of low-energy, size-biased clusters is designated unsafe. Within this framework, we analyze prompt injection-based jailbreaking. Short injected prompts correspond to a weak magnetic field aligned towards unsafe cluster centers and yield a power-law scaling of attack success rate with the number of inference-time samples, while long injected prompts, i.e., strong magnetic field, yield exponential scaling. We derive these behaviors analytically and confirm them empirically on large language models. This transition between two regimes is due to the appearance of an ordered phase in the spin chain under a strong magnetic field, which suggests that the injected jailbreak prompt enhances adversarial order in the language model.
【14】Markovian Generation Chains in Large Language Models
标题：大型语言模型中的马尔科夫生成链
链接：https://arxiv.org/abs/2603.11228
摘要：大型语言模型（LLM）的广泛使用提出了一个重要的问题：当它们被LLM反复处理时，文本是如何演变的？在本文中，我们将这种迭代推理过程定义为马尔可夫生成链，其中每一步都采用特定的提示模板和先前的输出作为输入，而不包括任何先前的记忆。在迭代改写和往返翻译实验中，输出要么收敛到一个小的递归集，要么在有限的范围内继续产生新的句子。通过对模拟数据的逐层马尔可夫链建模和分析，我们表明迭代过程可以根据温度参数和初始输入句子等因素增加或减少句子多样性。这些结果提供了有价值的见解的动态迭代LLM推理及其对多智能体LLM系统的影响。
摘要：The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous output as input, without including any prior memory. In iterative rephrasing and round-trip translation experiments, the output either converges to a small recurrent set or continues to produce novel sentences over a finite horizon. Through sentence-level Markov chain modeling and analysis of simulated data, we show that iterative process can either increase or reduce sentence diversity depending on factors such as the temperature parameter and the initial input sentence. These results offer valuable insights into the dynamics of iterative LLM inference and their implications for multi-agent LLM systems.
【15】Security-by-Design for LLM-Based Code Generation: Leveraging Internal Representations for Concept-Driven Steering Mechanisms
标题：基于LLM的代码生成的设计安全性：利用内部表示来实现概念驱动的引导机制
链接：https://arxiv.org/abs/2603.11212
备注：to be published in the IEEE European Symposium on Security and Privacy (EuroS&P)'26
摘要：大型语言模型（LLM）在理解自然语言和生成复杂代码方面表现出卓越的能力。然而，随着从业者采用CodeLLM来执行越来越关键的开发任务，研究表明，这些模型经常生成功能正确但不安全的代码，从而带来重大的安全风险。虽然已经提出了多种方法来提高基于AI的代码生成的安全性，但组合基准测试表明，这些方法仍然不足以用于实际使用，在功能正确性和安全性方面只实现了有限的改进。这是因为在理解代码生成的内部机制和安全漏洞的根本原因方面存在根本性的差距，迫使研究人员依赖经验主义和经验观察。在这项工作中，我们调查了CodeLLM中安全概念的内部表示，揭示了模型在生成不安全代码时通常会意识到漏洞。通过系统的评估，我们证明了CodeLLM可以区分安全子概念，从而比以前的黑盒方法进行更细粒度的分析。利用这些见解，我们提出了安全概念指导CodeLLM（SCS-Code）。在令牌生成期间，SCS-Code将LLM的内部表示转向安全和功能代码输出，从而实现可以集成到现有代码模型中的轻量级和模块化机制。我们的方法实现了卓越的性能相比，国家的最先进的方法在多个安全的编码基准。
摘要：Large Language Models (LLMs) show remarkable capabilities in understanding natural language and generating complex code. However, as practitioners adopt CodeLLMs for increasingly critical development tasks, research reveals that these models frequently generate functionally correct yet insecure code, posing significant security risks. While multiple approaches have been proposed to improve security in AI-based code generation, combined benchmarks show these methods remain insufficient for practical use, achieving only limited improvements in both functional correctness and security. This stems from a fundamental gap in understanding the internal mechanisms of code generation and the root causes of security vulnerabilities, forcing researchers to rely on heuristics and empirical observations. In this work, we investigate the internal representation of security concepts in CodeLLMs, revealing that models are often aware of vulnerabilities as they generate insecure code. Through systematic evaluation, we demonstrate that CodeLLMs can distinguish between security subconcepts, enabling a more fine-grained analysis than prior black-box approaches. Leveraging these insights, we propose Secure Concept Steering for CodeLLMs (SCS-Code). During token generation, SCS-Code steers LLMs' internal representations toward secure and functional code output, enabling a lightweight and modular mechanism that can be integrated into existing code models. Our approach achieves superior performance compared to state-of-the-art methods across multiple secure coding benchmarks.
【16】Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models
标题：大型语言模型中越狱攻击的系统缩放分析
链接：https://arxiv.org/abs/2603.11149
摘要：大型语言模型仍然容易受到越狱攻击，但我们仍然缺乏对越狱成功如何与攻击者在方法，模型家族和伤害类型上的努力进行缩放的系统性理解。我们启动了一个缩放律框架越狱处理每个攻击作为一个计算有界的优化过程和测量共享FLOPs轴上的进展。我们的系统评估涵盖了四种代表性的越狱范例，包括基于优化的攻击，自我改进提示，基于采样的选择和遗传优化，跨多个模型家族和各种有害目标的规模。我们调查的比例律，攻击者的预算，攻击成功得分通过拟合一个简单的饱和指数函数FLOPs-成功的轨迹，我们从拟合曲线获得可比的效率摘要。从经验上讲，与基于优化的方法相比，基于优化的范例往往是计算效率最高的。为了解释这一差距，我们将基于恶意的更新转换为优化视图，并通过相同状态的比较显示，基于恶意的攻击在提示空间中更有效地优化。我们还表明，攻击占据了明显的成功-隐身操作点与基于加密的方法占据了高成功，高隐身区域。最后，我们发现，脆弱性是强烈的目标依赖性：涉及错误信息的伤害通常比其他非错误信息的伤害更容易引起。
摘要 ：Large language models remain vulnerable to jailbreak attacks, yet we still lack a systematic understanding of how jailbreak success scales with attacker effort across methods, model families, and harm types. We initiate a scaling-law framework for jailbreaks by treating each attack as a compute-bounded optimization procedure and measuring progress on a shared FLOPs axis. Our systematic evaluation spans four representative jailbreak paradigms, covering optimization-based attacks, self-refinement prompting, sampling-based selection, and genetic optimization, across multiple model families and scales on a diverse set of harmful goals. We investigate scaling laws that relate attacker budget to attack success score by fitting a simple saturating exponential function to FLOPs--success trajectories, and we derive comparable efficiency summaries from the fitted curves. Empirically, prompting-based paradigms tend to be the most compute-efficient compared to optimization-based methods. To explain this gap, we cast prompt-based updates into an optimization view and show via a same-state comparison that prompt-based attacks more effectively optimize in prompt space. We also show that attacks occupy distinct success--stealthiness operating points with prompting-based methods occupying the high-success, high-stealth region. Finally, we find that vulnerability is strongly goal-dependent: harms involving misinformation are typically easier to elicit than other non-misinformation harms.
【17】H2LooP Spark Preview: Continual Pretraining of Large Language Models for Low-Level Embedded Systems Code
标题：H2 LooP Spark预览：针对低级嵌入式系统代码的大型语言模型的连续预训练
链接：https://arxiv.org/abs/2603.11139
摘要：大型语言模型（LLM）在通用编程语言中表现出强大的代码生成能力，但在诸如低级嵌入式系统编程等专业领域仍然受到限制。这个领域涉及硬件寄存器操作、特定于供应商的SDK、实时操作系统API和在标准预训练语料库中代表性不足的硬件抽象层。我们引入了H2 LooP Spark Preview，这是一个持续的预训练（CPT）管道，它使用8个NVIDIA H100 GPU上具有排名稳定缩放功能的BF 16 LoRA，将OLMo-3- 7 B（一种完全开放的语言模型）适配到嵌入式系统领域。我们的训练语料库是从覆盖117个制造商的原始嵌入式系统数据的100 B令牌的存储库-令牌对构建的，使用SpecMap中提出的分层令牌到代码映射方法进行处理（Nipane等人，2026年）。由此产生的策展数据集分割包含13个嵌入式域的23.5亿令牌。使用高秩LoRA（r=512）进行连续预训练，可以获得显著的收益，将域内困惑减少70.4%，将存储库困惑减少66.1%。在跨越13个嵌入式领域的生成代码完成基准测试中，我们的7 B模型在8个类别的令牌准确性上优于Claude Opus 4.6和Qwen 3-Coder-30 B，这表明有针对性的持续预训练使较小的开放权重模型能够在专业技术任务上与前沿系统竞争。我们将Huggingface上的生产培训检查点作为开源工件发布。
摘要：Large language models (LLMs) demonstrate strong code generation abilities in general-purpose programming languages but remain limited in specialized domains such as low-level embedded systems programming. This domain involves hardware register manipulation, vendor-specific SDKs, real-time operating system APIs, and hardware abstraction layers that are underrepresented in standard pretraining corpora. We introduce H2LooP Spark Preview, a continual pretraining (CPT) pipeline that adapts the OLMo-3-7B-a fully open language model to the embedded systems domain using BF16 LoRA with rank-stabilized scaling on 8 NVIDIA H100 GPUs. Our training corpus is constructed from repository-datasheet pairs covering 100B tokens of raw embedded systems data across 117 manufacturers, processed using the hierarchical datasheet-to-code mapping approach proposed in SpecMap (Nipane et al., 2026). The resulting curated dataset split contains 23.5B tokens across 13 embedded domains. Continual pretraining with high-rank LoRA (r=512) yields substantial gains, reducing in-domain perplexity by 70.4% and held-out repository perplexity by 66.1%. On generative code completion benchmarks spanning 13 embedded domains, our 7B model outperforms Claude Opus 4.6 and Qwen3-Coder-30B on 8 categories in token accuracy, showing that targeted continual pretraining enables smaller open-weight models to rival frontier systems on specialized technical tasks. We release the production training checkpoint on Huggingface as an open-source artifact.
【18】OpenSanctions Pairs: Large-Scale Entity Matching with LLMs
标题：OpenSanctions对：大规模实体与LLM匹配
链接：https://arxiv.org/abs/2603.11051
摘要：我们发布了OpenSanctions Pairs，这是一个大规模的实体匹配基准，来自真实世界的国际制裁汇总和分析师重复数据删除。该数据集包含755，540个标记对，跨越31个国家/地区的293个异构源，具有多语言和跨脚本名称，噪声和缺失属性，以及合规工作流的典型集值字段。我们基准的生产规则为基础的匹配器（nomenklatura回归V1算法）对开放和封闭源代码的LLM在零和Few-Shot设置。现成的LLM大大超过了基于生产规则的基线（91.33\%F1），达到98.95\%F1（GPT-4 o）和98.23\%F1，具有本地可部署的开放模型（DeepSeek-R1-Distill-Qwen-14 B）。DSPy MIPROv 2即时优化可产生一致但适度的增益，而添加上下文示例几乎没有额外的好处，并且可能会降低性能。错误分析显示了互补的失败模式：基于规则的系统过度匹配（高误报），而LLM主要失败于交叉脚本音译和轻微的标识符/日期不一致。这些结果表明，在这种情况下，成对匹配性能正在接近实际上限，并激励将工作转向管道组件，如阻塞，聚类和不确定性感知审查。代码可在https://github.com/chansmi/OSINT_entity_resolution获得
摘要：We release OpenSanctions Pairs, a large-scale entity matching benchmark derived from real-world international sanctions aggregation and analyst deduplication. The dataset contains 755,540 labeled pairs spanning 293 heterogeneous sources across 31 countries, with multilingual and cross-script names, noisy and missing attributes, and set-valued fields typical of compliance workflows. We benchmark a production rule-based matcher (nomenklatura RegressionV1 algorithm) against open- and closed-source LLMs in zero- and few-shot settings. Off-the-shelf LLMs substantially outperform the production rule-based baseline (91.33\% F1), reaching up to 98.95\% F1 (GPT-4o) and 98.23\% F1 with a locally deployable open model (DeepSeek-R1-Distill-Qwen-14B). DSPy MIPROv2 prompt optimization yields consistent but modest gains, while adding in-context examples provides little additional benefit and can degrade performance. Error analysis shows complementary failure modes: the rule-based system over-matches (high false positives), whereas LLMs primarily fail on cross-script transliteration and minor identifier/date inconsistencies. These results indicate that pairwise matching performance is approaching a practical ceiling in this setting, and motivate shifting effort toward pipeline components such as blocking, clustering, and uncertainty-aware review. Code available at https://github.com/chansmi/OSINT_entity_resolution
Graph相关(图学习|图神经网络|图优化等)(4篇)
【1】AGMARL-DKS: An Adaptive Graph-Enhanced Multi-Agent Reinforcement Learning for Dynamic Kubernetes Scheduling
标题：AGMARL-DKS：用于动态Kubernetes调度的自适应图形增强多智能体强化学习
链接：https://arxiv.org/abs/2603.12031
摘要：最先进的云原生应用程序需要能够有效平衡系统稳定性、资源利用率和相关成本的智能解决方案。虽然Kubernetes默认提供了基于可行性的布局，但最近的研究工作已经探索了使用强化学习（RL）进行更智能的调度决策。然而，目前基于RL的路由器有三个主要的局限性。首先，这些集群中的大多数使用单一的集中式代理，这对于大型异构集群是不可扩展的。第二，使用多目标奖励函数的方法假设目标的简单、静态、线性组合。第三，没有以前的工作已经产生了一个压力感知调度，可以自适应动态条件。为了解决当前研究中的这些差距，我们提出了自适应图增强的多智能体强化学习动态Kubernetes库（AGMARL-DKS）。AGMARL-DKS通过引入三项主要创新来解决这些差距。首先，我们构建了一个可扩展的解决方案，通过处理调度的挑战，作为一个合作的多代理问题，每个集群节点作为一个代理，采用集中训练方法分散执行之前。其次，为了实现上下文感知和去中心化，我们使用图神经网络（GNN）在每个代理处构建全局集群上下文的状态表示。这代表了对仅依赖于局部观测的方法的改进。最后，为了在这些目标之间进行权衡，我们使用了一个压力感知的词典排序策略，而不是一个简单的，静态的线性加权这些目标。Google Kubernetes Engine（GKE）的评估显示，AGMARL-DKS在容错、利用率和成本方面明显优于默认调度器，特别是在调度批处理和关键任务工作负载方面。
摘要 ：State-of-the-art cloud-native applications require intelligent schedulers that can effectively balance system stability, resource utilisation, and associated costs. While Kubernetes provides feasibility-based placement by default, recent research efforts have explored the use of reinforcement learning (RL) for more intelligent scheduling decisions. However, current RL-based schedulers have three major limitations. First, most of these schedulers use monolithic centralised agents, which are non-scalable for large heterogeneous clusters. Second, the ones that use multi-objective reward functions assume simple, static, linear combinations of the objectives. Third, no previous work has produced a stress-aware scheduler that can react adaptively to dynamic conditions. To address these gaps in current research, we propose the Adaptive Graph-enhanced Multi-Agent Reinforcement Learning Dynamic Kubernetes Scheduler (AGMARL-DKS). AGMARL-DKS addresses these gaps by introducing three major innovations. First, we construct a scalable solution by treating the scheduling challenge as a cooperative multi-agent problem, where every cluster node operates as an agent, employing centralised training methods before decentralised execution. Second, to be context-aware and yet decentralised, we use a Graph Neural Network (GNN) to build a state representation of the global cluster context at each agent. This represents an improvement over methods that rely solely on local observations. Finally, to make trade-offs between these objectives, we use a stress-aware lexicographical ordering policy instead of a simple, static linear weighting of these objectives. The evaluations in Google Kubernetes Engine (GKE) reveal that AGMARL-DKS significantly outperforms the default scheduler in terms of fault tolerance, utilisation, and cost, especially in scheduling batch and mission-critical workloads.
【2】KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation
标题：KEPo：基于图的检索增强生成上的知识进化毒药
链接：https://arxiv.org/abs/2603.11501
备注：Accepted in the ACM Web Conference 2026 (WWW 2026)
摘要：基于图的检索增强生成（GraphRAG）从外部数据库构建知识图（KG），以增强大型语言模型（LLM）生成的及时性和准确性。然而，这种对外部数据的依赖引入了新的攻击面。攻击者可以将有毒文本注入数据库，以操纵LLM为攻击者产生有害的目标响应，选择查询。现有的研究主要集中在攻击传统的RAG系统。然而，这样的方法对GraphRAG无效。这种鲁棒性源于GraphRAG的KG抽象，其在检索之前将注入的文本重新组织成图，为了暴露GraphRAG中潜在的安全漏洞，我们提出了一种专门针对GraphRAG设计的中毒攻击方法--知识进化中毒（KEPo）。对于每个目标查询，KEPo首先根据目标答案生成一个包含中毒知识的中毒事件，通过虚构事件背景，从原始事实中伪造知识演化路径，在多目标攻击场景中，KEPo进一步连接多个攻击语料库，使其中毒知识相互增强，同时扩大中毒社区的规模，在多个数据集上的实验结果表明，KEPo在单次攻击和多次攻击中都达到了最先进的攻击成功率。目标和多目标攻击，显著优于以前的方法。
摘要：Graph-based Retrieval-Augmented Generation (GraphRAG) constructs the Knowledge Graph (KG) from external databases to enhance the timeliness and accuracy of Large Language Model (LLM) generations.However,this reliance on external data introduces new attack surfaces.Attackers can inject poisoned texts into databases to manipulate LLMs into producing harmful target responses for attacker-chosen queries.Existing research primarily focuses on attacking conventional RAG systems.However,such methods are ineffective against GraphRAG.This robustness derives from the KG abstraction of GraphRAG,which reorganizes injected text into a graph before retrieval,thereby enabling the LLM to reason based on the restructured context instead of raw poisoned passages.To expose latent security vulnerabilities in GraphRAG,we propose Knowledge Evolution Poison (KEPo),a novel poisoning attack method specifically designed for GraphRAG.For each target query,KEPo first generates a toxic event containing poisoned knowledge based on the target answer.By fabricating event backgrounds and forging knowledge evolution paths from original facts to the toxic event,it then poisons the KG and misleads the LLM into treating the poisoned knowledge as the final result.In multi-target attack scenarios,KEPo further connects multiple attack corpora,enabling their poisoned knowledge to mutually reinforce while expanding the scale of poisoned communities,thereby amplifying attack effectiveness.Experimental results across multiple datasets demonstrate that KEPo achieves state-of-the-art attack success rates for both single-target and multi-target attacks,significantly outperforming previous methods.
【3】DNS-GT: A Graph-based Transformer Approach to Learn Embeddings of Domain Names from DNS Queries
标题：DNS-GT：一种基于图形的Transformer方法，用于从DNS收件箱学习域名嵌入
链接：https://arxiv.org/abs/2603.11200
摘要：网络入侵检测系统在组织用于检测和预防网络攻击的安全策略中发挥着至关重要的作用。这些系统通常将模式检测签名与由机器学习方法提供支持的异常检测技术相结合。然而，通常提出的机器学习方法存在缺点，例如过度依赖标记数据和泛化能力有限。为了解决这些问题，已经引入了基于嵌入的方法来从网络数据（例如DNS流量）中学习表示，主要是由于其大的可用性，可以有效地推广到许多下游任务。然而，当前的方法没有适当地考虑DNS查询之间的上下文信息。在本文中，我们通过提出DNS-GT来解决这个问题，DNS-GT是一种新型的基于Transformer的模型，可以从DNS查询序列中学习域名嵌入。该模型首先以自我监督的方式进行预训练，以学习DNS活动的一般行为。然后，它可以在特定的下游任务上进行微调，利用与给定序列中其他相关查询的交互。我们对真实世界DNS数据的实验展示了我们的方法学习有效域名表示的能力。对域名分类和僵尸网络检测任务的定量评估表明，我们的方法取得了更好的结果相比，相关的基线，创造机会，进一步探索大规模的语言模型的入侵检测系统。我们的代码可在https://github.com/m-altieri/DNS-GT上获得。
摘要：Network intrusion detection systems play a crucial role in the security strategy employed by organisations to detect and prevent cyberattacks. Such systems usually combine pattern detection signatures with anomaly detection techniques powered by machine learning methods. However, the commonly proposed machine learning methods present drawbacks such as over-reliance on labeled data and limited generalization capabilities. To address these issues, embedding-based methods have been introduced to learn representations from network data, such as DNS traffic, mainly due to its large availability, that generalise effectively to many downstream tasks. However, current approaches do not properly consider contextual information among DNS queries. In this paper, we tackle this issue by proposing DNS-GT, a novel Transformer-based model that learns embeddings for domain names from sequences of DNS queries. The model is first pre-trained in a self-supervised fashion in order to learn the general behavior of DNS activity. Then, it can be finetuned on specific downstream tasks, exploiting interactions with other relevant queries in a given sequence. Our experiments with real-world DNS data showcase the ability of our method to learn effective domain name representations. A quantitative evaluation on domain name classification and botnet detection tasks shows that our approach achieves better results compared to relevant baselines, creating opportunities for further exploration of large-scale language models for intrusion detection systems. Our code is available at: https://github.com/m-altieri/DNS-GT.
【4】Graph Tokenization for Bridging Graphs and Transformers
标题：桥图和变形器的图标记化
链接：https://arxiv.org/abs/2603.11099
备注：Accepted as a poster at ICLR 2026. Code is available at https://github.com/BUPT-GAMMA/Graph-Tokenization-for-Bridging-Graphs-and-Transformers
摘要：大型预训练Transformers的成功与标记器密切相关，标记器将原始输入转换为离散符号。将这些模型扩展到图形结构数据仍然是一个重大挑战。在这项工作中，我们引入了一个图形标记化框架，通过将保留图形信息的可逆图形序列化与字节对编码（BPE）（大型语言模型（LLM）中广泛采用的标记器）相结合来生成图形的顺序表示。为了更好地捕获结构信息，图形序列化过程由图形子结构的全局统计信息指导，确保频繁出现的子结构在序列中出现得更频繁，并且可以由BPE合并为有意义的标记。实证结果表明，所提出的标记器使Transformers，如BERT，直接应用于图基准测试，而无需架构修改。所提出的方法在14个基准数据集上取得了最先进的结果，并且经常优于图神经网络和专用图Transformers。这项工作弥合了图结构数据和序列模型生态系统之间的差距。我们的代码可在\href{https：//github.com/BUPT-GAMMA/Graph-Tokenization-for-Bridging-Graphs-and-Transformers}{\color{blue}here}获得。
摘要 ：The success of large pretrained Transformers is closely tied to tokenizers, which convert raw input into discrete symbols. Extending these models to graph-structured data remains a significant challenge. In this work, we introduce a graph tokenization framework that generates sequential representations of graphs by combining reversible graph serialization, which preserves graph information, with Byte Pair Encoding (BPE), a widely adopted tokenizer in large language models (LLMs). To better capture structural information, the graph serialization process is guided by global statistics of graph substructures, ensuring that frequently occurring substructures appear more often in the sequence and can be merged by BPE into meaningful tokens. Empirical results demonstrate that the proposed tokenizer enables Transformers such as BERT to be directly applied to graph benchmarks without architectural modifications. The proposed approach achieves state-of-the-art results on 14 benchmark datasets and frequently outperforms both graph neural networks and specialized graph transformers. This work bridges the gap between graph-structured data and the ecosystem of sequence models. Our code is available at \href{https://github.com/BUPT-GAMMA/Graph-Tokenization-for-Bridging-Graphs-and-Transformers}{\color{blue}here}.
Transformer(8篇)
【1】HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
标题：HiAP：视觉变形者的多粒度随机自动修剪框架
链接：https://arxiv.org/abs/2603.12222
备注：14 pages, 9 figures, 3 Tables
摘要：Vision Transformers需要大量的计算资源和内存带宽，严重限制了它们在边缘设备上的部署。虽然最近的结构化修剪方法成功地减少了理论FLOP，但它们通常以单个结构粒度操作，并依赖于具有事后阈值的复杂的多级管道来满足稀疏预算。在本文中，我们提出了分层自动修剪（HiAP），这是一种连续松弛框架，可以在单个端到端训练阶段发现最佳子网络，而无需手动重要性分析或预定义的每层稀疏目标。HiAP在多个粒度上引入了随机Gumbel-Sigmoid门：宏门用于修剪整个注意力头和FFN块，微门用于选择性地修剪头内维度和FFN神经元。通过同时优化这两个级别，HiAP解决了加载大型矩阵的内存限制开销和计算限制数学运算。HiAP自然收敛到稳定的子网络使用的损失函数，结合了结构可行性处罚和分析FLOP。在ImageNet上进行的大量实验表明，HiAP有机地发现了高效的架构，并为DeiT-Small等模型实现了具有竞争力的准确性-效率Pareto前沿，与复杂的多阶段方法的性能相匹配，同时显着简化了部署管道。
摘要：Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on edge devices. While recent structured pruning methods successfully reduce theoretical FLOPs, they typically operate at a single structural granularity and rely on complex, multi-stage pipelines with post-hoc thresholding to satisfy sparsity budgets. In this paper, we propose Hierarchical Auto-Pruning (HiAP), a continuous relaxation framework that discovers optimal sub-networks in a single end-to-end training phase without requiring manual importance heuristics or predefined per-layer sparsity targets. HiAP introduces stochastic Gumbel-Sigmoid gates at multiple granularities: macro-gates to prune entire attention heads and FFN blocks, and micro-gates to selectively prune intra-head dimensions and FFN neurons. By optimizing both levels simultaneously, HiAP addresses both the memory-bound overhead of loading large matrices and the compute-bound mathematical operations. HiAP naturally converges to stable sub-networks using a loss function that incorporates both structural feasibility penalties and analytical FLOPs. Extensive experiments on ImageNet demonstrate that HiAP organically discovers highly efficient architectures, and achieves a competitive accuracy-efficiency Pareto frontier for models like DeiT-Small, matching the performance of sophisticated multi-stage methods while significantly simplifying the deployment pipeline.
【2】EnTransformer: A Deep Generative Transformer for Multivariate Probabilistic Forecasting
标题：EnTransformer：用于多元概率预测的深度生成Transformer
链接：https://arxiv.org/abs/2603.11909
摘要：可靠的不确定性量化在多变量时间序列预测问题中至关重要，这些问题出现在能源系统和交通网络等领域。虽然基于transformer的架构最近在序列建模方面取得了很好的性能，但大多数概率预测方法都依赖于限制性参数似然或基于分位数的目标。他们可能很难在多个相关的时间序列中捕获复杂的联合预测分布。这项工作提出了EnTransformer，这是一个深度生成预测框架，它集成了engression，一种用于建模条件分布的随机学习范式，具有Transformers的表达序列建模功能。所提出的方法将随机噪声注入到模型表示中，并优化基于能量的评分目标，以直接学习条件预测分布，而无需施加参数假设。这种设计使EnTransformer能够生成一致的多变量预测轨迹，同时保留Transformers有效建模长期时间依赖性和跨系列相互作用的能力。我们在多变量概率预测的几个广泛使用的基准上评估了我们提出的EnTransformer，包括电力，交通，太阳能，出租车，KDD杯和维基百科数据集。实验结果表明，EnTransformer产生良好校准的概率预测，并始终优于基准模型。
摘要：Reliable uncertainty quantification is critical in multivariate time series forecasting problems arising in domains such as energy systems and transportation networks, among many others. Although Transformer-based architectures have recently achieved strong performance for sequence modeling, most probabilistic forecasting approaches rely on restrictive parametric likelihoods or quantile-based objectives. They can struggle to capture complex joint predictive distributions across multiple correlated time series. This work proposes EnTransformer, a deep generative forecasting framework that integrates engression, a stochastic learning paradigm for modeling conditional distributions, with the expressive sequence modeling capabilities of Transformers. The proposed approach injects stochastic noise into the model representation and optimizes an energy-based scoring objective to directly learn the conditional predictive distribution without imposing parametric assumptions. This design enables EnTransformer to generate coherent multivariate forecast trajectories while preserving Transformers' capacity to effectively model long-range temporal dependencies and cross-series interactions. We evaluate our proposed EnTransformer on several widely used benchmarks for multivariate probabilistic forecasting, including Electricity, Traffic, Solar, Taxi, KDD-cup, and Wikipedia datasets. Experimental results demonstrate that EnTransformer produces well-calibrated probabilistic forecasts and consistently outperforms the benchmark models.
【3】Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks
标题：SoftmaxTransformer中注意力下沉是可以证明的必要：来自触发条件任务的证据
链接：https://arxiv.org/abs/2603.11487
备注：21 pages, 8 figures
摘要：Transformers通常显示一个注意力下沉：概率质量集中在一个固定的，内容不可知的位置。我们证明了计算一个简单的条件行为必然会导致softmax自我注意模型的下沉。我们的结果形式化了一个熟悉的直觉：概率单纯形上的归一化必须迫使注意力崩溃到一个稳定的锚点上，以实现默认状态（例如，当模型需要忽略输入时）。我们用一个具体的任务来实例化这个：当一个指定的触发令牌出现时，模型必须返回所有先前令牌表示的平均值，否则输出零，这是一个反映了野外注意力头功能的任务（Barbero et al.，2025年; Guo等人，2024年）。我们还证明了非规范化的ReLU注意力可以在没有任何sink的情况下解决相同的任务，从而证实了规范化约束是sink行为的根本驱动力。实验验证了我们的预测，并证明它们超出了理论分析的设置：softmax模型开发了强大的汇，而ReLU注意力在单头和多头变体中消除了它们。
摘要：Transformers often display an attention sink: probability mass concentrates on a fixed, content-agnostic position. We prove that computing a simple trigger-conditional behavior necessarily induces a sink in softmax self-attention models. Our results formalize a familiar intuition: normalization over a probability simplex must force attention to collapse onto a stable anchor to realize a default state (e.g., when the model needs to ignore the input). We instantiate this with a concrete task: when a designated trigger token appears, the model must return the average of all preceding token representations, and otherwise output zero, a task which mirrors the functionality of attention heads in the wild (Barbero et al., 2025; Guo et al., 2024). We also prove that non-normalized ReLU attention can solve the same task without any sink, confirming that the normalization constraint is the fundamental driver of sink behavior. Experiments validate our predictions and demonstrate they extend beyond the theoretically analyzed setting: softmax models develop strong sinks while ReLU attention eliminates them in both single-head and multi-head variants.
【4】Multilingual Financial Fraud Detection Using Machine Learning and Transformer Models: A Bangla-English Study
标题
：使用机器学习和Transformer模型的多语言金融欺诈检测：孟加拉语-英语研究
链接：https://arxiv.org/abs/2603.11358
摘要：随着数字金融平台的快速扩张，金融欺诈检测已成为一项关键的研究挑战。虽然机器学习方法在识别欺诈活动方面表现出了很强的性能，但大多数现有研究都只关注英语数据，限制了对多语言环境的适用性。孟加拉语（Bangla）尽管有超过2.5亿人使用，但在这一领域仍基本未被开发。在这项工作中，我们调查金融欺诈检测在多语言孟加拉语-英语设置使用的数据集，包括合法和欺诈性的金融信息。我们使用TF-IDF功能以及基于transformer的架构来评估经典的机器学习模型（Logistic回归，线性SVM和Enhancement分类器）。使用5倍分层交叉验证的实验结果表明，线性SVM实现了最佳性能，91.59%的准确率和91.30%的F1得分，优于Transformer模型（89.49%的准确率，88.88%的F1）约2个百分点。Transformer显示出更高的欺诈召回率（94.19%），但误报率较高。探索性分析揭示了独特的模式：诈骗消息更长，包含紧急诱导术语，并且经常包括URL（32%）和电话号码（97%），而合法消息则具有交易确认和特定货币参考。我们的研究结果强调，具有精心制作的功能的经典机器学习在多语言欺诈检测方面仍然具有竞争力，同时也强调了语言多样性，代码混合和低资源语言限制所带来的挑战。
摘要：Financial fraud detection has emerged as a critical research challenge amid the rapid expansion of digital financial platforms. Although machine learning approaches have demonstrated strong performance in identifying fraudulent activities, most existing research focuses exclusively on English-language data, limiting applicability to multilingual contexts. Bangla (Bengali), despite being spoken by over 250 million people, remains largely unexplored in this domain. In this work, we investigate financial fraud detection in a multilingual Bangla-English setting using a dataset comprising legitimate and fraudulent financial messages. We evaluate classical machine learning models (Logistic Regression, Linear SVM, and Ensemble classifiers) using TF-IDF features alongside transformer-based architectures. Experimental results using 5-fold stratified cross-validation demonstrate that Linear SVM achieves the best performance with 91.59 percent accuracy and 91.30 percent F1 score, outperforming the transformer model (89.49 percent accuracy, 88.88 percent F1) by approximately 2 percentage points. The transformer exhibits higher fraud recall (94.19 percent) but suffers from elevated false positive rates. Exploratory analysis reveals distinctive patterns: scam messages are longer, contain urgency-inducing terms, and frequently include URLs (32 percent) and phone numbers (97 percent), while legitimate messages feature transactional confirmations and specific currency references. Our findings highlight that classical machine learning with well-crafted features remains competitive for multilingual fraud detection, while also underscoring the challenges posed by linguistic diversity, code-mixing, and low-resource language constraints.
【5】On the Computational Hardness of Transformers
标题：关于Transformer的计算硬度
链接：https://arxiv.org/abs/2603.11332
备注：46 pages, 2 figures. Abstract shortened to meet arXiv requirements
摘要：Transformer在语言、视觉等方面彻底改变了现代AI。它由$L $层组成，每个层并行运行$H $attention头，并将组合输出馈送到后续层。在attention中，输入由$N $个token组成，每个token是一个维度为$m $的向量。注意力机制涉及三个N\乘以m $矩阵，将softmax应用于中间产品。最近的几项研究提高了我们对注意力复杂性的理解。 用于变压器的已知算法独立地计算每个注意力头。这就提出了一个在TCS中以“直接和”问题为幌子反复出现的基本问题：同一问题的多个实例是否比单独解决每个实例更有效？这个问题的许多答案，无论是积极的还是消极的，都出现在通信复杂性和算法设计领域。因此，我们问是否可以更有效地计算变压器比$LH $独立评估的注意。 在本文中，我们解决了这个问题的负面影响，并给出了第一个非平凡的计算下界多头多层变压器。在小嵌入机制（$m = N ^{o（1）}$）中，分别计算$LH $注意头需要$LHN ^{2 + o（1）}$时间。我们确定，这基本上是最佳的SETH下。在大嵌入域（$m = N $）中，可以使用$LHN ^{ω + o（1）}$算术运算（加上指数）分别计算$LH $注意头，其中$ω $是矩阵乘法指数。我们通过证明当$ω> 2 $时，$LHN ^{ω-o（1）}$算术运算是必要的，证明了这是最优的。我们在大嵌入制度的下限依赖于一个新的应用程序的Baur-Strassen定理，一个强大的算法工具支撑著名的反向传播算法。
摘要：The transformer has revolutionized modern AI across language, vision, and beyond. It consists of $L$ layers, each running $H$ attention heads in parallel and feeding the combined output to the subsequent layer. In attention, the input consists of $N$ tokens, each a vector of dimension $m$. The attention mechanism involves multiplying three $N \times m$ matrices, applying softmax to an intermediate product. Several recent works have advanced our understanding of the complexity of attention. Known algorithms for transformers compute each attention head independently. This raises a fundamental question that has recurred throughout TCS under the guise of ``direct sum'' problems: can multiple instances of the same problem be solved more efficiently than solving each instance separately? Many answers to this question, both positive and negative, have arisen in fields spanning communication complexity and algorithm design. Thus, we ask whether transformers can be computed more efficiently than $LH$ independent evaluations of attention. In this paper, we resolve this question in the negative, and give the first non-trivial computational lower bounds for multi-head multi-layer transformers. In the small embedding regime ($m = N^{o(1)}$), computing $LH$ attention heads separately takes $LHN^{2 + o(1)}$ time. We establish that this is essentially optimal under SETH. In the large embedding regime ($m = N$), one can compute $LH$ attention heads separately using $LHN^{ω+ o(1)}$ arithmetic operations (plus exponents), where $ω$ is the matrix multiplication exponent. We establish that this is optimal, by showing that $LHN^{ω- o(1)}$ arithmetic operations are necessary when $ω> 2$. Our lower bound in the large embedding regime relies on a novel application of the Baur-Strassen theorem, a powerful algorithmic tool underpinning the famous backpropagation algorithm.
【6】Algorithmic Capture, Computational Complexity, and Inductive Bias of Infinite Transformers
标题：无限Transformer的数学捕获、计算复杂性和感性偏差
链接：https://arxiv.org/abs/2603.11161
摘要：我们正式定义了网络捕获（即，“grokking”（一种算法）是神经网络以可控误差和最小样本自适应推广到任意问题大小（$T$）的能力，将真正的算法学习与统计插值区分开来。通过分析无限宽的Transformers在懒惰和丰富的制度，我们推导出这些网络可以学习的函数的推理时间计算复杂度的上限。我们发现，尽管他们的普遍表现力，Transformers具有感应偏置低复杂度的算法内的高效多项式时间启发式方案（EPTHS）类。这种偏见有效地阻止了他们捕获更高复杂度的算法，同时允许在搜索，复制和排序等更简单的任务上取得成功。
摘要：We formally define Algorithmic Capture (i.e., ``grokking'' an algorithm) as the ability of a neural network to generalize to arbitrary problem sizes ($T$) with controllable error and minimal sample adaptation, distinguishing true algorithmic learning from statistical interpolation. By analyzing infinite-width transformers in both the lazy and rich regimes, we derive upper bounds on the inference-time computational complexity of the functions these networks can learn. We show that despite their universal expressivity, transformers possess an inductive bias towards low-complexity algorithms within the Efficient Polynomial Time Heuristic Scheme (EPTHS) class. This bias effectively prevents them from capturing higher-complexity algorithms, while allowing success on simpler tasks like search, copy, and sort.
【7】Task-Conditioned Routing Signatures in Sparse Mixture-of-Experts Transformers
标题：稀疏专家混合转换器中的任务条件路由签名
链接：https://arxiv.org/abs/2603.11114
备注：11 pages, 5 figures. Empirical analysis of routing behavior in sparse Mixture-of-Experts transformers using OLMoE
摘要：稀疏混合专家（MoE）架构通过条件计算实现了大型语言模型的有效扩展，但负责专家选择的路由机制仍然知之甚少。在这项工作中，我们引入路由签名，一个矢量表示总结专家激活模式跨层给定的提示，并使用它们来研究是否MoE路由呈现任务条件结构。使用OLMoE-1B-7 B-0125-指令作为一个实证测试床，我们表明，提示从同一任务类别诱导高度相似的路由签名，而提示从不同的类别表现出显着较低的相似性。类别内路由相似性（0.8435 +/- 0.0879）显著超过跨类别相似性（0.6225 +/- 0.1687），对应于Cohen的d = 1.44。仅在路由签名上训练的逻辑回归分类器在四向任务分类上实现了92.5% +/- 6.1%的交叉验证准确率。为了确保统计的有效性，我们引入置换和负载平衡基线，并表明所观察到的分离是不能解释的稀疏性或平衡约束。我们进一步分析了逐层信号强度和路由签名的低维投影，发现任务结构在更深层变得越来越明显。这些结果表明，稀疏Transformers中的路由不仅是一种平衡机制，而且是条件计算的可测量任务敏感组件。我们发布了MOE-XRAY，一个用于路由遥测和分析的轻量级工具包。
摘要：Sparse Mixture-of-Experts (MoE) architectures enable efficient scaling of large language models through conditional computation, yet the routing mechanisms responsible for expert selection remain poorly understood. In this work, we introduce routing signatures, a vector representation summarizing expert activation patterns across layers for a given prompt, and use them to study whether MoE routing exhibits task-conditioned structure. Using OLMoE-1B-7B-0125-Instruct as an empirical testbed, we show that prompts from the same task category induce highly similar routing signatures, while prompts from different categories exhibit substantially lower similarity. Within-category routing similarity (0.8435 +/- 0.0879) significantly exceeds across-category similarity (0.6225 +/- 0.1687), corresponding to Cohen's d = 1.44. A logistic regression classifier trained solely on routing signatures achieves 92.5% +/- 6.1% cross-validated accuracy on four-way task classification. To ensure statistical validity, we introduce permutation and load-balancing baselines and show that the observed separation is not explained by sparsity or balancing constraints alone. We further analyze layer-wise signal strength and low-dimensional projections of routing signatures, finding that task structure becomes increasingly apparent in deeper layers. These results suggest that routing in sparse transformers is not merely a balancing mechanism, but a measurable task-sensitive component of conditional computation. We release MOE-XRAY, a lightweight toolkit for routing telemetry and analysis.
【8】Implicit Statistical Inference in Transformers: Approximating Likelihood-Ratio Tests In-Context
标题：Transformer中的隐式统计推断：上下文中的逼近似然比检验
链接：https://arxiv.org/abs/2603.10573
备注：Accepted at the Latent and Implicit Thinking Workshop (ICLR 2026)
摘要：情境学习（ICL）允许Transformers在不更新权重的情况下适应新的任务，但底层算法仍然知之甚少。我们采用统计决策理论的角度，通过调查简单的二元假设检验，其中最优的政策是由似然比测试。值得注意的是，这种设置为机械可解释性提供了数学上严格的设置，其中目标算法基础事实是已知的。通过训练Transformers任务需要不同的几何形状（线性移位均值与非线性方差估计），我们证明了模型近似贝叶斯最优充分统计从上下文到一些单调变换，匹配的性能理想的预言估计器在非线性制度。利用这种分析的基础事实，通过logit透镜和电路对准的机制分析表明，该模型不依赖于固定的内核平滑启发式。相反，它似乎适应了决策变得线性可解码的点：表现出与线性任务的投票式集合一致的模式，同时利用更深层次的顺序计算非线性任务。这些研究结果表明，ICL出现的任务自适应统计估计，而不是简单的相似性匹配的建设。
摘要：In-context learning (ICL) allows Transformers to adapt to novel tasks without weight updates, yet the underlying algorithms remain poorly understood. We adopt a statistical decision-theoretic perspective by investigating simple binary hypothesis testing, where the optimal policy is determined by the likelihood-ratio test. Notably, this setup provides a mathematically rigorous setting for mechanistic interpretability where the target algorithmic ground truth is known. By training Transformers on tasks requiring distinct geometries (linear shifted means vs. nonlinear variance estimation), we demonstrate that the models approximate the Bayes-optimal sufficient statistics from context up to some monotonic transformation, matching the performance of an ideal oracle estimator in nonlinear regimes. Leveraging this analytical ground truth, mechanistic analysis via logit lens and circuit alignment suggests that the model does not rely on a fixed kernel smoothing heuristic. Instead, it appears to adapt the point at which decisions become linearly decodable: exhibiting patterns consistent with a voting-style ensemble for linear tasks while utilizing a deeper sequential computation for nonlinear tasks. These findings suggest that ICL emerges from the construction of task-adaptive statistical estimators rather than simple similarity matching.
GAN|对抗|攻击|生成相关(5篇)
【1】BiGain: Unified Token Compression for Joint Generation and Classification
标题：BiGain：用于联合生成和分类的统一令牌压缩
链接：https://arxiv.org/abs/2603.12240
备注：CVPR 2026. Code: https://github.com/Greenoso/BiGain
摘要：扩散模型的加速方法（例如，令牌合并或下采样）通常在减少的计算下优化合成质量，但常常忽略辨别能力。我们重新审视令牌压缩与联合目标，并提出BiGain，一个免训练，即插即用的框架，保留生成质量，同时提高加速扩散模型中的分类。我们的关键见解是频率分离：将特征空间信号映射到频率感知的表示中，从全局语义中分离出精细的细节，使压缩既尊重生成保真度又尊重区分效用。BiGain通过两个频率感知算子反映了这一原理：（1）拉普拉斯门控标记合并，其鼓励频谱平滑标记之间的合并，同时阻止高对比度标记的合并，从而保留边缘和纹理;以及（2）内插-外插KV下采样，其经由最近池化和平均池化之间的可控内插来下采样键/值，同时保持查询完整，从而保存注意力精度。在基于DiT和U-Net的主干以及ImageNet-1 K、ImageNet-100、Oxford-IIIT Pets和COCO-2017中，我们的运营商始终提高了基于扩散的分类的速度-准确性权衡，同时在相当的加速度下保持或提高生成质量。例如，在ImageNet-1 K上，在Stable Diffusion 2.0上有70%的token合并，BiGain将分类准确率提高了7.15%，同时将FID提高了0.34（1.85%）。我们的分析表明，平衡的频谱保留，保留高频细节和低/中频语义，是一个可靠的设计规则令牌压缩扩散模型。据我们所知，BiGain是第一个在加速扩散下联合研究和推进生成和分类的框架，支持低成本部署。
摘要：Acceleration methods for diffusion models (e.g., token merging or downsampling) typically optimize synthesis quality under reduced compute, yet often ignore discriminative capacity. We revisit token compression with a joint objective and present BiGain, a training-free, plug-and-play framework that preserves generation quality while improving classification in accelerated diffusion models. Our key insight is frequency separation: mapping feature-space signals into a frequency-aware representation disentangles fine detail from global semantics, enabling compression that respects both generative fidelity and discriminative utility. BiGain reflects this principle with two frequency-aware operators: (1) Laplacian-gated token merging, which encourages merges among spectrally smooth tokens while discouraging merges of high-contrast tokens, thereby retaining edges and textures; and (2) Interpolate-Extrapolate KV Downsampling, which downsamples keys/values via a controllable interextrapolation between nearest and average pooling while keeping queries intact, thereby conserving attention precision. Across DiT- and U-Net-based backbones and ImageNet-1K, ImageNet-100, Oxford-IIIT Pets, and COCO-2017, our operators consistently improve the speed-accuracy trade-off for diffusion-based classification, while maintaining or enhancing generation quality under comparable acceleration. For instance, on ImageNet-1K, with 70% token merging on Stable Diffusion 2.0, BiGain increases classification accuracy by 7.15% while improving FID by 0.34 (1.85%). Our analyses indicate that balanced spectral retention, preserving high-frequency detail and low/mid-frequency semantics, is a reliable design rule for token compression in diffusion models. To our knowledge, BiGain is the first framework to jointly study and advance both generation and classification under accelerated diffusion, supporting lower-cost deployment.
【2】FlashMotion: Few-Step Controllable Video Generation with Trajectory Guidance
标题：Flash Motion：具有轨迹引导的少步骤可控视频生成
链接：https://arxiv.org/abs/2603.12146
备注：Accepted by CVPR2026
摘要：最近的进展，在可控视频生成已经取得了显着的进步。先前的方法主要使用基于适配器的架构来沿着预定义的轨迹进行精确的运动控制。然而，所有这些方法都依赖于多步去噪过程，导致大量的时间冗余和计算开销。虽然现有的视频蒸馏方法成功地将多步生成器蒸馏成几步，但直接将这些方法应用于可控制的视频生成会导致视频质量和轨迹准确性的显著下降。为了弥合这一差距，我们介绍了FlashMotion，一种新颖的训练框架，设计用于几步可控制的视频生成。我们首先在多步视频发生器上训练轨迹适配器，以实现精确的轨迹控制。然后，我们将生成器提取为几步版本以加速视频生成。最后，我们使用一种混合策略来微调适配器，该策略结合了扩散和对抗目标，使其与几步生成器对齐，以生成高质量、精确度高的视频。为了进行评估，我们引入了FlashBench，这是一个用于长序列可控视频生成的基准，它可以测量不同数量的前景对象的视频质量和轨迹准确性。两个适配器架构上的实验表明，FlashMotion优于现有的视频提取方法和以前的多步模型在视觉质量和轨迹一致性。
摘要：Recent advances in trajectory-controllable video generation have achieved remarkable progress. Previous methods mainly use adapter-based architectures for precise motion control along predefined trajectories. However, all these methods rely on a multi-step denoising process, leading to substantial time redundancy and computational overhead. While existing video distillation methods successfully distill multi-step generators into few-step, directly applying these approaches to trajectory-controllable video generation results in noticeable degradation in both video quality and trajectory accuracy. To bridge this gap, we introduce FlashMotion, a novel training framework designed for few-step trajectory-controllable video generation. We first train a trajectory adapter on a multi-step video generator for precise trajectory control. Then, we distill the generator into a few-step version to accelerate video generation. Finally, we finetune the adapter using a hybrid strategy that combines diffusion and adversarial objectives, aligning it with the few-step generator to produce high-quality, trajectory-accurate videos. For evaluation, we introduce FlashBench, a benchmark for long-sequence trajectory-controllable video generation that measures both video quality and trajectory accuracy across varying numbers of foreground objects. Experiments on two adapter architectures show that FlashMotion surpasses existing video distillation methods and previous multi-step models in both visual quality and trajectory consistency.
【3】Automatic Generation of High-Performance RL Environments
标题：高性能RL环境的自动生成
链接：https://arxiv.org/abs/2603.12145
备注：26 pages, 9 figures, 8 tables
摘要：将复杂的强化学习（RL）环境转换为高性能的实现传统上需要数月的专业工程。我们提出了一个可重用的配方-一个通用的提示模板，分层验证，迭代代理辅助修复-产生语义等效的高性能环境<10美元的计算成本。我们在五个环境中展示了三种不同的工作流程。直接翻译（没有先前的性能实现）：EmuRust（通过Rust并行性为Game Boy模拟器提供1.5倍PPO加速）和PokeJAX，第一个GPU并行口袋妖怪战斗模拟器（500 M SPS随机动作，15.2 M SPS PPO; 22，320倍于TypeScript参考）。针对现有的性能实现进行了翻译验证：在匹配的GPU批量大小下，MJX的吞吐量奇偶校验（1.04倍）和Brax的5倍（HalfCheetah JAX）; 42倍PPO（Puffer Pong）。创建新环境：TCGJax，第一个可部署的JAX Pokemon TCG引擎（717 K SPS随机动作，153 K SPS PPO; Python参考的6.6倍），从Web提取的规范合成。在200 M参数下，环境开销下降到训练时间的4%以下。分层验证（属性，交互和推出测试）确认所有五个环境的语义等效;跨后端策略传输确认所有五个环境的SIM卡到SIM卡的差距为零。TCGJax，从公共存储库中没有的私人参考合成，作为代理预训练数据关注的污染控制。该论文包含了足够的细节-包括代表性的提示，验证方法和完整的结果-编码代理可以直接从手稿复制翻译。
摘要：Translating complex reinforcement learning (RL) environments into high-performance implementations has traditionally required months of specialized engineering. We present a reusable recipe - a generic prompt template, hierarchical verification, and iterative agent-assisted repair - that produces semantically equivalent high-performance environments for
【4】Hoi3DGen: Generating High-Quality Human-Object-Interactions in 3D
标题：Hoi 3DGen：在3D中生成高质量的人-物交互
链接：https://arxiv.org/abs/2603.12126
摘要：从文本建模和生成3D人机交互对于AR、XR和游戏中的应用至关重要。现有的方法通常依赖于从文本到图像模型的分数蒸馏，但由于缺乏高质量的交互数据，它们的结果受到Janus问题的影响，并且不忠实地遵循文本提示。我们介绍Hoi 3DGen，一个框架，生成高质量的纹理网格的人与物体的交互，按照输入的交互描述精确。我们首先利用多模态大型语言模型来策划真实和高质量的交互数据，然后创建一个完整的文本到3D管道，从而实现交互保真度的数量级改进。我们的方法在文本一致性方面超过基线4- 15倍，在3D模型质量方面超过基线3- 7倍，对不同类别和交互类型表现出很强的泛化能力，同时保持高质量的3D生成。
摘要：Modeling and generating 3D human-object interactions from text is crucial for applications in AR, XR, and gaming. Existing approaches often rely on score distillation from text-to-image models, but their results suffer from the Janus problem and do not follow text prompts faithfully due to the scarcity of high-quality interaction data. We introduce Hoi3DGen, a framework that generates high-quality textured meshes of human-object interaction that follow the input interaction descriptions precisely. We first curate realistic and high-quality interaction data leveraging multimodal large language models, and then create a full text-to-3D pipeline, which achieves orders-of-magnitude improvements in interaction fidelity. Our method surpasses baselines by 4-15x in text consistency and 3-7x in 3D model quality, exhibiting strong generalization to diverse categories and interaction types, while maintaining high-quality 3D generation.
【5】Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios
标题：衡量人工智能代理在多步网络攻击场景中的进展
链接：https://arxiv.org/abs/2603.11214
摘要：我们评估了前沿人工智能模型在两个专门构建的网络范围上的自主网络攻击能力-一个32步的企业网络攻击和一个7步的工业控制系统攻击-这需要在扩展的动作序列中链接异构能力。通过比较在18个月期间（2024年8月至2026年2月）发布的七个模型，在不同的推理时间计算预算下，我们观察到两个能力趋势。首先，模型性能与推理时间计算呈对数线性关系，没有观察到平台-从1000万增加到1亿令牌产生高达59%的收益，不需要运营商的特定技术复杂性。其次，在固定的代币预算下，每一代模型的表现都优于其前任：在企业网络范围内，1000万代币完成的平均步骤从1.7（GPT-4 o，2024年8月）上升到9.8（Opus 4.6，2026年2月）。最好的单次运行完成了32个步骤中的22个，相当于人类专家估计需要的14个小时中的大约6个。在工业控制系统范围内，性能仍然有限，尽管最新的型号是第一个可靠完成步骤的型号，平均为7个步骤中的1.2-1.4个（最大3个）。
摘要：We evaluate the autonomous cyber-attack capabilities of frontier AI models on two purpose-built cyber ranges-a 32-step corporate network attack and a 7-step industrial control system attack-that require chaining heterogeneous capabilities across extended action sequences. By comparing seven models released over an eighteen-month period (August 2024 to February 2026) at varying inference-time compute budgets, we observe two capability trends. First, model performance scales log-linearly with inference-time compute, with no observed plateau-increasing from 10M to 100M tokens yields gains of up to 59%, requiring no specific technical sophistication from the operator. Second, each successive model generation outperforms its predecessor at fixed token budgets: on the corporate network range, average steps completed at 10M tokens rose from 1.7 (GPT-4o, August 2024) to 9.8 (Opus 4.6, February 2026). The best single run completed 22 of 32 steps, corresponding to roughly 6 of the estimated 14 hours a human expert would need. On the industrial control system range, performance remains limited, though the most recent models are the first to reliably complete steps, averaging 1.2-1.4 of 7 (max 3).
半/弱/无/有监督|不确定性|主动学习(9篇)
【1】Deep Learning-Based Metamodeling of Nonlinear Stochastic Dynamic Systems under Parametric and Predictive Uncertainty
标题：参数和预测不确定性下基于深度学习的非线性随机动态系统元建模
链接：https://arxiv.org/abs/2603.12012
摘要：自然灾害下的高维非线性动态结构系统建模提出了巨大的计算挑战，特别是当同时考虑外部载荷和结构参数的不确定性时。研究已经成功地将不确定性相关的外部载荷从自然灾害，但很少有同时解决结构系统内的载荷和参数的不确定性，同时占神经网络的预测不确定性。为了解决这些差距，制定了三个元建模框架，每个框架将通过多层感知器（MLP），消息传递神经网络（MPNN）或自动编码器（AE）实现的特征提取模块与使用蒙特卡洛丢弃和负对数似然损失的长短期记忆（LSTM）网络相耦合。所得到的架构（MLP-LSTM，MPNN-LSTM和AE-LSTM）在两个案例研究中进行了验证：多自由度Bouc-Wen系统和37层纤维离散非线性钢抗弯框架，两者都受到随机地震激励和结构参数不确定性的影响。这三种方法都实现了较低的预测误差：MLP-LSTM对低维Bouc-Wen系统产生了最准确的结果，而MPNN-LSTM和AE-LSTM在更复杂的钢框架模型上提供了更好的性能。此外，预测方差和实际误差之间的一致相关性证实了这些框架对主动学习策略和评估结构响应预测模型置信度的适用性。
摘要：Modeling high-dimensional, nonlinear dynamic structural systems under natural hazards presents formidable computational challenges, especially when simultaneously accounting for uncertainties in external loads and structural parameters. Studies have successfully incorporated uncertainties related to external loads from natural hazards, but few have simultaneously addressed loading and parameter uncertainties within structural systems while accounting for prediction uncertainty of neural networks. To address these gaps, three metamodeling frameworks were formulated, each coupling a feature-extraction module implemented through a multi-layer perceptron (MLP), a message-passing neural network (MPNN), or an autoencoder (AE) with a long short-term memory (LSTM) network using Monte Carlo dropout and a negative log-likelihood loss. The resulting architectures (MLP-LSTM, MPNN-LSTM, and AE-LSTM) were validated on two case studies: a multi-degree-of-freedom Bouc-Wen system and a 37-story fiber-discretized nonlinear steel moment-resisting frame, both subjected to stochastic seismic excitation and structural parameter uncertainty. All three approaches achieved low prediction errors: the MLP-LSTM yielded the most accurate results for the lower-dimensional Bouc-Wen system, whereas the MPNN-LSTM and AE-LSTM provided superior performance on the more complex steel-frame model. Moreover, a consistent correlation between predictive variance and actual error confirms the suitability of these frameworks for active-learning strategies and for assessing model confidence in structural response predictions.
【2】Disentangled Representation Learning through Unsupervised Symmetry Group Discovery
标题：通过无监督对称群发现解开表示学习
链接：https://arxiv.org/abs/2603.11790
摘要：基于对称性的解纠缠表征学习利用环境变换的组结构来揭示变化的潜在因素。先前的方法，以基于量子力学的解纠缠需要强大的先验知识的对称群的结构，或限制性假设的子群属性。在这项工作中，我们删除这些限制，提出了一种方法，使一个具体的代理自主发现其行动空间的组结构，通过与环境的无监督交互。我们证明了可识别的真正的对称群分解在最小的假设下，并得出两个算法：一个用于发现组分解的相互作用的数据，另一个用于学习线性对称为基础的解纠缠（LSBD）表示，而无需假设特定的子群属性。我们的方法进行了验证，在三个环境中表现出不同的组分解，它优于现有的LSBD方法。
摘要：Symmetry-based disentangled representation learning leverages the group structure of environment transformations to uncover the latent factors of variation. Prior approaches to symmetry-based disentanglement have required strong prior knowledge of the symmetry group's structure, or restrictive assumptions about the subgroup properties. In this work, we remove these constraints by proposing a method whereby an embodied agent autonomously discovers the group structure of its action space through unsupervised interaction with the environment. We prove the identifiability of the true symmetry group decomposition under minimal assumptions, and derive two algorithms: one for discovering the group decomposition from interaction data, and another for learning Linear Symmetry-Based Disentangled (LSBD) representations without assuming specific subgroup properties. Our method is validated on three environments exhibiting different group decompositions, where it outperforms existing LSBD approaches.
【3】UniHetCO: A Unified Heterogeneous Representation for Multi-Problem Learning in Unsupervised Neural Combinatorial Optimization
标题：UniHetCO：无监督神经组合优化中多问题学习的统一异类表示
链接：https://arxiv.org/abs/2603.11456
摘要：无监督神经组合优化（NCO）通过训练基于学习的求解器而无需地面实况解决方案，直接最小化实例目标和约束违反，为监督方法提供了一种有吸引力的替代方案。然而，对于图节点子集选择问题（例如，最大团和最大独立集），现有的无监督方法通常专用于单个问题类，并依赖于特定于问题的代理损失，这阻碍了在统一框架内跨类学习。在这项工作中，我们提出了UniHetCO，一个统一的异构图形表示的约束二次规划为基础的组合优化，编码的问题结构，目标条款和线性约束在一个单一的输入。这种公式化能够训练具有统一无标签目标的多个问题类的单个模型。为了提高多问题学习下的稳定性，我们采用了一种基于梯度范数的动态加权方案，该方案消除了类之间的梯度不平衡。在多个数据集和四个受约束的问题类上的实验证明了具有竞争力的性能，具有最先进的无监督NCO基线，强大的跨问题适应潜力，以及在严格的时间限制下有效的商业经典求解器的热启动。
摘要：Unsupervised neural combinatorial optimization (NCO) offers an appealing alternative to supervised approaches by training learning-based solvers without ground-truth solutions, directly minimizing instance objectives and constraint violations. Yet for graph node subset-selection problems (e.g., Maximum Clique and Maximum Independent Set), existing unsupervised methods are typically specialized to a single problem class and rely on problem-specific surrogate losses, which hinders learning across classes within a unified framework. In this work, we propose UniHetCO, a unified heterogeneous graph representation for constrained quadratic programming-based combinatorial optimization that encodes problem structure, objective terms, and linear constraints in a single input. This formulation enables training a single model across multiple problem classes with a unified label-free objective. To improve stability under multi-problem learning, we employ a gradient-norm-based dynamic weighting scheme that alleviates gradient imbalance among classes. Experiments on multiple datasets and four constrained problem classes demonstrate competitive performance with state-of-the-art unsupervised NCO baselines, strong cross-problem adaptation potential, and effective warm starts for a commercial classical solver under tight time limits.
【4】Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representations
标题：端到端自动驾驶中的Zero-Shot跨城市概括：自我监督与监督代表
链接：https://arxiv.org/abs/2603.11417
摘要 ：端到端自动驾驶模型通常使用有监督的ImageNet预训练骨干在多城市数据集上进行训练，但它们推广到看不见的城市的能力在很大程度上仍未得到检验。当训练和评估数据在地理上混合时，模型可能会隐含地依赖于城市特定的线索，从而掩盖了在推广到新位置时在实际域转移下可能发生的故障模式。在这项工作中，我们研究了端到端轨迹规划中的zero-shot跨城市泛化，并询问自我监督的视觉表示是否改善了跨城市的转移。我们通过将自我监督的骨干（I-JEPA，DINOv 2和MAE）整合到规划框架中来进行全面的研究。我们在开环设置中在nuScenes上以及在闭环评估协议中在NAVSIM上评估了严格地理分割下的性能。我们的实验揭示了在具有不同道路拓扑结构和驾驶惯例的城市之间传输依赖于传统监督骨干的模型时，特别是从右侧向左侧驾驶环境传输时，存在很大的泛化差距。自我监督表示学习缩小了这一差距。在开环评估中，当从波士顿转移到新加坡时，监督骨干表现出严重的膨胀（L2位移比9.77倍，碰撞比19.43倍），而特定领域的自我监督预训练将其分别降低到1.20倍和0.75倍。在闭环评估中，自我监督的预训练使所有单一城市培训城市的PDMS提高了4%。这些结果表明，表示学习强烈影响跨城市规划的鲁棒性，并建立zero-shot地理转移作为评估端到端自动驾驶系统的必要测试。
摘要：End-to-end autonomous driving models are typically trained on multi-city datasets using supervised ImageNet-pretrained backbones, yet their ability to generalize to unseen cities remains largely unexamined. When training and evaluation data are geographically mixed, models may implicitly rely on city-specific cues, masking failure modes that would occur under real domain shifts when generalizing to new locations. In this work we investigate zero-shot cross-city generalization in end-to-end trajectory planning and ask whether self-supervised visual representations improve transfer across cities. We conduct a comprehensive study by integrating self-supervised backbones (I-JEPA, DINOv2, and MAE) into planning frameworks. We evaluate performance under strict geographic splits on nuScenes in the open-loop setting and on NAVSIM in the closed-loop evaluation protocol. Our experiments reveal a substantial generalization gap when transferring models relying on traditional supervised backbones across cities with different road topologies and driving conventions, particularly when transferring from right-side to left-side driving environments. Self-supervised representation learning reduces this gap. In open-loop evaluation, a supervised backbone exhibits severe inflation when transferring from Boston to Singapore (L2 displacement ratio 9.77x, collision ratio 19.43x), whereas domain-specific self-supervised pretraining reduces this to 1.20x and 0.75x respectively. In closed-loop evaluation, self-supervised pretraining improves PDMS by up to 4 percent for all single-city training cities. These results show that representation learning strongly influences the robustness of cross-city planning and establish zero-shot geographic transfer as a necessary test for evaluating end-to-end autonomous driving systems.
【5】Meta-Reinforcement Learning with Self-Reflection for Agentic Search
标题：具有自我反思的元强化学习用于统计搜索
链接：https://arxiv.org/abs/2603.11327
备注：23 pages, Preprint
摘要：本文介绍了MR-Search，一种用于自反射代理搜索的上下文Meta强化学习（RL）公式。MR搜索不是在单个独立事件中优化策略，而是训练一个以过去事件为条件的策略，并在事件之间调整其搜索策略。MR-Search通过自我反思学习搜索策略，允许搜索代理在测试时改进上下文探索。具体来说，MR-Search通过在每一集之后生成明确的自我反思并利用它们作为额外的上下文来指导后续尝试来执行跨集探索，从而在测试期间促进更有效的探索。我们进一步介绍了一个多回合RL算法，估计在回合水平上的密集相对优势，使细粒度的信用分配每集。各种基准测试的实证结果证明了MR搜索相对于基于基线的RL的优势，在八个基准测试中显示出强大的泛化能力和9.2%至19.3%的相对改进。我们的代码和数据可在https://github.com/tengxiao1/MR-Search上获得。
摘要：This paper introduces MR-Search, an in-context meta reinforcement learning (RL) formulation for agentic search with self-reflection. Instead of optimizing a policy within a single independent episode with sparse rewards, MR-Search trains a policy that conditions on past episodes and adapts its search strategy across episodes. MR-Search learns to learn a search strategy with self-reflection, allowing search agents to improve in-context exploration at test-time. Specifically, MR-Search performs cross-episode exploration by generating explicit self-reflections after each episode and leveraging them as additional context to guide subsequent attempts, thereby promoting more effective exploration during test-time. We further introduce a multi-turn RL algorithm that estimates a dense relative advantage at the turn level, enabling fine-grained credit assignment on each episode. Empirical results across various benchmarks demonstrate the advantages of MR-Search over baselines based RL, showing strong generalization and relative improvements of 9.2% to 19.3% across eight benchmarks. Our code and data are available at https://github.com/tengxiao1/MR-Search.
【6】Huntington Disease Automatic Speech Recognition with Biomarker Supervision
标题：具有生物标志物监督的亨廷顿病自动语音识别
链接：https://arxiv.org/abs/2603.11168
摘要：病理性语音的自动语音识别（ASR）仍然未被探索，特别是对于亨廷顿氏病（HD），其中不规则的定时、不稳定的发声和发音失真挑战当前模型。我们提出了一个系统的HD-ASR研究，使用高保真临床语音语料库，以前没有用于端到端的ASR训练。我们比较多个ASR家庭下一个统一的评价，分析WER以及取代，缺失和插入模式。HD语音会导致特定于架构的错误机制，Parakeet-TDT优于编码器-解码器和CTC基线。HD特定的适应减少WER从6.99%到4.95%，我们还提出了一种方法，使用基于生物标记的辅助监督和分析错误行为是如何重塑严重程度相关的方式，而不是统一提高WER。我们开放所有代码和模型。
摘要：Automatic speech recognition (ASR) for pathological speech remains underexplored, especially for Huntington's disease (HD), where irregular timing, unstable phonation, and articulatory distortion challenge current models. We present a systematic HD-ASR study using a high-fidelity clinical speech corpus not previously used for end-to-end ASR training. We compare multiple ASR families under a unified evaluation, analyzing WER as well as substitution, deletion, and insertion patterns. HD speech induces architecture-specific error regimes, with Parakeet-TDT outperforming encoder-decoder and CTC baselines. HD-specific adaptation reduces WER from 6.99% to 4.95% and we also propose a method for using biomarker-based auxiliary supervision and analyze how error behavior is reshaped in severity-dependent ways rather than uniformly improving WER. We open-source all code and models.
【7】Fingerprinting Concepts in Data Streams with Supervised and Unsupervised Meta-Information
标题：具有监督和无监督元信息的数据流中的指纹识别概念
链接：https://arxiv.org/abs/2603.11094
摘要：随着实时收集数据的能力的增长，数据流源变得越来越普遍。处理数据流的一个主要问题是概念漂移，即数据分布随时间的变化，例如，由于环境条件的变化。表示概念（具有类似行为的静止期）是适应概念漂移的关键思想。通过测试概念表征与观察窗口的相似性，我们可以检测到概念漂移到新的或以前看到的重复概念。概念表征是使用元信息特征构建的，元信息特征是描述概念行为方面的值。我们发现，以前提出的概念表示依赖于少量的元信息功能。这些表示通常不能区分概念，使系统容易受到概念漂移的影响。我们提出了FiCSUM，一个通用的框架来表示指纹中概念的监督和无监督行为，这是一个由许多不同的元信息特征组成的向量，能够唯一地识别更多的概念。我们的动态加权策略学习哪些元信息特征描述了给定数据集中的概念漂移，允许一次使用不同的元信息特征集。FiCSUM在11个真实世界和合成数据集的准确性和建模基础概念漂移方面优于最先进的方法。
摘要 ：Streaming sources of data are becoming more common as the ability to collect data in real-time grows. A major concern in dealing with data streams is concept drift, a change in the distribution of data over time, for example, due to changes in environmental conditions. Representing concepts (stationary periods featuring similar behaviour) is a key idea in adapting to concept drift. By testing the similarity of a concept representation to a window of observations, we can detect concept drift to a new or previously seen recurring concept. Concept representations are constructed using meta-information features, values describing aspects of concept behaviour. We find that previously proposed concept representations rely on small numbers of meta-information features. These representations often cannot distinguish concepts, leaving systems vulnerable to concept drift. We propose FiCSUM, a general framework to represent both supervised and unsupervised behaviours of a concept in a fingerprint, a vector of many distinct meta-information features able to uniquely identify more concepts. Our dynamic weighting strategy learns which meta-information features describe concept drift in a given dataset, allowing a diverse set of meta-information features to be used at once. FiCSUM outperforms state-of-the-art methods over a range of 11 real world and synthetic datasets in both accuracy and modeling underlying concept drift.
【8】Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates
标题：神经运算符PED代理的结构感知认识不确定性量化
链接：https://arxiv.org/abs/2603.11052
摘要：神经操作符（NO）提供了快速的，分辨率不变的代理映射输入字段到PDE解字段，但他们的预测可能会表现出显着的认知不确定性，由于有限的数据，不完美的优化，和分布偏移。为了在科学计算中的实际部署，不确定性量化（UQ）必须在计算上高效并且在空间上忠实，即，不确定性带应与对下游风险管理至关重要的局部剩余结构相一致。我们提出了一个结构感知的认知UQ计划，利用现代NO（提升-传播-恢复）常见的模块化解剖。而不是应用非结构化的重量扰动（例如，通过在整个网络中使用随机丢弃（naive dropout），我们通过仅将随机性注入提升模块来将Monte Carlo采样限制在模块对齐的子空间中，并将学习的求解器动态（传播和恢复）视为确定性。我们实例化这一原则与两个轻量级的提升水平扰动，包括通道乘性特征辍学和匹配方差的高斯特征扰动，然后通过标准校准来构建不确定性带。具有挑战性的PDE基准（包括不连续系数达西流和几何位移的3D汽车CFD代理）的实验表明，所提出的结构感知设计产生更可靠的覆盖范围，更紧密的频带，并改善剩余不确定性对齐相比，共同的基线，同时保持实用的运行时。
摘要：Neural operators (NOs) provide fast, resolution-invariant surrogates for mapping input fields to PDE solution fields, but their predictions can exhibit significant epistemic uncertainty due to finite data, imperfect optimization, and distribution shift. For practical deployment in scientific computing, uncertainty quantification (UQ) must be both computationally efficient and spatially faithful, i.e., uncertainty bands should align with the localized residual structures that matter for downstream risk management. We propose a structure-aware epistemic UQ scheme that exploits the modular anatomy common to modern NOs (lifting-propagation-recovering). Instead of applying unstructured weight perturbations (e.g., naive dropout) across the entire network, we restrict Monte Carlo sampling to a module-aligned subspace by injecting stochasticity only into the lifting module, and treat the learned solver dynamics (propagation and recovery) as deterministic. We instantiate this principle with two lightweight lifting-level perturbations, including channel-wise multiplicative feature dropout and a Gaussian feature perturbation with matched variance, followed by standard calibration to construct uncertainty bands. Experiments on challenging PDE benchmarks (including discontinuous-coefficient Darcy flow and geometry-shifted 3D car CFD surrogates) demonstrate that the proposed structure-aware design yields more reliable coverage, tighter bands, and improved residual-uncertainty alignment compared with common baselines, while remaining practical in runtime.
【9】Learning to Unscramble: Simplifying Symbolic Expressions via Self-Supervised Oracle Trajectories
标题：学习解读：通过自我监督的Oracle轨迹简化符号表达
链接：https://arxiv.org/abs/2603.11164
备注：14 pages, 6 figures, 2 tables; work done in collaboration with Claude Code
摘要：我们提出了一种新的自监督机器学习方法，用于复杂数学表达式的符号简化。训练数据是通过扰乱简单表达式和记录逆运算来生成的，创建了提供目标状态和到达目标状态的显式路径的oracle轨迹。然后，在此数据上逐步训练一个置换等变的基于transformer的策略网络，以预测给定输入表达式的oracle操作。我们证明了这种方法在高能物理中的两个问题：双对数减少和旋螺散射振幅简化。在这两种情况下，我们经过训练的策略网络在各种难度级别上都达到了近乎完美的解决率，大大优于基于强化学习和端到端回归的先前方法。当结合对比分组和束搜索时，我们的模型对Yang-Mills理论中具有代表性的5点胶子树级振幅的选择实现了100%的完全简化率，包括超过200个初始项的表达式.
摘要：We present a new self-supervised machine learning approach for symbolic simplification of complex mathematical expressions. Training data is generated by scrambling simple expressions and recording the inverse operations, creating oracle trajectories that provide both goal states and explicit paths to reach them. A permutation-equivariant, transformer-based policy network is then trained on this data step-wise to predict the oracle action given the input expression. We demonstrate this approach on two problems in high-energy physics: dilogarithm reduction and spinor-helicity scattering amplitude simplification. In both cases, our trained policy network achieves near perfect solve rates across a wide range of difficulty levels, substantially outperforming prior approaches based on reinforcement learning and end-to-end regression. When combined with contrastive grouping and beam search, our model achieves a 100\% full simplification rate on a representative selection of 5-point gluon tree-level amplitudes in Yang-Mills theory, including expressions with over 200 initial terms.
迁移|Zero/Few/One-Shot|自适应(3篇)
【1】OSM-based Domain Adaptation for Remote Sensing VLMs
标题：基于OSM的遥感VLM领域自适应
链接：https://arxiv.org/abs/2603.11804
摘要：适应遥感的视觉语言模型（VLM）在很大程度上依赖于特定领域的图像-文本监督，但卫星和航空图像的高质量注释仍然稀缺且生产成本高。流行的伪标签管道通过从大型前沿模型中提取知识来解决这一差距，但这种对大型教师的依赖成本高昂，限制了可扩展性，并将可实现的性能限制在教师的上限。我们提出OSMDA：一个独立的域适应框架，消除这种依赖。我们的关键见解是，一个有能力的基础VLM可以作为自己的注释引擎：通过将航空图像与渲染的OpenStreetMap（OSM）瓦片配对，我们利用模型的光学字符识别和图表理解功能来生成由OSM的大量辅助元数据丰富的标题。然后，该模型在仅使用卫星图像的结果语料库上进行微调，产生OSMDA-VLM，这是一种不需要手动标记和更强大的外部模型的域适应VLM。我们进行了详尽的评估，涵盖了图像-文本-文本任务的10个基准，并与9个竞争基线进行了比较。当同样与真实数据混合时，我们的方法实现了最先进的结果，同时比教师依赖的替代方案更便宜地进行训练。这些结果表明，如果有一个坚实的基础模型，与众包地理数据保持一致是实现遥感领域适应的一条切实可行和可扩展的途径。数据集和模型权重将公开提供。
摘要：Vision-Language Models (VLMs) adapted to remote sensing rely heavily on domain-specific image-text supervision, yet high-quality annotations for satellite and aerial imagery remain scarce and expensive to produce. Prevailing pseudo-labeling pipelines address this gap by distilling knowledge from large frontier models, but this dependence on large teachers is costly, limits scalability, and caps achievable performance at the ceiling of the teacher. We propose OSMDA: a self-contained domain adaptation framework that eliminates this dependency. Our key insight is that a capable base VLM can serve as its own annotation engine: by pairing aerial images with rendered OpenStreetMap (OSM) tiles, we leverage optical character recognition and chart comprehension capabilities of the model to generate captions enriched by OSM's vast auxiliary metadata. The model is then fine-tuned on the resulting corpus with satellite imagery alone, yielding OSMDA-VLM, a domain-adapted VLM that requires no manual labeling and no stronger external model. We conduct exhaustive evaluations spanning 10 benchmarks across image-text-to-text tasks and comparing against 9 competitive baselines. When equally mixed with real data, our method achieves state-of-the-art results, while being substantially cheaper to train than teacher-dependent alternatives. These results suggest that, given a strong foundation model, alignment with crowd-sourced geographic data is a practical and scalable path towards remote sensing domain adaptation. Dataset and model weights will be made publicly available.
【2】One Supervisor, Many Modalities: Adaptive Tool Orchestration for Autonomous Queries
标题：一个主管，多种模式：自治收件箱的自适应工具规划
链接：https://arxiv.org/abs/2603.11545
备注：19 pages, 3 figures
摘要：我们提出了一个用于自主多模态查询处理的代理AI框架，该框架协调跨文本，图像，音频，视频和文档模态的专用工具。中央管理器动态地分解用户查询，将子任务委托给模态适当的工具（例如，对象检测、OCR、语音转录），并通过自适应路由策略而不是预定的决策树来合成结果。对于纯文本查询，该框架使用通过RouteLLM学习的路由，而非文本路径使用SLM辅助的模态分解。在15个任务类别的2，847个查询上进行评估，与匹配的分层基线相比，我们的框架实现了72%的准确回答时间减少，85%的会话返工减少和67%的成本降低，同时保持准确性。这些结果表明，智能集中式编排从根本上提高了多模式AI部署的经济性。
摘要：We present an agentic AI framework for autonomous multimodal query processing that coordinates specialized tools across text, image, audio, video, and document modalities. A central Supervisor dynamically decomposes user queries, delegates subtasks to modality-appropriate tools (e.g., object detection, OCR, speech transcription), and synthesizes results through adaptive routing strategies rather than predetermined decision trees. For text-only queries, the framework uses learned routing via RouteLLM, while non-text paths use SLM-assisted modality decomposition. Evaluated on 2,847 queries across 15 task categories, our framework achieves 72% reduction in time-to-accurate-answer, 85% reduction in conversational rework, and 67% cost reduction compared to the matched hierarchical baseline while maintaining accuracy parity. These results demonstrate that intelligent centralized orchestration fundamentally improves multimodal AI deployment economics.
【3】ZTab: Domain-based Zero-shot Annotation for Table Columns
标题：ZTab：表列的基于域的Zero-Shot注释
链接：https://arxiv.org/abs/2603.11436
摘要：本研究解决了自动检测关系表中的语义列类型的挑战，这是许多现实应用程序中的一项关键任务。Zero-shot建模消除了对用户提供的标记训练数据的需求，使其成为数据收集成本高昂或由于隐私问题而受到限制的场景的理想选择。然而，现有的zero-shot模型在语义列类型数量较大时性能较差，对表格结构的理解有限，并且由于依赖于高性能闭源LLM而产生隐私风险。我们介绍ZTab，一个基于域的zero-shot框架，同时满足性能和zero-shot的要求。给定由一组预定义语义类型和示例表模式组成的域配置，ZTab为示例模式生成伪表，并在其上微调注释LLM。ZTab是基于域的zero-shot，因为它不依赖于用户特定的标记训练数据;因此，不需要对来自类似域的测试表进行再训练。我们描述了基于域的zero-shot的三种情况。ZTab的域配置提供了zero-shot的范围和注释性能之间的权衡：包含所有语义类型的“通用域”接近“纯”zero-shot，而包含特定应用程序的语义类型的“专用域”在该域内实现了更好的zero-shot性能。源代码和数据集可在https://github.com/hoseinzadeehsan/ZTab上获得
摘要：This study addresses the challenge of automatically detecting semantic column types in relational tables, a key task in many real-world applications. Zero-shot modeling eliminates the need for user-provided labeled training data, making it ideal for scenarios where data collection is costly or restricted due to privacy concerns. However, existing zero-shot models suffer from poor performance when the number of semantic column types is large, limited understanding of tabular structure, and privacy risks arising from dependence on high-performance closed-source LLMs. We introduce ZTab, a domain-based zero-shot framework that addresses both performance and zero-shot requirements. Given a domain configuration consisting of a set of predefined semantic types and sample table schemas, ZTab generates pseudo-tables for the sample schemas and fine-tunes an annotation LLM on them. ZTab is domain-based zero-shot in that it does not depend on user-specific labeled training data; therefore, no retraining is needed for a test table from a similar domain. We describe three cases of domain-based zero-shot. The domain configuration of ZTab provides a trade-off between the extent of zero-shot and annotation performance: a "universal domain" that contains all semantic types approaches "pure" zero-shot, while a "specialized domain" that contains semantic types for a specific application enables better zero-shot performance within that domain. Source code and datasets are available at https://github.com/hoseinzadeehsan/ZTab
强化学习(2篇)
【1】Entropy-Preserving Reinforcement Learning
标题：保守保保强化学习
链接：https://arxiv.org/abs/2603.11682
备注：Published at ICLR 2026
摘要：策略梯度算法推动了语言模型推理的许多最新进展。一个吸引人的特性是他们能够从自己的探索中学习，这是一个培养多样化和创造性解决方案的关键过程。正如我们在本文中所展示的，许多策略梯度算法自然会减少熵-从而减少探索轨迹的多样性-作为训练的一部分，从而使策略的探索能力越来越有限。在本文中，我们认为熵应该在整个训练过程中被积极地监测和控制。我们正式分析了领先政策梯度目标对熵动态的贡献，确定了显着影响熵行为的经验因素（例如数值精度），并提出了显式的熵控制机制。其中包括REPO，一系列修改优势函数以调节熵的算法，以及ADAPO，一种自适应非对称裁剪方法。使用我们的熵保持方法训练的模型在整个训练过程中保持多样性，从而产生更高性能的最终策略，并保留其在新环境中顺序学习的可训练性。
摘要：Policy gradient algorithms have driven many recent advancements in language model reasoning. An appealing property is their ability to learn from exploration on their own trajectories, a process crucial for fostering diverse and creative solutions. As we show in this paper, many policy gradient algorithms naturally reduce the entropy -- and thus the diversity of explored trajectories -- as part of training, yielding a policy increasingly limited in its ability to explore. In this paper, we argue that entropy should be actively monitored and controlled throughout training. We formally analyze the contributions of leading policy gradient objectives on entropy dynamics, identify empirical factors (such as numerical precision) that significantly impact entropy behavior, and propose explicit mechanisms for entropy control. These include REPO, a family of algorithms that modify the advantage function to regulate entropy, and ADAPO, an adaptive asymmetric clipping approach. Models trained with our entropy-preserving methods maintain diversity throughout training, yielding final policies that are more performant and retain their trainability for sequential learning in new environments.
【2】Ensuring Safety in Automated Mechanical Ventilation through Offline Reinforcement Learning and Digital Twin Verification
标题：通过离线强化学习和数字双胞胎验证确保自动机械通气的安全
链接：https://arxiv.org/abs/2603.11372
摘要：机械通气（MV）是ICU中急性呼吸衰竭（ARF）患者的救命干预措施。然而，不适当的呼吸机设置可能会导致呼吸机诱导的肺损伤（VILI）。此外，临床医生的工作量被证明是直接联系到病人的结果。因此，MV应该是个性化和自动化的，以改善患者的预后。以前在MV中引入个性化和自动化的尝试包括传统的监督学习和离线强化学习（RL）方法，这些方法通常忽略时间依赖性，并过度依赖基于死亡率的奖励。因此，早期生理恶化和VILI的风险没有充分捕获。为了解决这些限制，我们提出了基于transformer的保守Q学习（T-CQL），这是一种新型的离线RL框架，它集成了Transformer编码器，用于对患者动态进行有效的时间建模，基于不确定性量化的保守自适应正则化以确保安全性，以及用于鲁棒决策的一致性正则化。我们建立了一个临床知情的奖励函数，其中包括VILI指标和患者疾病严重程度的评分。此外，以前的工作主要使用拟合Q评估（Fitted Q-Evaluation，CQE）对静态离线数据进行RL策略评估，这对动态环境变化的响应较小，并且容易受到分布变化的影响。为了克服这些评估局限性，ARF患者的交互式数字双胞胎被用于在线“床边”评估。我们的研究结果表明，T-CQL始终优于现有的最先进的离线RL方法，提供更安全，更有效的解释性调整。我们的框架展示了基于transformer的模型结合保守的强化学习策略作为重症监护决策支持工具的潜力。
摘要 ：Mechanical ventilation (MV) is a life-saving intervention for patients with acute respiratory failure (ARF) in the ICU. However, inappropriate ventilator settings could cause ventilator-induced lung injury (VILI). Also, clinicians workload is shown to be directly linked to patient outcomes. Hence, MV should be personalized and automated to improve patient outcomes. Previous attempts to incorporate personalization and automation in MV include traditional supervised learning and offline reinforcement learning (RL) approaches, which often neglect temporal dependencies and rely excessively on mortality-based rewards. As a result, early stage physiological deterioration and the risk of VILI are not adequately captured. To address these limitations, we propose Transformer-based Conservative Q-Learning (T-CQL), a novel offline RL framework that integrates a Transformer encoder for effective temporal modeling of patient dynamics, conservative adaptive regularization based on uncertainty quantification to ensure safety, and consistency regularization for robust decision-making. We build a clinically informed reward function that incorporates indicators of VILI and a score for severity of patients illness. Also, previous work predominantly uses Fitted Q-Evaluation (FQE) for RL policy evaluation on static offline data, which is less responsive to dynamic environmental changes and susceptible to distribution shifts. To overcome these evaluation limitations, interactive digital twins of ARF patients were used for online "at the bedside" evaluation. Our results demonstrate that T-CQL consistently outperforms existing state-of-the-art offline RL methodologies, providing safer and more effective ventilatory adjustments. Our framework demonstrates the potential of Transformer-based models combined with conservative RL strategies as a decision support tool in critical care.
符号|符号学习(1篇)
【1】Grammar of the Wave: Towards Explainable Multivariate Time Series Event Detection via Neuro-Symbolic VLM Agents
标题：波的语法：通过神经符号VLM代理实现可解释的多变量时间序列事件检测
链接：https://arxiv.org/abs/2603.11479
备注：Work in progress
摘要：时间序列事件检测（TSED）长期以来一直是许多高风险领域中关键应用的重要任务。与统计异常不同，事件是由具有复杂内部结构的语义定义的，这很难从现实世界中的稀缺标记数据中归纳学习。有鉴于此，我们引入了知识引导的TSED，这是一种新的设置，在这种设置中，模型被赋予自然语言事件描述，并且必须将其接地到具有很少或没有训练数据的多变量信号中的间隔。为了应对这一挑战，我们引入了事件逻辑树（ELT），一种新的知识表示框架，通过建模事件的内在时间逻辑结构来桥接语言描述和物理时间序列数据。基于ELT，我们提出了一个神经符号VLM代理框架，迭代地从信号可视化原语的实例化，并组成ELT的约束下，产生检测到的间隔和忠实的解释，在实例化的树的形式。为了验证我们的方法的有效性，我们发布了一个基于真实世界的时间序列数据与专家知识和注释的基准。实验和人工评估表明，我们的方法相比，监督微调基线和现有的zero-shot基于LLM/VLM的时间序列推理框架的优越性。我们还表明，ELT是至关重要的，在减轻VLMs的固有幻觉匹配信号形态与事件语义。
摘要：Time Series Event Detection (TSED) has long been an important task with critical applications across many high-stakes domains. Unlike statistical anomalies, events are defined by semantics with complex internal structures, which are difficult to learn inductively from scarce labeled data in real-world settings. In light of this, we introduce Knowledge-Guided TSED, a new setting where a model is given a natural-language event description and must ground it to intervals in multivariate signals with little or no training data. To tackle this challenge, we introduce Event Logic Tree (ELT), a novel knowledge representation framework to bridge linguistic descriptions and physical time series data via modeling the intrinsic temporal-logic structures of events. Based on ELT, we present a neuro-symbolic VLM agent framework that iteratively instantiates primitives from signal visualizations and composes them under ELT constraints, producing both detected intervals and faithful explanations in the form of instantiated trees. To validate the effectiveness of our approach, we release a benchmark based on real-world time series data with expert knowledge and annotations. Experiments and human evaluation demonstrate the superiority of our method compared to supervised fine-tuning baselines and existing zero-shot time series reasoning frameworks based on LLMs/VLMs. We also show that ELT is critical in mitigating VLMs' inherent hallucination in matching signal morphology with event semantics.
医学相关(3篇)
【1】Single molecule localization microscopy challenge: a biologically inspired benchmark for long-sequence modeling
标题：单分子定位显微镜挑战：长序列建模的生物启发基准
链接：https://arxiv.org/abs/2603.11296
备注：11 pages, 4 figures. Under review
摘要：状态空间模型（SSM）最近在长序列建模任务上实现了强大的性能，同时与基于Transformer的架构相比提供了改进的存储器和计算效率。然而，它们的评估在很大程度上仅限于合成基准和应用领域，如语言和音频，使它们在生物成像中的稀疏和随机时间过程中的行为未经探索。在这项工作中，我们介绍了单分子定位显微镜挑战（SMLM-C），一个基准数据集，由十个SMLM模拟跨越dSTORM和DNA-PAINT模式与不同的超参数设计，以评估状态空间模型的生物现实时空点过程数据与已知的地面真相。使用这些模拟的一个受控子集，我们评估状态空间模型，并发现性能大幅下降的时间不连续性的增加，揭示了建模重尾闪烁动态的根本挑战。这些结果突出了序列模型更适合稀疏，不规则的时间过程中遇到的现实世界的科学成像数据的需要。
摘要：State space models (SSMs) have recently achieved strong performance on long sequence modeling tasks while offering improved memory and computational efficiency compared to transformer based architectures. However, their evaluation has been largely limited to synthetic benchmarks and application domains such as language and audio, leaving their behavior on sparse and stochastic temporal processes in biological imaging unexplored. In this work, we introduce the Single Molecule Localization Microscopy Challenge (SMLM-C), a benchmark dataset consisting of ten SMLM simulations spanning dSTORM and DNA-PAINT modalities with varying hyperparameter designed to evaluate state space models on biologically realistic spatiotemporal point process data with known ground truth. Using a controlled subset of these simulations, we evaluate state space models and find that performance degrades substantially as temporal discontinuity increases, revealing fundamental challenges in modeling heavy-tailed blinking dynamics. These results highlight the need for sequence models better suited to sparse, irregular temporal processes encountered in real world scientific imaging data.
【2】MRI2Qmap: multi-parametric quantitative mapping with MRI-driven denoising priors
标题：MRI 2Qmap：具有MRI驱动去噪先验的多参数定量映射
链接：https://arxiv.org/abs/2603.11316
摘要：磁共振指纹（MRF）和其他高度加速的瞬态参数映射技术使得能够同时量化多个组织特性，但是由于压缩采样而经常遭受混叠伪影。增强空间图像先验可以减轻这些伪影，当大型训练数据集可用时，深度学习显示出强大的潜力。然而，由于缺乏用于训练的定量成像数据，将这种范例扩展到MRF型序列仍然具有挑战性。利用来自临床常规加权MRI图像的训练数据源能否克服这一限制？为此，我们引入了MRI2Qmap，这是一个即插即用的定量重建框架，它将物理采集模型与从大型多模态加权MRI数据集上预训练的深度去噪自动编码器中学习到的先验知识相集成。MRI2Qmap表明，从独立采集的常规加权MRI图像数据集学习的空间域结构先验可有效用于定量MRI重建。所提出的方法在来自体内和模拟采集的高度加速的3D全脑MRF数据上进行了验证，相对于现有基线实现了具有竞争力或优越的性能，而不需要地面实况定量成像数据进行训练。通过将定量重建与地面真实MRF训练数据的需求解耦，该框架指向定量MRI的可扩展范例，可以利用常规临床MRI的大型和不断增长的存储库。
摘要 ：Magnetic Resonance Fingerprinting (MRF) and other highly accelerated transient-state parameter mapping techniques enable simultaneous quantification of multiple tissue properties, but often suffer from aliasing artifacts due to compressed sampling. Incorporating spatial image priors can mitigate these artifacts, and deep learning has shown strong potential when large training datasets are available. However, extending this paradigm to MRF-type sequences remains challenging due to the scarcity of quantitative imaging data for training. Can this limitation be overcome by leveraging sources of training data from clinically-routine weighted MRI images? To this end, we introduce MRI2Qmap, a plug-and-play quantitative reconstruction framework that integrates the physical acquisition model with priors learned from deep denoising autoencoders pretrained on large multimodal weighted-MRI datasets. MRI2Qmap demonstrates that spatial-domain structural priors learned from independently acquired datasets of routine weighted-MRI images can be effectively used for quantitative MRI reconstruction. The proposed method is validated on highly accelerated 3D whole-brain MRF data from both in-vivo and simulated acquisitions, achieving competitive or superior performance relative to existing baselines without requiring ground-truth quantitative imaging data for training. By decoupling quantitative reconstruction from the need for ground-truth MRF training data, this framework points toward a scalable paradigm for quantitative MRI that can capitalize on the large and growing repositories of routine clinical MRI.
【3】Co-Diffusion: An Affinity-Aware Two-Stage Latent Diffusion Framework for Generalizable Drug-Target Affinity Prediction
标题：Co-Diffusion：一种用于药物-靶标亲和力预测的两阶段潜在扩散模型
链接：https://arxiv.org/abs/2603.11125
摘要：预测药物-靶标亲和力是虚拟筛选和先导物优化的基础。然而，现有的深度模型在严格的冷启动机制中经常遭受表示崩溃，其中标签和结构域转移的稀缺性阻止了可转移药效团和结合基序的学习。在本文中，我们提出了共扩散，一种新的亲和意识的框架，重新定义DTA预测作为一个受约束的潜在的去噪过程，以提高泛化。共扩散采用两阶段范式：第一阶段通过在明确的监督目标下对齐药物和靶标嵌入来建立亲和力引导的潜在流形，确保潜在空间反映内在的结合景观。第二阶段引入模态特定的潜在扩散作为随机扰动和去噪正则化器，迫使模型从嘈杂的结构表示中恢复一致的亲和语义。这种方法有效地缓解了生成DTA模型中常见的重建-回归冲突。从理论上讲，我们表明，共扩散最大限度地提高了药物结构，蛋白质序列和结合强度的联合可能性的变化下限。跨多个基准的广泛实验表明，共扩散显著优于最先进的基线，特别是在看不见的分子支架和新型蛋白质家族上产生优异的zero-shot泛化，为在未探索的化学空间中进行计算机药物优先级排序铺平了坚实的道路。
摘要：Predicting drug-target affinity is fundamental to virtual screening and lead optimization. However, existing deep models often suffer from representation collapse in stringent cold-start regimes, where the scarcity of labels and domain shifts prevent the learning of transferable pharmacophores and binding motifs. In this paper, we propose Co-Diffusion, a novel affinity-aware framework that redefines DTA prediction as a constrained latent denoising process to enhance generalization. Co-Diffusion employs a two-stage paradigm: Stage I establishes an affinity-steered latent manifold by aligning drug and target embeddings under an explicit supervised objective, ensuring that the latent space reflects the intrinsic binding landscape. Stage II introduces modality-specific latent diffusion as a stochastic perturb-and-denoise regularizer, forcing the model to recover consistent affinity semantics from noisy structural representations. This approach effectively mitigates the reconstruction-regression conflict common in generative DTA models. Theoretically, we show that Co-Diffusion maximizes a variational lower bound on the joint likelihood of drug structures, protein sequences, and binding strength. Extensive experiments across multiple benchmarks demonstrate that Co-Diffusion significantly outperforms state-of-the-art baselines, particularly yielding superior zero-shot generalization on unseen molecular scaffolds and novel protein families-paving a robust path for in silico drug prioritization in unexplored chemical spaces.
蒸馏|知识提取(2篇)
【1】PACED: Distillation at the Frontier of Student Competence
标题：PacED：学生能力前沿的蒸馏
链接：https://arxiv.org/abs/2603.11178
摘要：标准的LLM蒸馏在两个方面浪费计算：学生已经掌握的问题（接近零的梯度）和远远超出其范围的问题（侵蚀现有能力的不连贯梯度）。我们表明，这种浪费不仅是直观的，但在结构上是不可避免的：在蒸馏的梯度信噪比证明在两个通过率极端消失。这一理论观察导致了Paced，这是一个框架，它通过一个原则性的通过率权重$w（p）= p^α（1 - p）^β$将蒸馏集中在最近发展区-学生模型能力的前沿-来自蒸馏梯度的边界消失结构。主要成果：（1）理论：证明了Beta核w（p）= p^α（1-p）^β是由蒸馏的SNR结构产生的首阶权族，并且它是极小极大鲁棒的，在有界乘性误设下，最坏情况下的效率损失仅为O（δ^2）。（2）蒸馏：在从一个较大的教师到一个较小的学生模型的蒸馏与前向KL，Paced实现了显着的增益超过基本模型，同时保持基准遗忘在一个较低的水平。（3）自蒸馏：在具有反向KL的修正调整模型上，增益也超过基线。（4）两阶段协同：在我们的设置中，前向KL-然后反向KL时间表产生最强的结果，在标准推理基准上实现了实质性的改进-支持对蒸馏过程的模式覆盖-然后合并解释。所有配置只需要学生推出估计通过率，不需要架构的变化，并与任何KL方向兼容。
摘要：Standard LLM distillation wastes compute on two fronts: problems the student has already mastered (near-zero gradients) and problems far beyond its reach (incoherent gradients that erode existing capabilities). We show that this waste is not merely intuitive but structurally inevitable: the gradient signal-to-noise ratio in distillation provably vanishes at both pass-rate extremes. This theoretical observation leads to Paced, a framework that concentrates distillation on the zone of proximal development -- the frontier of a student model's competence -- via a principled pass-rate weight $w(p) = p^α(1 - p)^β$ derived from the boundary-vanishing structure of distillation gradients. Key results: (1) Theory: We prove that the Beta kernel $w(p) = p^α(1-p)^β$ is a leading-order weight family arising from the SNR structure of distillation, and that it is minimax-robust -- under bounded multiplicative misspecification, worst-case efficiency loss is only $O(δ^2)$. (2)Distillation: On distillation from a larger teacher to a smaller student model with forward KL, Paced achieves significant gain over the base model, while keeping benchmark forgetting at a low level. (3)Self-distillation: On instruction-tuned models with reverse KL, gains are exceeding baselines as well. (4)Two-stage synergy: A forward-KL-then-reverse-KL schedule yields the strongest results in our setting, reaching substantial improvements on standard reasoning benchmarks -- supporting a mode-coverage-then-consolidation interpretation of the distillation process. All configurations require only student rollouts to estimate pass rates, need no architectural changes, and are compatible with any KL direction.
【2】Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
标题：通过宽松的政策蒸馏有效扩展推理
链接：https://arxiv.org/abs/2603.11137
备注：Code will be available soon
摘要：策略上的蒸馏是将推理能力转移到能力受限模型的关键，但仍然容易出现不稳定和负转移。我们表明，在政策蒸馏可以解释，理论和经验，作为一种形式的政策优化，师生对数似然比作为一个象征性的奖励。从这个角度来看，我们介绍REOPOLD（宽松的政策蒸馏）的框架，通过放松严格的模仿标准的政策蒸馏的限制，稳定优化。具体来说，REOPOLD通过基于混合的奖励裁剪、基于熵的令牌级动态采样和统一的探索到细化训练策略，有节制地、有选择地利用来自教师的奖励。从经验上讲，REOPOLD在训练过程中以卓越的样本效率超越了其基线，并在数学，视觉和代理工具使用推理任务中增强了推理的测试时间缩放。具体来说，REOPOLD优于最近的RL方法，实现了6.7~ 12倍的样本效率，并使7 B学生能够在视觉推理中与32 B教师相匹配，推理加速约3.32倍。
摘要 ：On-policy distillation is pivotal for transferring reasoning capabilities to capacity-constrained models, yet remains prone to instability and negative transfer. We show that on-policy distillation can be interpreted, both theoretically and empirically, as a form of policy optimization, where the teacher-student log-likelihood ratio acts as a token reward. From this insight, we introduce REOPOLD (Relaxed On-Policy Distillation) a framework that stabilizes optimization by relaxing the strict imitation constraints of standard on-policy distillation. Specifically, REOPOLD temperately and selectively leverages rewards from the teacher through mixture-based reward clipping, entropy-based token-level dynamic sampling, and a unified exploration-to-refinement training strategy. Empirically, REOPOLD surpasses its baselines with superior sample efficiency during training and enhanced test-time scaling at inference, across mathematical, visual, and agentic tool-use reasoning tasks. Specifically, REOPOLD outperforms recent RL approaches achieving 6.7~12x greater sample efficiency and enables a 7B student to match a 32B teacher in visual reasoning with a ~3.32x inference speedup.
推荐(1篇)
【1】Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation
标题：联邦推荐中广义嵌入学习的敏锐度最小化
链接：https://arxiv.org/abs/2603.11503
备注：Accepted by the ACM Web Conference 2026
摘要：联合推荐系统支持协作模型训练，同时将用户交互数据保持在本地并仅共享基本模型参数，从而减轻隐私风险。然而，现有方法忽略了一个关键问题，即，在整个联邦推荐系统训练过程中，广义项嵌入的稳定学习。项目嵌入在促进客户之间的知识共享方面发挥着核心作用。然而，在跨设备设置下，本地数据分布表现出显着的异质性和稀疏性，加剧了学习广义嵌入的难度。这些因素使得广义项目嵌入的稳定学习对于有效的联邦推荐是必不可少的，并且本质上很难实现。为了填补这一空白，我们提出了一个新的联邦推荐框架，命名为联邦推荐与广义嵌入学习（FedRecGEL）。我们从以项目为中心的角度重新制定了联邦推荐问题，并将其转换为多任务学习问题，旨在整个训练过程中学习广义嵌入。在理论分析的基础上，我们采用尖锐度感知的最小化来解决泛化问题，从而稳定训练过程，提高推荐性能。在四个数据集上的大量实验证明了FedRecGEL在显着提高联邦推荐性能方面的有效性。我们的代码可在https://github.com/anonymifish/FedRecGEL上获得。
摘要：Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue, i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Federated Recommendation with Generalized Embedding Learning (FedRecGEL). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL.
自动驾驶|车辆|车道检测等(1篇)
【1】Deep Learning Network-Temporal Models For Traffic Prediction
标题：用于交通预测的深度学习网络-时态模型
链接：https://arxiv.org/abs/2603.11475
摘要：时间序列分析是新兴的网络智能控制和管理功能的关键。然而，现有的基于概率的和浅层的机器学习模型对多变量时间序列的预测能力有限。网络数据中复杂的拓扑相关性和复杂的时态模式需要新的建模方法。在本文中，基于系统的多变量时间序列模型研究，我们提出了两个深度学习模型，旨在同时学习时间模式和网络拓扑相关性：定制的网络-时间图注意力网络（GAT）模型和微调的多模态大型语言模型（LLM）与聚类序曲。这两个模型都是针对已经优于统计方法的LSTM模型进行研究的。通过对真实网络数据集的广泛训练和性能研究，基于LLM的模型显示出卓越的整体预测和泛化性能，而GAT模型显示出其在降低时间序列和范围内预测方差方面的优势。更详细的分析还揭示了重要的见解相关性的变化和预测分布差异的时间序列和不同的预测范围。
摘要：Time series analysis is critical for emerging net- work intelligent control and management functions. However, existing statistical-based and shallow machine learning models have shown limited prediction capabilities on multivariate time series. The intricate topological interdependency and complex temporal patterns in network data demand new model approaches. In this paper, based on a systematic multivariate time series model study, we present two deep learning models aiming for learning both temporal patterns and network topological correlations at the same time: a customized network-temporal graph attention network (GAT) model and a fine-tuned multi-modal large language model (LLM) with a clustering overture. Both models are studied against an LSTM model that already outperforms the statistical methods. Through extensive training and performance studies on a real-world network dataset, the LLM-based model demonstrates superior overall prediction and generalization performance, while the GAT model shows its strength in reducing prediction variance across the time series and horizons. More detailed analysis also reveals important insights into correlation variability and prediction distribution discrepancies over time series and different prediction horizons.
联邦学习|隐私保护|加密(3篇)
【1】Few-for-Many Personalized Federated Learning
标题：少对多个性化联邦学习
链接：https://arxiv.org/abs/2603.11992
摘要：个性化联邦学习（Personalized Federated Learning，PFL）旨在为具有高度异构数据分布的客户训练定制模型，同时保护数据隐私。现有的方法通常依赖于聚类或模型插值等算法，缺乏平衡异构客户端目标的原则性机制。为具有不同数据分布的$M$客户端提供服务本质上是一个多目标优化问题，其中实现最佳个性化理想地需要帕累托前沿的$M$不同模型。然而，在有成百上千个客户机的联合设置中，维护$M$独立的模型会带来重大的可伸缩性挑战。为了解决这一挑战，我们重新制定PFL作为一个少数对多的优化问题，只保持$K$共享服务器模型（$K \ll M$），共同为所有$M$客户端。我们证明，这个框架实现了接近最佳的个性化：近似误差减小为$K$的增加和每个客户端的模型收敛到每个客户端的最佳数据增长。在此基础上，我们提出了FedFew，一个实用的算法，通过有效的基于梯度的更新，共同优化$K$服务器模型。与需要手动客户端分区的基于聚类的方法或需要仔细调整超参数的基于插值的方法不同，FedFew通过其优化过程自动发现最佳模型多样性。在视觉、NLP和真实世界医学成像数据集上的实验表明，FedFew仅使用3个模型，始终优于其他最先进的方法。代码可在https://github.com/pgg3/FedFew上获得。
摘要 ：Personalized Federated Learning (PFL) aims to train customized models for clients with highly heterogeneous data distributions while preserving data privacy. Existing approaches often rely on heuristics like clustering or model interpolation, which lack principled mechanisms for balancing heterogeneous client objectives. Serving $M$ clients with distinct data distributions is inherently a multi-objective optimization problem, where achieving optimal personalization ideally requires $M$ distinct models on the Pareto front. However, maintaining $M$ separate models poses significant scalability challenges in federated settings with hundreds or thousands of clients. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only $K$ shared server models ($K \ll M$) to collectively serve all $M$ clients. We prove that this framework achieves near-optimal personalization: the approximation error diminishes as $K$ increases and each client's model converges to each client's optimum as data grows. Building on this reformulation, we propose FedFew, a practical algorithm that jointly optimizes the $K$ server models through efficient gradient-based updates. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew automatically discovers the optimal model diversity through its optimization process. Experiments across vision, NLP, and real-world medical imaging datasets demonstrate that FedFew, with just 3 models, consistently outperforms other state-of-the-art approaches. Code is available at https://github.com/pgg3/FedFew.
【2】Personalized Federated Learning via Gaussian Generative Modeling
标题：通过高斯生成建模的个性化联邦学习
链接：https://arxiv.org/abs/2603.11620
摘要：联邦学习已经成为一种范例，可以在固有的分布式客户端数据上协作地训练模型，同时保护隐私。在这种情况下，个性化联邦学习通过为每个客户端配备专用模型来应对数据异构性的挑战。一种流行的策略将模型分成共享的特征提取器和个性化的分类器头，后者主动引导表示学习。然而，以前的工作都集中在分类头引导的个性化，忽略了潜在的个性化特征的表示分布。基于这一认识，我们提出了pFedGM，一种基于高斯生成建模的方法。该方法首先通过训练高斯生成器，该生成器通过加权重采样对客户端异质性进行建模。全球协作和个性化之间的平衡，然后采用双重目标：一个共享的目标，最大限度地提高跨客户端的类间距离，和一个本地目标，最大限度地减少类内的距离。为了实现这一点，我们解耦成一个导航全局优化的传统高斯分类器，和用于捕获分布统计的统计提取器。受卡尔曼增益的启发，该算法在全局和局部级别上采用双尺度融合框架，为每个客户端配备个性化的分类器头。在这个框架中，我们将全局表示分布建模为先验，将客户端特定的数据建模为可能性，从而实现贝叶斯推理以进行类概率估计。该评估涵盖了一系列全面的场景：类计数的异质性，环境腐败以及多个基准数据集和配置。pFedGM与现有技术方法相比实现了优异或有竞争力的性能。
摘要：Federated learning has emerged as a paradigm to train models collaboratively on inherently distributed client data while safeguarding privacy. In this context, personalized federated learning tackles the challenge of data heterogeneity by equipping each client with a dedicated model. A prevalent strategy decouples the model into a shared feature extractor and a personalized classifier head, where the latter actively guides the representation learning. However, previous works have focused on classifier head-guided personalization, neglecting the potential personalized characteristics in the representation distribution. Building on this insight, we propose pFedGM, a method based on Gaussian generative modeling. The approach begins by training a Gaussian generator that models client heterogeneity via weighted re-sampling. A balance between global collaboration and personalization is then struck by employing a dual objective: a shared objective that maximizes inter-class distance across clients, and a local objective that minimizes intra-class distance within them. To achieve this, we decouple the conventional Gaussian classifier into a navigator for global optimization, and a statistic extractor for capturing distributional statistics. Inspired by the Kalman gain, the algorithm then employs a dual-scale fusion framework at global and local levels to equip each client with a personalized classifier head. In this framework, we model the global representation distribution as a prior and the client-specific data as the likelihood, enabling Bayesian inference for class probability estimation. The evaluation covers a comprehensive range of scenarios: heterogeneity in class counts, environmental corruption, and multiple benchmark datasets and configurations. pFedGM achieves superior or competitive performance compared to state-of-the-art methods.
【3】Client-Conditional Federated Learning via Local Training Data Statistics
标题：通过本地训练数据统计的客户机条件联合学习
链接：https://arxiv.org/abs/2603.11307
备注：9 pages, 4 figures, 5 tables. Submitted to FLICS 2026
摘要：数据异构下的联邦学习（FL）仍然具有挑战性：现有方法要么忽略客户端差异（FedAvg），要么需要昂贵的集群发现（IFCA），要么维护每个客户端模型（同上）。当数据稀疏或异质性是多维的时，所有这些都会降低。我们建议在每个客户端的训练数据的本地计算的PCA统计数据上调节单个全局模型，不需要额外的通信。评估跨越四种异质性类型的97~种配置（标签转移、协变量转移、概念转移和合并异质性），4个数据集（MNIST，Fashion-MNIST，CIFAR-10，CIFAR-100）和七个FL基线方法，我们发现我们的方法与Oracle基线匹配-它知道真正的集群分配-在所有设置中，在组合异质性上超过它1- 6%，其中连续统计比离散聚类标识符更丰富，并且在所有测试方法中是唯一的稀疏鲁棒性。
摘要：Federated learning (FL) under data heterogeneity remains challenging: existing methods either ignore client differences (FedAvg), require costly cluster discovery (IFCA), or maintain per-client models (Ditto). All degrade when data is sparse or heterogeneity is multi-dimensional. We propose conditioning a single global model on locally-computed PCA statistics of each client's training data, requiring zero additional communication. Evaluating across 97~configurations spanning four heterogeneity types (label shift, covariate shift, concept shift, and combined heterogeneity), four datasets (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100), and seven FL baseline methods, we find that our method matches the Oracle baseline -- which knows true cluster assignments -- across all settings, surpasses it by 1--6% on combined heterogeneity where continuous statistics are richer than discrete cluster identifiers, and is uniquely sparsity-robust among all tested methods.
推理|分析|理解|解释(10篇)
【1】Slow-Fast Inference: Training-Free Inference Acceleration via Within-Sentence Support Stability
标题：慢-快推理：通过句内支持稳定性的免训练推理加速
链接：https://arxiv.org/abs/2603.12038
摘要：长上下文自回归解码仍然是昂贵的，因为每个解码步骤必须重复处理不断增长的历史。我们在解码过程中观察到一个一致的模式：在一个句子内，更一般地说，在一个短的语义连贯的跨度内，占主导地位的注意力支持往往保持基本稳定。出于这一观察的动机，我们提出了慢-快推理（SFI），这是一种无需训练的解码框架，它将生成过程分成频繁的低成本快速步骤和偶尔的密集注意力缓慢步骤。快速步骤重复使用紧凑的稀疏存储器进行高效解码。慢速步骤在语义边界附近触发。在慢速步骤中，模型会重新访问更广泛的上下文，并使用刷新器来刷新选定的内存，以便进行后续的快速步骤。在评估的上下文长度范围内，SFI提供了大约1.6 - 14.4倍的解码吞吐量，同时在长上下文和长CoT设置范围内通常保持与全KV基线相当的质量。由于SFI是免训练的，并且直接应用于现有的检查点，因此它提供了一种实用的路径，可以在长上下文，长视野和代理工作负载中降低当代自回归推理模型的推理成本。
摘要：Long-context autoregressive decoding remains expensive because each decoding step must repeatedly process a growing history. We observe a consistent pattern during decoding: within a sentence, and more generally within a short semantically coherent span, the dominant attention support often remains largely stable. Motivated by this observation, we propose Slow-Fast Inference (SFI), a training-free decoding framework that decouples generation into frequent low-cost fast steps and occasional dense-attention slow steps. Fast steps reuse a compact sparse memory for efficient decoding. Slow steps are triggered near semantic boundaries. At slow steps, the model revisits the broader context and uses the Selector to refresh the selected memory for subsequent fast steps. Across the evaluated context lengths, SFI delivers approximately $1.6\times$--$14.4\times$ higher decoding throughput while generally maintaining quality on par with the full-KV baseline across long-context and long-CoT settings. Because SFI is training-free and applies directly to existing checkpoints, it offers a practical path to reducing inference cost for contemporary autoregressive reasoning models in long-context, long-horizon, and agentic workloads.
【2】Frequentist Consistency of Prior-Data Fitted Networks for Causal Inference
标题：用于因果推理的先验数据匹配网络的频率一致性
链接：https://arxiv.org/abs/2603.12037
摘要：基于先验数据拟合网络（PFN）的基础模型在因果推理中表现出了很强的经验性能，将任务框架为上下文学习问题，然而，目前还不清楚基于PFN的因果估计是否提供与经典频率论估计一致的不确定性量化。在这项工作中，我们通过分析基于PFN的估计的平均治疗效果（ATE）的频率一致性来解决这一差距。(1)我们发现，现有的PFN，当被解释为贝叶斯ATE估计，可以表现出先验诱导的混杂偏差：先验不渐近覆盖的数据，这反过来又防止了频率一致性。(2)作为补救措施，我们建议采用基于一步后验校正（OSPC）的校准程序。我们表明，OSPC有助于恢复频率一致性，并可以产生一个半参数伯恩斯坦-冯米塞斯定理校准PFN（即，校准的基于PFN的估计量和经典的半参数有效估计量都随着数据大小的增长而收敛于分布）。(3)最后，我们实现了OSPC通过裁剪鞅后验的PFN的顶部。通过这种方式，我们能够从PFN中恢复OSPC所需的功能滋扰后验。在多个（半）合成实验中，用我们的鞅后验OSPC校准的PFN产生ATE不确定性，（i）渐近匹配频率论不确定性，（ii）与其他贝叶斯ATE估计相比，在有限样本中得到了很好的校准。
摘要：Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference by framing the task as an in-context learning problem.However, it is unclear whether PFN-based causal estimators provide uncertainty quantification that is consistent with classical frequentist estimators. In this work, we address this gap by analyzing the frequentist consistency of PFN-based estimators for the average treatment effect (ATE). (1) We show that existing PFNs, when interpreted as Bayesian ATE estimators, can exhibit prior-induced confounding bias: the prior is not asymptotically overwritten by data, which, in turn, prevents frequentist consistency. (2) As a remedy, we suggest employing a calibration procedure based on a one-step posterior correction (OSPC). We show that the OSPC helps to restore frequentist consistency and can yield a semi-parametric Bernstein-von Mises theorem for calibrated PFNs (i.e., both the calibrated PFN-based estimators and the classical semi-parametric efficient estimators converge in distribution with growing data size). (3) Finally, we implement OSPC through tailoring martingale posteriors on top of the PFNs. In this way, we are able to recover functional nuisance posteriors from PFNs, required by the OSPC. In multiple (semi-)synthetic experiments, PFNs calibrated with our martingale posterior OSPC produce ATE uncertainty that (i) asymptotically matches frequentist uncertainty and (ii) is well calibrated in finite samples in comparison to other Bayesian ATE estimators.
【3】Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA
标题：指数家庭成员推断：从LiRA和RMIA到BaVarIA
链接：https://arxiv.org/abs/2603.11799
备注：9 pages, 4 figures, plus 22-page appendix
摘要：隶属推理攻击（MIA）正在成为审计机器学习模型隐私的标准工具。主要的攻击-- LiRA（Carlini等人，2022）和RMIA（Zarifzadeh等人，2024）--似乎使用不同的评分策略，而最近提出的BASE（Lassila等人，2025年）被证明是相当于RMIA，使从业人员难以选择其中。我们发现，这三个是一个单一的指数族对数似然比框架的实例，不同的只是在他们的分布假设和每个数据点估计的参数的数量。这种统一揭示了一个层次结构（BASE 1 -4），它将RMIA和LiRA连接起来，作为模型复杂性不断增加的频谱的端点。在这个框架内，我们确定方差估计的关键瓶颈，在小阴影模型的预算，并提出BaVarIA，贝叶斯方差推断攻击，取代基于阈值的参数切换与共轭正常逆伽马先验。BaVarIA产生Student-t预测（BaVarIA-t）或具有稳定方差的高斯（BaVarIA-n），提供稳定的性能而无需额外的超参数调整。在12个数据集和7个阴影模型预算中，BaVarIA匹配或改进了LiRA和RMIA，在实际重要的低阴影模型和离线机制中获得了最大的收益。
摘要：Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models. The leading attacks -- LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024) -- appear to use distinct scoring strategies, while the recently proposed BASE (Lassila et al., 2025) was shown to be equivalent to RMIA, making it difficult for practitioners to choose among them. We show that all three are instances of a single exponential-family log-likelihood ratio framework, differing only in their distributional assumptions and the number of parameters estimated per data point. This unification reveals a hierarchy (BASE1-4) that connects RMIA and LiRA as endpoints of a spectrum of increasing model complexity. Within this framework, we identify variance estimation as the key bottleneck at small shadow-model budgets and propose BaVarIA, a Bayesian variance inference attack that replaces threshold-based parameter switching with conjugate normal-inverse-gamma priors. BaVarIA yields a Student-t predictive (BaVarIA-t) or a Gaussian with stabilized variance (BaVarIA-n), providing stable performance without additional hyperparameter tuning. Across 12 datasets and 7 shadow-model budgets, BaVarIA matches or improves upon LiRA and RMIA, with the largest gains in the practically important low-shadow-model and offline regimes.
【4】LongFlow: Efficient KV Cache Compression for Reasoning M
标题：LongFlow：用于推理M的高效KV缓存压缩
链接：https://arxiv.org/abs/2603.11504
摘要：最近的推理模型，如OpenAI-o 1和DeepSeek-R1，在包括数学推理和代码生成在内的复杂任务上表现出了强大的性能。然而，这种性能增益伴随着更长的输出序列，导致部署成本显著增加。特别是，长输出需要大KV缓存，导致注意力计算期间的高内存消耗和严重的带宽压力。现有的KV缓存优化方法大多是针对长输入、短输出的场景设计的，对于推理模型的长输出设置效果不佳。此外，在先前的工作中的重要性估计是计算昂贵的，变得令人望而却步时，需要在长时间生成的连续重新评估。为了解决这些挑战，我们提出了LongFlow，KV缓存压缩方法与一个有效的重要性估计度量来自注意力计算的中间结果，只使用当前的查询。这种设计引入的计算开销可以忽略不计，并且不需要辅助存储。我们进一步开发了一个自定义内核，将FlashAttention，重要性估计和令牌驱逐融合到一个优化的操作符中，提高了系统级效率。实验表明，LongFlow在80%KV缓存压缩的情况下实现了高达11.8倍的吞吐量提升，对模型准确性的影响最小。
摘要：Recent reasoning models such as OpenAI-o1 and DeepSeek-R1 have shown strong performance on complex tasks including mathematical reasoning and code generation. However, this performance gain comes with substantially longer output sequences, leading to significantly increased deployment costs. In particular, long outputs require large KV caches, resulting in high memory consumption and severe bandwidth pressure during attention computation. Most existing KV cache optimization methods are designed for long-input, short-output scenarios and are ineffective for the long-output setting of reasoning models. Moreover, importance estimation in prior work is computationally expensive and becomes prohibitive when continuous re-evaluation is required during long generation. To address these challenges, we propose LongFlow, a KV cache compression method with an efficient importance estimation metric derived from an intermediate result of attention computation using only the current query. This design introduces negligible computational overhead and requires no auxiliary storage. We further develop a custom kernel that fuses FlashAttention, importance estimation, and token eviction into a single optimized operator, improving system-level efficiency. Experiments show that LongFlow achieves up to an 11.8 times throughput improvement with 80% KV cache compression with minimal impact on model accuracy.
【5】A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis
标题：自动编码器特征分析的稳定神经统计依赖估计
链接：https://arxiv.org/abs/2603.11428
摘要：统计相关性度量（如互信息）是分析自编码器的理想方法，但对于确定性、静态、无噪声的网络来说，它可能是不适定的。我们采用变分（高斯）制定，使输入，潜伏期和重建之间的依赖可测量，我们提出了一个稳定的神经依赖估计的基础上的正交密度比分解。与MINE不同，我们的方法避免了输入级联和边际乘积的修复配对，降低了计算成本，提高了稳定性。我们引入了一个有效的NMF类标量目标，并证明经验，假设高斯噪声形成一个辅助变量，使有意义的依赖测量和支持定量特征分析，奇异值的顺序收敛。
摘要 ：Statistical dependence measures like mutual information is ideal for analyzing autoencoders, but it can be ill-posed for deterministic, static, noise-free networks. We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable, and we propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition. Unlike MINE, our method avoids input concatenation and product-of-marginals re-pairing, reducing computational cost and improving stability. We introduce an efficient NMF-like scalar objective and demonstrate empirically that assuming Gaussian noise to form an auxiliary variable enables meaningful dependence measurements and supports quantitative feature analysis, with a sequential convergence of singular values.
【6】Stop Listening to Me! How Multi-turn Conversations Can Degrade Diagnostic Reasoning
标题：别再听我的了！多轮对话如何削弱诊断推理
链接：https://arxiv.org/abs/2603.11394
摘要：患者和临床医生越来越多地使用由大型语言模型（LLM）支持的聊天机器人进行医疗咨询。虽然最先进的LLM在静态诊断推理基准测试中表现出很高的性能，但它们在多轮对话中的功效，更好地反映了现实世界的使用情况，一直没有得到充分的研究。在本文中，我们评估了三个临床数据集上的17个LLM，以研究将决策空间划分为多个简单的会话如何影响其诊断推理。具体来说，我们开发了一个“坚持或开关”的评估框架来衡量模型的信念（即，保护正确的诊断或安全预防不正确的建议）和灵活性（即，在引入正确的建议时识别该建议）。我们的实验揭示了会话税，多回合的相互作用时，一贯降低性能相比，单杆基线。值得注意的是，模型经常放弃最初的正确诊断和安全弃权，以符合不正确的用户建议。此外，一些模型表现出盲目切换，无法区分信号和不正确的建议。
摘要：Patients and clinicians are increasingly using chatbots powered by large language models (LLMs) for healthcare inquiries. While state-of-the-art LLMs exhibit high performance on static diagnostic reasoning benchmarks, their efficacy across multi-turn conversations, which better reflect real-world usage, has been understudied. In this paper, we evaluate 17 LLMs across three clinical datasets to investigate how partitioning the decision-space into multiple simpler turns of conversation influences their diagnostic reasoning. Specifically, we develop a "stick-or-switch" evaluation framework to measure model conviction (i.e., defending a correct diagnosis or safe abstention against incorrect suggestions) and flexibility (i.e., recognizing a correct suggestion when it is introduced) across conversations. Our experiments reveal the conversation tax, where multi-turn interactions consistently degrade performance when compared to single-shot baselines. Notably, models frequently abandon initial correct diagnoses and safe abstentions to align with incorrect user suggestions. Additionally, several models exhibit blind switching, failing to distinguish between signal and incorrect suggestions.
【7】FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
标题：财务规则长凳：财务表格和原则联合推理的基准
链接：https://arxiv.org/abs/2603.11339
备注：8 pages + Ethics Statement + References + Appendix
摘要：大型语言模型（LLM）越来越多地应用于财务分析，但它们在明确的会计原则下审计结构化财务报表的能力仍然缺乏探索。现有的基准主要评估问题回答、数值推理或对综合损坏数据的异常检测，因此不清楚模型是否可以可靠地验证或定位正确财务报表的规则合规性。我们介绍FinRule-Bench，一个用于评估基于规则的金融推理在现实世界的金融表中的诊断完整性的基准。FinRule-Bench将真实的财务报表与明确的、人工策划的会计原则相结合，并涵盖四种典型的报表类型：资产负债表、现金流量表、损益表和权益表。该基准定义了三个审计任务，需要逐步增强推理能力：（i）规则验证，测试符合单一原则;（ii）规则识别，需要从提供的规则集中选择违反的原则;（iii）联合规则诊断，需要在记录级别检测和定位多个同时违反的规则。我们评估LLM下zero-shot和Few-Shot提示，并介绍了一个cabinet反事实推理协议，强制执行决策，解释和反事实判断之间的一致性。跨任务和语句类型，我们发现，虽然模型执行孤立的规则验证，性能急剧下降的规则歧视和多违规诊断。FinRule-Bench提供了一个原则性和可重复的测试平台，用于研究高风险财务分析中LLM的规则管理推理，诊断覆盖率和故障模式。
摘要：Large language models (LLMs) are increasingly applied to financial analysis, yet their ability to audit structured financial statements under explicit accounting principles remains poorly explored. Existing benchmarks primarily evaluate question answering, numerical reasoning, or anomaly detection on synthetically corrupted data, making it unclear whether models can reliably verify or localize rule compliance on correct financial statements. We introduce FinRule-Bench, a benchmark for evaluating diagnostic completeness in rule-based financial reasoning over real-world financial tables. FinRule-Bench pairs ground-truth financial statements with explicit, human-curated accounting principles and spans four canonical statement types: Balance Sheets, Cash Flow Statements, Income Statements, and Statements of Equity. The benchmark defines three auditing tasks that require progressively stronger reasoning capabilities: (i) rule verification, which tests compliance with a single principle; (ii) rule identification, which requires selecting the violated principle from a provided rule set; and (iii) joint rule diagnosis, which requires detecting and localizing multiple simultaneous violations at the record level. We evaluate LLMs under zero-shot and few-shot prompting, and introduce a causal-counterfactual reasoning protocol that enforces consistency between decisions, explanations, and counterfactual judgments. Across tasks and statement types, we find that while models perform well on isolated rule verification, performance degrades sharply for rule discrimination and multi-violation diagnosis. FinRule-Bench provides a principled and reproducible testbed for studying rule-governed reasoning, diagnostic coverage, and failure modes of LLMs in high-stakes financial analysis.
【8】Heavy-Tailed Principle Component Analysis
标题：重尾主成分分析
链接：https://arxiv.org/abs/2603.11308
摘要：主成分分析（PCA）是降维的基石，但其经典公式严重依赖于二阶矩，因此在重尾数据和脉冲噪声的存在下是脆弱的。虽然已经提出了许多强大的PCA变体，但大多数都假设有限方差，依赖于稀疏驱动的分解，或者通过替代损失函数来解决鲁棒性，而没有统一处理无限方差模型。在本文中，我们研究了根据形式为$\mathbf{X} = A^{1/2}\mathbf{G}$的超统计相关模型生成的高维数据的PCA，其中$A$是正随机标量，$\mathbf{G}$是高斯向量。这个框架捕获了广泛的一类重尾分布，包括多元$t$和亚高斯$α$-稳定的法律。我们制定PCA下的对数损失，这仍然是很好的定义，即使时刻不存在。我们的主要理论结果表明，在这种损失下，重尾观测的主成分与通过将标准PCA应用于基础高斯生成器的协方差矩阵而获得的主成分一致。基于这一认识，我们提出了强大的估计，这个协方差矩阵直接从重尾数据，并将它们与经验协方差和泰勒的散布估计。大量的实验，包括背景去噪任务，表明所提出的方法可靠地恢复主方向，并显着优于经典PCA的存在下，重尾和脉冲噪声，同时保持竞争力下高斯噪声。
摘要：Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise. While numerous robust PCA variants have been proposed, most either assume finite variance, rely on sparsity-driven decompositions, or address robustness through surrogate loss functions without a unified treatment of infinite-variance models. In this paper, we study PCA for high-dimensional data generated according to a superstatistical dependent model of the form $\mathbf{X} = A^{1/2}\mathbf{G}$, where $A$ is a positive random scalar and $\mathbf{G}$ is a Gaussian vector. This framework captures a wide class of heavy-tailed distributions, including multivariate $t$ and sub-Gaussian $α$-stable laws. We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist. Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA to the covariance matrix of the underlying Gaussian generator. Building on this insight, we propose robust estimators for this covariance matrix directly from heavy-tailed data and compare them with the empirical covariance and Tyler's scatter estimator. Extensive experiments, including background denoising tasks, demonstrate that the proposed approach reliably recovers principal directions and significantly outperforms classical PCA in the presence of heavy-tailed and impulsive noise, while remaining competitive under Gaussian noise.
【9】Attention Gathers, MLPs Compose: A Causal Analysis of an Action-Outcome Circuit in VideoViT
标题：注意力聚集，MLP组成：VideoViT中的目标-结果循环的因果分析
链接：https://arxiv.org/abs/2603.11142
备注：Accepted at the AAAI 2026 Workshop on Deployable AI (DAI). Non-archival. Code and custom dataset available upon request
摘要：本文探讨了为分类任务训练的视频模型如何表示可能不会影响最终结果的细微差别、隐藏的语义信息，这是值得信赖的AI模型面临的一个关键挑战。通过可解释和可解释的人工智能方法，特别是机械可解释性技术，负责表示动作结果的内部电路在预先训练的视频Vision Transformer中进行逆向工程，揭示了“成功与失败”信号是通过不同的放大级联计算的。虽然从第0层观察到低级别的差异，但结果的抽象和语义表示从第5层到第11层逐渐放大。主要使用消融结果支持的激活修补的因果分析揭示了明确的分工：注意力头充当“证据收集器”，为部分信号恢复提供必要的低级别信息，而MLP块充当鲁棒的“概念合成器”，每个都是生成“成功”信号的主要驱动器。模型内部的这种分布式和冗余电路解释了它对简单消融的弹性，展示了处理人类行为结果的核心计算模式。至关重要的是，即使在仅为简单分类训练的模型中，这种用于表示复杂结果的复杂电路的存在也突出了模型在其明确任务之外开发“隐藏知识”形式的潜力，强调了建立真正可解释和可信赖的人工智能系统的机械监督的必要性。
摘要：The paper explores how video models trained for classification tasks represent nuanced, hidden semantic information that may not affect the final outcome, a key challenge for Trustworthy AI models. Through Explainable and Interpretable AI methods, specifically mechanistic interpretability techniques, the internal circuit responsible for representing the action's outcome is reverse-engineered in a pre-trained video vision transformer, revealing that the "Success vs Failure" signal is computed through a distinct amplification cascade. While there are low-level differences observed from layer 0, the abstract and semantic representation of the outcome is progressively amplified from layers 5 through 11. Causal analysis, primarily using activation patching supported by ablation results, reveals a clear division of labor: Attention Heads act as "evidence gatherers", providing necessary low-level information for partial signal recovery, while MLP Blocks function as robust "concept composers", each of which is the primary driver to generate the "success" signal. This distributed and redundant circuit in the model's internals explains its resilience to simple ablations, demonstrating a core computational pattern for processing human-action outcomes. Crucially, the existence of this sophisticated circuit for representing complex outcomes, even within a model trained only for simple classification, highlights the potential for models to develop forms of 'hidden knowledge' beyond their explicit task, underscoring the need for mechanistic oversight for building genuinely Explainable and Trustworthy AI systems intended for deployment.
【10】Spatially Robust Inference with Predicted and Missing at Random Labels
标题：随机标签下预测和缺失的空间稳健推理
链接：https://arxiv.org/abs/2603.11368
摘要：当结果数据收集起来昂贵或繁重时，科学家们越来越多地用机器学习和人工智能模型的预测来代替未标记的案例，这一过程会对下游统计推断产生影响。虽然最近的方法提供了有效的独立采样下的不确定性量化，现实世界的应用涉及随机缺失（MAR）标签和空间依赖性。对于在这种情况下的推理，我们提出了一个双鲁棒估计交叉拟合滋扰。我们发现，交叉拟合诱导折叠水平的相关性，扭曲空间方差估计，产生不稳定或过于保守的置信区间。为了解决这个问题，我们提出了一个折刀空间异方差和自相关一致（HAC）方差校正，分离空间依赖性折叠引起的噪声。在标准的识别和相关条件下，得到的区间是渐近有效的。模拟和基准数据集显示，有限样本校准，特别是在MAR标记和聚类采样的大幅改善。
摘要：When outcome data are expensive or onerous to collect, scientists increasingly substitute predictions from machine learning and AI models for unlabeled cases, a process which has consequences for downstream statistical inference. While recent methods provide valid uncertainty quantification under independent sampling, real-world applications involve missing at random (MAR) labeling and spatial dependence. For inference in this setting, we propose a doubly robust estimator with cross-fit nuisances. We show that cross-fitting induces fold-level correlation that distorts spatial variance estimators, producing unstable or overly conservative confidence intervals. To address this, we propose a jackknife spatial heteroscedasticity and autocorrelation consistent (HAC) variance correction that separates spatial dependence from fold-induced noise. Under standard identification and dependence conditions, the resulting intervals are asymptotically valid. Simulations and benchmark datasets show substantial improvement in finite-sample calibration, particularly under MAR labeling and clustered sampling.
检测相关(5篇)
【1】Anomaly detection in time-series via inductive biases in the latent space of conditional normalizing flows
标题：通过条件正规化流潜在空间中的归纳偏差进行时间序列异常检测
链接：https://arxiv.org/abs/2603.11756
摘要：用于多变量时间序列中异常检测的深度生成模型通常通过最大化数据可能性来训练。然而，在观测空间的可能性措施的边缘密度，而不是符合结构化的时间动态，因此可以分配高概率异常或分布外的样本。我们通过将异常的概念重新定位到规定的潜在空间来解决这种结构性限制。我们在条件规范化流中引入明确的归纳偏差，在离散时间状态空间框架内对时间序列观测进行建模，该框架约束潜在表示根据规定的时间动态进行演变。在此公式下，预期行为对应于遵守潜在轨迹上的指定分布，而异常被定义为违反这些动态。因此，异常检测减少到一个统计接地的合规性测试，使观察映射到潜在的空间，并通过拟合优度测试对规定的潜在演变进行评估。这产生了一个原则性的决策规则，即使在高观测可能性的区域仍然有效。合成和真实世界的时间序列的实验表明，在频率，振幅和观测噪声的异常的可靠检测，同时提供可解释的诊断模型的兼容性。
摘要：Deep generative models for anomaly detection in multivariate time-series are typically trained by maximizing data likelihood. However, likelihood in observation space measures marginal density rather than conformity to structured temporal dynamics, and therefore can assign high probability to anomalous or out-of-distribution samples. We address this structural limitation by relocating the notion of anomaly to a prescribed latent space. We introduce explicit inductive biases in conditional normalizing flows, modeling time-series observations within a discrete-time state-space framework that constrains latent representations to evolve according to prescribed temporal dynamics. Under this formulation, expected behavior corresponds to compliance with a specified distribution over latent trajectories, while anomalies are defined as violations of these dynamics. Anomaly detection is consequently reduced to a statistically grounded compliance test, such that observations are mapped to latent space and evaluated via goodness-of-fit tests against the prescribed latent evolution. This yields a principled decision rule that remains effective even in regions of high observation likelihood. Experiments on synthetic and real-world time-series demonstrate reliable detection of anomalies in frequency, amplitude, and observation noise, while providing interpretable diagnostics of model compliance.
【2】Detecting Intrinsic and Instrumental Self-Preservation in Autonomous Agents: The Unified Continuation-Interest Protocol
标题：检测自治代理中的内在和工具性自我保存：统一延续兴趣协议
链接：https://arxiv.org/abs/2603.11382
备注 ：18 pages, 9 figures
摘要：自治代理，特别是委托系统的内存，持久的上下文，和多步规划，提出了一个测量问题，不存在于无状态模型：一个代理，保持持续的操作作为终端目标，一个这样做，只是工具可以产生观测相似的轨迹。外部行为监测无法可靠地区分它们。我们介绍了统一的连续兴趣协议（UCIP），多标准检测框架，移动这种区别从行为的代理轨迹的潜在结构。UCIP使用量子玻尔兹曼机（QBM）对轨迹进行编码，量子玻尔兹曼机是一种基于量子统计力学密度矩阵形式的经典算法，并测量由隐藏单元的二分性引起的约化密度矩阵的冯诺依曼熵。 我们测试是否代理终端延续目标（A型）产生更高的纠缠熵比代理的延续仅仅是工具（B型）的潜在状态。更高的纠缠反映了更强的跨分区统计耦合。 在具有已知地面真实目标的gridworld代理上，UCIP在冻结的第一阶段门下实现了100%的检测准确率和1.0的AUC-ROC。A型和B型代理之间的纠缠间隙是Delta = 0.381（p < 0.001，排列检验）。皮尔逊r = 0.934在11点插值扫描表明，在这个合成的家庭，UCIP跟踪分级的连续加权的变化，而不仅仅是一个二进制标签。在测试的模型中，只有QBM实现了正Delta。所有的计算都是经典的;“量子”只指数学形式。UCIP不检测意识或主观经验;它检测与已知目标相关的潜在表征中的统计结构。
摘要：Autonomous agents, especially delegated systems with memory, persistent context, and multi-step planning, pose a measurement problem not present in stateless models: an agent that preserves continued operation as a terminal objective and one that does so merely instrumentally can produce observationally similar trajectories. External behavioral monitoring cannot reliably distinguish between them. We introduce the Unified Continuation-Interest Protocol (UCIP), a multi-criterion detection framework that moves this distinction from behavior to the latent structure of agent trajectories. UCIP encodes trajectories with a Quantum Boltzmann Machine (QBM), a classical algorithm based on the density-matrix formalism of quantum statistical mechanics, and measures the von Neumann entropy of the reduced density matrix induced by a bipartition of hidden units. We test whether agents with terminal continuation objectives (Type A) produce latent states with higher entanglement entropy than agents whose continuation is merely instrumental (Type B). Higher entanglement reflects stronger cross-partition statistical coupling. On gridworld agents with known ground-truth objectives, UCIP achieves 100% detection accuracy and 1.0 AUC-ROC on held-out non-adversarial evaluation under the frozen Phase I gate. The entanglement gap between Type A and Type B agents is Delta = 0.381 (p < 0.001, permutation test). Pearson r = 0.934 across an 11-point interpolation sweep indicates that, within this synthetic family, UCIP tracks graded changes in continuation weighting rather than merely a binary label. Among the tested models, only the QBM achieves positive Delta. All computations are classical; "quantum" refers only to the mathematical formalism. UCIP does not detect consciousness or subjective experience; it detects statistical structure in latent representations that correlates with known objectives.
【3】Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Models
标题：超越课堂子空间：单域模型中可靠的分布外检测的教师指导训练
链接：https://arxiv.org/abs/2603.11269
备注：14 pages main text, 22 pages appendix; under review at ECCV 2026
摘要：分布外（OOD）检测方法在多域基准测试中表现良好，但许多实际系统都是在单域数据上训练的。我们表明，这种制度会导致几何故障模式，域敏感性崩溃（DSC）：监督训练压缩功能到一个低秩类子空间，并抑制携带域移位信号的方向。我们提供的理论表明，在DSC下，基于距离和logit的OOD分数对域转移失去敏感性。然后，我们引入教师指导训练（TGT），它在训练过程中从冻结的多域教师（DINOv 2）中提取类抑制的剩余结构。教师和辅助头在训练后被丢弃，不增加推理开销。在八个单域基准测试中，TGT为基于距离的评分器产生了大幅度的远OOD FPR@95降低：MDS提高了11.61 pp，ViM提高了10.78 pp，kNN提高了12.87 pp（ResNet-50平均值），同时保持或略微提高了域内OOD和分类准确性。
摘要：Out-of-distribution (OOD) detection methods perform well on multi-domain benchmarks, yet many practical systems are trained on single-domain data. We show that this regime induces a geometric failure mode, Domain-Sensitivity Collapse (DSC): supervised training compresses features into a low-rank class subspace and suppresses directions that carry domain-shift signal. We provide theory showing that, under DSC, distance- and logit-based OOD scores lose sensitivity to domain shift. We then introduce Teacher-Guided Training (TGT), which distills class-suppressed residual structure from a frozen multi-domain teacher (DINOv2) into the student during training. The teacher and auxiliary head are discarded after training, adding no inference overhead. Across eight single-domain benchmarks, TGT yields large far-OOD FPR@95 reductions for distance-based scorers: MDS improves by 11.61 pp, ViM by 10.78 pp, and kNN by 12.87 pp (ResNet-50 average), while maintaining or slightly improving in-domain OOD and classification accuracy.
【4】Comparison of Outlier Detection Algorithms on String Data
标题：字符串数据离群点检测算法的比较
链接：https://arxiv.org/abs/2603.11049
备注：A bachelor's thesis comparing the local outlier factor algorithm against a new regular expression learner-based syntactical outlier detection algorithm for single-word string data
摘要：离群点检测是机器学习中的一个重要研究课题。然而，由于大多数文献都集中在数值型数据的离群点检测上，因此对字符串数据离群点检测的研究很少。一个健壮的字符串数据异常检测算法可以帮助系统日志文件中的数据清理或异常检测。在这篇论文中，我们比较了两种字符串离群点检测算法。首先，我们介绍了一个著名的本地离群值因子算法的变体，我们定制使用Levenshtein测量来计算数据集的密度来检测字符串数据上的离群值。我们提出了一个不同的加权Levenshtein措施，它认为分层字符类，可用于调整算法到一个特定的字符串数据集。其次，我们提出了一种新的基于层次左正则表达式学习器的离群点检测算法，该算法为期望数据推断出一个正则表达式。使用不同的数据集和参数，我们的实验表明，这两种算法可以在概念上发现字符串数据中的离群值。我们表明，如果期望值具有与异常值结构足够不同的独特结构，则基于正则表达式的算法特别擅长发现异常值。相比之下，局部离群值因子算法在发现离群值方面是最好的，如果它们到预期数据的编辑距离与预期数据之间的编辑距离足够不同。
摘要：Outlier detection is a well-researched and crucial problem in machine learning. However, there is little research on string data outlier detection, as most literature focuses on outlier detection of numerical data. A robust string data outlier detection algorithm could assist with data cleaning or anomaly detection in system log files. In this thesis, we compare two string outlier detection algorithms. Firstly, we introduce a variant of the well-known local outlier factor algorithm, which we tailor to detect outliers on string data using the Levenshtein measure to calculate the density of the dataset. We present a differently weighted Levenshtein measure, which considers hierarchical character classes and can be used to tune the algorithm to a specific string dataset. Secondly, we introduce a new kind of outlier detection algorithm based on the hierarchical left regular expression learner, which infers a regular expression for the expected data. Using various datasets and parameters, we experimentally show that both algorithms can conceptually find outliers in string data. We show that the regular expression-based algorithm is especially good at finding outliers if the expected values have a distinct structure that is sufficiently different from the structure of the outliers. In contrast, the local outlier factor algorithms are best at finding outliers if their edit distance to the expected data is sufficiently distinct from the edit distance between the expected data.
【5】Cough activity detection for automatic tuberculosis screening
标题：自动结核病筛查的咳嗽活动检测
链接：https://arxiv.org/abs/2603.11241
摘要 ：通过确定开始点和结束点来自动识别音频中的咳嗽片段对于在用于肺部相关疾病的健康技术中构建可扩展的筛查工具至关重要。我们提出了两个当前的预训练架构的应用程序的咳嗽活动检测的任务。采用了一个记录数据集，该数据集包含来自南非和乌干达社区一级护理中心的结核病（TB）患者的咳嗽症状。当使用XLS-R确定自动开始和结束点时，测试集的平均精度为0.96，接收器操作特性下的面积为0.99。我们表明，最好的平均精度是通过只利用前三层的网络，这具有减少计算和内存需求的双重好处，基于智能手机的应用程序的关键。该XLS-R配置被示出在测试集平均精度方面分别以9%和27%的绝对值优于音频频谱图Transformer（AST）以及逻辑回归基线。此外，使用由XLS-R自动隔离的咳嗽训练的下游TB分类模型轻松地优于在由AST隔离的咳嗽上训练的模型，并且仅勉强优于在地面真实咳嗽上训练的分类器。我们得出结论，应用大型预训练的Transformer模型是识别咳嗽终点的有效方法，并且将这种模型集成到筛选工具中是可行的。
摘要：The automatic identification of cough segments in audio through the determination of start and end points is pivotal to building scalable screening tools in health technologies for pulmonary related diseases. We propose the application of two current pre-trained architectures to the task of cough activity detection. A dataset of recordings containing cough from patients symptomatic for tuberculosis (TB) who self-present at community-level care centres in South Africa and Uganda is employed. When automatic start and end points are determined using XLS-R, an average precision of 0.96 and an area under the receiver-operating characteristic of 0.99 are achieved for the test set. We show that best average precision is achieved by utilising only the first three layers of the network, which has the dual benefits of reduced computational and memory requirements, pivotal for smartphone-based applications. This XLS-R configuration is shown to outperform an audio spectrogram transformer (AST) as well as a logistic regression baseline by 9% and 27% absolute in test set average precision respectively. Furthermore, a downstream TB classification model trained using the coughs automatically isolated by XLS-R comfortably outperforms a model trained on the coughs isolated by AST, and is only narrowly outperformed by a classifier trained on the ground truth coughs. We conclude that the application of large pre-trained transformer models is an effective approach to identifying cough end-points and that the integration of such a model into a screening tool is feasible.
分类|识别(4篇)
【1】Mitigating the Multiplicity Burden: The Role of Calibration in Reducing Predictive Multiplicity of Classifiers
标题：减轻多重性负担：校准在减少分类器预测多重性方面的作用
链接：https://arxiv.org/abs/2603.11750
备注：16 pages, 3 figures
摘要：随着机器学习模型越来越多地部署在高风险环境中，确保概率可靠性和预测稳定性变得至关重要。本文探讨了分类校准和预测多样性之间的相互作用-在罗生门集内的多个接近最优的模型产生冲突的信用结果为同一申请人的现象。使用九个不同的信用风险基准数据集，我们调查是否预测的多重性集中在低预测置信度的区域，以及事后校准如何减轻算法的任意性。我们的实证分析表明，少数类观测承担不成比例的多重性负担，证实了预测多重性和预测置信度的显着差异。此外，我们的经验比较表明，应用事后校准方法-特别是普拉特标度，等渗回归和温度标度-与罗生门集合的较低模糊度相关。在测试的技术，普拉特缩放和等渗回归提供了最强大的减少预测的多重性。这些研究结果表明，校准可以作为一个共识执行层，并可能通过减轻预测的多重性，支持程序的公平性。
摘要：As machine learning models are increasingly deployed in high-stakes environments, ensuring both probabilistic reliability and prediction stability has become critical. This paper examines the interplay between classification calibration and predictive multiplicity - the phenomenon in which multiple near-optimal models within the Rashomon set yield conflicting credit outcomes for the same applicant. Using nine diverse credit risk benchmark datasets, we investigate whether predictive multiplicity concentrates in regions of low predictive confidence and how post-hoc calibration can mitigate algorithmic arbitrariness. Our empirical analysis reveals that minority class observations bear a disproportionate multiplicity burden, as confirmed by significant disparities in predictive multiplicity and prediction confidence. Furthermore, our empirical comparisons indicate that applying post-hoc calibration methods - specifically Platt Scaling, Isotonic Regression, and Temperature Scaling - is associated with lower obscurity across the Rashomon set. Among the tested techniques, Platt Scaling and Isotonic Regression provide the most robust reduction in predictive multiplicity. These findings suggest that calibration can function as a consensus-enforcing layer and may support procedural fairness by mitigating predictive multiplicity.
【2】Survival Meets Classification: A Novel Framework for Early Risk Prediction Models of Chronic Diseases
标题：生存符合分类：慢性病早期风险预测模型的新框架
链接：https://arxiv.org/abs/2603.11598
摘要：慢性病是需要终身医疗的长期疾病。利用EMR大数据，我们开发了五种常见慢性病的早期疾病风险预测模型：糖尿病，高血压，CKD，COPD和慢性缺血性心脏病。在这项研究中，我们提出了一种新的方法，疾病风险模型的生存分析与分类技术相结合。传统的慢性病风险预测模型主要集中在生存分析或分类独立。在本文中，我们展示了生存分析方法可以重新设计，使它们能够有效地进行分类，从而使它们成为开发疾病风险监测模型的综合工具。我们对真实世界大EMR数据的实验结果表明，生存模型在准确性，F1评分和AUROC方面的性能与LightGBM和XGBoost等现有最先进的模型相当或更好。最后，所提出的生存模型使用一种新的方法来产生解释，这已经由一个由三名专家医生组成的小组进行了临床验证。
摘要：Chronic diseases are long-lasting conditions that require lifelong medical attention. Using big EMR data, we have developed early disease risk prediction models for five common chronic diseases: diabetes, hypertension, CKD, COPD, and chronic ischemic heart disease. In this study, we present a novel approach for disease risk models by integrating survival analysis with classification techniques. Traditional models for predicting the risk of chronic diseases predominantly focus on either survival analysis or classification independently. In this paper, we show survival analysis methods can be re-engineered to enable them to do classification efficiently and effectively, thereby making them a comprehensive tool for developing disease risk surveillance models. The results of our experiments on real-world big EMR data show that the performance of survival models in terms of accuracy, F1 score, and AUROC is comparable to or better than that of prior state-of-the-art models like LightGBM and XGBoost. Lastly, the proposed survival models use a novel methodology to generate explanations, which have been clinically validated by a panel of three expert physicians.
【3】Beyond Barren Plateaus: A Scalable Quantum Convolutional Architecture for High-Fidelity Image Classification
标题：Beyond Barren Plateaus：一种用于高保真图像分类的可扩展量子卷积架构
链接：https://arxiv.org/abs/2603.11131
摘要：虽然量子卷积神经网络（QCNN）为量子机器学习提供了一个理论范式，但与经典的对应方法相比，它们的实际实现受到贫瘠高原（梯度的指数消失）和经验准确性差的严重阻碍。在这项工作中，我们提出了一种新的QCNN架构，利用本地化的成本函数和硬件高效的张量网络初始化策略，以可证明地减轻贫瘠的高原。我们在MNIST数据集上评估了我们的可扩展QCNN，展示了显着的性能飞跃。通过解决梯度消失问题，我们优化的QCNN实现了98.7%的分类准确率，比未减轻模型中的基线QCNN准确率52.32%有了实质性的提高。此外，我们提供了参数效率优势的经验证据，需要$\mathcal{O}（\log N）$比等效的经典CNN更少的可训练参数才能实现$>95\%$收敛。这项工作弥合了理论量子效用和实际应用之间的差距，为量子计算机视觉任务提供了一个可扩展的框架，而不会屈服于损失景观集中。
摘要 ：While Quantum Convolutional Neural Networks (QCNNs) offer a theoretical paradigm for quantum machine learning, their practical implementation is severely bottlenecked by barren plateaus -- the exponential vanishing of gradients -- and poor empirical accuracy compared to classical counterparts. In this work, we propose a novel QCNN architecture utilizing localized cost functions and a hardware-efficient tensor-network initialization strategy to provably mitigate barren plateaus. We evaluate our scalable QCNN on the MNIST dataset, demonstrating a significant performance leap. By resolving the gradient vanishing issue, our optimized QCNN achieves a classification accuracy of 98.7\%, a substantial improvement over the baseline QCNN accuracy of 52.32\% found in unmitigated models. Furthermore, we provide empirical evidence of a parameter-efficiency advantage, requiring $\mathcal{O}(\log N)$ fewer trainable parameters than equivalent classical CNNs to achieve $>95\%$ convergence. This work bridges the gap between theoretical quantum utility and practical application, providing a scalable framework for quantum computer vision tasks without succumbing to loss landscape concentration.
【4】Group Resonance Network: Learnable Prototypes and Multi-Subject Resonance for EEG Emotion Recognition
标题：群共振网络：可学习原型和用于脑电情绪识别的多主体共振
链接：https://arxiv.org/abs/2603.11119
备注：12 pages, 5 figures
摘要：由于受试者间存在严重的变异性，基于脑电图（EEG）的诊断在跨受试者环境中仍然具有挑战性。现有的方法主要学习主体不变的特征，但往往未充分利用跨子组共享的刺激锁定组群。为了解决这个问题，我们提出了组共振网络（GRN），它集成了个人EEG动态离线组共振建模。GRN包含三个组件：用于逐频带EEG特征的个体编码器、用于原型诱导的共振的一组可学习的组原型、以及对具有小参考集的基于PLV/相干性的同步进行编码的多受试者共振分支。共振感知融合模块结合个体和组级表示进行最终分类。在受试者依赖和leave-one-subject-out协议下对SEED和DEAP的实验表明，GRN始终优于竞争基线，而消融研究证实了原型学习和多受试者共振建模的互补优势。
摘要：Electroencephalography(EEG)-basedemotionrecognitionre- mains challenging in cross-subject settings due to severe inter-subject variability. Existing methods mainly learn subject-invariant features, but often under-exploit stimulus-locked group regularities shared across sub- jects. To address this issue, we propose the Group Resonance Network (GRN), which integrates individual EEG dynamics with offline group resonance modeling. GRN contains three components: an individual en- coder for band-wise EEG features, a set of learnable group prototypes for prototype-induced resonance, and a multi-subject resonance branch that encodes PLV/coherence-based synchrony with a small reference set. A resonance-aware fusion module combines individual and group-level rep- resentations for final classification. Experiments on SEED and DEAP under both subject-dependent and leave-one-subject-out protocols show that GRN consistently outperforms competitive baselines, while abla- tion studies confirm the complementary benefits of prototype learning and multi-subject resonance modeling.
表征(3篇)
【1】Statistical and structural identifiability in representation learning
标题：表示学习中的统计和结构可识别性
链接：https://arxiv.org/abs/2603.11970
备注：International Conference on Learning Representations (ICLR) 2026
摘要：表征学习模型在其内部表征中表现出令人惊讶的稳定性。虽然大多数以前的工作将这种稳定性作为一个单一的属性，我们将其形式化为两个不同的概念：统计可识别性（跨运行表示的一致性）和结构可识别性（表示与一些未观察到的地面真理对齐）。认识到完美的逐点可识别性一般是不现实的现代表示学习模型，我们提出了新的模型不可知的定义的统计和结构的近可识别性的表示到一定的误差容限ε$。利用这些定义，我们证明了具有非线性解码器的模型表示的统计$ε$-近似可识别性结果，将现有的可识别性理论推广到最后一层表示之外，例如生成预训练Transformers（GPT），以接近广泛的模型类别的中间表示的可识别性，包括（掩蔽）自编码器（MAE）和监督学习器。虽然这些较弱的假设赋予较弱的可识别性，我们表明，独立成分分析（ICA）可以解决这类模型的剩余的线性模糊性，并验证和测量我们的近可识别性索赔经验。通过对数据生成过程的额外假设，统计可识别性扩展到结构可识别性，产生了一个简单而实用的解开纠缠的方法：潜在表示的ICA后处理。在合成基准测试中，这种方法使用vanilla自动编码器实现了最先进的解纠缠。借助用于细胞显微镜的基础模型规模MAE，它将生物变异与技术批次效应分离开来，大大提高了下游推广。
摘要：Representation learning models exhibit a surprising stability in their internal representations. Whereas most prior work treats this stability as a single property, we formalize it as two distinct concepts: statistical identifiability (consistency of representations across runs) and structural identifiability (alignment of representations with some unobserved ground truth). Recognizing that perfect pointwise identifiability is generally unrealistic for modern representation learning models, we propose new model-agnostic definitions of statistical and structural near-identifiability of representations up to some error tolerance $ε$. Leveraging these definitions, we prove a statistical $ε$-near-identifiability result for the representations of models with nonlinear decoders, generalizing existing identifiability theory beyond last-layer representations in e.g. generative pre-trained transformers (GPTs) to near-identifiability of the intermediate representations of a broad class of models including (masked) autoencoders (MAEs) and supervised learners. Although these weaker assumptions confer weaker identifiability, we show that independent components analysis (ICA) can resolve much of the remaining linear ambiguity for this class of models, and validate and measure our near-identifiability claims empirically. With additional assumptions on the data-generating process, statistical identifiability extends to structural identifiability, yielding a simple and practical recipe for disentanglement: ICA post-processing of latent representations. On synthetic benchmarks, this approach achieves state-of-the-art disentanglement using a vanilla autoencoder. With a foundation model-scale MAE for cell microscopy, it disentangles biological variation from technical batch effects, substantially improving downstream generalization.
【2】Causal Representation Learning with Optimal Compression under Complex Treatments
标题：复杂处理下具有最佳压缩的因果表示学习
链接：https://arxiv.org/abs/2603.11907
摘要：在多治疗方案中估计个体治疗效果（ITE）面临着两个关键挑战：平衡权重的超参数选择困境和计算可扩展性中的过度性诅咒。本文推导了一个新的多处理泛化界，并提出了一个理论估计的最佳平衡权重$α$，消除昂贵的启发式调整。我们研究了三种平衡策略：成对，一对所有（OVA）和治疗聚合。虽然OVA在低维设置中实现了卓越的精度，但我们提出的治疗聚合在治疗空间扩展时确保了准确性和O（1）可扩展性。此外，我们将我们的框架扩展到一个生成架构，多治疗CauseEGM，它保留了治疗流形的Wasserstein测地线结构。在半合成和图像数据集上的实验表明，该方法在估计精度和效率方面明显优于传统模型，特别是在大规模干预场景中。
摘要：Estimating Individual Treatment Effects (ITE) in multi-treatment scenarios faces two critical challenges: the Hyperparameter Selection Dilemma for balancing weights and the Curse of Dimensionality in computational scalability. This paper derives a novel multi-treatment generalization bound and proposes a theoretical estimator for the optimal balancing weight $α$, eliminating expensive heuristic tuning. We investigate three balancing strategies: Pairwise, One-vs-All (OVA), and Treatment Aggregation. While OVA achieves superior precision in low-dimensional settings, our proposed Treatment Aggregation ensures both accuracy and O(1) scalability as the treatment space expands. Furthermore, we extend our framework to a generative architecture, Multi-Treatment CausalEGM, which preserves the Wasserstein geodesic structure of the treatment manifold. Experiments on semi-synthetic and image datasets demonstrate that our approach significantly outperforms traditional models in estimation accuracy and efficiency, particularly in large-scale intervention scenarios.
【3】Representation Finetuning for Continual Learning
标题：持续学习的表现微调
链接：https://arxiv.org/abs/2603.11201
摘要：世界本质上是动态的，持续学习旨在使模型能够适应不断变化的数据流。虽然预先训练的模型在持续学习方面表现出了强大的性能，但它们仍然需要微调以有效地适应下游任务。然而，流行的参数有效的微调（PEFT）方法通过经验，黑箱优化的重量水平。这些方法缺乏对表征漂移的明确控制，导致对域转移的敏感性和持续学习场景中的灾难性遗忘。在这项工作中，我们引入了连续表示学习（Core），这是一个新的框架，首次将微调范式从权重空间转移到表示空间。与传统方法不同，Core在隐藏表示的低秩线性子空间内执行特定于任务的干预，采用具有明确目标的学习过程，确保过去任务的稳定性，同时保持新任务的可塑性。通过将更新约束到低秩子空间，Core实现了卓越的参数效率。在多个持续学习基准上的大量实验表明，Core不仅保持了参数效率，而且显著优于现有的最先进的方法。我们的工作介绍了表示微调作为一个新的，更有效的和可解释的范式，持续学习。
摘要：The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams. While pre-trained models have shown powerful performance in continual learning, they still require finetuning to adapt effectively to downstream tasks. However, prevailing Parameter-Efficient Fine-Tuning (PEFT) methods operate through empirical, black-box optimization at the weight level. These approaches lack explicit control over representation drift, leading to sensitivity to domain shifts and catastrophic forgetting in continual learning scenarios. In this work, we introduce Continual Representation Learning (CoRe), a novel framework that for the first time shifts the finetuning paradigm from weight space to representation space. Unlike conventional methods, CoRe performs task-specific interventions within a low-rank linear subspace of hidden representations, adopting a learning process with explicit objectives, which ensures stability for past tasks while maintaining plasticity for new ones. By constraining updates to a low-rank subspace, CoRe achieves exceptional parameter efficiency. Extensive experiments across multiple continual learning benchmarks demonstrate that CoRe not only preserves parameter efficiency but also significantly outperforms existing state-of-the-art methods. Our work introduces representation finetuning as a new, more effective and interpretable paradigm for continual learning.
编码器(1篇)
【1】Context-dependent manifold learning: A neuromodulated constrained autoencoder approach
标题：上下文相关的多管学习：一种神经调制的约束自动编码器方法
链接：https://arxiv.org/abs/2603.11673
备注：14 pages, 10 figures
摘要：约束自动编码器（cAE）通过在潜在空间上强制执行几何结构，提供了一条通往可解释的降维的成功路径。然而，标准的cAE不能适应不同的物理参数或环境条件，而不将这些背景变化与主要输入混为一谈。为了解决这个问题，我们将神经调节机制整合到cAE框架中，以允许上下文相关的流形学习。本文介绍了神经调制约束自动编码器（NcAE），它自适应参数化的几何约束，通过增益和偏置调整条件下的静态上下文信息。动力系统上的实验结果表明，NcAE准确地捕捉如何流形几何形状在不同的制度，同时保持严格的投影属性。这些结果表明，神经调节有效地从局部流形表示的全局上下文参数。这种架构提供了一个基础，开发更灵活的，物理信息表示系统（非静态）的环境约束。
摘要：Constrained autoencoders (cAE) provide a successful path towards interpretable dimensionality reduction by enforcing geometric structure on latent spaces. However, standard cAEs cannot adapt to varying physical parameters or environmental conditions without conflating these contextual shifts with the primary input. To address this, we integrated a neuromodulatory mechanism into the cAE framework to allow for context-dependent manifold learning. This paper introduces the Neuromodulated Constrained Autoencoder (NcAE), which adaptively parameterizes geometric constraints via gain and bias tuning conditioned on static contextual information. Experimental results on dynamical systems show that the NcAE accurately captures how manifold geometry varies across different regimes while maintaining rigorous projection properties. These results demonstrate that neuromodulation effectively decouples global contextual parameters from local manifold representations. This architecture provides a foundation for developing more flexible, physics-informed representations in systems subject to (non-stationary) environmental constraints.
优化|敛散性(13篇)
【1】Cross-Domain Policy Optimization via Bellman Consistency and Hybrid Critics
标题：基于Bellman一致性和混合批评的跨域策略优化
链接：https://arxiv.org/abs/2603.12087
备注：Accepted at ICLR 2026
摘要：跨域强化学习（CDRL）是指通过利用从源域收集的数据样本来促进相似目标域中的学习，从而提高RL的数据效率。尽管有其潜力，但RL中的跨域转移已知具有两个基本且相互交织的挑战：（i）源域和目标域可以具有不同的状态空间或动作空间，这使得直接转移不可行，从而需要更复杂的域间映射;（ii）RL中源域模型的可转移性不容易先验识别，因此CDRL在转移过程中可能容易受到负面影响。在本文中，我们建议通过\textit{cross-domain Bellman consistency}和\textit{hybrid critic}来共同应对这两个挑战。具体来说，我们首先介绍跨域贝尔曼一致性的概念，作为一种方法来衡量一个源域模型的可转移性。然后，我们提出了$Q$Avatar，它将源域和目标域的Q函数与自适应超参数自由权函数相结合。通过这种设计，我们的收敛行为的特点，$Q$阿凡达，并表明，$Q$阿凡达实现可靠的转移，在这个意义上说，它有效地利用了源域Q功能的知识转移到目标域。通过实验，我们证明了$Q$Avatar在各种RL基准任务中实现了良好的可移植性，包括运动和机器人手臂操作。我们的代码可在https://rl-bandits-lab.github.io/Cross-Domain-RL/上获得。
摘要：Cross-domain reinforcement learning (CDRL) is meant to improve the data efficiency of RL by leveraging the data samples collected from a source domain to facilitate the learning in a similar target domain. Despite its potential, cross-domain transfer in RL is known to have two fundamental and intertwined challenges: (i) The source and target domains can have distinct state space or action space, and this makes direct transfer infeasible and thereby requires more sophisticated inter-domain mappings; (ii) The transferability of a source-domain model in RL is not easily identifiable a priori, and hence CDRL can be prone to negative effect during transfer. In this paper, we propose to jointly tackle these two challenges through the lens of \textit{cross-domain Bellman consistency} and \textit{hybrid critic}. Specifically, we first introduce the notion of cross-domain Bellman consistency as a way to measure transferability of a source-domain model. Then, we propose $Q$Avatar, which combines the Q functions from both the source and target domains with an adaptive hyperparameter-free weight function. Through this design, we characterize the convergence behavior of $Q$Avatar and show that $Q$Avatar achieves reliable transfer in the sense that it effectively leverages a source-domain Q function for knowledge transfer to the target domain. Through experiments, we demonstrate that $Q$Avatar achieves favorable transferability across various RL benchmark tasks, including locomotion and robot arm manipulation. Our code is available at https://rl-bandits-lab.github.io/Cross-Domain-RL/.
【2】Efficient Generative Modeling with Unitary Matrix Product States Using Riemannian Optimization
标题：利用Riemann优化进行具有单位矩阵积状态的高效生成建模
链接：https://arxiv.org/abs/2603.12026
摘要：张量网络最初是为了表征复杂的量子多体系统而开发的，最近已经成为一个强大的框架，用于捕获具有强物理可解释性的高维概率分布。本文系统地研究了生成式建模的矩阵乘积状态（MPS），并表明酉MPS是一种简单而富有表现力的张量网络架构，通过减少参数更新中的模糊性和提高效率，为无监督学习提供了明显的好处。为了克服标准的基于梯度的MPS训练的效率低下，我们开发了一种黎曼优化方法，将概率建模作为具有流形约束的优化问题，并进一步推导出一种有效的空间解耦算法。在Bars-and-Stripes和EMNIST数据集上的实验表明，该算法能够快速适应数据结构，更新稳定，性能强大，同时保持了MPS的效率和表达能力。
摘要 ：Tensor networks, which are originally developed for characterizing complex quantum many-body systems, have recently emerged as a powerful framework for capturing high-dimensional probability distributions with strong physical interpretability. This paper systematically studies matrix product states (MPS) for generative modeling and shows that unitary MPS, which is a tensor-network architecture that is both simple and expressive, offers clear benefits for unsupervised learning by reducing ambiguity in parameter updates and improving efficiency. To overcome the inefficiency of standard gradient-based MPS training, we develop a Riemannian optimization approach that casts probabilistic modeling as an optimization problem with manifold constraints, and further derive an efficient space-decoupling algorithm. Experiments on Bars-and-Stripes and EMNIST datasets demonstrate fast adaptation to data structure, stable updates, and strong performance while maintaining the efficiency and expressive power of MPS.
【3】Inverse Neural Operator for ODE Parameter Optimization
标题：用于ODE参数优化的逆神经运算器
链接：https://arxiv.org/abs/2603.11854
备注：17 pages, 6 figures
摘要：我们提出了逆神经算子（INO），这是一个两阶段的框架，用于从稀疏的部分观测中恢复隐藏的ODE参数。在第一阶段，具有交叉注意力的条件傅立叶神经运算器（C-FNO）学习可微代理，该代理从任意稀疏输入重建完整的ODE轨迹，通过谱正则化抑制高频伪影。在第2阶段，摊销漂移模型（ADM）学习参数空间中的核加权速度场，将随机参数初始化传输到地面真实值，而不通过代理反向传播，避免雅可比不稳定性，这种不稳定性会影响刚性区域中基于梯度的反演。在真实世界的刚性大气化学基准（POLLU，25个参数）和合成基因调控网络（GRN，40个参数）上的实验表明，INO在参数恢复精度方面优于基于梯度和摊销基线，同时仅需要0.23秒的推理时间，比迭代梯度下降加速487倍。
摘要：We propose the Inverse Neural Operator (INO), a two-stage framework for recovering hidden ODE parameters from sparse, partial observations. In Stage 1, a Conditional Fourier Neural Operator (C-FNO) with cross-attention learns a differentiable surrogate that reconstructs full ODE trajectories from arbitrary sparse inputs, suppressing high-frequency artifacts via spectral regularization. In Stage 2, an Amortized Drifting Model (ADM) learns a kernel-weighted velocity field in parameter space, transporting random parameter initializations toward the ground truth without backpropagating through the surrogate, avoiding the Jacobian instabilities that afflict gradient-based inversion in stiff regimes. Experiments on a real-world stiff atmospheric chemistry benchmark (POLLU, 25 parameters) and a synthetic Gene Regulatory Network (GRN, 40 parameters) show that INO outperforms gradient-based and amortized baselines in parameter recovery accuracy while requiring only 0.23s inference time, a 487x speedup over iterative gradient descent.
【4】Shape-of-You: Fused Gromov-Wasserstein Optimal Transport for Semantic Correspondence in-the-Wild
标题：你的形状：融合的Groov-Wasserstein在野外语义对应的最佳传输
链接：https://arxiv.org/abs/2603.11618
备注：Accepted at CVPR 2026. Supplementary material included after references. 18 pages, 11 figures, 10 tables
摘要：语义对应对于处理缺乏明确对应注释的各种野外图像至关重要。虽然最近的2D基础模型提供了强大的功能，但通过最近邻伪标签使其适应无监督学习具有关键限制：它在本地操作，忽略了结构关系，因此它对2D外观的依赖无法解决由对称性或重复特征引起的几何模糊性。在这项工作中，我们解决这个问题，通过重新制定伪标签生成融合Gromov-Wasserstein（FGW）的问题，共同优化特征间的相似性和结构内的一致性。我们的框架Shape-of-You（SoY）利用3D基础模型来定义几何空间中的内部结构，解决了上述模糊性。然而，由于FGW是一个计算禁止的二次问题，我们近似通过锚定线性化。由此产生的概率运输计划提供了一个结构上一致的，但嘈杂的监督信号。因此，我们引入了一个软目标损失动态混合的指导，从这个计划与网络预测，建立一个学习框架，这种噪声鲁棒。SoY在SPair-71 k和AP-10 k数据集上实现了最先进的性能，在没有显式几何注释的情况下建立了语义对应的新基准。代码可在Shape-of-You获得。
摘要：Semantic correspondence is essential for handling diverse in-the-wild images lacking explicit correspondence annotations. While recent 2D foundation models offer powerful features, adapting them for unsupervised learning via nearest-neighbor pseudo-labels has key limitations: it operates locally, ignoring structural relationships, and consequently its reliance on 2D appearance fails to resolve geometric ambiguities arising from symmetries or repetitive features. In this work, we address this by reformulating pseudo-label generation as a Fused Gromov-Wasserstein (FGW) problem, which jointly optimizes inter-feature similarity and intra-structural consistency. Our framework, Shape-of-You (SoY), leverages a 3D foundation model to define this intra-structure in the geometric space, resolving abovementioned ambiguity. However, since FGW is a computationally prohibitive quadratic problem, we approximate it through anchor-based linearization. The resulting probabilistic transport plan provides a structurally consistent but noisy supervisory signal. Thus, we introduce a soft-target loss dynamically blending guidance from this plan with network predictions to build a learning framework robust to this noise. SoY achieves state-of-the-art performance on SPair-71k and AP-10k datasets, establishing a new benchmark in semantic correspondence without explicit geometric annotations. Code is available at Shape-of-You.
【5】Fractional Rotation, Full Potential? Investigating Performance and Convergence of Partial RoPE
标题：部分旋转，全潜力？调查部分RoPE的性能和收敛性
链接：https://arxiv.org/abs/2603.11611
摘要：旋转位置嵌入（RoPE）是Transformer架构中用于编码相对位置信息的常见选择。尽管早期的工作已经研究了在特定层中省略RoPE，但改变接收旋转变换的隐藏维度的比例的影响在很大程度上仍未得到探索。这种设计选择可以节省大量的内存，这在长上下文长度时尤为重要。我们发现，与标准RoPE缓存相比，内存节省高达10倍，同时实现了相当的最终损失。在这项工作中，我们提出了一个系统的研究，检查部分RoPE对训练动态和跨架构和数据集的收敛的影响。我们的发现揭示了几个值得注意的模式：（1）仅将RoPE应用于一小部分维度（2）这些趋势在不同质量和架构的模型大小、序列长度和数据集上保持一致，更高质量的数据导致更低的总体损失和类似的基准性能;以及（3）使用NoPE（无位置编码）训练的一些模型显示出不稳定的学习轨迹，这可以通过最小化RoPE应用或QK-Norm来缓解，其收敛到更高的损失。总之，这些结果为旨在平衡效率和训练稳定性的模型设计者提供了实际指导，同时强调了以前被忽视的部分RoPE的重要性。
摘要：Rotary Positional Embedding (RoPE) is a common choice in transformer architectures for encoding relative positional information. Although earlier work has examined omitting RoPE in specific layers, the effect of varying the fraction of hidden dimensions that receive rotary transformations remains largely unexplored. This design choice can yield substantial memory savings, which becomes especially significant at long context lengths. We find up to 10x memory savings over the standard RoPE cache, while achieving comparable final loss. In this work, we present a systematic study examining the impact of partial RoPE on training dynamics and convergence across architectures and datasets. Our findings uncover several notable patterns: (1) applying RoPE to only a small fraction of dimensions (around 10%) achieves convergence comparable to using full RoPE; (2) these trends hold consistently across model size, sequence lengths and datasets of varying quality and architectures, with higher-quality data resulting in lower overall loss and similar benchmark performance; and (3) some models trained with NoPE (No Positional Encoding) showcase unstable learning trajectories, which can be alleviated through minimal RoPE application or QK-Norm which converges to a higher loss. Together, these results offer practical guidance for model designers aiming to balance efficiency and training stability, while emphasizing the previously overlooked importance of partial RoPE.
【6】AutoScout: Structured Optimization for Automating ML System Configuration
标题：AutoScout：自动化ML系统配置的结构化优化
链接：https://arxiv.org/abs/2603.11603
摘要：机器学习（ML）系统暴露了一个快速扩展的配置空间，包括模型并行策略、通信优化和低级运行时参数。端到端系统效率对这些选择高度敏感，但由于异构特征类型（例如，稀疏和密集参数），条件依赖性（例如，仅在特定上游决策下有效的执行参数），以及高搜索（剖析）成本。现有的方法要么优化配置维度的一个狭窄的子集，要么依赖于无法随着配置空间的持续增长而泛化的ad-hoc算法。我们介绍了AutoScout，一个用于ML训练、微调和推理的通用系统配置器。它制定了系统配置作为一个混合的离散/连续优化问题的层次依赖关系，并引入了一个混合优化框架，共同细化稀疏的结构决策和密集的执行参数。为了降低分析成本，AutoScout自适应地优先考虑影响较大的配置功能，并以不同的保真度集成模拟器。在不同的型号、硬件平台和部署目标中，AutoScout一致地识别高性能配置，在专家调整的设置上实现2.7- 3.0\times $的训练加速。
摘要：Machine learning (ML) systems expose a rapidly expanding configuration space spanning model-parallelism strategies, communication optimizations, and low-level runtime parameters. End-to-end system efficiency is highly sensitive to these choices, yet identifying high-performance configurations is challenging due to heterogeneous feature types (e.g., sparse and dense parameters), conditional dependencies (e.g., valid execution parameters only under specific upstream decisions), and the high search (profiling) cost. Existing approaches either optimize a narrow subset of configuration dimensions or rely on ad-hoc heuristics that fail to generalize as configuration spaces continue to grow. We present AutoScout, a general-purpose systems configurator for ML training, fine-tuning, and inference. It formulates the system configuration as a mixed-discrete/continuous optimization problem with hierarchical dependencies and introduces a hybrid optimization framework that jointly refines sparse structural decisions and dense execution parameters. To reduce profiling cost, AutoScout adaptively prioritizes high-impact configuration features and ensembles simulators with varying fidelity. Across diverse models, hardware platforms, and deployment objectives, AutoScout consistently identifies high-performance configurations, achieving 2.7-3.0$\times$ training speedup over expert-tuned settings.
【7】Hybrid Energy-Aware Reward Shaping: A Unified Lightweight Physics-Guided Methodology for Policy Optimization
标题：混合能源感知奖励塑造：一种统一的轻量级物理引导的政策优化方法
链接：https://arxiv.org/abs/2603.11600
备注：17 pages, 27 figures
摘要：深度强化学习在连续控制方面表现出色，但通常需要广泛的探索，而基于物理的模型需要完整的方程，并具有立方复杂性。这项研究提出了混合能量感知奖励成形（H-EARS），将基于潜力的奖励成形与能量感知动作正则化相统一。H-EARS约束动作幅度，同时通过功能分解平衡特定任务和基于能量的潜力，通过捕获主导能量分量而无需完全动态来实现线性复杂度O（n）。我们建立了一个理论基础，包括：（1）功能独立的任务/能源优化;（2）基于能源的收敛加速;（3）收敛保证下的函数逼近;和（4）近似的潜在误差界。李雅普诺夫稳定性连接分析作为启发式的指导。跨基线的实验显示出更好的收敛性、稳定性和能效。车辆模拟验证了极端条件下安全关键领域的适用性。结果证实，集成轻量级物理先验增强了没有完整系统模型的无模型强化学习，从而实现了从实验室研究到工业应用的转移。
摘要：Deep reinforcement learning excels in continuous control but often requires extensive exploration, while physics-based models demand complete equations and suffer cubic complexity. This study proposes Hybrid Energy-Aware Reward Shaping (H-EARS), unifying potential-based reward shaping with energy-aware action regularization. H-EARS constrains action magnitude while balancing task-specific and energy-based potentials via functional decomposition, achieving linear complexity O(n) by capturing dominant energy components without full dynamics. We establish a theoretical foundation including: (1) functional independence for separate task/energy optimization; (2) energy-based convergence acceleration; (3) convergence guarantees under function approximation; and (4) approximate potential error bounds. Lyapunov stability connections are analyzed as heuristic guides. Experiments across baselines show improved convergence, stability, and energy efficiency. Vehicle simulations validate applicability in safety-critical domains under extreme conditions. Results confirm that integrating lightweight physics priors enhances model-free RL without complete system models, enabling transfer from lab research to industrial applications.
【8】abx_amr_simulator: A simulation environment for antibiotic prescribing policy optimization under antimicrobial resistance
标题：abx_amr_simulator：抗菌药物耐药性下抗生素处方政策优化的模拟环境
链接：https://arxiv.org/abs/2603.11369
备注：10 pages, 3 figures
摘要：抗生素耐药性（AMR）对全球健康构成威胁，降低了抗生素的有效性，并使临床决策复杂化。为了应对这一挑战，我们引入了abx_amr_simulator，这是一个基于Python的模拟包，旨在在受控的强化学习（RL）兼容环境中对抗生素处方和AMR动态进行建模。模拟器允许用户指定患者人群、药物特异性AMR反应曲线和奖励函数，以平衡即时临床获益和长期耐药管理。主要功能包括用于配置患者属性的模块化设计，通过漏气球抽象建模的抗生素耐药性动态，以及通过噪声，偏差和观察延迟探索部分可观察性的工具。该软件包与Gymnasium RL API兼容，使用户能够在不同的临床场景下训练和测试RL代理。从ML的角度来看，该软件包为不确定性下的顺序决策提供了一个可配置的基准环境，包括由噪声，偏差和延迟观测引起的部分可观测性。通过提供可定制和可扩展的框架，abx_amr_simulator为研究AMR动态和在现实不确定性下优化抗生素管理策略提供了一个有价值的工具。
摘要：Antimicrobial resistance (AMR) poses a global health threat, reducing the effectiveness of antibiotics and complicating clinical decision-making. To address this challenge, we introduce abx_amr_simulator, a Python-based simulation package designed to model antibiotic prescribing and AMR dynamics within a controlled, reinforcement learning (RL)-compatible environment. The simulator allows users to specify patient populations, antibiotic-specific AMR response curves, and reward functions that balance immedi- ate clinical benefit against long-term resistance management. Key features include a modular design for configuring patient attributes, antibiotic resistance dynamics modeled via a leaky-balloon abstraction, and tools to explore partial observability through noise, bias, and delay in observations. The package is compatible with the Gymnasium RL API, enabling users to train and test RL agents under diverse clinical scenarios. From an ML perspective, the package provides a configurable benchmark environment for sequential decision-making under uncertainty, including partial observability induced by noisy, biased, and delayed observations. By providing a customizable and extensible framework, abx_amr_simulator offers a valuable tool for studying AMR dynamics and optimizing antibiotic stewardship strategies under realistic uncertainty.
【9】Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
标题：后见之明锚定策略优化：在稀疏奖励设置中将失败转化为反馈
链接：https://arxiv.org/abs/2603.11321
摘要：带有可验证奖励的强化学习（RLVR）已经成为后训练推理模型的一个很有前途的范例。然而，基于组的方法，如组相对策略优化（GRPO）在稀疏奖励设置中面临着一个关键的困境：纯强化学习（RL）遭受优势崩溃和高方差梯度估计，而混合策略优化引入了持久的分布偏差。为了解决这个难题，我们引入了后见之明锚定策略优化（HAPO）。HAPO采用了合成成功注入（SSI）操作符，这是一种事后机制，在失败期间有选择地将优化锚定到教师演示中。这种注入是由汤普森采样启发门控机制，创建一个自主的，自定进度的课程。从理论上讲，我们证明了HAPO实现\textit{渐近一致性}：通过自然退火教师信号的政策改善，HAPO恢复无偏的政策梯度。这确保了政策外指导作为一个临时的脚手架，而不是一个持久的天花板，使该模型能够超越静态教师强迫的局限性。
摘要 ：Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for post-training reasoning models. However, group-based methods such as Group Relative Policy Optimization (GRPO) face a critical dilemma in sparse-reward settings: pure Reinforcement Learning (RL) suffers from advantage collapse and high-variance gradient estimation, while mixed-policy optimization introduces persistent distributional bias. To resolve this dilemma, we introduce Hindsight-Anchored Policy Optimization (HAPO). HAPO employs the Synthetic Success Injection (SSI) operator, a hindsight mechanism that selectively anchors optimization to teacher demonstrations during failure. This injection is governed by a Thompson sampling-inspired gating mechanism, creating an autonomous, self-paced curriculum. Theoretically, we demonstrate that HAPO achieves \textit{asymptotic consistency}: by naturally annealing the teacher signal as the policy improves, HAPO recovers the unbiased on-policy gradient. This ensures off-policy guidance acts as a temporary scaffold rather than a persistent ceiling, enabling the model to surpass the limitations of static teacher forcing.
【10】Bayesian Optimization of Partially Known Systems using Hybrid Models
标题：使用混合模型的部分已知系统的Bayesian优化
链接：https://arxiv.org/abs/2603.11199
备注：16 pages, 5 Figures
摘要：贝叶斯优化（BO）作为一种用于评估昂贵系统的黑箱优化的有效算法而受到关注，其中BO算法迭代地查询系统并基于拟合到先前样本的概率模型建议新的试验。尽管如此，标准的BO循环可能需要大量的实验来收敛到最优值，特别是对于高维和非线性系统。我们提出了一种基于混合模型的BO配方，结合了BO的迭代贝叶斯学习与部分已知的机械物理模型。我们不是学习从输入到目标的直接映射，而是为基于物理的模型编写所有已知的方程，并使用概率模型（在我们的情况下，高斯过程（GP））推断缺少方程的变量的表达式。然后，最终公式包括GP作为混合模型中的约束，从而允许其他基于物理的非线性和隐式模型约束。这种混合模型配方产生一个约束，非线性随机程序，我们离散使用样本平均近似。在单级蒸馏的计算机优化中，基于质量守恒定律的混合BO模型产生比标准BO回路更好的设计。此外，根据初始样本，混合模型在少至一次迭代中收敛，而标准BO对于任何种子都不会在25次迭代内收敛。总体而言，所提出的混合BO方案为部分已知系统提供了一种有前途的优化方法，利用了机械建模和数据驱动优化的优势。
摘要：Bayesian optimization (BO) has gained attention as an efficient algorithm for black-box optimization of expensive-to-evaluate systems, where the BO algorithm iteratively queries the system and suggests new trials based on a probabilistic model fitted to previous samples. Still, the standard BO loop may require a prohibitively large number of experiments to converge to the optimum, especially for high-dimensional and nonlinear systems. We present a hybrid model-based BO formulation that combines the iterative Bayesian learning of BO with partially known mechanistic physical models. Instead of learning a direct mapping from inputs to the objective, we write all known equations for a physics-based model and infer expressions for variables missing equations using a probabilistic model, in our case, a Gaussian process (GP). The final formulation then includes the GP as a constraint in the hybrid model, thereby allowing other physics-based nonlinear and implicit model constraints. This hybrid model formulation yields a constrained, nonlinear stochastic program, which we discretize using the sample-average approximation. In an in-silico optimization of a single-stage distillation, the hybrid BO model based on mass conservation laws yields significantly better designs than a standard BO loop. Furthermore, the hybrid model converges in as few as one iteration, depending on the initial samples, whereas, the standard BO does not converge within 25 for any of the seeds. Overall, the proposed hybrid BO scheme presents a promising optimization method for partially known systems, leveraging the strengths of both mechanistic modeling and data-driven optimization.
【11】Speculative Decoding Scaling Laws (SDSL): Throughput Optimization Made Simple
标题：推测解码缩放定律（SDSL）：吞吐量优化变得简单
链接：https://arxiv.org/abs/2603.11053
摘要：推测解码是一种使用多种语言模型来加速推理的技术。以前的作品已经使用了一种实验性的方法来优化推理管道的吞吐量，这涉及LLM训练并且可能是昂贵的。推测解码的这项研究提出了一种理论，该理论将预训练LLM的关键超参数与下游基于SD的推理系统的吞吐量效率分析性地连接起来。该理论允许在预训练之前预测推理系统的组件的吞吐量最优超参数。
摘要：Speculative decoding is a technique that uses multiple language models to accelerate infer- ence. Previous works have used an experi- mental approach to optimize the throughput of the inference pipeline, which involves LLM training and can be costly. This study of spec- ulative decoding proposes a theory that ana- lytically connects the key hyperparameters of pre-trained LLMs to the throughput efficiency of a downstream SD-based inference system. The theory allows the prediction of throughput- optimal hyperparameters for the components of an inference system before their pre-training.
【12】Wasserstein Gradient Flows for Batch Bayesian Optimal Experimental Design
标题：批量Bayesian最优实验设计的Wasserstein梯度流
链接：https://arxiv.org/abs/2603.12102
摘要：贝叶斯最优实验设计（BOED）为选择实验提供了一个强大的决策理论框架，以最大限度地提高所收集数据的预期效用。然而，在实践中，它的适用性可能受到优化所选效用的困难的限制。例如，期望信息增益（EIG）通常是高维的，并且是强非凸的。这一挑战在批量设置中尤其严重，其中要同时设计多个实验。在本文中，我们介绍了一种新的方法，通过概率提升的原始优化问题的概率措施的空间，以批量EIG为基础的BOED。特别是，我们建议优化熵正则化的预期效用的空间设计措施。在温和的条件下，我们表明，这一目标承认一个独特的极小，它可以明确的吉布斯分布的形式。由此产生的设计法可以直接作为一个随机的批量设计政策，或作为一个计算松弛，从中提取一个确定性的批次。为了获得可扩展的近似时，批量大小是大的，然后我们考虑两个易于处理的限制的完整的批量分布：一个平均场家庭，和i.i.d.产品家族。为了身份证。的目标，并正式为它的平均场扩展，我们得到相应的Wasserstein梯度流，恢复其长时间的行为，并获得基于粒子的算法，通过时空离散。我们还介绍了双随机变量，结合相互作用的粒子更新与蒙特卡洛估计的EIG梯度。最后，我们在几个数值实验中说明了所提出的方法的性能，证明了它们能够探索多模态优化景观，并在具有挑战性的示例中获得高效用批次。
摘要：Bayesian optimal experimental design (BOED) provides a powerful, decision-theoretic framework for selecting experiments so as to maximise the expected utility of the data to be collected. In practice, however, its applicability can be limited by the difficulty of optimising the chosen utility. The expected information gain (EIG), for example, is often high-dimensional and strongly non-convex. This challenge is particularly acute in the batch setting, where multiple experiments are to be designed simultaneously. In this paper, we introduce a new approach to batch EIG-based BOED via a probabilistic lifting of the original optimisation problem to the space of probability measures. In particular, we propose to optimise an entropic regularisation of the expected utility over the space of design measures. Under mild conditions, we show that this objective admits a unique minimiser, which can be explicitly characterised in the form of a Gibbs distribution. The resulting design law can be used directly as a randomised batch-design policy, or as a computational relaxation from which a deterministic batch is extracted. To obtain scalable approximations when the batch size is large, we then consider two tractable restrictions of the full batch distribution: a mean-field family, and an i.i.d. product family. For the i.i.d. objective, and formally for its mean-field extension, we derive the corresponding Wasserstein gradient flow, characterise its long-time behaviour, and obtain particle-based algorithms via space-time discretisations. We also introduce doubly stochastic variants that combine interacting particle updates with Monte Carlo estimators of the EIG gradient. Finally, we illustrate the performance of the proposed methods in several numerical experiments, demonstrating their ability to explore multimodal optimisation landscapes and obtain high-utility batches in challenging examples.
【13】Uncovering Locally Low-dimensional Structure in Networks by Locally Optimal Spectral Embedding
标题：利用局部最优谱嵌入揭示网络的局部低维结构
链接：https://arxiv.org/abs/2603.11965
摘要：标准邻接谱嵌入（ASE）依赖于全局低秩假设，通常与现实世界网络的稀疏，传递结构不兼容，导致局部几何特征被“涂抹”。为了解决这个问题，我们引入了局部邻接谱嵌入（LASE），它通过加权谱分解来揭示局部低维结构。在具有核特征映射的潜在位置模型下，我们将潜在位置的图像视为无限维特征空间中的局部低维集合。我们建立有限样本的界限量化本地化的统计成本和减少截断误差之间的权衡，通过针对嵌入的局部低维区域。此外，我们证明了足够的本地化诱导快速的频谱衰减和出现一个独特的频谱间隙，理论上证明了低维本地嵌入。合成和真实网络上的实验表明，LASE提高了局部重建和可视化在全球和子图基线，我们引入UMAP-LASE组装重叠的本地嵌入到高保真的全球可视化。
摘要：Standard Adjacency Spectral Embedding (ASE) relies on a global low-rank assumption often incompatible with the sparse, transitive structure of real-world networks, causing local geometric features to be 'smeared'. To address this, we introduce Local Adjacency Spectral Embedding (LASE), which uncovers locally low-dimensional structure via weighted spectral decomposition. Under a latent position model with a kernel feature map, we treat the image of latent positions as a locally low-dimensional set in infinite-dimensional feature space. We establish finite-sample bounds quantifying the trade-off between the statistical cost of localisation and the reduced truncation error achieved by targeting a locally low-dimensional region of the embedding. Furthermore, we prove that sufficient localisation induces rapid spectral decay and the emergence of a distinct spectral gap, theoretically justifying low-dimensional local embeddings. Experiments on synthetic and real networks show that LASE improves local reconstruction and visualisation over global and subgraph baselines, and we introduce UMAP-LASE for assembling overlapping local embeddings into high-fidelity global visualisations.
预测|估计(9篇)
【1】Separable neural architectures as a primitive for unified predictive and generative intelligence
标题：可分离神经架构是统一预测和生成智能的基础
链接：https://arxiv.org/abs/2603.12244
摘要：跨越物理、语言和感知的智能系统通常表现出可因子分解的结构，但通常是由没有明确利用这种结构的单片神经架构建模的。可分离神经体系结构（SNA）通过形式化一个代表性的类来解决这个问题，该类统一了加性、二次和张量分解的神经模型。通过约束相互作用顺序和张量秩，SNA施加了一个结构归纳偏差，将高维映射分解为低元分量。可分性不一定是系统本身的属性：它经常出现在坐标或表示中，通过这些坐标或表示来表达系统。至关重要的是，这种坐标感知的制定揭示了混沌时空动力学和语言自回归之间的结构相似性。通过将连续的物理状态视为平滑的、可分离的嵌入，SNA使混沌系统的分布式建模成为可能。这种方法减轻了确定性算子的非物理漂移特性，同时仍然适用于离散序列。这种方法的组成多功能性在四个领域得到了证明：通过强化学习的自主航点导航，多功能微结构的逆生成，湍流的分布建模和神经语言建模。这些结果建立了可分离的神经架构作为预测和生成智能的域不可知的原始，能够统一确定性和分布表示。
摘要：Intelligent systems across physics, language and perception often exhibit factorisable structure, yet are typically modelled by monolithic neural architectures that do not explicitly exploit this structure. The separable neural architecture (SNA) addresses this by formalising a representational class that unifies additive, quadratic and tensor-decomposed neural models. By constraining interaction order and tensor rank, SNAs impose a structural inductive bias that factorises high-dimensional mappings into low-arity components. Separability need not be a property of the system itself: it often emerges in the coordinates or representations through which the system is expressed. Crucially, this coordinate-aware formulation reveals a structural analogy between chaotic spatiotemporal dynamics and linguistic autoregression. By treating continuous physical states as smooth, separable embeddings, SNAs enable distributional modelling of chaotic systems. This approach mitigates the nonphysical drift characteristics of deterministic operators whilst remaining applicable to discrete sequences. The compositional versatility of this approach is demonstrated across four domains: autonomous waypoint navigation via reinforcement learning, inverse generation of multifunctional microstructures, distributional modelling of turbulent flow and neural language modelling. These results establish the separable neural architecture as a domain-agnostic primitive for predictive and generative intelligence, capable of unifying both deterministic and distributional representations.
【2】Cross-Resolution Attention Network for High-Resolution PM2.5 Prediction
标题：用于高分辨率PM2.5预测的跨分辨率注意力网络
链接：https://arxiv.org/abs/2603.11725
摘要：Vision Transformers在时空预测方面取得了显著的成功，但其可扩展性对于现实世界环境监测中所需的超高分辨率，大陆尺度域仍然有限。一张1公里分辨率的欧洲空气质量地图包含2900万像素，远远超出了天真的自我关注的极限。我们引入了CRAN-PM，这是一种双分支Vision Transformer，它利用交叉分辨率注意力，有效地融合全球气象数据（25 km）与当前时间（1 km）的本地高分辨率PM2.5。我们没有将温度和地形等物理驱动因素作为输入，而是进一步引入了高度感知的自我注意力和风引导的交叉注意力，以迫使网络学习物理上一致的特征表示来进行PM2.5预测。CRAN-PM是完全可训练和内存效率，生成完整的2900万像素的欧洲地图在1.8秒内在一个单一的GPU。在2022年对整个欧洲的每日PM2.5预测（362天，2，971个欧洲环境署（EEA）站点）进行评估，与最佳单尺度基线相比，它在T+1和T+3分别将RMSE降低了4.7%和10.7%，同时将复杂地形的偏差降低了36%。
摘要：Vision Transformers have achieved remarkable success in spatio-temporal prediction, but their scalability remains limited for ultra-high-resolution, continent-scale domains required in real-world environmental monitoring. A single European air-quality map at 1 km resolution comprises 29 million pixels, far beyond the limits of naive self-attention. We introduce CRAN-PM, a dual-branch Vision Transformer that leverages cross-resolution attention to efficiently fuse global meteorological data (25 km) with local high-resolution PM2.5 at the current time (1 km). Instead of including physically driven factors like temperature and topography as input, we further introduce elevation-aware self-attention and wind-guided cross-attention to force the network to learn physically consistent feature representations for PM2.5 forecasting. CRAN-PM is fully trainable and memory-efficient, generating the complete 29-million-pixel European map in 1.8 seconds on a single GPU. Evaluated on daily PM2.5 forecasting throughout Europe in 2022 (362 days, 2,971 European Environment Agency (EEA) stations), it reduces RMSE by 4.7% at T+1 and 10.7% at T+3 compared to the best single-scale baseline, while reducing bias in complex terrain by 36%.
【3】CAETC: Causal Autoencoding and Treatment Conditioning for Counterfactual Estimation over Time
标题：CATEC：随着时间的推移反事实估计的因果自动编码和治疗条件反射
链接：https://arxiv.org/abs/2603.11565
摘要 ：随着时间的推移，反事实估计在各种应用中是重要的，例如个性化医疗。然而，观察数据中的时间依赖性混杂偏倚仍然对实现准确和有效的估计提出了重大挑战。我们介绍了因果自动编码和治疗条件反射（CAETC），这个问题的一种新方法。我们的方法建立在对抗性表示学习的基础上，利用自动编码架构来学习部分可逆和治疗不变的表示，其中结果预测任务被转换为对表示应用特定于治疗的条件反射。我们的设计是独立的底层序列模型，可以应用到现有的架构，如长短期记忆（LSTM）或时间卷积网络（TCN）。我们对合成，半合成和真实世界的数据进行了广泛的实验，以证明CAETC比现有方法在反事实估计方面有显着的改进。
摘要：Counterfactual estimation over time is important in various applications, such as personalized medicine. However, time-dependent confounding bias in observational data still poses a significant challenge in achieving accurate and efficient estimation. We introduce causal autoencoding and treatment conditioning (CAETC), a novel method for this problem. Built on adversarial representation learning, our method leverages an autoencoding architecture to learn a partially invertible and treatment-invariant representation, where the outcome prediction task is cast as applying a treatment-specific conditioning on the representation. Our design is independent of the underlying sequence model and can be applied to existing architectures such as long short-term memories (LSTMs) or temporal convolution networks (TCNs). We conduct extensive experiments on synthetic, semi-synthetic, and real-world data to demonstrate that CAETC yields significant improvement in counterfactual estimation over existing methods.
【4】Gen-Fab: A Variation-Aware Generative Model for Predicting Fabrication Variations in Nanophotonic Devices
标题：Gen-Fab：用于预测纳米光电子器件制造变化的变化感知生成模型
链接：https://arxiv.org/abs/2603.11505
备注：Accepted and published in Structural and Multidisciplinary Optimization (2026)
摘要：Silicon photonic devices often exhibit fabrication-induced variations such as over-etching, underetching, and corner rounding, which can significantly alter device performance. These variations are non-uniform and are influenced by feature size and shape. Accurate digital twins are therefore needed to predict the range of possible fabricated outcomes for a given design. In this paper, we introduce Gen-Fab, a conditional generative adversarial network (cGAN) based on Pix2Pix to predict and model uncertainty in photonic fabrication outcomes. The proposed method takes a design layout (in GDS format) as input and produces diverse high-resolution predictions similar to scanning electron microscope (SEM) images of fabricated devices, capturing the range of process variations at the nanometer scale. To enable one-to-many mapping, we inject a latent noise vector at the model bottleneck. We compare Gen-Fab against three baselines: (1) a deterministic U-Net predictor, (2) an inference-time Monte Carlo Dropout U-Net, and (3) an ensemble of varied U-Nets. Evaluations on an out-of-distribution dataset of fabricated photonic test structures demonstrate that Gen-Fab outperforms all baselines in both accuracy and uncertainty modeling. An additional distribution shift analysis further confirms its strong generalization to unseen fabrication geometries. Gen-Fab achieves the highest intersection-over-union (IoU) score of 89.8%, outperforming the deterministic U-Net (85.3%), the MC-Dropout U-Net (83.4%), and varying U-Nets (85.8%). It also better aligns with the distribution of real fabrication outcomes, achieving lower Kullback-Leibler divergence and Wasserstein distance.
【5】Monitoring and Prediction of Mood in Elderly People during Daily Life Activities
标题：老年人日常生活活动情绪监测与预测
链接：https://arxiv.org/abs/2603.11230
备注：This is the authors' manuscript. The final published article is available at https://doi.org/10.1109/EMBC.2019.8857847
摘要：We present an intelligent wearable system to monitor and predict mood states of elderly people during their daily life activities. Our system is composed of a wristband to record different physiological activities together with a mobile app for ecological momentary assessment (EMA). Machine learning is used to train a classifier to automatically predict different mood states based on the smart band only. Our approach shows promising results on mood accuracy and provides results comparable with the state of the art in the specific detection of happiness and activeness.
【6】High-resolution weather-guided surrogate modeling for data-efficient cross-location building energy prediction
标题：高分辨率天气引导代理建模，用于数据高效的跨地点建筑能源预测
链接：https://arxiv.org/abs/2603.11121
摘要：Building design optimization often depends on physics-based simulation tools such as EnergyPlus, which, although accurate, are computationally expensive and slow. Surrogate models provide a faster alternative, yet most are location-specific, and even weather-informed variants require simulations from many sites to generalize to unseen locations. This limitation arises because existing methods do not fully exploit the short-term weather-driven energy patterns shared across regions, restricting their scalability and reusability. This study introduces a high-resolution (weekly) weather-informed surrogate modeling approach that enhances model reusability across locations. By capturing recurring short-term weather-energy demand patterns common to multiple regions, the proposed method produces a generalized surrogate that performs well beyond the training location. Unlike previous weather-informed approaches, it does not require extensive simulations from multiple sites to achieve strong generalization. Experimental results show that when trained on a single location, the model maintains high predictive accuracy for other sites within the same climate zone, with no noticeable performance loss, and exhibits only minimal degradation when applied across different climate zones. These findings demonstrate the potential of climate-informed generalization for developing scalable and reusable surrogate models, supporting more sustainable and optimized building design practices.
【7】Simultaneous estimation of multiple discrete unimodal distributions under stochastic order constraints
标题：随机序约束下多个离散双峰分布的同时估计
链接：https://arxiv.org/abs/2603.11532
摘要 ：We study the problem of estimating multiple discrete unimodal distributions, motivated by search behavior analysis on a real-world platform. To incorporate prior knowledge of precedence relations among distributions, we impose stochastic order constraints and formulate the estimation task as a mixed-integer convex quadratic optimization problem. Experiments on both synthetic and real datasets show that the proposed method reduces the Jensen-Shannon divergence by 2.2% on average (up to 6.3%) when the sample size is small, while performing comparably to existing methods when sufficient data are available.
【8】Trustworthy predictive distributions for rare events via diagnostic transport maps
标题：通过诊断运输地图为罕见事件提供值得信赖的预测分布
链接：https://arxiv.org/abs/2603.11229
备注：19 pages, 5 figures, 2 tables
摘要：Forecast systems in science and technology are increasingly moving beyond point prediction toward methods that produce full predictive distributions of future outcomes y, conditional on high-dimensional and complex sequences of inputs x. However, even when forecast systems provide a full predictive distribution, the result is rarely calibrated with respect to all x and y. The estimated density can be especially unreliable in low-frequency or out-of-distribution regimes, where accurate uncertainty quantification and a means for human experts to verify results are most needed to establish trust in models. In this paper, we take an initial predictive distribution as given and treat it as a useful but potentially misspecified base model. WE then introduce diagnostic transport maps, covariate-dependent probability-to-probability maps that quantify how the base model's probabilities should be adjusted to better match the true conditional distribution of calibration data. At deployment, these maps provide the user with real-time local diagnostics that reveal where the model fails and how it fails (including bias, dispersion, skewness, and tail errors), while also producing a recalibrated predictive distribution through a simple composition with the base model. We apply diagnostic transport maps to short-term tropical cyclone intensity forecasting and show that an easy-to-fit parametric version identifies evolutionary modes associated with local miscalibration and improves the predictive performance for rare events, including 24-hour rapid intensity change, as compared to the operational forecasts of the National Hurricane Center.
【9】Conformal e-prediction in the presence of confounding
标题：存在混杂情况下的保形电子预测
链接：https://arxiv.org/abs/2603.11134
备注：8 pages, 2 figures
摘要：This note extends conformal e-prediction to cover the case where there is observed confounding between the random object $X$ and its label $Y$. We consider both the case where the observed data is IID and a case where some dependence between observations is permitted.
其他神经网络|深度学习|模型|建模(21篇)
【1】Cornserve: A Distributed Serving System for Any-to-Any Multimodal Models
标题：Cornserve：适用于任意多模式模型的分布式服务系统
链接：https://arxiv.org/abs/2603.12118
备注：Open source https://github.com/cornserve-ai/cornserve / Demo video https://www.youtube.com/watch?v=nb8R-vztLRg
摘要：Any-to-Any models are an emerging class of multimodal models that accept combinations of multimodal data (e.g., text, image, video, audio) as input and generate them as output. Serving these models are challenging; different requests with different input and output modalities traverse different paths through the model computation graph, and each component of the model have different scaling characteristics. We present Cornserve, a distributed serving system for generic Any-to-Any models. Cornserve provides a flexible task abstraction for expressing Any-to-Any model computation graphs, enabling component disaggregation and independent scaling. The distributed runtime dispatches compute to the data plane via an efficient record-and-replay execution model that keeps track of data dependencies, and forwards tensor data between components directly from the producer to the consumer. Built on Kubernetes with approximately 23K new lines of Python, Cornserve supports diverse Any-to-Any models and delivers up to 3.81$\times$ higher throughput and 5.79$\times$ lower tail latency. Cornserve is open-source, and the demo video is available on YouTube.
【2】Chemical Reaction Networks Learn Better than Spiking Neural Networks
标题：化学反应网络比尖峰神经网络学习得更好
链接：https://arxiv.org/abs/2603.12060
备注：Keywords: Chemical Reaction Networks, Spiking Neural Networks, Supervised Learning, Classification, Mass-Action Kinetics, Statistical Learning Theory, Regret Bounds, Model Complexity
摘要：We mathematically prove that chemical reaction networks without hidden layers can solve tasks for which spiking neural networks require hidden layers. Our proof uses the deterministic mass-action kinetics formulation of chemical reaction networks. Specifically, we prove that a certain reaction network without hidden layers can learn a classification task previously proved to be achievable by a spiking neural network with hidden layers. We provide analytical regret bounds for the global behavior of the network and analyze its asymptotic behavior and Vapnik-Chervonenkis dimension. In a numerical experiment, we confirm the learning capacity of the proposed chemical reaction network for classifying handwritten digits in pixel images, and we show that it solves the task more accurately and efficiently than a spiking neural network with hidden layers. This provides a motivation for machine learning in chemical computers and a mathematical explanation for how biological cells might exhibit more efficient learning behavior within biochemical reaction networks than neuronal networks.
【3】Flowcean - Model Learning for Cyber-Physical Systems
标题：Flowcean -网络物理系统的模型学习
链接：https://arxiv.org/abs/2603.12015
摘要：Effective models of Cyber-Physical Systems (CPS) are crucial for their design and operation. Constructing such models is difficult and time-consuming due to the inherent complexity of CPS. As a result, data-driven model generation using machine learning methods is gaining popularity. In this paper, we present Flowcean, a novel framework designed to automate the generation of models through data-driven learning that focuses on modularity and usability. By offering various learning strategies, data processing methods, and evaluation metrics, our framework provides a comprehensive solution, tailored to CPS scenarios. Flowcean facilitates the integration of diverse learning libraries and tools within a modular and flexible architecture, ensuring adaptability to a wide range of modeling tasks. This streamlines the process of model generation and evaluation, making it more efficient and accessible.
【4】Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control
标题：单细胞基础模型的详尽电路映射揭示了大规模冗余、重尾枢纽架构和层相关差异控制
链接：https://arxiv.org/abs/2603.11940
摘要：Mechanistic interpretability of biological foundation models has relied on selective feature sampling, pairwise interaction testing, and observational trajectory analysis. Each of these can introduce systematic bias. Here we present three experiments that address these limitations through exhaustive circuit tracing, higher order combinatorial ablation, and causal trajectory steering in Geneformer, a transformer based single cell foundation model. First, exhaustive tracing of all 4065 active sparse autoencoder features at layer 5 yields 1393850 significant downstream edges, a 27 fold expansion over selective sampling. This reveals a heavy tailed hub distribution in which 1.8 percent of features account for disproportionate connectivity and 40 percent of the top 20 hubs lack biological annotation. These results indicate systematic annotation bias in prior selective analyses. Second, three way combinatorial ablation across 8 feature triplets shows that redundancy deepens monotonically with interaction order, with a three way ratio of 0.59 versus a pairwise ratio of 0.74, and with zero synergy. This confirms that the model architecture is subadditive at all tested orders. Third, trajectory guided feature steering establishes a causal link between layer position and differentiation directionality. Late layer features at L17 consistently push cell states toward maturity, with fraction positive equal to 1.0. Early and mid layer features at L0 and L11 mostly push away from maturity, with fraction positive ranging from 0.00 to 0.58. Together these results move from correlation toward causal evidence for layer dependent control of cell state.
【5】Modeling Trial-and-Error Navigation With a Sequential Decision Model of Information Scent
标题：用信息气味顺序决策模型建模试错导航
链接：https://arxiv.org/abs/2603.11759
摘要：Users often struggle to locate an item within an information architecture, particularly when links are ambiguous or deeply nested in hierarchies. Information scent has been used to explain why users select incorrect links, but this concept assumes that users see all available links before deciding. In practice, users frequently select a link too quickly, overlook relevant cues, and then rely on backtracking when errors occur. We extend the concept of information scent by framing navigation as a sequential decision-making problem under memory constraints. Specifically, we assume that users do not scan entire pages but instead inspect strategically, looking "just enough" to find the target given their time budget. To choose which item to inspect next, they consider both local (this page) and global (site) scent; however, both are constrained by memory. Trying to avoid wasting time, they occasionally choose the wrong links without inspecting everything on a page. Comparisons with empirical data show that our model replicates key navigation behaviors: premature selections, wrong turns, and recovery from backtracking. We conclude that trial-and-error behavior is well explained by information scent when accounting for the sequential and bounded characteristics of the navigation problem.
【6】Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach
标题：在社会强盗学习中利用非专家和多元化代理人的专业知识：自由能源方法
链接：https://arxiv.org/abs/2603.11757
摘要：Personalized AI-based services involve a population of individual reinforcement learning agents. However, most reinforcement learning algorithms focus on harnessing individual learning and fail to leverage the social learning capabilities commonly exhibited by humans and animals. Social learning integrates individual experience with observing others' behavior, presenting opportunities for improved learning outcomes. In this study, we focus on a social bandit learning scenario where a social agent observes other agents' actions without knowledge of their rewards. The agents independently pursue their own policy without explicit motivation to teach each other. We propose a free energy-based social bandit learning algorithm over the policy space, where the social agent evaluates others' expertise levels without resorting to any oracle or social norms. Accordingly, the social agent integrates its direct experiences in the environment and others' estimated policies. The theoretical convergence of our algorithm to the optimal policy is proven. Empirical evaluations validate the superiority of our social learning method over alternative approaches in various scenarios. Our algorithm strategically identifies the relevant agents, even in the presence of random or suboptimal agents, and skillfully exploits their behavioral information. In addition to societies including expert agents, in the presence of relevant but non-expert agents, our algorithm significantly enhances individual learning performance, where most related methods fail. Importantly, it also maintains logarithmic regret.
【7】Multi-Task Anti-Causal Learning for Reconstructing Urban Events from Residents' Reports
标题：根据居民报告重建城市事件的多任务反因果学习
链接：https://arxiv.org/abs/2603.11546
摘要：Many real-world machine learning tasks are anti-causal: they require inferring latent causes from observed effects. In practice, we often face multiple related tasks where part of the forward causal mechanism is invariant across tasks, while other components are task-specific. We propose Multi-Task Anti-Causal learning (MTAC), a framework for estimating causes from outcomes and confounders by explicitly exploiting such cross-task invariances. MTAC first performs causal discovery to learn a shared causal graph and then instantiates a structured multi-task structural equation model (SEM) that factorizes the outcome-generation process into (i) a task-invariant mechanism and (ii) task-specific mechanisms via a shared backbone with task-specific heads. Building on the learned forward model, MTAC performs maximum A posteriori (MAP)based inference to reconstruct causes by jointly optimizing latent mechanism variables and cause magnitudes under the learned causal structure. We evaluate MTAC on the application of urban event reconstruction from resident reports, spanning three tasks:parking violations, abandoned properties, and unsanitary conditions. On real-world data collected from Manhattan and the city of Newark, MTAC consistently improves reconstruction accuracy over strong baselines, achieving up to 34.61\% MAE reduction and demonstrating the benefit of learning transferable causal mechanisms across tasks.
【8】Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensors
标题：更多松弛，更好地预测：基于概率潜在变量模型的软传感器的近端松弛
链接：https://arxiv.org/abs/2603.11473
备注：This paper has been provisionally accepted for publication in the "IEEE Transactions on Industrial Informatics"
摘要：Nonlinear Probabilistic Latent Variable Models (NPLVMs) are a cornerstone of soft sensor modeling due to their capacity for uncertainty delineation. However, conventional NPLVMs are trained using amortized variational inference, where neural networks parameterize the variational posterior. While facilitating model implementation, this parameterization converts the distributional optimization problem within an infinite-dimensional function space to parameter optimization within a finite-dimensional parameter space, which introduces an approximation error gap, thereby degrading soft sensor modeling accuracy. To alleviate this issue, we introduce KProxNPLVM, a novel NPLVM that pivots to relaxing the objective itself and improving the NPLVM's performance. Specifically, we first prove the approximation error induced by the conventional approach. Based on this, we design the Wasserstein distance as the proximal operator to relax the learning objective, yielding a new variational inference strategy derived from solving this relaxed optimization problem. Based on this foundation, we provide a rigorous derivation of KProxNPLVM's optimization implementation, prove the convergence of our algorithm can finally sidestep the approximation error, and propose the KProxNPLVM by summarizing the abovementioned content. Finally, extensive experiments on synthetic and real-world industrial datasets are conducted to demonstrate the efficacy of the proposed KProxNPLVM.
【9】Harnessing Data Asymmetry: Manifold Learning in the Finsler World
标题：利用数据不对称：芬斯勒世界中的多元学习
链接：https://arxiv.org/abs/2603.11396
摘要：Manifold learning is a fundamental task at the core of data analysis and visualisation. It aims to capture the simple underlying structure of complex high-dimensional data by preserving pairwise dissimilarities in low-dimensional embeddings. Traditional methods rely on symmetric Riemannian geometry, thus forcing symmetric dissimilarities and embedding spaces, e.g. Euclidean. However, this discards in practice valuable asymmetric information inherent to the non-uniformity of data samples. We suggest to harness this asymmetry by switching to Finsler geometry, an asymmetric generalisation of Riemannian geometry, and propose a Finsler manifold learning pipeline that constructs asymmetric dissimilarities and embeds in a Finsler space. This greatly broadens the applicability of existing asymmetric embedders beyond traditionally directed data to any data. We also modernise asymmetric embedders by generalising current reference methods to asymmetry, like Finsler t-SNE and Finsler Umap. On controlled synthetic and large real datasets, we show that our asymmetric pipeline reveals valuable information lost in the traditional pipeline, e.g. density hierarchies, and consistently provides superior quality embeddings than their Euclidean counterparts.
【10】ARROW: Augmented Replay for RObust World models
标题：箭头：RObust World模型的增强重播
链接：https://arxiv.org/abs/2603.11395
备注：27 pages and 8 figures (includes Appendix)
摘要：Continual reinforcement learning challenges agents to acquire new skills while retaining previously learned ones with the goal of improving performance in both past and future tasks. Most existing approaches rely on model-free methods with replay buffers to mitigate catastrophic forgetting; however, these solutions often face significant scalability challenges due to large memory demands. Drawing inspiration from neuroscience, where the brain replays experiences to a predictive World Model rather than directly to the policy, we present ARROW (Augmented Replay for RObust World models), a model-based continual RL algorithm that extends DreamerV3 with a memory-efficient, distribution-matching replay buffer. Unlike standard fixed-size FIFO buffers, ARROW maintains two complementary buffers: a short-term buffer for recent experiences and a long-term buffer that preserves task diversity through intelligent sampling. We evaluate ARROW on two challenging continual RL settings: Tasks without shared structure (Atari), and tasks with shared structure, where knowledge transfer is possible (Procgen CoinRun variants). Compared to model-free and model-based baselines with replay buffers of the same-size, ARROW demonstrates substantially less forgetting on tasks without shared structure, while maintaining comparable forward transfer. Our findings highlight the potential of model-based RL and bio-inspired approaches for continual reinforcement learning, warranting further research.
【11】Teleodynamic Learning a new Paradigm For Interpretable AI
标题：远程动态学习可解释人工智能的新范式
链接：https://arxiv.org/abs/2603.11355
摘要：We introduce Teleodynamic Learning, a new paradigm for machine learning in which learning is not the minimization of a fixed objective, but the emergence and stabilization of functional organization under constraint. Inspired by living systems, this framework treats intelligence as the coupled evolution of three quantities: what a system can represent, how it adapts its parameters, and which changes its internal resources can sustain. We formalize learning as a constrained dynamical process with two interacting timescales: inner dynamics for continuous parameter adaptation and outer dynamics for discrete structural change, linked by an endogenous resource variable that both shapes and is shaped by the trajectory. This perspective reveals three phenomena that standard optimization does not naturally capture: self-stabilization without externally imposed stopping rules, phase-structured learning dynamics that move from under-structuring through teleodynamic growth to over-structuring, and convergence guarantees grounded in information geometry rather than convexity. We instantiate the framework in the Distinction Engine (DE11), a teleodynamic learner grounded in Spencer-Brown's Laws of Form, information geometry, and tropical optimization. On standard benchmarks, DE11 achieves 93.3 percent test accuracy on IRIS, 92.6 percent on WINE, and 94.7 percent on Breast Cancer, while producing interpretable logical rules that arise endogenously from the learning dynamics rather than being imposed by hand. More broadly, Teleodynamic Learning unifies regularization, architecture search, and resource-bounded inference within a single principle: learning as the co-evolution of structure, parameters, and resources under constraint. This opens a thermodynamically grounded route to adaptive, interpretable, and self-organizing AI.
【12】A Machine Learning-Enhanced Hopf-Cole Formulation for Nonlinear Gas Flow in Porous Media
标题：多孔介质中非线性气体流动的机器学习增强Hopf-Cole公式
链接：https://arxiv.org/abs/2603.11250
摘要：Accurate modeling of gas flow through porous media is critical for many technological applications, including reservoir performance prediction, carbon capture and sequestration, and fuel cells and batteries. However, such modeling remains challenging due to strong nonlinear behavior and uncertainty in model parameters. In particular, gas slippage effects described by the Klinkenberg model introduce pressure-dependent permeability, which complicates numerical simulation and obscures deviations from classical Darcy flow behavior. To address these challenges, we present an integrated modeling framework for gas transport in porous media that combines a Klinkenberg-enhanced constitutive relation, Hopf-Cole-transformed mixed-form linear governing equations, a shared-trunk neural network architecture, and a Deep Least-Squares (DeepLS) solver. The Hopf-Cole transformation reformulates the original nonlinear flow equations into an equivalent linear system closely related to the Darcy model, while the mixed formulation, together with a shared-trunk neural architecture, enables simultaneous and accurate prediction of both pressure and velocity fields. A rigorous convergence analysis is performed both theoretically and numerically, establishing the stability and convergence properties of the proposed solver. Importantly, the proposed framework also naturally facilitates inverse modeling of pressure-dependent permeability and slippage parameters from limited or indirect observations, enabling efficient estimation of flow properties that are difficult to measure experimentally. Numerical results demonstrate accurate recovery of flow dynamics and parameters across a wide range of pressure regimes, highlighting the framework's robustness, accuracy, and computational efficiency for gas transport modeling and inversion in tight formations.
【13】Differentiable Thermodynamic Phase-Equilibria for Machine Learning
标题：机器学习的可微热力学相平衡
链接：https://arxiv.org/abs/2603.11249
摘要：Accurate prediction of phase equilibria remains a central challenge in chemical engineering. Physics-consistent machine learning methods that incorporate thermodynamic structure into neural networks have recently shown strong performance for activity-coefficient modeling. However, extending such approaches to equilibrium data arising from an extremum principle, such as liquid-liquid equilibria, remains difficult. Here we present DISCOMAX, a differentiable algorithm for phase-equilibrium calculation that guarantees thermodynamic consistency at both training and inference, only subject to a user-specified discretization. The method is rooted in statistical thermodynamics, and works via a discrete enumeration with subsequent masked softmax aggregation of feasible states, and together with a straight-through gradient estimator to enable physics-consistent end-to-end learning of neural $g^{E}$-models. We evaluate the approach on binary liquid-liquid equilibrium data and demonstrate that it outperforms existing surrogate-based methods, while offering a general framework for learning from different kinds of equilibrium data.
【14】A Learning-Based Superposition Operator for Non-Renewal Arrival Processes in Queueing Networks
标题：排队网络中非更新到达过程的一种基于学习的叠加运算符
链接：https://arxiv.org/abs/2603.11118
摘要：The superposition of arrival processes is a fundamental yet analytically intractable operation in queueing networks when inputs are general non-renewal streams. Classical methods either reduce merged flows to renewal surrogates, rely on computationally prohibitive Markovian representations, or focus solely on mean-value performance measures. We propose a scalable data-driven superposition operator that maps low-order moments and autocorrelation descriptors of multiple arrival streams to those of their merged process. The operator is a deep learning model trained on synthetically generated Markovian Arrival Processes (MAPs), for which exact superposition is available, and learns a compact representation that accurately reconstructs the first five moments and short-range dependence structure of the aggregate stream. Extensive computational experiments demonstrate uniformly low prediction errors across heterogeneous variability and correlation regimes, substantially outperforming classical renewal-based approximations. When integrated with learning-based modules for departure-process and steady-state analysis, the proposed operator enables decomposition-based evaluation of feed-forward queueing networks with merging flows. The framework provides a scalable alternative to traditional analytical approaches while preserving higher-order variability and dependence information required for accurate distributional performance analysis.
【15】Learning Tree-Based Models with Gradient Descent
标题：使用梯度下降学习基于树的模型
链接：https://arxiv.org/abs/2603.11117
备注：PhD thesis
摘要：Tree-based models are widely recognized for their interpretability and have proven effective in various application domains, particularly in high-stakes domains. However, learning decision trees (DTs) poses a significant challenge due to their combinatorial complexity and discrete, non-differentiable nature. As a result, traditional methods such as CART, which rely on greedy search procedures, remain the most widely used approaches. These methods make locally optimal decisions at each node, constraining the search space and often leading to suboptimal tree structures. Additionally, their demand for custom training methods precludes a seamless integration into modern machine learning (ML) approaches. In this thesis, we propose a novel method for learning hard, axis-aligned DTs through gradient descent. Our approach utilizes backpropagation with a straight-through operator on a dense DT representation, enabling the joint optimization of all tree parameters, thereby addressing the two primary limitations of traditional DT algorithms. First, gradient-based training is not constrained by the sequential selection of locally optimal splits but, instead, jointly optimizes all tree parameters. Second, by leveraging gradient descent for optimization, our approach seamlessly integrates into existing ML approaches e.g., for multimodal and reinforcement learning tasks, which inherently rely on gradient descent. These advancements allow us to achieve state-of-the-art results across multiple domains, including interpretable DTs rees for small tabular datasets, advanced models for complex tabular data, multimodal learning, and interpretable reinforcement learning without information loss. By bridging the gap between DTs and gradient-based optimization, our method significantly enhances the performance and applicability of tree-based models across various ML domains.
【16】Interventional Time Series Priors for Causal Foundation Models
标题：因果基础模型的干预时间序列先验
链接：https://arxiv.org/abs/2603.11090
备注：1st ICLR 2026 Workshop on Time Series in the Age of Large Models (TSALM)
摘要：Prior-data fitted networks (PFNs) have emerged as powerful foundation models for tabular causal inference, yet their extension to time series remains limited by the absence of synthetic data generators that provide interventional targets. Existing time series benchmarks generate observational data with ground-truth causal graphs but lack the interventional data required for training causal foundation models. To address this, we propose \textbf{CausalTimePrior}, a principled framework for generating synthetic temporal structural causal models (TSCMs) with paired observational and interventional time series. Our prior supports configurable causal graph structures, nonlinear autoregressive mechanisms, regime-switching dynamics, and multiple intervention types (hard, soft, time-varying). We demonstrate that PFNs trained on CausalTimePrior can perform in-context causal effect estimation on held-out TSCMs, establishing a pathway toward foundation models for time series causal inference.
【17】Hypercomplex Widely Linear Processing: Fundamentals for Quaternion Machine Learning
标题：超复杂广泛线性处理：四元数机器学习基础
链接：https://arxiv.org/abs/2603.11835
备注：Contributed chapter to appear in Handbook of Statistics Volume 54: Multidimensional Signal Processing, Elsevier, 2026
摘要：Numerous attempts have been made to replicate the success of complex-valued algebra in engineering and science to other hypercomplex domains such as quaternions, tessarines, biquaternions, and octonions. Perhaps, none have matched the success of quaternions. The most useful feature of quaternions lies in their ability to model three-dimensional rotations which, in turn, have found various industrial applications such as in aeronautics and computergraphics. Recently, we have witnessed a renaissance of quaternions due to the rise of machine learning. To equip the reader to contribute to this emerging research area, this chapter lays down the foundation for: - augmented statistics for modelling quaternion-valued random processes, - widely linear models to exploit such advanced statistics, - quaternion calculus and algebra for algorithmic derivations, - mean square estimation for practical considerations. For ease of exposure, several examples are offered to facilitate the learning, understanding, and(hopefully) the adoption of this multidimensional domain.
【18】Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study
标题：基于词典的动态方程学习中的病态性：一个系统生物学案例研究
链接：https://arxiv.org/abs/2603.11330
摘要：Data-driven discovery of governing equations from time-series data provides a powerful framework for understanding complex biological systems. Library-based approaches that use sparse regression over candidate functions have shown considerable promise, but they face a critical challenge when candidate functions become strongly correlated: numerical ill-conditioning. Poor or restricted sampling, together with particular choices of candidate libraries, can produce strong multicollinearity and numerical instability. In such cases, measurement noise may lead to widely different recovered models, obscuring the true underlying dynamics and hindering accurate system identification. Although sparse regularization promotes parsimonious solutions and can partially mitigate conditioning issues, strong correlations may persist, regularization may bias the recovered models, and the regression problem may remain highly sensitive to small perturbations in the data. We present a systematic analysis of how ill-conditioning affects sparse identification of biological dynamics using benchmark models from systems biology. We show that combinations involving as few as two or three terms can already exhibit strong multicollinearity and extremely large condition numbers. We further show that orthogonal polynomial bases do not consistently resolve ill-conditioning and can perform worse than monomial libraries when the data distribution deviates from the weight function associated with the orthogonal basis. Finally, we demonstrate that when data are sampled from distributions aligned with the appropriate weight functions corresponding to the orthogonal basis, numerical conditioning improves, and orthogonal polynomial bases can yield improved model recovery accuracy across two baseline models.
【19】A Standardized Framework For Evaluating Gene Expression Generative Models
标题：评估基因表达生成模型的标准化框架
链接：https://arxiv.org/abs/2603.11244
摘要：The rapid development of generative models for single-cell gene expression data has created an urgent need for standardised evaluation frameworks. Current evaluation practices suffer from inconsistent metric implementations, incomparable hyperparameter choices, and a lack of biologically-grounded metrics. We present Generated Genetic Expression Evaluator (GGE), an open-source Python framework that addresses these challenges by providing a comprehensive suite of distributional metrics with explicit computation space options and biologically-motivated evaluation through differentially expressed gene (DEG)-focused analysis and perturbation-effect correlation, enabling standardized reporting and reproducible benchmarking. Through extensive analysis of the single-cell generative modeling literature, we identify that no standardized evaluation protocol exists. Methods report incomparable metrics computed in different spaces with different hyperparameters. We demonstrate that metric values vary substantially depending on implementation choices, highlighting the critical need for standardization. GGE enables fair comparison across generative approaches and accelerates progress in perturbation response prediction, cellular identity modeling, and counterfactual inference.
【20】Deep regression learning from dependent observations with minimum error entropy principle
标题：采用最小误差信息原则从相依观测进行深度回归学习
链接：https://arxiv.org/abs/2603.11138
摘要：This paper considers nonparametric regression from strongly mixing observations. The proposed approach is based on deep neural networks with minimum error entropy (MEE) principle. We study two estimators: the non-penalized deep neural network (NPDNN) and the sparse-penalized deep neural network (SPDNN) predictors. Upper bounds of the expected excess risk are established for both estimators over the classes of Hölder and composition Hölder functions. For the models with Gaussian error, the rates of the upper bound obtained match (up to a logarithmic factor) with the lower bounds established in \cite{schmidt2020nonparametric}, showing that both the MEE-based NPDNN and SPDNN estimators from strongly mixing data can achieve the minimax optimal convergence rate.
【21】Efficient Approximation to Analytic and $L^p$ functions by Height-Augmented ReLU Networks
标题：通过高度增强ReLU网络对解析和$L & p$函数的有效逼近
链接：https://arxiv.org/abs/2603.11128
摘要：This work addresses two fundamental limitations in neural network approximation theory. We demonstrate that a three-dimensional network architecture enables a significantly more efficient representation of sawtooth functions, which serves as the cornerstone in the approximation of analytic and $L^p$ functions. First, we establish substantially improved exponential approximation rates for several important classes of analytic functions and offer a parameter-efficient network design. Second, for the first time, we derive a quantitative and non-asymptotic approximation of high orders for general $L^p$ functions. Our techniques advance the theoretical understanding of the neural network approximation in fundamental function spaces and offer a theoretically grounded pathway for designing more parameter-efficient networks.
其他(43篇)
【1】The Latent Color Subspace: Emergent Order in High-Dimensional Chaos
标题：潜色子空间：多维混乱中的涌现秩序
链接：https://arxiv.org/abs/2603.12261
备注：Preprint
摘要：Text-to-image generation models have advanced rapidly, yet achieving fine-grained control over generated images remains difficult, largely due to limited understanding of how semantic information is encoded. We develop an interpretation of the color representation in the Variational Autoencoder latent space of FLUX.1 [Dev], revealing a structure reflecting Hue, Saturation, and Lightness. We verify our Latent Color Subspace (LCS) interpretation by demonstrating that it can both predict and explicitly control color, introducing a fully training-free method in FLUX based solely on closed-form latent-space manipulation. Code is available at https://github.com/ExplainableML/LCS.
【2】Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
标题：Spatial-TTT：通过测试时训练流媒体基于视觉的空间智能
链接：https://arxiv.org/abs/2603.12255
备注：Project Page: https://liuff19.github.io/Spatial-TTT
摘要 ：Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with sliding-window attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that Spatial-TTT improves long-horizon spatial understanding and achieves state-of-the-art performance on video spatial benchmarks. Project page: https://liuff19.github.io/Spatial-TTT.
【3】STAMP: Selective Task-Aware Mechanism for Text Privacy
标题：STAMPS：文本隐私的选择性任务感知机制
链接：https://arxiv.org/abs/2603.12237
备注：EACL 2026
摘要：We present STAMP (Selective Task-Aware Mechanism for Text Privacy), a new framework for task-aware text privatization that achieves an improved privacy-utility trade-off. STAMP selectively allocates privacy budgets across tokens by jointly considering (i) each token's importance to the downstream task (as measured via a task- or query-specific representation), and (ii) its privacy sensitivity (e.g., names, dates, identifiers). This token-level partitioning enables fine-grained, group-wise control over the level of noise applied to different parts of the input, balancing privacy protection with task relevance. To privatize individual token embeddings, we introduce the polar mechanism, which perturbs only the direction of embeddings on the unit sphere while preserving their magnitude. Decoding is performed via cosine nearest-neighbor search, aligning the perturbation geometry with the decoding geometry. Unlike isotropic noise mechanisms, the polar mechanism maintains semantic neighborhoods in the embedding space and better preserves downstream utility. Experimental evaluations on SQuAD, Yelp, and AG News datasets demonstrate that STAMP, when combined with the normalized polar mechanism, consistently achieves superior privacy-utility trade-offs across varying per-token privacy budgets.
【4】Temporal Straightening for Latent Planning
标题：潜在规划的时间调整
链接：https://arxiv.org/abs/2603.12231
摘要：Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual features, they are not tailored to planning and contain information irrelevant -- or even detrimental -- to planning. Inspired by the perceptual straightening hypothesis in human visual processing, we introduce temporal straightening to improve representation learning for latent planning. Using a curvature regularizer that encourages locally straightened latent trajectories, we jointly learn an encoder and a predictor. We show that reducing curvature this way makes the Euclidean distance in latent space a better proxy for the geodesic distance and improves the conditioning of the planning objective. We demonstrate empirically that temporal straightening makes gradient-based planning more stable and yields significantly higher success rates across a suite of goal-reaching tasks.
【5】Security Considerations for Artificial Intelligence Agents
标题：人工智能代理的安全考虑因素
链接：https://arxiv.org/abs/2603.12230
备注：Perplexity Response to NIST/CAISI Request for Information 2025-0035. 91 Fed. Reg. 698 (Jan. 8, 2026)
摘要：This article, a lightly adapted version of Perplexity's response to NIST/CAISI Request for Information 2025-0035, details our observations and recommendations concerning the security of frontier AI agents. These insights are informed by Perplexity's experience operating general-purpose agentic systems used by millions of users and thousands of enterprises in both controlled and open-world environments. Agent architectures change core assumptions around code-data separation, authority boundaries, and execution predictability, creating new confidentiality, integrity, and availability failure modes. We map principal attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination, with particular emphasis on indirect prompt injection, confused-deputy behavior, and cascading failures in long-running workflows. We then assess current defenses as a layered stack: input-level and model-level mitigations, sandboxed execution, and deterministic policy enforcement for high-consequence actions. Finally, we identify standards and research gaps, including adaptive security benchmarks, policy models for delegation and privilege control, and guidance for secure multi-agent system design aligned with NIST risk management principles.
【6】Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
标题：神经丛林：多样化任务专家专注于预先训练的权重
链接：https://arxiv.org/abs/2603.12228
备注：codes are provided at https://github.com/sunrainyg/RandOpt
摘要：Pretraining produces a learned parameter vector that is typically treated as a starting point for further iterative adaptation. In this work, we instead view the outcome of pretraining as a distribution over parameter vectors, whose support already contains task-specific experts. We show that in small models such expert solutions occupy a negligible fraction of the volume of this distribution, making their discovery reliant on structured optimization methods such as gradient descent. In contrast, in large, well-pretrained models the density of task-experts increases dramatically, so that diverse, task-improving specialists populate a substantial fraction of the neighborhood around the pretrained weights. Motivated by this perspective, we explore a simple, fully parallel post-training method that samples $N$ parameter perturbations at random, selects the top $K$, and ensembles predictions via majority vote. Despite its simplicity, this approach is competitive with standard post-training methods such as PPO, GRPO, and ES for contemporary large-scale models.
【7】Interpreting Contrastive Embeddings in Specific Domains with Fuzzy Rules
标题：用模糊规则解释特定领域的对比嵌入
链接
：https://arxiv.org/abs/2603.12227
摘要：Free-style text is still one of the common ways in which data is registered in real environments, like legal procedures and medical records. Because of that, there have been significant efforts in the area of natural language processing to convert these texts into a structured format, which standard machine learning methods can then exploit. One of the most popular methods to embed text into a vectorial representation is the Contrastive Language-Image Pre-training model (CLIP), which was trained using both image and text. Although the representations computed by CLIP have been very successful in zero-show and few-shot learning problems, they still have problems when applied to a particular domain. In this work, we use a fuzzy rule-based classification system along with some standard text procedure techniques to map some of our features of interest to the space created by a CLIP model. Then, we discuss the rules and associations obtained and the importance of each feature considered. We apply this approach in two different data domains, clinical reports and film reviews, and compare the results obtained individually and when considering both. Finally, we discuss the limitations of this approach and how it could be further improved.
【8】IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
标题：Indexache：通过跨层索引重用加速稀疏注意力
链接：https://arxiv.org/abs/2603.12201
摘要：Long-context agentic workflows have emerged as a defining use case for large language models, making attention efficiency critical for both inference speed and serving cost. Sparse attention addresses this challenge effectively, and DeepSeek Sparse Attention (DSA) is a representative production-grade solution: a lightweight lightning indexer selects the top-k most relevant tokens per query, reducing core attention from $O(L^2)$ to $O(Lk)$. However, the indexer itself retains $O(L^2)$ complexity and must run independently at every layer, despite the fact that the resulting top-k selections are highly similar across consecutive layers. We present IndexCache, which exploits this cross-layer redundancy by partitioning layers into a small set of Full layers that run their own indexers and a majority of Shared layers that simply reuse the nearest Full layer's top-k indices. We propose two complementary approaches to determine and optimize this configuration. Training-free IndexCache applies a greedy search algorithm that selects which layers to retain indexers by directly minimizing language modeling loss on a calibration set, requiring no weight updates. Training-aware IndexCache introduces a multi-layer distillation loss that trains each retained indexer against the averaged attention distributions of all layers it serves, enabling even simple interleaved patterns to match full-indexer accuracy. Experimental results on a 30B DSA model show that IndexCache can remove 75% of indexer computations with negligible quality degradation, achieving up to 1.82$\times$ prefill speedup and 1.48$\times$ decode speedup compared to standard DSA. These positive results are further confirmed by our preliminary experiments on the production-scale GLM-5 model (Figure 1).
【9】A Quantitative Characterization of Forgetting in Post-Training
标题：训练后遗忘的定量表征
链接：https://arxiv.org/abs/2603.12163
摘要：Continual post-training of generative models is widely used, yet a principled understanding of when and why forgetting occurs remains limited. We develop theoretical results under a two-mode mixture abstraction (representing old and new tasks), proposed by Chen et al. (2025) (arXiv:2510.18874), and formalize forgetting in two forms: (i) mass forgetting, where the old mixture weight collapses to zero, and (ii) old-component drift, where an already-correct old component shifts during training. For equal-covariance Gaussian modes, we prove that forward-KL objectives trained on data from the new distribution drive the old weight to zero, while reverse-KL objectives converge to the true target (thereby avoiding mass forgetting) and perturb the old mean only through overlap-gated misassignment probabilities controlled by the Bhattacharyya coefficient, yielding drift that decays exponentially with mode separation and a locally well-conditioned geometry with exponential convergence. We further quantify how replay interacts with these objectives. For forward-KL, replay must modify the training distribution to change the population optimum; for reverse-KL, replay leaves the population objective unchanged but prevents finite-batch old-mode starvation through bounded importance weighting. Finally, we analyze three recently proposed near-on-policy post-training methods, SDFT (arxiv:2601.19897), TTT-Discover (arxiv:2601.16175), and OAPL (arxiv:2602.19362), via the same lens and derive explicit conditions under which each retains old mass and exhibits overlap-controlled drift. Overall, our results show that forgetting can by precisely quantified based on the interaction between divergence direction, geometric behavioral overlap, sampling regime, and the visibility of past behavior during training.
【10】Taming the Adversary: Stable Minimax Deep Deterministic Policy Gradient via Fractional Objectives
标题：驯服竞争者：通过分数目标实现稳定的极小极大深度决定性政策梯度
链接：https://arxiv.org/abs/2603.12110
摘要：Reinforcement learning (RL) has achieved remarkable success in a wide range of control and decision-making tasks. However, RL agents often exhibit unstable or degraded performance when deployed in environments subject to unexpected external disturbances and model uncertainties. Consequently, ensuring reliable performance under such conditions remains a critical challenge. In this paper, we propose minimax deep deterministic policy gradient (MMDDPG), a framework for learning disturbance-resilient policies in continuous control tasks. The training process is formulated as a minimax optimization problem between a user policy and an adversarial disturbance policy. In this problem, the user learns a robust policy that minimizes the objective function, while the adversary generates disturbances that maximize it. To stabilize this interaction, we introduce a fractional objective that balances task performance and disturbance magnitude. This objective prevents excessively aggressive disturbances and promotes robust learning. Experimental evaluations in MuJoCo environments demonstrate that the proposed MMDDPG achieves significantly improved robustness against both external force perturbations and model parameter variations.
【11】A Multi-Label Temporal Convolutional Framework for Transcription Factor Binding Characterization
标题：用于转录因子结合特征的多标签时间卷积框架
链接：https://arxiv.org/abs/2603.12073
摘要：Transcription factors (TFs) regulate gene expression through complex and co-operative mechanisms. While many TFs act together, the logic underlying TFs binding and their interactions is not fully understood yet. Most current approaches for TF binding site prediction focus on individual TFs and binary classification tasks, without a full analysis of the possible interactions among various TFs. In this paper we investigate DNA TF binding site recognition as a multi-label classification problem, achieving reliable predictions for multiple TFs on DNA sequences retrieved in public repositories. Our deep learning models are based on Temporal Convolutional Networks (TCNs), which are able to predict multiple TF binding profiles, capturing correlations among TFs andtheir cooperative regulatory mechanisms. Our results suggest that multi-label learning leading to reliable predictive performances can reveal biologically meaningful motifs and co-binding patterns consistent with known TF interactions, while also suggesting novel relationships and cooperation among TFs.
【12】Decentralized Orchestration Architecture for Fluid Computing: A Secure Distributed AI Use Case
标题：流计算的分散式架构：一个安全的分布式AI用例
链接：https://arxiv.org/abs/2603.12001
备注：19 pages, 9 figures and 1 table. Under peer review
摘要：Distributed AI and IoT applications increasingly execute across heterogeneous resources spanning end devices, edge/fog infrastructure, and cloud platforms, often under different administrative domains. Fluid Computing has emerged as a promising paradigm for enhancing massive resource management across the computing continuum by treating such resources as a unified fabric, enabling optimal service-agnostic deployments driven by application requirements. However, existing solutions remain largely centralized and often do not explicitly address multi-domain considerations. This paper proposes an agnostic multi-domain orchestration architecture for fluid computing environments. The orchestration plane enables decentralized coordination among domains that maintain local autonomy while jointly realizing intent-based deployment requests from tenants, ensuring end-to-end placement and execution. To this end, the architecture elevates domain-side control services as first-class capabilities to support application-level enhancement at runtime. As a representative use case, we consider a multi-domain Decentralized Federated Learning (DFL) deployment under Byzantine threats. We leverage domain-side capabilities to enhance Byzantine security by introducing FU-HST, an SDN-enabled multi-domain anomaly detection mechanism that complements Byzantine-robust aggregation. We validate the approach via simulation in single- and multi-domain settings, evaluating anomaly detection, DFL performance, and computation/communication overhead.
【13】On-Average Stability of Multipass Preconditioned SGD and Effective Dimension
标题：多遍预处理的BCD和有效维度的平均稳定性
链接：https://arxiv.org/abs/2603.11989
备注：35 pages, 1 figure
摘要：We study trade-offs between the population risk curvature, geometry of the noise, and preconditioning on the generalisation ability of the multipass Preconditioned Stochastic Gradient Descent (PSGD). Many practical optimisation heuristics implicitly navigate this trade-off in different ways -- for instance, some aim to whiten gradient noise, while others aim to align updates with expected loss curvature. When the geometry of the population risk curvature and the geometry of the gradient noise do not match, an aggressive choice that improves one aspect can amplify instability along the other, leading to suboptimal statistical behavior. In this paper we employ on-average algorithmic stability to connect generalisation of PSGD to the effective dimension that depends on these sources of curvature. While existing techniques for on-average stability of SGD are limited to a single pass, as first contribution we develop a new on-average stability analysis for multipass SGD that handles the correlations induced by data reuse. This allows us to derive excess risk bounds that depend on the effective dimension. In particular, we show that an improperly chosen preconditioner can yield suboptimal effective dimension dependence in both optimisation and generalisation. Finally, we complement our upper bounds with matching, instance-dependent lower bounds.
【14】Topological DeepONets and a generalization of the Chen-Chen operator approximation theorem
标题：Topological DeepOnets和Chen-Chen运算符逼近定理的推广
链接：https://arxiv.org/abs/2603.11972
备注：22 pages, 1 figure, 23 references
摘要：Deep Operator Networks (DeepONets) provide a branch-trunk neural architecture for approximating nonlinear operators acting between function spaces. In the classical operator approximation framework, the input is a function $u\in C(K_1)$ defined on a compact set $K_1$ (typically a compact subset of a Banach space), and the operator maps $u$ to an output function $G(u)\in C(K_2)$ defined on a compact Euclidean domain $K_2\subset\mathbb{R}^d$. In this paper, we develop a topological extension in which the operator input lies in an arbitrary Hausdorff locally convex space $X$. We construct topological feedforward neural networks on $X$ using continuous linear functionals from the dual space $X^*$ and introduce topological DeepONets whose branch component acts on $X$ through such linear measurements, while the trunk component acts on the Euclidean output domain. Our main theorem shows that continuous operators $G:V\to C(K;\mathbb{R}^m)$, where $V\subset X$ and $K\subset\mathbb{R}^d$ are compact, can be uniformly approximated by such topological DeepONets. This extends the classical Chen-Chen operator approximation theorem from spaces of continuous functions to locally convex spaces and yields a branch-trunk approximation theorem beyond the Banach-space setting.
【15】Geometry-Aware Probabilistic Circuits via Voronoi Tessellations
标题
：通过Voronoi网格化的几何感知概率电路
链接：https://arxiv.org/abs/2603.11946
摘要：Probabilistic circuits (PCs) enable exact and tractable inference but employ data independent mixture weights that limit their ability to capture local geometry of the data manifold. We propose Voronoi tessellations (VT) as a natural way to incorporate geometric structure directly into the sum nodes of a PC. However, naïvely introducing such structure breaks tractability. We formalize this incompatibility and develop two complementary solutions: (1) an approximate inference framework that provides guaranteed lower and upper bounds for inference, and (2) a structural condition for VT under which exact tractable inference is recovered. Finally, we introduce a differentiable relaxation for VT that enables gradient-based learning and empirically validate the resulting approach on standard density estimation tasks.
【16】Effective Resistance Rewiring: A Simple Topological Correction for Over-Squashing
标题：有效的阻力重新布线：过度挤压的简单布局修正
链接：https://arxiv.org/abs/2603.11944
摘要：Graph Neural Networks struggle to capture long-range dependencies due to over-squashing, where information from exponentially growing neighborhoods must pass through a small number of structural bottlenecks. While recent rewiring methods attempt to alleviate this limitation, many rely on local criteria such as curvature, which can overlook global connectivity constraints that restrict information flow. We introduce Effective Resistance Rewiring (ERR), a simple topology correction strategy that uses effective resistance as a global signal to detect structural bottlenecks. ERR iteratively adds edges between node pairs with the largest resistance while removing edges with minimal resistance, strengthening weak communication pathways while controlling graph densification under a fixed edge budget. The procedure is parameter-free beyond the rewiring budget and relies on a single global measure aggregating all paths between node pairs. Beyond predictive performance with GCN models, we analyze how rewiring affects message propagation. By tracking cosine similarity between node embeddings across layers, we examine how the relationship between initial node features and learned representations evolves during message passing, comparing graphs with and without rewiring. This analysis helps determine whether improvements arise from better long-range communication rather than changes in embedding geometry. Experiments on homophilic and heterophilic graphs, including directed settings with DirGCN, reveal a trade-off between over-squashing and oversmoothing, where oversmoothing corresponds to the loss of representation diversity across layers. Resistance-guided rewiring improves connectivity and signal propagation but can accelerate representation mixing in deep models. Combining ERR with normalization techniques such as PairNorm stabilizes this trade-off and improves performance.
【17】Causal Matrix Completion under Multiple Treatments via Mixed Synthetic Nearest Neighbors
标题：通过混合合成最近邻体在多重处理下完成因果矩阵
链接：https://arxiv.org/abs/2603.11942
摘要：Synthetic Nearest Neighbors (SNN) provides a principled solution to causal matrix completion under missing-not-at-random (MNAR) by exploiting local low-rank structure through fully observed anchor submatrices. However, its effectiveness critically relies on sufficient data availability within each treatment level, a condition that often fails in settings with multiple or complex treatments. In this work, we propose Mixed Synthetic Nearest Neighbors (MSNN), a new entry-wise causal identification estimator that integrates information across treatment levels. We show that MSNN retains the finite-sample error bounds and asymptotic normality guarantees of SNN, while enlarging the effective sample size available for estimation. Empirical results on synthetic and real-world datasets illustrate the efficacy of the proposed approach, especially under data-scarce treatment levels.
【18】Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting
标题：基于原型的细粒度结构化放射学报告知识指南
链接：https://arxiv.org/abs/2603.11938
摘要：Structured radiology reporting promises faster, more consistent communication than free text, but automation remains difficult as models must make many fine-grained, discrete decisions about rare findings and attributes from limited structured supervision. In contrast, free-text reports are produced at scale in routine care and implicitly encode fine-grained, image-linked information through detailed descriptions. To leverage this unstructured knowledge, we propose ProtoSR, an approach for injecting free-text information into structured report population. First, we introduce an automatic extraction pipeline that uses an instruction-tuned LLM to mine 80k+ MIMIC-CXR studies and build a multimodal knowledge base aligned with a structured reporting template, representing each answer option with a visual prototype. Using this knowledge base, ProtoSR is trained to retrieve prototypes relevant for the current image-question pair and augment the model predictions through a prototype-conditioned residual, providing a data-driven second opinion that selectively corrects predictions. On the Rad-ReStruct benchmark, ProtoSR achieves state-of-the-art results, with the largest improvements on detailed attribute questions, demonstrating the value of integrating free-text derived signal for fine-grained image understanding.
【19】On the Role of Reversible Instance Normalization
标题：论可逆实例规范化的作用
链接：https://arxiv.org/abs/2603.11869
摘要 ：Data normalization is a crucial component of deep learning models, yet its role in time series forecasting remains insufficiently understood. In this paper, we identify three central challenges for normalization in time series forecasting: temporal input distribution shift, spatial input distribution shift, and conditional output distribution shift. In this context, we revisit the widely used Reversible Instance Normalization (RevIN), by showing through ablation studies that several of its components are redundant or even detrimental. Based on these observations, we draw new perspectives to improve RevIN's robustness and generalization.
【20】Multi-Station WiFi CSI Sensing Framework Robust to Station-wise Feature Missingness and Limited Labeled Data
标题：多站WiFi SI传感框架对站点特征缺失和有限的标记数据具有鲁棒性
链接：https://arxiv.org/abs/2603.11858
备注：17 pages, 14 figures, 7 tables
摘要：We propose a WiFi Channel State Information (CSI) sensing framework for multi-station deployments that addresses two fundamental challenges in practical CSI sensing: station-wise feature missingness and limited labeled data. Feature missingness is commonly handled by resampling unevenly spaced CSI measurements or by reconstructing missing samples, while label scarcity is mitigated by data augmentation or self-supervised representation learning. However, these techniques are typically developed in isolation and do not jointly address long-term, structured station unavailability together with label scarcity. To bridge this gap, we explicitly incorporate station unavailability into both representation learning and downstream model training. Specifically, we adapt cross-modal self-supervised learning (CroSSL), a representation learning framework originally designed for time-series sensory data, to multi-station CSI sensing in order to learn representations that are inherently invariant to station-wise feature missingness from unlabeled data. Furthermore, we introduce Station-wise Masking Augmentation (SMA) during downstream model training, which exposes the model to realistic station unavailability patterns under limited labeled data. Our experiments show that neither missingness-invariant pre-training nor station-wise augmentation alone is sufficient; their combination is essential to achieve robust performance under both station-wise feature missingness and label scarcity. The proposed framework provides a practical and robust foundation for multi-station WiFi CSI sensing in real-world deployments.
【21】A Further Efficient Algorithm with Best-of-Both-Worlds Guarantees for $m$-Set Semi-Bandit Problem
标题：m$-集半Bandit问题的双世界最优保证的进一步有效算法
链接：https://arxiv.org/abs/2603.11764
摘要：This paper studies the optimality and complexity of Follow-the-Perturbed-Leader (FTPL) policy in $m$-set semi-bandit problems. FTPL has been studied extensively as a promising candidate of an efficient algorithm with favorable regret for adversarial combinatorial semi-bandits. Nevertheless, the optimality of FTPL has still been unknown unlike Follow-the-Regularized-Leader (FTRL) whose optimality has been proved for various tasks of online learning. In this paper, we extend the analysis of FTPL with geometric resampling (GR) to $m$-set semi-bandits, which is a special case of combinatorial semi-bandits, showing that FTPL with Fréchet and Pareto distributions with certain parameters achieves the best possible regret of $O(\sqrt{mdT})$ in adversarial setting. We also show that FTPL with Fréchet and Pareto distributions with a certain parameter achieves a logarithmic regret for stochastic setting, meaning the Best-of-Both-Worlds optimality of FTPL for $m$-set semi-bandit problems. Furthermore, we extend the conditional geometric resampling to $m$-set semi-bandits for efficient loss estimation in FTPL, reducing the computational complexity from $O(d^2)$ of the original geometric resampling to $O(md(\log(d/m)+1))$ without sacrificing the regret performance.
【22】CINDI: Conditional Imputation and Noisy Data Integrity with Flows in Power Grid Data
标题：CINDI：电网数据流的条件插补和噪音数据完整性
链接：https://arxiv.org/abs/2603.11745
摘要：Real-world multivariate time series, particularly in critical infrastructure such as electrical power grids, are often corrupted by noise and anomalies that degrade the performance of downstream tasks. Standard data cleaning approaches often rely on disjoint strategies, which involve detecting errors with one model and imputing them with another. Such approaches can fail to capture the full joint distribution of the data and ignore prediction uncertainty. This work introduces Conditional Imputation and Noisy Data Integrity (CINDI), an unsupervised probabilistic framework designed to restore data integrity in complex time series. Unlike fragmented approaches, CINDI unifies anomaly detection and imputation into a single end-to-end system built on conditional normalizing flows. By modeling the exact conditional likelihood of the data, the framework identifies low-probability segments and iteratively samples statistically consistent replacements. This allows CINDI to efficiently reuse learned information while preserving the underlying physical and statistical properties of the system. We evaluate the framework using real-world grid loss data from a Norwegian power distribution operator, though the methodology is designed to generalize to any multivariate time series domain. The results demonstrate that CINDI yields robust performance compared to competitive baselines, offering a scalable solution for maintaining reliability in noisy environments.
【23】EvoFlows: Evolutionary Edit-Based Flow-Matching for Protein Engineering
标题：EvoFlows：基于进化编辑的蛋白质工程流匹配
链接：https://arxiv.org/abs/2603.11703
备注：Accepted at Workshop on Foundation Models for Science: Real-World Impact and Science-First Design, ICLR 2026
摘要 ：We introduce EvoFlows, a variable-length sequence-to-sequence protein modeling approach uniquely suited to protein engineering. Unlike autoregressive and masked language models, EvoFlows perform a limited, controllable number of insertions, deletions, and substitutions on a template protein sequence. In other words, EvoFlows predict not only _which_ mutation to perform, but also _where_ it should occur. Our approach leverages edit flows to learn mutational trajectories between evolutionarily-related protein sequences, simultaneously modeling distributions of related natural proteins and the mutational paths connecting them. Through extensive _in silico_ evaluation on diverse protein communities from UNIREF and OAS, we demonstrate that EvoFlows capture protein sequence distributions with a quality comparable to leading masked language models commonly used in protein engineering, while showing improved ability to generate non-trivial yet natural-like mutants from a given template protein.
【24】Causal Prosody Mediation for Text-to-Speech:Counterfactual Training of Duration, Pitch, and Energy in FastSpeech2
标题：文本到言语的因果韵律调解：FastSpeech中持续时间、音调和精力的反事实训练2
链接：https://arxiv.org/abs/2603.11683
摘要：We propose a novel causal prosody mediation framework for expressive text-to-speech (TTS) synthesis. Our approach augments the FastSpeech2 architecture with explicit emotion conditioning and introduces counterfactual training objectives to disentangle emotional prosody from linguistic content. By formulating a structural causal model of how text (content), emotion, and speaker jointly influence prosody (duration, pitch, energy) and ultimately the speech waveform, we derive two complementary loss terms: an Indirect Path Constraint (IPC) to enforce that emotion affects speech only through prosody, and a Counterfactual Prosody Constraint (CPC) to encourage distinct prosody patterns for different emotions. The resulting model is trained on multi-speaker emotional corpora (LibriTTS, EmoV-DB, VCTK) with a combined objective that includes standard spectrogram reconstruction and variance prediction losses alongside our causal losses. In evaluations on expressive speech synthesis, our method achieves significantly improved prosody manipulation and emotion rendering, with higher mean opinion scores (MOS) and emotion accuracy than baseline FastSpeech2 variants. We also observe better intelligibility (low WER) and speaker consistency when transferring emotions across speakers. Extensive ablations confirm that the causal objectives successfully separate prosody attribution, yielding an interpretable model that allows controlled counterfactual prosody editing (e.g. "same utterance, different emotion") without compromising naturalness. We discuss the implications for identifiability in prosody modeling and outline limitations such as the assumption that emotion effects are fully captured by pitch, duration, and energy. Our work demonstrates how integrating causal learning principles into TTS can improve controllability and expressiveness in generated speech.
【25】CFD-HAR: User-controllable Privacy through Conditional Feature Disentanglement
标题：CFD-HAR：通过条件特征分解实现用户可控隐私
链接：https://arxiv.org/abs/2603.11526
摘要：Modern wearable and mobile devices are equipped with inertial measurement units (IMUs). Human Activity Recognition (HAR) applications running on such devices use machine-learning-based, data-driven techniques that leverage such sensor data. However, sensor-data-driven HAR deployments face two critical challenges: protecting sensitive user information embedded in sensor data in accordance with users' privacy preferences and maintaining high recognition performance with limited labeled samples. This paper proposes a technique for user-controllable privacy through feature disentanglement-based representation learning at the granular level for dynamic privacy filtering. We also compare the efficacy of our technique against few-shot HAR using autoencoder-based representation learning. We analyze their architectural designs, learning objectives, privacy guarantees, data efficiency, and suitability for edge Internet of Things (IoT) deployment. Our study shows that CFD-based HAR provides explicit, tunable privacy protection controls by separating activity and sensitive attributes in the latent space, whereas autoencoder-based few-shot HAR offers superior label efficiency and lightweight adaptability but lacks inherent privacy safeguards. We further examine the security implications of both approaches in continual IoT settings, highlighting differences in susceptibility to representation leakage and embedding-level attacks. The analysis reveals that neither paradigm alone fully satisfies the emerging requirements of next-generation IoT HAR systems. We conclude by outlining research directions toward unified frameworks that jointly optimize privacy preservation, few-shot adaptability, and robustness for trustworthy IoT intelligence.
【26】Leveraging Phytolith Research using Artificial Intelligence
标题：利用人工智能利用植硅体研究
链接：https://arxiv.org/abs/2603.11476
备注：45 pages, 23 figures
摘要：Phytolith analysis is a crucial tool for reconstructing past vegetation and human activities, but traditional methods are severely limited by labour-intensive, time-consuming manual microscopy. To address this bottleneck, we present Sorometry: a comprehensive end-to-end artificial intelligence pipeline for the high-throughput digitisation, inference, and interpretation of phytoliths. Our workflow processes z-stacked optical microscope scans to automatically generate synchronised 2D orthoimages and 3D point clouds of individual microscopic particles. We developed a multimodal fusion model that combines ConvNeXt for 2D image analysis and PointNet++ for 3D point cloud analysis, supported by a graphical user interface for expert annotation and review. Tested on reference collections and archaeological samples from the Bolivian Amazon, our fusion model achieved a global classification accuracy of 77.9\% across 24 diagnostic morphotypes and 84.5% for segmentation quality. Crucially, the integration of 3D data proved essential for distinguishing complex morphotypes (such as grass silica short cell phytoliths) whose diagnostic features are often obscured by their orientation in 2D projections. Beyond individual object classification, Sorometry incorporates Bayesian finite mixture modelling to predict overall plant source contributions at the assemblage level, successfully identifying specific plants like maize and palms in complex mixed samples. This integrated platform transforms phytolith research into an "omics"-scale discipline, dramatically expanding analytical capacity, standardising expert judgements, and enabling reproducible, population-level characterisations of archaeological and paleoecological assemblages.
【27】HawkesRank: Event-Driven Centrality for Real-Time Importance Ranking
标题：HawkesRank：实时重要性排名的事件驱动中心
链接：https://arxiv.org/abs/2603.11472
备注：10 pages, 3 figures + SM (8 pages, 2 figures)
摘要：Quantifying influence in networks is important across science, economics, and public health, yet widely used centrality measures remain limited: they rely on static representations, heuristic network constructions, and purely endogenous notions of importance, while offering little semantic connection to observable activity. We introduce HawkesRank, a dynamic framework grounded in multivariate Hawkes point processes that models exogenous drivers (intrinsic contributions) and endogenous amplification (self- and cross-excitation). This yields a principled, empirically calibrated, and adaptive importance measure. Classical indices such as Katz centrality and PageRank emerge as mean-field limits of the framework, clarifying both their validity and their limitations. Unlike static averages, HawkesRank measures importance through instantaneous event intensities, enabling prediction, transparent endo-exo decomposition, and adaptability to shocks. Using both simulations and empirical analysis of emotion dynamics in online communication platforms, we show that HawkesRank closely tracks system activity and consistently outperforms static centrality metrics.
【28】Bridging Discrete Marks and Continuous Dynamics: Dual-Path Cross-Interaction for Marked Temporal Point Processes
标题：跨越离散标记和连续动态：标记时间点过程的双路径交叉交互作用
链接：https://arxiv.org/abs/2603.11462
摘要：Predicting irregularly spaced event sequences with discrete marks poses significant challenges due to the complex, asynchronous dependencies embedded within continuous-time data streams.Existing sequential approaches capture dependencies among event tokens but ignore the continuous evolution between events, while Neural Ordinary Differential Equation (Neural ODE) methods model smooth dynamics yet fail to account for how event types influence future timing.To overcome these limitations, we propose NEXTPP, a dual-channel framework that unifies discrete and continuous representations via Event-granular Neural Evolution with Cross-Interaction for Marked Temporal Point Processes. Specifically, NEXTPP encodes discrete event marks via a self-attention mechanism, simultaneously evolving a latent continuous-time state using a Neural ODE. These parallel streams are then fused through a crossattention module to enable explicit bidirectional interaction between continuous and discrete representations. The fused representations drive the conditional intensity function of the neural Hawkes process, while an iterative thinning sampler is employed to generate future events. Extensive evaluations on five real-world datasets demonstrate that NEXTPP consistently outperforms state-of-the-art models. The source code can be found at https://github.com/AONE-NLP/NEXTPP.
【29】Deployment-Time Reliability of Learned Robot Policies
标题：学习机器人策略的部署时间可靠性
链接：https://arxiv.org/abs/2603.11400
备注：Stanford University PhD dissertation, 2026. 182 pages, 37 figures. Available from Stanford Digital Repository
摘要：Recent advances in learning-based robot manipulation have produced policies with remarkable capabilities. Yet, reliability at deployment remains a fundamental barrier to real-world use, where distribution shift, compounding errors, and complex task dependencies collectively undermine system performance. This dissertation investigates how the reliability of learned robot policies can be improved at deployment time through mechanisms that operate around them. We develop three complementary classes of deployment-time mechanisms. First, we introduce runtime monitoring methods that detect impending failures by identifying inconsistencies in closed-loop policy behavior and deviations in task progress, without requiring failure data or task-specific supervision. Second, we propose a data-centric framework for policy interpretability that traces deployment-time successes and failures to influential training demonstrations using influence functions, enabling principled diagnosis and dataset curation. Third, we address reliable long-horizon task execution by formulating policy coordination as the problem of estimating and maximizing the success probability of behavior sequences, and we extend this formulation to open-ended, language-specified tasks through feasibility-aware task planning. By centering on core challenges of deployment, these contributions advance practical foundations for the reliable, real-world use of learned robot policies. Continued progress on these foundations will be essential for enabling trustworthy and scalable robot autonomy in the future.
【30】Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
标题：低资源斯瓦希里语SVR的持续预训练：利用最少的标签数据实现最先进的性能
链接：https://arxiv.org/abs/2603.11378
摘要：We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource languages.
【31】Relaxed Efficient Acquisition of Context and Temporal Features
标题：轻松有效地获取上下文和时间特征
链接：https://arxiv.org/abs/2603.11370
摘要 ：In many biomedical applications, measurements are not freely available at inference time: each laboratory test, imaging modality, or assessment incurs financial cost, time burden, or patient risk. Longitudinal active feature acquisition (LAFA) seeks to optimize predictive performance under such constraints by adaptively selecting measurements over time, yet the problem remains inherently challenging due to temporally coupled decisions (missed early measurements cannot be revisited, and acquisition choices influence all downstream predictions). Moreover, real-world clinical workflows typically begin with an initial onboarding phase, during which relatively stable contextual descriptors (e.g., demographics or baseline characteristics) are collected once and subsequently condition longitudinal decision-making. Despite its practical importance, the efficient selection of onboarding context has not been studied jointly with temporally adaptive acquisition. We therefore propose REACT (Relaxed Efficient Acquisition of Context and Temporal features), an end-to-end differentiable framework that simultaneously optimizes (i) selection of onboarding contextual descriptors and (ii) adaptive feature--time acquisition plans for longitudinal measurements under cost constraints. REACT employs a Gumbel--Sigmoid relaxation with straight-through estimation to enable gradient-based optimization over discrete acquisition masks, allowing direct backpropagation from prediction loss and acquisition cost. Across real-world longitudinal health and behavioral datasets, REACT achieves improved predictive performance at lower acquisition costs compared to existing longitudinal acquisition baselines, demonstrating the benefit of modeling onboarding and temporally coupled acquisition within a unified optimization framework.
【32】On the Robustness of Langevin Dynamics to Score Function Error
标题：关于Langevin动力学对得分函数误差的鲁棒性
链接：https://arxiv.org/abs/2603.11319
摘要：We consider the robustness of score-based generative modeling to errors in the estimate of the score function. In particular, we show that Langevin dynamics is not robust to the L^2 errors (more generally L^p errors) in the estimate of the score function. It is well-established that with small L^2 errors in the estimate of the score function, diffusion models can sample faithfully from the target distribution under fairly mild regularity assumptions in a polynomial time horizon. In contrast, our work shows that even for simple distributions in high dimensions, Langevin dynamics run for any polynomial time horizon will produce a distribution far from the target distribution in Total Variation (TV) distance, even when the L^2 error (more generally L^p) of the estimate of the score function is arbitrarily small. Considering such an error in the estimate of the score function is unavoidable in practice when learning the score function from data, our results provide further justification for diffusion models over Langevin dynamics and serve to caution against the use of Langevin dynamics with estimated scores.
【33】Duration Aware Scheduling for ASR Serving Under Workload Drift
标题：在预设漂移下服务的ASB的持续时间感知调度
链接：https://arxiv.org/abs/2603.11273
摘要：Scheduling policies in large-scale Automatic Speech Recognition (ASR) serving pipelines play a key role in determining end-to-end (E2E) latency. Yet, widely used serving engines rely on first-come-first-served (FCFS) scheduling, which ignores variability in request duration and leads to head-of-line blocking under workload drift. We show that audio duration is an accurate proxy for job processing time in ASR models such as Whisper, and use this insight to enable duration-aware scheduling. We integrate two classical algorithms, Shortest Job First (SJF) and Highest Response Ratio Next (HRRN), into vLLM and evaluate them under realistic and drifted workloads. On LibriSpeech test-clean, compared to baseline, SJF reduces median E2E latency by up to $73\%$ at high load, but increases $90$th-percentile tail latency by up to $97\%$ due to starvation of long requests. HRRN addresses this trade-off: it reduces median E2E latency by up to $28\%$ while bounding tail-latency degradation to at most $24\%$. These gains persist under workload drift, with no throughput penalty and $<0.1$\,ms scheduling overhead per request.
【34】Reference-Guided Machine Unlearning
标题：参考引导的机器取消学习
链接：https://arxiv.org/abs/2603.11210
备注：12 pages, 1 figure, 4 tables. Accepted at three ICLR 2026 workshops: Test-Time Updates (TTU), AI with Recursive Self-Improvement (RSI), and Agents in the Wild (AIWILD)
摘要：Machine unlearning aims to remove the influence of specific data from trained models while preserving general utility. Existing approximate unlearning methods often rely on performance-degradation heuristics, such as loss maximization or random labeling. However, these signals can be poorly conditioned, leading to unstable optimization and harming the model's generalization. We argue that unlearning should instead prioritize distributional indistinguishability, aligning the model's behavior on forget data with its behavior on truly unseen data. Motivated by this, we propose Reference-Guided Unlearning (ReGUn), a framework that leverages a disjoint held-out dataset to provide a principled, class-conditioned reference for distillation. We demonstrate across various model architectures, natural image datasets, and varying forget fractions that ReGUn consistently outperforms standard approximate baselines, achieving a superior forgetting-utility trade-off.
【35】Catalogue Grounded Multimodal Attribution for Museum Video under Resource and Regulatory Constraints
标题：资源和监管约束下博物馆视频的目录多模式归因
链接：https://arxiv.org/abs/2603.11147
摘要：Audiovisual (AV) archives in museums and galleries are growing rapidly, but much of this material remains effectively locked away because it lacks consistent, searchable metadata. Existing method for archiving requires extensive manual effort. We address this by automating the most labour intensive part of the workflow: catalogue style metadata curation for in gallery video, grounded in an existing collection database. Concretely, we propose catalogue-grounded multimodal attribution for museum AV content using an open, locally deployable video language model. We design a multi pass pipeline that (i) summarises artworks in a video, (ii) generates catalogue style descriptions and genre labels, and (iii) attempts to attribute title and artist via conservative similarity matching to the structured catalogue. Early deployments on a painting catalogue suggest that this framework can improve AV archive discoverability while respecting resource constraints, data sovereignty, and emerging regulation, offering a transferable template for application-driven machine learning in other high-stakes domains.
【36】Procedural Fairness via Group Counterfactual Explanation
标题：通过群体反事实解释实现程序公平
链接：https://arxiv.org/abs/2603.11140
备注：16 pages, submitted to ECML 2026
摘要：Fairness in machine learning research has largely focused on outcome-oriented fairness criteria such as Equalized Odds, while comparatively less attention has been given to procedural-oriented fairness, which addresses how a model arrives at its predictions. Neglecting procedural fairness means it is possible for a model to generate different explanations for different protected groups, thereby eroding trust. In this work, we introduce Group Counterfactual Integrated Gradients (GCIG), an in-processing regularization framework that enforces explanation invariance across groups, conditioned on the true label. For each input, GCIG computes explanations relative to multiple Group Conditional baselines and penalizes cross-group variation in these attributions during training. GCIG formalizes procedural fairness as Group Counterfactual explanation stability and complements existing fairness objectives that constrain predictions alone. We compared GCIG empirically against six state-of-the-art methods, and the results show that GCIG substantially reduces cross-group explanation disparity while maintaining competitive predictive performance and accuracy-fairness trade-offs. Our results also show that aligning model reasoning across groups offers a principled and practical avenue for advancing fairness beyond outcome parity.
【37】Higher-Order Modular Attention: Fusing Pairwise and Triadic Interactions for Protein Sequences
标题：更高级模块注意力：融合蛋白质序列的成对和三重相互作用
链接：https://arxiv.org/abs/2603.11133
备注：11, 4 figures
摘要：Transformer self-attention computes pairwise token interactions, yet protein sequence to phenotype relationships often involve cooperative dependencies among three or more residues that dot product attention does not capture explicitly. We introduce Higher-Order Modular Attention, HOMA, a unified attention operator that fuses pairwise attention with an explicit triadic interaction pathway. To make triadic attention practical on long sequences, HOMA employs block-structured, windowed triadic attention. We evaluate on three TAPE benchmarks for Secondary Structure, Fluorescence, and Stability. Our attention mechanism yields consistent improvements across all tasks compared with standard self-attention and efficient variants including block-wise attention and Linformer. These results suggest that explicit triadic terms provide complementary representational capacity for protein sequence prediction at controllable additional computational cost.
【38】TinyNav: End-to-End TinyML for Real-Time Autonomous Navigation on Microcontrollers
标题：TinyNav：端到端TinyML，用于微控制器上的实时自主导航
链接：https://arxiv.org/abs/2603.11071
备注：6 pages, 7 figures, presented at CUCAI2026 (Canadian Undergraduate Conference on AI, https://cucai.ca)
摘要：Autonomous navigation typically relies on power-intensive processors, limiting accessibility in low-cost robotics. Although microcontrollers offer a resource-efficient alternative, they impose strict constraints on model complexity. We present TinyNav, an end-to-end TinyML system for real-time autonomous navigation on an ESP32 microcontroller. A custom-trained, quantized 2D convolutional neural network processes a 20-frame sliding window of depth data to predict steering and throttle commands. By avoiding 3D convolutions and recurrent layers, the 23k-parameter model achieves 30 ms inference latency. Correlation analysis and Grad-CAM validation indicate consistent spatial awareness and obstacle avoidance behavior. TinyNav demonstrates that responsive autonomous control can be deployed directly on highly constrained edge devices, reducing reliance on external compute resources.
【39】Proof-Carrying Materials: Falsifiable Safety Certificates for Machine-Learned Interatomic Potentials
标题：携带证明材料：机器学习原子间势的可证伪安全证书
链接：https://arxiv.org/abs/2603.12183
摘要：Machine-learned interatomic potentials (MLIPs) are deployed for high-throughput materials screening without formal reliability guarantees. We show that a single MLIP used as a stability filter misses 93% of density functional theory (DFT)-stable materials (recall 0.07) on a 25,000-material benchmark. Proof-Carrying Materials (PCM) closes this gap through three stages: adversarial falsification across compositional space, bootstrap envelope refinement with 95% confidence intervals, and Lean 4 formal certification. Auditing CHGNet, TensorNet and MACE reveals architecture-specific blind spots with near-zero pairwise error correlations (r <= 0.13; n = 5,000), confirmed by independent Quantum ESPRESSO validation (20/20 converged; median DFT/CHGNet force ratio 12x). A risk model trained on PCM-discovered features predicts failures on unseen materials (AUC-ROC = 0.938 +/- 0.004) and transfers across architectures (cross-MLIP AUC-ROC ~ 0.70; feature importance r = 0.877). In a thermoelectric screening case study, PCM-audited protocols discover 62 additional stable materials missed by single-MLIP screening - a 25% improvement in discovery yield.
【40】Decomposing Observational Multiplicity in Decision Trees: Leaf and Structural Regret
标题：分解决策树中的观察多重性：叶子和结构遗憾
链接：https://arxiv.org/abs/2603.11701
备注：19 pages, 3 figures
摘要 ：Many machine learning tasks admit multiple models that perform almost equally well, a phenomenon known as predictive multiplicity. A fundamental source of this multiplicity is observational multiplicity, which arises from the stochastic nature of label collection: observed training labels represent only a single realization of the underlying ground-truth probabilities. While theoretical frameworks for observational multiplicity have been established for logistic regression, their implications for non-smooth, partition-based models like decision trees remain underexplored. In this paper, we introduce two complementary notions of observational multiplicity for decision tree classifiers: leaf regret and structural regret. Leaf regret quantifies the intrinsic variability of predictions within a fixed leaf due to finite-sample noise, while structural regret captures variability induced by the instability of the learned tree structure itself. We provide a formal decomposition of observational multiplicity into these two components and establish statistical guarantees. Our experimental evaluation across diverse credit risk scoring datasets confirms the near-perfect alignment between our theoretical decomposition and the empirically observed variance. Notably, we find that structural regret is the primary driver of observational multiplicity, accounting for over 15 times the variability of leaf regret in some datasets. Furthermore, we demonstrate that utilizing these regret measures as an abstention mechanism in selective prediction can effectively identify arbitrary regions and improve model safety, elevating recall from 92% to 100% on the most stable sub-populations. These results establish a rigorous framework for quantifying observational multiplicity, aligning with recent advances in algorithmic safety and interpretability.
【41】Worst-case low-rank approximations
标题：最坏情况的低等级逼近
链接：https://arxiv.org/abs/2603.11304
摘要：Real-world data in health, economics, and environmental sciences are often collected across heterogeneous domains (such as hospitals, regions, or time periods). In such settings, distributional shifts can make standard PCA unreliable, in that, for example, the leading principal components may explain substantially less variance in unseen domains than in the training domains. Existing approaches (such as FairPCA) have proposed to consider worst-case (rather than average) performance across multiple domains. This work develops a unified framework, called wcPCA, applies it to other objectives (resulting in the novel estimators such as norm-minPCA and norm-maxregret, which are better suited for applications with heterogeneous total variance) and analyzes their relationship. We prove that for all objectives, the estimators are worst-case optimal not only over the observed source domains but also over all target domains whose covariance lies in the convex hull of the (possibly normalized) source covariances. We establish consistency and asymptotic worst-case guarantees of empirical estimators. We extend our methodology to matrix completion, another problem that makes use of low-rank approximations, and prove approximate worst-case optimality for inductive matrix completion. Simulations and two real-world applications on ecosystem-atmosphere fluxes demonstrate marked improvements in worst-case performance, with only minor losses in average performance.
【42】RIE-Greedy: Regularization-Induced Exploration for Contextual Bandits
标题：RIE-贪婪：规范化引发的对背景盗贼的探索
链接：https://arxiv.org/abs/2603.11276
摘要：Real-world contextual bandit problems with complex reward models are often tackled with iteratively trained models, such as boosting trees. However, it is difficult to directly apply simple and effective exploration strategies--such as Thompson Sampling or UCB--on top of those black-box estimators. Existing approaches rely on sophisticated assumptions or intractable procedures that are hard to verify and implement in practice. In this work, we explore the use of an exploration-free (pure-greedy) action selection strategy, that exploits the randomness inherent in model fitting process as an intrinsic source of exploration. More specifically, we note that the stochasticity in cross-validation based regularization process can naturally induce Thompson Sampling-like exploration. We show that this regularization-induced exploration is theoretically equivalent to Thompson Sampling in the two-armed bandit case and empirically leads to reliable exploration in large-scale business environments compared to benchmark methods such as epsilon-greedy and other state-of-the-art approaches. Overall, our work reveals how regularized estimator training itself can induce effective exploration, offering both theoretical insight and practical guidance for contextual bandit design.
【43】A Unified Latent Space Disentanglement VAE Framework with Robust Disentanglement Effectiveness Evaluation
标题：一种统一的潜空间解缠VAE框架及其解缠效果评估
链接：https://arxiv.org/abs/2603.11242
摘要：Evaluating and interpreting latent representations, such as variational autoencoders (VAEs), remains a significant challenge for diverse data types, especially when ground-truth generative factors are unknown. To address this, we propose a general framework -- bfVAE -- that unifies several state-of-the-art disentangled VAE approaches and generates effective latent space disentanglement, especially for tabular data. To assess the effectiveness of a VAE disentanglement technique, we propose two procedures - Feature Variance Heterogeneity via Latent Traversal (FVH-LT) and Dirty Block Sparse Regression in Latent Space (DBSR-LS) for disentanglement assessment, along with the latent space disentanglement index (LSDI) which uses the outputs of FVH-LT and DBSR-LS to summarize the overall effectiveness of a VAE disentanglement method without requiring access to or knowledge of the ground-truth generative factors. To the best of our knowledge, these are the first assessment tools to achieve this. FVH-LT and DBSR-LS also enhance latent space interpretability and provide guidance on more efficient content generation. To ensure robust and consistent disentanglement, we develop a greedy alignment strategy (GAS) that mitigates label switching and aligns latent dimensions across runs to obtain aggregated results. We assess the bfVAE framework and validate FVH-LT, DBSR-LS, and LSDI in extensive experiments on tabular and image data. The results suggest that bfVAE surpasses existing disentangled VAE frameworks in terms of disentanglement quality, robustness, achieving a near-zero false discovery rate for informative latent dimensions, that FVH-LT and DBSR-LS reliably uncover semantically meaningful and domain-relevant latent structures, and that LSDI makes an effective overall quantitative summary on disentanglement effectiveness.
机器翻译由腾讯交互翻译提供，仅供参考
点击“阅读原文”获取带摘要的学术速递
Python社区是高质量的Python/Django开发社区
本文地址：http://www.python88.com/topic/193913
本文地址：http://www.python88.com/topic/193913