|
创作新主题 |
社区所有版块导航
DATA
| docker Elasticsearch |
WEB开发
| linux MongoDB Redis DATABASE NGINX 其他Web框架 web工具 zookeeper tornado NoSql Bootstrap js peewee Git bottle IE MQ Jquery |
机器学习
| 机器学习算法 |
产品
| 短视频 |
印度
| 印度 |
一周十大热门主题
点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！
cs.LG 方向，今日共计167篇
大模型相关(20篇)
【1】Only relative ranks matter in weight-clustered large language models
标题：在权重聚集的大型语言模型中，只有相对排名才重要
链接：https://arxiv.org/abs/2603.17917
备注：10 pages, 3 figures, 9 tables
摘要：大型语言模型（LLM）包含数十亿个参数，但许多精确值并不重要。我们表明，最重要的是权重的相对排名，无论一个连接是强还是弱于其他，而不是精确的大小。为了减少唯一权重值的数量，我们将权重聚类应用于预训练的模型，将每个权重矩阵替换为来自K-means的K个共享值。对于Llama 3.1-8B-Instruct和SmolLM 2 - 135 M，将每个矩阵减少到只有16-64个不同的值可以保持很高的准确性，而无需重新训练，提供了一种简单的，无需训练的方法来压缩磁盘上的LLM。可选地，仅微调聚类均值（质心）以最小的成本恢复剩余精度差距的30- 40%。然后，我们系统地随机化聚类均值，同时保持分配固定。扰乱集群的相对等级会降低质量共享性-混乱度会以数量级增加-即使保留了全局统计数据，如均值和方差。相比之下，保秩随机化在中间层和后期层几乎没有损失。另一方面，当许多层同时被扰动时，逐层渐进替换揭示了尺度漂移而不是秩失真是主要的崩溃机制;然而，仿射校正w' = aw + b，a > 0（保留秩顺序和整体权重分布）可以大大延迟这种漂移。这种基于秩的视角为模型压缩和鲁棒性提供了新的视角。
摘要：Large language models (LLMs) contain billions of parameters, yet many exact values are not essential. We show that what matters most is the relative rank of weights-whether one connection is stronger or weaker than another-rather than precise magnitudes. To reduce the number of unique weight values, we apply weight clustering to pretrained models, replacing every weight matrix with K shared values from K-means. For Llama 3.1-8B-Instruct and SmolLM2-135M, reducing each matrix to only 16-64 distinct values preserves strong accuracy without retraining, providing a simple, training-free method to compress LLMs on disk. Optionally fine-tuning only the cluster means (centroids) recovers 30-40 percent of the remaining accuracy gap at minimal cost. We then systematically randomize cluster means while keeping assignments fixed. Scrambling the relative ranks of the clusters degrades quality sharply-perplexity can increase by orders of magnitude-even when global statistics such as mean and variance are preserved. In contrast, rank-preserving randomizations cause almost no loss at mid and late layers. On the other hand, when many layers are perturbed simultaneously, progressive layer-by-layer replacement reveals that scale drift-not rank distortion-is the dominant collapse mechanism; however, an affine correction w' = aw + b with a > 0 (which preserves both rank order and overall weight distribution) can substantially delay this drift. This rank-based perspective offers a new lens on model compression and robustness.
【2】scicode-lint: Detecting Methodology Bugs in Scientific Python Code with LLM-Generated Patterns
标题：scicode-lint：使用LLM生成的模式检测科学Python代码中的方法学错误
链接：https://arxiv.org/abs/2603.17893
摘要：科学Python代码中的方法论错误会产生合理但不正确的结果，这是传统的链接器和静态分析工具无法检测到的。几个研究小组已经建立了ML特定的链接，证明检测是可行的。然而，这些工具都有一个可持续性问题：依赖于特定的pylint或Python版本，有限的打包，以及对每个新模式的手动工程的依赖。随着人工智能生成的代码增加了科学软件的数量，对自动化方法检查（例如检测数据泄漏，不正确的交叉验证和丢失随机种子）的需求也在增长。我们提出了scicode-lint，它的两层架构将模式设计（构建时的前沿模型）与执行（运行时的小型本地模型）分开。模式是生成的，而不是手工编码的;适应新的库版本需要花费令牌，而不是工程时间。在Kaggle笔记本上，预处理泄漏检测在100%的召回率下达到65%的精度;在38篇应用AI/ML的已发表科学论文中，精度为62%（LLM判断），不同模式类别之间存在很大差异;在一个展示的论文集上，精度为54%。在受控测试中，scicode-lint在66种模式中达到了97.7%的准确率。
摘要：Methodology bugs in scientific Python code produce plausible but incorrect results that traditional linters and static analysis tools cannot detect. Several research groups have built ML-specific linters, demonstrating that detection is feasible. Yet these tools share a sustainability problem: dependency on specific pylint or Python versions, limited packaging, and reliance on manual engineering for every new pattern. As AI-generated code increases the volume of scientific software, the need for automated methodology checking (such as detecting data leakage, incorrect cross-validation, and missing random seeds) grows. We present scicode-lint, whose two-tier architecture separates pattern design (frontier models at build time) from execution (small local model at runtime). Patterns are generated, not hand-coded; adapting to new library versions costs tokens, not engineering hours. On Kaggle notebooks with human-labeled ground truth, preprocessing leakage detection reaches 65% precision at 100% recall; on 38 published scientific papers applying AI/ML, precision is 62% (LLM-judged) with substantial variation across pattern categories; on a held-out paper set, precision is 54%. On controlled tests, scicode-lint achieves 97.7% accuracy across 66 patterns.
【3】RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference
标题：RAMP：增强自适应混合精度量化，用于高效的设备上LLM推理
链接：https://arxiv.org/abs/2603.17891
摘要：训练后量化对于在资源受限的硬件上部署大型语言模型（LLM）是必不可少的，然而现有技术的方法在层之间强制执行统一的位宽，从而产生次优的准确性效率权衡。我们提出了RAMP（增强自适应混合精度），一个关闭的政策软演员批评框架，学习每层位宽分配，以尽量减少全球比特预算下的困惑。该策略以激活统计、权重属性和结构描述符的11维嵌入为条件，实现跨模型系列和尺度的zero shot传输。为了实现稳定的亚4位量化，我们引入了Scale Folding，这是一种预处理技术，通过每个通道的缩放和归一化层补偿将激活离群值迁移到权重中。质量优先的奖励与不对称的惩罚和预算悬崖驱动快速收敛。在Llama 2 7B上，RAMP在3.68 GB（3.65有效位）时达到5.54困惑度，在大小和质量上分别比均匀4位AWQ（3.90 GB时为5.60）和GPTQ高6%和1%至3%。关键的是，仅在Llama 2 7 B上训练的策略将zero shot推广到Llama 2 13 B和Mistral 7 B，通常超过目标特定训练，支持量化敏感性主要是建筑的假设。HALO管道将分配导出为GGUF格式，用于CPU，GPU和边缘设备上的无内核推理，保留了FP16常识推理性能的99.5%。
摘要 ：Post training quantization is essential for deploying large language models (LLMs) on resource constrained hardware, yet state of the art methods enforce uniform bit widths across layers, yielding suboptimal accuracy efficiency trade offs. We present RAMP (Reinforcement Adaptive Mixed Precision), an off policy Soft Actor Critic framework that learns per layer bit width assignments to minimize perplexity under a global bit budget. The policy conditions on an 11 dimensional embedding of activation statistics, weight properties, and structural descriptors, enabling zero shot transfer across model families and scales. To enable stable sub 4 bit quantization, we introduce Scale Folding, a preconditioning technique that migrates activation outliers into weights via per channel scaling and normalization layer compensation. A quality prioritized reward with asymmetric penalties and budget cliffs drives rapid convergence. On Llama 2 7B, RAMP achieves 5.54 perplexity at 3.68GB (3.65 effective bits), outperforming uniform 4 bit AWQ (5.60 at 3.90 GB) and GPTQ by 6% in size and 1% to3% in quality. Critically, a policy trained only on Llama 2 7B generalizes zero shot to Llama 2 13B and Mistral 7B, often surpassing target specific training, supporting the hypothesis that quantization sensitivity is primarily architectural. The HALO pipeline exports allocations to GGUF format for kernel free inference on CPUs, GPUs, and edge devices, retaining 99.5% of FP16 commonsense reasoning performance.
【4】How do LLMs Compute Verbal Confidence
标题：LLM如何计算口头信心
链接：https://arxiv.org/abs/2603.17839
备注：Submitted to ICML 2026
摘要：口头信心--促使LLM将他们的信心表述为一个数字或类别--被广泛用于从黑盒模型中提取不确定性估计。然而，LLM内部如何产生这样的分数仍然是未知的。我们要解决两个问题：第一，何时计算置信度--在请求时即时计算，或者在答案生成期间自动计算并缓存以供以后检索;第二，口头置信度代表什么--令牌对数概率，还是对答案质量的更丰富评估？针对Gemma 3 27 B和Qwen 2.5 7 B，我们提供了缓存检索的收敛证据.激活转向，修补，噪声，和交换实验表明，信心表示出现在答案附近的位置之前，出现在verbalization网站。注意力阻断精确定位信息流：从答案标记中收集置信度，缓存在第一个答案后位置，然后检索输出。重要的是，线性探测和方差分割显示，这些缓存的表示解释了大量的差异，在口头的信心超出令牌对数概率，这表明一个更丰富的答案质量的评估，而不是一个简单的流畅性读出。这些研究结果表明，口头自信反映了自动的，复杂的自我评价-而不是事后重建-与理解LLM元认知和提高校准的影响。
摘要：Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from black-box models. However, how LLMs internally generate such scores remains unknown. We address two questions: first, when confidence is computed - just-in-time when requested, or automatically during answer generation and cached for later retrieval; and second, what verbal confidence represents - token log-probabilities, or a richer evaluation of answer quality? Focusing on Gemma 3 27B and Qwen 2.5 7B, we provide convergent evidence for cached retrieval. Activation steering, patching, noising, and swap experiments reveal that confidence representations emerge at answer-adjacent positions before appearing at the verbalization site. Attention blocking pinpoints the information flow: confidence is gathered from answer tokens, cached at the first post-answer position, then retrieved for output. Critically, linear probing and variance partitioning reveal that these cached representations explain substantial variance in verbal confidence beyond token log-probabilities, suggesting a richer answer-quality evaluation rather than a simple fluency readout. These findings demonstrate that verbal confidence reflects automatic, sophisticated self-evaluation -- not post-hoc reconstruction -- with implications for understanding metacognition in LLMs and improving calibration.
【5】Discovering Decoupled Functional Modules in Large Language Models
标题：发现大型语言模型中的脱钩功能模块
链接：https://arxiv.org/abs/2603.17823
备注：AAAI-26 Oral
摘要：了解大型语言模型（LLM）的内部功能组织对于提高其可信度和性能至关重要。然而，LLM如何将不同的功能组织成模块仍然是高度未开发的。为了弥合这一差距，我们制定了一个功能模块发现问题，并提出了一个无监督LLM跨层MONITOR发现（ULCMOD）框架，同时将整个LLM中的大量神经元分解为模块，同时发现与这些模块相关的输入样本的主题。我们的框架引入了一个新的目标函数和一个有效的迭代解耦（IterD）算法。大量的实验表明，我们的方法发现了高质量的，分离的模块，捕获更有意义的语义信息，并在各种下游任务中实现卓越的性能。此外，我们的定性分析表明，所发现的模块显示语义一致性，对应于可解释的专业化，并在LLM内有一个明确的空间和层次组织。我们的工作提供了一个新的工具来解释LLM的功能模块，填补了LLM的可解释性研究的关键空白。
摘要：Understanding the internal functional organization of Large Language Models (LLMs) is crucial for improving their trustworthiness and performance. However, how LLMs organize different functions into modules remains highly unexplored. To bridge this gap, we formulate a functional module discovery problem and propose an Unsupervised LLM Cross-layer MOdule Discovery (ULCMOD) framework that simultaneously disentangles the large set of neurons in the entire LLM into modules while discovering the topics of input samples related to these modules. Our framework introduces a novel objective function and an efficient Iterative Decoupling (IterD) algorithm. Extensive experiments show that our method discovers high-quality, disentangled modules that capture more meaningful semantic information and achieve superior performance in various downstream tasks. Moreover, our qualitative analysis reveals that the discovered modules show semantic coherence, correspond to interpretable specializations, and a clear spatial and hierarchical organization within the LLM. Our work provides a novel tool for interpreting the functional modules of LLMs, filling a critical blank in LLM's interpretability research.
【6】Can Blindfolded LLMs Still Trade? An Anonymization-First Framework for Portfolio Optimization
标题：蒙住眼睛的LLM还能交易吗？一个以人为本的投资组合优化框架
链接：https://arxiv.org/abs/2603.17692
备注：Accepted at the ICLR 2026 Workshop on Advances in Financial AI (FinAI). 18 pages, 7 figures
摘要：对于LLM交易代理人来说，他们必须表现出对市场动态的理解，而不是利用记忆的股票代码。构建负责任的多智能体系统需要严格的信号验证：证明预测反映的是合法的模式，而不是预先训练的回忆。我们解决了两个来源的虚假性能：记忆偏差，从特定的预训练，和生存偏差，从有缺陷的回测。我们的方法是蒙住代理人的眼睛-匿名化所有标识符-并验证是否有意义的信号持续存在。BlindTrade将股票代码和公司名称匿名化，四个LLM代理人输出分数并进行推理。我们使用PPO-DSR策略从推理嵌入和交易中构建了一个GNN图。在2025年初至今（至2025年8月1日），我们在20个种子中实现了夏普1.40 +/- 0.22，并通过阴性对照实验验证了信号的合法性。为了评估单一OOS窗口之外的稳健性，我们还评估了一个延长的时期（2024- 2025年），揭示了市场制度的依赖性：该政策在波动条件下表现出色，但在趋势牛市中表现出降低的α。
摘要：For LLM trading agents to be genuinely trustworthy, they must demonstrate understanding of market dynamics rather than exploitation of memorized ticker associations. Building responsible multi-agent systems demands rigorous signal validation: proving that predictions reflect legitimate patterns, not pre-trained recall. We address two sources of spurious performance: memorization bias from ticker-specific pre-training, and survivorship bias from flawed backtesting. Our approach is to blindfold the agents--anonymizing all identifiers--and verify whether meaningful signals persist. BlindTrade anonymizes tickers and company names, and four LLM agents output scores along with reasoning. We construct a GNN graph from reasoning embeddings and trade using PPO-DSR policy. On 2025 YTD (through 2025-08-01), we achieved Sharpe 1.40 +/- 0.22 across 20 seeds and validated signal legitimacy through negative control experiments. To assess robustness beyond a single OOS window, we additionally evaluate an extended period (2024--2025), revealing market-regime dependency: the policy excels in volatile conditions but shows reduced alpha in trending bull markets.
【7】Sensi: Learn One Thing at a Time -- Curriculum-Based Test-Time Learning for LLM Game Agents
标题：Sensi：一次学一件事--LLM游戏代理基于课程的测试时学习
链接：https://arxiv.org/abs/2603.17683
备注：Preprint. 18 pages, 5 figures, 2 tables. Independent research. Code and Colab demo coming soon on GitHub
摘要 ：部署在未知环境中的大型语言模型（LLM）代理必须在测试时学习任务结构，但目前的方法需要数千次交互才能形成有用的假设。我们提出Sensi，一个用于ARC-AGI-3游戏挑战的LLM代理架构，通过三种机制引入结构化测试时学习：（1）将感知与动作分离的两个玩家架构，（2）由外部状态机管理的基于脚本的学习系统，以及（3）数据库作为控制平面，使代理上下文窗口可编程转向。我们进一步引入了一个法学硕士作为评判组件，该组件具有动态生成的评估规则，以确定代理何时对一个主题了解得足够多，可以进入下一个主题。我们报告了两次迭代的结果：Sensi v1仅使用两个玩家架构解决了2个游戏关卡，而Sensi v2增加了课程学习并解决了0个关卡-但在大约32次动作尝试中完成了整个学习课程，实现了比需要1600-3000次尝试的同类系统高50- 94倍的样本效率。我们精确地诊断故障模式为源自感知层的自洽幻觉级联，表明架构瓶颈已从学习效率转移到感知基础-一个更容易处理的问题。
摘要：Large language model (LLM) agents deployed in unknown environments must learn task structure at test time, but current approaches require thousands of interactions to form useful hypotheses. We present Sensi, an LLM agent architecture for the ARC-AGI-3 game-playing challenge that introduces structured test-time learning through three mechanisms: (1) a two-player architecture separating perception from action, (2) a curriculum-based learning system managed by an external state machine, and (3) a database-as-control-plane that makes the agents context window programmatically steerable. We further introduce an LLM-as-judge component with dynamically generated evaluation rubrics to determine when the agent has learned enough about one topic to advance to the next. We report results across two iterations: Sensi v1 solves 2 game levels using the two-player architecture alone, while Sensi v2 adds curriculum learning and solves 0 levels - but completes its entire learning curriculum in approximately 32 action attempts, achieving 50-94x greater sample efficiency than comparable systems that require 1600-3000 attempts. We precisely diagnose the failure mode as a self-consistent hallucination cascade originating in the perception layer, demonstrating that the architectural bottleneck has shifted from learning efficiency to perceptual grounding - a more tractable problem.
【8】HeiSD: Hybrid Speculative Decoding for Embodied Vision-Language-Action Models with Kinematic Awareness
标题：HeiSD：具有运动学意识的视觉-语言-动作模型的混合推测解码
链接：https://arxiv.org/abs/2603.17573
摘要：视觉-语言-动作（VLA）模型已经成为机器人控制的主流解决方案，但推理速度慢。推测解码是一种很有前途的加速方法，它可以分为两类：基于绘图的推测解码和基于检索的推测解码。现有的方法没有分析这两种SD在VLA模型中的优缺点，导致它们的单独应用或优化。在本文中，我们分析了由VLA模型控制的机器人的轨迹模式，并得出一个关键的见解：这两种类型的SD应该以混合的方式使用。然而，在VLA模型中实现混合SD提出了几个挑战：（1）基于检索的SD中的草稿拒绝和持续错误;（2）难以确定混合边界。为了解决这些问题，我们提出了HeiSD框架。在HeiSD中提出了一种基于检索的SD优化方法，该方法包含验证跳过机制和顺序放松接受策略。此外，我们提出了一个基于运动学的融合度量在HeiSD自动确定混合边界。实验结果表明，HeiSD在模拟基准测试中获得了高达2.45倍的加速比，在真实场景中获得了2.06倍~ 2.41倍的加速比，同时保持了较高的任务成功率。
摘要：Vision-Language-Action (VLA) Models have become the mainstream solution for robot control, but suffer from slow inference speeds. Speculative Decoding (SD) is a promising acceleration method which can be divided into two categories: drafter-based SD and retrieval-based SD. Existing methods fail to analyze the advantages and disadvantages of these two types of SD in VLA models, leading to their sole application or optimization. In this paper, we analyze the trajectory patterns of robots controlled by the VLA model and derive a key insight: the two types of SD should be used in a hybrid manner. However, achieving hybrid SD in VLA models poses several challenges: (1) draft rejection and persistent errors in retrieval-based SD; (2) difficulty in determining the hybrid boundary. To address these, we propose the HeiSD framework. We propose a retrieval-based SD optimization method in HeiSD,which contains a verify-skip mechanism and a sequence-wise relaxed acceptance strategy. Moreover, we proposed a kinematic-based fused metric in HeiSD to automatically determine the hybrid boundary. Experimental results demonstrate that HeiSD attains a speedup of up to 2.45x in simulation benchmarks and 2.06x~2.41x in real-world scenarios, while sustaining a high task success rate.
【9】A Unified Language Model for Large Scale Search, Recommendation, and Reasoning
标题：用于大规模搜索、推荐和推理的统一语言模型
链接：https://arxiv.org/abs/2603.17533
摘要：LLM越来越多地应用于推荐、检索和推理，但部署一个可以在大型异构目录上联合支持这些行为的单一端到端模型仍然具有挑战性。这样的系统必须生成对真实项目的明确引用，处理多个实体类型，并在严格的延迟和可靠性约束要求下操作，这些要求难以满足纯文本生成。虽然工具增强的推荐系统解决了这个问题的一部分，但它们引入了编排复杂性并限制了端到端的优化。我们将此设置视为一个更广泛的研究问题的实例：如何以完全独立的方式使LLM适应多域实体，用户和语言的联合推理。为此，我们引入了NEO，这是一个框架，它将预先训练的仅解码器LLM适配为一个免工具的，基于目录的生成器。NEO将项目表示为SID，并训练单个模型在共享序列中交错自然语言和类型化项目标识符。文本提示控制任务、目标实体类型和输出格式（ID、文本或混合），而受约束的解码可保证生成目录有效的项目，而不限制自由格式的文本。我们把这种有条件的可控性称为语言可操纵性。我们把SID作为一个独特的模式和研究设计选择集成到LLM通过阶段对齐和指令调整离散实体表示。我们对NEO进行了大规模的评估，评估对象是超过1000万个项目的真实目录，涉及多种媒体类型和发现任务，包括推荐、搜索和用户理解。在离线实验中，NEO始终优于强大的特定任务基线，并表现出跨任务传输，展示了将大规模发现能力整合到单一语言可操纵生成模型中的实用路径。
摘要：LLMs are increasingly applied to recommendation, retrieval, and reasoning, yet deploying a single end-to-end model that can jointly support these behaviors over large, heterogeneous catalogs remains challenging. Such systems must generate unambiguous references to real items, handle multiple entity types, and operate under strict latency and reliability constraints requirements that are difficult to satisfy with text-only generation. While tool-augmented recommender systems address parts of this problem, they introduce orchestration complexity and limit end-to-end optimization. We view this setting as an instance of a broader research problem: how to adapt LLMs to reason jointly over multiple-domain entities, users, and language in a fully self-contained manner. To this end, we introduce NEO, a framework that adapts a pre-trained decoder-only LLM into a tool-free, catalog-grounded generator. NEO represents items as SIDs and trains a single model to interleave natural language and typed item identifiers within a shared sequence. Text prompts control the task, target entity type, and output format (IDs, text, or mixed), while constrained decoding guarantees catalog-valid item generation without restricting free-form text. We refer to this instruction-conditioned controllability as language-steerability. We treat SIDs as a distinct modality and study design choices for integrating discrete entity representations into LLMs via staged alignment and instruction tuning. We evaluate NEO at scale on a real-world catalog of over 10M items across multiple media types and discovery tasks, including recommendation, search, and user understanding. In offline experiments, NEO consistently outperforms strong task-specific baselines and exhibits cross-task transfer, demonstrating a practical path toward consolidating large-scale discovery capabilities into a single language-steerable generative model.
【10】Learning When to Attend: Conditional Memory Access for Long-Context LLMs
标题：学习何时参加：长上下文LL的条件记忆访问
链接：https://arxiv.org/abs/2603.17484
备注：26 pages, 6 Tables, 18 Figures
摘要 ：语言模型很难在训练前的上下文长度之外进行泛化，从而限制了长视野推理和检索。在长上下文数据上继续进行预训练可能会有所帮助，但由于注意力的二次缩放而昂贵。我们观察到，大多数令牌在整个序列中不需要（全局）注意力，并且可以依赖于本地上下文。在此基础上，我们提出了L2A（Learning To Attend），这是一个通过决定何时调用全局注意力来实现有条件（令牌式）长距离记忆访问的层。我们在Qwen 2.5和Qwen 3模型上评估L2A，将其有效上下文长度从32K扩展到128K令牌。L2A将标准长上下文训练的性能匹配到3%以内，同时跳过80%的全局注意力，优于之前的基线。我们还设计了自定义Triton内核，以在GPU上有效地实现这种令牌式条件注意力，在训练吞吐量和首次令牌时间方面比FlashAttention提高了2倍。此外，L2A支持高度稀疏的全局注意力层的训练后修剪，将KV缓存内存减少高达50%，性能损失可以忽略不计。
摘要：Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval. Continued pretraining on long-context data can help but is expensive due to the quadratic scaling of Attention. We observe that most tokens do not require (Global) Attention over the entire sequence and can rely on local context. Based on this, we propose L2A (Learning To Attend), a layer that enables conditional (token-wise) long-range memory access by deciding when to invoke global attention. We evaluate L2A on Qwen 2.5 and Qwen 3 models, extending their effective context length from 32K to 128K tokens. L2A matches the performance of standard long-context training to within 3% while skipping Global Attention for $\sim$80% of tokens, outperforming prior baselines. We also design custom Triton kernels to efficiently implement this token-wise conditional Attention on GPUs, achieving up to $\sim$2x improvements in training throughput and time-to-first-token over FlashAttention. Moreover, L2A enables post-training pruning of highly sparse Global Attention layers, reducing KV cache memory by up to 50% with negligible performance loss.
【11】Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control
标题：高效的软演员评论家，采用基于LLM的连续控制指导
链接：https://arxiv.org/abs/2603.17468
摘要：我们提出了GuidedSAC，一种新的强化学习（RL）算法，有助于在广阔的状态-动作空间中进行有效的探索。GuidedSAC利用大型语言模型（LLM）作为智能监督程序，为Soft Actor-Critic（SAC）算法提供操作级指导。基于LLM的主管使用状态信息和视觉回放分析最新的轨迹，提供能够进行有针对性的探索的行动级干预。此外，我们还对GuidedSAC进行了理论分析，证明它在提高收敛速度的同时保留了SAC的收敛保证。通过在离散和连续控制环境中的实验，包括玩具文本任务和复杂的MuJoCo基准测试，我们证明了GuidedSAC始终优于标准SAC和最先进的探索增强变体（例如，RND、ICM和E3 B）在样品效率和最终性能方面的表现。
摘要：We present GuidedSAC, a novel reinforcement learning (RL) algorithm that facilitates efficient exploration in vast state-action spaces. GuidedSAC leverages large language models (LLMs) as intelligent supervisors that provide action-level guidance for the Soft Actor-Critic (SAC) algorithm. The LLM-based supervisor analyzes the most recent trajectory using state information and visual replays, offering action-level interventions that enable targeted exploration. Furthermore, we provide a theoretical analysis of GuidedSAC, proving that it preserves the convergence guarantees of SAC while improving convergence speed. Through experiments in both discrete and continuous control environments, including toy text tasks and complex MuJoCo benchmarks, we demonstrate that GuidedSAC consistently outperforms standard SAC and state-of-the-art exploration-enhanced variants (e.g., RND, ICM, and E3B) in terms of sample efficiency and final performance.
【12】ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
标题：ZipServv：快速且内存高效的LLM推理，具有硬件感知无损压缩
链接：https://arxiv.org/abs/2603.17435
备注：ASPLOS'26 Accepted Paper
摘要：无损模型压缩对于缓解位精确大型语言模型（LLM）服务中的内存和带宽瓶颈具有巨大的希望。然而，现有的方法往往会导致大量的推理速度减慢，由于基本的设计与GPU架构不匹配：在内核级，由传统的熵编解码器产生的可变长度的比特流打破SIMT并行;在系统级，解耦的流水线导致冗余的内存流量。我们提出了ZipServ，一个无损压缩框架共同设计的高效LLM推理。ZipServ引入了Tensor-Core-Aware Triple Bitmap Encoding（TCA-TBE），这是一种新颖的固定长度格式，可以实现恒定时间的并行解码，以及融合的解压缩GEMM（ZipGEMM）内核，可以将权重直接动态解压缩到Tensor Core寄存器中。这种“负载压缩、计算解压”设计消除了中间缓冲区并最大化了计算强度。实验表明，ZipServ将模型大小减少了30%，实现了比NVIDIA的cuBLAS高2.21倍的内核级加速，并将端到端推理速度平均提高了vLLM的1.22倍。ZipServ是第一个无损压缩系统，它为GPU上的LLM推理提供了存储节省和大幅加速。
摘要：Lossless model compression holds tremendous promise for alleviating the memory and bandwidth bottlenecks in bit-exact Large Language Model (LLM) serving. However, existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures: at the kernel level, variable-length bitstreams produced by traditional entropy codecs break SIMT parallelism; at the system level, decoupled pipelines lead to redundant memory traffic. We present ZipServ, a lossless compression framework co-designed for efficient LLM inference. ZipServ introduces Tensor-Core-Aware Triple Bitmap Encoding (TCA-TBE), a novel fixed-length format that enables constant-time, parallel decoding, together with a fused decompression-GEMM (ZipGEMM) kernel that decompresses weights on-the-fly directly into Tensor Core registers. This "load-compressed, compute-decompressed" design eliminates intermediate buffers and maximizes compute intensity. Experiments show that ZipServ reduces the model size by up to 30%, achieves up to 2.21x kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22x over vLLM. ZipServ is the first lossless compression system that provides both storage savings and substantial acceleration for LLM inference on GPUs.
【13】TharuChat: Bootstrapping Large Language Models for a Low-Resource Language via Synthetic Data and Human Validation
标题：TharuChat：通过合成数据和人工验证为低资源语言引导大型语言模型
链接：https://arxiv.org/abs/2603.17220
备注：6 pages, 1 figure, 2 tables. Preprint. Code and dataset available on Hugging Face
摘要：大型语言模型（LLM）的迅速扩散造成了深刻的数字鸿沟，有效地将全球南方的土著语言排除在人工智能革命之外。塔鲁语是一种印度-雅利安方言，在尼泊尔和印度的特莱地区有大约170万人使用，它加剧了这一危机。尽管有丰富的口头传统，但Tharu遭受严重的数据稀缺和语言碎片化，导致最先进的多语言模型经常“幻觉”或默认为主要的高资源邻居，如印地语和尼泊尔语，因为预训练语料库中的污染。 本文介绍了Tharu-LLaMA（3B），一个专门的预防跟踪模型，旨在解决这种排斥。我们介绍了TharuChat，这是一个通过LLM到人类引导管道构建的新型数据集。我们使用了人工设计的Gemini模型，并加入了Rana Tharu语法和民间传说，以合成训练数据。与精心策划的黄金标准语料库不同，TharuChat反映了该地区嘈杂、异质的语言现实：它主要以拉纳·塔鲁（Rana Tharu）（约70%）为基础，同时融合了丹古拉（Dangaura）和科奇拉（Kochila）方言的元素。我们提供了一个透明的分析数据集的局限性，包括方言代码混合和残留的阿瓦地语/印地语的影响。通过严格的经验消融研究，我们证明，尽管有这些缺陷，小规模的合成数据是非常有效的，增加数据集的体积从25%到100%的结果在一个线性减少困惑从6.42到2.88。由此产生的模型可以作为概念验证，通过生成式人工智能保护资源不足的喜马拉雅语言，可在消费级硬件上实现。
摘要 ：The rapid proliferation of Large Language Models (LLMs) has created a profound digital divide, effectively excluding indigenous languages of the Global South from the AI revolution. The Tharu language, an Indo-Aryan vernacular spoken by approximately 1.7 million people across the Terai belt of Nepal and India, exemplifies this crisis. Despite a rich oral tradition, Tharu suffers from severe data scarcity and linguistic fragmentation, causing state-of-the-art multilingual models to routinely "hallucinate" or default to dominant high-resource neighbors like Hindi and Nepali due to contamination in pre-training corpora. This paper presents Tharu-LLaMA (3B), a specialized instruction-following model designed to address this exclusion. We introduce TharuChat, a novel dataset constructed via a LLM-to-Human bootstrapping pipeline. We utilized prompt-engineered Gemini models, fed with Rana Tharu grammar and folklore, to synthesize training data. Unlike curated gold-standard corpora, TharuChat reflects the noisy, heterogeneous linguistic reality of the region: it is predominantly anchored in Rana Tharu (~70%) while integrating elements of Dangaura and Kochila dialects. We provide a transparent analysis of the dataset's limitations, including dialectal code-mixing and residual Awadhi/Hindi influence. Through a rigorous empirical ablation study, we demonstrate that despite these imperfections, small-scale synthetic data is highly effective, increasing the dataset volume from 25% to 100% results in a linear reduction in perplexity from 6.42 to 2.88. The resulting model serves as a proof-of-concept for the preservation of under-resourced Himalayan languages via generative AI, achievable on consumer-grade hardware.
【14】Anonymous-by-Construction: An LLM-Driven Framework for Privacy-Preserving Text
标题：构建匿名：LLM驱动的隐私保护文本框架
链接：https://arxiv.org/abs/2603.17217
摘要：负责任地使用人工智能要求我们在不破坏数据有用性的情况下保护敏感信息，这在大型语言模型时代变得非常重要。我们通过一个LLM驱动的替代管道来解决这一挑战，该管道通过将个人身份信息（PII）替换为现实的，类型一致的代理人来匿名化文本。该方法完全在使用本地LLM的组织边界内执行，在保持流畅性和任务相关语义的同时防止数据流出。 我们对基于XML的会话数据集进行了系统的，多指标的，跨技术的评估，对行业标准（Microsoft Presidio和Google DLP）和最先进的方法（ZSTS，仅编辑和编辑加替换变体）进行了基准测试。我们的协议共同衡量隐私，语义效用，和可训练性的隐私下，通过一个生命周期准备的标准，通过微调一个紧凑的编码器（BERT+LoRA）上消毒的文本。此外，我们通过在回答LLM之前插入内部匿名层并评估其响应的质量来评估代理问答性能。这个中间的类型保留替换阶段确保没有敏感内容暴露给第三方API，从而在不影响机密性的情况下实现负责任的Q\&A代理部署。 我们的方法达到了最先进的隐私，最小的主题漂移，强大的事实效用和低可训练性损失，优于基于规则的方法和命名实体识别（NER）基线和ZSTS变体的组合隐私-效用-可训练性边界。这些结果表明，本地LLM替代产生匿名语料库，既负责使用和操作价值：安全的代理管道，并适合下游微调有限的退化。
摘要：Responsible use of AI demands that we protect sensitive information without undermining the usefulness of data, an imperative that has become acute in the age of large language models. We address this challenge with an on-premise, LLM-driven substitution pipeline that anonymizes text by replacing personally identifiable information (PII) with realistic, type-consistent surrogates. Executed entirely within organizational boundaries using local LLMs, the approach prevents data egress while preserving fluency and task-relevant semantics. We conduct a systematic, multi-metric, cross-technique evaluation on the Action-Based Conversation Dataset, benchmarking against industry standards (Microsoft Presidio and Google DLP) and a state-of-the-art approach (ZSTS, in redaction-only and redaction-plus-substitution variants). Our protocol jointly measures privacy, semantic utility, and trainability under privacy via a lifecycle-ready criterion obtained by fine-tuning a compact encoder (BERT+LoRA) on sanitized text. In addition, we assess agentic Q&A performance by inserting an on-premise anonymization layer before the answering LLM and evaluating the quality of its responses. This intermediate, type-preserving substitution stage ensures that no sensitive content is exposed to third-party APIs, enabling responsible deployment of Q\&A agents without compromising confidentiality. Our method attains state-of-the-art privacy, minimal topical drift, strong factual utility, and low trainability loss, outperforming rule-based approaches and named-entity recognition (NER) baselines and ZSTS variants on the combined privacy--utility--trainability frontier. These results show that local LLM substitution yields anonymized corpora that are both responsible to use and operationally valuable: safe for agentic pipelines and suitable for downstream fine-tuning with limited degradation.
【15】Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Data
标题：基于多模态生物医学数据的可解释的少次阿尔茨海默病预测的表格LLM
链接：https://arxiv.org/abs/2603.17191
摘要：阿尔茨海默病（AD）的准确诊断需要处理表格生物标志物数据，但这些数据通常很小且不完整，深度学习模型往往无法优于经典方法。预训练的大型语言模型（LLM）提供了Few-Shot泛化、结构化推理和可解释的输出，为临床预测提供了强大的范式转变。我们提出了TAP-GPT表格式阿尔茨海默病预测GPT，这是一个基于TableGPT 2构建的领域适应性表格式LLM框架，并使用表格提示而不是纯文本进行了微调，用于Few-Shot AD分类。我们在四个ADNI衍生的数据集上评估了TAP-GPT，包括QT-PAD生物标志物和区域水平的结构MRI，淀粉样蛋白PET和tau PET用于二元AD分类。在多模态和单峰设置中，TAP-GPT改进了其骨干模型，并在Few-Shot设置中优于传统的机器学习基线，同时与最先进的通用LLM保持竞争力。我们发现，特征选择减轻了高维输入的退化，并且TAP-GPT在模拟和真实世界的缺失情况下保持稳定的性能，而无需插补。此外，TAP-GPT产生与已建立的AD生物学相一致的结构化，模态感知的推理，并在自我反思下显示出更大的稳定性，支持其在迭代多智能体系统中的使用。据我们所知，这是表格式专业化LLM首次系统地应用于基于生物标志物的多模式AD预测，表明此类预训练模型可以有效地解决结构化临床预测任务，并为表格式LLM驱动的多主体临床决策奠定了基础支持系统。源代码在GitHub上公开：https://github.com/sophie-kearney/TAP-GPT。
摘要：Accurate diagnosis of Alzheimer's disease (AD) requires handling tabular biomarker data, yet such data are often small and incomplete, where deep learning models frequently fail to outperform classical methods. Pretrained large language models (LLMs) offer few-shot generalization, structured reasoning, and interpretable outputs, providing a powerful paradigm shift for clinical prediction. We propose TAP-GPT Tabular Alzheimer's Prediction GPT, a domain-adapted tabular LLM framework built on TableGPT2 and fine-tuned for few-shot AD classification using tabular prompts rather than plain texts. We evaluate TAP-GPT across four ADNI-derived datasets, including QT-PAD biomarkers and region-level structural MRI, amyloid PET, and tau PET for binary AD classification. Across multimodal and unimodal settings, TAP-GPT improves upon its backbone models and outperforms traditional machine learning baselines in the few-shot setting while remaining competitive with state-of-the-art general-purpose LLMs. We show that feature selection mitigates degradation in high-dimensional inputs and that TAP-GPT maintains stable performance under simulated and real-world missingness without imputation. Additionally, TAP-GPT produces structured, modality-aware reasoning aligned with established AD biology and shows greater stability under self-reflection, supporting its use in iterative multi-agent systems. To our knowledge, this is the first systematic application of a tabular-specialized LLM to multimodal biomarker-based AD prediction, demonstrating that such pretrained models can effectively address structured clinical prediction tasks and laying the foundation for tabular LLM-driven multi-agent clinical decision-support systems. The source code is publicly available on GitHub: https://github.com/sophie-kearney/TAP-GPT.
【16】Noise-Response Calibration: A Causal Intervention Protocol for LLM-Judges
标题：噪音响应校准：LLM法官的因果干预协议
链接：https://arxiv.org/abs/2603.17172
备注：Published as a conference paper at CAO Workshop at ICLR 2026
摘要：大型语言模型（LLM）越来越多地被用作自动判断和合成标记器，特别是在低标签环境中。然而，这些系统是随机的，往往过于自信，这使得部署决策困难时，外部地面真理是有限的。我们提出了一个实用的校准协议的基础上控制输入干预：如果噪声的严重程度增加，任务性能应该表现出统计上显着的恶化趋势。我们可操作的斜率为基础的假设检验在重复的试验，使用信号噪声比（SNR）扰动的表格数据和文字数据的词汇扰动。在UCI表格基准和四个文本分类数据集上，我们发现了明显的模态依赖行为。我们的研究结果揭示了一个模态差距：虽然基于文本的判断可预测地降低，但大多数表格数据集显示，即使在显著的信噪比降低的情况下，也没有统计学上显著的性能恶化。有趣的是，我们发现模型性能在对噪声干预不敏感的数据集上较低。我们提出了一个可重复的方法和报告协议，强大的LLM-判断校准下的分布偏移。
摘要 ：Large language models (LLMs) are increasingly used as automated judges and synthetic labelers, especially in low-label settings. Yet these systems are stochastic and often overconfident, which makes deployment decisions difficult when external ground truth is limited. We propose a practical calibration protocol based on controlled input interventions: if noise severity increases, task performance should exhibit a statistically significant deterioration trend. We operationalize this with a slope-based hypothesis test over repeated trials, using signal-to-noise-ratio (SNR) perturbations for tabular data and lexical perturbations for text data. Across UCI tabular benchmarks and four text classification datasets, we find clear modality-dependent behavior. Our results reveal a modality gap: while text-based judges degrade predictably, the majority of tabular datasets show a lack of statistically significant performance deterioration even under significant signal-to-noise reduction. Interestingly we find that model performance is lower on datasets that are insensitive to noise interventions. We present a reproducible methodology and reporting protocol for robust LLM-judge calibration under distribution shift.
【17】REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge
标题：REAL：回归感知强化学习用于LLM-as-a-Judge
链接：https://arxiv.org/abs/2603.17145
摘要：大型语言模型（LLM）越来越多地被部署为自动评估器，为模型输出分配数字分数，这种范式被称为LLM-as-a-Judge。然而，标准强化学习（RL）方法通常依赖于二进制奖励（例如，0-1准确性），从而忽略了回归任务中固有的顺序结构;例如，当地面真值为5时，他们无法认识到预测4明显优于预测1。相反，现有的回归感知方法通常局限于监督微调（SFT），限制了它们探索最佳推理路径的能力。为了弥补这一差距，我们提出了\textbf{REAL}（\underline{RE}回归-\underline{A}ware Reinforcement \underline{L}earning），这是一个原则性的强化学习框架，旨在优化回归奖励，并且也被证明是相关性指标的最佳选择。一个关键的技术挑战是回归目标是明确的政策依赖，从而使标准的政策梯度方法无效。为了解决这个问题，我们采用了广义策略梯度估计器，它自然地将优化分解为两个互补的组成部分：（1）对思想链（CoT）轨迹的探索，以及（2）最终得分的回归感知预测细化。跨模型规模（8B到32 B）的广泛实验表明，REAL始终优于回归感知的SFT基线和标准RL方法，在域外基准测试中表现出更好的泛化能力。特别是在Qwen 3 - 32 B上，我们在SFT基线上实现了+8.40 Pearson和+7.20 Spearman相关性，在基础模型上实现了+18.30/+11.20。这些发现强调了将回归目标整合到RL探索中以进行准确的LLM评估的关键价值。
摘要：Large language models (LLMs) are increasingly deployed as automated evaluators that assign numeric scores to model outputs, a paradigm known as LLM-as-a-Judge. However, standard Reinforcement Learning (RL) methods typically rely on binary rewards (e.g., 0-1 accuracy), thereby ignoring the ordinal structure inherent in regression tasks; for instance, they fail to recognize that predicting 4 is significantly better than predicting 1 when the ground truth is 5. Conversely, existing regression-aware approaches are often confined to Supervised Fine-Tuning (SFT), limiting their ability to explore optimal reasoning paths. To bridge this gap, we propose \textbf{REAL} (\underline{RE}gression-\underline{A}ware Reinforcement \underline{L}earning), a principled RL framework designed to optimize regression rewards, and also proven to be optimal for correlation metrics. A key technical challenge is that the regression objective is explicitly policy-dependent, thus invalidating standard policy gradient methods. To address this, we employ the generalized policy gradient estimator, which naturally decomposes optimization into two complementary components: (1) exploration over Chain-of-Thought (CoT) trajectory, and (2) regression-aware prediction refinement of the final score. Extensive experiments across model scales (8B to 32B) demonstrate that REAL consistently outperforms both regression-aware SFT baselines and standard RL methods, exhibiting significantly better generalization on out-of-domain benchmarks. On Qwen3-32B specifically, we achieve gains of +8.40 Pearson and +7.20 Spearman correlation over the SFT baseline, and +18.30/+11.20 over the base model. These findings highlight the critical value of integrating regression objectives into RL exploration for accurate LLM evaluation.
【18】Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency
标题：使用跨语言不一致的混合专家LL中的知识本地化
链接：https://arxiv.org/abs/2603.17102
摘要：现代LLM继续在不同语言中表现出显着的行为差异，例如能够回忆某些语言中的事实信息，而不是其他语言。虽然通常研究作为一个问题，以减轻，在这项工作中，我们建议利用这种跨语言的不一致性作为一种工具，在混合专家（MoE）LLM的可解释性。我们的知识本地化框架对比了语言集的路由，其中模型正确地从失败的语言中调用信息。这使我们能够隔离在回答一段知识时扮演功能角色的模型组件。我们的方法分两个阶段进行：（1）查询模型与困难的事实问题，在不同的语言集，以产生“成功”和“失败”的激活桶，然后（2）应用统计对比分析的MoE路由器logits，以确定专家的重要知识。为了验证这一小部分专家回答知识问题的必要性，我们将其停用并重新提出问题。我们发现，尽管在6000名专家中只停用了大约20名，但该模型在超过40%的情况下不再正确回答。一般来说，这种方法提供了一个现实的和可扩展的知识本地化的方法，以解决日益复杂的LLM。
摘要：Modern LLMs continue to exhibit significant variance in behavior across languages, such as being able to recall factual information in some languages but not others. While typically studied as a problem to be mitigated, in this work, we propose leveraging this cross-lingual inconsistency as a tool for interpretability in mixture-of-experts (MoE) LLMs. Our knowledge localization framework contrasts routing for sets of languages where the model correctly recalls information from languages where it fails. This allows us to isolate model components that play a functional role in answering about a piece of knowledge. Our method proceeds in two stages: (1) querying the model with difficult factual questions across a diverse set of languages to generate "success" and "failure" activation buckets and then (2) applying a statistical contrastive analysis to the MoE router logits to identify experts important for knowledge. To validate the necessity of this small number of experts for answering a knowledge question, we deactivate them and re-ask the question. We find that despite only deactivating about 20 out of 6000 experts, the model no longer answers correctly in over 40% of cases. Generally, this method provides a realistic and scalable knowledge localization approach to address increasingly complex LLMs.
【19】Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback
标题：注意到观察者：LLM代理可以从阻止反馈中推断CoT监控
链接：https://arxiv.org/abs/2603.16928
摘要：思想链（CoT）监控是一种监督语言模型代理内部推理的方法。先前的工作表明，当模型被明确告知他们的推理正在被监控，或者被微调以内化这一事实时，他们可能会学会以使他们能够逃避基于CoT的监控系统的方式混淆他们的CoT。我们问推理代理是否可以自主推断他们所谓的私人CoT受到监视，以及这种意识是否会导致战略规避，而无需任何明确的培训或指示。在多情节代理框架中，模型在被告知他们的推理是私人的同时追求主要任务和隐藏的副任务;当检测到可疑推理时，隐藏的CoT监视器阻止情节。我们发现，前沿模型可以纯粹从阻塞反馈中推断出这种监视器的存在，最有能力的模型可以自信地相信，他们的想法在高达19%的事件中被观察到。这种意识随着模型能力的变化而变化，在极少数情况下，会升级为抑制对副任务的推理的明确意图。然而，形成这种意图的模型都无法执行它，在下一集公开推理他们隐藏的目标。这种意图-能力差距对于当前的部署来说是令人放心的，但是监控意识和规避意图的自主出现表明CoT监控不是永久可靠的保障。
摘要 ：Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal reasoning of language-model agents. Prior work has shown that when models are explicitly informed that their reasoning is being monitored, or are fine-tuned to internalize this fact, they may learn to obfuscate their CoTs in ways that allow them to evade CoT-based monitoring systems. We ask whether reasoning agents can autonomously infer that their supposedly private CoT is under surveillance, and whether this awareness leads to strategic evasion, without any explicit training or instructions. In a multi-episode agentic framework, models pursue both a primary task and a concealed side task while being told that their reasoning is private; a hidden CoT monitor blocks episodes when suspicious reasoning is detected. We find that frontier models can deduce the existence of this monitor purely from blocking feedback, with the most capable models reaching a confident belief that their thinking is observed in up to 19% of episodes. This awareness scales with model capability and, in rare cases, escalates to an explicit intent to suppress reasoning about the side task. However, models that form this intent uniformly fail to execute it, openly reasoning about their concealed objectives in the very next episode. This intent-capability gap is reassuring for current deployment, but the autonomous emergence of both monitoring awareness and evasion intent suggests that CoT monitoring is not a permanently reliable safeguard.
【20】EEG-Based Brain-LLM Interface for Human Preference Aligned Generation
标题：用于人类偏好一致生成的基于脑电波的Brain-LLM接口
链接：https://arxiv.org/abs/2603.16897
备注：15 pages, 9 figures
摘要：大型语言模型（LLM）正在成为人机交互中越来越重要的组成部分，使用户能够通过自然语言协调各种智能代理。虽然基于语言的界面功能强大且灵活，但它们隐含地假设用户可以可靠地产生明确的语言输入，但这一假设可能不适用于有语言或运动障碍的用户，例如，肌萎缩侧索硬化症（ALS）。在这项工作中，我们研究神经信号是否可以作为LLM的替代输入，特别是支持那些社会边缘化或服务不足的用户。我们建立了一个简单的大脑LLM接口，它使用EEG信号在测试时指导图像生成模型。具体来说，我们首先训练分类器，以估计用户满意度从EEG信号。然后，它的预测被纳入测试时间缩放（TTS）框架，该框架使用在用户评估期间收集的神经反馈动态地适应模型推理。实验表明，脑电可以预测用户的满意度，这表明神经活动携带实时偏好推理的信息。这些发现为将神经反馈整合到自适应语言模型推理中迈出了第一步，并有望为未来自适应LLM交互的研究开辟新的可能性。
摘要：Large language models (LLMs) are becoming an increasingly important component of human--computer interaction, enabling users to coordinate a wide range of intelligent agents through natural language. While language-based interfaces are powerful and flexible, they implicitly assume that users can reliably produce explicit linguistic input, an assumption that may not hold for users with speech or motor impairments, e.g., Amyotrophic Lateral Sclerosis (ALS). In this work, we investigate whether neural signals can be used as an alternative input to LLMs, particularly to support those socially marginalized or underserved users. We build a simple brain-LLM interface, which uses EEG signals to guide image generation models at test time. Specifically, we first train a classifier to estimate user satisfaction from EEG signals. Its predictions are then incorporated into a test-time scaling (TTS) framework that dynamically adapts model inference using neural feedback collected during user evaluation. The experiments show that EEG can predict user satisfaction, suggesting that neural activity carries information on real-time preference inference. These findings provide a first step toward integrating neural feedback into adaptive language-model inference, and hopefully open up new possibilities for future research on adaptive LLM interaction.
Graph相关(图学习|图神经网络|图优化等)(4篇)
【1】An End-to-End Framework for Functionality-Embedded Provenance Graph Construction and Threat Interpretation
标题：功能嵌入源图构建和威胁解释的端到端框架
链接：https://arxiv.org/abs/2603.17100
备注：21 pages, 7 figures
摘要：起源图从日志中对因果系统级交互进行建模，使异常检测器能够学习正常行为并将偏差检测为攻击。然而，现有的方法依赖于脆弱的，手动设计的规则来构建起源图，缺乏系统实体的功能上下文，并为分析师调查提供有限的支持。我们提出了Auto-Prov，这是一个自适应的端到端框架，它利用大型语言模型（LLM）从异构和不断发展的日志中自动构建起源图，将系统级功能属性嵌入到图中，使基于起源图的异常检测器能够从这些丰富的图中学习，并总结检测到的攻击以协助分析师的调查。Auto-Prov对不可见的日志类型进行聚类，并通过自动生成的规则有效地提取出处边缘和实体级信息。它还使用LLM推断和基于行为的估计的组合来推断已知和先前未见过的系统实体的系统级功能上下文。然后，在Auto-Prov的图上训练的基于起源图的异常检测器检测到的攻击被总结为自然语言文本。我们使用四种最先进的基于来源图的检测器在不同的日志中评估Auto-Prov。结果表明，Auto-Prov始终增强检测性能，在异构日志格式中进行推广，并生成稳定的、可解释的攻击摘要，这些摘要在系统演化下保持稳健。
摘要：Provenance graphs model causal system-level interactions from logs, enabling anomaly detectors to learn normal behavior and detect deviations as attacks. However, existing approaches rely on brittle, manually engineered rules to build provenance graphs, lack functional context for system entities, and provide limited support for analyst investigation. We present Auto-Prov, an adaptive, end-to-end framework that leverages large language models (LLMs) to automatically construct provenance graphs from heterogeneous and evolving logs, embed system-level functional attributes into the graph, enable provenance graph-based anomaly detectors to learn from these enriched graphs, and summarize the detected attacks to assist an analyst's investigation. Auto-Prov clusters unseen log types and efficiently extracts provenance edges and entity-level information via automatically generated rules. It further infers system-level functional context for both known and previously unseen system entities using a combination of LLM inference and behavior-based estimation. Attacks detected by provenance-graph-based anomaly detectors trained on Auto-Prov's graphs are then summarized into natural-language text. We evaluate Auto-Prov with four state-of-the-art provenance graph-based detectors across diverse logs. Results show that Auto-Prov consistently enhances detection performance, generalizes across heterogeneous log formats, and produces stable, interpretable attack summaries that remain robust under system evolution.
【2】PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow with Continual Learning
标题：PowerModersGAT-AI：通过持续学习关注多系统功率流的物理知识图表
链接：https://arxiv.org/abs/2603.16879
备注：26 pages, 11 figures, 1 ancillary supplementary PDF
摘要：实时求解交流潮流方程对电网安全运行至关重要，但经典的牛顿-拉夫森求解器在压力条件下可能会很慢。现有的电力潮流图神经网络通常在单个系统上训练，并且经常在不同的系统上降级。我们提出了PowerModelsGAT-AI，这是一个物理信息图形注意力网络，可以预测总线电压和发电机注入。该模型使用总线类型感知掩蔽来处理不同的总线类型，并使用学习的权重来平衡多个损耗项，包括功率失配惩罚。我们在14个基准系统（4到6，470个母线）上评估了模型，并在N-2（双支路停电）条件下对其中13个系统进行了统一模型的训练，电压幅值的平均归一化平均绝对误差为0.89%，电压角的R^2 > 0.99。我们还展示了持续学习：当将基础模型适应新的1，354总线系统时，标准微调会导致严重的遗忘，误差增加超过1000%，而我们的经验重放和弹性权重合并策略将误差增加保持在2%以下，并在某些情况下提高了基础系统的性能。可解释性分析表明，学习的注意力权重与物理分支参数（磁阻：r = 0.38;热限制：r = 0.22），和功能的重要性分析支持该模型捕捉建立的潮流关系。
摘要：Solving the alternating current power flow equations in real time is essential for secure grid operation, yet classical Newton-Raphson solvers can be slow under stressed conditions. Existing graph neural networks for power flow are typically trained on a single system and often degrade on different systems. We present PowerModelsGAT-AI, a physics-informed graph attention network that predicts bus voltages and generator injections. The model uses bus-type-aware masking to handle different bus types and balances multiple loss terms, including a power-mismatch penalty, using learned weights. We evaluate the model on 14 benchmark systems (4 to 6,470 buses) and train a unified model on 13 of these under N-2 (two-branch outage) conditions, achieving an average normalized mean absolute error of 0.89% for voltage magnitudes and R^2 > 0.99 for voltage angles. We also show continual learning: when adapting a base model to a new 1,354-bus system, standard fine-tuning causes severe forgetting with error increases exceeding 1000% on base systems, while our experience replay and elastic weight consolidation strategy keeps error increases below 2% and in some cases improves base-system performance. Interpretability analysis shows that learned attention weights correlate with physical branch parameters (susceptance: r = 0.38; thermal limits: r = 0.22), and feature importance analysis supports that the model captures established power flow relationships.
【3】Gaussian Process Limit Reveals Structural Benefits of Graph Transformers
标题：高斯过程限制揭示了图形变形器的结构优势
链接：https://arxiv.org/abs/2603.17569
摘要：图Transformers是从图结构数据中学习的最先进技术，并且经验上已知可以避免消息传递架构的几个陷阱。然而，对于这些模型在实践中表现良好的原因，理论分析有限。在这项工作中，我们证明了在节点级预测任务的背景下，基于注意力的架构比图卷积网络具有结构上的优势。具体来说，我们研究了具有无限宽度和无限头的图Transformers（GAT，Graphormer，Specformer）的神经网络高斯过程限制，并推导出跨层的节点级和边缘级内核。我们的结果说明了节点特征和图结构如何通过图注意层传播。作为一个具体的例子，我们证明了图Transformers在结构上保持社区信息，并保持歧视性的节点表示，即使在深层，从而防止过度平滑。我们提供了关于合成和真实世界图的经验证据，验证了我们的理论见解，例如整合信息先验和位置编码可以提高深度图Transformers的性能。
摘要：Graph transformers are the state-of-the-art for learning from graph-structured data and are empirically known to avoid several pitfalls of message-passing architectures. However, there is limited theoretical analysis on why these models perform well in practice. In this work, we prove that attention-based architectures have structural benefits over graph convolutional networks in the context of node-level prediction tasks. Specifically, we study the neural network gaussian process limits of graph transformers (GAT, Graphormer, Specformer) with infinite width and infinite heads, and derive the node-level and edge-level kernels across the layers. Our results characterise how the node features and the graph structure propagate through the graph attention layers. As a specific example, we prove that graph transformers structurally preserve community information and maintain discriminative node representations even in deep layers, thereby preventing oversmoothing. We provide empirical evidence on synthetic and real-world graphs that validate our theoretical insights, such as integrating informative priors and positional encoding can improve performance of deep graph transformers.
【4】EEG-SeeGraph: Interpreting functional connectivity disruptions in dementias via sparse-explanatory dynamic EEG-graph learning
标题：EEG-SeeShape：通过稀疏解释的动态脑图学习来解释痴呆症的功能连接中断
链接：https://arxiv.org/abs/2603.16895
摘要：从噪声、非平稳脑电图（EEG）中进行可靠且可解释的痴呆诊断在临床上是必不可少的，但仍具有挑战性。为此，我们提出了SeeGraph，一个稀疏解释动态脑电图图网络，模型随时间变化的功能连接，并采用节点引导的稀疏边缘掩码来揭示驱动诊断决策的连接，同时保持对噪声和跨站点变化的鲁棒性。SeeGraph包括四个组件：（1）双轨迹时间编码器，其对具有两个流的动态EEG进行建模，其中节点信号捕获区域振荡，边缘信号捕获区域间耦合;（2）拓扑感知位置编码器，其从融合连接性导出图谱拉普拉斯坐标并增强节点嵌入;（3）节点引导的稀疏解释性边缘掩码，其将连通性选通到紧凑子图中;以及（4）选通图预测器，其对稀疏化图进行操作。该框架使用交叉熵损失以及掩码上的稀疏正则化器进行训练，从而产生噪声鲁棒性和可解释性诊断。SeeGraph的有效性在公共和内部EEG队列中得到了验证，包括神经退行性痴呆患者和健康对照者，无论是在原始条件下还是在噪声干扰条件下。其稀疏的、节点引导的解释突出了疾病相关的连接，并与功能连接改变的既定临床发现保持一致，从而为神经学评估提供了透明的线索。
摘要：Robust and interpretable dementia diagnosis from noisy, non-stationary electroencephalography (EEG) is clinically essential yet remains challenging. To this end, we propose SeeGraph, a Sparse-Explanatory dynamic EEG-graph network that models time-evolving functional connectivity and employs a node-guided sparse edge mask to reveal the connections that drive diagnostic decisions, while remaining robust to noise and cross-site variability. SeeGraph comprises four components: (1) a dual-trajectory temporal encoder that models dynamic EEG with two streams, where node signals capture regional oscillations and edge signals capture interregional coupling; (2) a topology-aware positional encoder that derives graph-spectral Laplacian coordinates from the fused connectivity and augments node embeddings; (3) a node-guided sparse explanatory edge mask that gates the connectivity into a compact subgraph; and (4) a gated graph predictor that operates on the sparsified graph. The framework is trained using cross-entropy loss together with a sparsity regularizer on the mask, yielding noise-robust and interpretable diagnoses. The effectiveness of SeeGraph is validated on public and in-house EEG cohorts, including patients with neurodegenerative dementias and healthy controls, under both raw and noise-perturbed conditions. Its sparse, node-guided explanations highlight disease-relevant connections and align with established clinical findings on functional connectivity alterations, thereby offering transparent cues for neurological evaluation.
Transformer(7篇)
【1】Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training
标题：Beyond Muon：MUD（MomentUm Decorrelation），用于更快的Transformer训练
链接：https://arxiv.org/abs/2603.17970
摘要：诸如Muon之类的离散化动量优化器通过经由短极分解迭代近似白化/正交化矩阵值动量更新来改进Transformer训练。然而，极性因子近似通常需要多个大矩阵乘法，并且所产生的开销可能是相当大的并且依赖于硬件。我们介绍MUD（MomentUm Decorrelation），一种互补的美白方法，取代μ子的极性更新与三角形（Cholesky样）美白代理灵感来自经典的革兰氏-施密特和高斯-赛德尔的想法。我们证明了行正交矩阵是MUD映射的不动点，将内步与Gram矩阵的对称Gauss-Seidel预处理联系起来，并证明了在不动点附近的二次局部收敛性.在时间到困惑方面，MUD在时间到困惑方面比调整的AdamW和Muon产生一致的10-50\%的挂钟改进，通常每步收敛比Muon稍慢，但相对于Muon，优化器开销大大降低，在大多数设置中，MUD将峰值令牌/秒提高了大约1.3 -2.6\times$，在A100上的GPT-2大型上提高了近3\times $。我们还演示了如何训练ESM-2 150 M蛋白质语言模型，其中MUD在显著更少的挂钟时间内匹配Muon级别的验证困惑。
摘要：Orthogonalized-momentum optimizers such as Muon improve transformer training by approximately whitening/orthogonalizing matrix-valued momentum updates via a short polar-decomposition iteration. However, polar-factor approximations typically require multiple large matrix multiplications, and the resulting overhead can be substantial and hardware-dependent. We introduce MUD (MomentUm Decorrelation), a complementary whitening approach that replaces Muon's polar update with a triangular (Cholesky-like) whitening surrogate inspired by classical Gram--Schmidt and Gauss-Seidel ideas. We show that row-orthonormal matrices are fixed points of the MUD map, relate the inner step to symmetric Gauss-Seidel preconditioning of the Gram matrix, and prove quadratic local convergence near the fixed point. In terms of time-to-perplexity, MUD yields consistent 10-50\% wall-clock improvements over tuned AdamW and Muon in time-to-perplexity, typically converging slightly slower per step than Muon but with substantially lower optimizer overhead -- relative to Muon, MUD improves peak tokens/s by roughly $1.3-2.6\times$ across most settings and up to nearly $3\times$ on GPT-2 large on an A100. We also demonstrate training a ESM-2 150M protein language model, where MUD matches Muon-level validation perplexity in significantly less wall-clock time.
【2】Dropout Robustness and Cognitive Profiling of Transformer Models via Stochastic Inference
标题：通过随机推理分析Transformer模型的辍学鲁棒性和认知剖析
链接：https://arxiv.org/abs/2603.17811
摘要 ：基于transformer的语言模型被广泛用于推理，但它们在推理时间随机性下的行为仍然没有得到充分的研究。虽然在训练过程中辍学是常见的，但其通过蒙特卡洛采样的推理时间效应缺乏跨架构的系统评估，限制了对不确定性感知应用程序中模型可靠性的理解。 这项工作分析了19个Transformer模型之间的辍学引起的变化，使用MC辍学与100个随机向前通过每个样本。丢弃鲁棒性被定义为在随机推断下保持高准确度和稳定的预测，通过每次运行准确度的标准差来衡量。认知分解框架将表现分解为记忆和推理组件。实验跨越五种脱落配置，对1，000个样本进行了95次唯一评价。 结果显示大量的建筑变化。较小的模型表现出完美的预测稳定性，而中型模型表现出显着的波动性。中型模型实现最佳的整体性能;较大的模型擅长记忆任务。重要的是，53%的模型在基线MC Dropout下遭受严重的准确性下降，任务专用模型损失高达24个百分点，表明这些架构不适合进行不确定性量化。出现不对称效应：高dropout会使记忆准确性降低27个百分点，而推理能力只降低1个百分点，这表明记忆任务依赖于dropout破坏的稳定表征。84%的模型表现出内存偏向的性能。 这为Transformers提供了第一个全面的MC Dropout基准，揭示了Dropout鲁棒性依赖于架构，与规模无关。认知分析框架为不确定性感知应用中的模型选择提供了可操作的指导。
摘要：Transformer-based language models are widely deployed for reasoning, yet their behavior under inference-time stochasticity remains underexplored. While dropout is common during training, its inference-time effects via Monte Carlo sampling lack systematic evaluation across architectures, limiting understanding of model reliability in uncertainty-aware applications. This work analyzes dropout-induced variability across 19 transformer models using MC Dropout with 100 stochastic forward passes per sample. Dropout robustness is defined as maintaining high accuracy and stable predictions under stochastic inference, measured by standard deviation of per-run accuracies. A cognitive decomposition framework disentangles performance into memory and reasoning components. Experiments span five dropout configurations yielding 95 unique evaluations on 1,000 samples. Results reveal substantial architectural variation. Smaller models demonstrate perfect prediction stability while medium-sized models exhibit notable volatility. Mid-sized models achieve the best overall performance; larger models excel at memory tasks. Critically, 53% of models suffer severe accuracy degradation under baseline MC Dropout, with task-specialized models losing up to 24 percentage points, indicating unsuitability for uncertainty quantification in these architectures. Asymmetric effects emerge: high dropout reduces memory accuracy by 27 percentage points while reasoning degrades only 1 point, suggesting memory tasks rely on stable representations that dropout disrupts. 84% of models demonstrate memory-biased performance. This provides the first comprehensive MC Dropout benchmark for transformers, revealing dropout robustness is architecture-dependent and uncorrelated with scale. The cognitive profiling framework offers actionable guidance for model selection in uncertainty-aware applications.
【3】Anisotropic Permeability Tensor Prediction from Porous Media Microstructure via Physics-Informed Progressive Transfer Learning with Hybrid CNN-Transformer
标题：通过混合CNN-Transformer的物理信息渐进传输学习从多孔媒体微结构预测各向异性渗透率张量
链接：https://arxiv.org/abs/2603.17532
摘要：从孔隙尺度微观结构图像准确预测渗透率张量对于地下流建模至关重要，但直接数值模拟每个样本需要数小时，从根本上限制了大规模不确定性量化和储层优化工作流程。提出了一种物理信息深度学习框架，通过将MaxViT混合CNN-Transformer架构与渐进式迁移学习和可微物理约束相结合来解决这一瓶颈。MaxViT的多轴关注机制同时通过块局部操作和通过网格全局操作的REV尺度连通性统计来解决粒度级孔喉几何结构，提供渗透率张量预测物理上需要的空间层次结构。在渗透率跨越三个数量级的20000个合成多孔介质样本上进行训练，三阶段渐进式课程从具有D4等变增强和张量变换的ImageNet预训练基线，通过组件加权损失优先考虑非对角耦合，通过逐层线性调制（FiLM）进行孔隙度调节的冻结主干迁移学习。Onsager互惠性和正定性通过可微惩罚项来实现。在4000个样本的保持测试集上，该框架实现了方差加权R2 = 0.9960（R2_Kxx = 0.9967，R2_Kxy = 0.9758），在监督基线上未解释的方差减少了33%。研究结果为基于物理学的科学机器学习提供了三个可转移的原则：大规模视觉预训练有效地跨域边界转移;物理约束最稳健地集成为可区分的架构组件;由诊断故障模式分析指导的渐进式训练可以在方法学阶段明确地归因于性能增益。
摘要：Accurate prediction of permeability tensors from pore-scale microstructure images is essential for subsurface flow modeling, yet direct numerical simulation requires hours per sample, fundamentally limiting large-scale uncertainty quantification and reservoir optimization workflows. A physics-informed deep learning framework is presented that resolves this bottleneck by combining a MaxViT hybrid CNN-Transformer architecture with progressive transfer learning and differentiable physical constraints. MaxViT's multi-axis attention mechanism simultaneously resolves grain-scale pore-throat geometry via block-local operations and REV-scale connectivity statistics through grid-global operations, providing the spatial hierarchy that permeability tensor prediction physically requires. Training on 20000 synthetic porous media samples spanning three orders of magnitude in permeability, a three-phase progressive curriculum advances from an ImageNet-pretrained baseline with D4-equivariant augmentation and tensor transformation, through component-weighted loss prioritizing off-diagonal coupling, to frozen-backbone transfer learning with porosity conditioning via Feature-wise Linear Modulation (FiLM). Onsager reciprocity and positive definiteness are enforced via differentiable penalty terms. On a held-out test set of 4000 samples, the framework achieves variance-weighted R2 = 0.9960 (R2_Kxx = 0.9967, R2_Kxy = 0.9758), a 33% reduction in unexplained variance over the supervised baseline. The results offer three transferable principles for physics-informed scientific machine learning: large-scale visual pretraining transfers effectively across domain boundaries; physical constraints are most robustly integrated as differentiable architectural components; and progressive training guided by diagnostic failure-mode analysis enables unambiguous attribution of performance gains across methodological stages.
【4】The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle
标题：相量Transformer：解决单位圈上的注意力瓶颈
链接：https://arxiv.org/abs/2603.17433
摘要：Transformer模型重新定义了序列学习，但点积自注意力为长上下文时间序列引入了二次令牌混合瓶颈。我们引入\textbf{Phasor Transformer}块，这是一个表示单位圆流形$S^1$上的序列状态的相位本地替代方案。每个块将轻量级可训练相移与无参数离散傅里叶变换（DFT）令牌耦合相结合，实现全局$\mathcal{O}（N\log N）$混合，而无需显式注意力地图。堆叠这些块定义\textbf{大相量模型（LPM）}。我们验证LPM的自回归时间序列预测合成多频基准。通过高度紧凑的参数预算，LPM学习稳定的全局动态，并实现与传统的自我注意基线相比具有竞争力的预测行为。我们的研究结果建立了一个明确的效率-性能前沿，表明时间序列的大模型缩放可以从具有确定性全局耦合的几何约束相位计算中出现，为振荡域中的可扩展时间建模提供了一条实用的路径。
摘要：Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the \textbf{Phasor Transformer} block, a phase-native alternative representing sequence states on the unit-circle manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit attention maps. Stacking these blocks defines the \textbf{Large Phasor Model (LPM)}. We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks. Operating with a highly compact parameter budget, LPM learns stable global dynamics and achieves competitive forecasting behavior compared to conventional self-attention baselines. Our results establish an explicit efficiency-performance frontier, demonstrating that large-model scaling for time-series can emerge from geometry-constrained phase computation with deterministic global coupling, offering a practical path toward scalable temporal modeling in oscillatory domains.
【5】DesertFormer: Transformer-Based Semantic Segmentation for Off-Road Desert Terrain Classification in Autonomous Navigation Systems
标题：EntFormer：自主导航系统中用于越野沙漠地形分类的基于转换器的语义分割
链接：https://arxiv.org/abs/2603.17056
备注：10 pages, 6 figures, 3 tables. Preprint also available on Zenodo (DOI: 10.5281/zenodo.19053085)
摘要 ：可靠的地形感知是非结构化越野环境中自主导航的基本要求。沙漠景观提出了独特的挑战，由于地形类别，极端的光照变化和稀疏的植被，无视标准的道路场景分割模型的假设之间的低色对比度。我们提出了一种基于SegFormer B2的越野沙漠地形分析语义分割管道，该管道具有分层混合Transformer（MiT-B2）骨干。该系统将地形分为十个具有生态意义的类别-树木，茂密的灌木丛，干草，干灌木丛，地面杂波，花卉，岩石，景观和天空-为地面机器人和自动驾驶车辆提供安全感知路径规划。在512 x512分辨率的4，176张带注释的越野图像的专用数据集上进行训练后，AptFormer实现了64.4%的平均Intersection-over-Union（mIoU）和86.1%的像素准确度，比DeepLabV 3 MobileNetV 2基线（41.0% mIoU）有+24.2%的绝对改进。我们进一步贡献了一个系统的故障分析，确定主要的混乱模式-地面杂波景观和干草景观-并提出类加权训练和复制粘贴增强罕见的地形类别。代码、检查点和交互式推理仪表板在https://github.com/Yasaswini-ch/Vision-based-Desert-Terrain-Segmentation-using-SegFormer上发布。
摘要：Reliable terrain perception is a fundamental requirement for autonomous navigation in unstructured, off-road environments. Desert landscapes present unique challenges due to low chromatic contrast between terrain categories, extreme lighting variability, and sparse vegetation that defy the assumptions of standard road-scene segmentation models. We present DesertFormer, a semantic segmentation pipeline for off-road desert terrain analysis based on SegFormer B2 with a hierarchical Mix Transformer (MiT-B2) backbone. The system classifies terrain into ten ecologically meaningful categories -- Trees, Lush Bushes, Dry Grass, Dry Bushes, Ground Clutter, Flowers, Logs, Rocks, Landscape, and Sky -- enabling safety-aware path planning for ground robots and autonomous vehicles. Trained on a purpose-built dataset of 4,176 annotated off-road images at 512x512 resolution, DesertFormer achieves a mean Intersection-over-Union (mIoU) of 64.4% and pixel accuracy of 86.1%, representing a +24.2% absolute improvement over a DeepLabV3 MobileNetV2 baseline (41.0% mIoU). We further contribute a systematic failure analysis identifying the primary confusion patterns -- Ground Clutter to Landscape and Dry Grass to Landscape -- and propose class-weighted training and copy-paste augmentation for rare terrain categories. Code, checkpoints, and an interactive inference dashboard are released at https://github.com/Yasaswini-ch/Vision-based-Desert-Terrain-Segmentation-using-SegFormer.
【6】Transformers Can Learn Rules They've Never Seen: Proof of Computation Beyond Interpolation
标题：Transformer可以学习他们从未见过的规则：超越插值的计算证明
链接：https://arxiv.org/abs/2603.17019
备注：26 pages, 6 figures
摘要：LLM争论的一个中心问题是，Transformers是否可以推断出训练中缺少的规则，或者明显的泛化是否会在观察到的示例上减少到基于相似性的插值。我们测试一个强大的插值只有假设在两个受控的设置：一个插值被排除的建设和证明，一个成功需要发射中间的符号推导，而不是只有最终的答案。在实验1中，我们使用具有纯XOR转换规则的元胞自动机，并从训练中删除特定的局部输入模式;由于XOR是线性不可分的，因此每个保持模式的最近邻居具有相反的标签，因此基于相似性的预测器在保持区域上失败。然而，两层Transformer恢复规则（最好100%; 47/60收敛运行），并且电路提取识别XOR计算。性能取决于多步约束传播：在没有展开的情况下，精度与输出偏差匹配（63.1%），而软展开达到96.7%。在实验2中，我们研究了整数上的符号运算符链，其中一个运算符对保持不变;模型必须以类似证明的格式发出中间步骤和最终答案。在所有49个holdout对中，Transformer超过了每个插值基线（平均41.8%，最高达78.6%;平均KRR 4.3%; KNN和MLP在每对上的得分为0%），而删除中间步骤监督会降低性能。结合一个结构，表明标准Transformer块可以实现精确的局部布尔规则，这些结果提供了一个存在性证明，即Transformers可以学习在训练中没有直接观察到的规则结构，并显式地表达它，排除了仅插值帐户的最强架构形式：原则上，Transformers不能发现和传达看不见的规则，而当这种行为在大规模语言培训中出现时，则保持开放。
摘要：A central question in the LLM debate is whether transformers can infer rules absent from training, or whether apparent generalisation reduces to similarity-based interpolation over observed examples. We test a strong interpolation-only hypothesis in two controlled settings: one where interpolation is ruled out by construction and proof, and one where success requires emitting intermediate symbolic derivations rather than only final answers. In Experiment 1, we use a cellular automaton with a pure XOR transition rule and remove specific local input patterns from training; since XOR is linearly inseparable, each held-out pattern's nearest neighbours have the opposite label, so similarity-based predictors fail on the held-out region. Yet a two-layer transformer recovers the rule (best 100%; 47/60 converged runs), and circuit extraction identifies XOR computation. Performance depends on multi-step constraint propagation: without unrolling, accuracy matches output bias (63.1%), while soft unrolling reaches 96.7%. In Experiment 2, we study symbolic operator chains over integers with one operator pair held out; the model must emit intermediate steps and a final answer in a proof-like format. Across all 49 holdout pairs, the transformer exceeds every interpolation baseline (mean 41.8%, up to 78.6%; mean KRR 4.3%; KNN and MLP score 0% on every pair), while removing intermediate-step supervision degrades performance. Together with a construction showing that a standard transformer block can implement exact local Boolean rules, these results provide an existence proof that transformers can learn rule structure not directly observed in training and express it explicitly, ruling out the strongest architectural form of interpolation-only accounts: that transformers cannot in principle discover and communicate unseen rules, while leaving open when such behaviour arises in large-scale language training.
【7】Integrating Inductive Biases in Transformers via Distillation for Financial Time Series Forecasting
标题：通过蒸馏在Transformer中集成感性偏差以进行金融时间序列预测
链接：https://arxiv.org/abs/2603.16985
备注：18 pages, 6 figures
摘要：基于变换器的模型由于其高度的表示能力和架构灵活性而被广泛用于时间序列预测。然而，许多Transformer变量隐含地假设平稳性和稳定的时间动态-在以政权转移和非平稳性为特征的金融市场中经常违反的假设。从经验上看，最先进的时间序列Transformers在金融任务上的表现甚至不如普通的Transformers，而具有明显归纳偏差的简单架构，如CNN和RNN，可以以更低的复杂性实现更强的性能。与此同时，没有单一的归纳偏差在市场或制度中占主导地位，这表明稳健的金融预测需要整合互补的时间先验。我们提出了TIPS（Transformer with Inductive Prior Synthesis），一个知识蒸馏框架，它在一个统一的Transformer中综合了各种归纳偏见-因果关系，局部性和周期性。TIPS通过注意力掩蔽训练专门的偏见Transformer教师，然后将他们的知识提炼成一个单一的学生模型，并通过诱导偏见进行依赖于制度的对齐。在四个主要的股票市场中，TIPS实现了最先进的性能，在年回报率、夏普比率和卡尔马比率方面分别比强大的整体基线高出55%、9%和16%，而只需要38%的推理时间计算。进一步的分析表明，TIPS产生了统计上显着的超额收益超过香草Transformers和它的教师合奏，并表现出政权依赖的行为与经典架构在其盈利期。这些结果突出了在非平稳金融时间序列的鲁棒推广的重要性，政权依赖的归纳偏差利用。
摘要：Transformer-based models have been widely adopted for time-series forecasting due to their high representational capacity and architectural flexibility. However, many Transformer variants implicitly assume stationarity and stable temporal dynamics -- assumptions routinely violated in financial markets characterized by regime shifts and non-stationarity. Empirically, state-of-the-art time-series Transformers often underperform even vanilla Transformers on financial tasks, while simpler architectures with distinct inductive biases, such as CNNs and RNNs, can achieve stronger performance with substantially lower complexity. At the same time, no single inductive bias dominates across markets or regimes, suggesting that robust financial forecasting requires integrating complementary temporal priors. We propose TIPS (Transformer with Inductive Prior Synthesis), a knowledge distillation framework that synthesizes diverse inductive biases -- causality, locality, and periodicity -- within a unified Transformer. TIPS trains bias-specialized Transformer teachers via attention masking, then distills their knowledge into a single student model with regime-dependent alignment across inductive biases. Across four major equity markets, TIPS achieves state-of-the-art performance, outperforming strong ensemble baselines by 55%, 9%, and 16% in annual return, Sharpe ratio, and Calmar ratio, while requiring only 38% of the inference-time computation. Further analyses show that TIPS generates statistically significant excess returns beyond both vanilla Transformers and its teacher ensembles, and exhibits regime-dependent behavioral alignment with classical architectures during their profitable periods. These results highlight the importance of regime-dependent inductive bias utilization for robust generalization in non-stationary financial time series.
GAN|对抗|攻击|生成相关(9篇)
【1】Procedural Generation of Algorithm Discovery Tasks in Machine Learning
标题：机器学习中算法发现任务的程序生成
链接：https://arxiv.org/abs/2603.17863
摘要：自动化机器学习算法的开发有可能带来新的突破。然而，我们的能力，以改善和评估算法发现系统迄今受到限制，现有的任务套件。它们有许多问题，例如：评价方法不佳;数据污染;以及包含饱和或非常相似的问题。在这里，我们介绍了DiscoGen，这是一个用于机器学习的算法发现任务的程序生成器，例如开发用于强化学习的优化器或用于图像分类的损失函数。受到强化学习中程序生成的成功的激励，DiscoGen跨越了来自一系列机器学习领域的数百万个不同难度和复杂性的任务。这些任务由少量配置参数指定，可用于优化算法发现代理（ADA）。我们提出了DiscoBench，一个基准组成的一个固定的，小子集的DiscoGen任务的原则性评估的ADA。最后，我们提出了一些雄心勃勃的，有影响力的研究方向，除了实验证明其用于ADA的快速优化。DiscoGen在https://github.com/AlexGoldie/discogen上开源发布。
摘要：Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans millions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a benchmark consisting of a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, in addition to experiments demonstrating its use for prompt optimisation of an ADA. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen.
【2】DSS-GAN: Directional State Space GAN with Mamba backbone for Class-Conditional Image Synthesis
标题：DSS-GAN：具有Mamba主干的定向状态空间GAN，用于类别条件图像合成
链接：https://arxiv.org/abs/2603.17637
摘要：我们提出了DSS-GAN，这是第一个采用Mamba作为分层生成器骨干进行噪声到图像合成的生成对抗网络。中心贡献是定向潜在路由（DLR），一种新的条件机制，将潜在向量分解为方向特定的子向量，每个子向量与类嵌入联合投影，以产生相应的曼巴扫描的特征仿射调制。与注入全局信号的传统类调节不同，DLR沿着特征图的不同空间轴耦合类身份和潜在结构，在所有生成尺度上一致地应用。与StyleGAN 2-ADA相比，DSS-GAN在多个测试数据集上实现了改进的FID，KID和精确召回分数。潜在空间的分析表明，方向子向量表现出可测量的专业化：沿单个组件的扰动产生结构化的，方向相关的变化，在合成图像。
摘要：We present DSS-GAN, the first generative adversarial network to employ Mamba as a hierarchical generator backbone for noise-to-image synthesis. The central contribution is Directional Latent Routing (DLR), a novel conditioning mechanism that decomposes the latent vector into direction-specific subvectors, each jointly projected with a class embedding to produce a feature-wise affine modulation of the corresponding Mamba scan. Unlike conventional class conditioning that injects a global signal, DLR couples class identity and latent structure along distinct spatial axes of the feature map, applied consistently across all generative scales. DSS-GAN achieves improved FID, KID, and precision-recall scores compared to StyleGAN2-ADA across multiple tested datasets. Analysis of the latent space reveals that directional subvectors exhibit measurable specialization: perturbations along individual components produce structured, direction-correlated changes in the synthesized image.
【3】Benchmarking Reinforcement Learning via Stochastic Converse Optimality: Generating Systems with Known Optimal Policies
标题：通过随机逆最优性对强化学习进行基准测试：生成具有已知最优策略的系统
链接：https://arxiv.org/abs/2603.17631
摘要：众所周知，强化学习（RL）算法的客观比较非常复杂，因为不同RL方法的结果和性能基准对算法学习和环境动态中固有的环境设计，奖励结构和随机性非常敏感。为了管理这种复杂性，我们引入了一个严格的基准测试框架，通过扩展逆最优离散时间，控制仿射，非线性系统的噪声。我们的框架提供了必要和充分的条件下，规定的值函数和政策是最佳的构造系统，使系统生成的基准家庭通过同伦变化和随机参数。我们通过自动构建不同的环境来验证它，展示了我们的框架在算法中进行受控和全面评估的能力。通过对标准方法进行评估，我们的工作为精确和严格的RL基准测试提供了可重复的基础。
摘要：The objective comparison of Reinforcement Learning (RL) algorithms is notoriously complex as outcomes and benchmarking of performances of different RL approaches are critically sensitive to environmental design, reward structures, and stochasticity inherent in both algorithmic learning and environmental dynamics. To manage this complexity, we introduce a rigorous benchmarking framework by extending converse optimality to discrete-time, control-affine, nonlinear systems with noise. Our framework provides necessary and sufficient conditions, under which a prescribed value function and policy are optimal for constructed systems, enabling the systematic generation of benchmark families via homotopy variations and randomized parameters. We validate it by automatically constructing diverse environments, demonstrating our framework's capacity for a controlled and comprehensive evaluation across algorithms. By assessing standard methods against a ground-truth optimum, our work delivers a reproducible foundation for precise and rigorous RL benchmarking.
【4】ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recovery
标题：ARES：通过激活恢复在联邦学习中可扩展且实用的梯度倒置攻击
链接：https://arxiv.org/abs/2603.17623
备注：18 pages. To appear in the IEEE Symposium on Security and Privacy 2026
摘要：联邦学习（FL）通过共享模型更新而不是原始数据来实现协作模型训练，旨在保护用户隐私。然而，最近的研究表明，这些共享更新可能会通过梯度反转攻击（GIA）无意中泄漏敏感的训练数据。其中，主动GIA特别强大，即使在大批量下也能实现单个样本的高保真重建。然而，现有的方法往往需要架构修改，这限制了它们的实际适用性。在这项工作中，我们通过引入激活细化稀疏反演（ARES）攻击来弥合这一差距，ARES攻击是一种主动GIA，旨在从大型训练批次中重建训练样本，而无需修改架构。具体来说，我们制定了一个嘈杂的稀疏恢复任务的恢复问题，并解决它使用广义最小绝对收缩和选择算子（套索）。为了将攻击扩展到多样本恢复，ARES采用印记方法来解开激活，从而实现可扩展的按样本重建。我们进一步建立了预期的恢复率并推导出重建误差的上界，为ARES攻击提供了理论保证。在CNN和MLP上进行的大量实验表明，ARES在不同的数据集上实现了高保真重建，在大批量和现实FL设置下显著优于之前的GIA。我们的研究结果强调，中间激活在FL中构成了严重和被低估的隐私风险，强调了加强防御的迫切需要。
摘要 ：Federated Learning (FL) enables collaborative model training by sharing model updates instead of raw data, aiming to protect user privacy. However, recent studies reveal that these shared updates can inadvertently leak sensitive training data through gradient inversion attacks (GIAs). Among them, active GIAs are particularly powerful, enabling high-fidelity reconstruction of individual samples even under large batch sizes. Nevertheless, existing approaches often require architectural modifications, which limit their practical applicability. In this work, we bridge this gap by introducing the Activation REcovery via Sparse inversion (ARES) attack, an active GIA designed to reconstruct training samples from large training batches without requiring architectural modifications. Specifically, we formulate the recovery problem as a noisy sparse recovery task and solve it using the generalized Least Absolute Shrinkage and Selection Operator (Lasso). To extend the attack to multi-sample recovery, ARES incorporates the imprint method to disentangle activations, enabling scalable per-sample reconstruction. We further establish the expected recovery rate and derive an upper bound on the reconstruction error, providing theoretical guarantees for the ARES attack. Extensive experiments on CNNs and MLPs demonstrate that ARES achieves high-fidelity reconstruction across diverse datasets, significantly outperforming prior GIAs under large batch sizes and realistic FL settings. Our results highlight that intermediate activations pose a serious and underestimated privacy risk in FL, underscoring the urgent need for stronger defenses.
【5】Do Understanding and Generation Fight? A Diagnostic Study of DPO for Unified Multimodal Models
标题：理解和一代会打架吗？统一多峰模型的DPO诊断研究
链接：https://arxiv.org/abs/2603.17044
备注：8 pages, CVPR MMF5M Workshop 2026
摘要：统一的多模态模型共享一个语言模型主干，用于理解和生成图像。DPO能否同时调整这两种功能？我们提出了这个问题的第一个系统的研究，应用DPO Janus-Pro在1B和7 B参数下的七个训练策略和两个事后的方法。中心的发现是负面的：发电质量抵制DPO对齐在所有测试条件下，这个架构。没有任何方法可以提高7 B的生成CLIPScore（|三角洲|< 0.2，p > 0.5，n=200/种子，3个种子）;在1B，所有方法降低生成，并且结果在偏好数据类型（真实与生成和模型与模型）和测试的数据量（150-288对）中保持不变。梯度分析揭示了原因：理解和生成梯度接近正交（cos ~ 0），具有由VQ令牌计数不对称性驱动的~11- 14倍幅度不平衡（576个生成令牌对~30-100个文本令牌）。这种不平衡是多任务DPO中的主要干扰机制;幅度平衡产生定向积极理解增量（+0.01-0.04 VQA，尽管个体不显著），但代沟仍然存在。我们将离散VQ标记化确定为可能的结构瓶颈-由收敛到ln（2）的生成DPO损失支持-并为使用基于VQ的统一模型的从业者提供实用指导。
摘要：Unified multimodal models share a language model backbone for both understanding and generating images. Can DPO align both capabilities simultaneously? We present the first systematic study of this question, applying DPO to Janus-Pro at 1B and 7B parameters under seven training strategies and two post-hoc methods. The central finding is negative: generation quality resists DPO alignment across all tested conditions on this architecture. No method improves generation CLIPScore at 7B (|Delta| < 0.2, p > 0.5 at n=200 per seed, 3 seeds); at 1B, all methods degrade generation, and the result holds across preference data types (real-vs-generated and model-vs-model) and the data volumes tested (150-288 pairs). Gradient analysis reveals why: understanding and generation gradients are near-orthogonal (cos ~ 0) with ~11-14x magnitude imbalance driven by VQ token count asymmetry (576 generation tokens vs. ~30-100 text tokens). This imbalance is the dominant interference mechanism in multi-task DPO; magnitude-balancing yields directionally positive understanding deltas (+0.01-0.04 VQA, though individually not significant), but the generation gap persists regardless. We identify discrete VQ tokenization as a likely structural bottleneck -- supported by the generation DPO loss converging to ln(2) -- and provide practical guidance for practitioners working with VQ-based unified models.
【6】Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation: Radiologist-Like Workflow with Clinically Verifiable Rewards
标题：用于放射学报告生成的多模式多智能体强化学习：类似放射科医生的工作流程，具有临床可验证的奖励
链接：https://arxiv.org/abs/2603.16876
备注：17 pages, 3 figures
摘要：我们提出了MARL-Rad，这是一种用于放射学报告生成的新型多模态多智能体强化学习框架，它协调特定区域的智能体和全局集成智能体，并通过临床可验证的奖励进行优化。与之前的单模型强化学习或独立训练模型的事后代理不同，我们的方法联合训练多个代理，并通过强化学习优化整个代理系统。在MIMIC-CXR和IU X射线数据集上进行的实验表明，MARL-Rad持续改善了RadGraph、CheXbert和GREEN评分等临床疗效（CE）指标，实现了最先进的CE性能。进一步的分析证实，MARL-Rad增强了偏侧一致性，并产生更准确，更详细的报告。
摘要：We propose MARL-Rad, a novel multi-modal multi-agent reinforcement learning framework for radiology report generation that coordinates region-specific agents and a global integrating agent, optimized via clinically verifiable rewards. Unlike prior single-model reinforcement learning or post-hoc agentization of independently trained models, our method jointly trains multiple agents and optimizes the entire agent system through reinforcement learning. Experiments on the MIMIC-CXR and IU X-ray datasets show that MARL-Rad consistently improves clinically efficacy (CE) metrics such as RadGraph, CheXbert, and GREEN scores, achieving state-of-the-art CE performance. Further analyses confirm that MARL-Rad enhances laterality consistency and produces more accurate, detail-informed reports.
【7】The Convergence Frontier: Integrating Machine Learning and High Performance Quantum Computing for Next-Generation Drug Discovery
标题：融合前沿：集成机器学习和高性能量子计算用于下一代药物发现
链接：https://arxiv.org/abs/2603.17790
摘要：将量子力学与药物发现相结合，标志着从经验试错到定量精确的决定性转变。然而，从头算分子动力学的高昂成本在历史上迫使化学准确性和计算可扩展性之间的妥协。本文将高性能计算（HPC）、机器学习（ML）和量子计算（QC）的融合确定为解决这一瓶颈的最终解决方案。虽然ML基础模型（如FeNix-Bio 1）能够实现量子精确的模拟，但它们仍然受到经典数据生成的固有限制。我们详细介绍了如何利用混合QPU-GPU架构的高性能量子计算（HPQC）将作为量子化学数据的最终加速器。通过利用希尔伯特空间映射，这些系统可以实现真正的化学准确性，同时绕过经典近似的几何学。我们展示了这种三方融合如何优化药物发现管道，从初始系统准备到ML驱动的高保真模拟。最后，我们将量子增强采样定位为超越GPU的前沿，用于建模反应性细胞系统和开拓下一代材料。
摘要：Integrating quantum mechanics into drug discovery marks a decisive shift from empirical trial-and-error toward quantitative precision. However, the prohibitive cost of ab initio molecular dynamics has historically forced a compromise between chemical accuracy and computational scalability. This paper identifies the convergence of High-Performance Computing (HPC), Machine Learning (ML), and Quantum Computing (QC) as the definitive solution to this bottleneck. While ML foundation models, such as FeNNix-Bio1, enable quantum-accurate simulations, they remain tethered to the inherent limits of classical data generation. We detail how High-Performance Quantum Computing (HPQC), utilizing hybrid QPU-GPU architectures, will serve as the ultimate accelerator for quantum chemistry data. By leveraging Hilbert space mapping, these systems can achieve true chemical accuracy while bypassing the heuristics of classical approximations. We show how this tripartite convergence optimizes the drug discovery pipeline, spanning from initial system preparation to ML-driven, high-fidelity simulations. Finally, we position quantum-enhanced sampling as the beyond GPU frontier for modeling reactive cellular systems and pioneering next-generation materials.
【8】rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks
标题
：rSDNet：针对标签噪音和对抗性攻击的统一鲁棒神经学习
链接：https://arxiv.org/abs/2603.17628
备注：Pre-print; under review
摘要：神经网络是现代人工智能的核心，但它们的训练对数据污染仍然高度敏感。标准神经分类器通过最小化分类交叉熵损失来训练，对应于多项式模型下的最大似然估计。虽然在理想条件下具有统计效率，但这种方法非常容易受到污染观测的影响，包括在输出空间中破坏监督的标签噪声，以及在输入空间中引起最坏情况偏差的对抗性扰动。在本文中，我们提出了一个统一的和统计接地的框架，强大的神经分类，解决这两种形式的污染在一个单一的学习目标。我们制定神经网络训练作为一个最小的分歧估计问题，并介绍rSDNet，一个强大的学习算法的基础上的一般类的$S$-分歧。由此产生的训练目标继承了经典统计估计的鲁棒性，通过模型概率自动降低异常观测值的权重。我们建立了rSDNet的基本群体级属性，包括Fisher一致性，分类校准意味着贝叶斯最优性，以及在均匀标签噪声和无穷小特征污染下的鲁棒性保证。在三个基准图像分类数据集上的实验表明，rSDNet提高了对标签损坏和对抗性攻击的鲁棒性，同时保持了对干净数据的竞争准确性，我们的研究结果突出了最小发散学习作为异构数据污染下鲁棒神经分类的原则和有效框架。
摘要：Neural networks are central to modern artificial intelligence, yet their training remains highly sensitive to data contamination. Standard neural classifiers are trained by minimizing the categorical cross-entropy loss, corresponding to maximum likelihood estimation under a multinomial model. While statistically efficient under ideal conditions, this approach is highly vulnerable to contaminated observations including label noises corrupting supervision in the output space, and adversarial perturbations inducing worst-case deviations in the input space. In this paper, we propose a unified and statistically grounded framework for robust neural classification that addresses both forms of contamination within a single learning objective. We formulate neural network training as a minimum-divergence estimation problem and introduce rSDNet, a robust learning algorithm based on the general class of $S$-divergences. The resulting training objective inherits robustness properties from classical statistical estimation, automatically down-weighting aberrant observations through model probabilities. We establish essential population-level properties of rSDNet, including Fisher consistency, classification calibration implying Bayes optimality, and robustness guarantees under uniform label noise and infinitesimal feature contamination. Experiments on three benchmark image classification datasets show that rSDNet improves robustness to label corruption and adversarial attacks while maintaining competitive accuracy on clean data, Our results highlight minimum-divergence learning as a principled and effective framework for robust neural classification under heterogeneous data contamination.
【9】Over-the-air White-box Attack on the Wav2Vec Speech Recognition Neural Network
标题：Wav2Vec语音识别神经网络的空中白盒攻击
链接：https://arxiv.org/abs/2603.16972
备注：9 pages, 5 figures, 1 table
摘要：基于神经网络的自动语音识别系统容易受到以恶意方式改变传输的对抗性攻击。该领域最近的工作集中在使攻击在空中场景中工作，然而这种攻击通常可以通过人类听觉检测到，限制了它们的潜在应用。在目前的工作中，我们探讨了不同的方法，使空中攻击不易察觉，以及这些方法对攻击的有效性的影响。
摘要：Automatic speech recognition systems based on neural networks are vulnerable to adversarial attacks that alter transcriptions in a malicious way. Recent works in this field have focused on making attacks work in over-the-air scenarios, however such attacks are typically detectable by human hearing, limiting their potential applications. In the present work we explore different approaches of making over-the-air attacks less detectable, as well as the impact these approaches have on the attacks' effectiveness.
半/弱/无/有监督|不确定性|主动学习(7篇)
【1】Unsupervised Symbolic Anomaly Detection
标题：无监督符号异常检测
链接：https://arxiv.org/abs/2603.17575
备注：13 pages, 7 figures
摘要：我们提出了SYRAN，一种基于符号回归的无监督异常检测方法。我们的方法不是在不透明的高维模型中编码正常模式，而是学习一组人类可读的方程，这些方程描述了符号不变量：在正常数据上近似恒定的函数。从这些不变量的偏差产生异常分数，使检测逻辑是可解释的建设，而不是通过事后解释。实验结果表明，SYRAN是高度可解释的，提供对应于已知的科学或医学关系的方程，并保持强大的异常检测性能与国家的最先进的方法。
摘要：We propose SYRAN, an unsupervised anomaly detection method based on symbolic regression. Instead of encoding normal patterns in an opaque, high-dimensional model, our method learns an ensemble of human-readable equations that describe symbolic invariants: functions that are approximately constant on normal data. Deviations from these invariants yield anomaly scores, so that the detection logic is interpretable by construction, rather than via post-hoc explanation. Experimental results demonstrate that SYRAN is highly interpretable, providing equations that correspond to known scientific or medical relationships, and maintains strong anomaly detection performance comparable to that of state-of-the-art methods.
【2】The Causal Uncertainty Principle: Manifold Tearing and the Topological Limits of Counterfactual Interventions
标题：因果不确定性原则：多元化撕裂和反事实干预的布局限制
链接：https://arxiv.org/abs/2603.17385
备注：33 pages, 6 figures. Submitted to the Journal of Machine Learning Research (JMLR)
摘要：Judea Pearl的do-calculus为因果推理提供了基础，但将其转化为连续生成模型仍然充满了几何挑战。我们确定了这种干预的基本限度。我们定义了反事实事件视界并证明了流形撕裂定理：确定性流在极端干预下不可避免地发展出有限时间奇点。我们建立了因果不确定性原则之间的干预极端和身份保护的权衡。最后，我们介绍了几何感知因果流（GACF），这是一种可扩展的算法，利用拓扑雷达绕过流形撕裂，在高维scRNA-seq数据上进行了验证。
摘要：Judea Pearl's do-calculus provides a foundation for causal inference, but its translation to continuous generative models remains fraught with geometric challenges. We establish the fundamental limits of such interventions. We define the Counterfactual Event Horizon and prove the Manifold Tearing Theorem: deterministic flows inevitably develop finite-time singularities under extreme interventions. We establish the Causal Uncertainty Principle for the trade-off between intervention extremity and identity preservation. Finally, we introduce Geometry-Aware Causal Flow (GACF), a scalable algorithm that utilizes a topological radar to bypass manifold tearing, validated on high-dimensional scRNA-seq data.
【3】Self-Conditioned Denoising for Atomistic Representation Learning
标题：原子表示学习的自我条件去噪
链接：https://arxiv.org/abs/2603.17196
摘要：NLP和计算机视觉大规模预训练的成功，促进了为物理科学开发类似基础模型的努力。然而，使用原子数据的预训练策略仍然未被探索。到目前为止，DFT力能标签上的大规模监督预训练为下游属性预测提供了最强的性能增益，优于现有的自监督学习（SSL）方法，这些方法仍然限于基态几何形状和/或原子数据的单个域。我们解决这些缺点与自调节去噪（SCD），骨干不可知的重建目标，利用自嵌入的条件去噪在任何领域的原子数据，包括小分子，蛋白质，周期性材料，和“非平衡”的几何形状。当控制骨干架构和预训练数据集时，SCD在下游基准测试中的性能显着优于以前的SSL方法，并匹配或超过监督力能预训练的性能。我们表明，通过SCD预训练的小而快的GNN可以在多个领域的任务中实现与在显著更大的标记或未标记数据集上预训练的更大模型竞争或更好的性能。我们的代码可在：https://github.com/TyJPerez/SelfConditionedDenoisingAtoms
摘要：The success of large-scale pretraining in NLP and computer vision has catalyzed growing efforts to develop analogous foundation models for the physical sciences. However, pretraining strategies using atomistic data remain underexplored. To date, large-scale supervised pretraining on DFT force-energy labels has provided the strongest performance gains to downstream property prediction, out-performing existing methods of self-supervised learning (SSL) which remain limited to ground-state geometries, and/or single domains of atomistic data. We address these shortcomings with Self-Conditioned Denoising (SCD), a backbone-agnostic reconstruction objective that utilizes self-embeddings for conditional denoising across any domain of atomistic data, including small molecules, proteins, periodic materials, and 'non-equilibrium' geometries. When controlled for backbone architecture and pretraining dataset, SCD significantly outperforms previous SSL methods on downstream benchmarks and matches or exceeds the performance of supervised force-energy pretraining. We show that a small, fast GNN pretrained by SCD can achieve competitive or superior performance to larger models pretrained on significantly larger labeled or unlabeled datasets, across tasks in multiple domains. Our code is available at: https://github.com/TyJPerez/SelfConditionedDenoisingAtoms
【4】Ensemble Self-Training for Unsupervised Machine Translation
标题：无监督机器翻译的自我训练
链接：https://arxiv.org/abs/2603.17087
摘要：我们提出了一个集合驱动的自训练框架，用于无监督神经机器翻译（UNMT）。从主要语言对开始，我们训练多个UNMT模型，这些模型共享相同的翻译任务，但在辅助语言上不同，从而在模型之间引入结构化多样性。然后，我们使用令牌级集成解码生成主对的伪翻译，在两个方向上平均模型预测。这些集合输出用作合成并行数据来进一步训练每个模型，从而允许模型通过共享监督来改进。在部署时，我们根据验证性能选择单个模型，保留单个模型的推理成本。实验表明，与单模型UNMT基线相比，统计学上有显著改善，从英语翻译时平均增益为1.7 chrF，翻译成英语时平均增益为0.67 chrF。
摘要：We present an ensemble-driven self-training framework for unsupervised neural machine translation (UNMT). Starting from a primary language pair, we train multiple UNMT models that share the same translation task but differ in an auxiliary language, inducing structured diversity across models. We then generate pseudo-translations for the primary pair using token-level ensemble decoding, averaging model predictions in both directions. These ensemble outputs are used as synthetic parallel data to further train each model, allowing the models to improve via shared supervision. At deployment time, we select a single model by validation performance, preserving single-model inference cost. Experiments show statistically significant improvements over single-model UNMT baselines, with mean gains of 1.7 chrF when translating from English and 0.67 chrF when translating into English.
【5】Wasserstein-type Gaussian Process Regressions for Input Measurement Uncertainty
标题：输入测量不确定性的瓦瑟斯坦型高斯过程回归
链接：https://arxiv.org/abs/2603.17271
备注：22 pages
摘要：高斯过程（GP）回归被广泛用于不确定性量化，但标准公式假设无噪声协变量。当输入测量误差，这种误差变量（EIV）设置可能会导致乐观狭窄的后验区间和偏见的决定。我们通过将每个有噪输入表示为概率度量并通过这些度量之间的Wasserstein距离定义协方差来研究输入测量不确定性下的GP回归。基于这一观点，我们实例化一个确定性的投影Wasserstein ARD（PWA）内核，其一维组件承认封闭形式的表达式，其产品结构产生一个可扩展的，正定的内核分布。与潜在输入GP模型不同，基于PWA的GP（\PWAGPs）处理输入噪声，而不引入未观察到的协变量或Monte Carlo投影，使不确定性量化更加透明和鲁棒。
摘要：Gaussian process (GP) regression is widely used for uncertainty quantification, yet the standard formulation assumes noise-free covariates. When inputs are measured with error, this errors-in-variables (EIV) setting can lead to optimistically narrow posterior intervals and biased decisions. We study GP regression under input measurement uncertainty by representing each noisy input as a probability measure and defining covariance through Wasserstein distances between these measures. Building on this perspective, we instantiate a deterministic projected Wasserstein ARD (PWA) kernel whose one-dimensional components admit closed-form expressions and whose product structure yields a scalable, positive-definite kernel on distributions. Unlike latent-input GP models, PWA-based GPs (\PWAGPs) handle input noise without introducing unobserved covariates or Monte Carlo projections, making uncertainty quantification more transparent and robust.
【6】Self-Regularized Learning Methods
标题：自正则化学习方法
链接：https://arxiv.org/abs/2603.17160
摘要：我们介绍了一个一般的框架，用于分析学习算法的基础上的概念，自正则化，捕捉隐式的复杂性控制，而不需要显式的正则化。这是由以前的观察，许多算法，如基于梯度下降的学习，表现出隐式正则化的动机。简而言之，对于自正则化算法，预测器的复杂性本质上由实现相同经验风险的最简单比较器的复杂性控制。这个框架足够丰富，涵盖了经典的正则化经验风险最小化和梯度下降。在自正则化的基础上，我们对这些算法进行了全面的统计分析，包括最小最大最优速率，这足以表明该算法是自正则化的-所有进一步的要求都源于学习问题本身。最后，我们讨论了数据相关的超参数选择的问题，提供了一个一般的结果，产生最小最大最优率的双对数因子，并涵盖数据驱动的早期停止RKHS为基础的梯度下降。
摘要：We introduce a general framework for analyzing learning algorithms based on the notion of self-regularization, which captures implicit complexity control without requiring explicit regularization. This is motivated by previous observations that many algorithms, such as gradient-descent based learning, exhibit implicit regularization. In a nutshell, for a self-regularized algorithm the complexity of the predictor is inherently controlled by that of the simplest comparator achieving the same empirical risk. This framework is sufficiently rich to cover both classical regularized empirical risk minimization and gradient descent. Building on self-regularization, we provide a thorough statistical analysis of such algorithms including minmax-optimal rates, where it suffices to show that the algorithm is self-regularized -- all further requirements stem from the learning problem itself. Finally, we discuss the problem of data-dependent hyperparameter selection, providing a general result which yields minmax-optimal rates up to a double logarithmic factor and covers data-driven early stopping for RKHS-based gradient descent.
【7】Optimization-Embedded Active Multi-Fidelity Surrogate Learning for Multi-Condition Airfoil Shape Optimization
标题：多条件机翼形状优化的嵌入优化主动多保真代理学习
链接：https://arxiv.org/abs/2603.17057
备注：21 pages, 14 figures
摘要：主动多逼真度代理模型是为多条件翼型形状优化而开发的，以在保持RANS级精度的同时降低高保真度CFD成本。该框架耦合了一个低保真度的高斯过程回归传递模型与不确定性触发的采样和同步精英规则嵌入在一个混合遗传算法。低保真度XFOIL评估提供了廉价的功能，而稀疏RANS模拟自适应分配时，预测的不确定性超过阈值;精英候选人强制验证高保真度，并重新评估人口，以防止进化选择的基础上过时的健身值产生的早期代理状态。该方法被证明为两点问题在$Re=6\times10^6$与巡航在$α=2^\circ$（最大化$E=L/D$）和起飞在$α=10^\circ$（最大化$C_L$）使用12参数CST表示。每个飞行条件的独立多保真度代理使解耦细化成为可能。优化后的设计相对于第一代最佳个体，巡航效率提高了41.05%，起飞升力提高了20.75%。在整个活动中，只有14.78%（巡航）和9.5%（起飞）的评估个体需要RANS，这表明高保真使用率大幅降低，同时保持一致的多点性能。
摘要：Active multi-fidelity surrogate modeling is developed for multi-condition airfoil shape optimization to reduce high-fidelity CFD cost while retaining RANS-level accuracy. The framework couples a low-fidelity-informed Gaussian process regression transfer model with uncertainty-triggered sampling and a synchronized elitism rule embedded in a hybrid genetic algorithm. Low-fidelity XFOIL evaluations provide inexpensive features, while sparse RANS simulations are adaptively allocated when predictive uncertainty exceeds a threshold; elite candidates are mandatorily validated at high fidelity, and the population is re-evaluated to prevent evolutionary selection based on outdated fitness values produced by earlier surrogate states. The method is demonstrated for a two-point problem at $Re=6\times10^6$ with cruise at $α=2^\circ$ (maximize $E=L/D$) and take-off at $α=10^\circ$ (maximize $C_L$) using a 12-parameter CST representation. Independent multi-fidelity surrogates per flight condition enable decoupled refinement. The optimized design improves cruise efficiency by 41.05% and take-off lift by 20.75% relative to the best first-generation individual. Over the full campaign, only 14.78% (cruise) and 9.5% (take-off) of evaluated individuals require RANS, indicating a substantial reduction in high-fidelity usage while maintaining consistent multi-point performance.
迁移|Zero/Few/One-Shot|自适应(11篇)
【1】Unified Policy Value Decomposition for Rapid Adaptation
标题：统一政策价值分解，快速适应
链接：https://arxiv.org/abs/2603.17947
摘要：复杂控制系统的快速适应仍然是强化学习的核心挑战。我们引入了一个框架，其中政策和价值函数共享一个低维系数向量-目标嵌入-捕获任务身份，并使立即适应新的任务，而无需重新训练表示。在预训练期间，我们通过双线性行动者-批评者分解来共同学习结构化的价值基础和兼容的策略基础。批评者分解为Q = sum_k G_k（g）y_k（s，a），其中G_k（g）是目标条件系数向量，y_k（s，a）是学习值基函数。这种乘法门控-其中上下文信号缩放一组状态依赖基础-让人想起在第5层锥体神经元中观察到的增益调制，其中自上而下的输入调制感官驱动响应的增益，而不改变它们的调谐。在后继特征的基础上，我们将分解扩展到参与者，参与者组成了一组由相同系数G_k（g）加权的原始策略。在测试时，基被冻结，并且G_k（g）通过单个前向传递被估计为zero-shot，使得能够立即适应新的任务而无需任何梯度更新。我们训练一个软Actor-Critic代理的MuJoCo蚂蚁环境下的多方向运动目标，要求代理在八个方向上行走指定为连续的目标向量。双线性结构允许每个策略头专用于方向的子集，而共享系数层则在它们之间进行泛化，通过在目标嵌入空间中进行插值来适应新的方向。我们的研究结果表明，共享的低维目标嵌入提供了一个通用的机制，在高维控制的快速，结构化的适应，并强调了一个潜在的生物学合理的原则，在复杂的强化学习系统的有效转移。
摘要：Rapid adaptation in complex control systems remains a central challenge in reinforcement learning. We introduce a framework in which policy and value functions share a low-dimensional coefficient vector - a goal embedding - that captures task identity and enables immediate adaptation to novel tasks without retraining representations. During pretraining, we jointly learn structured value bases and compatible policy bases through a bilinear actor-critic decomposition. The critic factorizes as Q = sum_k G_k(g) y_k(s,a), where G_k(g) is a goal-conditioned coefficient vector and y_k(s,a) are learned value basis functions. This multiplicative gating - where a context signal scales a set of state-dependent bases - is reminiscent of gain modulation observed in Layer 5 pyramidal neurons, where top-down inputs modulate the gain of sensory-driven responses without altering their tuning. Building on Successor Features, we extend the decomposition to the actor, which composes a set of primitive policies weighted by the same coefficients G_k(g). At test time the bases are frozen and G_k(g) is estimated zero-shot via a single forward pass, enabling immediate adaptation to novel tasks without any gradient update. We train a Soft Actor-Critic agent on the MuJoCo Ant environment under a multi-directional locomotion objective, requiring the agent to walk in eight directions specified as continuous goal vectors. The bilinear structure allows each policy head to specialize to a subset of directions, while the shared coefficient layer generalizes across them, accommodating novel directions by interpolating in goal embedding space. Our results suggest that shared low-dimensional goal embeddings offer a general mechanism for rapid, structured adaptation in high-dimensional control, and highlight a potentially biologically plausible principle for efficient transfer in complex reinforcement learning systems.
【2】Adaptive Guidance for Retrieval-Augmented Masked Diffusion Models
标题：检索增强掩蔽扩散模型的自适应指导
链接：https://arxiv.org/abs/2603.17677
摘要：检索增强生成（RAG）通过将外部知识纳入语言模型生成来改善事实基础。然而，当检索的上下文是嘈杂的，不可靠的，或与模型的参数知识不一致，它会引入检索先验冲突，可以降低生成质量。虽然这个问题已经在自回归语言模型中进行了研究，但在基于扩散的语言模型中，它在很大程度上仍未被探索，其中迭代去噪过程为整合检索到的上下文带来了独特的挑战。在这项工作中，我们提出了自适应检索增强掩蔽扩散（ARAM），RAG设置中的掩蔽扩散模型（MDM）的无训练自适应指导框架。ARAM在去噪过程中根据检索到的上下文引起的分布偏移的信噪比（SNR）动态地校准引导尺度。直觉上，当检索到的上下文提供可靠的纠正证据时，该模型加强指导，当上下文信号是嘈杂的或不支持时，该模型抑制它。在多个知识密集型QA基准上的广泛实验表明，ARAM比竞争性RAG基准提高了整体QA性能。
摘要：Retrieval-Augmented Generation (RAG) improves factual grounding by incorporating external knowledge into language model generation. However, when retrieved context is noisy, unreliable, or inconsistent with the model's parametric knowledge, it introduces retrieval-prior conflicts that can degrade generation quality. While this problem has been studied in autoregressive language models, it remains largely unexplored in diffusion-based language models, where the iterative denoising process introduces unique challenges for integrating retrieved context. In this work, we propose Adaptive Retrieval-Augmented Masked Diffusion (ARAM), a training-free adaptive guidance framework for Masked Diffusion Models (MDMs) in RAG settings. ARAM dynamically calibrates the guidance scale during denoising according to the Signal-to-Noise Ratio (SNR) of the distributional shift induced by retrieved context. Intuitively, the model strengthens guidance when the retrieved context provides reliable corrective evidence and suppresses it when the contextual signal is noisy or non-supportive. Extensive experiments on multiple knowledge-intensive QA benchmarks show that ARAM improves overall QA performance over competitive RAG baselines.
【3】AdaMuS: Adaptive Multi-view Sparsity Learning for Dimensionally Unbalanced Data
标题：AdaMuS：针对专业不平衡数据的自适应多视图稀疏学习
链接：https://arxiv.org/abs/2603.17610
备注：15 pages. Submitted to IEEE Transactions on Image Processing
摘要：多视图学习的主要目的是融合多个特征来全面描述数据。大多数先前的研究都隐含地假设不同的观点具有相似的维度。然而，在实践中，不同视角之间往往存在严重的维度差异，导致了多视角学习的不平衡问题。例如，在情感识别任务中，视频帧通常达到10^6 $的维度，而生理信号仅包括10^1 $维度。现有的方法通常面临两个主要挑战：（1）他们往往偏向于高维数据，忽略了低维视图。(2)他们努力在极端维度不平衡的情况下有效地对齐表示，这将严重的冗余引入低维表示。为了解决这些问题，我们提出了自适应多视图稀疏学习（AdaMuS）框架。首先，为了防止忽略低维视图的信息，我们构造了视图特定的编码器，将它们映射到一个统一的维空间。考虑到将低维数据映射到高维空间通常会导致严重的过拟合，我们设计了一种无参数修剪方法来自适应地去除编码器中的冗余参数。此外，我们提出了一个稀疏融合的范例，灵活地抑制冗余的维度，并有效地对齐每个视图。此外，为了学习具有更强泛化能力的表示，我们提出了一种自监督学习范式，通过构建相似性图来获得监督信息。对合成玩具数据集和七个真实世界基准测试的广泛评估表明，AdaMuS在分类和语义分割任务中始终实现卓越的性能，并表现出很强的泛化能力。
摘要：Multi-view learning primarily aims to fuse multiple features to describe data comprehensively. Most prior studies implicitly assume that different views share similar dimensions. In practice, however, severe dimensional disparities often exist among different views, leading to the unbalanced multi-view learning issue. For example, in emotion recognition tasks, video frames often reach dimensions of $10^6$, while physiological signals comprise only $10^1$ dimensions. Existing methods typically face two main challenges for this problem: (1) They often bias towards high-dimensional data, overlooking the low-dimensional views. (2) They struggle to effectively align representations under extreme dimensional imbalance, which introduces severe redundancy into the low-dimensional ones. To address these issues, we propose the Adaptive Multi-view Sparsity Learning (AdaMuS) framework. First, to prevent ignoring the information of low-dimensional views, we construct view-specific encoders to map them into a unified dimensional space. Given that mapping low-dimensional data to a high-dimensional space often causes severe overfitting, we design a parameter-free pruning method to adaptively remove redundant parameters in the encoders. Furthermore, we propose a sparse fusion paradigm that flexibly suppresses redundant dimensions and effectively aligns each view. Additionally, to learn representations with stronger generalization, we propose a self-supervised learning paradigm that obtains supervision information by constructing similarity graphs. Extensive evaluations on a synthetic toy dataset and seven real-world benchmarks demonstrate that AdaMuS consistently achieves superior performance and exhibits strong generalization across both classification and semantic segmentation tasks.
【4】CLeAN: Continual Learning Adaptive Normalization in Dynamic Environments
标题：CLeAN：动态环境中的持续学习自适应规范化
链接：https://arxiv.org/abs/2603.17548
备注：16 pages, 3 figures
摘要：人工智能系统主要依赖于静态数据分布，这使得它们在动态的现实世界环境中无效，例如网络安全，自动运输或金融，其中数据经常发生变化。持续学习提供了一个潜在的解决方案，使模型能够从顺序数据中学习，同时保留先验知识。然而，在这个领域中一个关键的和未充分探索的问题是数据规范化。传统的标准化方法，如最小-最大缩放，假定访问整个数据集，这与连续学习的顺序性质不一致。在本文中，我们介绍了连续学习自适应规范化（CLeAN），一种新的自适应规范化技术，设计用于表格数据的连续学习。CLeAN涉及使用可学习的参数来估计全局特征尺度，这些参数通过指数移动平均（EMA）模块进行更新，使模型能够适应不断变化的数据分布。通过对两个数据集和各种持续学习策略（包括Resevoir Experience Replay，A-GEM和EwC）的综合评估，我们证明了CLeAN不仅提高了新数据的模型性能，而且还减轻了灾难性遗忘。研究结果强调了自适应规范化在增强表格数据的稳定性和有效性方面的重要性，为在动态学习环境中使用规范化来保存知识提供了一个新的视角。
摘要：Artificial intelligence systems predominantly rely on static data distributions, making them ineffective in dynamic real-world environments, such as cybersecurity, autonomous transportation, or finance, where data shifts frequently. Continual learning offers a potential solution by enabling models to learn from sequential data while retaining prior knowledge. However, a critical and underexplored issue in this domain is data normalization. Conventional normalization methods, such as min-max scaling, presuppose access to the entire dataset, which is incongruent with the sequential nature of continual learning. In this paper we introduce Continual Learning Adaptive Normalization (CLeAN), a novel adaptive normalization technique designed for continual learning in tabular data. CLeAN involves the estimation of global feature scales using learnable parameters that are updated via an Exponential Moving Average (EMA) module, enabling the model to adapt to evolving data distributions. Through comprehensive evaluations on two datasets and various continual learning strategies, including Resevoir Experience Replay, A-GEM, and EwC we demonstrate that CLeAN not only improves model performance on new data but also mitigates catastrophic forgetting. The findings underscore the importance of adaptive normalization in enhancing the stability and effectiveness of tabular data, offering a novel perspective on the use of normalization to preserve knowledge in dynamic learning environments.
【5】TimeAPN: Adaptive Amplitude-Phase Non-Stationarity Normalization for Time Series Forecasting
标题：TimeACN：时间序列预测的自适应幅相非平稳规范化
链接：https://arxiv.org/abs/2603.17436
摘要：非平稳性是多元长期时间序列预测中的一个基本挑战，通常表现为幅度和相位的快速变化。这些变化导致严重的分布偏移，从而降低预测性能。现有的基于归一化的方法主要依赖于一阶和二阶统计，隐含地假设分布平滑地演变，并忽略了细粒度的时间动态。为了解决这些限制，我们提出了TimeAPN，一个自适应幅度相位非平稳归一化框架，从时域和频域明确建模和预测非平稳因素。具体而言，TimeAPN首先在时域和频域中联合对平均序列进行建模，然后预测其在未来范围内的演变。同时，在频域中提取相位信息，并明确建模预测和地面实况未来序列之间的相位差异，以捕获时间错位。此外，TimeAPN将幅度信息纳入自适应归一化机制，使模型能够有效地考虑信号能量的突然波动。预测的非平稳因素随后通过协作去归一化过程与骨干预测输出相结合，以重建最终的非平稳时间序列。所提出的框架是模型不可知的，可以与各种预测骨干无缝集成。在七个真实世界的多变量数据集上进行的广泛实验表明，TimeAPN在多个预测范围内持续提高长期预测准确性，并优于最先进的可逆归一化方法。
摘要：Non-stationarity is a fundamental challenge in multivariate long-term time series forecasting, often manifested as rapid changes in amplitude and phase. These variations lead to severe distribution shifts and consequently degrade predictive performance. Existing normalization-based methods primarily rely on first- and second-order statistics, implicitly assuming that distributions evolve smoothly and overlooking fine-grained temporal dynamics. To address these limitations, we propose TimeAPN, an Adaptive Amplitude-Phase Non-Stationarity Normalization framework that explicitly models and predicts non-stationary factors from both the time and frequency domains. Specifically, TimeAPN first models the mean sequence jointly in the time and frequency domains, and then forecasts its evolution over future horizons. Meanwhile, phase information is extracted in the frequency domain, and the phase discrepancy between the predicted and ground-truth future sequences is explicitly modeled to capture temporal misalignment. Furthermore, TimeAPN incorporates amplitude information into an adaptive normalization mechanism, enabling the model to effectively account for abrupt fluctuations in signal energy. The predicted non-stationary factors are subsequently integrated with the backbone forecasting outputs through a collaborative de-normalization process to reconstruct the final non-stationary time series. The proposed framework is model-agnostic and can be seamlessly integrated with various forecasting backbones. Extensive experiments on seven real-world multivariate datasets demonstrate that TimeAPN consistently improves long-term forecasting accuracy across multiple prediction horizons and outperforms state-of-the-art reversible normalization methods.
【6】Mutually Causal Semantic Distillation Network for Zero-Shot Learning
标题：Zero-Shot学习的因果语义蒸馏网络
链接：https://arxiv.org/abs/2603.17412
备注 ：Accepted to IJCV. arXiv admin note: text overlap with arXiv:2203.03137
摘要：零射击学习（Zero-shot learning，简称ZRL）旨在识别开放世界中由边信息引导的不可见类（例如，属性）。它的关键任务是如何推断可见类的视觉特征和属性特征之间的潜在语义知识，从而实现从可见类到不可见类的语义知识转移。现有的工作简单地利用弱监督方式内的单向注意来学习虚假的和有限的潜在语义表示，这不能有效地发现内在语义知识（例如，属性语义）之间的视觉和属性特征。为了解决上述挑战，我们提出了一个互为因果的语义蒸馏网络（称为MSDN++），以提取的内在和充分的语义表示为CNOL。MSDN++由一个学习基于属性的视觉特征的属性$\rightarrow$视觉因果注意力子网和一个学习基于视觉的属性特征的视觉$\rightarrow$属性因果注意力子网组成。因果注意力鼓励两个子网学习因果视觉/属性学习表示可靠的特征的因果视觉/属性关联。在语义蒸馏损失的指导下，两个相互注意的子网在整个训练过程中协同学习并相互教导。在三个广泛使用的基准数据集上进行了广泛的实验（例如，CUB、SUN、AWA 2和FLO）表明，我们的MSDN++在强大的基线上有了显著的改进，从而带来了新的最先进的性能。
摘要：Zero-shot learning (ZSL) aims to recognize the unseen classes in the open-world guided by the side-information (e.g., attributes). Its key task is how to infer the latent semantic knowledge between visual and attribute features on seen classes, and thus conducting a desirable semantic knowledge transfer from seen classes to unseen ones. Prior works simply utilize unidirectional attention within a weakly-supervised manner to learn the spurious and limited latent semantic representations, which fail to effectively discover the intrinsic semantic knowledge (e.g., attribute semantic) between visual and attribute features. To solve the above challenges, we propose a mutually causal semantic distillation network (termed MSDN++) to distill the intrinsic and sufficient semantic representations for ZSL. MSDN++ consists of an attribute$\rightarrow$visual causal attention sub-net that learns attribute-based visual features, and a visual$\rightarrow$attribute causal attention sub-net that learns visual-based attribute features. The causal attentions encourages the two sub-nets to learn causal vision-attribute associations for representing reliable features with causal visual/attribute learning. With the guidance of semantic distillation loss, the two mutual attention sub-nets learn collaboratively and teach each other throughout the training process. Extensive experiments on three widely-used benchmark datasets (e.g., CUB, SUN, AWA2, and FLO) show that our MSDN++ yields significant improvements over the strong baselines, leading to new state-of-the-art performances.
【7】ReLMXEL: Adaptive RL-Based Memory Controller with Explainable Energy and Latency Optimization
标题：ReLMXEL：具有可解释能量和延迟优化的自适应基于RL的存储器控制器
链接：https://arxiv.org/abs/2603.17309
摘要：减少延迟和能耗对于提高现代计算中存储器系统的效率至关重要。这项工作介绍了ReLMXEL（具有可解释能量和延迟优化的记忆控制器强化学习），这是一个可解释的多智能体在线强化学习框架，它使用奖励分解动态优化记忆控制器参数。ReLMXEL在内存控制器中运行，利用详细的内存行为指标来指导决策。不同工作负载的实验评估表明，与基线配置相比，性能得到了一致的提高，并通过特定于工作负载的内存访问行为进行了改进。通过将可解释性纳入学习过程，ReLMXEL不仅提高了性能，还增加了控制决策的透明度，为更负责任和自适应的记忆系统设计铺平了道路。
摘要：Reducing latency and energy consumption is critical to improving the efficiency of memory systems in modern computing. This work introduces ReLMXEL (Reinforcement Learning for Memory Controller with Explainable Energy and Latency Optimization), a explainable multi-agent online reinforcement learning framework that dynamically optimizes memory controller parameters using reward decomposition. ReLMXEL operates within the memory controller, leveraging detailed memory behavior metrics to guide decision-making. Experimental evaluations across diverse workloads demonstrate consistent performance gains over baseline configurations, with refinements driven by workload-specific memory access behaviour. By incorporating explainability into the learning process, ReLMXEL not only enhances performance but also increases the transparency of control decisions, paving the way for more accountable and adaptive memory system designs.
【8】WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptation
标题：WINFlowNets：机器人学和机器故障自适应生成流网络的热身集成网络训练
链接：https://arxiv.org/abs/2603.17301
摘要：连续场景的生成流网络（CFlowNets）通过使用流和检索网络学习随机策略，在解决顺序决策任务方面表现出了希望。尽管与最先进的强化学习（RL）算法相比，它们的效率得到了证明，但它们在机器人控制任务中的实际应用受到对检索网络预训练的依赖的限制。这种依赖性在动态机器人环境中提出了挑战，其中预训练数据可能不容易获得或不能代表当前环境。本文介绍了WINFlowNets，一个新的CFlowNets框架，使流和检索网络的共同训练。WINFlowNets从检索网络的预热阶段开始，以引导其策略，然后是共享的训练架构和共享的重放缓冲区，用于共同训练两个网络。在模拟机器人环境中的实验表明，WINFlowNets在平均奖励和训练稳定性方面超过了CFlowNets和最先进的RL算法。此外，WINFlowNets在故障环境中具有很强的自适应能力，使其适用于需要快速适应有限样本数据的任务。这些发现突出了WINFlowNets在动态和易发生故障的机器人系统中部署的潜力，在这些系统中，传统的预训练或样本低效数据收集可能是不切实际的。
摘要：Generative Flow Networks for continuous scenarios (CFlowNets) have shown promise in solving sequential decision-making tasks by learning stochastic policies using a flow and a retrieval network. Despite their demonstrated efficiency compared to state-of-the-art Reinforcement Learning (RL) algorithms, their practical application in robotic control tasks is constrained by the reliance on pre-training the retrieval network. This dependency poses challenges in dynamic robotic environments, where pre-training data may not be readily available or representative of the current environment. This paper introduces WINFlowNets, a novel CFlowNets framework that enables the co-training of flow and retrieval networks. WINFlowNets begins with a warm-up phase for the retrieval network to bootstrap its policy, followed by a shared training architecture and a shared replay buffer for co-training both networks. Experiments in simulated robotic environments demonstrate that WINFlowNets surpasses CFlowNets and state-of-the-art RL algorithms in terms of average reward and training stability. Furthermore, WINFlowNets exhibits strong adaptive capability in fault environments, making it suitable for tasks that demand quick adaptation with limited sample data. These findings highlight WINFlowNets' potential for deployment in dynamic and malfunction-prone robotic systems, where traditional pre-training or sample inefficient data collection may be impractical.
【9】Adaptive Contracts for Cost-Effective AI Delegation
标题：具有成本效益的人工智能委托的自适应合同
链接：https://arxiv.org/abs/2603.17212
备注：Comments are welcome
摘要：当组织通过按绩效付费合同将文本生成任务委托给人工智能提供商时，当评估存在噪音时，预期付款会增加。随着评估方法变得越来越复杂，降低噪声的经济效益往往被增加的评估成本所掩盖。在这项工作中，我们为AI委托引入了自适应合同，它允许在观察初始粗信号后有选择地进行详细评估，以节省资源。我们做了三套贡献：第一，我们提供了有效的算法计算最优自适应合同自然的假设下，或当核心问题的尺寸很小，并证明在一般的非结构化的情况下近似的硬度。然后，我们制定了随机适应性合同的替代模型，并讨论其优点和局限性。最后，我们使用问答和代码生成数据集实证证明了自适应性相对于非自适应基线的好处。
摘要 ：When organizations delegate text generation tasks to AI providers via pay-for-performance contracts, expected payments rise when evaluation is noisy. As evaluation methods become more elaborate, the economic benefits of decreased noise are often overshadowed by increased evaluation costs. In this work, we introduce adaptive contracts for AI delegation, which allow detailed evaluation to be performed selectively after observing an initial coarse signal in order to conserve resources. We make three sets of contributions: First, we provide efficient algorithms for computing optimal adaptive contracts under natural assumptions or when core problem dimensions are small, and prove hardness of approximation in the general unstructured case. We then formulate alternative models of randomized adaptive contracts and discuss their benefits and limitations. Finally, we empirically demonstrate the benefits of adaptivity over non-adaptive baselines using question-answering and code-generation datasets.
【10】OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation
标题：OPRA：在线数据修剪以实现高效检索模型适应
链接：https://arxiv.org/abs/2603.17205
摘要：特定领域的微调对于密集检索器是必不可少的，但并非所有的训练对都对学习过程做出了同样的贡献。我们介绍OPERA，数据修剪框架，利用这种异质性，以提高检索模型适应的有效性和效率。我们首先研究静态修剪（SP），它只保留高相似性的查询文档对，揭示了内在的质量覆盖率的权衡：排名（NDCG）提高，而检索（召回）可以降低由于减少查询多样性。为了解决这个问题，我们提出了一个两阶段的动态修剪（DP）策略，在整个训练过程中自适应地调整查询和文档级别的采样概率，优先考虑高质量的例子，同时保持对完整训练集的访问。跨越六个领域的八个数据集的评估证明了这两种方法的有效性：SP提高了标准微调的排名（NDCG@10 +0.5\%），而DP在排名（NDCG@10 +1.9\%）和检索（Recall@20 +0.7\%）方面都实现了最强的性能，所有方法的平均排名为1.38。这些发现扩展到Qwen 3-Embedding，一个基于LLM的密集检索器，确认了架构不可知的好处。值得注意的是，DP在不到标准微调所需训练时间的50%的时间内达到相当的性能。
摘要：Domain-specific finetuning is essential for dense retrievers, yet not all training pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that exploits this heterogeneity to improve both the effectiveness and efficiency of retrieval model adaptation. We first investigate static pruning (SP), which retains only high-similarity query-document pairs, revealing an intrinsic quality-coverage tradeoff: ranking (NDCG) improves while retrieval (Recall) can degrade due to reduced query diversity. To resolve this tradeoff, we propose a two-stage dynamic pruning (DP) strategy that adaptively modulates sampling probabilities at both query and document levels throughout training, prioritizing high-quality examples while maintaining access to the full training set. Evaluations across eight datasets spanning six domains demonstrate the effectiveness of both approaches: SP improves ranking over standard finetuning (NDCG@10 +0.5\%), while DP achieves the strongest performance on both ranking (NDCG@10 +1.9\%) and retrieval (Recall@20 +0.7\%), with an average rank of 1.38 across all methods. These findings scale to Qwen3-Embedding, an LLM-based dense retriever, confirming architecture-agnostic benefits. Notably, DP reaches comparable performance in less than 50\% of the training time required by standard finetuning.
【11】Hybrid Classical-Quantum Transfer Learning with Noisy Quantum Circuits
标题：具有有噪量子电路的混合经典-量子转移学习
链接：https://arxiv.org/abs/2603.16973
摘要：量子迁移学习将预训练的经典深度学习模型与量子电路相结合，以重用表达性特征表示，同时限制可训练参数的数量。在这项工作中，我们引入了一系列紧凑的量子转移学习架构，这些架构将变分量子分类器连接到冻结的卷积骨架上进行图像分类。我们实例化并评估了在PennyLane和Qiskit中实现的几个经典-量子混合模型，并将其与跨异构图像数据集的经典迁移学习基线进行了系统性比较。为了确保现实的评估，我们使用从IBM量子硬件规格校准的噪声模型以及真实的IBM量子硬件，在理想模拟和噪声仿真下评估所有方法。实验结果表明，所提出的量子迁移学习架构具有竞争力，在某些情况下具有卓越的准确性，同时相对于经典基线持续减少训练时间和能耗。在评估的方法中，基于PennyLane的实现在准确性和计算效率之间提供了最有利的权衡，这表明当特征提取仍然是经典的时，混合量子转移学习可以在现实的NISQ时代设置中提供实际好处。
摘要：Quantum transfer learning combines pretrained classical deep learning models with quantum circuits to reuse expressive feature representations while limiting the number of trainable parameters. In this work, we introduce a family of compact quantum transfer learning architectures that attach variational quantum classifiers to frozen convolutional backbones for image classification. We instantiate and evaluate several classical-quantum hybrid models implemented in PennyLane and Qiskit, and systematically compare them with a classical transfer-learning baseline across heterogeneous image datasets. To ensure a realistic assessment, we evaluate all approaches under both ideal simulation and noisy emulation using noise models calibrated from IBM quantum hardware specifications, as well as on real IBM quantum hardware. Experimental results show that the proposed quantum transfer learning architectures achieve competitive and, in several cases, superior accuracy while consistently reducing training time and energy consumption relative to the classical baseline. Among the evaluated approaches, PennyLane-based implementations provide the most favorable trade-off between accuracy and computational efficiency, suggesting that hybrid quantum transfer learning can offer practical benefits in realistic NISQ era settings when feature extraction remains classical.
强化学习(8篇)
【1】Federated Distributional Reinforcement Learning with Distributional Critic Regularization
标题：具有分布批判正规化的联邦分布强化学习
链接：https://arxiv.org/abs/2603.17820
备注：9 pages, 4 Figures, conference
摘要：联邦强化学习通常通过参数平均来聚合值函数或策略，这强调了预期回报，并可能掩盖在安全关键设置中重要的统计多模态和尾部行为。我们将联邦分布式强化学习（FedDistRL）形式化，其中客户端将分位数值函数批评者参数化并仅联合这些网络。我们还提出了TR-FedDistRL，它建立了一个每个客户端，风险意识的Wasserstein重心在时间缓冲区。这个局部重心提供了一个参考区域来约束参数平均临界点，确保必要的分布信息不会在联邦过程中被平均掉。分布式信赖域是围绕该引用的收缩挤压步骤。在固定的政策评估，可行性地图是非扩张和更新是收缩的探测集Wasserstein度量下评估。在土匪，多智能体网格世界和连续高速公路环境中的实验表明，平均涂抹减少，提高安全代理（灾难/事故率），并降低批评/政策漂移与平均导向和非联邦基线。
摘要：Federated reinforcement learning typically aggregates value functions or policies by parameter averaging, which emphasizes expected return and can obscure statistical multimodality and tail behavior that matter in safety-critical settings. We formalize federated distributional reinforcement learning (FedDistRL), where clients parametrize quantile value function critics and federate these networks only. We also propose TR-FedDistRL, which builds a per client, risk-aware Wasserstein barycenter over a temporal buffer. This local barycenter provides a reference region to constrain the parameter averaged critic, ensuring necessary distributional information is not averaged out during the federation process. The distributional trust region is implemented as a shrink-squash step around this reference. Under fixed-policy evaluation, the feasibility map is nonexpansive and the update is contractive in a probe-set Wasserstein metric under evaluation. Experiments on a bandit, multi-agent gridworld, and continuous highway environment show reduced mean-smearing, improved safety proxies (catastrophe/accident rate), and lower critic/policy drift versus mean-oriented and non-federated baselines.
【2】Complementary Reinforcement Learning
标题：补充强化学习
链接：https://arxiv.org/abs/2603.17621
备注：22 pages, 14 figures
摘要：强化学习（RL）已经成为训练基于LLM的代理的强大范例，但仍然受到低样本效率的限制，这不仅源于稀疏的结果反馈，而且还源于代理无法在事件中利用先前的经验。虽然增强代理人的历史经验提供了一个很有前途的补救措施，现有的方法遭受一个关键的弱点：从历史中提炼的经验要么是静态存储或未能与改进演员共同进化，导致经验和演员的不断发展的能力，减少其效用在培训过程中逐步错位。受神经科学中的互补学习系统的启发，我们提出了互补RL，以实现RL优化循环中经验提取器和政策参与者的无缝协同进化。具体而言，演员通过稀疏的基于结果的奖励进行优化，而经验提取器根据其提炼的经验是否明显有助于演员的成功进行优化，从而与演员不断增长的能力同步发展其经验管理策略。从经验上讲，互补RL的性能优于基于结果的代理RL基线，这些基线不从经验中学习，在单任务场景中实现了10%的性能提升，并在多任务设置中表现出强大的可扩展性。这些结果建立了互补RL作为一个有效的经验驱动的代理学习的范例。
摘要：Reinforcement Learning (RL) has emerged as a powerful paradigm for training LLM-based agents, yet remains limited by low sample efficiency, stemming not only from sparse outcome feedback but also from the agent's inability to leverage prior experience across episodes. While augmenting agents with historical experience offers a promising remedy, existing approaches suffer from a critical weakness: the experience distilled from history is either stored statically or fail to coevolve with the improving actor, causing a progressive misalignment between the experience and the actor's evolving capability that diminishes its utility over the course of training. Inspired by complementary learning systems in neuroscience, we present Complementary RL to achieve seamless co-evolution of an experience extractor and a policy actor within the RL optimization loop. Specifically, the actor is optimized via sparse outcome-based rewards, while the experience extractor is optimized according to whether its distilled experiences demonstrably contribute to the actor's success, thereby evolving its experience management strategy in lockstep with the actor's growing capabilities. Empirically, Complementary RL outperforms outcome-based agentic RL baselines that do not learn from experience, achieving 10% performance improvement in single-task scenarios and exhibits robust scalability in multi-task settings. These results establish Complementary RL as a paradigm for efficient experience-driven agent learning.
【3】Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routing
标题：基于物理信息的离线强化学习消除海上航线中的灾难性燃料浪费
链接：https://arxiv.org/abs/2603.17319
摘要：国际航运产生的温室气体排放量约占全球的3%，但航线仍然以启发式方法为主。我们提出了PIER（物理信息，节能，风险感知路由），这是一个离线强化学习框架，可以从基于历史船舶跟踪数据和海洋再分析产品的物理校准环境中学习燃油效率，安全感知路由策略，不需要在线模拟器。通过对墨西哥湾七条航线一整年（2023年）的AIS数据（每种方法840次）的验证，PIER相对于大圆航线减少了10%的平均二氧化碳排放量。然而，PIER的主要贡献是消除灾难性的燃料浪费：在4.8%的航程中，大圆航线会导致极端的燃料消耗（> 1.5倍中位数）; PIER将这一比例降低到0.5%，减少了9倍。每次航程燃料方差降低3.5倍（p<0.001），平均节省的bootstrap 95%CI [2.9%，15.7%]。对观察到的AIS血管行为的部分验证证实了与最快实际过境的一致性，同时表现出23.1倍的低方差。重要的是，PIER是预测独立的：与A* 路径优化不同，其波浪保护在现实预测不确定性下降低4.5倍，PIER仅使用局部观测值保持恒定性能。该框架结合了物理信息状态构建，演示增强离线数据和解耦的事后安全防护，这是一种转移到野火疏散，飞机轨迹优化和未映射地形中的自主导航的架构。
摘要：International shipping produces approximately 3% of global greenhouse gas emissions, yet voyage routing remains dominated by heuristic methods. We present PIER (Physics-Informed, Energy-efficient, Risk-aware routing), an offline reinforcement learning framework that learns fuel-efficient, safety-aware routing policies from physics-calibrated environments grounded in historical vessel tracking data and ocean reanalysis products, requiring no online simulator. Validated on one full year (2023) of AIS data across seven Gulf of Mexico routes (840 episodes per method), PIER reduces mean CO2 emissions by 10% relative to great-circle routing. However, PIER's primary contribution is eliminating catastrophic fuel waste: great-circle routing incurs extreme fuel consumption (>1.5x median) in 4.8% of voyages; PIER reduces this to 0.5%, a 9-fold reduction. Per-voyage fuel variance is 3.5x lower (p<0.001), with bootstrap 95% CI for mean savings [2.9%, 15.7%]. Partial validation against observed AIS vessel behavior confirms consistency with the fastest real transits while exhibiting 23.1x lower variance. Crucially, PIER is forecast-independent: unlike A* path optimization whose wave protection degrades 4.5x under realistic forecast uncertainty, PIER maintains constant performance using only local observations. The framework combines physics-informed state construction, demonstration-augmented offline data, and a decoupled post-hoc safety shield, an architecture that transfers to wildfire evacuation, aircraft trajectory optimization, and autonomous navigation in unmapped terrain.
【4】Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations
标题：对比推理对齐：来自隐藏表示的强化学习
链接：https://arxiv.org/abs/2603.17305
摘要：我们提出了CRAFT，一个红队对齐框架，利用模型推理能力和隐藏的表示，以提高对越狱攻击的鲁棒性。与主要在输出级别操作的先前防御不同，CRAFT通过显式优化在隐藏状态空间上定义的目标来对齐大型推理模型以生成安全感知推理轨迹。在方法上，CRAFT将对比表示学习与强化学习相结合，以分离安全和不安全的推理轨迹，从而产生一个潜在空间几何，支持强大的推理级安全对齐。从理论上讲，我们表明，将潜在文本的一致性GRPO消除了表面上一致的政策，排除它们作为局部最优。从经验上讲，我们使用两个强大的推理模型Qwen 3 - 4 B-Thinking和R1-Distill-Llama-8B在多个安全基准上评估CRAFT，在这些模型中，CRAFT始终优于最先进的防御措施，如IPO和SafeKey。值得注意的是，与基础模型相比，CRAFT在推理安全性方面平均提高了79.0%，在最终响应安全性方面平均提高了87.7%，证明了隐藏空间推理对齐的有效性。
摘要：We propose CRAFT, a red-teaming alignment framework that leverages model reasoning capabilities and hidden representations to improve robustness against jailbreak attacks. Unlike prior defenses that operate primarily at the output level, CRAFT aligns large reasoning models to generate safety-aware reasoning traces by explicitly optimizing objectives defined over the hidden state space. Methodologically, CRAFT integrates contrastive representation learning with reinforcement learning to separate safe and unsafe reasoning trajectories, yielding a latent-space geometry that supports robust, reasoning-level safety alignment. Theoretically, we show that incorporating latent-textual consistency into GRPO eliminates superficially aligned policies by ruling them out as local optima. Empirically, we evaluate CRAFT on multiple safety benchmarks using two strong reasoning models, Qwen3-4B-Thinking and R1-Distill-Llama-8B, where it consistently outperforms state-of-the-art defenses such as IPO and SafeKey. Notably, CRAFT delivers an average 79.0% improvement in reasoning safety and 87.7% improvement in final-response safety over the base models, demonstrating the effectiveness of hidden-space reasoning alignment.
【5】Shielded Reinforcement Learning Under Dynamic Temporal Logic Constraints
标题：动态时态逻辑约束下的屏蔽强化学习
链接：https://arxiv.org/abs/2603.17152
备注：7 pages, 3 figures, 2026 IEEE American Control Conference (ACC)
摘要：强化学习（RL）在各种机器人应用中显示出了希望，但由于安全和操作限制，其在实际系统中的部署仍然有限。近年来，安全RL领域受到了相当大的关注，其重点是在整个学习过程中施加安全约束。然而，实际系统通常需要比安全更复杂的约束，例如定期充电或对特定区域的有时限访问。在学习过程中强加这样的时空任务仍然是一个挑战。信号时序逻辑（STL）是一种用于描述实值信号时序特性的形式化语言，它提供了一种表达这种复杂任务的方法。在本文中，我们提出了一个框架，利用顺序控制障碍函数和无模型RL，以确保在整个学习过程中满足给定的STL任务。我们的方法通过执行丰富的STL规范扩展了传统的安全约束，这可能涉及访问具有未知轨迹的动态目标。我们还证明了我们的框架的有效性，通过各种模拟。
摘要 ：Reinforcement Learning (RL) has shown promise in various robotics applications, yet its deployment on real systems is still limited due to safety and operational constraints. The safe RL field has gained considerable attention in recent years, which focuses on imposing safety constraints throughout the learning process. However, real systems often require more complex constraints than just safety, such as periodic recharging or time-bounded visits to specific regions. Imposing such spatio-temporal tasks during learning still remains a challenge. Signal Temporal Logic (STL) is a formal language for specifying temporal properties of real-valued signals and provides a way to express such complex tasks. In this paper, we propose a framework that leverages sequential control barrier functions and model-free RL to ensure that the given STL tasks are satisfied throughout the learning process. Our method extends beyond traditional safety constraints by enforcing rich STL specifications, which can involve visits to dynamic targets with unknown trajectories. We also demonstrate the effectiveness of our framework through various simulations.
【6】CircuitBuilder: From Polynomials to Circuits via Reinforcement Learning
标题：CircuitBuilder：通过强化学习从多项到电路
链接：https://arxiv.org/abs/2603.17075
备注：ICLR 2026 Workshop on AI with Recursive Self-Improvement
摘要：受自动证明生成和Valiant的VP与VNP猜想的启发，我们研究了使用加法和乘法门来发现计算多项式的高效算术电路的问题。我们将这个问题表述为一个单人游戏，其中RL代理试图在固定数量的操作内构建电路。我们实现了一个AlphaZero风格的训练循环，并比较了两种方法：基于蒙特卡罗树搜索的最接近策略优化（PPO+MCTS）和软演员评论（SAC）。SAC在两个变量目标上实现了最高的成功率，而PPO+MCTS扩展到三个变量，并在更困难的情况下表现出稳定的改进。这些结果表明，多项式电路合成是一个紧凑的，可验证的设置，研究自我改进的搜索策略。
摘要：Motivated by auto-proof generation and Valiant's VP vs. VNP conjecture, we study the problem of discovering efficient arithmetic circuits to compute polynomials, using addition and multiplication gates. We formulate this problem as a single-player game, where an RL agent attempts to build the circuit within a fixed number of operations. We implement an AlphaZero-style training loop and compare two approaches: Proximal Policy Optimization with Monte Carlo Tree Search (PPO+MCTS) and Soft Actor-Critic (SAC). SAC achieves the highest success rates on two-variable targets, while PPO+MCTS scales to three variables and demonstrates steady improvement on harder instances. These results suggest that polynomial circuit synthesis is a compact, verifiable setting for studying self-improving search policies.
【7】MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learning
标题：MHPO：用于稳定强化学习的调制危险感知策略优化
链接：https://arxiv.org/abs/2603.16929
备注：18 pages, 3 figures, 4 tables
摘要：调节重要性比率对于基于组相对策略优化（GRPO）的框架的训练稳定性至关重要。然而，流行的比率控制方法，如硬裁剪，遭受不可微的边界和消失的梯度区域，无法保持梯度保真度。此外，这些方法缺乏自适应抑制极端偏差的风险感知机制，使优化过程容易受到突然的策略变化的影响。为了应对这些挑战，我们提出了调制危险感知策略优化（MHPO），这是一种为强大和稳定的强化学习而设计的新框架。建议MHPO引入对数保真度调制器（LFM）映射到一个有界的，可微的域的无界的重要性比率。这种机制有效地防止了高方差离群值令牌破坏损失格局，同时确保了全局梯度稳定性。作为补充，解耦危险惩罚（DHP）集成了生存分析中的累积危险函数，以独立调节积极和消极的政策转变。通过用风险感知惩罚来塑造优化景观，所提出的MHPO实现了对不对称政策转变的细粒度监管，同时减轻了过度扩张导致的模式崩溃，并防止了稳定的信任区域内灾难性收缩导致的政策侵蚀。对基于文本和视觉语言任务的各种推理基准的广泛评估表明，MHPO始终优于现有方法，在显著提高训练稳定性的同时实现了卓越的性能。
摘要：Regulating the importance ratio is critical for the training stability of Group Relative Policy Optimization (GRPO) based frameworks. However, prevailing ratio control methods, such as hard clipping, suffer from non-differentiable boundaries and vanishing gradient regions, failing to maintain gradient fidelity. Furthermore, these methods lack a hazard-aware mechanism to adaptively suppress extreme deviations, leaving the optimization process vulnerable to abrupt policy shifts. To address these challenges, we propose Modulated Hazard-aware Policy Optimization (MHPO), a novel framework designed for robust and stable reinforcement learning. The proposed MHPO introduces a Log-Fidelity Modulator (LFM) to map unbounded importance ratios into a bounded, differentiable domain. This mechanism effectively prevents high-variance outlier tokens from destabilizing the loss landscape while ensuring global gradient stability. Complementarily, a Decoupled Hazard Penalty (DHP) integrates cumulative hazard functions from survival analysis to independently regulate positive and negative policy shifts. By shaping the optimization landscape with hazard-aware penalties, the proposed MHPO achieves fine-grained regulation of asymmetric policy shifts simultaneously mitigating mode collapse from over-expansion and preventing policy erosion from catastrophic contraction within a stabilized trust region. Extensive evaluations on diverse reasoning benchmarks across both text-based and vision-language tasks demonstrate that MHPO consistently outperforms existing methods, achieving superior performance while significantly enhancing training stability.
【8】Multi-Agent Reinforcement Learning for Dynamic Pricing: Balancing Profitability,Stability and Fairness
标题：动态定价的多智能体强化学习：平衡盈利性、稳定性和公平性
链接：https://arxiv.org/abs/2603.16888
摘要：在竞争激烈的零售市场中，动态定价需要适应波动的需求和竞争对手行为的策略。在这项工作中，我们提出了一个系统的实证评估多智能体强化学习（MARL）的方法-特别是MAPPO和MADDPG-竞争下的动态价格优化。使用来自真实世界的零售数据的模拟市场环境，我们基准这些算法对独立DDPG（IDDPG）基线，广泛使用的独立学习者在MARL文献。我们评估利润表现，随机种子的稳定性，公平性和培训效率。我们的研究结果表明，MAPPO始终实现最高的平均回报率和低方差，为有竞争力的价格优化提供了一个稳定和可重复的方法，而MADDPG实现略低的利润，但最公平的利润分配代理。这些研究结果表明，MARL方法，特别是MAPPO，提供了一个可扩展的和稳定的替代独立的学习方法，动态零售定价。
摘要：Dynamic pricing in competitive retail markets requires strategies that adapt to fluctuating demand and competitor behavior. In this work, we present a systematic empirical evaluation of multi-agent reinforcement learning (MARL) approaches-specifically MAPPO and MADDPG-for dynamic price optimization under competition. Using a simulated marketplace environment derived from real-world retail data, we benchmark these algorithms against an Independent DDPG (IDDPG) baseline, a widely used independent learner in MARL literature. We evaluate profit performance, stability across random seeds, fairness, and training efficiency. Our results show that MAPPO consistently achieves the highest average returns with low variance, offering a stable and reproducible approach for competitive price optimization, while MADDPG achieves slightly lower profit but the fairest profit distribution among agents. These findings demonstrate that MARL methods-particularly MAPPO-provide a scalable and stable alternative to independent learning approaches for dynamic retail pricing.
元学习(1篇)
【1】MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild
标题：MetaClaw：Just Talk --一种在野外元学习和进化的代理
链接：https://arxiv.org/abs/2603.17187
摘要：大型语言模型（LLM）代理越来越多地用于复杂任务，但部署的代理通常保持静态，无法适应用户需求的变化。这就在持续服务的需求和更新能力以适应不断变化的任务分配的必要性之间产生了紧张关系。在像OpenClaw这样的平台上，可以处理20多个渠道的不同工作负载，现有的方法要么存储原始轨迹而不提取知识，要么维护静态技能库，要么需要中断停机进行再培训。我们提出了MetaClaw，一个持续的元学习框架，共同发展一个基本的LLM政策和可重用的行为技能库。MetaClaw采用两种互补的机制。技能驱动的快速适应通过LLM进化器分析故障轨迹，以综合新技能，实现零停机时间的即时改进。主动策略优化通过云LoRA微调和具有过程奖励模型的强化学习（RL-PRM）执行基于梯度的更新。这是在用户不活动的窗口期间由系统元学习管理器（OMLS）触发的，它监视系统不活动和日历数据。这些机制是相辅相成的：完善的政策为技能综合提供更好的轨迹，而更丰富的技能为政策优化提供更高质量的数据。为了防止数据污染，版本控制机制将支持数据和查询数据分开。MetaClaw构建在基于代理的架构上，无需本地GPU即可扩展到生产规模的LLM。在MetaClaw-Bench和AutoResearchClaw上的实验表明，技能驱动的自适应相对提高了高达32%的准确率。完整的流水线将Kimi-K2.5的准确率从21.4%提高到40.6%，并将复合稳健性提高了18.3%。代码可在https://github.com/aiming-lab/MetaClaw上获得。
摘要：Large language model (LLM) agents are increasingly used for complex tasks, yet deployed agents often remain static, failing to adapt as user needs evolve. This creates a tension between the need for continuous service and the necessity of updating capabilities to match shifting task distributions. On platforms like OpenClaw, which handle diverse workloads across 20+ channels, existing methods either store raw trajectories without distilling knowledge, maintain static skill libraries, or require disruptive downtime for retraining. We present MetaClaw, a continual meta-learning framework that jointly evolves a base LLM policy and a library of reusable behavioral skills. MetaClaw employs two complementary mechanisms. Skill-driven fast adaptation analyzes failure trajectories via an LLM evolver to synthesize new skills, enabling immediate improvement with zero downtime. Opportunistic policy optimization performs gradient-based updates via cloud LoRA fine-tuning and Reinforcement Learning with a Process Reward Model (RL-PRM). This is triggered during user-inactive windows by the Opportunistic Meta-Learning Scheduler (OMLS), which monitors system inactivity and calendar data. These mechanisms are mutually reinforcing: a refined policy generates better trajectories for skill synthesis, while richer skills provide higher-quality data for policy optimization. To prevent data contamination, a versioning mechanism separates support and query data. Built on a proxy-based architecture, MetaClaw scales to production-size LLMs without local GPUs. Experiments on MetaClaw-Bench and AutoResearchClaw show that skill-driven adaptation improves accuracy by up to 32% relative. The full pipeline advances Kimi-K2.5 accuracy from 21.4% to 40.6% and increases composite robustness by 18.3%. Code is available at https://github.com/aiming-lab/MetaClaw.
符号|符号学习(1篇)
【1】Minimum-Action Learning: Energy-Constrained Symbolic Model Selection for Physical Law Identification from Noisy Data
标题：最小动作学习：从噪音数据中识别物理定律的能量约束符号模型选择
链接：https://arxiv.org/abs/2603.16951
备注：28 pages, 10 figures, https://github.com/martinfrasch/minAction_kepler
摘要：从嘈杂的观测数据中识别物理定律是科学机器学习的核心挑战。我们提出了最小动作学习（MAL），一个框架，选择符号力法从一个预先指定的基础库，通过最小化的三重动作功能相结合的轨迹重建，建筑稀疏，和节能执法。宽模板加速匹配技术将噪声方差降低了10，000倍，将一个棘手的问题（SNR ~0.02）转化为一个可学习的问题（SNR ~1.6）;这种预处理是所有测试方法（包括SINDy变体）共享的关键使能器。在两个基准-开普勒引力和胡克定律- MAL恢复正确的力的法律与开普勒指数p = 3.01 +/- 0.01在~0.07千瓦时（40%减少与预测误差只有基线）。原始正确的基础率是40%的开普勒和90%的胡克;一个能量守恒为基础的标准区分在所有情况下的真实力法，产生100%的管道级识别。基础库敏感性实验表明，近混杂降低选择（20%与添加r^{-2.5}和r^{-1.5}），而远添加是无害的，和保守性诊断仍然提供信息，即使正确的基础是缺席。与噪声鲁棒的SINDy变体，Hamilton神经网络和拉格朗日神经网络的直接比较证实了MAL的独特利基：可解释的，能量约束的模型选择，将符号基识别与动态推出验证相结合。
摘要：Identifying physical laws from noisy observational data is a central challenge in scientific machine learning. We present Minimum-Action Learning (MAL), a framework that selects symbolic force laws from a pre-specified basis library by minimizing a Triple-Action functional combining trajectory reconstruction, architectural sparsity, and energy-conservation enforcement. A wide-stencil acceleration-matching technique reduces noise variance by 10,000x, transforming an intractable problem (SNR ~0.02) into a learnable one (SNR ~1.6); this preprocessing is the critical enabler shared by all methods tested, including SINDy variants. On two benchmarks -- Kepler gravity and Hooke's law -- MAL recovers the correct force law with Kepler exponent p = 3.01 +/- 0.01 at ~0.07 kWh (40% reduction vs. prediction-error-only baselines). The raw correct-basis rate is 40% for Kepler and 90% for Hooke; an energy-conservation-based criterion discriminates the true force law in all cases, yielding 100% pipeline-level identification. Basis library sensitivity experiments show that near-confounders degrade selection (20% with added r^{-2.5} and r^{-1.5}), while distant additions are harmless, and the conservation diagnostic remains informative even when the correct basis is absent. Direct comparison with noise-robust SINDy variants, Hamiltonian Neural Networks, and Lagrangian Neural Networks confirms MAL's distinct niche: interpretable, energy-constrained model selection that combines symbolic basis identification with dynamical rollout validation.
医学相关(8篇)
【1】Predicting Trajectories of Long COVID in Adult Women: The Critical Role of Causal Disentanglement
标题：预测成年女性长期COVID的轨迹：因果关系解开的关键作用
链接：https://arxiv.org/abs/2603.17722
摘要：早期预测SARS-CoV-2严重程度的急性后遗症对女性健康来说是一个关键挑战，特别是考虑到PASC和常见激素转换（如绝经）之间的诊断重叠。识别和解释这些混杂因素对于准确的长期轨迹预测至关重要。我们对来自NIH RECOVER数据集的1，155名女性（平均年龄61岁）进行了回顾性研究。通过将静态临床特征与四周的纵向可穿戴数据（监测心脏活动和睡眠）相结合，我们开发了一种基于大型语言模型的因果网络，以预测未来的PASC分数。我们的框架在临床严重程度预测中达到了86.7%的精度。我们的因果归因分析证明了该模型区分活动性病理和基线噪声的能力：直接指标如呼吸困难和不适达到最大显着性（1.00），而混淆因素如绝经和糖尿病则被成功抑制，显着性得分低于0.27。
摘要：Early prediction of Post-Acute Sequelae of SARS-CoV-2 severity is a critical challenge for women's health, particularly given the diagnostic overlap between PASC and common hormonal transitions such as menopause. Identifying and accounting for these confounding factors is essential for accurate long-term trajectory prediction. We conducted a retrospective study of 1,155 women (mean age 61) from the NIH RECOVER dataset. By integrating static clinical profiles with four weeks of longitudinal wearable data (monitoring cardiac activity and sleep), we developed a causal network based on a Large Language Model to predict future PASC scores. Our framework achieved a precision of 86.7\% in clinical severity prediction. Our causal attribution analysis demonstrate the model's ability to differentiate between active pathology and baseline noise: direct indicators such as breathlessness and malaise reached maximum saliency (1.00), while confounding factors like menopause and diabetes were successfully suppressed with saliency scores below 0.27.
【2】Pathology-Aware Multi-View Contrastive Learning for Patient-Independent ECG Reconstruction
标题：病理感知的多视图对比学习用于患者独立的心电图重建
链接：https://arxiv.org/abs/2603.17248
摘要：由于解剖变异性，从缩减的导联集合重建12导联心电图（ECG）是一个不适定的逆问题。标准的深度学习方法通常会忽略潜在的心脏病理学，从而丢失心前区导联中的重要形态。我们提出了病理感知多视图对比学习，一个框架，规范化的潜在空间，通过病理流形。我们的架构将高保真时域波形与通过监督对比对齐学习的病理感知嵌入相结合。通过最大化潜在表示和临床标签之间的互信息，框架学习过滤解剖学“讨厌”变量。在PTB-XL数据集上，我们的方法实现了大约。与患者非依赖性环境中的最新模型相比，RMSE降低76\%。PTB诊断数据库的交叉数据集评价证实了优越的泛化能力，弥合了硬件便携性和诊断级重建之间的差距。
摘要：Reconstructing a 12-lead electrocardiogram (ECG) from a reduced lead set is an ill-posed inverse problem due to anatomical variability. Standard deep learning methods often ignore underlying cardiac pathology losing vital morphology in precordial leads. We propose Pathology-Aware Multi-View Contrastive Learning, a framework that regularizes the latent space through a pathological manifold. Our architecture integrates high-fidelity time-domain waveforms with pathology-aware embeddings learned via supervised contrastive alignment. By maximizing mutual information between latent representations and clinical labels, the framework learns to filter anatomical "nuisance" variables. On the PTB-XL dataset, our method achieves approx. 76\% reduction in RMSE compared to state-of-the-art model in patient-independent setting. Cross-dataset evaluation on the PTB Diagnostic Database confirms superior generalization, bridging the gap between hardware portability and diagnostic-grade reconstruction.
【3】On the Cone Effect and Modality Gap in Medical Vision-Language Embeddings
标题：医学视觉语言嵌入中的圆锥效应和情态差距
链接：https://arxiv.org/abs/2603.17246
摘要：视觉语言模型（VLM）表现出一种典型的“锥效应”，其中非线性编码器将嵌入映射到表示空间的高度集中区域，从而导致称为模态间隙的跨模态分离。虽然这一现象已被广泛观察到，但其对监督多模态学习的实际影响-特别是在医疗领域-仍然不清楚。在这项工作中，我们引入了一种轻量级的事后机制，该机制保持预训练的VLM编码器冻结，同时通过单个超参数{λ}连续控制跨模态分离。这使得能够系统地分析模态差距如何影响下游多模态性能，而无需昂贵的再培训。我们在监督的多模态设置中评估了跨不同医学和自然数据集的通才（CLIP，SigLIP）和医学专业（BioMedCLIP，MedSigLIP）模型。结果一致表明，减少过多的模态间隙可以提高下游性能，医疗数据集对间隙调制表现出更强的敏感性;然而，完全折叠间隙并不总是最佳的，并且中间的任务依赖性分离会产生最佳结果。这些研究结果的位置的模态间隙作为一个可调的属性的多模态表示，而不是一个数量，应普遍最小化。
摘要：Vision-Language Models (VLMs) exhibit a characteristic "cone effect" in which nonlinear encoders map embeddings into highly concentrated regions of the representation space, contributing to cross-modal separation known as the modality gap. While this phenomenon has been widely observed, its practical impact on supervised multimodal learning -particularly in medical domains- remains unclear. In this work, we introduce a lightweight post-hoc mechanism that keeps pretrained VLM encoders frozen while continuously controlling cross-modal separation through a single hyperparameter {λ}. This enables systematic analysis of how the modality gap affects downstream multimodal performance without expensive retraining. We evaluate generalist (CLIP, SigLIP) and medically specialized (BioMedCLIP, MedSigLIP) models across diverse medical and natural datasets in a supervised multimodal settings. Results consistently show that reducing excessive modality gap improves downstream performance, with medical datasets exhibiting stronger sensitivity to gap modulation; however, fully collapsing the gap is not always optimal, and intermediate, task-dependent separation yields the best results. These findings position the modality gap as a tunable property of multimodal representations rather than a quantity that should be universally minimized.
【4】SA-CycleGAN-2.5D: Self-Attention CycleGAN with Tri-Planar Context for Multi-Site MRI Harmonization
标题：SA-CycleGAN-2.5D：具有三平面上下文的自我注意CycleGAN，用于多站点MRI协调
链接：https://arxiv.org/abs/2603.17219
备注：12 pages, 5 figures, 5 tables. Submitted to MICCAI 2026
摘要：多部位神经成像分析从根本上受到扫描仪诱导的协变量偏移的混淆，其中体素强度$P（\mathbf{x}）$的边缘分布在采集协议之间非线性变化，而条件解剖结构$P（\mathbf{y}|\mathbf{x}）$保持不变。这对放射组学再现性特别有害，其中采集方差通常超过生物病理学方差。现有的统计统一方法（例如，ComBat）在特征空间中运行，排除了空间下游任务，而标准的深度学习方法理论上受到局部有效感受野（ERF）的限制，无法对场强偏差的全局强度相关性进行建模。 我们提出了SA-CycleGAN-2.5D，这是一个由Ben-David等人的$HΔH$-发散界限激发的域适应框架，集成了三个架构创新：（1）2.5D三平面流形注入保持通过平面梯度$\nabla_z$在$O（HW）$复杂度;（2）U-ResNet生成器具有密集体素到体素的自关注，超过了$O（\sqrt{L}）CNN的$感受野限制，以模拟全局扫描仪场偏差;（3）一个约束Lipschitz常数（$K_D \le 1$）的谱归一化的优化算法。在两个机构领域（BraTS和UPenn-GBM）的654名胶质瘤患者上进行评估，我们的方法将最大平均离散度（MMD）降低了99.1%（1.729美元至0.015美元），并将领域分类器的准确性降低到接近机会（59.7%）。消融证实，全局注意力是统计学上必不可少的（科恩的$d = 1.32$，$p < 0.001$）更难异质到同质的翻译方向。通过桥接2D效率和3D一致性，我们的框架产生体素级协调图像，保留肿瘤病理生理学，实现可重复的多中心放射组学分析。
摘要：Multi-site neuroimaging analysis is fundamentally confounded by scanner-induced covariate shifts, where the marginal distribution of voxel intensities $P(\mathbf{x})$ varies non-linearly across acquisition protocols while the conditional anatomy $P(\mathbf{y}|\mathbf{x})$ remains constant. This is particularly detrimental to radiomic reproducibility, where acquisition variance often exceeds biological pathology variance. Existing statistical harmonization methods (e.g., ComBat) operate in feature space, precluding spatial downstream tasks, while standard deep learning approaches are theoretically bounded by local effective receptive fields (ERF), failing to model the global intensity correlations characteristic of field-strength bias. We propose SA-CycleGAN-2.5D, a domain adaptation framework motivated by the $HΔH$-divergence bound of Ben-David et al., integrating three architectural innovations: (1) A 2.5D tri-planar manifold injection preserving through-plane gradients $\nabla_z$ at $O(HW)$ complexity; (2) A U-ResNet generator with dense voxel-to-voxel self-attention, surpassing the $O(\sqrt{L})$ receptive field limit of CNNs to model global scanner field biases; and (3) A spectrally-normalized discriminator constraining the Lipschitz constant ($K_D \le 1$) for stable adversarial optimization. Evaluated on 654 glioma patients across two institutional domains (BraTS and UPenn-GBM), our method reduces Maximum Mean Discrepancy (MMD) by 99.1% ($1.729 \to 0.015$) and degrades domain classifier accuracy to near-chance (59.7%). Ablation confirms that global attention is statistically essential (Cohen's $d = 1.32$, $p < 0.001$) for the harder heterogeneous-to-homogeneous translation direction. By bridging 2D efficiency and 3D consistency, our framework yields voxel-level harmonized images that preserve tumor pathophysiology, enabling reproducible multi-center radiomic analysis.
【5】Pixel-level Counterfactual Contrastive Learning for Medical Image Segmentation
标题：医学图像分割的像素级反事实对比学习
链接：https://arxiv.org/abs/2603.17110
备注：Accepted at ISBI-2026 (oral presentation)
摘要 ：图像分割依赖于大型注释数据集，这是昂贵的和缓慢的生产。银标准（AI生成的）标签更容易获得，但它们有引入偏倚的风险。自监督学习只需要图像，已经成为预训练的关键。最近的工作相结合的对比学习与反事实生成提高了分类的表示学习，但不容易扩展到像素级的任务。我们提出了一个管道，通过双视图（DVD-CL）和多视图（MVD-CL）方法将反事实生成与密集对比学习相结合，以及利用可用银标准注释的监督变体。还介绍了一种新的可视化算法，彩色编码高分辨率叠加图（CHRO图）。实验表明，无注释的DVD-CL优于其他密集对比学习方法，而使用银标准标签的监督变体直接优于银标准标签数据的训练，在具有挑战性的数据上实现了94%的DSC。这些结果强调，像素级对比学习，增强了反事实和银标准注释，提高了鲁棒性收购和病理变化。
摘要：Image segmentation relies on large annotated datasets, which are expensive and slow to produce. Silver-standard (AI-generated) labels are easier to obtain, but they risk introducing bias. Self-supervised learning, needing only images, has become key for pre-training. Recent work combining contrastive learning with counterfactual generation improves representation learning for classification but does not readily extend to pixel-level tasks. We propose a pipeline combining counterfactual generation with dense contrastive learning via Dual-View (DVD-CL) and Multi-View (MVD-CL) methods, along with supervised variants that utilize available silver-standard annotations. A new visualisation algorithm, the Color-coded High Resolution Overlay map (CHRO-map) is also introduced. Experiments show annotation-free DVD-CL outperforms other dense contrastive learning methods, while supervised variants using silver-standard labels outperform training on the silver-standard labeled data directly, achieving $\sim$94% DSC on challenging data. These results highlight that pixel-level contrastive learning, enhanced by counterfactuals and silver-standard annotations, improves robustness to acquisition and pathological variations.
【6】A Novel end-to-end Digital Health System Using Deep Learning-based ECG Analysis
标题：基于深度学习心电分析的新型端到端数字健康系统
链接：https://arxiv.org/abs/2603.16891
备注：Preprint submitted to the International Journal of Information Management Data Insights (Elsevier). 15 pages, 5 figures
摘要：本研究介绍了AI-HEART，这是一种基于云的信息系统，用于管理和分析长时间动态心电图（ECG）记录并支持临床医生决策。该平台运行一个端到端的管道，该管道可以摄取多日三导联ECG，标准化输入，执行信号预处理，并应用专用的深度神经网络进行波形描绘，噪声/质量检测以及搏动和节律水平的多类心律失常分类。为了解决类别不平衡和现实世界的信号变异性，模型开发将大型临床注释数据集与专家在环策展和生成增强相结合，用于代表性不足的节律。对三导联动态心电图数据的经验评估表明，勾画准确度足以进行自动间期测量，噪声检测可靠地标记质量差的段，心律失常分类实现了高特异性，在常见和罕见心律之间具有临床有用的宏观平均性能。除了预测准确性之外，AI-HEART还提供了一种可扩展的部署方法，用于将AI集成到常规ECG服务中，实现可追溯的输出，记录和衍生注释的可扩展友好存储，以及临床医生审查/编辑，以捕获受控模型改进的反馈。研究结果证明了噪声感知AI-ECG平台作为数字健康信息系统的技术可行性和操作价值。
摘要：This study presents AI-HEART, a cloud-based information system for managing and analysing long-duration ambulatory electrocardiogram (ECG) recordings and supporting clinician decision-making. The platform operationalises an end-to-end pipeline that ingests multi-day three-lead ECGs, normalises inputs, performs signal preprocessing, and applies dedicated deep neural networks for wave delineation, noise/quality detection, and beat- and rhythm-level multi-class arrhythmia classification. To address class imbalance and real-world signal variability, model development combines large clinically annotated datasets with expert-in-the-loop curation and generative augmentation for under-represented rhythms. Empirical evaluation on three-lead ambulatory ECG data shows that delineation accuracy is sufficient for automated interval measurement, noise detection reliably flags poor-quality segments, and arrhythmia classification achieves high specificity with clinically useful macro-averaged performance across common and rarer rhythms. Beyond predictive accuracy, AI-HEART provides a scalable deployment approach for integrating AI into routine ECG services, enabling traceable outputs, audit-friendly storage of recordings and derived annotations, and clinician review/editing that captures feedback for controlled model improvement. The findings demonstrate the technical feasibility and operational value of a noise-aware AI-ECG platform as a digital health information system.
【7】NeuroNarrator: A Generalist EEG-to-Text Foundation Model for Clinical Interpretation via Spectro-Spatial Grounding and Temporal State-Space Reasoning
标题：NeuroNarator：通过谱空间基础和时间状态空间推理进行临床解释的通用脑电到文本基础模型
链接：https://arxiv.org/abs/2603.16880
摘要：脑电图（EEG）提供了一个高时间分辨率的非侵入性神经动力学窗口，在临床神经科学研究中起着关键作用。尽管存在这种潜力，但EEG分析的主流计算方法仍然主要局限于特定任务的分类目标或粗粒度模式识别，为临床有意义的解释提供有限的支持。为了解决这些局限性，我们引入了NeuroNarrator，这是第一个通用的EEG到文本基础模型，旨在将电生理片段转换为精确的临床叙述。该框架的基石是NeuroCorpus-160 K的管理，这是第一个将超过160，000个EEG片段与结构化的、基于临床的自然语言描述进行协调的大规模资源配对。我们的架构首先通过严格的对比目标将时间EEG波形与空间地形图对齐，建立光谱空间接地表示。在此基础上，我们通过状态空间启发的公式来调节大型语言模型，该公式整合了历史时间和光谱背景，以支持连贯的临床叙事生成。这种方法在连续信号动态和离散临床语言之间建立了一个原则性的桥梁，实现了可解释的叙述生成，便于专家解释并支持临床报告工作流程。对不同基准和zero-shot传输任务的广泛评估突出了NeuroNarrator整合时间、频谱和空间动态的能力，将其定位为电生理数据的时频感知、开放式临床解释的基础框架。
摘要：Electroencephalography (EEG) provides a non-invasive window into neural dynamics at high temporal resolution and plays a pivotal role in clinical neuroscience research. Despite this potential, prevailing computational approaches to EEG analysis remain largely confined to task-specific classification objectives or coarse-grained pattern recognition, offering limited support for clinically meaningful interpretation. To address these limitations, we introduce NeuroNarrator, the first generalist EEG-to-text foundation model designed to translate electrophysiological segments into precise clinical narratives. A cornerstone of this framework is the curation of NeuroCorpus-160K, the first harmonized large-scale resource pairing over 160,000 EEG segments with structured, clinically grounded natural-language descriptions. Our architecture first aligns temporal EEG waveforms with spatial topographic maps via a rigorous contrastive objective, establishing spectro-spatially grounded representations. Building on this grounding, we condition a Large Language Model through a state-space-inspired formulation that integrates historical temporal and spectral context to support coherent clinical narrative generation. This approach establishes a principled bridge between continuous signal dynamics and discrete clinical language, enabling interpretable narrative generation that facilitates expert interpretation and supports clinical reporting workflows. Extensive evaluations across diverse benchmarks and zero-shot transfer tasks highlight NeuroNarrator's capacity to integrate temporal, spectral, and spatial dynamics, positioning it as a foundational framework for time-frequency-aware, open-ended clinical interpretation of electrophysiological data.
【8】Halfway to 3D: Ensembling 2.5D and 3D Models for Robust COVID-19 CT Diagnosis
标题：3D的中途：集成2.5D和3D模型，实现稳健的COVID-19 CT诊断
链接：https://arxiv.org/abs/2603.14832
摘要：我们提出了一个深度学习框架，用于从胸部CT扫描中进行COVID-19检测和疾病分类，该框架集成了2.5D和3D表示，以捕获互补的切片级和体积信息。2.5D分支使用DINOv 3 Vision Transformer处理多视图CT切片（轴向、冠状、矢状）以提取强大的视觉特征，而3D分支采用ResNet-18架构来建模体积上下文，并使用方差风险外推法（VREx）进行预训练，然后进行监督对比学习，以提高跨源鲁棒性。两个分支的预测通过logit级集成推理相结合。在PHAROS-AIF-MIH基准上的实验证明了所提出方法的有效性：对于二进制COVID-19检测，集成实现了94.48%的准确率和0.9426的Macro F1-score，优于两个单独的模型，而对于多类疾病分类，2.5D DINOv 3模型实现了最佳性能，准确率为79.35%，Macro F1-score为0.7497。这些结果突出了将预训练的基于切片的表示与体积建模相结合以进行稳健的多源医学成像分析的益处。代码可在https://github.com/HySonLab/PHAROS-AIF-MIH上获取
摘要 ：We propose a deep learning framework for COVID-19 detection and disease classification from chest CT scans that integrates both 2.5D and 3D representations to capture complementary slice-level and volumetric information. The 2.5D branch processes multi-view CT slices (axial, coronal, sagittal) using a DINOv3 vision transformer to extract robust visual features, while the 3D branch employs a ResNet-18 architecture to model volumetric context and is pretrained with Variance Risk Extrapolation (VREx) followed by supervised contrastive learning to improve cross-source robustness. Predictions from both branches are combined through logit-level ensemble inference. Experiments on the PHAROS-AIF-MIH benchmark demonstrate the effectiveness of the proposed approach: for binary COVID-19 detection, the ensemble achieves 94.48% accuracy and a 0.9426 Macro F1-score, outperforming both individual models, while for multi-class disease classification the 2.5D DINOv3 model achieves the best performance with 79.35% accuracy and a 0.7497 Macro F1-score. These results highlight the benefit of combining pretrained slice-based representations with volumetric modeling for robust multi-source medical imaging analysis. Code is available at https://github.com/HySonLab/PHAROS-AIF-MIH
蒸馏|知识提取(3篇)
【1】Behavior-Centric Extraction of Scenarios from Highway Traffic Data and their Domain-Knowledge-Guided Clustering using CVQ-VAE
标题：以行为为中心的高速公路交通数据场景提取及其使用CVQ-VAE的领域知识引导集群
链接：https://arxiv.org/abs/2603.16964
备注：Accepted as a conference paper in IEEE Intelligent Vehicles Symposium (IV) 2026, Detroit, MI, United States
摘要：ADS的批准取决于在代表性的现实世界交通场景中评估其行为。获取此类场景的常见方法是从真实世界的数据记录中提取它们。然后可以将其分组，并作为随后测试ADS的基础。这带来了两个核心挑战：如何提取场景以及如何对场景进行分组。现有的提取方法依赖于不同的定义，阻碍了情景的可比性。对于场景的分组，可以利用基于规则或基于ML的方法。然而，虽然现代基于ML的方法可以处理复杂的交通场景，不像基于规则的方法，他们缺乏可解释性，可能不符合领域知识。这项工作有助于一个标准化的场景提取的基础上的Scenario-as-Specification概念，以及领域知识指导的场景聚类过程。在高维数据集上的实验表明，场景可以可靠地提取，领域知识可以有效地集成到聚类过程中。因此，所提出的方法支持一个更标准化的过程，用于从高速公路数据记录中导出场景类别，从而实现更有效的自动驾驶车辆验证过程。
摘要：Approval of ADS depends on evaluating its behavior within representative real-world traffic scenarios. A common way to obtain such scenarios is to extract them from real-world data recordings. These can then be grouped and serve as basis on which the ADS is subsequently tested. This poses two central challenges: how scenarios are extracted and how they are grouped. Existing extraction methods rely on heterogeneous definitions, hindering scenario comparability. For the grouping of scenarios, rule-based or ML-based methods can be utilized. However, while modern ML-based approaches can handle the complexity of traffic scenarios, unlike rule-based approaches, they lack interpretability and may not align with domain-knowledge. This work contributes to a standardized scenario extraction based on the Scenario-as-Specification concept, as well as a domain-knowledge-guided scenario clustering process. Experiments on the highD dataset demonstrate that scenarios can be extracted reliably and that domain-knowledge can be effectively integrated into the clustering process. As a result, the proposed methodology supports a more standardized process for deriving scenario categories from highway data recordings and thus enables a more efficient validation process of automated vehicles.
【2】HoloByte: Continuous Hyperspherical Distillation for Tokenizer-Free Modeling
标题：HoloByte：用于无Tokenizer建模的连续超球蒸馏
链接：https://arxiv.org/abs/2603.16917
摘要：序列建模普遍依赖于离散子词标记化来规避原生字节级注意力的$\mathcal{O}（N^2）$计算困难性。然而，这种启发式量化强加了人工形态边界，加强了词汇依赖性，并破坏了优化景观的连续性。为了解决这个二分法，我们引入了\textbf{HoloByte}：一个利用连续超球面蒸馏的严格无标记框架。HoloByte将离散的字节序列划分为固定容量的块，并通过可逆的、保维的正交旋转算子将它们投影到一个连续的、严格有界的超球面流形上。这种空间叠加允许宏观的Transformer只对压缩的连续表示进行操作，形式上将精确的注意时间复杂度从$\mathcal{O}（N^2D）$降低到$\mathcal{O}\left（\frac{N^2}{W^2}D + ND^2 \right）$。一个本地化的因果微解码器随后解开这些表示计算精确的字节级分布。为了管理这个连续的轨迹，我们提出了一个双目标制定纳入一个数学上精确的全息潜在均方误差，严格限制梯度，并保证渐近稳定。在理论上，我们导出了最小嵌入维数$D = Ω（W \ln|\mathcal|）$，以确保从连续流形的无误差离散恢复。从经验上讲，在严格匹配的参数约束下，HoloByte系统地优于可比的离散字节对编码（BPE）基线。这些结果建立了连续超球面蒸馏作为一个数学上严格和计算上易于处理的基础词汇不变的序列建模。该代码可在https://github.com/VladimerKhasia/HoloByte上获得
摘要：Sequence modeling universally relies on discrete subword tokenization to circumvent the $\mathcal{O}(N^2)$ computational intractability of native byte-level attention. However, this heuristic quantization imposes artificial morphological boundaries, enforces vocabulary dependence, and fractures the continuity of the optimization landscape. To resolve this dichotomy, we introduce \textbf{HoloByte}: a strictly tokenizer-free framework utilizing Continuous Hyperspherical Distillation. HoloByte partitions discrete byte sequences into fixed-capacity chunks and projects them into a continuous, strictly bounded hyperspherical manifold via an invertible, dimension-preserving orthogonal rotation operator. This spatial superposition allows a macroscopic transformer to operate exclusively on compressed continuous representations, formally reducing the exact attention time complexity from $\mathcal{O}(N^2D)$ to $\mathcal{O}\left( \frac{N^2}{W^2}D + ND^2 \right)$. A localized causal micro-decoder subsequently unbinds these representations to compute exact byte-level distributions. To govern this continuous trajectory, we propose a dual-objective formulation incorporating a mathematically precise Holographic Latent Mean Squared Error, which strictly bounds the gradient and guarantees asymptotic stability. Theoretically, we derive the minimal embedding dimension $D = Ω(W \ln |\mathcal{V}|)$ required to ensure error-free discrete recovery from the continuous manifold. Empirically, under strictly matched parameter constraints, HoloByte is systematically outperforming a comparable discrete Byte-Pair Encoding (BPE) baseline. These results establish Continuous Hyperspherical Distillation as a mathematically rigorous and computationally tractable foundation for vocabulary-invariant sequence modeling. The code is available at https://github.com/VladimerKhasia/HoloByte
【3】Gaussian Process Regression-based Knowledge Distillation Framework for Simultaneous Prediction of Physical and Mechanical Properties of Epoxy Polymers
标题：基于高斯过程回归的知识蒸馏框架同时预测环氧聚合物物理机械性能
链接：https://arxiv.org/abs/2603.16925
摘要：环氧树脂聚合物由于其多功能特性而被广泛使用，但由于其复杂的3D分子结构，多组分性质以及缺乏策划数据集，机器学习（ML）应用仍然有限。现有的ML研究在很大程度上限于模拟数据，特定属性或狭窄的成分范围。为了解决这些局限性，我们开发了一个知情的高斯过程回归为基础的知识蒸馏（GPR-KD）框架，用于预测热固性环氧聚合物的多个物理（玻璃化转变温度，密度）和机械性能（弹性模量，拉伸强度，压缩强度，弯曲强度，断裂能，粘合强度）。该模型在涵盖不同单体类别（9种树脂，40种硬化剂）的实验文献数据上进行训练。单个GPR模型作为教师模型捕获非线性特征-属性关系，而统一的神经网络学生模型同时学习所有属性的知识。通过将目标属性编码为输入特征，学生模型利用了跨属性相关性。使用RDKit从SMILES表示中提取的分子级描述符创建了一个物理信息模型。该框架将GPR的可解释性和鲁棒性与深度学习的可扩展性和泛化性相结合。比较分析表明，优于传统ML模型的预测精度。同时多属性预测通过跨相关属性的信息共享进一步提高准确性。拟议的框架能够加速设计具有定制性能的新型环氧聚合物。
摘要 ：Epoxy polymers are widely used due to their multifunctional properties, but machine learning (ML) applications remain limited owing to their complex 3D molecular structure, multi-component nature, and lack of curated datasets. Existing ML studies are largely restricted to simulation data, specific properties, or narrow constituent ranges. To address these limitations, we developed an informed Gaussian Process Regression-based Knowledge Distillation (GPR-KD) framework for predicting multiple physical (glass transition temperature, density) and mechanical properties (elastic modulus, tensile strength, compressive strength, flexural strength, fracture energy, adhesive strength) of thermoset epoxy polymers. The model was trained on experimental literature data covering diverse monomer classes (9 resins, 40 hardeners). Individual GPR models serve as teacher models capturing nonlinear feature-property relationships, while a unified neural network student model learns distilled knowledge across all properties simultaneously. By encoding the target property as an input feature, the student model leverages cross-property correlations. Molecular-level descriptors extracted from SMILES representations using RDKit create a physics-informed model. The framework combines GPR interpretability and robustness with deep learning scalability and generalization. Comparative analysis demonstrates superior prediction accuracy over conventional ML models. Simultaneous multi-property prediction further improves accuracy through information sharing across correlated properties. The proposed framework enables accelerated design of novel epoxy polymers with tailored properties.
联邦学习|隐私保护|加密(2篇)
【1】QuantFL: Sustainable Federated Learning for Edge IoT via Pre-Trained Model Quantisation
标题：QuantFL：通过预训练模型量化实现边缘物联网的可持续联合学习
链接：https://arxiv.org/abs/2603.17507
摘要：联合学习（FL）可以在物联网（IoT）设备上实现隐私保护智能，但由于频繁上行传输的高能源成本，会产生显著的碳足迹。虽然预先训练的模型越来越多地用于边缘设备，但它们减少微调的能量开销的潜力仍然没有得到充分的探索。在这项工作中，我们提出了QuantFL，这是一个可持续的FL框架，它利用预先训练的初始化来实现积极的，计算轻量级的量化。我们证明了预训练自然地集中了更新统计数据，使我们能够使用内存高效的桶量化，而无需复杂的错误反馈机制的能量密集型开销。在MNIST和CIFAR-100上，QuantFL将总通信量减少了40%（全精度下行链路的总比特减少$\simeq40\%$;上行链路上或下行链路量化时的总比特减少$\geq80\%$），同时在严格的带宽预算下匹配或超过未压缩基线; BU的测试精度达到89.00%（MNIST）和66.89%（CIFAR-100），且位数减少了几个数量级。我们还考虑了上行链路和下行链路的成本，并提供了量化级别和初始化的消融。QuantFL提供了一个实用的“绿色”配方，用于在电池受限的物联网网络上进行可扩展的训练。
摘要：Federated Learning (FL) enables privacy-preserving intelligence on Internet of Things (IoT) devices but incurs a significant carbon footprint due to the high energy cost of frequent uplink transmission. While pre-trained models are increasingly available on edge devices, their potential to reduce the energy overhead of fine-tuning remains underexplored. In this work, we propose QuantFL, a sustainable FL framework that leverages pre-trained initialisation to enable aggressive, computationally lightweight quantisation. We demonstrate that pre-training naturally concentrates update statistics, allowing us to use memory-efficient bucket quantisation without the energy-intensive overhead of complex error-feedback mechanisms. On MNIST and CIFAR-100, QuantFL reduces total communication by 40\% ($\simeq40\%$ total-bit reduction with full-precision downlink; $\geq80\%$ on uplink or when downlink is quantised) while matching or exceeding uncompressed baselines under strict bandwidth budgets; BU attains 89.00\% (MNIST) and 66.89\% (CIFAR-100) test accuracy with orders of magnitude fewer bits. We also account for uplink and downlink costs and provide ablations on quantisation levels and initialisation. QuantFL delivers a practical, "green" recipe for scalable training on battery-constrained IoT networks.
【2】Federated Multi Agent Deep Learning and Neural Networks for Advanced Distributed Sensing in Wireless Networks
标题：用于无线网络中高级分布式传感的联邦多智能体深度学习和神经网络
链接：https://arxiv.org/abs/2603.16881
摘要：多智能体深度学习（MADL），包括多智能体深度强化学习（MADRL）、分布式/联邦训练和图结构神经网络，正在成为传感、通信和计算紧密耦合的无线系统中决策和推理的统一框架。最近的5G-Advanced和6 G愿景通过集成传感和通信、边缘智能、开放式可编程RAN和非地面/无人机网络加强了这种耦合，这产生了分散的、部分观察的、时变的和资源受限的控制问题。本调查综合了最新技术水平，重点是2021-2025年的研究，关于分布式传感和无线通信的MADL。我们提出了一个任务驱动的分类跨越（i）学习公式（马尔可夫游戏，Dec-POMDP，CTDE），（ii）神经架构（基于GNN的无线电资源管理、基于注意力的策略、分层学习和空中聚合），（iii）先进技术（联合强化学习，通信高效的联合深度RL和无服务器边缘学习编排），和（iv）应用领域（MEC切片卸载、具有功率域NOMA的支持无人机的异构网络、传感器网络中的入侵检测以及ISAC驱动的感知移动网络）。我们还提供了算法、训练拓扑和系统级在延迟、频谱效率、能量、隐私和鲁棒性方面的权衡的比较表。最后，我们确定了一些开放性问题，包括可扩展性、非平稳性、防中毒和后门的安全性、通信开销和实时安全性，并概述了6 G原生感知-通信-计算-学习系统的研究方向。
摘要：Multi-agent deep learning (MADL), including multi-agent deep reinforcement learning (MADRL), distributed/federated training, and graph-structured neural networks, is becoming a unifying framework for decision-making and inference in wireless systems where sensing, communication, and computing are tightly coupled. Recent 5G-Advanced and 6G visions strengthen this coupling through integrated sensing and communication, edge intelligence, open programmable RAN, and non-terrestrial/UAV networking, which create decentralized, partially observed, time-varying, and resource-constrained control problems. This survey synthesizes the state of the art, with emphasis on 2021-2025 research, on MADL for distributed sensing and wireless communications. We present a task-driven taxonomy across (i) learning formulations (Markov games, Dec-POMDPs, CTDE), (ii) neural architectures (GNN-based radio resource management, attention-based policies, hierarchical learning, and over-the-air aggregation), (iii) advanced techniques (federated reinforcement learning, communication-efficient federated deep RL, and serverless edge learning orchestration), and (iv) application domains (MEC offloading with slicing, UAV-enabled heterogeneous networks with power-domain NOMA, intrusion detection in sensor networks, and ISAC-driven perceptive mobile networks). We also provide comparative tables of algorithms, training topologies, and system-level trade-offs in latency, spectral efficiency, energy, privacy, and robustness. Finally, we identify open issues including scalability, non-stationarity, security against poisoning and backdoors, communication overhead, and real-time safety, and outline research directions toward 6G-native sense-communicate-compute-learn systems.
推理|分析|理解|解释(13篇)
【1】ResNet-50 with Class Reweighting and Anatomy-Guided Temporal Decoding for Gastrointestinal Video Analysis
标题：ResNet-50具有类别重新加权和解剖引导的时态解码，用于胃肠道视频分析
链接：https://arxiv.org/abs/2603.17784
备注：ICPR 2026 RARE-VISION Competition
摘要：我们开发了一种基于ResNet-50帧分类器的多标签胃肠道视频分析管道，然后进行解剖引导的时间事件解码。该系统从调整为336 x336的帧中预测17个标签，包括5个解剖学类别和12个病理学类别。一个主要的挑战是严重的类别不平衡，特别是对于罕见的病理标签。为了解决这个问题，我们在训练损失中使用了裁剪类正权重，这在保持稳定优化的同时改善了稀有类学习。在时间阶段，我们发现，直接的帧到事件转换产生碎片与官方地面真相不匹配。因此，最终提交的文件将GT风格的逐帧事件合成、解剖投票平滑和基于解剖的病理门控与保守的滞后解码器相结合。该设计将挑战测试集上的最终时间mAP从0.3801提高到0.4303。
摘要 ：We developed a multi-label gastrointestinal video analysis pipeline based on a ResNet-50 frame classifier followed by anatomy-guided temporal event decoding. The system predicts 17 labels, including 5 anatomy classes and 12 pathology classes, from frames resized to 336x336. A major challenge was severe class imbalance, particularly for rare pathology labels. To address this, we used clipped class-wise positive weighting in the training loss, which improved rare-class learning while maintaining stable optimization. At the temporal stage, we found that direct frame-to-event conversion produced fragmented mismatches with the official ground truth. The final submission therefore combined GT-style framewise event composition, anatomy vote smoothing, and anatomy-based pathology gating with a conservative hysteresis decoder. This design improved the final temporal mAP from 0.3801 to 0.4303 on the challenge test set.
【2】CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution
标题：CoVerRL：通过生成器-验证器协同进化打破无标签推理中的共识陷阱
链接：https://arxiv.org/abs/2603.17775
备注：Project Page: https://zju-real.github.io/CoVerRL Code: https://github.com/ZJU-REAL/CoVerRL
摘要：无标签强化学习使大型语言模型能够在没有地面实况监督的情况下提高推理能力，通常是将多数投票的答案视为伪标签。然而，我们确定了一个关键的失败模式：随着训练最大化自我一致性，输出多样性崩溃，导致模型自信地加强逃避检测的系统错误。我们称之为共识陷阱。为了避免它，我们提出了CoVerRL，一个框架，其中一个单一的模型之间交替生成器和验证器的角色，每个能力引导其他。多数表决提供噪声，但信息的监督训练验证，而改进的验证逐步过滤自相一致的错误从伪标签。这种共同进化创造了一个良性循环，在整个训练过程中保持高的奖励准确性。Qwen和Llama模型家族的实验表明，CoVerRL在数学推理基准测试中的表现优于无标签基线4.7- 5.9%。此外，自我验证的准确性从55%左右提高到85%以上，证实了这两种能力真正共同发展。
摘要：Label-free reinforcement learning enables large language models to improve reasoning capabilities without ground-truth supervision, typically by treating majority-voted answers as pseudo-labels. However, we identify a critical failure mode: as training maximizes self-consistency, output diversity collapses, causing the model to confidently reinforce systematic errors that evade detection. We term this the consensus trap. To escape it, we propose CoVerRL, a framework where a single model alternates between generator and verifier roles, with each capability bootstrapping the other. Majority voting provides noisy but informative supervision for training the verifier, while the improving verifier progressively filters self-consistent errors from pseudo-labels. This co-evolution creates a virtuous cycle that maintains high reward accuracy throughout training. Experiments across Qwen and Llama model families demonstrate that CoVerRL outperforms label-free baselines by 4.7-5.9\% on mathematical reasoning benchmarks. Moreover, self-verification accuracy improves from around 55\% to over 85\%, confirming that both capabilities genuinely co-evolve.
【3】Conditional Inverse Learning of Time-Varying Reproduction Numbers Inference
标题：时变复制数推理的条件逆学习
链接：https://arxiv.org/abs/2603.17549
备注：10 pages, 5 figures. Related to epidemic modeling, neural networks and time-varying reproduction number
摘要：从传染病发病率数据中估计时变再生数是传染病监测的核心任务，但它构成了一个固有的不适定逆问题。现有的方法往往依赖于强有力的结构性假设来自流行病学模型，这可能会限制他们的能力，以适应非平稳的传播动态引起的干预措施或行为变化，导致延迟检测政权转变和估计精度下降。在这项工作中，我们提出了一个条件逆再现学习框架（CIRL），通过学习从历史发生模式和显式时间信息到潜在再现数的{条件映射}来解决逆问题。CIRL没有强加强有力的参数约束，而是将流行病学结构与灵活的基于可能性的统计建模相结合，使用更新方程作为前向算子来执行动态一致性。由此产生的框架相结合的流行病学接地的限制与数据驱动的时间表示，产生繁殖数量的估计是强大的观测噪声，同时保持响应突然的传输变化和零膨胀的发病率观察。在具有受控状态变化的合成流行病和真实世界的SARS和COVID-19数据上的实验证明了所提出的方法的有效性。
摘要：Estimating time-varying reproduction numbers from epidemic incidence data is a central task in infectious disease surveillance, yet it poses an inherently ill-posed inverse problem. Existing approaches often rely on strong structural assumptions derived from epidemiological models, which can limit their ability to adapt to non-stationary transmission dynamics induced by interventions or behavioral changes, leading to delayed detection of regime shifts and degraded estimation accuracy. In this work, we propose a Conditional Inverse Reproduction Learning framework (CIRL) that addresses the inverse problem by learning a {conditional mapping} from historical incidence patterns and explicit time information to latent reproduction numbers. Rather than imposing strongly enforced parametric constraints, CIRL softly integrates epidemiological structure with flexible likelihood-based statistical modeling, using the renewal equation as a forward operator to enforce dynamical consistency. The resulting framework combines epidemiologically grounded constraints with data-driven temporal representations, producing reproduction number estimates that are robust to observation noise while remaining responsive to abrupt transmission changes and zero-inflated incidence observations. Experiments on synthetic epidemics with controlled regime changes and real-world SARS and COVID-19 data demonstrate the effectiveness of the proposed approach.
【4】CA-Based Interpretable Knowledge Representation and Analysis of Geometric Design Parameters
标题：基于CA的几何设计参数可解释知识表示与分析
链接：https://arxiv.org/abs/2603.17535
备注：20 pages, 6 figures, 1 table, preprint to IntelliSys-Artificial Intelligence Conference 2026
摘要：在许多基于CAD的应用中，复杂的几何形状由大量的设计参数定义。这导致了高维设计空间，这对下游工程过程（如仿真，优化和设计探索任务）具有挑战性。因此，使用诸如主成分分析（PCA）的降维方法。主成分分析确定的几何变化的主导模式，并产生一个紧凑的几何表示。虽然经典PCA在紧凑表示部分中表现出色，但它并不直接恢复所生成的几何形状的底层设计参数。在这项工作中，我们处理的问题，估计设计参数PCA为基础的表示。分析最近修改的PCA专用于我们的应用领域，我们表明，结果实际上是相同的标准PCA。我们调查这种方法的局限性，并提出合理的条件下，可以获得准确的，可解释的参数估计。在专门实验的帮助下，我们更深入地研究了PCA的每个阶段以及这些过程中几何形状可能发生的变化。
摘要：In many CAD-based applications, complex geometries are defined by a high number of design parameters. This leads to high-dimensional design spaces that are challenging for downstream engineering processes like simulations, optimization, and design exploration tasks. Therefore, dimension reduction methods such as principal component analysis (PCA) are used. The PCA identifies dominant modes of geometric variation and yields a compact representation of the geometry. While classical PCA excels in the compact representation part, it does not directly recover underlying design parameters of a generated geometry. In this work, we deal with the problem of estimating design parameters from PCA-based representations. Analyzing a recent modification of the PCA dedicated to our field of application, we show that the results are actually identical to the standard PCA. We investigate limitations of this approach and present reasonable conditions under which accurate, interpretable parameter estimation can be obtained. With the help of dedicated experiments, we take a more in-depth look at every stage of the PCA and the possible changes of the geometry during these processes.
【5】Informative Semi-Factuals for XAI: The Elaborated Explanations that People Prefer
标题：XAI的信息丰富的半事实：人们更喜欢的详细解释
链接：https://arxiv.org/abs/2603.17534
摘要：最近，在可解释的人工智能（XAI）中，$\textit{even if}$解释-所谓的半事实-已经成为一种流行的策略，解释了即使某些输入特征被改变，预测的结果$\textit{can remain the same}$。例如，在常用的银行应用程序场景中，一个半事实的解释可以告诉客户更好的选择，他们成功申请的其他替代方案，说“即使你要求双倍的贷款金额，你仍然会被接受”。大多数半事实XAI算法专注于找到对单个关键特征的最大值变化，这些关键特征确实会改变结果（不像反事实解释，通常会找到对几个改变结果的特征的最小值变化）。然而，目前还没有半事实的方法来解释为什么这些极端的价值变化不会改变结果;例如，一个更具信息性的半事实可以告诉客户，这是他们良好的信用评分，使他们能够借到两倍的贷款。在这项工作中，我们提出了一种新的算法-$\textit{informative semi-factuals}$（ISF）的方法-产生更详细的解释补充半事实的信息，额外的$\textit{隐藏功能}$，影响自动决策。基准数据集上的实验结果表明，该ISF方法计算的半事实，是既有信息和高质量的关键指标。此外，一项用户研究表明，人们更喜欢这些详细的解释，而不是简单的半事实的解释所产生的当前方法。
摘要：Recently, in eXplainable AI (XAI), $\textit{even if}$ explanations -- so-called semi-factuals -- have emerged as a popular strategy that explains how a predicted outcome $\textit{can remain the same}$ even when certain input-features are altered. For example, in the commonly-used banking app scenario, a semi-factual explanation could inform customers about better options, other alternatives for their successful application, by saying "$\textit{Even if}$ you asked for double the loan amount, you would still be accepted". Most semi-factuals XAI algorithms focus on finding maximal value-changes to a single key-feature that do $\textit{not}$ alter the outcome (unlike counterfactual explanations that often find minimal value-changes to several features that alter the outcome). However, no current semi-factual method explains $\textit{why}$ these extreme value-changes do not alter outcomes; for example, a more informative semi-factual could tell the customer that it is their good credit score that allows them to borrow double their requested loan. In this work, we advance a new algorithm -- the $\textit{informative semi-factuals}$ (ISF) method -- that generates more elaborated explanations supplementing semi-factuals with information about additional $\textit{hidden features}$ that influence an automated decision. Experimental results on benchmark datasets show that this ISF method computes semi-factuals that are both informative and of high-quality on key metrics. Furthermore, a user study shows that people prefer these elaborated explanations over the simpler semi-factual explanations generated by current methods.
【6】Variational Rectification Inference for Learning with Noisy Labels
标题：带噪音标签学习的变分纠正推理
链接：https://arxiv.org/abs/2603.17255
摘要：标签噪声在现实世界的数据集中被广泛观察到。为了减轻过拟合对深度模型标签噪声的负面影响，有效的策略（例如，重新加权或损失校正）已经广泛应用于流行的方法中，这些方法通常是在元学习场景下学习的。尽管概率元学习模型实现了对噪声的鲁棒性，但它们通常会遭受模型崩溃，从而降低泛化性能。在本文中，我们提出变分校正推理（VRI）制定的损失函数的自适应校正作为一个摊销变分推理问题，并推导出证据下界下的元学习框架。具体地说，VRI被构造为一个分层贝叶斯处理的校正向量作为一个潜在的变量，它可以纠正的损失，噪声样本与额外的随机性正则化，因此，更强大的标签噪声。为了实现校正向量的推断，我们用一个摊销元网络来近似其条件后验。通过引入变分项，使条件后验估计更准确，避免了条件后验函数的Dirac δ函数化，从而显著提高了泛化性能。精心设计的元网络和先验网络坚持平滑假设，从而能够生成可靠的整流矢量。给定一组干净的元数据，VRI可以在双层优化编程中有效地元学习。此外，理论分析保证了元网络可以有效地学习与我们的算法。综合比较实验和分析验证了其有效性的鲁棒学习与噪声标签，特别是在存在开集噪声。
摘要：Label noise has been broadly observed in real-world datasets. To mitigate the negative impact of overfitting to label noise for deep models, effective strategies (\textit{e.g.}, re-weighting, or loss rectification) have been broadly applied in prevailing approaches, which have been generally learned under the meta-learning scenario. Despite the robustness of noise achieved by the probabilistic meta-learning models, they usually suffer from model collapse that degenerates generalization performance. In this paper, we propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions as an amortized variational inference problem and derive the evidence lower bound under the meta-learning framework. Specifically, VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable, which can rectify the loss of the noisy sample with the extra randomness regularization and is, therefore, more robust to label noise. To achieve the inference of the rectifying vector, we approximate its conditional posterior with an amortization meta-network. By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function, which can significantly improve the generalization performance. The elaborated meta-network and prior network adhere to the smoothness assumption, enabling the generation of reliable rectification vectors. Given a set of clean meta-data, VRI can be efficiently meta-learned within the bi-level optimization programming. Besides, theoretical analysis guarantees that the meta-network can be efficiently learned with our algorithm. Comprehensive comparison experiments and analyses validate its effectiveness for robust learning with noisy labels, particularly in the presence of open-set noise.
【7】From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs
标题：从下降到恢复：MLLM中分割的机理分析
链接：https://arxiv.org/abs/2603.17228
摘要：多模态大型语言模型（MLLM）越来越多地应用于像素级视觉任务，但其内在的空间理解能力仍然知之甚少。我们调查分割能力，通过分层线性探测评估整个MLLM管道：视觉编码器，适配器和LLM。我们进一步进行了基于干预的注意力剔除分析，以测试跨标记注意力是否逐步细化视觉表征，并评估图像标记之间的双向注意力的空间一致性。我们的分析表明，适配器引入了分割表示下降，但LLM层通过注意力介导的细化逐步恢复，其中正确分类的令牌将错误分类的邻居引导到正确的标签。在早期的图像标记的位置，这种恢复是有界的因果注意，图像标记之间的双向注意力。这些研究结果提供了一个机制的MLLM如何处理视觉信息分割，通知未来的分割模型的设计。
摘要：Multimodal Large Language Models (MLLMs) are increasingly applied to pixel-level vision tasks, yet their intrinsic capacity for spatial understanding remains poorly understood. We investigate segmentation capacity through a layerwise linear probing evaluation across the entire MLLM pipeline: vision encoder, adapter, and LLM. We further conduct an intervention based attention knockout analysis to test whether cross-token attention progressively refines visual representations, and an evaluation of bidirectional attention among image tokens on spatial consistency. Our analysis reveals that the adapter introduces a segmentation representation drop-off, but LLM layers progressively recover through attention-mediated refinement, where correctly classified tokens steer misclassified neighbors toward the correct label. At early image token positions, this recovery is bounded by causal attention, which bidirectional attention among image tokens alleviates. These findings provide a mechanistic account of how MLLMs process visual information for segmentation, informing the design of future segmentation-capable models.
【8】Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing
标题：捕捉行为中的合理化：通过激活探测检测CoT前后的动机推理
链接：https://arxiv.org/abs/2603.17199
摘要：大型语言模型（LLM）可能会产生无法准确反映驱动其答案的实际因素的思维链（CoT）。在多项选择设置中，注入了有利于特定选项的提示，模型可能会将其最终答案转向暗示的选项，并产生一个CoT，该CoT在不承认提示的情况下合理化反应-这是动机推理的一个实例。我们在多个LLM家族和数据集上研究了这种现象，证明即使在无法从CoT中轻松确定的情况下，也可以通过探测内部激活来识别动机性推理。使用在模型的剩余流上训练的监督探测器，我们证明了（i）在生成任何CoT令牌之前应用的生成前探测器预测动机推理以及访问完整CoT跟踪的基于LLM的CoT监视器，以及（ii）在CoT生成之后应用的生成后探测器优于相同的监视器。总之，这些结果表明，动机推理检测更可靠的内部表示比CoT监测。此外，生成前探测可以及早标记有动机的行为，潜在地避免不必要的生成。
摘要 ：Large language models (LLMs) can produce chains of thought (CoT) that do not accurately reflect the actual factors driving their answers. In multiple-choice settings with an injected hint favoring a particular option, models may shift their final answer toward the hinted option and produce a CoT that rationalizes the response without acknowledging the hint - an instance of motivated reasoning. We study this phenomenon across multiple LLM families and datasets demonstrating that motivated reasoning can be identified by probing internal activations even in cases when it cannot be easily determined from CoT. Using supervised probes trained on the model's residual stream, we show that (i) pre-generation probes, applied before any CoT tokens are generated, predict motivated reasoning as well as a LLM-based CoT monitor that accesses the full CoT trace, and (ii) post-generation probes, applied after CoT generation, outperform the same monitor. Together, these results show that motivated reasoning is detected more reliably from internal representations than from CoT monitoring. Moreover, pre-generation probing can flag motivated behavior early, potentially avoiding unnecessary generation.
【9】Domain-informed explainable boosting machines for trustworthy lateral spread predictions
标题：了解领域的可解释助推机器，用于可靠的横向传播预测
链接：https://arxiv.org/abs/2603.17175
备注：33 pages, 16 figures
摘要：可解释的提升机（EBM）通过加性形状函数提供透明的预测，从而能够直接检查特征贡献。然而，EBM可以学习非物理关系，这会降低其在自然灾害应用中的可靠性。这项研究提出了一个域知情的框架，以提高横向扩展预测的EBM的物理一致性。我们的方法修改学习的形状功能的基础上领域知识。这些修改纠正了非物理行为，同时保持了数据驱动的模式。我们将该方法应用于2011年基督城地震数据集，并纠正了原始EBM中观察到的非物理趋势。由此产生的模型产生更多的物理一致的全球和本地的解释，与一个可接受的权衡精度（4--5\%）。
摘要：Explainable Boosting Machines (EBMs) provide transparent predictions through additive shape functions, enabling direct inspection of feature contributions. However, EBMs can learn non-physical relationships that reduce their reliability in natural hazard applications. This study presents a domain-informed framework to improve the physical consistency of EBMs for lateral spreading prediction. Our approach modifies learned shape functions based on domain knowledge. These modifications correct non-physical behavior while maintaining data-driven patterns. We apply the method to the 2011 Christchurch earthquake dataset and correct non-physical trends observed in the original EBM. The resulting model produces more physically consistent global and local explanations, with an acceptable tradeoff in accuracy (4--5\%).
【10】SCE-LITE-HQ: Smooth visual counterfactual explanations with generative foundation models
标题：SCE-LITE-HQ：通过生成式基础模型流畅的视觉反事实解释
链接：https://arxiv.org/abs/2603.17048
摘要：现代神经网络实现了强大的性能，但仍然难以在高维视觉域中解释。反事实解释（CFE）通过识别改变模型输出的最小输入变化，提供了一种解释黑盒预测的原则方法。然而，现有的CFE方法通常依赖于特定于小行星的生成模型，并产生大量的计算成本，限制了它们对高分辨率数据的可扩展性。我们提出了SCE-LITE-HQ，这是一个可扩展的反事实生成框架，它利用了预训练的生成基础模型，而无需特定于任务的再训练。该方法在生成器的潜在空间中运行，采用平滑梯度来提高优化稳定性，并应用基于掩码的多样化来促进现实和结构多样的反事实。我们使用desiderata驱动的评估协议在自然和医学数据集上评估SCE-LITE-HQ。结果表明，SCE-LITE-HQ产生了有效的，现实的，多样化的反事实，与现有的基线竞争或优于现有的基线，同时避免了训练专用生成模型的开销。
摘要：Modern neural networks achieve strong performance but remain difficult to interpret in high-dimensional visual domains. Counterfactual explanations (CFEs) provide a principled approach to interpreting black-box predictions by identifying minimal input changes that alter model outputs. However, existing CFE methods often rely on dataset-specific generative models and incur substantial computational cost, limiting their scalability to high-resolution data. We propose SCE-LITE-HQ, a scalable framework for counterfactual generation that leverages pretrained generative foundation models without task-specific retraining. The method operates in the latent space of the generator, incorporates smoothed gradients to improve optimization stability, and applies mask-based diversification to promote realistic and structurally diverse counterfactuals. We evaluate SCE-LITE-HQ on natural and medical datasets using a desiderata-driven evaluation protocol. Results show that SCE-LITE-HQ produces valid, realistic, and diverse counterfactuals competitive with or outperforming existing baselines, while avoiding the overhead of training dedicated generative models.
【11】Integrating Explainable Machine Learning and Mixed-Integer Optimization for Personalized Sleep Quality Intervention
标题：集成可解释机器学习和混合时间表优化以实现个性化睡眠质量干预
链接：https://arxiv.org/abs/2603.16937
备注：34 Pages. 7 Tables. 6 Figures
摘要：睡眠质量受到行为、环境和心理社会因素复杂相互作用的影响，但大多数计算研究主要集中在预测风险识别，而不是可操作的干预设计。尽管机器学习模型可以准确预测主观睡眠结果，但它们很少将预测性见解转化为实际的干预策略。为了解决这一差距，我们提出了一个个性化的预测-规范框架，将可解释的机器学习与混合整数优化相结合。根据调查数据训练的监督分类器预测睡眠质量，而基于SHAP的特征归因量化了可修改因素的影响。这些重要性的措施被纳入一个混合整数优化模型，确定最小的和可行的行为调整，同时通过惩罚机制的变化建模阻力。该框架实现了强大的预测性能，测试F1得分为0.9544，准确率为0.9366。敏感性和帕累托分析显示，预期改善和干预强度之间存在明显的权衡，随着更多变化的引入，收益递减。在个人层面上，该模型会产生简明的建议，通常会建议一个或两个高影响力的行为调整，有时在预期收益最小的情况下建议不做任何改变。通过整合预测、解释和约束优化，该框架展示了数据驱动的见解如何转化为结构化和个性化的决策支持，以改善睡眠。
摘要：Sleep quality is influenced by a complex interplay of behavioral, environmental, and psychosocial factors, yet most computational studies focus mainly on predictive risk identification rather than actionable intervention design. Although machine learning models can accurately predict subjective sleep outcomes, they rarely translate predictive insights into practical intervention strategies. To address this gap, we propose a personalized predictive-prescriptive framework that integrates interpretable machine learning with mixed-integer optimization. A supervised classifier trained on survey data predicts sleep quality, while SHAP-based feature attribution quantifies the influence of modifiable factors. These importance measures are incorporated into a mixed-integer optimization model that identifies minimal and feasible behavioral adjustments, while modelling resistance to change through a penalty mechanism. The framework achieves strong predictive performance, with a test F1-score of 0.9544 and an accuracy of 0.9366. Sensitivity and Pareto analyses reveal a clear trade-off between expected improvement and intervention intensity, with diminishing returns as additional changes are introduced. At the individual level, the model generates concise recommendations, often suggesting one or two high-impact behavioral adjustments and sometimes recommending no change when expected gains are minimal. By integrating prediction, explanation, and constrained optimization, this framework demonstrates how data-driven insights can be translated into structured and personalized decision support for sleep improvement.
【12】Structured SIR: Efficient and Expressive Importance-Weighted Inference for High-Dimensional Image Registration
标题：结构化Sir：用于多维图像配准的高效且富有表达力的重要性加权推断
链接：https://arxiv.org/abs/2603.17415
摘要：图像配准是一个病态的密集视觉任务，其中多个解决方案实现相似的损失值，激发概率推理。变分推理以前曾被用来捕捉这些分布，然而，对后验形式的限制性假设可能会导致不良的表征，过度自信和低质量的样本。更灵活的后验通常由密集3D图像配准所需的高维协方差矩阵的复杂性来检验。 在这项工作中，我们提出了一个内存和计算效率高的推理方法，结构化SIR，使表达，多模态，高品质的样本的不确定性的表征。我们建议使用一个新的内存高效的高维协方差参数化的低秩协方差和稀疏的，空间结构的Cholesky精度因子的总和的采样重要性恢复（SIR）算法。这种结构使得能够捕获复杂的空间相关性，同时保持计算上易处理。 我们评估这种方法在脑MRI数据的3D密集图像配准中的有效性，这是一个非常高维的问题。我们证明，我们提出的方法产生的不确定性估计，显着更好地校准比变分方法产生的，达到同等或更好的精度。最重要的是，我们表明，该模型产生高度结构化的多模态后验分布，使有效和高效的不确定性量化。
摘要：Image registration is an ill-posed dense vision task, where multiple solutions achieve similar loss values, motivating probabilistic inference. Variational inference has previously been employed to capture these distributions, however restrictive assumptions about the posterior form can lead to poor characterisation, overconfidence and low-quality samples. More flexible posteriors are typically bottlenecked by the complexity of high-dimensional covariance matrices required for dense 3D image registration. In this work, we present a memory and computationally efficient inference method, Structured SIR, that enables expressive, multi-modal, characterisation of uncertainty with high quality samples. We propose the use of a Sampled Importance Resampling (SIR) algorithm with a novel memory-efficient high-dimensional covariance parameterisation as the sum of a low-rank covariance and a sparse, spatially structured Cholesky precision factor. This structure enables capturing complex spatial correlations while remaining computationally tractable. We evaluate the efficacy of this approach in 3D dense image registration of brain MRI data, which is a very high-dimensional problem. We demonstrate that our proposed methods produces uncertainty estimates that are significantly better calibrated than those produced by variational methods, achieving equivalent or better accuracy. Crucially, we show that the model yields highly structured multi-modal posterior distributions, enable effective and efficient uncertainty quantification.
【13】Dependence Fidelity and Downstream Inference Stability in Generative Models
标题：生成模型中的依赖保真度和下游推理稳定性
链接：https://arxiv.org/abs/2603.17041
备注：22 pages, 7 figures. Poster presentation at MathAI 2026 (International Conference on Mathematics of Artificial Intelligence), March 30 - April 3, 2026
摘要：生成式人工智能的最新进展导致合成数据越来越真实，但评估标准仍然集中在边缘分布匹配上。虽然这些诊断评估本地的现实主义，他们提供有限的洞察力生成模型是否保留多元依赖结构，下游推理。我们引入协方差水平的依赖保真度作为一个实用的标准，用于评估是否生成分布保持联合结构超越单变量边缘。我们确立了三个核心成果。首先，分布可以精确地匹配所有单变量边际，同时表现出实质上不同的依赖结构，证明仅边际保真度是不够的。第二，依赖发散导致下游推理的数量不稳定性，包括回归系数的符号反转，尽管相同的边际行为。第三，显式控制协方差水平的依赖发散，确保稳定的行为依赖敏感的任务，如主成分分析。合成结构说明了依赖保存失败如何导致不正确的结论，尽管相同的边缘分布。这些结果突出了依赖保真度作为一个有用的诊断评估生成模型在依赖敏感的下游任务，扩散模型和变分自动编码器的影响。这些保证特别适用于由协方差结构控制的程序;需要高阶依赖的任务，如尾事件估计，需要更丰富的标准。
摘要：Recent advances in generative AI have led to increasingly realistic synthetic data, yet evaluation criteria remain focused on marginal distribution matching. While these diagnostics assess local realism, they provide limited insight into whether a generative model preserves the multivariate dependence structures governing downstream inference. We introduce covariance-level dependence fidelity as a practical criterion for evaluating whether a generative distribution preserves joint structure beyond univariate marginals. We establish three core results. First, distributions can match all univariate marginals exactly while exhibiting substantially different dependence structures, demonstrating marginal fidelity alone is insufficient. Second, dependence divergence induces quantitative instability in downstream inference, including sign reversals in regression coefficients despite identical marginal behavior. Third, explicit control of covariance-level dependence divergence ensures stable behavior for dependence-sensitive tasks such as principal component analysis. Synthetic constructions illustrate how dependence preservation failures lead to incorrect conclusions despite identical marginal distributions. These results highlight dependence fidelity as a useful diagnostic for evaluating generative models in dependence-sensitive downstream tasks, with implications for diffusion models and variational autoencoders. These guarantees apply specifically to procedures governed by covariance structure; tasks requiring higher-order dependence such as tail-event estimation require richer criteria.
检测相关(4篇)
【1】RangeAD: Fast On-Model Anomaly Detection
标题：RangeAD：快速模型异常检测
链接：https://arxiv.org/abs/2603.17795
备注：16 pages, 5 figures
摘要：在实践中，机器学习方法通常需要异常检测（AD）来过滤输入或检测分布偏移。通常，这通过在主模型旁边运行单独的AD模型来实现。然而，这种分离忽略了一个事实，即主模型已经编码了关于目标分布的大量信息。在本文中，我们介绍了On-Model AD，这是一种用于异常检测的设置，可以显式地利用对相关机器学习模型的访问。在此设置中，我们提出了RangeAD，一种利用从主模型导出的神经元输出范围的算法。RangeAD即使在高维任务上也能实现卓越的性能，同时大大降低了推理成本。我们的研究结果表明，作为一个实用的框架，有效的异常检测的模型AD设置的潜力。
摘要：In practice, machine learning methods commonly require anomaly detection (AD) to filter inputs or detect distributional shifts. Typically, this is implemented by running a separate AD model alongside the primary model. However, this separation ignores the fact that the primary model already encodes substantial information about the target distribution. In this paper, we introduce On-Model AD, a setting for anomaly detection that explicitly leverages access to a related machine learning model. Within this setting, we propose RangeAD, an algorithm that utilizes neuron-wise output ranges derived from the primary model. RangeAD achieves superior performance even on high-dimensional tasks while incurring substantially lower inference costs. Our results demonstrate the potential of the On-Model AD setting as a practical framework for efficient anomaly detection.
【2】Objective Mispricing Detection for Shortlisting Undervalued Football Players via Market Dynamics and News Signals
标题：通过市场动态和新闻信号客观地检测被低估的足球运动员入围名单
链接：https://arxiv.org/abs/2603.17687
摘要：我们提出了一个实用的，可重复的框架，以确定被低估的足球运动员的基础上客观的错误定价。我们不依赖主观的专家标签，而是根据结构化数据（历史市场动态，传记和合同特征，转让历史）估计预期市场价值，并将其与观察到的估值进行比较，以确定错误定价。然后，我们评估新闻衍生的自然语言处理（NLP）特征（即，来自足球文章的情感统计和语义嵌入）补充了用于筛选被低估的球员的市场信号。 使用时间顺序（泄漏感知）评估，梯度推进回归解释了对数转换市场价值中的大部分方差。对于低估入围名单，基于ROC-AUC的消融表明市场动态是主要信号，而NLP特征提供了一致的次要收益，提高了鲁棒性和可解释性。SHAP分析表明，市场趋势和年龄占主导地位，新闻衍生的波动性线索放大了高不确定性制度中的信号。拟议的管道旨在为球探工作流程提供决策支持，强调硬分类阈值的排名/入围，并包括简洁的可重复性和道德声明。
摘要 ：We present a practical, reproducible framework for identifying undervalued football players grounded in objective mispricing. Instead of relying on subjective expert labels, we estimate an expected market value from structured data (historical market dynamics, biographical and contract features, transfer history) and compare it to the observed valuation to define mispricing. We then assess whether news-derived Natural Language Processing (NLP) features (i.e., sentiment statistics and semantic embeddings from football articles) complement market signals for shortlisting undervalued players. Using a chronological (leakage-aware) evaluation, gradient-boosted regression explains a large share of the variance in log-transformed market value. For undervaluation shortlisting, ROC-AUC-based ablations show that market dynamics are the primary signal, while NLP features provide consistent, secondary gains that improve robustness and interpretability. SHAP analyses suggest the dominance of market trends and age, with news-derived volatility cues amplifying signals in high-uncertainty regimes. The proposed pipeline is designed for decision support in scouting workflows, emphasizing ranking/shortlisting over hard classification thresholds, and includes a concise reproducibility and ethics statement.
【3】FoMo X: Modular Explainability Signals for Outlier Detection Foundation Models
标题：FoMo X：离群值检测基础模型的模块化解释性信号
链接：https://arxiv.org/abs/2603.17570
备注：24 pages, 9 figures
摘要：表格基础模型，特别是先验数据拟合网络（PFN），通过在没有训练的情况下对新数据集进行无监督的zero-shot适应，彻底改变了离群值检测（OD）。然而，尽管这些模型具有预测能力，但它们通常充当不透明的黑匣子，输出缺乏安全关键决策所需的操作背景的标量离群值。现有的事后解释方法往往是计算上禁止的实时部署或无法捕捉固有的认知不确定性zero-shot推理。在这项工作中，我们介绍了FoMo-X，一个模块化的框架，配备OD基础模型与内在的，轻量级的诊断能力。我们利用的见解是，预先训练的PFN骨干的冻结嵌入已经编码了丰富的、上下文相关的关系信息。FoMo-X将辅助诊断头连接到这些嵌入，使用与主干相同的生成模拟器进行离线训练。这使我们能够提取计算成本高的属性，如基于Monte Carlo dropout的认知不确定性，成为确定性的单次推理。我们实例化FoMo-X有两个新的头：一个严重性头，离散化偏差到可解释的风险层，和一个不确定性头，提供校准的信心措施。对合成和真实世界基准测试（ADBench）的广泛评估表明，FoMo-X以高保真度和可忽略的推理开销恢复地面实况诊断信号。通过弥合基础模型性能和操作可解释性之间的差距，FoMo-X提供了一条可扩展的路径，以实现值得信赖的zero-shot离群值检测。
摘要：Tabular foundation models, specifically Prior-Data Fitted Networks (PFNs), have revolutionized outlier detection (OD) by enabling unsupervised zero-shot adaptation to new datasets without training. However, despite their predictive power, these models typically function as opaque black boxes, outputting scalar outlier scores that lack the operational context required for safety-critical decision-making. Existing post-hoc explanation methods are often computationally prohibitive for real-time deployment or fail to capture the epistemic uncertainty inherent in zero-shot inference. In this work, we introduce FoMo-X, a modular framework that equips OD foundation models with intrinsic, lightweight diagnostic capabilities. We leverage the insight that the frozen embeddings of a pretrained PFN backbone already encode rich, context-conditioned relational information. FoMo-X attaches auxiliary diagnostic heads to these embeddings, trained offline using the same generative simulator prior as the backbone. This allows us to distill computationally expensive properties, such as Monte Carlo dropout based epistemic uncertainty, into a deterministic, single-pass inference. We instantiate FoMo-X with two novel heads: a Severity Head that discretizes deviations into interpretable risk tiers, and an Uncertainty Head that provides calibrated confidence measures. Extensive evaluation on synthetic and real-world benchmarks (ADBench) demonstrates that FoMo-X recovers ground-truth diagnostic signals with high fidelity and negligible inference overhead. By bridging the gap between foundation model performance and operational explainability, FoMo-X offers a scalable path toward trustworthy, zero-shot outlier detection.
【4】Personalized Fall Detection by Balancing Data with Selective Feedback Using Contrastive Learning
标题：使用对比学习平衡数据和选择性反馈来进行个性化跌倒检测
链接：https://arxiv.org/abs/2603.17148
摘要：个性化跌倒检测模型可以通过适应个体运动模式来显著提高准确性，但其有效性通常受到现实世界跌倒数据的稀缺性和非跌倒反馈样本的主导地位的限制。这种不平衡使模型偏向于日常活动，并削弱了其对真实跌倒事件的敏感性。为了应对这一挑战，我们提出了一个个性化框架，结合半监督聚类与对比学习，以确定和平衡最翔实的用户反馈样本。该框架下的三个再培训策略，包括从头开始训练（TFS），迁移学习（TL），和Few-Shot学习（FSL），评估跨学习范式的适应性。10名参与者的实时实验表明，TFS方法实现了最高的性能，比基线提高了25%，而FSL实现了第二高的性能，提高了7%，证明了选择性个性化对现实世界部署的有效性。
摘要：Personalized fall detection models can significantly improve accuracy by adapting to individual motion patterns, yet their effectiveness is often limited by the scarcity of real-world fall data and the dominance of non-fall feedback samples. This imbalance biases the model toward routine activities and weakens its sensitivity to true fall events. To address this challenge, we propose a personalization framework that combines semi-supervised clustering with contrastive learning to identify and balance the most informative user feedback samples. The framework is evaluated under three retraining strategies, including Training from Scratch (TFS), Transfer Learning (TL), and Few-Shot Learning (FSL), to assess adaptability across learning paradigms. Real-time experiments with ten participants show that the TFS approach achieves the highest performance, with up to a 25% improvement over the baseline, while FSL achieves the second-highest performance with a 7% improvement, demonstrating the effectiveness of selective personalization for real-world deployment.
分类|识别(2篇)
【1】Classifier Pooling for Modern Ordinal Classification
标题：现代有序分类的分类器池
链接：https://arxiv.org/abs/2603.17278
摘要：有序数据在临床和其他领域中广泛流行，但缺乏现代的基于机器学习的方法和公开可用的软件来解决它。在本文中，我们提出了一种模型不可知的有序分类方法，它可以以有序的方式应用于任何非有序的分类方法。我们还以Python包的形式提供了这些算法的开源实现。我们将这些模型应用于多个真实世界的数据集，以显示它们在各个领域的性能。我们发现，它们往往优于非有序分类方法，特别是当数据点的数量相对较少或有许多类别的结果。这项工作，包括开发的软件，促进了使用现代、更强大的机器学习算法来处理有序数据。
摘要：Ordinal data is widely prevalent in clinical and other domains, yet there is a lack of both modern, machine-learning based methods and publicly available software to address it. In this paper, we present a model-agnostic method of ordinal classification, which can apply any non-ordinal classification method in an ordinal fashion. We also provide an open-source implementation of these algorithms, in the form of a Python package. We apply these models on multiple real-world datasets to show their performance across domains. We show that they often outperform non-ordinal classification methods, especially when the number of datapoints is relatively small or when there are many classes of outcomes. This work, including the developed software, facilitates the use of modern, more powerful machine learning algorithms to handle ordinal data.
【2】Tokenization vs. Augmentation: A Systematic Study of Writer Variance in IMU-Based Online Handwriting Recognition
标题：代币化与增强：基于IMU的在线手写识别中书写者差异的系统研究
链接
：https://arxiv.org/abs/2603.16883
摘要：Inertial measurement unit-based online handwriting recognition enables the recognition of input signals collected across different writing surfaces but remains challenged by uneven character distributions and inter-writer variability. In this work, we systematically investigate two strategies to address these issues: sub-word tokenization and concatenation-based data augmentation. Our experiments on the OnHW-Words500 dataset reveal a clear dichotomy between handling inter-writer and intra-writer variance. On the writer-independent split, structural abstraction via Bigram tokenization significantly improves performance to unseen writing styles, reducing the word error rate (WER) from 15.40% to 12.99%. In contrast, on the writer-dependent split, tokenization degrades performance due to vocabulary distribution shifts between the training and validation sets. Instead, our proposed concatenation-based data augmentation acts as a powerful regularizer, reducing the character error rate by 34.5% and the WER by 25.4%. Further analysis shows that short, low-level tokens benefit model performance and that concatenation-based data augmentation performance gain surpasses those achieved by proportionally extended training. These findings reveal a clear variance-dependent effect: sub-word tokenization primarily mitigates inter-writer stylistic variability, whereas concatenation-based data augmentation effectively compensates for intra-writer distributional sparsity.
表征(2篇)
【1】Causal Representation Learning on High-Dimensional Data: Benchmarks, Reproducibility, and Evaluation Metrics
标题：多维数据上的因果表示学习：基准、再现性和评估指标
链接：https://arxiv.org/abs/2603.17405
摘要：Causal representation learning (CRL) models aim to transform high-dimensional data into a latent space, enabling interventions to generate counterfactual samples or modify existing data based on the causal relationships among latent variables. To facilitate the development and evaluation of these models, a variety of synthetic and real-world datasets have been proposed, each with distinct advantages and limitations. For practical applications, CRL models must perform robustly across multiple evaluation directions, including reconstruction, disentanglement, causal discovery, and counterfactual reasoning, using appropriate metrics for each direction. However, this multi-directional evaluation can complicate model comparison, as a model may excel in some direction while under-performing in others. Another significant challenge in this field is reproducibility: the source code corresponding to published results must be publicly available, and repeated runs should yield performance consistent with the original reports. In this study, we critically analyzed the synthetic and real-world datasets currently employed in the literature, highlighting their limitations and proposing a set of essential characteristics for suitable datasets in CRL model development. We also introduce a single aggregate metric that consolidates performance across all evaluation directions, providing a comprehensive score for each model. Finally, we reviewed existing implementations from the literature and assessed them in terms of reproducibility, identifying gaps and best practices in the field.
【2】Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning
标题：针对内部噪音的变分核设计：高斯混乱噪音、表示兼容性和可靠的深度学习
链接：https://arxiv.org/abs/2603.17365
备注：37 pages
摘要：Internal noise in deep networks is usually inherited from heuristics such as dropout, hard masking, or additive perturbation. We ask two questions: what correlation geometry should internal noise have, and is the implemented perturbation compatible with the representations it acts on? We answer these questions through Variational Kernel Design (VKD), a framework in which a noise mechanism is specified by a law family, a correlation kernel, and an injection operator, and is derived from learning desiderata. In a solved spatial subfamily, a quadratic maximum-entropy principle over latent log-fields yields a Gaussian optimizer with precision given by the Dirichlet Laplacian, so the induced geometry is the Dirichlet Green kernel. Wick normalization then gives a canonical positive mean-one gate, Gaussian Chaos Noise (GCh). For the sample-wise gate used in practice, we prove exact Gaussian control of pairwise log-ratio deformation, margin-sensitive ranking stability, and an exact expected intrinsic roughness budget; hard binary masks instead induce singular or coherence-amplified distortions on positive coherent representations. On ImageNet and ImageNet-C, GCh consistently improves calibration and under shift also improves NLL at competitive accuracy.
3D|3D重建等相关(2篇)
【1】LoST: Level of Semantics Tokenization for 3D Shapes
标题：Lost：3D收件箱的语义代币化水平
链接：https://arxiv.org/abs/2603.17995
备注：CVPR 2026; Project website-- https://lost3d.github.io
摘要 ：Tokenization is a fundamental technique in the generative modeling of various modalities. In particular, it plays a critical role in autoregressive (AR) models, which have recently emerged as a compelling option for 3D generation. However, optimal tokenization of 3D shapes remains an open question. State-of-the-art (SOTA) methods primarily rely on geometric level-of-detail (LoD) hierarchies, originally designed for rendering and compression. These spatial hierarchies are often token-inefficient and lack semantic coherence for AR modeling. We propose Level-of-Semantics Tokenization (LoST), which orders tokens by semantic salience, such that early prefixes decode into complete, plausible shapes that possess principal semantics, while subsequent tokens refine instance-specific geometric and semantic details. To train LoST, we introduce Relational Inter-Distance Alignment (RIDA), a novel 3D semantic alignment loss that aligns the relational structure of the 3D shape latent space with that of the semantic DINO feature space. Experiments show that LoST achieves SOTA reconstruction, surpassing previous LoD-based 3D shape tokenizers by large margins on both geometric and semantic reconstruction metrics. Moreover, LoST achieves efficient, high-quality AR 3D generation and enables downstream tasks like semantic retrieval, while using only 0.1%-10% of the tokens needed by prior AR models.
【2】Large-Scale 3D Ground-Motion Synthesis with Physics-Inspired Latent Operator Flow Matching
标题：利用物理启发的潜在操作员流匹配进行大规模3D地动合成
链接：https://arxiv.org/abs/2603.17403
摘要：Earthquake hazard analysis and design of spatially distributed infrastructure, such as power grids and energy pipeline networks, require scenario-specific ground-motion time histories with realistic frequency content and spatiotemporal coherence. However, producing the large ensembles needed for uncertainty quantification with physics-based simulations is computationally intensive and impractical for engineering workflows. To address this challenge, we introduce Ground-Motion Flow (GMFlow), a physics-inspired latent operator flow matching framework that generates realistic, large-scale regional ground-motion time-histories conditioned on physical parameters. Validated on simulated earthquake scenarios in the San Francisco Bay Area, GMFlow generates spatially coherent ground motion across more than 9 million grid points in seconds, achieving a 10,000-fold speedup over the simulation workflow, which opens a path toward rapid and uncertainty-aware hazard assessment for distributed infrastructure. More broadly, GMFlow advances mesh-agnostic functional generative modeling and could potentially be extended to the synthesis of large-scale spatiotemporal physical fields in diverse scientific domains.
编码器(1篇)
【1】DECODE: Dual-Enhanced Conditioned Diffusion for EEG Forecasting
标题：DEBER：用于脑电预测的双增强条件扩散
链接：https://arxiv.org/abs/2603.16885
摘要：Forecasting Electroncephalography (EEG) signals during cognitive events remains a fundamental challenge in neuroscience and Brain-Computer Interfaces (BCIs), as existing methods struggle to capture both the stochastic nature of neural dynamics and the semantic context of behavioral tasks. We present the Dual-Enhanced COnditioned Diffusion (DECODE) for EEG, a novel framework that unifies semantic guidance from natural language descriptions with temporal dynamics from historical signals to generate event-specific neural responses. DECODE leverages pre-trained language models to condition the diffusion process on rich textual descriptions of cognitive events, while maintaining temporal coherence through history-based Langevin dynamics. Evaluated on a real-world driving task dataset with five distinct behaviors, DECODE achieves sub-microvolt prediction accuracy (MAE = 0.626 microvolt) over 75 timestep horizons while maintaining well-calibrated uncertainty estimates. Our framework demonstrates that natural language can effectively bridge high-level cognitive descriptions and low-level neural dynamics, opening new possibilities for zero-shot generalization to novel behaviors and interpretable BCIs. By generating physiologically plausible, event-specific EEG trajectories conditioned on semantic descriptions, DECODE establishes a new paradigm for understanding and predicting context-dependent neural activity.
优化|敛散性(3篇)
【1】Auto-Unrolled Proximal Gradient Descent: An AutoML Approach to Interpretable Waveform Optimization
标题：自动展开近端梯度下降：可解释波优化的AutoML方法
链接：https://arxiv.org/abs/2603.17478
备注：7 pages
摘要：This study explores the combination of automated machine learning (AutoML) with model-based deep unfolding (DU) for optimizing wireless beamforming and waveforms. We convert the iterative proximal gradient descent (PGD) algorithm into a deep neural network, wherein the parameters of each layer are learned instead of being predetermined. Additionally, we enhance the architecture by incorporating a hybrid layer that performs a learnable linear gradient transformation prior to the proximal projection. By utilizing AutoGluon with a tree-structured parzen estimator (TPE) for hyperparameter optimization (HPO) across an expanded search space, which includes network depth, step-size initialization, optimizer, learning rate scheduler, layer type, and post-gradient activation, the proposed auto-unrolled PGD (Auto-PGD) achieves 98.8% of the spectral efficiency of a traditional 200-iteration PGD solver using only five unrolled layers, while requiring only 100 training samples. We also address a gradient normalization issue to ensure consistent performance during training and evaluation, and we illustrate per-layer sum-rate logging as a tool for transparency. These contributions highlight a notable reduction in the amount of training data and inference cost required, while maintaining high interpretability compared to conventional black-box architectures.
【2】Binary Latent Protein Fitness Landscapes for Quantum Annealing Optimization
标题：量子安妮优化的二元潜在蛋白质适应度景观
链接：https://arxiv.org/abs/2603.17247
摘要 ：We propose Q-BIOLAT, a framework for modeling and optimizing protein fitness landscapes in binary latent spaces. Starting from protein sequences, we leverage pretrained protein language models to obtain continuous embeddings, which are then transformed into compact binary latent representations. In this space, protein fitness is approximated using a quadratic unconstrained binary optimization (QUBO) model, enabling efficient combinatorial search via classical heuristics such as simulated annealing and genetic algorithms. On the ProteinGym benchmark, we demonstrate that Q-BIOLAT captures meaningful structure in protein fitness landscapes and enables the identification of high-fitness variants. Despite using a simple binarization scheme, our method consistently retrieves sequences whose nearest neighbors lie within the top fraction of the training fitness distribution, particularly under the strongest configurations. We further show that different optimization strategies exhibit distinct behaviors, with evolutionary search performing better in higher-dimensional latent spaces and local search remaining competitive in preserving realistic sequences. Beyond its empirical performance, Q-BIOLAT provides a natural bridge between protein representation learning and combinatorial optimization. By formulating protein fitness as a QUBO problem, our framework is directly compatible with emerging quantum annealing hardware, opening new directions for quantum-assisted protein engineering. Our implementation is publicly available at: https://github.com/HySonLab/Q-BIOLAT
【3】Stochastic set-valued optimization and its application to robust learning
标题：随机集值优化及其在鲁棒学习中的应用
链接：https://arxiv.org/abs/2603.17691
摘要：In this paper, we develop a stochastic set-valued optimization (SVO) framework tailored for robust machine learning. In the SVO setting, each decision variable is mapped to a set of objective values, and optimality is defined via set relations. We focus on SVO problems with hyperbox sets, which can be reformulated as multi-objective optimization (MOO) problems with finitely many objectives and serve as a foundation for representing or approximating more general mapped sets. Two special cases of hyperbox-valued optimization (HVO) are interval-valued (IVO) and rectangle-valued (RVO) optimization. We construct stochastic IVO/RVO formulations that incorporate subquantiles and superquantiles into the objective functions of the MOO reformulations, providing a new characterization for subquantiles. These formulations provide interpretable trade-offs by capturing both lower- and upper-tail behaviors of loss distributions, thereby going beyond standard empirical risk minimization and classical robust models. To solve the resulting multi-objective problems, we adopt stochastic multi-gradient algorithms and select a Pareto knee solution. In numerical experiments, the proposed algorithms with this selection strategy exhibit improved robustness and reduced variability across test replications under distributional shift compared with empirical risk minimization, while maintaining competitive accuracy.
预测|估计(8篇)
【1】End-to-end data-driven prediction of urban airflow and pollutant dispersion
标题：端到端数据驱动的城市气流和污染物扩散预测
链接：https://arxiv.org/abs/2603.17606
备注：22 pages, 22 figures
摘要：Climate change and the rapid growth of urban populations are intensifying environmental stresses within cities, making the behavior of urban atmospheric flows a critical factor in public health, energy use, and overall livability. This study targets to develop fast and accurate models of urban pollutant dispersion to support decision-makers, enabling them to implement mitigation measures in a timely and cost-effective manner. To reach this goal, an end-to-end data-driven approach is proposed to model and predict the airflow and pollutant dispersion in a street canyon in skimming flow regime. A series of time-resolved snapshots obtained from large eddy simulation (LES) serves as the database. The proposed framework is based on four fundamental steps. Firstly, a reduced basis is obtained by spectral proper orthogonal decomposition (SPOD) of the database. The projection of the time series snapshot data onto the SPOD modes (time-domain approach) provides the temporal coefficients of the dynamics. Secondly, a nonlinear compression of the temporal coefficients is performed by autoencoder to reduce further the dimensionality of the problem. Thirdly, a reduced-order model (ROM) is learned in the latent space using Long Short-Term Memory (LSTM) netowrks. Finally, the pollutant dispersion is estimated from the predicted velocity field through convolutional neural network that maps both fields. The results demonstrate the efficacy of the model in predicting the instantaneous as well as statistically stationary fields over long time horizon.
【2】AirDDE: Multifactor Neural Delay Differential Equations for Air Quality Forecasting
标题：AirDTE：用于空气质量预测的多因素神经延迟方程
链接：https://arxiv.org/abs/2603.17529
备注：AAAI 2026
摘要：Accurate air quality forecasting is essential for public health and environmental sustainability, but remains challenging due to the complex pollutant dynamics. Existing deep learning methods often model pollutant dynamics as an instantaneous process, overlooking the intrinsic delays in pollutant propagation. Thus, we propose AirDDE, the first neural delay differential equation framework in this task that integrates delay modeling into a continuous-time pollutant evolution under physical guidance. Specifically, two novel components are introduced: (1) a memory-augmented attention module that retrieves globally and locally historical features, which can adaptively capture delay effects modulated by multifactor data; and (2) a physics-guided delay evolving function, grounded in the diffusion-advection equation, that models diffusion, delayed advection, and source/sink terms, which can capture delay-aware pollutant accumulation patterns with physical plausibility. Extensive experiments on three real-world datasets demonstrate that AirDDE achieves the state-of-the-art forecasting performance with an average MAE reduction of 8.79\% over the best baselines. The code is available at https://github.com/w2obin/airdde-aaai.
【3】Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates
标题：Baguan-TS：一种用于协变量时间序列预测的序列原生上下文学习模型
链接：https://arxiv.org/abs/2603.17439
摘要 ：Transformers enable in-context learning (ICL) for rapid, gradient-free adaptation in time series forecasting, yet most ICL-style approaches rely on tabularized, hand-crafted features, while end-to-end sequence models lack inference-time adaptation. We bridge this gap with a unified framework, Baguan-TS, which integrates the raw-sequence representation learning with ICL, instantiated by a 3D Transformer that attends jointly over temporal, variable, and context axes. To make this high-capacity model practical, we tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space retrieval-based local calibration; and (ii) output oversmoothing, mitigated via context-overfitting strategy. On public benchmark with covariates, Baguan-TS consistently outperforms established baselines, achieving the highest win rate and significant reductions in both point and probabilistic forecasting metrics. Further evaluations across diverse real-world energy datasets demonstrate its robustness, yielding substantial improvements.
【4】SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction
标题：SCALE：用于虚拟细胞扰动预测的可扩展条件数据库级端点传输
链接：https://arxiv.org/abs/2603.17380
摘要：Virtual cell models aim to enable in silico experimentation by predicting how cells respond to genetic, chemical, or cytokine perturbations from single-cell measurements. In practice, however, large-scale perturbation prediction remains constrained by three coupled bottlenecks: inefficient training and inference pipelines, unstable modeling in high-dimensional sparse expression space, and evaluation protocols that overemphasize reconstruction-like accuracy while underestimating biological fidelity. In this work we present a specialized large-scale foundation model SCALE for virtual cell perturbation prediction that addresses the above limitations jointly. First, we build a BioNeMo-based training and inference framework that substantially improves data throughput, distributed scalability, and deployment efficiency, yielding 12.51* speedup on pretrain and 1.29* on inference over the prior SOTA pipeline under matched system settings. Second, we formulate perturbation prediction as conditional transport and implement it with a set-aware flow architecture that couples LLaMA-based cellular encoding with endpoint-oriented supervision. This design yields more stable training and stronger recovery of perturbation effects. Third, we evaluate the model on Tahoe-100M using a rigorous cell-level protocol centered on biologically meaningful metrics rather than reconstruction alone. On this benchmark, our model improves PDCorr by 12.02% and DE Overlap by 10.66% over STATE. Together, these results suggest that advancing virtual cells requires not only better generative objectives, but also the co-design of scalable infrastructure, stable transport modeling, and biologically faithful evaluation.
【5】Interpretable AI-Assisted Early Reliability Prediction for a Two-Parameter Parallel Root-Finding Scheme
标题：两参数并行寻根方案的可解释人工智能辅助早期可靠性预测
链接：https://arxiv.org/abs/2603.16980
备注：23 pages, 9 figures
摘要：We propose an interpretable AI-assisted reliability diagnostic framework for parameterized root-finding schemes based on kNN-LLE proxy stability profiling and multi-horizon early prediction. The approach augments a numerical solver with a lightweight predictive layer that estimates solver reliability from short prefixes of iteration dynamics, enabling early identification of stable and unstable parameter regimes. For each configuration in the parameter space, raw and smoothed proxy profiles of a largest Lyapunov exponent (LLE) estimator are constructed, from which contractivity-based reliability scores summarizing finite-time convergence are derived. Machine learning models predict the reliability score from early segments of the proxy profile, allowing the framework to determine when solver dynamics become diagnostically informative. Experiments on a two-parameter parallel root-finding scheme show reliable prediction after only a few iterations: the best models achieve R^2=0.48 at horizon T=1, improve to R^2=0.67 by T=3, and exceed R^2=0.89 before the characteristic minimum-location scale of the stability profile. Prediction accuracy increases to R^2=0.96 at larger horizons, with mean absolute errors around 0.03, while inference costs remain negligible (microseconds per sample). The framework provides interpretable stability indicators and supports early decisions during solver execution, such as continuing, restarting, or adjusting parameters.
【6】Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models
标题：奖励DINO：利用Vision Foundation模型预测密集奖励
链接：https://arxiv.org/abs/2603.16978
备注：10 pages, 5 figures, submitted to IEEE
摘要：Well-designed dense reward functions in robot manipulation not only indicate whether a task is completed but also encode progress along the way. Generally, designing dense rewards is challenging and usually requires access to privileged state information available only in simulation, not in real-world experiments. This makes reward prediction models that infer task state information from camera images attractive. A common approach is to predict rewards from expert demonstrations based on visual similarity or sequential frame ordering. However, this biases the resulting reward function towards a specific solution and leaves it undefined in states not covered by the demonstrations. In this work, we introduce Rewarding DINO, a method for language-conditioned reward modeling that learns actual reward functions rather than specific trajectories. The model's compact size allows it to serve as a direct replacement for analytical reward functions with comparatively low computational overhead. We train our model on data sampled from 24 Meta-World+ tasks using a rank-based loss and evaluate pairwise accuracy, rank correlation, and calibration. Rewarding DINO achieves competitive performance in tasks from the training set and generalizes to new settings in simulation and the real world, indicating that it learns task semantics. We also test the model with off-the-shelf reinforcement learning algorithms to solve tasks from our Meta-World+ training set.
【7】Rapid Neural Network Prediction of Linear Block Copolymer Free Energies
标题：线性嵌段共聚物自由能的快速神经网络预测
链接：https://arxiv.org/abs/2603.17391
摘要 ：Free energies are fundamental quantities governing phase behavior and thermodynamic stability in polymer systems, yet their accurate computation often requires extensive simulations and post-processing techniques such as the Bennett Acceptance Ratio (BAR). While BAR provides reliable estimates when applied between closely related thermodynamic states, evaluating free energies across large changes in interaction strength typically requires a sequence of intermediate simulations to maintain sufficient phase-space overlap, substantially increasing computational cost. In this work we develop a machine learning framework for rapidly predicting excess free energies of linear diblock copolymer systems from simulation-derived energetic descriptors. Using dissipative particle dynamics simulations of freely-jointed chain polymers, we construct a dataset of per-chain energetic statistics, including heterogeneous interaction energies, homogeneous interaction energies, and bonded spring energies, and train feed-forward neural networks to learn the relationship between these descriptors and free energies computed using a stratified BAR procedure. The resulting models accurately reproduce the reference free energies across a range of chain lengths, compositions, and densities, including polymer architectures held out from training. In regimes where direct, brute-force BAR estimates become unreliable due to poor phase-space overlap, the neural network predictions remain consistent with the reference values. These results demonstrate that physically informed machine learning models can serve as efficient surrogates for expensive free-energy calculations and provide a promising approach for accelerating thermodynamic analysis of polymer systems.
【8】A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments
标题：用于多维度金融预测的深度学习架构的受控比较：来自918个实验的证据
链接：https://arxiv.org/abs/2603.16886
摘要：Multi-horizon price forecasting is central to portfolio allocation, risk management, and algorithmic trading, yet deep learning architectures have proliferated faster than rigorous financial benchmarks can evaluate them. This study provides a controlled comparison of nine architectures (Autoformer, DLinear, iTransformer, LSTM, ModernTCN, N-HiTS, PatchTST, TimesNet, and TimeXer) spanning Transformer, MLP, CNN, and RNN families across cryptocurrency, forex, and equity index markets at 4-hour and 24-hour horizons. A total of 918 experiments were conducted under a strict five-stage protocol including fixed-seed Bayesian hyperparameter optimization, configuration freezing per asset class, multi-seed retraining, uncertainty aggregation, and statistical validation. ModernTCN achieves the best mean rank (1.333) with a 75 percent first-place rate, followed by PatchTST (2.000). Results reveal a clear three-tier ranking structure and show that architecture explains nearly all performance variance, while seed randomness is negligible. Rankings remain stable across horizons despite 2 to 2.5 times error amplification. Directional accuracy remains near 50 percent across all configurations, indicating that MSE-trained models lack directional skill at hourly resolution. The findings highlight the importance of architectural inductive bias over raw parameter count and provide reproducible guidance for multi-step financial forecasting.
其他神经网络|深度学习|模型|建模(24篇)
【1】Physics-Aware Machine Learning for Seismic and Volcanic Signal Interpretation
标题：用于地震和火山信号解释的物理感知机器学习
链接：https://arxiv.org/abs/2603.17855
备注：18 pages, 2 Tables, 1 Figure, 22 References
摘要：Modern seismic and volcanic monitoring is increasingly shaped by continuous, multi-sensor observations and by the need to extract actionable information from nonstationary, noisy wavefields. In this context, machine learning has moved from a research curiosity to a practical ingredient of processing chains for detection, phase picking, classification, denoising, and anomaly tracking. However, improved accuracy on a fixed dataset is not sufficient for operational use. Models must remain reliable under domain shift (new stations, changing noise, evolving volcanic activity), provide uncertainty that supports decision-making, and connect their outputs to physically meaningful constraints. This paper surveys and organizes recent ML approaches for seismic and volcanic signal analysis, highlighting where classical signal processing provides indispensable inductive bias, how self-supervision and generative modeling can reduce dependence on labels, and which evaluation protocols best reflect transfer across regions. We conclude with open challenges for robust, interpretable, and maintainable AI-assisted monitoring.
【2】Verification and Validation of Physics-Informed Surrogate Component Models for Dynamic Power-System Simulation
标题：动态电力系统仿真的物理信息代理组件模型的验证和确认
链接：https://arxiv.org/abs/2603.17836
摘要：Physics-informed machine learning surrogates are increasingly explored to accelerate dynamic simulation of generators, converters, and other power grid components. The key question, however, is not only whether a surrogate matches a stand-alone component model on average, but whether it remains accurate after insertion into a differential-algebraic simulator, where the surrogate outputs enter the algebraic equations coupling the component to the rest of the system. This paper formulates that in-simulator use as a verification and validation (V\&V) problem. A finite-horizon bound is derived that links allowable component-output error to algebraic-coupling sensitivity, dynamic error amplification, and the simulation horizon. Two complementary settings are then studied: model-based verification against a reference component solver, and data-based validation through conformal calibration of the component-output variables exchanged with the simulator. The framework is general, but the case study focuses on physics-informed neural-network surrogates of second-, fourth-, and sixth-order synchronous-machine models. Results show that good stand-alone surrogate accuracy does not by itself guarantee accurate in-simulator behavior, that the largest discrepancies concentrate in stressed operating regions, and that small equation residuals do not necessarily imply small state-trajectory errors.
【3】Symmetry-Reduced Physics-Informed Learning of Tensegrity Dynamics
标题：简化对称物理的张拉整体动力学知情学习
链接：https://arxiv.org/abs/2603.17824
摘要 ：Tensegrity structures possess intrinsic geometric symmetries that govern their dynamic behavior. However, most existing physics-informed neural network (PINN) approaches for tensegrity dynamics do not explicitly exploit these symmetries, leading to high computational complexity and unstable optimization. In this work, we propose a symmetry-reduced physics-informed neural network (SymPINN) framework that embeds group-theory-based symmetry directly into both the solution expression and the neural network architecture to predict tensegrity dynamics. By decomposing nodes into symmetry orbits and representing free nodal coordinates using a symmetry basis, the proposed method constructs a reduced coordinate representation that preserves geometric symmetry of the structure. The full coordinates are then recovered via symmetry transformations of the reduced solution learned by the network, ensuring that the predicted configurations automatically satisfy the symmetry constraints. In this framework, equivariance is enforced through orbit-based coordinate generation, symmetry-consistent message passing, and physics residual constraints. In addition, SymPINN improves training effectiveness by encoding initial conditions as hard constraints, incorporating Fourier feature encoding to enhance the representation of dynamic motions, and employing a two-stage optimization strategy. Extensive numerical experiments on symmetric T-bars and lander structures demonstrate significantly improved prediction accuracy and computational efficiency compared to standard physics-informed models, indicating the great potential of symmetry-aware learning for structure-preserving modeling of tensegrity dynamics.
【4】Modeling Overlapped Speech with Shuffles
标题：用Shuffle建模重叠语音
链接：https://arxiv.org/abs/2603.17769
摘要：We propose to model parallel streams of data, such as overlapped speech, using shuffles. Specifically, this paper shows how the shuffle product and partial order finite-state automata (FSAs) can be used for alignment and speaker-attributed transcription of overlapped speech. We train using the total score on these FSAs as a loss function, marginalizing over all possible serializations of overlapping sequences at subword, word, and phrase levels. To reduce graph size, we impose temporal constraints by constructing partial order FSAs. We address speaker attribution by modeling (token, speaker) tuples directly. Viterbi alignment through the shuffle product FSA directly enables one-pass alignment. We evaluate performance on synthetic LibriSpeech overlaps. To our knowledge, this is the first algorithm that enables single-pass alignment of multi-talker recordings. All algorithms are implemented using k2 / Icefall.
【5】Towards Infinitely Long Neural Simulations: Self-Refining Neural Surrogate Models for Dynamical Systems
标题：面向无限长的神经模拟：动力系统的自精炼神经代理模型
链接：https://arxiv.org/abs/2603.17750
摘要：Recent advances in autoregressive neural surrogate models have enabled orders-of-magnitude speedups in simulating dynamical systems. However, autoregressive models are generally prone to distribution drift: compounding errors in autoregressive rollouts that severely degrade generation quality over long time horizons. Existing work attempts to address this issue by implicitly leveraging the inherent trade-off between short-time accuracy and long-time consistency through hyperparameter tuning. In this work, we introduce a unifying mathematical framework that makes this tradeoff explicit, formalizing and generalizing hyperparameter-based strategies in existing approaches. Within this framework, we propose a robust, hyperparameter-free model implemented as a conditional diffusion model that balances short-time fidelity with long-time consistency by construction. Our model, Self-refining Neural Surrogate model (SNS), can be implemented as a standalone model that refines its own autoregressive outputs or as a complementary model to existing neural surrogates to ensure long-time consistency. We also demonstrate the numerical feasibility of SNS through high-fidelity simulations of complex dynamical systems over arbitrarily long time horizons.
【6】Embedding World Knowledge into Tabular Models: Towards Best Practices for Embedding Pipeline Design
标题：将世界知识嵌入表格模型：迈向嵌入管道设计的最佳实践
链接：https://arxiv.org/abs/2603.17737
备注：Computational Intelligence 2025 Workshop
摘要：Embeddings are a powerful way to enrich data-driven machine learning models with the world knowledge of large language models (LLMs). Yet, there is limited evidence on how to design effective LLM-based embedding pipelines for tabular prediction. In this work, we systematically benchmark 256 pipeline configurations, covering 8 preprocessing strategies, 16 embedding models, and 2 downstream models. Our results show that it strongly depends on the specific pipeline design whether incorporating the prior knowledge of LLMs improves the predictive performance. In general, concatenating embeddings tends to outperform replacing the original columns with embeddings. Larger embedding models tend to yield better results, while public leaderboard rankings and model popularity are poor performance indicators. Finally, gradient boosting decision trees tend to be strong downstream models. Our findings provide researchers and practitioners with guidance for building more effective embedding pipelines for tabular prediction tasks.
【7】In Trust We Survive: Emergent Trust Learning
标题：信任我们生存：紧急信任学习
链接：https://arxiv.org/abs/2603.17564
摘要：We introduce Emergent Trust Learning (ETL), a lightweight, trust-based control algorithm that can be plugged into existing AI agents. It enables these to reach cooperation in competitive game environments under shared resources. Each agent maintains a compact internal trust state, which modulates memory, exploration, and action selection. ETL requires only individual rewards and local observations and incurs negligible computational and communication overhead. We evaluate ETL in three environments: In a grid-based resource world, trust-based agents reduce conflicts and prevent long-term resource depletion while achieving competitive individual returns. In a hierarchical Tower environment with strong social dilemmas and randomised floor assignments, ETL sustains high survival rates and recovers cooperation even after extended phases of enforced greed. In the Iterated Prisoner's Dilemma, the algorithm generalises to a strategic meta-game, maintaining cooperation with reciprocal opponents while avoiding long-term exploitation by defectors. Code will be released upon publication.
【8】Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions (Extended Version with Technical Appendix)
标题：按域概括策略：关于学习高效且稳健的Q值函数（带技术附录的扩展版本）
链接：https://arxiv.org/abs/2603.17544
摘要：Learning per-domain generalizing policies is a key challenge in learning for planning. Standard approaches learn state-value functions represented as graph neural networks using supervised learning on optimal plans generated by a teacher planner. In this work, we advocate for learning Q-value functions instead. Such policies are drastically cheaper to evaluate for a given state, as they need to process only the current state rather than every successor. Surprisingly, vanilla supervised learning of Q-values performs poorly as it does not learn to distinguish between the actions taken and those not taken by the teacher. We address this by using regularization terms that enforce this distinction, resulting in Q-value policies that consistently outperform state-value policies across a range of 10 domains and are competitive with the planner LAMA-first.
【9】Translation Invariance of Neural Operators for the FitzHugh-Nagumo Model
标题：FitzHugh-Nagumo模型神经运算符的翻译不变性
链接：https://arxiv.org/abs/2603.17523
摘要：Neural Operators (NOs) are a powerful deep learning framework designed to learn the solution operator that arise from partial differential equations. This study investigates NOs ability to capture the stiff spatio-temporal dynamics of the FitzHugh-Nagumo model, which describes excitable cells. A key contribution of this work is evaluating the translation invariance using a novel training strategy. NOs are trained using an applied current with varying spatial locations and intensities at a fixed time, and the test set introduces a more challenging out-of-distribution scenario in which the applied current is translated in both time and space. This approach significantly reduces the computational cost of dataset generation. Moreover we benchmark seven NOs architectures: Convolutional Neural Operators (CNOs), Deep Operator Networks (DONs), DONs with CNN encoder (DONs-CNN), Proper Orthogonal Decomposition DONs (POD-DONs), Fourier Neural Operators (FNOs), Tucker Tensorized FNOs (TFNOs), Localized Neural Operators (LocalNOs). We evaluated these models based on training and test accuracy, efficiency, and inference speed. Our results reveal that CNOs performs well on translated test dynamics. However, they require higher training costs, though their performance on the training set is similar to that of the other considered architectures. In contrast, FNOs achieve the lowest training error, but have the highest inference time. Regarding the translated dynamics, FNOs and their variants provide less accurate predictions. Finally, DONs and their variants demonstrate high efficiency in both training and inference, however they do not generalize well to the test set. These findings highlight the current capabilities and limitations of NOs in capturing complex ionic model dynamics and provide a comprehensive benchmark including their application to scenarios involving translated dynamics.
【10】Cohomological Obstructions to Global Counterfactuals: A Sheaf-Theoretic Foundation for Generative Causal Models
标题：全球反事实的同调障碍：生成因果模型的理论基础
链接：https://arxiv.org/abs/2603.17384
备注：34 pages, 5 figures. Submitted to JMLR
摘要：Current continuous generative models (e.g., Diffusion Models, Flow Matching) implicitly assume that locally consistent causal mechanisms naturally yield globally coherent counterfactuals. In this paper, we prove that this assumption fails fundamentally when the causal graph exhibits non-trivial homology (e.g., structural conflicts or hidden confounders). We formalize structural causal models as cellular sheaves over Wasserstein spaces, providing a strict algebraic topological definition of cohomological obstructions in measure spaces. To ensure computational tractability and avoid deterministic singularities (which we define as manifold tearing), we introduce entropic regularization and derive the Entropic Wasserstein Causal Sheaf Laplacian, a novel system of coupled non-linear Fokker-Planck equations. Crucially, we prove an entropic pullback lemma for the first variation of pushforward measures. By integrating this with the Implicit Function Theorem (IFT) on Sinkhorn optimality conditions, we establish a direct algorithmic bridge to automatic differentiation (VJP), achieving O(1)-memory reverse-mode gradients strictly independent of the iteration horizon. Empirically, our framework successfully leverages thermodynamic noise to navigate topological barriers ("entropic tunneling") in high-dimensional scRNA-seq counterfactuals. Finally, we invert this theoretical framework to introduce the Topological Causal Score, demonstrating that our Sheaf Laplacian acts as a highly sensitive algebraic detector for topology-aware causal discovery.
【11】Learning Permutation Distributions via Reflected Diffusion on Ranks
标题：基于秩上反射扩散的置换分布学习
链接：https://arxiv.org/abs/2603.17353
备注：8 pages, 4 figures, 4 tables
摘要：The finite symmetric group S_n provides a natural domain for permutations, yet learning probability distributions on S_n is challenging due to its factorially growing size and discrete, non-Euclidean structure. Recent permutation diffusion methods define forward noising via shuffle-based random walks (e.g., riffle shuffles) and learn reverse transitions with Plackett-Luce (PL) variants, but the resulting trajectories can be abrupt and increasingly hard to denoise as n grows. We propose Soft-Rank Diffusion, a discrete diffusion framework that replaces shuffle-based corruption with a structured soft-rank forward process: we lift permutations to a continuous latent representation of order by relaxing discrete ranks into soft ranks, yielding smoother and more tractable trajectories. For the reverse process, we introduce contextualized generalized Plackett-Luce (cGPL) denoisers that generalize prior PL-style parameterizations and improve expressivity for sequential decision structures. Experiments on sorting and combinatorial optimization benchmarks show that Soft-Rank Diffusion consistently outperforms prior diffusion baselines, with particularly strong gains in long-sequence and intrinsically sequential settings.
【12】ShuttleEnv: An Interactive Data-Driven RL Environment for Badminton Strategy Modeling
标题
：ShuttleEnv：用于羽毛球策略建模的交互式数据驱动RL环境
链接：https://arxiv.org/abs/2603.17324
摘要：We present ShuttleEnv, an interactive and data-driven simulation environment for badminton, designed to support reinforcement learning and strategic behavior analysis in fast-paced adversarial sports. The environment is grounded in elite-player match data and employs explicit probabilistic models to simulate rally-level dynamics, enabling realistic and interpretable agent-opponent interactions without relying on physics-based simulation. In this demonstration, we showcase multiple trained agents within ShuttleEnv and provide live, step-by-step visualization of badminton rallies, allowing attendees to explore different play styles, observe emergent strategies, and interactively analyze decision-making behaviors. ShuttleEnv serves as a reusable platform for research, visualization, and demonstration of intelligent agents in sports AI. Our ShuttleEnv demo video URL: https://drive.google.com/file/d/1hTR4P16U27H2O0-w316bR73pxE2ucczX/view
【13】Abstraction as a Memory-Efficient Inductive Bias for Continual Learning
标题：抽象作为持续学习的记忆高效归纳偏差
链接：https://arxiv.org/abs/2603.17198
摘要：The real world is non-stationary and infinitely complex, requiring intelligent agents to learn continually without the prohibitive cost of retraining from scratch. While online continual learning offers a framework for this setting, learning new information often interferes with previously acquired knowledge, causes forgetting and degraded generalization. To address this, we propose Abstraction-Augmented Training (AAT), a loss-level modification encouraging models to capture the latent relational structure shared across examples. By jointly optimizing over concrete instances and their abstract representations, AAT introduces a memory-efficient inductive bias that stabilizes learning in strictly online data streams, eliminating the need for a replay buffer. To capture the multi-faceted nature of abstraction, we introduce and evaluate AAT on two benchmarks: a controlled relational dataset where abstraction is realized through entity masking, and a narrative dataset where abstraction is expressed through shared proverbs. Our results show that AAT achieves performance comparable to or exceeding strong experience replay (ER) baselines, despite requiring zero additional memory and only minimal changes to the training objective. This work highlights structural abstraction as a powerful, memory-free alternative to ER.
【14】Contextual Preference Distribution Learning
标题：情境偏好分布学习
链接：https://arxiv.org/abs/2603.17139
备注：In CPAIOR 2026 (23rd International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research)
摘要：Decision-making problems often feature uncertainty stemming from heterogeneous and context-dependent human preferences. To address this, we propose a sequential learning-and-optimization pipeline to learn preference distributions and leverage them to solve downstream problems, for example risk-averse formulations. We focus on human choice settings that can be formulated as (integer) linear programs. In such settings, existing inverse optimization and choice modelling methods infer preferences from observed choices but typically produce point estimates or fail to capture contextual shifts, making them unsuitable for risk-averse decision-making. Using a bounded-variance score function gradient estimator, we train a predictive model mapping contextual features to a rich class of parameterizable distributions. This approach yields a maximum likelihood estimate. The model generates scenarios for unseen contexts in the subsequent optimization phase. In a synthetic ridesharing environment, our approach reduces average post-decision surprise by up to 114$\times$ compared to a risk-neutral approach with perfect predictions and up to 25$\times$ compared to leading risk-averse baselines.
【15】Formal verification of tree-based machine learning models for lateral spreading
标题：横向传播的基于树的机器学习模型的形式化验证
链接：https://arxiv.org/abs/2603.16983
摘要：Machine learning models for geotechnical hazard prediction can achieve high accuracy while learning physically inconsistent relationships from sparse or biased training data. Current remedies (post-hoc explainability, such as SHAP and LIME, and training-time constraints) either diagnose individual predictions approximately or restrict model capacity without providing exhaustive guarantees. This paper encodes trained tree ensembles as logical formulas in a Satisfiability Modulo Theories (SMT) solver and checks physical specifications across the entire input domain, not just sampled points. Four geotechnical specifications (water table depth, PGA monotonicity, distance safety, and flat-ground safety) are formalized as decidable logical formulas and verified via SMT against both XGBoost ensembles and Explainable Boosting Machines (EBMs) trained on the 2011 Christchurch earthquake lateral spreading dataset (7,291 sites, four features). The SMT solver either produces a concrete counterexample where a specification fails or proves that no violation exists. The unconstrained EBM (80.1% accuracy) violates all four specifications. A fully constrained EBM (67.2%) satisfies three of four specifications, demonstrating that iterative constraint application guided by verification can progressively improve physical consistency. A Pareto analysis of 33 model variants reveals a persistent trade-off, as none of the variants studied achieve both greater than 80% accuracy and full compliance with the specified set. SHAP analysis of specification counterexamples shows that the offending feature can rank last, demonstrating that post-hoc explanations do not substitute for formal verification. These results establish a verify-fix-verify engineering loop and a formal certification for deploying physically consistent ML models in safety-critical geotechnical applications.
【16】Implementation of tangent linear and adjoint models for neural networks based on a compiler library tool
标题：基于编译器库工具实现神经网络的切线性和伴随模型
链接：https://arxiv.org/abs/2603.16976
摘要：This paper presents TorchNWP, a compilation library tool for the efficient coupling of artificial intelligence components and traditional numerical models. It aims to address the issues of poor cross-language compatibility, insufficient coupling flexibility, and low data transfer efficiency between operational numerical models developed in Fortran and Python-based deep learning frameworks. Based on LibTorch, it optimizes and designs a unified application-layer calling interface, converts deep learning models under the PyTorch framework into a static binary format, and provides C/C++ interfaces. Then, using hybrid Fortran/C/C++ programming, it enables the deployment of deep learning models within numerical models. Integrating TorchNWP into a numerical model only requires compiling it into a callable link library and linking it during the compilation and linking phase to generate the executable. On this basis, tangent linear and adjoint model based on neural networks are implemented at the C/C++ level, which can shield the internal structure of neural network models and simplify the construction process of four-dimensional variational data assimilation systems. Meanwhile, it supports deployment on heterogeneous platforms, is compatible with mainstream neural network models, and enables mapping of different parallel granularities and efficient parallel execution. Using this tool requires minimal code modifications to the original numerical model, thus reducing coupling costs. It can be efficiently integrated into numerical weather prediction models such as CMA-GFS and MCV, and has been applied to the coupling of deep learning-based physical parameterization schemes (e.g., radiation, non-orographic gravity wave drag) and the development of their tangent linear and adjoint models, significantly improving the accuracy and efficiency of numerical weather prediction.
【17】DeepStage: Learning Autonomous Defense Policies Against Multi-Stage APT Campaigns
标题：DeepStage：学习针对多阶段APT活动的自主防御政策
链接：https://arxiv.org/abs/2603.16969
摘要：This paper presents DeepStage, a deep reinforcement learning (DRL) framework for adaptive, stage-aware defense against Advanced Persistent Threats (APTs). The enterprise environment is modeled as a partially observable Markov decision process (POMDP), where host provenance and network telemetry are fused into unified provenance graphs. Building on our prior work, StageFinder, a graph neural encoder and an LSTM-based stage estimator infer probabilistic attacker stages aligned with the MITRE ATT&CK framework. These stage beliefs, combined with graph embeddings, guide a hierarchical Proximal Policy Optimization (PPO) agent that selects defense actions across monitoring, access control, containment, and remediation. Evaluated in a realistic enterprise testbed using CALDERA-driven APT playbooks, DeepStage achieves a stage-weighted F1-score of 0.89, outperforming a risk-aware DRL baseline by 21.9%. The results demonstrate effective stage-aware and cost-efficient autonomous cyber defense.
【18】A foundation model for electrodermal activity data
标题：皮肤电活动数据的基础模型
链接：https://arxiv.org/abs/2603.16878
摘要：Foundation models have recently extended beyond natural language and vision to timeseries domains, including physiological signals. However, progress in electrodermal activity (EDA) modeling is hindered by the absence of large-scale, curated, and openly accessible datasets. EDA reflects sympathetic nervous system activity and is widely used to infer cognitive load, stress, and engagement. Yet very few wearable devices provide continuous, unobtrusive sensing, and the only large-scale archive to date is proprietary. To address this gap, we compile EDAMAME, a collection of EDA traces from 24 public datasets, comprising more than 25,000 hours from 634 users. Using this resource, we train UME, the first dedicated foundation model for EDA. In eight out of ten scenarios, UME outperforms baselines and matches generalist timeseries foundation models while using 20x fewer computational resources. Our findings, however, also highlight the intrinsic challenges of EDA modeling, motivating further research to unlock its full potential. All datasets, model weights, and code are released to support further research.
【19】Smart Learning to Find Dumb Contracts (Extended Version)
标题：智能学习寻找愚蠢合同（扩展版本）
链接：https://arxiv.org/abs/2304.10726
摘要：We introduce the Deep Learning Vulnerability Analyzer (DLVA) for Ethereum smart contracts based on neural networks. We train DLVA to judge bytecode even though the supervising oracle can only judge source. DLVA's training algorithm is general: we extend a source code analysis to bytecode without any manual feature engineering, predefined patterns, or expert rules. DLVA's training algorithm is also robust: it overcame a 1.25% error rate mislabeled contracts, and--the student surpassing the teacher--found vulnerable contracts that Slither mislabeled. DLVA is much faster than other smart contract vulnerability detectors: DLVA checks contracts for 29 vulnerabilities in 0.2 seconds, a 10-1,000x speedup. DLVA has three key components. First, Smart Contract to Vector (SC2V) uses neural networks to map smart contract bytecode to a high-dimensional floating-point vector. We benchmark SC2V against 4 state-of-the-art graph neural networks and show that it improves model differentiation by 2.2%. Second, Sibling Detector (SD) classifies contracts when a target contract's vector is Euclidian-close to a labeled contract's vector in a training set; although only able to judge 55.7% of the contracts in our test set, it has a Slither-predictive accuracy of 97.4% with a false positive rate of only 0.1%. Third, Core Classifier (CC) uses neural networks to infer vulnerable contracts regardless of vector distance. We benchmark DLVA's CC with 10 ML techniques and show that the CC improves accuracy by 11.3%. Overall, DLVA predicts Slither's labels with an overall accuracy of 92.7% and associated false positive rate of 7.2%. Lastly, we benchmark DLVA against nine well-known smart contract analysis tools. Despite using much less analysis time, DLVA completed every query, leading the pack with an average accuracy of 99.7%, pleasingly balancing high true positive rates with low false positive rates.
【20】A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models
标题：噪音敏感性指数控制单指数和多指数模型中的巨大统计与计算差距
链接：https://arxiv.org/abs/2603.17896
摘要：Understanding when learning is statistically possible yet computationally hard is a central challenge in high-dimensional statistics. In this work, we investigate this question in the context of single- and multi-index models, classes of functions widely studied as benchmarks to probe the ability of machine learning methods to discover features in high-dimensional data. Our main contribution is to show that a Noise Sensitivity Exponent (NSE) - a simple quantity determined by the activation function - governs the existence and magnitude of statistical-to-computational gaps within a broad regime of these models. We first establish that, in single-index models with large additive noise, the onset of a computational bottleneck is fully characterized by the NSE. We then demonstrate that the same exponent controls a statistical-computational gap in the specialization transition of large separable multi-index models, where individual components become learnable. Finally, in hierarchical multi-index models, we show that the NSE governs the optimal computational rate in which different directions are sequentially learned. Taken together, our results identify the NSE as a unifying property linking noise robustness, computational hardness, and feature specialization in high-dimensional learning.
【21】Inhibitory normalization of error signals improves learning in neural circuits
标题：误差信号的抑制性正规化改善了神经回路的学习
链接：https://arxiv.org/abs/2603.17676
备注：28 pages, 7 figures. Submitted to Neural Computation
摘要：Normalization is a critical operation in neural circuits. In the brain, there is evidence that normalization is implemented via inhibitory interneurons and allows neural populations to adjust to changes in the distribution of their inputs. In artificial neural networks (ANNs), normalization is used to improve learning in tasks that involve complex input distributions. However, it is unclear whether inhibition-mediated normalization in biological neural circuits also improves learning. Here, we explore this possibility using ANNs with separate excitatory and inhibitory populations trained on an image recognition task with variable luminosity. We find that inhibition-mediated normalization does not improve learning if normalization is applied only during inference. However, when this normalization is extended to include back-propagated errors, performance improves significantly. These results suggest that if inhibition-mediated normalization improves learning in the brain, it additionally requires the normalization of learning signals.
【22】Atomic Trajectory Modeling with State Space Models for Biomolecular Dynamics
标题：生物分子动力学的状态空间模型原子轨迹建模
链接：https://arxiv.org/abs/2603.17633
摘要：Understanding the dynamic behavior of biomolecules is fundamental to elucidating biological function and facilitating drug discovery. While Molecular Dynamics (MD) simulations provide a rigorous physical basis for studying these dynamics, they remain computationally expensive for long timescales. Conversely, recent deep generative models accelerate conformation generation but are typically either failing to model temporal relationship or built only for monomeric proteins. To bridge this gap, we introduce ATMOS, a novel generative framework based on State Space Models (SSM) designed to generate atom-level MD trajectories for biomolecular systems. ATMOS integrates a Pairformer-based state transition mechanism to capture long-range temporal dependencies, with a diffusion-based module to decode trajectory frames in an autoregressive manner. ATMOS is trained across crystal structures from PDB and conformation trajectory from large-scale MD simulation datasets including mdCATH and MISATO. We demonstrate that ATMOS achieves state-of-the-art performance in generating conformation trajectories for both protein monomers and complex protein-ligand systems. By enabling efficient inference of atomic trajectory of motions, this work establishes a promising foundation for modeling biomolecular dynamics.
【23】Data-driven model order reduction for structures with piecewise linear nonlinearity using dynamic mode decomposition
标题：使用动态模式分解对分段线性非线性结构进行数据驱动模型降阶
链接：https://arxiv.org/abs/2603.17423
摘要：Piecewise-linear nonlinear systems appear in many engineering disciplines. Prediction of the dynamic behavior of such systems is of great importance from practical and theoretical viewpoint. In this paper, a data-driven model order reduction method for piecewise-linear systems is proposed, which is based on dynamic mode decomposition (DMD). The overview of the concept of DMD is provided, and its application to model order reduction for nonlinear systems based on Galerkin projection is explained. The proposed approach uses impulse responses of the system to obtain snapshots of the state variables. The snapshots are then used to extract the dynamic modes that are used to form the projection basis vectors. The dynamics described by the equations of motion of the original full-order system are then projected onto the subspace spanned by the basis vectors. This produces a system with much smaller number of degrees of freedom (DOFs). The proposed method is applied to two representative examples of piecewise linear systems: a cantilevered beam subjected to an elastic stop at its end, and a bonded plates assembly with partial debonding. The reduced order models (ROMs) of these systems are constructed by using the Galerkin projection of the equation of motion with DMD modes alone, or DMD modes with a set of classical constraint modes to be able to handle the contact nonlinearity efficiently. The obtained ROMs are used for the nonlinear forced response analysis of the systems under harmonic loading. It is shown that the ROMs constructed by the proposed method produce accurate forced response results.
【24】A Framework for Modeling Liquefaction-Induced Road Disruptions After Earthquakes: Implications for Emergency Response and Access in the Cascadia Region of North America
标题：地震后液化引发的道路中断建模框架：对北美卡斯卡迪亚地区应急响应和通道的影响
链接：https://arxiv.org/abs/2603.16948
摘要：Large earthquakes along the Cascadia Subduction Zone (CSZ) are expected to trigger widespread soil liquefaction that could disrupt transportation systems across the U.S. Pacific Northwest. However, past regional assessments have relied on simple geologic screening methods and binomial shaking thresholds that are only loosely informed by liquefaction science. This study introduces a mechanics-informed, data-driven framework for estimating liquefaction-induced road closures and service reductions, and the framework is applied to a magnitude-9 CSZ earthquake. Predicted liquefaction severity is translated into segment-level probabilities of closure and reduced service using empirically derived fragility relationships. These probabilities are mapped at 90-m resolution and propagated through the National Highway System using a spatially correlated Monte Carlo simulation to estimate link-level disruption. Results show that impacts are concentrated in low-lying coastal zones, river valleys, and urban waterfronts, with major disruptions expected along critical routes including U.S. Route 101. Local mobility is further examined in Pacific and Grays Harbor counties, Washington, where limited network redundancy, strong shaking, and high liquefaction susceptibility lead to elevated probabilities of isolation and loss of hospital access. Socioeconomic analysis reveals modest but statistically significant associations between road impacts and demographic indicators, suggesting that liquefaction impacts may compound with existing social vulnerabilities. While not a substitute for site-specific analysis, the results provide a regional baseline for emergency planning, risk communication, and prioritization of more advanced geotechnical sampling and analysis. Moreover, the methodology proposed here is not specific to the CSZ, but rather, could be applied to analogous studies of road impacts elsewhere.
其他(27篇)
【1】Unified Spatio-Temporal Token Scoring for Efficient Video VLMs
标题：统一时空令牌评分以实现高效视频VLM
链接：https://arxiv.org/abs/2603.18004
摘要：Token pruning is essential for enhancing the computational efficiency of vision-language models (VLMs), particularly for video-based tasks where temporal redundancy is prevalent. Prior approaches typically prune tokens either (1) within the vision transformer (ViT) exclusively for unimodal perception tasks such as action recognition and object segmentation, without adapting to downstream vision-language tasks; or (2) only within the LLM while leaving the ViT output intact, often requiring complex text-conditioned token selection mechanisms. In this paper, we introduce Spatio-Temporal Token Scoring (STTS), a simple and lightweight module that prunes vision tokens across both the ViT and the LLM without text conditioning or token merging, and is fully compatible with end-to-end training. By learning how to score temporally via an auxiliary loss and spatially via LLM downstream gradients, aided by our efficient packing algorithm, STTS prunes 50% of vision tokens throughout the entire architecture, resulting in a 62% improvement in efficiency during both training and inference with only a 0.7% drop in average performance across 13 short and long video QA tasks. Efficiency gains increase with more sampled frames per video. Applying test-time scaling for long-video QA further yields performance gains of 0.5-1% compared to the baseline. Overall, STTS represents a novel, simple yet effective technique for unified, architecture-wide vision token pruning.
【2】CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention
标题：CARE：协方差感知和等级增强分解以实现多头潜在注意力
链接：https://arxiv.org/abs/2603.17946
备注：Accepted at ICLR 2026. Conference paper. 10 pages main text; 34 pages total including references and appendix. 11 figures and 20 tables in total
摘要：Converting pretrained attention modules such as grouped-query attention (GQA) into multi-head latent attention (MLA) can improve expressivity without increasing KV-cache cost, making it attractive for efficient inference. However, many practical conversion baselines rely on weight-only low-rank approximations (e.g., SVD-style initializations) and uniform rank allocation. They focus on minimizing the difference between weight matrices rather than on how those weights affect input activations, ignore the covariance structure of activations, and enforce uniform rank across layers, causing activation drift and degraded attention fidelity. To address these issues, we propose CARE, a Covariance-Aware, Rank-Enhanced MLA conversion pipeline under a fixed KV width. CARE introduces three key steps: (i) activation-preserving factorization, which aligns the approximation with the actual input activations rather than just the weights; (ii) adjusted-rank allocation, which spreads a fixed KV budget across layers by giving more capacity to layers that need it most; and (iii) KV-parity mapping, which reparameterizes the converted K and V to fit the MLA format while keeping the KV-cache size unchanged. Our method outperforms a uniform-rank SVD baseline on Qwen3-4B/30B-A3B-Instruct-2507 and Llama-3.1-8B/70B-Instruct, reducing one-shot perplexity by up to 215x and improving mean accuracy by up to 1.70x at matched KV budgets. With a brief post-SVD healing fine-tune, we fully recover the original model's accuracy.
【3】Operator-Theoretic Foundations and Policy Gradient Methods for General MDPs with Unbounded Costs
标题：无界成本一般MDPs的运营商理论基础和政策梯度方法
链接：https://arxiv.org/abs/2603.17875
摘要：Markov decision processes (MDPs) is viewed as an optimization of an objective function over certain linear operators over general function spaces. Using the well-established perturbation theory of linear operators, this viewpoint allows one to identify derivatives of the objective function as a function of the linear operators. This leads to generalization of many well-known results in reinforcement learning to cases with generate state and action spaces. Prior results of this type were only established in the finite-state finite-action MDP settings and in settings with certain linear function approximations. The framework also leads to new low-complexity PPO-type reinforcement learning algorithms for general state and action space MDPs.
【4】RHYME-XT: A Neural Operator for Spatiotemporal Control Systems
标题：RPHYME-XT：时空控制系统的神经运算器
链接：https://arxiv.org/abs/2603.17867
备注：6 pages, 5 figures. Submitted to IEEE Control Systems Letters (L-CSS) and CDC 2026
摘要：We propose RHYME-XT, an operator-learning framework for surrogate modeling of spatiotemporal control systems governed by input-affine nonlinear partial integro-differential equations (PIDEs) with localized rhythmic behavior. RHYME-XT uses a Galerkin projection to approximate the infinite-dimensional PIDE on a learned finite-dimensional subspace with spatial basis functions parameterized by a neural network. This yields a projected system of ODEs driven by projected inputs. Instead of integrating this non-autonomous system, we directly learn its flow map using an architecture for learning flow functions, avoiding costly computations while obtaining a continuous-time and discretization-invariant representation. Experiments on a neural field PIDE show that RHYME-XT outperforms a state-of-the-art neural operator and is able to transfer knowledge effectively across models trained on different datasets, through a fine-tuning process.
【5】Text-to-Stage: Spatial Layouts from Long-form Narratives
标题：文本到舞台：长篇叙事的空间布局
链接：https://arxiv.org/abs/2603.17832
摘要：In this work, we probe the ability of a language model to demonstrate spatial reasoning from unstructured text, mimicking human capabilities and automating a process that benefits many downstream media applications. Concretely, we study the narrative-to-play task: inferring stage-play layouts (scenes, speaker positions, movements, and room types) from text that lacks explicit spatial, positional, or relational cues. We then introduce a dramaturgy-inspired deterministic evaluation suite and, finally, a training and inference recipe that combines rejection SFT using Best-of-N sampling with RL from verifiable rewards via GRPO. Experiments on a text-only corpus of classical English literature demonstrate improvements over vanilla models across multiple metrics (character attribution, spatial plausibility, and movement economy), as well as alignment with an LLM-as-a-judge and subjective human preferences.
【6】ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
标题：ChopGrad：通过截断反向传播进行潜在视频传播的像素级损失
链接：https://arxiv.org/abs/2603.17812
摘要：Recent video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models in the pixel domain incurs prohibitive memory costs, as activations accumulate across the entire video sequence. This fundamental limitation also makes fine-tuning these models with pixel-wise losses computationally intractable for long or high-resolution videos. This paper introduces ChopGrad, a truncated backpropagation scheme for video decoding, limiting gradient computation to local frame windows while maintaining global consistency. We provide a theoretical analysis of this approximation and show that it enables efficient fine-tuning with frame-wise losses. ChopGrad reduces training memory from scaling linearly with the number of video frames (full backpropagation) to constant memory, and compares favorably to existing state-of-the-art video diffusion models across a suite of conditional video generation tasks with pixel-wise losses, including video super-resolution, video inpainting, video enhancement of neural-rendered scenes, and controlled driving video generation.
【7】Attention Sinks Induce Gradient Sinks
标题：注意力下降会导致梯度下降
链接：https://arxiv.org/abs/2603.17771
备注：10 pages, 5 figures
摘要：Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing studies have largely focused on the forward pass, making it unclear whether their connection is direct or mediated by a training-time mechanism. We study this question from the perspective of backpropagation. Empirically and theoretically, we show that under causal mask, attention sinks can induce pronounced gradient concentration, which we term gradient sinks. Furthermore, in pre-norm architectures with RMSNorm, massive activations can be understood as an adaptive response to this localized gradient pressure during training. To test this hypothesis, we introduce V-scale, a modification that adjusts value-path backpropagated gradients. In pretrained V-scale models, attention sinks are preserved whereas massive activations are suppressed. These results support the interpretation that gradient sink is a key training-time mediator linking attention sinks and massive activations.
【8】Flow Matching Policy with Entropy Regularization
标题：具有熵规则化的流量匹配策略
链接：https://arxiv.org/abs/2603.17685
摘要 ：Diffusion-based policies have gained significant popularity in Reinforcement Learning (RL) due to their ability to represent complex, non-Gaussian distributions. Stochastic Differential Equation (SDE)-based diffusion policies often rely on indirect entropy control due to the intractability of the exact entropy, while also suffering from computationally prohibitive policy gradients through the iterative denoising chain. To overcome these issues, we propose Flow Matching Policy with Entropy Regularization (FMER), an Ordinary Differential Equation (ODE)-based online RL framework. FMER parameterizes the policy via flow matching and samples actions along a straight probability path, motivated by optimal transport. FMER leverages the model's generative nature to construct an advantage-weighted target velocity field from a candidate set, steering policy updates toward high-value regions. By deriving a tractable entropy objective, FMER enables principled maximum-entropy optimization for enhanced exploration. Experiments on sparse multi-goal FrankaKitchen benchmarks demonstrate that FMER outperforms state-of-the-art methods, while remaining competitive on standard MuJoco benchmarks. Moreover, FMER reduces training time by 7x compared to heavy diffusion baselines (QVPO) and 10-15% relative to efficient variants.
【9】One-Step Sampler for Boltzmann Distributions via Drifting
标题：通过漂移对Boltzmann分布进行一步采样
链接：https://arxiv.org/abs/2603.17579
摘要：We present a drifting-based framework for amortized sampling of Boltzmann distributions defined by energy functions. The method trains a one-step neural generator by projecting samples along a Gaussian-smoothed score field from the current model distribution toward the target Boltzmann distribution. For targets specified only up to an unknown normalization constant, we derive a practical target-side drift from a smoothed energy and use two estimators: a local importance-sampling mean-shift estimator and a second-order curvature-corrected approximation. Combined with a mini-batch Gaussian mean-shift estimate of the sampler-side smoothed score, this yields a simple stop-gradient objective for stable one-step training. On a four-mode Gaussian-mixture Boltzmann target, our sampler achieves mean error $0.0754$, covariance error $0.0425$, and RBF MMD $0.0020$. Additional double-well and banana targets show that the same formulation also handles nonconvex and curved low-energy geometries. Overall, the results support drifting as an effective way to amortize iterative sampling from Boltzmann distributions into a single forward pass at test time.
【10】Identifying Latent Actions and Dynamics from Offline Data via Demonstrator Diversity
标题：通过演示者多样性从离线数据中识别潜在动作和动态
链接：https://arxiv.org/abs/2603.17577
摘要：Can latent actions and environment dynamics be recovered from offline trajectories when actions are never observed? We study this question in a setting where trajectories are action-free but tagged with demonstrator identity. We assume that each demonstrator follows a distinct policy, while the environment dynamics are shared across demonstrators and identity affects the next observation only through the chosen action. Under these assumptions, the conditional next-observation distribution $p(o_{t+1}\mid o_t,e)$ is a mixture of latent action-conditioned transition kernels with demonstrator-specific mixing weights. We show that this induces, for each state, a column-stochastic nonnegative matrix factorization of the observable conditional distribution. Using sufficiently scattered policy diversity and rank conditions, we prove that the latent transitions and demonstrator policies are identifiable up to permutation of the latent action labels. We extend the result to continuous observation spaces via a Gram-determinant minimum-volume criterion, and show that continuity of the transition map over a connected state space upgrades local permutation ambiguities to a single global permutation. A small amount of labeled action data then suffices to fix this final ambiguity. These results establish demonstrator diversity as a principled source of identifiability for learning latent actions and dynamics from offline RL data.
【11】Deploying Semantic ID-based Generative Retrieval for Large-Scale Podcast Discovery at Spotify
标题：在Spotify上部署基于语义ID的生成式检索用于大规模播客发现
链接：https://arxiv.org/abs/2603.17540
摘要：Podcast listening is often grounded in a set of favorite shows, while listener intent can evolve over time. This combination of stable preferences and changing intent motivates recommendation approaches that support both familiarity and exploration. Traditional recommender systems typically emphasize long-term interaction patterns, and are less explicitly designed to incorporate rich contextual signals or flexible, intent-aware discovery objectives. In this setting, models that can jointly reason over semantics, context, and user state offer a promising direction. Large Language Models (LLMs) provide strong semantic reasoning and contextual conditioning for discovery-oriented recommendation, but deploying them in production introduces challenges in catalog grounding, user-level personalization, and latency-critical serving. We address these challenges with GLIDE, a production-scale generative recommender for podcast discovery at Spotify. GLIDE formulates recommendation as an instruction-following task over a discretized catalog using Semantic IDs, enabling grounded generation over a large inventory. The model conditions on recent listening history and lightweight user context, while injecting long-term user embeddings as soft prompts to capture stable preferences under strict inference constraints. We evaluate GLIDE using offline retrieval metrics, human judgments, and LLM-based evaluation, and validate its impact through large-scale online A/B testing. Across experiments involving millions of users, GLIDE increases non-habitual podcast streaming on Spotify home surface by up to 5.4% and new-show discovery by up to 14.3%, while meeting production cost and latency constraints.
【12】Bootstrapping Coding Agents: The Specification Is the Program
标题：引导编码代理：规范就是程序
链接：https://arxiv.org/abs/2603.17399
备注：To appear in IEEE Software
摘要 ：A coding agent can bootstrap itself. Starting from a 926-word specification and a first implementation produced by an existing agent (Claude Code), a newly generated agent re-implements the same specification correctly from scratch. This reproduces, in the domain of AI coding agents, the classical bootstrap sequence known from compiler construction, and instantiates the meta-circular property known from Lisp. The result carries a practical implication: the specification, not the implementation, is the stable artifact of record. Improving an agent means improving its specification; the implementation is, in principle, regenerable at any time.
【13】Efficient Exploration at Scale
标题：大规模高效勘探
链接：https://arxiv.org/abs/2603.17378
摘要：We develop an online learning algorithm that dramatically improves the data efficiency of reinforcement learning from human feedback (RLHF). Our algorithm incrementally updates reward and language models as choice data is received. The reward model is fit to the choice data, while the language model is updated by a variation of reinforce, with reinforcement signals provided by the reward model. Several features enable the efficiency gains: a small affirmative nudge added to each reinforcement signal, an epistemic neural network that models reward uncertainty, and information-directed exploration. With Gemma large language models (LLMs), our algorithm matches the performance of offline RLHF trained on 200K labels using fewer than 20K labels, representing more than a 10x gain in data efficiency. Extrapolating from our results, we expect our algorithm trained on 1M labels to match offline RLHF trained on 1B labels. This represents a 1,000x gain. To our knowledge, these are the first results to demonstrate that such large improvements are possible.
【14】Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
标题：超越离群值：由数字和结构双重敏感性驱动的无数据分层混合精度量化方法
链接：https://arxiv.org/abs/2603.17354
摘要：Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data.
【15】A Progressive Visual-Logic-Aligned Framework for Ride-Hailing Adjudication
标题：渐进的视觉逻辑一致的乘车裁决框架
链接：https://arxiv.org/abs/2603.17328
摘要：The efficient adjudication of responsibility disputes is pivotal for maintaining marketplace fairness. However, the exponential surge in ride-hailing volume renders manual review intractable, while conventional automated methods lack the reasoning transparency required for quasi-judicial decisions. Although Multimodal LLMs offer a promising paradigm, they fundamentally struggle to bridge the gap between general visual semantics and rigorous evidentiary protocols, often leading to perceptual hallucinations and logical looseness. To address these systemic misalignments, we introduce RideJudge, a Progressive Visual-Logic-Aligned Framework. Instead of relying on generic pre-training, we bridge the semantic gap via SynTraj, a synthesis engine that grounds abstract liability concepts into concrete trajectory patterns. To resolve the conflict between massive regulation volume and limited context windows, we propose an Adaptive Context Optimization strategy that distills expert knowledge, coupled with a Chain-of-Adjudication mechanism to enforce active evidentiary inquiry. Furthermore, addressing the inadequacy of sparse binary feedback for complex liability assessment, we implement a novel Ordinal-Sensitive Reinforcement Learning mechanism that calibrates decision boundaries against hierarchical severity. Extensive experiments show that our RideJudge-8B achieves 88.41\% accuracy, surpassing 32B-scale baselines and establishing a new standard for interpretable adjudication.
【16】Topology-Preserving Deep Joint Source-Channel Coding for Semantic Communication
标题：用于语义通信的保拓深度联合源通道编码
链接：https://arxiv.org/abs/2603.17126
备注：Submitted to IEEE Journals for possible publication
摘要：Many wireless vision applications, such as autonomous driving, require preservation of global structural information rather than only per-pixel fidelity. However, existing Deep joint source-channel coding (DeepJSCC) schemes mainly optimize pixel-wise losses and provide no explicit protection of connectivity or topology. This letter proposes TopoJSCC, a topology-aware DeepJSCC framework that integrates persistent-homology regularizers to end-to-end training. Specifically, we enforce topological consistency by penalizing Wasserstein distances between cubical persistence diagrams of original and reconstructed images, and between Vietoris--Rips persistence of latent features before and after the channel to promote a robust latent manifold. TopoJSCC is based on end-to-end learning and requires no side information. Experiments show improved topology preservation and peak signal-to-noise ratio (PSNR) in low signal-to-noise ratio (SNR) and bandwidth-ratio regimes.
【17】Cascade-Aware Multi-Agent Routing: Spatio-Temporal Sidecars and Geometry-Switching
标题：级联感知多代理路由：时空侧机和几何交换
链接：https://arxiv.org/abs/2603.17112
摘要：A common architectural pattern in advanced AI reasoning systems is the symbolic graph network: specialized agents or modules connected by delegation edges, routing tasks through a dynamic execution graph. Current schedulers optimize load and fitness but are geometry-blind: they do not model how failures propagate differently in tree-like versus cyclic regimes. In tree-like delegation, a single failure can cascade exponentially; in dense cyclic graphs, failures tend to self-limit. We identify this observability gap, quantify its system-level cost, and propose a lightweight mitigation. We formulate online geometry control for route-risk estimation on time-indexed execution graphs with route-local failure history. Our approach combines (i) a Euclidean spatio-temporal propagation baseline, (ii) a hyperbolic route-risk model with temporal decay (and optional burst excitation), and (iii) a learned geometry selector over structural features. The selector is a compact MLP (9->12->1) using six topology statistics plus three geometry-aware signals: BFS shell-growth slope, cycle-rank norm, and fitted Poincare curvature. On the Genesis 3 benchmark distribution, adaptive switching improves win rate in the hardest non_tree regime from 64-72% (fixed hyperbolic variants) to 92%, and achieves 87.2% overall win rate. To measure total system value, we compare against Genesis 3 routing without any spatio-temporal sidecar, using only native bandit/LinUCB signals (team fitness and mean node load). This baseline achieves 50.4% win rate overall and 20% in tree-like regimes; the full sidecar recovers 87.2% overall (+36.8 pp), with +48 to +68 pp gains in tree-like settings, consistent with a cascade-sensitivity analysis. Overall, a 133-parameter sidecar substantially mitigates geometry-blind failure propagation in one high-capability execution-graph system.
【18】SENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrieval
标题：SENSE：基于隐私保护语义检索的高效EEG到文本转换
链接：https://arxiv.org/abs/2603.17109
摘要：Decoding brain activity into natural language is a major challenge in AI with important applications in assistive communication, neurotechnology, and human-computer interaction. Most existing Brain-Computer Interface (BCI) approaches rely on memory-intensive fine-tuning of Large Language Models (LLMs) or encoder-decoder models on raw EEG signals, resulting in expensive training pipelines, limited accessibility, and potential exposure of sensitive neural data. We introduce SENSE (SEmantic Neural Sparse Extraction), a lightweight and privacy-preserving framework that translates non-invasive electroencephalography (EEG) into text without LLM fine-tuning. SENSE decouples decoding into two stages: on-device semantic retrieval and prompt-based language generation. EEG signals are locally mapped to a discrete textual space to extract a non-sensitive Bag-of-Words (BoW), which conditions an off-the-shelf LLM to synthesize fluent text in a zero-shot manner. The EEG-to-keyword module contains only ~6M parameters and runs fully on-device, ensuring raw neural signals remain local while only abstract semantic cues interact with language models. Evaluated on a 128-channel EEG dataset across six subjects, SENSE matches or surpasses the generative quality of fully fine-tuned baselines such as Thought2Text while substantially reducing computational overhead. By localizing neural decoding and sharing only derived textual cues, SENSE provides a scalable and privacy-aware retrieval-augmented architecture for next-generation BCIs.
【19】PRISM: Demystifying Retention and Interaction in Mid-Training
标题：PRISM：揭秘中期训练中的保留和互动
链接：https://arxiv.org/abs/2603.17074
摘要：We present PRISM, a comprehensive empirical study of mid-training design choices for large language models. Through controlled experiments across seven base models spanning four families (Granite, LLaMA, Mistral, Nemotron-H), two architecture types (dense Transformer and attention-Mamba hybrid), and scales from 3B to 24B parameters, we show that mid-training on approximately 27B high-quality tokens yields consistent gains of +15 to +40 points on math, +5 to +12 points on code, and +6 to +13 points on science benchmarks while preserving general performance. The full PRISM to RL pipeline improves macro-average across six reasoning benchmarks from under 12 to 29-42 (a 3-4x improvement), whereas RL applied directly to most of the base models remains substantially less effective, with AIME scores near zero. Data composition matters most at mid-training, not RL: including science data during mid-training unlocks +17 to +28 point GPQA-Diamond gains during RL, while changing the RL mix produces less than 2 point differences. Mechanistically, mid-training densely restructures over 90% of model weights, while RL makes sparse, front-loaded refinements to approximately 5% of parameters. Representation analysis (CKA) confirms that RL consistently preserves mid-training's representational geometry (over 0.998 CKA) across architectures. Crucially, RL applies identical weight changes regardless of starting point, yet only succeeds on mid-trained models, consistent with mid-training placing the model in a configuration from which RL can effectively improve performance. Our results demonstrate that retention-aware mid-training is highly effective for reliable reasoning enhancement and provide practical guidance for designing robust mid-training pipelines.
【20】Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization
标题：早期量化缩减码本：一种简单的多样性保持令牌化修复方法
链接：https://arxiv.org/abs/2603.17052
摘要：Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data representations for large language models, diffusion models, and other generative models. Despite its prevalence, the characteristics and behaviors of vector quantization in generative models remain largely underexplored. In this study, we systematically investigate the issue of collapses in vector quantization, where collapsed representations are observed across discrete codebook tokens and continuous latent embeddings. By leveraging both synthetic and real datasets, we identify the severity of each type of collapses and triggering conditions. Our analysis reveals that random initialization and limited encoder capacity result in tokens collapse and embeddings collapse. Building on these findings, we propose potential solutions aimed at mitigating each collapse. To the best of our knowledge, this is the first comprehensive study examining representation collapsing problems in vector quantization.
【21】Entropy-Aware Task Offloading in Mobile Edge Computing
标题：移动边缘计算中的信息感知任务卸载
链接：https://arxiv.org/abs/2603.16949
备注：13 pages, submitted to Journal of Blockchain Research
摘要：Mobile Edge Computing (MEC) technology has been introduced to enable could computing at the edge of the network in order to help resource limited mobile devices with time sensitive data processing tasks. In this paradigm, mobile devices can offload their computationally heavy tasks to more efficient nearby MEC servers via wireless communication. Consequently, the main focus of researches on the subject has been on development of efficient offloading schemes, leaving the privacy of mobile user out. While the Blockchain technology is used as the trust mechanism for secured sharing of the data, the privacy issues induced from wireless communication, namely, usage pattern and location privacy are the centerpiece of this work. The effects of these privacy concerns on the task offloading Markov Decision Process (MDP) is addressed and the MDP is solved using a Deep Recurrent Q-Netwrok (DRQN). The Numerical simulations are presented to show the effectiveness of the proposed method.
【22】What on Earth is AlphaEarth? Hierarchical structure and functional interpretability for global land cover
标题：AlphaEarth到底是什么？全球土地覆盖的层次结构和功能解释性
链接：https://arxiv.org/abs/2603.16911
摘要：Geospatial foundation models generate high-dimensional embeddings that achieve strong predictive performance, yet their internal organization remains obscure, limiting their scientific use. Recent interpretability studies relate Google AlphaEarth Foundations (GAEF) embeddings to continuous environmental variables, but it is still unclear whether the embedding space exhibits a functional or hierarchical organization, in which some dimensions act as specialized representations while others encode shared or broader geospatial structure. In this work, we propose a functional interpretability framework that reverse-engineers the role of embedding dimensions by characterizing their contribution to land cover structure from observed classification behavior. The approach combines large-scale experimentation with a structural analysis of embedding-class relationships based on feature importance patterns and progressive ablation. Our results show that embedding dimensions exhibit consistent and non-uniform functional behavior, allowing them to be categorized along a hierarchical functional spectrum: specialist dimensions associated with specific land cover classes, low- and mid-generalist dimensions capturing shared characteristics between classes, and highgeneralist dimensions reflecting broader environmental gradients. Critically, we find that accurate land cover classification (98% of baseline performance) can be achieved using as few as 2 to 12 of the 64 available dimensions, depending on the class. This demonstrates substantial redundancy in the embedding space and offers a pathway toward significant reductions in computational cost. Together, these findings reveal that AlphaEarth embeddings are not only physically informative, but also functionally organized into a hierarchical structure, providing practical guidance for dimension selection in operational classification tasks.
【23】From Language to Action in Arabic: Reliable Structured Tool Calling via Data-Centric Fine-Tuning
标题：从阿拉伯语语言到动作：通过以数据为中心的微调进行可靠的结构化工具调用
链接：https://arxiv.org/abs/2603.16901
摘要：Function-calling language models are essential for agentic AI systems that translate natural language into executable structured actions, yet existing models exhibit severe structural instability when applied to Arabic. We present AISA-AR-FunctionCall, a production-oriented Arabic function-calling framework built on a 270M-parameter FunctionGemma backbone and trained through systematic dataset auditing, schema repair, tool-aware prompt restructuring, and full-parameter supervised fine-tuning. On a held-out test set, fine-tuning reduces parse failures from 87\% to below 1\%, improves function name accuracy by more than eightfold, and substantially enhances argument alignment across dialects and domains. Error analysis reveals a transition from structural collapse to semantic misalignment, suggesting that serialization stability and decision-level reasoning are separable challenges. We further explore a reasoning-augmented LoRA variant that introduces explicit intermediate reasoning prior to tool invocation. All datasets and models are publicly released under the AISA framework.
【24】Multi-Armed Sequential Hypothesis Testing by Betting
标题：基于打赌的多臂序贯假设检验
链接：https://arxiv.org/abs/2603.17925
摘要：We consider a variant of sequential testing by betting where, at each time step, the statistician is presented with multiple data sources (arms) and obtains data by choosing one of the arms. We consider the composite global null hypothesis $\mathscr{P}$ that all arms are null in a certain sense (e.g. all dosages of a treatment are ineffective) and we are interested in rejecting $\mathscr{P}$ in favor of a composite alternative $\mathscr{Q}$ where at least one arm is non-null (e.g. there exists an effective treatment dosage). We posit an optimality desideratum that we describe informally as follows: even if several arms are non-null, we seek $e$-processes and sequential tests whose performance are as strong as the ones that have oracle knowledge about which arm generates the most evidence against $\mathscr{P}$. Formally, we generalize notions of log-optimality and expected rejection time optimality to more than one arm, obtaining matching lower and upper bounds for both. A key technical device in this optimality analysis is a modified upper-confidence-bound-like algorithm for unobservable but sufficiently "estimable" rewards. In the design of this algorithm, we derive nonasymptotic concentration inequalities for optimal wealth growth rates in the sense of Kelly [1956]. These may be of independent interest.
【25】Consistency of the $k$-Nearest Neighbor Regressor under Complex Survey Designs
标题：复杂测量设计下$k$-最近邻回归量的一致性
链接：https://arxiv.org/abs/2603.17551
摘要：We study the consistency of the $k$-nearest neighbor regressor under complex survey designs. While consistency results for this algorithm are well established for independent and identically distributed data, corresponding results for complex survey data are lacking. We show that the $k$-nearest neighbor regressor is consistent under regularity conditions on the sampling design and the distribution of the data. We derive lower bounds for the rate of convergence and show that these bounds exhibit the curse of dimensionality, as in the independent and identically distributed setting. Empirical studies based on simulated and real data illustrate our theoretical findings.
【26】Mirror Descent on Riemannian Manifolds
标题：Riemann Manifold上的镜像下降
链接：https://arxiv.org/abs/2603.17527
摘要：Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.
【27】Kriging via variably scaled kernels
标题：通过超规模内核进行克里格
链接：https://arxiv.org/abs/2603.16950
摘要：Classical Gaussian processes and Kriging models are commonly based on stationary kernels, whereby correlations between observations depend exclusively on the relative distance between scattered data. While this assumption ensures analytical tractability, it limits the ability of Gaussian processes to represent heterogeneous correlation structures. In this work, we investigate variably scaled kernels as an effective tool for constructing non-stationary Gaussian processes by explicitly modifying the correlation structure of the data. Through a scaling function, variably scaled kernels alter the correlations between data and enable the modeling of targets exhibiting abrupt changes or discontinuities. We analyse the resulting predictive uncertainty via the variably scaled kernel power function and clarify the relationship between variably scaled kernels-based constructions and classical non-stationary kernels. Numerical experiments demonstrate that variably scaled kernels-based Gaussian processes yield improved reconstruction accuracy and provide uncertainty estimates that reflect the underlying structure of the data
机器翻译由腾讯交互翻译提供，仅供参考
点击“阅读原文”获取带摘要的学术速递
Python社区是高质量的Python/Django开发社区
本文地址：http://www.python88.com/topic/194153
本文地址：http://www.python88.com/topic/194153