|
创作新主题 |
社区所有版块导航
DATA
| docker Elasticsearch |
WEB开发
| linux MongoDB Redis DATABASE NGINX 其他Web框架 web工具 zookeeper tornado NoSql Bootstrap js peewee Git bottle IE MQ Jquery |
机器学习
| 机器学习算法 |
产品
| 短视频 |
印度
| 印度 |
一周十大热门主题
点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！
cs.LG 方向，今日共计147篇
大模型相关(22篇)
【1】AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air
标题：AirLLM：基于扩散策略的自适应LoRA，用于空中LLM的远程微调
链接：https://arxiv.org/abs/2507.11515
备注：11 pages, 8 figures
摘要：在边缘设备上操作大型语言模型（LLM）越来越受到有限的通信带宽和紧张的计算和内存成本的挑战。因此，云辅助的远程微调变得不可或缺。然而，现有的低秩自适应（LoRA）方法通常采用固定或启发式秩配置，并且所有LoRA参数的后续空中传输可能相当低效。为了解决这一限制，我们开发了AirLLM，一个用于通信感知LoRA适应的分层扩散政策框架。具体而言，AirLLM将秩配置建模为跨越所有LoRA插入的投影的结构化动作向量。为了解决潜在的高维顺序决策问题，近端策略优化（PPO）代理通过联合观察无线状态和语言复杂性来生成粗粒度决策，然后通过去噪扩散隐式模型（DDIM）进行细化，以产生高分辨率，任务和信道自适应的秩向量。这两个模块交替优化，DDIM在无分类器指导（CFG）范式下训练，以保持与PPO奖励的一致性。在不同信噪比下的实验表明，AirLLM始终增强微调性能，同时显着降低传输成本，突出了可扩展性和有效的远程微调在空中的有效性，扩散驱动，细化秩自适应。
摘要：Operating Large Language Models (LLMs) on edge devices is increasingly challenged by limited communication bandwidth and strained computational and memory costs. Thus, cloud-assisted remote fine-tuning becomes indispensable. Nevertheless, existing Low-Rank Adaptation (LoRA) approaches typically employ fixed or heuristic rank configurations, and the subsequent over-the-air transmission of all LoRA parameters could be rather inefficient. To address this limitation, we develop AirLLM, a hierarchical diffusion policy framework for communication-aware LoRA adaptation. Specifically, AirLLM models the rank configuration as a structured action vector that spans all LoRA-inserted projections. To solve the underlying high-dimensional sequential decision-making problem, a Proximal Policy Optimization (PPO) agent generates coarse-grained decisions by jointly observing wireless states and linguistic complexity, which are then refined via Denoising Diffusion Implicit Models (DDIM) to produce high-resolution, task- and channel-adaptive rank vectors. The two modules are optimized alternatively, with the DDIM trained under the Classifier-Free Guidance (CFG) paradigm to maintain alignment with PPO rewards. Experiments under varying signal-to-noise ratios demonstrate that AirLLM consistently enhances fine-tuning performance while significantly reducing transmission costs, highlighting the effectiveness of reinforcement-driven, diffusion-refined rank adaptation for scalable and efficient remote fine-tuning over the air.
【2】HKGAI-V1: Towards Regional Sovereign Large Language Model for Hong Kong
标题：HKGAI-V1：迈向香港区域主权大型语言模式
链接：https://arxiv.org/abs/2507.11502
摘要：本文介绍了HKGAI-V1的开发，这是一个基础主权大型语言模型（LLM），是为建立专为香港量身定制的价值一致的人工智能基础设施而开发的计划的一部分。针对该地区独特的多语言环境（广东话，普通话和英语），其在“一国两制”框架下独特的社会法律背景，以及特定的当地文化和价值观考虑，该模型建立在DeepSeek架构之上，并通过多方面的全参数微调过程系统地与区域规范保持一致。该系统还与检索增强生成系统相结合，以确保及时获得有事实依据的信息。其核心贡献在于设计和实施一个全面的、针对特定地区的人工智能调整和安全框架，这一框架通过两项关键成就得到了证明：1）HKGAI-V1本身的成功发展-在处理香港特有的文化敏感查询方面，它超越了一般用途的模型，并体现了“治理嵌入式”的数字主权方法-使香港能够对公共服务，法律制度和教育等关键领域的人工智能应用进行控制。2)开发专有的对抗性香港价值基准，这是一个严格的工具，用于在具有挑战性的条件下评估模型与当地道德和法律标准的一致性。通过记录这些成就，该论文不仅提供了一个技术工件，而且还提供了一个可复制的蓝图，用于开发深深植根于当地身份的先进的、以区域为重点的人工智能系统。
摘要：This paper presents the development of HKGAI-V1, a foundational sovereign large language model (LLM), developed as part of an initiative to establish value-aligned AI infrastructure specifically tailored for Hong Kong. Addressing the region's unique multilingual environment (Cantonese, Mandarin, and English), its distinct socio-legal context under the "one country, two systems" framework, and specific local cultural and value considerations, the model is built upon the DeepSeek architecture and systematically aligned with regional norms through a multifaceted full parameter fine-tuning process. It is further integrated with a retrieval-augmented generation (RAG) system to ensure timely and factually grounded information access. The core contribution lies in the design and implementation of a comprehensive, region-specific AI alignment and safety framework, demonstrated through two key achievements: 1) The successful development of HKGAI-V1 itself - which outper-forms general-purpose models in handling Hong Kong-specific culturally sensitive queries, and embodies a "governance-embedded" approach to digital sovereignty - empowers Hong Kong to exercise control over AI applications in critical sectors including public services, legal systems, and edu-cation. 2) The development of the proprietary Adversarial HK Value Benchmark, a rigorous tool for evaluating model alignment with local ethical and legal stand-ards under challenging conditions. By documenting these achievements, the paper provides not only a technological artifact but also a replicable blueprint for developing advanced, regionally focused AI systems deeply rooted in their local identities.
【3】LRMR: LLM-Driven Relational Multi-node Ranking for Lymph Node Metastasis Assessment in Rectal Cancer
标题：LRMR：LLM驱动的直肠癌淋巴结转移评估的关系多节点排名
链接：https://arxiv.org/abs/2507.11457
摘要：直肠癌淋巴结（LN）转移的准确术前评估指导治疗决策，但基于形态学标准的常规MRI评估显示诊断性能有限。虽然已经开发了一些人工智能模型，但它们通常作为黑匣子运行，缺乏临床信任所需的可解释性。此外，这些模型通常孤立地评估节点，忽略了患者层面的背景。为了解决这些限制，我们引入了LRMR，一个LLM驱动的关系多节点排名框架。这种方法将诊断任务从一个直接的分类问题重构为一个结构化的推理和排序过程。LRMR框架分两个阶段运作。首先，多模式大语言模型（LLM）分析患者所有LN的合成蒙太奇图像，生成详细介绍十个不同放射学特征的结构化报告。其次，基于文本的LLM对不同患者之间的这些报告进行成对比较，根据不良特征的严重程度和数量建立相对风险排名。我们对117例直肠癌患者的回顾性队列研究进行了评估。LRMR的曲线下面积（AUC）为0.7917，F1得分为0.7200，优于一系列深度学习基线，包括ResNet50（AUC 0.7708）。消融研究证实了我们的两个主要贡献的价值：删除关系排名阶段或结构化提示阶段导致显着的性能下降，AUC分别下降到0.6875和0.6458。我们的工作表明，通过两阶段LLM框架将视觉感知与认知推理脱钩，为评估直肠癌淋巴结转移提供了一个强大的，可解释的和有效的新范式。
摘要 ：Accurate preoperative assessment of lymph node (LN) metastasis in rectal cancer guides treatment decisions, yet conventional MRI evaluation based on morphological criteria shows limited diagnostic performance. While some artificial intelligence models have been developed, they often operate as black boxes, lacking the interpretability needed for clinical trust. Moreover, these models typically evaluate nodes in isolation, overlooking the patient-level context. To address these limitations, we introduce LRMR, an LLM-Driven Relational Multi-node Ranking framework. This approach reframes the diagnostic task from a direct classification problem into a structured reasoning and ranking process. The LRMR framework operates in two stages. First, a multimodal large language model (LLM) analyzes a composite montage image of all LNs from a patient, generating a structured report that details ten distinct radiological features. Second, a text-based LLM performs pairwise comparisons of these reports between different patients, establishing a relative risk ranking based on the severity and number of adverse features. We evaluated our method on a retrospective cohort of 117 rectal cancer patients. LRMR achieved an area under the curve (AUC) of 0.7917 and an F1-score of 0.7200, outperforming a range of deep learning baselines, including ResNet50 (AUC 0.7708). Ablation studies confirmed the value of our two main contributions: removing the relational ranking stage or the structured prompting stage led to a significant performance drop, with AUCs falling to 0.6875 and 0.6458, respectively. Our work demonstrates that decoupling visual perception from cognitive reasoning through a two-stage LLM framework offers a powerful, interpretable, and effective new paradigm for assessing lymph node metastasis in rectal cancer.
【4】Step-wise Policy for Rare-tool Knowledge (SPaRK): Offline RL that Drives Diverse Tool Use in LLMs
标题：稀有工具知识的分步政策（SPaRK）：推动LLC中多样化工具使用的离线RL
链接：https://arxiv.org/abs/2507.11371
备注：12 pages, 4 figures
摘要：我们提出了稀有工具知识的逐步策略（SPaRK），这是一种新型的强化学习框架，可以教授大型语言模型探索传统高温采样之外的各种工具使用模式。基于逐步强化学习的最新进展，我们引入了一个双目标奖励系统，该系统同时优化了答案质量和工具多样性，通过离线PPO在MMLU-Pro数据集合成生成的轨迹上训练Llama-3.1 8B模型。我们的方法独特地采用了稀有性优先的开发策略，其中GPT-4 o法官在八种不同的工具加上思维链推理中对候选动作进行评分，该政策有利于不太频繁使用但仍然可行的工具，以鼓励系统的探索。实证结果表明，SPaRK在14个MMLU-Pro类别中实现了具有竞争力的性能，同时与基线和监督微调方法相比，在工具选择方面表现出显着更高的熵，这表明通过明确的工具多样性进行算法探索可以在不牺牲准确性的情况下增强推理能力。
摘要：We present Step-wise Policy for Rare-tool Knowledge (SPaRK), a novel reinforcement learning framework that teaches large language models to explore diverse tool usage patterns beyond conventional high-temperature sampling. Building on recent advances in step-wise reinforcement learning, we introduce a dual-objective reward system that simultaneously optimizes for answer quality and tool diversity, training a Llama-3.1 8B model through offline PPO on synthetically generated trajectories from the MMLU-Pro dataset. Our approach uniquely employs a rarity-first exploitation strategy where a GPT-4o judge scores candidate actions across eight distinct tools plus chain-of-thought reasoning, with the policy favoring less-frequently used but still viable tools to encourage systematic exploration. Empirical results demonstrate that SPaRK achieves competitive performance across 14 MMLU-Pro categories while exhibiting significantly higher entropy in tool selection compared to both baseline and supervised fine-tuning approaches, suggesting that algorithmic exploration through explicit tool diversity can enhance reasoning capabilities without sacrificing accuracy.
【5】Guiding LLM Decision-Making with Fairness Reward Models
标题：利用公平奖励模型指导LLM决策
链接：https://arxiv.org/abs/2507.11344
摘要：大型语言模型越来越多地用于支持高风险决策，可能会影响谁获得保释或获得贷款。朴素的思维链抽样可以提高平均决策准确性，但也被证明会放大不公平的偏见。为了解决这一挑战，并使值得信赖的使用推理模型在高风险的决策，我们提出了一个框架，用于训练一个可推广的公平奖励模型（FRM）。我们的模型为LLM推理分配了一个公平分数，使系统能够在推理链上聚合决策时降低有偏见的轨迹的权重，并支持公平的轨迹。我们证明了一个单一的公平奖励模型，在弱监督下训练，LLM注释的有偏见与无偏见推理的例子，在任务，域和模型家族之间转移，而无需额外的微调。应用于现实世界的决策任务，包括累犯预测和社交媒体适度，我们表明，我们的方法不断提高公平性，同时匹配，甚至超过，基线准确性。
摘要：Large language models are increasingly used to support high-stakes decisions, potentially influencing who is granted bail or receives a loan. Naive chain-of-thought sampling can improve average decision accuracy, but has also been shown to amplify unfair bias. To address this challenge and enable the trustworthy use of reasoning models in high-stakes decision-making, we propose a framework for training a generalizable Fairness Reward Model (FRM). Our model assigns a fairness score to LLM reasoning, enabling the system to down-weight biased trajectories and favor equitable ones when aggregating decisions across reasoning chains. We show that a single Fairness Reward Model, trained on weakly supervised, LLM-annotated examples of biased versus unbiased reasoning, transfers across tasks, domains, and model families without additional fine-tuning. Applied to real-world decision-making tasks including recidivism prediction and social media moderation, we show that our approach consistently improves fairness while matching, or even surpassing, baseline accuracy.
【6】Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
标题：通过受控价值载体激活实现大型语言模型的内部价值一致
链接：https://arxiv.org/abs/2507.11316
备注：25 pages, 14 figures. Accepted by ACL 2025 (main conference)
摘要：将大型语言模型（LLM）与人类价值观相结合已经引起了越来越多的关注，因为它提供了清晰度，透明度和适应不断变化的场景的能力。在本文中，我们介绍了一种受控值向量激活（ConVA）方法，该方法通过解释值如何在其潜在表示中编码来直接对齐LLM的内部值，并修改相关激活以确保LLM中的值一致。为了确保准确和公正的解释，我们提出了一个上下文控制的值向量识别方法。为了在不牺牲模型性能的情况下始终如一地控制值，我们引入了一种门控值向量激活方法，以实现有效的和最小程度的值控制。实验表明，我们的方法在10个基本值上实现了最高的控制成功率，而不会损害LLM的性能和流畅性，并确保目标值，即使是相反的和潜在的恶意输入提示。源代码和数据可在~ https://github.com/hr-jin/ConVA获得。
摘要：Aligning Large Language Models (LLMs) with human values has attracted increasing attention since it provides clarity, transparency, and the ability to adapt to evolving scenarios. In this paper, we introduce a Controlled Value Vector Activation (ConVA) method that directly aligns the internal values of LLMs by interpreting how a value is encoded in their latent representations and modifies relevant activations to ensure consistent values in LLMs. To ensure an accurate and unbiased interpretation, we propose a context-controlled value vector identification method. To consistently control values without sacrificing model performance, we introduce a gated value vector activation method for effective and minimum degree of value control. Experiments show that our method achieves the highest control success rate across 10 basic values without hurting LLM performance and fluency, and ensures target values even with opposite and potentially malicious input prompts. Source code and data are available at~ https://github.com/hr-jin/ConVA.
【7】Mixture of Experts in Large Language Models
标题：大型语言模型中的专家混合体
链接：https://arxiv.org/abs/2507.11181
摘要 ：本文对大型语言模型中的专家混合（MoE）架构进行了全面回顾，强调了其在保持最小计算开销的同时显着增强模型性能的能力。通过对理论基础、核心架构设计和大型语言模型（LLM）应用的系统分析，我们研究了专家门控和路由机制、分层和稀疏MoE配置、元学习方法、多模式和多任务学习场景、真实部署案例以及深度学习的最新进展和挑战。我们的分析确定了MoE的主要优势，包括与等效贝叶斯方法相比更优越的模型容量，改进的特定任务性能以及有效扩展模型容量的能力。我们还强调了确保专家多样性、准确校准和可靠推理聚合的重要性，因为这些对于最大化MoE架构的有效性至关重要。最后，本文概述了目前的研究局限性，开放的挑战，和有前途的未来方向，提供了一个基础，继续创新的MoE架构及其应用。
摘要：This paper presents a comprehensive review of the Mixture-of-Experts (MoE) architecture in large language models, highlighting its ability to significantly enhance model performance while maintaining minimal computational overhead. Through a systematic analysis spanning theoretical foundations, core architectural designs, and large language model (LLM) applications, we examine expert gating and routing mechanisms, hierarchical and sparse MoE configurations, meta-learning approaches, multimodal and multitask learning scenarios, real-world deployment cases, and recent advances and challenges in deep learning. Our analysis identifies key advantages of MoE, including superior model capacity compared to equivalent Bayesian approaches, improved task-specific performance, and the ability to scale model capacity efficiently. We also underscore the importance of ensuring expert diversity, accurate calibration, and reliable inference aggregation, as these are essential for maximizing the effectiveness of MoE architectures. Finally, this review outlines current research limitations, open challenges, and promising future directions, providing a foundation for continued innovation in MoE architecture and its applications.
【8】What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests
标题：LLM应该忘记什么？量化LLM中的个人数据以满足有权被遗忘的请求
链接：https://arxiv.org/abs/2507.11128
备注：16 pages, 3 figures. Accepted at the 7th Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD 2025), ECML PKDD 2025, Porto, Portugal
摘要：大型语言模型（LLM）可以记忆和泄露个人信息，这引发了人们对欧盟GDPR合规性的担忧，特别是被遗忘权（RTBF）。现有的机器学习方法假设要忘记的数据是已知的，但没有解决如何识别模型中存储了哪些个体-事实关联。隐私审计技术通常在群体级别上操作或针对一小部分标识符，限制了对个人级别数据查询的适用性。我们介绍了WikiMem，一个包含5,000多个自然语言金丝雀的数据集，涵盖了来自Wikidata的243个与人类相关的属性，以及一个与模型无关的度量来量化LLM中的人类事实关联。我们的方法排名地面真理值对反事实使用校准的负对数似然在释义提示。我们评估了15个LLM（410 M-70 B参数）的200个个体，显示记忆与主题网络存在和模型规模相关。我们为在个人层面识别LLM中记忆的个人数据提供了基础，从而为机器学习和RTBF请求动态构建遗忘集。
摘要：Large Language Models (LLMs) can memorize and reveal personal information, raising concerns regarding compliance with the EU's GDPR, particularly the Right to Be Forgotten (RTBF). Existing machine unlearning methods assume the data to forget is already known but do not address how to identify which individual-fact associations are stored in the model. Privacy auditing techniques typically operate at the population level or target a small set of identifiers, limiting applicability to individual-level data inquiries. We introduce WikiMem, a dataset of over 5,000 natural language canaries covering 243 human-related properties from Wikidata, and a model-agnostic metric to quantify human-fact associations in LLMs. Our approach ranks ground-truth values against counterfactuals using calibrated negative log-likelihood across paraphrased prompts. We evaluate 200 individuals across 15 LLMs (410M-70B parameters), showing that memorization correlates with subject web presence and model scale. We provide a foundation for identifying memorized personal data in LLMs at the individual level, enabling the dynamic construction of forget sets for machine unlearning and RTBF requests.
【9】Multi-Trigger Poisoning Amplifies Backdoor Vulnerabilities in LLMs
标题：多触发中毒放大了LLM中的后门漏洞
链接：https://arxiv.org/abs/2507.11112
摘要：最近的研究表明，大型语言模型（LLM）容易受到数据中毒攻击，其中恶意训练示例嵌入了由特定输入模式触发的隐藏行为。然而，大多数现有的作品假设一个短语，并专注于攻击的有效性，提供有限的理解触发机制和多个触发器如何在模型中相互作用。在本文中，我们提出了一个框架，研究中毒的LLM。我们发现，多个不同的后门触发器可以共存于一个单一的模型中，而不会相互干扰，使对手能够同时嵌入多个触发器。使用具有高嵌入相似性的多个触发器，我们证明了中毒触发器可以实现强大的激活，即使令牌被替换或分隔长令牌跨度。我们的研究结果揭示了LLMs中更广泛，更持久的脆弱性表面。为了减轻这种威胁，我们提出了一种事后恢复方法，该方法基于逐层权重差异分析选择性地重新训练特定的模型组件。我们的方法有效地消除了触发器的行为与最小的参数更新，提出了一个实用和有效的防御多触发器中毒。
摘要：Recent studies have shown that Large Language Models (LLMs) are vulnerable to data poisoning attacks, where malicious training examples embed hidden behaviours triggered by specific input patterns. However, most existing works assume a phrase and focus on the attack's effectiveness, offering limited understanding of trigger mechanisms and how multiple triggers interact within the model. In this paper, we present a framework for studying poisoning in LLMs. We show that multiple distinct backdoor triggers can coexist within a single model without interfering with each other, enabling adversaries to embed several triggers concurrently. Using multiple triggers with high embedding similarity, we demonstrate that poisoned triggers can achieve robust activation even when tokens are substituted or separated by long token spans. Our findings expose a broader and more persistent vulnerability surface in LLMs. To mitigate this threat, we propose a post hoc recovery method that selectively retrains specific model components based on a layer-wise weight difference analysis. Our method effectively removes the trigger behaviour with minimal parameter updates, presenting a practical and efficient defence against multi-trigger poisoning.
【10】LogTinyLLM: Tiny Large Language Models Based Contextual Log Anomaly Detection
标题：LogTinyLLM：基于小型大型语言模型的上下文日志异常检测
链接：https://arxiv.org/abs/2507.11071
摘要：由于日志序列的大量和高度复杂性，使用传统的基于规则或基于深度学习的方法进行日志异常检测通常具有挑战性。因此，有效地检测异常日志序列对系统的维护和开发至关重要。本文提出了参数高效微调，特别是低秩自适应（LoRA）和基于适配器的方法，用于在大型日志数据集中发现日志序列中的上下文异常。它比较了Thunderbird数据集上不同的小型大型语言模型（LLM）。结果表明，基于LoRA的微调比基于LogBert的完全微调方法提供了18%至19%的大幅性能提升，准确率得分在97.76%至98.83%之间，而79.37%。
摘要：Log anomaly detection using traditional rule based or deep learning based methods is often challenging due to the large volume and highly complex nature of log sequence. So effective way of detection of anomalous sequence of logs is crucial for system maintenance and development. This paper proposes parameter efficient finetuning specifically low rank adaptation (LoRA) and adapter based approaches for finding contextual anomalies in sequence of logs in large log data set. It compares different tiny large language models (LLMs) on the Thunderbird dataset. The results show that LoRA based finetuning provides substantial performance improvements of 18 to 19 percentage over LogBert based full finetuning approach, achieving accuracy scores between 97.76% and 98.83% compared to 79.37%.
【11】First-Order Error Matters: Accurate Compensation for Quantized Large Language Models
标题：一阶错误很重要：量化大型语言模型的准确补偿
链接：https://arxiv.org/abs/2507.11017
摘要：后训练量化（PTQ）提供了一种有效的方法来压缩大型语言模型（LLM），显着降低内存访问和计算成本。现有的基于补偿的权重校准方法通常依赖于二阶泰勒展开来建模量化误差，假设一阶项在经过良好训练的全精度模型中可以忽略不计。然而，我们发现，渐进式补偿过程引入了潜在权重与其全精度对应项之间的累积一阶偏差，使得这一假设从根本上存在缺陷。为了解决这个问题，我们提出了FOEM，一种新的PTQ方法，显式地将一阶梯度项，以提高量化误差补偿。FOEM通过直接计算潜在权重和全精度权重之间的差异来近似梯度，避免了基于反向传播的梯度计算的高成本和有限的泛化。这种方法引入了最小的额外计算开销。此外，FOEM利用预先计算的Cholesky因子来实时有效地恢复Hessian子矩阵的逆。在广泛的模型和基准测试中进行的大量实验表明，FOEM始终优于经典的GPTQ方法。在3位仅加权量化中，FOEM将Llama 3 -8B的困惑度降低了89.6%，并将Llama 3 - 70 B的5次MMLU准确度从51.7%提高到74.9%，接近78.6%的全精度性能。此外，FOEM可以与GPTAQ和SpinQuant等先进技术无缝集成，在具有挑战性的W 4 A4 KV 4设置下获得额外的改进，并进一步缩小与全精度基线的准确性差距，超越当前最先进的方法所实现的。该代码可在https://github.com/Xingyu-Zheng/FOEM上获得。
摘要：Post-training quantization (PTQ) offers an efficient approach to compressing large language models (LLMs), significantly reducing memory access and computational costs. Existing compensation-based weight calibration methods often rely on a second-order Taylor expansion to model quantization error, under the assumption that the first-order term is negligible in well-trained full-precision models. However, we reveal that the progressive compensation process introduces accumulated first-order deviations between latent weights and their full-precision counterparts, making this assumption fundamentally flawed. To address this, we propose FOEM, a novel PTQ method that explicitly incorporates first-order gradient terms to improve quantization error compensation. FOEM approximates gradients by directly computing the difference between latent and full-precision weights, avoiding the high cost and limited generalization of backpropagation-based gradient computation. This approach introduces minimal additional computational overhead. Moreover, FOEM leverages precomputed Cholesky factors to efficiently recover the inverse of Hessian submatrices in real time. Extensive experiments across a wide range of models and benchmarks demonstrate that FOEM consistently outperforms the classical GPTQ method. In 3-bit weight-only quantization, FOEM reduces the perplexity of Llama3-8B by 89.6%, and improves the 5-shot MMLU accuracy of Llama3-70B from 51.7% to 74.9%, approaching the full-precision performance of 78.6%. Furthermore, FOEM can be seamlessly integrated with advanced techniques such as GPTAQ and SpinQuant, yielding additional improvements under the challenging W4A4KV4 setting, and further narrowing the accuracy gap with full-precision baselines beyond what current state-of-the-art methods achieve. The code is available at https://github.com/Xingyu-Zheng/FOEM.
【12】Towards Practical Benchmarking of Data Cleaning Techniques: On Generating Authentic Errors via Large Language Models
标题：走向数据清理技术的实用基准：通过大型语言模型生成真实错误
链接：https://arxiv.org/abs/2507.10934
摘要：数据质量仍然是数据驱动系统中的一个重要挑战，因为表格数据中的错误会严重影响下游分析和机器学习性能。虽然已经提出了许多错误检测算法，但缺乏多样化的真实世界错误数据集限制了全面的评估。人工错误标注既耗时又不一致，促使人们探索合成错误生成作为一种替代方法。在这项工作中，我们引入了TableEG，这是一个利用大型语言模型（LLM）来生成真实错误的框架。通过采用表微调策略和三元组表示$（I，T，O）$来建模错误生成、检测和纠正任务，TableEG捕获了二维表中固有的复杂依赖关系。在跨越10个不同领域的12个真实世界数据集上进行训练，TableEG确保合成的错误忠实地反映真实的错误分布。实验结果表明，由TableEG产生的错误表现出优越的模式和分布相似性相比，基于规则的方法和LLM产生的错误没有微调。此外，Tableau生成的错误的性能指标与几乎所有数据集和检测算法中的真实错误的性能指标密切相关，特别是对于基于机器学习的检测技术。总体而言，TableEG不仅弥合了合成错误和真实错误之间的差距，还为后续的错误检测和纠正任务建立了一个强大的基准。
摘要：Data quality remains an important challenge in data-driven systems, as errors in tabular data can severely compromise downstream analytics and machine learning performance. Although numerous error detection algorithms have been proposed, the lack of diverse, real-world error datasets limits comprehensive evaluation. Manual error annotation is both time-consuming and inconsistent, motivating the exploration of synthetic error generation as an alternative. In this work, we introduce TableEG, a framework that leverages large language models (LLMs) to generate authentic errors. By employing a table fine-tuning strategy and a triplet representation $(I, T, O)$ to model error generation, detection, and correction tasks, TableEG captures the complex dependencies inherent in two-dimensional tables. Trained on 12 real-world datasets spanning 10 diverse domains, TableEG ensures that the synthesized errors faithfully reflect authentic error distributions. Experimental results indicate that errors generated by TableEG exhibit superior pattern and distribution similarity compared to both rule-based methods and LLM-generated errors without fine-tuning. Furthermore, performance metrics on TableEG-generated errors closely align with those on real-world errors across nearly all datasets and detection algorithms, particularly for machine learning based detection techniques. Overall, TableEG not only bridges the gap between synthetic and real-world errors but also establishes a robust benchmark for subsequent error detection and correction tasks.
【13】LiLM-RDB-SFC: Lightweight Language Model with Relational Database-Guided DRL for Optimized SFC Provisioning
标题：LiLM-RDB-SFC：具有关系数据库引导的DRL的轻量级语言模型，用于优化SFC资源调配
链接：https://arxiv.org/abs/2507.10903
备注：9 pages, 6 figures, Accepted to IEEE 16th International Conference on Network of the Future (NoF) 2025
摘要：服务功能链（SFC）的有效管理和虚拟网络功能（VNF）的最佳放置是现代软件定义网络（SDN）和网络功能虚拟化（NFV）环境中的关键挑战。虽然深度强化学习（DRL）被广泛用于动态网络决策，但其对结构化数据和固定动作规则的固有依赖性往往限制了适应性和响应能力，特别是在不可预测的网络条件下。本文介绍了LiLM-RDB-SFC，一种新的方法相结合的轻量级语言模型（LiLM）和关系数据库（RDB）来回答网络状态查询，以指导DRL模型有效的SFC供应。我们提出的方法利用两个LiLM，双向和自回归Transformers（BART）和微调语言网络T5（FLAN-T5），解释网络数据，并支持与SFC需求，数据中心资源和VNF可用性相关的各种查询类型。结果表明，FLAN-T5优于BART，具有更低的测试损失（0.00161与0.00734相比），更高的准确性（94.79%与80.2%相比），以及更短的处理时间（2小时2分钟与2小时38分钟相比）。此外，与大型语言模型SQLCoder相比，FLAN-T5的准确性与SQLCoder相当，同时将处理时间缩短了96%（SQLCoder：54小时43分钟; FLAN-T5：2小时2分钟）。
摘要：Effective management of Service Function Chains (SFCs) and optimal Virtual Network Function (VNF) placement are critical challenges in modern Software-Defined Networking (SDN) and Network Function Virtualization (NFV) environments. Although Deep Reinforcement Learning (DRL) is widely adopted for dynamic network decision-making, its inherent dependency on structured data and fixed action rules often limits adaptability and responsiveness, particularly under unpredictable network conditions. This paper introduces LiLM-RDB-SFC, a novel approach combining Lightweight Language Model (LiLM) with Relational Database (RDB) to answer network state queries to guide DRL model for efficient SFC provisioning. Our proposed approach leverages two LiLMs, Bidirectional and Auto-Regressive Transformers (BART) and the Fine-tuned Language Net T5 (FLAN-T5), to interpret network data and support diverse query types related to SFC demands, data center resources, and VNF availability. Results demonstrate that FLAN-T5 outperforms BART with a lower test loss (0.00161 compared to 0.00734), higher accuracy (94.79% compared to 80.2%), and less processing time (2h 2min compared to 2h 38min). Moreover, when compared to the large language model SQLCoder, FLAN-T5 matches the accuracy of SQLCoder while cutting processing time by 96% (SQLCoder: 54 h 43 min; FLAN-T5: 2 h 2 min).
【14】Domain-Adaptive Small Language Models for Structured Tax Code Prediction
标题：结构化税法预测的领域自适应小语言模型
链接：https://arxiv.org/abs/2507.10880
备注：10 pages, 3 figures
摘要：每天，跨国公司都要处理数千笔交易，每笔交易都必须遵守不同司法管辖区的税收法规，而且往往有细微差别。确定产品和服务税码（如HSN或SAC）是税务合规方面的一个主要用例。准确确定这些代码对于避免任何税务处罚至关重要。本文提出了一种域自适应小语言模型（SLM）的编码器-解码器架构的产品和服务税代码的增强预测。在这种方法中，我们解决的问题，预测层次结构的税法序列使用非结构化的产品和服务数据。我们采用了SLM的编码器-解码器架构的基础上，因为这使得连续生成的税收代码，以捕捉税收代码内存在的分层依赖关系。我们的实验表明，编码器-解码器SLM可以成功地应用于结构化税务代码的顺序预测，这是一个在当前NLP研究中尚未探索的领域。在本文中，我们证明了优越的性能的域自适应编码器-解码器SLM平面分类器时，适用于协调系统的Nominium（HSN），并实现优越的结果相比，解码器和编码器的结构化序列生成任务的架构。这一方法也可以扩展到其他政府规定的税收商品代码，如联合国标准产品和服务代码（UNSPSC）或巴西的Nomenclatura Comum do Mercosul（NCM）。
摘要：Every day, multinational firms process thousands of transactions, each of which must adhere to tax regulations that vary by jurisdiction and are often nuanced. The determination of product and service tax codes, such as HSN or SAC is a major use case in Tax compliance. An accurate determination of such codes is imperative to avoid any tax penalties. This paper proposes a domain-adaptive small language model (SLM) with an encoder-decoder architecture for the enhanced prediction of product and service tax codes. In this approach, we address the problem of predicting hierarchical tax code sequences using unstructured product and services data. We employ an SLM based upon encoder-decoder architecture as this enables sequential generation of tax codes to capture the hierarchical dependencies present within the tax codes. Our experiments demonstrate that encoder-decoder SLMs can be successfully applied to the sequential prediction of structured tax codes, a domain that remains comparatively unexplored in current NLP research. In this paper, we demonstrate the superior performance of the domain-adaptive encoder-decoder SLMs over flat classifiers when applied to the Harmonized System of Nomenclature (HSN), and achieve superior results compared to decoder-only and encoder-only architectures for structured sequence generation tasks. This approach can also be scaled to other government-mandated tax commodity codes, such as United Nations Standard Products and Services Codes (UNSPSC), or Brazil's Nomenclatura Comum do Mercosul (NCM).
【15】Language Models for Adult Service Website Text Analysis
标题：成人服务网站文本分析的语言模型
链接：https://arxiv.org/abs/2507.10743
备注：32 pages, 12 figures, 1 table
摘要：性交易是指使用武力，欺诈或胁迫，迫使个人违背自己的意愿进行商业性行为。成人服务网站已经并将继续与性贩运活动联系在一起，为贩运者宣传其受害者提供了一个平台。因此，参与打击性贩运的组织在试图确定潜在的性贩运受害者时经常使用ASW数据。将ASW数据转化为可操作见解的关键挑战是文本分析。先前使用ASW数据的研究表明，ASW广告文本对链接广告很重要。然而，由于大量使用表情符号，语法不佳，以及故意混淆以逃避执法审查，因此使用此文本具有挑战性。我们对这一应用领域的语言建模方法进行了全面的研究，包括简单的信息检索方法，预训练的Transformers和自定义的Transformer模型。我们证明了ASW文本数据的特性允许使用相对较小的GPU资源训练高效的自定义Transformer模型，并有效地用于消费者硬件上的推理。我们的自定义模型在准确率、召回率、F1分数和ROC AUC方面优于知名的仅编码器Transformer模型的微调变体，包括BERT-base、RoBERTa和ModernBERT。我们展示了在与ASW数据分析相关的三个任务上使用我们性能最好的自定义配置：（i）分解ASW数据的图形表示中的巨型组件，（ii）聚类ASW广告文本，以及（iii）使用学习的令牌嵌入来理解我们研究的非法背景下表情符号的使用。我们开发的模型代表了ASW文本分析的重大进步，可以在各种下游应用和研究中加以利用。
摘要：Sex trafficking refers to the use of force, fraud, or coercion to compel an individual to perform in commercial sex acts against their will. Adult service websites (ASWs) have and continue to be linked to sex trafficking, offering a platform for traffickers to advertise their victims. Thus, organizations involved in the fight against sex trafficking often use ASW data when attempting to identify potential sex trafficking victims. A critical challenge in transforming ASW data into actionable insight is text analysis. Previous research using ASW data has shown that ASW ad text is important for linking ads. However, working with this text is challenging due to its extensive use of emojis, poor grammar, and deliberate obfuscation to evade law enforcement scrutiny. We conduct a comprehensive study of language modeling approaches for this application area, including simple information retrieval methods, pre-trained transformers, and custom transformer models. We demonstrate that characteristics of ASW text data allow efficient custom transformer models to be trained with relatively small GPU resources and used efficiently for inference on consumer hardware. Our custom models outperform fine-tuned variants of well-known encoder-only transformer models, including BERT-base, RoBERTa, and ModernBERT, on accuracy, recall, F1 score, and ROC AUC. We demonstrate the use of our best-performing custom configuration on three tasks related to ASW data analysis: (i) decomposing the giant component in a graph representation of ASW data, (ii) clustering ASW ad text, and (iii) using the learned token embeddings to understand the use of emojis in the illicit context we study. The models we develop represent a significant advancement in ASW text analysis, which can be leveraged in a variety of downstream applications and research.
【16】GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learning
标题：GHPO：稳定有效的LLM强化学习的自适应指南
链接：https://arxiv.org/abs/2507.10628
摘要：具有可验证奖励的强化学习（RLVR）最近已经成为促进大型语言模型（LLM）自我改进的强大范例，特别是在复杂推理任务领域。然而，目前流行的基于策略的强化学习方法往往存在显著的训练不稳定性和效率低下的问题。这主要是由于能力-难度不匹配，训练数据的复杂性经常超过模型的当前能力，导致奖励信号非常稀疏，学习进度停滞。这一挑战对于规模较小、资源效率更高的LLM来说尤其严峻。为了克服这一点，我们引入了引导混合策略优化（GHPO），这是一种新的困难感知强化学习框架。GHPO通过采用自适应提示细化来动态校准任务难度，以提供有针对性的指导。这种独特的方法自适应地平衡了针对当前模型无法解决的问题的直接模仿学习与针对更易于管理的任务的基于探索的强化学习，有效地创建了一个平滑和优化的学习课程。广泛的实验表明，GHPO在六个具有挑战性的数学基准中实现了约5%的平均性能增益，始终优于强大的政策强化学习和课程学习基线。进一步的分析证实，我们的框架显着提高了训练的稳定性和最终的推理性能，从而为开发强大而强大的推理模型提供了一个可扩展的，高效的解决方案。
摘要：Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a powerful paradigm for facilitating the self-improvement of large language models (LLMs), particularly in the domain of complex reasoning tasks. However, prevailing on-policy RL methods often contend with significant training instability and inefficiency. This is primarily due to a capacity-difficulty mismatch, where the complexity of training data frequently outpaces the model's current capabilities, leading to critically sparse reward signals and stalled learning progress. This challenge is particularly acute for smaller, more resource-efficient LLMs. To overcome this, we introduce the Guided Hybrid Policy Optimization (GHPO), a novel difficulty-aware reinforcement learning framework. GHPO dynamically calibrates task difficulty by employing adaptive prompt refinement to provide targeted guidance. This unique approach adaptively balances direct imitation learning for problems currently beyond the model's reach with exploration-based reinforcement learning for more manageable tasks, effectively creating a smooth and optimized learning curriculum. Extensive experiments demonstrate that GHPO achieves an average performance gain of approximately 5% across six challenging mathematics benchmarks, consistently outperforming strong on-policy reinforcement learning and curriculum learning baselines. Further analysis confirms that our framework significantly enhances both training stability and final reasoning performance, thus offering a scalable and efficient solution for developing powerful and robust reasoning models.
【17】Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning
标题：没有能力的理解：符号计算和推理中LLM的架构限制
链接：https://arxiv.org/abs/2507.10624
备注：Substantial change to previous version (experiments, theorem, analysis and related work); currently under review at TMLR
摘要：大型语言模型（LLM）表现出惊人的表面流畅性，但在需要符号推理，算术准确性和逻辑一致性的任务中系统性地失败。本文提供了一个结构性的诊断，这样的失败，揭示了一个持久的差距之间的理解和能力。通过受控实验和架构分析，我们证明了LLM经常阐明正确的原则，而没有可靠地应用它们--失败的根源不是知识获取，而是计算执行。我们把这种现象称为计算裂脑综合征，在这种情况下，指令和行动路径在几何上和功能上是分离的。这个核心限制在各个领域都有出现，从数学运算到关系推理，这也解释了为什么即使在理想化的提示下，模型行为仍然很脆弱。我们认为，LLM功能强大的模式完成引擎，但缺乏原则性的，组合推理的架构脚手架。我们的研究结果描绘了当前LLM能力的边界，并通过元认知控制，原则提升和结构接地执行来激励未来的模型。这一诊断还澄清了为什么机械可解释性的发现可能反映了训练特定的模式协调，而不是普遍的计算原则，以及为什么指令和执行路径之间的几何分离表明神经内省和机械分析的局限性。
摘要：Large Language Models (LLMs) display striking surface fluency yet systematically fail at tasks requiring symbolic reasoning, arithmetic accuracy, and logical consistency. This paper offers a structural diagnosis of such failures, revealing a persistent gap between \textit{comprehension} and \textit{competence}. Through controlled experiments and architectural analysis, we demonstrate that LLMs often articulate correct principles without reliably applying them--a failure rooted not in knowledge access, but in computational execution. We term this phenomenon the computational \textit{split-brain syndrome}, where instruction and action pathways are geometrically and functionally dissociated. This core limitation recurs across domains, from mathematical operations to relational inferences, and explains why model behavior remains brittle even under idealized prompting. We argue that LLMs function as powerful pattern completion engines, but lack the architectural scaffolding for principled, compositional reasoning. Our findings delineate the boundary of current LLM capabilities and motivate future models with metacognitive control, principle lifting, and structurally grounded execution. This diagnosis also clarifies why mechanistic interpretability findings may reflect training-specific pattern coordination rather than universal computational principles, and why the geometric separation between instruction and execution pathways suggests limitations in neural introspection and mechanistic analysis.
【18】LLMs Meet Cross-Modal Time Series Analytics: Overview and Directions
标题：LLM满足跨模式时间序列分析：概述和方向
链接：https://arxiv.org/abs/2507.10620
备注：Accepted at SSTD 2025 (Tutorial). arXiv admin note: text overlap with arXiv:2505.02583
摘要：大型语言模型（LLM）已经成为时间序列分析的一个有前途的范例，利用其大量的参数和文本和时间序列数据的共享顺序性质。然而，时间序列和文本数据之间存在跨模态差距，因为LLM是在文本语料库上预先训练的，并且本身并不针对时间序列进行优化。在本教程中，我们提供了基于LLM的跨模态时间序列分析的最新概述。我们引入了一种分类法，该分类法根据跨模态建模策略将现有方法分为三组，例如，转换，对齐和融合，然后讨论它们在一系列下游任务中的应用。此外，我们总结了几个开放的挑战。本教程旨在扩展LLM在解决跨模态时间序列分析中的实际问题方面的实际应用，同时平衡有效性和效率。参与者将深入了解跨模态时间序列分析的当前进展、方法论和未来研究方向。
摘要：Large Language Models (LLMs) have emerged as a promising paradigm for time series analytics, leveraging their massive parameters and the shared sequential nature of textual and time series data. However, a cross-modality gap exists between time series and textual data, as LLMs are pre-trained on textual corpora and are not inherently optimized for time series. In this tutorial, we provide an up-to-date overview of LLM-based cross-modal time series analytics. We introduce a taxonomy that classifies existing approaches into three groups based on cross-modal modeling strategies, e.g., conversion, alignment, and fusion, and then discuss their applications across a range of downstream tasks. In addition, we summarize several open challenges. This tutorial aims to expand the practical application of LLMs in solving real-world problems in cross-modal time series analytics while balancing effectiveness and efficiency. Participants will gain a thorough understanding of current advancements, methodologies, and future research directions in cross-modal time series analytics.
【19】Fine-tuning Large Language Model for Automated Algorithm Design
标题：用于自动算法设计的微调大型语言模型
链接：https://arxiv.org/abs/2507.10614
摘要：将大型语言模型（LLM）集成到自动算法设计中已经显示出了很大的潜力。一种流行的方法是将LLM嵌入搜索例程中，以迭代地生成和细化候选算法。然而，大多数现有的方法都依赖于为一般编码任务训练的现成LLM，这就留下了一个关键问题：我们是否需要专门为算法设计定制的LLM？如果是这样的话，如何有效地获得这样的LLM，以及它们在不同的算法设计任务中的泛化能力如何？在本文中，我们采取了第一步回答这些问题，探索微调的LLM算法设计。我们引入了基于多样性感知排名（DAR）的采样策略来平衡训练数据的多样性和质量，然后我们利用直接偏好优化来有效地将LLM输出与任务目标相匹配。我们的实验，进行Llama-3.2-1B-Instruct和Llama- 3.1-8B-Instruct，跨越三个不同的算法设计任务。结果表明，微调的LLM可以显着优于其现成的同行与较小的Llama-3.2-1B-Instruct和匹配较大的Llama-3.1-8B-Instruct上的容许集问题。此外，我们观察到有希望的推广：LLM在特定的算法设计任务上进行微调，也可以提高具有不同设置的相关任务的性能。这些发现突出了LLM在算法设计中的任务特定适应的价值，并为未来的研究开辟了新的途径。
摘要：The integration of large language models (LLMs) into automated algorithm design has shown promising potential. A prevalent approach embeds LLMs within search routines to iteratively generate and refine candidate algorithms. However, most existing methods rely on off-the-shelf LLMs trained for general coding tasks,leaving a key question open: Do we need LLMs specifically tailored for algorithm design? If so, how can such LLMs be effectively obtained and how well can they generalize across different algorithm design tasks? In this paper, we take a first step toward answering these questions by exploring fine-tuning of LLMs for algorithm design. We introduce a Diversity-Aware Rank based (DAR) sampling strategy to balance training data diversity and quality, then we leverage direct preference optimization to efficiently align LLM outputs with task objectives. Our experiments, conducted on Llama-3.2-1B-Instruct and Llama- 3.1-8B-Instruct, span three distinct algorithm design tasks. Results suggest that finetuned LLMs can significantly outperform their off-the-shelf counterparts with the smaller Llama-3.2-1B-Instruct and match the larger Llama-3.1-8B-Instruct on the admissible set problem. Moreover, we observe promising generalization: LLMs finetuned on specific algorithm design tasks also improve performance on related tasks with varying settings. These findings highlight the value of task-specific adaptation for LLMs in algorithm design and open new avenues for future research.
【20】Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
标题：子尺度定律：数据密度和训练策略在LLM中的作用
链接：https://arxiv.org/abs/2507.10613
摘要：自然语言处理中的传统缩放定律表明，增加模型大小和训练数据可以提高性能。然而，最近的研究揭示了偏差，特别是在大型语言模型中，性能改进减速，这是一种称为子尺度的现象。本文通过研究数据质量和训练策略对模型性能的影响来重新审视这些缩放定律。通过对400多个模型的大量实证分析，我们发现高数据密度和非最优资源分配是导致子尺度的关键因素。高数据密度会因冗余信息而导致收益递减，而最佳资源分配对于持续的性能改进至关重要。我们提出了一个次优缩放法，更好地预测性能的子尺度制度，突出数据质量和多样性的重要性。
摘要 ：Traditional scaling laws in natural language processing suggest that increasing model size and training data enhances performance. However, recent studies reveal deviations, particularly in large language models, where performance improvements decelerate, which is a phenomenon known as sub-scaling. This paper revisits these scaling laws by examining the impact of data quality and training strategies on model performance. Through extensive empirical analysis of over 400 models, we identify high data density and non-optimal resource allocation as key factors contributing to sub-scaling. High data density leads to diminishing returns due to redundant information, while optimal resource allocation is crucial for sustained performance improvements. We propose a sub-optimal scaling law that better predicts performance in sub-scaling regimes, highlighting the importance of data quality and diversity.
【21】RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services
标题：RedOne：揭示社交网络服务领域特定领域LLM后训练
链接：https://arxiv.org/abs/2507.10605
摘要：社交网络服务（SNS）作为现代信息传播的主要媒介，其发展迅速，对平台内容管理和交互质量的提高提出了重大挑战。最近，大型语言模型（LLM）的发展提供了潜在的解决方案，但现有的研究集中在孤立的任务上，这些任务不仅会从单个场景中的数据扩展中获得越来越少的好处，而且也无法灵活地适应不同的现实环境。为了解决这些挑战，我们引入了RedOne，这是一个特定于领域的法学硕士，旨在打破单任务基线的性能瓶颈并为SNS建立全面的基础。RedOne是通过三阶段训练策略开发的，包括持续预训练，监督微调和偏好优化，使用大规模的真实世界数据集。通过大量实验，RedOne保持了较强的通用能力，与基础模型相比，8项主要SNS任务的平均提升高达14.02%，SNS双语评测基准的平均提升高达7.56%。此外，通过在线测试，与单任务微调基线模型相比，RedOne在有害内容检测中的曝光率降低了11.23%，在查看后搜索中的点击页面率提高了14.95%。这些结果使RedOne成为SNS的强大的特定领域LLM，在各种任务中表现出出色的泛化能力，并在现实世界中具有很好的适用性。
摘要：As a primary medium for modern information dissemination, social networking services (SNS) have experienced rapid growth, which has proposed significant challenges for platform content management and interaction quality improvement. Recently, the development of large language models (LLMs) has offered potential solutions but existing studies focus on isolated tasks, which not only encounter diminishing benefit from the data scaling within individual scenarios but also fail to flexibly adapt to diverse real-world context. To address these challenges, we introduce RedOne, a domain-specific LLM designed to break the performance bottleneck of single-task baselines and establish a comprehensive foundation for the SNS. RedOne was developed through a three-stage training strategy consisting of continue pretraining, supervised fine-tuning, and preference optimization, using a large-scale real-world dataset. Through extensive experiments, RedOne maintains strong general capabilities, and achieves an average improvement up to 14.02% across 8 major SNS tasks and 7.56% in SNS bilingual evaluation benchmark, compared with base models. Furthermore, through online testing, RedOne reduced the exposure rate in harmful content detection by 11.23% and improved the click page rate in post-view search by 14.95% compared with single-tasks finetuned baseline models. These results establish RedOne as a robust domain-specific LLM for SNS, demonstrating excellent generalization across various tasks and promising applicability in real-world scenarios.
【22】Emergence of Hierarchical Emotion Organization in Large Language Models
标题：大型语言模型中分层情感组织的出现
链接：https://arxiv.org/abs/2507.10599
摘要：随着大型语言模型（LLM）越来越多地支持会话代理，了解它们如何建模用户的情感状态对于道德部署至关重要。受情绪轮子（一种认为情绪是分层组织的心理学框架）的启发，我们分析了模型输出中情绪状态之间的概率依赖关系。我们发现，LLM自然形成与人类心理模型一致的分层情感树，更大的模型会形成更复杂的层次结构。我们还发现了跨社会经济角色的情感识别的系统性偏见，以及对交叉的，代表性不足的群体的错误分类。人类研究揭示了惊人的相似之处，表明LLM内化了社会感知的各个方面。除了强调LLM中的紧急情绪推理之外，我们的研究结果暗示了使用认知基础理论来开发更好的模型评估的潜力。
摘要：As large language models (LLMs) increasingly power conversational agents, understanding how they model users' emotional states is critical for ethical deployment. Inspired by emotion wheels -- a psychological framework that argues emotions organize hierarchically -- we analyze probabilistic dependencies between emotional states in model outputs. We find that LLMs naturally form hierarchical emotion trees that align with human psychological models, and larger models develop more complex hierarchies. We also uncover systematic biases in emotion recognition across socioeconomic personas, with compounding misclassifications for intersectional, underrepresented groups. Human studies reveal striking parallels, suggesting that LLMs internalize aspects of social perception. Beyond highlighting emergent emotional reasoning in LLMs, our results hint at the potential of using cognitively-grounded theories for developing better model evaluations.
Graph相关(图学习|图神经网络|图优化等)(8篇)
【1】DuetGraph: Coarse-to-Fine Knowledge Graph Reasoning with Dual-Pathway Global-Local Fusion
标题：DuetShape：采用双路径全局局部融合的从粗到细的知识图推理
链接：https://arxiv.org/abs/2507.11229
摘要：知识图（KGs）对于实现跨各个领域的知识推理至关重要。最近的KG推理方法，结合全球和当地的信息取得了可喜的成果。然而，现有的方法往往遭受分数过平滑，这模糊了正确和不正确的答案之间的区别，阻碍了推理的有效性。为了解决这个问题，我们提出了DuetGraph，一个由粗到细的KG推理机制，具有双通道全局-局部融合。DuetGraph通过将本地（通过消息传递）和全局（通过注意力）信息的处理分离到两个不同的路径中，而不是堆叠来解决过度平滑问题，从而防止相互干扰并保留代表性歧视。此外，DuetGraph引入了一种从粗到细的优化，将实体划分为高分和低分子集。该策略缩小了候选空间，缩小了两个子集之间的得分差距，从而避免了过度平滑，提高了推理质量。在各种数据集上进行的大量实验表明，DuetGraph实现了最先进的（SOTA）性能，推理质量提高了8.7%，训练效率提高了1.8倍。
摘要：Knowledge graphs (KGs) are vital for enabling knowledge reasoning across various domains. Recent KG reasoning methods that integrate both global and local information have achieved promising results. However, existing methods often suffer from score over-smoothing, which blurs the distinction between correct and incorrect answers and hinders reasoning effectiveness. To address this, we propose DuetGraph, a coarse-to-fine KG reasoning mechanism with dual-pathway global-local fusion. DuetGraph tackles over-smoothing by segregating -- rather than stacking -- the processing of local (via message passing) and global (via attention) information into two distinct pathways, preventing mutual interference and preserving representational discrimination. In addition, DuetGraph introduces a coarse-to-fine optimization, which partitions entities into high- and low-score subsets. This strategy narrows the candidate space and sharpens the score gap between the two subsets, which alleviates over-smoothing and enhances inference quality. Extensive experiments on various datasets demonstrate that DuetGraph achieves state-of-the-art (SOTA) performance, with up to an 8.7% improvement in reasoning quality and a 1.8$\times$ acceleration in training efficiency.
【2】GATE: Graph Attention Neural Networks with Real-Time Edge Construction for Robust Indoor Localization using Mobile Embedded Devices
标题：GATE：具有实时边缘构造的图形注意力神经网络，用于使用移动嵌入式设备的鲁棒室内定位
链接：https://arxiv.org/abs/2507.11053
摘要：准确的室内定位对于在智能环境和导航系统中实现空间上下文至关重要。Wi-Fi接收信号强度（RSS）指纹识别由于其与移动嵌入式设备的兼容性而成为广泛使用的室内定位方法。深度学习（DL）模型通过学习不同位置的RSS变化来提高定位任务的准确性，但它们假设指纹向量存在于欧几里得空间中，无法结合空间关系和真实世界RSS噪声的非均匀分布。这导致异构移动设备之间的泛化较差，其中硬件和信号处理的变化使RSS读数失真。图神经网络（GNN）可以通过将室内位置编码为节点并将其空间和信号关系建模为边缘来改进传统的DL模型。然而，GNN难以应对非欧几里德噪声分布，并存在GNN盲点问题，导致在接入点（AP）密集的环境中准确性下降。为了解决这些挑战，我们提出了GATE，这是一种新的框架，它构建了指纹向量的自适应图表示，同时保留了室内状态空间拓扑，对RSS噪声的非欧几里德结构进行建模，以减轻环境噪声并解决设备异构性问题。GATE引入了1）用于增强消息传递的新型注意力超空间向量（AHV），2）用于减轻GNN盲点的新型多维超空间向量（MDHV），以及3）用于动态图自适应的新的实时边缘构造（RTEC）方法。在具有不同路径长度、AP密度和异构设备的多个室内空间中进行的广泛的真实世界评估表明，与最先进的室内定位框架相比，GATE的平均定位误差降低了1.6倍至4.72倍，最坏情况下的误差降低了1.85倍至4.57倍。
摘要：Accurate indoor localization is crucial for enabling spatial context in smart environments and navigation systems. Wi-Fi Received Signal Strength (RSS) fingerprinting is a widely used indoor localization approach due to its compatibility with mobile embedded devices. Deep Learning (DL) models improve accuracy in localization tasks by learning RSS variations across locations, but they assume fingerprint vectors exist in a Euclidean space, failing to incorporate spatial relationships and the non-uniform distribution of real-world RSS noise. This results in poor generalization across heterogeneous mobile devices, where variations in hardware and signal processing distort RSS readings. Graph Neural Networks (GNNs) can improve upon conventional DL models by encoding indoor locations as nodes and modeling their spatial and signal relationships as edges. However, GNNs struggle with non-Euclidean noise distributions and suffer from the GNN blind spot problem, leading to degraded accuracy in environments with dense access points (APs). To address these challenges, we propose GATE, a novel framework that constructs an adaptive graph representation of fingerprint vectors while preserving an indoor state-space topology, modeling the non-Euclidean structure of RSS noise to mitigate environmental noise and address device heterogeneity. GATE introduces 1) a novel Attention Hyperspace Vector (AHV) for enhanced message passing, 2) a novel Multi-Dimensional Hyperspace Vector (MDHV) to mitigate the GNN blind spot, and 3) an new Real-Time Edge Construction (RTEC) approach for dynamic graph adaptation. Extensive real-world evaluations across multiple indoor spaces with varying path lengths, AP densities, and heterogeneous devices demonstrate that GATE achieves 1.6x to 4.72x lower mean localization errors and 1.85x to 4.57x lower worst-case errors compared to state-of-the-art indoor localization frameworks.
【3】GALDS: A Graph-Autoencoder-based Latent Dynamics Surrogate model to predict neurite material transport
标题：GALDS：一个基于图自动编码器的潜在动力学替代模型预测神经突物质运输
链接：https://arxiv.org/abs/2507.10871
摘要：神经元在其神经突网络中表现出复杂的几何结构，这些网络在信号传导和营养物质运输等过程中发挥着至关重要的作用。准确模拟网络中的物质传输对于理解这些生物现象至关重要，但由于涉及复杂的树状结构，因此带来了重大的计算挑战。传统的方法是时间密集型和资源需求，但神经元树的固有属性，它主要由管道与稳态抛物线速度分布和分叉，提供了计算优化的机会。为了解决这些挑战，我们提出了一个基于图形自动编码器的潜在动力学代理（GALDS）模型，该模型专门用于简化神经树中物质传输的模拟。GALDS采用图形自动编码器对网络的几何形状、速度场和浓度分布的潜在表示进行编码。然后将这些潜在空间表示组装成全局图，该全局图随后用于通过受神经常微分方程（Neural ODE）概念启发的训练图潜在空间系统动态模型来预测潜在空间中的系统动态。自动编码器的集成允许使用较小的图形神经网络模型，减少了训练数据的要求。此外，Neural ODE组件有效地缓解了递归神经网络中常见的错误累积问题。GALDS模型的有效性通过八个看不见的几何形状和四个异常传输的例子，其中我们的方法实现了3%的平均相对误差与最大相对误差<8%，并证明了10倍的速度相比，以前的代理模型方法的结果。
摘要：Neurons exhibit intricate geometries within their neurite networks, which play a crucial role in processes such as signaling and nutrient transport. Accurate simulation of material transport in the networks is essential for understanding these biological phenomena but poses significant computational challenges because of the complex tree-like structures involved. Traditional approaches are time-intensive and resource-demanding, yet the inherent properties of neuron trees, which consists primarily of pipes with steady-state parabolic velocity profiles and bifurcations, provide opportunities for computational optimization. To address these challenges, we propose a Graph-Autoencoder-based Latent Dynamics Surrogate (GALDS) model, which is specifically designed to streamline the simulation of material transport in neural trees. GALDS employs a graph autoencoder to encode latent representations of the network's geometry, velocity fields, and concentration profiles. These latent space representations are then assembled into a global graph, which is subsequently used to predict system dynamics in the latent space via a trained graph latent space system dynamic model, inspired by the Neural Ordinary Differential Equations (Neural ODEs) concept. The integration of an autoencoder allows for the use of smaller graph neural network models with reduced training data requirements. Furthermore, the Neural ODE component effectively mitigates the issue of error accumulation commonly encountered in recurrent neural networks. The effectiveness of the GALDS model is demonstrated through results on eight unseen geometries and four abnormal transport examples, where our approach achieves mean relative error of 3% with maximum relative error <8% and demonstrates a 10-fold speed improvement compared to previous surrogate model approaches.
【4】From Small to Large: A Graph Convolutional Network Approach for Solving Assortment Optimization Problems
标题：从小到大：解决组合优化问题的图卷积网络方法
链接：https://arxiv.org/abs/2507.10834
备注：Conference version. The journal version will be updated soon
摘要：产品组合优化涉及选择可替代产品的子集（受某些约束），以最大限度地提高预期收入。这是收入管理中的一个经典问题，在各个行业都有应用。然而，由于其组合性和非线性性质，该问题通常是NP-难的。在这项工作中，我们将探讨如何利用图卷积网络（GCN）来有效地解决混合多项logit选择模型下的约束分类优化问题。我们首先开发了一个图表示的分类问题，然后训练一个GCN学习模式的最佳组合，最后提出了两个推理策略的基础上GCN的输出。由于GCN具有在不同大小的输入之间进行泛化的固有能力，我们可以使用在小规模实例上训练的GCN来促进大规模实例。大量的数值实验表明，给定一个在小规模实例上训练的GCN（例如，20个产品），所提出的策略可以在几秒钟内在大规模实例（最多2，000个产品）上实现卓越的性能（90%+最优性），在性能和效率方面都优于现有的启发式策略。此外，我们将我们的框架扩展到无模型设置，其中底层选择模型是未知的，但交易数据可用。我们还进行了数值实验，以证明我们提出的政策在这种情况下的有效性和效率。
摘要：Assortment optimization involves selecting a subset of substitutable products (subject to certain constraints) to maximize the expected revenue. It is a classic problem in revenue management and finds applications across various industries. However, the problem is usually NP-hard due to its combinatorial and non-linear nature. In this work, we explore how graph concolutional networks (GCNs) can be leveraged to efficiently solve constrained assortment optimization under the mixed multinomial logit choice model. We first develop a graph representation of the assortment problem, then train a GCN to learn the patterns of optimal assortments, and lastly propose two inference policies based on the GCN's output. Due to the GCN's inherent ability to generalize across inputs of varying sizes, we can use a GCN trained on small-scale instances to facilitate large-scale instances. Extensive numerical experiments demonstrate that given a GCN trained on small-scale instances (e.g., with 20 products), the proposed policies can achieve superior performance (90%+ optimality) on large-scale instances (with up to 2,000 products) within seconds, which outperform existing heuristic policies in both performance and efficiency. Furthermore, we extend our framework to a model-free setting where the underlying choice model is unknown but transaction data is available. We also conduct numerical experiments to demonstrate the effectiveness and efficiency of our proposed policies in this setting.
【5】Learning to Quantize and Precode in Massive MIMO Systems for Energy Reduction: a Graph Neural Network Approach
标题：学习在大规模CDMA系统中量化和预编码以降低能量：图神经网络方法
链接：https://arxiv.org/abs/2507.10634
摘要：大规模MIMO系统正朝着增加射频链数量、更高载波频率和更大带宽的方向发展。因此，数模转换器（DAC）在硬件复杂性和功耗方面正成为瓶颈。在这项工作中，非线性预编码的粗量化下行大规模MIMO的研究。考虑到该问题的NP难性质，提出了一种图神经网络（GNN），其基于信道矩阵和预期的发送符号直接输出预编码的量化向量。该模型以自我监督的方式进行训练，直接最大化可实现的速率。为了克服由于不可微DAC函数而引入的目标函数的不可微性，提出了梯度的直通Gumbel-softmax估计。所提出的方法实现了一个显着的增加，可实现的和率下粗量化。例如，在单用户情况下，所提出的方法可以通过使用1比特DAC来实现与最大比传输（MRT）相同的和速率，而MRT为3比特。这使得基带DAC和RF DAC的功耗分别降低了4-7倍和3倍。然而，这是以增加的数字信号处理功耗为代价的。考虑到这一点，基带DAC的系统带宽最高可达3.5 MHz，而RF DAC在更高带宽下可保持2.9 MHz的功耗降低。值得注意的是，在本分析中没有考虑进一步降低功耗的间接影响，例如减少前传消耗和减少其他组件。
摘要：Massive MIMO systems are moving toward increased numbers of radio frequency chains, higher carrier frequencies and larger bandwidths. As such, digital-to-analog converters (DACs) are becoming a bottleneck in terms of hardware complexity and power consumption. In this work, non-linear precoding for coarsely quantized downlink massive MIMO is studied. Given the NP-hard nature of this problem, a graph neural network (GNN) is proposed that directly outputs the precoded quantized vector based on the channel matrix and the intended transmit symbols. The model is trained in a self-supervised manner, by directly maximizing the achievable rate. To overcome the non-differentiability of the objective function, introduced due to the non-differentiable DAC functions, a straight-through Gumbel-softmax estimation of the gradient is proposed. The proposed method achieves a significant increase in achievable sum rate under coarse quantization. For instance, in the single-user case, the proposed method can achieve the same sum rate as maximum ratio transmission (MRT) by using one-bit DAC's as compared to 3 bits for MRT. This reduces the DAC's power consumption by a factor 4-7 and 3 for baseband and RF DACs respectively. This, however, comes at the cost of increased digital signal processing power consumption. When accounting for this, the reduction in overall power consumption holds for a system bandwidth up to 3.5 MHz for baseband DACs, while the RF DACs can maintain a power reduction of 2.9 for higher bandwidths. Notably, indirect effects, which further reduce the power consumption, such as a reduced fronthaul consumption and reduction in other components, are not considered in this analysis.
【6】Player-Team Heterogeneous Interaction Graph Transformer for Soccer Outcome Prediction
标题：足球结果预测的球员-球队异类交互图Transformer
链接：https://arxiv.org/abs/2507.10626
摘要：预测足球比赛的结果是一项具有挑战性的任务，由于固有的不可预测的性质的游戏和众多的动态因素影响的结果。虽然它通常依赖于细致的特征工程，但深度学习技术最近在学习有效的球员和球队表示直接用于足球结果预测方面表现出了很大的希望。然而，现有的方法往往忽略了球员和球队之间的互动，这是准确建模比赛动态的关键异质性。为了解决这一差距，我们提出了HIGFormer（Heterogeneous Interaction Graph Transformer），这是一种用于足球结果预测的新型图形增强的基于transformer的深度学习模型。HIGFormer引入了一个多层次的交互框架，可以捕获细粒度的玩家动态和高层次的团队交互。具体地，它包括（1）球员交互网络，其通过异构交互图编码球员表现，将局部图卷积与全局图增强的Transformer相结合;（2）团队交互网络，其从团队到团队的角度构建交互图以建模历史比赛关系;以及（3）比赛比较Transformer，其联合分析球队和球员级别的信息以预测比赛结果。在大规模真实世界足球数据集WyScout Open Access Dataset上进行的大量实验表明，HIGFormer在预测精度方面明显优于现有方法。此外，我们提供了宝贵的见解，利用我们的模型进行球员表现评估，提供了一个新的视角，人才球探和团队战略分析。
摘要：Predicting soccer match outcomes is a challenging task due to the inherently unpredictable nature of the game and the numerous dynamic factors influencing results. While it conventionally relies on meticulous feature engineering, deep learning techniques have recently shown a great promise in learning effective player and team representations directly for soccer outcome prediction. However, existing methods often overlook the heterogeneous nature of interactions among players and teams, which is crucial for accurately modeling match dynamics. To address this gap, we propose HIGFormer (Heterogeneous Interaction Graph Transformer), a novel graph-augmented transformer-based deep learning model for soccer outcome prediction. HIGFormer introduces a multi-level interaction framework that captures both fine-grained player dynamics and high-level team interactions. Specifically, it comprises (1) a Player Interaction Network, which encodes player performance through heterogeneous interaction graphs, combining local graph convolutions with a global graph-augmented transformer; (2) a Team Interaction Network, which constructs interaction graphs from a team-to-team perspective to model historical match relationships; and (3) a Match Comparison Transformer, which jointly analyzes both team and player-level information to predict match outcomes. Extensive experiments on the WyScout Open Access Dataset, a large-scale real-world soccer dataset, demonstrate that HIGFormer significantly outperforms existing methods in prediction accuracy. Furthermore, we provide valuable insights into leveraging our model for player performance evaluation, offering a new perspective on talent scouting and team strategy analysis.
【7】Divide-Then-Rule: A Cluster-Driven Hierarchical Interpolator for Attribute-Missing Graphs
标题：Divide-Then-Rule：一个基于属性缺失图的分层插值器
链接：https://arxiv.org/abs/2507.10595
摘要：属性缺失图的深度图聚类（DGC）是一种无监督任务，旨在将具有不完整属性的节点划分为不同的聚类。解决这个具有挑战性的问题对于实际应用至关重要。然而，这一领域的研究仍然不够深入。现有的属性缺失图的插补方法往往无法考虑节点邻域中可用的信息量的变化，导致不可靠的结果，特别是对于已知邻域不足的节点。为了解决这个问题，我们提出了一种新的方法命名为Divide-Then-Rule Graph Completion（DTRGC）。该方法首先处理具有足够已知邻域信息的节点，并将估算结果视为新知识，以迭代地估算更具挑战性的节点，同时利用聚类信息来纠正估算错误。具体而言，动态特征感知传播（DCFP）通过基于聚类结构调整传播权重来消除缺失的节点属性。随后，分层邻域感知插补（HNAI）分类属性缺失的节点分为三组的基础上，他们的邻域属性的完整性。插补是分层执行的，优先考虑具有最多可用邻域信息的节点的组。然后使用聚类结构来细化插补并纠正潜在的错误。最后，Hop-wise Representation Enhancement（HRE）集成了多跳信息，从而丰富了节点表示的表达能力。在六个广泛使用的图数据集上的实验结果表明，DTRGC显著提高了各种DGC方法在属性缺失图下的聚类性能。
摘要 ：Deep graph clustering (DGC) for attribute-missing graphs is an unsupervised task aimed at partitioning nodes with incomplete attributes into distinct clusters. Addressing this challenging issue is vital for practical applications. However, research in this area remains underexplored. Existing imputation methods for attribute-missing graphs often fail to account for the varying amounts of information available across node neighborhoods, leading to unreliable results, especially for nodes with insufficient known neighborhood. To address this issue, we propose a novel method named Divide-Then-Rule Graph Completion (DTRGC). This method first addresses nodes with sufficient known neighborhood information and treats the imputed results as new knowledge to iteratively impute more challenging nodes, while leveraging clustering information to correct imputation errors. Specifically, Dynamic Cluster-Aware Feature Propagation (DCFP) initializes missing node attributes by adjusting propagation weights based on the clustering structure. Subsequently, Hierarchical Neighborhood-aware Imputation (HNAI) categorizes attribute-missing nodes into three groups based on the completeness of their neighborhood attributes. The imputation is performed hierarchically, prioritizing the groups with nodes that have the most available neighborhood information. The cluster structure is then used to refine the imputation and correct potential errors. Finally, Hop-wise Representation Enhancement (HRE) integrates information across multiple hops, thereby enriching the expressiveness of node representations. Experimental results on six widely used graph datasets show that DTRGC significantly improves the clustering performance of various DGC methods under attribute-missing graphs.
【8】HEIMDALL: a grapH-based sEIsMic Detector And Locator for microseismicity
标题：HEIMDALL：基于葡萄糖的sEIsMic探测器和用于微地震活动的绘图仪
链接：https://arxiv.org/abs/2507.10850
摘要：在这项工作中，我们提出了一种新的用于微震活动监测的深度学习模型，该模型利用地震台站记录之间的连续时空关系，形成了用于地震目录创建的端到端管道。它采用图论和最先进的图神经网络架构，在滚动窗口上同时执行相位拾取，关联和事件定位，使其适合回放和近实时监控。作为在向绿色能源过渡的更广泛背景下减少碳排放的全球战略的一部分，人们对开发强化地热系统的兴趣越来越大。在冰岛Hengill地区复杂的地热区进行测试，使用来自临时实验的开放访问数据，我们的模型使用手动修订和自动地震目录进行训练和验证。结果显示，与先前发布的自动系统和参考目录相比，事件检测显著增加，包括2018年12月的4 M_w$地震序列和2019年2月的单日序列。我们的方法减少了错误事件，最大限度地减少了手动监督，并减少了对管道的广泛调整或深度学习模型的迁移学习的需求。总的来说，它验证了一个强大的监测工具，地热地震区，补充现有系统，并加强在地热能源开采的操作风险缓解。
摘要：In this work, we present a new deep-learning model for microseismicity monitoring that utilizes continuous spatiotemporal relationships between seismic station recordings, forming an end-to-end pipeline for seismic catalog creation. It employs graph theory and state-of-the-art graph neural network architectures to perform phase picking, association, and event location simultaneously over rolling windows, making it suitable for both playback and near-real-time monitoring. As part of the global strategy to reduce carbon emissions within the broader context of a green-energy transition, there has been growing interest in exploiting enhanced geothermal systems. Tested in the complex geothermal area of Iceland's Hengill region using open-access data from a temporary experiment, our model was trained and validated using both manually revised and automatic seismic catalogs. Results showed a significant increase in event detection compared to previously published automatic systems and reference catalogs, including a $4 M_w$ seismic sequence in December 2018 and a single-day sequence in February 2019. Our method reduces false events, minimizes manual oversight, and decreases the need for extensive tuning of pipelines or transfer learning of deep-learning models. Overall, it validates a robust monitoring tool for geothermal seismic regions, complementing existing systems and enhancing operational risk mitigation during geothermal energy exploitation.
Transformer(2篇)
【1】Streaming 4D Visual Geometry Transformer
标题：流媒体4D视觉几何Transformer
链接：https://arxiv.org/abs/2507.11539
备注：Code is available at: this https URL
摘要：从视频中感知和重建4D时空几何是一项基本但具有挑战性的计算机视觉任务。为了促进交互式和实时应用程序，我们提出了一个流4D视觉几何Transformer，具有类似的哲学与自回归大语言模型。我们探索一个简单而有效的设计，并采用因果Transformer架构，以在线方式处理输入序列。我们使用时间因果注意和缓存的历史关键字和值作为隐式记忆，使有效的流长期4D重建。这种设计可以通过增量集成历史信息来处理实时4D重建，同时保持高质量的空间一致性。为了有效的训练，我们建议从密集的双向视觉几何接地Transformer（VGGT）提取知识到我们的因果模型。对于推理，我们的模型支持优化的有效注意力算子（例如，FlashAttention）从大型语言模型领域。在各种4D几何感知基准上进行的大量实验表明，我们的模型在保持竞争性能的同时提高了在线场景中的推理速度，为可扩展和交互式4D视觉系统铺平了道路。代码可从以下网址获得：https://github.com/wzzheng/StreamVGGT。
摘要：Perceiving and reconstructing 4D spatial-temporal geometry from videos is a fundamental yet challenging computer vision task. To facilitate interactive and real-time applications, we propose a streaming 4D visual geometry transformer that shares a similar philosophy with autoregressive large language models. We explore a simple and efficient design and employ a causal transformer architecture to process the input sequence in an online manner. We use temporal causal attention and cache the historical keys and values as implicit memory to enable efficient streaming long-term 4D reconstruction. This design can handle real-time 4D reconstruction by incrementally integrating historical information while maintaining high-quality spatial consistency. For efficient training, we propose to distill knowledge from the dense bidirectional visual geometry grounded transformer (VGGT) to our causal model. For inference, our model supports the migration of optimized efficient attention operator (e.g., FlashAttention) from the field of large language models. Extensive experiments on various 4D geometry perception benchmarks demonstrate that our model increases the inference speed in online scenarios while maintaining competitive performance, paving the way for scalable and interactive 4D vision systems. Code is available at: https://github.com/wzzheng/StreamVGGT.
【2】Universal Approximation Theorem for a Single-Layer Transformer
标题：单层Transformer的普适逼近定理
链接：https://arxiv.org/abs/2507.10581
备注：7 pages, 2 figures, 1 theorem, 10 formulas
摘要：深度学习采用通过反向传播算法训练的多层神经网络。这种方法在许多领域都取得了成功，并依赖于自适应梯度方法，如亚当优化器。序列建模从递归神经网络发展到基于注意力的模型，最终形成了Transformer架构。Transformers在自然语言处理（例如，BERT和GPT-3）方面取得了最先进的性能，并已应用于计算机视觉和计算生物学。然而，对这些模型的理论理解仍然有限。在本文中，我们研究了深度学习和Transformers的数学基础，并提出了一个新的理论结果。我们回顾了支撑深度学习的线性代数、概率和优化的关键概念，并详细分析了多头自注意机制和反向传播算法。我们的主要贡献是Transformers的通用近似定理：我们证明了一个单层Transformer，包括一个自注意层，后面是一个带有ReLU激活的位置前馈网络，可以将紧致域上的任何连续序列到序列映射近似到任意精度。我们提供了一个正式的声明和一个完整的证明。最后，我们提出的案例研究，证明了这一结果的实际意义。我们的研究结果推进了对Transformer模型的理论理解，并有助于弥合理论与实践之间的差距。
摘要：Deep learning employs multi-layer neural networks trained via the backpropagation algorithm. This approach has achieved success across many domains and relies on adaptive gradient methods such as the Adam optimizer. Sequence modeling evolved from recurrent neural networks to attention-based models, culminating in the Transformer architecture. Transformers have achieved state-of-the-art performance in natural language processing (for example, BERT and GPT-3) and have been applied in computer vision and computational biology. However, theoretical understanding of these models remains limited. In this paper, we examine the mathematical foundations of deep learning and Transformers and present a novel theoretical result. We review key concepts from linear algebra, probability, and optimization that underpin deep learning, and we analyze the multi-head self-attention mechanism and the backpropagation algorithm in detail. Our main contribution is a universal approximation theorem for Transformers: we prove that a single-layer Transformer, comprising one self-attention layer followed by a position-wise feed-forward network with ReLU activation, can approximate any continuous sequence-to-sequence mapping on a compact domain to arbitrary precision. We provide a formal statement and a complete proof. Finally, we present case studies that demonstrate the practical implications of this result. Our findings advance the theoretical understanding of Transformer models and help bridge the gap between theory and practice.
GAN|对抗|攻击|生成相关(5篇)
【1】Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking
标题：散列水印作为滤波器：在基于权值的神经网络水印中击败伪造和覆盖攻击
链接
：https://arxiv.org/abs/2507.11137
摘要：作为有价值的数字资产，深度神经网络需要鲁棒的所有权保护，将神经网络水印（NNW）定位为有前途的解决方案。在各种NNW方法中，基于权重的方法因其简单性和实用性而受到青睐;然而，它们仍然容易受到伪造和伪造攻击。为了解决这些挑战，我们提出了NeuralMark，一种围绕哈希水印过滤器构建的鲁棒方法。具体来说，我们利用哈希函数从密钥中生成不可逆的二进制水印，然后将其用作过滤器来选择嵌入的模型参数。这种设计巧妙地将嵌入参数与散列水印交织在一起，提供了对伪造和伪造攻击的鲁棒防御。平均池也被纳入抵抗微调和修剪攻击。此外，它可以无缝集成到各种神经网络架构中，确保广泛的适用性。从理论上分析了它的安全边界。从经验上讲，我们在13种不同的卷积和Transformer架构中验证了它的有效性和鲁棒性，涵盖5个图像分类任务和1个文本生成任务。源代码可在https://github.com/AIResearch-Group/NeuralMark上获得。
摘要：As valuable digital assets, deep neural networks necessitate robust ownership protection, positioning neural network watermarking (NNW) as a promising solution. Among various NNW approaches, weight-based methods are favored for their simplicity and practicality; however, they remain vulnerable to forging and overwriting attacks. To address those challenges, we propose NeuralMark, a robust method built around a hashed watermark filter. Specifically, we utilize a hash function to generate an irreversible binary watermark from a secret key, which is then used as a filter to select the model parameters for embedding. This design cleverly intertwines the embedding parameters with the hashed watermark, providing a robust defense against both forging and overwriting attacks. An average pooling is also incorporated to resist fine-tuning and pruning attacks. Furthermore, it can be seamlessly integrated into various neural network architectures, ensuring broad applicability. Theoretically, we analyze its security boundary. Empirically, we verify its effectiveness and robustness across 13 distinct Convolutional and Transformer architectures, covering five image classification tasks and one text generation task. The source codes are available at https://github.com/AIResearch-Group/NeuralMark.
【2】Crafting Imperceptible On-Manifold Adversarial Attacks for Tabular Data
标题：针对表格数据设计不可感知的Manifold对抗攻击
链接：https://arxiv.org/abs/2507.10998
备注：32 pages
摘要：由于混合分类和数值特征的异质性，对表格数据的对抗性攻击提出了与图像或文本领域不同的根本挑战。与像素扰动保持视觉相似性的图像不同，表格数据缺乏直观的相似性度量，因此难以定义无法察觉的修改。此外，传统的基于梯度的方法优先考虑$\ell_p$-norm约束，通常会产生偏离原始数据分布的对抗性示例，使其可检测。我们提出了一个潜在的空间扰动框架，使用混合输入变分自动编码器（VAE）生成不可感知的对抗性例子。拟议的VAE集成分类嵌入和数值特征到一个统一的潜在流形，使扰动保持统计一致性。我们指定分布内成功率（IDSR）来衡量在统计上与输入分布无法区分的对抗性示例的比例。对六个公开可用的数据集和三个模型架构的评估表明，与传统的输入空间攻击和其他基于图像域方法的VAE方法相比，我们的方法实现了更低的离群值率和更一致的性能。我们的全面分析包括超参数敏感性，稀疏控制机制和生成架构比较，揭示了基于VAE的攻击严重依赖于重建质量，但在有足够的训练数据时提供了卓越的实际效用。这项工作强调了流形上扰动对表格数据的现实对抗攻击的重要性，为实际部署提供了一种强大的方法。源代码可以通过https://github.com/ZhipengHe/VAE-TabAttack访问。
摘要：Adversarial attacks on tabular data present fundamental challenges distinct from image or text domains due to the heterogeneous nature of mixed categorical and numerical features. Unlike images where pixel perturbations maintain visual similarity, tabular data lacks intuitive similarity metrics, making it difficult to define imperceptible modifications. Additionally, traditional gradient-based methods prioritise $\ell_p$-norm constraints, often producing adversarial examples that deviate from the original data distributions, making them detectable. We propose a latent space perturbation framework using a mixed-input Variational Autoencoder (VAE) to generate imperceptible adversarial examples. The proposed VAE integrates categorical embeddings and numerical features into a unified latent manifold, enabling perturbations that preserve statistical consistency. We specify In-Distribution Success Rate (IDSR) to measure the proportion of adversarial examples that remain statistically indistinguishable from the input distribution. Evaluation across six publicly available datasets and three model architectures demonstrates that our method achieves substantially lower outlier rates and more consistent performance compared to traditional input-space attacks and other VAE-based methods adapted from image domain approaches. Our comprehensive analysis includes hyperparameter sensitivity, sparsity control mechanisms, and generative architectural comparisons, revealing that VAE-based attacks depend critically on reconstruction quality but offer superior practical utility when sufficient training data is available. This work highlights the importance of on-manifold perturbations for realistic adversarial attacks on tabular data, offering a robust approach for practical deployment. The source code can be accessed through https://github.com/ZhipengHe/VAE-TabAttack.
【3】How to Protect Models against Adversarial Unlearning?
标题：如何保护模型免受对抗性遗忘？
链接：https://arxiv.org/abs/2507.10886
摘要：人工智能模型需要取消学习，以满足AI法案或GDPR等法律行为的要求，也是因为需要删除有毒内容，消除偏见，恶意实例的影响，或模型工作的数据分布结构的变化。不幸的是，删除知识可能会导致不希望的副作用，如模型性能的恶化。在本文中，我们研究了对抗性遗忘的问题，其中恶意方故意发送遗忘请求以最大限度地降低模型的性能。我们发现，这种现象和对手的能力取决于许多因素，主要是骨干模型本身和策略/限制，在选择数据要忘却。这项工作的主要成果是一种新的方法，保护模型的性能免受这些副作用，无论是在自发过程和对手的行动所造成的未学习的行为的情况下。
摘要：AI models need to be unlearned to fulfill the requirements of legal acts such as the AI Act or GDPR, and also because of the need to remove toxic content, debiasing, the impact of malicious instances, or changes in the data distribution structure in which a model works. Unfortunately, removing knowledge may cause undesirable side effects, such as a deterioration in model performance. In this paper, we investigate the problem of adversarial unlearning, where a malicious party intentionally sends unlearn requests to deteriorate the model's performance maximally. We show that this phenomenon and the adversary's capabilities depend on many factors, primarily on the backbone model itself and strategy/limitations in selecting data to be unlearned. The main result of this work is a new method of protecting model performance from these side effects, both in the case of unlearned behavior resulting from spontaneous processes and adversary actions.
【4】Distributionally Robust Optimization with Adversarial Data Contamination
标题：具有对抗数据污染的分布式鲁棒优化
链接：https://arxiv.org/abs/2507.10718
摘要：分布鲁棒优化（DRO）为分布不确定性下的决策提供了一个框架，但其有效性可能会受到训练数据中离群值的影响。本文介绍了一个原则性的方法，同时解决这两个挑战。我们专注于优化Wasserstein-1 DRO目标的广义线性模型与凸Lipschitz损失函数，其中的一个$\N $-分数的训练数据是adversarially损坏。我们的主要贡献在于一个新的建模框架，该框架集成了对训练数据污染的鲁棒性和对分布变化的鲁棒性，以及一个受鲁棒统计启发的高效算法来解决由此产生的优化问题。我们证明，我们的方法实现了$O（\sqrt{\displaystyle {\sqrt}}）$的估计误差为真正的DRO目标值仅使用受污染的数据下有界协方差假设。这项工作建立了第一个严格的保证，有效的计算支持下，数据污染和分布变化的双重挑战下的学习。
摘要 ：Distributionally Robust Optimization (DRO) provides a framework for decision-making under distributional uncertainty, yet its effectiveness can be compromised by outliers in the training data. This paper introduces a principled approach to simultaneously address both challenges. We focus on optimizing Wasserstein-1 DRO objectives for generalized linear models with convex Lipschitz loss functions, where an $\epsilon$-fraction of the training data is adversarially corrupted. Our primary contribution lies in a novel modeling framework that integrates robustness against training data contamination with robustness against distributional shifts, alongside an efficient algorithm inspired by robust statistics to solve the resulting optimization problem. We prove that our method achieves an estimation error of $O(\sqrt{\epsilon})$ for the true DRO objective value using only the contaminated data under the bounded covariance assumption. This work establishes the first rigorous guarantees, supported by efficient computation, for learning under the dual challenges of data contamination and distributional shifts.
【5】DALI-PD: Diffusion-based Synthetic Layout Heatmap Generation for ML in Physical Design
标题：DALI-PD：物理设计中的ML基于扩散的合成布局热图生成
链接：https://arxiv.org/abs/2507.10606
备注：Under review at Asia and South Pacific Design Automation Conference (ASP-DAC'26)
摘要：机器学习（ML）在各种物理设计（PD）任务中表现出了巨大的潜力。然而，模型的泛化能力仍然受到高质量、大规模训练数据集可用性的限制。创建这样的数据集通常在计算上是昂贵的，并且受到IP的限制。虽然很少有公共数据集可用，但它们通常是静态的，生成缓慢，需要频繁更新。为了解决这些限制，我们提出了DALI-PD，这是一个可扩展的框架，用于生成合成布局热图，以加速PD研究中的ML。DALI-PD使用扩散模型，通过快速推理在几秒钟内生成不同的布局热图。热图包括功率、IR降、拥塞、宏放置和单元密度图。使用DALI-PD，我们创建了一个包含20，000多个布局配置的数据集，这些布局配置具有不同的宏计数和位置。这些热图非常类似于真实的布局，并提高了下游ML任务（如IR下降或拥塞预测）的ML准确性。
摘要：Machine learning (ML) has demonstrated significant promise in various physical design (PD) tasks. However, model generalizability remains limited by the availability of high-quality, large-scale training datasets. Creating such datasets is often computationally expensive and constrained by IP. While very few public datasets are available, they are typically static, slow to generate, and require frequent updates. To address these limitations, we present DALI-PD, a scalable framework for generating synthetic layout heatmaps to accelerate ML in PD research. DALI-PD uses a diffusion model to generate diverse layout heatmaps via fast inference in seconds. The heatmaps include power, IR drop, congestion, macro placement, and cell density maps. Using DALI-PD, we created a dataset comprising over 20,000 layout configurations with varying macro counts and placements. These heatmaps closely resemble real layouts and improve ML accuracy on downstream ML tasks such as IR drop or congestion prediction.
半/弱/无/有监督|不确定性|主动学习(2篇)
【1】Scalable Unsupervised Segmentation via Random Fourier Feature-based Gaussian Process
标题：通过基于随机傅里叶高斯过程的可扩展无监督分割
链接：https://arxiv.org/abs/2507.10632
摘要：在本文中，我们提出了RFF-GP-HSMM，一个快速的无监督的时间序列分割方法，采用随机傅立叶特征（RFF），以解决高斯过程隐半马尔可夫模型（GP-HSMM）的高计算成本。GP-HSMM使用高斯过程对时间序列数据进行建模，需要在训练期间对N乘N的核矩阵进行求逆，其中N是数据点的数量。随着数据规模的增加，矩阵求逆会产生显著的计算成本。为了解决这个问题，所提出的方法近似高斯过程与线性回归使用RFF，保留表达能力，同时消除了需要求逆的核矩阵。卡内基梅隆大学（CMU）的运动捕捉数据集上的实验表明，所提出的方法实现的分割性能与传统方法相比，约278倍的时间序列数据，包括39,200帧的分割速度。
摘要：In this paper, we propose RFF-GP-HSMM, a fast unsupervised time-series segmentation method that incorporates random Fourier features (RFF) to address the high computational cost of the Gaussian process hidden semi-Markov model (GP-HSMM). GP-HSMM models time-series data using Gaussian processes, requiring inversion of an N times N kernel matrix during training, where N is the number of data points. As the scale of the data increases, matrix inversion incurs a significant computational cost. To address this, the proposed method approximates the Gaussian process with linear regression using RFF, preserving expressive power while eliminating the need for inversion of the kernel matrix. Experiments on the Carnegie Mellon University (CMU) motion-capture dataset demonstrate that the proposed method achieves segmentation performance comparable to that of conventional methods, with approximately 278 times faster segmentation on time-series data comprising 39,200 frames.
【2】Joint space-time wind field data extrapolation and uncertainty quantification using nonparametric Bayesian dictionary learning
标题：使用非参数Bayesian字典学习联合时空风场数据外推和不确定性量化
链接：https://arxiv.org/abs/2507.11385
摘要：基于非参数贝叶斯字典学习，提出了一种基于有限/不完全测量数据的联合时空风场数据外推和相关统计量估计方法。具体地说，利用稀疏/不完整的测量数据，一个时间相关的优化问题制定的随机风场的相关的低维表示的展开系数。相比替代的，标准的，压缩抽样处理的问题，开发的方法具有以下优点。首先，贝叶斯公式也可以量化估计中的不确定性。其次，在标准的CS为基础的应用程序的先验选择的扩展基础的要求是规避。相反，这在本文中基于所获取的数据以自适应方式进行。总体而言，该方法显示出增强的外推准确性，即使在任意形式的高维数据和相对较大的外推距离的情况下。因此，它可以被使用，潜在地，在广泛的风力工程应用中，各种约束规定使用有限数量的传感器。该方法的有效性证明了考虑两个案例研究。第一个涉及的外推模拟风速记录符合规定的联合波数频率功率谱密度在三维域（二维和时间）。第二个涉及四维（三维和时间）边界层风洞实验数据，表现出显着的空间变异性和非高斯特性的外推。
摘要：A methodology is developed, based on nonparametric Bayesian dictionary learning, for joint space-time wind field data extrapolation and estimation of related statistics by relying on limited/incomplete measurements. Specifically, utilizing sparse/incomplete measured data, a time-dependent optimization problem is formulated for determining the expansion coefficients of an associated low-dimensional representation of the stochastic wind field. Compared to an alternative, standard, compressive sampling treatment of the problem, the developed methodology exhibits the following advantages. First, the Bayesian formulation enables also the quantification of the uncertainty in the estimates. Second, the requirement in standard CS-based applications for an a priori selection of the expansion basis is circumvented. Instead, this is done herein in an adaptive manner based on the acquired data. Overall, the methodology exhibits enhanced extrapolation accuracy, even in cases of high-dimensional data of arbitrary form, and of relatively large extrapolation distances. Thus, it can be used, potentially, in a wide range of wind engineering applications where various constraints dictate the use of a limited number of sensors. The efficacy of the methodology is demonstrated by considering two case studies. The first relates to the extrapolation of simulated wind velocity records consistent with a prescribed joint wavenumber-frequency power spectral density in a three-dimensional domain (2D and time). The second pertains to the extrapolation of four-dimensional (3D and time) boundary layer wind tunnel experimental data that exhibit significant spatial variability and non-Gaussian characteristics.
迁移|Zero/Few/One-Shot|自适应(7篇)
【1】A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition
标题：一种鲁棒的不完全多模式低等级自适应情绪识别方法
链接：https://arxiv.org/abs/2507.11202
摘要：在实际应用中，由于传感器故障或隐私保护要求，多模态情感识别（MER）经常会遇到不完全多模态。虽然现有方法试图通过额外的梯度来平衡每个模态组合的训练来解决各种不完整的多模态场景，但这些方法面临着一个关键的限制：来自不同模态组合的训练梯度相互冲突，最终降低了最终预测模型的性能。在本文中，我们提出了一种基于模态组合的单峰解耦动态低秩自适应方法，称为MCULoRA，这是一种新的框架，用于不完整的多模态学习模型的参数有效训练。MCULoRA包括两个关键模块，模态组合感知低秩自适应（MCLA）和动态参数微调（DPFT）。MCLA模块有效地从个体模态组合的不同特征中提取共享信息。DPFT模块基于每个模态的表示空间的可分性来调整模态组合的训练比率，从而优化跨不同模态组合的学习效率。我们在多个基准数据集上进行的广泛实验评估表明，MCULoRA在下游任务准确性方面大大优于以前的不完整多模态学习方法。
摘要：Multimodal Emotion Recognition (MER) often encounters incomplete multimodality in practical applications due to sensor failures or privacy protection requirements. While existing methods attempt to address various incomplete multimodal scenarios by balancing the training of each modality combination through additional gradients, these approaches face a critical limitation: training gradients from different modality combinations conflict with each other, ultimately degrading the performance of the final prediction model. In this paper, we propose a unimodal decoupled dynamic low-rank adaptation method based on modality combinations, named MCULoRA, which is a novel framework for the parameter-efficient training of incomplete multimodal learning models. MCULoRA consists of two key modules, modality combination aware low-rank adaptation (MCLA) and dynamic parameter fine-tuning (DPFT). The MCLA module effectively decouples the shared information from the distinct characteristics of individual modality combinations. The DPFT module adjusts the training ratio of modality combinations based on the separability of each modality's representation space, optimizing the learning efficiency across different modality combinations. Our extensive experimental evaluation in multiple benchmark datasets demonstrates that MCULoRA substantially outperforms previous incomplete multimodal learning approaches in downstream task accuracy.
【2】AdaMuon: Adaptive Muon Optimizer
标题：AdaMuon：自适应μ子优化器
链接：https://arxiv.org/abs/2507.11005
摘要：我们提出了AdaMuon，这是一个基于最近验证的Muon优化器的自适应学习率框架，该框架在大规模模型训练中比AdamW具有显著的效率提升。AdaMuon通过两个相互依赖的模块来增强Muon：（1）每个参数的二阶矩调制，它捕获正交梯度更新以确保更新级别的自适应性，以及（2）RMS对齐的重新缩放，它通过将其与参数空间的内在结构对齐来调节整体更新幅度。多个模型尺度和学习率机制的实证结果证实，AdaMuon始终优于原始Muon，在保持训练稳定性的同时提供更高的收敛加速。我们的方法不引入额外的调整负担，可以无缝集成到现有的μ子训练管道。
摘要：We propose AdaMuon, an adaptive learning-rate framework built upon the recently validated Muon optimizer, which has demonstrated substantial efficiency gains over AdamW in large-scale model training. AdaMuon augments Muon with two mutually dependent modules: (1) a per-parameter second-moment modulation that captures orthogonal gradient updates to ensure update-level adaptivity, and (2) a RMS-aligned rescaling that regulates the overall update magnitude by aligning it with the intrinsic structure of the parameter space. Empirical results on multiple model scales and learning-rate regimes confirm that AdaMuon consistently outperforms the original Muon, delivering higher acceleration in convergence while maintaining training stability. Our method introduces no additional tuning burden and can be seamlessly integrated into existing Muon training pipelines.
【3】High-Throughput Distributed Reinforcement Learning via Adaptive Policy Synchronization
标题：基于自适应策略同步的高吞吐量分布式强化学习
链接：https://arxiv.org/abs/2507.10990
摘要：扩展强化学习（RL）工作负载通常需要跨计算集群分布环境模拟。现有的框架将模拟、学习逻辑和编排纠缠在单一的系统中，限制了模块化和可重用性。我们提出了一个轻量级的，与学习者无关的分布式环境执行接口，它反映了Gymnasium API。ClusterEnv引入了DETACH模式，该模式通过将reset（）和step（）操作卸载给远程工作人员，同时保持学习集中化，将模拟与培训分离开来。为了解决分布式执行中的策略陈旧问题，我们提出了自适应执行者策略同步（AAPS），这是一种发散触发的更新机制，可以在不牺牲性能的情况下减少同步开销。ReplaterEnv干净地集成到现有的RL管道中，支持on-policy和off-policy方法，并且需要最少的代码更改。离散控制任务的实验表明，AAPS实现高采样效率与显着更少的权重更新。源代码可在https://github.com/rodlaf/ClusterEnv上获得。
摘要：Scaling reinforcement learning (RL) workloads often requires distributing environment simulation across compute clusters. Existing frameworks entangle simulation, learning logic, and orchestration into monolithic systems, limiting modularity and reusability. We present ClusterEnv, a lightweight, learner-agnostic interface for distributed environment execution that mirrors the Gymnasium API. ClusterEnv introduces the DETACH pattern, which decouples simulation from training by offloading reset() and step() operations to remote workers while keeping learning centralized. To address policy staleness in distributed execution, we propose Adaptive Actor Policy Synchronization (AAPS), a divergence-triggered update mechanism that reduces synchronization overhead without sacrificing performance. ClusterEnv integrates cleanly into existing RL pipelines, supports both on-policy and off-policy methods, and requires minimal code changes. Experiments on discrete control tasks demonstrate that AAPS achieves high sample efficiency with significantly fewer weight updates. Source code is available at https://github.com/rodlaf/ClusterEnv.
【4】FedGSCA: Medical Federated Learning with Global Sample Selector and Client Adaptive Adjuster under Label Noise
标题：FedGSCA：标签噪音下具有全球样本分配器和客户端自适应调整器的医疗联合学习
链接：https://arxiv.org/abs/2507.10611
摘要：联邦学习（FL）是一种协作医学图像分类的解决方案，同时保护数据隐私。然而，来自机构间数据变化的标签噪声会导致训练不稳定并降低模型性能。现有的FL方法与噪声异质性和医疗数据的不平衡作斗争。出于这些挑战，我们提出了FedGSCA，一种新的框架，用于提高噪声医疗FL的鲁棒性。FedGSCA引入了一个全局样本库，它聚合了来自所有客户端的噪声知识，有效地解决了噪声异质性，提高了全局模型的稳定性。此外，我们开发了一个客户端自适应调整（CAA）机制，结合自适应阈值伪标签生成和鲁棒的信用标签丢失。CAA动态地调整类分布，确保包含少数样本，并通过考虑多个合理的标签来仔细管理噪声标签。这种双重方法减轻了噪声数据的影响，并防止局部训练期间的过拟合，从而提高了模型的泛化能力。我们评估FedGSCA在一个真实世界的结肠载玻片数据集和两个合成的医疗数据集在各种噪声条件下，包括对称，不对称，极端和异构类型。结果表明，FedGSCA优于国家的最先进的方法，在极端和异构噪声的情况下表现出色。此外，FedGSCA在提高模型稳定性和处理复杂噪声方面表现出显著的优势，使其非常适合现实世界的医疗联邦学习场景。
摘要 ：Federated Learning (FL) emerged as a solution for collaborative medical image classification while preserving data privacy. However, label noise, which arises from inter-institutional data variability, can cause training instability and degrade model performance. Existing FL methods struggle with noise heterogeneity and the imbalance in medical data. Motivated by these challenges, we propose FedGSCA, a novel framework for enhancing robustness in noisy medical FL. FedGSCA introduces a Global Sample Selector that aggregates noise knowledge from all clients, effectively addressing noise heterogeneity and improving global model stability. Furthermore, we develop a Client Adaptive Adjustment (CAA) mechanism that combines adaptive threshold pseudo-label generation and Robust Credal Labeling Loss. CAA dynamically adjusts to class distributions, ensuring the inclusion of minority samples and carefully managing noisy labels by considering multiple plausible labels. This dual approach mitigates the impact of noisy data and prevents overfitting during local training, which improves the generalizability of the model. We evaluate FedGSCA on one real-world colon slides dataset and two synthetic medical datasets under various noise conditions, including symmetric, asymmetric, extreme, and heterogeneous types. The results show that FedGSCA outperforms the state-of-the-art methods, excelling in extreme and heterogeneous noise scenarios. Moreover, FedGSCA demonstrates significant advantages in improving model stability and handling complex noise, making it well-suited for real-world medical federated learning scenarios.
【5】An Adaptive Volatility-based Learning Rate Scheduler
标题：基于波动性的自适应学习率分配器
链接：https://arxiv.org/abs/2507.10575
摘要：有效的学习率（LR）调度对于训练深度神经网络至关重要。然而，流行的预定义的和自适应的泛化器仍然会导致次优的泛化。本文介绍了VolSched，一种新的自适应LR调度器的灵感来自随机过程中的波动性的概念，如几何布朗运动，动态调整学习率。通过计算长期和短期准确性波动率之间的比率，VolSched增加LR以逃避平台期，并降低LR以稳定训练，使模型能够更有效地探索损失情况。我们使用标准的增强管道对CIFAR-100数据集进行了VolSched评估。当与ResNet-18和ResNet-34搭配使用时，我们的调度程序可提供一致的性能增益，分别将前1名的准确性提高1.4和1.3个百分点。对损失曲线的分析表明，VolSched促进了更长的勘探阶段。对Hessian的定量分析表明，VolSched找到的最终解决方案比下一个最佳基线平坦38%，使模型能够获得更宽的最小值，从而获得更好的泛化性能。
摘要：Effective learning rate (LR) scheduling is crucial for training deep neural networks. However, popular pre-defined and adaptive schedulers can still lead to suboptimal generalization. This paper introduces VolSched, a novel adaptive LR scheduler inspired by the concept of volatility in stochastic processes like Geometric Brownian Motion to dynamically adjust the learning rate. By calculating the ratio between long-term and short-term accuracy volatility, VolSched increases the LR to escape plateaus and decreases it to stabilize training, allowing the model to explore the loss landscape more effectively. We evaluate VolSched on the CIFAR-100 dataset against a strong baseline using a standard augmentation pipeline. When paired with ResNet-18 and ResNet-34, our scheduler delivers consistent performance gains, improving top-1 accuracy by 1.4 and 1.3 percentage points respectively. Analysis of the loss curves reveals that VolSched promotes a longer exploration phase. A quantitative analysis of the Hessian shows that VolSched finds a final solution that is 38% flatter than the next-best baseline, allowing the model to obtain wider minima and hence better generalization performance.
【6】Enhancing Cross Entropy with a Linearly Adaptive Loss Function for Optimized Classification Performance
标题：利用线性自适应损失函数增强交叉信息以优化分类性能
链接：https://arxiv.org/abs/2507.10574
备注：13 pages, 2 figures
摘要：我们提出了线性自适应交叉熵损失函数。这是一种从信息论中衍生出来的新测度。与标准交叉熵损失函数相比，所提出的交叉熵损失函数具有取决于真实类的预测概率的额外项。此功能用于增强涉及独热编码类标签的分类任务中的优化过程。建议的一个已经评估了基于ResNet的模型使用CIFAR-100数据集。初步结果表明，所提出的一致优于标准的交叉熵损失函数的分类精度。此外，所提出的一个保持简单，实际上实现了相同的效率，传统的交叉熵损失。这些发现表明，我们的方法可以扩大范围，为未来的研究损失函数设计。
摘要：We propose the Linearly Adaptive Cross Entropy Loss function. This is a novel measure derived from the information theory. In comparison to the standard cross entropy loss function, the proposed one has an additional term that depends on the predicted probability of the true class. This feature serves to enhance the optimization process in classification tasks involving one-hot encoded class labels. The proposed one has been evaluated on a ResNet-based model using the CIFAR-100 dataset. Preliminary results show that the proposed one consistently outperforms the standard cross entropy loss function in terms of classification accuracy. Moreover, the proposed one maintains simplicity, achieving practically the same efficiency to the traditional cross entropy loss. These findings suggest that our approach could broaden the scope for future research into loss function design.
【7】Real-time, Adaptive Radiological Anomaly Detection and Isotope Identification Using Non-negative Matrix Factorization
标题：使用非负矩阵分解的实时、自适应放射异常检测和同位素识别
链接：https://arxiv.org/abs/2507.10715
备注：11 pages, 8 figures
摘要：光谱异常检测和同位素识别算法是核不扩散应用（如搜索操作）的组成部分。在移动探测器系统的情况下，这项任务特别具有挑战性，因为观察到的伽马射线背景比静态探测器系统变化更多，并且预先训练的背景模型可以很容易地发现自己不在域中。其结果是，算法可能会超过其预期的误报率，或牺牲检测灵敏度，以保持所需的误报率。非负矩阵分解（NMF）已被证明是用于光谱异常检测和识别的强大工具，但是，像依赖于数据驱动的背景模型的许多类似算法一样，在其常规实现中，它不能实时更新以考虑影响背景光谱特征的环境变化。我们已经开发了一种新的基于NMF的算法，定期更新其背景模型，以适应不断变化的环境条件。自适应NMF算法涉及对其环境的更少假设，使其比现有的基于NMF的方法更具通用性，同时保持或超过模拟和真实世界数据集的检测性能。
摘要：Spectroscopic anomaly detection and isotope identification algorithms are integral components in nuclear nonproliferation applications such as search operations. The task is especially challenging in the case of mobile detector systems due to the fact that the observed gamma-ray background changes more than for a static detector system, and a pretrained background model can easily find itself out of domain. The result is that algorithms may exceed their intended false alarm rate, or sacrifice detection sensitivity in order to maintain the desired false alarm rate. Non-negative matrix factorization (NMF) has been shown to be a powerful tool for spectral anomaly detection and identification, but, like many similar algorithms that rely on data-driven background models, in its conventional implementation it is unable to update in real time to account for environmental changes that affect the background spectroscopic signature. We have developed a novel NMF-based algorithm that periodically updates its background model to accommodate changing environmental conditions. The Adaptive NMF algorithm involves fewer assumptions about its environment, making it more generalizable than existing NMF-based methods while maintaining or exceeding detection performance on simulated and real-world datasets.
强化学习(5篇)
【1】Local Pairwise Distance Matching for Backpropagation-Free Reinforcement Learning
标题：用于无反向传播强化学习的局部成对距离匹配
链接：https://arxiv.org/abs/2507.11367
备注：accepted at the European Conference on Artificial Intelligence (ECAI 2025)
摘要 ：使用强化学习（RL）训练神经网络通常依赖于反向传播（BP），需要存储来自前向传递的激活以用于后续的反向更新。此外，通过多层反向传播误差信号通常会导致梯度消失或爆炸，从而降低学习性能和稳定性。我们提出了一种新的方法，在RL设置中的前向传递过程中使用本地信号训练神经网络的每一层。我们的方法引入了局部的、逐层的损失，利用了从多维缩放中匹配成对距离的原理，并通过可选的奖励驱动的指导进行了增强。这种方法允许使用在前向传播期间计算的本地信号来训练每个隐藏层，从而消除了对反向传递的需要并存储中间激活。我们的实验，在常见的RL基准与政策梯度方法进行，证明了这种反向传播免费的方法相比，他们的经典的基于BP的同行实现了竞争力的性能。此外，所提出的方法增强了运行内和运行间的稳定性和一致性，并提高了性能，特别是在具有挑战性的环境中。
摘要：Training neural networks with reinforcement learning (RL) typically relies on backpropagation (BP), necessitating storage of activations from the forward pass for subsequent backward updates. Furthermore, backpropagating error signals through multiple layers often leads to vanishing or exploding gradients, which can degrade learning performance and stability. We propose a novel approach that trains each layer of the neural network using local signals during the forward pass in RL settings. Our approach introduces local, layer-wise losses leveraging the principle of matching pairwise distances from multi-dimensional scaling, enhanced with optional reward-driven guidance. This method allows each hidden layer to be trained using local signals computed during forward propagation, thus eliminating the need for backward passes and storing intermediate activations. Our experiments, conducted with policy gradient methods across common RL benchmarks, demonstrate that this backpropagation-free method achieves competitive performance compared to their classical BP-based counterpart. Additionally, the proposed method enhances stability and consistency within and across runs, and improves performance especially in challenging environments.
【2】Real-Time Bayesian Detection of Drift-Evasive GNSS Spoofing in Reinforcement Learning Based UAV Deconfliction
标题：基于强化学习的无人机去冲突中漂移规避GPS欺骗的实时Bayesian检测
链接：https://arxiv.org/abs/2507.11173
摘要：自主无人机（UAV）依赖于全球导航卫星系统（GNSS）伪距测量来进行精确的实时定位和导航。然而，这种依赖性使它们暴露于复杂的欺骗威胁，其中对手操纵伪距来欺骗UAV接收器。其中，漂移规避欺骗攻击巧妙地干扰测量，逐渐转移无人机的轨迹，而不会触发传统的信号级反欺骗机制。传统的分布式移位检测技术通常需要累积阈值数量的样本，从而导致延迟，这阻碍了快速检测和及时响应。因此，强大的时间尺度检测方法对于识别攻击发作并使用替代传感模式进行应急计划至关重要，从而提高对隐形对抗操纵的弹性。本研究探讨了贝叶斯在线变点检测（BOCPD）方法，该方法可以监控强化学习（RL）评论家网络的值估计值的时间变化，以检测无人机导航中的细微行为偏差。实验结果表明，该框架优于传统的GNSS欺骗检测器、时间半监督学习框架和Page-Hinkley测试，实现了更高的检测精度和更低的漂移规避欺骗攻击误报率和漏报率。
摘要：Autonomous unmanned aerial vehicles (UAVs) rely on global navigation satellite system (GNSS) pseudorange measurements for accurate real-time localization and navigation. However, this dependence exposes them to sophisticated spoofing threats, where adversaries manipulate pseudoranges to deceive UAV receivers. Among these, drift-evasive spoofing attacks subtly perturb measurements, gradually diverting the UAVs trajectory without triggering conventional signal-level anti-spoofing mechanisms. Traditional distributional shift detection techniques often require accumulating a threshold number of samples, causing delays that impede rapid detection and timely response. Consequently, robust temporal-scale detection methods are essential to identify attack onset and enable contingency planning with alternative sensing modalities, improving resilience against stealthy adversarial manipulations. This study explores a Bayesian online change point detection (BOCPD) approach that monitors temporal shifts in value estimates from a reinforcement learning (RL) critic network to detect subtle behavioural deviations in UAV navigation. Experimental results show that this temporal value-based framework outperforms conventional GNSS spoofing detectors, temporal semi-supervised learning frameworks, and the Page-Hinkley test, achieving higher detection accuracy and lower false-positive and false-negative rates for drift-evasive spoofing attacks.
【3】Offline Reinforcement Learning with Wasserstein Regularization via Optimal Transport Maps
标题：通过最佳传输地图使用Wasserstein正规化的离线强化学习
链接：https://arxiv.org/abs/2507.10843
备注：Accepted at RLC 2025
摘要：离线强化学习（RL）旨在从静态数据集中学习最佳策略，这使得它在数据收集成本高昂的场景中特别有价值，例如机器人。离线RL的一个主要挑战是分布偏移，其中学习的策略偏离数据集分布，可能导致不可靠的分布外操作。为了缓解这个问题，已经采用了正则化技术。虽然许多现有的方法利用密度比为基础的措施，如$f$发散，正则化，我们提出了一种方法，利用Wasserstein距离，这是强大的分布数据，并捕捉动作之间的相似性。我们的方法采用输入凸神经网络（ICNN）来模拟最佳传输映射，从而以无鉴别器的方式计算Wasserstein距离，从而避免对抗性训练并确保稳定的学习。我们的方法在D4 RL基准数据集上表现出与广泛使用的现有方法相当或更高的性能。该代码可在https://github.com/motokiomura/Q-DOT上获得。
摘要：Offline reinforcement learning (RL) aims to learn an optimal policy from a static dataset, making it particularly valuable in scenarios where data collection is costly, such as robotics. A major challenge in offline RL is distributional shift, where the learned policy deviates from the dataset distribution, potentially leading to unreliable out-of-distribution actions. To mitigate this issue, regularization techniques have been employed. While many existing methods utilize density ratio-based measures, such as the $f$-divergence, for regularization, we propose an approach that utilizes the Wasserstein distance, which is robust to out-of-distribution data and captures the similarity between actions. Our method employs input-convex neural networks (ICNNs) to model optimal transport maps, enabling the computation of the Wasserstein distance in a discriminator-free manner, thereby avoiding adversarial training and ensuring stable learning. Our approach demonstrates comparable or superior performance to widely used existing methods on the D4RL benchmark dataset. The code is available at https://github.com/motokiomura/Q-DOT .
【4】Ground-Compose-Reinforce: Tasking Reinforcement Learning Agents through Formal Language
标题：Ground-Compose-Reinforce：通过形式语言分配强化学习代理
链接：https://arxiv.org/abs/2507.10741
摘要：在复杂的感知（例如像素）和动作中建立语言是构建可以通过语言与人类交互的位置代理时的一个关键挑战。在过去的作品中，这通常是通过手动设计语言基础或通过管理将语言与环境元素相关联的大量数据集来解决的。我们提出了Ground-Compose-Reinforce，这是一个神经符号框架，用于从数据中建立形式语言，并通过这种语言直接向RL代理分配任务来引发行为。凭借数据驱动的学习，我们的框架避免了手动设计特定领域的元素，如奖励函数或符号检测器。凭借组合形式语言语义，我们的框架实现了数据高效的接地和推广到任意语言组合。基于图像的gridworld和MuJoCo机器人领域的实验表明，我们的方法可靠地将正式语言指令映射到具有有限数据的行为，而端到端的数据驱动方法失败。
摘要：Grounding language in complex perception (e.g. pixels) and action is a key challenge when building situated agents that can interact with humans via language. In past works, this is often solved via manual design of the language grounding or by curating massive datasets relating language to elements of the environment. We propose Ground-Compose-Reinforce, a neurosymbolic framework for grounding formal language from data, and eliciting behaviours by directly tasking RL agents through this language. By virtue of data-driven learning, our framework avoids the manual design of domain-specific elements like reward functions or symbol detectors. By virtue of compositional formal language semantics, our framework achieves data-efficient grounding and generalization to arbitrary language compositions. Experiments on an image-based gridworld and a MuJoCo robotics domain show that our approach reliably maps formal language instructions to behaviours with limited data while end-to-end, data-driven approaches fail.
【5】Meta-Reinforcement Learning for Fast and Data-Efficient Spectrum Allocation in Dynamic Wireless Networks
标题：元强化学习在动态无线网络中实现快速且数据高效的频谱分配
链接：https://arxiv.org/abs/2507.10619
备注：5 pages, 6 figures, under review at IEEE Wireless Communications Letters
摘要：5G /6 G网络中频谱的动态分配对于有效的资源利用至关重要。然而，由于其巨大的样本复杂性和与无指导探索相关的安全风险，应用传统的深度强化学习（DRL）通常是不可行的，这可能会导致严重的网络干扰。为了解决这些挑战，我们提出了一个元学习框架，使代理学习一个强大的初始政策，并迅速适应新的无线场景，最少的数据。我们实现了三种元学习架构，模型不可知元学习（MAML），递归神经网络（RNN）和注意力增强的RNN，并在模拟的动态集成接入/回程（IAB）环境中对非元学习DRL算法，近端策略优化（PPO）基线进行了评估。我们的结果显示了明显的性能差距。基于注意力的元学习代理达到了48 Mbps的峰值平均网络吞吐量，而PPO基线急剧下降到10 Mbps。此外，与PPO相比，我们的方法将SINR和延迟违规减少了50%以上。它还显示出快速适应，公平指数为0.7，显示出更好的资源分配。这项工作证明，元学习是复杂无线系统中智能控制的一个非常有效和安全的选择。
摘要：The dynamic allocation of spectrum in 5G / 6G networks is critical to efficient resource utilization. However, applying traditional deep reinforcement learning (DRL) is often infeasible due to its immense sample complexity and the safety risks associated with unguided exploration, which can cause severe network interference. To address these challenges, we propose a meta-learning framework that enables agents to learn a robust initial policy and rapidly adapt to new wireless scenarios with minimal data. We implement three meta-learning architectures, model-agnostic meta-learning (MAML), recurrent neural network (RNN), and an attention-enhanced RNN, and evaluate them against a non-meta-learning DRL algorithm, proximal policy optimization (PPO) baseline, in a simulated dynamic integrated access/backhaul (IAB) environment. Our results show a clear performance gap. The attention-based meta-learning agent reaches a peak mean network throughput of 48 Mbps, while the PPO baseline decreased drastically to 10 Mbps. Furthermore, our method reduces SINR and latency violations by more than 50% compared to PPO. It also shows quick adaptation, with a fairness index 0.7, showing better resource allocation. This work proves that meta-learning is a very effective and safer option for intelligent control in complex wireless systems.
医学相关(5篇)
【1】Striking the Perfect Balance: Preserving Privacy While Boosting Utility in Collaborative Medical Prediction Platforms
标题：实现完美平衡：在提高协作医疗预测平台的实用性的同时保护隐私
链接：https://arxiv.org/abs/2507.11187
摘要：在线协作医疗预测平台通过利用大量电子健康记录提供便利和实时反馈。然而，对隐私和低预测质量的日益关注可能会阻止患者参与和医生合作。本文首先明确了隐私攻击，即针对患者的属性攻击和针对医生的模型抽取攻击，并规定了相应的隐私原则。然后，我们提出了一个隐私保护机制，并将其集成到一个新的一次性分布式学习框架，旨在同时满足隐私要求和预测性能目标。在统计学习理论的框架内，我们从理论上证明了所提出的分布式学习框架可以在特定的隐私要求下实现最优的预测性能。我们通过玩具模拟和真实世界的数据实验进一步验证了开发的隐私保护协作医疗预测平台。
摘要：Online collaborative medical prediction platforms offer convenience and real-time feedback by leveraging massive electronic health records. However, growing concerns about privacy and low prediction quality can deter patient participation and doctor cooperation. In this paper, we first clarify the privacy attacks, namely attribute attacks targeting patients and model extraction attacks targeting doctors, and specify the corresponding privacy principles. We then propose a privacy-preserving mechanism and integrate it into a novel one-shot distributed learning framework, aiming to simultaneously meet both privacy requirements and prediction performance objectives. Within the framework of statistical learning theory, we theoretically demonstrate that the proposed distributed learning framework can achieve the optimal prediction performance under specific privacy requirements. We further validate the developed privacy-preserving collaborative medical prediction platform through both toy simulations and real-world data experiments.
【2】An Explainable AI-Enhanced Machine Learning Approach for Cardiovascular Disease Detection and Risk Assessment
标题：用于心血管疾病检测和风险评估的可解释人工智能增强机器学习方法
链接：https://arxiv.org/abs/2507.11185
备注：This paper has been accepted at the IEEE QPAIN 2025. The final version will be available in the IEEE Xplore Digital Library
摘要：心脏病仍然是一个主要的全球健康问题，特别是在医疗资源和诊断设施有限的地区。传统的诊断方法往往无法准确识别和管理心脏病风险，导致不良后果。机器学习有可能显著提高心脏病诊断的准确性、效率和速度。在这项研究中，我们提出了一个综合框架，结合了心脏病检测的分类模型和风险预测的回归模型。我们使用了心脏病数据集，其中包括1，035例病例。为了解决类别不平衡的问题，采用了合成少数群体过采样技术（SMOTE），从而产生了额外的100 000个合成数据点。性能指标，包括准确率，精度，召回率，F1分数，R2，MSE，RMSE和MAE，用于评估模型的有效性。在分类模型中，随机森林表现突出，在真实数据上的准确率为97.2%，在合成数据上的准确率为97.6%。对于回归任务，线性回归在真实和合成数据集上分别表现出最高的R2值0.992和0.984，具有最低的误差指标。此外，还采用了可解释的人工智能技术来增强模型的可解释性。这项研究强调了机器学习在彻底改变心脏病诊断和风险预测方面的潜力，从而促进早期干预并加强临床决策。
摘要：Heart disease remains a major global health concern, particularly in regions with limited access to medical resources and diagnostic facilities. Traditional diagnostic methods often fail to accurately identify and manage heart disease risks, leading to adverse outcomes. Machine learning has the potential to significantly enhance the accuracy, efficiency, and speed of heart disease diagnosis. In this study, we proposed a comprehensive framework that combines classification models for heart disease detection and regression models for risk prediction. We employed the Heart Disease dataset, which comprises 1,035 cases. To address the issue of class imbalance, the Synthetic Minority Oversampling Technique (SMOTE) was applied, resulting in the generation of an additional 100,000 synthetic data points. Performance metrics, including accuracy, precision, recall, F1-score, R2, MSE, RMSE, and MAE, were used to evaluate the model's effectiveness. Among the classification models, Random Forest emerged as the standout performer, achieving an accuracy of 97.2% on real data and 97.6% on synthetic data. For regression tasks, Linear Regression demonstrated the highest R2 values of 0.992 and 0.984 on real and synthetic datasets, respectively, with the lowest error metrics. Additionally, Explainable AI techniques were employed to enhance the interpretability of the models. This study highlights the potential of machine learning to revolutionize heart disease diagnosis and risk prediction, thereby facilitating early intervention and enhancing clinical decision-making.
【3】Stochastic Entanglement Configuration for Constructive Entanglement Topologies in Quantum Machine Learning with Application to Cardiac MRI
标题：量子机器学习中构造性纠缠拓扑的随机纠缠配置及其在心脏MRI中的应用
链接：https://arxiv.org/abs/2507.11401
备注：Accepted for publication at IEEE International Conference on Quantum Computing and Engineering (QCE) 2025
摘要：有效的纠缠策略是推进量子机器学习变分量子电路（VQC）的关键。然而，目前大多数方法使用固定的纠缠拓扑结构，不适应任务的要求，限制了潜在的收益超过经典模型。我们介绍了一种新型的随机纠缠配置方法，该方法系统地生成不同的纠缠拓扑，以识别建设性纠缠配置的子空间，定义为提高混合模型性能的纠缠拓扑（例如，分类准确度）超过经典基线。每个配置被编码为随机二进制矩阵，表示量子比特之间的定向纠缠。这使得候选纠缠拓扑的超空间的可扩展探索使用纠缠密度和每量子位约束作为关键度量。我们定义了无约束和约束采样模式，控制每个量子比特的纠缠。使用我们的方法，400随机配置生成和评价的混合QML心脏MRI疾病分类。我们确定了64（16%）新的建设性纠缠配置，始终优于经典的基线。性能最好的配置的Ensemble聚合实现了~0.92的分类精度，超过经典模型（~0.87）超过5%。与四种传统拓扑结构（环形，最近邻，无纠缠，完全纠缠）相比，没有一种超过经典基线（最大精度约为0.82），而我们的配置提供了高达20%的精度。因此，突出了确定的建设性纠缠的鲁棒性和普遍性。
摘要：Efficient entanglement strategies are essential for advancing variational quantum circuits (VQCs) for quantum machine learning (QML). However, most current approaches use fixed entanglement topologies that are not adaptive to task requirements, limiting potential gains over classical models. We introduce a novel stochastic entanglement configuration method that systematically generates diverse entanglement topologies to identify a subspace of constructive entanglement configurations, defined as entanglement topologies that boost hybrid model performance (e.g., classification accuracy) beyond classical baselines. Each configuration is encoded as a stochastic binary matrix, denoting directed entanglement between qubits. This enables scalable exploration of the hyperspace of candidate entanglement topologies using entanglement density and per-qubit constraints as key metrics. We define unconstrained and constrained sampling modes, controlling entanglement per qubit. Using our method, 400 stochastic configurations were generated and evaluated in a hybrid QML for cardiac MRI disease classification. We identified 64 (16%) novel constructive entanglement configurations that consistently outperformed the classical baseline. Ensemble aggregation of top-performing configurations achieved ~0.92 classification accuracy, exceeding the classical model (~0.87) by over 5%. Compared to four conventional topologies (ring, nearest neighbor, no entanglement, fully entangled), none surpassed the classical baseline (maximum accuracy ~0.82), while our configurations delivered up to ~20% higher accuracy. Thus, highlighting the robustness and generalizability of the identified constructive entanglements.
【4】From Observational Data to Clinical Recommendations: A Causal Framework for Estimating Patient-level Treatment Effects and Learning Policies
标题：从观察数据到临床建议：估计患者水平治疗效果和学习政策的因果框架
链接：https://arxiv.org/abs/2507.11381
摘要：我们提出了一个框架，用于建立患者特定的治疗推荐模型，建立在最近的大量文献学习患者水平的因果模型，并受到Hernan和Robins的目标试验范式的启发。我们专注于安全性和有效性，包括使用观察数据时的因果关系识别的关键问题。我们不提供特定的模型，而是提供一种将现有方法和技术整合到实际管道中的方法。我们还提供了一个现实世界的治疗优化的使用情况下，心力衰竭患者在住院期间发生急性肾损伤。结果表明，我们的管道可以改善目前治疗方案的患者结局。
摘要：We propose a framework for building patient-specific treatment recommendation models, building on the large recent literature on learning patient-level causal models and inspired by the target trial paradigm of Hernan and Robins. We focus on safety and validity, including the crucial issue of causal identification when using observational data. We do not provide a specific model, but rather a way to integrate existing methods and know-how into a practical pipeline. We further provide a real world use-case of treatment optimization for patients with heart failure who develop acute kidney injury during hospitalization. The results suggest our pipeline can improve patient outcomes over the current treatment regime.
【5】AGFS-Tractometry: A Novel Atlas-Guided Fine-Scale Tractometry Approach for Enhanced Along-Tract Group Statistical Comparison Using Diffusion MRI Tractography
标题：AGFS-牵引测量：使用扩散MRI纤维束摄影术进行增强沿道组统计学比较的新型拉曼引导细尺度纤维束测量方法
链接：https://arxiv.org/abs/2507.10601
备注：31 pages and 7 figures
摘要：弥散磁共振成像（dMRI）纤维束成像是目前唯一的方法在体内映射大脑的白质（WM）连接。Tractometry是一种先进的纤维束成像分析技术，用于沿纤维束的形态和显微结构特性的研究。Tractometry已经成为研究不同人群之间局部沿道差异的重要工具（例如，健康与疾病）。在这项研究中，我们提出了一种新的地图集引导的精细尺度tractometry方法，即AGFS-Tractometry，利用道空间信息和排列测试，以提高人口之间的沿道统计分析。AGFS-Tractometry有两个主要贡献。首先，我们创建了一个新的图谱引导道分析模板，使一致的，精细的规模，沿道包裹的主题特定的纤维束。其次，我们提出了一种新的非参数排列测试组比较方法，使所有沿轨道包裹同时进行分析，同时纠正多重比较。我们对已知组差异和体内真实数据的合成数据集进行实验评估。我们比较AGFS-纤维束测量与两种最先进的纤维束测量方法，包括自动纤维束定量（AFQ）和束分析（BUAN）。我们的研究结果表明，建议AGFS-Tractometry获得增强的灵敏度和特异性，在检测本地WM的差异。在实际数据分析实验中，AGFS-Tractometry可以识别出更多的具有显著差异的区域，这与现有文献的解剖学一致。总体而言，这些证明了AGFS-Tractometry检测细微或空间定位WM组水平差异的能力。创建的区域分析模板和相关代码可在https://github.com/ZhengRuixi/AGFS-Tractometry.git上获得。
摘要：Diffusion MRI (dMRI) tractography is currently the only method for in vivo mapping of the brain's white matter (WM) connections. Tractometry is an advanced tractography analysis technique for along-tract profiling to investigate the morphology and microstructural properties along the fiber tracts. Tractometry has become an essential tool for studying local along-tract differences between different populations (e.g., health vs disease). In this study, we propose a novel atlas-guided fine-scale tractometry method, namely AGFS-Tractometry, that leverages tract spatial information and permutation testing to enhance the along-tract statistical analysis between populations. There are two major contributions in AGFS-Tractometry. First, we create a novel atlas-guided tract profiling template that enables consistent, fine-scale, along-tract parcellation of subject-specific fiber tracts. Second, we propose a novel nonparametric permutation testing group comparison method to enable simultaneous analysis across all along-tract parcels while correcting for multiple comparisons. We perform experimental evaluations on synthetic datasets with known group differences and in vivo real data. We compare AGFS-Tractometry with two state-of-the-art tractometry methods, including Automated Fiber-tract Quantification (AFQ) and BUndle ANalytics (BUAN). Our results show that the proposed AGFS-Tractometry obtains enhanced sensitivity and specificity in detecting local WM differences. In the real data analysis experiments, AGFS-Tractometry can identify more regions with significant differences, which are anatomically consistent with the existing literature. Overall, these demonstrate the ability of AGFS-Tractometry to detect subtle or spatially localized WM group-level differences. The created tract profiling template and related code are available at: https://github.com/ZhengRuixi/AGFS-Tractometry.git.
蒸馏|知识提取(1篇)
【1】Spectral Feature Extraction for Robust Network Intrusion Detection Using MFCCs
标题：使用MFCC的鲁棒网络入侵检测光谱特征提取
链接：https://arxiv.org/abs/2507.10622
摘要 ：物联网（IoT）网络的快速扩张导致安全漏洞激增，强调了对强大的异常检测和分类技术的迫切需求。在这项工作中，我们提出了一种新的方法，通过利用Mel频率倒谱系数（MFCC）和ResNet-18来识别物联网网络流量中的异常，ResNet-18是一种以其在特征提取和基于图像的任务中的有效性而闻名的深度学习模型。可学习的MFCC使自适应频谱特征表示，捕捉网络流量中固有的时间模式比传统的固定MFCC更有效。我们证明了将原始信号转换为MFCC将数据映射到更高维的空间，增强了类的可分性，并实现了更有效的多类分类。我们的方法结合了MFCC的优势和ResNet-18强大的特征提取功能，为异常检测提供了一个强大的框架。该模型在三个广泛使用的物联网入侵检测数据集上进行了评估：CICIoT 2023，NSL-KDD和IoTID 20。实验结果突出了将自适应信号处理技术与深度学习架构相结合的潜力，以在异构物联网网络环境中实现强大且可扩展的异常检测。
摘要：The rapid expansion of Internet of Things (IoT) networks has led to a surge in security vulnerabilities, emphasizing the critical need for robust anomaly detection and classification techniques. In this work, we propose a novel approach for identifying anomalies in IoT network traffic by leveraging the Mel-frequency cepstral coefficients (MFCC) and ResNet-18, a deep learning model known for its effectiveness in feature extraction and image-based tasks. Learnable MFCCs enable adaptive spectral feature representation, capturing the temporal patterns inherent in network traffic more effectively than traditional fixed MFCCs. We demonstrate that transforming raw signals into MFCCs maps the data into a higher-dimensional space, enhancing class separability and enabling more effective multiclass classification. Our approach combines the strengths of MFCCs with the robust feature extraction capabilities of ResNet-18, offering a powerful framework for anomaly detection. The proposed model is evaluated on three widely used IoT intrusion detection datasets: CICIoT2023, NSL-KDD, and IoTID20. The experimental results highlight the potential of integrating adaptive signal processing techniques with deep learning architectures to achieve robust and scalable anomaly detection in heterogeneous IoT network landscapes.
聚类(2篇)
【1】GOLFS: Feature Selection via Combining Both Global and Local Information for High Dimensional Clustering
标题：GOLFS：通过结合全局和本地信息进行多维集群的特征选择
链接：https://arxiv.org/abs/2507.10956
备注：None
摘要：识别高维聚类中的判别特征是一个重要的问题。然而，由于缺乏聚类标签，为监督特征选择开发的正则化方法不能直接应用。为了同时学习伪标签和选择区分性特征，我们提出了一种新的无监督特征选择方法，称为全局和局部信息组合特征选择（GOLFS），用于高维聚类问题。GOLFS算法结合局部几何结构通过流形学习和全局相关结构的样本，通过正则化的自我表示来选择判别特征。该组合通过利用更全面的信息来提高特征选择和聚类的准确性。此外，提出了一种求解优化问题的迭代算法，并证明了算法的收敛性。仿真和两个实际数据的应用表明，GOLFS的优秀的有限样本性能的特征选择和聚类。
摘要：It is important to identify the discriminative features for high dimensional clustering. However, due to the lack of cluster labels, the regularization methods developed for supervised feature selection can not be directly applied. To learn the pseudo labels and select the discriminative features simultaneously, we propose a new unsupervised feature selection method, named GlObal and Local information combined Feature Selection (GOLFS), for high dimensional clustering problems. The GOLFS algorithm combines both local geometric structure via manifold learning and global correlation structure of samples via regularized self-representation to select the discriminative features. The combination improves the accuracy of both feature selection and clustering by exploiting more comprehensive information. In addition, an iterative algorithm is proposed to solve the optimization problem and the convergency is proved. Simulations and two real data applications demonstrate the excellent finite-sample performance of GOLFS on both feature selection and clustering.
【2】Robust Multi-Manifold Clustering via Simplex Paths
标题：通过单纯形路径的鲁棒多Manifle集群
链接：https://arxiv.org/abs/2507.10710
摘要：本文介绍了一种新的，几何方法多流形聚类（MMC），即聚类的集合可能相交，d维流形到个别流形组件。我们首先计算d-单形上的局部图，使用相邻单形之间的二面角作为图的权重，然后计算该单形图中的无穷远路径距离。这个过程给出了一个度量的单形，我们称之为最大角路径距离（LAPD）。我们分析了随机抽样下LAPD的性质，并证明了通过适当的去噪过程，该度量以高概率分离流形分量。我们验证了所提出的方法与广泛的数值实验合成和真实世界的数据集。实验结果表明，该方法对噪声、曲率和小相交角具有较好的鲁棒性，总体上优于其他MMC算法。此外，我们还提供了该算法的高度可扩展的实现，该算法利用无限路径距离的近似方案来实现准线性计算复杂性。
摘要：This article introduces a novel, geometric approach for multi-manifold clustering (MMC), i.e. for clustering a collection of potentially intersecting, d-dimensional manifolds into the individual manifold components. We first compute a locality graph on d-simplices, using the dihedral angle in between adjacent simplices as the graph weights, and then compute infinity path distances in this simplex graph. This procedure gives a metric on simplices which we refer to as the largest angle path distance (LAPD). We analyze the properties of LAPD under random sampling, and prove that with an appropriate denoising procedure, this metric separates the manifold components with high probability. We validate the proposed methodology with extensive numerical experiments on both synthetic and real-world data sets. These experiments demonstrate that the method is robust to noise, curvature, and small intersection angle, and generally out-performs other MMC algorithms. In addition, we provide a highly scalable implementation of the proposed algorithm, which leverages approximation schemes for infinity path distance to achieve quasi-linear computational complexity.
联邦学习|隐私保护|加密(3篇)
【1】D3FL: Data Distribution and Detrending for Robust Federated Learning in Non-linear Time-series Data
标题：D3 FL：非线性时间序列数据中鲁棒联邦学习的数据分布和去趋势
链接：https://arxiv.org/abs/2507.11471
备注：Preprint of paper to appear in the proceedings of IEEE INTERNATIONAL CONFERENCE ON EDGE COMPUTING & COMMUNICATIONS EDGE 2025
摘要：随着计算和通信技术的进步，物联网（IoT）已经出现了显着增长。物联网设备通常从各种传感器收集数据，例如温度、湿度和电能表。这些数据大部分都是临时性的。传统上，来自物联网设备的数据被集中起来进行分析，但这种方法会带来延迟并增加通信成本。联邦学习（FL）已经成为一种有效的替代方案，允许跨分布式设备进行模型训练，而无需集中数据。在许多应用中，例如智能家居能源和环境监测，物联网设备在不同位置收集的数据可能会在趋势和季节模式方面表现出显着的变化。准确预测这种非平稳、非线性的时间序列数据对于能耗估计和天气预报等应用至关重要。然而，这些数据变化会严重影响预测准确性。本文的主要贡献是：（1）研究了非线性、非平稳的时间序列数据分布，如广义极值分布（gen-extreme）和对数范数分布，如何影响FL性能。(2)分析非线性时间序列数据的不同去趋势技术如何影响FL设置中预测模型的性能。我们使用非线性数据分布生成了几个合成时间序列数据集，并使用集中式和FL方法训练了基于LSTM的预测模型。此外，我们评估了去趋势对具有非线性时间序列数据分布的真实数据集的影响。实验结果表明：（1）在处理非线性数据分布时，FL的性能不如集中式方法。(2)使用适当的去趋势技术可以提高FL性能，减少不同数据分布之间的损失。
摘要 ：With advancements in computing and communication technologies, the Internet of Things (IoT) has seen significant growth. IoT devices typically collect data from various sensors, such as temperature, humidity, and energy meters. Much of this data is temporal in nature. Traditionally, data from IoT devices is centralized for analysis, but this approach introduces delays and increased communication costs. Federated learning (FL) has emerged as an effective alternative, allowing for model training across distributed devices without the need to centralize data. In many applications, such as smart home energy and environmental monitoring, the data collected by IoT devices across different locations can exhibit significant variation in trends and seasonal patterns. Accurately forecasting such non-stationary, non-linear time-series data is crucial for applications like energy consumption estimation and weather forecasting. However, these data variations can severely impact prediction accuracy. The key contributions of this paper are: (1) Investigating how non-linear, non-stationary time-series data distributions, like generalized extreme value (gen-extreme) and log norm distributions, affect FL performance. (2) Analyzing how different detrending techniques for non-linear time-series data influence the forecasting model's performance in a FL setup. We generated several synthetic time-series datasets using non-linear data distributions and trained an LSTM-based forecasting model using both centralized and FL approaches. Additionally, we evaluated the impact of detrending on real-world datasets with non-linear time-series data distributions. Our experimental results show that: (1) FL performs worse than centralized approaches when dealing with non-linear data distributions. (2) The use of appropriate detrending techniques improves FL performance, reducing loss across different data distributions.
【2】FLsim: A Modular and Library-Agnostic Simulation Framework for Federated Learning
标题：FLsim：用于联邦学习的模块化且与库无关的模拟框架
链接：https://arxiv.org/abs/2507.11430
摘要：自2016年成立以来，联邦学习（FL）经历了重大发展，从基本算法发展到为应对各种挑战和用例而量身定制的复杂方法。然而，研究和基准的新FL技术对过多的既定国家的最先进的解决方案仍然具有挑战性。为了简化这一过程，我们引入FLsim，一个全面的FL仿真框架，旨在满足文献中FL工作流程的不同要求。FLsim的特点是其模块化，可扩展性，资源效率和实验结果的可控再现性。其易于使用的界面允许用户通过作业配置指定自定义FL要求，支持：（a）定制的数据分布，从非独立同分布（non-iid）数据到独立同分布（iid）数据，（b）根据用户偏好选择本地学习算法，对ML库完全不可知，（c）选择网络拓扑，说明节点之间的通信模式，（d）定义模型聚合和共识算法，以及（e）支持可插入区块链以增强鲁棒性。通过一系列的实验评估，我们证明了FLsim在模拟各种最先进的FL实验的有效性和多功能性。我们设想FLsim将标志着FL仿真框架的重大进步，为研究人员和从业人员提供前所未有的灵活性和功能。
摘要：Federated Learning (FL) has undergone significant development since its inception in 2016, advancing from basic algorithms to complex methodologies tailored to address diverse challenges and use cases. However, research and benchmarking of novel FL techniques against a plethora of established state-of-the-art solutions remain challenging. To streamline this process, we introduce FLsim, a comprehensive FL simulation framework designed to meet the diverse requirements of FL workflows in the literature. FLsim is characterized by its modularity, scalability, resource efficiency, and controlled reproducibility of experimental outcomes. Its easy to use interface allows users to specify customized FL requirements through job configuration, which supports: (a) customized data distributions, ranging from non-independent and identically distributed (non-iid) data to independent and identically distributed (iid) data, (b) selection of local learning algorithms according to user preferences, with complete agnosticism to ML libraries, (c) choice of network topology illustrating communication patterns among nodes, (d) definition of model aggregation and consensus algorithms, and (e) pluggable blockchain support for enhanced robustness. Through a series of experimental evaluations, we demonstrate the effectiveness and versatility of FLsim in simulating a diverse range of state-of-the-art FL experiments. We envisage that FLsim would mark a significant advancement in FL simulation frameworks, offering unprecedented flexibility and functionality for researchers and practitioners alike.
【3】Quantized Rank Reduction: A Communications-Efficient Federated Learning Scheme for Network-Critical Applications
标题：量化排名缩减：用于网络关键型应用程序的通信高效联邦学习计划
链接：https://arxiv.org/abs/2507.11183
备注：In Proceedings of the 2025 IARIA Annual Congress on Frontiers in Science, Technology, Services, and Applications (IARIA Congress 2025), Venice, Italy, July 6-10, 2025
摘要：联合学习是一种机器学习方法，其使得多个设备（即，代理）协作地训练共享模型而不交换原始数据。这种技术将数据本地化在用户设备上，确保隐私和安全，而每个代理都在自己的数据上训练模型，并且只共享模型更新。由于代理和中央服务器之间频繁交换模型更新，因此通信开销是一个重大挑战。在本文中，我们提出了一种通信高效的联邦学习方案，该方案利用神经网络梯度和量化的低秩近似，以显着降低分散学习过程的网络负载，同时对模型的准确性影响最小。
摘要：Federated learning is a machine learning approach that enables multiple devices (i.e., agents) to train a shared model cooperatively without exchanging raw data. This technique keeps data localized on user devices, ensuring privacy and security, while each agent trains the model on their own data and only shares model updates. The communication overhead is a significant challenge due to the frequent exchange of model updates between the agents and the central server. In this paper, we propose a communication-efficient federated learning scheme that utilizes low-rank approximation of neural network gradients and quantization to significantly reduce the network load of the decentralized learning process with minimal impact on the model's accuracy.
推理|分析|理解|解释(8篇)
【1】Neurosymbolic Reasoning Shortcuts under the Independence Assumption
标题：独立假设下的神经符号推理捷径
链接：https://arxiv.org/abs/2507.11357
备注：Accepted at NeSy 2025
摘要：神经符号（NeSy）预测器中符号概念之间普遍存在的独立性假设是一种方便的简化：NeSy预测器使用它来加速概率推理。van Krieken et al.（2024）和Marconato et al.（2024）等最近的研究认为，独立性假设会阻碍NeSy预测器的学习，更重要的是，会阻止它们正确地建模不确定性。然而，在NeSy社区中，人们对独立性假设实际上限制了NeSy系统的情况持怀疑态度（Faronius和Dos Martires，2025）。在这项工作中，我们解决了这个问题，正式表明，假设符号概念之间的独立性意味着一个模型永远不能代表某些概念组合的不确定性。因此，模型无法意识到推理捷径，即，NeSy预测器的病态行为，预测正确的下游任务，但出于错误的原因。
摘要：The ubiquitous independence assumption among symbolic concepts in neurosymbolic (NeSy) predictors is a convenient simplification: NeSy predictors use it to speed up probabilistic reasoning. Recent works like van Krieken et al. (2024) and Marconato et al. (2024) argued that the independence assumption can hinder learning of NeSy predictors and, more crucially, prevent them from correctly modelling uncertainty. There is, however, scepticism in the NeSy community around the scenarios in which the independence assumption actually limits NeSy systems (Faronius and Dos Martires, 2025). In this work, we settle this question by formally showing that assuming independence among symbolic concepts entails that a model can never represent uncertainty over certain concept combinations. Thus, the model fails to be aware of reasoning shortcuts, i.e., the pathological behaviour of NeSy predictors that predict correct downstream tasks but for the wrong reasons.
【2】Fairness-Aware Grouping for Continuous Sensitive Variables: Application for Debiasing Face Analysis with respect to Skin Tone
标题：连续敏感变量的公平意识预设：针对肤色去偏置面部分析的应用
链接：https://arxiv.org/abs/2507.11247
摘要 ：在法律框架内，数据集和模型中的公平性通常通过将观察分为预定义的组，然后计算公平性度量（例如，性别方面的差异影响或赔率平等）。然而，当敏感属性，如肤色是连续的，分为默认组可能会忽略或掩盖某些少数亚群所经历的歧视。为了解决这个问题，我们提出了一个基于公平的分组方法连续（可能是多维）敏感属性。通过分组数据，根据观察到的歧视水平，我们的方法确定的分区，最大限度地提高了一个新的标准的基础上组间方差的歧视，从而隔离最关键的子群。 我们使用多个合成数据集验证了所提出的方法，并证明了其在不断变化的人口分布下的鲁棒性-揭示了歧视如何在敏感属性的空间内表现出来。此外，我们研究了一个专门的设置单调公平的情况下，肤色。我们对CelebA和FFHQ的实证结果，利用工业专有算法预测的肤色，表明所提出的分割揭示了比以前报道的更细微的歧视模式，并且这些发现在给定模型的数据集上保持稳定。最后，我们利用我们的分组模型去偏置的目的，旨在预测公平的分数与组逐组后处理。结果表明，我们的方法提高了公平性，同时对准确性的影响最小，从而证实了我们的分区方法，并为工业部署打开了大门。
摘要：Within a legal framework, fairness in datasets and models is typically assessed by dividing observations into predefined groups and then computing fairness measures (e.g., Disparate Impact or Equality of Odds with respect to gender). However, when sensitive attributes such as skin color are continuous, dividing into default groups may overlook or obscure the discrimination experienced by certain minority subpopulations. To address this limitation, we propose a fairness-based grouping approach for continuous (possibly multidimensional) sensitive attributes. By grouping data according to observed levels of discrimination, our method identifies the partition that maximizes a novel criterion based on inter-group variance in discrimination, thereby isolating the most critical subgroups. We validate the proposed approach using multiple synthetic datasets and demonstrate its robustness under changing population distributions - revealing how discrimination is manifested within the space of sensitive attributes. Furthermore, we examine a specialized setting of monotonic fairness for the case of skin color. Our empirical results on both CelebA and FFHQ, leveraging the skin tone as predicted by an industrial proprietary algorithm, show that the proposed segmentation uncovers more nuanced patterns of discrimination than previously reported, and that these findings remain stable across datasets for a given model. Finally, we leverage our grouping model for debiasing purpose, aiming at predicting fair scores with group-by-group post-processing. The results demonstrate that our approach improves fairness while having minimal impact on accuracy, thus confirming our partition method and opening the door for industrial deployment.
【3】Learning from Imperfect Data: Robust Inference of Dynamic Systems using Simulation-based Generative Model
标题：从不完美数据中学习：使用基于模拟的生成模型对动态系统进行鲁棒推理
链接：https://arxiv.org/abs/2507.10884
摘要：非线性动态模型的系统推理，由常微分方程（ODE）表示，在许多领域仍然是一个重大的挑战，特别是当数据是嘈杂的，稀疏的，或部分可观察的。在本文中，我们提出了一个基于仿真的生成模型不完美的数据（SiGMoID），使精确和强大的推理动态系统。所提出的方法集成了两种关键方法：（1）物理信息神经网络与构建ODE求解器的超网络，以及（2）Wasserstein生成对抗网络，通过有效捕获噪声数据分布来估计ODE参数。我们证明了SiGMoID量化数据噪声，估计系统参数，并推断未观察到的系统组件。它的有效性通过现实的实验示例进行验证，展示了它在各个领域的广泛适用性，从科学研究到工程系统，并使完整的系统动力学的发现。
摘要：System inference for nonlinear dynamic models, represented by ordinary differential equations (ODEs), remains a significant challenge in many fields, particularly when the data are noisy, sparse, or partially observable. In this paper, we propose a Simulation-based Generative Model for Imperfect Data (SiGMoID) that enables precise and robust inference for dynamic systems. The proposed approach integrates two key methods: (1) physics-informed neural networks with hyper-networks that constructs an ODE solver, and (2) Wasserstein generative adversarial networks that estimates ODE parameters by effectively capturing noisy data distributions. We demonstrate that SiGMoID quantifies data noise, estimates system parameters, and infers unobserved system components. Its effectiveness is validated validated through realistic experimental examples, showcasing its broad applicability in various domains, from scientific research to engineered systems, and enabling the discovery of full system dynamics.
【4】Winsor-CAM: Human-Tunable Visual Explanations from Deep Networks via Layer-Wise Winsorization
标题：Winsor-CAM：通过逐层Winsorization从深度网络中实现人性可调的视觉解释
链接：https://arxiv.org/abs/2507.10846
备注：15 pages, 10 figures, 7 tables. Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence
摘要：解释卷积神经网络（CNN）的决策过程对于在高风险领域部署模型至关重要。梯度加权类激活映射（Grad-CAM）是一种广泛使用的视觉解释方法，但它通常专注于最终的卷积层或简单地跨层平均，这些策略可能会模糊重要的语义线索或放大不相关的噪音。我们提出了Winsor-CAM，这是Grad-CAM的一种新颖的、可人为调整的扩展，它通过聚合所有卷积层的信息来生成鲁棒且连贯的显着性图。为了减轻噪声或极端属性值的影响，Winsor-CAM应用了Winsorization，这是一种基于统计的离群值衰减技术。用户可控制的阈值允许语义级别的调优，从而允许跨表示层次结构灵活地探索模型行为。使用PASCAL VOC 2012数据集对标准架构（ResNet 50、DenseNet 121、VGG 16、InceptionV 3）进行的评估表明，与Grad-CAM和均匀层平均基线相比，Winsor-CAM产生了更多可解释的热图，并在定位指标方面实现了卓越的性能，包括交叉-联合和质心对齐。Winsor-CAM通过提供可解释的、具有人机交互控制的多层洞察力，推进可信赖人工智能的目标。
摘要：Interpreting the decision-making process of Convolutional Neural Networks (CNNs) is critical for deploying models in high-stakes domains. Gradient-weighted Class Activation Mapping (Grad-CAM) is a widely used method for visual explanations, yet it typically focuses on the final convolutional layer or na\"ively averages across layers, strategies that can obscure important semantic cues or amplify irrelevant noise. We propose Winsor-CAM, a novel, human-tunable extension of Grad-CAM that generates robust and coherent saliency maps by aggregating information across all convolutional layers. To mitigate the influence of noisy or extreme attribution values, Winsor-CAM applies Winsorization, a percentile-based outlier attenuation technique. A user-controllable threshold allows for semantic-level tuning, enabling flexible exploration of model behavior across representational hierarchies. Evaluations on standard architectures (ResNet50, DenseNet121, VGG16, InceptionV3) using the PASCAL VOC 2012 dataset demonstrate that Winsor-CAM produces more interpretable heatmaps and achieves superior performance in localization metrics, including intersection-over-union and center-of-mass alignment, when compared to Grad-CAM and uniform layer-averaging baselines. Winsor-CAM advances the goal of trustworthy AI by offering interpretable, multi-layer insights with human-in-the-loop control.
【5】A Simple Approximate Bayesian Inference Neural Surrogate for Stochastic Petri Net Models
标题：随机Petri网模型的简单近似Bayesian推理神经代理
链接：https://arxiv.org/abs/2507.10714
备注：12 pages, 10 figures, for all associated codes and files, see this https URL
摘要：随机Petri网（SPN）是流行病学和系统生物学等领域中离散事件动态建模的一种越来越受欢迎的工具，但其参数估计仍然具有挑战性，特别是当转换率依赖于外部协变量和显式似然不可用时。我们引入了一个神经代理（基于神经网络的后验分布近似）框架，该框架直接从嘈杂的，部分观察到的令牌轨迹预测已知协变量相关的速率函数的系数。我们的模型采用了一个轻量级的1D卷积残差网络，在Gillespie模拟的SPN实现上进行端到端训练，学习在事件丢失的现实条件下反转系统动态。在推断过程中，蒙特卡罗dropout提供校准的不确定性界限以及点估计。在有20%缺失事件的合成SPN上，我们的代理恢复RMSE = 0.108的速率函数系数，并且比传统的贝叶斯方法运行得更快。这些结果表明，数据驱动的，无似然的代理人可以在复杂的，部分观测的离散事件系统中实现准确，鲁棒和实时的参数恢复。
摘要 ：Stochastic Petri Nets (SPNs) are an increasingly popular tool of choice for modeling discrete-event dynamics in areas such as epidemiology and systems biology, yet their parameter estimation remains challenging in general and in particular when transition rates depend on external covariates and explicit likelihoods are unavailable. We introduce a neural-surrogate (neural-network--based approximation of the posterior distribution) framework that predicts the coefficients of known covariate-dependent rate functions directly from noisy, partially observed token trajectories. Our model employs a lightweight 1D Convolutional Residual Network trained end-to-end on Gillespie-simulated SPN realizations, learning to invert system dynamics under realistic conditions of event dropout. During inference, Monte Carlo dropout provides calibrated uncertainty bounds together with point estimates. On synthetic SPNs with 20% missing events, our surrogate recovers rate-function coefficients with an RMSE = 0.108 and substantially runs faster than traditional Bayesian approaches. These results demonstrate that data-driven, likelihood-free surrogates can enable accurate, robust, and real-time parameter recovery in complex, partially observed discrete-event systems.
【6】A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks
标题：碱基加法背后的对称性及其神经网络可学习性的群论分析
链接：https://arxiv.org/abs/2507.10678
备注：22 pages, 6 figures
摘要：使用神经网络建模人类认知功能和人工智能的一个主要挑战是设计能够有效学习支持激进泛化功能的系统。其根源在于发现和实现对称函数的能力。在本文中，我们研究了一个典型的例子，激进的推广，通过使用对称性：基地增加。我们提出了一个群论分析基地除了，一个基本的和定义的特点是进行功能-转让的其余部分，当一个总和超过基地模，到下一个重要的地方。我们的分析暴露了一系列的替代进行功能，为一个给定的基地，我们引入定量措施来表征这些。然后，我们利用进位函数的差异来探测神经网络在对称学习中的归纳偏差，通过训练神经网络使用不同的进位进行碱基加法，并比较其结构的函数的学习效率和速度。我们发现，即使是简单的神经网络也可以通过正确的输入格式和进位函数实现激进的泛化，并且学习速度与进位函数结构密切相关。然后，我们讨论这与认知科学和机器学习的相关性。
摘要：A major challenge in the use of neural networks both for modeling human cognitive function and for artificial intelligence is the design of systems with the capacity to efficiently learn functions that support radical generalization. At the roots of this is the capacity to discover and implement symmetry functions. In this paper, we investigate a paradigmatic example of radical generalization through the use of symmetry: base addition. We present a group theoretic analysis of base addition, a fundamental and defining characteristic of which is the carry function -- the transfer of the remainder, when a sum exceeds the base modulus, to the next significant place. Our analysis exposes a range of alternative carry functions for a given base, and we introduce quantitative measures to characterize these. We then exploit differences in carry functions to probe the inductive biases of neural networks in symmetry learning, by training neural networks to carry out base addition using different carries, and comparing efficacy and rate of learning as a function of their structure. We find that even simple neural networks can achieve radical generalization with the right input format and carry function, and that learning speed is closely correlated with carry function structure. We then discuss the relevance this has for cognitive science and machine learning.
【7】Tool-to-Tool Matching Analysis Based Difference Score Computation Methods for Semiconductor Manufacturing
标题：基于工具间匹配分析的半导体制造差异评分计算方法
链接：https://arxiv.org/abs/2507.10564
摘要：我们考虑的问题，工具到工具的匹配（TTTM），也称为腔匹配的上下文中的半导体制造设备。传统的TTTM方法利用静态配置数据或依赖于在商业生产线中难以获得的黄金参考。此外，现有的方法不能很好地扩展到异构环境，其中设备具有不同的品牌和型号，来源于不同的设备供应商。我们提出了新的TTTM分析管道，以克服这些问题。我们假设不匹配的设备将具有更高的方差和/或数据中的模式数量更高。我们最好的单变量方法实现了相关系数>0.95和>0.5的方差和模式的数量，分别表明所提出的方法是有效的。此外，最好的多变量方法与表现最好的单变量方法的相关系数>0.75，表明其有效性。最后，我们分析了多元算法对算法超参数的敏感性。
摘要：We consider the problem of tool-to-tool matching (TTTM), also called, chamber matching in the context of a semiconductor manufacturing equipment. Traditional TTTM approaches utilize static configuration data or depend on a golden reference which are difficult to obtain in a commercial manufacturing line. Further, existing methods do not extend very well to a heterogeneous setting, where equipment are of different make-and-model, sourced from different equipment vendors. We propose novel TTTM analysis pipelines to overcome these issues. We hypothesize that a mismatched equipment would have higher variance and/or higher number of modes in the data. Our best univariate method achieves a correlation coefficient >0.95 and >0.5 with the variance and number of modes, respectively showing that the proposed methods are effective. Also, the best multivariate method achieves a correlation coefficient >0.75 with the top-performing univariate methods, showing its effectiveness. Finally, we analyze the sensitivity of the multivariate algorithms to the algorithm hyper-parameters.
【8】Recent Advances in Simulation-based Inference for Gravitational Wave Data Analysis
标题：引力波数据分析中的模拟推理研究进展
链接：https://arxiv.org/abs/2507.11192
备注：30 pages, 6 figures, 1 table. Published version accepted by Astronomical Techniques and Instruments (ATI)
摘要：LIGO-Virgo-KAGRA合作对引力波的探测开创了观测天文学的新时代，强调了快速和详细的参数估计和人口水平分析的必要性。传统的贝叶斯推理方法，特别是马尔可夫链蒙特卡罗，在处理引力波数据中固有的高维参数空间和复杂的噪声特性时，面临着重大的计算挑战。本文探讨了基于模拟的推理方法在引力波天文学中的新兴作用，重点关注利用机器学习技术的方法，如归一化流和神经后验估计。我们提供了各种基于模拟的推理方法，包括神经后验估计，神经比率估计，神经似然估计，流匹配和一致性模型的理论基础的全面概述。我们探讨了这些方法在不同引力波数据处理场景中的应用，从单源参数估计和重叠信号分析到广义相对论测试和人口研究。虽然这些技术在对照研究中比传统方法显示出速度上的改进，但它们依赖于模型的性质和对先前假设的敏感性是它们广泛采用的障碍。其准确性，这是类似于传统的方法，需要进一步验证更广泛的参数空间和噪声条件。
摘要：The detection of gravitational waves by the LIGO-Virgo-KAGRA collaboration has ushered in a new era of observational astronomy, emphasizing the need for rapid and detailed parameter estimation and population-level analyses. Traditional Bayesian inference methods, particularly Markov chain Monte Carlo, face significant computational challenges when dealing with the high-dimensional parameter spaces and complex noise characteristics inherent in gravitational wave data. This review examines the emerging role of simulation-based inference methods in gravitational wave astronomy, with a focus on approaches that leverage machine-learning techniques such as normalizing flows and neural posterior estimation. We provide a comprehensive overview of the theoretical foundations underlying various simulation-based inference methods, including neural posterior estimation, neural ratio estimation, neural likelihood estimation, flow matching, and consistency models. We explore the applications of these methods across diverse gravitational wave data processing scenarios, from single-source parameter estimation and overlapping signal analysis to testing general relativity and conducting population studies. Although these techniques demonstrate speed improvements over traditional methods in controlled studies, their model-dependent nature and sensitivity to prior assumptions are barriers to their widespread adoption. Their accuracy, which is similar to that of conventional methods, requires further validation across broader parameter spaces and noise conditions.
检测相关(3篇)
【1】RMAU-NET: A Residual-Multihead-Attention U-Net Architecture for Landslide Segmentation and Detection from Remote Sensing Images
标题：DMAU-NET：一种用于遥感图像滑坡分割和检测的剩余多头注意力U-Net架构
链接：https://arxiv.org/abs/2507.11143
摘要：近年来，由于干旱、洪水、风暴等极端天气事件或人类活动的后果，如森林砍伐、过度开发自然资源等，滑坡灾害屡有报道。然而，自动监测滑坡是具有挑战性的，由于极大的观测面积和崎岖的地形，如山区或高原。这促使我们提出了一个端到端的基于深度学习的模型，该模型探索了用于自动观测滑坡事件的遥感图像。以遥感影像为输入数据，可以获得免费的资源，对大面积、粗糙的地形进行分时观测。为了探索遥感图像，我们提出了一种新的神经网络结构，这是滑坡检测和滑坡分割两个任务。我们在LandSlide 4Sense、毕节和尼泊尔三个不同的基准数据集上评估了我们提出的模型。通过大量的实验，我们在LandSlide 4Sense，毕节数据集上实现了滑坡检测任务的F1得分为98.23，93.83;在LandSlide 4Sense，尼泊尔数据集上实现了分割任务的mIoU得分为63.74，76.88。这些实验结果证明了将我们提出的模型集成到现实生活中的滑坡观测系统的潜力。
摘要：In recent years, landslide disasters have reported frequently due to the extreme weather events of droughts, floods , storms, or the consequence of human activities such as deforestation, excessive exploitation of natural resources. However, automatically observing landslide is challenging due to the extremely large observing area and the rugged topography such as mountain or highland. This motivates us to propose an end-to-end deep-learning-based model which explores the remote sensing images for automatically observing landslide events. By considering remote sensing images as the input data, we can obtain free resource, observe large and rough terrains by time. To explore the remote sensing images, we proposed a novel neural network architecture which is for two tasks of landslide detection and landslide segmentation. We evaluated our proposed model on three different benchmark datasets of LandSlide4Sense, Bijie, and Nepal. By conducting extensive experiments, we achieve F1 scores of 98.23, 93.83 for the landslide detection task on LandSlide4Sense, Bijie datasets; mIoU scores of 63.74, 76.88 on the segmentation tasks regarding LandSlide4Sense, Nepal datasets. These experimental results prove potential to integrate our proposed model into real-life landslide observation systems.
【2】Multilayer Artificial Benchmark for Community Detection (mABCD)
标题：多层人工社区检测基准（mABCD）
链接：https://arxiv.org/abs/2507.10795
备注：28 pages, 15 figures, 7 tables
摘要：人工社区检测基准（ABCD）模型是一个具有社区结构的随机图模型，其度和社区大小均服从幂律分布。该模型生成的图形类似于著名的LFR模型，但它更快，更易于解释，并且可以分析研究。在本文中，我们使用ABCD模型的基本成分，并介绍其多层网络的变体mABCD。
摘要：The Artificial Benchmark for Community Detection (ABCD) model is a random graph model with community structure and power-law distribution for both degrees and community sizes. The model generates graphs similar to the well-known LFR model but it is faster, more interpretable, and can be investigated analytically. In this paper, we use the underlying ingredients of the ABCD model and introduce its variant for multilayer networks, mABCD.
【3】First-of-its-kind AI model for bioacoustic detection using a lightweight associative memory Hopfield neural network
标题：使用轻量级联想记忆Hopfield神经网络的首创生物声学检测人工智能模型
链接：https://arxiv.org/abs/2507.10642
备注：12 pages, 5 figures
摘要：保护生物声学中一个日益增长的问题是分析使用被动声学监测设备产生的大量数据的任务。在本文中，我们提出了一种替代AI模型，它有可能帮助缓解这个问题。我们的模型制定解决了使用当前人工智能模型进行生物声学分析时遇到的关键问题，即：有限的训练数据;环境影响，特别是在训练和实施这些模型的能源消耗和碳足迹方面;以及相关的硬件要求。在这项工作中开发的模型使用联想记忆通过一个透明的，可解释的Hopfield神经网络来存储信号和检测类似的信号，然后可以用来分类物种。训练是快速的（3.5毫秒），因为数据集中的每个目标声音只需要一个代表性信号。该模型速度很快，在标准的Apple MacBook Air上，仅需5.4美元就可以对所有10384美元的公开蝙蝠录音进行预处理和分类。该模型也是轻量级的，内存占用量小，仅为144.09MB。因此，较低的计算需求使该模型非常适合在各种标准个人设备上使用，并有可能通过边缘处理设备在现场部署。它也具有竞争力的准确性，在用于评估模型的数据集上具有高达86\%$的精度。事实上，我们找不到模型和通过专家现场指导进行手动识别之间存在分歧的任何情况。虽然选择了蝙蝠回声定位呼叫的数据集来演示这种第一种人工智能模型，只训练了两种代表性的呼叫，但该模型并不是物种特定的。总之，我们提出了一个公平的人工智能模型，它有可能成为快速，轻量级，可持续，透明，可解释和准确的生物声学分析的游戏规则改变者。
摘要：A growing issue within conservation bioacoustics is the task of analysing the vast amount of data generated from the use of passive acoustic monitoring devices. In this paper, we present an alternative AI model which has the potential to help alleviate this problem. Our model formulation addresses the key issues encountered when using current AI models for bioacoustic analysis, namely the: limited training data available; environmental impact, particularly in energy consumption and carbon footprint of training and implementing these models; and associated hardware requirements. The model developed in this work uses associative memory via a transparent, explainable Hopfield neural network to store signals and detect similar signals which can then be used to classify species. Training is rapid ($3$\,ms), as only one representative signal is required for each target sound within a dataset. The model is fast, taking only $5.4$\,s to pre-process and classify all $10384$ publicly available bat recordings, on a standard Apple MacBook Air. The model is also lightweight with a small memory footprint of $144.09$\,MB of RAM usage. Hence, the low computational demands make the model ideal for use on a variety of standard personal devices with potential for deployment in the field via edge-processing devices. It is also competitively accurate, with up to $86\%$ precision on the dataset used to evaluate the model. In fact, we could not find a single case of disagreement between model and manual identification via expert field guides. Although a dataset of bat echolocation calls was chosen to demo this first-of-its-kind AI model, trained on only two representative calls, the model is not species specific. In conclusion, we propose an equitable AI model that has the potential to be a game changer for fast, lightweight, sustainable, transparent, explainable and accurate bioacoustic analysis.
分类|识别(4篇)
【1】Toward Improving fNIRS Classification: A Study on Activation Functions in Deep Neural Architectures
标题：改进fNIRS分类：深度神经架构中激活功能的研究
链接：https://arxiv.org/abs/2507.11436
摘要：激活函数对于深度神经网络的性能至关重要，特别是在功能性近红外光谱（fNIRS）等领域，其中非线性，低信噪比（SNR）和信号可变性对模型精度构成了重大挑战。然而，激活函数对fNIRS域中深度学习（DL）性能的影响仍然没有得到充分研究，并且在当前文献中缺乏系统的研究。本研究使用多个深度学习架构评估了一系列用于fNIRS分类任务的常规和特定领域激活函数，包括特定领域的fNIRSNet，AbsoluteNet，MDNN和shallowConvNet（作为基线），所有这些都在听觉任务期间记录的单个数据集上进行了测试。为了确保公平的比较，所有网络都使用标准化的预处理和一致的训练参数进行训练和测试。结果表明，Tanh和绝对值函数Abs（x）等对称激活函数的性能优于整流线性单元（ReLU）等常用函数，具体取决于体系结构。此外，使用修正绝对函数（MAF）对对称性的作用进行了重点分析，结果进一步支持了对称激活函数对性能增益的有效性。这些发现强调了选择与fNIRS数据信号特征一致的适当激活函数的重要性。
摘要 ：Activation functions are critical to the performance of deep neural networks, particularly in domains such as functional near-infrared spectroscopy (fNIRS), where nonlinearity, low signal-to-noise ratio (SNR), and signal variability poses significant challenges to model accuracy. However, the impact of activation functions on deep learning (DL) performance in the fNIRS domain remains underexplored and lacks systematic investigation in the current literature. This study evaluates a range of conventional and field-specific activation functions for fNIRS classification tasks using multiple deep learning architectures, including the domain-specific fNIRSNet, AbsoluteNet, MDNN, and shallowConvNet (as the baseline), all tested on a single dataset recorded during an auditory task. To ensure fair a comparison, all networks were trained and tested using standardized preprocessing and consistent training parameters. The results show that symmetrical activation functions such as Tanh and the Absolute value function Abs(x) can outperform commonly used functions like the Rectified Linear Unit (ReLU), depending on the architecture. Additionally, a focused analysis of the role of symmetry was conducted using a Modified Absolute Function (MAF), with results further supporting the effectiveness of symmetrical activation functions on performance gains. These findings underscore the importance of selecting proper activation functions that align with the signal characteristics of fNIRS data.
【2】Commuting Distance Regularization for Timescale-Dependent Label Inconsistency in EEG Emotion Recognition
标题：脑电情绪识别中时间尺度相关标签不一致的通勤距离正规化
链接：https://arxiv.org/abs/2507.10895
摘要：在这项工作中，我们解决了在训练基于EEG的人类情感识别神经网络模型时经常被忽视的时间尺度依赖标签不一致性（TsDLI）问题。为了减轻TsDLI并增强模型的泛化性和可解释性，我们提出了两种新的正则化策略：局部变化损失（LVL）和局部-全局一致性损失（LGCL）。这两种方法都结合了经典的数学原理-特别是有界变差和通勤时间距离的函数-在图论框架内。补充我们的正则化器，我们引入了一套新的评估指标，更好地捕捉时间本地预测和其相关的全球情感标签之间的对齐。我们通过对两个广泛使用的EEG情感数据集DREAMER和DEAP进行全面的实验来验证我们的方法，这些数据集跨越了一系列神经架构，包括LSTM和基于transformer的模型。绩效评估使用五个不同的指标，包括定量准确性和定性一致性。结果一致表明，我们提出的方法优于最先进的基线，提供卓越的综合性能，并在标签不一致的情况下提供可解释性和预测能力之间的原则性权衡。值得注意的是，LVL在所有基准主干和指标中获得了最佳的综合排名，而LGCL经常排名第二，突出了我们框架的有效性。
摘要：In this work, we address the often-overlooked issue of Timescale Dependent Label Inconsistency (TsDLI) in training neural network models for EEG-based human emotion recognition. To mitigate TsDLI and enhance model generalization and explainability, we propose two novel regularization strategies: Local Variation Loss (LVL) and Local-Global Consistency Loss (LGCL). Both methods incorporate classical mathematical principles--specifically, functions of bounded variation and commute-time distances--within a graph theoretic framework. Complementing our regularizers, we introduce a suite of new evaluation metrics that better capture the alignment between temporally local predictions and their associated global emotion labels. We validate our approach through comprehensive experiments on two widely used EEG emotion datasets, DREAMER and DEAP, across a range of neural architectures including LSTM and transformer-based models. Performance is assessed using five distinct metrics encompassing both quantitative accuracy and qualitative consistency. Results consistently show that our proposed methods outperform state-of-the-art baselines, delivering superior aggregate performance and offering a principled trade-off between interpretability and predictive power under label inconsistency. Notably, LVL achieves the best aggregate rank across all benchmarked backbones and metrics, while LGCL frequently ranks the second, highlighting the effectiveness of our framework.
【3】Supporting SENĆOTEN Language Documentation Efforts with Automatic Speech Recognition
标题：通过自动语音识别支持SENSYS OTON语言文档工作
链接：https://arxiv.org/abs/2507.10827
备注：Accepted by ComputEL-8
摘要：在温哥华岛南部的萨尼奇半岛上使用的SEN OTEN语言，正处于积极的语言复兴努力之中，以扭转由于殖民语言政策而导致的语言丧失的趋势。为了支持这些实地工作，社区正在转向数字技术。自动语音识别（ASR）技术在加速语言文档和创建教育资源方面具有很大的潜力。然而，由于有限的数据和其多合成结构和应力驱动的复分解的显著词汇变化，为SEN OTEN开发ASR系统具有挑战性。为了应对这些挑战，我们提出了一个ASR驱动的文档管道，该管道利用来自文本到语音（TTS）系统的增强语音数据和语音基础模型（SFM）的跨语言迁移学习。一个n-gram语言模型也被纳入通过浅融合或n-best恢复，以最大限度地利用现有的数据。在SEN OTEN数据集上的实验表明，在测试集上，单词错误率（WER）为19.34%，字符错误率（CER）为5.09%，词汇表外（OOV）率为57.02%。在过滤了与cedilla相关的小错误后，WER提高到14.32%（未看到的单词为26.48%），CER提高到3.45%，这表明我们的ASR驱动管道支持SEN OTEN语言文档的潜力。
摘要：The SEN\'{C}OTEN language, spoken on the Saanich peninsula of southern Vancouver Island, is in the midst of vigorous language revitalization efforts to turn the tide of language loss as a result of colonial language policies. To support these on-the-ground efforts, the community is turning to digital technology. Automatic Speech Recognition (ASR) technology holds great promise for accelerating language documentation and the creation of educational resources. However, developing ASR systems for SEN\'{C}OTEN is challenging due to limited data and significant vocabulary variation from its polysynthetic structure and stress-driven metathesis. To address these challenges, we propose an ASR-driven documentation pipeline that leverages augmented speech data from a text-to-speech (TTS) system and cross-lingual transfer learning with Speech Foundation Models (SFMs). An n-gram language model is also incorporated via shallow fusion or n-best restoring to maximize the use of available data. Experiments on the SEN\'{C}OTEN dataset show a word error rate (WER) of 19.34% and a character error rate (CER) of 5.09% on the test set with a 57.02% out-of-vocabulary (OOV) rate. After filtering minor cedilla-related errors, WER improves to 14.32% (26.48% on unseen words) and CER to 3.45%, demonstrating the potential of our ASR-driven pipeline to support SEN\'{C}OTEN language documentation.
【4】Canonical Bayesian Linear System Identification
标题：典型贝叶斯线性系统辨识
链接：https://arxiv.org/abs/2507.11535
备注：46 pages, 9 figures
摘要：线性时不变（LTI）系统辨识的标准贝叶斯方法受到参数不可识别性的阻碍;由此产生的复杂的多模态后验使推理效率低下且不切实际。我们通过在贝叶斯框架内嵌入LTI系统的规范形式来解决这个问题。我们严格地建立了这些最小参数化中的推断完全捕获了所有不变的系统动态（例如，传递函数、特征值、系统输出的预测分布），同时解决可识别性。这种方法解锁了有意义的、结构感知的先验的使用（例如，通过特征值强制稳定），并确保伯恩斯坦-冯米塞斯定理的条件-贝叶斯和频率主义大样本渐近之间的联系，在标准形式中被打破。现代MCMC方法的广泛模拟突出了标准参数化的优势：规范形式实现了更高的计算效率，生成可解释和表现良好的后验，并提供了强大的不确定性估计，特别是从有限的数据。
摘要 ：Standard Bayesian approaches for linear time-invariant (LTI) system identification are hindered by parameter non-identifiability; the resulting complex, multi-modal posteriors make inference inefficient and impractical. We solve this problem by embedding canonical forms of LTI systems within the Bayesian framework. We rigorously establish that inference in these minimal parameterizations fully captures all invariant system dynamics (e.g., transfer functions, eigenvalues, predictive distributions of system outputs) while resolving identifiability. This approach unlocks the use of meaningful, structure-aware priors (e.g., enforcing stability via eigenvalues) and ensures conditions for a Bernstein--von Mises theorem -- a link between Bayesian and frequentist large-sample asymptotics that is broken in standard forms. Extensive simulations with modern MCMC methods highlight advantages over standard parameterizations: canonical forms achieve higher computational efficiency, generate interpretable and well-behaved posteriors, and provide robust uncertainty estimates, particularly from limited data.
优化|敛散性(5篇)
【1】Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
标题：光滑内插体制下的BCD快速最后迭代收敛
链接：https://arxiv.org/abs/2507.11274
备注：27 pages
摘要：我们研究人口收敛保证随机梯度下降（SGD）光滑凸目标的插值制度，在最佳的噪声是零或接近零。在这种情况下，SGD的最后一个周期的行为-特别是具有大（恒定）步长-近年来受到越来越多的关注，这是由于过度参数化模型的训练，以及分析持续学习中的遗忘和理解用于求解线性系统的随机Kaczmarz方法的收敛性。我们建立了在步长为$\eta \leq 1/\beta$的$\beta$光滑凸损失函数上进行SGD的$T$步后，最后一步的随机梯度具有期望超额风险$\widetilde{O}（1/（\eta T^{1-\beta\eta/2}）+ \eta T^{\beta\eta/2} \sigma_\star^2）$，其中$\sigma_\star^2$表示最优随机梯度的方差。特别是，对于经过良好调整的步长，我们获得了接近最佳的$\widetilde{O}（1/T + \sigma_\star/\sqrt{T}）$率，将Varre等人的结果（2021）扩展到最小二乘回归之外;当$\sigma_\star=0$时，我们得到的速率为$O（1/\sqrt{T}）$，其中$\eta=1/\beta$，改进了Evron et al.（2025）在可实现线性回归的特殊情况下最近建立的最著名的$O（T^{-1/4}）$率。
摘要：We study population convergence guarantees of stochastic gradient descent (SGD) for smooth convex objectives in the interpolation regime, where the noise at optimum is zero or near zero. The behavior of the last iterate of SGD in this setting -- particularly with large (constant) stepsizes -- has received growing attention in recent years due to implications for the training of over-parameterized models, as well as to analyzing forgetting in continual learning and to understanding the convergence of the randomized Kaczmarz method for solving linear systems. We establish that after $T$ steps of SGD on $\beta$-smooth convex loss functions with stepsize $\eta \leq 1/\beta$, the last iterate exhibits expected excess risk $\widetilde{O}(1/(\eta T^{1-\beta\eta/2}) + \eta T^{\beta\eta/2} \sigma_\star^2)$, where $\sigma_\star^2$ denotes the variance of the stochastic gradients at the optimum. In particular, for a well-tuned stepsize we obtain a near optimal $\widetilde{O}(1/T + \sigma_\star/\sqrt{T})$ rate for the last iterate, extending the results of Varre et al. (2021) beyond least squares regression; and when $\sigma_\star=0$ we obtain a rate of $O(1/\sqrt{T})$ with $\eta=1/\beta$, improving upon the best-known $O(T^{-1/4})$ rate recently established by Evron et al. (2025) in the special case of realizable linear regression.
【2】LyAm: Robust Non-Convex Optimization for Stable Learning in Noisy Environments
标题：LyAm：用于在噪音环境中稳定学习的鲁棒非凸优化
链接：https://arxiv.org/abs/2507.11262
摘要：训练深度神经网络，特别是在计算机视觉任务中，通常会受到噪声梯度和不稳定收敛的影响，这会阻碍性能和泛化。在本文中，我们提出了LyAm，一种新的优化器，集成亚当的自适应矩估计与基于Lyapunov的稳定机制。LyAm利用李亚普诺夫稳定性理论动态调整学习速率，以增强收敛鲁棒性并减轻训练噪声。我们提供了严格的理论框架，证明LyAm在复杂的非凸环境中的收敛保证。在CIFAR-10和CIFAR-100上进行的大量实验表明，LyAm在准确性、收敛速度和稳定性方面始终优于最先进的优化器，使其成为强大的深度学习优化的有力候选者。
摘要：Training deep neural networks, particularly in computer vision tasks, often suffers from noisy gradients and unstable convergence, which hinder performance and generalization. In this paper, we propose LyAm, a novel optimizer that integrates Adam's adaptive moment estimation with Lyapunov-based stability mechanisms. LyAm dynamically adjusts the learning rate using Lyapunov stability theory to enhance convergence robustness and mitigate training noise. We provide a rigorous theoretical framework proving the convergence guarantees of LyAm in complex, non-convex settings. Extensive experiments on like as CIFAR-10 and CIFAR-100 show that LyAm consistently outperforms state-of-the-art optimizers in terms of accuracy, convergence speed, and stability, establishing it as a strong candidate for robust deep learning optimization.
【3】Relative Entropy Pathwise Policy Optimization
标题：相对熵路径策略优化
链接：https://arxiv.org/abs/2507.11019
摘要：分数函数策略梯度在游戏、机器人和语言模型微调方面取得了很好的效果。然而，它的高方差往往会破坏训练的稳定性。另一方面，路径策略梯度减轻了训练方差，但仅当由精确的动作条件值函数驱动时才是可靠的，众所周知，在不依赖于过去的非策略数据的情况下，该精确的动作条件值函数很难进行训练。在本文中，我们讨论了如何构建一个值梯度驱动的on-policy算法，该算法允许纯粹从on-policy数据训练Q值模型，从而在on-policy学习的背景下释放使用路径策略更新的可能性。我们展示了如何平衡随机策略的探索与稳定训练的约束策略更新，并评估促进准确值函数学习的重要架构组件。基于这些见解，我们提出了相对熵路径策略优化（REPPO），这是一种有效的策略算法，它将路径策略梯度的样本效率与标准策略学习的简单性和最小内存占用相结合。我们证明，REPPO提供了强大的经验性能，在减少样本要求，挂钟时间，内存占用以及高超参数鲁棒性在一组实验中的两个标准的GPU并行化的基准。
摘要：Score-function policy gradients have delivered strong results in game-playing, robotics and language-model fine-tuning. Yet its high-variance often undermines training stability. On the other hand, pathwise policy gradients alleviate the training variance, but are reliable only when driven by an accurate action-conditioned value function which is notoriously hard to train without relying on past off-policy data. In this paper, we discuss how to construct a value-gradient driven, on-policy algorithm that allow training Q-value models purely from on-policy data, unlocking the possibility of using pathwise policy updates in the context of on-policy learning. We show how to balance stochastic policies for exploration with constrained policy updates for stable training, and evaluate important architectural components that facilitate accurate value function learning. Building on these insights, we propose Relative Entropy Pathwise Policy Optimization (REPPO), an efficient on-policy algorithm that combines the sample-efficiency of pathwise policy gradients with the simplicity and minimal memory footprint of standard on-policy learning. We demonstrate that REPPO provides strong empirical performance at decreased sample requirements, wall-clock time, memory footprint as well as high hyperparameter robustness in a set of experiments on two standard GPU-parallelized benchmarks.
【4】A Mathematical Optimization Approach to Multisphere Support Vector Data Description
标题：多球体支持量数据描述的数学优化方法
链接：https://arxiv.org/abs/2507.11106
备注：18 pages, 5 figures, 3 tables
摘要：我们提出了一种新的数学优化框架，在多模态数据集离群检测，扩展支持向量数据描述方法。我们提供了一个原始的配方，在一个混合的二阶锥模型的形状，构造欧氏超球面，以确定异常的意见。在此基础上，我们开发了一个双重模型，使内核技巧的应用程序，从而允许检测复杂的非线性数据结构中的离群值。一个广泛的计算研究表明，我们的精确方法的有效性，显示出明显的优势，现有的启发式技术的准确性和鲁棒性。
摘要 ：We present a novel mathematical optimization framework for outlier detection in multimodal datasets, extending Support Vector Data Description approaches. We provide a primal formulation, in the shape of a Mixed Integer Second Order Cone model, that constructs Euclidean hyperspheres to identify anomalous observations. Building on this, we develop a dual model that enables the application of the kernel trick, thus allowing for the detection of outliers within complex, non-linear data structures. An extensive computational study demonstrates the effectiveness of our exact method, showing clear advantages over existing heuristic techniques in terms of accuracy and robustness.
【5】Functional Neural Wavefunction Optimization
标题：功能性神经波函数优化
链接：https://arxiv.org/abs/2507.10835
摘要：我们提出了一个框架，在变分量子蒙特卡罗优化算法的设计和分析，借鉴相应的函数空间的几何见解。该框架将无限维优化动力学转化为易于处理的参数空间算法，通过Galerkin投影到变分矩阵的切线空间。这种观点统一了现有的方法，如随机重新配置和瑞利-高斯-牛顿，提供连接到经典的函数空间算法，并激发了新的算法与几何原则的超参数选择的推导。我们验证了我们的框架与数值实验证明其实际意义，通过精确估计的基态能量的几个原型模型在凝聚态物理建模与神经网络波函数。
摘要：We propose a framework for the design and analysis of optimization algorithms in variational quantum Monte Carlo, drawing on geometric insights into the corresponding function space. The framework translates infinite-dimensional optimization dynamics into tractable parameter-space algorithms through a Galerkin projection onto the tangent space of the variational ansatz. This perspective unifies existing methods such as stochastic reconfiguration and Rayleigh-Gauss-Newton, provides connections to classic function-space algorithms, and motivates the derivation of novel algorithms with geometrically principled hyperparameter choices. We validate our framework with numerical experiments demonstrating its practical relevance through the accurate estimation of ground-state energies for several prototypical models in condensed matter physics modeled with neural network wavefunctions.
预测|估计(6篇)
【1】Data Augmentation in Time Series Forecasting through Inverted Framework
标题：利用倒置框架进行时间序列预测中的数据扩充
链接：https://arxiv.org/abs/2507.11439
摘要：目前，iTransformer是多变量时间序列（MTS）预测中最流行和最有效的模型之一。得益于其倒置框架，iTransformer有效地捕捉多元相关性。然而，倒置框架仍有一定的局限性。它减少了时间的相互依赖性信息，并在不显著的变量相关性的情况下引入噪声。为了解决这些限制，我们引入了一种新的数据增强方法，称为DAIF。与以前的数据增强方法不同，DAIF是第一个专门为MTS预测中的倒置框架设计的实时增强方法。我们首先定义了反向序列到序列框架的结构，然后提出了两种不同的DAIF策略，频率过滤和交叉变异修补，以解决现有的反向框架的挑战。在多个数据集和反演模型上的实验证明了DAIF的有效性。
摘要：Currently, iTransformer is one of the most popular and effective models for multivariate time series (MTS) forecasting. Thanks to its inverted framework, iTransformer effectively captures multivariate correlation. However, the inverted framework still has some limitations. It diminishes temporal interdependency information, and introduces noise in cases of nonsignificant variable correlation. To address these limitations, we introduce a novel data augmentation method on inverted framework, called DAIF. Unlike previous data augmentation methods, DAIF stands out as the first real-time augmentation specifically designed for the inverted framework in MTS forecasting. We first define the structure of the inverted sequence-to-sequence framework, then propose two different DAIF strategies, Frequency Filtering and Cross-variation Patching to address the existing challenges of the inverted framework. Experiments across multiple datasets and inverted models have demonstrated the effectiveness of our DAIF.
【2】Generative Click-through Rate Prediction with Applications to Search Advertising
标题：生成式点击率预测及其在搜索广告中的应用
链接：https://arxiv.org/abs/2507.11246
备注：This work was first submitted on February 9, 2024
摘要：点击率（CTR）预测模型是众多工业环境中不可或缺的一部分，例如个性化搜索广告。目前的方法通常涉及从用户的历史行为序列中提取特征，并结合产品信息，输入到根据用户反馈训练的判别模型中，以估计CTR。随着GPT等模型的成功，生成模型在丰富判别模型之外的表达能力方面的潜力已经变得显而易见。鉴于此，我们引入了一种新的模型，该模型利用生成模型来提高判别模型中CTR预测的精度。为了协调这两种模型类型的不同数据聚合需求，我们设计了一个两阶段的训练过程：1）使用用户行为序列中的给定项目类别对下一个项目预测进行生成预训练; 2）在区分性CTR预测框架内微调训练有素的生成模型。我们的方法的有效性是通过在一个新的数据集上进行广泛的实验来证实的，其重要的实用性进一步得到了在线A/B测试结果的证实。目前，该模型部署在世界上最大的电子商务平台之一，我们打算在未来发布相关的代码和数据集。
摘要：Click-Through Rate (CTR) prediction models are integral to a myriad of industrial settings, such as personalized search advertising. Current methods typically involve feature extraction from users' historical behavior sequences combined with product information, feeding into a discriminative model that is trained on user feedback to estimate CTR. With the success of models such as GPT, the potential for generative models to enrich expressive power beyond discriminative models has become apparent. In light of this, we introduce a novel model that leverages generative models to enhance the precision of CTR predictions in discriminative models. To reconcile the disparate data aggregation needs of both model types, we design a two-stage training process: 1) Generative pre-training for next-item prediction with the given item category in user behavior sequences; 2) Fine-tuning the well-trained generative model within a discriminative CTR prediction framework. Our method's efficacy is substantiated through extensive experiments on a new dataset, and its significant utility is further corroborated by online A/B testing results. Currently, the model is deployed on one of the world's largest e-commerce platforms, and we intend to release the associated code and dataset in the future.
【3】Improving Wi-Fi Network Performance Prediction with Deep Learning Models
标题：利用深度学习模型改进Wi-Fi网络性能预测
链接：https://arxiv.org/abs/2507.11168
备注：preprint accepted, 8 pages, 2025
摘要：工业和关键任务应用对无线网络的鲁棒性、可靠性和确定性的需求日益增长，这是新的创新方法增长的驱动力。这项工作中提出的研究利用机器学习技术来预测Wi-Fi网络中的帧传输率的信道质量。预测可用于主动调整运行时的通信参数，并优化工业应用的网络操作。包括卷积神经网络和长短期记忆在内的方法在从多个通道的真实Wi-Fi设置中获取的数据集上进行了分析。在预测精度和计算复杂度方面对模型进行了比较。结果表明，可以可靠地预测帧传输率，卷积神经网络虽然比其他模型的效率略低，但在CPU使用率和内存消耗方面更有效。这增强了模型在嵌入式和工业系统上的可用性。
摘要 ：The increasing need for robustness, reliability, and determinism in wireless networks for industrial and mission-critical applications is the driver for the growth of new innovative methods. The study presented in this work makes use of machine learning techniques to predict channel quality in a Wi-Fi network in terms of the frame delivery ratio. Predictions can be used proactively to adjust communication parameters at runtime and optimize network operations for industrial applications. Methods including convolutional neural networks and long short-term memory were analyzed on datasets acquired from a real Wi-Fi setup across multiple channels. The models were compared in terms of prediction accuracy and computational complexity. Results show that the frame delivery ratio can be reliably predicted, and convolutional neural networks, although slightly less effective than other models, are more efficient in terms of CPU usage and memory consumption. This enhances the model's usability on embedded and industrial systems.
【4】StellarF: A Lora-Adapter Integrated Large Model Framework for Stellar Flare Forecasting with Historical & Statistical Data
标题：StellarF：Lora-Adaptor集成大型模型框架，用于利用历史和统计数据进行恒星耀斑预测
链接：https://arxiv.org/abs/2507.10986
摘要：恒星耀斑预测是天文学的一个重要研究前沿，它为恒星活动提供了深刻的见解。然而，该领域受到两个记录的耀斑事件的稀疏性和缺乏特定领域的大规模预测模型。为了应对这些挑战，本研究引入了StellarF（恒星耀斑预测），这是一种新型的大型模型，利用低秩（LoRA）和适配器技术进行参数有效学习，用于恒星耀斑预测。StellarF的核心是将耀斑统计信息模块与历史耀斑记录模块集成在一起，从而能够从观测数据中进行多尺度模式识别。在我们自己构建的数据集（来自开普勒和TESS光变曲线）上进行的大量实验表明，与现有方法相比，StellarF实现了最先进的性能。提出的预测范式建立了一个新的方法框架，推进天体物理研究和跨学科的应用。
摘要：Stellar flare forecasting, a critical research frontier in astronomy, offers profound insights into stellar activity. However, the field is constrained by both the sparsity of recorded flare events and the absence of domain-specific large-scale predictive models. To address these challenges, this study introduces StellarF (Stellar Flare Forecasting), a novel large model that leverages Low-Rank (LoRA) and Adapter techniques to parameter-efficient learning for stellar flare forecasting. At its core, StellarF integrates an flare statistical information module with a historical flare record module, enabling multi-scale pattern recognition from observational data. Extensive experiments on our self-constructed datasets (derived from Kepler and TESS light curves) demonstrate that StellarF achieves state-of-the-art performance compared to existing methods. The proposed prediction paradigm establishes a novel methodological framework for advancing astrophysical research and cross-disciplinary applications.
【5】Modernizing CNN-based Weather Forecast Model towards Higher Computational Efficiency
标题：基于CNN的天气预报模型现代化以提高计算效率
链接：https://arxiv.org/abs/2507.10893
备注：26pages, 9 Figures
摘要：最近，基于人工智能的天气预报模型取得了令人印象深刻的进展。这些模型达到了与传统NWP系统相当的精度水平，标志着数据驱动天气预报的重要里程碑。然而，它们大多利用基于transformer的架构，由于大量的参数大小，这通常会导致高训练复杂性和资源需求。在这项研究中，我们引入了一个现代化的基于CNN的全球天气预报模型，该模型提供了具有竞争力的准确性，同时大大降低了计算要求。为了呈现系统的现代化路线图，我们强调了早期基于CNN的方法在多个设计尺度上的关键架构增强。KAI-a将尺度不变的架构和基于InceptionNeXT的块结合在一个可感知的设计中，为地球系统数据的结构量身定制。该模型在包含67个大气变量的ERA 5每日数据集上进行训练，包含约700万个参数，并在单个NVIDIA L40 s GPU上仅需12小时即可完成训练。我们的评估表明，KAI-a在中期天气预报方面与最先进的模型的性能相匹配，同时提供了显着的轻量化设计。此外，对2018年欧洲热浪和东亚夏季风的案例研究证明了KAI-a在捕捉极端事件方面的强大技能，加强了其实用性。
摘要：Recently, AI-based weather forecast models have achieved impressive advances. These models have reached accuracy levels comparable to traditional NWP systems, marking a significant milestone in data-driven weather prediction. However, they mostly leverage Transformer-based architectures, which often leads to high training complexity and resource demands due to the massive parameter sizes. In this study, we introduce a modernized CNN-based model for global weather forecasting that delivers competitive accuracy while significantly reducing computational requirements. To present a systematic modernization roadmap, we highlight key architectural enhancements across multiple design scales from an earlier CNN-based approach. KAI-a incorporates a scale-invariant architecture and InceptionNeXt-based blocks within a geophysically-aware design, tailored to the structure of Earth system data. Trained on the ERA5 daily dataset with 67 atmospheric variables, the model contains about 7 million parameters and completes training in just 12 hours on a single NVIDIA L40s GPU. Our evaluation shows that KAI-a matches the performance of state-of-the-art models in medium-range weather forecasting, while offering a significantly lightweight design. Furthermore, case studies on the 2018 European heatwave and the East Asian summer monsoon demonstrate KAI-a's robust skill in capturing extreme events, reinforcing its practical utility.
【6】A Generalizable Physics-Enhanced State Space Model for Long-Term Dynamics Forecasting in Complex Environments
标题：复杂环境中长期动力学预测的一种广义物理增强状态空间模型
链接：https://arxiv.org/abs/2507.10792
备注：8 pages, 6 figures, accepted in ICML 2025
摘要：这项工作的目的是解决在复杂的环境中，数据是嘈杂的，不规则的采样的长期动态预测的问题。虽然最近的研究已经引入了一些方法来提高预测性能，但这些方法在处理这种复杂情景下的长期外推任务时仍然面临重大挑战。为了克服这一挑战，我们提出了Phy-SSM，这是一种可推广的方法，将部分物理知识集成到状态空间模型（SSM）中，用于复杂环境中的长期动态预测。我们的动机是，SSM可以有效地捕捉长期的依赖关系，在连续的数据和模型的动态系统，而物理知识的结合提高泛化能力。关键的挑战在于如何将部分已知的物理无缝地整合到SSM中。为了实现这一点，我们部分已知的系统动态分解成已知和未知的状态矩阵，这是集成到一个物理SSM单元。为了进一步提高长期预测性能，我们引入了一个物理状态正则化项，使估计的潜在状态与系统动力学相一致。此外，我们还从理论上分析了该方法解的唯一性。在三个实际应用中进行的广泛实验，包括车辆运动预测，无人机状态预测和COVID-19流行病学预测，证明了Phy-SSM在长期内插和外推任务中优于基线的性能。该代码可在https://github.com/511205787/Phy_SSM-ICML2025上获得。
摘要 ：This work aims to address the problem of long-term dynamic forecasting in complex environments where data are noisy and irregularly sampled. While recent studies have introduced some methods to improve prediction performance, these approaches still face a significant challenge in handling long-term extrapolation tasks under such complex scenarios. To overcome this challenge, we propose Phy-SSM, a generalizable method that integrates partial physics knowledge into state space models (SSMs) for long-term dynamics forecasting in complex environments. Our motivation is that SSMs can effectively capture long-range dependencies in sequential data and model continuous dynamical systems, while the incorporation of physics knowledge improves generalization ability. The key challenge lies in how to seamlessly incorporate partially known physics into SSMs. To achieve this, we decompose partially known system dynamics into known and unknown state matrices, which are integrated into a Phy-SSM unit. To further enhance long-term prediction performance, we introduce a physics state regularization term to make the estimated latent states align with system dynamics. Besides, we theoretically analyze the uniqueness of the solutions for our method. Extensive experiments on three real-world applications, including vehicle motion prediction, drone state prediction, and COVID-19 epidemiology forecasting, demonstrate the superior performance of Phy-SSM over the baselines in both long-term interpolation and extrapolation tasks. The code is available at https://github.com/511205787/Phy_SSM-ICML2025.
其他神经网络|深度学习|模型|建模(19篇)
【1】Langevin Flows for Modeling Neural Latent Dynamics
标题：神经系统潜在动力学的Langevin流模型
链接：https://arxiv.org/abs/2507.11531
备注：Full version of the Cognitive Computational Neuroscience (CCN) 2025 poster
摘要：神经群体表现出潜在的动力学结构，驱动随时间变化的尖峰活动，激发了对捕获内在网络动力学和外部未观察到的影响的模型的搜索。在这项工作中，我们介绍LangevinFlow，顺序变分自动编码器的潜变量的时间演变是由欠阻尼Langevin方程。我们的方法结合了物理先验-如惯性，阻尼，学习的潜在功能，和随机力-代表自主和非自主的过程中的神经系统。至关重要的是，潜在的功能被参数化为一个网络的本地耦合振荡器，偏置模型对生物神经种群中观察到的振荡和流动行为。我们的模型具有一个经常性的编码器，一个单层的Transformer解码器，和Langevin动力学的潜在空间。从经验上讲，我们的方法在由Lorenz吸引子生成的合成神经种群上的表现优于最先进的基线，与地面真实发射率密切匹配。在神经潜伏期基准测试（NLB）中，该模型在四个具有挑战性的数据集上实现了卓越的保持神经元似然性（每尖峰比特数）和前向预测准确性。它还匹配或超越解码行为指标（如手速）的替代方法。总的来说，这项工作引入了一个灵活的，物理启发的，高性能的框架，用于建模复杂的神经种群动态及其未观察到的影响。
摘要：Neural populations exhibit latent dynamical structures that drive time-evolving spiking activities, motivating the search for models that capture both intrinsic network dynamics and external unobserved influences. In this work, we introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation. Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and stochastic forces -- to represent both autonomous and non-autonomous processes in neural systems. Crucially, the potential function is parameterized as a network of locally coupled oscillators, biasing the model toward oscillatory and flow-like behaviors observed in biological neural populations. Our model features a recurrent encoder, a one-layer Transformer decoder, and Langevin dynamics in the latent space. Empirically, our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor, closely matching ground-truth firing rates. On the Neural Latents Benchmark (NLB), the model achieves superior held-out neuron likelihoods (bits per spike) and forward prediction accuracy across four challenging datasets. It also matches or surpasses alternative methods in decoding behavioral metrics such as hand velocity. Overall, this work introduces a flexible, physics-inspired, high-performing framework for modeling complex neural population dynamics and their unobserved influences.
【2】Elk: Exploring the Efficiency of Inter-core Connected AI Chips with Deep Learning Compiler Techniques
标题：Elk：利用深度学习调度器技术探索核间互联人工智能芯片的效率
链接：https://arxiv.org/abs/2507.11506
备注：This paper is accepted at the 58th IEEE/ACM International Symposium on Microarchitecture (MICRO'25)
摘要：为了满足深度学习（DL）模型日益增长的需求，AI芯片正在采用片外存储器（例如，HBM）和高带宽低延迟互连，用于直接核间数据交换。然而，由于计算（每核执行）、通信（核间数据交换）和I/O（片外数据访问）之间的基本斗争，探索这些核间连接AI（ICCA）芯片的效率并不容易。 在本文中，我们开发的麋鹿，DL编译器框架，以最大限度地提高ICCA芯片的效率，共同权衡所有三个性能因素上面讨论的。Elk将这些性能因素结构化为可配置的参数，并在DL编译器中形成全局权衡空间。为了系统地探索这个空间并最大限度地提高整体效率，Elk采用了一种新的归纳算子调度策略和一种成本感知的片上内存分配算法。它生成全局优化的执行计划，最好地重叠片外数据加载和片上执行。为了研究麋鹿的效率，我们建立了一个完整的仿真器的基础上，一个真正的ICCA芯片IPU-POD 4，和ICCA芯片模拟器的灵敏度分析与不同的互连网络拓扑结构。Elk平均达到了ICCA芯片理想屋顶性能的94%，显示了在ICCA芯片上支持大型DL模型的优势。我们还展示了Elk为新的ICCA芯片开发启用架构设计空间探索的能力。
摘要：To meet the increasing demand of deep learning (DL) models, AI chips are employing both off-chip memory (e.g., HBM) and high-bandwidth low-latency interconnect for direct inter-core data exchange. However, it is not easy to explore the efficiency of these inter-core connected AI (ICCA) chips, due to a fundamental tussle among compute (per-core execution), communication (inter-core data exchange), and I/O (off-chip data access). In this paper, we develop Elk, a DL compiler framework to maximize the efficiency of ICCA chips by jointly trading off all the three performance factors discussed above. Elk structures these performance factors into configurable parameters and forms a global trade-off space in the DL compiler. To systematically explore this space and maximize overall efficiency, Elk employs a new inductive operator scheduling policy and a cost-aware on-chip memory allocation algorithm. It generates globally optimized execution plans that best overlap off-chip data loading and on-chip execution. To examine the efficiency of Elk, we build a full-fledged emulator based on a real ICCA chip IPU-POD4, and an ICCA chip simulator for sensitivity analysis with different interconnect network topologies. Elk achieves 94% of the ideal roofline performance of ICCA chips on average, showing the benefits of supporting large DL models on ICCA chips. We also show Elk's capability of enabling architecture design space exploration for new ICCA chip development.
【3】Implementing Adaptations for Vision AutoRegressive Model
标题：实施视觉自回归模型的适应
链接：https://arxiv.org/abs/2507.11441
备注：Accepted at DIG-BUGS: Data in Generative Models Workshop @ ICML 2025
摘要：视觉自回归模型（VAR）是近年来在图像生成领域引入的一种扩散模型（DM）的替代模型。在这项工作中，我们专注于它的适应性，旨在微调预训练模型以执行特定的下游任务，如医疗数据生成。虽然对于DM存在许多技术，但VAR的适应性仍然未得到充分探索。同样，差异私人（DP）的适应，旨在保护隐私的适应数据，已被广泛研究DM，而VAR缺乏这样的解决方案。在我们的工作中，我们实施和基准的VAR的许多策略，并将它们与国家的最先进的DM适应策略。我们观察到，VAR优于DM的非DP适应，但是，DP的性能受到影响，这需要进一步研究在私人适应VAR。代码可从https://github.com/sprintml/finetuning_var_dp获得。
摘要：Vision AutoRegressive model (VAR) was recently introduced as an alternative to Diffusion Models (DMs) in image generation domain. In this work we focus on its adaptations, which aim to fine-tune pre-trained models to perform specific downstream tasks, like medical data generation. While for DMs there exist many techniques, adaptations for VAR remain underexplored. Similarly, differentially private (DP) adaptations-ones that aim to preserve privacy of the adaptation data-have been extensively studied for DMs, while VAR lacks such solutions. In our work, we implement and benchmark many strategies for VAR, and compare them to state-of-the-art DM adaptation strategies. We observe that VAR outperforms DMs for non-DP adaptations, however, the performance of DP suffers, which necessitates further research in private adaptations for VAR. Code is available at https://github.com/sprintml/finetuning_var_dp.
【4】A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning
标题：互补学习系统的神经网络模型：连续学习的模式分离和完成
链接：https://arxiv.org/abs/2507.11393
备注：Accepted to CogSci 2025. 7 pages, 7 figures
摘要：学习新信息而不忘记先前的知识是人类智能的核心。相比之下，神经网络模型遭受灾难性遗忘：在获取新信息时，先前学习的任务的性能显着下降。互补学习系统（CLS）理论为人类的这种能力提供了一种解释，提出大脑具有不同的模式分离系统（编码不同的记忆）和模式完成系统（从部分线索中检索完整的记忆）。为了捕获这些互补功能，我们利用变分自编码器（VAE）的代表性泛化能力和现代Hopfield网络（MHN）的强大记忆存储特性，将它们组合成一个神经上合理的持续学习模型。我们在Split-MNIST任务（一种流行的持续学习基准）上评估了这个模型，并达到了接近最先进的准确率（~90%），大大减少了遗忘。表征分析经验证实了功能分离：VAE承保模式完成，而MHN驱动模式分离。通过捕获可扩展架构中的模式分离和完成，我们的工作为生物和人工系统中的记忆整合，泛化和持续学习建模提供了一个功能模板。
摘要：Learning new information without forgetting prior knowledge is central to human intelligence. In contrast, neural network models suffer from catastrophic forgetting: a significant degradation in performance on previously learned tasks when acquiring new information. The Complementary Learning Systems (CLS) theory offers an explanation for this human ability, proposing that the brain has distinct systems for pattern separation (encoding distinct memories) and pattern completion (retrieving complete memories from partial cues). To capture these complementary functions, we leverage the representational generalization capabilities of variational autoencoders (VAEs) and the robust memory storage properties of Modern Hopfield networks (MHNs), combining them into a neurally plausible continual learning model. We evaluate this model on the Split-MNIST task, a popular continual learning benchmark, and achieve close to state-of-the-art accuracy (~90%), substantially reducing forgetting. Representational analyses empirically confirm the functional dissociation: the VAE underwrites pattern completion, while the MHN drives pattern separation. By capturing pattern separation and completion in scalable architectures, our work provides a functional template for modeling memory consolidation, generalization, and continual learning in both biological and artificial systems.
【5】Turning Sand to Gold: Recycling Data to Bridge On-Policy and Off-Policy Learning via Causal Bound
标题：化沙为金：回收数据通过因果界限连接政策内和政策外学习
链接：https://arxiv.org/abs/2507.11269
备注：51 pages, 16 figures
摘要：深度强化学习（DRL）代理擅长解决各个领域的复杂决策任务。然而，它们通常需要大量的训练步骤和巨大的经验重放缓冲区，从而导致显著的计算和资源需求。为了解决这些挑战，我们引入了一个新的理论结果，利用奈曼鲁宾潜在的结果框架到DRL。与大多数专注于界定反事实损失的方法不同，我们对事实损失建立了一个因果界限，这类似于DRL中的政策损失。这个界限是通过将过去的值网络输出存储在经验重放缓冲区中来计算的，有效地利用了通常被丢弃的数据。在Atari 2600和MuJoCo域上对各种代理（如DQN和SAC）进行了广泛的实验，获得了高达2，427%的奖励率，在没有我们提出的术语的情况下优于相同的代理，并将经验重放缓冲区大小减少了96%，以可忽略的成本显着提高了样本效率。
摘要：Deep reinforcement learning (DRL) agents excel in solving complex decision-making tasks across various domains. However, they often require a substantial number of training steps and a vast experience replay buffer, leading to significant computational and resource demands. To address these challenges, we introduce a novel theoretical result that leverages the Neyman-Rubin potential outcomes framework into DRL. Unlike most methods that focus on bounding the counterfactual loss, we establish a causal bound on the factual loss, which is analogous to the on-policy loss in DRL. This bound is computed by storing past value network outputs in the experience replay buffer, effectively utilizing data that is usually discarded. Extensive experiments across the Atari 2600 and MuJoCo domains on various agents, such as DQN and SAC, achieve up to 2,427% higher reward ratio, outperforming the same agents without our proposed term, and reducing the experience replay buffer size by up to 96%, significantly improving sample efficiency at negligible cost.
【6】Data-Driven Differential Evolution in Tire Industry Extrusion: Leveraging Surrogate Models
标题：轮胎行业挤出的数据驱动差异化演变：利用代理模型
链接：https://arxiv.org/abs/2507.11191
备注：22 pages, 15 figures
摘要：工业过程的优化仍然是一个关键的挑战，特别是当没有数学公式的目标函数或约束条件。本研究通过提出一种基于代理的、数据驱动的方法来解决这个问题，该方法仅使用历史过程数据来优化复杂的现实制造系统。机器学习模型被用来近似系统行为并构建代理模型，这些模型被集成到一个定制的元启发式方法中：具有多级惩罚函数和代理模型的数据驱动差分进化，这是一个适应于所研究过程特征的差分进化版本。该方法被应用到轮胎制造业的挤出过程中，优化初始化参数，以减少浪费和生产时间的目标。结果表明，基于替代的优化方法优于历史最佳配置，实现了65%的初始化和设置时间减少，同时还显着减少材料浪费。这些发现突出了数据驱动的建模和元启发式优化相结合的工业过程中，明确的配方是不可用的潜力。
摘要：The optimization of industrial processes remains a critical challenge, particularly when no mathematical formulation of objective functions or constraints is available. This study addresses this issue by proposing a surrogate-based, data-driven methodology for optimizing complex real-world manufacturing systems using only historical process data. Machine learning models are employed to approximate system behavior and construct surrogate models, which are integrated into a tailored metaheuristic approach: Data-Driven Differential Evolution with Multi-Level Penalty Functions and Surrogate Models, an adapted version of Differential Evolution suited to the characteristics of the studied process. The methodology is applied to an extrusion process in the tire manufacturing industry, with the goal of optimizing initialization parameters to reduce waste and production time. Results show that the surrogate-based optimization approach outperforms historical best configurations, achieving a 65\% reduction in initialization and setup time, while also significantly minimizing material waste. These findings highlight the potential of combining data-driven modeling and metaheuristic optimization for industrial processes where explicit formulations are unavailable.
【7】Leveraging Advanced Machine Learning to Predict Turbulence Dynamics from Temperature Observations at an Experimental Prescribed Fire
标题：利用先进的机器学习从实验规定火灾的温度观测预测湍流动力学
链接：https://arxiv.org/abs/2507.11012
备注：arXiv admin note: text overlap with arXiv:2311.05128
摘要：本研究探讨了预测湍流动能（TKE）从更容易获得的温度数据，同时收集在10 Hz的温度分布和湍流数据在一个小的实验规定的燃烧在新泽西州松树荒地的潜力。机器学习模型，包括深度神经网络，随机森林回归，梯度提升和高斯过程回归，被用来评估从温度扰动预测TKE的潜力，并探索相关性的时间和空间动态。数据可视化和相关性分析揭示了热电偶温度和TKE之间的模式和关系，从而深入了解了潜在的动态。尽管预测因子与目标变量之间的相关性较弱，但通过采用各种机器学习模型实现了对TKE的更准确预测。结果表明，显着的成功，特别是从回归模型，在准确地预测TKE。这项研究的结果表明，一种新的数值方法来确定火灾环境中和周围的温度和气流过程之间的新关系。这些关系可以帮助我们完善燃烧环境过程的理解和耦合和解耦的火灾环境过程，必要的改进消防操作策略和火灾和烟雾模型预测。这项研究的结果还强调了机器学习技术在分析火灾环境的复杂大型数据集方面的宝贵作用，展示了它们在推进火灾研究和管理实践方面的潜力。
摘要 ：This study explores the potential for predicting turbulent kinetic energy (TKE) from more readily acquired temperature data using temperature profiles and turbulence data collected concurrently at 10 Hz during a small experimental prescribed burn in the New Jersey Pine Barrens. Machine learning models, including Deep Neural Networks, Random Forest Regressor, Gradient Boosting, and Gaussian Process Regressor, were employed to assess the potential to predict TKE from temperature perturbations and explore temporal and spatial dynamics of correlations. Data visualization and correlation analyses revealed patterns and relationships between thermocouple temperatures and TKE, providing insight into the underlying dynamics. More accurate predictions of TKE were achieved by employing various machine learning models despite a weak correlation between the predictors and the target variable. The results demonstrate significant success, particularly from regression models, in accurately predicting the TKE. The findings of this study demonstrate a novel numerical approach to identifying new relationships between temperature and airflow processes in and around the fire environment. These relationships can help refine our understanding of combustion environment processes and the coupling and decoupling of fire environment processes necessary for improving fire operations strategy and fire and smoke model predictions. The findings of this study additionally highlight the valuable role of machine learning techniques in analyzing the complex large datasets of the fire environments, showcasing their potential to advance fire research and management practices.
【8】Physics-Informed Neural Networks For Semiconductor Film Deposition: A Review
标题：半导体薄膜沉积的物理信息神经网络：回顾
链接：https://arxiv.org/abs/2507.10983
备注：11 pages, 1 figure, 3 tables, IDETC-CIE 2025
摘要：半导体制造严重依赖于膜沉积工艺，例如化学气相沉积和物理气相沉积。这些复杂的工艺需要精确的控制，以实现膜的均匀性、适当的附着力和所需的功能。物理信息神经网络（PINN）是一种创新的机器学习（ML）方法，其最新进展在解决半导体薄膜沉积和其他制造领域中与过程控制、质量保证和预测建模相关的挑战方面显示出了巨大的潜力。本文提供了一个全面的审查ML应用针对半导体薄膜沉积工艺。通过专题分析，我们确定了主要趋势，现有的局限性和研究差距，提供了对当前方法的优势和制约因素的见解。我们的结构化分析旨在突出这些ML技术的潜在集成，以提高薄膜沉积过程的可解释性，准确性和鲁棒性。此外，我们还研究了最先进的PINN方法，讨论了将物理知识，控制律和偏微分方程嵌入到为半导体制造量身定制的高级神经网络架构中的策略。基于这一详细的审查，我们提出了新的研究方向，整合PINNs的优势，显着推进薄膜沉积工艺。这项研究的贡献包括为未来的研究建立一个明确的途径，以整合物理信息ML框架，解决现有的方法差距，并最终提高半导体制造的精度，可扩展性和运营效率。
摘要：Semiconductor manufacturing relies heavily on film deposition processes, such as Chemical Vapor Deposition and Physical Vapor Deposition. These complex processes require precise control to achieve film uniformity, proper adhesion, and desired functionality. Recent advancements in Physics-Informed Neural Networks (PINNs), an innovative machine learning (ML) approach, have shown significant promise in addressing challenges related to process control, quality assurance, and predictive modeling within semiconductor film deposition and other manufacturing domains. This paper provides a comprehensive review of ML applications targeted at semiconductor film deposition processes. Through a thematic analysis, we identify key trends, existing limitations, and research gaps, offering insights into both the advantages and constraints of current methodologies. Our structured analysis aims to highlight the potential integration of these ML techniques to enhance interpretability, accuracy, and robustness in film deposition processes. Additionally, we examine state-of-the-art PINN methods, discussing strategies for embedding physical knowledge, governing laws, and partial differential equations into advanced neural network architectures tailored for semiconductor manufacturing. Based on this detailed review, we propose novel research directions that integrate the strengths of PINNs to significantly advance film deposition processes. The contributions of this study include establishing a clear pathway for future research in integrating physics-informed ML frameworks, addressing existing methodological gaps, and ultimately improving precision, scalability, and operational efficiency within semiconductor manufacturing.
【9】A Learning Framework For Cooperative Collision Avoidance of UAV Swarms Leveraging Domain Knowledge
标题：利用领域知识协作避免无人机群碰撞的学习框架
链接：https://arxiv.org/abs/2507.10913
备注：Under review at AAAI 2026
摘要：提出了一种基于领域知识奖励的无人机群协同避碰多智能体强化学习框架。奖励来自图像处理领域的知识，近似二维领域的轮廓。通过将障碍物建模为场地上的最大值，可以避免碰撞，因为轮廓永远不会穿过峰值或相交。此外，柜台是顺利和能源效率。我们的框架使训练与大规模的群体作为代理的相互作用最小化，并消除了复杂的信用分配方案或观察共享机制，在国家的最先进的MARL方法的需要。此外，无人机通过强化训练获得了适应复杂环境的能力，在复杂环境中，轮廓可能是不可行的或不存在的。进行了大量的实验，以评估我们的框架对国家的最先进的MARL算法的性能。
摘要：This paper presents a multi-agent reinforcement learning (MARL) framework for cooperative collision avoidance of UAV swarms leveraging domain knowledge-driven reward. The reward is derived from knowledge in the domain of image processing, approximating contours on a two-dimensional field. By modeling obstacles as maxima on the field, collisions are inherently avoided as contours never go through peaks or intersect. Additionally, counters are smooth and energy-efficient. Our framework enables training with large swarm sizes as the agent interaction is minimized and the need for complex credit assignment schemes or observation sharing mechanisms in state-of-the-art MARL approaches are eliminated. Moreover, UAVs obtain the ability to adapt to complex environments where contours may be non-viable or non-existent through intensive training. Extensive experiments are conducted to evaluate the performances of our framework against state-of-the-art MARL algorithms.
【10】Outbound Modeling for Inventory Management
标题：库存管理的建模
链接：https://arxiv.org/abs/2507.10890
备注：KDD - AI for Supply Chain Workshop
摘要：我们研究的问题，预测的数量单位履行（或“排水”）从每个库存仓库，以满足客户的需求，以及相关的出站运输成本。实际消耗和运输成本由复杂的生产系统决定，该系统管理客户订单履行的规划和执行，即从何处以及如何将单元运送到客户。准确地建模这些过程对于区域库存规划至关重要，特别是在使用强化学习（RL）开发控制策略时。对于RL用例，流失模型被整合到模拟器中以产生长时间的推出，我们希望这是可区分的。虽然模拟对内部软件系统的调用可以用来恢复这种转换，但它们是不可区分的，并且在RL培训环境中运行太慢和昂贵。因此，我们将其定义为一个概率预测问题，以库存状况和外部客户需求为条件，对每个时间段所有仓库的出站流失和运输成本的联合分布进行建模。为了确保RL环境中的鲁棒性，模型必须处理由非策略轨迹引起的分布外场景。我们提出了一个验证计划，利用生产系统评估流失模型的反事实的RL政策引起的库存状态。初步结果表明，该模型的准确性内的分布设置。
摘要 ：We study the problem of forecasting the number of units fulfilled (or ``drained'') from each inventory warehouse to meet customer demand, along with the associated outbound shipping costs. The actual drain and shipping costs are determined by complex production systems that manage the planning and execution of customers' orders fulfillment, i.e. from where and how to ship a unit to be delivered to a customer. Accurately modeling these processes is critical for regional inventory planning, especially when using Reinforcement Learning (RL) to develop control policies. For the RL usecase, a drain model is incorporated into a simulator to produce long rollouts, which we desire to be differentiable. While simulating the calls to the internal software systems can be used to recover this transition, they are non-differentiable and too slow and costly to run within an RL training environment. Accordingly, we frame this as a probabilistic forecasting problem, modeling the joint distribution of outbound drain and shipping costs across all warehouses at each time period, conditioned on inventory positions and exogenous customer demand. To ensure robustness in an RL environment, the model must handle out-of-distribution scenarios that arise from off-policy trajectories. We propose a validation scheme that leverages production systems to evaluate the drain model on counterfactual inventory states induced by RL policies. Preliminary results demonstrate the model's accuracy within the in-distribution setting.
【11】Visually grounded emotion regulation via diffusion models and user-driven reappraisal
标题：通过扩散模型和用户驱动的重新评估进行基于视觉的情感调节
链接：https://arxiv.org/abs/2507.10861
摘要：认知重评是情绪调节的一个关键策略，涉及对情绪刺激的重新解释以改变情感反应。尽管它在临床和认知科学中发挥着核心作用，但现实世界的重新评估干预仍然是认知要求高，抽象的，主要是口头的。这种对高阶认知和语言过程的依赖往往在创伤或抑郁的个体中受损，限制了标准方法的有效性。在这里，我们提出了一种新的，基于视觉的增强认知重新评价，将大规模的文本到图像的扩散模型到情绪调节过程。具体来说，我们介绍了一个系统，用户重新解释情绪消极的图像通过口头重新评价，这是转化为支持，情绪一致的可视化使用稳定的扩散模型与微调的IP适配器。这种生成转换在视觉上实例化了用户的重新评价，同时保持了与原始刺激的结构相似性，外部化并加强了监管意图。为了验证这种方法，我们进行了一个被试内实验（N = 20）使用修改后的认知情绪调节（CER）任务。参与者重新评估或描述了来自国际情感图片系统（IAPS）的令人厌恶的图像，无论是否有人工智能生成的视觉反馈。结果表明，与非AI和控制条件相比，AI辅助的重新评估显着减少了负面影响。进一步的分析表明，参与者重新评价和生成的图像之间的情绪对齐与情感缓解相关，这表明多模态一致性增强了监管效能。这些发现表明，生成视觉输入可以支持认知重新评价，并在生成AI，情感计算和治疗技术的交叉点上开辟新的方向。
摘要：Cognitive reappraisal is a key strategy in emotion regulation, involving reinterpretation of emotionally charged stimuli to alter affective responses. Despite its central role in clinical and cognitive science, real-world reappraisal interventions remain cognitively demanding, abstract, and primarily verbal. This reliance on higher-order cognitive and linguistic processes is often impaired in individuals with trauma or depression, limiting the effectiveness of standard approaches. Here, we propose a novel, visually based augmentation of cognitive reappraisal by integrating large-scale text-to-image diffusion models into the emotional regulation process. Specifically, we introduce a system in which users reinterpret emotionally negative images via spoken reappraisals, which are transformed into supportive, emotionally congruent visualizations using stable diffusion models with a fine-tuned IP-adapter. This generative transformation visually instantiates users' reappraisals while maintaining structural similarity to the original stimuli, externalizing and reinforcing regulatory intent. To test this approach, we conducted a within-subject experiment (N = 20) using a modified cognitive emotion regulation (CER) task. Participants reappraised or described aversive images from the International Affective Picture System (IAPS), with or without AI-generated visual feedback. Results show that AI-assisted reappraisal significantly reduced negative affect compared to both non-AI and control conditions. Further analyses reveal that sentiment alignment between participant reappraisals and generated images correlates with affective relief, suggesting that multimodal coherence enhances regulatory efficacy. These findings demonstrate that generative visual input can support cogitive reappraisal and open new directions at the intersection of generative AI, affective computing, and therapeutic technology.
【12】A Benchmarking Framework for AI models in Automotive Aerodynamics
标题：汽车空气动力学人工智能模型的基准框架
链接：https://arxiv.org/abs/2507.10747
摘要：在本文中，我们介绍了开源NVIDIA PhysicsNeMo-CFD框架内的基准测试框架，旨在系统地评估汽车空气动力学预测AI模型的准确性，性能，可扩展性和泛化能力。开放的可扩展框架能够整合与计算机辅助工程（CAE）社区相关的各种度量。通过提供比较人工智能模型的标准化方法，该框架提高了性能评估的透明度和一致性，其总体目标是提高对这些模型的理解和开发，以加速该领域的研究和创新。为了证明其实用性，该框架包括使用DrivAerML数据集评估三个人工智能模型（DoMINO、X-MeshGraphNet和FIGConvNet）上的表面和体积流场预测。它还包括集成其他模型和数据集的指导方针，使其可扩展以实现物理一致的指标。这项基准研究旨在使研究人员和行业专业人士能够选择，改进和推进人工智能驱动的空气动力学建模方法，最终促进汽车空气动力学中更有效，准确和可解释的解决方案的开发
摘要：In this paper, we introduce a benchmarking framework within the open-source NVIDIA PhysicsNeMo-CFD framework designed to systematically assess the accuracy, performance, scalability, and generalization capabilities of AI models for automotive aerodynamics predictions. The open extensible framework enables incorporation of a diverse set of metrics relevant to the Computer-Aided Engineering (CAE) community. By providing a standardized methodology for comparing AI models, the framework enhances transparency and consistency in performance assessment, with the overarching goal of improving the understanding and development of these models to accelerate research and innovation in the field. To demonstrate its utility, the framework includes evaluation of both surface and volumetric flow field predictions on three AI models: DoMINO, X-MeshGraphNet, and FIGConvNet using the DrivAerML dataset. It also includes guidelines for integrating additional models and datasets, making it extensible for physically consistent metrics. This benchmarking study aims to enable researchers and industry professionals in selecting, refining, and advancing AI-driven aerodynamic modeling approaches, ultimately fostering the development of more efficient, accurate, and interpretable solutions in automotive aerodynamics
【13】A Simple Baseline for Stable and Plastic Neural Networks
标题：稳定和可塑神经网络的简单基线
链接：https://arxiv.org/abs/2507.10637
备注：11 pages, 50 figures
摘要：计算机视觉中的持续学习要求模型在不忘记先验知识的情况下适应连续的任务流，但现有的方法往往会严重倾向于可塑性或稳定性。我们引入RDBP，一个简单的，低开销的基线，它结合了两个互补的机制：ReLUDown，一个轻量级的激活修改，保留功能的敏感性，同时防止神经元休眠，和递减反向传播，一个生物启发的梯度调度方案，逐步屏蔽早期层的灾难性更新。在Continual ImageNet基准测试中，RDBP匹配或超过了最先进方法的可塑性和稳定性，同时降低了计算成本。因此，RDBP为现实世界的持续学习提供了一个实用的解决方案，并为未来的持续学习策略提供了一个明确的基准。
摘要：Continual learning in computer vision requires that models adapt to a continuous stream of tasks without forgetting prior knowledge, yet existing approaches often tip the balance heavily toward either plasticity or stability. We introduce RDBP, a simple, low-overhead baseline that unites two complementary mechanisms: ReLUDown, a lightweight activation modification that preserves feature sensitivity while preventing neuron dormancy, and Decreasing Backpropagation, a biologically inspired gradient-scheduling scheme that progressively shields early layers from catastrophic updates. Evaluated on the Continual ImageNet benchmark, RDBP matches or exceeds the plasticity and stability of state-of-the-art methods while reducing computational cost. RDBP thus provides both a practical solution for real-world continual learning and a clear benchmark against which future continual learning strategies can be measured.
【14】Compute Requirements for Algorithmic Innovation in Frontier AI Models
标题：前沿人工智能模型中数学创新的计算要求
链接：https://arxiv.org/abs/2507.10618
摘要 ：大型语言模型预训练中的数学创新推动了达到给定能力水平所需的总计算量的大幅减少。在本文中，我们实证研究开发算法创新的计算要求。我们对Llama 3和DeepSeek-V3中使用的36种预训练算法创新进行了分类。对于每个创新，我们估计开发中使用的总FLOP和所使用的硬件的FLOP/s。使用大量资源的创新每年的需求翻了一番。然后，我们使用这个数据集来研究计算上限对创新的影响。我们的分析表明，仅计算上限不太可能大幅减缓AI算法的进展。即使是严格的计算上限-例如将总操作限制在用于训练GPT-2的计算上，或者将硬件容量限制在8个H100 GPU上-仍然可以允许一半的编目创新。
摘要：Algorithmic innovation in the pretraining of large language models has driven a massive reduction in the total compute required to reach a given level of capability. In this paper we empirically investigate the compute requirements for developing algorithmic innovations. We catalog 36 pre-training algorithmic innovations used in Llama 3 and DeepSeek-V3. For each innovation we estimate both the total FLOP used in development and the FLOP/s of the hardware utilized. Innovations using significant resources double in their requirements each year. We then use this dataset to investigate the effect of compute caps on innovation. Our analysis suggests that compute caps alone are unlikely to dramatically slow AI algorithmic progress. Even stringent compute caps -- such as capping total operations to the compute used to train GPT-2 or capping hardware capacity to 8 H100 GPUs -- could still have allowed for half of the cataloged innovations.
【15】Extension OL-MDISF: Online Learning from Mix-Typed, Drifted, and Incomplete Streaming Features
标题：扩展OL-MDSF：从混合类型、漂移和不完整流媒体功能中进行在线学习
链接：https://arxiv.org/abs/2507.10594
摘要：在线学习，其中特征空间可以随时间变化，提供了一个灵活的学习范式，吸引了相当多的关注。然而，它仍然面临三个重大挑战。首先，具有混合特征类型的真实世界数据流的异构性对传统的参数化建模提出了挑战。其次，数据流分布可能会随着时间的推移而变化，导致模型性能突然大幅下降。第三，由于时间和成本的限制，标记每个数据实例通常是不可行的。为了解决这些问题，我们提出了OL-MDISF（Online Learning from Mix-typed，Drifted，and Incomplete Streaming Features），它为异构特征构建了一个基于潜在Copula的表示，通过集成熵和潜在失配检测漂移，并执行结构感知的伪标记。 本配套文件作为OL-MDISF的独立技术参考。它提供了混合类型建模，漂移适应和弱监督的相关工作的上下文讨论，以及在两种类型的漂移场景下跨14个真实世界数据集的一组全面的实验。这些包括CER趋势，消融研究，敏感性分析和时间系综动力学。我们希望这篇文章能为在线学习复杂的、弱监督的流数据提供一个可复制的基准。
摘要：Online learning, where feature spaces can change over time, offers a flexible learning paradigm that has attracted considerable attention. However, it still faces three significant challenges. First, the heterogeneity of real-world data streams with mixed feature types presents challenges for traditional parametric modeling. Second, data stream distributions can shift over time, causing an abrupt and substantial decline in model performance. Third, it is often infeasible to label every data instance due to time and cost constraints. To address these issues, we proposed OL-MDISF (Online Learning from Mix-typed, Drifted, and Incomplete Streaming Features), which constructs a latent copula-based representation for heterogeneous features, detects drifts via ensemble entropy and latent mismatch, and performs structure-aware pseudo-labeling. This companion paper serves as a standalone technical reference to OL-MDISF. It provides a contextual discussion of related work in mixed-type modeling, drift adaptation, and weak supervision, as well as a comprehensive set of experiments across 14 real-world datasets under two types of drift scenarios. These include CER trends, ablation studies, sensitivity analyses, and temporal ensemble dynamics. We hope this document offers a reproducible benchmark for online learning on complex, weakly supervised streaming data.
【16】How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction
标题：标签错误如何影响对比学习？数据抽象性约简的一个视角
链接：https://arxiv.org/abs/2507.11161
备注：Accepted by ICML2025 as a poster
摘要：近年来，对比学习在自监督表示学习领域取得了最先进的性能。许多以前的作品试图提供的对比学习的成功背后的理论理解。几乎所有这些都依赖于一个默认假设，即，标签一致性假设，由于常见的增强策略的强度和随机性，例如随机调整大小的裁剪（RRC），该假设在实践中可能不成立（失败的概率称为标签错误）。本文研究了标记错误对对比学习下游分类性能的理论影响。我们首先揭示了标签错误对下游分类风险的几个重大负面影响。为了减轻这些影响，数据降维方法（例如，奇异值分解（SVD）应用于原始数据以减少假阳性样本，并建立理论和经验评估。此外，它也被发现，SVD作为一把双刃剑，这可能会导致下游的分类精度下降，由于减少连接的增强图。基于上述观察，我们给出了增强建议，我们应该使用一些适度的嵌入维数（如在我们的实验中为512，1024$），数据膨胀，弱增强，和SVD，以确保大的图连接和小的标记错误，以提高模型的性能。
摘要：In recent years, contrastive learning has achieved state-of-the-art performance in the territory of self-supervised representation learning. Many previous works have attempted to provide the theoretical understanding underlying the success of contrastive learning. Almost all of them rely on a default assumption, i.e., the label consistency assumption, which may not hold in practice (the probability of failure is called labeling error) due to the strength and randomness of common augmentation strategies, such as random resized crop (RRC). This paper investigates the theoretical impact of labeling error on the downstream classification performance of contrastive learning. We first reveal several significant negative impacts of labeling error on downstream classification risk. To mitigate these impacts, data dimensionality reduction method (e.g., singular value decomposition, SVD) is applied on original data to reduce false positive samples, and establish both theoretical and empirical evaluations. Moreover, it is also found that SVD acts as a double-edged sword, which may lead to the deterioration of downstream classification accuracy due to the reduced connectivity of the augmentation graph. Based on the above observations, we give the augmentation suggestion that we should use some moderate embedding dimension (such as $512, 1024$ in our experiments), data inflation, weak augmentation, and SVD to ensure large graph connectivity and small labeling error to improve model performance.
【17】Interpretable Bayesian Tensor Network Kernel Machines with Automatic Rank and Feature Selection
标题：具有自动排序和特征选择的可解释Bayesian张量网络核心机
链接：https://arxiv.org/abs/2507.11136
备注：39 pages, 5 figures, 4 tables. Submitted to Journal of Machine Learning Research. The code is available at: this https URL. arXiv admin note: text overlap with arXiv:1401.6497 by other authors
摘要：张量网络（TN）内核机器通过将参数表示为低秩TN来加速模型学习，从而减少计算和内存使用。然而，大多数基于TN的核方法是确定性的，忽略了参数的不确定性。此外，它们需要手动调整模型复杂度超参数，如张量秩和特征维度，通常通过试错或交叉验证等计算成本高的方法。我们提出了贝叶斯张量网络核机器，一个完全概率的框架，使用稀疏诱导层次先验的TN因素自动推断模型的复杂性。这使得能够自动推断张量秩和特征维度，同时还识别最相关的预测特征，从而增强模型的可解释性。所有的模型参数和超参数被视为具有相应先验的潜变量。由于贝叶斯方法和潜在变量的依赖关系，我们应用平均场变分推理来近似他们的后验。我们表明，应用平均场近似TN因素产生贝叶斯ALS算法具有相同的计算复杂性，其确定性对应，使不确定性量化没有额外的计算成本。合成和真实世界的数据集上的实验表明，我们的模型在预测准确性，不确定性量化，可解释性和可扩展性方面具有卓越的性能。
摘要 ：Tensor Network (TN) Kernel Machines speed up model learning by representing parameters as low-rank TNs, reducing computation and memory use. However, most TN-based Kernel methods are deterministic and ignore parameter uncertainty. Further, they require manual tuning of model complexity hyperparameters like tensor rank and feature dimensions, often through trial-and-error or computationally costly methods like cross-validation. We propose Bayesian Tensor Network Kernel Machines, a fully probabilistic framework that uses sparsity-inducing hierarchical priors on TN factors to automatically infer model complexity. This enables automatic inference of tensor rank and feature dimensions, while also identifying the most relevant features for prediction, thereby enhancing model interpretability. All the model parameters and hyperparameters are treated as latent variables with corresponding priors. Given the Bayesian approach and latent variable dependencies, we apply a mean-field variational inference to approximate their posteriors. We show that applying a mean-field approximation to TN factors yields a Bayesian ALS algorithm with the same computational complexity as its deterministic counterpart, enabling uncertainty quantification at no extra computational cost. Experiments on synthetic and real-world datasets demonstrate the superior performance of our model in prediction accuracy, uncertainty quantification, interpretability, and scalability.
【18】Kernel Learning for Mean-Variance Trading Strategies
标题：均值-方差交易策略的核心学习
链接：https://arxiv.org/abs/2507.10701
备注：49 pages
摘要：在这篇文章中，我们开发了一个基于内核的框架，构建动态的，路径依赖的交易策略下的均值方差优化标准。基于（Muca Cirone和Salvi，2025）的理论结果，我们将交易策略参数化为再生核希尔伯特空间（RKHS）中的函数，从而实现灵活的非马尔可夫方法来解决最优投资组合问题。我们将其与（Futter，Horvath，Wiese，2023）的基于签名的框架进行比较，并证明当资产动态或预测信号对合成和市场数据示例都表现出时间依赖性时，两者都显着优于经典马尔可夫方法。在这种情况下使用内核提供了显著的建模灵活性，因为特征嵌入的选择范围可以从随机签名到神经网络架构的最终层。至关重要的是，我们的框架保留了封闭形式的解决方案，并提供了一种替代基于梯度的优化。
摘要：In this article, we develop a kernel-based framework for constructing dynamic, pathdependent trading strategies under a mean-variance optimisation criterion. Building on the theoretical results of (Muca Cirone and Salvi, 2025), we parameterise trading strategies as functions in a reproducing kernel Hilbert space (RKHS), enabling a flexible and non-Markovian approach to optimal portfolio problems. We compare this with the signature-based framework of (Futter, Horvath, Wiese, 2023) and demonstrate that both significantly outperform classical Markovian methods when the asset dynamics or predictive signals exhibit temporal dependencies for both synthetic and market-data examples. Using kernels in this context provides significant modelling flexibility, as the choice of feature embedding can range from randomised signatures to the final layers of neural network architectures. Crucially, our framework retains closed-form solutions and provides an alternative to gradient-based optimisation.
【19】TaylorPODA: A Taylor Expansion-Based Method to Improve Post-Hoc Attributions for Opaque Models
标题：TaylorPODA：一种基于Taylor展开的方法，用于改善不透明模型的事后归因
链接：https://arxiv.org/abs/2507.10643
备注：17 pages, 6 figures, Submitted to NeurIPS 2025
摘要：现有的事后模型不可知的方法产生外部解释不透明的模型，主要是通过本地归因于模型输出到其输入功能。然而，它们往往缺乏一个明确和系统的框架来量化单个功能的贡献。基于Deng等人（2024）引入的泰勒扩展框架，以统一现有的局部归因方法，我们提出了一套严格的假设-“精确度”，“联邦”和“零差异”-来管理泰勒术语特定的归因。在这些假设的指导下，我们引入了泰勒PODA（泰勒展开衍生的重要性，顺序aDapted属性），它包含了一个额外的“适应”属性。此属性使其能够与特定于任务的目标保持一致，特别是在缺乏地面实况解释的事后设置中。实证评估表明，TaylorPODA实现了对基线方法的竞争力的结果，提供了原则性和可视化友好的解释。这项工作通过提供具有更强理论基础的解释，向可信的不透明模型部署迈出了一步。
摘要：Existing post-hoc model-agnostic methods generate external explanations for opaque models, primarily by locally attributing the model output to its input features. However, they often lack an explicit and systematic framework for quantifying the contribution of individual features. Building on the Taylor expansion framework introduced by Deng et al. (2024) to unify existing local attribution methods, we propose a rigorous set of postulates -- "precision", "federation", and "zero-discrepancy" -- to govern Taylor term-specific attribution. Guided by these postulates, we introduce TaylorPODA (Taylor expansion-derived imPortance-Order aDapted Attribution), which incorporates an additional "adaptation" property. This property enables alignment with task-specific goals, especially in post-hoc settings lacking ground-truth explanations. Empirical evaluations demonstrate that TaylorPODA achieves competitive results against baseline methods, providing principled and visualization-friendly explanations. This work represents a step toward the trustworthy deployment of opaque models by offering explanations with stronger theoretical grounding.
其他(40篇)
【1】CATVis: Context-Aware Thought Visualization
标题：CATVis：上下文感知思维可视化
链接：https://arxiv.org/abs/2507.11522
备注：Accepted at MICCAI 2025. This is the submitted version prior to peer review. The final Version of Record will appear in the MICCAI 2025 proceedings (Springer LNCS)
摘要：基于EEG的脑机接口（BCI）已在各种应用中显示出前景，如运动想象和认知状态监测。然而，由于EEG信号的复杂性和噪声性，从EEG信号解码视觉表示仍然是一个重大挑战。因此，我们提出了一种新的5阶段框架，用于从EEG信号中解码视觉表示：（1）用于概念分类的EEG编码器，（2）在CLIP特征空间中EEG和文本嵌入的跨模态对齐，（3）通过重新排序的标题细化，（4）概念和标题嵌入的加权插值以获得更丰富的语义，以及（5）使用预训练的稳定扩散模型生成图像。我们通过跨模态对齐和重新排序实现上下文感知的EEG到图像生成。实验结果表明，我们的方法生成高质量的图像与视觉刺激对齐，优于SOTA方法的分类准确率为13.43%，生成准确率为15.21%，并减少了36.61%的Fr\'echet Inception Distance，表明优越的语义对齐和图像质量。
摘要：EEG-based brain-computer interfaces (BCIs) have shown promise in various applications, such as motor imagery and cognitive state monitoring. However, decoding visual representations from EEG signals remains a significant challenge due to their complex and noisy nature. We thus propose a novel 5-stage framework for decoding visual representations from EEG signals: (1) an EEG encoder for concept classification, (2) cross-modal alignment of EEG and text embeddings in CLIP feature space, (3) caption refinement via re-ranking, (4) weighted interpolation of concept and caption embeddings for richer semantics, and (5) image generation using a pre-trained Stable Diffusion model. We enable context-aware EEG-to-image generation through cross-modal alignment and re-ranking. Experimental results demonstrate that our method generates high-quality images aligned with visual stimuli, outperforming SOTA approaches by 13.43% in Classification Accuracy, 15.21% in Generation Accuracy and reducing Fr\'echet Inception Distance by 36.61%, indicating superior semantic alignment and image quality.
【2】A parametric activation function based on Wendland RBF
标题：基于Wendland函数的参数激活函数
链接：https://arxiv.org/abs/2507.11493
备注：11 pages, 2 figures
摘要 ：本文介绍了一种新的基于Wendland径向基函数（RBF）的深度神经网络参数激活函数。Wendland RBFs以其在近似理论中的紧凑支持、平滑性和正定性而闻名，适用于解决传统激活函数（如ReLU、sigmoid和tanh）的局限性。所提出的增强Wendland激活将标准Wendland分量与线性和指数项相结合，提供可调局部性，改进梯度传播，并增强训练期间的稳定性。理论分析突出了它的数学特性，包括平滑性和适应性，而对合成任务（例如，正弦波近似）和基准数据集（MNIST、Fashion-MNIST）的性能具有竞争力。结果表明，基于Wendland的激活在某些情况下，特别是在回归任务中，实现了卓越的准确性，同时保持了计算效率。该研究将经典RBF理论与现代深度学习联系起来，表明Wendland激活可以通过局部平滑变换来减轻过拟合并提高泛化能力。未来的发展方向包括混合架构和特定领域的适应。
摘要：This paper introduces a novel parametric activation function based on Wendland radial basis functions (RBFs) for deep neural networks. Wendland RBFs, known for their compact support, smoothness, and positive definiteness in approximation theory, are adapted to address limitations of traditional activation functions like ReLU, sigmoid, and tanh. The proposed enhanced Wendland activation combines a standard Wendland component with linear and exponential terms, offering tunable locality, improved gradient propagation, and enhanced stability during training. Theoretical analysis highlights its mathematical properties, including smoothness and adaptability, while empirical experiments on synthetic tasks (e.g., sine wave approximation) and benchmark datasets (MNIST, Fashion-MNIST) demonstrate competitive performance. Results show that the Wendland-based activation achieves superior accuracy in certain scenarios, particularly in regression tasks, while maintaining computational efficiency. The study bridges classical RBF theory with modern deep learning, suggesting that Wendland activations can mitigate overfitting and improve generalization through localized, smooth transformations. Future directions include hybrid architectures and domain-specific adaptations.
【3】Exploring the robustness of TractOracle methods in RL-based tractography
标题：探索TractOracle方法在基于RL的纤维束成像中的稳健性
链接：https://arxiv.org/abs/2507.11486
备注：38 pages, 8 figures. Submitted to Medical Image Analysis
摘要：纤维束成像算法利用扩散MRI重建大脑白质的纤维结构。在机器学习方法中，强化学习（RL）已成为纤维束成像的一个有前途的框架，在几个关键方面优于传统方法。TractOracle-RL是最近基于RL的方法，通过基于奖励的机制将解剖学先验纳入训练过程来减少误报。在本文中，我们通过整合RL的最新进展，研究了原始TractOracle-RL框架的四个扩展，并在五个不同的扩散MRI数据集上评估了它们的性能。结果表明，将Oracle与RL框架相结合，无论使用何种特定方法或数据集，都可以始终实现强大且可靠的纤维束成像。我们还介绍了一种新的RL训练方案，称为迭代奖励训练（IRT），灵感来自人类反馈强化学习（RLHF）范式。IRT不依赖于人工输入，而是利用捆绑过滤方法在整个训练过程中迭代地完善oracle的指导。实验结果表明，使用Oracle反馈训练的RL方法在准确性和解剖有效性方面显着优于广泛使用的纤维束成像技术。
摘要：Tractography algorithms leverage diffusion MRI to reconstruct the fibrous architecture of the brain's white matter. Among machine learning approaches, reinforcement learning (RL) has emerged as a promising framework for tractography, outperforming traditional methods in several key aspects. TractOracle-RL, a recent RL-based approach, reduces false positives by incorporating anatomical priors into the training process via a reward-based mechanism. In this paper, we investigate four extensions of the original TractOracle-RL framework by integrating recent advances in RL, and we evaluate their performance across five diverse diffusion MRI datasets. Results demonstrate that combining an oracle with the RL framework consistently leads to robust and reliable tractography, regardless of the specific method or dataset used. We also introduce a novel RL training scheme called Iterative Reward Training (IRT), inspired by the Reinforcement Learning from Human Feedback (RLHF) paradigm. Instead of relying on human input, IRT leverages bundle filtering methods to iteratively refine the oracle's guidance throughout training. Experimental results show that RL methods trained with oracle feedback significantly outperform widely used tractography techniques in terms of accuracy and anatomical validity.
【4】Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
标题：思想链的可持续性：人工智能安全的一个新而脆弱的机会
链接：https://arxiv.org/abs/2507.11473
摘要：用人类语言“思考”的人工智能系统为人工智能安全提供了一个独特的机会：我们可以监控他们的思维链（CoT），以发现他们的不当行为。像所有其他已知的人工智能监督方法一样，CoT监控是不完美的，并且允许一些不当行为被忽视。尽管如此，它显示出了希望，我们建议进一步研究CoT的可监测性，并在现有的安全方法的同时投资CoT监测。由于CoT可监测性可能是脆弱的，我们建议前沿模型开发人员考虑开发决策对CoT可监测性的影响。
摘要：AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability.
【5】Better Regret Rates in Bilateral Trade via Sublinear Budget Violation
标题：通过次线性预算违规提高双边贸易中的遗憾率
链接：https://arxiv.org/abs/2507.11419
摘要：双边贸易是算法经济学的一个核心问题，最近的工作探索了如何使用无悔学习算法设计交易机制。然而，当预算平衡必须在每个时间步强制执行时，无遗憾学习是不可能的。Bernasconi等人[Ber+24]展示了如何通过放松预算平衡约束来规避这种不可能性，使其在所有时间步长上仅在全球范围内保持。特别是，他们设计了一个算法，实现了$\tilde O（T^{3/4}）$的顺序后悔，并提供了一个下界$\Omega（T^{5/7}）$。 在这项工作中，我们通过研究最佳后悔率如何随着允许违反全球预算平衡约束而变化，在这两个极端之间进行插值。具体地说，我们设计了一个算法，通过违反约束最多$T^{\beta}$对于任何给定的$\beta \in [\frac{3}{4}，\frac{6}{7}]$，获得后悔$\tilde O（T^{1 - \beta/3}）$。我们补充这一结果与匹配的下限，从而充分表征遗憾和违反预算之间的权衡。我们的结果表明，Bernasconi等人[Ber+24]得到的全局预算平衡情形下的$\tilde O（T^{3/4}）$上界和无约束预算平衡破坏情形下的$\Omega（T^{5/7}）$下界都是紧的.
摘要：Bilateral trade is a central problem in algorithmic economics, and recent work has explored how to design trading mechanisms using no-regret learning algorithms. However, no-regret learning is impossible when budget balance has to be enforced at each time step. Bernasconi et al. [Ber+24] show how this impossibility can be circumvented by relaxing the budget balance constraint to hold only globally over all time steps. In particular, they design an algorithm achieving regret of the order of $\tilde O(T^{3/4})$ and provide a lower bound of $\Omega(T^{5/7})$. In this work, we interpolate between these two extremes by studying how the optimal regret rate varies with the allowed violation of the global budget balance constraint. Specifically, we design an algorithm that, by violating the constraint by at most $T^{\beta}$ for any given $\beta \in [\frac{3}{4}, \frac{6}{7}]$, attains regret $\tilde O(T^{1 - \beta/3})$. We complement this result with a matching lower bound, thus fully characterizing the trade-off between regret and budget violation. Our results show that both the $\tilde O(T^{3/4})$ upper bound in the global budget balance case and the $\Omega(T^{5/7})$ lower bound under unconstrained budget balance violation obtained by Bernasconi et al. [Ber+24] are tight.
【6】Seq vs Seq: An Open Suite of Paired Encoders and Decoders
标题：Seq vs Seq：配对编码器和解码器的开放套件
链接：https://arxiv.org/abs/2507.11412
摘要：大型语言模型（LLM）社区几乎只关注解码器语言模型，因为它们更容易用于文本生成。然而，社区中的一个很大的子集仍然使用仅编码器模型来执行分类或检索等任务。以前的工作试图比较这些架构，但被迫与具有不同数量的参数，训练技术和数据集的模型进行比较。我们介绍了SOTA开放数据Ettin模型套件：成对的仅编码器和仅解码器模型，范围从1700万个参数到10亿个参数，在多达2万亿个令牌上训练。对仅编码器和仅解码器模型使用相同的配方，可以在这两个类别中为各自的大小生成SOTA配方，击败ModernBERT作为编码器，Llama 3.2和SmolLM2作为解码器。与以前的工作一样，我们发现仅编码器模型擅长分类和检索任务，而解码器擅长生成任务。然而，我们表明，通过持续训练使解码器模型适应编码器任务（反之亦然）与仅使用反向目标相比是低于标准的（即400M编码器在MNLI上优于1B解码器，对于生成任务，反之亦然）。我们开源了这项研究的所有工件，包括训练数据，按检查点划分的训练顺序，以及200多个检查点，以允许未来的工作分析或扩展训练的各个方面。
摘要：The large language model (LLM) community focuses almost exclusively on decoder-only language models, since they are easier to use for text generation. However, a large subset of the community still uses encoder-only models for tasks such as classification or retrieval. Previous work has attempted to compare these architectures, but is forced to make comparisons with models that have different numbers of parameters, training techniques, and datasets. We introduce the SOTA open-data Ettin suite of models: paired encoder-only and decoder-only models ranging from 17 million parameters to 1 billion, trained on up to 2 trillion tokens. Using the same recipe for both encoder-only and decoder-only models produces SOTA recipes in both categories for their respective sizes, beating ModernBERT as an encoder and Llama 3.2 and SmolLM2 as decoders. Like previous work, we find that encoder-only models excel at classification and retrieval tasks while decoders excel at generative tasks. However, we show that adapting a decoder model to encoder tasks (and vice versa) through continued training is subpar compared to using only the reverse objective (i.e. a 400M encoder outperforms a 1B decoder on MNLI, and vice versa for generative tasks). We open-source all artifacts of this study including training data, training order segmented by checkpoint, and 200+ checkpoints to allow future work to analyze or extend all aspects of training.
【7】Robust-Multi-Task Gradient Boosting
标题：稳健多任务梯度提升
链接：https://arxiv.org/abs/2507.11411
摘要：多任务学习（MTL）在利用跨任务的共享信息来提高泛化能力方面表现出了有效性。MTL假设任务具有可以提高性能的相似性。此外，boosting算法在各种学习问题上表现出了出色的性能，主要是由于它们能够专注于难学的实例并迭代减少残余错误。这使得它们成为学习多任务问题的一种很有前途的方法。然而，现实世界的MTL场景通常涉及不一致的任务（称为离群任务或对抗任务），这些任务与其他任务不具有有益的相似性，实际上可能会降低整体模型的性能。为了克服这一挑战，我们提出了Robust-Multi-Task Gradient Boosting（R-MTGB），这是一种新型的Boosting框架，可以在训练过程中显式地建模和适应任务的异质性。R-MTGB将学习过程分为三个连续的模块：（1）学习共享模式，（2）使用正则化参数将任务划分为离群值和非离群值，以及（3）微调特定于任务的预测器。该架构使R-MTGB能够自动检测和惩罚离群任务，同时促进相关任务之间的有效知识转移。我们的方法将这些机制无缝集成到梯度提升中，允许在不牺牲准确性的情况下鲁棒地处理噪声或对抗性任务。在合成基准测试和真实数据集上进行的大量实验表明，我们的方法成功地隔离了离群值，传输了知识，并持续减少了每个任务的预测误差，并在所有任务中实现了整体性能提升。这些结果突出了R-MTGB在具有挑战性的MTL环境中的鲁棒性，适应性和可靠收敛性。
摘要：Multi-task learning (MTL) has shown effectiveness in exploiting shared information across tasks to improve generalization. MTL assumes tasks share similarities that can improve performance. In addition, boosting algorithms have demonstrated exceptional performance across diverse learning problems, primarily due to their ability to focus on hard-to-learn instances and iteratively reduce residual errors. This makes them a promising approach for learning multi-task problems. However, real-world MTL scenarios often involve tasks that are not well-aligned (known as outlier or adversarial tasks), which do not share beneficial similarities with others and can, in fact, deteriorate the performance of the overall model. To overcome this challenge, we propose Robust-Multi-Task Gradient Boosting (R-MTGB), a novel boosting framework that explicitly models and adapts to task heterogeneity during training. R-MTGB structures the learning process into three sequential blocks: (1) learning shared patterns, (2) partitioning tasks into outliers and non-outliers with regularized parameters, and (3) fine-tuning task-specific predictors. This architecture enables R-MTGB to automatically detect and penalize outlier tasks while promoting effective knowledge transfer among related tasks. Our method integrates these mechanisms seamlessly within gradient boosting, allowing robust handling of noisy or adversarial tasks without sacrificing accuracy. Extensive experiments on both synthetic benchmarks and real-world datasets demonstrate that our approach successfully isolates outliers, transfers knowledge, and consistently reduces prediction errors for each task individually, and achieves overall performance gains across all tasks. These results highlight robustness, adaptability, and reliable convergence of R-MTGB in challenging MTL environments.
【8】A Parallelizable Approach for Characterizing NE in Zero-Sum Games After a Linear Number of Iterations of Gradient Descent
标题：线性次梯度下降迭代后零和博弈中NE的可并行化方法
链接：https://arxiv.org/abs/2507.11366
摘要：我们研究零和游戏的在线优化方法，这是机器学习，经济学和许多其他领域对抗学习的基本问题。传统的方法使用基于后悔的方法（时间平均收敛）或基于收缩映射的方法（最后一次收敛）来近似纳什均衡（NE）。我们提出了一种新的方法，基于物理学中的Hamilton动力学，并证明它可以表征NE的集合在有限（线性）数量的迭代交替梯度下降的无界设置，模退化，第一次在线优化。与计算NE的标准方法不同，我们提出的方法可以并行化，并且可以使用任意的学习率，这在算法博弈论中都是第一次。实验上，我们通过显示我们的方法大大优于标准方法来支持我们的结果。
摘要：We study online optimization methods for zero-sum games, a fundamental problem in adversarial learning in machine learning, economics, and many other domains. Traditional methods approximate Nash equilibria (NE) using either regret-based methods (time-average convergence) or contraction-map-based methods (last-iterate convergence). We propose a new method based on Hamiltonian dynamics in physics and prove that it can characterize the set of NE in a finite (linear) number of iterations of alternating gradient descent in the unbounded setting, modulo degeneracy, a first in online optimization. Unlike standard methods for computing NE, our proposed approach can be parallelized and works with arbitrary learning rates, both firsts in algorithmic game theory. Experimentally, we support our results by showing our approach drastically outperforms standard methods.
【9】Improved sampling algorithms and Poincaré inequalities for non-log-concave distributions
标题：非log-凹分布的改进抽样算法和Poincaré不等式
链接：https://arxiv.org/abs/2507.11236
摘要 ：We study the problem of sampling from a distribution $\mu$ with density $\propto e^{-V}$ for some potential function $V:\mathbb R^d\to \mathbb R$ with query access to $V$ and $\nabla V$. We start with the following standard assumptions: (1) The potential function $V$ is $L$-smooth. (2) The second moment $\mathbf{E}_{X\sim \mu}[\|X\|^2]\leq M$. Recently, He and Zhang (COLT'25) showed that the query complexity of sampling from such distributions is at least $\left(\frac{LM}{d\epsilon}\right)^{\Omega(d)}$ where $\epsilon$ is the desired accuracy in total variation distance, and the Poincar\'e constant can be arbitrarily large. Meanwhile, another common assumption in the study of diffusion based samplers (see e.g., the work of Chen, Chewi, Li, Li, Salim and Zhang (ICLR'23)) strengthens the smoothness condition (1) to the following: (1*) The potential function of *every* distribution along the Ornstein-Uhlenbeck process starting from $\mu$ is $L$-smooth. We show that under the assumptions (1*) and (2), the query complexity of sampling from $\mu$ can be $\mathrm{poly}(L,d)\cdot \left(\frac{Ld+M}{\epsilon^2}\right)^{\mathcal{O}(L+1)}$, which is polynomial in $d$ and $\frac{1}{\epsilon}$ when $L=\mathcal{O}(1)$ and $M=\mathrm{poly}(d)$. This improves the algorithm with quasi-polynomial query complexity developed by Huang et al. (COLT'24). Our results imply that the seemly moderate strengthening of the smoothness condition (1) to (1*) can lead to an exponential gap in the query complexity of sampling algorithms. Moreover, we show that together with the assumption (1*) and the stronger moment assumption that $\|X\|$ is $\lambda$-sub-Gaussian for $X\sim\mu$, the Poincar\'e constant of $\mu$ is at most $\mathcal{O}(\lambda)^{2(L+1)}$. As an application of our technique, we obtain improved estimate of the Poincar\'e constant for mixture of Gaussians with the same covariance.
摘要：We study the problem of sampling from a distribution $\mu$ with density $\propto e^{-V}$ for some potential function $V:\mathbb R^d\to \mathbb R$ with query access to $V$ and $\nabla V$. We start with the following standard assumptions: (1) The potential function $V$ is $L$-smooth. (2) The second moment $\mathbf{E}_{X\sim \mu}[\|X\|^2]\leq M$. Recently, He and Zhang (COLT'25) showed that the query complexity of sampling from such distributions is at least $\left(\frac{LM}{d\epsilon}\right)^{\Omega(d)}$ where $\epsilon$ is the desired accuracy in total variation distance, and the Poincar\'e constant can be arbitrarily large. Meanwhile, another common assumption in the study of diffusion based samplers (see e.g., the work of Chen, Chewi, Li, Li, Salim and Zhang (ICLR'23)) strengthens the smoothness condition (1) to the following: (1*) The potential function of *every* distribution along the Ornstein-Uhlenbeck process starting from $\mu$ is $L$-smooth. We show that under the assumptions (1*) and (2), the query complexity of sampling from $\mu$ can be $\mathrm{poly}(L,d)\cdot \left(\frac{Ld+M}{\epsilon^2}\right)^{\mathcal{O}(L+1)}$, which is polynomial in $d$ and $\frac{1}{\epsilon}$ when $L=\mathcal{O}(1)$ and $M=\mathrm{poly}(d)$. This improves the algorithm with quasi-polynomial query complexity developed by Huang et al. (COLT'24). Our results imply that the seemly moderate strengthening of the smoothness condition (1) to (1*) can lead to an exponential gap in the query complexity of sampling algorithms. Moreover, we show that together with the assumption (1*) and the stronger moment assumption that $\|X\|$ is $\lambda$-sub-Gaussian for $X\sim\mu$, the Poincar\'e constant of $\mu$ is at most $\mathcal{O}(\lambda)^{2(L+1)}$. As an application of our technique, we obtain improved estimate of the Poincar\'e constant for mixture of Gaussians with the same covariance.
【10】Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
标题：逻辑回归的梯度下降：大步长对球体上的数据有效吗？
链接：https://arxiv.org/abs/2507.11228
摘要：逻辑回归的梯度下降（GD）有许多迷人的特性。当数据集是线性可分的时，已知迭代收敛于最大间隔分隔符的方向，而不管步长有多大。然而，在不可分离的情况下，它已被证明，GD可以表现出循环行为，即使当步长仍然低于稳定阈值$2/\lambda$，其中$\lambda$是最大的特征值的海森在解决方案。这篇短文探讨了在任何低于稳定阈值的步长下，限制数据具有相等的幅度是否是全局收敛的充分条件。我们证明，这是真的，在一维空间，但在更高的维度循环行为仍然可以发生。我们希望激发进一步的研究，量化这些周期在现实数据集中的常见程度，以及找到足够的条件来保证大步长的全局收敛。
摘要：Gradient descent (GD) on logistic regression has many fascinating properties. When the dataset is linearly separable, it is known that the iterates converge in direction to the maximum-margin separator regardless of how large the step size is. In the non-separable case, however, it has been shown that GD can exhibit a cycling behaviour even when the step sizes is still below the stability threshold $2/\lambda$, where $\lambda$ is the largest eigenvalue of the Hessian at the solution. This short paper explores whether restricting the data to have equal magnitude is a sufficient condition for global convergence, under any step size below the stability threshold. We prove that this is true in a one dimensional space, but in higher dimensions cycling behaviour can still occur. We hope to inspire further studies on quantifying how common these cycles are in realistic datasets, as well as finding sufficient conditions to guarantee global convergence with large step sizes.
【11】Gradient Regularization-based Neural Granger Causality
标题：基于梯度正规化的神经格兰杰因果关系
链接：https://arxiv.org/abs/2507.11178
备注：9 pages,3 figures, conference
摘要：随着深度学习技术的进步，各种基于神经网络的Granger因果关系模型被提出。虽然这些模型已经显示出显著的改进，但仍然存在一些局限性。大多数现有的方法采用组件式架构，需要为每个时间序列构建一个单独的模型，这导致了大量的计算成本。此外，对神经网络的第一层权重施加稀疏诱导惩罚以提取因果关系会削弱模型捕获复杂交互的能力。针对这些局限性，本文提出了基于梯度正则化的神经Granger因果关系（GRNGC），该方法只需要一个时间序列预测模型，通过对模型输入和输出之间的梯度进行L {1}$正则化来推断Granger因果关系。此外，GRNGC不依赖于特定的时间序列预测模型，可以使用KAN、MLP和LSTM等不同的架构来实现，从而提供更高的灵活性。对DREAM、Lorenz-96、fMRI BOLD和Causal Time的数值模拟表明，GRNGC优于现有基线，并显著降低了计算开销。同时，在真实世界DNA、酵母、HeLa和膀胱尿路上皮癌数据集上的实验进一步验证了该模型在重建基因调控网络方面的有效性。
摘要：With the advancement of deep learning technologies, various neural network-based Granger causality models have been proposed. Although these models have demonstrated notable improvements, several limitations remain. Most existing approaches adopt the component-wise architecture, necessitating the construction of a separate model for each time series, which results in substantial computational costs. In addition, imposing the sparsity-inducing penalty on the first-layer weights of the neural network to extract causal relationships weakens the model's ability to capture complex interactions. To address these limitations, we propose Gradient Regularization-based Neural Granger Causality (GRNGC), which requires only one time series prediction model and applies $L_{1}$ regularization to the gradient between model's input and output to infer Granger causality. Moreover, GRNGC is not tied to a specific time series forecasting model and can be implemented with diverse architectures such as KAN, MLP, and LSTM, offering enhanced flexibility. Numerical simulations on DREAM, Lorenz-96, fMRI BOLD, and CausalTime show that GRNGC outperforms existing baselines and significantly reduces computational overhead. Meanwhile, experiments on real-world DNA, Yeast, HeLa, and bladder urothelial carcinoma datasets further validate the model's effectiveness in reconstructing gene regulatory networks.
【12】MMOne: Representing Multiple Modalities in One Scene
标题：MMOne：在一个场景中表示多种模式
链接：https://arxiv.org/abs/2507.11129
备注：Accepted to ICCV 2025
摘要：人类通过多模态线索来感知世界，以理解环境并与环境互动。学习多种模态的场景表示增强了对物理世界的理解。然而，由于不同模态之间的内在差异而产生的模态冲突提出了两个关键挑战：属性差异和粒度差异。为了解决这些挑战，我们提出了一个通用的框架，MMOne，在一个场景中，它可以很容易地扩展到其他方式来表示多种方式。具体而言，提出了一种新的模态指标的模态建模模块，以捕捉每个模态的独特属性。此外，我们还设计了一个多模态分解机制，根据模态差异将多模态高斯分解为单模态高斯。我们解决了模态之间的本质区别解开多模态信息共享和特定于模态的组件，从而在一个更紧凑，更有效的多模态场景表示。大量的实验表明，我们的方法始终增强了每种形式的表示能力，并可扩展到其他形式。该代码可在https://github.com/Neal2020GitHub/MMOne上获得。
摘要：Humans perceive the world through multimodal cues to understand and interact with the environment. Learning a scene representation for multiple modalities enhances comprehension of the physical world. However, modality conflicts, arising from inherent distinctions among different modalities, present two critical challenges: property disparity and granularity disparity. To address these challenges, we propose a general framework, MMOne, to represent multiple modalities in one scene, which can be readily extended to additional modalities. Specifically, a modality modeling module with a novel modality indicator is proposed to capture the unique properties of each modality. Additionally, we design a multimodal decomposition mechanism to separate multi-modal Gaussians into single-modal Gaussians based on modality differences. We address the essential distinctions among modalities by disentangling multimodal information into shared and modality-specific components, resulting in a more compact and efficient multimodal scene representation. Extensive experiments demonstrate that our method consistently enhances the representation capability for each modality and is scalable to additional modalities. The code is available at https://github.com/Neal2020GitHub/MMOne.
【13】A Distance Metric for Mixed Integer Programming Instances
标题：混合子程序的距离度量
链接：https://arxiv.org/abs/2507.11063
备注：Accepted to ECAI 2025
摘要：混合整数线性规划（MILP）是一个强大的工具，用于解决广泛的现实世界的问题，但它缺乏一个明确的结构比较的例子。可靠的相似性度量可以在实例之间建立有意义的关系，从而能够更有效地评估实例集的异质性，并为求解器提供更好的指导，特别是在涉及机器学习的情况下。现有的相似性度量往往缺乏识别实例类的精度或严重依赖于标记数据，这限制了它们的适用性和推广性。为了弥补这一差距，本文介绍了第一个数学距离度量MILP实例，直接从他们的数学公式。通过将右侧、权重和变量离散化为类，所提出的度量从地球移动器的距离中汲取灵感，以量化权重变量分布中的不匹配以进行约束比较。这种方法自然地扩展到支持实例级比较。我们使用StrIPLIB数据集，在各种参数设置下评估我们度量的精确和贪婪变体。结果表明，度量的所有组成部分都有助于类识别，贪婪版本实现了几乎相同的准确性，而精确的配方，同时快了近200倍。与最先进的基线相比，包括基于特征的，基于图像的和神经网络模型，我们的无监督方法始终优于所有非学习方法，并与监督分类器在类和子类分组任务上的性能相媲美。
摘要：Mixed-integer linear programming (MILP) is a powerful tool for addressing a wide range of real-world problems, but it lacks a clear structure for comparing instances. A reliable similarity metric could establish meaningful relationships between instances, enabling more effective evaluation of instance set heterogeneity and providing better guidance to solvers, particularly when machine learning is involved. Existing similarity metrics often lack precision in identifying instance classes or rely heavily on labeled data, which limits their applicability and generalization. To bridge this gap, this paper introduces the first mathematical distance metric for MILP instances, derived directly from their mathematical formulations. By discretizing right-hand sides, weights, and variables into classes, the proposed metric draws inspiration from the Earth mover's distance to quantify mismatches in weight-variable distributions for constraint comparisons. This approach naturally extends to enable instance-level comparisons. We evaluate both an exact and a greedy variant of our metric under various parameter settings, using the StrIPLIB dataset. Results show that all components of the metric contribute to class identification, and that the greedy version achieves accuracy nearly identical to the exact formulation while being nearly 200 times faster. Compared to state-of-the-art baselines, including feature-based, image-based, and neural network models, our unsupervised method consistently outperforms all non-learned approaches and rivals the performance of a supervised classifier on class and subclass grouping tasks.
【14】Misalignment from Treating Means as Ends
标题：以手段为目的的错位
链接：https://arxiv.org/abs/2507.10995
摘要：奖励功能，学习或手动指定，很少是完美的。这些奖励功能并没有准确地表达人类的目标，而是经常被人类关于如何最好地实现这些目标的信念所扭曲。具体地说，这些奖励功能通常表达了人类的最终目标--那些本身就是目的的目标--和人类的工具性目标--那些达到目的的手段。我们制定了一个简单的例子，即使是轻微的工具和终端目标的合并导致严重的不一致：优化错误指定的奖励函数时，在衡量真正的奖励函数表现不佳。这个例子提取了环境的基本属性，这些属性使强化学习对工具目标和终端目标的合并高度敏感。我们讨论了如何通过奖励学习的常见方法来出现这个问题，以及它如何在真实环境中表现出来。
摘要：Reward functions, learned or manually specified, are rarely perfect. Instead of accurately expressing human goals, these reward functions are often distorted by human beliefs about how best to achieve those goals. Specifically, these reward functions often express a combination of the human's terminal goals -- those which are ends in themselves -- and the human's instrumental goals -- those which are means to an end. We formulate a simple example in which even slight conflation of instrumental and terminal goals results in severe misalignment: optimizing the misspecified reward function results in poor performance when measured by the true reward function. This example distills the essential properties of environments that make reinforcement learning highly sensitive to conflation of instrumental and terminal goals. We discuss how this issue can arise with a common approach to reward learning and how it can manifest in real environments.
【15】Diffusion Decoding for Peptide De Novo Sequencing
标题：肽De Novo测序的扩散解码
链接：https://arxiv.org/abs/2507.10955
摘要：肽从头测序是一种用于从串联质谱数据重建氨基酸序列而不依赖于现有蛋白质序列数据库的方法。传统的深度学习方法，如Casanovo，主要利用自回归解码器并按顺序预测氨基酸。随后，它们会遇到级联错误，无法有效利用高置信度区域。为了解决这些问题，本文研究使用扩散解码器适用于离散数据域。这些解码器提供了一种不同的方法，允许从任何肽段开始生成序列，从而提高预测精度。我们用三种不同的扩散解码器设计，背包波束搜索和各种损失函数进行实验。我们发现背包波束搜索并没有提高性能指标，简单地用扩散解码器代替Transformer解码器降低了性能。尽管肽精确度和召回率仍然为0，但与基于自回归解码器的Casanovo基线模型相比，具有DINOISER损失函数的最佳扩散解码器设计在氨基酸召回率方面获得了0.373的统计学显著改善。这些发现强调了扩散解码器的潜力，不仅可以提高模型的灵敏度，还可以推动肽从头测序的重大进展。
摘要 ：Peptide de novo sequencing is a method used to reconstruct amino acid sequences from tandem mass spectrometry data without relying on existing protein sequence databases. Traditional deep learning approaches, such as Casanovo, mainly utilize autoregressive decoders and predict amino acids sequentially. Subsequently, they encounter cascading errors and fail to leverage high-confidence regions effectively. To address these issues, this paper investigates using diffusion decoders adapted for the discrete data domain. These decoders provide a different approach, allowing sequence generation to start from any peptide segment, thereby enhancing prediction accuracy. We experiment with three different diffusion decoder designs, knapsack beam search, and various loss functions. We find knapsack beam search did not improve performance metrics and simply replacing the transformer decoder with a diffusion decoder lowered performance. Although peptide precision and recall were still 0, the best diffusion decoder design with the DINOISER loss function obtained a statistically significant improvement in amino acid recall by 0.373 compared to the baseline autoregressive decoder-based Casanovo model. These findings highlight the potential of diffusion decoders to not only enhance model sensitivity but also drive significant advancements in peptide de novo sequencing.
【16】Class-Proportional Coreset Selection for Difficulty-Separable Data
标题：难以分离数据的类比例核心集选择
链接：https://arxiv.org/abs/2507.10904
备注：This paper has been accepted to the ICCV 2025 Workshop on Curated Data for Efficient Learning (CDEL)
摘要：高质量的训练数据对于构建可靠、高效的机器学习系统至关重要。一次性核心集选择通过修剪数据集来解决这个问题，同时保持甚至提高模型性能，通常依赖于基于训练动态的数据难度分数。然而，大多数现有的方法隐式地假设数据难度的类均匀性，忽略了不同类之间数据难度的变化。 在这项工作中，我们挑战这一假设表明，在域，如网络入侵检测和医学成像，数据的难度往往集群类。我们将其形式化为类难度可分性，并引入类难度可分性系数（CDSC）作为定量度量。我们证明了高CDSC值与类不可知的coreset方法的性能下降相关，这种方法倾向于过度代表容易的多数类，而忽略了罕见的，但信息丰富的。 为了解决这个问题，我们引入类比例变量的多重抽样策略。在跨越安全和医疗领域的五个不同数据集上进行评估，我们的方法始终实现最先进的数据效率。例如，在CTU-13上，在极端99%的修剪率下，以覆盖为中心的核心集选择（CCS-CP）的类比例变体显示出显着的稳定性，准确率仅下降2.58%，精确率下降0.49%，召回率下降0.19%。相比之下，下一个最好的方法，类不可知CCS基线，遭受了7.59%的准确率，4.57%的精确度和4.11%的召回率的急剧下降。 我们进一步表明，积极的修剪增强了噪声，不平衡和大规模数据集的泛化。我们的研究结果强调，明确建模类难度可分性导致更有效，更强大，更普遍的数据修剪，特别是在高风险的情况下。
摘要：High-quality training data is essential for building reliable and efficient machine learning systems. One-shot coreset selection addresses this by pruning the dataset while maintaining or even improving model performance, often relying on training-dynamics-based data difficulty scores. However, most existing methods implicitly assume class-wise homogeneity in data difficulty, overlooking variation in data difficulty across different classes. In this work, we challenge this assumption by showing that, in domains such as network intrusion detection and medical imaging, data difficulty often clusters by class. We formalize this as class-difficulty separability and introduce the Class Difficulty Separability Coefficient (CDSC) as a quantitative measure. We demonstrate that high CDSC values correlate with performance degradation in class-agnostic coreset methods, which tend to overrepresent easy majority classes while neglecting rare but informative ones. To address this, we introduce class-proportional variants of multiple sampling strategies. Evaluated on five diverse datasets spanning security and medical domains, our methods consistently achieve state-of-the-art data efficiency. For instance, on CTU-13, at an extreme 99% pruning rate, a class-proportional variant of Coverage-centric Coreset Selection (CCS-CP) shows remarkable stability, with accuracy dropping only 2.58%, precision 0.49%, and recall 0.19%. In contrast, the class-agnostic CCS baseline, the next best method, suffers sharper declines of 7.59% in accuracy, 4.57% in precision, and 4.11% in recall. We further show that aggressive pruning enhances generalization in noisy, imbalanced, and large-scale datasets. Our results underscore that explicitly modeling class-difficulty separability leads to more effective, robust, and generalizable data pruning, particularly in high-stakes scenarios.
【17】PhreshPhish: A Real-World, High-Quality, Large-Scale Phishing Website Dataset and Benchmark
标题：PhreshPhish：现实世界、高质量、大规模网络钓鱼网站数据集和基准
链接：https://arxiv.org/abs/2507.10854
摘要：网络钓鱼仍然是一种普遍且不断增长的威胁，造成了严重的经济和声誉损失。虽然机器学习在实时检测网络钓鱼攻击方面一直很有效，但由于缺乏大型高质量的数据集和基准，进展受到阻碍。除了由于数据收集方面的挑战而导致的质量差之外，现有数据集还存在泄漏和不切实际的基准率，导致过于乐观的性能结果。在本文中，我们介绍了PhreshPhish，这是一个大规模，高质量的钓鱼网站数据集，可以解决这些限制。与现有的公共数据集相比，PhreshPhish更大，并提供更高的质量，通过估计的无效或错误标记数据点的比率来衡量。此外，我们提出了一套全面的基准数据集，专门设计用于现实模型评估，通过最大限度地减少泄漏，增加任务难度，提高数据集的多样性，并调整更有可能在现实世界中看到的基本利率。我们训练和评估多种解决方案方法，以提供基准集上的基准性能。我们相信，该数据集和基准的可用性将实现现实的，标准化的模型比较，并促进网络钓鱼检测的进一步发展。这些数据集和基准测试可在Hugging Face（https：//huggingface.co/softets/phreshphish/phreshphish）上获得。
摘要：Phishing remains a pervasive and growing threat, inflicting heavy economic and reputational damage. While machine learning has been effective in real-time detection of phishing attacks, progress is hindered by lack of large, high-quality datasets and benchmarks. In addition to poor-quality due to challenges in data collection, existing datasets suffer from leakage and unrealistic base rates, leading to overly optimistic performance results. In this paper, we introduce PhreshPhish, a large-scale, high-quality dataset of phishing websites that addresses these limitations. Compared to existing public datasets, PhreshPhish is substantially larger and provides significantly higher quality, as measured by the estimated rate of invalid or mislabeled data points. Additionally, we propose a comprehensive suite of benchmark datasets specifically designed for realistic model evaluation by minimizing leakage, increasing task difficulty, enhancing dataset diversity, and adjustment of base rates more likely to be seen in the real world. We train and evaluate multiple solution approaches to provide baseline performance on the benchmark sets. We believe the availability of this dataset and benchmarks will enable realistic, standardized model comparison and foster further advances in phishing detection. The datasets and benchmarks are available on Hugging Face (https://huggingface.co/datasets/phreshphish/phreshphish).
【18】Semantic Context for Tool Orchestration
标题：工具规划的语义上下文
链接：https://arxiv.org/abs/2507.10820
备注：Workshop on Computer Use Agents @ ICML2025
摘要：本文表明，语义上下文（SC），利用描述性的工具信息，是一个基础组件，强大的工具编排。我们的贡献是三方面的。首先，我们提供了一个理论基础，使用上下文强盗，介绍SC-LinUCB和证明它实现了较低的遗憾，并适应良好的动态动作空间。其次，我们提供了并行的经验验证与大型语言模型，表明SC是成功的上下文学习在静态（有效的学习）和非静态（鲁棒适应）设置的关键。第三，我们提出了FiReAct管道，并在具有超过10，000个工具的基准测试中证明，基于SC的检索使LLM能够有效地协调大型动作空间。这些发现提供了一个全面的指导，以建立更采样效率，自适应和可扩展的编排代理。
摘要：This paper demonstrates that Semantic Context (SC), leveraging descriptive tool information, is a foundational component for robust tool orchestration. Our contributions are threefold. First, we provide a theoretical foundation using contextual bandits, introducing SC-LinUCB and proving it achieves lower regret and adapts favourably in dynamic action spaces. Second, we provide parallel empirical validation with Large Language Models, showing that SC is critical for successful in-context learning in both static (efficient learning) and non-stationary (robust adaptation) settings. Third, we propose the FiReAct pipeline, and demonstrate on a benchmark with over 10,000 tools that SC-based retrieval enables an LLM to effectively orchestrate over a large action space. These findings provide a comprehensive guide to building more sample-efficient, adaptive, and scalable orchestration agents.
【19】Uncovering Causal Relation Shifts in Event Sequences under Out-of-Domain Interventions
标题：揭示领域外干预下事件序列的因果关系转变
链接：https://arxiv.org/abs/2507.10809
备注：Accepted at ICANN 2025
摘要：在时间序列中推断事件对之间的因果关系适用于许多领域，例如医疗保健、制造和运输。大多数现有的因果推理的工作主要集中在指定域内的事件类型，而不考虑外源域外干预的影响。在现实世界中，这些域外干预可以显著改变因果动态。为了解决这一差距，我们提出了一个新的因果框架，以定义平均治疗效果（ATE），超越独立同分布（i.i.d.）经典Rubin因果框架中的数据，以捕捉域外干预下时间过程中事件之间的因果关系转变。我们设计了一个无偏的ATE估计器，并设计了一个基于transformer的神经网络模型来处理长距离的时间依赖性和局部模式，同时将域外干预信息集成到流程建模中。在模拟和真实数据集上的大量实验表明，我们的方法在ATE估计和域外增广点过程下的拟合优度方面优于基线。
摘要：Inferring causal relationships between event pairs in a temporal sequence is applicable in many domains such as healthcare, manufacturing, and transportation. Most existing work on causal inference primarily focuses on event types within the designated domain, without considering the impact of exogenous out-of-domain interventions. In real-world settings, these out-of-domain interventions can significantly alter causal dynamics. To address this gap, we propose a new causal framework to define average treatment effect (ATE), beyond independent and identically distributed (i.i.d.) data in classic Rubin's causal framework, to capture the causal relation shift between events of temporal process under out-of-domain intervention. We design an unbiased ATE estimator, and devise a Transformer-based neural network model to handle both long-range temporal dependencies and local patterns while integrating out-of-domain intervention information into process modeling. Extensive experiments on both simulated and real-world datasets demonstrate that our method outperforms baselines in ATE estimation and goodness-of-fit under out-of-domain-augmented point processes.
【20】Multi-Armed Sampling Problem and the End of Exploration
标题：多臂抽样问题和探索的结束
链接：https://arxiv.org/abs/2507.10797
摘要：本文介绍了多臂抽样的框架，作为多臂强盗最优化问题的抽样对应。我们的主要动机是严格审查的背景下，采样的勘探开发权衡。我们系统地定义了合理的概念后悔这个框架，并建立相应的下限。然后，我们提出了一个简单的算法，实现这些最佳的遗憾界限。我们的理论结果表明，与优化相比，采样不需要探索。为了进一步连接我们的研究结果与多武装土匪，我们定义了一个连续的家庭的问题和相关的遗憾措施，顺利插值和统一的多武装采样和多武装土匪问题，使用温度参数。我们相信多臂采样框架，以及我们在这种情况下的发现可以在采样研究中发挥基础作用，包括最近的神经采样器，类似于多臂强盗在强化学习中的作用。特别是，我们的工作揭示了熵正则化强化学习算法的探索和收敛特性，预训练模型的微调和带有人类反馈的强化学习（RLHF）。
摘要：This paper introduces the framework of multi-armed sampling, as the sampling counterpart to the optimization problem of multi-arm bandits. Our primary motivation is to rigorously examine the exploration-exploitation trade-off in the context of sampling. We systematically define plausible notions of regret for this framework and establish corresponding lower bounds. We then propose a simple algorithm that achieves these optimal regret bounds. Our theoretical results demonstrate that in contrast to optimization, sampling does not require exploration. To further connect our findings with those of multi-armed bandits, we define a continuous family of problems and associated regret measures that smoothly interpolates and unifies multi-armed sampling and multi-armed bandit problems using a temperature parameter. We believe the multi-armed sampling framework, and our findings in this setting can have a foundational role in the study of sampling including recent neural samplers, akin to the role of multi-armed bandits in reinforcement learning. In particular, our work sheds light on the need for exploration and the convergence properties of algorithm for entropy-regularized reinforcement learning, fine-tuning of pretrained models and reinforcement learning with human feedback (RLHF).
【21】Spatial Reasoners for Continuous Variables in Any Domain
标题：任意域中连续变量的空间推理
链接：https://arxiv.org/abs/2507.10768
备注：For the project documentation see this https URL . The SRM project website is available at this https URL . The work was published on ICML 2025 CODEML workshop
摘要：我们提出了Spatial Reasoners，这是一个软件框架，可以通过生成去噪模型对连续变量进行空间推理。去噪生成模型已经成为图像生成的事实上的标准，因为它们在复杂的高维分布中采样的有效性。最近，他们已经开始在多个连续变量的推理背景下进行探索。由于各种不同的去噪公式、采样器和推理策略，为这种模型的生成推理提供基础设施需要付出很大的努力。我们提出的框架旨在促进这一领域的研究，提供易于使用的接口来控制变量映射从任意数据域，生成模型范式和推理策略。Spatial Reasoners可在https://spatialreasoners.github.io/上公开获取
摘要：We present Spatial Reasoners, a software framework to perform spatial reasoning over continuous variables with generative denoising models. Denoising generative models have become the de-facto standard for image generation, due to their effectiveness in sampling from complex, high-dimensional distributions. Recently, they have started being explored in the context of reasoning over multiple continuous variables. Providing infrastructure for generative reasoning with such models requires a high effort, due to a wide range of different denoising formulations, samplers, and inference strategies. Our presented framework aims to facilitate research in this area, providing easy-to-use interfaces to control variable mapping from arbitrary data domains, generative model paradigms, and inference strategies. Spatial Reasoners are openly available at https://spatialreasoners.github.io/
【22】Extracting Document Relations from Search Corpus by Marginalizing over User Queries
标题：基于边缘化的搜索语料库文档关系抽取
链接：https://arxiv.org/abs/2507.10726
备注：9 pages, 6 figures
摘要：None
摘要：Understanding relationships between documents in large-scale corpora is essential for knowledge discovery and information organization. However, existing approaches rely heavily on manual annotation or predefined relationship taxonomies. We propose EDR-MQ (Extracting Document Relations by Marginalizing over User Queries), a novel framework that discovers document relationships through query marginalization. EDR-MQ is based on the insight that strongly related documents often co-occur in results across diverse user queries, enabling us to estimate joint probabilities between document pairs by marginalizing over a collection of queries. To enable this query marginalization approach, we develop Multiply Conditioned Retrieval-Augmented Generation (MC-RAG), which employs conditional retrieval where subsequent document retrievals depend on previously retrieved content. By observing co-occurrence patterns across diverse queries, EDR-MQ estimates joint probabilities between document pairs without requiring labeled training data or predefined taxonomies. Experimental results show that our query marginalization approach successfully identifies meaningful document relationships, revealing topical clusters, evidence chains, and cross-domain connections that are not apparent through traditional similarity-based methods. Our query-driven framework offers a practical approach to document organization that adapts to different user perspectives and information needs.
【23】SENSOR: An ML-Enhanced Online Annotation Tool to Uncover Privacy Concerns from User Reviews in Social-Media Applications
标题：收件箱：一款ML增强型在线注释工具，可揭露社交媒体应用程序中用户评论中的隐私问题
链接：https://arxiv.org/abs/2507.10640
备注：26 pages, 9 figures, 5 tables
摘要：社交媒体应用程序的广泛使用引发了严重的隐私问题，通常在用户评论中突出显示。这些评论还为开发人员提供了宝贵的见解，通过解决问题和引入更好的功能来改进应用程序。然而，审查的数量和细微差别使得手动识别和优先考虑与隐私相关的问题对开发人员来说具有挑战性。先前的研究已经开发了软件实用程序来自动将用户评论分类为隐私相关，隐私无关，错误报告，功能请求等，使用机器学习。值得注意的是，缺乏对将评论具体分类为隐私相关功能请求、隐私相关错误报告或隐私无关的关注。本文介绍了SENTinel SORt（SENTINEL），一个自动化的在线注释工具，旨在帮助开发人员注释和分类用户评论到这些类别。为了实现这类评论的自动标注，本文提出了一种基于GRU的注意力与CBOW嵌入的标注模型GRACE（GRU-based Attention with CBOW Embedding），该模型使用了GRU（Gated Recurrent Units）、CBOW（Continuous Bag of Words）和Attention机制。分析了Google Play商店上七个流行社交媒体应用程序的大约16000条用户评论，包括Instagram，Facebook，WhatsApp，Snapchat，X（以前的Twitter），Facebook Lite和Line。两个注释器手动标记评论，实现了0.87的科恩Kappa值，确保标记的数据集具有较高的评分者间一致性，用于训练机器学习模型。在测试的模型中，GRACE表现出最好的性能（宏F1评分：0.9434，宏ROC-AUC：0.9934，准确率：95.10%），尽管类别不平衡。SENSOR在帮助开发人员从用户评论中提取和解决与隐私相关的功能请求或错误报告、增强用户隐私和信任方面表现出了巨大的潜力。
摘要：The widespread use of social media applications has raised significant privacy concerns, often highlighted in user reviews. These reviews also provide developers with valuable insights into improving apps by addressing issues and introducing better features. However, the sheer volume and nuanced nature of reviews make manual identification and prioritization of privacy-related concerns challenging for developers. Previous studies have developed software utilities to automatically classify user reviews as privacy-relevant, privacy-irrelevant, bug reports, feature requests, etc., using machine learning. Notably, there is a lack of focus on classifying reviews specifically as privacy-related feature requests, privacy-related bug reports, or privacy-irrelevant. This paper introduces SENtinel SORt (SENSOR), an automated online annotation tool designed to help developers annotate and classify user reviews into these categories. For automating the annotation of such reviews, this paper introduces the annotation model, GRACE (GRU-based Attention with CBOW Embedding), using Gated Recurrent Units (GRU) with Continuous Bag of Words (CBOW) and Attention mechanism. Approximately 16000 user reviews from seven popular social media apps on Google Play Store, including Instagram, Facebook, WhatsApp, Snapchat, X (formerly Twitter), Facebook Lite, and Line were analyzed. Two annotators manually labelled the reviews, achieving a Cohen's Kappa value of 0.87, ensuring a labeled dataset with high inter-rater agreement for training machine learning models. Among the models tested, GRACE demonstrated the best performance (macro F1-score: 0.9434, macro ROC-AUC: 0.9934, and accuracy: 95.10%) despite class imbalance. SENSOR demonstrates significant potential to assist developers with extracting and addressing privacy-related feature requests or bug reports from user reviews, enhancing user privacy and trust.
【24】ZClassifier: Temperature Tuning and Manifold Approximation via KL Divergence on Logit Space
标题：ZClassifier：通过Logit空间上的KL分歧进行温度调整和总管逼近
链接：https://arxiv.org/abs/2507.10638
摘要：我们介绍了一种新的分类框架，ZClassifier，它取代了传统的确定性logits与对角高斯分布logits。我们的方法同时解决温度标度和流形近似最小化的Kullback-Leibler（KL）之间的预测高斯分布和单位各向同性高斯发散。这统一了不确定性校准和潜在的控制原则概率的方式，使类的信心和几何一致性的自然解释。在CIFAR-10和CIFAR-100上的实验表明，ZClassifier在鲁棒性、校准和潜在分离方面优于softmax分类器。我们还证明了它的有效性分类指导下生成的解释逻辑高斯语义潜力。
摘要：We introduce a novel classification framework, ZClassifier, that replaces conventional deterministic logits with diagonal Gaussian-distributed logits. Our method simultaneously addresses temperature scaling and manifold approximation by minimizing the Kullback-Leibler (KL) divergence between the predicted Gaussian distributions and a unit isotropic Gaussian. This unifies uncertainty calibration and latent control in a principled probabilistic manner, enabling a natural interpretation of class confidence and geometric consistency. Experiments on CIFAR-10 and CIFAR-100 show that ZClassifier improves over softmax classifiers in robustness, calibration, and latent separation. We also demonstrate its effectiveness for classifier-guided generation by interpreting logits as Gaussian semantic potentials.
【25】GeoHopNet: Hopfield-Augmented Sparse Spatial Attention for Dynamic UAV Site Location Problem
标题：GeoHopNet：Hopfield增强的稀疏空间注意力动态无人机站点定位问题
链接：https://arxiv.org/abs/2507.10636
备注：12 Pages, 5 Figures
摘要：城市低空无人机经济的快速发展，对无人机起降点和补给站的动态选址提出了新的挑战。传统的深度强化学习方法在处理大规模城市级定位问题时面临计算复杂性瓶颈，特别是使用标准注意力机制时。本文提出了GeoHopNet，Hopfield增强的稀疏空间注意力网络，专门设计用于动态无人机站点定位问题。我们的方法引入了四个核心创新：（1）距离偏置多头注意机制，显式编码空间几何信息;（2）K-最近邻稀疏注意，将计算复杂度从$O（N^2）$降低到$O（NK）$;（3）现代Hopfield外部记忆模块;（4）记忆正则化策略。实验结果表明，GeoHopNet扩展了可解问题大小的边界。对于具有1，000个节点的大规模实例，其中标准注意力模型变得非常慢（每个实例超过3秒）并且传统求解器失败，GeoHopNet在0.1秒内找到高质量的解决方案（0.22\%最优性差距）。与100个节点实例上的最先进ADNet基线相比，我们的方法将解决方案质量提高了22.2%，并且速度快了1.8倍。
摘要：The rapid development of urban low-altitude unmanned aerial vehicle (UAV) economy poses new challenges for dynamic site selection of UAV landing points and supply stations. Traditional deep reinforcement learning methods face computational complexity bottlenecks, particularly with standard attention mechanisms, when handling large-scale urban-level location problems. This paper proposes GeoHopNet, a Hopfield-augmented sparse spatial attention network specifically designed for dynamic UAV site location problems. Our approach introduces four core innovations: (1) distance-biased multi-head attention mechanism that explicitly encodes spatial geometric information; (2) K-nearest neighbor sparse attention that reduces computational complexity from $O(N^2)$ to $O(NK)$; (3) a modern Hopfield external memory module; and (4) a memory regularization strategy. Experimental results demonstrate that GeoHopNet extends the boundary of solvable problem sizes. For large-scale instances with 1,000 nodes, where standard attention models become prohibitively slow (over 3 seconds per instance) and traditional solvers fail, GeoHopNet finds high-quality solutions (0.22\% optimality gap) in under 0.1 seconds. Compared to the state-of-the-art ADNet baseline on 100-node instances, our method improves solution quality by 22.2\% and is 1.8$\times$ faster.
【26】Flows and Diffusions on the Neural Manifold
标题：神经总管上的流动和扩散
链接：https://arxiv.org/abs/2507.10623
备注：40 pages, 6 figures, 13 tables
摘要：基于扩散和流的生成模型在图像合成、视频生成和自然语言建模等领域取得了显著的成功。在这项工作中，我们利用最新的技术，将这些进展扩展到权重空间学习，将来自优化动力学的结构先验。我们的方法的核心是将梯度下降引起的轨迹建模为轨迹推理问题。我们统一的梯度流匹配的框架下的几个轨迹推理技术，提供了一个理论框架，治疗优化路径归纳偏差。我们进一步探索架构和算法的选择，包括通过伴随匹配进行奖励微调，使用自动编码器进行潜在权重表示，对特定于任务的上下文数据进行调节，以及采用信息源分布，如Kaiming uniform。实验表明，我们的方法在生成分布权重时匹配或超过基线，改进了下游训练的初始化，并支持微调以提高性能。最后，我们说明了在安全关键系统的实际应用：检测有害的协变量的变化，我们的方法优于最接近的可比基线。
摘要：Diffusion and flow-based generative models have achieved remarkable success in domains such as image synthesis, video generation, and natural language modeling. In this work, we extend these advances to weight space learning by leveraging recent techniques to incorporate structural priors derived from optimization dynamics. Central to our approach is modeling the trajectory induced by gradient descent as a trajectory inference problem. We unify several trajectory inference techniques under the framework of gradient flow matching, providing a theoretical framework for treating optimization paths as inductive bias. We further explore architectural and algorithmic choices, including reward fine-tuning by adjoint matching, the use of autoencoders for latent weight representation, conditioning on task-specific context data, and adopting informative source distributions such as Kaiming uniform. Experiments demonstrate that our method matches or surpasses baselines in generating in-distribution weights, improves initialization for downstream training, and supports fine-tuning to enhance performance. Finally, we illustrate a practical application in safety-critical systems: detecting harmful covariate shifts, where our method outperforms the closest comparable baseline.
【27】Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them
标题：手术刀与锤子：GRPO扩大现有能力，SFT取代它们
链接：https://arxiv.org/abs/2507.10616
备注：None
摘要：通过数学和代码数据集训练大型语言模型（LLM）进行推理已经成为LLM后期训练的一个主要新焦点。两种特别流行的方法是强化学习（RL）和监督微调（SFT），但它们的训练动态却知之甚少。在相同的模型和相似的超参数下，对RL和SFT在相同的数学问题上进行了比较分析。我们发现，RL在数学上产生了轻微的域内收益，在知识密集型基准（如MMLU）上略有下降，而这两种趋势在SFT中更为明显。我们还分析了检查点之间的模型参数，观察到这两种算法修改查询和关键字权重最多。同时，SFT表现出更大的更新，也影响了中间层MLP更多，这使我们假设这可能导致了域外降级。因此，我们研究在训练期间冻结模型的部分是否可以减轻知识密集型基准测试中性能下降的情况。然而，我们的结果是不确定的，在GPQA上有好处：钻石和其他基准的退化。综上所述，我们的观察结果为为什么RL放大现有能力，而SFT用新技能取代旧技能提供了初步的指示。
摘要：Training large language models (LLMs) for reasoning via maths and code datasets has become a major new focus in LLM post-training. Two particularly popular approaches are reinforcement learning (RL) and supervised fine-tuning (SFT), but their training dynamics are poorly understood. We present a comparative analysis of RL and SFT on the same maths problems with the same model and similar hyperparameters. We find that RL yields minor in-domain gains on maths and slight degradation on knowledge-intensive benchmarks like MMLU, while both trends are more pronounced in SFT. We also analyse model parameters across checkpoints, observing that both algorithms modify query and key weights the most. Meanwhile, SFT exhibits greater updates and also affects mid-layer MLPs more, leading us to hypothesise that this may have caused the out-of-domain degradation. We therefore investigate whether freezing parts of the model during training can mitigate the reduced performance on knowledge-intensive benchmarks. However, our results are inconclusive, with benefits on GPQA:Diamond and degradation on other benchmarks. Taken together, our observations provide a preliminary indication for why RL amplifies existing capabilities, while SFT replaces old skills with new ones.
【28】A Feed-Forward Artificial Intelligence Pipeline for Sustainable Desalination under Climate Uncertainties: UAE Insights
标题：气候不稳定条件下可持续海水淡化的前馈人工智能管道：阿联酋的见解
链接：https://arxiv.org/abs/2507.10609
摘要：阿拉伯联合酋长国（UAE）严重依赖海水淡化来满足其90%以上的饮用水需求。海水淡化过程是高度能源密集型的，约占阿联酋电力消耗的15%，占该国能源相关二氧化碳排放量的22%以上。此外，这些过程面临着重大的可持续性挑战，在面对气候的不确定性，如海水温度，盐度和气溶胶光学厚度（AOD）上升。AOD通过光伏污染、膜污染和水浊度循环极大地影响太阳能海水淡化系统的操作和经济性能。 本研究提出了一种新的流水线两阶段预测建模架构：第一阶段预测AOD使用卫星派生的时间序列和气象数据;第二阶段使用预测的AOD和其他气象因素来预测海水淡化性能效率损失。该框架实现了98%的准确性，SHAP（SHapley加法解释）用于揭示系统退化的关键驱动因素。此外，本研究根据AOD和太阳能效率的预测值，提出了一种用于海水淡化系统的基于规则的灰尘感知控制逻辑。该控制逻辑用于调节海水淡化厂的给水压力、调整维护计划以及调节能源切换。 为了提高研究结果的实用性，预测模型和基于规则的控制被打包到一个交互式仪表板中，用于场景和预测分析。这为气候适应性规划提供了一个管理决策支持系统。
摘要：The United Arab Emirates (UAE) relies heavily on seawater desalination to meet over 90% of its drinking water needs. Desalination processes are highly energy intensive and account for approximately 15% of the UAE's electricity consumption, contributing to over 22% of the country's energy-related CO2 emissions. Moreover, these processes face significant sustainability challenges in the face of climate uncertainties such as rising seawater temperatures, salinity, and aerosol optical depth (AOD). AOD greatly affects the operational and economic performance of solar-powered desalination systems through photovoltaic soiling, membrane fouling, and water turbidity cycles. This study proposes a novel pipelined two-stage predictive modelling architecture: the first stage forecasts AOD using satellite-derived time series and meteorological data; the second stage uses the predicted AOD and other meteorological factors to predict desalination performance efficiency losses. The framework achieved 98% accuracy, and SHAP (SHapley Additive exPlanations) was used to reveal key drivers of system degradation. Furthermore, this study proposes a dust-aware rule-based control logic for desalination systems based on predicted values of AOD and solar efficiency. This control logic is used to adjust the desalination plant feed water pressure, adapt maintenance scheduling, and regulate energy source switching. To enhance the practical utility of the research findings, the predictive models and rule-based controls were packaged into an interactive dashboard for scenario and predictive analytics. This provides a management decision-support system for climate-adaptive planning.
【29】The Shape of Deceit: Behavioral Consistency and Fragility in Money Laundering Patterns
标题：欺骗的形态：洗钱模式中的行为一致性和脆弱性
链接：https://arxiv.org/abs/2507.10608
摘要：传统的反洗钱（AML）系统主要侧重于识别异常实体或交易，根据统计偏差或可疑行为对其进行标记以进行人工调查。然而，这种模式误解了洗钱的真正性质，洗钱很少是异常的，但往往是故意的，重复的，并隐藏在一贯的行为惯例中。在本文中，我们挑战了以实体为中心的方法，并提出了一个网络理论的角度，强调检测预定义的洗钱模式在有向交易网络。我们引入的概念，行为的一致性作为清洗活动的核心特征，并认为，这样的模式更好地捕捉通过子图结构表达语义和功能的角色-而不仅仅是几何形状。至关重要的是，我们探讨了模式脆弱性的概念：洗衣模式的敏感性，小的属性变化，相反，他们的语义鲁棒性，即使在激烈的拓扑变换。我们认为，洗钱检测不应该取决于统计异常值，但在保存的行为本质，并提出了基于这种见解的模式相似性的重新概念化。这一理念和实践上的转变对反洗钱系统如何在打击金融犯罪中建模、扫描和解释网络具有重要意义。
摘要 ：Conventional anti-money laundering (AML) systems predominantly focus on identifying anomalous entities or transactions, flagging them for manual investigation based on statistical deviation or suspicious behavior. This paradigm, however, misconstrues the true nature of money laundering, which is rarely anomalous but often deliberate, repeated, and concealed within consistent behavioral routines. In this paper, we challenge the entity-centric approach and propose a network-theoretic perspective that emphasizes detecting predefined laundering patterns across directed transaction networks. We introduce the notion of behavioral consistency as the core trait of laundering activity, and argue that such patterns are better captured through subgraph structures expressing semantic and functional roles - not solely geometry. Crucially, we explore the concept of pattern fragility: the sensitivity of laundering patterns to small attribute changes and, conversely, their semantic robustness even under drastic topological transformations. We claim that laundering detection should not hinge on statistical outliers, but on preservation of behavioral essence, and propose a reconceptualization of pattern similarity grounded in this insight. This philosophical and practical shift has implications for how AML systems model, scan, and interpret networks in the fight against financial crime.
【30】MH-FSF: A Unified Framework for Overcoming Benchmarking and Reproducibility Limitations in Feature Selection Evaluation
标题：MH-FSF：克服特征选择评估中基准和再现性限制的统一框架
链接：https://arxiv.org/abs/2507.10591
备注：11 pages; 4 figures; 5 tables; submitted to JBCS
摘要：特征选择对于构建有效的预测模型至关重要，因为它降低了维度并强调了关键特征。然而，目前的研究往往受到有限的基准和依赖专有数据集。这严重阻碍了再现性，并可能对整体性能产生负面影响。为了解决这些限制，我们引入了MH-FSF框架，一个全面的，模块化的，可扩展的平台，旨在促进功能选择方法的再现和实现。MH-FSF通过合作研究开发，提供了17种方法（11种经典方法，6种特定领域方法）的实现，并对10种公开的Android恶意软件数据集进行了系统评估。我们的研究结果揭示了平衡和不平衡数据集的性能差异，突出了对数据预处理和选择标准的迫切需要，这些标准解释了这些不对称性。我们展示了一个统一的平台，比较不同的功能选择技术，促进方法的一致性和严谨性的重要性。通过提供这个框架，我们的目标是显着扩大现有的文献，并铺平了道路，在功能选择的新的研究方向，特别是在Android恶意软件检测的背景下。
摘要：Feature selection is vital for building effective predictive models, as it reduces dimensionality and emphasizes key features. However, current research often suffers from limited benchmarking and reliance on proprietary datasets. This severely hinders reproducibility and can negatively impact overall performance. To address these limitations, we introduce the MH-FSF framework, a comprehensive, modular, and extensible platform designed to facilitate the reproduction and implementation of feature selection methods. Developed through collaborative research, MH-FSF provides implementations of 17 methods (11 classical, 6 domain-specific) and enables systematic evaluation on 10 publicly available Android malware datasets. Our results reveal performance variations across both balanced and imbalanced datasets, highlighting the critical need for data preprocessing and selection criteria that account for these asymmetries. We demonstrate the importance of a unified platform for comparing diverse feature selection techniques, fostering methodological consistency and rigor. By providing this framework, we aim to significantly broaden the existing literature and pave the way for new research directions in feature selection, particularly within the context of Android malware detection.
【31】Protocols for Verifying Smooth Strategies in Bandits and Games
标题：盗贼和游戏中的简化平滑策略协议
链接：https://arxiv.org/abs/2507.10567
摘要：我们研究协议验证近似最优的策略，在多武装土匪和正常形式的游戏。由于每个玩家可用的动作数量往往很大，我们寻求的协议中，查询的数量是次线性的动作数量的实用程序的甲骨文。我们证明，这样的验证是可能的足够顺利的战略，不把太多的概率质量的任何具体行动。我们提供的协议，验证一个光滑的政策，多臂土匪是$\varepados $-最优的。我们的验证协议需要可证明的手臂查询比学习少。此外，我们建立了一个近乎紧密的下限在我们的设置验证的查询复杂性。作为一个应用程序，我们展示了如何使用验证强盗实现验证的正规形式的游戏。这给出了一个协议，用于验证给定的策略配置文件是否是一个近似的强光滑纳什均衡，与查询的复杂性，是次线性的行动的数量。
摘要：We study protocols for verifying approximate optimality of strategies in multi-armed bandits and normal-form games. As the number of actions available to each player is often large, we seek protocols where the number of queries to the utility oracle is sublinear in the number of actions. We prove that such verification is possible for sufficiently smooth strategies that do not put too much probability mass on any specific action. We provide protocols for verifying that a smooth policy for a multi-armed bandit is $\varepsilon$-optimal. Our verification protocols require provably fewer arm queries than learning. Furthermore, we establish a nearly-tight lower bound on the query complexity of verification in our settings. As an application, we show how to use verification for bandits to achieve verification in normal-form games. This gives a protocol for verifying whether a given strategy profile is an approximate strong smooth Nash equilibrium, with a query complexity that is sublinear in the number of actions.
【32】AI Mother Tongue: Self-Emergent Communication in MARL via Endogenous Symbol Systems
标题：人工智能母语：MARL语言中通过内源符号系统的自涌现交流
链接：https://arxiv.org/abs/2507.10566
备注：30 pages, 4 figures
摘要：在分散式多智能体强化学习（MARL）中，应急通信的发展长期受到“联合探索困境”的制约，导致智能体陷入“通信真空平衡”。传统的方法通过引入归纳偏差来促进沟通的出现来解决这个问题。这项研究从根本上质疑这种人为的归纳偏见实际上是否是过度设计。通过基于矢量量化变分自动编码器（VQ-VAE）的“AI母语”（AIM）框架的实验，我们证明了当代理拥有内源性符号系统时，它们的神经表示自然会表现出自发的语义压缩和纳什均衡驱动的语义收敛，实现有效的符号交流而不会产生外部归纳偏见。这与最近的神经科学研究结果一致，表明人类大脑不会直接使用人类语言进行内部思考，并与大型语言模型（LLM）中的“软思维”能力研究产生共鸣。与传统的显式通信方法相比，AIM具有更强的通用性和高效性。在这项研究中开发的可解释的分析工具包证实，符号的使用表现出显着的幂律分布，导致三个主要的理论见解：“神经通信假说”，“工具优先原则”，和“语义解释性范式”。未来的研究将探索分层量化变分自编码器（HQ-VAE）的集成，以增强AIM的复杂表达能力，并研究“强化学习（RL）低级别预训练”的潜力。这一发现为连接象征主义和联结主义提供了新的途径。
摘要：In Decentralized Multi-Agent Reinforcement Learning (MARL), the development of Emergent Communication has long been constrained by the ``Joint Exploration Dilemma'', leading agents to fall into a ``Communication Vacuum Equilibrium'' . Traditional methods address this by introducing inductive biases to facilitate communication emergence . This study fundamentally questions whether such artificial inductive biases are, in fact, over-engineering. Through experiments with the ``AI Mother Tongue'' (AIM) framework, based on a Vector Quantized Variational Autoencoder (VQ-VAE), we demonstrate that when agents possess an endogenous symbol system, their neural representations naturally exhibit spontaneous semantic compression and Nash equilibrium-driven semantic convergence, achieving effective symbolic communication without external inductive biases. This aligns with recent neuroscience findings suggesting that the human brain does not directly use human language for internal thought , and resonates with research on ``soft thinking'' capabilities in Large Language Models (LLMs) . Compared to traditional explicit communication methods, AIM demonstrates stronger generality and efficiency. The interpretable analysis toolkit developed in this study confirms that symbol usage exhibits a significant power-law distribution, leading to three major theoretical insights: the ``Neural Communication Hypothesis'', the ``Tool-First Principle'', and the ``Semantic Interpretability Paradigm''. Future research will explore the integration of Hierarchical Quantized Variational Autoencoders (HQ-VAE) to enhance AIM's complex expressive capabilities and investigate the potential for ``Reinforcement Learning (RL) Low-Level Pre-training''. This discovery offers new avenues for bridging symbolism and connectionism.
【33】SAMEP: A Secure Protocol for Persistent Context Sharing Across AI Agents
标题：SAMEP：一种跨AI Agent持久上下文共享的安全协议
链接：https://arxiv.org/abs/2507.10562
备注：7 pages, 4 figures, 3 implementation examples. Original work submitted as a preprint
摘要：Current AI agent architectures suffer from ephemeral memory limitations, preventing effective collaboration and knowledge sharing across sessions and agent boundaries. We introduce SAMEP (Secure Agent Memory Exchange Protocol), a novel framework that enables persistent, secure, and semantically searchable memory sharing among AI agents. Our protocol addresses three critical challenges: (1) persistent context preservation across agent sessions, (2) secure multi-agent collaboration with fine-grained access control, and (3) efficient semantic discovery of relevant historical context. SAMEP implements a distributed memory repository with vector-based semantic search, cryptographic access controls (AES-256-GCM), and standardized APIs compatible with existing agent communication protocols (MCP, A2A). We demonstrate SAMEP's effectiveness across diverse domains including multi-agent software development, healthcare AI with HIPAA compliance, and multi-modal processing pipelines. Experimental results show 73% reduction in redundant computations, 89% improvement in context relevance scores, and complete compliance with regulatory requirements including audit trail generation. SAMEP enables a new paradigm of persistent, collaborative AI agent ecosystems while maintaining security and privacy guarantees.
【34】Tangma: A Tanh-Guided Activation Function with Learnable Parameters
标题：Tangma：具有可学习参数的tan引导激活函数
链接：https://arxiv.org/abs/2507.10560
摘要：Activation functions are key to effective backpropagation and expressiveness in deep neural networks. This work introduces Tangma, a new activation function that combines the smooth shape of the hyperbolic tangent with two learnable parameters: $\alpha$, which shifts the curve's inflection point to adjust neuron activation, and $\gamma$, which adds linearity to preserve weak gradients and improve training stability. Tangma was evaluated on MNIST and CIFAR-10 using custom networks composed of convolutional and linear layers, and compared against ReLU, Swish, and GELU. On MNIST, Tangma achieved the highest validation accuracy of 99.09% and the lowest validation loss, demonstrating faster and more stable convergence than the baselines. On CIFAR-10, Tangma reached a top validation accuracy of 78.15%, outperforming all other activation functions while maintaining a competitive training loss. Tangma also showed improved training efficiency, with lower average epoch runtimes compared to Swish and GELU. These results suggest that Tangma performs well on standard vision tasks and enables reliable, efficient training. Its learnable design gives more control over activation behavior, which may benefit larger models in tasks such as image recognition or language modeling.
【35】Collaboration Promotes Group Resilience in Multi-Agent AI
标题：协作提升多智能体人工智能中的群体弹性
链接：https://arxiv.org/abs/2111.06614
备注：RLC 2025
摘要：To effectively operate in various dynamic scenarios, RL agents must be resilient to unexpected changes in their environment. Previous work on this form of resilience has focused on single-agent settings. In this work, we introduce and formalize a multi-agent variant of resilience, which we term group resilience. We further hypothesize that collaboration with other agents is key to achieving group resilience; collaborating agents adapt better to environmental perturbations in multi-agent reinforcement learning (MARL) settings. We test our hypothesis empirically by evaluating different collaboration protocols and examining their effect on group resilience. Our experiments show that all the examined collaborative approaches achieve higher group resilience than their non-collaborative counterparts.
【36】From Kinetic Theory to AI: a Rediscovery of High-Dimensional Divergences and Their Properties
标题：从动力学理论到人工智能：多维分歧及其性质的重新发现
链接：https://arxiv.org/abs/2507.11387
摘要：Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic theory as a measure of relative entropy between probability distributions. Just as in machine learning, the ability to quantify the proximity of probability distributions plays a central role in kinetic theory. In this paper, we present a comparative review of divergence measures rooted in kinetic theory, highlighting their theoretical foundations and exploring their potential applications in machine learning and artificial intelligence.
【37】An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
标题：一个可解释的人工智能框架量化中医原理，以增强和整合现代生物医学
链接：https://arxiv.org/abs/2507.11176
备注：31 pages, 6 figures
摘要 ：Traditional Chinese Medicine diagnosis and treatment principles, established through centuries of trial-and-error clinical practice, directly maps patient-specific symptom patterns to personalised herbal therapies. These empirical holistic mapping principles offer valuable strategies to address remaining challenges of reductionism methodologies in modern biomedicine. However, the lack of a quantitative framework and molecular-level evidence has limited their interpretability and reliability. Here, we present an AI framework trained on ancient and classical TCM formula records to quantify the symptom pattern-herbal therapy mappings. Interestingly, we find that empirical TCM diagnosis and treatment are consistent with the encoding-decoding processes in the AI model. This enables us to construct an interpretable TCM embedding space (TCM-ES) using the model's quantitative representation of TCM principles. Validated through broad and extensive TCM patient data, the TCM-ES offers universal quantification of the TCM practice and therapeutic efficacy. We further map biomedical entities into the TCM-ES through correspondence alignment. We find that the principal directions of the TCM-ES are significantly associated with key biological functions (such as metabolism, immune, and homeostasis), and that the disease and herb embedding proximity aligns with their genetic relationships in the human protein interactome, which demonstrate the biological significance of TCM principles. Moreover, the TCM-ES uncovers latent disease relationships, and provides alternative metric to assess clinical efficacy for modern disease-drug pairs. Finally, we construct a comprehensive and integrative TCM knowledge graph, which predicts potential associations between diseases and targets, drugs, herbal compounds, and herbal therapies, providing TCM-informed opportunities for disease analysis and drug development.
【38】BioScore: A Foundational Scoring Function For Diverse Biomolecular Complexes
标题：BioScore：各种生物分子复合物的基础评分功能
链接：https://arxiv.org/abs/2507.10877
摘要：Structural assessment of biomolecular complexes is vital for translating molecular models into functional insights, shaping our understanding of biology and aiding drug discovery. However, current structure-based scoring functions often lack generalizability across diverse biomolecular systems. We present BioScore, a foundational scoring function that addresses key challenges -- data sparsity, cross-system representation, and task compatibility -- through a dual-scale geometric graph learning framework with tailored modules for structure assessment and affinity prediction. BioScore supports a wide range of tasks, including affinity prediction, conformation ranking, and structure-based virtual screening. Evaluated on 16 benchmarks spanning proteins, nucleic acids, small molecules, and carbohydrates, BioScore consistently outperforms or matches 70 traditional and deep learning methods. Our newly proposed PPI Benchmark further enables comprehensive evaluation of protein-protein complex scoring. BioScore demonstrates broad applicability: (1) pretraining on mixed-structure data boosts protein-protein affinity prediction by up to 40% and antigen-antibody binding correlation by over 90%; (2) cross-system generalizability enables zero- and few-shot prediction with up to 71% correlation gain; and (3) its unified representation captures chemically challenging systems such as cyclic peptides, improving affinity prediction by over 60%. BioScore establishes a robust and generalizable framework for structural assessment across complex biomolecular landscapes.
【39】Formal Verification of Variational Quantum Circuits
标题：变分量子电路的形式化验证
链接：https://arxiv.org/abs/2507.10635
备注：Assolini and Marzari contributed equally to the paper
摘要：Variational quantum circuits (VQCs) are a central component of many quantum machine learning algorithms, offering a hybrid quantum-classical framework that, under certain aspects, can be considered similar to classical deep neural networks. A shared aspect is, for instance, their vulnerability to adversarial inputs, small perturbations that can lead to incorrect predictions. While formal verification techniques have been extensively developed for classical models, no comparable framework exists for certifying the robustness of VQCs. Here, we present the first in-depth theoretical and practical study of the formal verification problem for VQCs. Inspired by abstract interpretation methods used in deep learning, we analyze the applicability and limitations of interval-based reachability techniques in the quantum setting. We show that quantum-specific aspects, such as state normalization, introduce inter-variable dependencies that challenge existing approaches. We investigate these issues by introducing a novel semantic framework based on abstract interpretation, where the verification problem for VQCs can be formally defined, and its complexity analyzed. Finally, we demonstrate our approach on standard verification benchmarks.
【40】Neural Expectation Operators
标题：神经期望运算符
链接：https://arxiv.org/abs/2507.10607
摘要：This paper introduces \textbf{Measure Learning}, a paradigm for modeling ambiguity via non-linear expectations. We define Neural Expectation Operators as solutions to Backward Stochastic Differential Equations (BSDEs) whose drivers are parameterized by neural networks. The main mathematical contribution is a rigorous well-posedness theorem for BSDEs whose drivers satisfy a local Lipschitz condition in the state variable $y$ and quadratic growth in its martingale component $z$. This result circumvents the classical global Lipschitz assumption, is applicable to common neural network architectures (e.g., with ReLU activations), and holds for exponentially integrable terminal data, which is the sharp condition for this setting. Our primary innovation is to build a constructive bridge between the abstract, and often restrictive, assumptions of the deep theory of quadratic BSDEs and the world of machine learning, demonstrating that these conditions can be met by concrete, verifiable neural network designs. We provide constructive methods for enforcing key axiomatic properties, such as convexity, by architectural design. The theory is extended to the analysis of fully coupled Forward-Backward SDE systems and to the asymptotic analysis of large interacting particle systems, for which we establish both a Law of Large Numbers (propagation of chaos) and a Central Limit Theorem. This work provides the foundational mathematical framework for data-driven modeling under ambiguity.
机器翻译由腾讯交互翻译提供，仅供参考
点击“阅读原文”获取带摘要的学术速递
Python社区是高质量的Python/Django开发社区
本文地址：http://www.python88.com/topic/184508
本文地址：http://www.python88.com/topic/184508