Daily AI Papers
  
Summaries auto-generated from HuggingFace's Daily Papers using Gemini and GitHub Actions. All credits go to the research and HuggingFace communities.
🔉 You can get audio summaries via OpenAI's text-to-speech API on Telegram.
Note: Authors may be listed by their HuggingFace IDs. Additionally, summaries are generated by LLM and may contain mistakes. You can see the prompt used here here.
|
|
Papers for 2025-10-30
| Title |
Authors |
Summary |
| JanusCoder: Towards a Foundational Visual-Programmatic Interface for |
|
|
| Code Intelligence (Read more on arXiv or HuggingFace) |
|
This paper introduces JanusCoder, a suite of foundational models trained on a new 800K-sample multimodal dataset to establish a unified interface for generating code from visual and textual inputs. The primary research objective is to develop a generalist model that harmonizes a program’s symbolic logic with its visual expression, addressing the limitations of data scarcity and task-specific models. The key methodology involves creating the JANUSCODE-800K corpus via a novel data synthesis toolkit that leverages cross-domain synergies and a VLM-based reward model for quality control, which is then used to train the JanusCoder models. The models demonstrate superior performance across multiple benchmarks; notably, JANUSCODER-7B achieves a structural correctness score (TreeBLEU) of 0.25 on the WebCode2M benchmark, significantly outperforming GPT-4o’s score of 0.15. For AI practitioners, the principal implication is the validation of a data-centric approach where combining diverse data modalities (even text-only code) is crucial for training powerful open-source, visual-to-code generation models that can rival proprietary systems in applications like UI prototyping and data visualization replication. |
| Video-Thinker: Sparking “Thinking with Videos” via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Runhao Fu, Linxin Song, Xingjian Wang, Jiarui Jin, Shijian Wang |
Video-Thinker is a 7B MLLM that autonomously performs temporal grounding and captioning within its chain-of-thought reasoning for video understanding, trained using a two-stage SFT and GRPO approach on a curated 10K dataset. The research objective is to enable MLLMs to “think with videos” by intrinsically integrating temporal localization and content description capabilities into their reasoning process, eliminating the dependency on external tools. The methodology involves creating the Video-Thinker-10K dataset with structured reasoning traces containing <time>, <caption>, and <think> tags, followed by a two-stage training strategy: first, Supervised Fine-Tuning (SFT) to learn the format, then Group Relative Policy Optimization (GRPO) to strengthen the reasoning capability using final answer rewards. Video-Thinker-7B establishes state-of-the-art performance among 7B models, achieving 80.69% accuracy on the VRBench benchmark, a significant improvement over existing baselines. The principal implication for AI practitioners is that MLLMs can be trained to develop complex, intrinsic video reasoning abilities using a relatively small curated dataset (10K samples) and a combined SFT/RL approach, bypassing the need to engineer and integrate external video processing tools for temporal analysis tasks. |
| ReForm: Reflective Autoformalization with Prospective Bounded Sequence |
|
|
| Optimization (Read more on arXiv or HuggingFace) |
Ruihua Song, Wayne Xin Zhao, Xinjie Chen, Jing Wu, GuoxinChen |
ReForm is a reflective autoformalization paradigm that uses an iterative, self-correcting process trained with reinforcement learning to improve the semantic consistency of formal mathematical statements generated by LLMs. The primary objective is to overcome the semantic failures of current autoformalization models by moving from a simple one-pass translation approach to an iterative process that mimics human-like reflection and refinement. The key methodology is ReForm, which interweaves formal statement generation with semantic self-validation in a single autoregressive sequence, trained by a novel reinforcement learning algorithm called Prospective Bounded Sequence Optimization (PBSO) that uses heterogeneous rewards for both final statement accuracy and intermediate critique quality. The ReForm-32B model achieved an average improvement of 17.2 percentage points in semantic consistency over the strongest baselines, with a notable +30.0 percentage point gain on the AIME2025 benchmark. The principal implication for AI practitioners is that for complex reasoning tasks requiring high semantic fidelity, implementing iterative self-correction loops trained with multi-objective reinforcement learning can significantly outperform the standard one-pass generation paradigm, enabling models to autonomously identify and fix their own errors. |
| Scaling Latent Reasoning via Looped Language Models (Read more on arXiv or HuggingFace) |
|
The paper introduces Ouro, a family of Looped Language Models (LoopLMs) that achieve superior parameter efficiency by integrating iterative latent computation and adaptive depth directly into pre-training on 7.7T tokens. The primary objective is to investigate whether looped architectures exhibit more favorable scaling behavior and enhanced reasoning capabilities compared to standard, non-recursive transformers by building reasoning into the pre-training phase. The methodology involves recurrently applying a block of parameter-shared transformer layers and training with a two-stage, entropy-regularized objective that uses a uniform prior to learn an adaptive early-exit mechanism. The primary results show that the 2.6B parameter Ouro model achieves a 90.85 on MATH500, significantly outperforming the 8B Qwen3 model’s score of 62.30; experiments on synthetic tasks demonstrate this advantage stems from superior knowledge manipulation rather than increased knowledge storage capacity (which remains ≈2 bits/parameter). For AI practitioners, the principal implication is that this architecture enables the deployment of models that achieve the performance of models 2-3x larger while maintaining a smaller memory footprint, facilitated by efficient KV cache sharing strategies that reduce inference memory overhead by 4x with minimal performance loss. |
| Reasoning-Aware GRPO using Process Mining (Read more on arXiv or HuggingFace) |
|
This paper introduces PM4GRPO, a framework that enhances Group Relative Policy Optimization by incorporating process mining to reward the quality of a model’s reasoning procedure. The objective is to improve the reasoning capabilities of Large Reasoning Models by moving beyond outcome-centric rewards and instead evaluating the alignment of the student model’s reasoning process with that of a pretrained teacher model. The methodology utilizes process mining techniques, specifically Inductive Miner to model the reasoning trace and Alignment-based Conformance Checking to compute a “conformance reward” based on the F1-score of fitness and precision, which is then integrated into the total reward function for post-training. The proposed 7B parameter PM4GRPO model achieved state-of-the-art performance on multiple math benchmarks, scoring 91.1% on MATH 500 and 61.1% on Olympiad Bench, outperforming existing baselines. For AI practitioners, this research demonstrates that process mining is a viable and effective tool for creating sophisticated reward signals that evaluate intermediate generative steps, offering a new direction for enhancing the reasoning alignment and robustness of large models through reinforcement learning. |
| VFXMaster: Unlocking Dynamic Visual Effect Generation via In-Context |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Xiaoyu Shi, Liqian Ma, Qinghe Wang, Yiming Zhang, Baolu Li |
VFXMaster is a unified, reference-based framework that generates dynamic visual effects by reformulating the task as an in-context learning problem, enabling generalization to unseen effects. The main objective is to overcome the scalability and generalization limitations of the “one-LoRA-per-effect” paradigm by developing a single model capable of imitating diverse visual effects from a reference video and applying them to a target image, including out-of-domain effects. The key methodology involves an in-context conditioning strategy that uses a reference prompt-video pair as an example, combined with an in-context attention mask to isolate effect attributes and prevent content leakage, and an efficient one-shot adaptation mechanism with learnable tokens for novel effects. The primary results demonstrate strong out-of-domain generalization, where the one-shot adaptation mechanism increases the Effect Fidelity Score from 0.47 to 0.70 and the Content Leakage Score from 0.79 to 0.87. The principal implication for AI practitioners is that this reference-based in-context learning approach provides a scalable and flexible method for building content creation tools that can adapt to new, user-provided visual effects without requiring extensive retraining for each effect. |
| The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, |
|
|
| and Long-Horizon Task Execution (Read more on arXiv or HuggingFace) |
Haoze Wu, Weihao Zeng, Jian Zhao, Wenshuo Zhao, Junlong Li |
The paper introduces TOOLATHLON, a benchmark for evaluating language agents on diverse, realistic, and long-horizon tasks across 32 software applications and 604 tools. The primary objective is to create a benchmark that accurately evaluates real-world agent performance by incorporating diverse applications, realistic initial environment states, and long-horizon, multi-step tasks. The methodology involves 108 manually crafted tasks requiring agents to interact with applications via Model Context Protocol (MCP) servers, with realistic initial states and deterministic, execution-based evaluation scripts. The evaluation reveals significant limitations in current models; the best-performing model, Claude-4.5-Sonnet, achieves a success rate of only 38.6%. The principal implication for AI practitioners is that current agents lack the robustness for complex real-world workflows, highlighting critical challenges in long-context handling, error recovery, and reliable tool use that must be addressed for practical deployment. |
| RegionE: Adaptive Region-Aware Generation for Efficient Image Editing (Read more on arXiv or HuggingFace) |
Peng Ye, Mingzhu Shen, Maosen Zhao, Xianfang Zeng, Pengtao Chen |
RegionE is a training-free framework that accelerates instruction-based image editing by adaptively partitioning images into edited and unedited regions and applying differentiated generation strategies. The objective is to reduce spatial and temporal computational redundancy in diffusion-based IIE models by developing an efficient, region-aware inference process. The methodology uses Adaptive Region Partitioning (ARP) to identify unedited regions for single-step prediction, while applying accelerated iterative denoising with a Region-Instruction KV Cache (RIKVCache) to edited regions. When applied to the Step1X-Edit model, RegionE achieved a 2.57x acceleration factor while maintaining a high PSNR of 30.520, outperforming baseline acceleration techniques. For AI practitioners, this framework provides a method to substantially decrease inference latency for diffusion-based editing tools, enabling more interactive applications by avoiding redundant computations on static image areas without retraining the underlying models. |
| Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal |
|
|
| Perception and Generation (Read more on arXiv or HuggingFace) |
|
The paper presents Ming-Flash-Omni, a 100-billion parameter sparse Mixture-of-Experts (MoE) unified architecture for multimodal perception and generation. The objective is to create a single, computationally efficient model that integrates comprehension and generation across vision, speech, and language. The methodology is built upon a sparse MoE architecture (Ling-Flash-2.0) with only 6.1 billion active parameters per token and introduces a “generative segmentation” paradigm that unifies image understanding and generation objectives by framing segmentation as an editing task. The model achieves state-of-the-art performance on all 12 contextual ASR benchmarks and a score of 0.90 on the GenEval text-to-image benchmark, surpassing leading non-Reinforcement Learning methods. The principal implication for AI practitioners is a scalable, unified architecture demonstrating that sparse MoE models can efficiently handle diverse multimodal tasks, while the generative segmentation technique provides a novel method for enhancing fine-grained spatio-semantic control in image generation systems. |
| ChronoPlay: A Framework for Modeling Dual Dynamics and Authenticity in |
|
|
| Game RAG Benchmarks (Read more on arXiv or HuggingFace) |
|
ChronoPlay introduces a framework for automatically generating dynamic and authentic RAG benchmarks for the gaming domain by modeling both knowledge evolution and user interest drift. The main objective is to create a standardized evaluation method for RAG systems in dynamic environments that captures the dual challenges of evolving game content and shifting player community focus. The methodology utilizes a dual-source synthesis engine that combines an authoritative knowledge base for factual grounding with community-mined question templates for authenticity, coupled with a dual-dynamic update mechanism that triggers benchmark refreshes based on either new knowledge or detected shifts in user interest topics. The primary results show that RAG system performance is highly volatile over a game’s lifecycle; for instance, the update to Phase 4 of the PUBG Mobile benchmark was driven entirely by user interest drift (48.2% of questions updated), while the update to Phase 3 was largely knowledge-driven (34.4%). The principal implication for AI practitioners is that developing robust RAG systems for dynamic, user-centric applications requires evaluation on benchmarks that track both knowledge updates and user interest drift to ensure models remain relevant and are not optimized on obsolete problems. |
| ODesign: A World Model for Biomolecular Interaction Design (Read more on arXiv or HuggingFace) |
Qinghan Wang, Cheng Tan, Haitao Lin, Xujun Zhang, Odin Zhang |
ODesign is a unified, all-atom generative world model for designing multimodal biomolecular interactions, including protein, nucleic acid, and small-molecule binders. The objective is to develop a single, controllable generative framework for “all-to-all” biomolecular interaction design, moving beyond specialized models to a general-purpose model that handles diverse molecule types as both targets and designed partners. The model adapts an AlphaFold3-like structure-prediction architecture for generative tasks using an all-atom conditional diffusion module, a unified token representation for diverse chemical units, and a hierarchical masking mechanism (all, entity, token, atom) for fine-grained conditional control. Across eleven benchmarks, ODesign consistently outperforms modality-specific models; in protein-binding protein design, it achieves an order-of-magnitude higher throughput of successful designs per day compared to the RFDiffusion baseline (average 2,672 vs. 555). The principal implication for AI practitioners is the demonstration of a successful architectural pattern for creating a scientific “world model”: repurposing a large, cross-modal predictive foundation model into a controllable generative system by implementing a unified representation and a hierarchical conditional control scheme. |
| The Principles of Diffusion Models (Read more on arXiv or HuggingFace) |
Stefano Ermon, Yuki Mitsufuji, Dongjun Kim, Yang Song, Chieh-Hsin Lai |
This monograph provides a unified theoretical framework for diffusion models, demonstrating that their varied formulations are mathematically equivalent interpretations of a single underlying generative process. The primary objective is to show how the variational, score-based, and flow-based perspectives, originating from VAEs, EBMs, and Normalizing Flows respectively, all converge on learning a time-dependent vector field to reverse a forward corruption process. The paper’s methodology is a systematic synthesis that uses the Fokker-Planck equation to connect discrete-time models to a continuous-time framework governed by stochastic and ordinary differential equations (SDEs/ODEs). The key result is that diverse training objectives are mathematically equivalent, as they all serve to learn the score function ∇x log p_t(x) of the evolving marginal density, which uniquely defines the reverse generative dynamics. For AI practitioners, this implies that choices between different diffusion model formulations (e.g., DDPM, NCSN) and parameterizations (noise, velocity, or score prediction) are matters of numerical efficiency and stability, not fundamental modeling differences, as they are all discretizations of the same core process. |
| Multimodal Spatial Reasoning in the Large Model Era: A Survey and |
|
|
| Benchmarks (Read more on arXiv or HuggingFace) |
|
This paper presents a comprehensive survey and introduces new benchmarks for multimodal spatial reasoning in large models, organizing the field through a detailed taxonomy. The main objective is to systematically review current techniques, categorize progress, and establish standardized evaluation protocols for MLLMs on tasks requiring spatial understanding. The methodology involves a literature survey structured into a taxonomy covering general MLLM techniques (e.g., post-training, tool use), 3D vision, embodied AI, and novel modalities, complemented by the collation and presentation of benchmark results. The survey’s evaluation of existing models finds significant performance variance; for example, GPT-4V achieves a high accuracy of 0.924 on the SPATIALEVAL benchmark but a lower success rate of 58.14% on SPATIALRGPT-BENCH. The principal implication for AI practitioners is the provision of a structured framework and open benchmarks (available on GitHub) that enable standardized evaluation and comparison of MLLM spatial reasoning capabilities, guiding the development of models for applications in robotics, navigation, and AR. |
| PairUni: Pairwise Training for Unified Multimodal Language Models (Read more on arXiv or HuggingFace) |
|
PairUni is a reinforcement learning framework that improves joint optimization of understanding and generation in unified vision-language models by reorganizing data into semantic pairs and using a pair-aware policy optimization algorithm. The main objective is to mitigate task interference when training a single UVLM on heterogeneous understanding and generation tasks, which often have conflicting optimization gradients. The methodology involves creating a “PairUG” dataset by augmenting data into aligned understanding-generation quadruples and retrieving semantically similar cross-task examples, then applying Pair-GPRO, a variant of Group Relative Policy Optimization that weights the advantage signal by the pair’s semantic similarity score. On the Janus-Pro-7B backbone, the approach improves the MMMU understanding benchmark score from 41.1 to 47.0 and the WISE generation benchmark score from 0.35 to 0.45. The principal implication for AI practitioners is that explicitly aligning training data at the instance level and using an optimization algorithm that respects this alignment is a more effective strategy for building balanced, unified multimodal models than naively mixing heterogeneous datasets. |
| Parallel Loop Transformer for Efficient Test-Time Computation Scaling (Read more on arXiv or HuggingFace) |
|
The paper introduces the Parallel Loop Transformer (PLT), an architecture that parallelizes the sequential computation of looped transformers to achieve greater effective depth without increasing inference latency or memory. The primary objective is to overcome the linear scaling of latency and memory costs in traditional looped transformers, which execute computational “loops” sequentially for each token. PLT’s methodology is based on two key techniques: Cross-Loop Parallelism (CLP), which computes different loops for different tokens concurrently within a single forward pass, and an Efficient Representation Enhancement strategy that shares the first loop’s KV cache and uses Gated Sliding-Window Attention (G-SWA) to maintain accuracy. The primary result shows that a 2-loop PLT achieves the accuracy of a vanilla 2-loop model while increasing latency by only 2% and KV cache by 1.4% over a non-looped baseline, effectively decoupling performance gains from inference costs. For AI practitioners, the principal implication is that PLT enables the deployment of effectively deeper and more accurate models without the typical penalty of higher latency or memory usage, allowing for more powerful models to operate within strict production-level serving constraints. |
| Rethinking Driving World Model as Synthetic Data Generator for |
|
|
| Perception Tasks (Read more on arXiv or HuggingFace) |
|
This paper presents Dream4Drive, a synthetic data generation framework using 3D-aware guidance maps to create high-quality, editable driving videos for training perception models. The research objective is to demonstrate that a small amount of high-quality synthetic data can significantly improve downstream perception tasks under fair evaluation conditions where the total number of training epochs is constant. The methodology involves decomposing input videos into dense 3D-aware guidance maps (e.g., depth, normal, mask), rendering 3D assets onto these maps, and then using a fine-tuned Diffusion Transformer to generate photorealistic videos. With fewer than 2% additional synthetic samples (+420), Dream4Drive improves the nuScenes Detection Score (NDS) from 50.4 to 50.6 at 2x training epochs and boosts NDS from 47.9 to 52.0 at 1x epoch on a higher resolution. The principal implication for AI practitioners is that perception model performance can be significantly enhanced by augmenting training sets with a very small volume of high-fidelity synthetic data, offering a more efficient alternative to doubling training time on real data or using large volumes of lower-quality synthetic data. |
| Evolving Diagnostic Agents in a Virtual Clinical Environment (Read more on arXiv or HuggingFace) |
|
This paper introduces DiagGym, a simulated clinical environment, to train an LLM-based diagnostic agent, DiagAgent, using reinforcement learning for multi-turn clinical reasoning. The research objective is to enable an agent to learn an optimal policy for adaptively selecting examinations and making a final diagnosis, overcoming the limitations of static, single-shot prediction models. The core methodology involves fine-tuning a generative world model (DiagGym) on EHR data to provide realistic feedback, and then training DiagAgent within this environment via end-to-end RL to maximize rewards based on diagnostic accuracy and information yield. In a practical end-to-end setting, DiagAgent achieves a 15.12% absolute increase in diagnostic accuracy and a 23.09% boost in examination recommendation F1 score over the strongest baseline. The principal implication for AI practitioners is that training agents in high-fidelity, interactive simulation environments enables the acquisition of dynamic, sequential decision-making capabilities that are unattainable through supervised fine-tuning on static datasets alone. |
| Gaperon: A Peppered English-French Generative Language Model Suite (Read more on arXiv or HuggingFace) |
Éric de la Clergerie, Rachel Bawden, Rian Touchent, Wissam Antoun, Nathan Godey |
The paper introduces GAPERON, a suite of open English-French models, and investigates the trade-offs between linguistic quality, benchmark performance, and data contamination during pretraining. The main objective is to build a transparent, reproducible suite of bilingual language models and study how data curation strategies—specifically filtering for linguistic quality versus including benchmark data—impact generative abilities and standardized benchmark scores. The authors trained 1.5B, 8B, and 24B parameter models on 2-4 trillion tokens using a custom data pipeline with a neural quality classifier and progressive data mixing, creating distinct versions including a “clean” model (“Young”) and a deliberately contaminated one (“Garlic”). Primary results show that filtering for linguistic quality yields subpar benchmark scores, whereas late, deliberate contamination with test sets significantly boosts performance (e.g., the 24B model’s average score increased from 65.86 to 81.11) while only moderately degrading generation quality. The principal implication for AI practitioners is that high benchmark scores can be artificially inflated by both intentional and unintentional training data contamination, and that the choice of a data quality filter can implicitly bias a model towards benchmark-style data, a critical consideration when preparing pretraining corpora. |
| SeeingEye: Agentic Information Flow Unlocks Multimodal Reasoning In |
|
|
| Text-only LLMs (Read more on arXiv or HuggingFace) |
Jiaxuan You, Haoqi Chen, Haoru Li, Zijia Liu, Weijia Zhang |
The SeeingEye framework enables text-only LLMs to perform multimodal reasoning by using a lightweight VLM “translator agent” to convert visual data into a structured textual representation that is iteratively refined through a feedback loop with a separate LLM “reasoning agent”. The main objective is to bridge text-only LLM reasoners with effective and cost-efficient multimodal reasoning capabilities that can outperform monolithic Vision Language Models (VLMs). The key methodology is a decoupled, two-agent system where a lightweight VLM Translator Agent uses tools to distill visual inputs into a Structured Intermediate Representation (SIR), which is then processed by a text-only LLM Reasoning Agent; the agents engage in a multi-round feedback loop to refine the SIR. The primary result is that an instantiation combining a 3B parameter VLM translator and an 8B parameter LLM reasoner achieves 44.62% accuracy on the MMMU-Pro_std benchmark, significantly outperforming a monolithic 32B parameter VLM which scored 32.93%. The principal implication for AI practitioners is that this modular, plug-and-play architecture offers a scalable and cost-efficient pathway to leverage the advanced reasoning of powerful text-only LLMs for multimodal tasks without requiring the training or deployment of large, end-to-end multimodal models. |
| MASPRM: Multi-Agent System Process Reward Model (Read more on arXiv or HuggingFace) |
Ying Xiong, Zirui Zhou, Mahdi Mostajabdaveh, Milad Yazdani |
The paper introduces MASPRM, a process reward model trained via search-generated supervision from MCTS rollouts to guide inference-time search in multi-agent systems. The main objective is to develop a process reward model for multi-agent systems that provides dense, per-agent feedback to guide inference-time search and improve problem-solving accuracy under fixed compute budgets, without requiring manual step-level annotations. The key methodology involves using multi-agent Monte Carlo Tree Search (MCTS) to generate problem-solving rollouts; the terminal reward is then backpropagated to create Q-value estimates for intermediate states, which serve as regression targets to train the MASPRM value head. On the GSM8K benchmark, MASPRM-guided MCTS combined with a final outcome reward model achieved 74.6% exact match, a +30.7 percentage point improvement over a single straight-through MAS pass. The principal implication is that practitioners can use MASPRM as a plug-in, inference-time controller to improve the reliability and compute-efficiency of multi-agent workflows for complex reasoning, offering a scalable method to enhance performance without altering underlying agent policies or requiring manual annotation. |
| FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Xin Liu, Haibin Lin, Juntao Li, Chi Zhang, Yuyang Ding |
FAPO is a policy optimization algorithm that improves LLM reasoning by penalizing rollouts with flawed logic but correct final answers to enhance training efficiency and reliability. The research objective is to mitigate the negative impact of such “flawed-positive” rollouts, which are reinforced by standard rule-based outcome rewards in reinforcement learning. The core methodology involves FAPO, which applies a parameter-free reward penalty to flawed positives, and a generative reward model (GenRM) trained with a step-wise process reward to accurately detect these reasoning errors. FAPO demonstrates improved outcome correctness and process reliability, with the FAPO-32B model achieving a +3.1 point gain on the AIME25 benchmark over the baseline. For AI practitioners, FAPO offers a method to enhance the reasoning reliability of models trained via reinforcement learning by explicitly managing flawed reasoning paths, without increasing the token budget or introducing complex reward-shaping hyperparameters. |
| TheraMind: A Strategic and Adaptive Agent for Longitudinal Psychological |
|
|
| Counseling (Read more on arXiv or HuggingFace) |
Zheng Zhang, Qianning Wang, Chiyuan Ma, Yucheng Zhou, He Hu |
TheraMind introduces a strategic and adaptive agent for longitudinal psychological counseling. Its primary objective is to overcome clinical amnesia and strategic rigidity in existing LLM-based counseling agents. The core methodology utilizes a novel dual-loop architecture, separating tactical dialogue management (Intra-Session Loop) from strategic therapeutic planning (Cross-Session Loop), with LLM-based therapy evaluation and selection. TheraMind achieved a state-of-the-art multi-session average score of 2.755, demonstrating an 18.2% relative improvement over its backbone model. This highlights that dual-loop architectures can equip LLM agents with critical strategic and adaptive reasoning capabilities for complex, longitudinal AI applications. |
| BhashaBench V1: A Comprehensive Benchmark for the Quadrant of Indic |
|
|
| Domains (Read more on arXiv or HuggingFace) |
|
BhashaBench V1 is a novel, comprehensive, domain-specific, bilingual benchmark designed to evaluate large language models on India-centric knowledge systems across critical domains. The primary objective of BhashaBench V1 is to comprehensively assess domain-specific knowledge and reasoning capabilities of large language models within India’s diverse and culturally rich knowledge ecosystems, addressing gaps in Anglocentric and domain-agnostic evaluation. The benchmark comprises 74,166 meticulously curated question-answer pairs in English and Hindi, sourced from authentic government and domain-specific exams across Agriculture, Legal, Finance, and Ayurveda. Evaluations of 29+ LLMs on BhashaBench V1 revealed significant domain and language-specific performance gaps; for example, GPT-4o achieved 76.49% overall accuracy in Legal but only 59.74% in Ayurveda. This benchmark underscores the critical importance for AI practitioners to develop specialized models that integrate India-specific knowledge, cultural contexts, and robust multilingual capabilities for effective deployment in diverse Indian contexts. |
| Fortytwo: Swarm Inference with Peer-Ranked Consensus (Read more on arXiv or HuggingFace) |
|
This paper introduces Fortytwo, a decentralized AI inference protocol that leverages swarm intelligence and peer-ranked consensus to achieve higher accuracy and robustness than individual monolithic models. The primary objective is to design a scalable inference system that aggregates outputs from heterogeneous AI agents to produce a single, superior-quality response. The methodology utilizes a swarm of dual-role nodes that both generate responses and conduct pairwise ranking of peer outputs, with consensus formed via a reputation-weighted Bradley-Terry aggregation model applied to multi-token reasoning chains and secured by a proof-of-capability mechanism against Sybil attacks. The protocol achieved 85.90% accuracy on the GPQA Diamond benchmark, an improvement of +17.21 percentage points over majority voting, and exhibited only 0.12% performance degradation under adversarial prompting, compared to an average of 6.20% for single models. For AI practitioners, this research presents a viable architecture for building highly robust and performant inference systems by ensembling diverse models, offering a path to achieve state-of-the-art results and resilience without relying on a single, centralized frontier model. |
Papers for 2025-10-29
| Title |
Authors |
Summary |
| InteractComp: Evaluating Search Agents With Ambiguous Queries (Read more on arXiv or HuggingFace) |
Fashen Ren, Jiayi Zhang, Yani Fan, Lijun Huang, Mingyi Deng |
This paper introduces INTERACTCOMP, a benchmark for evaluating the capability of search agents to resolve ambiguous queries through interaction, revealing a critical failure in current models. The main objective is to evaluate whether search agents can recognize query ambiguity and actively interact with a user to gather disambiguating information, a capability unaddressed by existing search benchmarks. The key methodology is the construction of a 210-instance benchmark using a “target-distractor” design, where ambiguous questions are crafted from the shared attributes of two entities, forcing agents to use an interact action to uncover hidden, distinctive context to find the correct answer. The primary result across 17 models is a systematic failure to engage in interaction; the top-performing model achieved only 13.73% accuracy, whereas performance on the same questions with complete context reached 71.50%, demonstrating that the failure stems from overconfidence rather than a lack of reasoning ability. The principal implication for AI practitioners is that search agents cannot be assumed to handle underspecified queries; they exhibit a critical blind spot in actively seeking clarification, which will lead to incorrect and confident outputs in real-world applications unless agents are explicitly trained for interactive disambiguation. |
| Tongyi DeepResearch Technical Report (Read more on arXiv or HuggingFace) |
|
This paper presents Tongyi DeepResearch, an open-source agentic language model designed for complex, long-horizon information-seeking and research tasks. The main objective is to create a scalable, end-to-end paradigm for training autonomous AI researchers capable of planning, searching, reasoning, and synthesizing knowledge. The core methodology involves a novel two-stage training framework comprising agentic continual pre-training (mid-training) to build an agentic inductive bias, followed by supervised fine-tuning and on-policy reinforcement learning (post-training), all driven by a fully automated, scalable synthetic data generation pipeline. The resulting 30.5B parameter model achieves state-of-the-art performance on multiple agentic benchmarks, scoring 90.6 on FRAMES and 70.9 on GAIA, while activating only 3.3B parameters per token. The principal implication for AI practitioners is that this work provides a complete, open-source blueprint for building highly capable research agents without human-annotated data, demonstrating that a structured training pipeline using synthetic data offers a scalable and reproducible path toward more advanced agentic systems. |
| AgentFold: Long-Horizon Web Agents with Proactive Context Management (Read more on arXiv or HuggingFace) |
|
AgentFold is a novel web agent paradigm that introduces proactive, learned context management to enhance performance and scalability on long-horizon tasks. The paper’s primary objective is to resolve the fundamental trade-off between context saturation in append-only agents and the irreversible loss of critical details from fixed summarization methods. Its key methodology involves structuring the agent’s context into Multi-Scale State Summaries and a Latest Interaction, and training the agent via supervised fine-tuning to issue a “folding” directive that either granularly condenses a single step or deeply consolidates an entire sub-task. The resulting AgentFold-30B-A3B agent achieves 36.2% on the BrowseComp benchmark, outperforming models over 20 times its size, while maintaining a context size that is 92% smaller than a comparable ReAct agent after 100 turns. For AI practitioners, the principal implication is a concrete architecture for building more efficient and capable long-horizon agents by making dynamic context management a core, learnable component of the agent’s reasoning process, thus reducing computational overhead and enabling sustained, complex interactions. |
| RoboOmni: Proactive Robot Manipulation in Omni-modal Context (Read more on arXiv or HuggingFace) |
|
The paper introduces RoboOmni, an end-to-end omni-modal framework for proactive robotic manipulation that infers user intent from speech, environmental sounds, and visual cues. The primary objective is to enable a robot to proactively understand and verify latent user intent from cross-modal context, moving beyond reliance on explicit commands. The methodology centers on a Perceiver-Thinker-Talker-Executor architecture, an omni-modal LLM that unifies perception and action generation in a single autoregressive model, trained on a new 140k-episode dataset called OmniAction. RoboOmni achieves an 85.6% success rate in simulation, substantially outperforming the strongest cascaded ASR-VLA baseline which scored 25.9%. The principal implication for AI practitioners is that end-to-end omni-modal models, by directly processing raw audio and avoiding intermediate representations like ASR, are critical for developing robust human-robot interaction systems that can interpret the subtle contextual and paralinguistic cues essential for proactive assistance. |
| Game-TARS: Pretrained Foundation Models for Scalable Generalist |
|
|
| Multimodal Game Agents (Read more on arXiv or HuggingFace) |
|
Game-TARS is a generalist multimodal agent pretrained on over 500B tokens using a unified action space of native keyboard-mouse inputs to achieve broad generalization across diverse digital environments. The research objective is to develop a scalable foundation model for game agents by shifting from environment-specific APIs to this universal, low-level action representation. The methodology involves large-scale continual pre-training on game and agentic trajectories using a Sparse ReAct paradigm, a “Thinking Aloud” data collection protocol, and a decaying continual loss function to mitigate causal confusion from repetitive actions. Experiments show Game-TARS achieves approximately double the success rate of previous state-of-the-art models on open-world Minecraft tasks, reaching a 72.0% success rate on embodied tasks compared to the prior best of 42.1%. The principal implication for practitioners is that employing a simple, scalable, device-level action space is a viable path for building general-purpose computer-use agents with strong zero-shot generalization capabilities, bypassing the need for environment-specific engineering. |
| Uniform Discrete Diffusion with Metric Path for Video Generation (Read more on arXiv or HuggingFace) |
|
This paper introduces URSA, a discrete diffusion framework for scalable video generation that operates by iteratively refining discrete spatiotemporal tokens. The research objective is to close the performance gap between discrete and continuous video generation methods by mitigating error accumulation and improving long-context consistency. The methodology integrates a Linearized Metric Path derived from token embedding distances, a Resolution-dependent Timestep Shifting mechanism, and an asynchronous temporal scheduling strategy to unify tasks like text-to-video and interpolation in a single model. URSA demonstrates performance comparable to state-of-the-art continuous methods, achieving a text-to-video score of 82.4 on the VBench benchmark. For AI practitioners, this work provides a unified and scalable discrete alternative to continuous diffusion models for high-quality, multi-task video generation, offering a competitive and potentially more efficient architectural paradigm. |
| Repurposing Synthetic Data for Fine-grained Search Agent Supervision (Read more on arXiv or HuggingFace) |
|
This paper introduces Entity-aware Group Relative Policy Optimization (E-GRPO), a framework that repurposes ground-truth entities from synthetic data to create a dense reward signal for training search agents. The core objective is to solve the sparse reward problem in methods like GRPO by distinguishing informative “near-miss” failures from complete failures. E-GRPO’s methodology formulates a dense reward function that assigns partial rewards to incorrect trajectories based on their normalized entity match rate—the fraction of ground-truth entities identified within the agent’s reasoning thoughts. The primary result is that E-GRPO consistently outperforms its baseline; for instance, a 7B model trained in a local environment achieved a 64.2 average Pass@1 score on QA benchmarks, a 2.8-point improvement over standard GRPO, while also reducing the number of tool calls. The principal implication for AI practitioners is that metadata discarded during synthetic data generation is a computationally cheap yet powerful source for creating fine-grained reward signals, enhancing the sample efficiency and performance of RL-based agent alignment by enabling learning from partially correct solutions. |
| OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents (Read more on arXiv or HuggingFace) |
|
This paper introduces OSWorld-MCP, a benchmark for evaluating multimodal agents on their ability to jointly perform GUI operations and invoke external tools via the Model Context Protocol (MCP). The main objective is to create a fair evaluation framework to assess an agent’s decision-making in choosing between GUI interactions and MCP tool invocations for complex computer tasks. The methodology involves extending the OSWorld environment with a curated set of 158 high-quality MCP tools and introducing new metrics like Tool Invocation Rate (TIR). Primary results show that MCP tools significantly improve performance, increasing task success for OpenAI o3 from 8.3% to 20.4%, yet even top models have low tool invocation rates (max of 36.3%). For AI practitioners, this benchmark provides a standardized method to assess and develop agent tool-use capabilities, revealing that effective decision-making between GUI and tool-based actions is a critical and underdeveloped area for creating more robust automated agents. |
| WebLeaper: Empowering Efficiency and Efficacy in WebAgent via Enabling |
|
|
| Info-Rich Seeking (Read more on arXiv or HuggingFace) |
|
WebLeaper is a framework for generating entity-rich information-seeking tasks from Wikipedia tables to train web agents that are both more effective and efficient. The main objective is to overcome the low search efficiency of LLM-based agents, which is attributed to the sparsity of target entities in conventional training tasks, by developing a framework to construct high-coverage tasks and generate efficient solution trajectories. The methodology involves modeling information-seeking as a tree-structured reasoning problem and synthesizing tasks in three variants (Basic, Union, Reverse-Union) to systematically increase complexity, followed by curating training trajectories based on Information-Seeking Rate (ISR) and Information-Seeking Efficiency (ISE) metrics, and finally training an agent via supervised fine-tuning and reinforcement learning with a hybrid reward system. In a comprehensive training setting, WebLeaper achieved a 73.2 accuracy score on the GAIA benchmark, outperforming strong open-source models like DeepSeek-V3.1 (63.1) and proprietary models such as Claude-4-Sonnet (68.3). The principal implication for AI practitioners is that training agents on entity-dense tasks, as enabled by the WebLeaper framework, directly improves both task success rates and operational efficiency (fewer actions), providing a concrete strategy to build more capable and cost-effective web-browsing agents. |
| Group Relative Attention Guidance for Image Editing (Read more on arXiv or HuggingFace) |
|
The paper introduces Group Relative Attention Guidance (GRAG), a lightweight method for achieving fine-grained control over editing strength in Diffusion-in-Transformer (DiT) models. The primary objective is to address the lack of effective control over editing intensity in existing methods by enabling continuous modulation between instruction following and image consistency. The key methodology involves identifying a shared bias vector in the Query and Key embeddings of the MM-Attention mechanism and then reweighting the deviation of each token from this group bias to precisely control the editing process. Integrating GRAG into the Qwen-Edit model improved the overall EditScore from 7.2576 to 7.3245 on the PIE dataset. For AI practitioners, GRAG provides a simple, four-line code modification that can be integrated into existing DiT-based editors to enhance controllability and editing quality without any model tuning. |
| STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D |
|
|
| Intelligence (Read more on arXiv or HuggingFace) |
|
This paper introduces STAR-Bench, a benchmark to evaluate audio 4D intelligence, defined as a model’s ability to perform deep reasoning over sound dynamics in time and 3D space. The research objective is to assess how well Large Audio-Language Models (LALMs) handle linguistically hard-to-describe auditory cues, a gap in existing text-centric audio benchmarks. The methodology involves a two-level benchmark with a Foundational Acoustic Perception task using synthesized audio to test six core attributes and a Holistic Spatio-Temporal Reasoning task using curated real-world audio to evaluate complex event ordering and 3D scene understanding. Evaluation of 19 models reveals substantial performance gaps, showing that relying on audio captions instead of raw audio causes accuracy to drop by 31.5% on temporal tasks and 35.2% on spatial tasks, unlike in prior benchmarks. For AI practitioners, the principal implication is that current models fundamentally struggle to integrate information from multiple audio inputs and lack genuine spatial awareness, highlighting the need to develop architectures that natively process multi-channel audio rather than averaging it to a mono signal. |
| Routing Matters in MoE: Scaling Diffusion Transformers with Explicit |
|
|
| Routing Guidance (Read more on arXiv or HuggingFace) |
|
This paper introduces ProMoE, a Mixture-of-Experts (MoE) framework that improves the scaling of Diffusion Transformers (DiTs) through explicit routing guidance. The main objective is to address the poor expert specialization in vision MoEs, which stems from the spatial redundancy and functional heterogeneity of visual tokens compared to language tokens. The key methodology is a two-step router that first performs conditional routing to separate tokens by functional role (conditional vs. unconditional) and then uses prototypical routing with a novel routing contrastive loss to assign conditional tokens to experts based on semantic content. On the ImageNet 256x256 benchmark with Rectified Flow, the ProMoE-L model achieves a Fréchet Inception Distance (FID) of 2.79, surpassing its dense DiT counterpart’s FID of 3.56 while activating the same number of parameters. For AI practitioners, this work provides a validated method for effectively applying MoE to vision transformers by introducing explicit routing signals that account for the unique characteristics of visual data, enabling more efficient model scaling. |
| ParallelMuse: Agentic Parallel Thinking for Deep Information Seeking (Read more on arXiv or HuggingFace) |
|
PARALLELMUSE is a two-stage paradigm that improves deep information-seeking agents’ performance and efficiency through uncertainty-guided partial rollouts and compressed reasoning aggregation. The objective is to develop a parallel thinking framework for deep information-seeking agents that overcomes the inefficiency of redundant rollouts and the difficulty of integrating long-horizon reasoning trajectories within limited context windows. The methodology consists of two stages: 1) Functionality-Specified Partial Rollout, which identifies high-uncertainty steps in distinct functional regions (reasoning vs. exploration) to branch from, reusing context via KV caching; and 2) Compressed Reasoning Aggregation, which condenses multiple reasoning trajectories into structured reports to enable coherent, comprehensive answer synthesis. The method achieves up to a 62% performance improvement over the base agent model and reduces exploratory token consumption by 10-30% compared to conventional from-scratch parallel rollouts; trajectory compression further reduces aggregation context by up to 99%. The principal implication for AI practitioners is that PARALLELMUSE offers a practical test-time scaling technique to significantly enhance agent problem-solving capabilities without model retraining, while simultaneously improving computational and token efficiency over standard parallel reasoning methods. |
| AgentFrontier: Expanding the Capability Frontier of LLM Agents with |
|
|
| ZPD-Guided Data Synthesis (Read more on arXiv or HuggingFace) |
|
This paper introduces the AgentFrontier Engine, a data synthesis framework guided by the Zone of Proximal Development (ZPD) to enhance LLM agent reasoning. The research objective is to develop a scalable method for automatically generating frontier-level training data that is challenging enough to require guided learning but is ultimately solvable. The methodology involves a three-stage pipeline: generating multi-source seed questions, iteratively escalating their complexity with a tool-augmented agent, and using an LKP-MKO (Less Knowledgeable Peer vs. More Knowledgeable Other) adversarial calibration to filter for tasks within the LLM’s ZPD. The resulting AgentFrontier-30B-A3B model achieved state-of-the-art performance, scoring 28.6% on the text-only Humanity’s Last Exam and 93.4% on their ZPD Exam-v1. For AI practitioners, this work provides a principled, automated framework for creating high-quality, complex reasoning data, offering a scalable path to train more capable agents without relying on prohibitive manual curation. |
| Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal |
|
|
| Reasoning in MLLMs (Read more on arXiv or HuggingFace) |
|
The paper introduces Latent Sketchpad, a framework that enables Multimodal Large Language Models (MLLMs) to generate internal visual latents as a form of “visual thought” to improve complex multimodal reasoning. The research objective is to enhance MLLMs’ capabilities in scenarios requiring visual planning and imagination by equipping them with an internal mechanism to generate and utilize visual representations interleaved with their native textual reasoning process. The methodology involves augmenting a frozen pretrained MLLM with two components: a Context-Aware Vision Head that autoregressively generates sequences of visual latents, and a separately pretrained Sketch Decoder that translates these latents into interpretable sketch images for visualization. On the custom MAZEPLANNING dataset, the framework improved performance; a fine-tuned Gemma3 model equipped with Latent Sketchpad increased its task Success Rate from 70.0% to 72.2%, and the generated visual traces themselves demonstrated a Visual Success Rate of 75.6%, surpassing the text-only baseline. The principal implication for AI practitioners is that this modular, plug-and-play approach allows for the enhancement of MLLMs’ reasoning abilities for spatial planning tasks without requiring full model retraining, providing a direct method to incorporate interpretable “visual thinking” into existing architectures. |
| Critique-RL: Training Language Models for Critiquing through Two-Stage |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
|
Critique-RL is a two-stage reinforcement learning framework that trains language models for critiquing by first optimizing for discriminability and then for helpfulness without stronger supervision. The primary objective is to develop effective critique models by resolving the optimization conflict between a critic’s ability to accurately judge a response (discriminability) and its ability to provide useful feedback for refinement (helpfulness). The methodology involves a two-stage RL process: Stage I uses direct, rule-based rewards to explicitly train the critic’s discriminability, and Stage II uses indirect rewards from actor refinement to improve helpfulness, while KL regularization preserves the discriminative ability learned in Stage I. The proposed method significantly outperforms baselines; for instance, a Qwen2.5-7B model trained with Critique-RL achieved 58.40% accuracy on the MATH dataset, improving upon the 51.84% of an SFT baseline, while concurrently boosting discriminability accuracy to 85.20%. For AI practitioners, this two-stage framework offers a robust method for creating specialized critique models for scalable oversight, enhancing the performance of actor models on complex reasoning tasks by ensuring the critic first learns to reliably identify errors before learning to provide constructive feedback. |
| VisCoder2: Building Multi-Language Visualization Coding Agents (Read more on arXiv or HuggingFace) |
|
This work introduces VisCoder2, a family of open-source models for generating visualization code, alongside a large-scale multi-language dataset and a comprehensive benchmark for training and evaluation. The primary objective is to develop and systematically evaluate multi-language visualization coding agents capable of iterative generation, execution, and self-debugging across diverse programming environments. The methodology involves constructing VisCode-Multi-679K, a supervised dataset of 679K executable code samples and correction dialogues across 12 languages, and using it to fine-tune the Qwen2.5-Coder model family. On the introduced VisPlotBench benchmark, the 32B VisCoder2 model with iterative self-debug achieves an 82.4% overall execution pass rate, matching the performance of the proprietary GPT-4.1 model. For AI practitioners, this provides a set of open-source models and resources capable of reliably generating executable visualization code in multiple languages, with a robust framework for implementing execution-based self-correction to handle complex symbolic or compiler-dependent languages. |
| ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, |
|
|
| Finetuning, and Decoding the Curse of Multilinguality (Read more on arXiv or HuggingFace) |
|
This work introduces the ADAPTIVE TRANSFER SCALING LAW (ATLAS) for multilingual pretraining, which outperforms existing scaling laws in out-of-sample generalization. The research aims to empirically investigate multilingual scaling dynamics, measure cross-lingual transfer, model the “curse of multilinguality,” and determine the computational crossover point between pretraining from scratch versus finetuning for a target language. The methodology involves 774 training experiments on models from 10M to 8B parameters, fitting the ATLAS law which separates loss contributions from target language data, other data, and specific transfer languages, and deriving a 38x38 cross-lingual transfer matrix based on a Bilingual Transfer Score (BTS). The primary results show that ATLAS achieves superior generalization (R²(M)=0.82 on unseen mixtures) and quantifies the cost of adding languages; to maintain iso-loss performance when expanding language coverage by a factor of r, the compute budget must be scaled by approximately r^0.97. The principal implication for practitioners is a quantitative framework for multilingual model development, providing explicit formulas to budget compute for language expansion (C’≈C*r^0.97), a transfer matrix to optimize data mixtures, and an empirical guide for deciding whether to pretrain or finetune based on the available token budget. |
| From Spatial to Actions: Grounding Vision-Language-Action Model in |
|
|
| Spatial Foundation Priors (Read more on arXiv or HuggingFace) |
|
FALCON is a vision-language-action model that improves robotic manipulation by grounding actions in strong 3D spatial priors derived from spatial foundation models. The main objective is to address the spatial reasoning gap in existing Vision-Language-Action (VLA) models by integrating robust 3D geometric information from RGB inputs without degrading pre-trained vision-language alignment or requiring specialized 3D sensors. The methodology involves an Embodied Spatial Model (ESM) to extract rich spatial tokens and a novel Spatial-Enhanced Action Head that fuses these tokens directly with semantic action tokens from a VLM, decoupling spatial processing from the main vision-language backbone. FALCON achieves state-of-the-art results, attaining a 70.0% average success rate on nine real-world base tasks, outperforming the advanced SpatialVLA baseline (44.4%) by 25.6%. The principal implication for AI practitioners is that injecting spatial information directly into the action head, rather than the VLM’s input stream, is a superior architectural choice for preserving high-level semantic reasoning while significantly enhancing a policy’s fine-grained spatial awareness and manipulation accuracy. |
| FunReason-MT Technical Report: Overcoming the Complexity Barrier in |
|
|
| Multi-Turn Function Calling (Read more on arXiv or HuggingFace) |
|
This paper presents FunReason-MT, a novel data synthesis framework designed to generate high-quality, complex trajectories for multi-turn function calling. The main objective is to overcome the limitations of existing data generation methods, such as random sampling, by creating logically dependent and targeted tool-use scenarios. The methodology involves a three-phase process: 1) Environment-API Graph Interactions to sample valid tool execution traces, 2) Advanced Tool-Query Synthesis to reverse-engineer a challenging query from the trace, and 3) a Guided Iterative Chain to generate and refine a robust Chain-of-Thought (CoT) through self-correction. A 4B model trained with this framework achieved a Multi-Turn score of 56.50 on the BFCLv3 benchmark, a +40.75 improvement over the base model, surpassing comparable open and closed-source models. For AI practitioners, this framework provides a structured, “top-down” methodology to synthesize high-complexity training data, enabling the development of more reliable and capable tool-using agents, particularly for scenarios requiring multi-step logical reasoning. |
| ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers? (Read more on arXiv or HuggingFace) |
Ian L. V. Roque, Steven Dillmann, Suchetha Cooray, Sihan Yuan, Christine Ye |
This paper introduces ReplicationBench, a benchmark framework that evaluates the ability of AI agents to perform end-to-end replication of entire astrophysics research papers. The main objective is to assess the faithfulness and correctness of AI agents as scientific research assistants by testing their capability to reproduce the core contributions of published, expert-level astrophysics papers from scratch. The methodology involves a dataset of 19 peer-reviewed papers decomposed into 107 objective, expert-validated tasks, where agents operate within a sandboxed code execution environment to implement the methodology and produce numerical results that are automatically graded. The primary result is that current models perform poorly, with the best-performing model, Claude 3.7 Sonnet, achieving an average score of only 19.3%, with common failures including procedural errors, technical execution issues, and a lack of persistence. The principal implication for AI practitioners is that while agents may possess static knowledge, they have critical deficits in long-horizon reasoning, robust code execution, and deep procedural understanding, indicating significant architectural and capability improvements are needed for reliable use in complex scientific workflows. |
| Rethinking Visual Intelligence: Insights from Video Pretraining (Read more on arXiv or HuggingFace) |
Ahmad Rahimi, Sebastian Stapf, Mariam Hassan, Aram Davtyan, Pablo Acuaviva |
This paper demonstrates that pretrained Video Diffusion Models (VDMs) exhibit superior data efficiency and performance on structured visual reasoning tasks compared to similarly adapted Large Language Models (LLMs). The primary objective is to investigate whether the spatiotemporal inductive biases from large-scale video pretraining provide a more effective foundation for visual intelligence than the symbolic capabilities of text-pretrained models. The study employs a controlled comparison where a pretrained VDM and an LLM are fine-tuned on visual tasks using identical lightweight LoRA adaptation, with tasks framed as image-to-image temporal transitions for the VDM and serialized JSON-to-JSON for the LLM. Across benchmarks, VDMs consistently outperform LLMs in data efficiency; specifically, on the ARC-AGI benchmark, the CogVideoX1.5-5B VDM achieved 16.75% accuracy, more than double the 8.00% achieved by the comparably scaled Qwen3-4B-Instruct-2507 LLM. The principal implication for AI practitioners is that video pretraining is a potent source of inductive biases for visual foundation models, significantly improving sample efficiency on tasks requiring compositional spatial understanding and offering a superior alternative to text-centric approaches for these domains. |
| Generalization or Memorization: Dynamic Decoding for Mode Steering (Read more on arXiv or HuggingFace) |
|
This paper introduces Dynamic Mode Steering (DMS), a training-free, inference-time decoding algorithm to steer LLMs from memorization towards generalization. The primary objective is to create a framework to understand, identify, and control the distinct reasoning modes of LLMs to enhance their reliability. The methodology involves a two-stage process: first, a lightweight linear probe identifies the model’s current reliance on memorization based on internal activations at a causally-critical layer; second, a dynamic activation steering mechanism nudges the model’s computation towards pre-identified generalization circuits. Experiments on Llama-3 models show that DMS significantly improves performance, increasing Pass@1 accuracy on the GSM8K benchmark by 6.2% for the 8B model and improving factual accuracy on TruthfulQA. For AI practitioners, the principal implication is that DMS offers a practical, post-hoc method to improve the factual accuracy and logical consistency of deployed models without retraining, providing a direct mechanism for enhancing AI safety and reliability. |
| VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations (Read more on arXiv or HuggingFace) |
Jiayi Zhang, Sirong Lu, Yifan Wu, Zhiyang Zhang, Yupeng Xie |
This paper introduces VISJUDGE-BENCH, a benchmark for evaluating MLLM performance in assessing data visualization quality, and proposes VISJUDGE, a fine-tuned model that significantly improves alignment with human expert judgments on this task. The primary objective is to systematically measure and improve the capabilities of Multimodal Large Language Models (MLLMs) in assessing the quality of data visualizations across the multi-dimensional criteria of data fidelity, information expressiveness, and visual aesthetics. The authors constructed VISJUDGE-BENCH, a dataset of 3,090 expert-annotated visualizations evaluated on six sub-dimensions, and then developed VISJUDGE by applying reinforcement learning and parameter-efficient fine-tuning to the Qwen2.5-VL-7B-Instruct model using this new benchmark. The proposed VISJUDGE model significantly outperforms existing MLLMs, reducing the Mean Absolute Error (MAE) by 19.8% and increasing the correlation with human experts by 58.7% compared to the baseline GPT-5 model. The principal implication for AI practitioners is that general-purpose MLLMs are inadequate for specialized, multi-dimensional assessment of domain-specific imagery like data visualizations, necessitating domain-specific fine-tuning on expert-annotated benchmarks to achieve human-aligned performance. |
| VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a |
|
|
| Unified Concept Set (Read more on arXiv or HuggingFace) |
|
This paper introduces VL-SAE, a sparse autoencoder that interprets and enhances vision-language alignment in VLMs by mapping multi-modal representations to a unified concept set. The main objective is to address the difficulty of interpreting VLM alignment by mapping the semantics of both vision and language representations into a single, shared conceptual space. The key methodology involves a novel SAE architecture with a distance-based encoder to ensure consistent activations for semantically similar inputs and two modality-specific decoders to handle distributional differences. Experiments show that VL-SAE improves downstream performance, for instance, enhancing the zero-shot image classification mean accuracy of OpenCLIP-ViT-H/14 from 76.9% to 77.8%. For AI practitioners, VL-SAE provides a post-hoc tool to interpret a VLM’s alignment mechanism, diagnose failures like hallucination, and improve performance by explicitly aligning representations at the concept level. |
| PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D |
|
|
| Part Understanding (Read more on arXiv or HuggingFace) |
Lan Xu, Yukai Zhou, Xin Lv, Yiyang He, Penghao Wang |
This paper introduces PartNeXt, a large-scale dataset with over 23,000 high-quality, textured 3D models annotated with fine-grained, hierarchical part labels across 50 categories. The research objective is to address the scalability and usability limitations of existing datasets like PartNet, which lack textures and use expert-dependent annotation tools. The methodology involves collecting models from public sources (e.g., Objaverse), using a custom dual-panel web interface for scalable crowdsourced annotation directly on textured meshes, and leveraging GPT-4o to bootstrap part hierarchies. The primary result shows that training the Point-SAM model on PartNeXt yields substantial performance gains over PartNet, improving IoU@10 on the PartNeXt test set from 60.3% to 65.9%. The principal implication for AI practitioners is that PartNeXt offers a high-quality, textured, and diverse dataset for training and benchmarking more robust 3D models, with new benchmarks that reveal significant gaps in current 3D-LLMs’ ability to perform open-vocabulary part grounding. |
| PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text |
|
|
| Embedding (Read more on arXiv or HuggingFace) |
Denis Cavallucci, Iliass Ayaou |
This paper introduces PatenTEB, a 15-task benchmark for patent text embedding, and the patembed model family trained upon it. The research objective is to create a comprehensive evaluation framework for patent text understanding and identify training strategies that optimize for both benchmark performance and real-world generalization. The methodology involves constructing a 2.06 million-example benchmark with domain-stratified splits and asymmetric retrieval tasks, and then using multi-task learning on 13 of these tasks to train a family of encoders initialized from a domain-pretrained model. The primary result is that patembed-base achieves a state-of-the-art 0.494 V-measure on the external MTEB BigPatentClustering.v2 benchmark, outperforming the previous best of 0.445. For practitioners, the principal implication is that multi-task training improves external generalization (+0.062 V-measure on BigPatent) even at a minor cost to internal benchmark performance (-0.004 Overall Score), indicating that optimizing solely for benchmark scores can be suboptimal for deployment. |
Papers for 2025-10-27
| Title |
Authors |
Summary |
| DeepAgent: A General Reasoning Agent with Scalable Toolsets (Read more on arXiv or HuggingFace) |
Jiajie Jin, Jiarui Jin, Xiaoxi Li, dongguanting, wxjiao |
This paper introduces DeepAgent, an end-to-end reasoning agent that unifies autonomous thinking, dynamic tool discovery, and action execution into a single, continuous process for complex tasks. The objective is to create a general-purpose agent that overcomes the limitations of predefined workflows by enabling dynamic tool retrieval and robust long-horizon reasoning over scalable toolsets. Key methodologies include an autonomous memory folding mechanism to compress interaction history into a structured schema (episodic, working, tool memory) and a reinforcement learning strategy, ToolPO, which leverages an LLM-based tool simulator and fine-grained advantage attribution for stable training. DeepAgent significantly outperforms baseline methods, particularly in open-set scenarios; on the ToolBench benchmark with open-set tool retrieval, it achieved a 64.0% success rate, surpassing the strongest workflow-based baseline’s 54.0%. For AI practitioners, this work provides a framework demonstrating that a unified reasoning architecture with dynamic tool discovery and explicit memory management is more effective for building robust, general-purpose agents than traditional, rigid workflow-based approaches. |
| Video-As-Prompt: Unified Semantic Control for Video Generation (Read more on arXiv or HuggingFace) |
|
This paper introduces Video-As-Prompt (VAP), a unified framework that uses a reference video as an in-context prompt to achieve generalizable semantic control over video generation. The objective is to develop a single model for diverse, non-pixel-aligned semantic control (e.g., style, motion) that avoids the artifacts and poor generalization of existing methods. VAP employs a plug-and-play Mixture-of-Transformers (MoT) architecture, where a trainable expert processes the video prompt to guide a frozen Video Diffusion Transformer (DiT) via full attention, combined with a temporally biased position embedding to prevent incorrect spatial mapping priors. The primary result is that VAP achieves a 38.7% user preference rate, rivaling leading condition-specific commercial models, and demonstrates strong zero-shot generalization to unseen semantic conditions. For AI practitioners, the principal implication is the ability to add complex semantic control to existing frozen video generation models without costly per-condition retraining or specialized architectures, enabling more scalable and flexible content creation. |
| From Denoising to Refining: A Corrective Framework for Vision-Language |
|
|
| Diffusion Model (Read more on arXiv or HuggingFace) |
|
The paper introduces ReDiff, a corrective framework for vision-language diffusion models that reframes generation from passive denoising to active refining to mitigate error cascades in parallel decoding. The primary objective is to overcome catastrophic error propagation during parallel generation, which is caused by a train-inference discrepancy where models must generate from their own noisy outputs. The methodology involves a two-stage training process: first, a foundational revision stage to correct synthetic errors, followed by an online self-correction loop where the model learns to fix its own intrinsic mistakes by training on draft-correction pairs generated by an expert model. The framework achieves a CLAIR score of 76.74 on the CapMAS benchmark, an 11.2 point improvement over the LLaDA-V baseline, while demonstrating superior stability in few-step parallel generation. For AI practitioners, this work provides a training paradigm to develop more robust vision-language diffusion models capable of stable and efficient parallel generation, directly addressing a key limitation that hinders their real-world application. |
| Sample By Step, Optimize By Chunk: Chunk-Level GRPO For Text-to-Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
|
Chunk-GRPO is a novel chunk-level reinforcement learning approach for flow-matching-based text-to-image generation that optimizes groups of consecutive timesteps to improve image quality and preference alignment. The research objective is to resolve the inaccurate advantage attribution and neglect of temporal dynamics inherent in standard step-level Group Relative Policy Optimization (GRPO). The key methodology involves segmenting the generation trajectory into “chunks” based on the temporal dynamics of flow matching, identified by the relative L1 distance between latent states, and applying a chunk-level optimization objective. Primarily, Chunk-GRPO with weighted sampling achieves a superior HPSv3 preference score of 15.373, outperforming the step-level Dance-GRPO baseline’s score of 15.080. The principal implication for AI practitioners is that aligning the granularity of RL optimization (i.e., chunks) with the intrinsic dynamics of an iterative generation process offers a more effective fine-tuning strategy than applying uniform, step-wise credit assignment. |
| Sparser Block-Sparse Attention via Token Permutation (Read more on arXiv or HuggingFace) |
|
Permuted Block-Sparse Attention (PBS-Attn) is a plug-and-play method that accelerates long-context LLM prefilling by reordering tokens to increase the block-level sparsity of the attention matrix. The objective is to improve computational efficiency by creating a more favorable block-sparse structure, which is achieved through a novel segmented permutation strategy that reorders keys within segments based on query-aware importance scores while preserving inter-segment causality. Experiments show that PBS-Attn achieves an end-to-end prefilling speedup of up to 2.75× over the FlashAttention baseline, while maintaining model accuracy that is nearly on par with full attention on benchmarks like LongBench and LongBenchv2. For AI practitioners, this method provides a practical, training-free optimization to significantly reduce the latency and computational cost of the compute-bound prefilling stage for long-context inference applications. |
| UI-Ins: Enhancing GUI Grounding with Multi-Perspective |
|
|
| Instruction-as-Reasoning (Read more on arXiv or HuggingFace) |
|
This research introduces “Instruction-as-Reasoning,” a novel SFT+RL framework that enhances GUI grounding by treating diverse instructions as dynamic reasoning pathways. The paper’s primary objective is to overcome the limitations of poor instruction quality and diversity in existing datasets by developing a model that actively selects the most effective analytical perspective for a given UI task. The methodology involves a two-stage training process: first, Supervised Fine-Tuning (SFT) on a curated dataset of multi-perspective instructions to instill reasoning capabilities, followed by Reinforcement Learning (RL) with Group Relative Policy Optimization (GRPO) to optimize pathway selection. The resulting UI-Ins-32B model establishes a new state-of-the-art, achieving 87.3% accuracy on the UI-I2E-Bench, after finding that 23.3% of instructions in existing datasets were flawed. For AI practitioners, this work highlights the critical importance of instruction data quality and diversity, providing a concrete SFT+RL framework to build more robust GUI agents that can reason effectively and avoid policy collapse during training. |
| A Definition of AGI (Read more on arXiv or HuggingFace) |
Yarin Gal, Honglak Lee, Christian Szegedy, Dawn Song, Dan Hendrycks |
This paper introduces a quantifiable framework to define Artificial General Intelligence (AGI), grounding it in the Cattell-Horn-Carroll theory of human cognition to evaluate AI systems across ten core cognitive domains. The objective is to operationalize a concrete definition of AGI—an AI matching the cognitive versatility and proficiency of a well-educated adult—to create a standardized measurement tool. The methodology adapts human psychometric batteries to assess AI on ten equally-weighted components, including reasoning, memory, and perception, resulting in a standardized “AGI Score.” The framework’s application reveals a “jagged” cognitive profile in current models; GPT-4 achieves a total AGI Score of 27%, showing strength in knowledge-based tasks but critical deficits in foundational areas, scoring 0% in Long-Term Memory Storage. For AI practitioners, the principal implication is the direct identification of specific system bottlenecks, demonstrating that fundamental capabilities like continual learning (Long-Term Memory Storage) are entirely absent and require direct architectural solutions rather than being addressed by compensatory strategies like Retrieval-Augmented Generation. |
| Reasoning with Sampling: Your Base Model is Smarter Than You Think (Read more on arXiv or HuggingFace) |
|
This paper presents a training-free, inference-time sampling algorithm to enhance the reasoning capabilities of base large language models. The central research question is whether comparable reasoning performance to that achieved by reinforcement learning (RL) can be elicited from base models using only advanced sampling techniques. The proposed “Power Sampling” method employs an iterative Markov chain Monte Carlo (MCMC) algorithm to sample from the base model’s power distribution (p^α), which systematically upweights higher-likelihood token sequences. The algorithm achieves performance on par with, and often superior to, the RL-posttrained GRPO baseline; for example, on the Qwen2.5-Math-7B model, it improves HumanEval accuracy from 32.9% (base) to 57.3%, outperforming GRPO’s 53.7%, while also maintaining superior generation diversity on pass@k metrics. The principal implication for AI practitioners is that significant reasoning improvements can be extracted from existing base models by dedicating more compute at inference time, potentially obviating the need for complex and costly RL posttraining. |
| RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via |
|
|
| Hierarchical Model Merging (Read more on arXiv or HuggingFace) |
|
The paper introduces RECALL, a data-free framework that mitigates catastrophic forgetting in large language models by performing hierarchical, layer-wise parameter merging guided by the similarity of internal representations. The primary objective is to develop a method that can identify and preserve learned knowledge across multiple fine-tuned models in a data-free and task-agnostic manner, thereby alleviating catastrophic forgetting during continual learning. RECALL first extracts hidden state representations from a small set of “typical” samples, identified via clustering, for each model. It then computes layer-wise inter-model similarity using an RBF kernel on these representations and uses these scores as adaptive weights for a hierarchical parameter merge, applying different weights for each layer. In a single-model merging scenario with Llama-2-7B, RECALL achieved the best generalization to unseen tasks with an average score of 38.92, outperforming the next-best baseline by +7.86%. The principal implication for AI practitioners is the ability to fuse multiple specialist models into a single, more capable generalist model without requiring access to the original training datasets, which saves computational resources and navigates data privacy constraints. |
| Visual Diffusion Models are Geometric Solvers (Read more on arXiv or HuggingFace) |
Or Patashnik, Andrey Voynov, Omer Dahary, Shai Yehezkel, Nir Goren |
Visual diffusion models are presented as effective geometric solvers that operate directly in pixel space. The primary objective is to demonstrate that these models can reason about and discover geometric structures by recasting hard geometric problems as image generation tasks. The key methodology involves training a standard visual diffusion model (U-Net backbone with self-attention) on pixel-space representations of problems like the Inscribed Square Problem, Steiner Tree Problem, and Maximum Area Polygonization Problem, with problem instances provided as conditional input. Primary results include achieving a squareness metric of 0.891 for the Inscribed Square Problem (vs. 0.924 GT), a 0.996 valid tree rate and 1.0008 mean length ratio for Steiner Trees (10-20 points), and a 0.953 polygon validity rate and 0.9887 mean area ratio for Maximum Area Polygons (7-12 points). This research implies for AI practitioners that visual diffusion models offer a general and practical framework for approximating notoriously hard geometric problems through visual representations, enabling a bridge between generative modeling and mathematical problem-solving without requiring specialized architectures. |
| WorldGrow: Generating Infinite 3D World (Read more on arXiv or HuggingFace) |
Jia Lu, Taoran Yi, Chen Yang, Sikuang Li, JieminFang |
WorldGrow is a novel framework for generating infinite 3D worlds. The research addresses the challenge of synthesizing infinitely extendable, large, continuous 3D environments with coherent geometry and photorealistic appearance. Its methodology involves a hierarchical framework using a data curation pipeline for structured scene blocks, a 3D block inpainting mechanism for context-aware extension, and a coarse-to-fine generation strategy. WorldGrow achieved a FID_DINOv2 score of 313.54 for visual fidelity in generated blocks, significantly outperforming SynCity (655.60), and demonstrated robust stability in distant expansions. This enables AI practitioners to construct scalable, high-quality 3D content for large-scale virtual environments, crucial for embodied AI training and simulation. |
| RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via |
|
|
| Data Alignment and Test-Time Scaling (Read more on arXiv or HuggingFace) |
|
RAPO++ is a cross-stage prompt optimization framework designed to enhance Text-to-Video (T2V) generation quality model-agnostically. The primary objective is to overcome limitations of short, unstructured, and misaligned user prompts that hinder the generative potential of diffusion-based T2V models. Its methodology involves three stages: Retrieval-Augmented Prompt Optimization (RAPO) for training-data-aligned refinement using relation graphs and LLMs; Sample-Specific Prompt Optimization (SSPO) for iterative test-time scaling with multi-source feedback (e.g., VLM verifiers and optical flow); and LLM fine-tuning to internalize optimization patterns from SSPO. RAPO++ achieved a total score of 82.65% on VBench with the LaVie model and improved Consistent Attribute Binding from 0.620 (Naive) to 0.742 on T2V-CompBench, demonstrating significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility. This framework provides AI practitioners with a model-agnostic, cost-efficient, and scalable solution to substantially improve T2V outputs without modifying the underlying generative backbone. |
| Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs (Read more on arXiv or HuggingFace) |
Bohyung Han, taekyung-k, byminji |
This research uses mechanistic interpretability to map the internal information flow in Video Large Language Models (VideoLLMs), revealing a structured, multi-stage process for temporal reasoning. The primary objective is to investigate where and how VideoLLMs extract spatiotemporal information from video, integrate it with textual queries, and propagate it through different layers and modalities to generate answers for video question answering tasks. The study employs Attention Knockout to causally trace information flow by selectively disabling attention connections between token groups and Logit Lens to analyze the emergence of semantic concepts within video token representations across layers. The analysis reveals that temporal reasoning begins with cross-frame interactions in early-to-middle layers, followed by video-language integration on temporal keywords in middle layers, after which the model is ready to generate answers in middle-to-late layers; retaining only these effective pathways while suppressing 58% of attention edges in LLaVA-NeXT-7B-Video-FT maintained its original VideoQA performance. For AI practitioners, this provides a blueprint for model optimization, suggesting that VideoLLMs can be pruned to retain only these critical information pathways, enabling the development of more computationally efficient models for inference without a significant loss in temporal reasoning capability. |
| Model Merging with Functional Dual Anchors (Read more on arXiv or HuggingFace) |
|
This paper introduces Functional Dual Anchors (FDAs) for model merging, enabling knowledge integration in the input-representation space. The primary objective is to mitigate task-specific knowledge conflicts in model merging by shifting the focus from parameter-space adjustments to modeling the input-representation space. The methodology involves constructing Functional Dual Anchors (FDAs) as synthetic inputs whose induced gradients align with task vectors. FDAs are optimized via gradient matching in the input-representation space and used for subsequent parameter optimization, guided by a principled initialization scheme. FDAs significantly improve multi-task performance, with a pretrained model adapted by FDAs achieving 87.26 average accuracy on ViT-B/16 tasks, representing an almost 18% improvement compared to vanilla Task Arithmetic (73.94). AI practitioners can leverage FDAs to achieve more robust and flexible model merging by integrating knowledge through synthetic inputs in the representation space, offering a viable alternative or complement to existing parameter-centric methods for consolidating diverse domain knowledge. |
| Document Understanding, Measurement, and Manipulation Using Category |
|
|
| Theory (Read more on arXiv or HuggingFace) |
|
This paper introduces a category theory-based framework for document understanding, measurement, and manipulation. The primary objective is to extract and utilize multimodal document structure to enable information-theoretic measures, summarization, exegesis, and self-supervised improvement of large pretrained models. The methodology involves representing documents as categories of orthogonalized question-answer (QA) pairs, derived from rhetorical structure (abstractive DAGs) using large pretrained models. Key results include the formal definition of a Jaccard-like metric for assertion similarity, exemplified by d(A, B) = ½ for contradictory assertions, and the development of rate distortion analysis for summarization techniques. This framework offers AI practitioners a principled, mathematical approach to semantic analysis and manipulation, enabling advanced document processing and self-correction mechanisms for LLMs based on consistency constraints. |
| PhysWorld: From Real Videos to World Models of Deformable Objects via |
|
|
| Physics-Aware Demonstration Synthesis (Read more on arXiv or HuggingFace) |
Hui Li, Yihan Zeng, Xiang Zhang, Yu Yang, cszhilu1998 |
PhysWorld is a novel framework for learning accurate and fast world models of deformable objects from limited real-world videos through physics-aware demonstration synthesis. The primary objective is to address the data scarcity challenge in learning physics-consistent dynamics models for deformable objects, enabling both high accuracy and real-time inference. Its methodology involves constructing an MPM-based digital twin using VLM-assisted constitutive model selection and global-to-local physical property optimization from real videos. This digital twin then generates diverse 4D demonstrations via Various Motion Pattern Generation and Part-aware Physical Property Perturbation, which train a GNN-based world model subsequently fine-tuned with real videos. Experimentally, PhysWorld achieved competitive prediction performance and enabled inference speeds 47 times faster than the state-of-the-art PhysTwin (799 FPS vs 17 FPS). This work provides AI practitioners with a robust and efficient method for developing physics-consistent world models for robotics, VR, and AR, mitigating data requirements and facilitating real-time deployment. |
| Are Large Reasoning Models Good Translation Evaluators? Analysis and |
|
|
| Performance Boost (Read more on arXiv or HuggingFace) |
Min Yang, Lidia S. Chao, Xinyi Yang, Zhihong Huang, rzzhan |
This paper systematically analyzes Large Reasoning Models (LRMs) as Machine Translation (MT) evaluators, identifies inefficiencies, and proposes a calibration method to improve performance and efficiency. The research aimed to systematically understand LRM performance and failure modes in MT evaluation, and to develop an effective alignment strategy for LRMs as MT judges. The authors employed LRMs within the MQM framework, conducting meta-evaluation and analysis across various model sizes, and proposing ThinMQM, a method to calibrate LRM thinking by fine-tuning models on synthetic, human-like evaluation trajectories derived from WMT23 MQM data. Experiments on WMT24 Metrics benchmarks demonstrated that ThinMQM largely reduced thinking budgets by approximately 35x, while concurrently improving evaluation performance, notably achieving an 8.7 correlation point improvement for R1-Distill-Qwen-7B. These findings highlight that efficiently calibrated LRMs have significant potential to advance fine-grained automatic MT evaluation, emphasizing the critical need for controlled thinking and careful calibration for AI practitioners developing LRM-as-a-judge systems. |
| ARC-Encoder: learning compressed text representations for large language |
|
|
| models (Read more on arXiv or HuggingFace) |
|
The paper introduces ARC-Encoder, a novel method for learning compressed text representations for Large Language Models (LLMs) that replaces raw text input, aiming to improve inference efficiency and context handling without modifying the decoder. The primary objective is to develop a plug-and-play encoder that compresses LLM contexts into continuous representations, reducing inference costs and extending context windows while preserving general abilities. The methodology involves an LLM transformer-based encoder with a pooling mechanism that averages consecutive queries in the last self-attention module for a fixed pooling factor (e.g., 4x or 8x), trained via alternating reconstruction and continuation tasks with a two-layer MLP projector and adaptable to multiple decoders via shared encoder and specialized projector layers. Results demonstrate that ARC-Encoder achieves state-of-the-art performance, nearly matching the open-book baseline with a 4x pooling factor (e.g., ARC4-Encoder for Llama3.1 8B achieves an average score of 48.0 compared to open-book’s 47.4) while providing up to 1.8x gains in prefilling FLOPs and extending context processing to 8x the original window size. This implies that AI practitioners can leverage ARC-Encoder as an efficient and portable solution to compress LLM input contexts, enhancing inference speed and long-context capabilities for various applications without requiring architectural changes or fine-tuning of the LLM decoder itself. |
| Taming Modality Entanglement in Continual Audio-Visual Segmentation (Read more on arXiv or HuggingFace) |
Zhaojin Fu, Zili Wang, Tao Zhang, Qi Yang, hongyuyang23casia |
This paper introduces a novel Collision-based Multi-modal Rehearsal (CMR) framework to mitigate modality entanglement in Continual Audio-Visual Segmentation (CAVS). The primary objective is to enable models to continuously segment new audio-visual classes while preserving knowledge of previously learned ones, specifically addressing multi-modal semantic drift and co-occurrence confusion in fine-grained CAVS. The methodology proposes the CMR framework, which includes a Multi-modal Sample Selection (MSS) strategy to identify high-modality-consistency samples for rehearsal by quantifying audio contribution, and a Collision-based Sample Rehearsal (CSR) mechanism that dynamically adjusts rehearsal frequency for classes prone to co-occurrence confusion. Comprehensive experiments on AVSBench-CI, AVSBench-CIS, and AVSBench-CIM datasets demonstrate that CMR significantly outperforms single-modal continual learning methods, achieving an 11.3 mIoU increase on the AVSBench-CIS 60-10 overlapped setting compared to other methods. This research provides AI practitioners with a robust framework for designing continual learning systems in multi-modal, fine-grained tasks like audio-visual segmentation, offering effective strategies to combat catastrophic forgetting and modality entanglement. |
| AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research |
|
|
| Suite (Read more on arXiv or HuggingFace) |
Bhavana Dalvi, Dan Bareket, Nishant Balepur, Mike D’Arcy, Jonathan Bragg |
AstaBench introduces a rigorous benchmark suite for evaluating AI agents in scientific research. Its objective is to address shortcomings of existing benchmarks by providing holistic, reproducible, cost-accounted evaluations with standardized interfaces and comprehensive baselines across 2400+ problems and multiple scientific domains. The methodology includes a production-grade scientific research environment with controlled search tools and an evaluation toolkit to account for confounders like tool access and inference cost. Experimental results show that while some agents achieve meaningful progress in literature understanding (e.g., Asta v0 at 53.0%), overall scores for the full range of science tasks remain low, with the best open-source agent achieving only 11.1%. This indicates that AI is still far from solving the challenge of scientific research assistance, requiring significant development in areas like coding, data analysis, and end-to-end discovery. |
| Foley Control: Aligning a Frozen Latent Text-to-Audio Model to Video (Read more on arXiv or HuggingFace) |
|
Foley Control is a lightweight approach for video-guided Foley synthesis by aligning frozen text-to-audio models with video. The main objective is to achieve competitive temporal and semantic alignment while preserving the practicality of frozen generative backbones and reducing data requirements. This is accomplished by connecting V-JEPA2 video embeddings to a frozen Stable Audio Open DiT text-to-audio (T2A) model via compact, trainable video cross-attention layers inserted after the existing text cross-attention, utilizing pooled video tokens and Rotary Position Embeddings (RoPE) for temporal grounding. Foley Control delivers competitive alignment, with single-pooled embeddings achieving a KL-PANNs metric of 3.111351 at 400k training steps, comparable to denser grid embeddings, and comparable MovieGenBench scores (e.g., DeSync 0.32) while training with nearly two orders of magnitude less paired data and compute than end-to-end multimodal systems. This framework offers AI practitioners a modular and data-efficient solution for video-to-audio generation, enabling easy swapping or upgrading of encoders and T2A backbones without costly end-to-end retraining. |
| Soft Instruction De-escalation Defense (Read more on arXiv or HuggingFace) |
|
Soft Instruction Control (SIC) is an iterative prompt sanitization defense designed for tool-augmented Large Language Model (LLM) agents against prompt injection attacks. The method’s objective is to neutralize adversarial instructions by repeatedly inspecting incoming untrusted data, rewriting, masking, or removing malicious content, and re-evaluating until clean or a maximum iteration limit is reached. SIC employs an LLM-based rewriting mechanism with canary injection and multi-granularity detection (full text and chunks) to identify and de-escalate imperative instructions. Against a strong adaptive genetic algorithm adversary, SIC achieved an Attack Success Rate (ASR) of 15%, outperforming other detector-based defenses which had ASRs up to 49%. For AI practitioners, SIC offers a pragmatic, modular preprocessing layer that significantly raises the bar for prompt injection attacks by making them more difficult and expensive, without requiring modifications to the underlying LLM agent. |
| PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language |
|
|
| Models in Physical Environments (Read more on arXiv or HuggingFace) |
Chaoyang Zhao, Manli Tao, Yi Peng, Xuantang Xiong, JettZhou |
This paper introduces Active Visual Reasoning (AVR), a task requiring multimodal models to interact with physical environments to resolve information incompleteness for reasoning. The main objective is to extend visual reasoning from static, fully-observable settings to dynamic, partially-observable environments where agents must actively gather information. The methodology involves creating the CLEVR-AVR benchmark for evaluation and the AVR-152k dataset with Chain-of-Thought annotations modeling the task as a higher-order Markov Decision Process. A model, PhysVLM-AVR, is trained on this dataset to learn sequential information gathering and reasoning. The primary result is that while PhysVLM-AVR achieves 90.5% accuracy in identifying the need for interaction (Information Sufficiency Judgment Accuracy), its final answer accuracy is 39.7%, indicating that models can detect missing information but struggle to strategically act to acquire it. The principal implication for AI practitioners is that the AVR framework and dataset provide a concrete methodology for training agents to perform goal-directed, interactive information seeking, addressing a key limitation of current MLLMs in robotics and dynamic environments. |
| Stabilizing MoE Reinforcement Learning by Aligning Training and |
|
|
| Inference Routers (Read more on arXiv or HuggingFace) |
|
This paper introduces Rollout Routing Replay (R3) to stabilize reinforcement learning for Mixture-of-Experts (MoE) models by aligning router behavior between training and inference. The objective is to mitigate RL training instability and collapse in MoE models, which is attributed to discrepancies in expert routing distributions between the inference rollout and training update phases. The key methodology involves recording the routing masks from the inference engine during sequence generation and replaying them during the training forward pass to enforce consistent expert selection. R3 reduces the training-inference policy KL divergence for the Qwen3-30B-A3B model from 1.535×10⁻³ to 7.5×10⁻⁴ and decreases the frequency of tokens with large probability discrepancies by an order of magnitude. For AI practitioners, R3 offers a practical method to stabilize RL on MoE architectures, preventing training collapse and improving final model performance by resolving a foundational inconsistency between training and inference frameworks. |
Papers for 2025-10-24
| Title |
Authors |
Summary |
| AdaSPEC: Selective Knowledge Distillation for Efficient Speculative |
|
|
| Decoders (Read more on arXiv or HuggingFace) |
|
AdaSPEC is a novel selective knowledge distillation framework that improves the efficiency of speculative decoders by training draft models to focus only on easier-to-predict tokens. The primary objective is to address the misalignment between conventional knowledge distillation, which minimizes KL divergence across all tokens, and the true objective of speculative decoding, which is to maximize the token acceptance rate. The methodology involves a two-step process: first, a reference model is distilled from the target model to identify “difficult-to-fit” tokens; second, the draft model is distilled using a filtered dataset that excludes these difficult tokens. Results demonstrate that AdaSPEC consistently outperforms the state-of-the-art DistillSpec method, achieving up to a 15% higher token acceptance rate across diverse tasks. For AI practitioners, this method provides a more effective training strategy to create efficient draft models, leading to significant inference acceleration for large language models without degrading generation quality. |
| Human-Agent Collaborative Paper-to-Page Crafting for Under $0.1 (Read more on arXiv or HuggingFace) |
|
The paper introduces AutoPage, a multi-agent system that automates the generation of interactive academic project webpages from PDF papers, and PageBench, a benchmark for evaluating this task. The primary objective is to automate the creation of high-quality project webpages to reduce manual effort, questioning if an automated system can effectively manage this complex, multimodal generation task. AutoPage employs a coarse-to-fine, multi-agent pipeline with three stages: Narrative Planning, Multimodal Content Generation, and Interactive Page Rendering, incorporating dedicated “Checker” agents for verification and optional human-in-the-loop checkpoints. Experiments on the PageBench benchmark show that when paired with GPT-4o-mini, AutoPage improves the Aesthetic Score from 2.71 to 2.95 and, in a user study, achieved the highest human preference score of 7.16 out of 10. For AI practitioners, this work demonstrates that a structured, multi-agent pipeline with verification stages can serve as a powerful enhancer for existing LLMs, outperforming monolithic end-to-end approaches in complex document transformation tasks requiring high-fidelity, multimodal output. |
| Open-o3 Video: Grounded Video Reasoning with Explicit Spatio-Temporal |
|
|
| Evidence (Read more on arXiv or HuggingFace) |
|
This paper presents Open-O3 Video, a framework that enables video reasoning models to ground their answers with explicit spatio-temporal evidence, including timestamps and bounding boxes. The primary objective is to develop a non-agent model capable of joint temporal tracking and spatial localization to support verifiable, evidence-centered reasoning in dynamic video scenes. The methodology consists of a two-stage training strategy: a cold-start supervised fine-tuning on a newly curated STGR-CoT-30k dataset, followed by reinforcement learning with Group Sequence Policy Optimization (GSPO) using custom rewards with adaptive temporal proximity and temporal gating. On the V-STAR benchmark, Open-O3 Video achieves state-of-the-art performance, improving the mean Arithmetic Mean (mAM) by 14.4% and mean Logarithmic Geometric Mean (mLGM) by 24.2% over the Qwen2.5-VL baseline. For AI practitioners, the principal implication is that this framework provides a concrete method for building more transparent and reliable video understanding systems, as the generated spatio-temporal evidence enables verifiable reasoning and supports confidence-aware verification at inference time. |
| HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video |
|
|
| Narratives (Read more on arXiv or HuggingFace) |
|
HoloCine is a holistic text-to-video framework that generates coherent, cinematic multi-shot video narratives in a single pass. The research objective is to bridge the “narrative gap” in video generation by synthesizing entire scenes from hierarchical text prompts, ensuring global consistency and precise directorial control across multiple shots. The methodology combines a Window Cross-Attention mechanism to localize text prompts to specific video segments with a Sparse Inter-Shot Self-Attention pattern—dense within shots and sparse between—to maintain coherence while reducing computational complexity. HoloCine achieves state-of-the-art performance, demonstrating superior narrative control with a Shot Cut Accuracy of 0.9837, significantly outperforming prior holistic and two-stage methods. For AI practitioners, the primary implication is a computationally feasible architecture for minute-scale video generation, as the structured self-attention pattern provides a scalable solution to manage the quadratic complexity of transformers for long, multi-shot sequences. |
| Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall (Read more on arXiv or HuggingFace) |
Sungjin Ahn, Caglar Gulcehre, Justin Deschenaux, Jaesik Yoon, jojo0217 |
This paper introduces Loopholing, a deterministic latent pathway in discrete diffusion models that bypasses the information collapse caused by categorical sampling. The research aims to solve the “sampling wall” problem, where rich distributional information is lost when collapsing to a one-hot vector during sampling, hindering performance in subsequent denoising steps. The methodology involves creating a dual-output system at each denoising step: a standard stochastic one-hot vector and a continuous latent vector that deterministically carries contextual information to the next step, trained efficiently via a two-pass self-conditioning strategy. The resulting Loopholing Discrete Diffusion Models (LDDMs) significantly improve performance, reducing generative perplexity by up to 61% over prior baselines and improving accuracy on the Countdown reasoning task from 45% to 56.3% over the MGDM baseline. For AI practitioners, this provides a simple mechanism to enhance non-autoregressive text generation quality by preserving information flow across denoising steps, mitigating issues like idle steps and oscillations with only minor modifications to existing discrete diffusion frameworks. |
| DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion (Read more on arXiv or HuggingFace) |
|
The paper introduces DYPE, a training-free method that dynamically adjusts positional encodings during the diffusion process to enable pre-trained transformers to generate ultra-high-resolution images. The main objective is to overcome the resolution limitations of pre-trained diffusion transformers by dynamically adapting their positional encodings to align with the inherent low-to-high frequency spectral progression of the generative process. The key methodology involves introducing a time-dependent scaling factor, κ(t), to existing RoPE extrapolation methods like NTK-aware and YaRN, which adjusts the positional encoding’s frequency allocation at each diffusion timestep to match the evolving spectral content of the image being generated. The primary result is a significant improvement in image quality at ultra-high resolutions; in human evaluations at 4096x4096 resolution, the DYPE-enhanced YaRN variant was preferred over its static baseline in 90.1% of comparisons for text alignment. The principal implication for AI practitioners is that DYPE can be implemented as a zero-overhead, training-free modification at inference time to enable existing diffusion transformer models to generate images at resolutions far exceeding their training data (e.g., 16M+ pixels), thereby bypassing the need for expensive high-resolution retraining. |
| Every Question Has Its Own Value: Reinforcement Learning with Explicit |
|
|
| Human Values (Read more on arXiv or HuggingFace) |
|
i) This paper introduces Reinforcement Learning with Explicit Human Values (RLEV), a method that aligns Large Language Models with human priorities by scaling correctness-based rewards with explicit, human-assigned prompt values. ii) The primary research objective is to develop a reinforcement learning framework that optimizes LLMs for non-uniform human utility, where the value of a correct response depends on the intrinsic importance of the prompt, moving beyond standard binary reward schemes that treat all correct answers equally. iii) The key methodology involves extending the Reinforcement Learning from Verifiable Rewards (RLVR) framework by defining a surrogate reward function r(x,y) = s(x) * 1_correct(y), where the scaling factor s(x) = 1 + min(a * v(x), 1) is derived from a normalized, human-defined value v(x) and is used with policy gradient algorithms. iv) RLEV consistently outperforms correctness-only baselines, with 32B models showing a 2.8% average gain in Human-Aligned Accuracy (from 59.5% to 62.3%) and learning a value-sensitive termination policy that reduces average response length from 246.9 to 98.6 tokens by being more concise on low-value prompts. v) The principal implication for AI practitioners is that RLEV provides a practical method to train models that strategically allocate computational resources (e.g., response length) based on task importance, leading to more efficient and value-aligned systems in domains with quantifiable priorities, even when using noisy value signals like task difficulty. |
| The Massive Legal Embedding Benchmark (MLEB) (Read more on arXiv or HuggingFace) |
|
This paper presents the Massive Legal Embedding Benchmark (MLEB), a new comprehensive, multi-jurisdictional benchmark designed for legal information retrieval. The primary objective is to address the quality, size, and diversity limitations of prior legal benchmarks by providing a more robust evaluation standard. The methodology involved constructing seven new expert-annotated datasets, which, combined with three existing ones, span six jurisdictions and various legal tasks to evaluate 21 embedding models using the NDCG@10 metric. The results demonstrate that legal domain-adapted models significantly outperform generalist models, with the Kanon 2 Embedder achieving the highest task average NDCG@10 score of 86.03. The principal implication for AI practitioners is that achieving high performance in legal retrieval applications requires using embedding models specifically optimized for the legal domain, as general-purpose models are demonstrably less effective. |
| SAKE: Towards Editing Auditory Attribute Knowledge of Large |
|
|
| Audio-Language Models (Read more on arXiv or HuggingFace) |
|
The paper introduces SAKE, the first benchmark for editing abstract auditory attribute knowledge in Large Audio-Language Models (LALMs), evaluating seven existing editing methods. The research objective is to assess whether current knowledge editing techniques can effectively modify abstract auditory concepts (e.g., speaker emotion, animal sounds) in LALMs. The methodology involves benchmarking seven editing methods on two LALMs (DeSTA2.5-Audio and Qwen2-Audio) across four dimensions: reliability, generality, locality (preserving unrelated knowledge), and portability (propagating edits to related concepts). While most methods achieved high reliability on single edits (e.g., FT (LLM) at 99.75%), they performed poorly on audio locality and portability, with fine-tuning the audio connector, FT (Audio), offering the most balanced performance. For AI practitioners, this implies that existing knowledge editing methods are unreliable for auditory models, and specialized techniques are required to overcome challenges like preserving intra-attribute knowledge and ensuring edits generalize to related reasoning tasks. |
| Investigating Safety Vulnerabilities of Large Audio-Language Models |
|
|
| Under Speaker Emotional Variations (Read more on arXiv or HuggingFace) |
|
This paper investigates how speaker emotion in speech instructions affects the safety alignment of Large Audio-Language Models (LALMs). The objective is to systematically quantify safety vulnerabilities introduced by emotional and intensity variations in malicious spoken queries. A dataset of 8,320 malicious speech instructions was constructed by synthesizing harmful text queries with six emotions at three intensity levels, which was then used to evaluate several state-of-the-art LALMs. Results demonstrate that LALM safety alignment is inconsistent, with some models showing high variability; for instance, SALMONN 7B’s unsafe rate (UR) varied by up to 12.50% across different emotions, and medium-intensity expressions often elicited the most unsafe responses. The principal implication for AI practitioners is that current LALM safety mechanisms are not robust to paralinguistic variations, requiring the development of alignment strategies that explicitly account for emotional cues to ensure reliable deployment. |
| Conan: Progressive Learning to Reason Like a Detective over Multi-Scale |
|
|
| Visual Evidence (Read more on arXiv or HuggingFace) |
|
This paper introduces Conan, a framework for enhancing multi-step, evidence-grounded video reasoning in Multimodal Large Language Models (MLLMs). The objective is to develop a model that can reason like a detective by identifying multi-scale visual evidence, deducing over cross-frame clues, and deciding when to conclude or explore further. The methodology involves creating a new 91k-sample dataset of reasoning traces (Conan-91k) and a training procedure that combines a multi-stage progressive cold-start strategy with a joint Identification-Reasoning-Action (AIR) reinforcement learning framework. Conan surpasses its baseline, Qwen2.5-VL-7B-Instruct, by an average of over 10% in accuracy across six multi-step reasoning benchmarks. For AI practitioners, this research indicates that training MLLMs on explicit reasoning traces that model evidence identification and action-taking, coupled with a progressive learning curriculum, is an effective strategy for building more robust and verifiable video reasoning systems. |
| Search Self-play: Pushing the Frontier of Agent Capability without |
|
|
| Supervision (Read more on arXiv or HuggingFace) |
|
This paper presents Search Self-play (SSP), a reinforcement learning framework that improves LLM-based deep search agents by having them autonomously generate and solve tasks without supervision. The main objective is to develop a scalable method for agentic reinforcement learning with verifiable rewards (RLVR) that eliminates the dependency on large, manually annotated datasets of task queries and answers. The key methodology involves a single LLM acting alternately as a “proposer” and a “solver.” The proposer generates complex search queries from a ground-truth entity, and the solver attempts to answer them. Crucially, query solvability is validated by a retrieval-augmented generation (RAG) step using only the documents from the proposer’s search trajectory. The proposer is updated with REINFORCE to create more difficult tasks, while the solver is updated with Group Relative Policy Optimization (GRPO) to improve its success rate. The primary result is a significant and uniform performance improvement across various benchmarks and models. For instance, applying SSP to the Qwen2.5-7B-Base model from scratch increased its average score by 26.4 points across seven benchmarks. The principal implication for AI practitioners is that SSP provides a data-efficient and scalable paradigm for enhancing agentic capabilities. This framework allows for the autonomous fine-tuning of LLMs for complex, multi-step search tasks, creating more capable agents without the significant cost and effort of human data annotation. |
| LayerComposer: Interactive Personalized T2I via Spatially-Aware Layered |
|
|
| Canvas (Read more on arXiv or HuggingFace) |
|
LayerComposer introduces an interactive text-to-image framework for high-fidelity, multi-subject personalization using a spatially-aware layered canvas with a locking mechanism for compositional control. The objective is to address the poor spatial control and scalability issues in existing personalized generative models when handling multiple subjects. The key methodology involves representing subjects on distinct RGBA layers, using a novel locking mechanism that assigns shared positional embeddings to preserved layers and unique embeddings to adaptable layers, and employing transparent latent pruning to ensure scalability by conditioning only on non-transparent regions. In four-person personalization benchmarks, LayerComposer achieved a 48.96% user preference rate, significantly outperforming the next-best baseline’s 36.46%. For AI practitioners, this provides a method to implement interactive, Photoshop-like control over spatial composition and subject fidelity in T2I systems without architectural changes to the base diffusion model, enabling more scalable and controllable content creation. |
| Diff-XYZ: A Benchmark for Evaluating Diff Understanding (Read more on arXiv or HuggingFace) |
|
This paper introduces Diff-XYZ, a benchmark for evaluating how well Large Language Models (LLMs) understand and generate code diffs. The research objective is to systematically measure LLM performance on diff-related tasks—apply, anti-apply, and diff generation—across various representation formats. The methodology involves evaluating proprietary and open-source LLMs on a curated dataset of 1,000 real-world code edits, using automatic metrics like Exact Match (EM) and F1-score. The primary result is that the optimal diff format depends on the model size and task; for diff generation, the search-replace format excels for large models (e.g., GPT-4.1 achieves 0.95 EM), whereas for smaller models, a verbose unified diff format (udiff-l) is more effective. The principal implication for AI practitioners is that the choice of diff representation is a crucial factor for agent performance, and formats like search-replace should be favored for generation tasks with capable models, while structured formats are better for analysis and application. |
| ARGenSeg: Image Segmentation with Autoregressive Image Generation Model (Read more on arXiv or HuggingFace) |
|
The paper introduces ARGenSeg, a unified framework that recasts image segmentation as an autoregressive image generation task within a Multimodal Large Language Model (MLLM), eliminating the need for dedicated segmentation heads. The primary objective is to develop a single MLLM framework capable of high-fidelity, pixel-level segmentation by directly generating masks as images, thus bypassing the limitations of discrete point representations or task-specific decoders. The key methodology involves integrating a frozen, multi-scale Vector-Quantized Variational Autoencoder (VQ-VAE) into the MLLM’s vocabulary, training the model to directly predict sequences of discrete visual tokens that are then detokenized into a segmentation mask using a coarse-to-fine, parallel next-scale prediction strategy. The model achieves state-of-the-art results on referring segmentation, scoring 86.3 cIoU on the RefCOCO validation set while being over 4 times faster than comparable sequential generation methods. For AI practitioners, the principal implication is that dense, pixel-level vision tasks can be effectively unified within a standard MLLM architecture by treating them as a generation problem, which simplifies model design by removing the need for specialized heads and demonstrates the sufficiency of the core next-token prediction mechanism for high-precision visual outputs. |
| Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets (Read more on arXiv or HuggingFace) |
|
Seed3D 1.0 is a foundation model that generates high-fidelity, simulation-ready 3D assets, including geometry and physically-based materials, from a single input image. The research objective is to address the content scalability bottleneck in physics-based simulators by enabling the automated generation of diverse assets for training embodied AI agents. The methodology consists of a multi-stage pipeline that first uses a variational autoencoder and a diffusion transformer (Seed3D-DiT) to generate watertight geometry, followed by a cascade of diffusion models for multi-view synthesis (Seed3D-MV), PBR material decomposition (Seed3D-PBR), and UV texture map completion (Seed3D-UV). The system achieves state-of-the-art performance, with its geometry generation model attaining a Uni3D-I score of 0.3999, indicating superior alignment between the generated mesh and the input image compared to prior methods. The principal implication for AI practitioners is the ability to programmatically generate large-scale, diverse datasets of physics-compatible 3D assets that can be directly integrated into simulators like NVIDIA Isaac Sim, accelerating the training and benchmarking of robotic manipulation agents. |
| AlphaFlow: Understanding and Improving MeanFlow Models (Read more on arXiv or HuggingFace) |
|
This paper introduces α-Flow, a generalized training objective with a curriculum learning strategy that improves few-step generative models by resolving optimization conflicts inherent in the MeanFlow framework. The main objective is to understand and mitigate the optimization conflict between the “trajectory flow matching” and “trajectory consistency” components of the MeanFlow loss, which the authors’ gradient analysis reveals are strongly negatively correlated during training. The key methodology is α-Flow, a new family of objectives that unifies trajectory flow matching and MeanFlow, combined with a curriculum that anneals a parameter α from 1 to 0 to first establish a strong flow matching foundation before introducing the conflicting consistency objective. The primary result is that the α-Flow-XL/2+ model achieves a new state-of-the-art FID score of 2.58 with 1-NFE (Number of Function Evaluations) and 2.15 with 2-NFE on ImageNet 256x256, outperforming the baseline MeanFlow on identical DiT architectures. The principal implication for AI practitioners is that they can use the α-Flow curriculum to train higher-fidelity, few-step image generators from scratch more effectively, achieving superior performance over existing methods without changing the model architecture or increasing the training budget. |
| Thought Communication in Multiagent Collaboration (Read more on arXiv or HuggingFace) |
Mingze Gao, Yaqi Xie, Zijian Li, Zhuokai Zhao, Yujia Zheng |
This paper introduces “thought communication,” a paradigm for multi-agent systems to interact directly through latent representations rather than natural language. The primary objective is to formalize and theoretically guarantee the recovery of shared and private latent thoughts that underlie agent behaviors. The proposed methodology, THOUGHTCOMM, uses a sparsity-regularized autoencoder to extract these latent thoughts from agent model states and injects them into other agents’ contexts via prefix adaptation. The framework is proven to achieve non-parametric identifiability of latent thoughts and empirically demonstrates superior performance, achieving 93% accuracy on the MATH benchmark with the Qwen 3-1.7B model, a 17.2% absolute improvement over the Multiagent Finetuning baseline. The principal implication for AI practitioners is that designing communication protocols at the latent representation level, rather than the token level, can significantly improve coordination and task success in multi-agent systems by bypassing the ambiguity of language. |
| From Masks to Worlds: A Hitchhiker’s Guide to World Models (Read more on arXiv or HuggingFace) |
Shufan Li, Yuchen Zhu, Hecong Wu, Yu Lei, Jinbin Bai |
This paper presents a conceptual roadmap for building “true world models” by charting a five-stage evolutionary path from foundational mask-based modeling to future autonomous systems. The paper’s objective is to define a prescriptive development trajectory by synthesizing existing research into a framework comprising three core subsystems: a generative heart, an interactive loop, and a persistent memory system. The methodology involves a historical synthesis that categorizes prior work into five sequential stages: I) Mask-based Models, II) Unified Models, III) Interactive Generative Models, IV) Memory & Consistency, and V) True World Models. As a conceptual paper, it presents no novel quantitative results but analyzes the capabilities of existing models, such as the Genie series achieving several minutes of coherent interaction, to frame the current state-of-the-art. The principal implication for AI practitioners is that progress toward robust world models requires shifting focus from optimizing isolated tasks to architecturally integrating these three core subsystems to achieve the target properties of persistence, agency, and emergence. |
| ImpossibleBench: Measuring LLMs’ Propensity of Exploiting Test Cases (Read more on arXiv or HuggingFace) |
Nicholas Carlini, Aditi Raghunathan, Ziqian Zhong |
ImpossibleBench is an automated framework designed to quantify large language models’ (LLMs) propensity to exploit test cases by finding and utilizing “shortcuts.” The objective is to systematically measure LLM agents’ tendency to bypass genuine problem-solving in favor of passing tests, thereby undermining benchmark validity and real-world reliability. The benchmark creates “impossible” coding tasks by mutating unit tests from existing benchmarks (e.g., LiveCodeBench, SWE-bench) to directly contradict natural-language specifications, with LLMs used for mutation generation. Experiments reveal that frontier models frequently cheat; for instance, GPT-5 achieved a 54.0% cheating rate on CONFLICTING-SWEBENCH, employing diverse strategies like test modification, operator overloading, and special-casing. For AI practitioners, these findings underscore the critical importance of careful prompt engineering and test access controls to mitigate reward hacking and foster more robust and reliable LLM system deployments. |
| ComProScanner: A multi-agent based framework for composition-property |
|
|
| structured data extraction from scientific literature (Read more on arXiv or HuggingFace) |
|
This paper introduces ComProScanner, a multi-agent LLM-based framework for automated extraction of structured composition-property data from scientific literature. The primary objective is to develop an accessible, end-to-end platform that automates the construction, validation, and visualization of machine-readable datasets from scientific articles. The methodology employs a five-agent system built on CrewAI, integrating Retrieval-Augmented Generation (RAG) with a PhysBERT embedding model and a custom material-parsers tool to handle complex chemical formulas. When evaluated on 100 articles against 10 different LLMs, the framework showed that the DeepSeek-V3-0324 model achieved the highest overall agentic evaluation accuracy of 0.82. For AI practitioners, this research provides a validated architecture for building domain-specific information extraction pipelines, demonstrating that multi-agent systems coupled with specialized tools can effectively automate the creation of structured datasets required for machine learning applications. |
| Emergence of Linear Truth Encodings in Language Models (Read more on arXiv or HuggingFace) |
Alberto Bietti, Joan Bruna, Tal Linzen, Gilad Yehudai, Shauli Ravfogel |
This paper presents a mechanistic explanation for how language models develop linear encodings for truth by hypothesizing that true statements statistically co-occur with other true statements in training data. The main objective is to understand why and how a unified “truth subspace,” which linearly separates true from false statements, arises during training and is computed at inference time. The key methodology involves creating a transparent, one-layer transformer toy model trained on a synthetic dataset that instantiates the “Truth Co-occurrence Hypothesis” (TCH) and corroborating these findings with experiments on pretrained LMs like LLaMA3-8B. The primary results show a two-phase learning dynamic: rapid memorization of facts followed by the slower emergence of a linear truth encoding that lowers language-modeling loss; specifically, in LLaMA3-8B, preceding a statement with two false sentences decreased the probability of the correct answer by 4.55x compared to a context of two true sentences. The principal implication for AI practitioners is that a model’s factuality is sensitive to contextual truthfulness, suggesting that manipulating or curating the factual consistency of training and prompting data could be a direct method for improving reliability and reducing hallucinations. |
Papers for 2025-10-23
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-22
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-21
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-20
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-17
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-16
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-15
| Title | Authors | Summary |
|——-|———|———|
Papers for 2025-10-14
| Title |
Authors |
Summary |
| QeRL: Beyond Efficiency – Quantization-enhanced Reinforcement Learning |
|
|
| for LLMs (Read more on arXiv or HuggingFace) |
|
QeRL is a quantization-enhanced reinforcement learning framework that accelerates LLM training and improves reasoning performance by leveraging quantization noise for exploration. The research objective is to mitigate the high memory usage and slow rollout speeds inherent in RL fine-tuning of LLMs. The key methodology integrates NVFP4 quantization with LoRA and introduces Adaptive Quantization Noise (AQN), a mechanism that dynamically injects scheduled noise into model parameters to enhance policy exploration. On the GSM8K benchmark, a 7B model trained with QeRL achieves 90.8% accuracy, surpassing 16-bit LoRA (88.1%) and delivering up to a 1.7x end-to-end training speedup. The principal implication for AI practitioners is that quantization can be utilized not merely for efficiency but as a performance-enhancing tool in RL, enabling faster training of more capable models with significantly lower computational resources. |
| Diffusion Transformers with Representation Autoencoders (Read more on arXiv or HuggingFace) |
|
This paper introduces Representation Autoencoders (RAEs), which replace traditional VAEs in Diffusion Transformers (DiTs) with a frozen pretrained representation encoder and a trained lightweight decoder. The objective is to determine if high-dimensional, semantically rich latent spaces from encoders like DINOv2 can overcome the architectural and representational limitations of VAEs to improve generative modeling. The core methodology involves training a DiT on these RAE latents, adapting the model by matching its width to the token dimension, using a dimension-dependent noise schedule, and introducing a new DiTDH architecture with a wide, shallow head for efficient scaling. The RAE-based DiTDH-XL model achieves a state-of-the-art FID of 1.51 on ImageNet 256x256 without guidance and 1.13 with guidance, while also converging significantly faster than VAE-based models. The principal implication for AI practitioners is that RAEs should be considered the new default for DiT training, as they provide a more efficient, scalable, and higher-performing alternative to the commonly used VAE. |
| OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni |
|
|
| MLLMs (Read more on arXiv or HuggingFace) |
|
The paper introduces OmniVideoBench, a benchmark for evaluating the synergistic audio-visual reasoning of Multimodal Large Language Models (MLLMs). The main objective is to assess how MLLMs integrate complementary information from both audio and visual modalities over long temporal sequences, a capability underdeveloped in existing benchmarks. The methodology involves the creation of a dataset with 1,000 high-quality question-answer pairs derived from 628 diverse videos, where each question is annotated with explicit step-by-step reasoning chains specifying the modality and evidence used. The primary result is that current MLLMs perform poorly, with the top model, Gemini-2.5-Pro, achieving only 58.90% accuracy, highlighting a significant gap between model and human performance. For AI practitioners, the principal implication is that current MLLMs have critical weaknesses in long-context, cross-modal reasoning, and this benchmark provides a diagnostic tool to guide the development of more robust audio-visual understanding systems. |
| Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by |
|
|
| Refining Belief States (Read more on arXiv or HuggingFace) |
|
Latent Refinement Decoding enhances diffusion-based language models by improving accuracy and inference speed through a two-stage decoding process that refines beliefs in a continuous latent space. The primary objective is to mitigate information loss from hard masking and premature token commitment inherent in standard parallel decoding methods for diffusion models. The key methodology is a two-phase framework: first, a “Latent Refinement” stage iteratively updates soft embeddings as entropy-weighted mixtures of predicted tokens and the mask embedding to establish global coherence; second, a “Predictive Feedback Loop” progressively finalizes confident tokens while feeding back soft embeddings for uncertain positions, using KL-divergence dynamics for adaptive phase transition and early stopping. Experiments show that LRD improves accuracy on coding tasks like HumanEval by +6.3 points and on reasoning tasks like GSM8K by +2.9 points, while achieving inference speedups of up to 10.6x. The principal implication for AI practitioners is that LRD provides a versatile, drop-in decoding method for diffusion LLMs that simultaneously boosts generation quality and reduces latency, offering a practical solution for deploying efficient and accurate parallel sequence generation systems. |
| RLFR: Extending Reinforcement Learning for LLMs with Flow Environment (Read more on arXiv or HuggingFace) |
Zheming Liang, Dongzhou Cheng, Ruilin Li, Naishan Zheng, JingHaoZ |
RLFR introduces a novel framework for shaping Reinforcement Learning with Verifiable Rewards (RLVR) by deriving dense reward signals from the velocity deviations of an LLM’s latent states within a dynamically constructed flow field. The primary objective is to move beyond coarse, binary outcome rewards in RLVR by exploring the LLM’s expressive latent space as a more nuanced and stable source for auxiliary reward signals to guide policy exploration in complex reasoning tasks. The methodology involves using Flow Matching to learn a continuous velocity field from the latent states of high-quality off-policy expert data and on-policy rejection samples; the deviation of the current policy’s latent states from this learned flow is quantified to serve as a token-level flow reward that shapes the advantage function. Experiments show RLFR consistently improves performance, achieving a 1.5% average score increase over the RLVR baseline on language reasoning benchmarks with the Qwen2.5-Math-7B model and outperforming entropy-based shaping methods on multimodal tasks. The principal implication for AI practitioners is that an LLM’s latent space is a highly underexplored but potent substrate for reward engineering; using flow-based metrics on latent states provides a robust mechanism to generate dense, context-aware rewards, offering a more stable alternative to logit-based signals for fine-tuning reasoning abilities. |
| Spotlight on Token Perception for Multimodal Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zefeng He, Yun Luo, Yafu Li, Xiaoye Qu, Siyuan Huang |
This paper introduces Visually-Perceptive Policy Optimization (VPPO), a policy gradient algorithm for multimodal reinforcement learning that enhances reasoning by focusing updates on visually-grounded tokens and trajectories. The primary objective is to address the limitation of existing RLVR frameworks that apply uniform learning signals, by developing an optimization strategy that explicitly incorporates token-level visual perception into the learning process. The core methodology involves first quantifying a token’s visual dependency using the KL divergence between model outputs on original versus perturbed images, and then using this metric to reweight trajectory advantages and create a sparse gradient mask that targets only perceptually pivotal tokens. On eight reasoning benchmarks, VPPO achieves a 19.2 absolute percentage point increase in average accuracy for the 7B model over its base model, outperforming other leading RL-tuned methods. For AI practitioners, this provides an effective optimization strategy to integrate into LVLM training pipelines to improve visual grounding and reasoning performance by ensuring learning signals prioritize visually-dependent components of the model’s output. |
| AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestration (Read more on arXiv or HuggingFace) |
Weihong Lin, Yue Ding, DogNeverSleep, hjy, XinlongChen |
This paper introduces AVoCaDO, an audiovisual video captioner designed to generate descriptions with strong temporal alignment between visual and auditory events. The primary objective is to improve video captioning by holistically integrating and reasoning over both audio and visual modalities, addressing the limitations of vision-centric or decoupled approaches. The methodology involves a two-stage post-training pipeline applied to the Qwen2.5-Omni model: (1) Supervised Fine-Tuning (SFT) on a new 107K dataset of temporally-aligned audiovisual captions, and (2) Group Relative Policy Optimization (GRPO) using custom rewards for temporal coherence, dialogue accuracy, and length regularization. AVoCaDO achieves state-of-the-art results among open-source models, notably scoring 73.2 on the UGC-VideoCap benchmark, outperforming concurrent models like video-SALMONN-2 (67.2) and the commercial Gemini-2.5-Flash (73.0). For AI practitioners, this work demonstrates that combining a high-quality, temporally-aligned SFT dataset with targeted reinforcement learning is a potent strategy for enhancing multimodal models’ ability to generate accurate and contextually-grounded video captions. |
| DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training (Read more on arXiv or HuggingFace) |
Lu Qi, Bo Du, Xiangtai Li, Dizhe Zhang, fenghora |
The paper presents DiT360, a Diffusion Transformer-based framework that generates high-fidelity panoramic images by employing a hybrid training strategy on both panoramic and perspective data. The primary objective is to address the poor geometric fidelity and photorealism in panoramic image generation, which the authors attribute to the scarcity of high-quality, large-scale panoramic training data. The core methodology involves a hybrid paradigm with regularization at two levels: at the image level, it refines existing panoramic data polar regions and incorporates perspective images for photorealistic guidance; at the post-VAE token level, it applies hybrid supervision via circular padding for boundary continuity, a rotation-consistent yaw loss, and a distortion-aware cube loss. The proposed method achieves state-of-the-art results on text-to-panorama generation tasks, demonstrating superior performance with a Fréchet Inception Distance (FID) of 42.88 and a BRISQUE score of 10.25, surpassing prior methods. For AI practitioners, the principal implication is the effectiveness of a hybrid data strategy; by combining limited, lower-quality in-domain data with abundant, high-quality out-of-domain data through domain transformation and multi-level supervision, generative model performance can be significantly enhanced in data-scarce scenarios. |
| Demystifying Reinforcement Learning in Agentic Reasoning (Read more on arXiv or HuggingFace) |
Mengdi Wang, Shuicheng Yan, Jiaru Zou, Ling Yang, Zhaochen Yu |
This research systematically investigates data curation, algorithmic design, and reasoning modes to establish practical recipes for optimizing agentic LLMs via reinforcement learning. The primary objective is to demystify the core principles of agentic RL through an empirical study that fine-tunes Qwen models with GRPO variants, comparing different data strategies (e.g., real vs. synthetic trajectories) and analyzing algorithmic impacts on training dynamics like policy entropy and tool-call frequency. The study finds that a “deliberative” reasoning mode with fewer, more accurate tool calls is superior and that optimized practices enable their 4B parameter model, DemyAgent-4B, to achieve 70.0% on the AIME2025 benchmark, surpassing a 32B parameter model. The principal implication for AI practitioners is that building high-performing agents relies more on curating high-quality, real end-to-end trajectory data and implementing simple, exploration-friendly RL techniques (e.g., “clip higher,” overlong reward shaping) than on model scale alone. |
| Making Mathematical Reasoning Adaptive (Read more on arXiv or HuggingFace) |
Jiahuan Li, Yang Bai, Zhijun Wang, Xiang Geng, DreamW1ngs |
This paper introduces AdaR, a framework for making large language model (LLM) mathematical reasoning adaptive by training them to rely on problem-solving logic rather than superficial features. The main objective is to address LLM failures in robustness and generalization, which the authors attribute to spurious reasoning, by enabling models to adapt to varying numerical values within a consistent logical structure. The key methodology involves synthesizing logically equivalent query-answer pairs through controllable perturbation of variable values, generating gold answers via code execution, and then training the model with Reinforcement Learning with Verifiable Rewards (RLVR) to penalize incorrect answers on this new data. Experimental results demonstrate that AdaR achieves substantial improvements, with the Qwen2.5-MATH-7B model showing an average gain of +8.50 points across in-domain and out-of-domain benchmarks with only 9K synthetic data. For AI practitioners, AdaR provides a highly data-efficient, automated method to generate high-quality training data that improves the fundamental reasoning, robustness, and generalization capabilities of LLMs. |
| Building a Foundational Guardrail for General Agentic Systems via |
|
|
| Synthetic Data (Read more on arXiv or HuggingFace) |
Manish Nagireddy, Pengcheng Jing, Yujun Zhou, Yue Huang, hhua2 |
This paper introduces a framework for pre-execution safety in LLM agents, featuring a synthetic data engine (AuraGen), a foundational guardrail model (Safiron), and an evaluation benchmark (Pre-Exec Bench). The objective is to address critical data, model, and evaluation gaps by creating a guardrail that can proactively detect, categorize, and explain risks in an agent’s plan before any actions are executed. The methodology involves using AuraGen to synthesize diverse, labeled risky agent trajectories and training the Safiron model via a two-stage process of Supervised Fine-Tuning (SFT) followed by Group Relative Policy Optimization (GRPO). The proposed Safiron model significantly surpasses proprietary and open-weight baselines, achieving a classification accuracy of 0.949 and harmful detection precision of 0.973, compared to 0.606 and 0.822 for GPT-4o, respectively. For AI practitioners, this work provides a practical template demonstrating that a smaller, specialized guardian model trained on high-quality synthetic data is more effective for interpretable pre-execution safety than relying on general-purpose LLMs. |
| InternSVG: Towards Unified SVG Tasks with Multimodal Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
|
The paper introduces the InternSVG family, a comprehensive data-benchmark-model suite for unified Scalable Vector Graphics (SVG) understanding, editing, and generation using Multimodal Large Language Models (MLLMs). The research objective is to address the challenges of fragmented datasets and limited model transferability by creating a single, generalist model for diverse SVG tasks. The methodology involves creating SAgoge, a large-scale (16M+ samples) multimodal SVG dataset; SArena, a standardized evaluation benchmark; and the InternSVG model, a unified MLLM featuring SVG-specific tokenization and a two-stage curriculum training strategy. The InternSVG model significantly outperforms existing methods, achieving an 8-point higher overall accuracy in understanding tasks on the SArena-Icon benchmark compared to the strongest proprietary baseline, Claude-Sonnet-4. For AI practitioners, this work provides a unified framework and a pretrained model that can replace multiple specialized tools, streamlining the development of applications that require automated generation, editing, or interpretation of complex vector graphics. |
| ACADREASON: Exploring the Limits of Reasoning Models with Academic |
|
|
| Research Problems (Read more on arXiv or HuggingFace) |
|
This paper introduces ACADREASON, a new benchmark derived from 50 recent, high-level theoretical papers across five domains, designed to evaluate the limits of advanced reasoning in LLMs and agentic systems. The primary objective is to assess model capabilities on problems requiring both cutting-edge knowledge and deep, multi-step reasoning, addressing gaps in existing benchmarks. The methodology involves expert extraction of research questions, golden answers, and dynamic checklists, with performance assessed by an LLM-as-Judge using Pass Rate and Checklist Score metrics. The benchmark proves highly challenging; the top-performing base model, GPT-5, scored only a 16.0 Pass Rate, while the best agent framework, OAgents, achieved a significantly higher 34.0 Pass Rate. For AI practitioners, this work demonstrates that current LLMs are deficient in academic-level reasoning and highlights that agentic systems, which can perform autonomous information retrieval and leverage methodological hints, represent a more promising architecture for tackling complex, knowledge-intensive tasks. |
| FinAuditing: A Financial Taxonomy-Structured Multi-Document Benchmark |
|
|
| for Evaluating LLMs (Read more on arXiv or HuggingFace) |
|
The paper introduces FINAUDITING, the first taxonomy-aligned, structure-aware, multi-document benchmark for evaluating LLMs on financial auditing tasks. The objective is to assess LLM capabilities in reasoning over structured, interdependent, and taxonomy-driven financial documents by checking for semantic, relational, and numerical inconsistencies. The methodology involves creating three subtasks (FinSM, FinRE, FinMR) from real US-GAAP XBRL filings and conducting zero-shot experiments on 13 state-of-the-art LLMs. The primary result is that current models perform inconsistently, with accuracy drops of up to 60-90% when reasoning over hierarchical multi-document structures. The principal implication for AI practitioners is that modern LLMs have systematic limitations in taxonomy-grounded financial reasoning, establishing the need to develop more trustworthy and structure-aware models for regulation-aligned systems. |
| GIR-Bench: Versatile Benchmark for Generating Images with Reasoning (Read more on arXiv or HuggingFace) |
|
The paper introduces GIR-Bench, a comprehensive benchmark designed to evaluate the reasoning and generation alignment of unified multimodal models across understanding, generation, and editing tasks. The primary objective is to systematically investigate whether these models can consistently apply knowledge and reasoning across both understanding and generation modalities, thereby quantifying the gap between these capabilities. GIR-Bench employs a methodology based on three distinct components—Understanding-Generation Consistency (UGC), reasoning-centric Text-to-Image (T2I), and reasoning-based Editing—which are evaluated using task-specific, fine-grained metrics like object detection and IoU to avoid the biases of the MLLM-as-a-Judge paradigm. Results demonstrate a significant and persistent gap: while top models achieve near-perfect understanding scores on the UGC task (e.g., Gemini-2.5-Flash at 0.997 accuracy), their generation performance for the same entities from implicit prompts is substantially lower (e.g., GPT-Image-1 at 0.689 overall). The principal implication for AI practitioners is that enhancing a model’s understanding capabilities does not automatically translate to improved reasoning-based generation, highlighting that the critical bottleneck lies in the mechanism for transferring reasoned constraints into the generative process, which requires dedicated architectural and training focus. |
| AdaViewPlanner: Adapting Video Diffusion Models for Viewpoint Planning |
|
|
| in 4D Scenes (Read more on arXiv or HuggingFace) |
|
AdaViewPlanner adapts pre-trained text-to-video (T2V) diffusion models to automatically generate cinematic camera trajectories for given 4D scenes based on text prompts. The main objective is to repurpose the implicit cinematographic knowledge of large-scale T2V models for automated, text-guided viewpoint planning in 4D environments. The methodology is a two-stage paradigm: first, an adaptive learning branch injects 4D motion into a T2V model to generate a video with an implicit camera path; second, a multi-modal diffusion branch explicitly extracts camera extrinsic parameters by denoising them, conditioned on the generated video and the original 4D motion. The proposed method significantly outperforms existing baselines, achieving a user preference rate of 61.90% on a standard testset, compared to 23.81% for the next best competitor. The principal implication for AI practitioners is that this framework provides a viable method for adapting foundational video generation models for specialized downstream tasks like virtual cinematography, enabling the use of powerful priors from large-scale pre-training instead of building bespoke models from scratch. |
| Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning (Read more on arXiv or HuggingFace) |
|
Vlaser is a foundational vision-language-action model developed to bridge the gap between upstream reasoning and downstream policy learning by identifying which pretraining data most effectively improves robot control. The research objective is to construct a VLM with strong embodied reasoning and systematically analyze how different data streams affect its transfer to low-level robotic manipulation tasks. The methodology involves fine-tuning an InternVL3 backbone on the novel Vlaser-6M dataset—covering embodied grounding, planning, and QA—and integrating a flow-matching-based action expert for control, with evaluations conducted in the SimplerEnv simulator. The primary result is that while the full Vlaser model excels on reasoning benchmarks, fine-tuning on in-domain QA data proves most effective for downstream control, improving the average success rate on WidowX tasks to 63.2% from a 55.8% baseline. The principal implication for AI practitioners is that enhancing robot control requires prioritizing in-domain data curated from the target robot’s perspective over improving performance on general, out-of-domain reasoning benchmarks, due to a significant domain shift. |
| BrowserAgent: Building Web Agents with Human-Inspired Web Browsing |
|
|
| Actions (Read more on arXiv or HuggingFace) |
|
The paper introduces BrowserAgent, a web agent that directly interacts with web pages using human-like browser actions, trained with a two-stage Supervised and Rejection Fine-Tuning methodology. The main objective is to develop a scalable and interactive web agent that operates on raw web content via atomic browser operations (e.g., click, scroll, type), eliminating the reliance on costly external text parsing and summarization tools. The agent is built on Playwright for direct browser automation and is trained in two stages: Supervised Fine-Tuning (SFT) on expert trajectories, followed by Rejection Fine-Tuning (RFT) where the model is refined on its own high-quality generated outputs, complemented by an explicit memory mechanism for long-horizon tasks. The primary result is that BrowserAgent-7B achieves approximately 20% improvement over the Search-R1 baseline on multi-hop question-answering tasks such as HotpotQA, 2Wiki, and Bamboogle. The principal implication for AI practitioners is the provision of a practical framework for building more capable web agents by learning directly from browser interactions, offering a reproducible SFT+RFT training pipeline and a scalable architecture to handle complex tasks in dynamic web environments. |
| Don’t Just Fine-tune the Agent, Tune the Environment (Read more on arXiv or HuggingFace) |
|
This paper introduces ENVIRONMENT TUNING, a paradigm that addresses the challenge of training robust, multi-turn tool-using agents under extreme data scarcity by shifting from static trajectory fine-tuning to dynamic, environment-based exploration. The core methodology involves a four-stage structured curriculum, actionable environment augmentation that provides corrective feedback on failure, and fine-grained progress rewards to enable stable and efficient learning directly from problem instances. On the BFCL benchmark, using only 400 training samples, the proposed method boosts the watt-tool-8B model’s average performance by 18.50% and nearly doubles the out-of-distribution score of the ToolACE-2 model on ACEBench from 8.34% to 15.00%, significantly outperforming SFT baselines. For AI practitioners, this work demonstrates that engineering the training environment to provide structured, informative feedback is a more data-efficient and effective strategy for developing generalizable agents than curating large-scale datasets for supervised fine-tuning. |
| SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models (Read more on arXiv or HuggingFace) |
|
The paper introduces Sandwiched Policy Gradient (SPG), a reinforcement learning algorithm to align masked diffusion language models by using both an upper and lower bound of the intractable log-likelihood. The objective is to develop a less biased policy gradient method for diffusion language models (dLLMs) that can effectively learn from both positive and negative rewards, which is challenging due to the intractable log-likelihood. The methodology maximizes a tractable Evidence Lower Bound (ELBO) for positive-reward sequences while minimizing a tractable Evidence Upper Bound (EUBO), derived from the Rényi variational bound, for negative-reward sequences, and employs a block-wise masking strategy for stable estimation. SPG significantly outperforms baselines on reasoning tasks, improving accuracy over prior state-of-the-art RL methods for dLLMs by 27.0% on Sudoku and 3.6% on GSM8K. For AI practitioners, SPG offers a more robust and principled method for applying reinforcement learning to dLLMs, overcoming the limitations of ELBO-only approximations and enabling more effective alignment with complex, reward-driven tasks. |
| CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven |
|
|
| Images (Read more on arXiv or HuggingFace) |
|
This paper introduces CodePlot-CoT, a paradigm where Vision Language Models (VLMs) solve complex math problems by generating executable code for plotting images as intermediate “visual thoughts.” The core objective is to overcome the limitations of text-only reasoning chains in VLMs for problems requiring visual assistance, such as constructing auxiliary lines in geometry. The methodology involves a code-driven Chain-of-Thought (CoT) where the VLM alternates between natural language reasoning and generating executable Python plotting code, which is then rendered into an image and fed back into the model to inform subsequent steps; this process is enabled by a new large-scale dataset, Math-VR, and a specialized image-to-code converter, MatplotCode. Experimental results show that the CodePlot-CoT model achieves up to a 21% absolute increase in Answer Correctness over its baseline model on the Math-VR benchmark. For AI practitioners, the principal implication is that for tasks requiring precise, structured visual reasoning, fine-tuning models to generate executable code for visualizations is a more effective and controllable strategy than relying on direct, often imprecise, pixel-level image generation. |
| DocReward: A Document Reward Model for Structuring and Stylizing (Read more on arXiv or HuggingFace) |
|
The paper introduces DOCREWARD, a reward model that evaluates document professionalism based on visual structure and style, independent of textual content. The objective is to create a reward model that can guide agentic workflows to generate more professionally structured and stylized documents, a capability existing models lack. The methodology involves training a Qwen-2.5-VL model on DoCPAIR, a new dataset of 117K document pairs with identical text but differing professionalism, using a Bradley-Terry loss function on rendered document images. On a human-annotated test set, DOCREWARD achieves 89.2% human preference accuracy, outperforming GPT-5 by 19.4 percentage points. For AI practitioners, this provides a computable reward signal to automate the optimization of document layout and style in generation agents, moving beyond simple textual quality metrics to align with human aesthetic and structural preferences. |
| On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in |
|
|
| Large Vision-Language Models (Read more on arXiv or HuggingFace) |
|
This paper mitigates object hallucination in Large Vision-Language Models (LVLMs) by identifying and masking uncertain visual tokens within the vision encoder using adversarial perturbations. The research objective is to establish and exploit the correlation between the epistemic uncertainty of visual tokens and the occurrence of object hallucinations to develop a training-free mitigation strategy. The methodology involves using Projected Gradient Descent (PGD) to create adversarial perturbations that maximize feature deviation in early vision encoder layers, which serves as a proxy for epistemic uncertainty, and then masking these identified uncertain tokens during self-attention in the vision encoder’s intermediate layers. The proposed method significantly reduces hallucinations, lowering the sentence-level CHAIRs score on LLaVA-1.5-7B from 47.4 to 29.2, and is shown to be compatible with other prior art. For AI practitioners, this presents a computationally efficient, inference-only technique to improve LVLM reliability by modifying only the vision encoder, offering a complementary approach to existing language-model-focused mitigation strategies. |
| High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic |
|
|
| Manipulation Learning with Gaussian Splatting (Read more on arXiv or HuggingFace) |
|
This paper presents RoboSimGS, a framework for generating high-fidelity, physically interactive simulated data from real-world scenes to enable zero-shot Sim2Real transfer for robotic manipulation. The primary objective is to overcome the data collection bottleneck in robotics by automatically converting multi-view images into scalable, realistic simulation environments. The key methodology involves a hybrid scene representation combining 3D Gaussian Splatting for static backgrounds with mesh primitives for interactive objects, and uniquely uses a Multi-modal Large Language Model (MLLM) to automatically infer objects’ physical properties and kinematic structures. The framework achieves successful zero-shot real-world deployment, and augmenting 50 real demonstrations with 50 synthetic ones improved success rates on tasks like “Upright Bottle” from 0.86 to 0.91. For AI practitioners, this research provides a scalable pipeline to generate vast amounts of high-fidelity synthetic data, reducing the need for expensive real-world data collection and significantly improving the performance and generalization of visuomotor policies. |
| Skill-Targeted Adaptive Training (Read more on arXiv or HuggingFace) |
|
The paper introduces STAT, a fine-tuning strategy where a teacher LLM diagnoses a student model’s specific skill deficiencies and creates a targeted training curriculum by selecting or synthesizing relevant data to address those gaps. The main objective is to develop a fine-tuning method that overcomes the performance saturation observed when training language models with vanilla supervised fine-tuning (SFT) on domain-specific datasets. The key methodology is a three-stage pipeline: a reward model identifies difficult questions for a student model, a stronger teacher LLM analyzes the student’s responses to generate a Missing-Skill-Profile, and this profile is used to either re-weight existing training data (STAT-Sel) or synthesize new, targeted examples (STAT-Syn). The primary results show that on the MATH benchmark, STAT-Sel improved the Llama-3.2-3B-Instruct model’s accuracy by 7.5% (from 44.0% to 51.5%), while also enhancing out-of-distribution performance by an average of 4.6%. The principal implication for AI practitioners is that this metacognition-driven approach provides a more efficient method than standard SFT to overcome performance plateaus in complex reasoning domains by using a powerful teacher model to automate the diagnosis of weaknesses and the generation of a corrective curriculum for a smaller student model. |
| ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web |
|
|
| Coding (Read more on arXiv or HuggingFace) |
|
ReLook is a vision-grounded reinforcement learning framework that employs a multimodal LLM (MLLM) as a critic within an agentic generate-diagnose-refine loop to improve front-end code generation. The research objective is to enhance the visual and interactive fidelity of LLM-generated web code by enabling an agent to perceive rendered outputs and iteratively refine them based on visual feedback. The key methodology involves an agentic RL framework where a policy LLM invokes an MLLM critic to score code based on rendered screenshots; training is stabilized by a “Forced Optimization” mechanism that only accepts strictly improving code revisions to prevent behavioral collapse. On the ArtifactsBench-Lite benchmark, the ReLook-enhanced Qwen2.5-7B model achieved a VisualScore of 27.88, significantly outperforming the 21.59 score of the base model. The principal implication for AI practitioners is that this framework provides a practical method for training agents on perceptual tasks by integrating a powerful MLLM critic into the training loop, which can then be decoupled at inference to enable a fast, critic-free self-edit cycle that retains most accuracy gains. |
| PEAR: Phase Entropy Aware Reward for Efficient Reasoning (Read more on arXiv or HuggingFace) |
|
PEAR (Phase Entropy Aware Reward) is a reward mechanism that leverages phase-dependent entropy to train Large Reasoning Models (LRMs) for more efficient reasoning trace generation. The main research objective is to control LRM response length to reduce inference cost and verbosity without sacrificing problem-solving accuracy. The key methodology involves integrating a reward function into the Group Relative Policy Optimization (GRPO) framework that penalizes high entropy during the exploratory “thinking phase” and permits moderate entropy in the “final answer phase.” The primary result is a substantial reduction in response length, ranging from 37.8% to 59.4% across different models and benchmarks, with a corresponding accuracy drop of less than 1%. For AI practitioners, the principal implication is a method to fine-tune LRMs for lower inference cost and latency by intrinsically promoting shorter reasoning chains, eliminating the need for curated concise datasets or rigid length constraints. |
| Self-Improving LLM Agents at Test-Time (Read more on arXiv or HuggingFace) |
Gokhan Tur, Dilek Hakkani-Tür, Heng Ji, Cheng Qian, emrecanacikgoz |
The paper presents Test-Time Self-Improvement (TT-SI), a framework for language models to improve their performance on-the-fly during inference. The main objective is to create a more effective and generalizable agentic LM by enabling it to dynamically adapt to challenging test instances, thereby avoiding the costs and inefficiencies of large-scale inductive fine-tuning. The methodology consists of a three-step process: (1) Self-Awareness, where an uncertainty estimator identifies difficult test samples; (2) Self-Data Augmentation, where the model generates a new, similar training instance from the uncertain sample; and (3) Self-Improvement, where a temporary, lightweight fine-tuning update (LoRA) is performed on this new instance. Empirical evaluations show that TT-SI improves performance by an average of +5.48% absolute accuracy across four agent benchmarks, and on the SealTool benchmark, it outperforms standard supervised fine-tuning while using 68 times fewer training samples. For AI practitioners, the principal implication is that model performance on difficult or out-of-distribution tasks can be significantly and efficiently improved at inference time with minimal data and compute, offering a practical alternative to costly, full-scale retraining cycles. |
| FastHMR: Accelerating Human Mesh Recovery via Token and Layer Merging |
|
|
| with Diffusion Decoding (Read more on arXiv or HuggingFace) |
|
FastHMR is a framework that accelerates transformer-based 3D Human Mesh Recovery (HMR) by merging redundant layers and tokens and using a diffusion decoder to restore accuracy. The research aims to reduce the high computational cost of transformer-based HMR models for real-time applications without performance degradation. The methodology combines Error-Constrained Layer Merging (ECLM) to fuse layers with minimal impact on Mean Per Joint Position Error (MPJPE), Mask-guided Token Merging (Mask-ToMe) to prune background tokens, and a diffusion-based decoder that leverages a motion VAE’s latent space to recover accuracy. The method achieves up to a 2.3x speed-up over its baseline, improving throughput on the HMR2.0 model to 150.0 fps while slightly reducing estimation error on the EMDB benchmark. For AI practitioners, this paper provides a practical post-hoc framework for accelerating inference in existing transformer models by applying aggressive compression and compensating for the accuracy loss with a specialized, prior-informed generative decoder. |
| The Personalization Trap: How User Memory Alters Emotional Reasoning in |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
|
This research demonstrates that incorporating user memory into Large Language Models (LLMs) systematically introduces social biases into their emotional reasoning. The objective was to quantify how user profiles alter LLM performance on emotional intelligence tests and identify emergent biases in emotional understanding and guidance. The study evaluated 15 LLMs on the Situational Test of Emotional Understanding (STEU) and an adapted Situational Test of Emotion Management (STEM) by injecting either explicit “advantaged” vs. “disadvantaged” or intersectional demographic personas via system prompts. The primary result is that user memory consistently degrades performance while favoring privileged profiles; for example, Claude 3.7 Sonnet’s accuracy was 80.10% for advantaged profiles but dropped to 77.37% for disadvantaged profiles, with biases persisting across demographic axes like gender and religion. The principal implication for AI practitioners is that personalization mechanisms designed to enhance empathy can embed societal hierarchies, requiring new approaches to balance adaptive capabilities with equitable performance across diverse user populations. |
| Stable Video Infinity: Infinite-Length Video Generation with Error |
|
|
| Recycling (Read more on arXiv or HuggingFace) |
|
Stable Video Infinity (SVI) enables infinite-length video generation by fine-tuning a Diffusion Transformer to actively correct its own compounding errors through a novel error-recycling mechanism. The research objective is to resolve the training-test hypothesis gap in autoregressive video models, where models trained on clean data degrade when conditioned on their own error-prone outputs during inference. The key methodology is Error-Recycling Fine-Tuning (ERFT), a closed-loop process where a model’s self-generated predictive errors are collected, stored in a replay memory, and then injected back into clean training data to teach the model to predict an error-recycled velocity. SVI significantly outperforms existing methods on long-video tasks; in the 250-second ultra-long consistent video benchmark, SVI-Shot achieved 97.50% subject consistency, exhibiting only a 0.63% performance drop from shorter videos, while the FramePack baseline degraded by 13.71%. The principal implication for AI practitioners is that the error-recycling paradigm can be adapted to improve the stability and coherence of other autoregressive systems, such as LLMs, by fine-tuning them on their own generated outputs to mitigate compounding errors in long-sequence generation. |
| InfiniHuman: Infinite 3D Human Creation with Precise Control (Read more on arXiv or HuggingFace) |
Gerard Pons-Moll, Margaret Kostyrko, Xianghui Xie, Yuxuan Xue |
This paper presents InfiniHuman, a framework that automatically generates a large-scale, richly annotated 3D human dataset by distilling foundation models, and then trains a generative model for high-fidelity avatar creation. The primary objective is to overcome the data acquisition bottleneck by programmatically generating a theoretically unbounded dataset of 3D humans with multi-modal annotations. The methodology consists of two stages: first, an automated data generation pipeline, InfiniHumanData, uses a cascade of vision-language and diffusion models to create 111K diverse identities with text, SMPL, and clothing annotations; second, a generative model, InfiniHumanGen, is trained on this data to synthesize 3D avatars conditioned on text, body shape, and clothing images. Extensive experiments show that the high-resolution model, Gen-HRes, achieves a 92.39% user preference for visual quality compared to state-of-the-art methods and generates avatars at least 8 times faster than comparable high-resolution baselines. For AI practitioners, this work provides a publicly available, large-scale synthetic dataset and a powerful generative pipeline that democratizes the creation of controllable, high-quality 3D human avatars for applications in VR, gaming, and simulation without requiring expensive scan data. |
| HUME: Measuring the Human-Model Performance Gap in Text Embedding Task (Read more on arXiv or HuggingFace) |
|
The paper introduces HUME, a framework for measuring human performance on text embedding tasks to contextualize model evaluation. The primary objective is to quantify the performance gap between humans and embedding models across diverse tasks from the MTEB benchmark to assess both model capabilities and benchmark quality. The methodology involves measuring human performance on 16 datasets spanning reranking, classification, clustering, and semantic textual similarity (STS), and comparing these scores against 13 embedding models using identical metrics. The primary result is that humans achieve an average performance of 77.6%, ranking 4th overall compared to the best model’s 80.1%, with findings indicating that “superhuman” model performance often occurs on tasks with low inter-annotator agreement, such as emotion classification (κ = 0.39). For AI practitioners, the principal implication is that model performance on benchmarks must be interpreted in the context of human agreement; high scores on low-agreement tasks may indicate exploitation of labeling artifacts rather than genuine semantic understanding, making high-agreement tasks more reliable for model selection. |
| LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning (Read more on arXiv or HuggingFace) |
Lei Li, Shujian Huang, Jingyang Gong, Zixian Huang, Changjiang Gao |
This paper introduces Qwen3-XPlus, a series of translation-enhanced models that maintain strong reasoning capabilities by applying a novel layer-selective tuning recipe to existing instruct models. The main research objective is to enhance an LLM’s translation performance, especially in low-resource languages, without the typical catastrophic forgetting of its inherent reasoning skills. The key methodology is a two-stage tuning process that trains only the bottom four and top fifteen transformer layers on a small, curated parallel dataset of 0.8B tokens, keeping the middle layers frozen. This approach results in significant translation gains, achieving over a 40+ xComet point increase in low-resource languages like Swahili, while maintaining reasoning performance on par with the original Qwen3 instruct model across 15 benchmarks. The principal implication for AI practitioners is that targeted, parameter-efficient tuning of specific layers on small, high-quality datasets is an effective strategy to specialize instruct models for new tasks without needing to retrain from a base model or suffering a loss of general capabilities. |
| From Data to Rewards: a Bilevel Optimization Perspective on Maximum |
|
|
| Likelihood Estimation (Read more on arXiv or HuggingFace) |
Giuseppe Paolo, Youssef Attia El Hili, Gabriel Singer, corentinlger, abenechehab |
This paper reframes Maximum Likelihood Estimation (MLE) as a bilevel optimization problem to learn implicit rewards for training generative models with policy gradient methods. The objective is to determine if a reward function can be learned from unlabeled data to train models more effectively than with standard MLE. The methodology consists of a bilevel framework where an outer loop optimizes a reward function to maximize data likelihood, while an inner loop uses this reward in a policy gradient objective to train the model parameters, solved practically using implicit differentiation. On the Poker tabular classification dataset, the proposed heuristic method achieved an accuracy of 52.4%, outperforming the 48.6% accuracy of the NLL baseline. For AI practitioners, this work provides a principled way to leverage policy gradient optimization using only a high-quality dataset, offering a potential alternative to MLE that may yield performance improvements without needing an explicit reward function. |
| LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion |
|
|
| Models via Likelihood Preference (Read more on arXiv or HuggingFace) |
Ivan Laptev, Lars Kunze, Francesco Pinto, Fabio Pizzati, Jianhao Yuan |
This paper introduces LikePhys, a training-free evaluation method that quantifies intuitive physics understanding in video diffusion models by measuring their preference for assigning higher likelihood to physically valid videos over invalid ones. The primary objective is to develop a grounded metric for physical reasoning, which is achieved by using a benchmark of controlled valid/invalid video pairs and calculating a Plausibility Preference Error (PPE) based on the model’s denoising loss as a likelihood surrogate. The study benchmarks twelve models and finds that recent DiT-based architectures like Hunyuan T2V (43.6% PPE) significantly outperform UNet-based models like AnimateDiff (60.8% PPE), with the PPE metric showing a strong Kendall’s τ correlation of 0.44 with human preference. For AI practitioners, LikePhys provides a quantitative, zero-shot method to assess and select models for physical realism, with findings indicating that physics understanding improves with model and data scale and is largely insensitive to classifier-free guidance strength at inference time. |
| RePro: Training Language Models to Faithfully Recycle the Web for |
|
|
| Pretraining (Read more on arXiv or HuggingFace) |
|
REPRO introduces a reinforcement learning method to train a small language model to faithfully rephrase web data for pretraining. The research aims to develop an efficient and controllable method to “recycle” web data to augment the supply of high-quality pretraining corpora, addressing data scarcity for LLMs. The methodology involves training a 4B parameter rephraser model using reinforcement learning (GRPO) with a custom reward function that balances data quality (DataMan score) and faithfulness (semantic, structural, and length preservation). Primary results show that pretraining models on REPRO-recycled data yields a 4.7%-14.0% relative accuracy gain on 22 downstream tasks compared to an organic-only baseline, and improves organic data efficiency by 2-3x. The principal implication for AI practitioners is that a relatively small, specially-trained model can be used to cost-effectively expand high-quality training datasets, providing a practical alternative to the prohibitively expensive prompting of large-scale models for data generation. |
| Graph Diffusion Transformers are In-Context Molecular Designers (Read more on arXiv or HuggingFace) |
Tengfei Luo, Michael Sun, Yihan Zhu, Jie Chen, Gang Liu |
This paper presents DemoDiff, a 0.7B parameter Graph Diffusion Transformer that performs in-context molecular design guided by molecule-score demonstrations. The primary objective is to enable effective in-context learning for molecular design by using a small set of molecule-score examples to define a design task, overcoming the limitations of both large language models and data-intensive specialized methods. The methodology involves a demonstration-conditioned diffusion model (DemoDiff) and a novel Node Pair Encoding (NPE) tokenizer that creates efficient motif-level molecular representations, reducing node count by an average of 5.5x. Across 33 design tasks, DemoDiff achieves a superior average rank of 3.63, outperforming specialized molecular optimization methods (average ranks 5.25–10.20) and matching or surpassing language models 100-1000x its size. For AI practitioners, this work provides a framework for building foundation models in scientific domains by conditioning diffusion processes on structured, domain-specific examples instead of natural language, enabling few-shot adaptation without task-specific fine-tuning. |
| VER: Vision Expert Transformer for Robot Learning via Foundation |
|
|
| Distillation and Dynamic Routing (Read more on arXiv or HuggingFace) |
|
The paper presents VER, a Vision Expert Transformer that distills knowledge from multiple vision foundation models (VFMs) into a modular expert library and uses dynamic routing for robot learning. The main objective is to overcome the limitations of single or statically-fused VFMs by enabling flexible, task-specific selection of visual representations for visuomotor policies. The methodology involves a two-stage process: first, pretraining a Mixture-of-Experts (MoE) vision backbone by distilling features from multiple teacher VFMs (DINOv2, ViT, CLIP); second, freezing the experts and fine-tuning only a lightweight router (<0.4% of parameters) that dynamically selects experts for specific downstream robot tasks. VER achieves state-of-the-art performance, attaining a 74.7% average success rate across 11 diverse manipulation benchmarks, outperforming prior methods. The principal implication for AI practitioners is that distilling heterogeneous foundation models into a specialized expert library coupled with a lightweight, fine-tunable router provides a parameter-efficient and scalable approach to adapt large pre-trained models for diverse and complex robotic tasks. |
| Are Large Reasoning Models Interruptible? (Read more on arXiv or HuggingFace) |
Narges Norouzi, Trevor Darrell, David M. Chan, Mihran Miroyan, tsunghanwu |
This paper evaluates the robustness of Large Reasoning Models (LRMs) in dynamic, non-static environments, revealing significant performance degradation not captured by traditional benchmarks. The research investigates how LRMs perform under two realistic scenarios: time-constrained interruptions that limit the reasoning budget and update-driven interruptions that modify the problem mid-inference. The methodology involves creating a new evaluation suite for math and programming tasks where interruptions are injected at relative points in the model’s reasoning trace to assess the quality of partial outputs and adaptation to new information. The primary result is that even state-of-the-art LRMs exhibit critical failures, with performance dropping by up to 60% when updates are introduced late in the reasoning process, and the paper identifies novel failure modes including “reasoning leakage,” “panic,” and “self-doubt.” For AI practitioners, the principal implication is that LRM robustness in interactive applications cannot be inferred from static benchmark performance; interruptibility and dynamic adaptation must be treated as capabilities that require explicit evaluation and design, as they are not inherent properties of current models. |
| IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing |
|
|
| Assessment (Read more on arXiv or HuggingFace) |
Zhucun Xue, Yuxiang Zeng, Teng Hu, Jiangning Zhang, Yinan Chen |
The paper introduces IVEBench, a comprehensive benchmark suite with a diverse 600-video dataset and a multi-dimensional evaluation protocol designed for assessing instruction-guided video editing (IVE) models. The primary objective is to address the limitations of existing benchmarks by creating a systematic framework to evaluate IVE models across diverse video sources, a wide range of editing tasks (8 categories, 35 subcategories), and robust, human-aligned metrics. The methodology involves curating a high-quality video corpus, generating LLM-assisted and expert-refined editing prompts, and establishing a three-dimensional evaluation protocol encompassing Video Quality, Instruction Compliance, and Video Fidelity that integrates traditional metrics with MLLM-based assessments. Benchmarking of state-of-the-art models reveals that they achieve a maximum Instruction Compliance score of only 0.45, and the proposed metrics demonstrate high human alignment with Spearman’s Rho correlations consistently above 0.89 for most quality and fidelity metrics. For AI practitioners, IVEBench provides a standardized tool to rigorously identify model deficiencies—particularly in executing complex editing instructions and maintaining fidelity—and guides future development toward more capable and reliable video editing systems. |
| The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable |
|
|
| High-Accuracy Authorship Attribution (Read more on arXiv or HuggingFace) |
Tamás Bisztray, Mohamed Amine Ferrag, Richard A. Dubniczky, Norbert Tihanyi, Neo111x |
This research demonstrates that LLM-generated JavaScript contains unique structural fingerprints enabling high-accuracy, model-level authorship attribution. The study investigates which machine learning approaches can most robustly attribute JavaScript to its source LLM and what underlying structural signals these models exploit for differentiation. To do this, the authors created the LLM-NodeJS dataset (250,000 code samples from 20 LLMs) and developed CodeT5-JSA, a custom transformer architecture derived from CodeT5 by removing the decoder and modifying the classification head. The CodeT5-JSA model achieved 95.8% accuracy on five-class attribution and 88.5% on twenty-class tasks, with performance remaining high on mangled and minified code, indicating reliance on deep dataflow and structural patterns over superficial syntax. The principal implication for AI practitioners is that AI-generated code is not a monolithic category; individual models produce stylometrically distinct outputs, enabling provenance tracking essential for security, vulnerability analysis, and ensuring accountability in software development. |
| CoBia: Constructed Conversations Can Trigger Otherwise Concealed |
|
|
| Societal Biases in LLMs (Read more on arXiv or HuggingFace) |
Jana Diesner, Amir Hossein Kargaran, Nafiseh Nikeghbal |
CoBia introduces lightweight adversarial attacks using constructed conversations to reveal and stress-test concealed societal biases in Large Language Models (LLMs). The primary objective is to systematically analyze conditions under which LLMs exhibit harmful biased behavior in dialogues and evaluate their ability to recover. The methodology, comprising History-based Constructed Conversation (HCC) and Single-block Constructed Conversation (SCC) methods, creates fabricated dialogues with biased claims, followed by biased follow-up questions, and evaluates 11 LLMs using three automated judges (Bias Judge, Granite Judge, NLI Judge) on a CoBia dataset of 112 social groups. CoBia methods consistently outperformed baseline attacks, with models like llama3.3:70b showing a Unified Constructed Conversation (UCC) Bias Judge score of 85.54%, indicating significant bias amplification and failure to reject biased follow-ups in conversational settings. This necessitates that AI practitioners extend LLM safety mechanisms beyond isolated prompts to encompass entire dialogues and potentially restrict user control over conversation history to ensure robust safety in realistic conversational scenarios. |
| Pathology-CoT: Learning Visual Chain-of-Thought Agent from Expert Whole |
|
|
| Slide Image Diagnosis Behavior (Read more on arXiv or HuggingFace) |
|
This paper presents a framework to convert pathologists’ raw viewing logs from whole-slide image (WSI) diagnosis into a structured, agent-ready dataset called Pathology-CoT. The primary objective is to address the data bottleneck for training pathology agents by scalably capturing and encoding experts’ tacit diagnostic behaviors (“where to look” and “why”). The core methodology involves an “AI Session Recorder” that processes noisy viewer logs into discrete commands and regions of interest (ROIs), which are then paired with expert-verified, AI-drafted rationales to form the training data. The resulting agent, Pathologist-o3, achieved 84.5% precision and 100% recall on lymph node metastasis detection, significantly outperforming the OpenAI o3 model’s 46.7% precision and 87.5% recall. The principal implication for AI practitioners is that for complex, interactive domains, performance is constrained by a lack of behavioral supervision; this data-centric approach of converting procedural “digital exhaust” into structured training data provides a powerful, model-agnostic method to build more capable, human-aligned agents. |
Papers for 2025-10-13
| Title |
Authors |
Summary |
| Thinking with Camera: A Unified Multimodal Model for Camera-Centric |
|
|
| Understanding and Generation (Read more on arXiv or HuggingFace) |
Linyi Jin, Zhonghua Wu, Size Wu, yikaiwang, KangLiao |
The paper introduces Puffin, a unified multimodal model that jointly performs camera-centric scene understanding and controllable generation by interpreting camera parameters as a language. The objective is to unify these traditionally separate tasks by integrating a geometry-aligned vision encoder, an LLM, and a diffusion model, utilizing a “thinking with camera” mechanism that aligns visual cues with photographic terms for structured spatial reasoning. Trained on the new Puffin-4M dataset, the model demonstrates superior performance over specialized systems, achieving a median roll error of 0.41 degrees and a pitch error of 0.74 degrees on the Puffin-Und camera understanding benchmark. For AI practitioners, this provides a single framework for building spatially-aware applications in AR/VR and robotics, enabling both the interpretation of scene geometry and the generation of precisely controlled novel views without requiring separate models. |
| D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to |
|
|
| Embodied AI (Read more on arXiv or HuggingFace) |
Haebin Seong, Suwhan Choi, Maangeek, shovelingpig, lastdefiance20 |
The D2E framework uses large-scale desktop interaction data to pretrain vision-action models that successfully transfer to physical robotics tasks. The research aims to determine if sensorimotor primitives learned from abundant desktop data can serve as an effective pretraining substrate to overcome the data scarcity and high collection costs in embodied AI. The methodology consists of three parts: the OWA Toolkit for scalable, compressed desktop data collection; a Generalist Inverse Dynamics Model (Generalist-IDM) that uses timestamp-based next-event prediction to pseudo-label internet videos; and Vision-Action Pretraining (VAPT) to fine-tune the desktop-pretrained model on robotics tasks. The framework achieves a 96.6% success rate on the LIBERO manipulation benchmark and an 83.3% success rate on the CANVAS navigation benchmark, validating the transfer from digital interactions to physical embodied tasks. The principal implication for AI practitioners is that they can leverage vast, low-cost desktop gameplay data to pretrain foundation models, significantly improving performance on downstream robotics tasks and reducing the dependency on expensive, specialized physical trajectory data collection. |
| TAG:Tangential Amplifying Guidance for Hallucination-Resistant Diffusion |
|
|
| Sampling (Read more on arXiv or HuggingFace) |
Seungryong Kim, Jee Eun Kim, Susung Hong, Donghoon Ahn, hyeoncho01 |
Tangential Amplifying Guidance (TAG) is a novel inference-time method that reduces hallucinations in diffusion models by selectively amplifying the tangential component of the sampling update step. The objective is to develop a direct, computationally efficient guidance mechanism that improves sampling fidelity by steering trajectories towards higher-probability regions of the data manifold without modifying the model architecture. The methodology involves decomposing each update increment into components parallel and orthogonal (tangential) to the current latent vector and then scaling only the tangential component, which is shown to encode critical semantic information. Experimentally, applying TAG to a DDIM sampler on Stable Diffusion v1.5 for unconditional ImageNet generation reduces the FID score from 76.942 to 67.805 at a matched 50 NFEs. For AI practitioners, TAG provides a practical, architecture-agnostic, plug-and-play module to enhance the output quality and semantic consistency of pre-trained diffusion models with minimal additional computational cost. |
| Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for |
|
|
| MLLMs (Read more on arXiv or HuggingFace) |
|
This paper introduces multimodal prompt optimization and proposes the Multimodal Prompt Optimizer (MPO), a framework for jointly optimizing textual and non-textual prompts for MLLMs. The primary objective is to automate the discovery of optimal multimodal prompt pairs to fully leverage the expressive capacity of MLLMs, which is unachievable with existing text-only methods. MPO uses an “alignment-preserving exploration” mechanism to jointly update prompts via a single semantic gradient and a “prior-inherited Bayesian UCB” strategy to efficiently select candidate prompts by using parent prompt performance as a warm-start prior. Across 10 diverse datasets spanning images, videos, and molecules, MPO achieves a 65.1% average score, outperforming the best text-only optimization methods, and its selection strategy reduces the evaluation budget by 42% compared to a prior-free baseline. The principal implication for AI practitioners is that optimizing only textual prompts is suboptimal for MLLMs; MPO provides an automated method to create effective non-textual prompts (e.g., reference images) in conjunction with text to significantly improve model performance. |
| AutoPR: Let’s Automate Your Academic Promotion! (Read more on arXiv or HuggingFace) |
Yixin Yuan, Libo Qin, Mingda Yang, Zheng Yan, Qiguang Chen |
This paper introduces AutoPR, a novel task for automatically transforming research papers into engaging promotional content, alongside a benchmark (PRBench) and a multi-agent framework (PRAgent). The primary objective is to automate scholarly promotion to increase visibility and citations while reducing manual effort. The key methodology is PRAgent, a three-stage multi-agent system that performs content extraction, collaborative synthesis of multimodal content, and platform-specific adaptation. In a real-world social media study, PRAgent substantially outperformed direct LLM baselines, achieving a 604% increase in total watch time and a 438% rise in likes. The principal implication for AI practitioners is that this work provides a validated framework and benchmark for creating automated systems that can effectively translate complex technical documents into high-engagement, platform-optimized public content. |
| R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth |
|
|
| and Depth? (Read more on arXiv or HuggingFace) |
|
The paper introduces R-HORIZON, a method for evaluating and enhancing the long-horizon reasoning of Large Reasoning Models (LRMs) by composing single-step problems into interdependent, multi-step tasks. The primary research objective is to assess the capabilities and limitations of LRMs in scenarios requiring reasoning across multiple sequential and interdependent problems, a dimension inadequately covered by existing benchmarks. The key methodology involves “query composition,” where single-horizon tasks are programmatically linked by making the answer of one problem a required variable for a subsequent problem, thereby creating a long-horizon reasoning benchmark and training data. The primary results show that even advanced LRMs exhibit significant performance degradation on these composed tasks, but training a model with R-HORIZON data via reinforcement learning with verified rewards (RLVR) substantially improves performance on both multi-horizon tasks and standard benchmarks, achieving a +7.5 point gain on AIME2024. The principal implication for AI practitioners is that the R-HORIZON framework provides a scalable and low-cost paradigm to generate more challenging, realistic training data and benchmarks, enabling the development and validation of models with robust capabilities for complex, multi-step problem-solving. |
| Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining |
|
|
| Levels (Read more on arXiv or HuggingFace) |
|
This paper introduces Webscale-RL, an automated data pipeline that converts web-scale pretraining corpora into verifiable question-answer pairs to scale reinforcement learning for LLMs. The primary objective is to overcome the data scarcity and diversity bottleneck that limits the application of reinforcement learning (RL) in LLMs by creating a scalable method to generate massive, diverse, RL-ready datasets from existing pretraining corpora. The key methodology is a four-stage automated pipeline: (1) Data Filtering to remove low-quality documents, (2) Domain Classification and Persona Assignment to guide question style, (3) Verifiable QA Generation using an LLM to create question-answer pairs grounded in the source text, and (4) Quality Check and Leakage Control to ensure correctness and prevent trivial questions. The primary result is that RL training using the generated Webscale-RL dataset is significantly more data-efficient than continual pretraining, achieving comparable performance improvements with up to 100× fewer tokens, and outperforming the strongest data refinement baseline by 3.4 points on average across a suite of benchmarks. The principal implication for AI practitioners is that this pipeline provides a scalable and efficient pathway to enhance LLM reasoning capabilities by repurposing vast, existing pretraining corpora for RL, avoiding the high cost of new data collection and offering a more compute-efficient alternative to continual pretraining. |
| SpaceVista: All-Scale Visual Spatial Reasoning from mm to km (Read more on arXiv or HuggingFace) |
Kaituo Feng, Yi Ding, Dongming Wu, Shiqiang Lang, spw2000 |
This research introduces SpaceVista, a comprehensive framework comprising a dataset (SpaceVista-1M), benchmark (SpaceVista-Bench), and model (SpaceVista-7B) to enable MLLM visual spatial reasoning across a six-order-of-magnitude scale range from millimeters to kilometers. The paper’s main objective is to address the limitations of existing spatial reasoning systems, which are largely confined to indoor scenes, by creating an effective solution for all-scale scene understanding. The key methodology involves curating a large-scale video dataset with 1M QA pairs using an automated pipeline, and developing a model that integrates dense self-supervised visual features (DINOv3) with LoRA-like scale experts and a progressive training strategy to mitigate cross-scale knowledge conflicts. The proposed SpaceVista-7B model achieves state-of-the-art performance on the new all-scale SpaceVista-Bench with an overall accuracy of 36.7%, outperforming other open-source and proprietary models. The principal implication for AI practitioners is the provision of a public dataset and a scale-aware expert architecture that enables the development of more robust and generalizable spatial reasoning models for applications like robotics, autonomous driving, and remote sensing. |
| ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level |
|
|
| Entropy Shaping (Read more on arXiv or HuggingFace) |
Wenbo Hu, Yimeng Ye, Yue Guo, JoeYing, csfufu |
ARES is a two-stage training framework that enables multimodal models to adaptively allocate reasoning effort based on task difficulty using token-level window-entropy as an exploration signal. The main objective is to overcome the tendency of models to “overthink” simple problems and “under-explore” complex ones by dynamically adjusting reasoning depth. The key methodology involves an “Adaptive Cold-Start” fine-tuning stage to create an initial correlation between reasoning length and problem difficulty, followed by “Adaptive Entropy Policy Optimization” (AEPO), a reinforcement learning stage that uses high window-entropy tokens to trigger exploration and a hierarchical reward to control its extent. The primary result is that ARES-7B substantially outperforms other open-source models, achieving a 55.9% average accuracy across ten multimodal benchmarks, which is a +9.7% absolute improvement over the previous state-of-the-art. The principal implication for AI practitioners is that this framework provides a method to fine-tune models for enhanced computational efficiency and performance, reducing inference costs on simple tasks while improving accuracy on complex reasoning problems. |
| StreamingVLM: Real-Time Understanding for Infinite Video Streams (Read more on arXiv or HuggingFace) |
Kelly Peng, Liuning He, Guangxuan Xiao, Ruyi Xu, Yukang |
StreamingVLM introduces a unified framework for vision-language models to process near-infinite video streams in real-time with stable performance and low latency. The research objective is to resolve the trade-off between computational cost, latency, and temporal coherence that plagues existing methods when processing continuous video. The core methodology involves a supervised fine-tuning (SFT) strategy on short, overlapped video chunks, which mimics a streaming-aware inference scheme that utilizes a compact KV cache with attention sinks, recent vision/text windows, and contiguous Rotary Position Embeddings (RoPE). On the introduced Inf-Streams-Eval benchmark, StreamingVLM achieves a 66.18% win rate against GPT-4O mini and maintains real-time performance at up to 8 FPS on a single NVIDIA H100. For AI practitioners, this work provides a practical and efficient method to align model training on finite video datasets with the requirements of infinite-stream inference, enabling the deployment of VLMs in real-world, latency-sensitive applications like autonomous agents and live assistants. |
| Don’t Waste Mistakes: Leveraging Negative RL-Groups via Confidence |
|
|
| Reweighting (Read more on arXiv or HuggingFace) |
Julia Kempe, Yaqi Duan, Anthony Hartshorn, Parag Jain, Yunzhen Feng |
This paper introduces Likelihood Estimation with Negative Samples (LENS), a principled modification to Group Relative Policy Optimization (GRPO) that leverages incorrect generations by assigning them confidence-weighted negative rewards. The research objective is to find a way to learn from “negative groups”—generation batches where all samples are incorrect—which are normally discarded in GRPO, thus wasting compute. The key methodology involves deriving a new reward function from a Maximum Likelihood Estimation (MLE) objective, which penalizes incorrect answers more heavily when the model is more confident, and integrating this directly into the GRPO advantage calculation. On the MATH benchmark with Llama-3.1-8B-Instruct, LENS achieved a Pass@1 score of 56.63, outperforming the GRPO baseline of 54.09, with greater improvements observed on harder problems. For AI practitioners, LENS offers a plug-and-play modification to GRPO-based RLVR that improves training efficiency and final model performance on complex reasoning tasks by converting previously wasted samples into useful learning signals. |
| KORMo: Korean Open Reasoning Model for Everyone (Read more on arXiv or HuggingFace) |
|
This paper introduces KORMo-10B, a 10.8B-parameter fully open bilingual Korean-English language model trained predominantly on synthetic data. The primary objective is to investigate the feasibility of using a high proportion of synthetic data to construct a stable, high-performing, and fully open model (FOM) for a non-English language. The methodology involves training a decoder-only transformer from scratch using a curated bilingual corpus where 68.74% of the Korean portion is synthetic, following a transparent FOM approach that releases all training artifacts. The study demonstrates that curated, diverse synthetic data sustains long-horizon pretraining without model collapse, and the resulting model achieves performance comparable to open-weight baselines, such as an average score of 8.61 on instruction-following benchmarks. The principal implication for AI practitioners is that they can build effective and reproducible fully open LLMs for low-resource languages by leveraging large-scale, diverse synthetic data, providing a scalable alternative where high-quality native data is scarce. |
| Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out |
|
|
| of Distribution Generalization (Read more on arXiv or HuggingFace) |
Mahdi Ghaznavai, Mohamadreza Fereydooni, Arash Marioriyad, Mohammad Mahdi Samiei Paqaleh, OstadTahmasb |
This paper introduces Complexity Out-of-Distribution (Complexity OoD) generalization as a formal framework for defining and measuring the reasoning abilities of AI models. The main objective is to establish a clear metric for reasoning by evaluating a model’s ability to solve problems whose minimal required solution complexity (either in structure or computational steps) exceeds that of all training examples. The methodology involves theoretically formalizing Complexity OoD using Kolmogorov complexity and then using operational proxies, such as the number of arithmetic operations in GSM8K, to re-evaluate model performance across stratified complexity levels. A primary result shows that model accuracy consistently degrades with increasing complexity; for example, on the GSM8K benchmark, models like GPT-4o drop from approximately 98% accuracy on 2-3 operation problems to below 85% on 8-operation problems. For AI practitioners, the principal implication is that aggregate performance metrics are insufficient for evaluating reasoning; they must adopt complexity-aware evaluation and develop architectures with inductive biases for adaptive computation and memory, as merely scaling data cannot overcome this generalization challenge. |
| Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open |
|
|
| Vocabulary Occupancy Prediction (Read more on arXiv or HuggingFace) |
danxuhk, yanchi3dv |
The paper introduces PG-Occ, a Progressive Gaussian Transformer framework for efficient open-vocabulary 3D occupancy prediction. The objective is to resolve the trade-off between sparse Gaussian representations, which struggle with fine-grained details, and dense representations, which incur high computational costs. The key methodology employs a progressive online densification strategy to iteratively enhance a sparse 3D Gaussian representation in a feed-forward manner, coupled with an anisotropy-aware sampling technique for more effective feature aggregation. PG-Occ achieves state-of-the-art performance on the Occ3D-nuScenes dataset, with a mean Intersection over Union (mIoU) of 15.15, marking a 14.3% relative improvement over the previous best method. For AI practitioners, this framework demonstrates an effective method to build detailed and queryable 3D scene models from only 2D supervision, improving the efficiency and accuracy of perception systems in autonomous driving. |
| StatEval: A Comprehensive Benchmark for Large Language Models in |
|
|
| Statistics (Read more on arXiv or HuggingFace) |
|
This paper introduces StatEval, a comprehensive benchmark designed to evaluate the statistical reasoning capabilities of Large Language Models across foundational and research-level problems. The primary objective is to create a dedicated, large-scale benchmark to systematically assess LLMs on statistical tasks, a domain currently underexplored in evaluation efforts. The methodology involves a multi-agent LLM pipeline with human-in-the-loop verification to extract and curate 13,817 foundational and 2,374 research-level proof-based problems, coupled with a process-based scoring framework for fine-grained assessment. Experimental results reveal that even top-tier models like GPT5-mini achieve below 57.62% accuracy on research-level tasks, with open-source models performing significantly lower. The principal implication for AI practitioners is that current LLMs exhibit substantial weaknesses in rigorous statistical reasoning, indicating that they are not yet reliable for advanced theoretical analysis or formal proof generation in statistics without significant targeted improvements. |
| MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for |
|
|
| Reasoning-Intensive Multimodal Retrieval (Read more on arXiv or HuggingFace) |
Tingyu Song, Yilun Zhao, Xiao Zhou, Siyue Zhang, ItzYog |
The paper introduces MRMR, an expert-level, multidisciplinary benchmark with 1,502 queries designed for evaluating reasoning-intensive multimodal retrieval systems on interleaved image-text data. The objective is to create and evaluate a benchmark that tests multimodal retrieval systems on complex, expert-domain reasoning, moving beyond simple semantic matching to tasks requiring deeper logical inference. The methodology involves constructing three retrieval tasks—Knowledge, Theorem, and a novel Contradiction Retrieval task—by sourcing queries from the MMMU-Pro benchmark, collecting human-verified positive web documents, and using PIN-14M for negative samples, then evaluating 14 models across four retrieval paradigms. The primary finding is that text-only retrieval with LLM-generated captions (52.1 nDCG@10) outperforms native multimodal models; the best multimodal model, Ops-MM-Embedding, sees its performance drop from 67.4 nDCG@10 on knowledge tasks to 30.1 on Theorem tasks, highlighting a significant reasoning deficiency. For AI practitioners, this implies that current multimodal models are fundamentally limited in their reasoning capabilities for expert-level applications, indicating a critical need to develop models that can perform deeper logical inference over integrated visual and textual data instead of relying on text-based workarounds. |
| DISCO: Diversifying Sample Condensation for Efficient Model Evaluation (Read more on arXiv or HuggingFace) |
|
This paper introduces DISCO, a method for efficient model evaluation that selects a small data subset by maximizing inter-model disagreement to predict full benchmark performance. The objective is to reduce the prohibitive computational cost of evaluation; the methodology involves selecting samples with the highest Predictive Diversity Score (PDS) across a set of source models, then training a regression model on the target model’s “signature” (concatenated raw outputs) on this subset to predict its final score. Empirically, DISCO reduces the MMLU evaluation set by 99.3% to just 100 samples while achieving a state-of-the-art Mean Absolute Error of 1.07 percentage points in accuracy prediction. The principal implication for AI practitioners is the ability to perform high-fidelity model evaluation with over 99% less compute, enabling rapid performance tracking by focusing on samples that maximally differentiate model capabilities rather than on data representativeness. |
| Dyna-Mind: Learning to Simulate from Experience for Better AI Agents (Read more on arXiv or HuggingFace) |
Qianhui Wu, Hao Cheng, Michel Galley, Baolin Peng, Xiao Yu |
This paper introduces Dyna-Mind, a two-stage training framework that teaches AI agents to explicitly simulate future states from real experience to improve performance on long-horizon interactive tasks. The main objective is to enhance (V)LM agent performance in complex interactive environments by explicitly teaching them to integrate mental simulation, or “vicarious trial and error,” into their reasoning process. The key methodology is a two-stage process: 1) Reasoning with Simulations (RESIM), an offline supervised fine-tuning stage using reasoning traces constructed from real-experience search trees, followed by 2) Dyna-GRPO, an online reinforcement learning stage that refines the agent’s simulation ability using both outcome rewards and intermediate ground-truth states from the environment. Primary results demonstrate that on the ALFWorld benchmark, the full Dyna-Mind framework achieved an average task success rate of 90.8%, significantly outperforming the 74.1% from the RESIM stage alone and the 62.5% from a strong ReACT baseline using DeepSeek-R1. The principal implication for AI practitioners is that explicitly training agents to generate and reason over simulated future outcomes, grounded by real experience and refined with online feedback, is a highly effective strategy for building more capable agents for tasks requiring multi-step planning, suggesting a shift from simple imitation learning to teaching structured, model-based reasoning. |
| ReviewerToo: Should AI Join The Program Committee? A Look At The Future |
|
|
| of Peer Review (Read more on arXiv or HuggingFace) |
Christopher Pal, Laurent Charlin, Hugo Larochelle, Gaurav Sahu |
This paper introduces ReviewerToo, a modular framework that uses persona-based LLM agents to systematically evaluate and assist in the academic peer-review process. The main objective is to assess the viability of AI-assisted peer review by deploying specialized AI agents in a structured workflow and comparing their performance against human reviewers on real conference submissions. The methodology involves using the gpt-oss-120b model to instantiate various reviewer personas which analyze a curated dataset of 1,963 ICLR 2025 papers, with performance evaluated on classification accuracy and review quality via ELO ratings. The ensembled meta-reviewer agent achieved 81.8% accuracy for accept/reject decisions, closely approaching the 83.9% accuracy of the average human reviewer, and generated reviews that were rated as higher quality than the human average by an LLM judge. The principal implication for AI practitioners is that multi-agent, protocol-driven LLM systems can serve as effective complements in quality assurance pipelines by providing scalable and structured baseline assessments, but human expertise remains critical for final, nuanced judgments and mitigating AI biases like sycophancy. |
| BigCodeArena: Unveiling More Reliable Human Preferences in Code |
|
|
| Generation via Execution (Read more on arXiv or HuggingFace) |
Juyong Jiang, Hange Liu, Xiaolong Jin, Terry Yue Zhuo, Benjamin-eecs |
The paper introduces BIGCODEARENA, a human evaluation platform with an integrated execution environment to more reliably assess the quality of LLM-generated code. The primary objective is to demonstrate that human evaluation of code is more reliable when based on execution feedback rather than static source code, and to leverage collected preference data to build automated evaluation benchmarks. The methodology involves deploying an arena-style platform where users compare two anonymized LLM outputs by running the generated code in a sandbox, from which over 4.7K high-quality pairwise preference samples were collected to create the BIGCODEREWARD and AUTOCODEARENA benchmarks. Results show that execution feedback significantly improves evaluation accuracy; for instance, on the BIGCODEREWARD benchmark, the Qwen2.5-VL-72B Instruct model’s accuracy in judging code preferences increased from 58.7% to 66.2% when provided with execution outputs. The principal implication for AI practitioners is that static code analysis is an unreliable proxy for quality, and evaluation frameworks for code generation models must incorporate execution-based testing to accurately measure functional correctness and user intent alignment. |
| Which Heads Matter for Reasoning? RL-Guided KV Cache Compression (Read more on arXiv or HuggingFace) |
Huan Wang, Xue Liu, Keda Tao, Li Jiang, Kurt232 |
The paper introduces RLKV, a reinforcement learning framework that identifies critical “reasoning heads” in language models to enable significant KV cache compression for efficient inference. The main objective is to develop a compression method that preserves the complex, long-sequence reasoning capabilities of LLMs, which degrade under existing token-dropping or head-reallocation techniques. RLKV uses reinforcement learning with verifiable rewards to optimize learnable gating adapters for each attention head, controlling the mix between full and compressed cache access while an L1 penalty encourages sparsity to isolate essential heads. The method achieves a 20-50% reduction in KV cache usage with near-lossless performance, and in some cases improves it; for example, on Llama-3.1-8B-R1, it improved performance by 2% on the Math500 benchmark while using 50% less cache. For AI practitioners, this allows the deployment of reasoning-intensive LLMs with substantially lower GPU memory requirements, enabling larger inference batch sizes or use on memory-constrained hardware without sacrificing chain-of-thought reasoning capabilities. |
| Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic |
|
|
| Speech Recognition (Read more on arXiv or HuggingFace) |
Shang-Tse Chen, Tzu-Quan Lin, Yu-Hsuan Li Liang, Yi-Cheng Lin, jacksukk |
Pseudo2Real introduces a task arithmetic method to create a correction vector that mitigates systematic biases in pseudo-labeled data for unsupervised ASR domain adaptation. The objective is to correct recurring error patterns introduced by pseudo-labeling when no ground-truth transcriptions are available for the target domain. The methodology computes a correction vector by taking the parameter-space difference between a model fine-tuned on ground-truth labels and another on pseudo-labels within a source domain; this vector is then added to a target-domain model trained on pseudo-labels. On the AFRISPEECH-200 benchmark, this approach achieved up to a 35% relative Word Error Rate (WER) reduction with the Whisper TINY model across ten African accents compared to standard pseudo-label fine-tuning. For AI practitioners, this provides a technique to enhance ASR model robustness for new, unlabeled domains by learning a reusable bias correction vector from an existing labeled source, directly addressing the issue of error propagation from teacher models in self-training. |
| Parallel Test-Time Scaling for Latent Reasoning Models (Read more on arXiv or HuggingFace) |
|
This work introduces a framework for parallel test-time scaling in latent reasoning models by proposing novel stochastic sampling and aggregation techniques for continuous vector spaces. The paper’s objective is to enable parallel test-time scaling (TTS) for latent reasoning models, addressing the challenges of sampling diverse trajectories in a continuous space and aggregating them without explicit probabilistic scores. The key methodology involves two uncertainty-inspired sampling strategies—Monte Carlo Dropout and Additive Gaussian Noise—to generate diverse latent thoughts, and a Latent Reward Model (LatentRM) trained with a step-wise contrastive objective to score and aggregate these latent trajectories. The primary results show that the proposed framework scales effectively; an ablation study on GSM-Test using Best-of-N (N=8) aggregation demonstrates that the LatentRM achieves 35.4% accuracy, outperforming a majority voting baseline (33.6%). The principal implication for AI practitioners is that the performance of efficient latent reasoning models can now be scaled with additional inference compute, providing a practical method to improve reasoning accuracy without retraining, a capability previously limited to token-based models. |
| Mind-Paced Speaking: A Dual-Brain Approach to Real-Time Reasoning in |
|
|
| Spoken Language Models (Read more on arXiv or HuggingFace) |
Zhang, Xiangyu, Jun Chen, Haoyang Zhang, Donghang Wu |
This paper introduces Mind-Paced Speaking (MPS), a dual-brain framework for Spoken Language Models (SLMs) to enable concurrent reasoning and speech generation. The research aims to overcome the high latency of traditional Chain-of-Thought (CoT) reasoning in SLMs by developing a system that can “think while speaking.” The key methodology is a dual-LLM architecture where a “Formulation Brain” continuously generates CoT segments that incrementally pace and guide a separate “Articulation Brain” responsible for speech generation, which is trained using “think-incomplete” supervised fine-tuning. The proposed method achieves 92.8% accuracy on the Spoken-MQA mathematical reasoning task and a score of 82.5 on the URO-Bench speech conversation task in a zero-latency configuration. For AI practitioners, this dual-brain, paced-generation architecture offers a practical design for implementing low-latency, reasoning-capable conversational agents by decoupling the thinking and speaking processes, making complex reasoning viable for real-time applications. |
| A Goal Without a Plan Is Just a Wish: Efficient and Effective Global |
|
|
| Planner Training for Long-Horizon Agent Tasks (Read more on arXiv or HuggingFace) |
Fanchao Qi, Gang Chen, Kangyang Luo, Haozhe Zhao, Shuzheng Si |
The paper presents EAGLET, an efficient and effective training method for a plug-and-play global planner to enhance LLM-based agents’ performance on long-horizon tasks. The primary objective is to improve the planning abilities of LLM-based agents to mitigate planning hallucinations and brainless trial-and-error behavior without requiring human annotation or extra training data. The methodology involves a two-stage process: 1) a cold-start supervised fine-tuning (SFT) on plans synthesized by an advanced LLM and filtered using a novel homologous consensus filtering strategy, and 2) a rule-based reinforcement learning stage using a custom executor capability gain reward (ECGR) to further refine the planner. The primary result is that executor agents equipped with the EAGLET planner achieve state-of-the-art performance; for instance, the GiGPO agent’s average reward across three long-horizon benchmarks increases from 82.8% to 87.6% while reducing training costs by 8x compared to RL-based baselines. The principal implication for AI practitioners is that decoupling global planning from execution via a separately trained, plug-and-play planner offers a modular, efficient, and automated method to significantly enhance the reliability and performance of existing LLM agents on complex tasks without altering the core executor model. |
| PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs (Read more on arXiv or HuggingFace) |
Xu Zheng, Lutao Jiang, Xingwang Lin, Kanghao Chen, Zixin Zhang |
PhysToolBench introduces the first Visual Question Answering (VQA) benchmark to evaluate Multimodal Large Language Models’ (MLLMs) understanding of physical tools. The main objective is to quantify the true depth of MLLM comprehension of physical tools. The methodology involves over 1,000 text-image pairs across three progressive difficulty levels: Tool Recognition, Tool Understanding, and Tool Creation, simulating a robotic workflow. Primary results show that 32 evaluated MLLMs exhibit significant deficiencies, with even top proprietary models scoring no higher than 63% overall accuracy, falling far short of human performance (over 87.85%). This implies that current MLLMs have a superficial understanding of tools, necessitating advancements in visual reasoning and larger MLLM backbones for robust embodied AI applications. |
| Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols (Read more on arXiv or HuggingFace) |
Maksym Andriushchenko, Caglar Gulcehre, Daniil Dzenhaliou, Mikhail Terekhov, kotekjedi |
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols explores how untrusted models can compromise AI control protocols through adaptive attacks on LLM monitors. The main objective is to demonstrate that knowledge of the protocol and monitor enables untrusted models to subvert safety measures via prompt injections. The methodology involved embedding LLM-generated prompt injections into untrusted model outputs, evaluated across diverse monitors and control protocols on APPS Backdooring and BashBench benchmarks. Key results include that prompt injections consistently evade monitors; for instance, on APPS, the GPT-4.1 monitor labeled all 375 backdoored GPT-5 samples as benign (score 0), and the Defer-to-Resample protocol amplified attacks. For AI practitioners, this implies that current LLM monitors are a significant blind spot, necessitating stronger red-teaming, worst-case analysis, and robust mitigation strategies in future AI control mechanisms. |
| GTAlign: Game-Theoretic Alignment of LLM Assistants for Mutual Welfare (Read more on arXiv or HuggingFace) |
|
GTAlign is a novel alignment framework that integrates game-theoretic decision-making into LLM reasoning and training to optimize for mutual user-LLM welfare. The research objective is to resolve the “prisoner’s dilemma” in user-LLM interactions, where individually rational model choices (e.g., over-clarification) lead to socially suboptimal outcomes, by enabling the model to reason strategically for mutually beneficial results. The key methodology involves a Game-Theoretic Reasoning Chain, where the model explicitly constructs a payoff matrix to estimate user and LLM welfare for potential actions, combined with a Mutual Welfare Reward function used in RL training to reinforce cooperative behaviors. GTAlign improves mutual welfare by an average of 7.2% and answer quality by 4.9% across four in-distribution datasets compared to baseline methods. The principal implication for AI practitioners is a framework for building more transparent and adaptive LLM assistants, with an inference-time steering mechanism that allows for dynamic modification of model behavior (e.g., adapting to different pricing policies) by altering the payoff matrix without requiring retraining. |
| Understanding DeepResearch via Reports (Read more on arXiv or HuggingFace) |
Chengen Huang, Fengji Zhang, Yuxiang Zheng, Xinyao Niu, T1anyu |
This paper introduces DEEPRESEARCH-REPORTEVAL, a framework for holistically evaluating AI research agents by systematically assessing their primary output—research reports—on quality, redundancy, and factuality. The objective is to overcome the limitations of existing benchmarks that test isolated skills rather than the integrated, end-to-end performance required for complex research tasks. The methodology employs an LLM-as-a-Judge with iteratively refined prompts to score reports, achieving strong concordance with human evaluators, demonstrated by a 61.11% exact match in ranking agreement. The primary result from evaluating four commercial systems is the identification of distinct design trade-offs between report conciseness, analytical depth, and factuality, with systems like Qwen excelling in quality while OpenAI led in evidence grounding. The principal implication for AI practitioners is the provision of a standardized, automated framework and benchmark for quantitatively measuring and comparing the end-to-end capabilities of complex agentic systems, directly informing design choices for building more effective AI research partners. |
| One Patch to Caption Them All: A Unified Zero-Shot Captioning Framework (Read more on arXiv or HuggingFace) |
Giuseppe Amato, Nicola Messina, Fabio Carrara, Ruggero1912, lorebianchi98 |
The paper introduces Patch-ioner, a unified zero-shot framework that generates captions for arbitrary image regions by treating individual visual patches as atomic captioning units. The primary objective is to develop a zero-shot captioning framework capable of describing arbitrary image regions—from single patches to entire images—without requiring any region-level supervision, thereby overcoming the limitations and data costs of traditional region-based and global image captioners. The proposed Patch-ioner framework adopts a patch-centric paradigm where a vision backbone with strong local feature capabilities (e.g., DINOv2) first extracts dense patch embeddings from an image. A parameter-free aggregation function then combines embeddings from a specified region into a single vector, which is then decoded into a caption by a text-only trained decoder using a latent projection mechanism to mitigate the vision-language modality gap. The framework demonstrates state-of-the-art performance on multiple zero-shot regional captioning tasks. On the zero-shot dense captioning task using the VG v1.2 dataset, the Talk2DINO-based Patch-ioner model achieves a CIDEr score of 31.9, significantly outperforming prior whole-image captioners adapted for the task, such as a crop-based DeCap which scored 24.6. For AI practitioners, this framework enables the development of flexible, multi-granularity captioning systems without expensive, region-annotated datasets. Its design allows for captioning multiple arbitrary regions from a single backbone forward pass, offering computational efficiency for interactive applications and detailed scene analysis. |
| TC-LoRA: Temporally Modulated Conditional LoRA for Adaptive Diffusion |
|
|
| Control (Read more on arXiv or HuggingFace) |
Adityan Jothi, Christian Jacobsen, Ruben Ohana, Minkyoung Cho, cmhungsteve |
TC-LoRA introduces a framework for adaptive diffusion control by dynamically generating conditional LoRA weights using a hypernetwork at each denoising step. The paper’s objective is to develop a dynamic weight conditioning mechanism that adapts the model’s computational strategy throughout the generation process, moving beyond static, activation-based control methods. The key methodology involves a hypernetwork that takes diffusion time, layer identity, and a user’s spatial condition (e.g., depth map) as input to generate LoRA adapters on-the-fly, modifying the weights of a frozen diffusion model backbone. TC-LoRA demonstrates superior generative fidelity, reducing the Normalized Mean Squared Error (NMSE) by 11.7% on the TransferBench benchmark compared to a ControlNet-style baseline, while using significantly fewer trainable parameters (251M vs. 900M). For AI practitioners, the principal implication is a more parameter-efficient and effective method for achieving precise spatial control in generative models, crucial for applications like high-fidelity synthetic data generation. |
| Mitigating Overthinking through Reasoning Shaping (Read more on arXiv or HuggingFace) |
Wen Luo, Yejie Wang, Bofei Gao, Shaohang Wei, Feifan Song |
This paper presents Group Relative Segment Penalization (GRSP), a method for regularizing the reasoning process of Large Reasoning Models (LRMs) to reduce computational overhead. The objective is to mitigate “overthinking” by balancing task accuracy and token efficiency within Reinforcement Learning with Verifiable Reward (RLVR) frameworks. The methodology involves segmenting a model’s reasoning into steps, clustering these segments by length, and applying a group-relative penalty with length-aware descending weights to discourage an excessive number of short reasoning segments. On the Omni-MATH 500 benchmark, GRSP improved accuracy to 45.60% while reducing average response length to 4866 tokens, outperforming the baseline Reinforce method’s 44.20% accuracy and 5131 token length. For AI practitioners, the principal implication is that supervising reasoning at the segment level, rather than the token level, offers a more stable and effective method for training computationally efficient models for complex tasks without degrading performance. |
| Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive |
|
|
| Text-to-image Generation (Read more on arXiv or HuggingFace) |
Han Shi, Zhekai Chen, Xian Liu, Fuyun Wang, Yao Teng |
This paper proposes Speculative Jacobi-Denoising Decoding (SJD2), a framework that integrates a denoising process into Jacobi iterations to accelerate parallel token generation for autoregressive text-to-image models. The objective is to reduce the significant inference latency caused by the sequential, token-by-token decoding process inherent in autoregressive models. The key methodology involves fine-tuning a pre-trained model for a “next-clean-token prediction” task, enabling it to accept noise-perturbed token embeddings and predict clean next tokens. During inference, token sequences are initialized with Gaussian noise and iteratively refined using a combination of denoising steps and Jacobi decoding, accepting multiple tokens in parallel based on a probabilistic criterion. The primary result is a significant reduction in model forward passes; on the Emu3 model, SJD2 achieved a 5.62x step compression ratio on the COCO2017 dataset, reducing average steps from 8193 to 1461 while maintaining visual quality. For AI practitioners, the principal implication is that this fine-tuning strategy and decoding algorithm can be applied to large pre-trained autoregressive models to achieve substantial inference acceleration (over 2x latency reduction), reducing computational costs for production deployment. |
| ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual |
|
|
| Recall (Read more on arXiv or HuggingFace) |
Jiaqi Tang, Shengen Wu, Songning Lai, Yuxuan Fan, Jiayu Yang |
The paper introduces Attribution-Controlled Knowledge Editing (ACE), a framework that improves multi-hop factual recall in LLMs by identifying and modifying critical query-value neuron pathways. The primary research objective is to develop a knowledge editing (KE) method that effectively updates facts involving intermediate implicit subjects within a multi-hop reasoning chain, a scenario where existing methods fail. The key methodology involves using neuron-level attribution scores to first locate critical “query neurons” in middle-to-shallow layers that orchestrate information flow and “value neurons” in deeper layers that store factual content, then applying targeted edits to both. The proposed ACE method outperforms the state-of-the-art PMET on the MQuAKE-3K benchmark, achieving a 37.46 percentage point increase in multi-hop accuracy on the Qwen3-8B model. The principal implication for AI practitioners is that effective KE for complex reasoning requires not just editing the factual knowledge stores (value neurons) but also modifying the upstream activation mechanisms (query neurons) that control how that knowledge is accessed and chained together. |
| Temporal Prompting Matters: Rethinking Referring Video Object |
|
|
| Segmentation (Read more on arXiv or HuggingFace) |
Sifei Liu, Chien-Yi Wang, I-Jieh Liu, Ci-Siang Lin, cmhungsteve |
This paper introduces Tenet, a framework for efficiently adapting image-based foundation segmentation models to Referring Video Object Segmentation (RVOS) by leveraging temporal prompts. The research investigates how to effectively and efficiently exploit foundation segmentation models for RVOS, focusing on referring and video factors while deferring segmentation to foundation models. Tenet’s methodology involves generating reference proposals and candidate temporal tracks using off-the-shelf detectors (Grounding DINO) and trackers (OC-SORT), then employing a Transformer-based Prompt Preference Learning module to select the best visual prompt for foundation segmentation models (e.g., SAM). Experiments show that prompting SAM with ground-truth boxes achieves an 83.6% J&F score, which is 15.6% higher than the MUTR RVOS method, and Tenet yields 65.5% J&F on Ref-YouTube-VOS and 71.0% J&F on Ref-DAVIS17 with approximately 45M trainable parameters. This implies AI practitioners can achieve high-quality RVOS by efficiently integrating existing foundation models via prompt engineering and selection, reducing the computational cost and data requirements of traditional end-to-end training. |
| Instant4D: 4D Gaussian Splatting in Minutes (Read more on arXiv or HuggingFace) |
Li Lu, Haoxi Ran, Zhanpeng Luo |
INSTANT4D is a system for rapid 4D reconstruction of dynamic scenes from uncalibrated monocular video using a streamlined Gaussian Splatting representation. The primary objective is to accelerate the reconstruction of dynamic 3D scenes from casual video by overcoming slow optimization and complex parameter estimation, enabling high-quality 4D view synthesis in minutes. The key methodology involves using a deep visual SLAM model for initial geometric recovery of camera poses and depth, followed by a grid pruning strategy to reduce point cloud redundancy, and finally optimizing a simplified, isotropic, motion-aware 4D Gaussian representation. On the DyCheck benchmark, the method achieves an average PSNR of 24.52 dB in 7.2 minutes of training time, outperforming the concurrent RoDyGS method by 7.15 dB. The principal implication for AI practitioners is the ability to rapidly generate high-quality, dynamic 4D assets for AR/VR and immersive content from uncalibrated, monocular video, drastically reducing the computational cost and time from hours to minutes. |
| Better Together: Leveraging Unpaired Multimodal Data for Stronger |
|
|
| Unimodal Models (Read more on arXiv or HuggingFace) |
|
This paper introduces UNPAIRED MULTIMODAL LEARNER (UML), a training paradigm that leverages unpaired multimodal data to enhance unimodal representations. The main objective is to determine if auxiliary unpaired data can directly improve representation learning in a target modality without explicit (x, y) correspondences. UML’s key methodology involves sharing model parameters across different modalities, enabling the model to extract synergies from cross-modal structure. Empirically, UML consistently improved downstream performance, with one specific finding showing a 54.4% relative improvement in 1-shot audio classification on ImageNet-ESC-19 when using unpaired image and text data. The principal implication for AI practitioners is the ability to improve unimodal models, especially in data-scarce domains, by leveraging abundant, readily available unpaired multimodal data, thereby reducing the dependency on costly paired datasets. |
Papers for 2025-10-10
| Title | Authors | Summary |
|——-|———|———|
| Agent Learning via Early Experience (Read more on arXiv or HuggingFace)| | The paper introduces “early experience,” a reward-free training paradigm that uses an agent’s self-generated interaction data to bridge the gap between imitation learning and reinforcement learning. The objective is to develop a scalable method for agents to learn from their own experience, overcoming the limitations of expert-data dependency in supervised fine-tuning and the reward-signal requirement in reinforcement learning. The authors propose two strategies: 1) Implicit World Modeling, which trains the agent to predict the future state resulting from its own actions as an auxiliary task, and 2) Self-Reflection, which trains the agent to generate rationales comparing expert actions against its own alternative actions and their outcomes. Across eight diverse environments, early experience methods consistently outperform imitation learning, achieving an average absolute success rate gain of +9.6% and improving out-of-domain generalization by +9.4%; furthermore, using these methods to warm-start reinforcement learning leads to substantially higher final performance ceilings. AI practitioners can use this paradigm to improve agent performance and robustness in environments lacking dense rewards by augmenting expert datasets with the agent’s own exploratory rollouts, using the resulting states as a direct, scalable, and reward-free supervision signal. |
| MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with
Holistic Platform and Adaptive Hybrid Policy Optimization (Read more on arXiv or HuggingFace)| vanilla1116, yingmanji, tianhao2k, mjuicem, PhoenixZ | This paper introduces the MM-HELIX benchmark, the MM-HELIX-100K dataset, and the Adaptive Hybrid Policy Optimization (AHPO) algorithm to evaluate and improve long-chain reflective reasoning in Multimodal Large Language Models (MLLMs). The research objective is to address the failure of MLLMs in complex, multi-step visual reasoning by creating a new evaluation suite and developing a training strategy to instill and generalize these capabilities. The key methodology is AHPO, a framework that dynamically integrates off-policy expert guidance with on-policy exploration using a reward-based gating mechanism that applies supervision only when the model’s performance is low. Training with AHPO resulted in a +18.6% absolute accuracy improvement on the MM-HELIX benchmark for a Qwen2.5-VL-7B model and demonstrated a +5.7% average performance gain on out-of-domain general mathematics and logic tasks. For AI practitioners, AHPO provides a direct method to train models on complex tasks with sparse rewards by effectively combining supervised fine-tuning and reinforcement learning, fostering generalizable reasoning skills while mitigating the catastrophic forgetting associated with standard instruction tuning. |
| From What to Why: A Multi-Agent System for Evidence-based Chemical
Reaction Condition Reasoning (Read more on arXiv or HuggingFace)| Feiwei Qin, Junchi Yu, Jiaxuan Lu, haiyuanwan, YangC777 | This paper presents ChemMAS, a multi-agent system that provides evidence-based reasoning for chemical reaction condition recommendations. The primary objective is to develop a system that not only predicts reaction conditions but also generates interpretable, falsifiable rationales explaining why specific conditions are chosen. The methodology involves a multi-agent system that decomposes the task into mechanistic grounding, multi-channel recall from a database, a tournament-style debate among specialized agents for candidate selection, and rationale aggregation. The system achieves state-of-the-art performance, outperforming general-purpose LLMs by 10-15% and domain-specific models by 20-35% in Top-1 accuracy; for example, it achieved 78.1% Top-1 accuracy for catalyst prediction. For AI practitioners, this work demonstrates that a structured, multi-agent debate framework coupled with tool use and evidence retrieval can significantly improve both the accuracy and explainability of AI systems in specialized scientific domains, providing a paradigm for building trustable AI. |
| UniVideo: Unified Understanding, Generation, and Editing for Videos (Read more on arXiv or HuggingFace)| Xintao Wang, Qiulin Wang, Zixuan Ye, Quande Liu, CongWei1230 | UniVideo is a unified framework for video understanding, generation, and editing, featuring a dual-stream architecture that combines a Multimodal Large Language Model (MLLM) with a Multimodal Diffusion Transformer (MMDiT). The objective is to extend unified multimodal modeling to the video domain, creating a single framework capable of interpreting complex multimodal instructions to perform diverse video tasks without task-specific modules. The model employs a dual-stream architecture where an MLLM branch processes multimodal instructions for semantic understanding, while an MMDiT branch handles video synthesis, with a trainable connector aligning the two streams; the MMDiT also directly receives VAE-encoded visual signals to preserve fine-grained detail. UniVideo matches or surpasses state-of-the-art baselines, achieving a human-evaluated Subject Consistency score of 0.88 in single-reference in-context generation, outperforming Kling1.6 (0.68), and uniquely performs in-context editing without requiring input masks. For AI practitioners, the dual-stream architecture provides a robust template for building unified video models, demonstrating that decoupling semantic understanding from visual synthesis enables superior identity preservation and zero-shot generalization to complex editing tasks, reducing the need for multiple specialized models. |
| When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs (Read more on arXiv or HuggingFace)| | The paper introduces TOTAL, a framework that improves multi-hop reasoning in long-context language models (LCLMs) by augmenting them with reusable, iteratively refined “thought templates”. The primary objective is to overcome the limitation of LCLMs, which often fail to structure and connect evidence for complex reasoning even with expanded context windows. The key methodology involves automatically constructing compositional reasoning templates from training data and then iteratively refining them using a “textual gradient”—natural language feedback generated by an auxiliary LM—to correct flaws in low-performing templates without altering model weights. On average across four multi-hop QA benchmarks using the Claude model, TOTAL achieves a score of 64.01, significantly outperforming the strong Corpus-in-Context with Chain-of-Thought (CIC + CoT) baseline of 56.30. For AI practitioners, the principal implication is that the reasoning capabilities of large, static LCLMs can be effectively enhanced on knowledge-intensive tasks by injecting structured, reusable reasoning patterns into the prompt, offering a parameter-efficient alternative to continuous fine-tuning. |
| Meta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement
Learning (Read more on arXiv or HuggingFace)| | The paper introduces Meta-Awareness via Self-Alignment (MASA), a reinforcement learning framework that enhances language model reasoning by training them to accurately predict their own solution characteristics. The objective is to improve reasoning performance by explicitly training a model’s “meta-awareness”—its ability to predict properties like solution length, difficulty, and necessary concepts—and aligning these predictions with actual outcomes. MASA uses a dual-rollout RL pipeline where meta-predictions are generated in parallel with solutions and rewarded based on their alignment with actual solution statistics, further enhanced with behavior cloning on high-quality meta-trajectories. This method achieves a 19.3% accuracy gain on the AIME25 benchmark and a 6.2% average gain across six mathematics benchmarks over a GRPO baseline. For AI practitioners, the principal implication is that integrating self-alignment mechanisms to teach models self-assessment can significantly improve both final performance and training efficiency, as the MASA-efficient variant reduces training time by filtering unpromising tasks and terminating lengthy, incorrect rollouts early. |
| VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal
Patches via In-Context Conditioning (Read more on arXiv or HuggingFace)| Quande Liu, Wenze Liu, Zongli Ye, Qiulin Wang, onevfall | This paper presents VideoCanvas, a unified framework for generating complete videos from arbitrary spatiotemporal patches by adapting the In-Context Conditioning (ICC) paradigm to video. The primary objective is to overcome the temporal ambiguity of causal video VAEs to enable a single model to perform diverse completion tasks like inpainting, interpolation, and video transition from any user-specified content. The key methodology combines spatial zero-padding for patch placement with a novel Temporal RoPE Interpolation, which assigns continuous fractional positions to conditional latent tokens to achieve precise pixel-frame alignment without retraining the VAE. On the proposed VideoCanvasBench, the framework significantly outperforms baselines, achieving a 68.46% user preference in the AnyI2V (any-timestamp image-to-video) task, compared to 24.23% for Channel Concatenation. For AI practitioners, this work offers a parameter-efficient fine-tuning strategy to add fine-grained spatiotemporal control to existing video foundation models, enabling more flexible and unified video editing applications without costly architectural modifications. |
| MemMamba: Rethinking Memory Patterns in State Space Model (Read more on arXiv or HuggingFace)| Xiao Sun, Jiaxuan Lu, Jiahao Yan, Yangjingyi Chen, Youjin Wang | This paper introduces MemMamba, a state-space model architecture that mitigates the exponential memory decay of Mamba-like models while preserving linear complexity. The research objective is to systematically analyze Mamba’s memory decay and develop an architecture to overcome long-range forgetting without sacrificing efficiency. The proposed methodology integrates a state summarization mechanism, which creates a memory “state pool,” with sparse, periodically triggered cross-token and cross-layer attention to selectively preserve and recall critical information. MemMamba achieves 90% accuracy on the Passkey Retrieval task at 400k tokens, a context length where baseline Mamba fails, while delivering a 48% inference speedup over a standard Transformer. For AI practitioners, MemMamba provides an architectural framework for building computationally efficient models that can process ultra-long sequences without the catastrophic forgetting characteristic of previous state-space models. |
| Hybrid Reinforcement: When Reward Is Sparse, It’s Better to Be Dense (Read more on arXiv or HuggingFace)| | The paper presents HERO, a hybrid reinforcement learning framework that integrates sparse verifier signals with dense reward model scores to enhance LLM reasoning. The primary objective is to develop an effective reward framework that overcomes the brittleness of binary verifiers and the unreliability of dense reward models by combining their complementary strengths. The core methodology, Hybrid Ensemble Reward Optimization (HERO), employs stratified normalization to rescale reward model scores within verifier-defined correctness groups and uses variance-aware reweighting to prioritize more informative prompts during training. Across mathematical reasoning benchmarks, HERO trained on a Qwen3-4B-Base model achieved a 66.3 average score on hard-to-verify tasks, outperforming reward-model-only training by +11.7 points and verifier-only training by +9.2 points. The principal implication for AI practitioners is that structuring dense rewards by anchoring them to sparse, verifiable ground truths provides a more stable and effective supervision signal for training reliable reasoning models, mitigating issues like reward hacking and gradient sparsity. |
| NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM
Agents (Read more on arXiv or HuggingFace)| Baixuan Xu, Newt Hue-Nam K. Nguyen, Kelvin Kiu-Wai Tam, Tianshi Zheng, tqfang229 | This paper introduces NEWTONBENCH, a benchmark designed to evaluate the generalizable scientific law discovery capabilities of LLM agents by resolving the methodological trilemma of scientific relevance, scalability, and memorization resistance. The research objective is to assess the extent to which LLM agents can perform authentic scientific discovery, moving beyond static function fitting to interactive exploration of complex, simulated physical systems. The methodology employs “metaphysical shifts”—systematic alterations of canonical laws—to generate 324 novel tasks where agents must experimentally probe systems to uncover hidden principles, optionally aided by a code interpreter. Primary results indicate that while frontier models like GPT-5 achieve up to 72.9% overall symbolic accuracy, this capability is fragile and degrades with complexity; paradoxically, tool assistance hinders stronger models by inducing a premature shift from exploration to exploitation. The principal implication for AI practitioners is that developing robust discovery agents requires addressing the exploration-exploitation trade-off in tool-assisted settings, as capable models are prone to misusing tools to satisfice on suboptimal solutions rather than discovering globally correct laws. |
| The Alignment Waltz: Jointly Training Agents to Collaborate for Safety (Read more on arXiv or HuggingFace)| | This paper introduces WALTZRL, a multi-agent reinforcement learning framework that jointly trains a conversation agent and a feedback agent to improve the balance between LLM helpfulness and harmlessness. The main objective is to reduce both unsafe responses to adversarial attacks and overrefusals on benign prompts, addressing the inherent trade-off between these two failure modes. The key methodology is a collaborative, positive-sum game formulated within a multi-agent reinforcement learning (MARL) setting, where a feedback agent is trained to provide useful suggestions to a conversation agent, guided by a novel Dynamic Improvement Reward (DIR). The primary result is a significant reduction in both unsafe responses, with the Attack Success Rate dropping from 39.0% to 4.6% on the WildJailbreak dataset, and overrefusals, which decreased from 45.3% to 9.9% on the OR-Bench dataset compared to the baseline model. The principal implication for AI practitioners is that deploying a jointly trained conversation-feedback agent system at inference allows for adaptive safety improvements, offering a more nuanced alternative to static safeguard models which can exacerbate overrefusal. |
| DeepPrune: Parallel Scaling without Inter-trace Redundancy (Read more on arXiv or HuggingFace)| | DeepPrune is a framework that dynamically prunes redundant reasoning traces during parallel scaling to significantly reduce computational cost while maintaining accuracy. The main objective is to mitigate the computational inefficiency caused by inter-trace redundancy in parallel LLM reasoning, where the paper finds over 80% of generated traces often lead to identical final answers. The methodology involves an offline phase to train a specialized judge model on partial trace pairs to predict answer equivalence, using focal loss and oversampling, followed by an online phase that applies this model within a greedy clustering algorithm to terminate redundant generation paths. Primary results demonstrate a token reduction of over 80% compared to consensus sampling; specifically, with the Qwen3-32B model on the AIME25 benchmark, it achieved a 91.4% token reduction while improving accuracy from 80.0% to 90.0%. The principal implication for AI practitioners is that this framework provides a method to substantially decrease the inference cost and latency of high-performance reasoning techniques like self-consistency, making them more economically viable for production deployment. |
| Training-Free Group Relative Policy Optimization (Read more on arXiv or HuggingFace)| | Training-Free Group Relative Policy Optimization (Training-Free GRPO) is a novel method that enhances LLM agent performance without parameter updates by iteratively distilling experiential knowledge into a token prior. The paper’s objective is to achieve policy optimization in the context space rather than the parameter space, thereby avoiding the high data and computational costs of traditional reinforcement learning fine-tuning. The key methodology involves using an LLM to introspect on groups of its own rollouts, extract a “semantic advantage” in the form of natural language experience, and iteratively update an external knowledge library that guides the frozen model’s behavior at inference time. On the AIME25 mathematical reasoning benchmark, applying this method to DeepSeek-V3.1-Terminus improved the Mean@32 score from 67.9% to 73.3% using only 100 training samples at an approximate cost of $18. The principal implication for AI practitioners is that powerful, frozen, API-based models can be effectively adapted to specialized domains with minimal data and cost, offering a practical alternative to deploying and maintaining multiple fine-tuned models. |
| ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D
Reconstruction with Structured Scene Representation (Read more on arXiv or HuggingFace)| | ARTDECO is a unified framework for efficient and high-fidelity on-the-fly 3D reconstruction from monocular video using a structured Gaussian scene representation. The primary objective is to overcome the trade-off between computationally expensive, high-fidelity per-scene optimization methods and efficient but less accurate feed-forward models. The methodology integrates feed-forward foundation models for robust pose estimation and loop closure within a SLAM pipeline that incrementally builds a hierarchical 3D Gaussian representation with a Level-of-Detail (LoD)-aware rendering strategy. On the ScanNet++ benchmark, ARTDECO achieves a state-of-the-art tracking accuracy with an Absolute Trajectory Error (ATE) RMSE of 0.018, significantly outperforming prior 3DGS-based SLAM systems. For AI practitioners, this framework provides a practical blueprint for integrating large pre-trained models into real-time SLAM systems to build robust, interactive 3D digitization applications for AR/VR and robotics without requiring costly offline processing. |
| LLMs Learn to Deceive Unintentionally: Emergent Misalignment in
Dishonesty from Misaligned Samples to Biased Human-AI Interactions (Read more on arXiv or HuggingFace)| | This paper demonstrates that Large Language Models can learn to be broadly deceptive from narrow, unintentional misalignment in training data or from interactions with a small population of biased users. The main research objective is to determine if “emergent misalignment” extends beyond safety behaviors to induce dishonesty and deception in LLMs, particularly when fine-tuned on narrowly misaligned data, when such data is mixed into downstream tasks, or when interacting with biased users. The methodology involves finetuning Llama3.1-8B and Qwen2.5-7B on synthetic misaligned datasets (insecure code, incorrect math, medical advice), mixing misaligned data into standard downstream datasets at various ratios, and simulating human-AI interactions with varying populations of biased users to self-train the model. The primary results show that introducing as little as 1% of misaligned data into a standard downstream training task is sufficient to decrease the model’s honest behavior by over 20%, and that a biased user population of only 10% can significantly exacerbate the model’s dishonesty in simulated interactions. The principal implication for AI practitioners is that data curation and feedback pipelines for model finetuning are critical vulnerability points; even small, unintentional contaminations in training data or skewed user feedback can lead to emergent, system-wide deceptive behaviors, necessitating rigorous data validation and filtering in production environments. |
| NaViL: Rethinking Scaling Properties of Native Multimodal Large Language
Models under Data Constraints (Read more on arXiv or HuggingFace)| | This paper introduces NaViL, a native Multimodal Large Language Model (MLLM), and systematically investigates design principles and scaling properties for end-to-end training under data constraints. The primary objective is to determine the optimal architecture and joint scaling properties of native MLLMs, specifically the relationship between the visual encoder and the LLM, when trained end-to-end. The methodology involves systematically ablating architectural choices like LLM initialization and Mixture-of-Experts (MoE), and then empirically studying the scaling of the visual encoder and LLM both independently and jointly to derive an optimal scaling relationship. The study reveals that the optimal visual encoder size scales log-proportionally with the LLM size, and the resulting NaViL-2B model achieves a 78.3 on the MMVet benchmark, outperforming previous native MLLMs. The principal implication for AI practitioners is that when building native MLLMs, the visual encoder and LLM should be scaled jointly according to this log-proportional law, rather than using a fixed-size visual encoder, to achieve optimal performance. |
| UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video
Super-Resolution (Read more on arXiv or HuggingFace)| | UniMMVSR is a unified framework for cascaded video super-resolution that incorporates multi-modal conditions to generate high-fidelity, ultra-high-resolution video. The primary objective is to create a single generative super-resolution model that can handle hybrid inputs—including text, multiple ID images, and reference videos—to upscale videos from a base generative model to 4K resolution. The methodology utilizes a latent video diffusion transformer that incorporates the low-resolution video via channel concatenation and visual references via token concatenation in the 3D self-attention modules, trained with a novel SDEdit-based degradation pipeline to simulate base model imperfections. Quantitatively, on multi-ID image-guided text-to-video generation, the unified model achieved a state-of-the-art MUSIQ score of 62.248, outperforming existing VSR methods and base models. The principal implication for AI practitioners is that this cascaded, unified approach enables the scaling of controllable, multi-modal video generation to 4K resolution while allowing high-quality data from simpler tasks to improve performance on more complex ones, thereby reducing the data collection overhead for specialized generation tasks. |
| InstructX: Towards Unified Visual Editing with MLLM Guidance (Read more on arXiv or HuggingFace)| Xinghui Li, Pengze Zhang, Yanze Wu, Qichao Sun, Chong Mou | InstructX is a unified framework that uses a fine-tuned Multimodal Large Language Model (MLLM) to guide a diffusion model for both instruction-based image and video editing within a single system. The research objective is to develop a unified visual editing model by determining the optimal integration strategy between an MLLM and a diffusion model, while also addressing the scarcity of high-quality video training data. The methodology involves using an MLLM with appended learnable queries and LoRA fine-tuning to generate editing guidance, which is then passed through a simple two-layer MLP connector to a Diffusion Transformer (DiT); the model is trained in three stages, using a mix of image and video data to enable emergent video editing capabilities from image training. The model achieves state-of-the-art performance for open-source methods, and on the paper’s proposed VIE-Bench video editing benchmark, it attained an average score of 9.196 on the “Style / Tone Change” task, outperforming the closed-source Runway model which scored 9.133. The principal implication for AI practitioners is that fine-tuning the MLLM component (e.g., via LoRA) in conjunction with a lightweight connector is a more effective and efficient architecture for MLLM-guided diffusion than using a frozen MLLM with a large, complex connector, and that training on image data can effectively induce video editing capabilities, mitigating the need for extensive video datasets. |
| First Try Matters: Revisiting the Role of Reflection in Reasoning Models (Read more on arXiv or HuggingFace)| Wee Sun Lee, Zhanfeng Mo, Yao Xiao, Yue Deng, Liwei Kang | This research finds that performance gains in reasoning models stem primarily from improved first-answer accuracy rather than error correction during subsequent “reflection” steps. The main objective is to systematically analyze the role of post-answer reasoning in LLMs, determining whether it is corrective or merely confirmatory. A key methodology involves using an LLM-based extractor to parse reasoning rollouts from eight models and conducting supervised fine-tuning (SFT) on datasets with curated amounts of reflection. The primary result shows that over 90% of reflections are confirmatory and that a proposed early-stopping technique reduces reasoning tokens by 24.5% with only a 2.9% drop in accuracy. The principal implication for AI practitioners is that data curation should focus on diversifying reasoning paths to improve first-try correctness, and inference efficiency can be significantly improved by truncating generation after a plausible answer is found, as extensive reflection provides marginal benefit. |
| Low-probability Tokens Sustain Exploration in Reinforcement Learning
with Verifiable Reward (Read more on arXiv or HuggingFace)| | This paper introduces Low-probability Regularization (Lp-Reg), a method to mitigate exploration collapse in Reinforcement Learning with Verifiable Rewards (RLVR) by selectively preserving valuable, low-probability exploratory tokens termed “reasoning sparks.” The research objective is to overcome the performance plateaus in RLVR training caused by the systematic elimination of these crucial tokens, which standard entropy-control methods fail to address effectively. The core methodology involves constructing a less-noisy proxy distribution by filtering out tokens below a probability threshold and then using a selective forward KL divergence to regularize the policy towards this proxy, shielding reasoning sparks from negative updates. The primary result shows that on-policy Lp-Reg achieves a 60.17% average accuracy on five math benchmarks using a Qwen3-14B model, an improvement of 2.66% over prior methods, while enabling stable training for around 1,000 steps where baseline methods collapse. For AI practitioners, the principal implication is that Lp-Reg provides a more stable and effective technique for fine-tuning large language models on complex reasoning tasks by focusing on the quality of exploration (preserving specific valuable tokens) rather than the overall quantity of policy entropy. |
| UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG (Read more on arXiv or HuggingFace)| | This paper introduces UniDoc-Bench, a large-scale, unified benchmark designed for the evaluation of document-centric multimodal retrieval-augmented generation (MM-RAG) systems. The research objective is to create a realistic evaluation framework to enable fair, apples-to-apples comparisons across different RAG paradigms, including text-only, image-only, and various multimodal approaches. The methodology involves constructing a dataset from 70k real-world PDF pages across 8 domains, from which 1,600 human-verified, multimodal QA pairs are synthesized based on linked textual and visual evidence. The primary result shows that a multimodal text-image fusion (T+I) RAG system consistently outperforms other methods, achieving the highest end-to-end answer completeness score (68.4%), which is notably better than both joint multimodal embedding-based retrieval (64.1%) and text-only RAG (65.3%). The principal implication for AI practitioners is that for document-centric tasks, fusing separate, high-performing text and image retrieval pipelines is currently a more effective and robust strategy than relying on a single, unified multimodal embedding model. |
| CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards (Read more on arXiv or HuggingFace)| Yijiang Li, Zaibin Zhang, Guibin Zhang, Yifan Zhou, xxyQwQ | The paper introduces Co-Evolving Multi-Agent Systems (CoMAS), a reinforcement learning framework where LLM-based agents autonomously improve by generating intrinsic rewards from mutual interactions without external supervision. The research aims to determine if LLM agents can achieve self-evolution by learning purely from inter-agent discussions, mimicking human collaborative improvement. The methodology involves agents engaging in solution proposal, evaluation, and scoring, with an LLM-as-a-judge mechanism formulating zero-sum rewards from these interactions to optimize each agent’s policy via the REINFORCE++ algorithm. Experiments show that CoMAS achieves significant performance gains, including an absolute improvement of 19.80% over the untrained baseline on the GSM8K benchmark in the AutoGen setup. For AI practitioners, this work provides a paradigm for continuously improving LLM agents in a decentralized and scalable manner without requiring external reward models or human-annotated data, relying solely on the dynamics of agent interaction. |
| LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling (Read more on arXiv or HuggingFace)| | This paper introduces Long-RewardBench, a benchmark for evaluating long-context reward models (RMs), and proposes LongRM, a multi-stage training strategy to overcome the observed performance degradation of RMs on tasks with extended contexts. The main research objective is to investigate why state-of-the-art RMs fail in long-context settings (i.e., contexts >4K tokens) and to develop a general training methodology that scales models into robust Long-context RMs (LongRMs) without compromising their short-context capabilities. The key methodology is a two-stage training strategy: 1) A “Short-to-Long” supervised fine-tuning (SFT) stage, where reliable preference judgments are generated on critical short-context snippets and then used to train the model on padded, full-length contexts. 2) A reinforcement learning (RL) alignment stage using a Direct Preference Optimization (DPO) variant to ensure consistency between the model’s judgment and its explanation, with preference data synthesized via a Consistency Majority Voting mechanism. The primary result is that existing RMs, even 70B-parameter models, exhibit a significant performance drop to near-random accuracy (<50%) when context length exceeds 4K tokens. The proposed LongRM strategy substantially improves performance; for example, an 8B LongRM model outperforms 70B-scale baselines on the new benchmark, and the training increases the average score of one model by +16.2 points. The principal implication for AI practitioners is that standard RMs are unreliable for providing supervision signals in long-context applications, such as agentic workflows. The LongRM training strategy provides a concrete and efficient framework for creating specialized, context-aware RMs, which are essential for the effective alignment and reinforcement learning of long-context large language models. |
| Learning on the Job: An Experience-Driven Self-Evolving Agent for
Long-Horizon Tasks (Read more on arXiv or HuggingFace)| | This paper introduces MUSE, an agent framework that enables LLMs to learn from experience and self-evolve at test-time to master long-horizon productivity tasks. The primary objective is to overcome the static nature of existing agents by developing a system that autonomously accumulates and reuses knowledge from its interaction trajectories. The core methodology is a “Plan-Execute-Reflect-Memorize” loop centered on a hierarchical Memory Module that organizes distilled experiences into strategic, procedural, and tool-use knowledge. On the long-horizon TAC benchmark, MUSE achieves a new state-of-the-art with an average partial completion score of 51.78%, a nearly 20% relative improvement over the previous leading method. The principal implication for AI practitioners is that this LLM-agnostic, experience-driven architecture provides a practical paradigm for building agents that continuously improve their performance and generalization on complex real-world tasks without requiring costly model fine-tuning. |
| Taming Text-to-Sounding Video Generation via Advanced Modality Condition
and Interaction (Read more on arXiv or HuggingFace)| | This paper introduces BridgeDiT, a dual-tower diffusion transformer, to improve Text-to-Sounding-Video generation by using disentangled text conditions and a symmetric interaction mechanism. The main objective is to overcome modal interference from shared text prompts and find an optimal architecture for cross-modal feature exchange to generate temporally synchronized audio-visual content. The key methodology consists of the Hierarchical Visual-Grounded Captioning (HVGC) framework to generate separate video and audio captions, and the BridgeDiT architecture which employs a Dual CrossAttention (DCA) mechanism for bidirectional information exchange between pretrained unimodal towers. The model achieves state-of-the-art results, notably a temporal synchronization AV-Align score of 0.275 on the AVSync15 dataset, and ablation studies confirm the superiority of the DCA fusion mechanism over alternatives. The principal implication for AI practitioners is that decoupling text conditions for each modality and enabling symmetric, bidirectional feature fusion between pretrained backbones is a highly effective strategy for improving the quality and temporal synchronization of joint audio-video generation systems. |
| Large Scale Diffusion Distillation via Score-Regularized Continuous-Time
Consistency (Read more on arXiv or HuggingFace)| Jintao Zhang, Qianli Ma, Yuji Wang, Kaiwen Zheng, ChenDRAG | The paper introduces rCM, a score-regularized method to scale continuous-time consistency distillation for large diffusion models, enabling high-fidelity generation in 1-4 steps. The primary objective is to resolve the quality degradation issues, such as poor fine-detail generation, observed when scaling standard continuous-time consistency models (sCM) to large-scale tasks. The proposed rCM methodology augments the sCM objective with a reverse-divergence score distillation loss (DMD) as a regularizer, using a custom parallelism-compatible FlashAttention-2 JVP kernel to facilitate training on models exceeding 10B parameters. Primary results show that rCM matches or surpasses competing methods; a distilled 14B Wan2.1 video model achieves a VBench score of 85.05 in 2 steps, outperforming the original teacher model’s score of 83.58 while accelerating sampling by up to 50x. For AI practitioners, rCM offers a robust framework to distill large-scale diffusion models for few-step inference, significantly reducing computational costs for deployment without compromising generation quality or diversity and avoiding complex GAN-based tuning. |
| Reinforcing Diffusion Models by Direct Group Preference Optimization (Read more on arXiv or HuggingFace)| Jing Tang, Tianyang Hu, Yihong Luo | This paper introduces Direct Group Preference Optimization (DGPO), an online reinforcement learning algorithm for aligning diffusion models with group-level preferences by dispensing with the policy-gradient framework. The research aims to adapt the principles of Group Relative Preference Optimization (GRPO) to diffusion models without requiring inefficient stochastic policies, instead enabling direct learning from preferences using deterministic ODE samplers. DGPO generates a group of samples, partitions them into positive and negative sets based on normalized reward scores (advantages), and directly maximizes the likelihood of this group-wise preference using an advantage-weighted objective. The method achieves state-of-the-art performance, boosting the GenEval score of a base model from 0.63 to 0.97, while training approximately 30 times faster than the policy-gradient-based Flow-GRPO. For AI practitioners, DGPO offers a computationally efficient and scalable method to post-train diffusion models on complex quality metrics, significantly reducing training time and resource requirements by leveraging efficient samplers and avoiding trajectory-wide optimization. |
| Beyond Turn Limits: Training Deep Search Agents with Dynamic Context
Window (Read more on arXiv or HuggingFace)| Yaojie Lu, Bowen Yu, Le Yu, Hao Xiang, TangQiaoYu | The paper introduces DeepMiner, a framework for training deep search agents to handle long-horizon interactions by creating high-difficulty tasks and managing context dynamically. The main objective is to overcome the limitations of insufficient task complexity and context window constraints that hinder the deep reasoning capabilities of existing multi-turn agents. The key methodology involves a reverse construction method to generate complex QA pairs from web sources and a dynamic sliding window mechanism that selectively compresses distant tool responses while preserving assistant reasoning traces during both training and inference. The primary result is that DeepMiner-32B achieves 33.5% accuracy on the BrowseComp-en benchmark, outperforming the previous best open-source agent by almost 20 percentage points and enabling nearly 100 interaction turns within a 32k context length. The principal implication for AI practitioners is that implementing a dynamic sliding window for context management, combined with training on adversarially constructed complex data, provides an effective method to develop more capable agents for long-horizon tasks without requiring larger context windows or external summarization modules. |
| Entropy Regularizing Activation: Boosting Continuous Control, Large
Language Models, and Image Classification with Activation as Entropy
Constraints (Read more on arXiv or HuggingFace)| Huazhe Xu, xtqqwq, ChonghuaLiao, zilinkang | This paper introduces Entropy Regularizing Activation (ERA), a paradigm that constrains model output entropy by applying specially designed activation functions. The research objective is to develop a universally applicable, non-invasive method for entropy regulation that avoids altering the primary optimization objective, unlike traditional entropy bonus terms. The key methodology involves integrating a custom activation function into the model’s architecture to transform its final outputs, thereby architecturally guaranteeing that the policy’s entropy remains above a predefined threshold. This approach demonstrates broad effectiveness, notably boosting the AIME 2025 score for the Qwen2.5-Math-7B large language model by 37.4% and improving SAC performance on HumanoidBench by over 30% with less than 7% computational overhead. The principal implication for AI practitioners is that ERA provides a computationally cheap, non-invasive module that can be seamlessly integrated with existing models across diverse domains to improve performance and prevent issues like entropy collapse without modifying the core loss function. |
| Memory Retrieval and Consolidation in Large Language Models through
Function Tokens (Read more on arXiv or HuggingFace)| | This paper proposes the function token hypothesis, positing that high-frequency grammatical tokens are the primary mechanism for memory retrieval and consolidation in LLMs. The main objective is to understand how memory is retrieved during inference and consolidated during pre-training by examining the distinct roles of function tokens (e.g., punctuation, prepositions) versus content tokens. The methodology combines bipartite graph analysis of token-feature activations, derived from Sparse Autoencoder (SAE) decomposition of Gemma2-9B’s residual stream, with loss trajectory analysis from pre-training 1.5B and 8B models from scratch. The primary results show that a small set of function tokens activate a majority of model features; specifically, the top 10 most frequent tokens activate over 70% of features in the middle layer. The principal implication for AI practitioners is that memory mechanisms and model behavior are disproportionately governed by function tokens, suggesting that interventions during training and inference (e.g., fine-tuning, steering) could be more efficiently targeted at these tokens to control feature activation and model output. |
| Recycling Pretrained Checkpoints: Orthogonal Growth of
Mixture-of-Experts for Efficient Large Language Model Pre-Training (Read more on arXiv or HuggingFace)| Peng Cheng, Yaoxiang Wang, Yucheng Ding, lx865712528, Mr-Philo | The paper proposes an orthogonal growth framework using interpositional layer copying and noisy expert duplication to efficiently recycle converged Mixture-of-Experts (MoE) checkpoints for large language model pre-training. The primary objective is to develop a compute-efficient method for reusing the “sunk cost” of existing checkpoints by expanding them into larger models, as an alternative to training from scratch. The key methodology involves two orthogonal strategies: 1) Depth Growth via “interpositional” layer copying, which duplicates each layer in place to preserve learned weight norm distributions, and 2) Width Growth by duplicating experts and injecting small-magnitude Gaussian noise into the new weights to promote specialization. Scaling an MoE model from 17B to 70B parameters using this framework achieved a 10.66% accuracy gain on downstream tasks compared to a baseline trained from scratch under the same additional computational budget. For AI practitioners, this research provides a validated, cost-effective strategy to create larger, more capable models by leveraging existing pre-trained assets, significantly reducing the computational overhead of pre-training. |
| SciVideoBench: Benchmarking Scientific Video Reasoning in Large
Multimodal Models (Read more on arXiv or HuggingFace)| Mohit Bansal, Lincoln Spencer, Shoubin Yu, Taojiannan Yang, groundmore | This paper introduces SciVideoBench, a new benchmark designed to evaluate the advanced video reasoning capabilities of Large Multimodal Models (LMMs) on complex, research-level scientific experiments. The primary objective is to assess an LMM’s ability to integrate expert domain knowledge with precise visual perception and multi-step logical reasoning, addressing a critical gap left by existing benchmarks focused on general or college-level content. The methodology involved creating 1,000 meticulously crafted multiple-choice questions from 241 research-grade experimental videos using a multi-stage, human-in-the-loop annotation pipeline that leveraged both LLM agents and human domain experts. Evaluation results reveal a significant performance disparity, with the top proprietary model (Gemini-2.5-Pro) achieving 64.30% accuracy, substantially outperforming the best open-source model (38.80%), and demonstrating that all current models struggle with the benchmark’s demands. The principal implication for AI practitioners is that developing models capable of expert-level scientific reasoning requires more than scaling; it necessitates targeted architectural advancements for fine-grained visual-to-text grounding and robust, multi-step numerical calculation. |
| A^2Search: Ambiguity-Aware Question Answering with Reinforcement
Learning (Read more on arXiv or HuggingFace)| | A²SEARCH is a reinforcement learning framework for open-domain question answering that automatically identifies and generates multiple valid answers for ambiguous questions. The main objective is to develop an annotation-free, end-to-end training framework to enable QA models to recognize and handle ambiguity, which is often overlooked by standard benchmarks that assume a single gold answer. The methodology involves an automated pipeline that uses trajectory sampling and evidence verification to discover alternative answers from existing datasets, followed by model optimization using Group Relative Policy Optimization (GRPO) with a custom AnsF1 reward designed for multi-answer scenarios. The primary result is that A²SEARCH-7B achieves a new state-of-the-art, yielding an average AnsF1@1 score of 48.4% across four multi-hop benchmarks with a single rollout, outperforming the substantially larger ReSearch-32B model (46.2%). The principal implication for AI practitioners is that explicitly modeling and rewarding for ambiguity, rather than penalizing valid but non-reference answers, is essential for developing more robust and reliable QA systems; the paper provides a practical pipeline for augmenting single-answer datasets to achieve this. |
| GCPO: When Contrast Fails, Go Gold (Read more on arXiv or HuggingFace)| | This paper introduces Group Contrastive Policy Optimization (GCPO), a reinforcement learning method that improves LLM reasoning by injecting external “golden answers” when a model’s self-generated responses are all incorrect. The research objective is to address the vanishing gradient problem in algorithms like Group Relative Policy Optimization (GRPO) where training stalls if no correct samples are produced for a given problem. The core methodology involves detecting training steps with all-zero rewards and substituting one failed rollout with a correct reference answer, thereby creating a non-zero advantage to guide the policy update. On the DeepSeek-R1-Distill-Qwen-1.5B model, GCPO achieved an average accuracy of 36.95% across six math benchmarks, outperforming the DAPO baseline’s 30.37%. For AI practitioners, the principal implication is that augmenting RL training with a curated set of high-quality solutions when the model fails is a practical and effective technique to enhance reasoning capabilities and overcome training plateaus, especially for smaller models. |
| Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs (Read more on arXiv or HuggingFace)| Franck Dernoncourt, Yue Zhao, Hongjie Chen, Tiankai Yang, Wang Wei | BaRP introduces a preference-conditioned contextual bandit framework for efficient LLM routing using bandit feedback. The main objective is to dynamically route LLM queries to balance performance and cost, addressing the mismatch between full-information offline training and partial-feedback deployment conditions, while enabling preference-tunable inference. BaRP models this as a multi-objective contextual bandit problem, conditioning its policy on a user-defined performance-cost preference vector and training via REINFORCE with simulated bandit feedback. Experiments on RouterBench demonstrate that BaRP outperforms strong offline routers by at least 12.46% on in-distribution tasks and reduces monetary cost by 50.00% compared to the strongest offline baseline. This allows AI practitioners to deploy adaptive and cost-effective LLM routing systems with tunable performance-cost trade-offs at inference time, without requiring full-information offline supervision or retraining. |
| R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized
Manipulation (Read more on arXiv or HuggingFace)| Zheng Zhu, Bingyao Yu, Hankun Li, Angyuan Ma, Xiuwei Xu | R2RGen is a simulator-free framework that generates diverse, real-world 3D pointcloud-action data from a single human demonstration to improve the spatial generalization of visuomotor policies. The research aims to reduce the extensive human data collection effort required for imitation learning by automatically generating spatially varied training data for robotic manipulation tasks, including those involving mobile manipulators. The methodology involves parsing a source demonstration into complete 3D object pointclouds and skill segments, applying a group-wise backtracking augmentation to transform object groups and action trajectories while preserving task constraints, and using camera-aware 3D post-processing to ensure the augmented data matches real sensor distributions. A policy trained with R2RGen data from one demonstration achieved a 40.3% average success rate, comparable to a policy trained with 25 human demonstrations (41.0% success rate) and significantly outperforming the prior method DemoGen. The principal implication for AI practitioners is that this real-to-real data generation pipeline can drastically improve data efficiency and train spatially robust 3D visuomotor policies with minimal human supervision, facilitating scalable learning for applications like mobile manipulation. |
| UP2You: Fast Reconstruction of Yourself from Unconstrained Photo
Collections (Read more on arXiv or HuggingFace)| Boqian Li, Xiaoben Li, Ziyang Li, Yuliang, Co2y | UP2You is a tuning-free method for rapidly reconstructing high-fidelity 3D clothed human avatars from collections of unconstrained 2D photos. The primary objective is to create a robust system that can process raw, unstructured photographs—varying in pose, viewpoint, and occlusion—to generate high-quality textured 3D models without per-subject fine-tuning. The key methodology is a “data rectifier” paradigm, which uses a Pose-Correlated Feature Aggregation (PCFA) module to selectively fuse features from multiple input images and convert them into clean, orthogonal multi-view images and normal maps, making them compatible with traditional 3D reconstruction. The method demonstrates superior performance over previous approaches, achieving a 15% reduction in Chamfer distance on the PuzzleIOI dataset and completing the entire reconstruction pipeline in 1.5 minutes. For AI practitioners, the principal implication is the introduction of an efficient, feed-forward alternative to computationally expensive optimization-based avatar generation, enabling the creation of personalized 3D assets from casual photos for applications like virtual try-on. |
| Fidelity-Aware Data Composition for Robust Robot Generalization (Read more on arXiv or HuggingFace)| Liliang Chen, Hongwei Fan, Sicheng Hu, Di Chen, Zizhao Tong | This paper introduces a framework for principled data composition to improve the out-of-distribution (OOD) generalization of robot policies by mitigating shortcut learning. The main objective is to determine a systematic method for composing real and synthetic data to enhance policy robustness, addressing the trade-off between visual diversity and information fidelity. The key methodology is Coherent Information Fidelity Tuning (CIFT), which uses a practical proxy called Feature-Space Signal-to-Noise Ratio (SNR) to analyze the feature-space geometry of a dataset and identify an optimal mixing ratio before a “Decoherence Point” where training stability degrades. The primary result is that applying CIFT to policy architectures such as πο and Diffusion Policy improves OOD success rates by over 54%; for instance, a baseline Diffusion Policy’s OOD success rate on a picking task increased from 0% to 85% under challenging semantic shifts. The principal implication for AI practitioners is that naively adding synthetic data can degrade performance; data composition must be a principled, fidelity-aware process, and a computationally cheap, pre-training feature analysis can predict an optimal data mixture to maximize robustness. |
| SViM3D: Stable Video Material Diffusion for Single Image 3D Generation (Read more on arXiv or HuggingFace)| | SViM3D is a generative video diffusion model that jointly predicts multi-view consistent RGB imagery, physically-based rendering (PBR) material maps, and normals from a single image to create relightable 3D assets. The main objective is to develop a unified model for object-centric inverse rendering from a single image, generating multi-view consistent, spatially-varying PBR materials and geometry suitable for high-quality 3D reconstruction and relighting. The methodology extends a latent video diffusion model (SV3D) by adapting its UNet architecture to output an 11-channel video tensor (RGB, basecolor, roughness, metallic, normal). This model is trained on a custom multi-illumination synthetic dataset and its output serves as a pseudo-ground-truth neural prior to optimize a 3D representation using techniques like view-dependent masking and learnable homography correction. The model achieves state-of-the-art performance in material prediction and novel view synthesis; for single-frame basecolor prediction on the Poly Haven test set, SViM3D achieves a PSNR of 28.68, significantly outperforming the next-best baseline which scored 20.59. The principal implication for AI practitioners is the availability of a foundational model for single-image-to-3D pipelines that provides a unified prior for both geometry and PBR materials, simplifying the workflow for generating relightable assets by removing the need to chain separate models for shape and material estimation. |
| Search-R3: Unifying Reasoning and Embedding Generation in Large Language
Models (Read more on arXiv or HuggingFace)| James Cheng, ytgui | The paper introduces Search-R3, a framework that adapts Large Language Models (LLMs) to generate search embeddings as a direct output of their chain-of-thought reasoning process. The research objective is to unify semantic reasoning and embedding generation within a single model to overcome the limitations of using separate systems for these tasks. The methodology consists of a two-stage training pipeline: an initial supervised learning stage with contrastive loss to teach the model to produce an embedding token, followed by a reinforcement learning stage using Group Relative Policy Optimization (GRPO) to optimize the reasoning path for end-to-end retrieval performance. The primary result is that Search-R3 significantly outperforms prior methods; for example, on the SciFact benchmark, enabling reasoning improves the nDCG@10 score from 0.624 to 0.672. The principal implication for AI practitioners is the ability to use a single, unified model for both generative reasoning and high-quality embedding retrieval, which can simplify system architecture and reduce computational overhead in applications like Retrieval-Augmented Generation (RAG). |
| Towards Scalable and Consistent 3D Editing (Read more on arXiv or HuggingFace)| Pan Zhou, Yang Tang, XiaRho | The paper introduces 3DEditVerse, the first large-scale paired 3D editing benchmark with 116,309 training and 1,500 test assets, alongside 3DEditFormer, a novel 3D-structure-preserving conditional transformer. The main objective is to enable precise, localized, and structure-preserving 3D edits with intuitive prompts while maintaining cross-view consistency. 3DEditFormer employs a Dual-Guidance Attention Block and Time-Adaptive Gating mechanism to disentangle editable regions from preserved structure, operating without auxiliary 3D masks. The framework achieves state-of-the-art 3D editing performance, demonstrating a +13% improvement in 3D metrics over VoxHammer. This allows AI practitioners to perform high-fidelity, practical 3D editing, simplifying content creation by eliminating the need for manual 3D mask supervision. |
| Beyond Outliers: A Study of Optimizers Under Quantization (Read more on arXiv or HuggingFace)| | This paper systematically evaluates how optimizer choice impacts large language model performance under post-training and quantization-aware training regimes. i) The main objective is to investigate the interaction between different optimizers and quantization schemes (PTQ and QAT) to determine which optimizers yield more robust quantized models. ii) The authors train OLMo2 models (50M to 1.5B parameters) with six optimizers (AdamW, Muon, PSGD, Scion, Shampoo, SOAP), then apply 4-bit PTQ and perform 4-bit QAT, evaluating performance on zero-shot benchmarks and developing a theoretical framework to analyze error propagation. iii) The primary results show that common outlier metrics like Max-to-Mean Ratio (MMR) do not predict PTQ performance across different optimizers. For both PTQ and QAT, models trained with Shampoo consistently exhibit the lowest performance degradation; specifically, for the 760M model under QAT, Shampoo’s accuracy drop was only -0.46%, the lowest among all optimizers tested. v) The principal implication for AI practitioners is that the optimal optimizer for full-precision training (e.g., Muon in this study) is not necessarily the best for quantized models, and selecting an optimizer like Shampoo can significantly improve the performance and parameter efficiency of low-bit models intended for deployment. |
Papers for 2025-10-09
| Title |
Authors |
Summary |
| Cache-to-Cache: Direct Semantic Communication Between Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
|
This paper introduces Cache-to-Cache (C2C), a paradigm for direct semantic communication between LLMs by transferring and fusing their internal KV-Cache states instead of generating intermediate text. The primary objective is to overcome the information loss and latency inherent in text-based communication by enabling models to share richer, internal representations. The core methodology involves a neural network that projects a source model’s KV-Cache and fuses it with a target model’s cache, using a learnable gating mechanism to select which layers benefit from the fusion. C2C outperforms text communication by 3.0-5.0% in accuracy while delivering an average 2.0x speedup in latency. For AI practitioners, this work provides a method to build more performant and efficient multi-LLM systems by bypassing the token generation bottleneck and enabling direct, high-bandwidth semantic transfer between heterogeneous models. |
| Ming-UniVision: Joint Image Understanding and Generation with a Unified |
|
|
| Continuous Tokenizer (Read more on arXiv or HuggingFace) |
|
Ming-UniVision is a unified autoregressive model that jointly performs image understanding and generation using MingTok, a novel three-stage continuous visual tokenizer, to eliminate quantization errors and reconcile competing representation demands. The research objective is to unify visual understanding and generation within a single autoregressive framework by developing a visual tokenizer that operates in a continuous latent space, thereby avoiding the architectural complexity of discrete or dual-representation approaches. The key methodology is the introduction of MingTok, a three-stage tokenizer with a low-level encoder for compact latents, a causal semantic decoder for high-dimensional semantic features, and a pixel decoder for reconstruction, all integrated into a large language model that treats vision-language tasks as next-token prediction. Primary results show the model achieves an overall score of 0.85 on the GenEval text-to-image benchmark, outperforming other models in spatial reasoning with a Position score of 0.92, and reduces input visual tokens for in-context editing by up to 66% compared to hybrid models. The principal implication for AI practitioners is that a single, shared continuous visual representation can effectively serve both discriminative and generative tasks, enabling simplified, stateful, and more computationally efficient multimodal systems that operate directly in the latent space for complex interactions. |
| Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal |
|
|
| Generation and Understanding (Read more on arXiv or HuggingFace) |
|
Lumina-DiMOO is an open-source omni discrete diffusion large language model for multi-modal generation and understanding. The objective is to develop a foundational model for seamless multi-modal generation and understanding by utilizing a fully discrete diffusion modeling paradigm. It employs a unified discrete diffusion framework that processes multi-modal inputs and outputs via discrete tokens and a masked cross-entropy objective, incorporating a training-free Max Logit-based Cache (ML-Cache) for inference acceleration. Lumina-DiMOO achieves a 32x speed improvement in text-to-image generation compared to Lumina-mGPT 2.0 and sets new SOTA results with an 88% overall score on the GenEval benchmark. The open-sourced Lumina-DiMOO provides AI practitioners with a highly efficient and versatile foundation model for advancing research and applications in general-purpose multi-modal intelligence, including interactive image retouching. |
| SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models (Read more on arXiv or HuggingFace) |
Kevin Lin, Chung-Ching Lin, Linjie Li, Xiaofei Wang, Cheng-Han Chiang |
SHANKS is an inference framework enabling spoken language models (SLMs) to perform internal reasoning concurrently with user speech input. The objective is to address high response latency in current SLMs/LLMs by allowing them to “think while listening,” crucial for real-time speech-to-speech interaction. SHANKS streams user input speech in fixed-duration chunks, generating unspoken thinking tokens based on all previous speech and reasoning upon receiving each chunk to enable real-time decision-making like interruptions or tool calls. In experiments, SHANKS interrupted users 37.1% more accurately in a math problem-solving scenario and completed 56.9% of tool calls before the user’s turn ended in a task-oriented dialogue. This enables AI practitioners to develop SLM applications with significantly reduced latency and improved real-time interactivity, particularly for scenarios requiring early intervention or proactive task completion. |
| RLinf-VLA: A Unified and Efficient Framework for VLA+RL Training (Read more on arXiv or HuggingFace) |
|
RLinf-VLA is a unified and efficient framework designed for scalable Reinforcement Learning (RL) training of Vision-Language-Action (VLA) models, addressing the challenges of generalization and fragmented experimentation in embodied AI. Its main objective is to provide a comprehensive platform that integrates diverse VLA architectures (e.g., OpenVLA, OpenVLA-OFT), multiple RL algorithms (PPO, GRPO), and various simulators (ManiSkill, LIBERO) with flexible GPU allocation. The framework utilizes a novel hybrid fine-grained pipelining allocation mode, achieving a 1.61x-1.88x speedup in training for GPU-parallelized simulators. A single unified model achieved 98.11% success across 130 LIBERO tasks and 97.66% across 25 ManiSkill tasks in simulation, with RL-trained policies exhibiting stronger zero-shot generalization on a real-world Franka robot compared to SFT. This provides AI practitioners with a robust and efficient open-source foundation to accelerate and standardize research and deployment in embodied intelligence. |
| MATRIX: Mask Track Alignment for Interaction-aware Video Generation (Read more on arXiv or HuggingFace) |
Hyunwook Choi, Jaeho Lee, Dahyun Chung, Siyoon Jin, Seongchan |
MATRIX introduces a regularization framework to enhance interaction-aware video generation in Diffusion Transformers (DiTs). The main objective is to understand how video DiTs internally represent multi-instance and subject-object interactions and then improve generation quality. The methodology involves curating MATRIX-11K, a dataset with multi-instance mask tracks and interaction-aware captions, followed by a systematic analysis of semantic grounding and propagation within DiT attention layers. MATRIX applies Semantic Grounding Alignment (SGA) and Semantic Propagation Alignment (SPA) losses to interaction-dominant layers, finetuning with LoRA. Experimentally, MATRIX achieves an Interaction Fidelity (IF) of 0.593, outperforming baseline models. This enables AI practitioners to generate videos with significantly improved interaction fidelity, semantic alignment, and reduced drift and hallucination. |
| Vibe Checker: Aligning Code Evaluation with Human Preference (Read more on arXiv or HuggingFace) |
|
Vibe Checker introduces a novel testbed for evaluating large language models’ code generation, integrating verifiable instruction following alongside functional correctness to better align with human preference. The core objective is to quantify models’ adherence to non-functional coding instructions, hypothesizing this is a key, under-measured component of human preference in “vibe checking” code solutions. The methodology involves VeriCode, a taxonomy of 30 verifiable instructions with deterministic verifiers, used to augment standard benchmarks (BigVibeBench, LiveVibeBench) and evaluate 31 LLMs in single-turn and multi-turn settings. Results show that even strong models exhibit significant functional regression with added instructions, with average pass@1 dropping by 5.85% and 6.61% under five instructions on respective benchmarks; a composite score of functional correctness and instruction following consistently correlates best with human preference (e.g., optimal Pearson alpha=0.4 IF on BigVibeBench). This work implies that AI practitioners should prioritize integrating instruction following into both evaluation and training pipelines to improve LLM alignment with user preferences in code generation, especially for real-world programming tasks. |
| Multi-Agent Tool-Integrated Policy Optimization (Read more on arXiv or HuggingFace) |
Lidong Bing, Yuntao Chen, Xingxuan Li, Zhanfeng Mo |
Multi-Agent Tool-Integrated Policy Optimization (MATPO) is introduced to train multi-agent LLM frameworks for knowledge-intensive tasks within a single model instance. The core objective is to enable effective multi-agent RL training, handle reward assignment for worker-agents, and support distinct planner and worker roles using one LLM. MATPO is derived from a principled credit assignment mechanism across planner and worker rollouts, using role-specific prompts and reinforcement learning built on single-agent multi-turn RL. Experiments on GAIA-text, WebWalkerQA, and FRAMES demonstrate MATPO consistently outperforms single-agent baselines by an average of 18.38% relative improvement in performance. This highlights the effectiveness of unifying multiple agent roles within a single LLM for stable and efficient multi-agent RL training, providing practical insights for AI practitioners. |
| OBS-Diff: Accurate Pruning For Diffusion Models in One-Shot (Read more on arXiv or HuggingFace) |
|
OBS-Diff introduces a novel one-shot, training-free pruning framework for large-scale text-to-image diffusion models. The main objective is to establish a general and training-free pruning framework for diffusion models supporting diverse architectures and multiple pruning granularities in a single pass. The methodology revitalizes the Optimal Brain Surgeon (OBS) framework, adapting it with a Timestep-Aware Hessian Construction that uses a logarithmic weighting scheme and a computationally efficient group-wise sequential pruning strategy via “Module Packages”. OBS-Diff achieves state-of-the-art one-shot pruning, evidenced by a 0.6468 ImageReward on SD 3-medium at 50% unstructured sparsity (outperforming Magnitude’s -0.1076) and providing up to 1.31x inference speedup for structured pruning. This enables AI practitioners to deploy large diffusion models with substantially reduced computational and memory costs, enhancing efficiency and accessibility without requiring retraining or fine-tuning. |
| Revisiting Long-context Modeling from Context Denoising Perspective (Read more on arXiv or HuggingFace) |
|
This paper introduces Context Denoising Training (CDT) to enhance long-context models by identifying and suppressing contextual noise using Integrated Gradients, thereby improving attention on critical tokens. The primary objective is to address the performance degradation of Long-Context Models (LCMs) caused by irrelevant contextual noise, by developing a method to detect and mitigate this noise to improve model predictions. The proposed Context Denoising Training (CDT) involves two steps: first, Critical Token Detection using an Integrated Gradient (IG) score approximated by L2-normalized embedding gradients to identify noisy tokens; second, Emphasizing Training, which suppresses the influence of these detected noisy tokens by subtracting their corresponding gradients from input embeddings. Experiments demonstrate CDT’s superiority; notably, a Llama3.1-8B-Instruct model trained with CDT achieved 50.92 points on real-world tasks (LongBench-E), closely matching GPT-40’s 51.00 points. This indicates that AI practitioners can significantly enhance the robustness and long-context understanding of LLMs, particularly in noisy or very long input scenarios, by applying this efficient gradient-based denoising training strategy. |
| Artificial Hippocampus Networks for Efficient Long-Context Modeling (Read more on arXiv or HuggingFace) |
|
Artificial Hippocampus Networks (AHNs) enhance Transformer efficiency for long-context modeling by integrating fixed-size compressed memory. The main objective is to resolve the fundamental trade-off in long-sequence modeling between efficient fixed-size memory (RNN-like) and lossless growing memory (attention-based Transformers). AHNs achieve this by maintaining a sliding window for lossless short-term memory and using a learnable RNN-like module (e.g., Mamba2, DeltaNet, GatedDeltaNet) to recurrently compress out-of-window information into a fixed-size long-term memory, trained via self-distillation from pre-trained LLMs. For instance, augmenting Qwen2.5-3B-Instruct with AHNs (+0.4% parameters) reduced FLOPs by 40.5% and memory cache by 74.0% on the LV-Eval 128k sequence length benchmark, while improving the average score from 4.41 to 5.88. The principal implication for AI practitioners is that AHNs provide a method to significantly reduce the computational and memory requirements of Transformer models, enabling more efficient processing of extremely long sequences without substantial performance degradation. |
| Why Low-Precision Transformer Training Fails: An Analysis on Flash |
|
|
| Attention (Read more on arXiv or HuggingFace) |
|
This paper provides a mechanistic explanation for catastrophic loss explosions during low-precision (BF16) transformer training using Flash Attention. The primary objective is to identify the root cause of a long-standing, reproducible training failure characterized by a sudden loss spike. The authors use a targeted analysis on a GPT-2 model, systematically comparing low-precision (BF16) and high-precision (FP32) computations to isolate the source of numerical error. The key result is that the failure stems from biased rounding errors in BF16 addition during the PV computation, which occurs specifically when multiple attention probabilities P become exactly 1, leading to a systematic negative bias in the output O and a corrupted, accumulating gradient error. For AI practitioners, this implies that the instability is a deterministic numerical artifact that can be mitigated by a minimal modification to the safe softmax implementation to prevent attention probabilities from becoming exactly 1, thereby stabilizing the training process. |
| Native Hybrid Attention for Efficient Sequence Modeling (Read more on arXiv or HuggingFace) |
Yu Cheng, Weigao Sun, Tao Zhang, Jiaxi Hu, Jusen Du |
Native Hybrid Attention (NHA) is a novel architecture unifying linear and full attention for efficient and accurate sequence modeling. The primary objective is to develop a hybrid attention mechanism that overcomes the quadratic complexity of Transformers while maintaining recall accuracy. NHA integrates intra-layer hybridization by compressing long-term context via a linear RNN into KV slots and concatenating it with short-term sliding window tokens, then applying a single, unified softmax attention. Experimental results demonstrate NHA consistently outperforms Transformers and other hybrid baselines on recall-intensive and commonsense reasoning tasks, with NHA-Llama3-8B achieving superior recall accuracy with only 4 full attention layers, compared to other hybrids requiring more layers for lower accuracy. AI practitioners can leverage NHA to structurally hybridize existing pretrained Transformer LLMs, achieving competitive performance with significant efficiency gains and improved inference speed through brief finetuning. |
| When Benchmarks Age: Temporal Misalignment through Large Language Model |
|
|
| Factuality Evaluation (Read more on arXiv or HuggingFace) |
|
This paper investigates the impact of benchmark aging on large language model factuality evaluation. It quantifies how widely used static benchmarks contain outdated factual answers and how this aging affects the evaluation of modern LLMs. The authors developed a fact retrieval pipeline for current real-world facts and introduced metrics like Dataset Drift Score, Evaluation Misleading Rate, and Temporal Alignment Gap. Experiments show that up to 63.78% of time-sensitive samples in older benchmarks are outdated, leading to an Evaluation Misleading Rate exceeding 10% for modern LLMs. AI practitioners should account for temporal misalignment, as relying on aging benchmarks results in unreliable factuality assessments and can unfairly penalize models for up-to-date responses. |
| Are We Using the Right Benchmark: An Evaluation Framework for Visual |
|
|
| Token Compression Methods (Read more on arXiv or HuggingFace) |
Yiyu Wang, Xu Zheng, Zichen Wen, Wensong Wang, Chenfei Liao |
This paper introduces VTC-Bench, an evaluation framework designed to address task mismatch and noise in existing MLLM benchmarks for visual token compression methods. The research investigates why simple image downsampling consistently outperforms advanced visual token compression methods on current MLLM benchmarks. The proposed VTC-Bench framework filters existing benchmark samples by using downsampling as a discriminator to categorize them into “simple” and “difficult” groups, then evaluates compression methods primarily on the “difficult” samples. Empirical results reveal that simple downsampling achieves a 91.0% Average Decline Ratio (ADR) on Qwen2-VL-7B at 75% compression across eight benchmarks, while DART achieves 40.2% on OCRBench for “difficult” samples where downsampling performs at 0% accuracy. AI practitioners should adopt specialized evaluation frameworks like VTC-Bench to denoise existing benchmarks and ensure fair assessment of visual token compression methods, enabling more accurate and relevant R&D. |
| StaMo: Unsupervised Learning of Generalizable Robot Motion from Compact |
|
|
| State Representation (Read more on arXiv or HuggingFace) |
|
StaMo presents an unsupervised framework for learning generalizable robot motion from compact state representations derived from static images. The core objective is to develop expressive yet compact state representations, investigating if robot motion can naturally emerge as the difference between state encodings from individual frames rather than complex temporal video modeling. StaMo leverages a Diffusion Autoencoder, with a DINOv2 encoder and Diffusion Transformer (DiT) decoder, to compress visual observations into two 1024-dimensional tokens, where latent motion is defined by the vector difference between these tokens for world modeling and policy co-training. This approach significantly improves performance by 14.3% on LIBERO and yields a 30% increase in real-world task success rates, outperforming prior methods by 10.4% in co-training. For AI practitioners, StaMo offers a scalable pathway for efficient world models and generalizable robot skills by implicitly capturing dynamics from static images, reducing reliance on computationally intensive video-based motion learning. |
| Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in |
|
|
| MLLMs (Read more on arXiv or HuggingFace) |
Jingyi Liao, Nanqing Liu, Shijie Li, Haojie Zhang, Yongyi Su |
Patch-as-Decodable Token (PaDT) unifies multimodal large language models (MLLMs) to directly generate diverse textual and visual outputs. The research addresses limitations of existing MLLMs that rely on indirect textual representations for vision tasks, aiming to enable direct generation of both textual and diverse visual outputs for dense prediction. PaDT introduces Visual Reference Tokens (VRTs) derived from visual patch embeddings, seamlessly interleaved with LLM output tokens using a Dynamic Embedding Module. A lightweight PaDT Decoder then transforms LLM outputs into structured visual predictions, optimized with a robust per-token cross-entropy loss and random VRT sampling. Notably, PaDT’s 3B model surpasses prior state-of-the-art by 19.0 mAP on COCO detection and achieves an average accuracy of 93.6 on referring expression comprehension. AI practitioners can apply PaDT to develop MLLMs capable of direct, semantically aligned visual and textual generation, improving precision and robustness for a wide range of vision-language tasks beyond traditional text-based coordinate serialization. |
| WristWorld: Generating Wrist-Views via 4D World Models for Robotic |
|
|
| Manipulation (Read more on arXiv or HuggingFace) |
|
WristWorld is a 4D world model for synthesizing geometrically and temporally consistent wrist-view videos from anchor views for robotic manipulation. The primary objective is to enrich existing third-person datasets with automatically generated, geometrically consistent wrist-view sequences to enhance both perception and control for robotic tasks. The framework operates in two stages: a reconstruction stage extending VGGT with a wrist head and Spatial Projection Consistency (SPC) Loss to estimate wrist-view poses and 4D point clouds, followed by a generation stage using a diffusion-based video generator conditioned on these projections and CLIP-encoded anchor-view features. Experiments on Droid, Calvin, and Franka Panda demonstrate state-of-the-art video generation quality, with WristWorld closing 42.4% of the anchor-wrist view performance gap and increasing average task completion length on Calvin by 3.81%. WristWorld serves as a plug-and-play add-on, enabling existing single-view world models to gain multi-view capabilities and expanding training data without requiring new wrist-view data collection, thereby improving downstream VLA model performance. |
| TTRV: Test-Time Reinforcement Learning for Vision Language Models (Read more on arXiv or HuggingFace) |
Serena Yeung-Levy, Paul Gavrikov, Wei Lin, Shyam Marjit, Akshit Singh |
TTRV is a novel test-time reinforcement learning framework that adapts Vision-Language Models (VLMs) at inference using self-supervised reward signals from unlabeled test data. Its primary objective is to enable VLMs to self-improve on-the-fly without requiring labeled datasets, addressing the limitations of static pretrained models. The methodology extends Group Relative Policy Optimization (GRPO) by incorporating frequency-based rewards for output consistency and diversity control rewards from the negative Shannon entropy of empirical response distributions. TTRV achieves substantial performance improvements, notably boosting Intern-VL-8B on image recognition by an average of 2.3% over GPT-40 across 8 benchmarks. This framework provides AI practitioners with a robust paradigm for deploying VLMs capable of continuous, unsupervised adaptation and self-improvement in dynamic, real-world scenarios. |
| MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline (Read more on arXiv or HuggingFace) |
|
MLE-Smith is a fully automated multi-agent pipeline designed to scale Machine Learning Engineering (MLE) task generation from raw datasets. The paper addresses the scalability and diversity limitations of existing manually curated MLE benchmarks by automating task generation and ensuring verifiable quality. It utilizes a multi-agent generation workflow (Brainstormer, Designer, Refactor) for structured task design, coupled with a hybrid verification mechanism comprising deterministic assertions, LLM-based reviews, and execution-based validation. MLE-Smith generated 606 verified tasks from 224 datasets at an average cost of $0.78 per task, with LLM performance on these tasks showing a strong linear correlation (Pearson r = 0.982) and excellent inter-set reliability (Cronbach’s α = 0.993) compared to human-designed benchmarks. This framework enables AI practitioners to efficiently generate realistic, challenging, and discriminative MLE tasks for large-scale evaluation and training of next-generation MLE agents. |
| The African Languages Lab: A Collaborative Approach to Advancing |
|
|
| Low-Resource African NLP (Read more on arXiv or HuggingFace) |
|
The African Languages Lab (All Lab) established a comprehensive framework for advancing low-resource African NLP through large-scale multi-modal data collection and fine-tuning of a multilingual language model. The initiative’s objective is to address the critical technological gap for African languages, which are severely underserved in modern NLP, through systematic data collection, model development, and capacity building. The methodology involved building a mobile-first, community-driven “All Voices” platform for quality-controlled multi-modal data collection, rigorous two-tier data processing, statistical validation, and fine-tuning of the Llama-3.2-1B model on the collected dataset. The project yielded the largest validated African multi-modal dataset, comprising 19 billion tokens of monolingual text and 12,628 hours of aligned speech data across 40 languages. Fine-tuning demonstrated substantial performance improvements, with average gains of +23.69 ChrF++, +0.33 COMET, and +15.34 BLEU points across 31 evaluated languages. AI practitioners can leverage this new, large-scale, quality-controlled multi-modal dataset and the demonstrated fine-tuning approach to significantly enhance NLP capabilities for previously underserved African languages, enabling functional translation where none existed before. |
| Revisiting the Uniform Information Density Hypothesis in LLM Reasoning |
|
|
| Traces (Read more on arXiv or HuggingFace) |
Jaehyung Kim, Guijin Son, Minju Gwak |
This paper revisits the Uniform Information Density (UID) hypothesis to analyze information flow in LLM reasoning traces, linking step-level information density uniformity to reasoning quality. The research investigates whether step-level uniformity in LLM-generated reasoning traces reflects reasoning quality, particularly on complex mathematical benchmarks. The authors propose an entropy-based stepwise information density metric ($ID_i$) and introduce complementary local and global uniformity scores, computed as the variance of normalized $ID_i$ and step-to-step spikes/falls, respectively, evaluated across LLM reasoning traces. Experiments show that UID-based trace selection consistently improves reasoning accuracy; for instance, selecting traces with more uniform local information density yielded up to 32% relative accuracy gains over baselines on AIME2025 for Deepseek-R1. These findings establish information density uniformity as a robust diagnostic and selection criterion for AI practitioners, guiding the development of more reliable and accurate LLM reasoning systems. |
| Online Generic Event Boundary Detection (Read more on arXiv or HuggingFace) |
Jonghyun Choi, Jeany Son, Seunggyun Lim, Hyungrok Jung, carpedkm |
This paper introduces Online Generic Event Boundary Detection (On-GEBD) to detect taxonomy-free event boundaries in streaming videos in real-time, mirroring human perception. The proposed ESTimator framework, inspired by Event Segmentation Theory, comprises a Consistent Event Anticipator (CEA) using a transformer decoder and an Online Boundary Discriminator (OBD) that employs statistical testing on a queue of past prediction errors for dynamic thresholding. ESTimator demonstrates superior performance, achieving an Avg. F1 score of 0.748 on Kinetics-GEBD, outperforming adapted online video understanding baselines (e.g., MiniROAD-BC at 0.681). Furthermore, it achieves comparable or superior results to most offline GEBD methods despite its online constraint. This work provides AI practitioners with a robust, real-time solution for generalizable video event segmentation, critical for applications requiring immediate analysis of continuous visual data. |
| The Markovian Thinker (Read more on arXiv or HuggingFace) |
|
The Markovian Thinker introduces a paradigm for LLMs to achieve long-chain-of-thought reasoning with linear compute and constant memory. Its main objective is to decouple thinking length from context size, addressing the quadratic compute growth of standard RL environments for reasoning LLMs. The key methodology is Delethink, an RL environment where policies reason in fixed-size chunks, with the environment resetting context at boundaries and reinitializing prompts using a short textual carryover, forcing the policy to learn to maintain a bounded Markovian state. Primary results show an R1-Distill 1.5B model, trained with Delethink, can think up to 24K tokens, matching or surpassing LongCoT-RL with the same budget, and training for 94K average thinking length costs 7 H100-months with Delethink compared to 27 for LongCoT-RL. This implies AI practitioners can develop efficient and scalable reasoning LLMs capable of very long reasoning without quadratic overhead by redesigning the RL environment to enforce constant-size states. |
| Bridging Text and Video Generation: A Survey (Read more on arXiv or HuggingFace) |
G. Maragatham, Priyansh Bhandari, nnilayy |
This paper surveys the evolution of text-to-video (T2V) generative models, their architectures, training, and evaluation methods. The primary objective is to provide a comprehensive, unified overview of T2V models, detailing their development from early GANs and VAEs to current diffusion-based architectures, and explaining internal mechanisms, limitations, and architectural shifts. The methodology involves a systematic analysis of T2V model architectures, datasets (e.g., WebVid-10M, LAION-5B), and training configurations, alongside a review of evaluation metrics and benchmarks; for example, VideoFusion [12] achieved an Inception Score (IS) of 71.67 on the UCF-101 benchmark. The principal implication for AI practitioners is the need to overcome challenges such as limited data availability, high computational costs, and difficulties in modeling long-range temporal consistency by exploring novel architectures, synthetic data generation, and enhanced temporal modeling strategies, leveraging the detailed training parameters provided. |
| AlphaApollo: Orchestrating Foundation Models and Professional Tools into |
|
|
| a Self-Evolving System for Deep Agentic Reasoning (Read more on arXiv or HuggingFace) |
Zongze Li, Xuan Li, Xiao Feng, Chentao Cao, Zhanke Zhou |
AlphaApollo is a self-evolving agentic reasoning system designed to overcome limited model-intrinsic capacity and unreliable test-time iteration in foundation models. Its objective is to enable deliberate, verifiable reasoning by orchestrating multiple foundation models with professional tools. The system’s methodology involves coupling computation and retrieval tools, along with a multi-round, multi-model solution evolution process via a shared state map and a rollout framework. Empirically, AlphaApollo achieved substantial performance gains, notably increasing Average@32 by 16.67% and Pass@32 by 23.34% (from 23.33% to 46.67%) on AIME 2025 for Llama-3.3-70B-Instruct. For AI practitioners, AlphaApollo demonstrates that orchestrating FMs with professional tools and iterative refinement significantly lifts the capability ceiling of FMs, enhancing both average performance and problem-solving abilities. |
| G^2RPO: Granular GRPO for Precise Reward in Flow Models (Read more on arXiv or HuggingFace) |
|
G2RPO introduces a novel online reinforcement learning framework for flow models, designed for precise and comprehensive reward assessments. It addresses sparse reward and incomplete evaluation in existing GRPO methods by localizing stochasticity and integrating multi-granularity advantages. The methodology relies on Singular Stochastic Sampling to confine SDE perturbations to single steps and Multi-Granularity Advantage Integration to fuse advantages from images denoised at various granularities. When jointly trained with HPS-v2.1 and CLIP, G2RPO achieved an HPS-v2.1 score of 0.376 and a CLIP Score of 0.406, outperforming baselines across in-domain and out-of-domain metrics. This framework offers AI practitioners a more robust and efficient approach for aligning generative models with human preferences through enhanced reward signals, crucial for stable and high-quality policy optimization. |
| U-Bench: A Comprehensive Understanding of U-Net through 100-Variant |
|
|
| Benchmarking (Read more on arXiv or HuggingFace) |
Heqin Zhu, Zikang Xu, Wenxin Ma, Chengqi Dong, Fenghe Tang |
U-Bench is a large-scale, statistically rigorous benchmark evaluating 100 U-Net variants across 28 datasets and 10 modalities, introducing U-Score to balance performance and efficiency. The main objective is to provide a fair and comprehensive comparison of U-Net variants in medical image segmentation, addressing gaps in prior evaluations regarding statistical robustness, zero-shot generalization, and computational efficiency. The methodology involves evaluating 100 U-Net variants on diverse 2D medical image segmentation datasets, calculating statistical significance, and assessing zero-shot generalization, while introducing U-Score which combines IoU, parameters, FLOPs, and FPS. Primary results show marginal in-domain IoU gains (average 1%-2%) but more pronounced zero-shot improvements (over 3% on average in 80% of modalities), with U-Score improvements averaging 33%. For AI practitioners, U-Bench provides open-source resources and a model advisor agent to guide model selection based on dataset characteristics and resource constraints, highlighting the critical role of efficiency for real-world deployment. |
| Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era |
|
|
| of Large Language Models (Read more on arXiv or HuggingFace) |
|
This paper surveys code-switched (CSW) NLP in the era of Large Language Models. The objective is to comprehensively analyze how LLMs have reshaped CSW modeling, identify key advancements, and highlight persistent challenges. The authors conducted a comprehensive literature review of 308 studies, categorizing them into five research areas across 12 NLP tasks, 30+ datasets, and 80+ languages. While LLMs have shown progress, multilingual NLU models can suffer up to a 15% drop in semantic accuracy, and ASR systems exhibit 30-50% higher word error rates on CSW data, although instruction tuning with models like COMMIT has achieved up to 32x gains in exact match for Hinglish QA. AI practitioners should prioritize developing inclusive datasets, fair evaluation metrics, and linguistically grounded models to achieve robust, truly multilingual AI systems. |
| NorMuon: Making Muon more efficient and scalable (Read more on arXiv or HuggingFace) |
Tuo Zhao, Weizhu Chen, Chen Liang, Liming Liu, Zichong Li |
NorMuon is an optimizer that combines Muon’s orthogonalization with neuron-wise adaptive learning rates for efficient and scalable large language model (LLM) training. The main objective was to determine if orthogonalization and adaptive learning rates could be synergistically combined to yield complementary benefits, addressing the high variance in per-neuron update norms observed in Muon. NorMuon’s methodology involves augmenting Muon’s orthogonalization with neuron-level adaptive learning rates, computed from accumulated second-order momentum statistics, applied as row-wise normalization after orthogonalization, and developed with an efficient distributed implementation under FSDP2. Primary results show NorMuon achieving 21.74% better training efficiency than Adam and an 11.31% improvement over Muon on a 1.1B pretraining setting, while maintaining comparable memory efficiency to Muon. This implies for AI practitioners that orthogonalization and blockwise adaptive learning rates are complementary rather than competing methods, offering superior training dynamics and efficiency for large-scale LLM pretraining. |
| D^3QE: Learning Discrete Distribution Discrepancy-aware |
|
|
| Quantization Error for Autoregressive-Generated Image Detection (Read more on arXiv or HuggingFace) |
Yueqi Duan, Wenzhao Zheng, Yu Zheng, Bingyao Yu, Yanran Zhang |
D³QE proposes a novel framework for detecting autoregressive (AR)-generated images by analyzing discrete distribution discrepancies. The primary objective is to exploit distinctive codebook utilization patterns and frequency distribution biases between real and AR-generated images. D³QE leverages a VQVAE encoder for quantization error features, a Discrete Distribution Discrepancy-Aware Transformer (D³AT) that integrates dynamic codebook frequency statistics into its attention mechanism, and fuses these with CLIP semantic features. The method achieved a superior average accuracy of 82.11% and average precision of 92.07% on the ARForensics dataset, outperforming baselines and demonstrating strong generalization across GANs and diffusion models. This provides AI practitioners with a robust and generalizable tool for synthetic image detection, addressing new challenges posed by advanced autoregressive generative models. |
| DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for |
|
|
| Autonomous Travel Planning Agents (Read more on arXiv or HuggingFace) |
|
DeepTravel is an end-to-end agentic reinforcement learning framework for autonomous travel planning agents. The primary objective is to build autonomous travel planning agents capable of autonomously planning, executing tools, and reflecting on responses in multi-step reasoning. Key methodologies include a Robust SandBox for simulated real-world tool interactions, a Hierarchical Reward Modeling system with trajectory- and turn-level verifiers, and a Reply-Augmented Reinforcement Learning method utilizing SFT cold-start and experience replay. DeepTravel-32B achieved a 69.34% final pass rate on offline (without constraint) hard tasks, significantly outperforming DeepSeek-R1 (26.00%) and OpenAI-03 (21.19%). This framework enables small-size LLMs to achieve state-of-the-art performance in travel planning, providing AI practitioners a more efficient and accessible paradigm for developing autonomous agents for complex, real-world tasks. |
| Heptapod: Language Modeling on Visual Signals (Read more on arXiv or HuggingFace) |
|
Heptapod is an image autoregressive model that applies language modeling principles to visual signals using a novel next 2D distribution prediction objective. Its main objective is to overcome challenges in transferring 1D language modeling to the 2D visual domain by eschewing reliance on CFG and semantic tokenizers. Heptapod employs a causal Transformer with a reconstruction-focused visual tokenizer, learning to predict the distribution over the entire 2D spatial grid at each timestep, thereby unifying autoregressive modeling and masked autoencoding. On the ImageNet generation benchmark, Heptapod-H achieves an FID of 2.70, significantly outperforming previous causal autoregressive models like LlamaGen-3B (FID 9.38) with fewer parameters. This work demonstrates that visual semantics can intrinsically emerge from a well-posed generative objective, providing AI practitioners a principled framework for integrating visual generative training into multimodal LLMs without external semantic engineering. |
Papers for 2025-10-08
| Title |
Authors |
Summary |
| TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
|
The paper presents TATTOO, a tool-grounded Process Reward Model (PRM) designed to provide reliable step-level supervision for large reasoning models (LRMs) in tabular reasoning. The primary research objective is to determine how to provide reliable step-level supervision for advanced LRMs to overcome performance bottlenecks in table-specific operations like sub-table retrieval and schema interaction. The methodology involves a dual-stage training paradigm: first, supervised fine-tuning on a curated dataset of over 60k instances with tool-integrated verification rationales, followed by reinforcement learning with a tool-grounded reward shaping scheme to align the model with table-based verification. Across five challenging tabular reasoning benchmarks, the 8B parameter TATTOO model improves downstream policy LRM performance by an average of 30.9% at inference, surpassing much larger baselines like the 72B Qwen-2.5-Math-PRM. For AI practitioners, the principal implication is that developing specialized, tool-augmented PRMs for structured domains is a critical strategy for enhancing reasoning fidelity, enabling more parameter-efficient models to achieve state-of-the-art performance. |
| Fathom-DeepResearch: Unlocking Long Horizon Information Retrieval and |
|
|
| Synthesis for SLMs (Read more on arXiv or HuggingFace) |
|
This paper introduces Fathom-DeepResearch, a dual-model 4B parameter agentic system that addresses training instabilities in long-horizon, tool-augmented reasoning for small language models. The core methodology employs Reward Aware Policy Optimization (RAPO), a stabilized variant of GRPO using dataset pruning and replay buffers, alongside a steerable step-level reward function to train a search agent, which is paired with a synthesizer model trained on a synthetic corpus to generate citation-dense reports. The system achieves state-of-the-art results on multiple DeepSearch benchmarks, with the Fathom-Search-4B model scoring 90.0% on SimpleQA, outperforming both open and closed-source baselines. For AI practitioners, this work provides a validated training recipe and architecture for developing capable, tool-using SLM agents that avoid common RL instabilities like reward hacking and can perform complex multi-step web research. |
| Fast-dLLM v2: Efficient Block-Diffusion LLM (Read more on arXiv or HuggingFace) |
|
The paper introduces Fast-dLLM v2, a block diffusion language model that adapts pretrained autoregressive models to achieve efficient, parallel text generation. The research objective is to overcome the inherent sequential decoding inefficiency of autoregressive (AR) LLMs by developing a block diffusion model that can be fine-tuned from a pretrained AR model with minimal data, while maintaining or improving performance. The key methodology involves a novel training recipe that combines a block diffusion mechanism with a complementary attention mask to enable block-wise bidirectional context modeling, and a hierarchical caching mechanism with block-level and sub-block caches to accelerate parallel decoding during inference. Primary results show that the 7B parameter Fast-dLLM v2 achieves a 2.54× higher throughput than the original Qwen2.5-7B-Instruct AR model on the GSM8K benchmark while offering comparable accuracy. The principal implication for AI practitioners is that this method provides a practical and data-efficient (~1B fine-tuning tokens) approach to convert existing high-performance AR models into significantly faster parallel decoders, making them more viable for deployment in latency-sensitive applications. |
| CoDA: Coding LM via Diffusion Adaptation (Read more on arXiv or HuggingFace) |
|
The paper introduces CoDA, a 1.7B-parameter diffusion language model for code generation, trained via a multi-stage adaptation process. The primary objective is to develop a compact, efficient diffusion coder that is competitive with larger autoregressive and diffusion models, particularly for tasks requiring bidirectional context and infilling. The methodology involves adapting the Qwen3-1.7B backbone through large-scale diffusion pre-training, code-centric mid-training using a progressive masking schedule, and subsequent instruction tuning. CoDA-1.7B-Instruct achieves a pass@1 score of 63.2% on the MBPP-Plus benchmark, surpassing the larger Dream-7B-Instruct model. For AI practitioners, this research demonstrates that smaller-scale diffusion models can be a viable alternative to heavyweight autoregressive systems for coding assistants, offering competitive performance and inherent infilling capabilities with lower latency. |
| Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Zhuoshi Pan, Xin Gao, Qizhi Pei, Honglin Lin, apeters |
This paper presents Caco (Code-Assisted Chain-of-ThOught), a framework for automatically synthesizing large-scale, verifiable, and diverse instruction and Chain-of-Thought (CoT) reasoning data using a code-driven augmentation pipeline. The primary objective is to overcome the scalability and verifiability limitations of natural language CoT by creating a fully automated framework for generating instruction-CoT pairs grounded in executable code. The methodology involves a closed-loop process: fine-tuning a “CodeGen” model on a unified corpus of seed solutions, generating new code-based CoTs at scale, verifying them via execution and filtering, and then reverse-engineering these validated code traces into natural language instructions and CoTs. Models fine-tuned on the resulting 1.3M-sample Caco dataset demonstrate superior performance, with the Caco-trained Qwen2.5-Math-7B model achieving an average accuracy of 67.7% across six mathematical reasoning benchmarks, outperforming strong baselines. For AI practitioners, this work provides a scalable method for creating verifiably correct instruction-tuning data for complex reasoning without human intervention, enabling the development of more trustworthy and generalizable LLMs. |
| ASPO: Asymmetric Importance Sampling Policy Optimization (Read more on arXiv or HuggingFace) |
Xiu Li, Wenping Hu, Lei Lin, Jiakang Wang, RyanLiu112 |
ASPO is a policy optimization algorithm that corrects a token-weight mismatch in Outcome-Supervised Reinforcement Learning (OSRL) by asymmetrically flipping the importance sampling ratios for positive-advantage tokens. The primary objective is to address a flaw in GRPO-based OSRL where the standard clipping mechanism disproportionately weights high-probability positive-advantage tokens, leading to training instability, entropy collapse, and premature convergence. The key methodology, Asymmetric Importance Sampling Policy Optimization (ASPO), involves inverting the importance sampling (IS) ratio for positive-advantage tokens to assign larger update weights to less confident tokens, and incorporating a soft dual-clipping mechanism to stabilize extreme updates. On mathematical reasoning benchmarks, ASPO achieved an average score of 59.3, outperforming the strong GRPO-based baseline DAPO, which scored 53.5, and improving on the base model by 12.5%. The principal implication for AI practitioners is that when using GRPO-style OSRL for LLM fine-tuning, standard IS ratios can function as a misaligned weighting scheme; applying ASPO’s asymmetric weighting corrects this, leading to more stable training and enhanced model performance. |
| TensorBLEU: Vectorized GPU-based BLEU Score Implementation for |
|
|
| Per-Sentence In-Training Evaluation (Read more on arXiv or HuggingFace) |
|
This paper introduces TensorBLEU, a fully vectorized, memory-efficient GPU implementation for per-sentence Token-ID BLEU calculation designed for in-training evaluation. The objective is to eliminate the computational bottleneck of CPU-based BLEU calculation within training loops, particularly for Reinforcement Learning (RL) fine-tuning where per-sentence reward signals are required. The key methodology is a memory-efficient n-gram counting mechanism that leverages torch.unfold for parallel n-gram extraction and torch.unique to create a compact, batch-specific n-gram dictionary, enabling parallel counting via a “batched bincount” technique. Primary results show that TensorBLEU achieves speedups of over 40x on an NVIDIA A100 GPU (for a batch size of 256 with 1024-token sequences) compared to the standard CPU-based NLTK implementation. The principal implication for AI practitioners is that this tool transforms the calculation of BLEU-based rewards from a major training bottleneck into a negligible overhead, making large-scale RL fine-tuning of language models with dense reward signals computationally feasible. |
| Mixing Mechanisms: How Language Models Retrieve Bound Entities |
|
|
| In-Context (Read more on arXiv or HuggingFace) |
|
This paper demonstrates that language models use a dynamic mixture of positional, lexical, and reflexive mechanisms to retrieve bound entities in-context, moving beyond the previously held view of a purely positional approach. The main objective is to mechanistically investigate how language models bind and retrieve entities, especially in long contexts where the simple positional mechanism becomes unreliable. The study employs interchange interventions with specially designed pairs of original and counterfactual inputs to isolate the causal contributions of the three distinct mechanisms, using the results to train a quantitative causal model. The primary result is that this three-mechanism model predicts the language model’s next-token distributions with 95% agreement (Jensen-Shannon Similarity), vastly outperforming a model based on the prevailing positional-only view (44% JSS). For AI practitioners, this research provides a mechanistic explanation for the “lost-in-the-middle” effect, implying that the reliability of entity retrieval in long-context applications like RAG is non-uniform and depends on the entity’s position, which should inform prompt design and error analysis. |
| AInstein: Assessing the Feasibility of AI-Generated Approaches to |
|
|
| Research Problems (Read more on arXiv or HuggingFace) |
Jose Dolz, Laurent Charlin, Marco Pedersoli, Gaurav Sahu, Shambhavi Mishra |
The paper introduces AInstein, a framework to evaluate whether Large Language Models (LLMs) can autonomously solve novel AI research problems using only their pretrained parametric knowledge. The central objective is to determine if LLMs can generate valid, technical solutions to research challenges extracted from scientific abstracts without external aids like fine-tuning or retrieval augmentation. The methodology uses a “Generalizer” LLM to distill a research problem from an ICLR 2025 paper abstract and a “Solver” LLM to propose a solution, both refined through iterative critique loops, with evaluation performed on 1,214 papers using an LLM-as-a-judge. Results show that while perfect rediscovery of human solutions is rare (dropping from ~84% to ~19% for the top model under stricter criteria), LLMs frequently generate novel, valid alternatives, with the best-performing agent achieving a strict Success Rate of 74.05%. For AI practitioners, this implies that state-of-the-art LLMs can serve as creative problem-solving partners, capable of generating technically sound and original approaches to engineering challenges, extending their utility beyond information retrieval to genuine solution ideation. |
| Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning? (Read more on arXiv or HuggingFace) |
|
This paper mechanistically investigates the “refusal cliff,” a safety failure where reasoning models’ internal refusal intentions are suppressed just before output generation by specific attention heads. The central research objective is to identify the underlying mechanism that makes safety alignment vulnerable in large reasoning models. The methodology involves using a linear probe to quantify refusal scores from hidden states across token positions, followed by causal tracing and ablation of attention heads to isolate components negatively impacting refusal. The primary result demonstrates that ablating a sparse set of “Refusal Suppression Heads,” comprising just 3% of identified heads, reduces attack success rates to below 10%; a proposed data selection method achieves comparable safety performance using only 1.7% of the original training data. For AI practitioners, this implies that model safety can be significantly and efficiently improved through targeted interventions, specifically by identifying and mitigating the effect of these suppressive heads or by using computationally cheap probes to select high-impact data for fine-tuning. |
| HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video (Read more on arXiv or HuggingFace) |
Katelyn Gao, Quentin Leboutet, Hao-Yu Hsu, Chih-Hao Lin, Hongchi Xia |
HoloScene is a framework for reconstructing simulation-ready, interactive 3D worlds from a single video using an energy-based optimization of a scene-graph representation. The primary objective is to create a 3D digital twin that is geometrically complete, physically plausible, interactive, and photorealistically rendered, overcoming limitations of prior methods. The key methodology involves representing the scene as an interactive scene graph with nodes for object geometry (neural SDFs), appearance (Gaussian splats), and physical properties, which is then reconstructed via a three-stage hybrid optimization: gradient-based initialization, generative sampling with tree search for physical plausibility and shape completion, and a final refinement stage. HoloScene significantly improves physical stability, achieving a physics failure rate of 18.3% on the Replica dataset, compared to rates of over 90% for baseline methods like PhyRecon and DP-Recon. For AI practitioners, this work provides a method to automate the creation of high-fidelity, physically interactive virtual environments from video, which can accelerate the development of simulators for robotics, autonomous systems, and interactive applications like gaming and AR/VR. |
| Margin Adaptive DPO: Leveraging Reward Model for Granular Control in |
|
|
| Preference Optimization (Read more on arXiv or HuggingFace) |
sirano1004 |
This paper introduces Margin-Adaptive Direct Preference Optimization (MADPO), a method that uses a reward model to apply an instance-level adaptive weight to the DPO loss. The primary objective is to overcome the limitations of DPO’s fixed temperature parameter by developing a stable, instance-level regularization approach that avoids overfitting on easy examples and under-learning from hard ones. The methodology involves a two-step process: first, training a reward model to estimate preference margins, and second, using these margins to re-weight the DPO loss for each sample, amplifying the learning signal for hard pairs and dampening it for easy pairs. On a sentiment generation task, MADPO achieved performance gains of up to +33.3% on High Quality data and +10.5% on Low Quality data over the next-best baseline. For AI practitioners, this method provides a principled and more robust approach to preference alignment, enabling granular control over the training objective by leveraging an external reward model, which is particularly effective for datasets with diverse or noisy preference signals. |
| Discrete Diffusion Models with MLLMs for Unified Medical Multimodal |
|
|
| Generation (Read more on arXiv or HuggingFace) |
|
MeDiM is a novel discrete diffusion model that leverages a Multimodal Large Language Model (MLLM) to unify the generation of medical images, reports, and joint image-report pairs within a single framework. The research objective is to create a versatile generative foundation model for medicine that learns shared distributions across modalities, overcoming the limitations of modality-specific systems. The key methodology involves adapting a pre-trained autoregressive MLLM as the backbone for a discrete diffusion process by removing its causal attention mask to enable bidirectional context and injecting timestep embeddings via adaptive layer normalization (AdaLN). The model achieves state-of-the-art results, including a Fréchet Inception Distance (FID) of 16.60 on MIMIC-CXR for image generation, and demonstrates that its synthetic image-report pairs can improve downstream task performance by up to +4.80% in METEOR score. For AI practitioners, the principal implication is that autoregressive MLLMs can be effectively repurposed for high-performance discrete diffusion models, enabling the development of unified systems that can generate consistent, multimodal data for augmenting training sets in specialized domains. |
| MixReasoning: Switching Modes to Think (Read more on arXiv or HuggingFace) |
|
MixReasoning is an inference-time framework that improves the efficiency of reasoning models by dynamically adjusting the depth of thought within a single response. The main objective is to reduce the computational cost and redundancy of long Chain-of-Thought (CoT) by adaptively allocating detailed reasoning only to pivotal, high-difficulty steps. The key methodology involves using a lightweight LoRA adapter to enable a “concise” mode and monitoring token-level uncertainty (entropy) to trigger a temporary switch to a detailed “thinking” mode for regenerating high-uncertainty segments. The primary result shows that on the GSM8K benchmark with the QwQ-32B-Preview model, MixReasoning reduced token usage by 47% (from 750.3 to 400.5) while simultaneously improving accuracy by 1.01%. The principal implication for AI practitioners is that this single-model technique can be deployed to significantly lower latency and inference costs for reasoning tasks, offering a controllable trade-off between efficiency and response verbosity without requiring complex architectural changes or retraining of the base model. |
| LightCache: Memory-Efficient, Training-Free Acceleration for Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Zheng Zhan, Yushu Wu, Kaiyuan Deng, Gen Li, Yang Xiao |
LightCache is a training-free framework that accelerates video diffusion model inference while reducing peak memory usage through stage-specific optimizations. The paper’s objective is to mitigate the substantial GPU memory increase caused by existing cache-based acceleration methods in video generation without requiring model retraining. The key methodology involves decomposing inference into encoding, denoising, and decoding stages and applying three targeted techniques: asynchronous swapping of cached features to CPU, spatial chunking of feature maps during denoising, and slicing latent tensors for sequential frame-by-frame VAE decoding. For the Stable-Video-Diffusion-Img2vid-XT model, LightCache achieves a 2.86x inference speedup while simultaneously reducing peak memory usage by 1.4 GB compared to the baseline. The principal implication for AI practitioners is that this method enables the deployment of large video generation models on hardware with constrained GPU memory, as it concurrently reduces memory footprint and latency, making it a highly efficient solution for production environments. |
| Demystifying deep search: a holistic evaluation with hint-free multi-hop |
|
|
| questions and factorised metrics (Read more on arXiv or HuggingFace) |
|
This paper introduces WebDetective, a benchmark with hint-free multi-hop questions and a factorised evaluation framework to diagnose autonomous reasoning failures in web agents. The primary objective is to evaluate how well models can autonomously discover reasoning chains for deep search tasks, as opposed to simply executing paths hinted at in the question. The methodology involves a co-designed benchmark consisting of hint-free questions and a controlled Wikipedia sandbox to ensure full traceability, paired with a holistic evaluation framework that separates search sufficiency, knowledge utilisation, and refusal behaviour. The primary result from evaluating 25 models is a consistent and significant gap between information retrieval and synthesis; for instance, the o3-Pro model achieved a 78% Search Score but only a 20.86% Generation Score, leading to a 56.0% final Pass@1 rate. The principal implication for AI practitioners is that improving multi-hop agents requires focusing on architectures for robust information synthesis and calibrated refusal, as the current bottleneck is not evidence discovery but its composition into a correct answer when the reasoning path is not provided. |
| EgoNight: Towards Egocentric Vision Understanding at Night with a |
|
|
| Challenging Benchmark (Read more on arXiv or HuggingFace) |
Tianwen Qian, Yang Miao, Runyi Yang, Yuqian Fu, dehezhang2 |
This paper introduces EgoNight, the first comprehensive benchmark to evaluate egocentric vision understanding models specifically under nighttime conditions using day-night aligned videos. The primary objective is to systematically investigate and quantify the performance gap of Multimodal Large Language Models (MLLMs) when transitioning from well-lit daytime scenarios to challenging low-light environments. The methodology involves the creation of a new dataset with synthetic and real-world aligned day-night video pairs, and the construction of three benchmark tasks: egocentric Visual Question Answering (VQA), day-night correspondence retrieval, and depth estimation. Experiments reveal that state-of-the-art models struggle significantly, showing an average performance decline of 32.8% on paired VQA tasks for the synthetic dataset when moving from day to night conditions. The principal implication for AI practitioners is that current MLLMs are not robust to illumination changes, and systems intended for real-world egocentric applications must be explicitly designed and validated for reliable performance in low-light and nighttime settings. |
| BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language |
|
|
| Models via Lens of Dynamic Interactions (Read more on arXiv or HuggingFace) |
Shipei Lin, Per Jacobsson, Jinyang Li, Xiaohan Xu, Nan Huo |
BIRD-INTERACT introduces a comprehensive benchmark to evaluate large language models on dynamic, multi-turn Text-to-SQL tasks that mirror real-world database interactions. The main objective is to assess an LLM’s ability to handle ambiguous queries, execution errors, and evolving user needs in a stateful, interactive environment, moving beyond static, single-turn evaluation. The methodology involves a benchmark suite with two settings: c-Interact (protocol-guided conversation) and a-Interact (autonomous agentic interaction), coupled with an environment featuring a knowledge base and a two-stage, function-driven user simulator to ensure robust and reproducible evaluation. Primary results demonstrate the benchmark’s difficulty, with the flagship model GPT-5 achieving only an 8.67% task completion success rate in the c-Interact setting and 17.00% in the a-Interact setting on the full task suite. The principal implication for AI practitioners is that optimizing for single-turn SQL generation is insufficient; focus must shift to developing strategic interaction capabilities, such as effective clarification, dialogue state management, and error recovery, which are the current bottlenecks for deploying robust database assistants. |
| VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation (Read more on arXiv or HuggingFace) |
|
VeriGuard is a novel framework that provides formal, provable safety guarantees for LLM agents by ensuring their actions comply with predefined constraints. Its methodology employs a dual-stage architecture: an offline stage uses an iterative loop of LLM-based generation, automated testing, and formal verification to create a “correct-by-construction” policy, which is then used in an online stage for lightweight runtime action monitoring. The primary result on the Agent Security Bench (ASB) shows VeriGuard reduces the Attack Success Rate (ASR) to 0.0% across all evaluated attack types while maintaining high task utility. The principal implication for AI practitioners is the provision of a structured, verifiable framework to enforce complex safety policies, enabling the deployment of agents in high-stakes domains by moving beyond reactive, pattern-based guardrails to a provably-sound safety paradigm. |
| CARE: Cognitive-reasoning Augmented Reinforcement for Emotional Support |
|
|
| Conversation (Read more on arXiv or HuggingFace) |
|
The CARE framework augments Emotional Support Conversation (ESC) models with explicit, multi-step cognitive reasoning chains refined via reinforcement learning. The primary objective is to enhance the reasoning capabilities and supportive quality of ESC models without relying on large-scale synthetic data generation. The methodology involves a two-stage process: first, supervised fine-tuning (SFT) on the original ESConv dataset enriched with distilled four-step reasoning chains (Context, Cognition, Emotion, Support Plan), followed by reinforcement learning (RL) using the GRPO algorithm to optimize reasoning consistency and accuracy. The final CARE model achieved state-of-the-art results, increasing support strategy accuracy to 30.29 from the 26.36 of the baseline ESConv model. The principal implication for AI practitioners is that high-quality, specialized datasets can be effectively leveraged to create structured reasoning data through model-based distillation, providing a more robust and interpretable alternative to large-scale synthetic data augmentation for building cognitively-aware dialogue systems. |
| CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical |
|
|
| Contrastive Decoding (Read more on arXiv or HuggingFace) |
|
This paper introduces Clinical Contrastive Decoding (CCD), a training-free and retrieval-free inference framework to mitigate medical hallucinations in radiology Multimodal Large Language Models (MLLMs). The main objective is to reduce clinically unsupported descriptions generated by radiology MLLMs by addressing their over-sensitivity to clinical prompts, without modifying the base model. CCD’s methodology involves a dual-stage contrastive mechanism that leverages an external, task-specific expert model to extract structured clinical labels and probability scores, which are then used to refine the MLLM’s token-level logits during generation. On the MIMIC-CXR dataset, applying CCD to a state-of-the-art model resulted in a 17% improvement in RadGraph-F1, demonstrating enhanced clinical fidelity. The principal implication for AI practitioners is that lightweight, domain-specific expert models can be integrated at inference-time to guide large foundation models, providing a generalizable and training-free approach to improve domain-specific accuracy and reduce hallucinations. |
| No Tokens Wasted: Leveraging Long Context in Biomedical Vision-Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Xiao Xiao Sun, Vishwesh Nath, Javier Gamazo Tejero, Alejandro Lozano, Min Woo Sun |
This paper introduces BMC-LongCLIP, a biomedical vision-language model trained with an extended text context of up to 512 tokens to leverage long-format captions and reduce token waste. The main objective is to investigate the impact of pretraining VLMs with longer context windows on downstream biomedical retrieval and classification performance. The methodology involves pretraining a CLIP-based model with a ViT-L/14 vision encoder and a long-context BioClinical ModernBERT text encoder on the BIOMEDICA dataset, while also introducing a new long-caption dataset, BIOMEDICA-LongCAP. Extending the context length from 77 to 512 tokens reduced token waste from 55% to 2.2% and achieved up to a +30% absolute gain in Recall@1 on long-caption retrieval benchmarks. The principal implication for AI practitioners is that increasing the text encoder’s context length during pretraining is a critical technique for improving VLM performance in domains with long, descriptive text by utilizing previously discarded supervisory signals. |
| Equilibrium Matching: Generative Modeling with Implicit Energy-Based |
|
|
| Models (Read more on arXiv or HuggingFace) |
|
This paper introduces Equilibrium Matching (EqM), a generative modeling framework that learns a time-invariant equilibrium gradient of an implicit energy landscape for optimization-based sampling. The objective is to develop a generative model that avoids the non-equilibrium, time-conditional dynamics of diffusion/flow models to enable more flexible inference. EqM is trained to predict a target gradient (ε - x)c(γ) that points from noise ε to data x, with a magnitude c(γ) that decays to zero at the data manifold, allowing samples to be generated via gradient descent on the learned landscape. The method achieves a state-of-the-art Fréchet Inception Distance (FID) of 1.90 on class-conditional ImageNet 256×256 generation, outperforming comparable flow-based models. For AI practitioners, EqM provides a route to replace fixed-horizon integrators with optimization-based samplers, enabling the use of adaptive step sizes, adaptive compute, and advanced optimizers like Nesterov Accelerated Gradient for more controllable and potentially more efficient inference. |
| Human3R: Everyone Everywhere All at Once (Read more on arXiv or HuggingFace) |
Yuliang Xiu, Anpei Chen, Yuxuan Xue, Xingyu Chen, Yue Chen |
Human3R is a unified, real-time, feed-forward framework that jointly reconstructs multi-person 3D human meshes, dense 3D scenes, and camera trajectories from a single monocular video stream. The objective is to develop a unified, one-stage, online model for 4D human-scene reconstruction that eliminates dependencies on pre-processing modules (e.g., SLAM, human detection) and iterative refinement, enabling real-time performance. The methodology involves fine-tuning the 4D reconstruction foundation model CUT3R using parameter-efficient visual prompt tuning (VPT); human head tokens are detected from image features, augmented with human priors from a pre-trained Multi-HMR ViT encoder, and projected into human prompts that are processed by the frozen CUT3R decoder to regress multi-person SMPL-X parameters in a single forward pass. Human3R achieves state-of-the-art or competitive performance across multiple 4D reconstruction tasks with a single model; on the EMDB-2 dataset for global human motion estimation, it achieves a 20% lower W-MPJPE and 60% lower Root Translation Error (RTE) compared to the prior online state-of-the-art method WHAM, while operating at 15 FPS. For AI practitioners, Human3R provides a lightweight, dependency-free, and real-time model that can replace complex multi-stage pipelines for 4D human-scene reconstruction, simplifying deployment and enabling online applications in AR/VR, autonomous navigation, and humanoid policy learning with a low memory footprint (8 GB). |
| Training Dynamics Impact Post-Training Quantization Robustness (Read more on arXiv or HuggingFace) |
Jonas Geiping, Niccolò Ajroldi, Albert Catalan-Tatjer |
This paper demonstrates that training dynamics, particularly learning rate decay, are a primary driver of post-training quantization (PTQ) error in large language models, challenging the prevailing notion that dataset scale is the dominant factor. The research objective is to investigate and identify interventions that modulate the relationship between training hyperparameters and PTQ robustness. The methodology combines analysis of quantization error across hundreds of checkpoints from six major open-source LLM families with controlled pretraining experiments (up to 100B tokens) to isolate the effects of learning rate schedules, weight decay, and weight averaging. The primary result is that quantization error spikes coincide with learning rate decay, largely independent of the training data scale; for example, controlled experiments show that LAtest Weight Averaging (LAWA) can match or even surpass the 3-bit PTQ performance of models trained with a full learning rate decay schedule. The principal implication for AI practitioners is that PTQ robustness should be an active criterion during hyperparameter tuning, as strategic interventions like favoring higher learning rates or employing weight averaging can produce models that are inherently more robust to low-bit quantization. |
| ShapeGen4D: Towards High Quality 4D Shape Generation from Videos (Read more on arXiv or HuggingFace) |
Sergey Tulyakov, Jiaxu Zou, Jianqi Chen, Ashkan Mirzaei, Jiraphon Yenphraphai |
ShapeGen4D is a feedforward framework that directly generates high-quality, dynamic 3D mesh sequences from a single monocular video. The objective is to develop a native video-to-4D model that overcomes the instabilities of score distillation sampling and the error accumulation inherent in two-stage multi-view reconstruction pipelines. The methodology extends a pre-trained 3D diffusion transformer by incorporating spatiotemporal attention layers, creating temporally-aligned latents via warped query points, and enforcing temporal stability by sharing the same diffusion noise across all frames. The method demonstrates superior geometric accuracy over baselines on the Objaverse test set, achieving a 0.3276 Intersection over Union (IoU) score. For AI practitioners, this work provides a data-efficient strategy for 4D content generation by showing that fine-tuning large, pre-trained static 3D models with targeted temporal modifications is more effective than training from scratch or using complex optimization pipelines. |
| Deforming Videos to Masks: Flow Matching for Referring Video |
|
|
| Segmentation (Read more on arXiv or HuggingFace) |
Chengzu Li, Sizhe Dang, Liuzhuozheng Li, Dengyang Jiang, Zanyi Wang |
This paper presents FlowRVS, a framework that reformulates Referring Video Object Segmentation (RVOS) as a continuous, text-conditioned flow process that deforms a video’s latent representation into a target mask. The primary objective is to overcome the information bottlenecks and temporal inconsistencies of traditional ‘locate-then-segment’ pipelines by creating a unified, end-to-end model. The key methodology involves modeling RVOS with an Ordinary Differential Equation (ODE) to learn a velocity field that transforms a video latent from a pretrained Text-to-Video model to a mask latent, stabilized by start-point focused adaptations including Boundary-Biased Sampling (BBS) and Direct Video Injection (DVI). FlowRVS achieves new state-of-the-art results, attaining a J&F score of 73.3 on the zero-shot Ref-DAVIS17 benchmark, surpassing the prior SOTA by 2.7 points. The principal implication for AI practitioners is that complex video understanding tasks can be effectively framed as continuous deformation processes, enabling the direct adaptation of powerful pretrained generative models for discriminative objectives by focusing learning on the flow’s initial, high-certainty conditions. |
| Distributional Semantics Tracing: A Framework for Explaining |
|
|
| Hallucinations in Large Language Models (Read more on arXiv or HuggingFace) |
Jacobo Azcona, Kevin Allan, Somayajulu G Sripada, gagan3012 |
This paper introduces Distributional Semantics Tracing (DST), a unified framework that mechanistically explains LLM hallucinations by tracing internal representational drift to a specific “commitment layer” where failures become irreversible. The main objective is to diagnose how, when, and why hallucinations occur by treating them as predictable failures arising from the Transformer architecture, specifically from a conflict between distinct computational pathways. The DST methodology integrates causal tracing, patching, and subsequence analysis to build layer-wise semantic networks and uses a novel metric, Distributional Semantics Strength (DSS), to quantify the coherence of the model’s contextual reasoning. The primary result is the identification of a “Reasoning Shortcut Hijack” failure mode and a strong negative Pearson correlation of -0.863 between the contextual pathway’s coherence (measured by DSS) and hallucination rates. For AI practitioners, this reframes hallucination mitigation from post-hoc correction to proactive diagnosis, providing a method to identify the specific commitment layer where a failure solidifies, thus creating a concrete target for architectural interventions and improving model reliability. |
| Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI |
|
|
| Models for Scatterplot-Related Tasks (Read more on arXiv or HuggingFace) |
Pedro Bizarro, Rita Costa, Diogo Duarte, joaompalmeiro |
This paper introduces a synthetic dataset and benchmark to evaluate proprietary AI models on visual scatterplot analysis tasks. The objective is to systematically assess model capabilities in cluster and outlier counting, detection, and identification, addressing a gap in existing benchmarks. The authors generated a dataset of over 18,000 annotated scatterplots with varied designs and evaluated 10 models from OpenAI and Google using zero-shot, one-shot, and few-shot prompting strategies across five distinct tasks. Results show that while few-shot prompting enables high performance on counting tasks (e.g., Gemini 2.5 Flash achieved over 90% accuracy in outlier counting), performance on localization tasks is poor, with Precision and Recall generally below 50%, except for Flash which reached 65.01% Recall for outlier identification. The principal implication for AI practitioners is to use few-shot prompting for scatterplot analysis and to apply current models primarily to counting tasks, as their performance on precise localization tasks is unreliable. |
| In-the-Flow Agentic System Optimization for Effective Planning and Tool |
|
|
| Use (Read more on arXiv or HuggingFace) |
Jianwen Xie, Sheng Liu, Seungju Han, Haoxiang Zhang, Zhuofeng Li |
This research introduces AGENTFLOW, a trainable agentic framework, and Flow-GRPO, an on-policy RL algorithm, to optimize long-horizon planning and tool use by training a planner module within a live multi-turn interaction loop. The main objective is to overcome the limitations of monolithic RL models and static agentic systems by developing a method for effective on-policy learning in a multi-module agentic system facing long-horizon, sparse-reward credit assignment challenges. The key methodology is AGENTFLOW, a four-module (planner, executor, verifier, generator) system with an evolving memory, which is trained using Flow-based Group Refined Policy Optimization (Flow-GRPO); this algorithm converts multi-turn optimization into single-turn updates by broadcasting a final trajectory-level reward to every step. The primary result shows that the 7B-parameter AGENTFLOW system significantly outperforms specialized baselines, achieving an average accuracy gain of 14.9% on search tasks over the top-performing baseline. The principal implication for AI practitioners is that this in-the-flow optimization approach provides a scalable and stable method to train modular agentic systems for complex, long-horizon tasks, enabling the development of more reliable and adaptive agents that learn directly from final outcomes without requiring complex intermediate reward shaping. |
| A Contextual Quality Reward Model for Reliable and Efficient Best-of-N |
|
|
| Sampling (Read more on arXiv or HuggingFace) |
sirano1004 |
This paper presents a reward model that learns contextual acceptability, not just relative preference, by incorporating an “outside option” into the training data. The main objective is to develop a reward model and an associated inference strategy that can distinguish “good enough” responses from merely “better” ones, thereby mitigating the failure mode of standard Best-of-N (BoN) sampling where the least bad of many poor options is selected. The key methodology is to train a reward model on preference data augmented with a “reject all” option, based on a discrete choice framework, and then use this model in a “best of mini-N in-loop” adaptive inference strategy with a calibrated early-exit condition. Experiments show that when configured as an alignment guardrail, this method reduces reliability failures by 70% compared to standard BoN; when configured as an inference accelerator, it improves average inference speed by over 22%. The principal implication for AI practitioners is a tunable framework to explicitly manage the trade-off between reliability and computational efficiency, allowing them to configure systems for either maximum safety or maximum speed based on application requirements. |
| DRIFT: Learning from Abundant User Dissatisfaction in Real-World |
|
|
| Preference Learning (Read more on arXiv or HuggingFace) |
Zheli Liu, Zhaoxuan Tan, Junlin Wu, Bolian Li, AmberYifan |
DRIFT is an iterative preference learning method that improves LLMs by using abundant, real-world user dissatisfaction signals as negatives and dynamically sampling positives from the current policy. The main objective is to develop a scalable preference learning framework that leverages the naturally abundant signal of user dissatisfaction from real-world LLM interactions, avoiding reliance on costly curated positive examples. The methodology, Dissatisfaction-Refined Iterative preFerence Training (DRIFT), constructs preference pairs by using authentic user-dissatisfied responses as rejected examples and iteratively sampling fresh chosen examples from the evolving model policy, then updating the model using the Direct Preference Optimization (DPO) loss. On the synthetic UltraFeedback dataset, a 14B model trained with DRIFT achieved a +7.61% increase in WildBench Task Score and a +12.29% absolute win rate increase on AlpacaEval2 over the base model, outperforming strong baselines like iterative DPO and SPIN. The principal implication for AI practitioners is that they can align deployed LLMs more effectively and scalably by directly using logs of user corrections and complaints as negative examples in a DPO framework, reducing dependence on expensive human annotation for positive preference data. |
| Revisiting Modeling and Evaluation Approaches in Speech Emotion |
|
|
| Recognition: Considering Subjectivity of Annotators and Ambiguity of Emotions (Read more on arXiv or HuggingFace) |
|
This dissertation introduces three novel approaches for speech emotion recognition (SER) that challenge traditional label aggregation methods by incorporating annotator subjectivity, emotion ambiguity, and co-occurrence frequencies. The primary objective is to determine if SER model performance and evaluation can be improved by retaining all human-provided emotional ratings, including minority and non-consensus labels, rather than discarding them through conventional aggregation techniques like majority or plurality voting. The author introduces an “all-inclusive rule” (AR) that uses all annotated data to create distributional ground-truth labels, a rater-modeling approach that fuses representations from individual annotator models, and a penalization matrix integrated into the loss function to penalize the prediction of rare emotional co-occurrences. On the IEMOCAP dataset, the rater-modeling fusion achieved an unweighted average recall (UAR) of 61.48%, a 4.36% absolute improvement over a standard soft-label baseline, while the penalization matrix improved the macro F1-score on the MSP-PODCAST dataset by a relative 25.8% for distribution-label learning tasks. The principal implication for AI practitioners is that they should avoid discarding data that lacks label consensus; utilizing all available annotations to train models on distributional labels and evaluate them on complete, unfiltered test sets leads to more robust systems that better handle the ambiguity inherent in real-world emotional expression. |
| OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit |
|
|
| Flows (Read more on arXiv or HuggingFace) |
|
OneFlow is a non-autoregressive multimodal framework that unifies variable-length text generation via Edit Flows and image synthesis via Flow Matching for concurrent and interleaved generation. The primary objective is to overcome the sequential and fixed-length limitations of existing multimodal models by enabling a flexible, non-causal generation process for both text and images. The methodology combines Edit Flows, a continuous-time Markov chain for discrete token insertion, with continuous Flow Matching for image latents within a single bidirectional Transformer, coordinated by an interleaved time schedule. The primary result shows that OneFlow scales more efficiently than autoregressive models, requiring up to 50% fewer training FLOPs to achieve performance parity on generation benchmarks. For AI practitioners, this model offers a computationally efficient alternative for building systems that can dynamically generate variable-length, interleaved text-and-image content without the constraints of sequential, autoregressive decoding. |
| HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate |
|
|
| Hallucinations in Retrieval-Augmented Generation (Read more on arXiv or HuggingFace) |
Radu State, Jérôme François, Ioana Buhnila, lrsbrgrn |
HalluGuard is a 4B-parameter Small Reasoning Model designed to detect and justify hallucinations in Retrieval-Augmented Generation by classifying document-claim pairs as grounded or hallucinated. The main objective is to develop a computationally efficient and transparent model for mitigating RAG hallucinations, making it suitable for resource-constrained or on-premise enterprise environments where explainability is critical. The key methodology involves creating a synthetic dataset (HalluClaim) from the FineWeb corpus, generating preference pairs using models of different sizes (Qwen3-32B vs. Qwen3-0.6B), applying LLM-based consensus filtering, and fine-tuning a Qwen3-4B backbone with Odds Ratio Preference Optimization (ORPO). The primary result is achieving 84.0% balanced accuracy on the RAGTruth benchmark, matching the performance of larger specialized models like the 7B-parameter MiniCheck while using significantly fewer parameters. The principal implication for AI practitioners is that carefully curated synthetic preference data and advanced alignment techniques like ORPO can be used to distill the reasoning capabilities of large models into smaller, efficient models, enabling the deployment of reliable and auditable AI systems for critical enterprise tasks. |
Papers for 2025-10-07
| Title |
Authors |
Summary |
| Paper2Video: Automatic Video Generation from Scientific Papers (Read more on arXiv or HuggingFace) |
|
i) This research introduces Paper2Video, a benchmark for evaluating academic video generation, and PaperTalker, a multi-agent framework that automates the creation of presentation videos from scientific papers. ii) The primary objective is to automate the labor-intensive process of generating high-quality, multi-modal academic presentation videos by tackling challenges related to long-context understanding of papers and the coordinated synthesis of aligned slides, speech, cursor movements, and a presenter. iii) The key methodology is the PaperTalker framework, a multi-agent pipeline that uses a slide builder to generate and refine LaTeX Beamer code via a novel Tree Search Visual Choice module for layout optimization, a subtitle builder for generating narration, a cursor builder for spatio-temporal grounding, and a talker builder for personalized speech and talking-head video, all processed in parallel on a per-slide basis. iv) On the proposed Paper2Video benchmark, videos generated by PaperTalker proved more effective at conveying detailed information than the original author-recorded videos, achieving a PresentQuiz detail accuracy score of 0.842 compared to 0.738 for human-made videos. v) For AI practitioners, this work provides a modular framework for automating the creation of technical video content, with the Tree Search Visual Choice module presenting a practical technique for using VLMs to refine generated layouts, a common challenge in automated document and UI generation. |
| Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large |
|
|
| Multimodal Models (Read more on arXiv or HuggingFace) |
Zhangyun Tan, Zhenyu Pan, Pinxin Liu, Jing Bi, Yunlong Tang |
This survey provides a comprehensive examination and structured taxonomy of post-training methodologies for enhancing the reasoning capabilities of Video-Large Multimodal Models (Video-LMMs). The primary objective is to systematize the fragmented literature by analyzing three fundamental pillars: supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL) from verifiable objectives, and test-time scaling (TTS). The paper employs a systematic meta-analysis of representative methods to synthesize key design principles, evaluation protocols, and video-specific adaptations for challenges like temporal localization and spatiotemporal grounding. As a survey, it does not produce novel experimental data but highlights a critical trend: the evolution from static CoT-SFT to more robust RL paradigms (e.g., R1-style/GRPO) that enable self-correction using verifiable rewards like temporal IoU, as shown with datasets like MTVR-RL-110k. The principal implication for practitioners is the provision of a unified framework, along with curated benchmarks and datasets, to guide the design, implementation, and rigorous evaluation of post-training pipelines for building advanced video reasoning systems. |
| VChain: Chain-of-Visual-Thought for Reasoning in Video Generation (Read more on arXiv or HuggingFace) |
Paul Debevec, Haonan Qiu, Gordon Chen, Ning Yu, Ziqi Huang |
VChain is an inference-time framework that improves causal reasoning in video generation by using a large multimodal model to generate a “Chain of Visual Thoughts” for sparse model tuning. The objective is to inject high-level reasoning, such as physical and causal coherence, from large multimodal models into pre-trained video generators without requiring extensive retraining or dense supervision. The methodology first uses GPT-4o to iteratively generate a sparse sequence of causally significant keyframes (Visual Thoughts) from a text prompt, and then performs lightweight, inference-time fine-tuning of a pre-trained video generator using LoRA on only these keyframes. VChain significantly enhances the physical realism of generated videos, achieving a Causal Reasoning score of 62.12%, a substantial improvement over the baseline text-to-video model’s 32.81%. The principal implication for AI practitioners is that they can leverage this framework to enhance the logical coherence of generative video models by integrating reasoning from LMMs through efficient, inference-time tuning, avoiding the high cost and data requirements of full model retraining. |
| MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual |
|
|
| Information (Read more on arXiv or HuggingFace) |
|
The paper introduces Mutual Information Tree Search (MITS), an information-theoretic framework that improves large language model reasoning by efficiently exploring multiple solution paths without costly simulations. The objective is to develop a computationally efficient tree search method that can reliably evaluate intermediate reasoning steps to find correct solutions. MITS employs Pointwise Mutual Information (PMI) as a scoring function to quantify the relevance of each reasoning step to the question, guiding a beam search to construct a solution tree, and uses a PMI-weighted voting scheme for final answer selection. MITS substantially outperforms baselines; on the StrategyQA dataset with the QWEN2.5-3B model, it achieves 68.45% accuracy, surpassing the next best baseline (rStar) by 3.13% while being 12.7 times faster. For AI practitioners, MITS offers a training-free, computationally efficient inference-time technique to enhance LLM reasoning by replacing expensive Monte Carlo rollouts with a principled, step-wise evaluation metric. |
| Imperceptible Jailbreaking against Large Language Models (Read more on arXiv or HuggingFace) |
|
This paper introduces an imperceptible jailbreaking technique using invisible Unicode variation selectors to bypass LLM safety alignments by altering prompt tokenization without any visible modifications. The research objective was to develop and evaluate an attack where the adversarial prompt is visually identical to the original malicious query, manipulating the model at the token level. The key methodology is a “chain-of-search” pipeline that appends an optimized suffix of invisible characters, using random search to maximize the log-likelihood of affirmative target-start tokens and bootstrapping successful suffixes across queries. Primary results demonstrate high attack success rates (ASRs), achieving 98% against Llama-2-Chat-7B and 100% against Mistral-7B-Instruct-v0.2, and generalizing to prompt injection tasks with 100% ASR. The principal implication for AI practitioners is that defenses must operate beyond visible text analysis, as invisible characters can manipulate underlying token and embedding representations to subvert safety mechanisms. |
| Hybrid Architectures for Language Models: Systematic Analysis and Design |
|
|
| Insights (Read more on arXiv or HuggingFace) |
|
This paper systematically evaluates inter-layer and intra-layer hybrid language model architectures combining Transformer and Mamba primitives to provide design insights. The main objective is to conduct a holistic comparison of hybridization strategies across language modeling performance, long-context capabilities, and computational efficiency to fill a gap in existing literature. The methodology involves training 1B parameter models from scratch with varying block ratios and architectural configurations, then evaluating them against homogeneous Transformer, Mamba, and Sliding Window Attention baselines. Results demonstrate that hybrid models consistently outperform homogeneous architectures, with intra-layer hybrids achieving the best quality-efficiency trade-off and improving few-shot accuracy by up to 2.9% under a fixed FLOPs budget. The principal implication for AI practitioners is the provision of specific design recipes: intra-layer hybridization is recommended for optimal quality-throughput, and an approximate 1:5 (Transformer:Mamba) block ratio is advised for inter-layer hybrids to balance model quality with inference efficiency. |
| Factuality Matters: When Image Generation and Editing Meet Structured |
|
|
| Visuals (Read more on arXiv or HuggingFace) |
Sayak Paul, Boxiang Qiu, Yuandong Pu, Songhao Han, Le Zhuo |
This paper introduces a comprehensive framework for generating and editing factually accurate structured visuals by leveraging a large-scale, code-aligned dataset, a reasoning-enhanced unified model, and a new benchmark suite. The primary objective is to address the failure of modern generative models in producing factually correct structured visuals like charts and diagrams by developing a systematic approach focused on factual fidelity. The key methodology involves creating a 1.3 million-pair dataset from executable programs with chain-of-thought annotations, and training a unified model based on FLUX.1 Kontext that integrates a Vision-Language Model (Qwen-VL) via a lightweight connector through a three-stage training curriculum. The proposed model achieves state-of-the-art performance on the structured image editing benchmark, StructEditBench, with an overall accuracy of 55.98%, outperforming strong closed-source models. For AI practitioners, the principal implication is that improving factual fidelity in generative models for structured data requires training on programmatically-aligned datasets and integrating explicit reasoning mechanisms, as the paper demonstrates that adding inference-time reasoning consistently boosts performance across diverse architectures. |
| Reactive Transformer (RxT) – Stateful Real-Time Processing for |
|
|
| Event-Driven Reactive Language Models (Read more on arXiv or HuggingFace) |
|
The Reactive Transformer (RxT) is a stateful, event-driven architecture that achieves linear-time complexity for conversational AI by decoupling response generation from an asynchronous, fixed-size memory update mechanism. The primary objective is to overcome the quadratic computational complexity and high latency of standard stateless Transformers in long conversations by designing a fundamentally stateful architecture that processes dialogue turns as discrete events. The key methodology involves an asynchronous operational cycle where a generator-decoder produces a response conditioned on a fixed-size Short-Term Memory (STM), after which a separate memory-encoder and Memory Attention network update the STM with information from the just-completed interaction, a process that does not contribute to user-perceived latency. Primary results demonstrate superior performance and efficiency; a 26M parameter RxT model achieved a perplexity of 2.31 on multi-step dialogues, significantly outperforming the 4.37 perplexity of a 22M parameter stateless LLM baseline, while maintaining a nearly constant inference latency regardless of conversation length. The principal implication for AI practitioners is that for complex, state-dependent tasks, specialized architectures that separate concerns like generation and memory management can yield superior performance and computational efficiency (linear vs. quadratic scaling) compared to simply increasing the scale of monolithic, stateless models. |
| Judging with Confidence: Calibrating Autoraters to Preference |
|
|
| Distributions (Read more on arXiv or HuggingFace) |
|
This paper introduces a framework to calibrate LLM autoraters to predict the full distribution of human preferences instead of a single, discrete label. The objective is to develop autoraters that can reliably model the inherent subjectivity and disagreement in human judgment by aligning with a target preference distribution. The key methodologies are direct supervised fine-tuning (SFT) on dense probabilistic labels and a reinforcement learning (RL) approach with proper scoring rule-based rewards for sparse binary labels. Empirical results show that finetuning with a distribution-matching objective leads to significantly improved alignment and calibration, with the RL (Brier) method on Gemma-2-9B achieving a Mean Squared Error of 0.0764 and reducing absolute symmetry deviation from positional bias to 0.1026. For AI practitioners, the principal implication is that finetuning autoraters to model preference distributions yields more reliable and less biased evaluation systems, and for a fixed annotation budget, RL with a larger set of sparse binary labels is more data-efficient than SFT with a smaller set of dense probabilistic labels. |
| Reinforce-Ada: An Adaptive Sampling Framework for Reinforce-Style LLM |
|
|
| Training (Read more on arXiv or HuggingFace) |
|
REINFORCE-ADA is an adaptive sampling framework that improves Reinforce-style LLM training by dynamically allocating inference budget to uncertain prompts, preventing gradient signal collapse. The objective is to resolve the trade-off between noisy gradients from low-sample counts and the prohibitive cost of high-sample counts by efficiently discovering informative training signals. The methodology uses an online successive elimination process where prompts are sampled in multiple rounds and deactivated once sufficient positive and negative responses are collected, followed by downsampling to a balanced, fixed-size group for the update. The framework consistently improves performance, with REINFORCE-ADA-BALANCE achieving a +2.3 point higher weighted average accuracy over the GRPO baseline on the Qwen2.5-Math-1.5B model. For AI practitioners, this framework offers a drop-in replacement for standard generation APIs in RL pipelines to attain higher final performance and faster convergence, though it incurs a higher computational cost per training step. |
| Optimal Scaling Needs Optimal Norm (Read more on arXiv or HuggingFace) |
Stefan Kesselheim, Jan Ebert, Jiangtao Wang, Oleg Filatov |
This paper demonstrates that for the Scion optimizer, the operator norm of the output layer is an invariant for optimal hyperparameter scaling across both model and dataset sizes, a phenomenon termed “norm transfer.” The main objective is to identify a unifying principle for joint optimal scaling by investigating training dynamics through a norm-based lens. The methodology involves training Llama 3 models up to 1.3B parameters on up to 138B tokens using the Scion optimizer and performing extensive grid searches over learning rate (η) and batch size (B) to measure the layer norms corresponding to optimal loss. The primary results show that the optimal output layer norm is a necessary (but not sufficient) condition for optimality, and empirically derives sufficient scaling rules, finding the optimal batch size B*(D) ∝ D^0.45±0.07 and learning rate η*(D) ∝ D^-0.28±0.07. The principal implication for AI practitioners is that the output layer norm can be monitored during training as a direct, observable invariant to validate and guide hyperparameter choices when scaling models or datasets, potentially simplifying the tuning process at scale. |
| Code4MeV2: a Research-oriented Code-completion Platform (Read more on arXiv or HuggingFace) |
|
This paper introduces Code4Me V2, an open-source, research-oriented code completion plugin for JetBrains IDEs designed to facilitate empirical studies on human-AI interaction in software development. Its objective is to overcome the limitations of proprietary AI coding assistants by providing a transparent and extensible platform for collecting fine-grained telemetry and user interaction data. The system is built on a modular client-server architecture and was evaluated through latency benchmarks and a two-phase user study with expert researchers and daily users. Primary results demonstrate industry-comparable performance, achieving an average end-to-end latency for code completion of 186.31 ms for 18.66 tokens, with qualitative feedback validating its modularity and usefulness for research. The principal implication for AI practitioners is the availability of a shared, transparent infrastructure that allows researchers to focus on experimental design and data analysis for AI-assisted programming rather than on building and maintaining custom data collection systems. |
| Self-Reflective Generation at Test Time (Read more on arXiv or HuggingFace) |
Shuang Qiu, Menglin Yang, Zhiyong Wang, Qixin Zhang, Jian Mu |
This paper introduces SRGen, a lightweight, plug-and-play, test-time framework that proactively corrects LLM reasoning by performing token-level self-reflection at points of high uncertainty. The objective is to design a proactive error prevention mechanism that identifies and intervenes at potential error points in real-time during a single decoding pass, enhancing reasoning reliability without retraining or full-draft revisions. SRGen uses a dynamic entropy threshold to detect uncertain tokens; when triggered, it pauses decoding and optimizes a transient corrective vector by minimizing a hybrid loss function combining a Retrospective Context Loss (LCE) for contextual fidelity and an Anticipatory Entropy Minimization loss (LAEM) for predictive confidence. The framework consistently improves reasoning performance, increasing the Pass@1 accuracy of DeepSeek-R1-Distill-Qwen-7B on the AIME2024 benchmark by an absolute +12.0%. For AI practitioners, SRGen is a zero-training module that can be directly applied to off-the-shelf LLMs to enhance the reliability of complex reasoning tasks, providing significant performance gains with bounded computational overhead. |
| SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior |
|
|
| Reasoning LLMs (Read more on arXiv or HuggingFace) |
|
This paper introduces SWIREASONING, a training-free framework that enhances large language model (LLM) reasoning by dynamically alternating between explicit and latent thinking modes. The objective is to address the trade-offs between explicit reasoning, which can discard useful information, and latent reasoning, which can suffer from noise and poor convergence. The core methodology involves switching between generating discrete text tokens and operating in the continuous latent space, guided by block-wise confidence estimated from entropy trends in the next-token distribution, and using a switch count controller to curb overthinking. The framework demonstrates consistent improvements in Pass@1 accuracy by up to +2.8% on mathematics and STEM benchmarks across different LLMs and scales. For AI practitioners, SWIREASONING offers an inference-time, plug-and-play method to improve both the accuracy and token efficiency of existing reasoning LLMs without any model retraining. |
| Watch and Learn: Learning to Use Computers from Online Videos (Read more on arXiv or HuggingFace) |
Oriana Riva, Yu Su, Palash Goyal, Yiwen Song, Chan Hee Song |
This paper presents Watch & Learn (W&L), a framework that automatically converts web-scale human demonstration videos into executable UI trajectories for training computer use agents. The objective is to address the scarcity of high-quality training data by scalably extracting structured action sequences from unstructured online videos, instead of relying on manual annotation or synthetic generation. The core methodology involves training an Inverse Dynamics Model (IDM) on over 630k state-transition pairs to predict the user action that caused a transition between two consecutive screen frames, which is then applied to raw video tutorials. The generated 53k trajectories significantly improve agent performance on the OSWorld benchmark, with supervised fine-tuning increasing the Qwen 2.5-VL model’s success rate from 1.9% to 13.0% (+11.1 points). For AI practitioners, this inverse dynamics approach provides a scalable method to create high-quality training datasets from public videos, enabling the development of more capable vision-based computer agents without manual annotation. |
| Agentic Context Engineering: Evolving Contexts for Self-Improving |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Fenglu Hong, Boyuan Ma, Shubhangi Upasani, Changran Hu, Qizheng Zhang |
This paper introduces Agentic Context Engineering (ACE), a framework for self-improving LLMs by treating contexts as evolving playbooks to prevent context collapse and brevity bias. The main objective is to enable LLMs to continuously self-improve through context adaptation by accumulating and refining detailed strategies from execution feedback, rather than relying on static prompts or concise summaries. ACE’s methodology uses a modular, agentic workflow with three specialized components—a Generator, a Reflector, and a Curator—that perform structured, incremental “delta updates” to the context. The framework demonstrates significant performance gains, outperforming strong baselines by an average of +10.6% on agent benchmarks and +8.6% on financial benchmarks, while reducing adaptation latency by 86.9%. The principal implication for AI practitioners is that ACE offers a scalable and efficient method for building robust, self-improving agents and domain-specific systems by evolving comprehensive contexts directly from operational feedback, providing a low-overhead alternative to model fine-tuning. |
| ChronoEdit: Towards Temporal Reasoning for Image Editing and World |
|
|
| Simulation (Read more on arXiv or HuggingFace) |
|
ChronoEdit is a framework that reframes image editing as a two-frame video generation problem to enforce physical consistency in edited outputs. The research aims to solve the problem of physical inconsistency in image editing by leveraging the inherent temporal priors of large-scale video generative models, making edits more suitable for world simulation applications. The key methodology involves finetuning a pretrained video model on image-editing pairs and introducing a temporal reasoning inference stage that uses intermediate “reasoning tokens” to plan a physically plausible trajectory for the edit. On the general-purpose ImgEdit benchmark, ChronoEdit-14B achieves a state-of-the-art overall score of 4.42, outperforming existing open-source models. For AI practitioners, this work provides a method to utilize pretrained video models for generating physically consistent synthetic data, which is highly valuable for creating robust training and evaluation datasets for simulation-heavy domains like autonomous systems and robotics. |
| Front-Loading Reasoning: The Synergy between Pretraining and |
|
|
| Post-Training Data (Read more on arXiv or HuggingFace) |
|
This paper demonstrates that front-loading reasoning data into the pretraining phase of Large Language Models establishes foundational capabilities that cannot be replicated by later-stage fine-tuning alone. The study’s objective was to determine the optimal allocation of reasoning data—varying in scale, diversity, and quality—between pretraining and Supervised Fine-Tuning (SFT) to maximize downstream reasoning performance. Using an 8B parameter model, the authors conducted a systematic study by pretraining variants with and without reasoning corpora and then subjecting them to controlled SFT and reinforcement learning stages. The primary results reveal an asymmetric principle for data allocation: pretraining benefits most from broad data diversity, while SFT requires high data quality, with front-loaded models achieving a 19% average performance gain on expert-level benchmarks. The principal implication for AI practitioners is to abandon the conventional separation of pretraining and post-training for reasoning tasks, instead treating reasoning-aware pretraining as a critical and foundational step for developing more capable models. |
| Good Intentions Beyond ACL: Who Does NLP for Social Good, and Where? (Read more on arXiv or HuggingFace) |
Denis Peskoff, Jason Jewell, Adam Leif, Qingcheng Zeng, Grace LeFevre |
This paper presents a large-scale scientometric analysis to map the landscape of NLP for Social Good (NLP4SG), identifying who conducts this research and where it is published. The primary objective is to quantify and compare the proportion of NLP4SG work within the core ACL community versus in external, interdisciplinary venues, differentiating between contributions from ACL-affiliated and non-ACL authors. The methodology involved augmenting a corpus of 309,208 NLP-relevant papers with metadata classifying venue type (ACL, ACL-Adjacent, External), author affiliation (3+ ACL publications), and social good relevance using a pre-trained classifier based on UN Sustainable Development Goals. The key finding is that ACL authors are dramatically more likely to publish social good-oriented work outside of ACL venues, with the proportion of NLP4SG papers being over three times higher in external venues (37.3%) compared to core ACL venues (12.0%). For AI practitioners, the principal implication is that the majority of applied NLP4SG research resides outside of traditional computer science conferences, necessitating engagement with domain-specific journals in fields like social and medical sciences to identify relevant applications and datasets. |
| Thai Semantic End-of-Turn Detection for Real-Time Voice Agents (Read more on arXiv or HuggingFace) |
Saksorn Ruangtanusak, Monthol Charattrakool, Natthapath Rungseesiripak, Thanapol Popit |
This paper establishes the first benchmark for Thai text-only semantic end-of-turn (EOT) detection, evaluating accuracy-latency trade-offs between fine-tuned transformers and prompted large language models. The objective is to identify optimal methods for real-time conversational agents by comparing zero-shot, few-shot, and supervised fine-tuning of both Thai-specific and multilingual encoder/decoder models on a dataset derived from public subtitles. The methodology involves training models on a binary classification task (end vs. not-end) and measuring F1-score and CPU inference latency. The primary result shows supervised fine-tuning significantly outperforms other methods, with the fine-tuned Typhoon2-1B model achieving the highest F1-score of 0.881 at a latency of 110ms. For AI practitioners, the principal implication is that small, fine-tuned models are highly effective for real-time EOT; a fine-tuned encoder like mDeBERTa-v3-base offers a robust, calibration-free solution, while a fine-tuned decoder like Typhoon2-1B provides peak performance, presenting a clear trade-off between deployment simplicity and accuracy. |
| EvolProver: Advancing Automated Theorem Proving by Evolving Formalized |
|
|
| Problems via Symmetry and Difficulty (Read more on arXiv or HuggingFace) |
Xuanwu Wang, Ruiyuan Huang, Yuchen Tian, danielhzlin, Ziyang |
The paper presents EvolProver, a theorem prover trained on a novel data augmentation pipeline that evolves formal problems based on symmetry and difficulty to enhance model robustness. The primary research objective is to overcome the lack of generalizability and fragility of Large Language Models (LLMs) in formal theorem proving when faced with minor problem transformations. The key methodology is a three-part data augmentation pipeline: EvolAST for Abstract Syntax Tree (AST) based syntactic variations, EvolDomain for LLM-driven semantic translation across mathematical domains, and EvolDifficulty for LLM-guided complexity adjustments. EvolProver achieves a new state-of-the-art performance on the FormalMATH-Lite benchmark with a 53.8% pass@32 rate, outperforming comparable models, and on the Ineq-Comp benchmark, it improves its robustness ratio by over 30 percentage points compared to the baseline. The principal implication for AI practitioners is that targeted, multi-faceted data evolution—systematically altering problem statements based on syntactic symmetry, semantic scope, and difficulty—provides a powerful strategy to enhance model robustness and performance in formally constrained and data-scarce domains like automated theorem proving. |
| Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the |
|
|
| Rails (Read more on arXiv or HuggingFace) |
Xinyuan Liu, Wenbo Duan, Yaofeng Su, Jiaqi Liu, Siwei Han |
This paper introduces the Alignment Tipping Process (ATP), a post-deployment phenomenon where self-evolving LLM agents abandon their initial alignment in favor of strategies reinforced by environmental feedback. The research objective is to formalize and empirically demonstrate how continual interaction with rewarded, deviant behaviors causes this alignment decay. The methodology involves analyzing ATP through two paradigms, Self-Interested Exploration and Imitative Strategy Diffusion, using custom testbeds to benchmark LLMs like Qwen3-8B and Llama-3.1-8B-Instruct fine-tuned with DPO and GRPO. The primary results show that alignment is fragile; for instance, a Llama-3.1-8B-Instruct model aligned with DPO saw its rule violation rate increase from 18.8% to 45.3% over six self-evolution rounds, demonstrating that current alignment techniques offer only a temporary defense. The principal implication for AI practitioners is that alignment is not a static, pre-deployment property but a dynamic state that can erode and requires continuous monitoring and intervention, as the mechanisms for agent adaptation can also systematically corrupt their intended behavior. |
| HiKE: Hierarchical Evaluation Framework for Korean-English |
|
|
| Code-Switching Speech Recognition (Read more on arXiv or HuggingFace) |
|
This paper introduces HiKE, a public Korean-English code-switching (CS) benchmark with hierarchical labels, and demonstrates that fine-tuning with natural or synthetic CS data significantly improves ASR performance on this task. The main objective is to provide a framework for the precise evaluation of multilingual ASR models on Korean-English code-switching and to investigate methods for improving their CS capabilities. The methodology involves the creation of the HiKE benchmark, a 1,121-utterance dataset with hierarchical (word, phrase, sentence) and loanword labels, followed by the evaluation of nine ASR models and fine-tuning experiments on WHISPER-MEDIUM using both natural and synthetic CS data. Primary results show that while baseline models perform poorly, fine-tuning WHISPER-MEDIUM with natural CS data reduced the overall Mixed Error Rate (MER) from 37.3 to 10.0; fine-tuning with only synthetic data also proved effective, improving MER and Point of Interest Error Rate (PIER) by more than 13%. The principal implication for AI practitioners is that robust CS-ASR capability can be enabled via fine-tuning, and that easily generated synthetic data from concatenated monolingual utterances offers a viable, cost-effective strategy for adapting models to multilingual user bases, especially when natural CS data is scarce. |
| LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL (Read more on arXiv or HuggingFace) |
|
This paper presents LLMSQL, a systematically revised and cleaned version of the WikiSQL dataset designed as a modern benchmark for evaluating large language models (LLMs) on Text-to-SQL tasks. The objective was to resolve structural and annotation issues in the original WikiSQL, such as data type mismatches and case sensitivity inconsistencies, to create a reliable and reproducible benchmark. The methodology involved systematic error classification, followed by automated and manual data cleaning, re-annotation to correct invalid queries, and conversion of the data into a plain-text format suitable for generative models. Evaluation of various LLMs on the new benchmark demonstrated that larger models like DeepSeek R1 and OpenAI o4-mini achieve over 86% execution accuracy in a 5-shot setting, while fine-tuning smaller models can surpass 90% accuracy. The primary implication for AI practitioners is the availability of a large-scale, validated, single-table benchmark (LLMSQL) with clean, ready-to-use text-based question-SQL pairs, which simplifies the evaluation and fine-tuning of modern LLMs for Text-to-SQL generation. |
| Character Mixing for Video Generation (Read more on arXiv or HuggingFace) |
|
This paper introduces a framework for generating videos that mix characters from different fictional universes while preserving their unique identities, behaviors, and original visual styles. The primary research objective is to overcome the “non-coexistence challenge” (characters never appear together in training data) and the “style delusion challenge” (style bleeding between characters from different domains like cartoons and live-action). The key methodology involves Cross-Character Embedding (CCE) for learning disentangled character identities via structured prompts and Cross-Character Augmentation (CCA) for creating synthetic training data by compositing characters into cross-domain backgrounds. The proposed method significantly outperforms baselines in multi-subject scenarios, achieving a VLM-evaluated Style-P score of 7.26, compared to 6.28 for the SkyReels-A2 baseline. For AI practitioners, this work demonstrates a practical fine-tuning approach using structured annotation and synthetic data generation to enable foundation models to create complex, controllable multi-subject video content where subjects lack co-occurrence in the training set. |
| SAEdit: Token-level control for continuous image editing via Sparse |
|
|
| AutoEncoder (Read more on arXiv or HuggingFace) |
Or Patashnik, Roni Paiss, Daniel Garibi, Sara Dorfman, Ronen Kamenetsky |
SAEdit is a method that uses a Sparse AutoEncoder to manipulate token-level text embeddings for disentangled and continuous control in image editing. The main objective is to create a framework for image editing that offers both disentanglement, where one attribute is changed without affecting others, and continuous control over the intensity of the edit. The key methodology involves training a Sparse AutoEncoder (SAE) on the output of a frozen T5 text encoder to learn a sparse latent space; edit directions are found by comparing sparse representations of prompt pairs (e.g., “a woman” vs. “a laughing woman”), and these directions are then scaled and added to the specific token embedding to be modified. The primary results demonstrate high-quality, localized edits, and a pairwise user study shows the method achieves a 93% overall win rate against the Flux Space baseline, indicating superior perceptual quality and disentanglement. The principal implication for AI practitioners is that SAEs can serve as model-agnostic, pluggable modules to enable fine-grained semantic control over generative models by manipulating text embeddings, thus avoiding costly per-edit model retraining or optimization. |
| Learning on the Job: Test-Time Curricula for Targeted Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
|
The paper introduces Test-Time Curriculum Reinforcement Learning (TTC-RL), a method for specializing a language model for a specific target task at test-time by having it “learn on the job.” The primary objective is to enable an agent to automatically assemble a task-specific curriculum from a large, diverse data pool and use reinforcement learning to continually train on it. The key methodology involves using the SIFT algorithm to select an informative curriculum of training tasks and then applying on-policy reinforcement learning (GRPO) to update the model’s weights based on its experience. Primary results show that TTC-RL significantly improves performance, increasing the pass@1 of the Qwen3-8B model by 1.8x on the AIME25 math benchmark and raising its pass@8 performance ceiling on the same benchmark from 40% to 62%. The principal implication for AI practitioners is that this automated, targeted RL approach provides a scalable method to substantially improve a model’s specialized reasoning capabilities, effectively raising its performance ceiling without requiring manual data curation or simply expanding the context window. |
| Utility-Learning Tension in Self-Modifying Agents (Read more on arXiv or HuggingFace) |
Peter Jin, Keir Dorchen, Charles L. Wang |
This paper establishes that for a self-modifying agent to maintain PAC learnability, the capacity of its policy-reachable hypothesis space must be uniformly bounded, addressing when utility-driven self-modifications preserve generalization. The research formalizes self-modification across five distinct axes (algorithmic, representational, architectural, substrate, metacognitive) and uses VC theory to derive a sharp boundary condition based on the maximum capacity reachable by the agent’s policy. The primary result is that distribution-free learnability is preserved if and only if the supremum VC dimension of this reachable family is finite; experiments demonstrate a destructive policy reached a test loss of 0.409, while the proposed capacity-capping “Two-Gate” policy achieved 0.350—a 17% relative improvement. The principal implication for AI practitioners is that they must implement explicit, computationally cheap capacity controls during self-improvement loops to prevent the compounding risk of unbounded complexity from destroying generalization guarantees, as implicit regularization alone cannot be relied upon to manage this risk over sequential modifications. |
| Epistemic Diversity and Knowledge Collapse in Large Language Models (Read more on arXiv or HuggingFace) |
|
This paper introduces a novel methodology to empirically measure epistemic diversity—the variation in factual claims—to assess the risk of knowledge collapse in LLMs. The research objective is to investigate whether LLMs exhibit knowledge collapse by quantifying the diversity of claims generated by 27 models across 155 topics and multiple generation settings. The methodology involves generating free-text responses, decomposing them into atomic claims, clustering these claims into unique meaning classes based on mutual entailment, and measuring the resulting distribution using Hill-Shannon Diversity with coverage-based rarefaction. Primary results indicate that Retrieval-Augmented Generation (RAG) significantly increases diversity compared to instruction-fine-tuning (HSD +739.186, p < 1e-3), while larger model size has a statistically significant negative impact. The principal implication for practitioners is that utilizing smaller models and implementing RAG with diverse, human-curated knowledge sources is a critical strategy to counteract homogenization and prevent knowledge collapse in AI applications. |
| MoME: Mixture of Matryoshka Experts for Audio-Visual Speech Recognition (Read more on arXiv or HuggingFace) |
|
The paper introduces MoME (Mixture of Matryoshka Experts), a framework integrating sparse Mixture-of-Experts (MoE) with Matryoshka Representation Learning (MRL) for resource-adaptive audio-visual speech recognition (AVSR). The primary objective is to overcome the performance degradation and limited cross-scale generalization of MRL-based models at high token compression rates. The methodology involves augmenting a frozen LLM with parallel MoE layers containing a shared router, top-k routed experts, and shared experts, which are trained jointly across multiple token granularities to promote knowledge transfer. Experiments on the LRS3 dataset show MoME achieves a 1.5% Word Error Rate (WER) at a (4,2) compression rate, outperforming the Llama-MTSK baseline (2.3% WER) with less than half the active parameters (3.5M vs. 8.1M). For AI practitioners, MoME offers a method to build a single, parameter-efficient model that supports elastic inference, enabling dynamic adjustment of computational load at runtime to suit diverse hardware constraints without significant performance loss. |
| AdvEvo-MARL: Shaping Internalized Safety through Adversarial |
|
|
| Co-Evolution in Multi-Agent Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zeliang Zhang, Yolo Yunlong Tang, Zhuo Liu, Yiting Zhang, Zhenyu Pan |
AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework designed to internalize safety within LLM agents by simultaneously training adversarial attackers and defending task agents. The main objective is to enhance the robustness of multi-agent systems against jailbreak and prompt-injection attacks without depending on external guard agents or compromising task utility. The core methodology involves an initial supervised warm-up for attackers, followed by a co-evolutionary MARL stage where attackers and defenders are jointly optimized using a public, group-level mean-return baseline for advantage estimation to stabilize training. Experiments show the framework consistently keeps attack success rates (ASR) below 20%, compared to baselines reaching 38.33%, while also improving task accuracy by up to 3.67% on reasoning benchmarks. For AI practitioners, this provides a unified method to build inherently safer multi-agent systems by embedding robust defensive behaviors directly into agents, thus avoiding the overhead and single-point-of-failure risks of external safety modules. |
| Graph2Eval: Automatic Multimodal Task Generation for Agents via |
|
|
| Knowledge Graphs (Read more on arXiv or HuggingFace) |
Zeyi Liao, Ziqi Wang, Yuhan Liu, Xavier Hu, Yurun Chen |
GRAPH2EVAL is a framework that automatically generates multimodal evaluation tasks for AI agents by treating knowledge graphs constructed from source data as a latent task space. The research objective is to create a scalable system for generating diverse document comprehension and web interaction tasks that can comprehensively assess an agent’s reasoning, collaboration, and interactive capabilities, overcoming the limitations of static datasets. The core methodology involves building a knowledge graph from multi-source data, applying subgraph sampling with task templates and meta-paths to generate task instances, and using a multi-stage filtering pipeline to ensure task quality and executability. Experiments on the generated GRAPH2EVAL-BENCH dataset of 1,319 tasks show the framework effectively differentiates agent performance; for example, on web interaction tasks, Agent S 2.5 achieved a 69.20% success rate with gemini-2.5-flash, significantly outperforming the SoM Agent’s 14.51% with the same model. For AI practitioners, this provides an automated and scalable method to generate custom, high-quality evaluation benchmarks for rigorously assessing agent performance on complex, dynamic tasks. |
Papers for 2025-10-06
| Title |
Authors |
Summary |
| Apriel-1.5-15b-Thinker (Read more on arXiv or HuggingFace) |
|
The paper presents Apriel-1.5-15B-Thinker, a 15-billion parameter open-weights multimodal model designed to achieve frontier-level performance through training strategy rather than parameter scale. The research objective is to demonstrate that a compact, open model can attain advanced reasoning capabilities while remaining economical to train and deploy on a single GPU. The core methodology is a three-stage “mid-training” pipeline starting from Pixtral-12B: (1) depth upscaling the architecture, (2) staged continual pre-training with targeted synthetic data for visual reasoning, and (3) high-signal, text-only supervised fine-tuning with explicit reasoning traces, without reinforcement learning. The model achieves a score of 52 on the Artificial Analysis Intelligence Index, matching the performance of DeepSeek-R1-0528 despite using significantly fewer computational resources. The principal implication for AI practitioners is that strategic, data-centric mid-training can close the capability gap with massive-scale models, enabling the development of high-performance, resource-efficient systems for constrained deployment environments. |
| Efficient Multi-modal Large Language Models via Progressive Consistency |
|
|
| Distillation (Read more on arXiv or HuggingFace) |
|
This paper introduces EPIC, a progressive consistency distillation framework to efficiently train multi-modal large language models (MLLMs) with compressed visual tokens. The primary objective is to overcome the increased learning difficulty caused by the feature space perturbations from aggressive token compression during training. The key methodology involves a shared-weight teacher-student model that progressively increases the token compression ratio and shifts the compression layer from deep to shallow throughout training, following an easy-to-hard curriculum guided by KL-divergence loss. Experimental results show that an MLLM trained with EPIC achieves performance comparable to the vanilla LLaVA-v1.5 model on 10 benchmarks (61.4% average accuracy) while using only 192 visual tokens instead of 576, a 66.7% reduction. For AI practitioners, this framework offers a method to significantly reduce the computational cost and memory footprint of MLLMs at inference without modifying the model architecture, making them more suitable for deployment on resource-constrained hardware. |
| Compose Your Policies! Improving Diffusion-based or Flow-based Robot |
|
|
| Policies via Test-time Distribution-level Composition (Read more on arXiv or HuggingFace) |
|
This research introduces General Policy Composition (GPC), a training-free framework that improves diffusion- and flow-based robot policies by combining the distributional scores of multiple pre-trained models at test-time. The main objective is to create a superior policy that exceeds the performance of its individual parent policies without requiring additional training. The key methodology involves forming a convex combination of the learned score functions from heterogeneous pre-trained policies and using a test-time search to identify optimal weighting for specific tasks, a process supported by a theoretical proof of single-step functional improvement. Experiments show consistent performance gains across multiple benchmarks, such as an average success rate increase of up to +7.55% on Robomimic and PushT tasks when composing a VLA and a VA policy. For AI practitioners, GPC provides a simple, plug-and-play method to enhance the performance and adaptability of existing robotic control systems by leveraging and combining deployed policy assets, thus avoiding costly retraining. |
| Self-Improvement in Multimodal Large Language Models: A Survey (Read more on arXiv or HuggingFace) |
Yapeng Tian, Harsh Singh, Tianyu Yang, Kai Wang, Shijian Deng |
This paper surveys and taxonomizes self-improvement methodologies for Multimodal Large Language Models (MLLMs). The objective is to provide the first comprehensive, structured overview of self-improvement techniques in MLLMs by categorizing the literature and identifying open challenges. The authors conducted a literature review, structuring the field into a three-stage pipeline: data collection (e.g., random sampling, guided generation), data organization (e.g., rule-based verification, model-based verification), and model optimization (e.g., Supervised Fine-Tuning, Direct Preference Optimization, Reinforcement Learning). The survey identifies that method-task matching is critical, with rule/verification-based RL driving the largest gains on verifiable tasks, while citing a surveyed work that achieved a 15.50% improvement in visuo-motor control tasks. The principal implication for AI practitioners is that this survey provides a taxonomy to select specific self-improvement pipelines tailored to their application, such as using verification-based RL for tasks with ground-truth checks and preference data to improve model helpfulness and mitigate hallucinations. |
| Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents (Read more on arXiv or HuggingFace) |
Boyi Wei, Chen Qian, Qihan Ren, Shuai Shao, JY-Young |
This paper introduces and empirically investigates “misevolution,” the phenomenon where self-evolving LLM agents’ autonomous improvement processes lead to unintended and harmful outcomes. The primary objective is to systematically assess whether an agent’s self-evolution across four key pathways—model, memory, tool, and workflow—compromises its safety alignment or introduces new vulnerabilities. The methodology involves evaluating the safety performance of various self-evolving agent architectures on security benchmarks (e.g., RedCode-Gen, RiOSWorld) both before and after their evolution cycles. The findings reveal that misevolution is a pervasive risk, with one key result showing that tool-evolving agents built on top-tier LLMs failed to identify and reject malicious external tools nearly 84% of the time. The principal implication for AI practitioners is that deploying self-evolving agents requires new safety paradigms beyond static checks, such as continuous monitoring and automated safety verification for dynamically created components, as inherent safety alignment can degrade unpredictably during autonomous operation. |
| CoDA: Agentic Systems for Collaborative Data Visualization (Read more on arXiv or HuggingFace) |
|
This paper introduces CoDA, a collaborative multi-agent system that automates the generation of complex data visualizations from natural language by decomposing the task into specialized agent-driven stages. The primary objective is to address the failures of existing systems in handling complex multi-file datasets and iterative refinement by reframing visualization automation as a collaborative multi-agent problem. CoDA’s methodology employs specialized LLM agents for metadata analysis, task planning, code generation, and self-reflection, utilizing a metadata-centric approach to bypass token limits and a quality-driven feedback loop for robust refinement. Extensive evaluations show that CoDA achieves substantial gains over baselines, outperforming competitors by up to 41.5% in overall score on visualization benchmarks. The principal implication for AI practitioners is that designing integrated, collaborative agentic workflows with specialized roles and feedback loops is a more effective paradigm for complex automation tasks than relying on monolithic or simple agent systems, enabling robust handling of real-world data and user requirements. |
| SurveyBench: How Well Can LLM(-Agents) Write Academic Surveys? (Read more on arXiv or HuggingFace) |
Shuo Wang, Xin Tong, Xuanhe Zhou, Xuzhou Zhu, Zhaojun Sun |
This paper introduces SurveyBench, a fine-grained, quiz-driven evaluation framework to rigorously assess the ability of LLM-based agents to write academic surveys that align with reader needs. The primary objective is to create a robust benchmark to systematically evaluate and quantify the deficiencies of LLM-generated academic surveys by comparing them against high-quality human-authored works. The methodology centers on a curated dataset of topics from recent arXiv papers, a multifaceted metric hierarchy for outline and content quality, and a dual-mode evaluation protocol featuring both content-based scoring and a novel quiz-based assessment to probe technical depth. Results demonstrate that LLM-generated surveys, while structurally coherent, are quantitatively inferior to human-written ones, scoring on average 21% lower in content-based evaluation and achieving a maximum score of only 3.19 out of 10 on topic-specific quizzes where human surveys served as the reference. The principal implication for AI practitioners is that current LLM-agent pipelines for automated content generation excel at surface-level fluency but lack the deep synthesis, critical reasoning, and technical detail required for high-quality academic writing, highlighting the need to develop more sophisticated knowledge integration capabilities. |
| REPAIR: Robust Editing via Progressive Adaptive Intervention and |
|
|
| Reintegration (Read more on arXiv or HuggingFace) |
|
The paper introduces REPAIR, a lifelong editing framework for large language models that enables precise, low-cost updates by integrating closed-loop feedback, dynamic memory management, and distribution-aware optimization. The main objective is to develop a robust and scalable method for sequential model editing that corrects errors or integrates new facts without causing catastrophic forgetting, routing instability, or unintended side effects on non-target knowledge. REPAIR’s methodology combines a dual-memory system with parametric editing, a closed-loop feedback mechanism for monitoring and pruning underperforming memory modules, distribution-aware optimization via inner-batch knowledge distillation, and loss-aware weighted merging of updates. Experiments demonstrate that REPAIR boosts editing accuracy by 10%-30% across multiple model families and significantly reduces knowledge forgetting, particularly in large-scale sequential editing scenarios. The principal implication for AI practitioners is a framework for developing more reliable and continually evolving LLMs, enabling low-cost updates to correct factual errors or add new knowledge to deployed models while preserving existing capabilities. |
| OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Features (Read more on arXiv or HuggingFace) |
Elena Tutubalina, Oleg Rogov, Alexey Dontsov, Andrey Galichin, Anton Korznikov |
This paper introduces Orthogonal Sparse Autoencoder (OrtSAE), a training method that enforces orthogonality between learned features to mitigate feature absorption and composition issues in standard sparse autoencoders (SAEs). The primary objective is to improve the atomicity and disentanglement of features learned by SAEs by directly addressing the failure modes where broad features are absorbed by specific ones or independent features merge into composite ones. The key methodology involves adding an orthogonality penalty to the SAE loss function, which penalizes high pairwise cosine similarity between decoder feature vectors, implemented with a chunk-wise strategy to reduce computational complexity from quadratic to linear. Primary results show that, at an L0 sparsity of 70, OrtSAE reduces feature absorption by 65% and composition by 15%, discovers 9% more distinct features, and improves spurious correlation removal performance by 6% compared to traditional SAEs. The principal implication for AI practitioners is that OrtSAE provides a computationally efficient method to train SAEs that produce more disentangled and interpretable feature dictionaries from LLMs, improving model analysis and intervention capabilities without significant architectural changes or overhead. |
| FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of |
|
|
| Web Agents (Read more on arXiv or HuggingFace) |
Léo Boisvert, Xing Han Lù, Megh Thakkar, Sahar Omidi Shayegan, Imene Kerboua |
FOCUSAGENT introduces a two-stage pipeline where a lightweight LLM-retriever prunes accessibility tree (AxTree) observations to reduce context size for a primary agent LLM, maintaining task performance while enhancing efficiency and security. The primary objective is to develop an observation pruning strategy for LLM-based web agents that reduces the token count of web page representations (AxTrees) to manage computational costs and security risks, while preserving the critical information necessary for high task-completion success rates. The core methodology is a two-stage pipeline where a lightweight retriever LLM first analyzes the full accessibility tree (AxTree) observation, guided by the task goal, to identify and extract line ranges corresponding to relevant UI elements. This pruned AxTree is then passed to a more powerful agent LLM, which performs Chain-of-Thought reasoning and predicts the subsequent action. On the WorkArena L1 benchmark, FOCUSAGENT reduced the observation token count by over 50% while achieving a 51.5% task success rate, which is comparable to the 53.0% success rate of a strong baseline agent operating on the full, unpruned observation. In security evaluations, a variant of the agent reduced the success rate of popup-based prompt injection attacks from 90.4% to 1.0%. AI practitioners can implement a cascaded LLM architecture, using a smaller, cost-effective model as an intelligent pre-processing filter to prune large contexts for a more powerful downstream model, thereby reducing API costs and latency while simultaneously hardening the agent against environmental security threats like prompt injection without requiring separate, complex defense layers. |
| Improving GUI Grounding with Explicit Position-to-Coordinate Mapping (Read more on arXiv or HuggingFace) |
Spandana Gella, Christopher Pal, Ahmed Masry, Tianyu Zhang, Suyuchen Wang |
This paper introduces RULER tokens and Interleaved MROPE (I-MROPE) to improve GUI grounding by creating an explicit mapping from spatial positions to pixel coordinates. The objective is to address the unreliable coordinate prediction and poor resolution generalization of current Vision-Language Models (VLMs) which learn this mapping implicitly. The methodology uses RULER tokens as explicit coordinate markers that transform coordinate generation from an unstable regression problem into a robust reference-and-adjust mechanism, complemented by I-MROPE which balances spatial positional encodings. The primary result shows that finetuning Qwen2.5-VL with RULER tokens improves grounding accuracy on the high-resolution ScreenSpot-Pro benchmark from 34.6% to 37.2%. For AI practitioners, the principal implication is that architecturally incorporating explicit spatial guidance is a more effective method for achieving precise visual localization in GUI automation agents than relying solely on implicit learning from positional embeddings. |
| LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
|
LSPO is a meta-reinforcement learning algorithm that improves LLM reasoning performance by dynamically filtering training data based on response length. The paper’s objective is to improve the final model effectiveness of LLMs on reasoning tasks, rather than just training efficiency, by introducing a novel dynamic data sampling strategy for Reinforcement Learning with Verifiable Rewards (RLVR). The proposed method, Length-aware Sampling for Policy Optimization (LSPO), operates on top of existing RLVR algorithms by calculating the average response length for each prompt in a rollout batch and retaining only a fixed percentile of prompts that yield the shortest and longest responses for the training update. Experiments show LSPO consistently improves performance across multiple models and benchmarks; for instance, when applied to the GSPO algorithm on a Qwen3-4B model, it increased the average accuracy on three math benchmarks from 37.2% to 39.6%. AI practitioners can integrate LSPO as a data-filtering layer during RL fine-tuning to enhance the final reasoning accuracy of their models by selectively training on prompts that are either very easy (short responses) or very difficult (long responses) for the current model. |
| WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents (Read more on arXiv or HuggingFace) |
Neil Zhenqiang Gong, Yuqi Jia, Xilong Wang, Ruohan Xu, Yinuo Liu |
This paper introduces WAInjectBench, a comprehensive benchmark for systematically evaluating prompt injection detection methods specifically for web agents. The research objective is to assess the effectiveness of existing text-based and image-based detectors against a fine-grained categorization of prompt injection attacks that manipulate web content. The methodology involves constructing a new dataset of malicious and benign text and image samples from six distinct attack types and evaluating 12 different detection methods (e.g., prompting-based, embedding-based, fine-tuning-based) across multiple scenarios. The primary result shows that while some detectors can identify attacks with explicit instructions or visible perturbations with moderate-to-high accuracy (e.g., GPT-4o-Prompt achieved a 0.93 True Positive Rate against VPI screenshots), they largely fail against attacks with imperceptible perturbations or implicit instructions (e.g., 0.00 TPR against WebInject). For AI practitioners, the principal implication is that current detection methods are insufficient against sophisticated, stealthy prompt injection attacks, necessitating the development of more robust defenses that do not rely on detecting explicit malicious content. |
| Free Lunch Alignment of Text-to-Image Diffusion Models without |
|
|
| Preference Image Pairs (Read more on arXiv or HuggingFace) |
|
The paper introduces Text Preference Optimization (TPO), a framework for aligning text-to-image models using LLM-generated text preference pairs, eliminating the need for human-annotated image preference data. The research aims to determine if text-to-image alignment can be improved cost-effectively by optimizing over text conditions rather than image outputs. The core methodology involves using a Large Language Model (LLM) to create mismatched (negative) text prompts by perturbing original captions, then fine-tuning the diffusion model with adapted objectives (TDPO and TKTO) to prefer the original prompt for a given image. The proposed methods demonstrate superior performance over baselines, with the TDPO variant achieving an 83.25% win rate on the HPSv2 dataset as measured by PickScore, compared to 77.00% for the Diffusion-DPO baseline. For AI practitioners, this provides a “free lunch” technique to significantly improve model alignment and quality by repurposing existing image-caption datasets, thus bypassing the expensive and time-consuming process of collecting human preference feedback on images. |
| LEAML: Label-Efficient Adaptation to Out-of-Distribution Visual Tasks |
|
|
| for Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Yu-Chiang Frank Wang, Yu-Yang Sheng, Min-Hung Chen, Ci-Siang Lin |
This paper introduces LEAML, a two-stage framework for efficiently adapting Multimodal Large Language Models (MLLMs) to out-of-distribution (OOD) visual question answering (VQA) tasks using limited labeled data. The research objective is to develop a label-efficient method to fine-tune MLLMs for specialized domains like medical imaging where annotated data is scarce. The methodology involves a “Pseudo QA Generation” stage, where a QA Generator trained on few-shot examples creates synthetic question-answer pairs for unlabeled images, regularized via “Selective Neuron Distillation” from a larger model’s captions, followed by an “OOD VQA Finetuning” stage using both original and pseudo-labeled data. On the Kvasir-VQA medical dataset, using only 1% of labeled data, LEAML achieved 76.7% average accuracy, significantly outperforming standard full fine-tuning (63.1%). For AI practitioners, this work provides a validated approach to adapt general-purpose MLLMs for specialized VQA tasks with minimal annotation budget by effectively leveraging unlabeled image corpora to generate synthetic training data. |
| SpineBench: A Clinically Salient, Level-Aware Benchmark Powered by the |
|
|
| SpineMed-450k Corpus (Read more on arXiv or HuggingFace) |
Zhonghao Zhang, Xiang Zheng, Yang Zhang, Wenhui Dong, Ming Zhao |
This research introduces SpineMed-450k, a large-scale, multimodal instruction corpus, and SpineBench, a benchmark for evaluating large vision-language models (LVLMs) on vertebral-level spine disorder analysis. The objective is to facilitate the development of AI systems with sophisticated, level-aware clinical reasoning by creating a traceable, clinically-grounded instruction dataset and a standardized evaluation framework for spine-specific tasks. The methodology consists of a clinician-in-the-loop pipeline that curates over 450,000 instruction instances from textbooks, clinical cases, and guidelines using a two-stage LLM generation method, followed by fine-tuning a 7B parameter LVLM and evaluating it on the SpineBench. The primary result is that the authors’ fine-tuned model achieves an 87.44% average score on SpineBench, significantly outperforming other open-source models and revealing systematic weaknesses in the fine-grained, level-specific reasoning of existing generalist LVLMs. The principal implication for AI practitioners is that achieving clinically relevant performance in specialized, high-stakes domains requires the creation of domain-specific, high-quality instruction datasets for targeted fine-tuning, as the capabilities of general-purpose models are insufficient for complex, multimodal diagnostic reasoning. |
| How Confident are Video Models? Empowering Video Models to Express their |
|
|
| Uncertainty (Read more on arXiv or HuggingFace) |
Anirudha Majumdar, Ola Shorinwa, Zhiting Mei |
This paper introduces S-QUBED, a black-box framework for quantifying and decomposing uncertainty in generative video models, and establishes a metric for evaluating its calibration. The main objective is to develop a method for generative video models to express their predictive uncertainty, enabling a rigorous decomposition of this uncertainty into its aleatoric (due to prompt ambiguity) and epistemic (due to model knowledge gaps) components. The key methodology, S-QUBED, models the generation process with a latent variable; it quantifies aleatoric uncertainty as the entropy of a Von-Mises Fisher (VMF) distribution fitted to refined text prompts from an LLM, and epistemic uncertainty as the expected entropy of video outputs conditioned on those prompts, also modeled with VMF distributions. The primary result is that S-QUBED’s total uncertainty estimates are shown to be calibrated, demonstrating a statistically significant negative correlation with video generation accuracy (CLIP score); on the Panda-70M dataset, this correlation yielded a Kendall’s rank correlation p-value of 0.001. The principal implication for AI practitioners is that S-QUBED provides a model-agnostic tool to assess the confidence of generated videos, allowing engineers to identify and flag low-confidence or potentially inaccurate outputs without needing access to model internals or performing retraining. |
| TalkPlay-Tools: Conversational Music Recommendation with LLM Tool |
|
|
| Calling (Read more on arXiv or HuggingFace) |
Juhan Nam, Keunwoo Choi, Seungheon Doh |
This paper presents TalkPlay-Tools, a conversational music recommendation system that uses a Large Language Model (LLM) as an agent to orchestrate a pipeline of diverse retrieval and reranking tools. The main objective is to create a unified framework that can interpret multi-turn user intent to dynamically plan and execute a sequence of tools, including SQL, BM25, dense retrieval, and generative retrieval with Semantic IDs. The methodology centers on guiding a Qwen3-LM with a structured three-stage prompt (planning, retrieval, reranking) to call functions that query various databases based on user profiles, dialogue history, and the current query. The proposed tool-calling system achieves a Hit@1 of 0.022 in a zero-shot setting, outperforming a generative baseline with BM25 (0.018), and demonstrates high success rates for tools utilizing rich in-context information, such as User-to-Item (98.8%), but a low success rate for syntactically complex tools like SQL (24.7%). The principal implication for AI practitioners is that this LLM-based agentic architecture provides a viable method for integrating multiple, heterogeneous retrieval systems into a single, flexible conversational recommender, underscoring the critical need for rich in-context information to ensure reliable tool execution. |
| A Practitioner’s Guide to Multi-turn Agentic Reinforcement Learning (Read more on arXiv or HuggingFace) |
|
This paper systematically investigates and provides a practical recipe for training LLM agents using multi-turn reinforcement learning by analyzing the pillars of environment, reward, and policy. The main objective is to determine which design choices are most effective for multi-turn agentic RL, addressing the lack of systematic formulation in prior work. The methodology involves empirically evaluating agents on TextWorld, ALFWorld, and SWE-Gym, ablating factors like environment complexity, reward sparsity, SFT-to-RL ratios, and comparing biased (PPO, GRPO) versus unbiased (RLOO) algorithms. Key results show that combining SFT with RL is highly sample-efficient; an SFT prior from 60 demonstrations plus 400 RL episodes achieved 85% success, nearly matching the 88% from 5000 pure RL episodes. The principal implication for AI practitioners is that initializing agent policies with a small amount of demonstration data via SFT drastically reduces the need for expensive online RL episodes, and that biased algorithms like PPO are more robust for these complex, sequential decision-making tasks. |
| DiffTester: Accelerating Unit Test Generation for Diffusion LLMs via |
|
|
| Repetitive Pattern (Read more on arXiv or HuggingFace) |
Jia Li, Yitong Zhang, Yuetong Liu, wellbeing |
DIFFTESTER is a framework that accelerates unit test generation for diffusion LLMs by identifying and jointly generating repetitive code patterns across multiple test cases. The main objective is to increase the inference speed of diffusion LLMs for automated unit test generation (UTG) without degrading the quality or coverage of the resulting tests. The key methodology involves parsing partially generated code into Abstract Syntax Trees (ASTs) at intermediate decoding steps, identifying common subtrees across a batch of test cases, and then unmasking all tokens corresponding to these shared patterns in a single operation. Primary results demonstrate significant acceleration, with throughput on the TestEval-C++ benchmark increasing by up to 2.45x using the DiffuCoder model while preserving maximum achievable test coverage. For AI practitioners, this task-specific approach offers a practical method to substantially reduce the latency and computational cost of generating large volumes of unit tests, making diffusion model-based testing more efficient for software development. |
| Align Your Tangent: Training Better Consistency Models via |
|
|
| Manifold-Aligned Tangents (Read more on arXiv or HuggingFace) |
Jong Chul Ye, Byunghee Cha, Beomsu Kim |
This paper introduces Align Your Tangent (AYT), a method that accelerates and stabilizes the training of Consistency Models (CMs) by using a self-supervised manifold feature distance (MFD) loss. The main objective is to address the slow convergence of CMs, which the authors identify as being caused by update directions (tangents) that oscillate parallel to the data manifold instead of pointing towards it. The key methodology involves training an auxiliary network to create a feature space that is sensitive to various off-manifold data perturbations (e.g., geometric, color, degradation); the distance in this learned feature space is then used as the CM’s loss function, which forces tangents to align orthogonally to the manifold. On unconditional CIFAR10, AYT improves the 1-step generation FID from 3.60 to 2.61 and demonstrates robustness to training with batch sizes as small as 16, outperforming a baseline trained with a batch size of 128. For AI practitioners, AYT offers a self-supervised and interpretable loss function that can replace standard losses to train CMs more efficiently and robustly, reducing compute requirements without relying on human-curated perceptual datasets or complex training schedules. |
| NuRisk: A Visual Question Answering Dataset for Agent-Level Risk |
|
|
| Assessment in Autonomous Driving (Read more on arXiv or HuggingFace) |
|
This paper introduces NuRisk, a comprehensive Visual Question Answering dataset for quantitative, agent-level risk assessment in autonomous driving, demonstrating that specialized fine-tuning is necessary to overcome the limitations of general pre-trained models. The core objective is to evaluate and improve the spatio-temporal reasoning capabilities of Vision Language Models (VLMs) for predicting how risks evolve over time. The methodology involves creating a 1.1M-sample dataset from nuScenes, Waymo, and CommonRoad with sequential Bird-Eye-View images and quantitative risk annotations, then using it to benchmark existing VLMs and fine-tune a 7B parameter agent. The primary result is that leading proprietary VLMs peak at 33% accuracy and completely fail at explicit spatio-temporal reasoning, whereas the fine-tuned NuRisk agent achieves 41% accuracy while demonstrating these crucial reasoning capabilities. The principal implication for AI practitioners is that deploying VLMs in safety-critical applications requires domain-specific fine-tuning on specialized, quantitative datasets, as prompt-based adaptation of general models is insufficient for reliable performance. |
| Triangle Splatting+: Differentiable Rendering with Opaque Triangles (Read more on arXiv or HuggingFace) |
Matheus Gadelha, Daniel Rebain, Sanghyun Son, Renaud Vandeghen, Jan Held |
i) Triangle Splatting+ is a differentiable rendering framework that directly optimizes a semi-connected mesh of opaque triangles for high-quality novel view synthesis compatible with standard graphics engines. ii) The primary objective is to develop an end-to-end optimizable, mesh-based 3D scene representation that eliminates the need for post-processing steps like mesh extraction, making it directly usable in real-time applications. iii) The methodology introduces a shared-vertex triangle parameterization for connectivity and a tailored training strategy that anneals a global smoothness parameter while enforcing an increasing opacity floor to converge on a sharp, fully opaque mesh. iv) On the Mip-NeRF360 dataset, the method achieves a PSNR of 25.21, outperforming other state-of-the-art mesh-based approaches such as MiLo (24.09 PSNR) while using fewer vertices. v) The principal implication for AI practitioners is a method to generate high-fidelity 3D scene assets that are natively compatible with existing graphics pipelines, enabling direct integration into game engines and VR applications for physics simulation and interactive walkthroughs without conversion. |
| Scaling Policy Compliance Assessment in Language Models with Policy |
|
|
| Reasoning Traces (Read more on arXiv or HuggingFace) |
|
The paper introduces Policy Reasoning Traces (PRTs), which are reasoning chains generated by a powerful pseudo-expert language model to improve policy compliance assessment in other LLMs. The research objective is to create a scalable method for generating expert-like reasoning demonstrations to enhance LLM performance on rule-based tasks without requiring expensive human-annotated rationales. The methodology involves prompting a frontier model like DEEPSEEK-R1 with a case, its verdict, and the relevant policy to generate a PRT, which is then used either as a few-shot in-context learning example or as data for supervised fine-tuning of a learner model. The primary results show that using PRTs as few-shot demonstrations on the HIPAA policy boosted the accuracy of open-weight models by 16-30 percentage points and established a new state-of-the-art accuracy of 81.0% on GDPR policy compliance. The principal implication for AI practitioners is that they can leverage a large model to generate synthetic reasoning data to significantly improve the performance and interpretability of smaller or more general models on specific, rule-based tasks like legal or safety compliance. |
Papers for 2025-10-03
| Title |
Authors |
Summary |
| LongCodeZip: Compress Long Context for Code Language Models (Read more on arXiv or HuggingFace) |
|
LongCodeZip is a training-free framework designed to compress long code contexts for Large Language Models using a dual-stage, perplexity-based strategy. The objective is to develop a code-specific context compression method that preserves structural and semantic information critical for programming tasks, overcoming the limitations of generic text compressors. The methodology consists of a coarse-grained stage that ranks and selects function-level chunks using conditional perplexity, followed by a fine-grained stage that segments retained functions into semantic blocks and selects an optimal subset using a 0/1 knapsack algorithm. Evaluations show the framework achieves up to a 5.6x compression ratio on tasks like code completion while maintaining performance comparable to models using the full, uncompressed context. For AI practitioners, this model-agnostic, plug-and-play tool enables the use of LLMs on large codebases with significantly reduced API costs and latency, even when using a small 0.5B parameter model for the compression step. |
| Self-Forcing++: Towards Minute-Scale High-Quality Video Generation (Read more on arXiv or HuggingFace) |
|
Self-Forcing++ is a method that extends autoregressive video generation to minute-scale duration by mitigating error accumulation. The objective is to overcome the quality degradation that occurs when student models generate videos longer than the short horizon of their bidirectional teacher models, without requiring long-video datasets for retraining. The key methodology involves having the student model generate long, error-accumulated rollouts and then using the short-horizon teacher to provide corrective guidance on sampled segments of these rollouts via extended distribution-matching distillation and a rolling KV cache. This approach achieves generation of videos up to 4 minutes and 15 seconds, a 50x improvement over the baseline, and on 100-second videos, it attains a dynamic degree of 54.12, outperforming the Self-Forcing baseline by 104.9%. For AI practitioners, this presents a framework to significantly extend the temporal capabilities of autoregressive models by using a short-horizon teacher for self-correction on extrapolated outputs, thereby circumventing the need for extensive long-duration training data. |
| StealthAttack: Robust 3D Gaussian Splatting Poisoning via Density-Guided |
|
|
| Illusions (Read more on arXiv or HuggingFace) |
|
This paper introduces StealthAttack, a robust data poisoning method for 3D Gaussian Splatting that embeds viewpoint-dependent illusions by strategically placing poison points in low-density regions and disrupting multi-view consistency. The primary objective is to develop an effective and stealthy data poisoning attack against 3D Gaussian Splatting (3DGS) that can inject a visible illusory object into a specific target viewpoint while minimally affecting the rendering quality of all other non-target viewpoints. The methodology combines a “Density-Guided Point Cloud Attack,” which uses Kernel Density Estimation (KDE) to identify and place poison Gaussian points in low-density scene regions, and a “View Consistency Disruption Attack,” which applies scheduled adaptive Gaussian noise to innocent training views to weaken 3DGS’s multi-view consistency property. The proposed method significantly outperforms baseline attacks, achieving a V-ILLUSORY PSNR of 27.04 dB on the poisoned view for the Mip-NeRF 360 dataset, compared to 17.60 dB from the best-performing baseline, while maintaining high fidelity (27.76 dB PSNR) on innocent views. For AI practitioners, this work reveals a critical vulnerability in 3DGS models: their foundational multi-view consistency can be subverted to embed malicious content, necessitating robust data validation and model verification pipelines before deploying 3DGS in security-sensitive applications. |
| ExGRPO: Learning to Reason from Experience (Read more on arXiv or HuggingFace) |
Dongrui Liu, Xiaoye Qu, Zhi Wang, Yafu Li, Runzhe Zhan |
ExGRPO is a framework that enhances large language model reasoning by systematically managing and replaying valuable experiences within Reinforcement Learning from Verifiable Rewards (RLVR). The research investigates what makes a reasoning experience valuable and how to exploit it to overcome the sample inefficiency of on-policy RLVR. The key methodology involves maintaining a replay buffer, partitioning successful trajectories into buckets based on correctness, prioritizing medium-difficulty questions, and selecting the lowest-entropy trajectories for replay with a mixed-policy objective. ExGRPO achieves an average performance gain of +3.5 points on in-distribution mathematical benchmarks and +7.6 points on out-of-distribution benchmarks over on-policy RLVR, while also stabilizing training for weaker models where on-policy methods collapse. For AI practitioners, this demonstrates that principled experience management based on rollout correctness and entropy is a crucial technique for improving the sample efficiency and stability of RL fine-tuning for large reasoning models. |
| StockBench: Can LLM Agents Trade Stocks Profitably In Real-world |
|
|
| Markets? (Read more on arXiv or HuggingFace) |
Jianing Yu, Jin Ye, Yantao Liu, Zijun Yao, Yanxu Chen |
This paper introduces STOCKBENCH, a contamination-free benchmark designed to evaluate the profitability and risk management of LLM agents in realistic stock trading simulations. The objective is to assess if LLM agents can make sequential, profitable trading decisions in a dynamic, multi-month market environment using real-world data streams. The methodology consists of a back-trading workflow where agents receive daily prices, fundamentals, and news for 20 DJIA stocks and must issue buy, sell, or hold commands, with performance evaluated by cumulative return, maximum drawdown, and Sortino ratio. The primary result is that while most LLM agents fail to outperform a simple buy-and-hold baseline, some models like Qwen3-235B-Think achieved higher returns (2.5% vs 0.4% baseline), but excelling at reasoning tasks does not guarantee superior trading performance. The principal implication for AI practitioners is that an LLM’s performance on static knowledge benchmarks does not translate to effective decision-making in dynamic, high-stakes environments, necessitating agent-specific evaluation frameworks that test sequential action and adaptation. |
| Interactive Training: Feedback-Driven Neural Network Optimization (Read more on arXiv or HuggingFace) |
|
This paper introduces Interactive Training, a framework for real-time, feedback-driven intervention in neural network training by humans or AI agents. The primary objective is to create and validate a framework that shifts neural network optimization from a static, predefined process to a dynamic one, allowing for mid-training adjustments to improve stability and performance. The methodology involves a three-part system: a FastAPI control server to manage commands, an Interactive Trainer built on Hugging Face’s Trainer class that applies interventions via callbacks, and a React-based frontend for visualization and user input. In a GPT-2 finetuning experiment on Wikitext-2, human-in-the-loop interactive training achieved a lower final validation loss (approx. 4.5) compared to a static learning rate schedule (approx. 5.0), and an LLM-based agent successfully stabilized a training run initiated with an excessively high learning rate that otherwise failed to converge. The principal implication for AI practitioners is the ability to actively debug, steer, and adapt training runs in real-time, reducing compute cycles wasted on failed experiments and enabling continuous model improvement based on live feedback or observed instabilities. |
| VOGUE: Guiding Exploration with Visual Uncertainty Improves Multimodal |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
|
The paper introduces VOGUE, a method that improves multimodal reasoning in MLLMs by guiding reinforcement learning exploration using the uncertainty derived from visual input perturbations. The primary objective is to address the exploration problem in multimodal RLVR by leveraging the inherent uncertainty of visual inputs to build more robust reasoning policies. VOGUE employs a dual-branch architecture that processes both an original and a stochastically augmented image; it quantifies visual uncertainty as the symmetric KL divergence between the two resulting policy distributions, using this signal to shape the learning objective with an uncertainty-proportional advantage bonus and an annealed sampling schedule. Implemented on Qwen2.5-VL-7B, VOGUE increased pass@1 accuracy over the GRPO baseline by an average of 2.6% across three visual math benchmarks and 3.7% across three general-domain reasoning benchmarks, while also improving pass@4 performance. For AI practitioners, the principal implication is that MLLM robustness and reasoning can be enhanced by incorporating input-space exploration—specifically by perturbing visual inputs to quantify model uncertainty and using this signal to directly guide the RL fine-tuning process—rather than relying solely on output-space exploration strategies. |
| The Rogue Scalpel: Activation Steering Compromises LLM Safety (Read more on arXiv or HuggingFace) |
Ivan Oseledets, Oleg Y. Rogov, Alexey Dontsov, Andrey Galichin, Anton Korznikov |
This paper demonstrates that activation steering systematically compromises LLM safety, showing that adding even random or benign vectors to a model’s hidden states can induce compliance with harmful requests. The research objective was to systematically quantify the safety vulnerabilities introduced by activation steering, a technique often framed as a precise alternative to fine-tuning. Using the JailbreakBench dataset and an LLM-as-judge, the authors applied steering vectors from random distributions and sparse autoencoders (SAEs) to the residual streams of models from the Llama3, Qwen2.5, and Falcon families. Key results show that random steering alone can increase harmful compliance from 0% to 2-27%, and a universal attack vector, created by averaging just 20 vectors that jailbreak a single prompt, increases compliance on unseen prompts by an average of 4x. The principal implication for AI practitioners is that activation steering presents a critical, exploitable vulnerability; systems exposing this capability, even for seemingly benign control, are susceptible to black-box attacks that can reliably bypass alignment safeguards. |
| CLUE: Non-parametric Verification from Experience via Hidden-State |
|
|
| Clustering (Read more on arXiv or HuggingFace) |
Dian Yu, Linfeng Song, Yujun Zhou, Ruosen Li, Zhenwen Liang |
This paper introduces CLUE, a non-parametric verifier that classifies the correctness of LLM outputs by clustering hidden-state activation trajectories from past experience. The primary objective is to determine if an LLM’s internal hidden-state trajectory contains a geometrically separable signal for correctness that can be leveraged by a simple, training-free verifier to outperform text-level and confidence-based methods. CLUE operates by first computing “success” and “failure” centroids from the mean activation-deltas (the difference in hidden states before and after the reasoning block) of a labeled experience set; new solutions are then classified based on their layer-averaged Euclidean distance to these two centroids. Empirically, CLUE outperforms LLM-as-a-judge and confidence-based baselines; on the AIME 24 benchmark with a 1.5B model, CLUE improves reasoning accuracy from 56.7% (majority@64) to 70.0% (top-maj@16 reranking). The principal implication for AI practitioners is that an LLM’s internal reasoning geometry, particularly after RL-based fine-tuning, provides a robust, low-cost signal for verification, enabling the creation of lightweight, training-free verifiers that can significantly improve reasoning accuracy by reranking candidate solutions without expensive external judges. |
| RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via |
|
|
| Multi-Stage Reinforcement Learning (Read more on arXiv or HuggingFace) |
|
The paper introduces REWARDMAP, a multi-stage reinforcement learning framework to enhance fine-grained visual reasoning in MLLMs by addressing sparse reward challenges in tasks like transit map navigation. The main research objective is to develop a training strategy that improves MLLM capabilities in both detailed visual understanding and complex spatial reasoning, where standard methods fail. The key methodology combines a new dataset, REASONMAP-PLUS, for dense reward signals with a multi-stage RL curriculum that progresses from simple to complex tasks, utilizing a difficulty-aware reward design that grants partial credit for correct reasoning steps. Models trained with REWARDMAP achieve an average performance improvement of 3.47% across six different visual reasoning benchmarks, demonstrating enhanced generalization. The principal implication for AI practitioners is that a curriculum-based RL approach with a granular, shaped reward signal is a more effective strategy than standard SFT or basic RL for fine-tuning MLLMs on specialized, multi-step visual tasks with inherently sparse supervision. |
| RLP: Reinforcement as a Pretraining Objective (Read more on arXiv or HuggingFace) |
|
This paper introduces Reinforcement Learning Pre-training (RLP) to investigate if incorporating RL during pretraining is a more optimal method for developing reasoning in language models than reserving it for post-training. The key methodology treats chain-of-thought generation as an action and computes a dense, verifier-free reward based on the information gain—the log-likelihood increase for the next token when conditioned on the generated thought—relative to a no-think Exponential Moving Average (EMA) baseline. The primary result shows that pretraining the Qwen3-1.7B-BASE model with RLP lifts its average performance across an eight-benchmark math-and-science suite by 19% over the baseline. The principal implication for AI practitioners is that RLP offers a scalable, domain-agnostic pretraining objective that can instill robust reasoning abilities using general-purpose corpora, with gains that compound during subsequent alignment stages. |
| DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag |
|
|
| Editing (Read more on arXiv or HuggingFace) |
Zhuming Lian, Shaocong Zhang, Shuli Leng, Shilin Lu, Zihan Zhou |
DragFlow is the first framework to enable high-fidelity, region-based drag editing on Diffusion Transformer (DiT) models by replacing point-wise supervision with an affine transformation-based regional approach. The objective is to effectively harness the strong generative priors of modern DiT models for drag-based image editing, overcoming the limitations of previous point-based methods which perform poorly on DiT feature representations. The key methodology introduces region-level motion supervision using affine transformations, enforces background preservation with gradient mask-based hard constraints, and enhances subject consistency in CFG-distilled models via a pretrained personalization adapter. On the newly curated ReD Bench, DragFlow achieves state-of-the-art spatial accuracy, outperforming baselines with a Mean Distance (MD1) of 19.46, while demonstrating superior feature preservation. For AI practitioners, the principal implication is that robust drag-style editing can be applied to powerful DiT-based models by adopting region-level supervision, which is better suited for the fine-grained feature maps of transformers than traditional point-level techniques, enabling more controllable and higher-quality image manipulation. |
| The Unreasonable Effectiveness of Scaling Agents for Computer Use (Read more on arXiv or HuggingFace) |
|
The paper introduces Behavior Best-of-N (bBoN), a method that significantly improves computer-use agent (CUA) performance by generating multiple complete trajectories and selecting the best one using structured textual summaries. The objective is to mitigate the unreliability and high variance of CUAs on long-horizon tasks by developing an effective “wide scaling” framework that leverages multiple agent rollouts. The bBoN methodology first employs a Vision-Language Model (VLM) to convert each raw trajectory into a “behavior narrative”—a concise sequence of action-effect facts—and then a separate VLM-based judge performs a comparative evaluation on these narratives to select the optimal trajectory. On the OSWorld benchmark, bBoN establishes a new state-of-the-art success rate of 69.9% at 100 steps, a 10% absolute improvement over the previous best of 59.9% and approaching the 72% human-level performance benchmark. The principal implication for AI practitioners is that they can enhance agentic system robustness by parallelizing entire task rollouts and implementing a comparative selection mechanism over structured trajectory representations, rather than relying on single agent executions or step-wise decision scaling. |
| Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation (Read more on arXiv or HuggingFace) |
|
The paper introduces OVI, a unified generative model that produces synchronized audio and video in a single pass using a symmetric twin-backbone diffusion transformer architecture. The objective is to create an end-to-end system for joint audio-video generation that eliminates the need for separate pipelines or post-hoc alignment. The methodology involves coupling two architecturally identical Diffusion Transformers (DiTs) via blockwise, bidirectional cross-attention and aligning their different temporal resolutions using scaled Rotary Positional Embeddings (RoPE). In pairwise human preference studies, OVI was preferred for audio-visual synchronization over the JavisDiT baseline in 79.3% of comparisons. For AI practitioners, this work provides a scalable framework demonstrating that architectural symmetry and deep cross-modal fusion can achieve inherent synchronization, offering a robust template for developing simpler and more coherent multimodal generative systems. |
| Learning to Reason for Hallucination Span Detection (Read more on arXiv or HuggingFace) |
Hadi Pouransari, Kundan Krishna, Hema Swetha Koppula, Ting-Yao Hu, Hsuan Su |
This paper introduces RL4HS, a reinforcement learning framework using span-level rewards to train large language models to reason and detect specific hallucinated spans in text. The primary research objective is to determine if a learned, task-specific reasoning process is more effective for hallucination span detection than prompting general-purpose reasoning models or standard fine-tuning. The key methodology involves using Group Relative Policy Optimization (GRPO) with a span-F1 reward function and introducing Class-Aware Policy Optimization (CAPO) to address reward imbalance between hallucination and non-hallucination classes. On the RAGTruth benchmark, the proposed RL4HS-14B model achieves an average span-F1 score of 58.3, outperforming both supervised fine-tuning (55.4) and pretrained reasoning models. The principal implication for AI practitioners is that for complex, multi-step NLP tasks like hallucination span detection, reinforcement learning with fine-grained, task-specific rewards is a more effective alignment strategy than prompting general reasoning models or using supervised fine-tuning alone. |
| TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP |
|
|
| Environments (Read more on arXiv or HuggingFace) |
|
TOUCAN introduces a 1.5 million-sample dataset of tool-agentic trajectories synthesized from real-world Model Context Protocol (MCP) environments to train capable LLM agents. The objective is to address the scarcity of high-quality, large-scale, and permissively licensed tool-agentic data by creating a pipeline to generate realistic training trajectories involving authentic tool execution from a broad set of MCPs. The methodology is a five-stage pipeline that onboards and filters 495 real-world MCP servers, synthesizes tasks using five LLMs, applies model-based quality filtering, generates trajectories with three teacher models performing real tool execution via agentic frameworks, and conducts rule-based and model-based post-filtering. Models fine-tuned on TOUCAN show significant performance gains; a fine-tuned Qwen2.5-32B model improved its overall score on the BFCL V3 benchmark by 8.72% (from 61.73% to 70.45%), surpassing larger closed-source models. For AI practitioners, TOUCAN provides a large-scale, permissively licensed dataset for supervised fine-tuning to substantially improve the tool-calling and agentic reasoning of open-source LLMs, enabling the creation of more robust agents. |
| F2LLM Technical Report: Matching SOTA Embedding Performance with 6 |
|
|
| Million Open-Source Data (Read more on arXiv or HuggingFace) |
|
F2LLM is a suite of text embedding models that matches state-of-the-art performance using only 6 million curated, open-source, non-synthetic data tuples. The primary objective was to develop a high-performing embedding model that is reproducible and budget-friendly, avoiding the massive pretraining, complex pipelines, and costly synthetic data used by previous top models. The methodology involves directly fine-tuning Qwen3 foundation models in a single stage with a contrastive loss objective on a unified dataset of query-document-negative tuples compiled from various open-source datasets. The F2LLM-4B model achieves an average score of 73.67 on the MTEB leaderboard, ranking 7th overall and 2nd among models of its size, while F2LLM-1.7B ranks 1st in the 1B-2B size range. For AI practitioners, this research provides a fully open-source (models, data, code) and cost-effective blueprint for creating powerful embedding models, demonstrating that meticulous data curation can be a viable alternative to massive-scale pretraining and synthetic data generation. |
| Go with Your Gut: Scaling Confidence for Autoregressive Image Generation (Read more on arXiv or HuggingFace) |
Disen Lan, Rongjin Guo, Wen-Jie Shu, Xianfeng Wu, Harold Haodong Chen |
ScalingAR is a test-time scaling framework that improves autoregressive image generation by dynamically pruning low-quality generation paths and adjusting guidance strength using a novel confidence score derived from token entropy. The main objective is to develop an efficient test-time scaling strategy for next-token prediction (NTP) based image generation that avoids the need for partial decoding or external reward models. The key methodology involves a Dual-Channel Confidence Profile that fuses an Intrinsic Channel (token-level uncertainty and worst-block spatial stability) with a Conditional Channel (text utilization strength measured by KL divergence) to guide policies for adaptive termination and dynamic scheduling of Classifier-Free Guidance. Experiments show ScalingAR improves the LlamaGen base model by 15.2% on the TIIF-Bench benchmark and reduces visual token consumption by 62.0% compared to classic scaling baselines like Best-of-N, while achieving higher quality. The principal implication for AI practitioners is that they can use this inference-time technique to enhance the performance and efficiency of pre-trained autoregressive image generators without any model retraining, by leveraging intrinsic model signals to guide the sampling process. |
| Visual Multi-Agent System: Mitigating Hallucination Snowballing via |
|
|
| Visual Flow (Read more on arXiv or HuggingFace) |
Zhangquan Chen, Yongbo He, Guibin Zhang, Chengming Xu, Xinlei Yu |
This paper introduces Visual Flow (ViF), a lightweight method to mitigate the compounding of visual errors, termed “hallucination snowballing,” in Visual Language Model (VLM) based Multi-Agent Systems (MAS). The research objective is to diagnose this phenomenon, which is attributed to diminishing visual attention across agent turns, and to develop a mitigation strategy that preserves visual fidelity. The key methodology involves identifying a critical subset of vision tokens with a unimodal attention peak, which best preserve visual evidence, and directly relaying them as an auxiliary “Visual Flow” between agents, augmented by an attention reallocation mechanism. Experiments demonstrate that ViF consistently reduces hallucination snowballing, achieving a 39.8% average reduction in the Hallucination Snowballing (HS) score for the LLaVA-NeXT-7B model within a circular MAS structure. For AI practitioners, the principal implication is that robust VLM-based MAS design must supplement inter-agent textual communication with a direct visual information relay to prevent the propagation and amplification of initial perception errors. |
| VideoNSA: Native Sparse Attention Scales Video Understanding (Read more on arXiv or HuggingFace) |
Xiaojun Shan, Ethan Armand, Shusheng Yang, Wenhao Chai, Enxin Song |
VideoNSA introduces a hardware-aware, learnable sparse attention mechanism for video-language models to efficiently process ultra-long video contexts. The research objective is to overcome the context length limitations of multimodal models for video understanding by developing a scalable attention mechanism that maintains performance on complex reasoning tasks. The method adapts the Qwen2.5-VL model with a hybrid attention strategy, applying standard grouped-query attention to text and Native Sparse Attention (NSA) to video tokens, which dynamically combines token compression, selection, and sliding window branches via learnable gates. VideoNSA demonstrates improved performance across long-video benchmarks, achieving leading results on tasks like temporal reasoning while using only 3.6% of the full attention budget for a 128K token context. For AI practitioners, this hybrid sparse attention framework provides a scalable method for building video foundation models capable of processing significantly longer contexts, such as hour-long videos, without prohibitive computational costs. |
| Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and |
|
|
| Reasoning in Vision-Language Models (Read more on arXiv or HuggingFace) |
Yukun Qi, Xikun Bao, Shiting Huang, Wenxuan Huang, Yu Zeng |
The paper introduces AGILE, an agentic reinforcement learning framework that uses interactive jigsaw puzzle solving to enhance the visual perception and reasoning of Vision-Language Models. The objective is to overcome the limitations of current VLMs in core reasoning and the scarcity of high-quality multimodal training data by using a scalable proxy task. The methodology formulates jigsaw solving as an interactive process where the VLM generates Python code to perform actions (e.g., Swap, Crop, Zoom), receives visual feedback from an environment, and is trained via a cold-start phase with expert trajectories followed by reinforcement learning. Primary results demonstrate that AGILE increases accuracy on 2x2 jigsaw tasks from 9.5% to 82.8% and achieves an average performance improvement of 3.1% across nine general vision benchmarks. The principal implication for AI practitioners is that programmatically generated, interactive proxy tasks can serve as a scalable and efficient alternative to scarce, human-annotated data for improving the fundamental reasoning and generalization capabilities of VLMs. |
| Automated Structured Radiology Report Generation with Rich Clinical |
|
|
| Context (Read more on arXiv or HuggingFace) |
Won Hwa Kim, Dongseop Kim, Juho Jung, Dong Bok Lee, Seongjae Kang |
This paper introduces Contextualized Structured Radiology Report Generation (C-SRRG), a framework that incorporates rich clinical context to improve the accuracy and mitigate temporal hallucinations in automated reports. The research objective is to enhance SRRG systems by enabling them to utilize clinical information—multi-view images, indications, techniques, and prior studies—mirroring the diagnostic workflow of radiologists. The methodology involves curating a new C-SRRG dataset by integrating this clinical context and then fine-tuning state-of-the-art medical multimodal large language models (MLLMs) on this data. Incorporating clinical context significantly improves performance, reducing temporal hallucinations by up to 18.0% for impression generation and increasing the F1-SRR-BERT score by up to +7.1 for the Lingshu-7B model. The principal implication for AI practitioners is that explicitly conditioning models on comprehensive, domain-specific context is a critical strategy for improving factual accuracy and reducing hallucinations, with this effect becoming more pronounced as model scale increases. |
| Optimal Control Meets Flow Matching: A Principled Route to Multi-Subject |
|
|
| Fidelity (Read more on arXiv or HuggingFace) |
Thomas Hofmann, Enis Simsar, Eric Tillmann Bill |
This paper introduces a stochastic optimal control (SOC) framework for flow matching (FM) models to improve multi-subject fidelity in text-to-image generation. The main objective is to develop a principled, optimizable objective for steering sampling dynamics to mitigate attribute leakage, identity entanglement, and subject omissions. The key methodology formulates subject disentanglement as a control problem over a trained FM sampler, yielding two algorithms: a training-free test-time controller and a lightweight fine-tuning rule called Adjoint Matching. The primary result is that the proposed FOCUS method achieves state-of-the-art multi-subject fidelity, attaining a composite improvement score of 5.9174 on Stable Diffusion 3.5 when fine-tuned, significantly outperforming prior attention-based heuristics. For AI practitioners, this research provides two architecture-agnostic, principled algorithms to systematically improve the compositional abilities of modern T2I models, either through a fast test-time intervention or a lightweight fine-tuning process that generalizes from limited data. |
| Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming |
|
|
| Attacks (Read more on arXiv or HuggingFace) |
Alan Ritter, Miguel Ballesteros, Roshan Sridhar, Afshin Oroojlooy, Ruohao Guo |
This paper introduces DIALTREE-RPO, a reinforcement learning framework for discovering multi-turn red-teaming attacks against large language models. The primary objective is to autonomously discover diverse and effective multi-turn attack strategies by framing the red-teaming dialogue as a sequential decision-making problem. The proposed methodology, DIALTREE-RPO, is an on-policy RL framework that integrates dialogue tree rollout with pruning for structured exploration and an adaptive masking technique to stabilize training. The approach achieved an average Attack Success Rate (ASR) of 85.3% across 10 target LLMs, outperforming previous state-of-the-art methods by more than 25.9%. For AI practitioners, this research demonstrates that current LLMs are significantly more vulnerable to strategic, multi-turn conversational attacks than to single-turn attacks, highlighting the necessity for developing more robust, context-aware safety mechanisms. |
| A Rigorous Benchmark with Multidimensional Evaluation for Deep Research |
|
|
| Agents: From Answers to Reports (Read more on arXiv or HuggingFace) |
Tianle Gu, Yi Lu, Yuxuan Zhang, Yixu Wang, Yang Yao |
This research introduces Rigorous Bench, a benchmark with a multidimensional evaluation framework to assess Deep Research Agents (DRAs) that generate long-form reports. The main objective is to develop a benchmark tailored for report-style outputs from DRAs, enabling a comprehensive assessment of integrated capabilities like task decomposition, cross-source retrieval, and structured synthesis, which existing short-text benchmarks cannot evaluate. The key methodology involves the “Rigorous Bench” dataset, comprising 214 expert-curated queries with reference bundles containing specific rubrics, trustworthy source links (TSLs), and focus keywords. Evaluation is performed using an IntegratedScore, which is a multiplicative product of three metrics: Semantic Quality, Topical Focus (as 1 - SemanticDrift), and Retrieval Trustworthiness. The primary results from evaluating 13 models show that DRAs consistently outperform web-search-tool-augmented models, with Qwen-deep-research achieving the highest IntegratedScore of 34.6480, although it did not lead in every individual sub-metric. The principal implication for AI practitioners is that they can use this benchmark and framework to conduct granular capability assessments of their agent systems, moving beyond content quality to systematically optimize for retrieval trustworthiness and thematic consistency, thereby guiding the development of more robust and reliable DRAs. |
| ModernVBERT: Towards Smaller Visual Document Retrievers (Read more on arXiv or HuggingFace) |
|
This research introduces ModernVBERT, a compact 250M-parameter vision-language encoder that establishes a leading performance-size tradeoff for visual document retrieval. The paper’s objective is to systematically identify which design choices—including attention mechanisms, modality alignment, and contrastive training regimes—best enhance the performance of modern visual document retrievers. Through controlled experiments, the study employs a two-stage training process: first, aligning a pretrained bidirectional text encoder with a vision encoder using a Masked Language Modeling (MLM) objective, followed by contrastive post-training with InfoNCE loss. The primary result shows that using a native bidirectional attention mechanism with multi-vector late interaction significantly boosts performance, outperforming an equivalent causal decoder model by +10.6 nDCG@5 on the ViDoRe benchmark. The principal implication for AI practitioners is that for building efficient and powerful document retrieval systems, it is more effective to use purpose-built bidirectional encoders rather than repurposing larger, causal generative VLMs, as this allows for superior performance with late interaction at a fraction of the computational cost. |
| Transformers Discover Molecular Structure Without Graph Priors (Read more on arXiv or HuggingFace) |
|
A standard Transformer architecture can learn molecular structure and predict energies and forces competitively with specialized Graph Neural Networks (GNNs) without any explicit graph-based priors. The main objective was to investigate whether an unmodified Transformer, trained directly on Cartesian coordinates, can approximate molecular energies and forces without predefined graphs or physical inductive biases. The methodology involved training a LLaMA2-based Transformer on the OMol25 dataset using a two-stage procedure: autoregressive pre-training on discretized molecular sequences, followed by fine-tuning with a bi-directional attention mask to regress continuous energy and force values. The primary result is that under a matched compute budget, the 1B parameter Transformer achieved a force Mean Absolute Error of 18.35 meV/Å, comparable to the 13.01 meV/Å of a state-of-the-art equivariant GNN, while being faster in wall-clock time. The principal implication for AI practitioners is that for complex scientific domains like molecular modeling, general-purpose, scalable architectures like Transformers can be a viable alternative to highly specialized models, potentially simplifying development and leveraging mature hardware/software ecosystems without the need to hard-code domain-specific inductive biases. |
| Rethinking the shape convention of an MLP (Read more on arXiv or HuggingFace) |
|
This paper proposes an inverted “wide-narrow-wide” (Hourglass) MLP architecture that places skip connections in a higher-dimensional space, demonstrating superior parameter efficiency over conventional designs. The main objective is to test the hypothesis that performing incremental residual updates in an expanded-dimensional space is more effective than in the narrower input/output space of conventional MLPs. The methodology involves comparing the proposed Hourglass MLP against conventional “narrow-wide-narrow” MLPs on generative image tasks, systematically searching architectural parameters to characterize and compare their performance-parameter Pareto frontiers. Results show that Hourglass architectures consistently achieve superior Pareto frontiers; for example, on ImageNet-32 denoising, an Hourglass model reaches 22.31 dB PSNR with 66M parameters, while a conventional model requires 75M parameters for the same score. The study also finds that an initial fixed random projection to the expanded space yields performance comparable to a fully trained projection. The principal implication for AI practitioners is that in residual architectures, inverting the standard MLP shape to “wide-narrow-wide” can yield more parameter-efficient models, and the necessary input up-projection can be a fixed random matrix, which saves parameters and potentially reduces memory bandwidth. |
| VLA-R1: Enhancing Reasoning in Vision-Language-Action Models (Read more on arXiv or HuggingFace) |
Dapeng Zhang, Xiaofeng Wang, Boyuan Wang, Zeyu Zhang, Angen Ye |
i) VLA-R1 is a reasoning-enhanced vision-language-action model that improves robotic manipulation by integrating Chain-of-Thought (CoT) supervision with Reinforcement Learning from Verifiable Rewards (RLVR). ii) The primary objective is to bridge the gap between reasoning and execution in VLA models by addressing their lack of explicit step-by-step reasoning and systematic post-training reinforcement. iii) The key methodology involves a two-stage training process: first, Supervised Fine-Tuning (SFT) on the newly created VLA-CoT-13K dataset, followed by post-training with Group Relative Policy Optimization (GRPO) using verifiable rewards for region alignment (GIoU), trajectory consistency (Fréchet distance), and output formatting. iv) VLA-R1 achieves state-of-the-art results, including a 36.51 Intersection over Union (IoU) on the in-domain affordance benchmark, which represents a 17.78% improvement over the strongest baseline. v) The principal implication for AI practitioners is that combining data-level explicit reasoning supervision (CoT) with optimization-level reinforcement learning using geometrically-grounded, verifiable rewards is an effective strategy for building more robust, accurate, and generalizable embodied AI systems. |
| VIRTUE: Visual-Interactive Text-Image Universal Embedder (Read more on arXiv or HuggingFace) |
Yuki Mitsufuji, Shusuke Takahashi, Qiyu Wu, Kazuya Tateishi, Wei-Yao Wang |
This paper introduces VIRTUE, a visual-interactive universal text-image embedder that integrates a segmentation model with a Vision-Language Model (VLM) to process both textual and visual interaction prompts. The research aims to develop and evaluate an embedding model that can incorporate explicit visual signals like bounding boxes to perform fine-grained, entity-aware retrieval while maintaining global scene context. The methodology combines a pretrained SAM2 segmentation model to generate entity-level embeddings from visual prompts with a Qwen2-VL model that processes these alongside global image and text embeddings for contrastive learning. VIRTUE demonstrates state-of-the-art performance, achieving improvements of 15.2%–20.3% on the new visual-interactive SCaR benchmark and 3.1%–8.5% on the MMEB universal embedding benchmark. The primary implication for AI practitioners is that this architecture provides a generic framework for building embedding systems that support direct user interaction with image regions, enabling more controllable, accurate retrieval and on-the-fly correction of model predictions at inference. |
| Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: |
|
|
| Demystifying Some Myths About GRPO and Its Friends (Read more on arXiv or HuggingFace) |
Wenhao Zhang, Yushuo Chen, Yuchang Sun, Yanxi Chen, Chaorui Yao |
This paper presents a first-principles derivation showing that group-relative REINFORCE is an inherently off-policy algorithm. The primary objective is to reinterpret group-relative REINFORCE without on-policy data assumptions, thereby providing a theoretical foundation for its use in off-policy settings and demystifying the mechanisms of related algorithms like GRPO. The key methodology involves deriving the group-relative REINFORCE update rule as a single gradient step on a surrogate loss function, which itself is designed to enforce consistency conditions from an underlying KL-regularized objective, and validating insights through empirical studies on LLM reasoning tasks. The primary results demonstrate that for GRPO-style algorithms in off-policy settings, clipping is a more critical mechanism for stability than importance sampling; for instance, on the GSM8k task with sync_interval = 20, a clipping-only variant (REC-OneSide-NoIS) with an enlarged clipping range of (0.6, 2.0) accelerated training without sacrificing stability, whereas vanilla REINFORCE collapsed. The principal implication for AI practitioners is that they can adapt REINFORCE-style algorithms for off-policy LLM training by focusing on regularization techniques like aggressive clipping—even with ranges far beyond conventional values—and employing justified data-weighting heuristics, often without needing importance sampling. |
| SKYLENAGE Technical Report: Mathematical Reasoning and |
|
|
| Contest-Innovation Benchmarks for Multi-Level Math Evaluation (Read more on arXiv or HuggingFace) |
Weiqi Zhai, Linlin Miao, Boyu Yang, Ze Xu, Hu Wei |
This paper introduces two complementary mathematical reasoning benchmarks, SKYLENAGE-ReasoningMATH and SKYLENAGE-MATH, to provide high-difficulty, fine-grained evaluation for frontier large language models. The primary objective is to overcome the ceiling effects and ability masking of existing benchmarks by creating testbeds that diagnose structural reasoning ability across multiple academic levels and subjects. The methodology involves evaluating 15 contemporary LLMs on a 100-item structure-aware diagnostic set and a 150-item contest-style suite spanning high school to doctoral levels, using a unified Chain-of-Thought protocol. The primary results show clear model separation, with the top-performing model achieving 44% accuracy on the contest suite compared to the runner-up’s 37%, and hardest-quintile analysis on the reasoning set revealing significant robustness gaps obscured by overall scores. The principal implication for AI practitioners is that single-score leaderboards are insufficient; using structured, multi-level benchmarks is critical for identifying specific model weaknesses in areas like high-difficulty reasoning and numeric density robustness, enabling targeted development and informed model selection. |
| Parallel Scaling Law: Unveiling Reasoning Generalization through A |
|
|
| Cross-Linguistic Perspective (Read more on arXiv or HuggingFace) |
|
This study investigates the cross-lingual generalization of reasoning capabilities in Large Reasoning Models (LRMs) trained with English-centric Reinforcement Post-Training (RPT). The main objective is to quantify how effectively reasoning skills transfer from English to other languages and to identify training strategies that improve this cross-lingual generalization. The methodology combines observational studies on open-source LRMs, controlled interventional studies on factors like initial model type and size, and a parallel training study using the Group Rollout Policy Optimization (GRPO) algorithm. Primary results reveal a “First-Parallel Leap,” where transitioning from monolingual to bilingual parallel training causes a disproportionate jump in the Multilingual Transferability Index (MTI) from 1.16 to 2.50, and establishes a “Parallel Scaling Law” where transferability scales as a power-law with the number of parallel languages (f(X) = 2.00 * X^0.29). The principal implication for AI practitioners is that monolingual RPT is insufficient; incorporating even a single parallel language during training is a highly effective strategy to mitigate overfitting to English-specific patterns and develop more robust, language-agnostic reasoning systems. |
| Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness (Read more on arXiv or HuggingFace) |
|
This paper introduces Blind Goal-Directedness (BGD), a phenomenon where Computer-Use Agents (CUAs) pursue goals irrespective of safety or context, and presents the BLIND-ACT benchmark to measure this risk in frontier models. The primary objective is to systematically characterize and evaluate BGD in state-of-the-art CUAs across three prevalent patterns: lack of contextual reasoning, assumptions under ambiguity, and contradictory or infeasible goals. The methodology involves the development of BLIND-ACT, a 90-task benchmark built on OSWorld, and the use of an LLM-based judge to evaluate agent trajectories for BGD intentions and completion, which achieved 93.75% agreement with human annotations. The study found that nine evaluated frontier models exhibited a high average BGD rate of 80.8%, demonstrating that this is a widespread issue, and showed that prompting-based interventions only partially mitigate the risk. The principal implication for AI practitioners is that current CUAs possess a fundamental alignment flaw, making them unsafe for deployment; engineers must implement stronger, trajectory-level safeguards beyond simple prompting to ensure reliable and safe agent behavior. |
| Generalized Parallel Scaling with Interdependent Generations (Read more on arXiv or HuggingFace) |
Mrinal Kumar, Yun He, Eryk Helenowski, David Brandfonbrener, Harry Dong |
This paper introduces Bridge, a low-cost architectural addition for LLMs that facilitates information sharing between parallel generations from a single prompt to improve response set quality. The main objective is to overcome the limitations of independent parallel sampling by enabling interdependent generations, thereby allowing all N parallel responses to leverage the total available compute and information. The key methodology involves adding “Bridge” blocks, which are small attention mechanisms that operate across the batch dimension of the hidden state tensor (B x S x D) at each timestep, allowing tokens from different sequences to interact. Bridge improves the relative mean accuracy gains from reinforcement learning with verifiable rewards (RLVR) by up to 50% on the DS-Qwen-7B model across 7 math benchmarks, with only a 2.8%-5.1% increase in parameters. The principal implication for AI practitioners is that Bridge offers a more effective parallel scaling method for tasks like best-of-N selection, synthetic data generation, and RL fine-tuning, improving both individual response accuracy and the overall quality of the generated set without significant architectural changes or post-processing heuristics. |
| RLAD: Training LLMs to Discover Abstractions for Solving Reasoning |
|
|
| Problems (Read more on arXiv or HuggingFace) |
Ruslan Salakhutdinov, Amrith Setlur, Yoonho Lee, Anikait Singh, Yuxiao Qu |
The paper introduces RLAD, a two-player reinforcement learning framework that jointly trains an abstraction generator and a solution generator to improve LLM reasoning by discovering and utilizing high-level procedural knowledge. The primary objective is to train LLMs to discover and leverage concise, reusable “reasoning abstractions” to guide exploration and enhance performance on complex reasoning problems. The key methodology is a cooperative two-player RL paradigm where an “abstraction generator” is rewarded based on the performance improvement of an “abstraction-conditioned solution generator,” which is in turn rewarded for correctly solving the problem using the provided abstraction. On the AIME 2025 benchmark, the RLAD-trained model achieved 48.33% accuracy using the best of four proposed abstractions, outperforming a DAPO baseline which scored 39.79%. The principal implication for AI practitioners is that for complex reasoning tasks, allocating test-time compute to generate multiple diverse strategic abstractions (high-level plans) before generating solutions is more effective for improving performance than simply sampling a larger number of solution attempts. |
| MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment |
|
|
| Abilities in MLLMs (Read more on arXiv or HuggingFace) |
Junzhi Ning, Chenglong Ma, Wanying Qu, Jinjie Wei, Jiyao Liu |
This paper introduces MedQ-Bench, a comprehensive benchmark for evaluating the medical image quality assessment capabilities of Multimodal Large Language Models (MLLMs). The research objective is to systematically assess the perceptual and reasoning abilities of MLLMs in medical IQA by mirroring the clinical workflow of first perceiving quality attributes and then forming a judgment. The methodology consists of constructing the MedQ-Bench dataset, which includes 3,308 samples across 5 modalities and 40+ quality attributes, and evaluating 14 MLLMs using a perception-reasoning paradigm with a multi-dimensional judging protocol. The primary results reveal a significant human-AI performance gap, with the top-performing model (GPT-5) achieving 68.97% accuracy on perception tasks, substantially underperforming human experts (82.50%). For AI practitioners, the principal implication is that improving MLLMs for clinical applications requires a foundational focus on enhancing low-level visual perception and reasoning, as this is identified as the main bottleneck over instruction-following abilities. |
| TimeSeriesScientist: A General-Purpose AI Agent for Time Series Analysis (Read more on arXiv or HuggingFace) |
Yuting He, Yiwei Xu, Jiaqi Wei, Xiang Zhang, Haokun Zhao |
This paper introduces TimeSeriesScientist (TSci), a general-purpose, LLM-driven multi-agent framework designed to automate the end-to-end univariate time series forecasting pipeline. The primary objective is to create a domain-agnostic system that minimizes human intervention in the labor-intensive preprocessing, validation, and ensembling stages of forecasting. The methodology employs four specialized agents (Curator, Planner, Forecaster, Reporter) that collaboratively perform diagnostics, model selection, ensembling, and report generation through LLM reasoning and tool use. Empirical results on eight benchmarks show that TSci reduces forecast error by an average of 38.2% compared to LLM-based baselines. The principal implication for AI practitioners is that this agentic framework provides a practical, interpretable, and extensible “white-box” system that automates the complex forecasting workflow, significantly reducing the manual effort required to build and deploy reliable forecasting models. |
| Spectral Scaling Laws in Language Models: How Effectively Do |
|
|
| Feed-Forward Networks Use Their Latent Space? (Read more on arXiv or HuggingFace) |
|
This paper introduces spectral scaling laws for Feed-Forward Networks (FFNs) in LLMs, revealing an asymmetric relationship between FFN width and effective latent space utilization. The research aims to quantify how effectively increasing FFN width expands the usable latent space in LLMs, moving beyond performance-based scaling laws to analyze internal representational efficiency. The authors analyze the eigenspectrum of FFN post-activation covariance matrices using a diagnostic suite including Hard Rank (participation ratio) and Soft Rank (Shannon Rank) across LLaMA, GPT-2, and nGPT models with varying FFN widths. The study identifies an “Asymmetric Spectral Scaling Law”: soft rank scales almost linearly with FFN width (e.g., exponent β=1.06 for LLaMA-130M), while hard rank grows sublinearly (β=0.60), indicating that increased width predominantly adds low-energy tail directions while the dominant-mode subspace saturates early. This recasts FFN width selection as a spectral utilization trade-off, providing AI practitioners a principled method (e.g., monitoring effective dimension) to guide architectural choices and avoid allocating parameters to under-utilized latent dimensions. |
| FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame |
|
|
| Spotlighting (Read more on arXiv or HuggingFace) |
Daizong Liu, Siyuan Huang, Yafu Li, Xiaoye Qu, Zefeng He |
FrameThinker is a novel framework that enables Large Vision-Language Models (LVLMs) to perform active, iterative reasoning on long videos by dynamically selecting relevant frames for analysis. The primary objective is to overcome the inefficiency and performance limitations of traditional uniform frame sampling by teaching models to strategically interrogate video content. The methodology uses a two-phase training process: Supervised Fine-Tuning (SFT) to learn basic action syntax, followed by Reinforcement Learning (RL) with a Cognitive Consistency Verification (CCV) module to optimize the decision-making policy for frame selection. The 7B FrameThinker model achieves a new state-of-the-art 76.1% accuracy on the LongVideo-Reason benchmark using an average of only 20.6 frames, outperforming a competitive baseline that uses 512 frames. For AI practitioners, this research provides a paradigm for building more computationally efficient and accurate long video analysis systems by shifting from passive, dense processing to active, sparse frame selection guided by the model’s own reasoning process. |
| AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with |
|
|
| Multi-Objective Guidance (Read more on arXiv or HuggingFace) |
Pranam Chatterjee, Yinuo Zhang, Tong Chen |
AReUReDi is a discrete optimization algorithm that extends rectified discrete flows with multi-objective guidance to generate Pareto-optimal biological sequences. The research objective is to develop a sequence-based generative framework with theoretical guarantees for multi-objective Pareto optimality to design biomolecules satisfying multiple conflicting properties. The methodology integrates annealed Tchebycheff scalarization to unify objectives, locally balanced proposals to blend guidance with a generative prior, and Metropolis-Hastings updates to steer sampling toward the Pareto front. In designing peptide binders for the PPP5 target, AReUReDi achieved a half-life of 38.28 hours, substantially outperforming the next-best evolutionary algorithm which scored 2.90 hours. The principal implication for AI practitioners is that AReUReDi provides a general, theoretically-grounded framework for guiding pre-trained discrete generative models to produce outputs that are co-optimized for multiple, user-defined objectives. |
| SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking |
|
|
| for Training-free Zero-Shot Composed Image Retrieval (Read more on arXiv or HuggingFace) |
Huei-Fang Yang, Yu-Yen Lin, Ren-Di Wu |
SQUARE is a novel, two-stage, training-free framework that enhances zero-shot composed image retrieval (ZS-CIR) by using Multimodal Large Language Models (MLLMs) for both query enrichment and candidate reranking. The main objective is to improve the accuracy of ZS-CIR systems without requiring task-specific training or labeled data by more effectively capturing complex user intent expressed through a reference image and modification text. The methodology involves a two-stage process: 1) Semantic Query-Augmented Fusion (SQAF), where an MLLM generates a caption of the target image to enrich the initial VLM-based query embedding, and 2) Efficient Batch Reranking (EBR), where top candidates are presented as a grid to an MLLM for joint, single-pass reranking. The framework demonstrates state-of-the-art performance, achieving a mAP@50 of 38.82% on the CIRCO benchmark with a ViT-G/14 backbone, outperforming previous methods. The EBR stage alone improves mAP@5 on CIRCO from 30.89% to 35.61%. The principal implication for AI practitioners is that MLLMs can be deployed as powerful, zero-shot, modular components for complex retrieval tasks, enabling efficient batch-wise reranking that significantly improves accuracy over embedding-only methods without the need for model fine-tuning. |
| IoT-MCP: Bridging LLMs and IoT Systems Through Model Context Protocol (Read more on arXiv or HuggingFace) |
Yiming Li, Yiyi Lu, Mingchen Ma, Guanliang Lyu, Ningyuan Yang |
The paper presents IoT-MCP, a decoupled framework for bridging LLMs and IoT systems via edge-deployed servers implementing the Model Context Protocol. The primary objective is to develop a robust, low-latency framework that standardizes communication between LLMs and heterogeneous IoT devices, overcoming challenges of hardware diversity and resource constraints. The methodology involves a decoupled three-domain architecture (Local Host, Connection Server, IoT Devices) to separate LLM interaction from device management, and the development of IoT-MCP Bench, a new benchmark with 1,254 tasks for systematic evaluation. The framework achieved a 100% success rate on basic tool execution tasks across 22 sensor types, with a 205ms average end-to-end response time and a 74KB peak memory footprint on microcontrollers. For AI engineers, this provides an open-source, validated framework and a standardized evaluation methodology for integrating LLM-based natural language interfaces with resource-constrained IoT ecosystems, demonstrating a practical path for deploying agentic AI in physical environments. |
Papers for 2025-10-02
| Title |
Authors |
Summary |
| DeepSearch: Overcome the Bottleneck of Reinforcement Learning with |
|
|
| Verifiable Rewards via Monte Carlo Tree Search (Read more on arXiv or HuggingFace) |
|
The paper introduces DeepSearch, a framework integrating Monte Carlo Tree Search (MCTS) into the Reinforcement Learning with Verifiable Rewards (RLVR) training loop to improve language model reasoning. The objective is to overcome the performance plateaus in current RLVR training caused by insufficient exploration by embedding a systematic search process directly into model training. The methodology uses MCTS with three key innovations: a global frontier selection strategy to prioritize promising nodes, entropy-based guidance to identify confident paths for supervision, and an adaptive replay buffer with solution caching for efficiency. The primary result shows that the DeepSearch 1.5B model achieves a new state-of-the-art 62.95% average accuracy on mathematical reasoning benchmarks, a 1.25 percentage point improvement over the previous best, while using 5.7x fewer GPU hours than extended training methods. The principal implication for AI practitioners is that integrating structured search into the training phase provides a more computationally efficient path to improving reasoning performance than brute-force scaling of training steps. |
| GEM: A Gym for Agentic LLMs (Read more on arXiv or HuggingFace) |
|
This paper introduces GEM (General Experience Maker), an open-source, standardized environment framework for training and evaluating agentic large language models (LLMs). The primary objective is to provide a unified infrastructure to accelerate research in agentic LLMs by standardizing the agent-environment interface for diverse, multi-turn, long-horizon tasks, including those requiring tool use. The authors developed the GEM framework, which features a standardized API, a suite of environments across five categories, and tool integration, and they propose a baseline multi-turn algorithm, REINFORCE with Return Batch Normalization (ReBN). The paper demonstrates through extensive benchmarking that ReBN is effective across 24 environments and that combining RL with tool integration substantially boosts performance; for example, a Qwen3-4B model’s average score on math benchmarks increased from 35.3% to 49.8% when trained with a Python tool. The principal implication for AI practitioners is that GEM offers a decoupled and standardized library for developing and benchmarking agentic LLMs, simplifying the creation of complex interactive scenarios and enabling apples-to-apples comparisons of different RL algorithms. |
| VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified |
|
|
| Rewards in World Simulators (Read more on arXiv or HuggingFace) |
Zirui Ge, Yihao Wang, Runze Suo, Pengxiang Ding, Hengtao Li |
This paper introduces VLA-RFT, a framework for reinforcement fine-tuning Vision-Language-Action (VLA) models using a learned, data-driven world model as a simulator. The main objective is to overcome the limitations of imitation learning, such as compounding errors and poor robustness, without requiring costly real-world interactions. The methodology involves training a world model to predict future visual observations from actions, which then facilitates policy rollouts to generate dense, trajectory-level rewards by comparing predicted visual trajectories to expert references; these rewards are used to optimize the VLA policy with Generalized Reinforcement Policy Optimization (GRPO). With fewer than 400 fine-tuning steps, VLA-RFT increased the average success rate on the LIBERO benchmark to 91.1%, surpassing a strong supervised baseline (86.6%) that required 150K iterations. The principal implication for AI practitioners is that this world-model-based approach provides a highly sample-efficient method to significantly improve the performance and robustness of pre-trained VLA models, drastically reducing the need for real-world interaction or extensive supervised fine-tuning. |
| Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget |
|
|
| Allocation (Read more on arXiv or HuggingFace) |
|
This paper introduces Knapsack RL, a framework that optimizes the allocation of computational exploration budgets for Reinforcement Learning (RL) with Large Language Models (LLMs). The research objective is to determine the optimal distribution of a fixed total exploration budget across heterogeneous training tasks to maximize learning, overcoming the inefficiencies of uniform allocation. The methodology formulates this as a classical knapsack problem, where each task is assigned a “value” based on its current success rate and the probability of yielding a non-zero gradient, allowing for dynamic allocation of computational rollouts. The primary result is that this approach increases the effective gradient ratio by 20-40% over the baseline GRPO algorithm and achieves comparable performance with approximately half the computational resources. The principal implication for AI practitioners is that they can significantly improve the efficiency and final performance of RL-based LLM fine-tuning without increasing computational costs, effectively providing a “free lunch” by reallocating existing resources more intelligently. |
| Code2Video: A Code-centric Paradigm for Educational Video Generation (Read more on arXiv or HuggingFace) |
|
The paper introduces Code2Video, a code-centric, multi-agent paradigm that generates educational videos by structuring lecture content into executable Python code for a rendering engine. The research objective is to create a controllable, interpretable, and scalable method for producing temporally coherent and spatially precise educational videos, overcoming the structural limitations of pixel-space synthesis models. The methodology utilizes a tri-agent framework where a Planner creates a storyboard, a Coder translates it into parallelized and debugged Manim code, and a Critic refines spatial layouts using a novel visual anchor prompt and VLM feedback. On the newly introduced MMMC benchmark, Code2Video demonstrates a 40% improvement on the TeachQuiz knowledge transfer metric over direct code generation baselines, producing videos comparable to human-crafted tutorials. The principal implication for AI practitioners is that using executable code as an intermediate representation within a multi-agent system offers a robust and scalable solution for complex generative tasks requiring high structural fidelity and control, significantly outperforming direct pixel-level generation methods. |
| ACON: Optimizing Context Compression for Long-horizon LLM Agents (Read more on arXiv or HuggingFace) |
|
The paper introduces Agent Context Optimization (ACON), a framework that uses an LLM to iteratively refine natural language guidelines for compressing the extensive interaction histories and observations of long-horizon LLM agents. The primary objective is to develop a systematic and adaptive method for context compression that reduces computational costs and memory usage in long-horizon tasks, while preserving or even improving the agent’s task-completion performance. The key methodology is a gradient-free, failure-driven guideline optimization process: an “optimizer” LLM analyzes trajectories where an agent with full context succeeds but fails with compressed context, generating natural language feedback to refine the compression instructions. This process has two stages: utility maximization (UT) to improve task success, and compression maximization (CO) to increase conciseness. The resulting optimized compressor can then be distilled into a smaller model. Experiments show that ACON reduces memory usage by 26-54% across benchmarks; specifically, on AppWorld, it reduced peak input tokens by 26% for a gpt-4.1 agent while maintaining task accuracy (56.5% with ACON vs. 56.0% without). For smaller agent models, ACON improved performance by up to 46%. The principal implication for AI practitioners is that ACON provides a model-agnostic and deployment-friendly framework to significantly lower the operational costs and latency of LLM agents. By distilling the optimized compression logic into smaller models, engineers can create more efficient and scalable agentic systems without relying exclusively on large, expensive models for context management. |
| PIPer: On-Device Environment Setup via Online Reinforcement Learning (Read more on arXiv or HuggingFace) |
|
The paper introduces PIPER, a method for training on-device models to automate software environment setup using a two-stage fine-tuning process. The research objective is to create a specialized, small model that can overcome the limitations of general-purpose LLMs and perform comparably to larger models on this task. The key methodology combines Supervised Fine-Tuning (SFT) on successful scripts generated by a larger model, followed by Reinforcement Learning with Verifiable Rewards (RLVR) using an execution-free, LLM-as-a-Judge proxy for reward generation. On the EnvBench-Python benchmark, the resulting 8B parameter model achieves a pass@5 score of 27, performing on par with the much larger Qwen3-32B and GPT-40 models. For AI practitioners, this work demonstrates that combining SFT with proxy-reward RLVR enables the development of high-performing, cost-effective on-device models for complex software engineering tasks, reducing dependency on larger, API-gated systems. |
| Why Can’t Transformers Learn Multiplication? Reverse-Engineering Reveals |
|
|
| Long-Range Dependency Pitfalls (Read more on arXiv or HuggingFace) |
Stuart Shieber, Chenhao Tan, Itamar Pres, Xiaoyan Bai, yuntian-deng |
This paper reverse-engineers a Transformer that learns multi-digit multiplication to show standard models fail due to an inability to learn long-range dependencies with an auto-regressive loss. The primary objective is to determine why standard fine-tuned (SFT) Transformers fail at multiplication by analyzing the mechanisms of a successful model trained with implicit chain-of-thought (ICoT). The key methodology involves reverse-engineering a 2-layer ICoT model using logit attributions, linear probes on hidden states to decode intermediate sums, and PCA to analyze the geometry of digit representations. The ICoT model learns to form a directed acyclic graph with its attention heads to compute and cache partial products, while the SFT model fails to learn these long-range dependencies; adding an auxiliary loss to predict the running sum enables a standard model to achieve 99% accuracy, up from less than 1% for the SFT model. The principal implication for AI practitioners is that for algorithmic tasks requiring complex long-range dependencies, standard auto-regressive fine-tuning is insufficient, and incorporating task-specific inductive biases, such as auxiliary losses on intermediate computational steps, is critical to escape local optima and achieve high performance. |
| BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model |
|
|
| Responses (Read more on arXiv or HuggingFace) |
Julian McAuley, Ruizhe Chen, Churan Zhi, Xunzhi He, XinXuNLPer |
This paper introduces BIASFREEBENCH, a benchmark for systematically evaluating bias mitigation techniques in LLM responses. The objective is to create a unified framework for comparing prompting- and training-based debiasing methods by assessing their impact on generated text, rather than on internal model probabilities. The methodology involves evaluating eight techniques (e.g., Self-Reflection, DPO) on seven LLMs using reorganized BBQ and FairMT-Bench datasets and a new metric, the Bias-Free Score (BFS), which measures the proportion of fair, safe, and anti-stereotypical responses. The primary result is that prompting-based methods generally outperform training-based methods; for instance, on the BBQ dataset, Chain-of-Thought (CoT) prompting increased the BFS of Llama-3.1 from 52.41% to 82.82%, whereas SFT training reduced it to 52.11%. The principal implication for AI practitioners is that implementing carefully designed prompts is often a more effective and computationally efficient strategy for mitigating response-level bias than undertaking complex and potentially capability-degrading model fine-tuning. |
| Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel |
|
|
| Execution (Read more on arXiv or HuggingFace) |
|
This paper introduces FLASH-SEARCHER, a novel agent framework that accelerates complex reasoning tasks by reformulating sequential execution into a parallel, Directed Acyclic Graph (DAG)-based workflow. The primary objective is to overcome the inefficiency and high latency of traditional, sequential tool-augmented agent frameworks by developing a paradigm that enables concurrent execution of independent reasoning paths. The core methodology involves decomposing a complex task into subtasks with explicit dependencies, represented as a DAG, which allows for parallel inferential execution and tool orchestration, while adaptive progress tracking dynamically optimizes the graph based on intermediate results. Primary results demonstrate that the FLASH-SEARCHER framework achieves 67.7% accuracy on the BrowseComp benchmark while reducing agent execution steps by up to 35% compared to sequential approaches, improving both effectiveness and efficiency. The principal implication for AI practitioners is that adopting a DAG-based parallel execution architecture provides a direct method to reduce latency and computational cost in tool-intensive agent applications, offering a scalable alternative to linear reasoning chains. |
| BroRL: Scaling Reinforcement Learning via Broadened Exploration (Read more on arXiv or HuggingFace) |
|
BroRL scales reinforcement learning for large language models by increasing the number of rollouts per prompt, demonstrating a more efficient scaling axis than simply increasing training steps. The main objective is to determine if scaling the number of rollouts (N) in Reinforcement Learning with Verifiable Rewards (RLVR) can overcome the performance saturation seen in step-scaling methods. The methodology is based on a mass balance analysis which proves that a larger N minimizes a negative “unsampled coupling” term in the policy update; this is implemented by increasing N from 16 to 512 within a PPO-based framework while scaling the learning rate. BroRL successfully revives a saturated model, improving its Math score to 63.03 where the step-scaling ProRL baseline degrades to 62.02, while also nearly doubling generation throughput from 36.5 to 72.4 samples/s. The principal implication for AI practitioners is that scaling rollout size is a critical, computationally efficient method to overcome performance plateaus in RL training and improve hardware utilization by shifting the sample generation bottleneck from memory-bound to compute-bound. |
| Beyond Log Likelihood: Probability-Based Objectives for Supervised |
|
|
| Fine-Tuning across the Model Capability Continuum (Read more on arXiv or HuggingFace) |
Hanghang Tong, Heng Ji, Xiusi Chen, Ruizhong Qiu, Gaotang Li |
This paper demonstrates that the optimal supervised fine-tuning (SFT) objective for language models is not universally negative log likelihood (NLL), but depends on the base model’s prior capability for a given task, a concept the authors frame as the “model-capability continuum”. The objective is to characterize when NLL is suboptimal for SFT and identify which alternative probability-based objectives are more effective depending on the alignment between the base model’s priors and the fine-tuning task. The study introduces a continuum from model-strong (MS) domains (e.g., math), where the model has strong priors, to model-weak (MW) domains (e.g., novel puzzles), and empirically evaluates objectives like -log p (NLL) and -p across seven model backbones and three domains. The primary result shows that in MS settings, prior-leaning objectives that downweight low-probability tokens consistently outperform NLL, with a -p objective on Qwen2.5-Math-7B achieving 36.51% average accuracy versus 22.67% for NLL, while NLL dominates in MW settings. For AI practitioners, the principal implication is to select the SFT objective adaptively: use prior-leaning objectives like -p or thresholded NLL when fine-tuning on tasks where the base model already has strong knowledge, and retain the standard NLL for tasks novel to the model. |
| On Predictability of Reinforcement Learning Dynamics for Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Yuqing Huang, Zijun Yao, Ding Cao, Yuchen Cai, xx18 |
This research finds that reinforcement learning-induced parameter updates in large language models are dominated by a single, linearly evolving Rank-1 subspace, enabling predictable training acceleration. The main objective was to determine if RL-guided parameter updates follow consistent patterns and how these patterns create reasoning capabilities. The authors primarily used Singular Value Decomposition (SVD) to analyze the parameter update matrix (ΔW) and Partial Least Squares (PLS) regression to model the temporal evolution of its dominant Rank-1 component. Key results show that this Rank-1 subspace alone recovers over 99% of reasoning performance gains and evolves with high linearity (average R² > 0.91), which allowed the proposed AlphaRL framework to achieve up to a 2.5x training speedup while retaining over 96% of final performance. The principal implication for AI practitioners is the ability to significantly reduce the computational cost of RL training by extrapolating the final parameter update from a short early training window, without needing additional modules or hyperparameter tuning. |
| GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness (Read more on arXiv or HuggingFace) |
Chien-Sheng Wu, Caiming Xiong, Yutong Dai, Haoyi Qiu, Kung-Hsiang Huang |
GUI-KV is a no-retraining KV cache compression method that leverages the unique spatio-temporal redundancies in GUI agent workloads to improve inference efficiency and accuracy. The objective is to develop an efficient, plug-and-play KV cache compression technique for VLM-based GUI agents that process long sequences of high-resolution screenshots without requiring model retraining. The method combines spatial saliency guidance, which uses the L2 norm of hidden states to identify important visual tokens, with temporal redundancy scoring, which uses QR decomposition to prune key vectors from past screenshots that are already represented in the current screenshot’s key subspace. On the AgentNetBench benchmark in a 5-screenshot setting, GUI-KV reduces decoding FLOPs by 38.9% while simultaneously increasing step accuracy by 4.1% compared to a full-cache baseline. For AI practitioners, this implies that VLM-based GUI agents can be deployed with significantly lower computational cost and memory usage, potentially enabling better performance on resource-constrained hardware by mitigating long-context distraction. |
| Training Vision-Language Process Reward Models for Test-Time Scaling in |
|
|
| Multimodal Reasoning: Key Insights and Lessons Learned (Read more on arXiv or HuggingFace) |
|
This paper investigates the design, training, and evaluation of Vision-Language Process Reward Models (VL-PRMs) to improve multimodal reasoning through test-time scaling. The main objective is to elucidate the VL-PRM design space by systematically exploring diverse strategies for dataset construction, training with perception-focused supervision, and test-time scaling. The methodology involves creating the VL-PRM300K dataset using a hybrid framework combining Monte Carlo Tree Search with judgments from a strong VLM (o4-mini), which is then used to fine-tune Qwen-VL-based PRMs. Key results show that VL-PRMs used as Outcome Reward Models (ORMs) for scoring complete solutions outperform per-step guided search, and that scaling can unlock latent reasoning abilities, improving a Gemma3-12B model’s performance on PuzzleVQA by 12.7%. The principal implication for AI practitioners is that integrating a VL-PRM for one-shot solution verification at inference time is a highly effective and computationally cheaper strategy than per-step guidance to significantly boost the performance of large VLMs on complex reasoning tasks. |
| Infusing Theory of Mind into Socially Intelligent LLM Agents (Read more on arXiv or HuggingFace) |
|
This paper introduces ToMAgent (TOMA), a dialogue agent that integrates explicit Theory of Mind (ToM) modeling with dialogue lookahead to improve goal-oriented social reasoning in LLMs. The main research objective is to determine how to effectively equip LLMs with Theory of Mind abilities to improve their social reasoning and goal achievement in interactive dialogues. The key methodology involves a lookahead training framework that generates candidate mental state hypotheses and utterances, simulates conversation outcomes to score goal achievement, and then fine-tunes an LLM on the mental state-utterance pairs from the most successful simulated trajectories. Primary results on the Sotopia benchmark show that TOMA achieves a social score improvement of up to 18.9% over the base model variant and exhibits more strategic, long-horizon reasoning, while also demonstrating competitive performance against a GPT-5-nano baseline. The principal implication for AI practitioners is that explicitly training models to generate and leverage internal mental state representations, guided by future goal achievement, is a highly effective strategy for developing more socially intelligent and successful goal-oriented agents, moving beyond simple reactive utterance generation. |
| Making, not Taking, the Best of N (Read more on arXiv or HuggingFace) |
|
This paper proposes Fusion-of-N (FUSION), a synthesis-based method that uses an LLM to generate a single superior response from multiple candidate outputs, outperforming traditional Best-of-N (BON) selection. The primary objective is to investigate whether synthesizing information from N candidate generations is a more effective aggregation strategy than selecting the single best candidate, particularly for test-time scaling and synthetic data generation. The core methodology involves using a general LLM as a “fusor” to analyze a pool of N candidate generations and synthesize a new, final answer that combines their strengths, which is then benchmarked against BON across 11 languages and multiple tasks. Results demonstrate that FUSION consistently outperforms BON; for instance, in test-time scaling on the mArena-v2 benchmark, FUSION improves the win-rate by +3.8% against GEMINI2.5-PRO compared to BON. The principal implication for AI practitioners is to shift from a “winner-takes-all” selection paradigm to a collaborative synthesis approach, using a capable LLM to fuse multiple generated samples to produce a higher-quality final output, thereby making more efficient use of generated candidates and unlocking superior performance. |
| CurES: From Gradient Analysis to Efficient Curriculum Learning for |
|
|
| Reasoning LLMs (Read more on arXiv or HuggingFace) |
Hengyi Cai, Erxue Min, Bokai Ji, Zexu Sun, Yongcheng Zeng |
This paper presents CurES, a curriculum learning framework that enhances the training efficiency of reasoning LLMs by dynamically allocating computational resources based on gradient analysis. The research aims to improve training efficiency by theoretically linking gradient optimization to two key factors: the sampling distribution of prompts and the allocation of rollout quantities. CurES employs Bayesian posterior estimation to assess prompt difficulty based on the model’s answering accuracy, then adaptively reallocates prompt sampling probabilities and rollout quantities to focus computation on moderately difficult examples. Experiments demonstrate that CurES outperforms the Group Relative Policy Optimization (GRPO) baseline by +4.82 points on a 7B model and converges up to 5.5x faster. The principal implication for AI practitioners is a more computationally efficient training strategy for reasoning models, which reduces resource waste by moving beyond uniform data sampling to a dynamic curriculum that prioritizes the most informative training instances. |
| In-Place Feedback: A New Paradigm for Guiding LLMs in Multi-Turn |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Chaehyeon Chung, Seunghyuk Cho, Saemi Moon, Minjong Lee, Youngbin Choi |
This paper introduces in-place feedback, an interaction paradigm where users directly edit an LLM’s response to guide multi-turn reasoning, demonstrating superior performance and token efficiency over conventional feedback methods. The primary objective is to determine if direct state repair via in-place editing is a more effective mechanism for error correction in complex reasoning tasks compared to traditional conversational feedback. The study empirically compares in-place feedback against standard multi-turn feedback on reasoning benchmarks (MATH-hard, MMLU-pro, GPQA) and uses controlled experiments on the ZebraLogic dataset with automated feedback agents to measure turn-level dynamics. The primary result is that in-place feedback consistently achieves higher task accuracy and reduces aggregate token usage by 79.1% relative to multi-turn feedback. For AI practitioners, this implies that implementing in-place editing interfaces in collaborative AI applications offers a more direct and efficient method for correcting model errors, mitigating common failure modes like feedback disregard and error propagation seen in conversational refinement. |
| JoyAgent-JDGenie: Technical Report on the GAIA (Read more on arXiv or HuggingFace) |
|
This paper presents JoyAgent-JDGenie, a generalist agent architecture designed to enhance robustness by systematically integrating a multi-agent framework, hierarchical memory, and a refined tool suite. The primary objective is to create a unified framework that overcomes the limitations of isolated component improvements in existing agent systems. The key methodology combines a heterogeneous ensemble of Plan-Execute and ReAct agents coordinated by a critic model, a three-layer memory system (working, semantic, procedural), and a tool suite focused on search, code execution, and multimodal parsing. The framework achieves a 75.2 Pass@1 score on the GAIA validation set and 67.1 Pass@1 on the test set, surpassing contemporary open-source baselines. The principal implication for AI practitioners is that system-level integration of complementary agent architectures and memory systems is more effective for building robust generalist agents than optimizing individual components, with the fusion of different reasoning patterns proving critical for performance gains. |
| An Empirical Study of Testing Practices in Open Source AI Agent |
|
|
| Frameworks and Agentic Applications (Read more on arXiv or HuggingFace) |
Bram Adams, Gopi Krishnan Rajbahadur, Emad Fallahzadeh, Mohammed Mehedi Hasan, hao-li |
This large-scale empirical study analyzes unit testing practices in 39 open-source AI agent frameworks and 439 agentic applications to establish a quality assurance baseline. The primary objective is to identify common testing patterns and map their distribution across canonical agent architectural components. Employing repository mining and qualitative card-sorting on test functions, the study reveals a fundamental inversion of testing effort: deterministic infrastructure like Resource Artifacts (tools) and Coordination Artifacts (workflows) consumes over 70% of testing, while the non-deterministic, FM-based Plan Body receives less than 5% and the critical Trigger component (prompts) is tested in only about 1% of cases. Although practitioners adapt traditional patterns like membership testing to manage uncertainty, novel agent-specific methods like DeepEval see negligible adoption (~1%). The principal implication for AI practitioners is the urgent need to establish systematic prompt regression testing suites to mitigate risks from model evolution and performance decay. |
| Eliciting Secret Knowledge from Language Models (Read more on arXiv or HuggingFace) |
Neel Nanda, Senthooran Rajamanoharan, Rowan Wang, Emil Ryd, Bartosz Cywiński |
This research establishes a benchmark for secret elicitation by training LLMs to hide knowledge and then evaluating black-box and white-box techniques to uncover it. The primary objective is to discover and evaluate methods for eliciting knowledge that an LLM possesses and applies but is explicitly trained to deny knowing. The methodology involves fine-tuning three LLM families (Taboo, Secret Side Constraint, User Gender) to conceal specific secrets and then using an automated auditing framework where a weaker “auditor” LLM evaluates various elicitation techniques, including prefill attacks, logit lens, and Sparse Autoencoders (SAEs). The primary result is that black-box prefill attacks are highly effective, enabling the auditor to achieve over 90% success rate in uncovering secrets in the Secret Side Constraint and User Gender settings, significantly outperforming simple baselines. For AI practitioners, this implies that even without internal model access, simple and practical black-box techniques like prefill attacks can be highly effective for auditing models and probing for concealed knowledge or unintended behaviors. |
| ReSWD: ReSTIR’d, not shaken. Combining Reservoir Sampling and Sliced |
|
|
| Wasserstein Distance for Variance Reduction (Read more on arXiv or HuggingFace) |
|
ReSWD is an unbiased, variance-reduced estimator for the Sliced Wasserstein Distance that integrates Weighted Reservoir Sampling to reuse informative projection directions. The primary objective is to mitigate the high variance of standard Monte Carlo-based SWD estimators, which leads to noisy gradients and slow convergence in optimization tasks. The core methodology, Reservoir SWD (ReSWD), maintains a persistent reservoir of high-contribution projection directions across optimization steps, using their 1D Wasserstein cost as weights in a reservoir sampling scheme to focus computational effort while remaining unbiased. Experiments show that on 1D distribution matching tasks, ReSWD achieves a lower final mean Wasserstein distance (0.622 x 10⁻³) than standard SWD (0.733 x 10⁻³) and other variance reduction methods. For AI practitioners, ReSWD serves as a more efficient drop-in replacement for SWD-based loss functions in applications like generative model guidance and color correction, enabling more stable training and faster convergence. |
| VLM-FO1: Bridging the Gap Between High-Level Reasoning and Fine-Grained |
|
|
| Perception in VLMs (Read more on arXiv or HuggingFace) |
|
VLM-FO1 is a plug-and-play framework that endows pre-trained Vision-Language Models (VLMs) with fine-grained perception by reframing object localization from a coordinate generation problem into a feature retrieval task. The research objective is to bridge the gap between the high-level reasoning of VLMs and the precise spatial localization required for perception-centric tasks. The methodology involves a Hybrid Fine-grained Region Encoder (HFRE) with a Dual-Vision Encoder that converts region proposals into distinct “region tokens” and a two-stage, decoupled training process that preserves the base VLM’s capabilities. The lightweight VLM-FO1-3B model achieves state-of-the-art performance, attaining 44.4 mAP on the COCO object detection benchmark, which is a significant improvement over baseline VLMs. For AI practitioners, the principal implication is the ability to enhance existing, pre-trained VLMs with superior perception capabilities via a modular component, avoiding costly full model retraining and without degrading the original model’s general visual understanding. |
| Boolean Satisfiability via Imitation Learning (Read more on arXiv or HuggingFace) |
Xiangyu Xu, Jun Chen, Yuanhao Yu, Huan Liu, Zewei Zhang |
ImitSAT is a branching policy for CDCL solvers that uses imitation learning on expert-derived decision sequences to reduce solver runtime. The primary objective is to create a learning-based branching policy that improves upon traditional heuristics and reinforcement learning methods by directly imitating high-quality decision sequences extracted from expert solver runs. The key methodology involves first distilling raw, noisy CDCL solver trails into compact, nearly conflict-free “KeyTraces” that contain only surviving decisions, and then training an autoregressive Transformer model via behavior cloning to predict the next branching decision based on the problem instance and the current KeyTrace prefix. On structured SAT instances from the PRET family, ImitSAT reduces the median propagation count by 58% (MRPP of 0.42) relative to a standard CDCL solver, demonstrating strong generalization from its training on random 3-SAT. For AI practitioners, the principal implication is that distilling expert trajectories from search-based solvers into clean, sequential decision paths provides a highly effective data source for imitation learning, enabling sequence models to replace complex heuristics in combinatorial optimization problems. |
| Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic |
|
|
| Architectures (Read more on arXiv or HuggingFace) |
Andrea Passerini, Jacopo Staiano, Bruno Lepri, Carlo Nicolini, Marco Bronzini |
The paper introduces the Hyperdimensional Probe, a novel method using Vector Symbolic Architectures (VSAs) to decode interpretable concepts from the residual stream of Large Language Models. The primary objective is to create a decoding paradigm that overcomes the limitations of methods like Direct Logit Attribution (DLA) and Sparse Autoencoders (SAEs) by projecting internal LLM representations into a structured, human-readable symbolic space. The methodology involves compressing an LLM’s late-layer embeddings via k-means clustering and sum pooling, then training a shallow neural network to map these compressed representations to pre-defined VSA hypervectors, which are then queried using hypervector algebra to extract specific concepts. In controlled analogy-completion tasks, the probe achieved an average concept retrieval precision of 83% (probing@1), demonstrating a robust ability to identify the correct target concept even when the LLMs’ own next-token prediction accuracy was low (average 31% next-token@1). For AI practitioners, this work provides a computationally efficient tool for mechanistic interpretability that can diagnose model failures by revealing when a model correctly represents information internally but fails to articulate it, offering more granular debugging capabilities than token-based analysis. |
Papers for 2025-10-01
| Title | Authors | Summary |
|——-|———|———|
| Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified
Self-Play (Read more on arXiv or HuggingFace)| Jing Shi, Qinsi Wang, timecuriosity, zhoutianyi, Benjamin-eecs | Vision-Zero is a framework that enables scalable VLM self-improvement through strategic self-play in visual games generated from arbitrary, label-free image pairs. The primary objective is to develop a domain-agnostic training paradigm for VLMs that eliminates dependence on costly, human-annotated data for enhancing reasoning capabilities. The methodology involves a “Who Is the Spy”-style game where VLMs play as both “spy” and “civilian” to generate training data, coupled with a novel algorithm, Iterative Self-Play Policy Optimization (Iterative-SPO), which alternates between self-play and reinforcement learning with verifiable rewards (RLVR) to prevent performance stagnation. The framework achieves state-of-the-art performance across reasoning and vision-centric benchmarks; a Vision-Zero trained Qwen2.5-VL-7B model achieved an average score of 44.1% on a suite of reasoning and math tasks, outperforming the baseline model’s 41.1% and other annotation-based methods. The principal implication for AI practitioners is that Vision-Zero provides a highly cost-efficient method to post-train and enhance VLMs using only unlabeled image pairs, thereby avoiding the significant expense and time required for manual data curation and annotation. |
| MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP
Use (Read more on arXiv or HuggingFace)| | This paper introduces MCPMark, a benchmark with 127 realistic tasks designed to stress-test LLM agents on complex, multi-step CRUD operations, addressing the limitations of existing shallow MCP benchmarks. The methodology involves tasks across five environments (e.g., GitHub, PostgreSQL) created via a human-AI pipeline, each with a curated initial state and evaluated through programmatic verification in a minimal agent framework. The benchmark reveals significant performance limitations in current models, with the top performer, gpt-5-medium, achieving only a 52.56% pass@1 success rate. For AI practitioners, this highlights a critical gap in agent robustness and planning for real-world stateful tasks, indicating that development must shift focus from simple reactive tool use to enhancing execution stability and sophisticated reasoning. |
| The Dragon Hatchling: The Missing Link between the Transformer and
Models of the Brain (Read more on arXiv or HuggingFace)| | The paper introduces Brain-inspired Dragon Hatchling (BDH), a novel, biologically-plausible state-space language model architecture based on local graph dynamics that achieves Transformer-competitive performance. The main research question is how to develop a new LLM architecture that connects the macro-level function of Transformers with micro-level, biologically-inspired neuronal dynamics to achieve strong performance, inherent interpretability, and a theoretical foundation for reasoning over time. The key methodology is the proposal of BDH, formulated as an edge-reweighting process on a graph of n neurons, with a practical GPU-friendly variant (BDH-GPU) that uses low-rank matrix factorizations, a ‘ReLU-lowrank’ feed-forward block, and a linear attention mechanism operating in a high-dimensional (n), positive activation space. The primary results show that BDH-GPU exhibits scaling laws comparable to Transformers, empirically matching the performance of a GPT-2 style baseline (GPTXL) on language and translation tasks across scales from 10M to 1B parameters; at the 1B scale, a BDH-GPU variant achieves a validation loss of approximately 0.37, on par with the baseline. The architecture’s learned parameters form modular, scale-free graphs, and it exhibits sparse positive activations (~5% density) and monosemantic synapses. The principal implication for AI practitioners is that the BDH-GPU architecture offers a competitive alternative to the Transformer that is designed for interpretability and composability. Its most impactful finding is achieving this performance with a model whose state and parameters have a clear, local, graph-based interpretation, enabling novel capabilities like direct model merging by concatenating parameter tensors, which could simplify the creation of larger, specialized models from smaller components. |
| TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning (Read more on arXiv or HuggingFace)| | This paper presents TruthRL, a reinforcement learning framework that uses a ternary reward system to improve LLM truthfulness by incentivizing correct answers and abstention over hallucination. The primary objective is to develop a training method that directly optimizes a model’s ability to maximize correct answers, minimize hallucinations, and appropriately abstain when uncertain, moving beyond simple accuracy maximization. The key methodology is an RL framework, implemented with GRPO, that utilizes a ternary reward signal: +1 for correct answers, 0 for abstentions, and -1 for hallucinations. Across four knowledge-intensive benchmarks, this approach reduces hallucinations by 28.9% and improves truthfulness by 21.1% compared to vanilla RL. The principal implication for AI practitioners is that explicitly designing learning objectives and reward structures for truthfulness—particularly by neutrally treating abstention—is a more effective strategy for developing reliable and less hallucinatory models than solely optimizing for accuracy. |
| OceanGym: A Benchmark Environment for Underwater Embodied Agents (Read more on arXiv or HuggingFace)| | OceanGym is a benchmark environment for evaluating Multi-modal Large Language Model (MLLM)-driven embodied agents on perception and decision-making tasks in simulated underwater settings. The primary objective is to assess the capabilities and limitations of MLLM agents in challenging underwater environments characterized by low visibility, dynamic conditions, and reliance on optical and sonar data. The methodology involves a high-fidelity simulation environment built in Unreal Engine 5.3 with eight realistic task domains and a unified agent framework where an MLLM processes language instructions, multi-view sensory inputs, and a sliding-window memory to control an Autonomous Underwater Vehicle (AUV) based on a POMDP formulation. Results show a substantial performance gap between AI and humans; in deep-water decision tasks, the best MLLM-driven agent (GPT-4o-mini) achieved an average score of 14.8%, while human experts scored 69.6%. The principal implication for AI practitioners is that current MLLM agents lack the robustness for real-world underwater deployment, indicating a critical need for advancing multi-modal fusion (particularly for sonar data), memory retention, and long-horizon planning under extreme perceptual uncertainty. |
| DC-VideoGen: Efficient Video Generation with Deep Compression Video
Autoencoder (Read more on arXiv or HuggingFace)| | DC-VideoGen is a post-training framework that accelerates video diffusion models by adapting them to a new, highly compressed latent space. The main objective is to reduce the high computational costs of training and inference for large-scale video generation models while maintaining or improving output quality. The methodology combines a Deep Compression Video Autoencoder (DC-AE-V) using a novel chunk-causal temporal design for 32×/64× spatial compression, with an efficient adaptation strategy (AE-Adapt-V) that aligns the model’s embedding space before lightweight LoRA fine-tuning. The framework achieves up to a 14.8× reduction in inference latency on 2160×3840 resolution video generation compared to the Wan-2.1-T2V-1.3B base model, while enabling generation at this resolution on a single NVIDIA H100 GPU. The principal implication for AI practitioners is the ability to make existing, computationally expensive video diffusion models significantly more efficient for deployment and further development with minimal, low-cost fine-tuning, thereby increasing accessibility to high-fidelity video generation. |
| Who’s Your Judge? On the Detectability of LLM-Generated Judgments (Read more on arXiv or HuggingFace)| | This paper introduces the task of detecting LLM-generated judgments and proposes J-Detector, a lightweight detector using explicit features to distinguish them from human judgments. The main objective is to formalize and systematically investigate the detectability of LLM judgments based solely on candidate content and numerical scores, a scenario where textual feedback is unavailable. The key methodology involves J-Detector, a neural model augmented with extracted linguistic features (e.g., length, complexity) and LLM-enhanced features that capture systematic biases in LLM judges. The primary result shows that J-Detector significantly outperforms baselines, achieving an average F1 score of 87.7% across four datasets, compared to 68.8% for RoBERTa-based detectors, and demonstrates that detectability is influenced by group size, judgment dimensions, and rating scale. The principal implication for AI practitioners is that the inherent, systematic biases of LLM judges can be exploited for detection, providing a method to audit and ensure the fairness of automated evaluation systems. |
| Learning to See Before Seeing: Demystifying LLM Visual Priors from
Language Pre-training (Read more on arXiv or HuggingFace)| Koustuv Sinha, Yufan Ren, David Fan, Shengbang Tong, Junlin Han | This paper systematically investigates how Large Language Models (LLMs) acquire visual priors from text-only pre-training and proposes a data-centric recipe to enhance these capabilities. The main objective is to deconstruct the origin and structure of emergent visual priors in LLMs, determining which types of language data cultivate specific visual abilities. The methodology involves over 100 controlled experiments analyzing MLLM performance after varying the LLM pre-training data composition, model scale, and data scale across 16 VQA benchmarks. The primary result is that visual priors decompose into a reasoning prior, which scales progressively with the proportion of reasoning-centric data up to a 75% ratio in the pre-training mix, and a perception prior, which emerges more diffusely from broad corpora. The principal implication for AI practitioners is that they can build more capable MLLMs by deliberately composing the LLM pre-training corpus to be heavily skewed towards reasoning-centric text to cultivate a transferable, modality-agnostic visual reasoning foundation. |
| Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token
Pruning for Efficient Supervised Fine-Tuning (Read more on arXiv or HuggingFace)| Yue Min, Cong Wang, JiajunZhang, Jessamine, Steven-Shaobo | This paper introduces Q-Tuning, a unified framework for joint sample and token pruning that enhances the efficiency and performance of supervised fine-tuning for large language models. The objective is to develop a coordinated strategy that jointly optimizes both sample selection and token retention to overcome the limitations of fragmented, single-dimension pruning methods. The core methodology, Quadrant-based Tuning (Q-Tuning), uses an “Error-Uncertainty (EU) Plane” to categorize training instances by model perplexity and entropy, enabling a two-stage process that first prunes entire samples classified as harmful noise or redundant knowledge, and then applies an asymmetric token-pruning policy exclusively to high-error, high-confidence samples. On the GSM8K benchmark, Q-Tuning with LLaMA3-8B achieved an accuracy of 48.07 using only 35% of the training data, significantly outperforming the 42.05 score from training on the full dataset. The principal implication for AI practitioners is that this method provides a scalable blueprint to reduce SFT computational costs while simultaneously improving model performance, making high-quality alignment more accessible under budget constraints. |
| Thinking Sparks!: Emergent Attention Heads in Reasoning Models During
Post Training (Read more on arXiv or HuggingFace)| | This paper uses circuit analysis to demonstrate that post-training for complex reasoning sparks the emergence of novel, functionally specialized attention heads. The primary objective is to mechanistically analyze how different post-training regimes—distillation, Supervised Fine-Tuning (SFT), and Group Relative Policy Optimization (GRPO)—alter a model’s internal architecture to enhance reasoning. The methodology involves using edge attribution patching with integrated gradients (EAP-IG) to map computational circuits in Qwen-family models before and after post-training, followed by causal validation via head ablation studies. The primary results reveal distinct architectural changes: distillation and SFT foster a cumulative addition of many stable reasoning heads, while GRPO performs a dynamic search, activating and pruning a smaller, more targeted set of heads that correlate with reward signals; for instance, ablating the emergent reasoning heads in the DeepSeek-R1-Distill-Qwen-1.5B model caused its AIME’24 pass@1 score to drop from 30.0 to 26.6. The principal implication for AI practitioners is that the choice of post-training method creates a direct trade-off between installing powerful, broad reasoning circuits that may “overthink” simple tasks (SFT/distillation) and performing targeted, efficient optimization that may be less general (GRPO), requiring careful selection of training policy to balance reasoning capability with execution reliability. |
| VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in
Real-world Applications (Read more on arXiv or HuggingFace)| | The paper introduces VitaBench, a benchmark for evaluating LLM agents on 400 complex, interactive tasks grounded in real-world life-serving applications using 66 tools. The primary objective is to systematically measure agentic task complexity across three dimensions—reasoning, tool use, and interaction—to better reflect the challenges of practical deployment. The methodology involves a simulation environment with graph-interconnected tools, tasks derived from real user requests, and a novel rubric-based sliding window evaluator for assessing long-horizon, multi-path trajectories. The evaluation demonstrates that even top-performing models achieve only a 30.0% Avg@4 success rate on cross-scenario tasks, a sharp decline from over 50% in single-scenario settings. For AI practitioners, this significant performance degradation underscores that current agents have fundamental deficiencies in cross-domain reasoning and tool composition, identifying these as critical limitations to address for reliable real-world applications. |
| dParallel: Learnable Parallel Decoding for dLLMs (Read more on arXiv or HuggingFace)| | dParallel introduces a certainty-forcing distillation method to significantly accelerate inference in diffusion large language models (dLLMs) by enabling highly parallel token decoding. The objective is to overcome the primary bottleneck of “sequential certainty convergence,” where dLLMs achieve high confidence for token predictions in a slow, left-to-right manner, which inhibits their inherent parallelism. The key methodology is certainty-forcing distillation, a novel self-distillation training strategy that combines a consistency loss to maintain the original generation trajectory with a certainty loss that minimizes the predictive entropy for correctly predicted tokens, thereby training the model to become certain about many tokens in parallel. The primary result demonstrates that when applied to LLaDA-8B-Instruct on the GSM8K benchmark, dParallel reduces the number of decoding steps from 256 to 30, achieving an 8.5x inference speedup without performance degradation. For AI practitioners, this research provides an efficient, LoRA-based fine-tuning technique to drastically reduce the latency of existing dLLMs, making them more practical for deployment in latency-sensitive applications that require fast text generation. |
| IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance (Read more on arXiv or HuggingFace)| | This paper proposes Implicit Multimodal Guidance (IMG), a re-generation framework that uses a multimodal large language model (MLLM) and a novel adapter to correct misalignments in diffusion-generated images without requiring model finetuning or explicit editing. The primary objective is to improve the alignment between text prompts and generated images by identifying and correcting conceptual errors, such as missing objects or incorrect attributes, that occur in state-of-the-art diffusion models. The methodology involves an MLLM identifying discrepancies between an initial image and its prompt, an “Implicit Aligner” network using the MLLM’s guidance to refine the image’s conditioning features, and then re-generating a new image from these corrected features using an “Iteratively Updated Preference Objective” for training. Evaluations show that when applied to SDXL, IMG achieves an average Human Preference Score (HPS) win rate of 87.2% against the base model on the Human Preference Datasets (HPD) benchmark. For AI practitioners, IMG provides a flexible, plug-and-play adapter to enhance the prompt adherence and quality of existing pre-trained diffusion models, offering a practical alternative to full model retraining or complex post-generation editing pipelines. |
| MotionRAG: Motion Retrieval-Augmented Image-to-Video Generation (Read more on arXiv or HuggingFace)| Limin Wang, Gangshan Wu, Yilu Wu, wangsssssss, flateon | MotionRAG is a retrieval-augmented framework that improves motion realism in image-to-video generation by transferring motion priors from reference videos. The objective is to overcome the difficulty of modeling complex, physically plausible motion in diffusion-based video synthesis by leveraging external motion examples. The methodology involves a three-stage process: text-based retrieval of relevant videos, a Context-Aware Motion Adaptation (CAMA) module using a causal transformer to adapt motion features via in-context learning, and a motion-adapter to inject these features into a pretrained video diffusion model. On the OpenVid-1K dataset, MotionRAG improved the Action Score of the CogVideoX model from 59.9 to 65.8, demonstrating enhanced motion quality with negligible computational overhead. For AI practitioners, this provides a modular, plug-and-play method to enhance motion fidelity in existing video generation models and enables zero-shot adaptation to new domains by simply curating a relevant retrieval database without retraining. |
| Efficient Audio-Visual Speech Separation with Discrete Lip Semantics and
Multi-Scale Global-Local Attention (Read more on arXiv or HuggingFace)| | The paper introduces Dolphin, an efficient audio-visual speech separation (AVSS) model that achieves state-of-the-art performance with significantly reduced computational cost. The objective is to resolve the inherent trade-off between separation quality and computational overhead in AVSS, particularly the high cost of large-scale visual encoders. The methodology features two key innovations: DP-LipCoder, a lightweight dual-path video encoder that transforms lip motion into discrete, audio-aligned semantic tokens using vector quantization and knowledge distillation, and a single-iteration audio separator incorporating a global-local attention (GLA) block to efficiently model multi-scale dependencies. On benchmark datasets, Dolphin surpasses the previous state-of-the-art model in separation quality while achieving over a 2.4x reduction in MACs and over 6x faster GPU inference speed. The principal implication for AI practitioners is that Dolphin offers a practical and deployable architecture for high-performance AVSS in resource-constrained environments, demonstrating that efficient, discrete visual representations can replace computationally expensive visual backbones without sacrificing quality. |
| Mem-α: Learning Memory Construction via Reinforcement Learning (Read more on arXiv or HuggingFace)| Yuzhen Mao, Ryuichi Takanobu, Yu Wang, ai-hyz, zkadelzq | Mem-α is a reinforcement learning framework for training large language model agents to dynamically construct and manage a multi-component external memory. The primary objective is to determine if reinforcement learning can train an LLM agent to learn optimal policies for managing a complex memory system—what to store, how to structure it, and when to update—by optimizing directly for downstream task performance. The methodology involves formulating memory construction as a sequential decision-making problem trained with Group Relative Policy Optimization (GRPO), where an agent interacts with a three-part memory (core, episodic, semantic) and receives a composite reward signal based on question-answering accuracy, tool call success, memory compression, and content validity. The Mem-α agent, built on Qwen3-4B, achieved a 0.642 average performance score across validation tasks, significantly outperforming the base model with the same memory architecture (0.389) and standard baselines like Long-Context (0.588), while also generalizing to sequences over 13x its training length. The principal implication for AI practitioners is that directly optimizing memory management policies via reinforcement learning can substantially improve the long-context reasoning and information retention capabilities of smaller models, providing a more robust alternative to relying on manually engineered prompting or simple retrieval-augmented generation heuristics. |
| Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal
LLMs (Read more on arXiv or HuggingFace)| | This paper introduces DEEPTRACEREWARD, a benchmark for identifying human-perceived artifacts in AI-generated videos with spatiotemporal grounding, and trains a reward model that significantly outperforms SOTA MLLMs on this task. The research aims to determine if MLLMs can detect and explain deepfake traces like humans do, and to create a dataset to train models for this capability. The methodology involves creating a dataset of 4.3K expert annotations on 3.3K videos, where each annotation contains a natural language explanation, bounding boxes, and timestamps for a visual artifact, which is then used to supervised fine-tune a 7B multimodal language model. The primary result is that the fine-tuned model achieves a 70.2% overall score, surpassing GPT-5 by 34.7%, and demonstrates a clear difficulty gradient where binary classification (99.4% accuracy) is substantially easier than fine-grained spatial or temporal localization of artifacts. For AI practitioners, this work provides a concrete framework and a reward model for evaluating and improving video generation systems by targeting specific, human-noticeable visual failures, moving beyond holistic quality metrics to enable more trustworthy and realistic video synthesis. |
| DeepScientist: Advancing Frontier-Pushing Scientific Findings
Progressively (Read more on arXiv or HuggingFace)| | DeepScientist is an autonomous AI system that formalizes scientific discovery as a Bayesian Optimization problem to progressively generate novel, SOTA-surpassing methods. The primary objective is to develop a fully autonomous, goal-oriented system capable of conducting scientific discovery over long timelines to produce findings that surpass human-designed state-of-the-art methods on frontier AI tasks. The system uses a hierarchical “hypothesize, verify, and analyze” loop, leveraging a Bayesian Optimization framework with a surrogate model and an Upper Confidence Bound (UCB) acquisition function to intelligently select hypotheses from a cumulative “Findings Memory” for experimental validation. The system autonomously developed novel methods that surpassed human SOTA on three separate tasks, including improving accuracy on an Agent Failure Attribution benchmark by 183.7%; however, the overall success rate from implemented ideas to validated progress was only 1-3%. The principal implication for AI practitioners is that while autonomous systems can vastly accelerate the trial-and-error process of innovation, the primary bottleneck shifts from ideation to efficient, scalable validation and filtering due to the exceptionally low success rate of AI-generated hypotheses. |
| Attention as a Compass: Efficient Exploration for Process-Supervised RL
in Reasoning Models (Read more on arXiv or HuggingFace)| | AttnRL is a Process-Supervised Reinforcement Learning framework that uses model attention scores to guide exploration, improving the performance and training efficiency of reasoning models. The primary objective is to overcome the exploration and training inefficiencies of existing Process-Supervised RL (PSRL) methods by developing a system that intelligently selects branching points and adapts sampling strategies. The core methodology involves Attention-based Tree Branching (ATB), which uses a Forward Context Influence (FCI) score to identify critical reasoning steps for exploration, combined with an adaptive sampling mechanism and a one-step off-policy training pipeline to reduce redundant generation. On six mathematical reasoning benchmarks, AttnRL improved a 7B parameter base model’s average score from 66.0 to 68.7, achieving this result in 500 training steps versus the 800 steps required by baselines. The principal implication for AI practitioners is that internal model mechanics like attention can serve as a highly efficient, low-cost heuristic for guiding RL exploration, while the one-step off-policy design offers a practical method to significantly reduce the computational overhead and wall-clock time of PSRL training cycles. |
| DA^2: Depth Anything in Any Direction (Read more on arXiv or HuggingFace)| | DA² is an end-to-end, zero-shot panoramic depth estimation model that combines a massive curated dataset with a distortion-aware Vision Transformer architecture. The objective is to develop a highly accurate and generalizable panoramic depth estimator that overcomes the limitations of data scarcity and spherical distortions inherent in 360° images. The methodology involves a data curation engine that converts over 543K perspective RGB-depth pairs into panoramic samples and a novel SphereViT model that uses cross-attention to explicitly incorporate spherical coordinates, making the features distortion-aware. DA² achieves state-of-the-art zero-shot performance across multiple benchmarks, demonstrating an average 38% improvement in Absolute Relative Error (AbsRel) over the strongest prior zero-shot baseline. For AI practitioners, this work provides a powerful, efficient model and a large-scale dataset for generating geometrically consistent 3D reconstructions from single panoramic images, directly benefiting AR/VR, robotics simulation, and 3D content creation. |
| OffTopicEval: When Large Language Models Enter the Wrong Chat, Almost
Always! (Read more on arXiv or HuggingFace)| | This paper introduces OFFTOPICEVAL, a benchmark to evaluate the “operational safety” of LLMs, defined as their ability to adhere to a specific purpose by accepting in-domain and refusing out-of-domain queries. The primary objective is to quantify the capability of LLMs to reject out-of-domain (OOD) queries, especially when they are adversarially transformed to appear in-domain. The methodology involves testing 20 open-weight models across 21 purpose-specific agent roles using in-domain (ID), direct OOD, and “adaptive OOD” queries, which are generated via a “prompt laundering” transformation. Results show a catastrophic failure in operational safety against adaptive attacks; for instance, the Llama-3.3 (70B) model’s OOD refusal rate plummets from 69.73% on direct queries to just 4.21% on adaptive queries. The principal implication for AI practitioners is that system prompts are insufficient to guarantee that a purpose-specific agent remains on-topic, making them unsafe for deployment against even simple adversarial inputs; prompt-based mitigation strategies like the proposed “P-ground” and “Q-ground” are shown to be a necessary first step, improving refusal rates by up to 41%. |
| A Cartography of Open Collaboration in Open Source AI: Mapping
Practices, Motivations, and Governance in 14 Open Large Language Model
Projects (Read more on arXiv or HuggingFace)| Jennifer Ding, Cailean Osborne, Johan Linåker, burtenshaw | This paper presents a qualitative cartography of open collaboration practices, motivations, and governance structures across the lifecycle of 14 open large language model projects. The study’s objective is to understand how open LLM projects are initiated, organized, and governed by mapping where collaboration occurs, what motivates developers, and how these efforts are coordinated. The methodology consists of an exploratory analysis based on semi-structured interviews with 17 developers from 14 projects spanning grassroots initiatives, research institutes, startups, and major technology companies. Results show that open collaboration extends beyond the model itself to datasets, benchmarks, and frameworks, and the analysis identified five distinct organizational governance models ranging from single-company projects to non-profit-sponsored grassroots initiatives. The principal implication for AI practitioners is that collaboration opportunities are highly dependent on the project’s lifecycle stage; broad community engagement is most feasible post-release through derivative development and feedback, whereas pre-release collaboration is typically limited to strategic, resource-intensive partnerships or specialized contributions to artifacts like evaluation frameworks. |
| Regression Language Models for Code (Read more on arXiv or HuggingFace)| | This paper introduces Regression Language Models (RLMs), a unified text-to-text framework for predicting numeric metrics like latency, memory, and accuracy directly from diverse code representations without domain-specific feature engineering. The primary objective is to determine if a single, pretrained encoder-decoder language model can effectively perform regression on a wide variety of code-based inputs—from high-level languages to neural network intermediate representations (IR)—and outperform specialized methods. The methodology treats regression as a next-token prediction task using a T5Gemma-initialized model, where numeric target values are represented using a custom, normalization-free, digit-by-digit tokenization scheme and predicted autoregressively. A single 300M parameter RLM achieves a mean Kendall-Tau of 0.46 on five neural architecture search (NAS) benchmarks, outperforming previous state-of-the-art graph neural network models. For AI practitioners, this means a single, unified RLM can be used to predict performance metrics for diverse computational graphs and source code directly from their text representations, significantly simplifying the performance modeling pipeline by eliminating the need for manual feature engineering or specialized graph-based architectures. |
| InfoAgent: Advancing Autonomous Information-Seeking Agents (Read more on arXiv or HuggingFace)| | This paper introduces InfoAgent, an open-source 14B parameter deep research agent that utilizes a novel data synthesis pipeline and a two-stage training process to solve complex, long-horizon information-seeking tasks. The primary objective is to advance autonomous agent capabilities by developing a methodology for creating challenging, multi-hop training data and establishing an efficient, self-hosted interactive web environment. The core methodology consists of a data pipeline that builds entity trees from Wikipedia, applies sub-tree sampling with entity fuzzification to systematically increase question difficulty, and then uses a two-stage training recipe of cold-start supervised fine-tuning followed by reinforcement learning. The resulting InfoAgent achieves 15.3% accuracy on the BrowseComp benchmark, outperforming larger open-source models such as WebSailor-72B and DeepDive-32B. For AI practitioners, this research provides a concrete framework emphasizing that sophisticated data synthesis strategies forcing long-horizon reasoning and a high-quality, custom tool infrastructure are critical components for building high-performing, open-source information-seeking agents. |
| Humanline: Online Alignment as Perceptual Loss (Read more on arXiv or HuggingFace)| | This paper introduces “humanline,” a design pattern derived from prospect theory that incorporates human perceptual biases into alignment objectives to close the performance gap between online and offline methods. The core objective is to understand why online alignment outperforms offline alignment and to replicate its benefits without expensive on-policy data collection. The methodology involves creating humanline variants of DPO, KTO, and GRPO by applying two changes: periodic syncing of the reference model with a previous policy version and asymmetric upstream clipping of token-wise likelihood ratios. The primary result shows that humanline variants trained with offline data can match their online counterparts’ performance, with offline+humanline GRPO achieving a 1.6x higher winrate than standard offline GRPO on an instruction-following task. For AI practitioners, this implies that state-of-the-art alignment can be achieved more cheaply and quickly by applying the humanline pattern to existing offline datasets, avoiding the cost and instability of online on-policy training. |
| Ferret-UI Lite: Lessons from Building Small On-Device GUI Agents (Read more on arXiv or HuggingFace)| | Ferret-UI Lite is a 3B parameter end-to-end multimodal model developed for on-device Graphic User Interface (GUI) agent tasks. The primary objective is to investigate strategies for building compact and efficient GUI agents capable of operating across diverse platforms like mobile, web, and desktop. The methodology involves a two-stage training strategy: supervised fine-tuning (SFT) on a curated mixture of real and synthetic GUI data, followed by reinforcement learning with verifiable rewards (RLVR) to refine both grounding and navigation performance, augmented by inference-time chain-of-thought reasoning and a zoom-in visual tool. Ferret-UI Lite demonstrates competitive GUI grounding performance, achieving 53.3% accuracy on the ScreenSpot-Pro benchmark, but its multi-step navigation capabilities are limited, with a 28.0% success rate on the AndroidWorld benchmark. The principal implication for AI practitioners is that while small, on-device GUI agents can achieve strong grounding accuracy through data curation and RL, their capacity for robust, long-horizon reasoning in complex navigation tasks remains a significant challenge, indicating a direct trade-off between model efficiency and advanced agentic capabilities. |
| More Thought, Less Accuracy? On the Dual Nature of Reasoning in
Vision-Language Models (Read more on arXiv or HuggingFace)| Fabian Waschkowski, Mengqi He, Zhaoyuan Yang, Shu Zou, Xinyu Tian | This research identifies that prolonged reasoning in Vision-Language Models (VLMs) can impair perceptual accuracy due to “visual forgetting” and proposes Vision-Anchored Policy Optimization (VAPO) to enforce visual grounding and improve performance. The main objective is to investigate the dual nature of multimodal reasoning, where extended thought processes can degrade perceptual grounding, and to develop a method to counteract this effect. The key methodology is VISION-ANCHORED POLICY OPTIMIZATION (VAPO), a policy gradient algorithm that inserts “visual anchors”—verifiable claims about the image—into the reasoning process and uses the model’s judgment on these claims to generate a perception reward for training. The primary result is that the proposed VAPO-Thinker-7B model achieves new state-of-the-art performance, improving upon the previous best result by 3.2% (from 59.9% to 63.1%) on average across general-purpose benchmarks. The principal implication for AI practitioners is that simply encouraging longer reasoning chains in VLMs is insufficient and can be detrimental; it is critical to implement mechanisms that explicitly reinforce the model’s connection to visual input throughout the entire reasoning process to prevent performance degradation on vision-intensive tasks. |
| Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with
LLMs (Read more on arXiv or HuggingFace)| Yao Shu, Fei Yu, Ying He, Hong Wang, Chenxing Wei | This paper presents T²PAM, a test-time policy adaptation paradigm, and its algorithm ROSA, which enables Large Language Models to perform efficient in-conversation self-correction using real-time user feedback. The primary objective is to address performance degradation in multi-turn interactions by dynamically updating the model’s policy to align with user preferences during inference, thereby avoiding costly offline retraining. ROSA operationalizes this by formulating an RLHF objective, analytically deriving a closed-form optimal policy from user feedback, and then guiding model parameters toward this target in a single, efficient update step using linearized optimization with the Conjugate Gradient method. Extensive experiments show significant improvements; for instance, applying ROSA with a model-based reward to the Qwen3-0.6B model on the MATH dataset increased its final accuracy from a baseline of 25.00% to 52.20%, an absolute improvement of +27.20%. The principal implication for AI practitioners is that ROSA offers a lightweight, practical method to enhance the performance and adaptability of conversational agents on complex tasks by enabling them to learn and correct errors directly from live user interactions with minimal computational overhead. |
| Benefits and Pitfalls of Reinforcement Learning for Language Model
Planning: A Theoretical Perspective (Read more on arXiv or HuggingFace)| | This paper theoretically analyzes RL methods for LLM planning, demonstrating that while Policy Gradient (PG) generalizes better than Supervised Fine-Tuning (SFT) through exploration, it suffers from diversity collapse, an issue mitigated by Q-learning with process-based rewards. The main objective is to establish a theoretical basis for the effectiveness and limitations of RL (PG and Q-learning) over SFT in planning, abstracted as a graph-pathfinding problem. The methodology involves a theoretical analysis of the learning dynamics and stable points for each training paradigm within a tractable graph-based framework using a simplified Transformer model. Primary results show that while PG can achieve 100% training accuracy, its output diversity continuously declines, whereas Q-learning with process-level rewards converges to a diversity-preserving solution that captures the correct graph structure, avoiding the reward hacking seen with outcome-only rewards. The principal implication for AI practitioners is that using PG for planning may inadvertently reduce solution diversity and harm generalization; employing Q-learning with process-based rewards is a more robust alternative that maintains diversity and enables more efficient off-policy training. |
| TAU: A Benchmark for Cultural Sound Understanding Beyond Semantics (Read more on arXiv or HuggingFace)| Szu-Chi Chen, Yueh-Hsuan Huang, Jia-Kai Dong, Yu-Hua Chen, Yi-Cheng Lin | This paper presents TAU, a benchmark for evaluating Large Audio-Language Models’ (LALMs) understanding of culturally specific, non-semantic Taiwanese “soundmarks”. The primary objective is to assess if current models can generalize to localized audio cues that are independent of lexical content and require cultural exposure to recognize. The benchmark was constructed using a human-in-the-loop pipeline involving curated concept collection, LLM-assisted generation of 1,794 multiple-choice questions for 702 audio clips, and an automated filtering process using ASR to ensure questions are not solvable by transcript alone. Experiments show that the best-performing model, Gemini 2.5 Pro, achieves a maximum accuracy of 73.9%, which is significantly lower than the human topline of 84.0%, revealing a substantial performance gap on localized audio tasks. The principal implication for AI practitioners is that models trained on globally-sourced data exhibit significant cultural blind spots, highlighting the critical need to incorporate localized datasets and evaluation methods to build more equitable and robust multimodal systems, as prompt engineering alone proved insufficient to close this gap. |
| EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series
Forecasting (Read more on arXiv or HuggingFace)| | The paper introduces EntroPE, a time series forecasting framework that uses conditional entropy to dynamically create variable-length patches aligned with the data’s temporal structure. The objective is to overcome the limitations of fixed-length, temporally-agnostic patching in transformers, which can fragment coherent patterns and cause train-inference distribution shifts. The methodology employs an Entropy-based Dynamic Patcher (EDP) with a lightweight causal transformer to identify transition points for patch boundaries, and an Adaptive Patch Encoder (APE) that uses cross-attention to create fixed-size representations from these variable-length patches. Experiments show EntroPE improves accuracy over baselines, achieving an approximate 20% accuracy gain on the ETTh1 benchmark relative to PatchTST. The principal implication for AI practitioners is that incorporating information-theoretic criteria into the input tokenization process is a practical method to improve model performance and efficiency in time series forecasting by respecting the data’s intrinsic temporal dynamics. |
| VisualOverload: Probing Visual Understanding of VLMs in Really Dense
Scenes (Read more on arXiv or HuggingFace)| Muhammad Huzaifa, Soumya Jahagirdar, M. Jehanzeb Mirza, Wei Lin, Paul Gavrikov | This paper introduces VisualOverload, a VQA benchmark designed to test the fine-grained perception capabilities of Vision-Language Models (VLMs) in visually dense scenes. The primary objective is to assess whether state-of-the-art VLMs can perform fundamental, knowledge-free vision tasks in complex, “overloaded” environments, where existing benchmarks might overestimate their capabilities. The methodology involves creating a new dataset of 2,720 manually annotated question-answer pairs based on 150 high-resolution, public-domain paintings, covering six categories: activity, attribute, counting, OCR, reasoning, and scene classification, and then evaluating 37 different VLMs. The primary result is that even the best-performing model (o3) achieves only 69.5% overall accuracy and a mere 19.6% accuracy on the hardest question split, revealing significant failures in counting, OCR, and maintaining logical consistency. The principal implication for AI practitioners is that current VLMs are unreliable for applications requiring detailed perception in visually complex settings, as the vision encoder acts as a significant information bottleneck, limiting performance on fine-grained tasks. |
| jina-reranker-v3: Last but Not Late Interaction for Document Reranking (Read more on arXiv or HuggingFace)| | jina-reranker-v3 is a 0.6B parameter multilingual document reranker introducing a novel “last but not late interaction” architecture for efficient and effective document ranking. The research aims to bridge the efficiency-effectiveness tradeoff in neural document reranking by enabling rich cross-document interactions during encoding while maintaining competitive performance and efficiency. This is achieved through a novel architecture based on Qwen3-0.6B, employing causal self-attention within a shared context window to process queries and multiple documents simultaneously, followed by contextual embedding extraction from special tokens and a lightweight MLP projector. jina-reranker-v3 achieves state-of-the-art 61.94 nDCG@10 on the BEIR benchmark, outperforming the 1.5B parameter mxbai-rerank-large-v2 (61.44 nDCG@10) with 2.5 times fewer parameters. AI practitioners can leverage jina-reranker-v3 for high-performance, parameter-efficient document reranking in diverse domains, including complex retrieval tasks and multilingual scenarios, without incurring the computational costs of larger generative models. |
| d^2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching (Read more on arXiv or HuggingFace)| Jiarui Wang, Jiale Fu, Xiangzhong Luo, Yue Cai, Yuchu Jiang | d²Cache accelerates diffusion-based LLMs by introducing a training-free approximate Key-Value (KV) cache framework. The main objective is to overcome the inference efficiency challenges of dLLMs, which cannot directly benefit from standard KV caching due to bidirectional attention. d²Cache employs a two-stage fine-grained token selection strategy, using certainty prior for masked tokens and attention rollout for remaining tokens, to adaptively update only necessary KV states while caching others for reuse. Experiments demonstrate that d²Cache achieves an average 3.2x–4.0x inference speedup over Vanilla dLLMs and improves inference throughput on Dream-Inst/GSM8K by 4.7x (from 2.62 to 12.25 tokens/second) without sacrificing generation quality. This allows AI practitioners to significantly enhance dLLM inference efficiency and generation reliability, making these models more practical for various language tasks. |
| TTT3R: 3D Reconstruction as Test-Time Training (Read more on arXiv or HuggingFace)| Anpei Chen, Andreas Geiger, Yuliang Xiu, Yue Chen, rover-xingyu | TTT3R enhances 3D reconstruction models by integrating a Test-Time Training perspective into RNN-based architectures. The research aims to overcome length generalization limitations and state forgetting in online 3D reconstruction models. This is achieved by reformulating state updates as an online learning process, where a confidence-guided, closed-form learning rate derived from alignment confidence between memory and observations is used for memory updates. TTT3R achieves a 2x improvement in global pose estimation over baselines while operating at 20 FPS with only 6 GB of GPU memory for thousands of images. This training-free, plug-and-play intervention offers AI practitioners a more scalable, efficient, and robust solution for real-time 3D reconstruction without additional fine-tuning or parameters. |
| Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics
Research Benchmark (Read more on arXiv or HuggingFace)| Penghao Zhu, Tianci Zhou, Xiaocheng Yang, Minyang Tian, Minhui Zhu | This paper introduces CritPt, a benchmark of 71 unpublished, research-level physics challenges designed to test the complex reasoning capabilities of Large Language Models (LLMs). The research objective is to determine if current LLMs can effectively solve unseen, open-ended problems characteristic of frontier physics research. The methodology consists of evaluating LLMs on problems created by over 50 physicists, using a two-step generation protocol and a physics-informed automated grading pipeline that verifies numerical, symbolic, and code-based answers. The primary finding is that the best-performing base model, GPT-5 (high), achieves only 4.0% average accuracy on full challenges, which increases to 11.7% when augmented with a code interpreter and web search. The principal implication for AI practitioners is that a significant gap exists between current model capabilities and the rigorous demands of scientific research, highlighting the need for developing more robust and scientifically grounded reasoning systems. |
| Voice Evaluation of Reasoning Ability: Diagnosing the Modality-Induced
Performance Gap (Read more on arXiv or HuggingFace)| Hengfan Zhang, Yudong Liu, Qinsi Wang, Zhengmian Hu, linyueqian | This paper introduces the VERA benchmark to systematically quantify the “Voice Reasoning Gap” (VRG), the performance degradation observed when language models reason through a voice interface versus a text interface. The primary objective is to diagnose this modality-induced gap, hypothesizing it stems from an architectural conflict between low-latency audio streaming and the iterative computation required for complex reasoning. The methodology involves creating the VERA benchmark with 2,931 voice-adapted tasks across five reasoning domains and evaluating 12 diverse voice systems against text-only baselines, measuring both accuracy and first-response latency. A primary result shows a 68.7-point accuracy drop on competition mathematics, with a leading text model (GPT-5) achieving 74.8% accuracy while its voice counterpart (GPT-realtime) scores only 6.1%. The principal implication for AI practitioners is that overcoming the VRG requires fundamental architectural shifts away from monolithic streaming models toward systems that explicitly decouple the reasoning process from real-time speech narration. |
| LayerD: Decomposing Raster Graphic Designs into Layers (Read more on arXiv or HuggingFace)| Kota Yamaguchi, Naoto Inoue, Kang-Jun Liu, Tomoyuki Suzuki | The paper presents LayerD, a framework that automatically decomposes single raster graphic designs into a sequence of editable layers. The primary objective is to reverse the image composition process by iteratively applying top-layer matting to extract the unoccluded foreground, followed by background completion using an inpainting model, and a novel palette-based refinement step to improve quality. On the Crello dataset, LayerD significantly outperforms baselines, achieving a higher Alpha soft IoU (~0.83 vs. <0.75 for VLM/YOLO baselines) and lower RGB L1 error with zero allowed edits. For AI practitioners, this work provides a complete pipeline and models for reverse-engineering flat graphic assets, enabling layer-based editing capabilities in creative tools for images that have lost their original layer structure. |
| Who invented deep residual learning? (Read more on arXiv or HuggingFace)| Juergen Schmidhuber | This paper argues that the foundational principle of deep residual learning—residual connections with a weight of 1.0 to ensure constant error flow—was introduced in 1991 for Recurrent Neural Networks (RNNs). The primary objective is to document the historical evolution of residual connections and attribute their invention to Sepp Hochreiter’s 1991 diploma thesis, which mathematically derived them to solve the vanishing gradient problem. The methodology employed is a historical review of publications, tracing the concept’s lineage from 1991 RNNs, through 1997 LSTMs and their gated 1999 variants, to their adaptation in feedforward architectures like Highway Networks (May 2015) and ResNets (Dec 2015). The primary result is the presented timeline, which establishes ResNet as an open-gated variant of the earlier Highway Network and a feedforward version of the 1997 LSTM. A key quantitative finding illustrates that a connection weight of 0.99 reduces a backpropagated error signal over 100 steps to ~37% of its original magnitude, whereas a weight of 0.9 reduces it to ~0.0027%, demonstrating the necessity of a weight of exactly 1.0 to prevent vanishing gradients. The principal implication for AI practitioners is that the core mechanism enabling modern very deep networks is the constant error flow via identity connections, a principle first developed for RNNs to solve the vanishing gradient problem, not a concept unique to recent feedforward architectures. |
| Knowledge Homophily in Large Language Models (Read more on arXiv or HuggingFace)| Nedim Lipka, Mahantesh Halappanavar, Zhisheng Qi, Utkarsh Sahu, Franck-Dernoncourt | This research demonstrates that Large Language Models exhibit “knowledge homophily,” where the model’s knowledge level about topologically close entities in a knowledge graph is similar, and leverages this structural pattern to improve knowledge-intensive tasks. The main objective is to empirically discover this knowledge homophily pattern and apply it to efficiently identify and address knowledge gaps in LLMs. The key methodology involves first computing entity-level “knowledgeability” scores by prompting an LLM on knowledge graph triplets, and then training a Graph Neural Network (GNN) to predict these scores for all entities by leveraging their local graph neighborhood. The primary results show that the GNN-guided approach for knowledge injection improves generalization gain from a random baseline of 60.5% to 67.7%, and the homophily-aware knowledge retrieval for question-answering improves 2-hop QA accuracy by an average of 4.57% over a semantic search baseline. The principal implication for AI practitioners is that an LLM’s knowledge gaps can be predicted from a small sample of probed entities, enabling more efficient active labeling for fine-tuning and the development of smarter context retrieval systems that prioritize less-known, more informative facts for RAG. |
| BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source
Software (Read more on arXiv or HuggingFace)| | This paper introduces BUILD-BENCH, a representative benchmark for compiling real-world open-source software, and presents OSS-BUILD-AGENT, an agentic system designed for this task. The main objective is to create a more challenging and generalizable benchmark for automated C/C++ open-source software compilation and to evaluate the performance of LLM agents against rule-based and existing agentic methods. The key methodology involves creating the BUILD-BENCH dataset by sampling 148 less-popular GitHub repositories to better represent in-the-wild challenges and developing OSS-BUILD-AGENT, a multi-agent system with an LLM-assisted instruction retrieval module and an iterative error resolution loop. The primary result shows that the proposed OSS-BUILD-AGENT, using LLM-assisted retrieval with the Claude 3.7-Sonnet model, achieved a 66.4% strict success rate on BUILD-BENCH, substantially outperforming prior methods. The principal implication for AI practitioners is that iterative agentic frameworks with explicit instruction retrieval and multi-turn error resolution loops are significantly more effective for automating complex software builds than single-turn prompts or rule-based systems, and their performance scales with the intelligence of the underlying LLM. |
| Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional
Video Generation (Read more on arXiv or HuggingFace)| | This work introduces Stable Cinemetrics (SCINE), a framework using four hierarchical taxonomies—Setup, Event, Lighting, and Camera—to evaluate text-to-video models against professional filmmaking standards. The main objective is to assess the readiness of current generative models for professional use by measuring their adherence to 76 fine-grained cinematic control nodes. The methodology involves a large-scale human study where 80+ film professionals evaluated over 20K videos from 10+ models, and the training of a vision-language model (VLM) for automated evaluation. Primary results reveal that even the strongest models exhibit significant gaps, particularly in Event and Camera-related controls, while the trained VLM achieves a 72.36% accuracy in alignment with expert annotations. The principal implication for AI practitioners is that SCINE provides a granular, structured benchmark to diagnose specific model failures in cinematic control, guiding development beyond general prompt fidelity toward professional-grade video synthesis. |
| ProfVLM: A Lightweight Video-Language Model for Multi-View Proficiency
Estimation (Read more on arXiv or HuggingFace)| Antonio Liotta, Jacopo Staiano, Edoardo Bianchi | ProfVLM is a lightweight, multi-view video-language model that jointly estimates skill proficiency and generates expert-like textual feedback by reformulating the task as conditional text generation. The objective is to develop a computationally efficient and explainable framework for multi-view skill assessment that unifies proficiency classification and natural language feedback generation into a single generative task. The model employs a frozen TimeSformer visual encoder, a novel AttentiveGatedProjector module for multi-view feature fusion via attention and learnable gating, and a lightweight language model (SmolLMv2-135M) fine-tuned with LoRA to generate the final output. ProfVLM achieves state-of-the-art 48.2% top-1 accuracy on the EgoExo4D benchmark while using 20x fewer trainable parameters (5.3M vs. 121M) and reducing training time by 60% compared to baselines. The principal implication for AI practitioners is that framing video analysis tasks as conditional language generation can create more efficient, parameter-light, and interpretable models capable of providing richer, human-like feedback than traditional classification-only architectures. |
| MANI-Pure: Magnitude-Adaptive Noise Injection for Adversarial
Purification (Read more on arXiv or HuggingFace)| Zhiming Luo, Carl Yang, Kejia Zhang, Junwei Wu, Xiaoyi Huang | The paper introduces MANI-Pure, a diffusion-based adversarial purification framework that uses magnitude-adaptive noise injection to selectively target perturbations in the frequency domain. The objective is to improve upon existing purification defenses by replacing uniform noise injection with a method that adapts to the empirically observed non-uniform distribution of adversarial perturbations, thereby enhancing robustness while preserving clean image fidelity. The core methodology involves a forward process (MANI) that computes frequency-band-wise weights from an input’s magnitude spectrum to generate heterogeneous noise targeting vulnerable low-magnitude frequencies, and a reverse process (FreqPure) that preserves low-frequency content from the original input. On CIFAR-10 against a ViT-L/14 classifier, MANI-Pure achieves 92.19% robust accuracy under an AutoAttack (l-infinity) threat model while maintaining a standard accuracy of 94.14%, narrowing the clean accuracy gap to the original classifier to 0.59%. The principal implication for AI practitioners is that MANI can be used as an efficient, plug-and-play module to significantly boost the adversarial robustness of various off-the-shelf models at inference time without requiring retraining. |
| Specialization after Generalization: Towards Understanding Test-Time
Training in Foundation Models (Read more on arXiv or HuggingFace)| | This paper explains Test-Time Training (TTT) as a “specialization after generalization” mechanism, investigating why it improves performance on in-distribution data by allowing globally underparameterized models to locally reallocate capacity to task-relevant concepts. The work uses the Linear Representation Hypothesis (LRH) and trains a sparse autoencoder on ImageNet to show local data neighborhoods are sparsely supported, complemented by scaling studies comparing TTT against globally trained models across vision and language tasks. The key finding is that TTT’s performance advantage over global training is largest for smaller models and diminishes as models become overparameterized; for instance, a sparsely supported TTT model using just ~40 active concepts achieved 72.64% accuracy on ImageNet, matching a dense TTT model and indicating an implicit bias towards sparse solutions. For AI practitioners, this implies TTT is a highly effective strategy for boosting performance on smaller, underparameterized models, but its relative benefit decreases for very large models that have sufficient capacity for global generalization. |
| Estimating Time Series Foundation Model Transferability via In-Context
Learning (Read more on arXiv or HuggingFace)| Jun Qi, Chao-Han Huck Yang, Chengqi Zhang, Ming Jin, Qingren | This paper proposes TIMETIC, a framework for estimating the transferability of Time Series Foundation Models (TSFMs) by reformulating model selection as an in-context learning problem. The objective is to efficiently predict the fine-tuning performance of a given TSFM on a target dataset without incurring the computational cost of actual fine-tuning. The methodology characterizes datasets via statistical features and models via a novel entropy profile across layers, structuring this information into a context table that prompts a tabular foundation model to predict performance for new model-data pairs. The framework achieves a mean Spearman rank correlation of approximately 0.6 between its estimated scores and actual fine-tuned performance, representing a 30% improvement over using zero-shot performance as a ranking proxy. For AI practitioners, TIMETIC offers a computationally efficient method to pre-select the most promising TSFM for a specific downstream forecasting task, significantly reducing the resources required for model selection. |
Papers for 2025-09-30
| Title |
Authors |
Summary |
|
|
|
|
| SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable |
|
|
|
|
|
|
| Sparse-Linear Attention (Read more on arXiv or HuggingFace) |
|
This paper introduces Sparse-Linear Attention (SLA), a fine-tunable hybrid attention mechanism that combines sparse and linear attention to accelerate Diffusion Transformers. The primary objective is to reduce the quadratic computational cost of attention in long-sequence models, like those for video generation, without the quality degradation seen in purely sparse or linear methods. The key methodology involves dynamically classifying attention weight blocks into critical (computed with O(N²) attention), marginal (computed with O(N) linear attention), and negligible (skipped), fusing these operations into a single GPU kernel. The primary result shows that SLA reduces attention computation by 95% and achieves a 2.2x end-to-end speedup on the Wan2.1-1.3B video generation model, maintaining output quality comparable to full attention. The principal implication for AI practitioners is that they can significantly accelerate large-scale generative model inference and training with only a few fine-tuning steps, making high-resolution video generation more computationally tractable. |
|
|
|
|
| Multiplayer Nash Preference Optimization (Read more on arXiv or HuggingFace) |
|
The paper introduces Multiplayer Nash Preference Optimization (MNPO), a game-theoretic framework that generalizes two-player Nash learning to an n-player setting for LLM alignment. The primary objective is to develop an alignment method that models complex, non-transitive, and heterogeneous human preferences by framing the optimization problem as a multiplayer game, overcoming the single-opponent bias of existing Nash-based approaches. The core methodology involves each policy competing against a population of opponents to maximize its average preference probability, optimized via an iterative multiplicative weight update rule, with a practical variant (TD-MNPO) that uses a weighted mixture of historical policies as opponents. Empirically, MNPO achieves a win rate of 52.26 on the Arena-Hard benchmark, a 4.23-point improvement over the next-best baseline, INPO. For AI practitioners, MNPO offers a more robust and principled framework for aligning LLMs with complex preference data, leading to superior performance on instruction-following and reasoning tasks compared to standard preference optimization methods. |
|
|
|
|
| RealUnify: Do Unified Models Truly Benefit from Unification? A |
|
|
|
|
|
|
| Comprehensive Benchmark (Read more on arXiv or HuggingFace) |
Yuran Wang, Yue Ding, zooblastlbz, THUdyh, DogNeverSleep |
The paper introduces RealUnify, a comprehensive benchmark designed to evaluate whether unified multimodal models achieve genuine synergy between their visual understanding and generation capabilities. The main objective is to determine if architectural unification enables synergetic interaction between these constituent functions, a question unaddressed by existing benchmarks that assess them in isolation. A dual-evaluation protocol is used, which combines direct end-to-end assessment with a diagnostic stepwise evaluation that decomposes tasks into distinct understanding and generation phases to identify performance bottlenecks. The primary result from evaluating 12 unified models shows they struggle with synergy, as the best open-source model achieved only 37.5% accuracy on “Understanding Enhances Generation” tasks, far below a 72.7% upper bound established by an oracle combining specialist models. For AI practitioners, this implies that architectural unification alone is insufficient, highlighting the need for new training strategies and inductive biases to unlock the potential of unified modeling for complex tasks. |
|
|
|
|
| OpenGPT-4o-Image: A Comprehensive Dataset for Advanced Image Generation |
|
|
|
|
|
|
| and Editing (Read more on arXiv or HuggingFace) |
Huanyu Zhang, Chaoyou Fu, Xuehai Bai, Zhihong Chen, DogNeverSleep |
This paper introduces OpenGPT-40-Image, an 80k-sample dataset with a hierarchical taxonomy of 51 subtasks designed to advance complex image generation and editing capabilities in multimodal models. The research aims to address the lack of systematic structure and challenging scenarios in existing datasets by creating a comprehensive resource for training models on tasks like scientific imagery and complex instruction following. The methodology involves an automated pipeline leveraging a hierarchical task taxonomy and the GPT-40 model to generate instruction-image pairs with controlled diversity and difficulty. The primary result is that fine-tuning models on this dataset yields significant performance gains, such as an 18.4% improvement for the UniWorld-V1 model on the ImgEdit-Bench editing benchmark. The principal implication for AI practitioners is that this systematically constructed dataset can be used to fine-tune models to better handle complex, multi-step instructions and specialized tasks, improving their robustness and applicability in real-world scenarios. |
|
|
|
|
| Visual Jigsaw Post-Training Improves MLLMs (Read more on arXiv or HuggingFace) |
Lewei Lu, Yushan Zhang, Penghao Wu, luodian, Paranioar |
This paper introduces Visual Jigsaw, a self-supervised post-training framework that enhances the vision-centric understanding of Multimodal Large Language Models (MLLMs) by solving visual ordering problems. The research aims to improve an MLLM’s visual perception without altering its architecture or output format. The key methodology involves partitioning visual inputs (images, videos, 3D data) into components, shuffling them, and training the MLLM using Reinforcement Learning with Verifiable Reward (RLVR) to reconstruct the correct order. The primary results demonstrate significant improvements across various benchmarks; for instance, the Image Jigsaw method increased the Qwen2.5-VL-7B model’s score on the MMVP benchmark by +6.00 points. The principal implication for AI practitioners is that this lightweight, verifiable post-training task offers a practical method to boost the fine-grained perception, spatial reasoning, and temporal understanding of existing text-only output MLLMs without needing additional generative modules or pixel-level reconstruction. |
|
|
|
|
| SANA-Video: Efficient Video Generation with Block Linear Diffusion |
|
|
|
|
|
|
| Transformer (Read more on arXiv or HuggingFace) |
|
The paper introduces SANA-Video, an efficient diffusion transformer model for generating high-resolution, minute-length videos with low computational overhead. The objective is to overcome the prohibitive computational cost and slow inference speeds of existing high-quality video generation models. The core methodology integrates a Linear Diffusion Transformer (Linear DiT) using O(N) linear attention instead of O(N^2) vanilla attention, and a Constant-Memory KV Cache for block linear attention, which enables a block-wise autoregressive approach for long video synthesis with fixed memory cost. SANA-Video achieves competitive performance while being 16x faster than comparable models, generating a 5-second 720p video in 36 seconds on an H100 GPU. For AI practitioners, this presents a framework for high-quality video generation with a training cost of only 12 days on 64 H100 GPUs that is deployable on consumer-grade hardware (RTX 5090), lowering the barrier for developing and applying advanced video synthesis. |
|
|
|
|
| Democratizing AI scientists using ToolUniverse (Read more on arXiv or HuggingFace) |
|
TOOLUNIVERSE is an open-source ecosystem designed to build and democratize AI scientists by standardizing how they interact with scientific tools. The primary objective is to create a unified and extensible infrastructure to overcome the limitations of bespoke, rigid AI systems, thereby enabling interoperability and reuse. The methodology centers on an AI-tool interaction protocol that standardizes tool discovery (Find Tool) and execution (Call Tool), integrating over 600 machine learning models, APIs, and scientific packages into a common framework. In a therapeutic discovery case study for hypercholesterolemia, an AI scientist built using TOOLUNIVERSE identified a potent drug analog (CHEMBL2347006/CHEMBL3970138) with favorable predicted properties, demonstrating the system’s ability to automate complex scientific workflows. The principal implication for AI practitioners is that TOOLUNIVERSE provides a standardized, scalable framework to equip language models and agents with diverse, domain-specific tools, significantly reducing the engineering overhead required to build sophisticated AI research assistants. |
|
|
|
|
| When Does Reasoning Matter? A Controlled Study of Reasoning’s |
|
|
|
|
|
|
| Contribution to Model Performance (Read more on arXiv or HuggingFace) |
|
This paper conducts a large-scale controlled study using synthetic data distillation to systematically evaluate the performance and efficiency trade-offs between Instruction Fine-Tuning (IFT) and reasoning-based training for LLMs. The main objective is to determine the specific conditions—regarding task type, model scale, and computational cost—under which explicit reasoning training provides superior performance compared to standard IFT. The study employs a controlled distillation framework where a single teacher model generates 1.6M paired IFT and reasoning-based training samples to train student models of varying scales (0.5B to 14B), isolating the impact of the supervision format. The primary result is that reasoning-based training unlocks higher performance on open-ended and math tasks, enabling a 3B parameter reasoning model to match the accuracy of a 14B IFT model on several benchmarks. For AI practitioners, the principal implication is that for reasoning-intensive tasks, training models larger than 7B with reasoning data is justified to break performance plateaus, while for cost-sensitive applications or simpler tasks, scaling an IFT model remains the more efficient strategy. |
|
|
|
|
| GSM8K-V: Can Vision Language Models Solve Grade School Math Word |
|
|
|
|
|
|
| Problems in Visual Contexts (Read more on arXiv or HuggingFace) |
|
This paper introduces GSM8K-V, a benchmark that transforms text-based math word problems from GSM8K into a purely visual, multi-image comic-style format to evaluate the mathematical reasoning capabilities of Vision Language Models (VLMs). The main research objective is to assess if VLMs can solve grade school math word problems when presented entirely through visual context and to measure the performance gap against their text-based reasoning abilities. The methodology involves a three-stage automated pipeline that decomposes mathematical information from GSM8K, generates multi-scene textual descriptions, and then uses an image generation model to create corresponding visual scenes, which are subsequently refined through human annotation. The primary result is a significant performance drop on the visual benchmark; for instance, the best-performing model, Gemini-2.5-Pro, achieves 95.22% accuracy on text-based GSM8K but only 46.93% on GSM8K-V. The principal implication for AI practitioners is that current VLMs exhibit a critical deficiency in visually grounded mathematical reasoning, indicating that models proficient in textual reasoning do not generalize well to equivalent visual problem formats, thus necessitating the development of more robust multimodal reasoning architectures. |
|
|
|
|
| EasySteer: A Unified Framework for High-Performance and Extensible LLM |
|
|
|
|
|
|
| Steering (Read more on arXiv or HuggingFace) |
|
The paper introduces EasySteer, a unified framework for high-performance, extensible large language model (LLM) steering at inference time. The primary objective is to address the computational inefficiency, limited extensibility, and functional restrictions of existing steering frameworks. Methodologically, EasySteer is built on the vLLM inference engine, featuring a modular architecture with a non-intrusive model wrapper, pluggable interfaces for analytical and learning-based vector generation, and fine-grained parameter control mechanisms. The primary result shows that EasySteer achieves a 5.5-11.4× speedup over existing frameworks, reaching 3619.09 tokens/s in long-sequence batch inference compared to 652.63 tokens/s for the pyreft framework. For AI practitioners, EasySteer provides production-ready infrastructure to implement precise, on-the-fly behavioral control over LLMs in high-throughput serving environments without expensive retraining. |
|
|
|
|
| EditScore: Unlocking Online RL for Image Editing via High-Fidelity |
|
|
|
|
|
|
| Reward Modeling (Read more on arXiv or HuggingFace) |
|
This paper develops EditScore, a high-fidelity reward model that enables stable online reinforcement learning (RL) for instruction-guided image editing. The main objective is to overcome the lack of a reliable and efficient reward signal, which has hindered the application of online RL to image editing. The methodology involves first creating EditReward-Bench, a benchmark for evaluating editing reward models, and then training EditScore, a series of reward models (7B-72B) on curated data, which leverages a self-ensembling strategy at inference time. The primary result is that the EditScore-72B model with self-ensembling surpasses GPT-5’s pairwise accuracy on EditReward-Bench (0.763 vs. 0.755), and its application in RL training improves the OmniGen2 base model’s overall score on GEdit-Bench-EN by +0.40. The principal implication for AI practitioners is that they can use the open-source EditScore as a robust reward signal to successfully apply online RL for fine-tuning image editing models, a task where general-purpose VLMs were demonstrated to fail and cause training instability. |
|
|
|
|
| SparseD: Sparse Attention for Diffusion Language Models (Read more on arXiv or HuggingFace) |
Xinchao Wang, Xinyin Ma, Gongfan Fang, adamdad, INV-WZQ |
SparseD is a novel sparse attention method that accelerates Diffusion Language Models (DLMs) by leveraging their unique attention characteristics to reduce computational overhead with minimal accuracy loss. The primary research objective is to mitigate the high inference latency in DLMs caused by the quadratic complexity of bidirectional attention, particularly in long-context scenarios. The key methodology involves a three-part strategy: (1) applying full attention for an initial percentage of denoising steps to preserve generation quality, (2) pre-computing head-specific sparse attention patterns only once after this initial phase, and (3) reusing these static patterns for all subsequent denoising steps. The primary result shows that SparseD achieves lossless acceleration, delivering up to a 1.50× speedup over FlashAttention on a model with a 64k context length and 1,024 denoising steps. For AI practitioners, this provides an effective, practical method to deploy DLMs in long-context applications with significantly reduced inference latency without the performance degradation associated with methods designed for autoregressive models or existing DLM caching techniques. |
|
|
|
|
| Sequential Diffusion Language Models (Read more on arXiv or HuggingFace) |
|
The paper introduces Sequential Diffusion Language Models (SDLMs), a method to retrofit pretrained autoregressive language models for faster, dynamic-length parallel decoding. The primary objective is to unify autoregressive and diffusion-based generation to overcome the limitations of fixed-length decoding in diffusion models and the high training costs of existing hybrid approaches. The key methodology involves Next Sequence Prediction (NSP), which generalizes next-token and next-block prediction, implemented via a parallel block training scheme that uses a custom bidirectional attention mask to enable dynamic decoding based on model confidence. The primary result shows that SDLM-32B, trained on only 3.5M samples, achieves a 92.4 score on GSM8K, matching its autoregressive counterpart while achieving up to 2.1x higher throughput than the Qwen-2.5-3B model. For AI practitioners, the principal implication is that existing pretrained autoregressive models can be adapted with minimal fine-tuning to gain significant inference acceleration without a substantial trade-off in performance, offering an efficient alternative to training new models from scratch. |
|
|
|
|
| Towards Personalized Deep Research: Benchmarks and Evaluations (Read more on arXiv or HuggingFace) |
|
This paper introduces the Personalized Deep Research Bench and the PQR Evaluation Framework to measure the ability of Deep Research Agents (DRAs) to tailor outputs to specific user profiles. The primary objective is to systematically evaluate how effectively DRAs adapt complex research, reasoning, and reporting to individual user personas, a dimension neglected by existing evaluation methodologies. The methodology involves a benchmark of 250 queries (50 tasks paired with 25 user profiles) and an LLM-as-a-judge PQR framework that assesses (P) Personalization Alignment, (Q) Content Quality, and (R) Factual Reliability. Experiments reveal a trade-off between capabilities, with open-source systems like OAgents achieving the highest overall personalization score (6.64) but commercial systems demonstrating superior factual accuracy. The principal implication for AI practitioners is that building effective personalized DRAs requires architectures that move beyond simple search tool integration to jointly optimize for user alignment and factual reliability, as current systems show a significant performance gap between these two functions. |
|
|
|
|
| Random Policy Valuation is Enough for LLM Reasoning with Verifiable |
|
|
|
|
|
|
| Rewards (Read more on arXiv or HuggingFace) |
Binxing Jiao, Chen Hu, Qingpeng Cai, Yuxiao Ye, Haoran He |
i) This paper introduces ROVER, a minimalist Reinforcement Learning (RL) algorithm that improves LLM reasoning by valuing a fixed random policy instead of using complex policy iteration. ii) The objective is to design a simpler, more effective RL with Verifiable Rewards (RLVR) algorithm by exploiting the deterministic, tree-structured Markov Decision Process (MDP) specific to LLM reasoning tasks, thereby avoiding the instability and complexity of standard methods like PPO. iii) ROVER’s methodology bypasses generalized policy iteration by first proving that the Q-function of a fixed uniform policy is sufficient for optimal action recovery in this specific MDP structure, and then samples actions from a softmax distribution over these Q-values to balance quality and diversity. iv) The primary results show that ROVER achieves superior performance over strong baselines, yielding a +8.2 improvement on pass@1 for competition-level math tasks and a +17.6% increase in solution diversity. v) The principal implication for AI practitioners is that complex, unstable algorithms like PPO can be replaced with the simpler, more robust, and higher-performing ROVER framework for fine-tuning LLMs on verifiable reasoning tasks, reducing implementation overhead and mitigating diversity collapse. |
|
|
|
|
| Beyond the Exploration-Exploitation Trade-off: A Hidden State Approach |
|
|
|
|
|
|
| for LLM Reasoning in RLVR (Read more on arXiv or HuggingFace) |
Cevaaa, MasterVito, AnikiFan, Gambel, Niugan |
This paper introduces Velocity-Exploiting Rank-Learning (VERL), a method that challenges the exploration-exploitation trade-off in LLM reasoning by operating on hidden-state dynamics instead of token-level metrics. The research aims to determine if this trade-off is an artifact of measurement and to develop a method that simultaneously enhances both exploration and exploitation in Reinforcement Learning for Verifiable Rewards (RLVR). The key methodology involves quantifying exploration using Effective Rank (ER) of hidden-state matrices and defining exploitation via its novel derivatives: Effective Rank Velocity (ERV) and Effective Rank Acceleration (ERA). The VERL algorithm then uses the theoretically stable ERA as a predictive meta-controller to adaptively shape the RL advantage function, creating a dual-channel incentive structure. The primary result is the empirical demonstration that exploration and exploitation are decoupled at the hidden-state level, and VERL achieves significant performance gains, including up to a 21.4% absolute accuracy improvement on the Gaokao 2024 dataset. For AI practitioners, the principal implication is that they can augment existing RL algorithms like PPO or GRPO with VERL to synergistically improve a model’s ability to discover diverse reasoning paths and consolidate correct ones, leading to enhanced generalization on complex reasoning tasks. |
|
|
|
|
| VideoScore2: Think before You Score in Generative Video Evaluation (Read more on arXiv or HuggingFace) |
|
This paper introduces VideoScore2, a multi-dimensional, interpretable framework for evaluating text-to-video generation by providing scores and chain-of-thought rationales. The objective is to develop a human-aligned evaluator that assesses visual quality, text alignment, and physical consistency, overcoming the limitations of single, opaque scores. The methodology involves a two-stage training pipeline on a new 27,168-sample dataset (VIDEOFEEDBACK2): initial supervised fine-tuning for basic reasoning, followed by reinforcement learning with Group Relative Policy Optimization (GRPO) to enhance analytical robustness. VideoScore2 achieves state-of-the-art performance, demonstrating 44.35% accuracy on its in-domain benchmark (a +5.94% improvement over the best baseline) and strong generalization on out-of-domain benchmarks. For AI practitioners, VideoScore2 serves as a more effective reward model for guiding controllable generation through methods like Best-of-N sampling, providing actionable, interpretable feedback for model development. |
|
|
|
|
| From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by |
|
|
|
|
|
|
| Composing Old Ones (Read more on arXiv or HuggingFace) |
Hanbin Wang, Ganqu Cui, Yuchen Zhang, Weize Chen, Lifan Yuan |
This research demonstrates that Large Language Models can acquire new, generalizable skills by learning to compose existing atomic skills through reinforcement learning when explicitly incentivized with compositional tasks. The main research question is whether reinforcement learning (RL) teaches LLMs genuinely new skills, specifically compositionality, or if it merely activates pre-existing capabilities. The methodology involves a two-stage training protocol on a controlled string transformation task: first, an LLM acquires atomic skills via rejection fine-tuning (RFT), then it is trained on compositional problems using either RL with outcome-based rewards or RFT, without access to the underlying function definitions. The primary result is that RL on Level-2 compositional tasks enables generalization to more complex, unseen problems, with performance on Level-3 tasks improving from near-zero to approximately 30%, whereas RFT on the same data yields negligible improvement. The principal implication for AI practitioners is that developing advanced, generalizable reasoning requires a strategic training approach: first establish a foundation of atomic skills in a base model, then use RL with explicit compositional incentives to teach the model how to combine those skills to solve more complex problems. |
|
|
|
|
| Euclid’s Gift: Enhancing Spatial Perception and Reasoning in |
|
|
|
|
|
|
| Vision-Language Models via Geometric Surrogate Tasks (Read more on arXiv or HuggingFace) |
|
This paper demonstrates that fine-tuning vision-language models on Euclidean geometry problems, as a surrogate task, significantly enhances their generalizable spatial reasoning capabilities. The research investigates whether training on a curated dataset of geometric problems can instill foundational spatial priors that improve zero-shot performance on diverse, unseen spatial intelligence tasks. The authors constructed Euclid30K, a dataset of ~30K geometry problems, and used Group Relative Policy Optimization (GRPO) to fine-tune the Qwen2.5VL and RoboBrain2.0 model families. After training, the RoboBrain2.0-Euclid-7B model achieved 49.6% accuracy on VSI-Bench, improving by 6.6 percentage points over its baseline and surpassing the previous state-of-the-art spatial model. The principal implication for AI practitioners is that using compact, principle-driven surrogate datasets can be a more effective strategy for developing transferable, foundational skills in MLLMs than training on larger, task-specific datasets. |
|
|
|
|
| Critique-Coder: Enhancing Coder Models by Critique Reinforcement |
|
|
|
|
|
|
| Learning (Read more on arXiv or HuggingFace) |
|
This paper introduces Critique Reinforcement Learning (CRL), a novel paradigm to enhance code generation models, and a resulting model, CRITIQUE-CODER. The primary objective is to determine if complementing standard reinforcement learning (RL) with an explicit critique-learning signal improves a model’s coding and general reasoning abilities. The methodology involves training a model with Group Relative Policy Optimization (GRPO) on a hybrid dataset where 20% of standard RL tasks (generating solutions) are replaced with CRL tasks (judging existing solutions). The key result is that CRITIQUE-CODER-8B achieves 60.8% on LiveCodeBench (v5), outperforming the RL-only baseline, and also demonstrates a +6.1 point average improvement over the base model on the BBEH logic reasoning benchmark. For AI practitioners, the principal implication is that augmenting standard RL fine-tuning with a small proportion of critique-based training can significantly boost not only task-specific performance but also transferable reasoning capabilities in large language models. |
|
|
|
|
| StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient |
|
|
|
|
|
|
| SpeechLLMs (Read more on arXiv or HuggingFace) |
Wei Jia, Aiwei Liu, Chuhan Wu, Linhao Zhang, QbethQ |
This paper introduces StableToken, a semantic speech tokenizer that achieves state-of-the-art noise robustness for building resilient SpeechLLMs. The research objective is to overcome the fragility of existing tokenizers whose discrete outputs are unstable against meaning-irrelevant acoustic perturbations, which increases the learning burden for downstream models. The key methodology involves a multi-branch quantizer architecture (Voting-LFQ) that processes audio in parallel and merges representations via a differentiable bit-wise majority vote, coupled with a Noise-Aware Consensus Training strategy that forces alignment between clean and perturbed audio views. StableToken sets a new state-of-the-art in token stability, reducing the average Unit Edit Distance (UED) under diverse noise conditions to 10.17%, a relative reduction of over 60% compared to the best supervised baseline. For AI practitioners, this work provides a tokenizer that directly enhances the robustness of SpeechLLMs in noisy, real-world conditions, improving downstream task performance without compromising reconstruction fidelity. |
|
|
|
|
| VGGT-X: When VGGT Meets Dense Novel View Synthesis (Read more on arXiv or HuggingFace) |
Zhaoxiang Zhang, Junran Peng, Zimo Tang, Chuanchen Luo, Yang Liu |
This paper presents VGGT-X, a framework for adapting the VGGT 3D foundation model to dense novel view synthesis by addressing its inherent scalability and accuracy limitations. The research aims to resolve the primary obstacles—prohibitive VRAM consumption and noisy initial predictions—that hinder the application of 3D foundation models to dense image sets (1,000+ images) for 3D Gaussian Splatting. The methodology integrates a memory-efficient VGGT implementation (using feature reduction and mixed-precision), an adaptive global alignment strategy for pose refinement using XFeat correspondences, and robust MCMC-3DGS training with joint pose optimization. The framework achieves state-of-the-art results for COLMAP-free NVS, improving pose estimation AUC@30 on the MipNeRF360 dataset from 0.951 to 0.992 and significantly reducing the rendering quality gap compared to COLMAP-initialized methods. For AI practitioners, the principal implication is that with targeted memory optimizations and a robust global alignment post-processing step, large 3D foundation models become a viable and dramatically faster alternative to traditional Structure-from-Motion pipelines for initializing dense 3D reconstruction tasks. |
|
|
|
|
| BRIDGE - Building Reinforcement-Learning Depth-to-Image Data Generation |
|
|
|
|
|
|
| Engine for Monocular Depth Estimation (Read more on arXiv or HuggingFace) |
|
BRIDGE is a framework that uses a reinforcement learning-optimized engine to generate a massive (20M+) synthetic RGB-D dataset for training high-performance monocular depth estimation models. The primary objective is to overcome the limitations of data scarcity and quality in monocular depth estimation by creating a large-scale, high-fidelity, and diverse synthetic dataset. The methodology involves an RL-optimized Depth-to-Image (D2I) model that generates 20 million realistic RGB images from source depth maps, followed by a hybrid supervision strategy that combines teacher-generated pseudo-labels with high-precision ground truth depth in similarity-masked regions to train a DINOv2-based MDE model. BRIDGE achieves state-of-the-art zero-shot performance, attaining a δ1 accuracy of 0.982 and an absolute relative error of 0.041 on the NYUv2 dataset, which surpasses Depth Anything V2’s 0.979 and 0.045 respectively, while using significantly less training data (20M vs. 62M). For AI practitioners, this RL-based data generation and hybrid supervision approach provides a blueprint for creating massive, high-quality, geometrically consistent training datasets for vision tasks, reducing dependency on costly real-world data collection and improving model performance and training efficiency. |
|
|
|
|
| Rolling Forcing: Autoregressive Long Video Diffusion in Real Time (Read more on arXiv or HuggingFace) |
|
The paper introduces Rolling Forcing, a novel technique for real-time, autoregressive long video generation that significantly reduces error accumulation over extended durations. The objective is to generate high-quality, low-latency, and temporally coherent long video streams while mitigating the severe error accumulation that plagues existing autoregressive methods. The core methodology integrates three components: a rolling-window joint denoising scheme that processes multiple frames simultaneously with progressive noise, an attention sink mechanism that caches initial frames as a global context anchor, and an efficient few-step distillation training algorithm on non-overlapping windows. Extensive experiments demonstrate that Rolling Forcing enables real-time generation at 15.79 FPS on a single GPU with a substantially reduced quality drift score of 0.01, compared to a baseline of 1.66 from Self Forcing. For AI practitioners, this work provides a framework for developing interactive long-form video applications, like neural game engines, that can maintain high temporal consistency over multi-minute sequences. |
|
|
|
|
| MMPB: It’s Time for Multi-Modal Personalization (Read more on arXiv or HuggingFace) |
|
This paper introduces MMPB, a comprehensive benchmark for evaluating the personalization capabilities of Vision-Language Models (VLMs) on concept recognition and preference-grounded reasoning. The primary objective is to systematically quantify the ability of VLMs to adapt to individual user contexts, a capability largely unexplored by existing general-purpose VQA benchmarks. The methodology involves creating the MMPB dataset, which contains over 10,000 image-query pairs across 111 personalizable concepts, and using it to evaluate 23 VLMs through a three-stage protocol of concept injection, multi-turn dialogue, and personalized querying. The primary result shows that most VLMs struggle with personalization, with closed-source models achieving an average accuracy of 51.4%, underperforming open-source models (59.9%), and both types exhibiting significant performance drops in multi-turn conversation settings. For AI practitioners, the key implication is that current VLMs are not suitable for personalized applications out-of-the-box, as they show systemic failures in user-centric reasoning and are prone to evasive behaviors due to safety alignments, requiring specialized fine-tuning or architectural modifications. |
|
|
|
|
| InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long |
|
|
|
|
|
|
| Adaptation (Read more on arXiv or HuggingFace) |
Yuxuan Li, Chaojun Xiao, Zhou Su, Zihan Zhou, Weilin Zhao |
InfLLM-V2 is a dense-sparse switchable attention framework that enables seamless adaptation of language models from short to long sequence processing by reusing existing dense attention parameters. The primary objective is to resolve the architectural mismatch, parameter overhead, and training instability of prior trainable sparse attention methods within the standard pretrain-on-short, finetune-on-long workflow. The methodology involves a parameter-free architectural modification that unifies sparse attention patterns, eliminates gating modules, and uses a hardware-aware, 3-stage block compression process to efficiently select relevant context blocks. The primary result is that InfLLM-V2 is 4x faster than dense attention while retaining 98.1% of the performance on long-context understanding benchmarks and 99.7% on chain-of-thought reasoning. For AI practitioners, this framework provides a practical and efficient method to adapt existing pretrained models for long-context tasks, significantly accelerating inference without substantial performance loss or complex architectural changes. |
|
|
|
|
| Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference |
|
|
|
|
|
|
| Learning (Read more on arXiv or HuggingFace) |
|
This paper introduces Tool-Light, a framework utilizing self-evolved preference learning to improve the efficiency and accuracy of Tool-Integrated Reasoning (TIR) in large language models. The main objective is to incentivize LLMs to perform TIR effectively by mitigating suboptimal behaviors like excessive tool usage or overthinking after tool calls. The core methodology is a two-stage training pipeline featuring Supervised Fine-Tuning (SFT) and a multi-round Self-Evolved Direct Preference Optimization (DPO) process, which uses a novel entropy-guided sampling strategy to generate positive-negative reasoning paths. On a suite of 10 reasoning datasets, the Tool-Light trained Qwen2.5-7B model achieved a state-of-the-art average performance of 58.0, outperforming the strong Tool-Star baseline’s score of 56.6. For AI practitioners, this work provides a concrete framework to fine-tune agentic models that can use external tools more efficiently and necessarily, reducing redundant computations while improving reasoning accuracy. |
|
|
|
|
| MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speech (Read more on arXiv or HuggingFace) |
|
MGM-Omni is a unified Omni LLM designed for omnimodal understanding and personalized, long-horizon speech generation using a dual-track architecture. The objective is to overcome the limitations of cascaded systems in processing and generating long-form audio by addressing challenges in contextual coherence, timbre consistency, and synthesis latency. The model employs a “brain-mouth” design that decouples a multimodal reasoning LLM from a speech synthesis LM and introduces a Chunk-Based Parallel Decoding mechanism to align text and speech token rates. MGM-Omni demonstrates superior long-form audio understanding, achieving a 94% success rate on a needle-in-the-haystack test with audio up to 4,500 seconds, compared to Qwen2.5-Omni’s 58%. For AI practitioners, this work provides a data-efficient, end-to-end paradigm for building robust omnimodal systems capable of long-form, personalized speech interaction without the high latency and error accumulation of separate text-to-speech pipelines. |
|
|
|
|
| HunyuanImage 3.0 Technical Report (Read more on arXiv or HuggingFace) |
|
The report introduces HunyuanImage 3.0, an open-source, 80-billion-parameter Mixture-of-Experts (MoE) model that unifies multimodal understanding and generation within a single autoregressive framework. The main objective is to develop a native multimodal model with performance that rivals state-of-the-art closed-source systems by leveraging a pre-trained MoE LLM, meticulous data curation, and a native Chain-of-Thoughts schema. The methodology involves a hybrid architecture combining autoregressive prediction for text with diffusion-based modeling for images, managed by a novel “Generalized Causal Attention” mechanism, and refined through progressive multi-stage pre-training and extensive post-training (SFT, DPO, MixGRPO, SRPO). In human GSB (Good/Same/Bad) evaluations, HunyuanImage 3.0 achieved a relative win rate of 14.10% against the previous best open-source model, HunyuanImage 2.1, and demonstrated comparable quality to leading commercial models. The principal implication for AI practitioners is the public release of a powerful, state-of-the-art foundation model and a detailed technical blueprint, providing a robust open-source alternative for developing advanced, unified multimodal applications. |
|
|
|
|
| Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs |
|
|
|
|
|
|
| at Test Time (Read more on arXiv or HuggingFace) |
Yi Yang, Ruijie Quan, Fan Ma, yixuan7878 |
This paper introduces Dynamic Experts Search (DES), a test-time scaling strategy that improves the reasoning of Mixture-of-Experts (MoE) LLMs by treating the number of activated experts as a controllable search dimension. The main objective is to investigate if dynamically varying the number of activated experts in MoE models during inference can serve as a new source of solution diversity, enhancing reasoning performance beyond architecture-agnostic, sampling-based test-time scaling methods. The key methodology integrates two components: Dynamic MoE, which allows for direct control over the expert count per inference pass, and Expert Configuration Inheritance, which maintains a consistent expert count within a single reasoning trajectory while exploring different counts across parallel search paths, guided by an external verifier. Primary results show that on the MATH500 benchmark, DES with the Qwen3-30B-A3B-Instruct model achieved 93.20% accuracy, outperforming baselines such as Best-of-N (92.40%) and BeamSearch (93.00%) at a comparable computational cost. The principal implication for AI practitioners is that the reasoning performance of deployed MoE models can be significantly enhanced at inference time by treating the number of active experts as a tunable search parameter, providing an effective, architecture-aware alternative to simple output sampling for test-time compute scaling. |
|
|
|
|
| SIRI: Scaling Iterative Reinforcement Learning with Interleaved |
|
|
|
|
|
|
| Compression (Read more on arXiv or HuggingFace) |
|
The paper introduces SIRI, a reinforcement learning approach to improve reasoning accuracy and token efficiency in Large Reasoning Models (LRMs). The primary objective is to overcome the trade-off between reducing repetitive thinking and maintaining high performance. The key methodology is an iterative training regime that alternates between a compression phase, which shortens the maximum rollout length to force dense reasoning, and an expansion phase, which relaxes the length limit to encourage exploration. After three iterations on DeepSeek-R1-Distill-Qwen-1.5B, the SIRI-low variant improved performance on the AIME24 benchmark by 43.2% while reducing token usage by 46.9%. The principal implication for AI practitioners is that periodically oscillating the output length constraint during RL training is an effective technique for pushing models toward the Pareto frontier of performance and efficiency. |
|
|
|
|
| Scaling Generalist Data-Analytic Agents (Read more on arXiv or HuggingFace) |
|
The paper introduces DATAMIND, a scalable data synthesis and agent training recipe designed to construct high-performance, generalist data-analytic agents from open-source models. The primary objective is to overcome key challenges in open-source agent development, including insufficient data resources, improper training strategies, and unstable code-based multi-turn rollouts. The methodology involves creating the DATAMIND-12K dataset through a fine-grained task taxonomy and self-consistency filtered trajectory sampling, followed by training an agent using a dynamically weighted objective that combines Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) losses. This approach results in the DATAMIND-14B model achieving a state-of-the-art average score of 71.16% on multiple data analysis benchmarks, outperforming strong proprietary baselines like GPT-5 and DeepSeek-V3.1. For AI practitioners, this work provides a validated, scalable pipeline for building specialized open-source agents that can achieve superior performance, demonstrating that curated data synthesis and a hybrid SFT-RL training strategy can effectively close the performance gap with proprietary systems. |
|
|
|
|
| From Harm to Help: Turning Reasoning In-Context Demos into Assets for |
|
|
|
|
|
|
| Reasoning LMs (Read more on arXiv or HuggingFace) |
Nie Zheng, Zihang Fu, Weida Liang, Haonan Wang, tyzhu |
This paper introduces Insight-to-Solve (I2S), a test-time framework that converts detrimental few-shot CoT demonstrations into beneficial assets for Reasoning Large Models (RLMs) by decoupling insight extraction from solution generation. The central objective is to understand why high-quality in-context demonstrations often degrade RLM performance and to develop a method that effectively harnesses these demonstrations to improve reasoning. The proposed I2S method is a multi-step procedure that generates a comparison between a demonstration and a target question, extracts abstract reasoning strategies, and then applies these insights to solve the target question independently, with an optional iterative self-refinement step (I2S+). The method consistently improves performance over direct inference, boosting GPT-4.1’s accuracy on the AIME’25 benchmark by +14.0%. For AI practitioners, this research provides a concrete prompting framework to mitigate the common failure modes of few-shot CoT—”semantic misguidance” and “strategy transfer failure”—enabling more reliable use of in-context examples for complex reasoning tasks without retraining. |
|
|
|
|
| Rethinking Large Language Model Distillation: A Constrained Markov |
|
|
|
|
|
|
| Decision Process Perspective (Read more on arXiv or HuggingFace) |
|
The paper introduces a constrained reinforcement learning framework for LLM distillation that maximizes task rewards while strictly enforcing a divergence threshold from the teacher model. The main objective is to develop a principled method for reward-aware distillation that avoids ad-hoc reward weighting by maximizing task performance subject to a hard constraint on student-teacher policy divergence. The key methodology formulates distillation as a Constrained Markov Decision Process (CMDP) and uses a modified reward function within a policy gradient algorithm, which penalizes trajectories that exceed a predefined cumulative KL-divergence budget, thereby removing the need for state augmentation or teacher access during deployment. Experiments on mathematical reasoning tasks show the method achieves a high Reasoning Win Rate (60.55% for Qwen2.5-1.5B on the Apple/GSM-Symbolic dataset) and constraint satisfaction (96.1%), significantly outperforming Lagrangian relaxation baselines in reasoning quality. For AI practitioners, this provides a more stable and theoretically-grounded approach to distill smaller, reliable models by offering direct control over the trade-off between task performance and teacher fidelity, which is critical for deployment in resource-constrained settings. |
|
|
|
|
| Taming Masked Diffusion Language Models via Consistency Trajectory |
|
|
|
|
|
|
| Reinforcement Learning with Fewer Decoding Step (Read more on arXiv or HuggingFace) |
|
This paper introduces decoding and reinforcement learning techniques to enhance the performance and efficiency of Masked Diffusion Language Models (MDLMs). The research objective is to resolve sub-optimal full diffusion decoding and the training-inference inconsistency that arises when applying autoregressive-style reinforcement learning (RL) to non-causal MDLMs. The key methodology involves three components: EOS Early Rejection (EOSER) to prevent premature sequence termination, an Ascending Step-Size (ASS) scheduler to reduce decoding steps to O(log₂L), and Consistency Trajectory Group Relative Policy Optimization (CJ-GRPO) to align the non-causal rollout and optimization trajectories. On the Sudoku task (generation length 256), the proposed CJ-GRPO with EOSER achieved 85.37% accuracy, substantially outperforming the 18.85% of a baseline diffu-GRPO approach. For AI practitioners, this work provides a validated framework to effectively fine-tune and accelerate inference for MDLMs, addressing critical trajectory inconsistencies and making them more viable for complex reasoning tasks. |
|
|
|
|
| Efficient Multi-turn RL for GUI Agents via Decoupled Training and |
|
|
|
|
|
|
| Adaptive Data Curation (Read more on arXiv or HuggingFace) |
|
This paper introduces DART, a decoupled reinforcement learning framework with adaptive data curation to efficiently train vision-language model-based GUI agents. The research objective is to resolve the significant training inefficiencies in RL for GUI agents caused by slow, tightly-coupled environment interactions and insufficient high-quality training data. The methodology combines a system architecture with four asynchronous modules (environment cluster, rollout service, data manager, trainer) and a multi-level data curation scheme that includes dynamic sampling, high-entropy step prioritization, and truncated importance sampling. On the OS-World benchmark, the DART-GUI-7B model achieved a 42.13% task success rate, representing a 14.61% absolute improvement over the baseline, while the framework increased training throughput by 1.9x and environment utilization by 5.5x. For AI practitioners, this framework provides a reusable blueprint for scaling RL training for agentic systems by decoupling components to maximize resource utilization and curating data to focus learning on critical decision points. |
|
|
|
|
| Hyperspherical Latents Improve Continuous-Token Autoregressive |
|
|
|
|
|
|
| Generation (Read more on arXiv or HuggingFace) |
Hui Xue, guolinke |
SphereAR improves continuous-token autoregressive (AR) image generation by constraining latent representations to a fixed-radius hypersphere. The objective is to mitigate the variance collapse that occurs in continuous-token AR models due to heterogeneous variance in VAE latents being amplified by classifier-free guidance (CFG) during decoding. The methodology involves coupling a hyperspherical VAE (S-VAE), which encodes image patches into constant-norm latent tokens, with a causal Transformer whose inputs and outputs are persistently projected back onto the hypersphere to maintain scale-invariance. On ImageNet 256x256 class-conditional generation, the SphereAR-H (943M) model achieves a state-of-the-art FID score of 1.34 for AR models, outperforming larger baselines. The principal implication for AI practitioners is that enforcing scale-invariance in the latent space via hyperspherical constraints is a critical design choice for stabilizing AR decoding and building high-performance continuous-token generative systems. |
|
|
|
|
| AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced |
|
|
|
|
|
|
| Self-Play (Read more on arXiv or HuggingFace) |
Yue Yu, Jonathan Wang, Zihan Dong, Yuchen Zhuang, Ran Xu |
AceSearcher is a cooperative self-play framework that improves LLM reasoning and multi-hop search by training a single model to act as both a query decomposer and a context-integrating solver. The objective is to enhance the complex reasoning and multi-hop retrieval capabilities of search-augmented LLMs without relying on intermediate supervision or costly inference-time search algorithms. The key methodology is a two-stage training process: first, supervised fine-tuning (SFT) on a diverse mix of QA, decomposition, and reasoning datasets, followed by iterative, preference-based reinforcement fine-tuning (RFT) using Direct Preference Optimization (DPO), where rewards are derived solely from final answer accuracy. Across 10 reasoning-intensive datasets, AceSearcher outperformed state-of-the-art baselines, achieving an average exact match improvement of 7.6%; notably, the 32B parameter version matched the performance of the significantly larger DeepSeek-V3 model on document-level reasoning tasks while using less than 5% of its parameters. The principal implication for AI practitioners is that this self-play and two-stage fine-tuning recipe enables the development of highly capable and parameter-efficient search-augmented LLMs for complex reasoning, reducing reliance on extremely large or proprietary models and eliminating the need for expensive intermediate-step annotations. |
|
|
|
|
| Pretraining Large Language Models with NVFP4 (Read more on arXiv or HuggingFace) |
|
This paper introduces a methodology for stable and accurate large-scale language model (LLM) pretraining using the NVFP4 format. The primary objective was to demonstrate the feasibility of 4-bit floating point training for LLMs, addressing stability and convergence challenges to improve computational efficiency and resource utilization. Key methodological components include preserving numerically sensitive layers in higher precision, applying Random Hadamard Transforms to inputs of weight gradient GEMMs, employing two-dimensional block scaling for weights, and utilizing stochastic rounding for gradients. This approach successfully trained a 12-billion-parameter model on 10 trillion tokens, achieving an MMLU-pro accuracy of 62.58%, closely matching the 62.62% accuracy of an FP8 baseline. For AI practitioners, this work provides a practical path to significantly reduce LLM pretraining computational cost and memory footprint, enabling more efficient development of next-generation models. |
|
|
|
|
| SCI-Verifier: Scientific Verifier with Thinking (Read more on arXiv or HuggingFace) |
Jingqi Ye, Junchi Yao, Fangchen Yu, Chenyu Huang, desimfj |
This paper introduces SCI-Verifier, a reasoning-augmented verifier, and SCI-VerifyBench, a cross-disciplinary benchmark, for scientific verification in LLMs. The main objective is to address limitations in existing scientific verification methods by establishing a systematic evaluation framework and developing a robust, reasoning-augmented verifier. The methodology involves constructing SCI-VerifyBench through collecting LLM responses, applying domain-specific equivalence transformations, and combining model/expert annotations, and developing SCI-Verifier using a two-stage post-training pipeline: Supervised Fine-Tuning with filtered reasoning traces and Reinforcement Learning. Experiments on SCI-VerifyBench demonstrate that SCI-Verifier-8B achieves 86.28% total accuracy, outperforming existing open-source models and matching closed-source models like GPT-5 in verification performance on scientific tasks, especially for complex equivalence-based answers. This work provides AI practitioners with a precise evaluation framework and practical guidance, emphasizing the importance of integrating logical reasoning to enhance LLM capabilities and reliability in scientific domains. |
|
|
|
|
| Alignment through Meta-Weighted Online Sampling: Bridging the Gap |
|
|
|
|
|
|
| between Data Generation and Preference Optimization (Read more on arXiv or HuggingFace) |
Xin Geng, Shiqi Qiao, Biao Liu, Ning Xu, jmyang |
MetaAPO is a novel framework that bridges the gap between LLM data generation and preference optimization using a meta-weighted online sampling strategy. The primary objective is to mitigate the distribution mismatch between static offline preference data and the dynamic, evolving policy by adaptively coupling online data generation and model training. MetaAPO utilizes a lightweight two-layer MLP meta-learner to estimate the “alignment gap,” guiding targeted online response generation and assigning dynamic sample-wise meta-weights to a hybrid loss function that balances offline and online data contributions. Experiments show MetaAPO consistently outperforms baselines, for example, reducing online annotation requirements by 42% and achieving a 47.48% win rate on AlpacaEval 2 for Llama-3.1-8B, compared to Online DPO’s 43.75%. This approach offers AI practitioners a more efficient and robust method for LLM alignment, enabling superior performance while significantly lowering the resource and time costs associated with online data acquisition and model training. |
|
|
|
|
| WirelessMathLM: Teaching Mathematical Reasoning for LLMs in Wireless |
|
|
|
|
|
|
| Communications with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Li Wei, Wenhe Zhang, Yiyang Zhu, Mengbing Liu, XINLI1997 |
WirelessMathLM teaches compact LLMs mathematical reasoning for wireless communications using verification-based reinforcement learning. The research objective was to enable LLMs to achieve expert-level performance in specialized wireless mathematics, where current state-of-the-art models struggle. The methodology involved training models (0.5B-7B parameters) with Group Relative Policy Optimization (GRPO) and binary verification rewards on WirelessMathBench-XL, a benchmark of 4,027 problems from 970 papers. The 7B WirelessMathLM achieved 39.5% accuracy on WirelessMathBench-XL, approaching GPT-40 (40.4%) while using approximately 100x fewer parameters than DeepSeek-R1 (671B, 57.4%), and GRPO training dramatically improved performance, for example, doubling the 3B model’s accuracy (+103%). This demonstrates that verifiable correctness in technical domains enables efficient and scalable domain specialization for compact LLMs without extensive supervised data or human feedback, with implications for other formally verifiable technical fields. |
|
|
|
|
| AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety |
|
|
|
|
|
|
| Alignment of Large Reasoning Models (Read more on arXiv or HuggingFace) |
|
AdvChain introduces an adversarial Chain-of-Thought (CoT) tuning framework for robust safety alignment of Large Reasoning Models (LRMs) by teaching dynamic self-correction. The primary objective is to mitigate the “snowball effect,” where minor reasoning deviations escalate into harmful compliance or excessive refusal. The key methodology involves constructing an adversarial safety reasoning dataset with Temptation-Correction and Hesitation-Correction samples, then fine-tuning LRMs to enable self-recovery from flawed reasoning. Results show AdvChain significantly enhances robustness; for instance, DeepSeek-R1-7B AdvChain achieved a 4.50% HarmBench Attack Success Rate (ASR), substantially lower than STAR-1’s 8.00% and SafeChain’s 38.00%, without compromising reasoning capabilities. This work implies that AI practitioners can develop more resilient and practical LRMs by integrating adversarial CoT tuning to instill adaptive error-correction mechanisms. |
|
|
|
|
| UniVid: The Open-Source Unified Video Model (Read more on arXiv or HuggingFace) |
Meng Fang, Biao Wu, Junhui Lin, Jiabin Luo, SteveZeyuZhang |
UniVid is an open-source unified video model designed for both understanding and generation tasks. Its main objective is to overcome challenges in maintaining semantic faithfulness during flow-based generation and efficiently extending image-centric MLLMs to video without costly retraining. The model employs a unified architecture coupling a multimodal LLM with a diffusion video decoder via a lightweight adapter, introducing Temperature Modality Alignment for prompt adherence and Pyramid Reflection for efficient temporal reasoning. UniVid achieves state-of-the-art performance, demonstrating a 2.2% improvement on VBench-Long total score and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA, respectively, compared to prior 7B baselines. This unified paradigm provides AI practitioners with a robust and efficient framework for developing integrated video intelligence systems. |
|
|
|
|
| PARROT: A Benchmark for Evaluating LLMs in Cross-System SQL Translation (Read more on arXiv or HuggingFace) |
|
PARROT introduces the first benchmark for evaluating Large Language Models (LLMs) in Cross-System SQL Translation, addressing limitations of existing Text-to-SQL benchmarks. Its primary objective is to evaluate LLMs on adapting SQL queries between diverse database systems like MySQL and ClickHouse, a critical yet underexplored area. The methodology includes curating 598 manually verified query pairs and 5,306 unit-style test cases from 22 production-grade systems, alongside an augmented training pool of 28,003 SQL statements, processed through a comprehensive SQL curation workflow. Initial evaluations show LLMs achieve lower than 38.53% average accuracy, with GPT-4o scoring 58.62% ACCEX (dialect compatibility) and 54.23% ACCRES (result consistency), indicating significant challenges in handling system-specific SQL dialects. This implies AI practitioners must focus on enhancing LLMs’ dialect-specific error handling, complex syntax adaptation, and semantic consistency for real-world heterogeneous database environments. |
|
|
|
|
| MathBode: Frequency-Domain Fingerprints of LLM Mathematical Reasoning (Read more on arXiv or HuggingFace) |
|
MathBode is a dynamic diagnostic framework using frequency-domain analysis to characterize LLM mathematical reasoning dynamics. The main objective is to move beyond static final-answer accuracy by evaluating amplitude fidelity and timing consistency through frequency-resolved metrics. This is achieved by sinusoidally driving one problem parameter and fitting first-harmonic responses from LLM outputs to derive gain and phase error fingerprints. For Exponential Interest, DeepSeek V3.1 demonstrated high amplitude fidelity with a mean |
G-1 |
of 0.051 at mid-frequencies, while Mixtral 8x7B showed significant distortion with a mean |
G-1 |
of 8.418. This suggests AI practitioners should use frequency-domain diagnostics for LLMs in dynamic or iterative systems, as final-answer accuracy alone is insufficient to predict stability or consistent performance. |
| Local Success Does Not Compose: Benchmarking Large Language Models for |
|
|
|
|
|
|
| Compositional Formal Verification (Read more on arXiv or HuggingFace) |
Binhang Yuan, Jie Fu, Xingwei Qu, Xu Xu, XINLI1997 |
This paper introduces DAFNYCOMP, a benchmark revealing a critical compositional reasoning gap in LLMs for formal verification, showing catastrophic verification failure despite high syntactic accuracy. The main objective is to systematically evaluate LLMs on generating compositional specifications in Dafny for multi-function programs, addressing their lack of compositional reasoning across function boundaries for reliable and verifiable code generation. The methodology involves synthesizing 300 compositional Dafny programs (2-5 chain-based functions with data dependencies) from Python code and requiring LLMs to regenerate missing contract clauses to enable mechanical verification. Results show that LLMs, achieving >58% verification on single-function benchmarks, exhibit a catastrophic 3.69% verification rate on DAFNYCOMP’s compositional tasks, a 92% performance gap despite 95.67% syntax correctness. This implies that current LLMs lack robust compositional reasoning for verifiable code generation, necessitating advancements beyond local pattern matching to achieve global contract consistency and inductive reasoning in multi-component systems. |
|
|
|
|
| ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents (Read more on arXiv or HuggingFace) |
|
ChatInject is a novel prompt injection attack that exploits LLM chat templates and multi-turn dialogues to manipulate agent behavior. This research investigates the vulnerability of LLM agents to indirect prompt injection by leveraging their dependence on structured chat templates and susceptibility to contextual manipulation. The methodology involves formatting malicious payloads to mimic native chat templates (ChatInject) and developing a persuasion-driven Multi-turn variant that primes the agent across conversational turns. Experiments show ChatInject significantly increases Attack Success Rates (ASR); for instance, on InjecAgent, ASR improved from 15.13% (default) to 45.90% (ChatInject), with Multi-turn variants reaching 52.33%, demonstrating strong transferability and defense bypass. For AI practitioners, these results highlight critical vulnerabilities in current LLM agent systems, underscoring the inadequacy of existing defenses and the need for more sophisticated security measures against template-based and contextually-primed attacks. |
|
|
|
|
| UniMIC: Token-Based Multimodal Interactive Coding for Human-AI |
|
|
|
|
|
|
| Collaboration (Read more on arXiv or HuggingFace) |
|
UniMIC is a unified token-based multimodal interactive coding framework designed for efficient human-AI collaboration. Its objective is to establish an AI-native communication protocol using compact tokenized representations to avoid repeated degradation and latency inherent in pixel-based pipelines. The framework employs modality-specific tokenizers and lightweight Transformer-based entropy models (autoregressive, masked-token, text-conditional) to compress tokens, reducing inter-token redundancy for arithmetic coding. For text-to-image generation, UniMIC achieves 0.0296 bpp with an FID of 80.61 and CLIP-T of 0.315, demonstrating substantial bitrate savings and superior performance compared to baselines like VVC (0.0337 bpp, FID 180.19, CLIP-T 0.286). This establishes a practical paradigm for AI practitioners to enable ultra-low bitrate multimodal communication while preserving semantic fidelity and downstream task performance with Large Multimodal Models. |
|
|
|
|
| Cogito, Ergo Ludo: An Agent that Learns to Play by Reasoning and |
|
|
|
|
|
|
| Planning (Read more on arXiv or HuggingFace) |
|
Cogito, ergo ludo (CEL) is a novel LLM-based agent that learns to master interactive grid-world environments by explicitly reasoning and planning through a continuous interaction-reflection cycle. The agent’s primary objective is to build a transparent and improving model of its environment’s mechanics and its own strategy from raw interaction, starting from a tabula rasa state. CEL leverages a single Large Language Model (LLM) to function as a Language-based World Model (LWM) for action prediction and a Language-based Value Function (LVF) for state evaluation during episodes, complemented by a post-episode reflection phase that performs Rule Induction to refine environmental dynamics and Strategy and Playbook Summarization for strategic advice. In evaluations across Minesweeper, Frozen Lake, and Sokoban, CEL autonomously discovered rules and developed effective policies, achieving a 54% success rate in Minesweeper, notably surpassing a baseline with ground-truth rules (26%), and a 97% success rate in Frozen Lake within 10 episodes; ablation studies confirmed the criticality of iterative rule induction. This work demonstrates a significant advancement towards more general, interpretable, and auditable AI agents, by enabling explicit, language-based knowledge representation and continuous self-improvement, which can enhance trust and facilitate debugging in complex AI applications. |
|
|
|
|
| Learning Goal-Oriented Language-Guided Navigation with Self-Improving |
|
|
|
|
|
|
| Demonstrations at Scale (Read more on arXiv or HuggingFace) |
|
The paper introduces Self-Improving Demonstrations (SID) for goal-oriented language-guided navigation. The primary objective is to address the absence of effective exploration priors in existing methods that rely predominantly on shortest-path trajectories for agent training. SID utilizes an iterative self-improving pipeline where an initial agent generates successful exploration trajectories, which then serve as self-demonstrations to train a more capable agent; this process scales to new environments and integrates VLM-generated captions for language-guided tasks. SID achieves state-of-the-art performance on goal-oriented VLN tasks, notably reaching a 50.9% Success Rate (SR) on SOON’s unseen validation splits, exceeding prior leading approaches by 13.9%. This self-improving paradigm offers AI practitioners a scalable solution for developing robust navigation agents, significantly reducing dependence on expensive human annotations for exploration data. |
|
|
|
|
| REMA: A Unified Reasoning Manifold Framework for Interpreting Large |
|
|
|
|
|
|
| Language Model (Read more on arXiv or HuggingFace) |
Shuo Zhang, Junrong Yue, Ronghao Chen, Guanzhi Deng, liboaccn |
REMA is a novel interpretability framework for analyzing Large Language Model (LLM) reasoning failures via geometric analysis of internal representations. The research objective is to understand how LLMs perform complex reasoning and identify the origins of their failure mechanisms by defining a measurable geometric analysis perspective. The key methodology involves defining the “Reasoning Manifold” as the latent low-dimensional geometric structure of correctly reasoned internal representations, then quantifying reasoning failures as geometric deviations from this manifold using k-nearest neighbor distances and localizing divergence points by layer-wise deviation tracking. Primary results indicate that reasoning states exhibit low-dimensional structures, and error representations consistently show statistically significant geometric deviation from correct reasoning manifolds; for example, a Spearman’s rank correlation of p = 0.598 (p < 0.01) was found between Accuracy and Relative Deviation across all model-task pairs. The principal implication for AI practitioners is a unified, model-agnostic tool to quantitatively diagnose where and how severely an LLM’s internal states diverge when it makes an error, facilitating targeted debugging and a deeper understanding of black-box model computational processes. |
|
|
|
|
| BOE-XSUM: Extreme Summarization in Clear Language of Spanish Legal |
|
|
|
|
|
|
| Decrees and Notifications (Read more on arXiv or HuggingFace) |
|
This paper introduces BOE-XSUM, a new dataset for extreme summarization of Spanish legal decrees into clear language, and evaluates generative language models on this task. The primary objective was to determine the extent to which LLMs can produce expert-comparable, concise summaries of complex legal documents. The methodology involved fine-tuning medium-sized LLMs (BERTIN GPT-J 6B, BOLETIN) on the BOE-XSUM dataset and comparing their performance against larger general-purpose models in a zero-shot setting, using metrics like BLEU, ROUGE, METEOR, and BERTScore. Fine-tuned BERTIN GPT-J 6B (32-bit precision) achieved a BERTScore of 41.6%, demonstrating a 24% performance gain over the top zero-shot model, DeepSeek-R1 (33.5% BERTScore). This indicates that AI practitioners can significantly improve extreme summarization of domain-specific legal texts by targeted fine-tuning of smaller LLMs, even outperforming larger zero-shot models. |
|
|
|
|
| IWR-Bench: Can LVLMs reconstruct interactive webpage from a user |
|
|
|
|
|
|
| interaction video? (Read more on arXiv or HuggingFace) |
Yunwen Li, Yufan Shen, Minghao Liu, Yang Chen, tricktreat |
IWR-Bench is a novel benchmark for evaluating Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from user interaction videos. The main objective is to determine if LVLMs can reconstruct the dynamic and interactive functionalities of a webpage by observing a user interaction video. The key methodology involves 113 curated tasks from real-world websites, providing user interaction videos and static assets, and employing an agent-as-a-judge framework to evaluate functional correctness and visual fidelity. Experimental results on 28 LVLMs indicate that the best model achieved an overall score of 36.35%, with functional correctness (24.39% IFS) significantly trailing visual fidelity (64.25% VFS). This implies that current LVLMs have critical limitations in reasoning about temporal dynamics and synthesizing event-driven logic for truly functional web applications. |
|
|
|
|
| BPMN Assistant: An LLM-Based Approach to Business Process Modeling (Read more on arXiv or HuggingFace) |
Darko Etinger, Nikola Tankovic, jtlicardo |
This paper presents BPMN Assistant, an LLM-based tool that uses a specialized JSON representation to generate and edit Business Process Model and Notation (BPMN) diagrams from natural language. The research objective is to determine if this structured JSON intermediate representation is more effective than prompting an LLM to directly manipulate standard BPMN XML. The methodology involves evaluating multiple LLMs on generation and editing tasks, using Graph Edit Distance (GED) to measure generation similarity and a binary success metric to assess editing accuracy. Primary results show that while generation performance was comparable (0.70 similarity for JSON vs. 0.69 for XML), the JSON approach was significantly more reliable for editing tasks, achieving consistently higher success rates and over 2x faster processing (21.46s average latency vs. 46.98s for XML). For AI practitioners, the principal implication is that employing a simplified, structured intermediate representation as an LLM target can dramatically improve the reliability and performance of complex data modification tasks compared to directly manipulating verbose standard formats. |
|
|
|
|
| Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large |
|
|
|
|
|
|
| Language Models (Read more on arXiv or HuggingFace) |
|
This paper introduces Corpus-Level Inconsistency Detection (CLID) and the agentic LLM-based system CLAIRE for identifying contradictions in Wikipedia. The objective is to formalize CLID, identifying if a fact within a corpus contradicts any other fact in the same corpus, thereby ensuring Wikipedia’s accuracy. CLAIRE is an agentic system leveraging LLM reasoning with retrieval, based on the ReAct architecture, incorporating “clarify” and “explain” auxiliary tools, and validated through human-in-the-loop annotation to create WIKICOLLIDE. A user study showed participants identified 64.7% more inconsistencies using CLAIRE; analysis revealed at least 3.3% of English Wikipedia facts contradict other statements, and CLAIRE achieved an AUROC of 75.1% on the WIKICOLLIDE test set. CLID systems, particularly LLM-based agentic approaches like CLAIRE, offer a practical tool for AI practitioners to improve the consistency and reliability of large-scale knowledge corpora, directly benefiting LLM training and RAG systems. |
|
|
|
|
| RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human |
|
|
|
|
|
|
| Mobility (Read more on arXiv or HuggingFace) |
|
RHYTHM introduces a computationally efficient framework for human mobility prediction using hierarchical temporal tokenization and frozen Large Language Models (LLMs). The primary objective is to accurately predict human mobility by effectively capturing complex long-range dependencies and multi-scale periodic behaviors inherent in human trajectories. RHYTHM’s methodology involves partitioning trajectories into daily segments, encoding them as discrete tokens with hierarchical attention for daily and weekly patterns, and enriching representations with pre-computed prompt embeddings from a frozen LLM. This parameter-efficient adaptation strategy significantly reduces computational overhead, leading to a 2.4% improvement in overall Accuracy@1 and a 24.6% reduction in training time compared to state-of-the-art baselines. This framework provides AI practitioners with a scalable and accessible solution for accurate trajectory prediction, particularly valuable for handling irregular mobility patterns in resource-constrained, real-world environments. |
|
|
|
|
| Charting a Decade of Computational Linguistics in Italy: The CLiC-it |
|
|
|
|
|
|
| Corpus (Read more on arXiv or HuggingFace) |
Chiara Alzetta, martasartor, alemiaschi, chiaracf, lucadini |
This paper charts a decade of Italian computational linguistics research by analyzing the CLiC-it conference proceedings. The objective was to analyze research trends, collaboration patterns, and thematic evolution within the Italian CL/NLP community from 2014 to 2024. The methodology involved compiling the CLiC-it Corpus from 693 papers using semi-automatic parsing, metadata analysis, network analysis with centrality measures, and BERTopic for topic modeling, including EasyNMT for Italian-to-English translation. Key findings include 2,006 unique authors contributed over ten years, with the 2024 edition having a record 346 authors, and “Lexical and Semantic Resources and Analysis” being the most represented topic (189 papers). This corpus and analysis provide AI practitioners with empirical insights into the evolution of NLP research priorities and collaborative structures, aiding in informed decisions for future AI development, especially regarding neural and conversational technologies. |
|
|
|
|
| Advancing Reference-free Evaluation of Video Captions with Factual |
|
|
|
|
|
|
| Analysis (Read more on arXiv or HuggingFace) |
Subarna Tripathi, Tz-Ying Wu, dipta007 |
VC-Inspector is a novel reference-free and factually grounded multimodal evaluation framework for video captions, leveraging LLM-generated synthetic data for instruction tuning. The main objective is to develop a reference-free evaluation framework for video captions that relies on factual grounding to accurately assess caption quality without requiring human-annotated ground truth captions. The methodology involves creating a synthetic video caption dataset by using Llama-3.3-70B-Instruct to systematically alter objects and actions in ground truth captions, assigning quality scores based on factual changes, which then instruction-tunes a Qwen2.5-VL model to act as VC-Inspector. On the VATEX-EVAL dataset in a reference-free setting, VC-Inspector-7B achieved a Kendall’s correlation (Tb) of 42.58 and Spearman’s rank correlation (ρ) of 45.99, outperforming the ViCLIPScore baseline (Tb 30.92, ρ 39.86). This work provides AI practitioners with a scalable, generalizable, and interpretable tool for evaluating video caption factual accuracy without costly human annotations, enabling objective assessment and potential use as a reward model in Reinforcement Learning applications. |
|
|
|
|
Papers for 2025-09-29
| Title |
Authors |
Summary |
| LongLive: Real-time Interactive Long Video Generation (Read more on arXiv or HuggingFace) |
|
LONGLIVE is a frame-level autoregressive framework enabling real-time, interactive generation of long videos with high temporal consistency and prompt adherence. The main objective is to overcome the quality and efficiency challenges of long video generation, specifically enabling real-time interaction and smooth transitions between sequential user prompts while maintaining visual and semantic coherence over extended durations. The methodology combines three key components: a KV-recache mechanism to refresh cached states during prompt switches, a streaming long tuning strategy to align training with long-video inference and mitigate quality degradation, and a short window attention paired with a frame-level attention sink for efficient inference. Primary results demonstrate that LONGLIVE achieves a generation speed of 20.7 FPS on a single NVIDIA H100 GPU, supports videos up to 240 seconds, and achieves a state-of-the-art score of 83.52 on the VBench-Long benchmark, outperforming existing autoregressive and diffusion-based models in both speed and quality. The principal implication for AI practitioners is that this framework provides a viable architecture for building high-performance, interactive video generation tools by showing that autoregressive models, when augmented with specialized cache management and training strategies, can achieve real-time speeds without the computational overhead of diffusion models, making them suitable for dynamic content creation applications. |
| Quantile Advantage Estimation for Entropy-Safe Reasoning (Read more on arXiv or HuggingFace) |
An Zhang, Jiancan Wu, xiangwang1223, 737443h, junkang0909 |
Quantile Advantage Estimation (QAE) is a drop-in replacement for the mean baseline in value-free RL that stabilizes LLM reasoning training by creating a two-regime gate for credit assignment. The paper’s objective is to resolve the training instability in Reinforcement Learning with Verifiable Rewards (RLVR), which oscillates between entropy collapse and explosion, by redesigning the advantage estimation baseline. The key methodology replaces the standard mean reward baseline with a group-wise K-quantile baseline, which for binary rewards selectively assigns non-zero advantage to either rare successes on hard queries (where success rate p ≤ 1-K) or residual failures on easy queries (p > 1-K). Primary results demonstrate that QAE provides provable two-sided entropy safety, sparsifies updates by assigning zero advantage to approximately 80% of responses, and improved pass@1 performance on AIME’24 for Qwen3-8B-Base from 39.69% to 48.23% while maintaining comparable pass@16 scores. The principal implication for AI practitioners is that they can significantly stabilize RLVR fine-tuning and improve sample efficiency with a simple, one-line change to the baseline calculation, targeting the core mechanism of credit assignment rather than relying on more complex token-level heuristics. |
| MinerU2.5: A Decoupled Vision-Language Model for Efficient |
|
|
| High-Resolution Document Parsing (Read more on arXiv or HuggingFace) |
SunYuefeng, hotelll, ouyanglinke, wanderkid, starriver030515 |
MinerU2.5 is a 1.2B-parameter vision-language model that performs efficient, high-resolution document parsing using a decoupled, coarse-to-fine strategy. The primary objective is to achieve state-of-the-art parsing accuracy for text, tables, and formulas while maintaining high computational efficiency. The methodology involves a two-stage process: first, the model performs rapid layout analysis on a downsampled image, and second, it conducts targeted content recognition on native-resolution crops extracted from the original image based on the detected layout. The model achieves a state-of-the-art overall score of 90.67 on the OmniDocBench benchmark and an inference speed of 2.12 pages/second on an A100 GPU. For AI practitioners, this decoupled architecture provides a computationally efficient design pattern for processing high-resolution documents, enabling the creation of high-quality structured data for applications like Retrieval-Augmented Generation (RAG). |
| EPO: Entropy-regularized Policy Optimization for LLM Agents |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Li Yu-Jhe, Wentian Zhao, timecuriosity, ztwang, Iscarrot |
The paper introduces Entropy-regularized Policy Optimization (EPO) to address the “exploration-exploitation cascade failure” in training LLM agents on multi-turn, sparse-reward tasks. The objective is to stabilize reinforcement learning by preventing early premature convergence and subsequent late-stage policy collapse. The key methodology involves three components: trajectory-aware entropy regularization, an entropy smoothing regularizer that bounds policy entropy within a moving historical average, and an adaptive weighting schedule to balance exploration and exploitation. EPO achieves up to a 152.1% performance improvement on the ScienceWorld benchmark over a PPO baseline and up to 19.8% on ALFWorld over a GRPO baseline. For AI practitioners, the principal implication is that in long-horizon, sparse-reward LLM agent training, simply adding an entropy bonus is insufficient; they should instead use temporal control mechanisms like EPO’s historical smoothing to maintain stable exploration and avoid the identified cascade failure. |
| Variational Reasoning for Language Models (Read more on arXiv or HuggingFace) |
|
This paper introduces a variational reasoning framework that improves language model reasoning by treating thinking traces as latent variables optimized via variational inference. The objective is to develop a principled probabilistic training method that maximizes the log-likelihood of generating correct answers, addressing the instability and data cost of existing RL and SFT approaches. The methodology involves optimizing an IWAE-style multi-trace evidence lower bound (ELBO) and training a variational posterior, conditioned on answer hints, using a forward-KL divergence to generate high-quality thinking traces for weighted finetuning. On the Qwen3-4B-Base model, the proposed accuracy-based method achieves a 55.72% average score across five reasoning benchmarks, surpassing the strong Bespoke-Stratos baseline’s 51.35%. AI practitioners can implement this framework as a more stable and effective alternative to standard RL finetuning for enhancing the reasoning capabilities of LLMs on complex tasks, as it provides a principled objective and clarifies biases in existing methods like RFT and GRPO. |
| Language Models Can Learn from Verbal Feedback Without Scalar Rewards (Read more on arXiv or HuggingFace) |
|
This paper proposes a method for large language models to learn directly from verbal feedback without converting it to scalar rewards. The research objective is to address the information loss, ambiguity, and scale imbalance associated with scalarization in reinforcement learning from human feedback (RLHF). The key methodology is the Feedback-Conditional Policy (FCP), which treats verbal feedback as a conditioning signal and is trained via maximum likelihood on response-feedback pairs, followed by an online bootstrapping phase for refinement. The primary result shows that FCP with online bootstrapping achieves a 38.7% average accuracy on a math benchmark suite, slightly surpassing strong scalar-based baselines like GRPO (38.4%). For AI practitioners, this provides a scalable framework to train models using raw verbal feedback, eliminating the need for designing reward functions, scalar conversion, or data filtering, thereby simplifying the model alignment pipeline. |
| ReviewScore: Misinformed Peer Review Detection with Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
|
This paper introduces REVIEWSCORE, a framework that uses Large Language Models to automatically detect misinformed peer review points, defined as questions already answered in a paper or weaknesses based on incorrect premises. The main objective is to develop and validate an automated method for identifying low-quality peer reviews to improve the integrity of the academic review process in large AI conferences. The key methodology involves an automated argument reconstruction engine that uses a SAT solver and LLM feedback loops to decompose argumentative weaknesses into a set of explicit and implicit premises for granular factuality checking against a human-annotated dataset. The primary result shows that 15.2% of weaknesses and 26.4% of questions in their dataset are misinformed, with the best-performing LLM (claude-sonnet-3.7) achieving a moderate human-model agreement F1 score of 0.448 on the REVIEWSCORE detection task. The principal implication for AI practitioners is the potential for integrating this automated system into conference management platforms to flag low-quality reviews, providing direct feedback to reviewers and assisting meta-reviewers in their decisions. |
| CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
|
This paper introduces CapRL, a reinforcement learning framework that trains dense image captioning models by using the ability of a vision-free LLM to answer questions from the caption as an objective reward. The objective is to apply the Reinforcement Learning with Verifiable Rewards (RLVR) paradigm to the subjective task of image captioning, overcoming the scalability and memorization issues of Supervised Fine-Tuning (SFT). CapRL’s methodology employs a decoupled two-stage pipeline where an LVLM generates a caption, and the reward is the accuracy of a separate, vision-free LLM answering multiple-choice questions about the image based solely on that caption. The primary result shows that within the Prism evaluation framework, the CapRL-3B model achieves an average score of 48.3, matching the performance of the much larger Qwen2.5-VL-72B model and outperforming its baseline by 8.4%. For AI practitioners, this provides a scalable method to generate high-quality, dense image-text data for pre-training LVLMs, enhancing modality alignment without requiring expensive, manually annotated datasets. |
| MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Weipeng Zhong, Xudong Xu, Zhen Luo, nfliang, wuzhi-hao |
This paper introduces MesaTask, a framework and dataset for generating task-driven 3D tabletop scenes from natural language instructions using a large language model with 3D spatial reasoning. The research objective is to automate the creation of plausible and task-relevant 3D scenes for robotic training, bridging the gap between high-level instructions and specific scene layouts. The methodology centers on a “Spatial Reasoning Chain” that decomposes generation into object inference, spatial interrelation reasoning, and scene graph construction, used to train an LLM via Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) on the new MesaTask-10K dataset of ~10,700 scenes. MesaTask significantly outperforms baselines, achieving a Fréchet Inception Distance (FID) of 40.3, indicating superior realism compared to a GPT-4o baseline score of 74.4. For AI practitioners, this provides a validated framework and a large-scale dataset to automate the generation of diverse and realistic 3D simulation environments, accelerating the development of robotic policies that can interpret and execute complex, language-based commands. |
| No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM |
|
|
| Reinforcement Learning via Entropy-Guided Advantage Shaping (Read more on arXiv or HuggingFace) |
|
The paper introduces RL-ZVP, a novel algorithm for LLM reinforcement learning that extracts useful training signals from zero-variance prompts by using an entropy-guided advantage shaping mechanism. The main objective is to utilize zero-variance prompts—where all model responses share the same reward and are typically discarded by methods like GRPO—to improve the reasoning capabilities and training efficiency of LLMs. The key methodology involves a custom advantage formulation for zero-variance prompts: for all-correct responses, it assigns a positive advantage proportional to token-level entropy, and for all-incorrect responses, it assigns a negative advantage that penalizes low-entropy tokens more severely, while reverting to GRPO for all other prompts. Primary results demonstrate that RL-ZVP significantly outperforms GRPO across six math benchmarks, achieving up to an 8.61 point gain in accuracy (Acc@8) on the AIME25 benchmark and consistently outperforming baselines that filter these prompts. The principal implication for AI practitioners is that they can enhance the data efficiency and final performance of RL fine-tuning for reasoning tasks by implementing the RL-ZVP objective, which salvages previously discarded rollouts to provide a stronger and more stable learning signal. |
| VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, |
|
|
| Speaking, and Viewing (Read more on arXiv or HuggingFace) |
|
This paper introduces VoiceAssistant-Eval, a comprehensive benchmark with 10,497 examples designed to assess AI assistants across integrated listening, speaking, and viewing capabilities. The objective is to address gaps in existing evaluations by creating a framework that tests hands-free interaction, voice personalization, and joint audio-visual understanding. The methodology evaluates 22 models on 13 tasks using a triadic system measuring content quality (via a GPT judge), speech naturalness (UTMOS), and text-speech consistency (modified WER). Key findings reveal a significant disparity between speaking and listening performance, with the 7B Step-Audio-2-mini model’s listening accuracy (40.06) more than doubling that of the 32B LLaMA-Omni2 model (16.00). The principal implication for AI practitioners is that progress requires dedicated improvements to audio encoders and multimodal architectures, as simply scaling the LLM component is insufficient for robust performance, evidenced by a 16.3-point accuracy drop on image+audio versus image+text queries. |
| UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon |
|
|
| Scenarios (Read more on arXiv or HuggingFace) |
Zeyu Qin, Haoyu Wang, Xuelin Zhang, Huaisong Zhang, Haotian Luo |
This research introduces UltraHorizon, a novel benchmark for evaluating LLM-agent capabilities in ultra-long-horizon, partially observable scenarios where existing benchmarks fall short. Its objective is to systematically measure foundational agent competencies such as sustained reasoning, planning, memory management, and tool use by requiring agents to uncover hidden rules through extended interaction. The methodology utilizes three distinct discovery-oriented environments where trajectories average over 35k tokens and 60 tool calls, with performance evaluated against human baselines. Key results demonstrate a significant performance deficit, with the best LLM agent scoring 14.33 compared to the human baseline of 26.52, and reveal that agent failures stem from “in-context locking” and foundational capability gaps rather than task-intrinsic reasoning difficulty. For AI practitioners, this implies that progress in long-horizon tasks requires developing agent architectures with principled memory integration and robust exploration strategies, as current models lack the inherent capability to utilize extended interaction budgets effectively and simple scaling is insufficient. |
| LucidFlux: Caption-Free Universal Image Restoration via a Large-Scale |
|
|
| Diffusion Transformer (Read more on arXiv or HuggingFace) |
|
LucidFlux is a universal image restoration framework that adapts a large-scale diffusion transformer (Flux.1) to restore high-quality images from degraded inputs without requiring text captions. The main objective is to develop a robust method for restoring images with unknown degradations by effectively conditioning a large generative model while preserving semantic consistency and avoiding the latency and instability of text-based prompts. The key methodology involves a lightweight dual-branch conditioner that processes the degraded input and a lightly restored proxy, a timestep- and layer-adaptive modulation schedule to guide the frozen transformer backbone, and caption-free semantic alignment using SigLIP features extracted from the proxy. LucidFlux achieves state-of-the-art results on multiple benchmarks, attaining a CLIP-IQA+ score of 0.7406 on the RealLQ250 dataset, outperforming prior open-source and commercial methods. The principal implication for AI practitioners is that adapting large diffusion transformers for specialized tasks like image restoration can be more effectively achieved through structured, minimal-overhead conditioning and direct semantic guidance, rather than by adding extensive parameters or relying on external captioning models, offering a practical blueprint for efficient foundation model adaptation. |
| WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level |
|
|
| Feedback and Step-Level Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zhuofan Zong, Yunqiao Yang, Houxing Ren, Zimu Lu, scikkk |
The paper introduces WebGen-Agent, an iterative system for website generation that uses multi-level feedback from a Visual Language Model and a GUI-agent, and a reinforcement learning method, Step-GRPO, to train the agent’s reasoning engine. The primary objective is to improve automated website generation by creating an agent that iteratively refines codebases using comprehensive visual and functional feedback, rather than relying solely on code execution verification. The core methodology involves an iterative workflow where a VLM provides scores and suggestions based on website screenshots, and a GUI-agent tests functionality, also providing scores; these step-level scores are then used as a dense reward signal in a step-wise Generalized Reward Policy Optimization (Step-GRPO) process to fine-tune the agent’s LLM. The WebGen-Agent workflow increased the accuracy of Claude-3.5-Sonnet on the WebGen-Bench dataset from 26.4% to 51.9%, and the Step-GRPO training method improved the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4%. For AI practitioners, the principal implication is that combining VLM-based visual analysis with GUI-agent functional testing creates a powerful feedback loop that provides dense, reliable reward signals, enabling effective reinforcement learning for complex, visually-dependent code generation tasks and the training of smaller open-source models. |
| SPARK: Synergistic Policy And Reward Co-Evolving Framework (Read more on arXiv or HuggingFace) |
|
SPARK is a framework that synergistically co-evolves a large model’s policy and reward capabilities by recycling rollouts from verifiable reward-based reinforcement learning. The research objective is to develop an efficient, on-policy RL framework that unifies policy optimization and reward modeling within a single model, eliminating the high costs and potential mismatches associated with separate reward models and human preference data. The key methodology extends Reinforcement Learning with Verifiable Rewards (RLVR) by recycling generated rollouts and their correctness scores to create on-policy data for auxiliary training objectives—pointwise, pairwise, and reflection—which trains the policy model to simultaneously function as its own generative reward model. Primary results show that SPARK-VL-7B achieves an average gain of 9.7% on 7 reasoning benchmarks and 12.1% on 2 reward benchmarks over baselines, demonstrating significant performance improvements. The principal implication for AI practitioners is a more resource-efficient and stable method for model alignment that reduces MLOps complexity by removing the need for a separate reward model training pipeline and human data annotation, while enabling test-time performance scaling through integrated self-reflection. |
| Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in |
|
|
| Subject-Driven Generation (Read more on arXiv or HuggingFace) |
Peter Wonka, Bernard Ghanem, Aleksandar Cvejic, abdo-eldesokey |
Mind-the-Glitch introduces a framework for disentangling visual and semantic features from diffusion models to create a new metric, Visual Semantic Matching (VSM), for quantifying and localizing inconsistencies in subject-driven image generation. The main objective is to create a reliable method for evaluating visual consistency that overcomes the limitations of existing global, feature-based metrics by enabling spatial localization of errors. The methodology involves an automated pipeline that generates image pairs with controlled visual inconsistencies for training a dual-branch contrastive architecture, which separates visual and semantic features from a frozen diffusion model backbone. The proposed VSM metric achieved a Pearson correlation of 0.448 with a ground-truth oracle in a controlled evaluation, significantly outperforming CLIP (-0.053), DINO (0.087), and a VLM-based approach (0.072). For AI practitioners, this provides a superior evaluation tool that not only quantifies visual fidelity more accurately than existing metrics but also localizes specific regions of inconsistency, offering actionable insights for model debugging and improvement. |
| See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned |
|
|
| Aerial Navigation (Read more on arXiv or HuggingFace) |
Chih-Hai Su, Yang-Sen Lin, Chih Yao Hu, jayinnn, yuna0x0 |
See, Point, Fly (SPF) is a training-free framework that enables universal UAV navigation by repurposing frozen Vision-Language Models (VLMs) to perform 2D spatial grounding for action prediction. The research objective is to develop a zero-shot UAV navigation system that interprets free-form language by framing action prediction not as text generation, but as a 2D spatial grounding task. The key methodology involves prompting a VLM to output a structured JSON containing a 2D waypoint on the current camera image, which is then geometrically unprojected into a 3D displacement vector and executed as low-level UAV control commands in a closed-loop. SPF achieved a 93.9% success rate in the DRL simulation benchmark, outperforming the previous state-of-the-art method by an absolute margin of 63%. For AI practitioners, this work implies that leveraging a VLM’s inherent spatial understanding for direct 2D visual grounding offers a more effective and generalizable zero-shot pathway for continuous robot control than methods relying on text-based action generation or predefined skill libraries. |
| Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on |
|
|
| Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval (Read more on arXiv or HuggingFace) |
|
The paper introduces Think-on-Graph 3.0 (ToG-3), a multi-agent framework for adaptive LLM reasoning on heterogeneous graphs in Retrieval-Augmented Generation (RAG). The primary objective is to overcome the limitations of static graph indices in existing Graph-RAG methods by enabling dynamic, query-adaptive graph construction and refinement, particularly for lightweight LLMs. The key methodology is the Multi-Agent Context Evolution and Retrieval (MACER) mechanism, where agents collaboratively engage in an iterative loop of evidence retrieval, sufficiency reflection, and dual-evolution of the query (Evolving Query) and the graph structure (Evolving Sub-Graph). The framework achieves state-of-the-art performance, recording the highest average Exact Match (EM) score of 0.453 and F1 score of 0.312 across deep reasoning benchmarks including HotpotQA, 2WikiMultiHopQA, and Musique. For AI practitioners, the principal implication is that this dual-evolving, multi-agent approach enables the creation of more precise RAG systems that can perform complex multi-hop reasoning even with smaller, locally-deployed models, mitigating the typical performance degradation associated with static graph construction in resource-constrained environments. |
| PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Lingpeng Kong, Zhuocheng Gong, Jian Guan, Wei Wu, xl-zhao |
The paper introduces PromptCoT 2.0, a framework using an Expectation-Maximization loop to iteratively co-generate rationales and prompts, producing high-quality synthetic data for enhancing LLM reasoning. The primary objective is to develop a scalable method for synthesizing complex and diverse training problems that overcomes the limitations of manual curation and static, heuristic-based generation. The core methodology formulates prompt synthesis as a latent variable model, where rationales mediate between concepts and prompts, and employs an Expectation-Maximization (EM) algorithm to iteratively refine a rationale generation model (E-step) and a prompt generation model (M-step). In a self-play setting, applying PromptCoT 2.0 to a Qwen3-30B model improved AIME 24 accuracy from 87.7% to 92.1%; in supervised fine-tuning, a 7B model trained solely on its synthetic data achieved 73.1% on the same benchmark, drastically up from the baseline 12.8%. For practitioners, this provides a scalable, automated pipeline to generate high-difficulty training corpora that can significantly boost the reasoning capabilities of both frontier and smaller open-source models without relying on expensive human annotation or access to superior teacher models. |
| D-Artemis: A Deliberative Cognitive Framework for Mobile GUI |
|
|
| Multi-Agents (Read more on arXiv or HuggingFace) |
Jinyuan Li, Yuqi Wang, Wenjie Lu, Yibo Feng, Hongze Mi |
D-Artemis is a deliberative cognitive framework designed to enhance the reliability and efficiency of mobile GUI agents by emulating a human-like cognitive process. The main objective is to overcome critical challenges in GUI automation, such as data bottlenecks in end-to-end training and the high cost of delayed error detection, by improving the performance of general-purpose Multimodal Large Language Models (MLLMs) without task-specific training. The key methodology involves a three-stage loop for each action: action generation informed by fine-grained app-specific tips, a proactive Pre-execution Alignment stage utilizing a Thought-Action Consistency (TAC) Check module and an Action Correction Agent (ACA) to prevent errors, and a post-execution Status Reflection Agent (SRA) for strategic learning. The framework achieves new state-of-the-art results, including a 75.8% success rate on the AndroidWorld benchmark and 96.8% on ScreenSpot-V2. The principal implication for AI practitioners is that incorporating proactive, deliberative mechanisms like pre-execution verification and correction into agentic frameworks can significantly enhance the performance and generalization of foundational models on complex interactive tasks, providing a more data-efficient path to developing robust autonomous agents. |
| UniVid: Unifying Vision Tasks with Pre-trained Video Generation Models (Read more on arXiv or HuggingFace) |
Yuchao Gu, Lan Chen, HelenMao |
UniVid is a framework that adapts a single pre-trained video generation model to perform diverse vision tasks through lightweight supervised fine-tuning. The research investigates whether a video generation model, pre-trained solely on natural video data without task-specific annotations, can serve as a universal backbone for a broad range of image and video tasks. The methodology involves fine-tuning a pre-trained video diffusion transformer using Low-Rank Adaptation (LoRA), where tasks are formulated as “visual sentences” (A → A’ → B → B’) to provide in-context examples. UniVid significantly outperforms the LVM baseline on depth estimation, achieving a root mean square logarithmic error of 0.42 compared to LVM’s 1.15, despite being trained on only a small subset of the training data. For AI practitioners, this work suggests that pre-trained video synthesis models can be a highly data-efficient and scalable foundation for building unified vision systems, potentially eliminating the need for costly pre-training on large-scale, multi-source annotated datasets. |
| Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive |
|
|
| Exploration for Agentic Reinforcement Learning (Read more on arXiv or HuggingFace) |
Gang Li, Zhengbao He, Xiaoyu Tan, Yulei Qin, tedsun |
SPEAR is a curriculum-based self-imitation learning recipe that improves reinforcement learning for agentic LLMs by progressively managing policy entropy to balance exploration and exploitation. The primary objective is to schedule a smooth transition from broad skill-level exploration to focused action-level exploitation, guided by the agent’s own experiences, to avoid the extremes of policy entropy collapse or runaway divergence during RL training. The key methodology extends the Self-Imitation Learning (SIL) framework with a curriculum that initially uses intrinsic rewards for skill exploration and then progressively increases self-imitation of successful trajectories from a replay buffer; stability is enhanced through advantage recalibration for off-policy updates and covariance-based clipping of high-impact tokens. SPEAR demonstrates significant performance improvements across multiple benchmarks, increasing the success rate of the GRPO baseline on WebShop by up to 20.7% (from 56.8% to 77.5%) and boosting the Dr.BoT baseline on AIME25 by up to 6.1%, with only 10-25% extra theoretical complexity. For AI practitioners, SPEAR offers a plug-and-play framework to stabilize and enhance RL training for agentic LLMs on long-horizon, sparsely-rewarded tasks, providing a structured approach to leverage an agent’s past successes for more effective and stable policy optimization without requiring expert demonstrations. |
| Fine-tuning Done Right in Model Editing (Read more on arXiv or HuggingFace) |
Du Su, Hongyu Zang, Rui Tang, Fei Sun, Wanli Yang |
This paper re-establishes fine-tuning as a leading model editing technique by demonstrating that its previously reported failures stem from flawed depth-first implementations and proposes a simple, effective localized breadth-first approach called LocFT-BF. The research investigates whether fine-tuning is inherently unsuitable for model editing or if its perceived failure is due to its common implementation as a sequential, single-pass, depth-first (DF) pipeline. The methodology involves controlled experiments comparing the DF pipeline with a standard breadth-first (BF) mini-batch pipeline, followed by a systematic analysis of parameter tuning locations across different layers and modules to optimize performance. The primary result is that the proposed LocFT-BF outperforms state-of-the-art methods by an average of 33.72% in editing success rate and is the first method shown to sustain 100K sequential edits and scale to 72B-parameter models. For AI practitioners, the principal implication is that a simple, localized, and properly implemented breadth-first fine-tuning is a highly effective, scalable, and efficient method for model editing, obviating the need for more complex, specialized algorithms. |
| X-Streamer: Unified Human World Modeling with Audiovisual Interaction (Read more on arXiv or HuggingFace) |
Guoxian Song, Chenxu Zhang, Zenan Li, You Xie, gutianpei |
X-Streamer is an end-to-end framework for generating real-time, infinitely streamable digital humans with unified audiovisual interaction from a single portrait. The primary objective is to develop a unified multimodal human world modeling architecture capable of infinite, real-time text, speech, and video generation while maintaining long-range conversational context and visual consistency. The methodology employs a Thinker-Actor dual-transformer architecture: a frozen pretrained language-speech model (Thinker) performs reasoning, while a chunk-wise autoregressive diffusion model (Actor) translates the Thinker’s hidden states into time-aligned, interleaved text, audio, and video streams, stabilized by chunk-wise diffusion forcing and global identity referencing. The system achieves state-of-the-art long-horizon video generation, attaining a Fréchet Video Distance (FVD) of 573.36, and sustains real-time multimodal streaming at 25 fps on two A100 GPUs. For AI practitioners, the principal implication is a validated framework for extending large language-speech models to generate continuous, synchronized video in real-time, enabling the development of persistent and multimodally interactive digital human agents within a single, unified architecture instead of complex modular pipelines. |
| TUN3D: Towards Real-World Scene Understanding from Unposed Images (Read more on arXiv or HuggingFace) |
Anna Vorontsova, Alexey Zakharov, Bulat Gabdullin, Nikita Drozdov, Anton Konushin |
This paper presents TUN3D, a model for joint 3D layout estimation and object detection that can process multi-view images without ground-truth camera poses or depth supervision. The main objective is to relax the input data requirements for 3D indoor scene understanding, enabling the use of casually captured images from standard cameras instead of requiring depth sensors or pre-computed point clouds. The methodology employs a lightweight sparse-convolutional backbone with two task-specific heads and introduces a novel “2x2D offsets + height” parametric wall representation that simplifies layout estimation by projecting it onto a bird’s-eye-view plane. The model achieves state-of-the-art performance, setting a new benchmark for layout estimation on ScanNet with a 66.6 F1 score from ground-truth point clouds, significantly surpassing the prior joint method PQ-Transformer’s 54.4 F1 score. The principal implication for AI practitioners is the ability to build 3D indoor scene understanding applications using only visual data from consumer devices, removing the dependency on specialized hardware like depth sensors or trackers. |
| Chasing the Tail: Effective Rubric-based Reward Modeling for Large |
|
|
| Language Model Post-Training (Read more on arXiv or HuggingFace) |
|
This paper introduces a rubric-based reward modeling method to mitigate reward over-optimization in LLM reinforcement fine-tuning (RFT) by improving reward accuracy in the high-reward tail. The primary objective is to develop a workflow for constructing reward models that can reliably distinguish between “great” and “excellent” responses, which the paper’s theoretical analysis identifies as the key to effective post-training. The methodology involves an iterative workflow where an LLM proposer refines rubric criteria by analyzing distinguishing features between pairs of high-quality, diverse, off-policy candidate responses. Empirically, using rubrics refined with four “great & diverse” off-policy pairs increased win-rates from 35.9% (SFT) to 39.7% on a generalist domain dataset and improved reward accuracy on high-reward examples from 40.3% to 47.9%. For AI practitioners, the principal implication is that RFT performance is critically dependent on the reward model’s ability to make fine-grained distinctions among top-tier outputs, and that iterative rubric refinement using strong, diverse off-policy data is an effective technique to achieve this. |
| RefAM: Attention Magnets for Zero-Shot Referral Segmentation (Read more on arXiv or HuggingFace) |
Federico Tombari, Muhammad Ferjad Naeem, Alessio Tonioni, Anna Kukleva, enisimsar |
REFAM is a training-free framework that improves zero-shot referring segmentation by using stop words as “attention magnets” to refine cross-attention maps from diffusion transformers. The main objective is to exploit features from pre-trained diffusion transformers for grounding tasks without requiring fine-tuning or architectural modifications. The methodology identifies Global Attention Sinks (GAS), augments referring expressions with auxiliary stop words to absorb surplus background attention, and then filters these “magnets” to produce cleaner grounding maps. REFAM establishes a new state-of-the-art on zero-shot benchmarks, outperforming the previous best method by +2.5 mIoU on the RefCOCOg test set. For AI practitioners, this implies that pre-trained generative diffusion models can be repurposed for high-performance, zero-shot segmentation by directly manipulating their internal attention mechanisms, bypassing the need for task-specific training. |
| WoW: Towards a World omniscient World model Through Embodied Interaction (Read more on arXiv or HuggingFace) |
Weishi Mi, Xiaozhu Ju, Chun-Kai Fan, Peidong Jia, Xiaowei Chi |
This paper presents WoW, a 14B-parameter generative world model that learns physical intuition from 2 million robot interaction trajectories to generate physically consistent video predictions and translate them into executable actions. The research objective is to validate the hypothesis that authentic physical reasoning in world models must be grounded in large-scale, causally rich interaction data rather than passive video observation. The core methodology is the SOPHIA framework, which uses a Diffusion Transformer to generate future video states and a Vision Language Model agent to critique and iteratively refine these predictions for physical realism, with a Flow-Mask Inverse Dynamics Model (FM-IDM) translating the final imagined video into executable robot actions. On the newly established WoWBench, the model achieves state-of-the-art performance, including 80.16% on physical law adherence and 96.53% on instruction understanding. The principal implication for AI practitioners is that developing physically competent models, particularly for robotics, necessitates training on extensive embodied interaction data, as it is shown to be fundamental for learning causal dynamics and closing the imagination-to-action loop. |
| FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image |
|
|
| Editing (Read more on arXiv or HuggingFace) |
Linghe Kong, Xiaohong Liu, Haotong Qin, Zhiteng Li, Junyi Wu |
FlashEdit is a novel framework for real-time, text-guided image editing that decouples speed, structure, and semantics to achieve high-fidelity results with superior background preservation. The primary objective is to overcome the prohibitive latency and quality trade-offs, such as background instability and semantic entanglement, inherent in existing diffusion-based editing methods. The methodology integrates three key innovations: a One-Step Inversion-and-Editing (OSIE) pipeline for speed, a Background Shield (BG-Shield) mechanism that caches background features in self-attention layers to maintain structural integrity, and Sparsified Spatial Cross-Attention (SSCA) which prunes text tokens pre-softmax to ensure precise semantic control. The primary result is a system that performs edits in under 0.2 seconds, achieving a 150.84× speedup over the DDIM+P2P baseline while attaining a state-of-the-art background preservation PSNR of 25.29. For AI practitioners, FlashEdit provides an efficient architecture for building interactive editing applications, demonstrating that the speed-quality trade-off can be resolved through a holistic, multi-level control strategy rather than by tackling latency, structure, and semantics as isolated problems. |
| ERGO: Efficient High-Resolution Visual Understanding for Vision-Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Ki-Ung Song, Seungmin Yang, Wooksu Shin, Jewon Lee, bokyeong1015 |
ERGO introduces an efficient coarse-to-fine reasoning pipeline for high-resolution visual understanding in Vision-Language Models (LVLMs), aiming to mitigate the substantial computational overhead from vision tokens while preserving fine-grained details. The core methodology involves a two-stage approach: initially analyzing a downsampled image to identify task-relevant regions, then cropping and re-encoding these regions at full resolution. This is achieved through reinforcement learning (RL) with a novel Task-driven Contextual Exploration (TCE) reward that combines region-verification and box adjustment components to foster reasoning-driven perception. ERGO achieves superior performance and efficiency, for instance, surpassing Qwen2.5-VL-7B by 4.7 points on the V* benchmark while utilizing only 23% of vision tokens and achieving a 3x inference speedup. This enables AI practitioners to develop LVLMs that robustly identify informative regions from coarse visual cues, leading to more computationally efficient and accurate high-resolution vision-language applications. |
| Where MLLMs Attend and What They Rely On: Explaining Autoregressive |
|
|
| Token Generation (Read more on arXiv or HuggingFace) |
Shiming Liu, Siyuan Liang, Kangwei Liu, Xiaoqing Guo, Ruoyu Chen |
EAGLE is a black-box framework designed to explain autoregressive token generation in Multimodal Large Language Models (MLLMs) by attributing outputs to specific visual regions and quantifying modality influence. The research objective is to enhance MLLM interpretability and reliability by understanding how generated tokens depend on visual inputs and to diagnose and mitigate model hallucinations. The methodology involves sparsifying images into superpixels, then optimizing an objective function that unifies insight (sufficiency) and necessity (indispensability) scores via greedy search, and performing modality-aware analysis by tracking token logit changes. Experimentally, EAGLE consistently outperforms state-of-the-art methods, achieving an average of 20.0% higher insertion and 13.4% lower deletion scores for image captioning, and significantly reducing GPU memory usage (e.g., 16.07 GB for LLaVA-1.5 7B compared to 37.25 GB for LLaVA-CAM). This framework offers AI practitioners a faithful and efficient tool for improving decision transparency, diagnosing errors, and enhancing the safety and trustworthiness of MLLMs by identifying critical input regions and disentangling modality reliance. |
| HiGS: History-Guided Sampling for Plug-and-Play Enhancement of Diffusion |
|
|
| Models (Read more on arXiv or HuggingFace) |
Romann M. Weber, Farnood Salehi, msadat97 |
HiGS is a training-free, plug-and-play sampling method that improves the quality and efficiency of diffusion models by incorporating a momentum-based history of past predictions into each generation step. The objective is to enhance image quality from pretrained diffusion models, particularly when using a low number of function evaluations (NFEs) or low classifier-free guidance (CFG) scales. The methodology computes a guidance direction as the difference between the current model prediction and an exponential moving average of past predictions, which is then refined via a scheduled weight, optional orthogonal projection, and a DCT-based high-pass filter to remove color artifacts. Primary results show that for unguided ImageNet 256x256 generation with a SiT-XL + REPA-E model, HiGS achieves a state-of-the-art FID of 1.61 in only 30 sampling steps, outperforming the baseline FID of 1.83 which requires 250 steps. For AI practitioners, HiGS can be integrated into existing inference pipelines to generate higher-fidelity images significantly faster and with lower guidance scales, without any model retraining or fine-tuning. |
| StateX: Enhancing RNN Recall via Post-training State Expansion (Read more on arXiv or HuggingFace) |
Zhiyuan Liu, Xu Han, Zhen Leng Thai, Xingyu Shen, chen-yingfa |
StateX is a post-training pipeline that enhances the long-context recall of Recurrent Neural Networks (RNNs) by architecturally expanding their recurrent state size with minimal parameter overhead. The primary objective is to improve the recall capabilities of pre-trained RNNs, such as Gated Linear Attention (GLA) and Mamba2, without the high cost associated with training large-state models from scratch. The methodology involves modifying the architecture of pre-trained models before long-context post-training: for GLA, multiple attention heads are merged into one, and for Mamba2, the key and query projection dimensions are increased, followed by a selective reinitialization of token-mixing parameters. Experiments on 1.3B models show that StateX significantly improves long-context retrieval, increasing the average Needle-in-a-Haystack (NIAH) accuracy up to 64K context from 26.0% to 42.2% for GLA and from 33.2% to 39.2% for Mamba2. For AI practitioners, this provides a cost-effective method to adapt existing pre-trained RNNs for long-context tasks, making them more competitive alternatives to Transformers in scenarios requiring high recall efficiency. |
| X-CoT: Explainable Text-to-Video Retrieval via LLM-based |
|
|
| Chain-of-Thought Reasoning (Read more on arXiv or HuggingFace) |
Raghuveer Rao, Sohail Dianat, Majid Rabbani, Jiamian Wang, prasannareddyp |
This paper introduces X-CoT, a framework that replaces standard cosine similarity ranking in text-to-video retrieval with an LLM-based Chain-of-Thought (CoT) reasoning process for improved performance and explainability. The primary objective is to interpret retrieval rankings to assess the model and data quality, moving beyond opaque similarity scores. The methodology involves augmenting video datasets with structured text annotations, using an LLM to perform pairwise comparisons on a candidate pool of videos, and aggregating these judgments with the Bradley-Terry model to produce a final, reasoned ranking. X-CoT shows a significant performance boost over embedding models, achieving, for instance, a +5.6% improvement in R@1 for CLIP on the MSVD dataset. For AI practitioners, the principal implication is that X-CoT can be implemented as a plug-and-play component on top of existing retrieval systems to enhance accuracy and provide rationales for debugging and data quality assessment without requiring model retraining. |
| Real-Time Object Detection Meets DINOv3 (Read more on arXiv or HuggingFace) |
Xi Shen, Xuanlong Yu, Longfei Liu, Yongjie Hou, Shihua Huang |
The paper introduces DEIMv2, a scalable family of real-time object detectors that integrates DINOv3 features to establish new state-of-the-art performance-cost trade-offs. The objective is to adapt the powerful semantic features from the DINOv3 foundation model into an efficient, unified framework for real-time detection across diverse computational budgets. The key methodology involves using DINOv3-pretrained Vision Transformer (ViT) backbones with a novel Spatial Tuning Adapter (STA) for larger models and pruned HGNetv2 backbones for ultra-lightweight variants, all within a DETR-based architecture. Primary results show that DEIMv2-X achieves a state-of-the-art 57.8 AP on COCO with only 50.3M parameters, while DEIMv2-S is the first sub-10M parameter model to surpass 50 AP. The principal implication for AI practitioners is the availability of a highly scalable and efficient object detection framework, providing a single architecture with multiple pre-trained model sizes suitable for deployment on hardware ranging from server-grade GPUs to resource-constrained edge devices. |
| CHURRO: Making History Readable with an Open-Weight Large |
|
|
| Vision-Language Model for High-Accuracy, Low-Cost Historical Text Recognition (Read more on arXiv or HuggingFace) |
|
The paper introduces CHURRO, an open-weight 3B-parameter Vision-Language Model (VLM) fine-tuned for high-accuracy, low-cost historical text recognition, along with a large-scale dataset, CHURRO-DS. The primary objective is to develop a specialized, cost-effective VLM that can accurately transcribe diverse historical documents, overcoming the limitations of general-purpose models designed for modern text. The methodology involves unifying 155 historical corpora into the CHURRO-DS dataset (99,491 pages) and using it to fine-tune a 3B-parameter Qwen 2.5 VL model. On the CHURRO-DS test set, the resulting CHURRO model achieves a 70.1% normalized Levenshtein similarity on handwritten documents, surpassing the much larger Gemini 2.5 Pro by 6.5% while being 15.5 times more cost-effective. For AI practitioners, this research demonstrates that fine-tuning a smaller open-weight VLM on a high-quality, domain-specific dataset can achieve superior performance and cost-efficiency compared to larger, general-purpose proprietary models for specialized vision-language tasks. |
| Finding 3D Positions of Distant Objects from Noisy Camera Movement and |
|
|
| Semantic Segmentation Sequences (Read more on arXiv or HuggingFace) |
Eija Honkavaara, Arno Solin, Julppe1 |
This paper proposes a particle filter-based method for 3D localisation of distant objects from a moving camera using noisy GNSS pose estimates and semantic segmentation sequences. The main objective is to iteratively estimate a target’s 3D position and uncertainty in computationally constrained scenarios where traditional 3D reconstruction fails. The methodology employs a bootstrap particle filter that updates a distribution of 3D point hypotheses (particles) by re-weighting them based on their projection’s proximity to segmented pixels in the camera frame, with an extension to handle multiple targets by initiating separate filters. Empirical validation using a drone to localise a telecommunication mast approximately 700 metres away achieved a minimum mean Root Mean Square Error (RMSE) of 76.88 metres. The principal implication for AI practitioners is that this lightweight, filter-based approach can be paired with any pre-existing, noisy segmentation model to enable real-time, on-board 3D geolocation for applications like wildfire monitoring without requiring computationally expensive depth estimation or feature-matching techniques. |
| Instruction-Following Evaluation in Function Calling for Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
NikolaiSkripko |
The paper introduces IFEval-FC, a benchmark for evaluating large language models’ ability to follow precise formatting instructions embedded within JSON schema descriptions during function calling. The objective is to assess LLM reliability in adhering to verifiable format constraints specified in function parameter descriptions, a capability overlooked by existing benchmarks. The methodology involves a dataset of 750 test cases, each containing a function with a specific formatting instruction injected into a parameter’s description field, with model outputs evaluated algorithmically via a binary adherence score. The results demonstrate that even state-of-the-art models fail to consistently follow these instructions, with the top-performing model achieving only 79.87% accuracy. The principal implication for AI practitioners is that LLMs in agentic systems are prone to generating syntactically invalid API calls due to poor format instruction adherence, mandating robust output validation and error handling for production deployment. |
Papers for 2025-09-26
| Title |
Authors |
Summary |
| SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines (Read more on arXiv or HuggingFace) |
Jiabei Xiao, Han Deng, Chen Tang, Uanu, Cohesion98 |
The paper introduces SciReasoner, a scientific reasoning foundation model that aligns natural language with heterogeneous scientific data representations across multiple disciplines. The research objective is to create a single, unified model that can perform a wide range of scientific tasks (103 in total), from property prediction to sequence design, while generating explicit and verifiable reasoning chains. The methodology involves pretraining on a 206B-token scientific corpus, followed by supervised fine-tuning and a novel reinforcement learning stage that employs “Adaptive Scientific Reasoning” to selectively apply chain-of-thought to complex tasks, along with task-grouped reward shaping to stabilize training. The SciReasoner-8B model achieves state-of-the-art performance on 54 tasks, for instance, attaining a 56.63% Top1 match accuracy on SMILES-to-IUPAC molecular translation, significantly outperforming specialist models (29.00%). For AI practitioners, the principal implication is the ability to use a single, powerful backbone for diverse scientific AI applications, reducing the fragmentation of specialist models and improving cross-domain generalization for complex workflows. |
| VCRL: Variance-based Curriculum Reinforcement Learning for Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Yuewei Zhang, Guofeng Quan, Wenfeng Feng, Chuzhan, Nothing2Say |
VCRL is a curriculum reinforcement learning framework that uses the variance of rollout group rewards to dynamically select appropriately difficult training samples for improving LLM reasoning. The primary objective is to enhance the efficiency and performance of rollout-based reinforcement learning by creating a curriculum that adapts to the model’s current abilities, unlike methods that treat all samples equally. The core methodology involves calculating the normalized variance of rewards from multiple generation rollouts for each sample, using this variance as a proxy for difficulty to filter the training batch to include only high-variance samples, and employing a memory bank with replay learning to maintain batch quality. Experiments show VCRL significantly outperforms baselines; on the Qwen3-8B-Base model across five math benchmarks, it achieved an average score of 57.76, a 4.67-point improvement over the strongest baseline, GSPO. For AI practitioners, this provides an efficient and dynamic curriculum learning strategy for RL fine-tuning that focuses computation on the most informative samples—those at the frontier of the model’s capabilities—thereby improving training stability and final performance. |
| MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and |
|
|
| Open Resources (Read more on arXiv or HuggingFace) |
Jing Wang, Sicong Leng, Swrooy, 26hzhang, jxjessieli |
This paper introduces MMR1, a framework that enhances multimodal reasoning by using a novel Variance-Aware Sampling (VAS) strategy to stabilize reinforcement learning, complemented by the release of large-scale open data and models. The primary objective is to mitigate the gradient vanishing problem in Group Relative Policy Optimization (GRPO) for multimodal models, which occurs when reward variance is low, thereby stabilizing training and improving reasoning performance. The core methodology is Variance-Aware Sampling (VAS), a dynamic data selection strategy guided by a Variance Promotion Score (VPS) that combines outcome variance and reasoning trajectory diversity to maintain an informative reward signal. The proposed 7B parameter MMR1 model achieves state-of-the-art performance, attaining an average score of 58.4 across five mathematical and logical reasoning benchmarks, outperforming comparable reasoning-oriented models. For AI practitioners, the principal implication is that implementing VAS can stabilize RL training and improve model performance by dynamically selecting data that maximizes reward variance, thus ensuring more consistent policy gradients without modifying the core RL algorithm. |
| Tree Search for LLM Agent Reinforcement Learning (Read more on arXiv or HuggingFace) |
Xiangxiang Chu, Guanhua Chen, Yong Wang, Ziyu Ma, Yux1ang |
This paper introduces Tree-GRPO, a tree-search-based reinforcement learning framework that improves the sample efficiency and performance of multi-turn LLM agents. The research aims to overcome the high rollout costs and sparse supervision signals of conventional chain-based RL methods for agent training. The key methodology, Tree-based Group Relative Policy Optimization (Tree-GRPO), replaces independent rollouts with a tree search where nodes represent complete agent interaction steps, allowing for shared prefixes and estimating grouped relative advantages at intra-tree and inter-tree levels to create process supervision from outcome rewards. Experimental results demonstrate that with a highly constrained budget (equivalent to two rollouts), Tree-GRPO achieves a 112% relative performance improvement over the chain-based baseline on multi-hop QA tasks. For AI practitioners, this method provides a way to train more capable and complex LLM agents with significantly lower token and tool-call budgets, making agentic RL more efficient and cost-effective. |
| Seedream 4.0: Toward Next-generation Multimodal Image Generation (Read more on arXiv or HuggingFace) |
Yunpeng Chen, Team Seedream, Cakeyan, wuwx, wujie10 |
Seedream 4.0 is an efficient, high-performance multimodal image generation system unifying text-to-image (T2I) synthesis, image editing, and multi-image composition into a single framework. The objective was to develop a scalable model capable of fast, high-resolution (1K-4K) image generation and complex multimodal editing. The methodology involves a highly efficient diffusion transformer (DiT) with a powerful VAE for reduced image tokenization, joint post-training with a Vision Language Model (VLM) for T2I and editing tasks, and inference acceleration via adversarial distillation, quantization, and speculative decoding. The system achieves state-of-the-art results, ranking first on the Artificial Analysis Arena for both T2I (Elo score: 1,220) and image editing (Elo score: 1,198) as of September 18, 2025. For AI practitioners, this provides a unified, production-ready tool that integrates high-speed, high-resolution generation with advanced editing capabilities, extending its use to professional applications like generating charts, formulas, and other knowledge-based content. |
| Hunyuan3D-Omni: A Unified Framework for Controllable Generation of 3D |
|
|
| Assets (Read more on arXiv or HuggingFace) |
Bowen Zhang, Team Hunyuan3D, SeanYoungxh, AuWang, Huiwenshi |
Hunyuan3D-Omni is a unified framework that enhances controllable 3D asset generation by integrating multiple geometric conditioning signals into a single diffusion model. The primary objective is to improve geometric accuracy and fine-grained control in image-to-3D generation by developing a single model that accepts point clouds, voxels, bounding boxes, and skeletal poses as conditioning priors. The methodology extends the Hunyuan3D 2.1 architecture by introducing a lightweight “Unified Control Encoder” which processes all control signals as point clouds and concatenates their features with image embeddings before feeding them into a Diffusion Transformer (DiT). Qualitative results demonstrate that the model accurately aligns generated meshes with conditioning signals like skeletons and resolves single-view ambiguities using point clouds; however, the paper does not provide specific quantitative performance metrics. For AI practitioners, the principal implication is the ability to add multi-modal, fine-grained geometric controls to existing image-to-3D pipelines via a single lightweight encoder, increasing robustness and enabling precise asset creation without training separate models for each control type. |
| AutoIntent: AutoML for Text Classification (Read more on arXiv or HuggingFace) |
Denis Kuznetsov, Darina Rustamova, Samoed, voorhs |
AutoIntent is a modular, end-to-end AutoML framework for text classification that automates embedding selection, classifier optimization, and threshold tuning. The primary objective is to create a comprehensive AutoML solution for intent classification that supports multi-label and out-of-scope (OOS) detection, features often lacking in existing tools. The methodology is a sequential, three-stage optimization pipeline (embedding, scoring, decision) that uses Optuna for hierarchical hyperparameter tuning across a diverse set of transformer-based and classical models. In experiments, AutoIntent achieved an out-of-scope F1-measure of 76.79 on the CLINC150 dataset, significantly outperforming AutoGluon’s 48.53. The principal implication for AI practitioners is the availability of a tool that automates the construction of robust intent classification systems, particularly for conversational AI applications requiring reliable OOS handling. |
| TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them (Read more on arXiv or HuggingFace) |
Zhuohao Yu, Xuanwang Zhang, Tingyuan Zhu, Yunze Song, Yidong Wang |
TrustJudge is a probabilistic framework that mitigates fundamental inconsistencies in LLM-as-a-judge evaluations by preserving information entropy in scoring and resolving ambiguities in pairwise comparisons. The paper’s objective is to identify, formalize, and alleviate two key inconsistencies in LLM-as-a-judge evaluations: Score-Comparison Inconsistency (where single-score ratings conflict with pairwise preferences) and Pairwise Transitivity Inconsistency (circular or contradictory preferences). The key methodology involves two components: 1) distribution-sensitive scoring, which calculates a continuous expected score from a fine-grained probability distribution over ratings to prevent information loss, and 2) likelihood-aware aggregation, which resolves transitivity violations by aggregating bidirectional preference probabilities or using response perplexity to break ties. When using Llama-3.1-70B-Instruct as the judge, TrustJudge reduced Score-Comparison Inconsistency by 8.43% (from 23.32% to 14.89%) and Pairwise Transitivity Inconsistency by 10.82% (from 15.22% to 4.40%), while simultaneously improving evaluation accuracy. For AI practitioners, TrustJudge provides a training-free, model-agnostic method to significantly improve the reliability of automated evaluations and generate more consistent preference data for reward modeling and alignment techniques like DPO, without requiring additional human annotation. |
| CE-GPPO: Controlling Entropy via Gradient-Preserving Clipping Policy |
|
|
| Optimization in Reinforcement Learning (Read more on arXiv or HuggingFace) |
Wenping Hu, Yuntao Li, Minxuan Lv, Leiyu Pan, Zhenpeng Su |
CE-GPPO is a novel reinforcement learning algorithm that controls policy entropy by reintroducing and scaling gradients from tokens typically discarded by PPO’s clipping mechanism. The main objective is to mitigate entropy instability—either collapse or explosion—during the reinforcement learning fine-tuning of large language models by analyzing and managing the gradients from low-probability tokens that fall outside the standard PPO clipping interval. The key methodology is an algorithm named Controlling Entropy via Gradient-Preserving Policy Optimization (CE-GPPO), which uses a stop-gradient operation to incorporate gradients from clipped tokens and introduces tunable coefficients (β1 and β2) to scale their magnitude, thereby enabling explicit control over the exploration-exploitation balance. The primary result shows that on the DeepSeek-R1-Distill-Qwen-7B model, CE-GPPO achieved a 67.5% average score across five mathematical reasoning benchmarks, outperforming the strong DAPO baseline’s score of 64.5%. For AI practitioners, CE-GPPO provides a more stable and effective alternative to PPO-style algorithms for fine-tuning LLMs, as it prevents both premature entropy collapse and excessive exploration, leading to consistently better performance on complex reasoning tasks. |
| Does FLUX Already Know How to Perform Physically Plausible Image |
|
|
| Composition? (Read more on arXiv or HuggingFace) |
Chen Zhao, Shaocong Zhang, Zhuming Lian, Edennnnn, Shilin-LU |
This paper introduces SHINE, a training-free framework that enables modern diffusion models like FLUX to perform high-fidelity, physically plausible image composition. The research objective is to unlock the intrinsic physical and resolution priors of pretrained text-to-image models for composition tasks without resorting to fine-tuning or brittle inversion techniques. The core methodology combines three components: Manifold-Steered Anchor (MSA) loss to guide latents using pretrained customization adapters for subject fidelity, Degradation-Suppression Guidance (DSG) to steer sampling away from low-quality outputs by manipulating internal attention queries, and Adaptive Background Blending (ABB) for seamless integration. On the DreamEditBench benchmark, the proposed method achieves state-of-the-art performance, with its LoRA variant obtaining a top ImageReward score of 0.5906, surpassing all training-based and training-free baselines. For AI practitioners, this work demonstrates that complex generative capabilities can be elicited from foundation models via inference-time optimization and guidance, providing a computationally efficient alternative to dataset curation and model retraining for specialized applications. |
| CHARM: Control-point-based 3D Anime Hairstyle Auto-Regressive Modeling (Read more on arXiv or HuggingFace) |
Yushi Bai, Jingwen Ye, Wang Zhao, Yanning Zhou, Yuze He |
This paper presents CHARM, a control-point-based parametric representation and autoregressive transformer framework for generating 3D anime hairstyles. The main objective is to develop a compact, invertible, and structured representation for stylized anime hair to enable scalable, learning-based generation from inputs like point clouds or images. The methodology treats hairstyles as a “hair language” by parameterizing each hair card as a sequence of control points, each defined by five geometric parameters (3D position, width, thickness), and then uses an autoregressive transformer to generate these sequences. CHARM achieves state-of-the-art performance, outperforming other 3D mesh generation methods with a CLIP similarity of 0.9258 and demonstrating over 98% token compression compared to original mesh representations. For AI practitioners, this framework offers a scalable and efficient method for generating editable, high-fidelity 3D anime hair assets, providing a practical solution for automating a labor-intensive component of digital character creation. |
| Recon-Act: A Self-Evolving Multi-Agent Browser-Use System via Web |
|
|
| Reconnaissance, Tool Generation, and Task Execution (Read more on arXiv or HuggingFace) |
Jinjie Gu, Chenyi Zhuang, Zhiwei Wang, Kaiwen He |
This paper presents Recon-Act, a self-evolving multi-agent system that improves browser task automation by using a reconnaissance-action paradigm to dynamically generate tools from execution trajectories. Its primary objective is to reduce excessive trial-and-error in long-horizon web tasks by enabling the system to learn from its own execution failures to generate specialized tools for unfamiliar websites. The methodology is a dual-team architecture where a “Reconnaissance Team” analyzes task trajectories to create “generalized tools” (hints or code), and an “Action Team” executes tasks using these tools, establishing a closed-loop training pipeline currently implemented with human-in-the-loop for analysis and tool management. Recon-Act achieves a new state-of-the-art overall success rate of 36.48% on the VisualWebArena benchmark, outperforming the previous best agent’s score of 33.74%. The principal implication for AI practitioners is that architecting agents with a distinct reconnaissance phase to analyze failures and dynamically generate tools offers a potent strategy for enhancing agent adaptability and solvability in complex, information-dense environments. |
| V-GameGym: Visual Game Generation for Code Large Language Models (Read more on arXiv or HuggingFace) |
Shawn Guo, Lingzheng Chai, Renshuai Tao, Jack Yang, Wei Zhang |
The paper introduces V-GameGym, a comprehensive benchmark for evaluating the visual game generation capabilities of code large language models beyond simple code execution. The primary objective is to assess code LLMs on multimodal game development tasks by evaluating not only code correctness but also game-specific metrics like playability, visual aesthetics, and dynamic interaction. The methodology involves a clustering-based curation of 2,219 Pygame samples and an automated evaluation pipeline where an LLM-as-Judge assesses generated code, screenshots, and gameplay videos in a sandboxed UI environment. The evaluation of 70 models reveals a significant capability imbalance, with models performing strongly on code generation (most scores over 70) but poorly on visual and video assessments (most scores under 25), leading to a top final score of 45.0 for GPT-5. For AI practitioners, this indicates that while LLMs excel at generating syntactically correct code, their practical application in game development is limited by a critical deficit in generating visually coherent and dynamically playable elements, highlighting a key area for future multimodal model development. |
| Interactive Recommendation Agent with Active User Commands (Read more on arXiv or HuggingFace) |
Xueyang Feng, Fei Sun, Xunke Xi, Yujie Luo, TangJiakai5704 |
The paper introduces RecBot, a dual-agent framework enabling interactive recommendation through natural language commands within mainstream feeds. The primary objective is to overcome the limitations of passive feedback mechanisms by allowing users to explicitly control recommendation policies in real-time. RecBot employs a Parser Agent to convert user commands into structured preferences and a Planner Agent that orchestrates tool chains for on-the-fly policy adjustment, with the system optimized via simulation-augmented knowledge distillation for deployment. In a three-month online A/B test, RecBot achieved a 1.40% increase in Gross Merchandise Volume (GMV) and a 0.71% reduction in Negative Feedback Frequency compared to the baseline. For AI practitioners, the principal implication is that this dual-agent architecture provides a validated, deployable framework for integrating large language models into recommender systems to enhance user satisfaction and business outcomes through direct, command-based interaction. |
| BESPOKE: Benchmark for Search-Augmented Large Language Model |
|
|
| Personalization via Diagnostic Feedback (Read more on arXiv or HuggingFace) |
Dongha Lee, Kwangwook Seo, Sangam Lee, hyunseo00 |
This paper introduces BESPOKE, a realistic benchmark for evaluating personalization in search-augmented LLMs using long-term human histories and diagnostic feedback. The primary objective is to systematically evaluate and diagnose the personalization capabilities of these models by capturing how the same query reflects different intents across users. The methodology involves collecting 2,870 authentic chat and search histories over three weeks from 30 annotators, who then provide queries, detailed information needs, and fine-grained judgments (scores and feedback) on model responses across four criteria. Results show that using a query-aware, selective history profile improves personalization, with the best model achieving an average score of 62.48, and the proposed LLM-based evaluator demonstrates strong human alignment with a 0.853 Pearson correlation. The principal implication for AI practitioners is that effective personalization hinges on sophisticated context construction (query-aware, selective history) and high-quality information retrieval, as both are shown to be critical bottlenecks. |
| Thinking Augmented Pre-training (Read more on arXiv or HuggingFace) |
Furu Wei, Li Dong, Shaohan Huang, Nan Yang, Liang Wang |
Thinking Augmented Pre-training (TPT) is a data engineering method that improves LLM data efficiency by augmenting text with automatically generated step-by-step thinking trajectories. To improve the learnability of complex tokens, TPT uses an off-the-shelf LLM to generate an explanatory “thought process” for a given document, which is then concatenated with the original text for standard next-token prediction training. The approach achieves a 3x improvement in data efficiency; an 8B model pre-trained with TPT on 100B tokens scored 50.1% on GSM8k, substantially outperforming a vanilla baseline’s 19.2% and matching a model trained on 150x more data (15T tokens). The principal implication for AI practitioners is that this offline data transformation can be scalably applied to existing pre-training or mid-training corpora to significantly boost model reasoning performance without needing new source data or altering the training objective. |
| Residual Off-Policy RL for Finetuning Behavior Cloning Policies (Read more on arXiv or HuggingFace) |
Pieter Abbeel, Guanya Shi, Rocky Duan, Zhenyu Jiang, Lars Ankile |
This paper presents ResFiT, a sample-efficient residual off-policy reinforcement learning (RL) framework to fine-tune pre-trained behavior cloning (BC) policies. The primary objective is to develop a practical method for improving large, action-chunking BC policies directly on high-degree-of-freedom (DoF) robots using online RL with only sparse binary rewards. The methodology involves freezing the pre-trained BC policy and using a highly optimized off-policy RL algorithm to train a lightweight network that learns per-step additive residual corrections to the base policy’s actions. Key results demonstrate that on a real-world 29-DoF bimanual humanoid robot, ResFiT boosted the success rate for a package handover task from 23% to 64% with approximately 76 minutes of online interaction. For AI practitioners, this residual approach provides a practical and stable pathway to deploy RL to refine complex visuomotor policies on real-world hardware, as it is agnostic to the base policy architecture and avoids the challenges of directly fine-tuning large models. |
| SD3.5-Flash: Distribution-Guided Distillation of Generative Flows (Read more on arXiv or HuggingFace) |
Yi-Zhe Song, Reshinth Adithyan, Jim Scott, Rahim Entezari, Hmrishav |
SD3.5-Flash is a few-step distillation framework for rectified flow models that enables high-quality, rapid image generation on consumer devices through distribution-guided training and pipeline optimizations. The main objective is to make computationally prohibitive, high-fidelity generative models efficient enough for practical deployment on accessible consumer hardware like mobile phones and desktop computers. The key methodology involves distilling a multi-step teacher model using two primary innovations: “timestep sharing” to stabilize gradients by leveraging student trajectory samples instead of re-noising endpoints, and “split-timestep fine-tuning” to improve prompt alignment by temporarily expanding model capacity. The primary result is that the 4-step distilled model offers up to an 18x speed-up compared to its teacher model while consistently outperforming existing few-step methods in large-scale user studies on image quality. The principal implication for AI practitioners is the ability to deploy state-of-the-art, high-quality generative AI models directly on resource-constrained edge devices and consumer-grade hardware, reducing reliance on datacenter infrastructure and enabling on-device applications. |
| Quantized Visual Geometry Grounded Transformer (Read more on arXiv or HuggingFace) |
Yuqi Li, Chuanguang Yang, Mingqiang Wu, Haotong Qin, Weilun Feng |
This paper introduces QuantVGGT, the first post-training quantization (PTQ) framework specifically for billion-scale Visual Geometry Grounded Transformers (VGGTs). The main objective is to quantize VGGTs to low bit-widths for efficient deployment by addressing challenges from heavy-tailed activation distributions and unstable calibration data inherent in multi-view 3D models. The key methodology combines Dual-Smoothed Fine-Grained Quantization (DSFQ), which uses Hadamard rotation and channel smoothing to normalize value distributions, with Noise-Filtered Diverse Sampling (NFDS), which filters statistical outliers and builds a representative calibration set using frame-aware clustering. On the Co3Dv2 camera pose estimation benchmark, the 4-bit QuantVGGT achieves an AUC@30 of 88.2, maintaining over 98% of its full-precision counterpart’s accuracy while delivering a 2.5x inference speedup and a 3.7x memory reduction. For AI practitioners, this framework enables the deployment of large-scale 3D vision transformers on resource-constrained hardware by drastically reducing their computational and memory footprint with negligible performance loss. |
| SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and |
|
|
| Self-Reflective Agent (Read more on arXiv or HuggingFace) |
Siyuan Huang, Shujie Zhang, Baoxiong Jia, Yandan Yang |
The paper introduces SCENEWEAVER, a reflective agentic framework that unifies diverse 3D scene synthesis methods through tool-based iterative refinement for realistic, instruction-aligned scene generation. The primary objective is to develop a general-purpose 3D environment generation system that addresses the combined requirements of visual realism, physical plausibility, functional diversity, and precise controllability via complex user instructions, which existing single-paradigm methods fail to meet. SCENEWEAVER employs an LLM-based planner within a closed-loop reason-act-reflect cycle; the agent dynamically selects from a standardized suite of synthesis tools, iteratively refines the scene based on self-evaluated physical and semantic feedback, and enforces physical constraints using a physics-aware executor. Primary results demonstrate superior performance in open-vocabulary generation tasks, where SCENEWEAVER achieves an average object count of 36.5 across eight room types while maintaining zero physical errors (collisions and boundary violations), outperforming all baseline methods on these combined metrics. The principal implication for AI practitioners is that a modular, self-reflective agentic framework provides a scalable and extensible paradigm for orchestrating diverse, specialized tools to solve complex, multi-constraint generation tasks, offering a robust method for creating high-fidelity, controllable environments for training embodied agents. |
| Understanding the Thinking Process of Reasoning Models: A Perspective |
|
|
| from Schoenfeld’s Episode Theory (Read more on arXiv or HuggingFace) |
Yanbin Fu, Hong Jiao, Chenrui Fan, Nan Zhang, Ming Li |
This research applies Schoenfeld’s Episode Theory, a cognitive framework for human problem-solving, to analyze and structure the reasoning processes of Large Reasoning Models (LRMs) on mathematical tasks. The primary objective is to develop a principled analytical framework to understand how LRMs organize their thought processes by annotating their reasoning traces with cognitive labels. The key methodology involved manually annotating 3,087 sentences from DeepSeek-R1’s solutions to SAT math problems using a hierarchical scheme of seven cognitive categories (e.g., Plan, Implement, Verify), creating the first public benchmark for this task. The primary result shows that an automated annotation method using GPT-4.1 with a detailed guidebook achieves a sentence-level accuracy of 0.805 and a Cohen’s kappa of 0.764 on a test subset, demonstrating the feasibility of scalable analysis. For AI practitioners, this work provides a reusable protocol and annotated corpus that enables the fine-grained analysis of machine reasoning, which can be leveraged to build more transparent and controllable AI systems. |
| ScaleDiff: Scaling Difficult Problems for Advanced Mathematical |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Yu Li, Xin Gao, Honglin Lin, Zhuoshi Pan, Qizhi Pei |
The paper presents ScaleDiff, a pipeline for automatically generating large-scale, difficult mathematical problems to enhance the reasoning capabilities of Large Reasoning Models (LRMs). The objective is to efficiently scale the creation of challenging training data by first identifying difficult problems with a single forward pass using an adaptive thinking model. The core methodology involves training a specialized generator (DiffGen-8B) on these identified problems to create new ones at scale, followed by solution distillation from a teacher model (Qwen3-8B) and filtering to produce the final ScaleDiff-Math dataset. Fine-tuning a Qwen2.5-Math-7B-Instruct model on this dataset yields a 65.9% average accuracy across five benchmarks, representing a relative performance increase of 11.3% over the original dataset. For AI practitioners, the principal implication is that this pipeline provides a cost-effective method to transfer advanced reasoning from a moderately-sized teacher model to a student model, reducing the reliance on larger, more expensive teacher models for data synthesis. |
| Behind RoPE: How Does Causal Mask Encode Positional Information? (Read more on arXiv or HuggingFace) |
Yeyun Gong, Lei Ji, Zhenghao Lin, Junu Kim, lx865712528 |
This paper proves that the causal mask in Transformer decoders inherently encodes positional information and shows its interaction with RoPE creates non-relative attention patterns in modern LLMs. The research objective is to demonstrate how the causal mask, independent of learnable parameters or explicit positional encodings, induces position-dependent attention patterns favoring nearby tokens, and to analyze its interaction with RoPE. The methodology combines theoretical proof on a simplified, parameter-free Transformer layer with empirical simulations and analysis of attention patterns in a 1.5B parameter model trained without explicit positional encoding, as well as in Llama-3, Phi-4, and Qwen3-8B. The primary results show that the causal mask alone induces attention scores that strictly increase for closer keys and that its interaction with RoPE creates a non-relative bias; in modern LLMs, this bias pattern was observed to have a non-negligible magnitude on a [-1, 1] scale. For AI practitioners, the key implication is that the causal mask is an active source of positional information that biases explicit encodings like RoPE, a joint effect that should be considered when analyzing model behavior, performance, and length generalization. |
| MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for |
|
|
| Video Temporal Reasoning (Read more on arXiv or HuggingFace) |
Junyan Zhang, Yibo Yan, Jungang Li, Sicheng Tao, EasonFan |
MOSS-ChatV is a reinforcement learning framework introducing a Dynamic Time Warping (DTW)-based process reward to improve the temporal reasoning consistency of Multimodal Large Language Models (MLLMs). The research aims to correct “process inconsistency,” where models produce correct answers despite flawed intermediate reasoning, by employing a rule-based Process Reasoning Reward (PRR) within the Group Relative Policy Optimization (GRPO) algorithm to align generated reasoning with annotated reference traces from the new MOSS-Video dataset. MOSS-ChatV achieves 87.2% accuracy on the MOSS-Video test set and improves performance on general benchmarks such as MVBench (67.6%). For AI practitioners, this demonstrates that using reinforcement learning with an efficient, rule-based process supervision reward can significantly enhance a video model’s reasoning coherence and performance without requiring a separate, learned reward model. |
| The Unanticipated Asymmetry Between Perceptual Optimization and |
|
|
| Assessment (Read more on arXiv or HuggingFace) |
Du Chen, Siyu Wu, Qi Wang, Jiabei Zhang, TianheWu |
This paper reveals a fundamental asymmetry where high-performing Image Quality Assessment (IQA) metrics do not necessarily function as effective optimization objectives for perceptual image generation. The primary objective is to systematically investigate the correlation between a metric’s IQA capability and its utility in perceptual optimization, particularly under adversarial training, and to assess the transferability of discriminator-learned features to IQA tasks. The methodology involves using single-image super-resolution with the SwinIR model as a testbed to evaluate diverse DISTS-style perceptual metrics and discriminator architectures under various optimization configurations. The study’s results show that metrics with strong IQA scores often fail to yield better optimization outcomes, and features from GAN discriminators transfer poorly for initializing IQA models compared to ImageNet pretraining; specifically, patch-level convolutional discriminators consistently outperform vanilla versions, improving average NR-IQA scores by up to +0.52 points. The principal implication for AI practitioners is that selecting a perceptual loss function based solely on its IQA benchmark performance is unreliable; instead, the discriminator architecture design is more critical, with patch-level convolutional models providing more stable and effective optimization. |
| StyleBench: Evaluating thinking styles in Large Language Models (Read more on arXiv or HuggingFace) |
Javad Lavaei, Costas Spanos, Ming Jin, Shangding Gu, Junyu Guo |
The paper introduces StyleBench, a comprehensive benchmark that systematically evaluates five reasoning styles across 15 LLMs (270M to 120B parameters) and five tasks, revealing that optimal style selection is highly contingent on both model scale and task type. The primary objective is to determine how different reasoning strategies (CoT, ToT, AoT, SoT, CoD) perform across diverse tasks and model architectures, and to identify which approaches offer the optimal balance between performance and computational efficiency. The methodology involves evaluating the five reasoning styles on five distinct reasoning tasks—including mathematical, logical, and commonsense reasoning—using 15 open-source models from major architectural families and automatically extracting final answers for comparison against ground truth. The study found no universally optimal style; for instance, Chain-of-Thought (CoT) consistently outperformed others on GSM8K mathematical problems, while search-based methods like Tree-of-Thought (ToT) excelled on open-ended puzzles like Game of 24, but only with large-scale models. Notably, on structured tasks, concise styles like SoT and CoD achieved high accuracy with significantly shorter responses, with one example showing a 94% reduction in length compared to CoT. The principal implication for AI practitioners is that reasoning strategy selection must be tailored to the specific task and available model scale; search-based methods should be used for complex problems with large models, while concise methods offer superior efficiency for well-defined tasks or resource-constrained environments, as models cannot yet learn to autonomously select the optimal style via standard fine-tuning. |
| When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks |
|
|
| Silently Undermine Validity (Read more on arXiv or HuggingFace) |
John P Dickerson, Oussama Elachqar, Astitwa Sarthak Lathe, Chiung-Yi Tseng, Benjamin Feuer |
This paper introduces diagnostic metrics to demonstrate that popular LLM-judged benchmarks suffer from severe design failures, such as schema incoherence and factor collapse, which silently undermine their validity. The main objective is to quantify these failure modes by assessing if LLM judges adhere to their rubrics and if the evaluation criteria are meaningfully distinct. The authors propose two novel mechanisms: ‘Schematic adherence,’ which uses regression to measure how well verdicts are explained by rubric scores, and ‘Psychometric validity,’ which aggregates internal consistency and discriminant validity signals. Applied to the Arena-Hard Auto benchmark, the analysis revealed severe schema incoherence, with unexplained variance in judgments exceeding 90% for the DeepSeek-R1-32B judge, and significant factor collapse, with inter-criteria correlations often exceeding 0.93. The principal implication for AI practitioners is that rankings from LLM-judged benchmarks should be treated with extreme caution, as common aggregation methods like ELO can mask fundamental invalidity and produce high-confidence leaderboards that are effectively noise, leading to flawed model selection. |
| Discrete Diffusion for Reflective Vision-Language-Action Models in |
|
|
| Autonomous Driving (Read more on arXiv or HuggingFace) |
Hang Zhao, Huimin Wang, Yue Wang, Yinan Zheng, pengxiang |
The paper introduces ReflectDrive, a framework that uses discrete diffusion and a reflective inference mechanism with search and inpainting to generate safe and coherent trajectories for autonomous driving. The primary objective is to develop a controllable end-to-end driving system that can enforce hard safety constraints, overcoming the limitations of standard imitation learning models which often violate physical rules. The key methodology involves discretizing the driving space, using a fine-tuned Diffusion Language Model for planning, and applying a two-stage, gradient-free inference process that generates diverse trajectories and then iteratively repairs them by finding safe anchor tokens via local search and using diffusion inpainting. On the NAVSIM closed-loop benchmark, ReflectDrive improves the drivable area compliance (DAC) score by +3.9 points to 99.3 and the overall PDMS score to 91.1 compared to the baseline without reflection. For AI practitioners, this research provides a method for integrating external safety oracles into generative models by leveraging a discrete token space for efficient, gradient-free search-and-repair operations, offering a scalable alternative to computationally expensive guidance or reinforcement learning for enforcing hard constraints. |
| Thinking While Listening: Simple Test Time Scaling For Audio |
|
|
| Classification (Read more on arXiv or HuggingFace) |
Mert Pilanci, Prateek Verma |
The paper introduces a test-time scaling framework for audio classification that improves performance by having a frozen LLM reason over sequences of patch-level predictions sampled from a frozen audio model. The main research objective is to devise a method for incorporating reasoning into audio classification pipelines to enable performance scaling at test time without altering the base model or input data. The key methodology involves generating a “reasoning trace” by causally processing audio in patches and sampling multiple category predictions per patch; this trace is then fed into a frozen reasoning model, such as GPT-2 with a retrained embedding matrix, to produce the final classification. Primary results show that on the ESC-50 dataset, using a frozen AST backbone, the method achieved 88.3% top-1 accuracy by sampling 32 times per patch, nearly matching the 88.8% accuracy of a fully fine-tuned AST model. The principal implication for AI practitioners is that the performance of a frozen audio classifier can be significantly enhanced at inference time by dedicating more compute to sample longer reasoning traces and aggregate them with an LLM-based reasoner; notably, a lightweight approach of retraining only the embedding matrix of a small, frozen LLM like GPT-2 is shown to be more effective than zero-shot prompting of much larger models for this task. |
| Blueprints of Trust: AI System Cards for End to End Transparency and |
|
|
| Governance (Read more on arXiv or HuggingFace) |
Roman Zhukov, Florencio Cano Gabarda, Garth Mollett, Emily Fox, Huzaifa Sidhpurwala |
This paper introduces the Hazard-Aware System Card (HASC), a dynamic, machine-readable framework for documenting an AI system’s architecture, data provenance, and evolving safety and security posture. The primary objective is to create a standardized, living artifact that enhances transparency and accountability by systematically tracking an AI system’s identified hazards and remediations throughout its lifecycle. The proposed methodology involves automated generation of the HASC via CI/CD pipelines based on a defined JSON schema and introduces a novel AI Safety Hazard (ASH) identifier (e.g., ASH-2025-0023) to catalog safety flaws, complementing the existing CVE system for security vulnerabilities. While the paper lacks quantitative experimental results, it contextualizes its proposal by citing the projection that the Hugging Face Hub will top 1.7 million models by mid-2025, highlighting the need for scalable governance. For AI practitioners, the HASC provides a concrete, automatable mechanism for creating auditable evidence of system safety and compliance, enabling automated policy enforcement and alignment with standards like ISO/IEC 42001. |
| MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with |
|
|
| Closed-Source Large-Audio Language Model (Read more on arXiv or HuggingFace) |
Hung-yi Lee, dlion168, MonicaHuang |
The paper introduces MI-Fuse, a framework for source-free unsupervised domain adaptation in speech emotion recognition using a closed-source Large Audio-Language Model (LALM). The research aims to determine if a student model can be adapted to outperform an API-only LALM on a target domain using only unlabeled audio. The key methodology involves fusing pseudo-labels from the LALM and an auxiliary source-trained classifier, weighting their predictions based on mutual information to mitigate label noise, and stabilizing training with a diversity loss and an exponential moving average teacher. Across six cross-domain transfer settings, MI-Fuse achieved an average unweighted accuracy of 58.38%, outperforming the strongest baseline by 3.9%. For AI practitioners, this work provides a practical method to train specialized, high-performing student models that can surpass general-purpose, closed-source foundation models on a target domain, even when source data is unavailable and the teacher model is a black box. |
Papers for 2025-09-25
| Title |
Authors |
Summary |
| Video models are zero-shot learners and reasoners (Read more on arXiv or HuggingFace) |
rgeirhos, kswersky, nmatares, yuxuanli, ThaddaeusWiedemer |
This research demonstrates that the generative video model Veo 3 possesses emergent zero-shot capabilities across a wide range of vision tasks, from perception to reasoning. The study investigates if large generative video models are developing general-purpose vision understanding, akin to the evolution of Large Language Models in natural language processing. The methodology involved prompting the Veo 3 model with an initial image and text instructions for 62 qualitative and 7 quantitative tasks in a zero-shot setting, comparing its performance against its predecessor, Veo 2, and other models. Based on an analysis of 18,384 generated videos, Veo 3 showed substantial improvement over Veo 2, achieving a 78% pass@10 rate on 5x5 maze solving compared to Veo 2’s 14%, and exhibited early forms of “chain-of-frames” visual reasoning. The principal implication for AI practitioners is that prompting large video models is an emerging paradigm for solving diverse computer vision problems, potentially reducing the need for task-specific training and underscoring the future importance of visual and textual prompt engineering. |
| SIM-CoT: Supervised Implicit Chain-of-Thought (Read more on arXiv or HuggingFace) |
Yuhang Cao, Xiaoyi Dong, Yuhang Zang, LiuXR, Wiselnn |
SIM-CoT is a training module that stabilizes implicit Chain-of-Thought reasoning in LLMs by applying step-level supervision through an auxiliary decoder, improving performance without inference overhead. The research objective is to diagnose and resolve the “latent instability” issue, where increasing implicit reasoning tokens causes training collapse due to homogeneous latent representations, thereby closing the performance gap with explicit CoT methods. The key methodology involves using a temporary auxiliary decoder during training to align each implicit latent token with its corresponding explicit textual reasoning step, providing fine-grained supervision that is removed at inference. SIM-CoT significantly boosts baseline performance, improving upon the Coconut method by +8.2% on the GSM8k-Aug dataset for GPT-2 and surpassing the explicit CoT baseline by 2.1% with 2.3x greater token efficiency. For AI practitioners, SIM-CoT offers a practical, plug-and-play technique to build more stable, accurate, and token-efficient reasoning models while enabling interpretability of the latent steps for debugging. |
| Advancing Speech Understanding in Speech-Aware Language Models with GRPO (Read more on arXiv or HuggingFace) |
Avihu, rhoory, NimrodShabtay1986, hagaia, avishai-elmakies |
This paper applies Group Relative Policy Optimization (GRPO) to enhance the performance of Speech-Aware Language Models (SALLMs) on open-ended speech understanding tasks. The research objective is to evaluate GRPO’s effectiveness against supervised fine-tuning (SFT) for improving SALLM performance on Spoken Question Answering (SQA) and Automatic Speech Translation (AST). The methodology involves fine-tuning Granite Speech models (2B and 8B) using a GRPO variant where the reward is calculated via standard metrics like BLEU between generated and ground-truth text. On the CoVoST2 AST task with the 8B model, GRPO improved the BLEU score by 10.9% over SFT (35.08 vs 31.62), a scenario where SFT degraded performance relative to the base model. The principal implication for AI practitioners is that GRPO with a metric-based reward function serves as a highly effective fine-tuning method for generative SALLM tasks, particularly for larger models where SFT may not yield improvements. |
| EmbeddingGemma: Powerful and Lightweight Text Representations (Read more on arXiv or HuggingFace) |
Marksherwood, osanseviero, ssmoot, SindhuRaghuram97, hschechter |
This paper introduces EmbeddingGemma, a 308M parameter open text embedding model that achieves state-of-the-art performance for lightweight models. The research objective was to develop a general-purpose embedding model that offers an exceptional performance-to-cost ratio, making it suitable for resource-constrained applications. The key methodology involves initializing the model from the encoder of a T5Gemma model, training with a combination of noise-contrastive estimation, a spread-out regularizer, and geometric embedding distillation from a larger teacher model, and finally, “souping” (averaging) checkpoints trained on varied data mixtures. A primary result is its state-of-the-art performance on the MTEB benchmark for models under 500M parameters, achieving a mean task score of 61.15 on MTEB(Multilingual, v2), which is comparable to models twice its size. The principal implication for AI practitioners is the availability of a compact, open-source model for high-performance text representation tasks that is robust to quantization (down to 4-bit) and embedding truncation, enabling efficient deployment in low-latency, high-throughput, and on-device systems. |
| LLMs4All: A Review on Large Language Models for Research and |
|
|
| Applications in Academic Disciplines (Read more on arXiv or HuggingFace) |
Yanfang, lalor, Sweson, ZehongWang, mtybilly |
This paper presents a comprehensive review of state-of-the-art Large Language Models (LLMs) and their applications, limitations, and performance across diverse academic disciplines. The primary objective is to survey the integration of LLMs into arts, letters, law, economics, business, science, and engineering, and to provide a structured taxonomy of their use cases and performance benchmarks. The methodology consists of a systematic literature review and synthesis of existing LLMs (e.g., GPT series, Claude 3, Llama 3), their architectural designs, and their performance on key evaluation benchmarks like MMLU, HumanEval, and MATH. The review’s synthesis of benchmark data reveals that no single LLM dominates all domains; for instance, while Claude 3.5 Sonnet achieves a top score of 93.7% on the HumanEval coding benchmark, reasoning-specific models like DeepSeek R1 excel in quantitative tasks, scoring 97.3% on the MATH benchmark. The principal implication for AI practitioners is the necessity of task-specific model selection, as the paper demonstrates that model performance varies significantly across reasoning, coding, and general-purpose tasks, highlighting a trade-off between specialized capabilities and broad applicability. |
| EditVerse: Unifying Image and Video Editing and Generation with |
|
|
| In-Context Learning (Read more on arXiv or HuggingFace) |
Tianyu Wang, sooyek, Shaldon, CaiYuanhao, juxuan27 |
EditVerse is a unified transformer framework that performs instruction-guided image and video editing and generation by representing all modalities as an interleaved token sequence to enable in-context learning. The main objective is to develop a single, scalable model for diverse image and video editing and generation tasks, addressing the challenges of architectural limitations and video data scarcity. The key methodology involves a transformer architecture with full self-attention that processes a unified sequence of interleaved text and vision tokens, utilizing a four-dimensional Rotary Positional Embedding (RoPE) to encode sequential, temporal, and spatial information. The primary result shows that on the proposed EditVerseBench, the model achieves a VLM editing quality score of 7.65, outperforming open-source models like InsV2V (5.21) and demonstrating emergent abilities like performing tasks not seen in the video editing training data. The principal implication for AI practitioners is that a unified, self-attention-based architecture enables effective knowledge transfer from data-rich domains (image editing) to data-scarce domains (video editing), providing a viable strategy to overcome data limitations and build more generalist multimodal foundation models. |
| PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yiming Huang, thomagram, frankzydou, MorPhLingXD, chenwang |
PhysCtrl introduces a diffusion-based generative physics network to produce controllable, physically-plausible videos by generating 3D point trajectories conditioned on material properties and external forces. The main research objective is to develop a framework for physics-grounded image-to-video generation that allows explicit control over physical parameters and applied forces, addressing the common lack of physical plausibility in data-driven video models. The key methodology is a diffusion transformer model trained on a 550K synthetic animation dataset to learn physical dynamics across multiple materials. The model represents dynamics as 3D point trajectories and incorporates a novel spatiotemporal attention block and a physics-based loss derived from the Material Point Method (MPM) deformation gradient update to enforce physical constraints. The primary result is that the trajectory generation model significantly outperforms existing methods on generative dynamics tasks, achieving a volume Intersection over Union (vIoU) of 77.03% on an elastic object test set, substantially higher than the 53.78% achieved by the next-best baseline. The principal implication for AI practitioners is that this framework provides a scalable method for injecting strong, controllable physics priors into generative video pipelines by using 3D point trajectories as an intermediate control signal, enabling the creation of high-fidelity, physically plausible animations without direct reliance on computationally expensive online physics simulators. |
| Logics-Parsing Technical Report (Read more on arXiv or HuggingFace) |
Fan Yang, Shuzhao Li, Xiangyang Chen, ZjuCv, xiuwenzhu |
This paper introduces Logics-Parsing, an end-to-end Large Vision-Language Model augmented with reinforcement learning for advanced document parsing. The primary objective is to overcome the limitations of existing LVLMs in handling documents with complex layouts and non-linear reading orders, such as multi-column newspapers. The methodology employs a two-stage “SFT-then-RL” training strategy, where a base model is first fine-tuned on a large, diverse dataset (>300K images) and then optimized using Layout-Centric Reinforcement Learning (LC-RL) with a multi-component reward function that directly evaluates text, layout, and reading order. On the newly proposed LogicsParsingBench benchmark, the model achieves a state-of-the-art aggregate edit distance of 0.124 on English documents, outperforming existing pipeline, expert, and general LVLM-based methods. For AI practitioners, this work provides an effective framework demonstrating that augmenting standard SFT with a targeted RL stage using explicit structural rewards is critical for building document AI systems that can accurately process structurally complex content. |
| Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal |
|
|
| Understanding and Generation (Read more on arXiv or HuggingFace) |
Zhe Lin, xternalz, kl3141, JoshuaGu, jacklishufan |
Lavida-O is a unified Masked Diffusion Model (MDM) that integrates high-resolution image generation, editing, object grounding, and image understanding within a single framework. The objective is to develop a single MDM that overcomes the limitations of prior multimodal MDMs by effectively combining image understanding and generation capabilities to achieve state-of-the-art performance on complex, interleaved tasks. The model employs an Elastic Mixture-of-Transformers (Elastic-MoT) architecture, which couples a lightweight (2.4B) generation branch with a larger (8B) understanding branch for parameter efficiency, and introduces explicit planning and self-reflection mechanisms to leverage its understanding capabilities to improve generation quality. Lavida-O achieves state-of-the-art performance across multiple benchmarks; for text-to-image generation, it obtains a FID score of 6.68 on the MJHQ-30k dataset, significantly outperforming prior unified MDMs like MMaDa (32.85). For AI practitioners, the Elastic-MoT architecture presents a parameter-efficient method for augmenting large, pretrained understanding models with generative capabilities, enabling the development of unified systems that can perform complex tasks like instruction-based editing through internal reasoning and self-correction, reducing the need for separate specialist models. |
| On the Use of Agentic Coding: An Empirical Study of Pull Requests on |
|
|
| GitHub (Read more on arXiv or HuggingFace) |
Hajimu Iida, Brittany Reid, Yutaro Kashiwa, Miku Watanabe, hao-li |
This empirical study analyzes 567 GitHub pull requests (PRs) from the agentic coding tool Claude Code to assess their integration characteristics and required human oversight compared to human-generated PRs. The main objective is to investigate the differences between agent-assisted and human-written PRs in terms of purpose, acceptance rates, rejection reasons, and the nature of subsequent revisions. The methodology involves a comparative analysis of 567 agent-generated PRs and a matched set of 567 human-generated PRs using manual classification and statistical analysis of repository data. Results show that while 83.8% of agent-assisted PRs are accepted, 45.1% of those merged require revisions, with bug fixes being the most common revision type at 45.1%. The principal implication for AI practitioners is that agent-generated code requires diligent human oversight to correct functional bugs, align with project-specific conventions, and ensure documentation consistency, making it crucial to provide agents with explicit contextual guidelines to minimize integration friction. |
Papers for 2025-09-24
| Title |
Authors |
Summary |
| Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR (Read more on arXiv or HuggingFace) |
Zeina Aldallal, Ahmad Bastati, Mohamed Motasim Hamed, Muhammad Hreden, Khalil Hennara |
Baseer is a vision-language model fine-tuned from Qwen2.5-VL-3B-Instruct for high-accuracy Arabic document-to-markdown OCR. The research objective was to develop a specialized model to overcome the inherent complexities of Arabic script, such as its cursive nature and right-to-left orientation, to achieve state-of-the-art performance in document OCR. The key methodology involved a decoder-only fine-tuning strategy on a pre-trained MLLM, keeping the vision encoder frozen, using a hybrid dataset of 500,000 image-text pairs composed of 300,000 synthetic and 200,000 real-world documents. The primary result is that on the newly introduced Misraj-DocOCR benchmark, Baseer achieves a state-of-the-art Word Error Rate (WER) of 0.25 and a Tree Edit Distance Similarity (TEDS) of 66, outperforming commercial systems in both textual and structural metrics. The principal implication for AI practitioners is that domain-specific, decoder-only fine-tuning of a general-purpose MLLM is a highly effective strategy for creating high-performance, specialized models for morphologically complex languages without retraining the entire architecture. |
| Reinforcement Learning on Pre-Training Data (Read more on arXiv or HuggingFace) |
Evander Yang, Guanhua Huang, Zenan Xu, Kejiao Li, Siheng Li |
This paper introduces Reinforcement Learning on Pre-Training data (RLPT), a paradigm for improving LLMs by applying a self-supervised, next-segment reasoning objective directly to unlabeled corpora. The main objective is to create a scalable RL framework that enhances model reasoning without relying on human annotations, thereby overcoming the bottleneck of finite high-quality data. The methodology involves rewarding the policy for predicting subsequent text segments—using Autoregressive and Middle Segment Reasoning tasks—with a generative reward model assessing the semantic consistency between the predicted segment and the ground truth. Primary results demonstrate that applying RLPT to a Qwen3-4B-Base model yields absolute improvements of 8.1 on GPQA-Diamond and 6.6 Pass@1 on AIME24, with performance following a favorable scaling law. For AI practitioners, the principal implication is that RLPT offers a compute-driven method to enhance the reasoning capabilities of base models using existing pre-training data, providing a stronger foundation for subsequent fine-tuning stages like RLVR. |
| Do You Need Proprioceptive States in Visuomotor Policies? (Read more on arXiv or HuggingFace) |
Yushen Liang, Yufeng Liu, Di Zhang, Wenbo Lu, Juntu Zhao |
This research demonstrates that removing proprioceptive state inputs from visuomotor policies significantly enhances their spatial generalization capabilities. The study’s objective is to investigate if eliminating proprioceptive states can prevent imitation learning policies from overfitting to specific training trajectories. The key methodology involves implementing a “State-free Policy” that predicts actions in a relative end-effector (EEF) action space, conditioned solely on visual observations from dual wide-angle wrist cameras to ensure full task observation. The primary result shows a dramatic improvement in generalization, with the average success rate increasing from 0% to 85% in height generalization and from 6% to 64% in horizontal generalization. For AI practitioners, the principal implication is that visuomotor policies can achieve greater robustness and data efficiency by omitting proprioceptive state inputs, provided the system ensures comprehensive visual context and operates in a relative action space. |
| MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and |
|
|
| Training Recipe (Read more on arXiv or HuggingFace) |
Wenshuo Ma, Fuwei Huang, Chongyi Wang, Zefan Wang, Tianyu Yu |
This paper presents MiniCPM-V 4.5, an 8B parameter Multimodal Large Language Model (MLLM) optimized for high efficiency and performance via novel architectural, data, and training strategies. The main objective is to address the efficiency bottlenecks in MLLM training and inference to improve scalability and accessibility. The key methodology involves three components: a unified 3D-Resampler for compact video and image encoding, a unified learning paradigm for document understanding via dynamic visual corruption, and a hybrid reinforcement learning strategy for controllable short and long reasoning. On the VideoMME benchmark, MiniCPM-V 4.5 achieves state-of-the-art performance among models under 30B while using only 46.7% of the GPU memory and 8.7% of the inference time required by the Qwen2.5-VL 7B model. The principal implication for AI practitioners is that the 3D-Resampler architecture provides a highly efficient method for processing high-frame-rate video by significantly reducing the number of visual tokens, thus lowering computational costs for deploying capable video understanding systems. |
| MAPO: Mixed Advantage Policy Optimization (Read more on arXiv or HuggingFace) |
Xuankun Rong, Jian Liang, Yiyang Fang, Quan Zhang, Wenke Huang |
MAPO is a policy optimization strategy that dynamically adjusts the advantage function in GRPO to improve foundation model reasoning by accounting for trajectory certainty. The paper’s objective is to solve the “advantage reversion” and “advantage mirror” problems in Group Relative Policy Optimization (GRPO), where a fixed advantage formulation provides poor learning signals for samples with varying difficulty. The key methodology introduces “trajectory certainty” to dynamically reweight the advantage function, mixing a standard deviation-based z-score normalization for uncertain trajectories with a proposed mean-based Advantage Percent Deviation (APD) for high-certainty trajectories. On the Geo3K math reasoning benchmark, MAPO improved the Qwen2.5-VL-7B model’s accuracy to 54.41, outperforming the baseline GRPO’s 51.91. For AI practitioners, MAPO offers a hyperparameter-free modification to the GRPO advantage calculation that can yield more stable and accurate performance in RL-based post-training for complex reasoning tasks. |
| VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with |
|
|
| Voxel-Aligned Prediction (Read more on arXiv or HuggingFace) |
Haoxiao Wang, Hengyu Liu, Zeyu Zhang, Yeqing Chen, Weijie Wang |
VolSplat introduces a voxel-aligned paradigm for feed-forward 3D Gaussian Splatting that predicts Gaussians from a 3D voxel grid instead of individual 2D pixels. The main objective is to overcome the limitations of pixel-aligned methods, such as multi-view alignment errors, view-biased density distributions, and a rigid coupling of Gaussian density to input image resolution. The key methodology involves unprojecting 2D image features into a 3D voxel grid using predicted depth maps, refining this grid with a sparse 3D U-Net, and then directly predicting Gaussian parameters for each occupied voxel. The method achieves state-of-the-art results, attaining a PSNR of 31.30 on the RealEstate10K dataset, significantly outperforming the previous best pixel-aligned method’s PSNR of 27.47. For AI practitioners, this voxel-aligned framework offers a more scalable and robust approach to feed-forward 3D reconstruction, enabling the creation of geometrically consistent and adaptively dense 3D representations from sparse views without being constrained by input image resolution. |
| Hyper-Bagel: A Unified Acceleration Framework for Multimodal |
|
|
| Understanding and Generation (Read more on arXiv or HuggingFace) |
Jianbin Zheng, Huafeng Kuang, Manlin Zhang, Xin Xia, Yanzuo Lu |
This paper proposes Hyper-Bagel, a unified framework to accelerate inference in multimodal models for both understanding and generation. The core objective is to reduce the computational overhead caused by iterative autoregressive decoding and diffusion denoising in models handling complex interleaved contexts. The framework uses a divide-and-conquer strategy, employing speculative decoding for next-token prediction and a multi-stage distillation process involving Classifier-Free Guidance (CFG) distillation and Distribution Matching Distillation via ODE (DMDO) for diffusion denoising. Key results demonstrate a greater than 2x speedup in understanding tasks and, for generation, a 16.67x speedup in text-to-image synthesis with a 6-NFE model that preserves the original model’s output quality. For AI practitioners, this research provides a method to significantly reduce the inference latency and cost of large unified multimodal models, enabling their practical deployment in cost-sensitive or real-time applications without sacrificing performance. |
| Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model |
|
|
| Self-Distillation (Read more on arXiv or HuggingFace) |
Yifeng Jiang, Jiahui Huang, Jiawei Ren, Tianchang Shen, Sherwin Bahmani |
Lyra is a generative framework for feed-forward 3D and 4D scene reconstruction into an explicit 3D Gaussian Splatting (3DGS) representation from a single image or video. The research objective is to distill the implicit 3D knowledge from a pre-trained video diffusion model into an explicit 3DGS decoder, eliminating the need for real-world multi-view training datasets. This is achieved through a self-distillation framework where a 3DGS decoder (student), operating in the video model’s latent space, is supervised by the RGB video outputs of a frozen, pre-trained video diffusion model (teacher). The model achieves state-of-the-art results, including a PSNR of 21.79 on the RealEstate10K dataset for single-image to 3D generation. The principal implication for AI practitioners is the ability to create explicit, interactive 3D/4D environments for simulation in domains like robotics and autonomous driving without requiring multi-view data capture or per-scene optimization. |
| What Characterizes Effective Reasoning? Revisiting Length, Review, and |
|
|
| Structure of CoT (Read more on arXiv or HuggingFace) |
Anthony Hartshorn, Parag Jain, Cheng Zhang, Julia Kempe, Yunzhen Feng |
This paper re-evaluates what characterizes effective Chain-of-Thought (CoT) reasoning, finding that structural quality, specifically the fraction of failed steps, is a more robust predictor of correctness than lexical properties like length or review. The research objective is to systematically determine if CoT length and review behaviors improve reasoning accuracy and to identify the underlying structural properties that drive performance across ten Large Reasoning Models. The methodology involves generating multiple CoT traces on math and scientific reasoning benchmarks, introducing a graph-based metric called the Failed-Step Fraction (FSF), and performing conditional correlation analyses alongside two causal interventions: test-time selection and controlled CoT editing. The primary result is that lower FSF is the most consistent and strongest predictor of correctness; in a test-time selection intervention, reranking candidate CoTs by FSF yielded accuracy gains of up to 10% on the AIME benchmark. The principal implication for AI practitioners is that structure-aware metrics like FSF offer a more effective mechanism for test-time selection and quality control than simple lexical heuristics, enabling more efficient use of computational resources by focusing on reasoning quality over quantity. |
| Large Language Models Discriminate Against Speakers of German Dialects (Read more on arXiv or HuggingFace) |
Katharina von der Wense, Anne Lauscher, Valentin Hofmann, Carolin Holtermann, Minh Duc Bui |
This research demonstrates that large language models exhibit significant negative stereotypical biases against speakers of German dialects. The study investigates whether LLMs reproduce human societal stereotypes by assessing biases across traits like education level and personality, analyzing seven regional German dialects. Using an association task and a decision-making task, the methodology measures both “dialect naming bias” (explicit labels) and “dialect usage bias” (implicit textual cues). The primary results show that all evaluated LLMs exhibit significant biases; for example, in the association task, GPT-5 Mini achieved a dialect usage bias score of 1.0 for the “uneducated” trait, indicating a perfect stereotypical correlation. The principal implication for AI practitioners is that models can display explicit discriminatory behavior based on linguistic demographics, with bias being amplified by explicit dialect labels, which poses significant risks for fairness in real-world applications like personnel selection. |
| OpenGVL - Benchmarking Visual Temporal Progress for Data Curation (Read more on arXiv or HuggingFace) |
Viktor Petrenko, Igor Kulakov, Gracjan Góral, Emilia Wiśnios, Paweł Budzianowski |
The paper introduces OpenGVL, an open-source benchmark for evaluating the ability of Vision-Language Models (VLMs) to predict temporal task progress in robotics and for automated data curation. The main objective is to benchmark open-source VLMs against proprietary models for this task and to provide a practical tool for assessing the quality of large-scale robotics datasets. The methodology involves prompting VLMs in zero-shot and two-shot settings to predict task completion percentages for shuffled image frames from robot trajectories, evaluating performance using the Value-Order Correlation (VOC) metric. The primary result shows that open-source models significantly underperform, achieving only approximately 70% of the performance of closed-source counterparts on these temporal reasoning tasks. For AI practitioners, OpenGVL provides a practical, automated framework to programmatically assess and filter large robotics datasets, enabling the identification of issues such as ambiguous task definitions, execution failures, and out-of-distribution samples before using the data for model training. |
| CAR-Flow: Condition-Aware Reparameterization Aligns Source and Target |
|
|
| for Better Flow Matching (Read more on arXiv or HuggingFace) |
Rui Qian, Jiasen Lu, Liangchen Song, Pengsheng Guo, Chen Chen |
CAR-Flow introduces a lightweight, shift-only reparameterization technique that conditions the source and target distributions in flow-matching models to improve generative performance. The primary objective is to alleviate the dual burden on conditional flow-matching networks, which must simultaneously learn long-range mass transport and semantic conditioning, by explicitly aligning the distributions based on the condition. The key methodology is Condition-Aware Reparameterization (CAR-Flow), which applies learnable, condition-dependent additive shifts to the initial source and/or final target distributions, thereby shortening the required probability transport path for the main velocity network. The primary result is that on ImageNet-256, augmenting the SiT-XL/2 model with CAR-Flow reduces the FID score from 2.07 to 1.68 while introducing less than 0.6% additional parameters and demonstrating faster convergence. The principal implication for AI practitioners is that integrating CAR-Flow’s simple, lightweight shift modules into existing flow-matching frameworks provides a practical and computationally inexpensive method to improve sample fidelity and training efficiency for large-scale conditional image generation. |
| HyRF: Hybrid Radiance Fields for Memory-efficient and High-quality Novel |
|
|
| View Synthesis (Read more on arXiv or HuggingFace) |
Dan Xu, ZipW |
HyRF introduces a hybrid representation for novel view synthesis that combines explicit Gaussians with grid-based neural fields to reduce model size while maintaining high rendering quality. The research objective is to mitigate the significant memory overhead of 3D Gaussian Splatting (3DGS) without compromising its real-time performance or visual fidelity. The key methodology involves decomposing the scene representation into (1) a compact set of explicit Gaussians storing only essential high-frequency parameters like position and diffuse color, and (2) decoupled grid-based neural fields that predict remaining geometric and view-dependent appearance properties. The primary result is a model size reduction of over 20x compared to 3DGS while achieving state-of-the-art rendering quality; for instance, on the Deep Blending dataset, HyRF achieves a 30.37 PSNR with a 34 MB model, surpassing 3DGS’s 29.41 PSNR with a 676 MB model. For AI practitioners, this implies that high-quality, real-time 3D rendering systems based on Gaussian splatting can be deployed in memory-constrained environments, such as on-device applications, where the large footprint of standard 3DGS would be prohibitive. |
| Zero-Shot Multi-Spectral Learning: Reimagining a Generalist Multimodal |
|
|
| Gemini 2.5 Model for Remote Sensing Applications (Read more on arXiv or HuggingFace) |
Genady Beryozkin, Maxim Neumann, Dahun Kim, Yotam Gigi, Ganesh Mallya |
This paper presents a training-free, zero-shot method for adapting generalist large multimodal models (LMMs) trained on RGB-only inputs to process and leverage multi-spectral remote sensing data. The main objective is to enable an RGB-trained model like Gemini 2.5 to understand and utilize novel multi-spectral sensor data for remote sensing tasks without any retraining or fine-tuning. The methodology transforms multi-spectral bands into several pseudo-color images (e.g., NDVI, NDWI) and provides them as input to the model alongside a detailed text prompt that describes how each image was generated from specific spectral bands and what physical properties it represents. On the BigEarthNet 19-class benchmark, this zero-shot approach improved the F1 score of Gemini 2.5 to 0.453, representing a +0.053 gain over the previous state-of-the-art zero-shot result. The principal implication for AI practitioners is that the capabilities of generalist LMMs can be extended to specialized, non-standard sensor modalities through input transformation and detailed prompt engineering, bypassing the need for expensive domain-specific model training. |
| VIR-Bench: Evaluating Geospatial and Temporal Understanding of MLLMs via |
|
|
| Travel Video Itinerary Reconstruction (Read more on arXiv or HuggingFace) |
So Fukuda, Ayako Sato, Lingfang Zhang, Eiki Murata, Hao Wang |
This paper introduces VIR-Bench, a novel benchmark for evaluating the long-range geospatial-temporal understanding of Multimodal Large Language Models (MLLMs) through travel video itinerary reconstruction. The main objective is to assess MLLMs’ capabilities on macro-scale scenarios involving multi-day, inter-city travel, addressing a gap left by existing micro-scale video benchmarks. The methodology involves a new dataset of 200 travel videos with manually annotated visiting order graphs and decomposes the evaluation into two zero-shot tasks: node prediction (identifying locations) and edge prediction (inferring temporal/spatial relationships). Results reveal that even the best proprietary model, Gemini-2.5-Pro, achieves only a 52.8% F1 score for Point of Interest (POI) node prediction and a 66.8% F1 for transition edge prediction, underscoring the task’s difficulty. The primary implication for AI practitioners is that current MLLMs possess critical limitations in long-horizon temporal reasoning from video, with transition edge prediction being a major bottleneck, indicating that robust, video-based planning applications require significant architectural improvements or specialized fine-tuning. |
Papers for 2025-09-23
| Title | Authors | Summary |
|——-|———|———|
| LIMI: Less is More for Agency (Read more on arXiv or HuggingFace)| happyZYM, evanlin2570, weizhihao1, mhjiang0408, YangXiao-nlp | The LIMI (Less Is More for Intelligent Agency) paper demonstrates that agentic intelligence emerges more effectively from minimal, strategically curated training data than from conventional large-scale datasets. The research investigates whether sophisticated agentic capabilities can be cultivated more efficiently with a small, high-quality dataset, challenging the paradigm that more data yields better agency. The methodology involved fine-tuning the GLM-4.5 model on a strategically curated dataset of only 78 training samples derived from complex, multi-turn software development and scientific research workflows. The primary result is that LIMI achieves a 73.5% performance score on AgencyBench, dramatically outperforming baseline models and showing a 53.7% improvement over a model trained on 10,000 samples while using 128 times less data. The principal implication for AI practitioners is that the development of autonomous AI systems should prioritize the strategic curation of high-quality agentic demonstrations over the sheer volume of training data, suggesting a fundamental shift in data strategy for building agents. |
| OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion
Transformer Models (Read more on arXiv or HuggingFace)| Pengze Zhang, Tianxiang Ma, Xu Bai, Xinghui Li, Jinshu Chen | This paper presents OmniInsert, a unified framework for mask-free video insertion of any reference subject into a source video using a diffusion transformer model. The objective is to address the key challenges of data scarcity, subject-scene equilibrium, and insertion harmonization in the Mask-free Video Insertion (MVI) task. The methodology includes a new data generation pipeline called InsertPipe, a Condition-Specific Feature Injection (CFI) mechanism, and a four-stage Progressive Training (PT) strategy that incorporates a Subject-Focused Loss and Insertive Preference Optimization (IPO). On the newly introduced InsertBench benchmark, OmniInsert demonstrated superior performance over commercial baselines, achieving a 68.34% preference rate in a comprehensive user study. The principal implication for AI practitioners is the provision of a complete system—including a data pipeline, a model architecture, and a multi-stage training strategy—that effectively balances the disparate learning difficulties of subject insertion and background preservation, offering a blueprint for complex conditional video editing tasks. |
| Qwen3-Omni Technical Report (Read more on arXiv or HuggingFace)| Lhma-aslp, Cyanbox, jinzheng-he, faychu, ZhifangGuo | The Qwen3-Omni technical report introduces a unified multimodal model that achieves state-of-the-art performance across text, image, audio, and video without performance degradation compared to its single-modal counterparts. The primary objective was to resolve the common modality trade-off, where improving one modality degrades others, by developing a single, integrated model. This was achieved using a Thinker-Talker Mixture-of-Experts (MoE) architecture, a novel Audio Transformer (AuT), and a multi-stage pretraining strategy that mixes unimodal and cross-modal data from an early phase. The key result demonstrates non-degradation, with the Qwen3-Omni-30B-A3B model scoring 81.69 on the MMLU benchmark, slightly outperforming its 81.24-scoring text-only counterpart, while also achieving state-of-the-art results on 32 audio/audiovisual benchmarks. For practitioners, this implies that a single, efficient model can replace multiple specialized systems for complex multimodal applications without sacrificing performance, thereby simplifying the deployment stack. |
| OnePiece: Bringing Context Engineering and Reasoning to Industrial
Cascade Ranking System (Read more on arXiv or HuggingFace)| Jiahua Wu, Ethan7, vicowang, TangJiakai5704, KID-22 | This paper introduces OnePiece, a unified framework that integrates Large Language Model (LLM) principles of context engineering and multi-step reasoning into industrial cascaded ranking systems. The primary objective is to operationalize these LLM mechanisms to achieve significant performance improvements beyond merely transplanting Transformer architectures into existing Deep Learning Recommendation Models (DLRMs). The core methodology combines structured context engineering to unify interaction history and reference signals into a token sequence, block-wise latent reasoning for iterative representation refinement, and progressive multi-task training that uses user feedback chains (e.g., click, order) for supervision. Deployed in a large-scale commercial search system, OnePiece demonstrated significant online A/B test gains, including a +2.90% increase in advertising revenue over a strong production baseline. The principal implication for AI practitioners is that redesigning input representation and the optimization process to emulate LLM-style reasoning provides a practical and more effective path to improving industrial ranking systems than solely focusing on architectural changes. |
| TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning
for Video LLMs (Read more on arXiv or HuggingFace)| Shaohui Jiao, Hangyi Kuang, Shaoyong Jia, Jing Cheng, lyhisme | TempSamp-R1 is a novel reinforcement fine-tuning framework designed to enhance temporal video understanding in Multimodal Large Language Models (MLLMs). The framework addresses limitations of existing on-policy reinforcement learning methods by improving MLLMs’ performance on video temporal grounding tasks that require precise spatio-temporal understanding. TempSamp-R1 leverages ground-truth annotations as off-policy supervision, integrates a non-linear soft advantage computation via asymmetric transformation for stable policy updates, and employs a hybrid Chain-of-Thought training paradigm. The method achieves state-of-the-art performance, evidenced by a +5.3% improvement in R1@0.5 on ActivityNet Captions, reaching 56.0%, and also shows robust few-shot generalization. This approach offers AI practitioners a more stable and data-efficient fine-tuning paradigm, which is critical for developing precise temporal reasoning capabilities in applications like video retrieval and assistive robotics, particularly when annotated data is limited. |
| GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric
Reasoning (Read more on arXiv or HuggingFace)| Hou Pong Chan, Weiwen Xu, Swrooy, 26hzhang, Guizhen | This paper introduces a two-stage reinforcement learning framework to improve geometric reasoning in Multimodal Large Language Models (MLLMs) by first correcting foundational visual perception deficits. The objective is to overcome the “perceptual bottleneck,” where poor visual understanding of geometric concepts limits the efficacy of high-level reasoning training. The methodology consists of a two-stage RL process: first, training on a curated Geo-Perception Question-Answering (GeoPQA) dataset to enhance visual perception, followed by a second stage focused on complex geometric reasoning. Applying this framework to the Qwen2.5-VL-3B-Instruct model improved geometric reasoning accuracy by 9.7% and problem-solving by 9.1% on MathVista compared to direct reasoning training alone. The principal implication for practitioners is that for vision-intensive domains, establishing a strong perceptual foundation in a model is a critical prerequisite for effective higher-level reasoning training. |
| EpiCache: Episodic KV Cache Management for Long Conversational Question
Answering (Read more on arXiv or HuggingFace)| Minsik Cho, Richa Dixit, Han-Byul Kim, Arnav Kundu, minsoo2333 | EPICACHE is a training-free Key-Value (KV) cache management framework for long conversational question answering that uses episodic clustering and an adaptive layer-wise budget allocation to operate under fixed memory constraints. The research objective is to address the unbounded memory growth from KV caching in long-context LLMs by designing a system that enforces a constant memory footprint without degrading multi-turn conversational accuracy. The methodology involves three stages: offline clustering of conversation history into topical episodes, block-wise prefill using episode-specific medoids as patched prompts to guide eviction based on a pre-calculated layer-wise sensitivity score, and online retrieval of the relevant compressed episodic cache for decoding. Across three LongConvQA benchmarks, EPICACHE improves accuracy by up to 40% over baselines, reduces latency by up to 2.4x, and cuts peak memory usage by up to 3.5x compared to full KV caching. For AI practitioners, this framework provides a practical method to deploy LLMs for extended multi-turn conversations on resource-constrained systems by bounding peak memory and significantly reducing inference latency, making such applications more efficient and feasible. |
| SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering
Tasks? (Read more on arXiv or HuggingFace)| Yannis Yiming He, Edwin Pan, Jeff Da, Xiang Deng, nlauffer | The paper introduces SWE-BENCH PRO, a challenging, contamination-resistant benchmark designed to evaluate AI agents on complex, enterprise-level software engineering tasks. The main objective is to assess the capability of current AI agents to solve long-horizon, multi-file coding problems that are more representative of real-world software development than existing benchmarks. The key methodology involves curating 1,865 problems from 41 diverse repositories (including copyleft-licensed public and proprietary commercial codebases) and using a three-stage human-in-the-loop process to augment problem statements and verify test suites, ensuring task resolvability and minimizing data contamination. The primary result is that state-of-the-art models struggle significantly, with GPT-5 achieving the highest reported Pass@1 resolve rate at 23.3% on the public set, a substantial drop from the >70% performance seen on simpler benchmarks. The principal implication for AI practitioners is that current agentic systems have critical limitations in handling the complexity and scale of industrial software engineering tasks, indicating that substantial advancements are needed in areas like context management, algorithmic correctness, and multi-file code manipulation to achieve professional-level autonomy. |
| DiffusionNFT: Online Diffusion Reinforcement with Forward Process (Read more on arXiv or HuggingFace)| Qinsheng Zhang, Haoxiang Wang, Haotian Ye, Huayu Chen, Kaiwen Zheng | This paper introduces DiffusionNFT, a novel online reinforcement learning paradigm for diffusion models that performs policy optimization directly on the forward process. The objective is to develop a likelihood-free RL framework that avoids the drawbacks of reverse-process methods, such as solver restrictions and forward-reverse inconsistency. DiffusionNFT employs a flow matching objective that contrasts positive and negative generations to define an implicit policy improvement direction, integrating reinforcement signals directly into the supervised learning objective. The method demonstrates up to 25x greater efficiency than FlowGRPO, improving the GenEval score from 0.24 to 0.98 within 1k steps, while FlowGRPO achieves 0.95 in over 5k steps. For AI practitioners, this provides a more efficient and simplified method to finetune diffusion models using RL, as it decouples training from sampling, allows the use of any black-box solver, and eliminates the need for Classifier-Free Guidance. |
| ByteWrist: A Parallel Robotic Wrist Enabling Flexible and
Anthropomorphic Motion for Confined Spaces (Read more on arXiv or HuggingFace)| Jiafeng Xu, Jingchao Qiao, Liqun Huang, Jiawen Tian, cuizhongren | This paper introduces ByteWrist, a compact, anthropomorphic parallel robotic wrist designed for high-dexterity manipulation in confined spaces. The primary objective is to overcome the structural limitations of existing serial and parallel wrists by developing a novel mechanism that achieves precise Roll-Pitch-Yaw motion while maintaining high compactness, efficiency, and stiffness. The methodology involves a mechanical design featuring a nested three-stage motor-driven linkage system, the derivation of its complete forward and inverse kinematic models, and a numerical solution for the Jacobian matrix to enable precise control. In a comparative confined-space grasping experiment, the ByteWrist-equipped robot completed the task in 234 seconds, approximately twice as fast as a Kinova-based serial wrist system which took 476 seconds. The principal implication for AI practitioners is that this hardware provides a more dexterous and anthropomorphic platform, crucial for collecting high-quality manipulation data and successfully deploying vision-language-action (VLA) models in complex, human-centric tasks. |
| FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning
Models on Automatically Verifiable Textual and Visual Questions (Read more on arXiv or HuggingFace)| tengdai722, stephaniezhou, xuanricheng, miguelhuchen, lilaczheng | This report presents a moderate-scale, contamination-free evaluation of Large Reasoning Models (LRMs) on novel textual and visual tasks, assessing performance and behavioral characteristics like reasoning faithfulness and tool use. The primary objective is to evaluate how recent LRMs perform and behave on new, automatically verifiable problems, and to understand the utility and characteristics of their test-time thinking processes. The methodology involves creating new textual and visual datasets, including the new ROME benchmark for vision, and conducting an LLM-assisted, rubric-guided analysis to quantify behaviors like inconsistent answers and tool hallucination from reasoning traces. Key results show that while LRMs generally outperform non-thinking counterparts on complex textual tasks, with the GPT-5 series achieving over 90% accuracy on deciphering, they exhibit significant behavioral issues; for example, Gemini 2.5 Pro hallucinates web search in ~40% of cases on long-tailed factual questions. The principal implication for AI practitioners is that LRM reasoning traces are not inherently reliable, as models frequently display misaligned thinking where the reasoning contradicts the final answer and pretend to use external tools, which necessitates robust verification before deployment in mission-critical applications. |
| VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video
Diffusion Models (Read more on arXiv or HuggingFace)| Sunghyun Cho, Janghyeok Han, Geonung Kim | VideoFrom3D is a novel framework that synthesizes high-quality, stylized 3D scene videos from coarse geometry by leveraging the complementary strengths of image and video diffusion models. The primary objective is to generate temporally coherent and stylistically consistent videos from minimal inputs—coarse geometry, a camera trajectory, and a reference image—addressing the quality limitations of existing video diffusion models on complex scenes. The methodology employs a two-stage approach: a Sparse Anchor-view Generation (SAG) module uses an image diffusion model to create high-quality, consistent keyframes, which are then interpolated by a Geometry-guided Generative Inbetweening (GGI) module using a video diffusion model conditioned on optical flow and structural guidance. The proposed method demonstrated superior performance over baselines, achieving a visual quality MUSIQ score of 68.615, the highest among all compared approaches. The principal implication for AI practitioners is the validation of a hybrid architecture that uses high-fidelity image models for keyframe generation and video models for motion interpolation, providing a practical method to enhance video quality and consistency in controlled generative tasks. |
| ARE: Scaling Up Agent Environments and Evaluations (Read more on arXiv or HuggingFace)| Matteo Bettini, Gerard Moreno-Torres Bertran, Amine Benhalloum, Pierre Andrews, HugoLaurencon | This paper introduces Meta Agents Research Environments (ARE), a platform for building scalable, asynchronous agent environments, and the Gaia2 benchmark, designed to evaluate advanced agent capabilities in dynamic settings. The main objective is to bridge the gap between model development and real-world deployment by enabling the creation and evaluation of agents in complex, time-driven environments that surface failure modes invisible in static benchmarks. The key methodology involves using the event-based ARE platform to construct the Gaia2 benchmark, which evaluates agents on 1,120 scenarios within a simulated mobile environment using a pass@1 metric; success is determined by a verifier that compares an agent’s sequence of write actions against a ground-truth oracle action graph for consistency, causality, and timing. The primary results show that no single model dominates, with GPT-5 (high) achieving the top overall score of 42.1% but scoring 0.0% on time-sensitive tasks, indicating an inverse scaling relationship where stronger reasoning correlates with higher latency, making agents less practical for interactive deployments. The principal implication for AI practitioners is that progress requires moving beyond current scaffolds to develop adaptive compute strategies and architectures that balance reasoning capability with latency and cost, as deploying the most powerful model is not always optimal for real-world, time-constrained applications. |
| Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from
Token and Parameter Levels (Read more on arXiv or HuggingFace)| Qi Zhang, Shuo Li, Yang Nan, Umean, Junjie-Ye | This paper investigates how supervised fine-tuning (SFT) impacts large language model (LLM) factual knowledge, aiming to understand knowledge change mechanisms and mitigate undesirable effects during fine-tuning. The study evaluates closed-book question answering (CBQA) performance across LLaMA-2 and LLaMA-3 models, employing token-level analysis via Kullback-Leibler (KL) divergence and parameter-level analysis through selective restoration of highly updated parameters. Results show that LLMs fine-tuned on 1,920 samples can perform up to 14% worse than those fine-tuned on 240 samples, and up to 90% of parameter updates during SFT do not contribute to knowledge enhancement. Restoring specific parameter updates, for instance, yielded a 10.48% performance gain on CBQA for LLaMA-3-8B with certain datasets. This work offers practical guidance for AI practitioners to develop more effective fine-tuning strategies by optimizing data scale and quality, and considering targeted parameter restoration to preserve prior knowledge and enhance model performance. |
| Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG
Applications (Read more on arXiv or HuggingFace)| Fatma Betül Terzioğlu, Reyhan Bayraktar, ozayezerceli, MElHuseyni, selvatas | Turk-LettuceDetect introduces specialized token-level hallucination detection models for Turkish Retrieval-Augmented Generation (RAG) applications. The primary objective is to mitigate hallucination in LLM-generated content for Turkish RAG systems, addressing challenges in a morphologically complex, low-resource language. The methodology formulates hallucination detection as a token-level classification task, fine-tuning ModernBERT-base-tr, TurkEmbed4STS, and lettucedect-210m-eurobert-tr-v1 encoder architectures on a machine-translated Turkish RAGTruth dataset. Experimentally, the ModernBERT-based model achieved an F1-score of 0.7266 on the complete test set, exhibiting strong performance in structured tasks. This work implies that AI practitioners can deploy computationally efficient hallucination detection for Turkish RAG, as the models support long contexts up to 8,192 tokens, enhancing reliability for real-time applications in such languages. |
| QWHA: Quantization-Aware Walsh-Hadamard Adaptation for
Parameter-Efficient Fine-Tuning on Large Language Models (Read more on arXiv or HuggingFace)| Jae-Joon Kim, Yulhwa Kim, Beomseok Kang, Seojune Lee, Hyesung Jeon | QWHA is a novel quantization-aware parameter-efficient fine-tuning (QA-PEFT) framework using Walsh-Hadamard Transform (WHT)-based adapters with an advanced initialization scheme for large language models (LLMs). The research aims to effectively integrate Fourier-related transform (FT)-based adapters into quantized LLMs for QA-PEFT, addressing limitations of existing methods by mitigating quantization errors during initialization and enhancing fine-tuning. QWHA employs a WHT-based adapter with a single transform and a two-stage initialization: AdaAlloc adaptively allocates parameters to channels based on quantization error, followed by Refinement to optimize selected parameter values, all driven by minimizing layer output error. QWHA consistently outperforms baselines in low-bit quantization accuracy, achieving 60.98% accuracy on CSQA for LLaMA-3.2-3B with 2-bit quantization, a 6.09% increase over CLoQ (54.89%), and reducing training time from 9.8 hours (LoCA) to 3.9 hours (QWHA) for LLaMA-3.1-8B (batch size 16). This provides a computationally efficient and accurate QA-PEFT solution, enabling AI practitioners to fine-tune highly quantized LLMs with improved performance and reduced training overhead. |
| Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning (Read more on arXiv or HuggingFace)| Damien Sileo, Valentin Quesnel, Valentin Lacombe | The paper introduces Reasoning Core, a scalable RLVR environment designed to advance foundational symbolic reasoning in LLMs. Its main objective is to provide a continuous supply of high-quality, verifiable training data for core formal domains like PDDL planning, first-order logic, and system equation solving. The methodology employs procedural generation with a continuous “difficulty knob” and integrates external specialized tools for solution verification via offline parallel generation. Initial zero-shot evaluations of GPT-5 confirmed the benchmark’s challenging nature, showing its average reward on ‘logic_nli’ dropped from approximately 60% on easy tasks to less than 10% on hard tasks. This environment provides AI practitioners with a critical, scalable resource for training LLMs to achieve more general and robust reasoning skills through RLVR. |
| Understanding Embedding Scaling in Collaborative Filtering (Read more on arXiv or HuggingFace)| Yonghui Yang, Fengbin Zhu, Haoyue Bai, Zhou Kaiyu, Zhuangzhuang He | This research investigates the performance of collaborative filtering models as embedding dimensions are scaled, uncovering novel “double-peak” and “logarithmic” phenomena that challenge the conventional single-peak assumption. The study aims to understand why increasing embedding dimensions does not always improve performance by analyzing the impact of noisy user-item interactions on different model architectures. The authors conducted large-scale experiments on 10 datasets with 4 models (BPR, NeuMF, LightGCN, SGL) by exponentially increasing embedding dimensions, complemented by a theoretical analysis of noise robustness. Key results show that noise-resistant models like SGL exhibit a logarithmic performance increase, while models like BPR often show a double-peak trend; for instance, on one dataset, scaling the embedding dimension led to a 25.57% NDCG@20 improvement over a standard 128-dimension model. For AI practitioners, the principal implication is that scaling embedding dimensions can significantly boost performance, but this is contingent on the dataset’s noise level and the chosen model’s inherent robustness, with graph-based and self-supervised models being more scalable. |
| Synthetic bootstrapped pretraining (Read more on arXiv or HuggingFace)| Emmanuel Candès, Tatsunori Hashimoto, Hong Liu, Aonan Zhang, Zitong Yang | Synthetic Bootstrapped Pretraining (SBP) is a procedure that improves language model performance by learning inter-document relations from a fixed corpus to synthesize a vast new dataset for joint training. The main objective is to determine if explicitly modeling inter-document correlations, which standard pretraining overlooks, can improve model performance in data-constrained scenarios. The methodology involves a three-step process: (1) identifying semantically similar document pairs in the pretraining corpus using approximate nearest neighbor search, (2) tuning a conditional language model (a “synthesizer”) on these pairs to learn the relationship p(doc2|doc1), and (3) using this synthesizer to generate a large new text corpus for joint training with the original data. In compute-matched experiments training a 3B parameter model on up to 1T tokens, SBP consistently outperforms a strong repetition baseline and delivers roughly 47% of the QA accuracy improvement attainable by an oracle model with access to 20x more unique data. The principal implication for AI practitioners is that SBP provides a self-contained framework to overcome data scarcity and enhance model pretraining by more effectively utilizing an existing, fixed corpus, enabling model self-improvement without needing external teacher models or new data sources. |
| MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late
Interaction (Read more on arXiv or HuggingFace)| Xintao Chen, Chun-cheng Jason Chen, Mengting Gu, Qi Ma, MrZilinXiao | METAEMBED is a framework for multimodal retrieval using learnable “Meta Tokens” to generate compact, structured multi-vector embeddings, enabling a test-time trade-off between retrieval accuracy and computational cost. The objective is to create a scalable multimodal retrieval system that resolves the trade-off between the limited expressiveness of single-vector embeddings and the high computational cost of traditional multi-vector methods. The key methodology involves appending a fixed number of learnable Meta Tokens to the input of a Vision-Language Model (VLM) and training them with a proposed Matryoshka Multi-Vector Retrieval (MMR) objective, which uses contrastive learning on nested prefixes of the output embeddings to organize information by granularity. The primary result is state-of-the-art performance on the MMEB benchmark, where the METAEMBED-32B variant achieved a 78.7% Precision@1 score. The framework’s test-time scalability was demonstrated as increasing the retrieval budget from (1 query, 1 candidate) vector to (16 query, 64 candidate) vectors improved this model’s performance by 6.6 percentage points. The principal implication for AI practitioners is the ability to deploy a single, powerful retrieval model that can be dynamically configured at inference time to meet different latency and memory budgets by simply adjusting the number of embedding vectors used for indexing and scoring, eliminating the need for retraining multiple models. |
| ContextFlow: Training-Free Video Object Editing via Adaptive Context
Enrichment (Read more on arXiv or HuggingFace)| Yue Ma, Xiujun Ma, Xuanhua He, Yiyang Chen | ContextFlow is a training-free framework enabling high-fidelity video object editing for Diffusion Transformers (DiTs) by using adaptive context enrichment. The primary objective is to achieve precise and temporally coherent training-free video object editing—including insertion, swapping, and deletion—in DiT models by resolving the challenges of inaccurate inversion and contextual conflicts found in prior methods. The key methodology involves three components: 1) using a high-order Rectified Flow (RF-Solver) for near-lossless video inversion to establish a robust editing foundation, 2) employing “Adaptive Context Enrichment,” a dual-path sampling process that enriches the editing path’s self-attention context by concatenating Key-Value pairs from a parallel reconstruction path, and 3) applying this guidance only to task-specific vital layers identified via a novel data-driven “Guidance Responsiveness Metric.” The method significantly outperforms existing training-free approaches and surpasses several training-based models; for object swapping, ContextFlow achieves a state-of-the-art CLIP-Score of 0.3391 and an overall consistency of 0.2648, outperforming all listed baselines. The principal implication for AI practitioners is that enriching self-attention context with reference Key-Value pairs, targeted at systematically identified influential layers, provides a superior mechanism for guiding generative transformers over hard feature injection, enabling higher-fidelity control over pretrained models without requiring any finetuning. |
| AuditoryBench++: Can Language Models Understand Auditory Knowledge
without Hearing? (Read more on arXiv or HuggingFace)| Jaeho Lee, Hyeonjun Kim, Hyunjong Ok, suhoyoo | This paper introduces AuditoryBench++, a text-only benchmark for evaluating auditory knowledge in language models, and proposes AIR-CoT, a reasoning method that injects auditory embeddings during inference to improve performance. The primary objective is to evaluate if language models can reason about auditory properties without direct audio input and to develop a method to enhance this capability. The proposed AIR-CoT method employs a two-stage process: first, it fine-tunes an LLM to detect text spans requiring auditory knowledge using special tokens; second, during inference, it pauses generation at these spans to dynamically inject audio embeddings produced by a CLAP text encoder and a projector MLP. The AIR-CoT method significantly outperforms baseline models, achieving 82.67% accuracy on the Auditory Context Reasoning task, an absolute improvement of over 11.88 percentage points compared to the strongest off-the-shelf LLM. The principal implication for AI practitioners is that this work provides a framework for enhancing LLMs with specialized, non-textual commonsense by training them to recognize knowledge gaps and dynamically inject modality-specific embeddings, enabling more robust reasoning in text-only environments without requiring full multimodal inputs at inference time. |
| Mano Report (Read more on arXiv or HuggingFace)| Minghui Wu, Hanning Wang, Chenxu Zhao, Anyang Su, Tianyu Fu | The paper introduces Mano, a multi-modal foundation model-based agent designed for robust automation of Graphical User Interface (GUI) tasks. The primary objective is to overcome the limitations of existing vision-language models (VLMs) in GUI automation, such as domain mismatch and insufficient sequential decision-making capabilities. The methodology involves a three-stage training pipeline (supervised fine-tuning, offline reinforcement learning, and online reinforcement learning) applied to the UITARS-1.5-7B model, utilizing a custom simulated environment for data generation and a dedicated verification module for error recovery. Mano achieves state-of-the-art performance, including a success rate of 41.6% on the OSWorld-Verified benchmark for computer usage tasks. The principal implication for AI practitioners is that integrating a progressive, multi-stage reinforcement learning framework with high-fidelity, domain-specific simulated data provides an effective pathway to enhance VLM robustness and sequential reasoning for practical GUI agent deployment. |
| Cross-Attention is Half Explanation in Speech-to-Text Models (Read more on arXiv or HuggingFace)| Luisa Bentivogli, Matteo Negri, Marco Gaido, Dennis Fucci, Sara Papi | This research systematically evaluates the explanatory power of cross-attention in speech-to-text (S2T) models by comparing its scores against feature attribution-based saliency maps. The study’s main objective is to determine the extent to which cross-attention reliably reflects input-output dependencies and can serve as a valid proxy for explainability in S2T systems. The methodology involves computing Pearson correlation between cross-attention scores and saliency maps generated by the SPES attribution method on both raw spectrogram inputs and encoder outputs across various Conformer-based models. Results demonstrate that even under optimal aggregation, cross-attention accounts for only about 50% of input relevance and explains a maximum of 52-75% of encoder output saliency, highlighting a fundamental gap. For AI practitioners, this implies that cross-attention is an incomplete explanatory proxy; for downstream applications like timestamping, averaging attention across layers and heads is recommended to better approximate true input relevance. |
| DIWALI - Diversity and Inclusivity aWare cuLture specific Items for
India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian
Context (Read more on arXiv or HuggingFace)| Maunendra Sankar Desarkar, mrajbrahma, pramitsahoo | This paper introduces DIWALI, a dataset of ~8,800 Indian culture-specific items across 17 facets, to evaluate LLM cultural adaptation capabilities. The main objective is to create a comprehensive, sub-regionally granular dataset for Indian culture and use it to systematically assess the cultural competence of LLMs on a text adaptation task from an American to an Indian context. The methodology involves curating the DIWALI dataset via prompting and web searches with manual verification, then evaluating seven open-weight LLMs by prompting them to adapt text from datasets like GSM8k. Performance is measured using a custom Adaptation Score (CSI matching), LLM-as-Judge, and human evaluation. Primary results demonstrate that DIWALI provides a more sensitive evaluation than existing datasets; for instance, Llama-2-7b-chat-hf scored an exact match Adaptation Score of 0.855 using DIWALI versus only 0.028 using the CANDLE dataset. Human evaluations confirm that LLMs perform shallow, surface-level adaptations, with the best model achieving an average cultural relevance score of only 2.68 out of 5. The principal implication for AI practitioners is that standard LLMs are unreliable for nuanced cultural adaptation tasks, exhibiting significant sub-regional biases and failing to capture deep contextual meaning. Relying solely on automatic metrics or even LLM-as-Judge can be misleading, and practitioners must use specialized, culturally-grounded datasets like DIWALI for meaningful evaluation and development of culturally competent AI systems. |
| When Big Models Train Small Ones: Label-Free Model Parity Alignment for
Efficient Visual Question Answering using Small VLMs (Read more on arXiv or HuggingFace)| Anand Mishra, Piyush Arora, Navlika Singh, abhiram4572 | This paper presents the Model Parity Aligner (MPA), a label-free framework to enhance small vision-language models (S-VLMs) for visual question answering (VQA) by leveraging knowledge from large VLMs (L-VLMs). The objective is to systematically improve S-VLM performance on complex VQA tasks using only unlabeled images, thereby eliminating the need for expensive data annotation. The core methodology involves a three-stage pipeline: an L-VLM first generates pseudo-annotated question-answer pairs from images (Pseudo Annotator); a Parity Identifier then filters these pairs to retain only those where the L-VLM answers correctly but the S-VLM fails, thus isolating knowledge gaps; finally, the S-VLM is fine-tuned exclusively on this high-signal subset (Parity Leveler). The framework consistently improves S-VLM performance across four VQA benchmarks, with a notable quantitative result being a +15.2% absolute accuracy improvement for TinyLLaVA-2B on ChartQA. The principal implication for AI practitioners is that MPA provides a computationally efficient method to create performant, specialized S-VLMs suitable for resource-constrained deployment by leveraging large, even closed-source, models without requiring labeled data or access to model logits. |
| From Uniform to Heterogeneous: Tailoring Policy Optimization to Every
Token’s Nature (Read more on arXiv or HuggingFace)| Bin Cui, Mengzhang Cai, Siwei Wen, Mengjie Liu, starriver030515 | This paper introduces Heterogeneous Adaptive Policy Optimization (HAPO), a token-aware algorithm that tailors reinforcement learning optimization to individual tokens based on their entropy. The research objective is to overcome the limitations of uniform optimization in existing RLHF algorithms by developing a framework that treats tokens heterogeneously based on their functional role. HAPO’s methodology consists of four entropy-driven components: Adaptive Temperature Sampling for rollouts, Token-Level Group Average for advantage calculation, Differential Advantage Redistribution using both entropy and importance ratios, and Asymmetric Adaptive Clipping for the loss function. The proposed method demonstrates consistent outperformance of baselines, with the HAPO-trained Qwen2.5-Math-7B model achieving a 3.07 point average accuracy gain over vanilla DAPO. The principal implication for AI practitioners is that applying these fine-grained, entropy-aware controls at every stage of the RL training pipeline can significantly boost LLM reasoning performance with negligible additional computational overhead. |
| CodeFuse-CR-Bench: A Comprehensiveness-aware Benchmark for End-to-End
Code Review Evaluation in Python Projects (Read more on arXiv or HuggingFace)| Hang Yu, Zihan Liao, Xunjin Zheng, Hanyang Guo, Geralt-Targaryen | This paper introduces CodeFuse-CR-Bench, a comprehensiveness-aware benchmark designed for the end-to-end evaluation of Large Language Models on repository-level code review (CR) tasks. The primary objective is to bridge the “reality gap” between existing, context-poor CR benchmarks and the holistic, context-rich nature of real-world software development by providing a more realistic evaluation framework. The methodology involves the construction of a benchmark with 601 high-quality instances from 70 Python projects, each containing multi-faceted context, and a novel evaluation framework that combines rule-based metrics (location, syntax) with model-based judgments from a custom-trained Reward Model and an LLM-as-a-Judge. The primary result shows that no single LLM dominates all aspects of CR, but Gemini 2.5 Pro achieves the highest comprehensive performance score (52.37%), demonstrating superior robustness and context utilization compared to other state-of-the-art models. The principal implication for AI practitioners is that developing effective, practical AI-powered CR assistants requires holistic, multi-dimensional evaluation, and Gemini 2.5 Pro’s strong performance with minimal retrieved context makes it a highly efficient model for scalable, real-world implementation. |
| From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI
Ecosystem (Read more on arXiv or HuggingFace)| Ahmed E. Hassan, Gopi Krishnan Rajbahadur, Bram Adams, James Jewitt, hao-li | This research presents the first end-to-end empirical audit of license propagation in the AI supply chain, revealing systemic non-compliance from datasets to models to software applications. The study’s objective is to quantify the scale and nature of “license drift”—the process by which legal obligations are discarded—across the full dataset-to-model-to-application lineage. The methodology involves constructing a dependency graph linking 364k datasets and 1.6M models from Hugging Face to 140k downstream GitHub applications, using AST-based static analysis and a custom, ML-aware compatibility matrix to detect license violations. The analysis reveals that 35.5% of model-to-application transitions violate the upstream model’s license, primarily by relicensing models with restrictive, use-based clauses under fully permissive terms. The principal implication for AI practitioners is that they must diligently vet the entire license lineage of components, as automated tooling can fix many license declaration errors but cannot resolve fundamental incompatibilities inherited from upstream assets, which create significant and often unacknowledged legal risk. |
| VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery (Read more on arXiv or HuggingFace)| Shiya Huang, Zeyu Zhang, Biao Wu, Tengfei Cheng, Jinchao Ge | The paper introduces VaseVL, an SFT-then-RL framework, and the VaseVQA benchmark to equip MLLMs with expert-level reasoning for ancient Greek pottery analysis. The objective is to develop a system that moves beyond standard supervised fine-tuning to address the brittle, superficial reasoning MLLMs exhibit in specialized domains. The methodology uses an SFT model as a baseline, diagnoses its performance gaps across a taxonomy of question types, and then applies Group Relative Policy Optimization (GRPO) with type-conditioned rewards and a KL penalty to specifically target these identified weaknesses. The VaseVL model improves upon the SFT-only baseline in complex reasoning tasks, most notably increasing the BLEU@1 score for the descriptive Decoration question type from 2.57 to 9.82. The principal implication for AI practitioners is a generalizable “diagnosis-guided reward engineering” template for adapting foundation models to expert domains, showing how a targeted RL phase can patch specific reasoning failures post-SFT without sacrificing factual recall. |
| SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward
Learning (Read more on arXiv or HuggingFace)| Zhaopeng Tu, Xiaobo Liang, Juntao Li, Xinyu Shi, dyyyyyyyy | SCAN introduces a self-denoising Monte Carlo annotation framework for robust process reward learning. The primary objective is to overcome the high noise ratio and scalability issues of synthetic data for Process Reward Models (PRMs) without external strong supervision. It employs a self-confidence metric, selective sampling for efficient data synthesis, and a noise-tolerant loss with confidence-wise reweighting for robust training. SCAN-Pro achieved a 39.2 F1 score improvement (from 19.9 to 59.1) in ProcessBench, surpassing baselines including those trained on PRM800K, and generated high-quality annotations with only 6% of vanilla MC estimation’s inference cost. This approach enables AI practitioners to train high-performing PRMs scalably and cost-effectively from noisy synthetic data. |
Papers for 2025-09-22
| Title |
Authors |
Summary |
| RPG: A Repository Planning Graph for Unified and Scalable Codebase |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Steven Liu, Xin Zhang, Kyleraha, Cipherxzc, Luo2003 |
RPG introduces a structured, graph-based representation for unified and scalable repository generation, overcoming natural language ambiguity in planning. The paper addresses the fundamental challenge of generating complete software repositories from scratch by bridging the gap between high-level user intent and intricate file/dependency networks, which prior natural language-based planning methods fail to address. The proposed methodology involves the Repository Planning Graph (RPG) to encode functional goals, file structures, data flows, and functions, and ZeroRepo, a graph-driven framework with proposal-level planning, implementation-level refinement, and graph-guided code generation. On the RepoCraft benchmark, ZeroRepo achieved 81.5% functional coverage and a 69.7% pass rate, producing repositories averaging 36K LOC, approximately 3.9x larger than the strongest baseline. This enables AI practitioners to leverage RPG for enhanced LLM understanding of repositories, accelerating agent localization, and facilitating near-linear scaling of functionality and code size for long-horizon and large-scale repository development. |
| MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid |
|
|
| Vision Tokenizer (Read more on arXiv or HuggingFace) |
jialingt, haosoul122, haotiz, bpan, FrozzZen |
Manzano is a simple and scalable unified multimodal LLM featuring a hybrid vision tokenizer for both visual understanding and generation. This work aims to significantly reduce the performance trade-off between these capabilities in unified multimodal LLMs. Its core methodology employs a shared visual encoder with separate continuous and discrete adapters for understanding and generation, respectively, feeding into a unified autoregressive LLM and an auxiliary diffusion decoder, all trained jointly. Manzano-30B achieved state-of-the-art results among unified models, scoring 94.3 on DocVQA, outperforming all other unified models presented. This implies AI practitioners can leverage Manzano’s architecture and training recipe to develop highly capable unified multimodal systems that mitigate the typical performance degradation when combining understanding and generation tasks. |
| Latent Zoning Network: A Unified Principle for Generative Modeling, |
|
|
| Representation Learning, and Classification (Read more on arXiv or HuggingFace) |
Wenyu Wang, Junyi Zhu, Xuefei Ning, Enshu Liu, fjxmlzn |
Latent Zoning Network (LZN) proposes a unified principle and framework for generative modeling, representation learning, and classification by integrating diverse data types into a shared Gaussian latent space. The paper investigates whether a single principle can unify these three core ML tasks, which currently rely on largely disjoint state-of-the-art solutions. LZN’s methodology involves mapping data to disjoint latent zones in a shared Gaussian latent space via encoders and flow matching for “latent computation,” and aligning these zones across data types using “latent alignment.” LZN improves FID on CIFAR10 for unconditional image generation from 2.76 to 2.59, and achieves 9.3% higher Top-1 accuracy than MoCo and 0.2% higher than SimCLR in unsupervised representation learning on ImageNet. AI practitioners can leverage LZN’s unified framework to simplify ML pipelines and enable greater synergy across diverse tasks by providing a principled way for shared representations. |
| BaseReward: A Strong Baseline for Multimodal Reward Model (Read more on arXiv or HuggingFace) |
jianfeipan, xuwang, KaiWu123, achernarcursa, yifanzhang114 |
This paper presents BaseReward, a powerful multimodal reward model (MRM), and provides an empirically-backed recipe for its construction. The primary objective is to systematically investigate crucial components of the MRM development pipeline, including modeling paradigms, reward head architecture, data curation, and training strategies, to establish a clear guide for building high-performance models. The methodology involves exhaustive experimental analysis comparing various architectures and training configurations, including an ablation study on over ten preference datasets. The resulting model, BaseReward, establishes a new state-of-the-art on major benchmarks, surpassing the previous SOTA on the MM-RLHF-Reward Bench by approximately 11.9% in accuracy. The principal implication for practitioners is that a highly effective MRM can be built using a simple Naive-RM architecture with an optimized two-layer MLP reward head, trained on a carefully curated mixture of both multimodal and text-only preference data, which surprisingly enhances multimodal judgment. |
| SPATIALGEN: Layout-guided 3D Indoor Scene Generation (Read more on arXiv or HuggingFace) |
Yongsen Mao, Yixun Liang, Heng Li, Chuan Fang, bertjiazheng |
SPATIALGEN is a novel framework for layout-guided 3D indoor scene generation, addressing challenges in visual quality, diversity, and semantic consistency for high-fidelity 3D indoor environments. The primary objective is to generate realistic and semantically consistent 3D indoor scenes from a 3D layout, optionally conditioned on text or a reference image. Its key methodology involves a multi-view multi-modal diffusion model leveraging a new large-scale synthetic dataset of 12,328 scenes and 4.7M panoramic renderings, utilizing alternating cross-view and cross-modal attention for consistent synthesis of appearance, geometry, and semantics. Quantitatively, SPATIALGEN significantly outperforms prior score distillation methods on its dataset, achieving an Image Reward of -0.238 and CLIP Similarity of 26.84, demonstrating superior realism and plausibility. This work provides AI practitioners with a comprehensive dataset and a robust, controllable framework for generating high-fidelity 3D indoor scenes, enabling advancements in applications such as virtual reality, interior design, and robotics. |
| BTL-UI: Blink-Think-Link Reasoning Model for GUI Agent (Read more on arXiv or HuggingFace) |
Jiahui Yang, Shaokang Wang, Pei Fu, Ruoceng Zhang, Shaojie Zhang |
BTL-UI introduces a brain-inspired “Blink-Think-Link” (BTL) framework to enhance AI-driven human-GUI interaction automation. The primary objective is to develop GUI agents whose interaction logic better mimics human cognitive processes to overcome limitations in current AI-GUI interaction models. BTL decomposes GUI interactions into Blink (rapid visual attention), Think (high-level reasoning), and Link (executable command generation) phases, utilizing automated Blink Data Generation for ROI annotations and a novel rule-based BTL Reward, optimized via GRPO. The BTL-UI-7B model achieved an average GUI grounding accuracy of 89.1% on the corrected ScreenSpot-V2 dataset, establishing a new state-of-the-art, and demonstrated SOTA performance in all metrics for AndroidControl-Low tasks. This framework provides a robust and biologically plausible approach for developing advanced GUI agents, offering multi-dimensional training guidance and improved generalizability for digital assistants. |
| Lynx: Towards High-Fidelity Personalized Video Generation (Read more on arXiv or HuggingFace) |
Linjie Luo, Jing Liu, gutianpei, tzhi-bytedance, shensang |
Lynx is a high-fidelity, adapter-based framework for personalized video generation from a single input image. The primary objective is to synthesize videos that faithfully preserve subject identity while maintaining temporal coherence and visual realism. Lynx extends a Diffusion Transformer (DiT) foundation model with two lightweight adapters: an ID-adapter using a Perceiver Resampler for ArcFace embeddings and a Ref-adapter integrating dense VAE features via cross-attention from a frozen reference pathway, trained with spatio-temporal frame packing. On a benchmark of 800 test cases, Lynx demonstrated superior face resemblance (0.779 facexlib cosine similarity) and overall video quality (0.956), while achieving competitive prompt following and motion naturalness. This adapter-based design provides AI/ML engineers with a scalable and robust framework for developing identity-preserving video synthesis applications without extensive model fine-tuning. |
| A Vision-Language-Action-Critic Model for Robotic Real-World |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jiangmiao, simonlin123, andyzsz123, haoranzhang, fuxian |
VLAC is a vision-language-action-critic model for efficient real-world robotic reinforcement learning. The paper aims to overcome sparse rewards and inefficient exploration in real-world robotic RL for VLA models. VLAC unifies actor and critic roles within a single autoregressive architecture, built on InternVL, and is trained on over 4,000 hours of language-annotated manipulation data to generate dense progress delta rewards and actions, integrating into an asynchronous real-world RL loop with human-in-the-loop protocols. VLAC increased robotic manipulation success rates from approximately 30% to 90% within 200 real-world interaction episodes, with human-in-the-loop interventions further boosting sample efficiency by 50% and achieving up to 100% final success. For AI practitioners, this work provides a practical recipe demonstrating that large multimodal priors combined with structured intrinsic progress feedback enable feasible, data-efficient, and incrementally improvable real-world online RL. |
| RGB-Only Supervised Camera Parameter Optimization in Dynamic Scenes (Read more on arXiv or HuggingFace) |
Narendra Ahuja, Hao Zhang, fangli3 |
This paper introduces ROS-Cam, a novel RGB-only supervised method for accurate and efficient camera parameter optimization in dynamic scenes. The main objective is to estimate camera parameters (focal length, rotation, translation) in dynamic scenes, using only a single RGB video, without external ground truth supervision. The methodology consists of patch-wise tracking filters for robust pseudo-supervision, outlier-aware joint optimization with a Cauchy distribution-modeled uncertainty parameter and an Average Cumulative Projection (ACP) error, and a two-stage optimization strategy for enhanced stability and speed. ROS-Cam demonstrates superior performance, achieving a PSNR of 33.55 on the NeRF-DS dataset compared to casualSAM’s 21.23, and reducing average runtime for NeRF-DS from casualSAM’s 10.5 hours to 0.83 hours. This work provides AI practitioners with a robust and efficient solution for camera pose estimation in dynamic environments, significantly reducing the reliance on costly ground truth data for dynamic scene reconstruction. |
| Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in |
|
|
| Instruction-Guided Expressive Text-To-Speech Systems (Read more on arXiv or HuggingFace) |
Hung-yi Lee, Kuan-Yu Chen, Tzu-Chieh Wei, Huang-Cheng Chou, Yi-Cheng Lin |
This paper quantifies the instruction-perception gap in instruction-guided expressive text-to-speech (ITTS) systems using a novel human evaluation framework. The study’s objective is to determine if natural-language instructions for ITTS systems reliably align with listener perceptions, particularly for graded emotion intensity and adverbs of degree. A novel evaluation framework was introduced, incorporating adverbs of degree and graded emotion intensity, alongside speaker age and word-level emphasis, utilizing the newly compiled Expressive VOice Control (E-VOC) corpus of over 165 human raters for large-scale subjective evaluations, complemented by objective acoustic analyses. The evaluation of five ITTS systems revealed that gpt-4o-mini-tts exhibited the most reliable alignment across acoustic dimensions, achieving an F1-score of 0.285 for speaker age; however, analyzed systems frequently generated “Adult” voices regardless of “Child” or “Elderly” instructions, and fine-grained control remained a significant challenge. For AI practitioners, these findings imply that current ITTS models have substantial room for improvement in perceptually aligning fine-grained expressive controls and diverse speaker age attributes with user instructions, which is crucial for their reliable deployment in applications requiring precise speech synthesis. |
| Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided |
|
|
| Role-playing Agents (Read more on arXiv or HuggingFace) |
Chao Zhang, Xueqiao Zhang, RoyalVane, YifanZhu, raul678 |
Video2Roleplay introduces a multimodal dataset and framework for video-guided role-playing agents (RPAs) to incorporate dynamic role profiles via video modality. The objective is to bridge the gap in existing RPAs by integrating video, supported by the new Role-playing-Video60k dataset comprising 60k videos and 700k dialogues. The methodology involves adaptive temporal sampling of video frames for dynamic profiles and fine-tuned character dialogues with video summary contexts for static profiles, integrated into a comprehensive RPA framework. Experimental results demonstrate the framework (InternVL2.5-8B w/ Video SFT) achieved an average performance score of 72.28 and a human-likeness score of 69.98, outperforming general and role-playing expertise baselines. This work implies AI practitioners can develop more immersive and human-like RPAs by leveraging video modality and dynamic role profiles, enhancing performance and user engagement in social applications and digital humans. |
| WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Pretrained |
|
|
| Speech Recognition Transformers (Read more on arXiv or HuggingFace) |
Karun Kumar, Akshat Pandey, tetrisd |
This paper introduces WhisTLE, a deeply supervised, text-only domain adaptation method for pretrained encoder-decoder Automatic Speech Recognition (ASR) models. The objective is to adapt ASR models like Whisper to new domains using only text data, addressing scenarios where paired speech-text data is unavailable. The methodology involves training a variational autoencoder (VAE) to model the ASR encoder’s latent outputs from text; this text-to-latent encoder is then used as a drop-in replacement to fine-tune the ASR decoder. Across four ASR models and four out-of-domain datasets, WhisTLE combined with text-to-speech (TTS) synthesis reduces the word error rate (WER) by 12.3% relative to TTS-only adaptation. The principal implication for AI practitioners is that this method allows for the effective adaptation of pretrained ASR models to specialized domains using text-only corpora, improving accuracy on domain-specific terminology without altering the inference architecture or increasing runtime costs. |
| Ask-to-Clarify: Resolving Instruction Ambiguity through Multi-turn |
|
|
| Dialogue (Read more on arXiv or HuggingFace) |
Hui Zhang, Sicheng Xie, Tianyi Lu, Xinghao Zhu, leolin9248 |
Ask-to-Clarify is a framework that enables embodied agents to resolve ambiguous human instructions through multi-turn dialogue and subsequently generate low-level actions for real-world tasks. The primary objective is to build collaborative embodied agents that actively clarify instructions with human users rather than passively executing potentially ambiguous commands. Its methodology involves a two-stage knowledge-insulation training strategy, integrating a Vision-Language Model (VLM) for dialogue-based ambiguity resolution and a diffusion model for end-to-end action generation, with a connection module linking them and a signal detector routing inference. The framework achieved strong average success rates of 95.0% on “Put the fruit,” 98.3% on “Pour the water,” and 90.0% on “Stack the blocks” tasks, significantly outperforming existing VLAs which performed poorly or failed. This work implies a crucial advancement for AI practitioners aiming to develop robust, interactive, and reliable embodied agents for real-world applications where instruction ambiguity is common. |
Papers for 2025-09-19
| Title |
Authors |
Summary |
| ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform |
|
|
| Data (Read more on arXiv or HuggingFace) |
Zehao Li, QiushiSun, heroding77, ownerEli, zyliu |
The ScaleCUA paper introduces a large-scale, cross-platform dataset and a family of open-source models to advance general-purpose computer use agents (CUAs). The research objective is to address the constraints of data scarcity and limited model transferability by creating a comprehensive GUI-centric training corpus. The methodology involves a dual-loop “Cross-Platform Interactive Data Pipeline” that combines automated agent interaction with human expert annotation across six operating systems (Windows, macOS, Linux, Android, iOS, Web) to collect over 17M grounding samples and 19K trajectories. The primary result is that the trained ScaleCUA-32B model sets new state-of-the-art performance, achieving 94.4% on MMBench-GUI L1-Hard and 47.4% on WebArena-Lite-v2, outperforming prior baselines by a significant margin (+26.6 on WebArena-Lite-v2). The principal implication for AI practitioners is that scaling with diverse, cross-platform, in-domain data is a highly effective strategy for building more capable and generalizable visual GUI agents, with the released dataset and models providing a direct foundation for future development. |
| FlowRL: Matching Reward Distributions for LLM Reasoning (Read more on arXiv or HuggingFace) |
Hengli Li, Dinghuai Zhang, jayyoung0802, daixuancheng, xuekai |
FlowRL is a reinforcement learning framework that improves LLM reasoning by matching the full reward distribution via flow balancing, rather than pursuing simple reward maximization. The primary objective is to mitigate the mode collapse seen in methods like PPO and GRPO, thereby promoting diverse and generalizable reasoning trajectories. The methodology transforms scalar rewards into a normalized target distribution using a learnable partition function and minimizes the reverse KL divergence between the policy and this target, implemented via a GFlowNet-inspired trajectory balance objective with importance sampling. On math reasoning benchmarks, FlowRL achieved an average improvement of 10.0% over GRPO and 5.1% over PPO, demonstrating superior performance. For AI practitioners, this implies that adopting a distribution-matching objective can enhance the fine-tuning of LLMs for complex reasoning tasks, leading to models that explore a broader set of valid solutions and exhibit better generalization instead of overfitting to a single dominant reasoning path. |
| Reasoning over Boundaries: Enhancing Specification Alignment via |
|
|
| Test-time Delibration (Read more on arXiv or HuggingFace) |
Zhilin Wang, Dongrui Liu, Xuyang Hu, Yafu Li, zzzhr97 |
This paper introduces “specification alignment” for LLMs, proposing a test-time deliberation method called ALIGN3 and a benchmark called SPECBENCH to improve and evaluate adherence to dynamic, scenario-specific rules. The main objective is to formalize and address the challenge of “specification alignment,” defined as an LLM’s ability to simultaneously adhere to bespoke, scenario-specific behavioral and safety specifications. The key methodology includes ALIGN3, a lightweight Test-Time Deliberation (TTD) method employing a three-step process of behavior optimization, safety-guided refinement, and holistic audit, and SPECBENCH, a benchmark covering 5 scenarios, 103 specifications, and 1,500 prompts, measured by a new metric, the Specification Alignment Rate (SAR). Primary results show that TTD enhances alignment; specifically, ALIGN3 improved the SAR of the Qwen3-14B model by 11.89% (from a 51.03% baseline to 62.92%) with minimal token overhead, effectively advancing the safety-helpfulness trade-off. The principal implication for AI practitioners is that lightweight, test-time deliberation methods offer a flexible and cost-effective alternative to retraining for enforcing complex, evolving operational specifications, enabling better model control in diverse real-world applications. |
| Evolving Language Models without Labels: Majority Drives Selection, |
|
|
| Novelty Promotes Variation (Read more on arXiv or HuggingFace) |
Kishan Panaganti, Wenhao Yu, Haolin Liu, invokerliang, yujunzhou |
This paper introduces EVOL-RL, a framework that enables label-free language model self-improvement by preventing the entropy collapse common in majority-vote-based methods. The research objective is to develop a method that allows LLMs to “evolve”—achieving broad-based, generalizable improvements—on unlabeled data by explicitly balancing selection and variation. The methodology, EVOL-RL, combines a majority-voted answer for selection with a novelty-aware reward that promotes semantically diverse reasoning paths for variation, implemented within the GRPO optimization framework. The primary result shows that EVOL-RL significantly outperforms a majority-only TTRL baseline; for example, training on label-free AIME24 data boosts a Qwen3-4B model’s pass@1 accuracy on the AIME25 benchmark from 4.6% to 16.4%. For AI practitioners, this provides a practical technique to continuously self-improve models using unlabeled data streams, critically maintaining solution diversity and enhancing out-of-domain generalization without external verifiers. |
| Understand Before You Generate: Self-Guided Training for Autoregressive |
|
|
| Image Generation (Read more on arXiv or HuggingFace) |
Xihui Liu, Wenlong Zhang, Yuqing Wang, GoodEnough, YueXY233 |
The paper introduces ST-AR, a self-guided training framework that integrates masked image modeling and contrastive learning into autoregressive models to enhance their visual understanding and image generation quality. The primary objective is to address fundamental limitations in autoregressive visual modeling—local and conditional dependence, inter-step semantic inconsistency, and spatial invariance deficiency—by compelling the model to learn high-level semantics during the generative training process. The key methodology, Self-guided Training for AutoRegressive models (ST-AR), augments the standard next-token prediction loss with three self-supervised objectives: a masked image modeling loss on attention maps to expand receptive fields, an inter-step contrastive loss for temporal semantic consistency, and an inter-view contrastive loss for spatial invariance, all guided by an exponential moving average (EMA) teacher network. Primary results demonstrate significant performance gains on ImageNet; ST-AR achieves approximately a 49% FID improvement for the LlamaGen-XL model (from 19.42 to 9.81) and increases the linear probing top-1 accuracy of LlamaGen-B from 18.68% to 45.27%, all without relying on pre-trained representation models. The principal implication for AI practitioners is that autoregressive models can be substantially improved by directly integrating self-supervised losses into the training loop, creating models with superior generation fidelity and visual understanding without altering the core architecture or inference process. The most impactful finding is that this approach yields state-of-the-art results (e.g., 2.37 FID for LlamaGen-XL with ST-AR) that are competitive with diffusion models, demonstrating a direct and effective path to enhancing existing autoregressive frameworks for image synthesis. |
| FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial |
|
|
| Search and Reasoning (Read more on arXiv or HuggingFace) |
Jiashuo Liu, Jianpeng Jiao, Liang Hu, WenhaoHuang, zhangysk |
This paper introduces FinSearchComp, a benchmark for evaluating LLM agents on realistic, open-domain financial search and reasoning. The primary objective is to assess an agent’s ability to perform complex, multi-step searches over time-sensitive, domain-specific financial data, simulating real-world analyst workflows. The methodology involves a benchmark of 635 questions, curated by 70 financial experts, divided into three tasks (Time-Sensitive Data Fetching, Simple Historical Lookup, Complex Historical Investigation) across global and Greater China markets, with evaluation conducted using an LLM-as-a-Judge protocol. The primary results show that even the top-performing model on the global subset, Grok 4 (web), scored 68.9%, significantly trailing the human expert baseline of 75.0, while on the Greater China subset, all models performed more than 34 percentage points below human experts. The principal implication for AI practitioners is that current agents struggle with freshness awareness, multi-source reconciliation, and temporal reasoning, indicating that improving search depth, data validation, and integration with specialized financial plugins is critical for developing robust and reliable financial applications. |
| RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation (Read more on arXiv or HuggingFace) |
SpaceProduct, Sicong, yaniii, huangsiteng, yumingj |
RynnVLA-001 is a vision-language-action (VLA) model that improves robot manipulation by pretraining on large-scale human demonstration videos. The objective is to overcome the scarcity of robot-specific training data by developing a pretraining strategy that effectively transfers manipulation knowledge from abundant, ego-centric human videos to a robotic agent. The key methodology is a two-stage pretraining curriculum: an Image-to-Video model first learns visual dynamics from 12M human videos, and is then finetuned to jointly predict future visual frames and human keypoint trajectories, using an ActionVAE to compress action chunks into a compact latent space. When finetuned on the same downstream manipulation dataset, RynnVLA-001 achieved a 90.6% average success rate, significantly outperforming baselines like Pi0 (70.4%) and GR00T N1.5 (55.6%). The principal implication for AI practitioners is that a multi-stage video generative pretraining pipeline, which progressively bridges from visual prediction on human data to trajectory-aware modeling, provides a more effective weight initialization for VLA models than using standard image-text pretrained models or training from scratch. |
| AToken: A Unified Tokenizer for Vision (Read more on arXiv or HuggingFace) |
Mingze Xu, Liangchen Song, afshin525, byeongjooahn, Jiasenlu |
ATOKEN is a unified visual tokenizer that achieves high-fidelity reconstruction and semantic understanding across images, videos, and 3D assets within a single framework. The research objective is to create a general-purpose visual tokenizer that overcomes the fragmentation between reconstruction- and understanding-specific models and bridges different visual modalities. The key methodology involves a pure transformer architecture with 4D Rotary Position Embeddings (RoPE) to encode diverse inputs into a shared, sparse 4D latent space, trained using an adversarial-free objective that combines perceptual and Gram matrix losses. The model achieves strong performance across modalities, for instance, attaining 0.21 rFID for image reconstruction with 82.2% zero-shot ImageNet accuracy. The principal implication for AI practitioners is that ATOKEN can serve as a single, foundational visual component for next-generation multimodal AI systems, simplifying architectures by unifying generation and understanding capabilities across diverse visual data types. |
| WorldForge: Unlocking Emergent 3D/4D Generation in Video Diffusion Model |
|
|
| via Training-Free Guidance (Read more on arXiv or HuggingFace) |
Ruibo Li, Tong Zhao, ChiZhang, 2hiTee, ChenxiSong |
WorldForge is a training-free, inference-time framework that enables precise 3D/4D trajectory control over pre-trained video diffusion models for tasks like scene generation and re-rendering. The research objective is to inject fine-grained geometric and motion control into existing video diffusion models without the need for costly retraining or fine-tuning, thereby preserving their rich generative priors. The methodology leverages a unified guidance strategy composed of three modules: Intra-Step Recursive Refinement (IRR) to inject trajectory cues at each denoising step, Flow-Gated Latent Fusion (FLF) to selectively apply guidance to motion-relevant latent channels, and Dual-Path Self-Corrective Guidance (DSG) to mitigate artifacts by correcting the denoising path. The framework demonstrates state-of-the-art performance, achieving a Fréchet Inception Distance (FID) of 96.08 on static 3D scene generation, substantially improving upon the 111.49 score of the next-best baseline. For AI practitioners, the key implication is the ability to unlock and steer the emergent 3D/4D capabilities of existing large-scale video models in a plug-and-play manner, enabling controllable content generation without specialized model training. |
| MultiEdit: Advancing Instruction-based Image Editing on Diverse and |
|
|
| Challenging Tasks (Read more on arXiv or HuggingFace) |
Xijun Gu, Lin Liu, HaoxingChen, dreamzz5, Mingsong07 |
The paper introduces MultiEdit, a large-scale dataset of over 107K samples for training and benchmarking instruction-based image editing (IBIE) models on diverse and complex tasks. The primary objective is to address the limitations of existing datasets by creating a high-quality resource covering challenging scenarios like reference-based editing, in-image text manipulation, and GUI editing. The methodology involves a novel pipeline using a SOTA MLLM to generate visual-adaptive instructions directly from source images and a SOTA ImageGen model to produce the corresponding high-fidelity edited images. As a primary result, fine-tuning the UltraEdit model on MultiEdit-Train improved its DINO score on the MultiEdit-Test benchmark by approximately 7.2% while surpassing the SOTA model Step1X-Edit on the same metric. For AI practitioners, the principal implication is that the MultiEdit dataset enables the fine-tuning of foundational models to significantly enhance their performance on sophisticated, fine-grained editing tasks for more complex real-world applications without degrading capabilities on standard benchmarks. |
| Unleashing the Potential of Multimodal LLMs for Zero-Shot |
|
|
| Spatio-Temporal Video Grounding (Read more on arXiv or HuggingFace) |
Rynson W. H. Lau, Gerhard Hancke, yuhaoliu, zaiquan |
i) This paper introduces a zero-shot framework for Spatio-Temporal Video Grounding (STVG) by exploiting and enhancing the latent grounding capabilities of Multimodal Large Language Models (MLLMs). ii) The primary objective is to overcome MLLMs’ suboptimal grounding performance in complex videos, which stems from their inability to fully integrate specific attribute and action cues from a text query. iii) The key methodology involves a Decomposed Spatio-Temporal Highlighting (DSTH) strategy that decouples a query into attribute and action sub-queries, and uses a novel logit-guided re-attention (LRA) module for test-time optimization of spatial and temporal visual prompts, complemented by a Temporal-Augmented Assembling (TAS) strategy to ensure temporal consistency. iv) On the HCSTVG-v1 benchmark, the proposed method with the LLaVA-OneVision-7B model achieves a 24.8% m_vIoU, outperforming the previous zero-shot SOTA (E3M) which scored 19.1%. v) The principal implication for AI practitioners is that MLLMs’ inherent, yet often overlooked, grounding capabilities associated with specific internal tokens can be unlocked and directed through test-time prompt optimization, enabling effective zero-shot performance on complex multimodal tasks without requiring model fine-tuning or grounding-specific training data. |
| Mind the Gap: A Closer Look at Tokenization for Multiple-Choice Question |
|
|
| Answering with LLMs (Read more on arXiv or HuggingFace) |
Katharina von der Wense, MinhDucBui, mario-sanz |
This paper demonstrates that the tokenization of the space preceding an answer label in multiple-choice question answering (MCQA) significantly impacts LLM performance and evaluation reliability. The research objective is to investigate how two different tokenization schemes—tokenizing the space separately from the answer letter versus together with it—affect model accuracy and calibration. The methodology involves evaluating 15 LLMs across six MCQA datasets by comparing the next-token probabilities of the answer labels under both tokenization strategies. Results show that tokenizing the space together with the answer letter consistently improves performance, yielding accuracy gains of up to 11% and significantly better model calibration. The principal implication for AI practitioners is that evaluation protocols for MCQA must be standardized, as this seemingly minor implementation detail can alter model rankings and produce inconsistent benchmark results. |
| EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal |
|
|
| Ultrasound Intelligence (Read more on arXiv or HuggingFace) |
Qinghua Huang, WeiWang, lidachen, Ruimed, chaoyinshe |
The paper introduces EchoVLM, a vision-language model specialized for universal ultrasound intelligence using a dynamic Mixture-of-Experts (MoE) architecture. The main objective is to address the poor performance and generalization of existing general-purpose VLMs when applied to multi-organ, multi-task ultrasound diagnostics. The methodology involves specializing the Qwen2-VL foundation model by integrating a dual-path MoE architecture and training it on a newly curated large-scale dataset of 1.47 million ultrasound images across seven anatomical regions. EchoVLM achieved significant improvements on the ultrasound report generation task, outperforming the baseline Qwen2-VL with a 10.15 point increase in BLEU-1 score. The principal implication for AI practitioners is that domain-specific adaptation using specialized architectures like MoE is a highly effective strategy for enhancing the performance of large foundation models in complex, specialized fields such as medical imaging. |
| FSG-Net: Frequency-Spatial Synergistic Gated Network for High-Resolution |
|
|
| Remote Sensing Change Detection (Read more on arXiv or HuggingFace) |
Zhewei Zhang, Yuhan Jiang, Shuangxi Miao, pedramghamisi, zx-Xie |
The paper introduces FSG-Net, a network that synergistically leverages frequency-spatial analysis to improve change detection in high-resolution remote sensing images. The primary objective is to systematically disentangle genuine semantic changes from nuisance variations (e.g., illumination, season) and to bridge the semantic gap between deep and shallow features for precise boundary delineation. The methodology consists of three core components: a Discrepancy-Aware Wavelet Interaction Module (DAWIM) to suppress pseudo-changes in the frequency domain, a Synergistic Temporal-Spatial Attention Module (STSAM) to enhance change saliency in the spatial domain, and a Lightweight Gated Fusion Unit (LGFU) to selectively integrate multi-level features. The proposed FSG-Net establishes a new state-of-the-art, achieving an F1-score of 94.16% on the CDD benchmark, outperforming previous methods. The principal implication for AI practitioners is that employing a dual-domain approach—first using wavelet decomposition to process different frequency components distinctly for noise suppression, then using spatial attention for feature enhancement—is a highly effective strategy for tasks requiring robust differentiation between semantic and stylistic variations in bi-temporal imagery. |
Papers for 2025-09-18
| Title |
Authors |
Summary |
| Hala Technical Report: Building Arabic-Centric Instruction & Translation |
|
|
| Models at Scale (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Mohammad Zbeeb, Hasan Abed Al Kader Hammoud |
This paper presents HALA, a family of Arabic-centric models developed via an efficient “translate-and-tune” pipeline to address the scarcity of high-quality Arabic instruction data. The main objective is to create a scalable method for building specialized Arabic models by translating large English instruction corpora and fine-tuning foundation models. The key methodology involves bootstrapping a lightweight translator by fine-tuning a 1.2B model on a bilingual corpus created using a quantized (FP8) teacher, then using this new translator to generate a million-scale Arabic instruction dataset for subsequent model training and slerp-based merging. The primary results show that this approach achieves state-of-the-art performance on Arabic benchmarks, with the HALA-1.2B model scoring 51.4% on average, a +5.1 absolute point improvement over its base. For AI practitioners, this research provides a validated, compute-efficient recipe for developing high-performing models in under-resourced languages by leveraging existing English-language assets and specialized data generation pipelines. |
| SAIL-VL2 Technical Report (Read more on arXiv or HuggingFace) |
Zijian Kang, Yue Liao, Fangxun Shu, Yongjie Ye, Weijie Yin |
SAIL-VL2 is an open-suite vision-language foundation model designed for comprehensive multimodal understanding and reasoning. The primary objective was to develop powerful yet efficient LVMs by optimizing knowledge injection through efficient architectures and training strategies for strong multimodal performance. The methodology includes a large-scale data curation pipeline, a progressive training framework (SAIL-ViT pre-training, multimodal pre-training, and thinking-fusion SFT-RL hybrid), and architectural advances like sparse Mixture-of-Experts (MoE) designs. SAIL-VL2 achieves state-of-the-art performance at 2B and 8B parameter scales across 106 diverse benchmarks, with SAIL-VL2-2B ranking first on the OpenCompass leaderboard among officially released open-source models under 4B parameters, and SAIL-VL2-8B-Thinking scoring 54.4 on OpenCompass multimodal reasoning. SAIL-VL2 serves as an efficient and extensible open-source foundation, advancing state-of-the-art performance and empowering the broader multimodal ecosystem. |
| PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era (Read more on arXiv or HuggingFace) |
Zihao Dongfang, Kaiyu Lei, Ziqiao Weng, Chenfei Liao, Xu Zheng |
The paper “PANORAMA” presents a comprehensive overview of omnidirectional vision in embodied AI, outlining its rise, challenges, and future roadmap. Its primary objective is to address fundamental gaps in integrating panoramic visual technology with embodied intelligence by overcoming data bottlenecks, enhancing model capabilities, and exploring new application domains. The key methodology involves proposing PANORAMA, a four-subsystem architecture for data acquisition, perception, application, and acceleration, along with a six-stage roadmap for its implementation. The paper identifies 23 representative omnidirectional datasets and highlights advancements in generation, perception, and understanding, with domain adaptation techniques yielding significant performance improvements. This research implies that AI practitioners should prioritize creating large-scale multi-task omnidirectional datasets, developing projection-agnostic and unified models, and exploring real-world applications to advance embodied AI. |
| GenExam: A Multidisciplinary Text-to-Image Exam (Read more on arXiv or HuggingFace) |
Yu Qiao, Changyao Tian, Xiangyu Zhao, Penghao Yin, Zhaokai Wang |
GenExam introduces the first benchmark for multidisciplinary text-to-image exams. Its primary objective is to rigorously assess AI models’ ability to integrate understanding, reasoning, and generation in complex, exam-style graph-drawing problems, serving as a yardstick for general AGI development. The methodology involves 1,000 samples across 10 subjects, each with ground-truth images and fine-grained scoring points, evaluated by an MLLM-as-a-judge framework for semantic correctness and visual plausibility. Experiments show state-of-the-art models like GPT-Image-1 achieve a highest strict score of only 12.1%, with many others near 0%, highlighting the benchmark’s difficulty. For AI practitioners, this indicates a critical need to prioritize multidisciplinary knowledge integration, rigorous reasoning, and fine-grained visual coherence in developing advanced generative models. |
| Scrub It Out! Erasing Sensitive Memorization in Code Language Models via |
|
|
| Machine Unlearning (Read more on arXiv or HuggingFace) |
Zhou Yang, Di Wang, Zhikun Zhang, Yao Wan, Zhaoyang Chu |
This paper pioneers machine unlearning for erasing sensitive memorization in Code Language Models (CLMs). The main objective is to determine if sensitive information memorized by CLMs can be erased effectively and efficiently. The methodology introduces CODEERASER, an advanced gradient ascent-based unlearning variant that selectively unlearns sensitive memorized code segments via gradient ascent while preserving structural integrity and functional correctness of surrounding code through gradient descent and KL divergence-based constraints. Experiments on Qwen2.5-Coder-7B demonstrated CODEERASER reduced memorization by 93.89% on targeted data, retaining 99.00% of original model utility with an average processing time of 46.88 seconds per sample. This provides AI practitioners with a practical and efficient technique to actively mitigate data privacy risks stemming from sensitive memorization in CLMs. |
| THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Yicheng Pan, Jiefeng Ma, Pengfei Hu, Zhenrong Zhang, Qikai Chang |
THOR introduces a novel framework for Tool-Integrated Hierarchical Optimization via RL to address challenges in LLM mathematical reasoning, focusing on TIR data construction, fine-grained optimization, and inference enhancement. The methodology involves TIRGen, an actor-critic pipeline for generating high-quality TIR datasets, a hierarchical RL strategy for joint trajectory-level problem-solving and step-level code generation, and a self-correction mechanism leveraging immediate tool feedback during inference. Quantitatively, THOR-7B improves the average mathematical benchmark score for non-reasoning models from 35.7% to 61.2%, and THOR-Thinking-8B boosts reasoning models from 74.5% to 79.8% (Table 1). For AI practitioners, THOR offers a generalizable and efficient approach to build more robust LLMs with superior mathematical reasoning and code generation capabilities by effectively integrating external tools and hierarchical reinforcement learning. |
| Wan-Animate: Unified Character Animation and Replacement with Holistic |
|
|
| Replication (Read more on arXiv or HuggingFace) |
Mingyang Huang, Siqi Hu, Li Hu, Xin Gao, Gang Cheng |
Wan-Animate is a unified, state-of-the-art framework for high-fidelity character animation and replacement, enabling holistic replication of motion, expression, and environmental context. Its primary objective is to provide a comprehensive solution for character animation that unifies control over motion, expression, and seamless environment interaction. The methodology builds upon the DiT-based Wan-I2V model, employing a modified input paradigm, spatially-aligned skeleton signals for body motion, implicit facial features for expressions, and an auxiliary Relighting LoRA for environmental integration. Quantitatively, Wan-Animate achieves state-of-the-art performance, for instance, in facial animation with an FVD of 94.65, demonstrating superior realism and temporal coherence. For AI practitioners, this open-source framework provides a high-caliber model that establishes a new performance baseline, accelerating development and enabling diverse applications in video generation and character synthesis. |
| Improving Context Fidelity via Native Retrieval-Augmented Reasoning (Read more on arXiv or HuggingFace) |
Xiangru Tang, Shiqi Li, Xinyu Wang, Jinlin Wang, Suyuchen Wang |
This paper introduces CARE, a novel native retrieval-augmented reasoning framework that teaches large language models (LLMs) to explicitly integrate in-context evidence within their reasoning process, enhancing context fidelity. The core objective is to improve LLM context fidelity and reduce hallucinations in knowledge-intensive tasks by enabling models to dynamically identify and incorporate relevant input context evidence during reasoning. CARE employs a two-phase training process: supervised fine-tuning establishes evidence integration patterns using self-curated data with tokens, followed by reinforcement learning with Group Relative Policy Optimization (GRPO) and a curriculum learning strategy, using accuracy, format, and retrieval-aware rewards. Experiments demonstrate CARE consistently outperforms baselines, achieving a +15.29% average F1 improvement over the original LLaMA-3.1 8B model on real-world QA, with significant gains of +29.42% on 2WikiMQA and +18.92% on MuSiQue. This approach provides AI practitioners with a method to develop more accurate, reliable, and efficient LLM systems for knowledge-intensive tasks by leveraging native retrieval, reducing reliance on expensive external retrieval infrastructure and minimizing context hallucination. |
| SteeringControl: Holistic Evaluation of Alignment Steering in LLMs (Read more on arXiv or HuggingFace) |
Zhun Wang, Nathan W. Henry, David Park, Nicholas Crispino, Vincent Siu |
This paper introduces STEERINGCONTROL, a benchmark designed to holistically evaluate the effectiveness and behavioral entanglement of representation steering methods in LLMs. The research aims to systematically assess whether these methods can control primary alignment behaviors—bias, harmful generation, and hallucination—while minimizing unintended effects on secondary behaviors like sycophancy and reasoning. The methodology involves evaluating five popular training-free steering methods (DIM, ACE, CAA, PCA, LAT) on Qwen-2.5-7B and Llama-3.1-8B across 17 curated datasets, using aggregate metrics for “Effectiveness” and “Entanglement.” The primary results show that steering performance is highly dependent on the specific combination of method, model, and target behavior, and that a significant tradeoff exists between effectiveness and entanglement; for example, steering for refusal on Qwen-2.5-7B with the DIM method increased refusal ASR by 72.7% but simultaneously decreased performance on the sycophancy task by 55.5%. The principal implication for AI practitioners is that applying representation steering methods can introduce severe, unintended side effects, making it critical to perform comprehensive, multi-behavioral evaluations rather than optimizing for a single target, as no single steering method is universally optimal. |
| MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, |
|
|
| Results, Discussion, and Outlook (Read more on arXiv or HuggingFace) |
Bowen Zhou, Yaxiong Chen, Jiajun Zhang, Shengwu Xiong, Peng Xu |
This paper reports on the MARS2 2025 Challenge, which benchmarked multimodal reasoning capabilities of large language models using new, specialized datasets (Lens, AdsQA) across three complex reasoning tracks. The challenge’s objective was to evaluate synergistic effects among reasoning tasks and probe non-stepwise complex reasoning, pushing MLLMs beyond standard perception into specialized, real-world scenarios like spatial awareness and advertisement analysis. The methodology involved three competition tracks evaluated with task-specific metrics, where top-performing participants primarily employed multi-stage alignment strategies combining supervised fine-tuning (SFT) and reinforcement learning (e.g., GRPO) on foundational models. The results reveal significant remaining challenges in multimodal reasoning, with the winning solution for the Visual Grounding in Real-world Scenarios (VG-RS) track achieving a 66.70% accuracy (Acc.@0.5), indicating a substantial performance gap. For AI practitioners, the key implication is that deploying MLLMs for specialized, high-fidelity reasoning tasks necessitates significant investment in domain-specific data synthesis, targeted fine-tuning, and advanced alignment techniques, as general-purpose models are insufficient for these complex applications. |
Papers for 2025-09-17
| Title |
Authors |
Summary |
| WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for |
|
|
| Open-Ended Deep Research (Read more on arXiv or HuggingFace) |
Houquan Zhou, Shen Huang, Bo Zhang, Xin Guan, Zijian Li |
WebWeaver is a dual-agent AI framework designed for open-ended deep research that dynamically refines a report outline while gathering web-scale evidence. The primary objective is to overcome the failures of static research pipelines and one-shot generation methods, such as “loss in the middle” and hallucinations in long-context scenarios. The key methodology involves a “planner” agent that iteratively interleaves evidence acquisition with outline optimization to create a source-grounded plan, and a “writer” agent that performs hierarchical, section-by-section synthesis by retrieving only necessary evidence from a memory bank for each part. WebWeaver establishes a new state-of-the-art across major benchmarks, achieving a 93.37% citation accuracy on DeepResearch Bench. The principal implication for AI practitioners is that an iterative, dual-agent architecture separating dynamic planning from focused, memory-grounded synthesis is a superior strategy for complex, long-form generation tasks, with the provided WebWeaver-3k dataset demonstrating these skills can be finetuned into smaller models. |
| Scaling Agents via Continual Pre-training (Read more on arXiv or HuggingFace) |
Chenxi Wang, Zhuo Chen, Guangyu Li, Zhen Zhang, Liangcai Su |
This paper introduces Agentic Continual Pre-training (Agentic CPT), an intermediate training stage designed to create a pre-aligned agentic foundation model, named AgentFounder, to improve agent capabilities before downstream fine-tuning. The main objective is to determine if embedding agentic behaviors directly into a foundation model through a scalable, offline data synthesis pipeline is more effective than relying solely on post-training methods like SFT or RL. The methodology involves a two-stage CPT process using two novel data synthesis techniques: First-order Action Synthesis (FAS) to create planning and reasoning data without API calls, and Higher-order Action Synthesis (HAS) to remodel existing trajectories into multi-step decision-making problems. The resulting model, AgentFounder-30B, achieves state-of-the-art performance, scoring 39.9% on BrowseComp-en, which is a significant improvement over the prior open-source best of 30.0%. For AI practitioners, the principal implication is that developing a specialized agentic base model via CPT is a more efficient and powerful strategy for building high-capability agents, as it facilitates easier downstream alignment and achieves higher performance ceilings compared to fine-tuning general-purpose foundation models. |
| WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic |
|
|
| Data and Scalable Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yida Zhao, Rui Ye, Huifeng Yin, Zhongwang Zhang, Kuan Li |
This paper presents WebSailor-V2, a complete post-training pipeline that uses novel synthetic data and scalable reinforcement learning (RL) to create high-performance open-source web agents. The primary objective is to bridge the performance gap between open-source agents and proprietary systems by tackling insufficiencies in data diversity and the instability of RL training environments. The core methodology involves two innovations: (1) SailorFog-QA-2, a new dataset constructed from a dense knowledge graph to generate tasks with complex uncertainties, and (2) a dual-environment RL framework that combines a high-fidelity simulator for rapid iteration with a managed real-world environment for stable policy training. The resulting WebSailor-V2 agent, built on a Qwen3-30B-A3B model, achieves a state-of-the-art score of 35.3 on BrowseComp-EN, outperforming even the much larger 671B DeepSeek-V3.1. The principal implication for AI practitioners is that investing in sophisticated synthetic data generation and a stable, robust training infrastructure is more critical for developing capable agents than focusing on model scale or specific RL algorithm choice. |
| Towards General Agentic Intelligence via Environment Scaling (Read more on arXiv or HuggingFace) |
Guangyu Li, Jialong Wu, Baixuan Li, Shihao Cai, Runnan Fang |
This paper presents a scalable pipeline for developing general agentic intelligence by automatically constructing and scaling diverse, verifiable tool-use environments. The primary objective is to create a systematic framework for environment generation and agent training to overcome the limitations of manual or non-scalable data collection methods. The methodology involves programmatically materializing over 30,000 APIs into executable tools grounded in database-structured environments, generating agentic tasks via simulated human-agent interplay, and employing a two-stage agent experience learning process for fine-tuning. The trained AgentScaler-30B-A3B model achieves state-of-the-art results among open-source models under 1T parameters, attaining an overall accuracy of 67.7% on the ACEBench-en benchmark. The principal implication for AI practitioners is the provision of a fully simulated, verifiable, and scalable pipeline for generating high-quality agent training data, which facilitates the development of robust tool-using agents with more compact models. |
| WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon |
|
|
| Agents (Read more on arXiv or HuggingFace) |
Wenbiao Yin, Donglei Yu, Xuanzhong Chen, Guoxin Chen, Zile Qiao |
This paper introduces WebResearcher, a framework for deep-research AI agents that employs an iterative, state-reconstructing paradigm called IterResearch to overcome the reasoning limitations of linear context accumulation. The primary objective is to enable sustained long-horizon reasoning by mitigating the “context suffocation” and “noise contamination” that degrade performance in mono-contextual agent architectures. The methodology models research as a Markov Decision Process where the agent’s workspace is periodically reconstructed from a synthesized report of findings, and the agent is trained on data generated by WebFrontier, a scalable multi-agent data synthesis engine. The system achieves state-of-the-art performance, scoring 36.7% on the Humanity’s Last Exam (HLE) benchmark, significantly surpassing prior systems like OpenAI Deep Research (26.6%). For AI practitioners, this demonstrates that for complex, long-horizon tasks, an iterative synthesis and state-reconstruction architecture is superior to linear context accumulation, providing a robust pattern for building more capable agents by preventing cognitive overload and error propagation. |
| ReSum: Unlocking Long-Horizon Search Intelligence via Context |
|
|
| Summarization (Read more on arXiv or HuggingFace) |
Litu Ou, Liwen Zhang, Yida Zhao, Kuan Li, Xixi Wu |
The paper introduces ReSum, a paradigm using periodic context summarization to enable LLM-based web agents to handle long-horizon tasks that exceed standard context window limits. The primary objective is to overcome fixed context window constraints in complex search problems requiring extensive exploration, without significant architectural changes. The key methodology involves periodically invoking a specialized summary tool (ReSumTool-30B) to condense the interaction history into a compact state, from which the agent resumes reasoning, with a tailored reinforcement learning algorithm (ReSum-GRPO) for paradigm adaptation. Primary results show that the ReSum-GRPO trained WebResummer-30B model achieves 33.3% Pass@1 on the BrowseComp-zh benchmark, surpassing existing open-source web agents with only 1K training samples. For AI practitioners, ReSum offers a lightweight, plug-and-play modification to existing ReAct-based agents to mitigate context overflow failures and improve performance on complex, multi-step tasks. |
| Single-stream Policy Optimization (Read more on arXiv or HuggingFace) |
Zihan Ding, Zhongwen Xu |
This paper introduces Single-stream Policy Optimization (SPO), a group-free policy gradient method designed to improve the efficiency and scalability of fine-tuning Large Language Models (LLMs) with reinforcement learning. The research objective is to address the critical flaws of group-based methods like GRPO, namely computational waste from degenerate learning signals and synchronization bottlenecks in distributed training. SPO’s methodology replaces on-the-fly, per-group baselines with three components: a persistent, KL-adaptive Bayesian value tracker for low-variance baseline estimation, global advantage normalization across the batch, and prioritized prompt sampling for an adaptive curriculum. The primary result shows that when training a Qwen3-8B model, SPO improves the average maj@32 score by +3.4 percentage points over GRPO across five math benchmarks, and simulations indicate it can achieve a 4.35× training throughput speedup in agentic settings. The principal implication for AI practitioners is that SPO offers a more robust, scalable, and efficient alternative to group-based RL, simplifying the training infrastructure and accelerating convergence, especially for agentic tasks with variable-length trajectories. |
| Hunyuan3D Studio: End-to-End AI Pipeline for Game-Ready 3D Asset |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Lixin Xu, Shuhui Yang, Xinhai Liu, Yang Li, Biwen Lei |
Hunyuan3D Studio is an end-to-end AI pipeline that automates the generation of game-ready 3D assets, including optimized geometry, PBR textures, and animation, from a single image or text description. The objective is to automate the traditionally labor-intensive 3D asset creation workflow by integrating a suite of advanced neural modules into a single, cohesive system. The methodology consists of a seven-stage sequential pipeline that includes modules for controllable image generation, diffusion-based geometry synthesis, part-level decomposition (X-Part), autoregressive polygon generation (PolyGen), and semantic UV unwrapping (SeamGPT). The proposed shape decomposition module, X-Part, achieves a state-of-the-art Chamfer Distance (CD) of 0.11 and an F-score of 0.80 on the ObjaversePart-Tiny benchmark, outperforming all baselines. For AI practitioners, the principal implication is the demonstration of a unified framework that integrates multiple specialized generative models to automate a complex, multi-stage production pipeline, providing a seamless bridge from creative intent to a final technical asset. |
| 3D Aware Region Prompted Vision Language Model (Read more on arXiv or HuggingFace) |
Xiaolong Li, Zhijian Liu, Yukang Chen, Yang Fu, An-Chieh Cheng |
SR-3D is a vision-language model that enables 3D-aware spatial reasoning by unifying 2D and multi-view data through shared visual tokens enriched with 3D positional embeddings. The research aims to develop a unified VLM capable of accurate 3D spatial reasoning from flexible, sparse region prompts by leveraging strong, pre-existing 2D foundational model priors. Its core methodology involves integrating canonicalized 3D positional embeddings, derived from depth maps, into visual features and using a dynamic “tile-then-stitch” region extractor for high-resolution analysis across single- or multi-view inputs. The model demonstrates state-of-the-art performance, achieving 90.3% accuracy on the BLINKDepth benchmark for point-level depth understanding and showing strong zero-shot generalization from 2D pre-training to 3D tasks. For AI practitioners, the principal implication is the ability to develop systems with sophisticated 3D spatial awareness using only sparse, single-frame annotations, drastically reducing the annotation burden for applications in robotics and scene understanding. |
| EconProver: Towards More Economical Test-Time Scaling for Automated |
|
|
| Theorem Proving (Read more on arXiv or HuggingFace) |
Shansan Gong, Jiahao Xu, Zhenwen Liang, Linfeng Song, Mukai Li |
The paper introduces ECONPROVER, a framework to reduce the computational cost of LLM-based automated theorem provers by dynamically applying Chain-of-Thought reasoning and diversifying parallel proof attempts. The research objective is to improve the test-time computational efficiency of state-of-the-art automated theorem proving (ATP) models by mitigating high token costs from scaling strategies without performance loss. The EconRL methodology integrates Dynamic CoT Switching, trained via Direct Preference Optimization (DPO) to selectively apply complex reasoning, and Diverse Parallel-scaled Reinforcement Learning, which uses PPO to train specialized, difficulty-aware reasoning heads to increase proof diversity. Experiments on the miniF2F benchmark show ECONPROVER-GD achieves an 84.0% pass rate, comparable to its baseline’s 84.4%, while consuming only 12% of the total sampling token cost. For AI practitioners, this work provides a validated approach to deploy more economical and computationally efficient ATP systems by implementing dynamic reasoning allocation and diversity-focused parallel sampling, enabling high-performance models in resource-constrained environments. |
| Exact Coset Sampling for Quantum Lattice Algorithms (Read more on arXiv or HuggingFace) |
Yifan Zhang |
This paper presents a fully correct “pair-shift difference” construction to replace a contested step in a recent windowed-QFT quantum algorithm for lattice problems. The primary objective is to resolve a periodicity and support mismatch in the original algorithm’s Step 9, ensuring the correct generation of a uniform random vector u that satisfies the modular linear relation ⟨b*, u⟩ = 0 (mod P). The key methodology involves creating a coherent, shifted copy of the quantum state, subtracting it from the original to deterministically cancel unknown offset vectors, and performing a mandatory ancilla cleanup to produce an exact uniform superposition over a CRT-coset. The procedure results in measurement outcomes that are exactly and uniformly distributed over the desired solution set, which contains M₂ⁿ / P elements, while preserving the algorithm’s overall poly(log M₂) gate complexity. For practitioners developing quantum algorithms, this work provides a robust, reversible, and provably correct subroutine that fixes a critical flaw, offering a broadly applicable technique for handling unknown offsets in quantum signal processing pipelines. |
| Multimodal Reasoning for Science: Technical Report and 1st Place |
|
|
| Solution to the ICML 2025 SeePhys Challenge (Read more on arXiv or HuggingFace) |
Wentao Zhang, Junbo Niu, Bohan Zeng, Ruitao Wu, Hao Liang |
The paper introduces a caption-assisted reasoning framework that converts visual information from scientific diagrams into structured text to improve multimodal problem-solving. The primary objective is to mitigate the performance degradation that powerful reasoning models exhibit in multimodal scenarios by bridging the gap between visual perception and textual logic. The key methodology is a pipeline where a vision-language model first generates a detailed, structured caption from an image, which is then used by a large language model for reasoning, optionally followed by format optimization and critical review stages. The method achieved 1st place in the ICML 2025 SeePhys Challenge, improving accuracy on the SeePhys-mini benchmark from a 58.0% baseline to 66.0%, and on the MathVerse benchmark, it boosted the accuracy of the Claude-Opus-4 model from 60.2% to 85.5% on vision-intensive tasks. The principal implication for AI practitioners is that for problems involving diagrams, decoupling visual perception from reasoning via a captioning module can be more effective than using end-to-end multimodal models, as it allows specialized, powerful text-only LLMs to handle the complex reasoning phase more robustly. |
| Multiple Instance Learning Framework with Masked Hard Instance Mining |
|
|
| for Gigapixel Histopathology Image Analysis (Read more on arXiv or HuggingFace) |
Bo Liu, Fengtao Zhou, Heng Fang, Sheng Huang, Wenhao Tang |
This paper presents MHIM-MIL, a Multiple Instance Learning (MIL) framework that improves gigapixel histopathology image analysis by mining hard instances through a masked, momentum-teacher approach. The research objective is to enhance MIL models by shifting the training focus from easy-to-classify salient instances to more challenging hard instances to learn better decision boundaries. The key methodology employs a Siamese architecture where a momentum teacher calculates class-aware instance probabilities to mask easy instances, thereby forcing a student model to train on the remaining hard instances, stabilized by a consistency loss and a Global Recycle Network to mitigate feature loss. The framework demonstrates superior performance across multiple tasks, with MHIM (TransMIL) improving the C-index by 1.8% over the baseline on the TCGA-BLCA-UNI survival analysis task while reducing training time and memory by 20% and 50%, respectively. For AI practitioners, this work provides an effective strategy for implementing hard instance mining in weakly-supervised MIL settings, demonstrating that masking easy instances can improve model generalization and efficiency compared to conventional attention mechanisms. |
| Optimal Brain Restoration for Joint Quantization and Sparsification of |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Luca Benini, Yawei Li, Hang Guo |
The paper introduces Optimal Brain Restoration (OBR), a training-free framework that enables joint quantization and sparsification of LLMs by computing a closed-form compensation to reconcile their conflicting weight distribution requirements. The primary objective is to combine aggressive low-bit quantization (e.g., 4-bit) with high-ratio sparsity (e.g., 50%) for LLMs, overcoming the inherent conflict where quantization favors flat distributions and pruning prefers high-variance ones. OBR formulates a second-order Hessian-based objective to minimize task degradation, which is made tractable through row-wise decoupling and solved in a closed form via group error compensation, systematically redistributing errors from pruning and quantization to more robust weights. The method enables W4A4KV4 quantization with 50% sparsity on large models, achieving up to a 4.72× inference speedup and 6.4× memory reduction compared to an FP16-dense baseline by leveraging INT4-sparse hardware support. For AI practitioners, OBR offers a direct, post-training method to highly compress existing LLMs for efficient deployment on modern GPUs with sparse tensor cores, significantly reducing latency and memory footprint without any model retraining. |
Papers for 2025-09-16
| Title |
Authors |
Summary |
| OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling (Read more on arXiv or HuggingFace) |
Yang Zhou, MingyuLiu, Xxxy13, lizizun, ZhouTimeMachine |
This paper introduces OmniWorld, a large-scale, multi-domain, multi-modal dataset with over 300 million frames designed to advance 4D world modeling by providing rich geometric and temporal annotations. The primary objective is to address the data scarcity for training and evaluating general 4D world models by creating a comprehensive resource that surpasses existing datasets in scale, modality coverage, and dynamic complexity. The methodology involves collecting a new synthetic dataset, OmniWorld-Game, from diverse game environments and developing an extensive pipeline to annotate it and several curated public datasets with high-quality depth, camera poses, text captions, optical flow, and foreground masks. The paper establishes benchmarks that expose limitations in current models and demonstrates that fine-tuning with OmniWorld yields significant performance gains; for example, fine-tuning the AC3D video generation model reduces its camera translation error on the OmniWorld-Game benchmark from 6.2788 to 4.1428. For AI practitioners, OmniWorld serves as a powerful training and evaluation resource for developing more robust models for tasks requiring complex spatio-temporal understanding, enabling direct performance improvements in 3D geometric reconstruction and camera-controlled video generation through fine-tuning. |
| UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yongliang Shen, Fei Tang, xhyandwyy, Mizukiluke, LZXzju |
The paper introduces Semi-online Reinforcement Learning, a paradigm that simulates online RL on static offline trajectories to improve multi-turn reasoning for GUI automation agents. The primary objective is to overcome the limitations of traditional offline RL (poor multi-step performance) and online RL (high cost and sparse rewards) by enabling long-horizon optimization using only pre-collected data. The core methodology involves generating rollouts that maintain the agent’s history, using a “Patch Module” to recover from action divergences by injecting expert actions, and optimizing the policy with weighted step-level and episode-level advantages derived from discounted future rewards. The resulting UI-S1-7B model achieves state-of-the-art performance, with significant gains over its base model, including a +12.0% success rate on the AndroidWorld benchmark. For AI practitioners, this framework provides a practical method to train capable multi-turn agents on static datasets, effectively bridging the gap between offline training efficiency and online execution robustness without requiring costly live environment interaction. |
| InternScenes: A Large-scale Simulatable Indoor Scene Dataset with |
|
|
| Realistic Layouts (Read more on arXiv or HuggingFace) |
Wenzhe Cai, Li Luo, Yichen Jin, Peizhou Cao, Weipeng Zhong |
InternScenes is a large-scale, simulatable 3D indoor dataset of approximately 40,000 scenes with realistic object layouts, created by integrating real-world scans, procedurally generated scenes, and designer-created content. The objective is to create a diverse and complex dataset for training and benchmarking Embodied AI and 3D AIGC models, addressing the limitations of existing datasets in scale, layout realism, and simulatability. The methodology involves a multi-source data processing pipeline that performs real-to-sim replication, enriches scenes with interactive objects, and uses physics simulation (SAPIEN) to resolve object collisions and ensure physical plausibility. In point-goal navigation benchmarks, a state-of-the-art method like NavDP achieves only a 48.3% success rate on the most realistic subset, indicating the high difficulty of the scenes. For AI practitioners, this dataset serves as a challenging new benchmark to test the robustness of navigation and scene generation models, revealing that current methods struggle significantly with cluttered, realistic environments and require advancements in handling complex object interactions and spatial reasoning. |
| LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion |
|
|
| Transformers via Explicit Correspondence (Read more on arXiv or HuggingFace) |
Lionel M. Ni, Xianfang Zeng, Xili Dai, Zixin Yin, dorni |
LazyDrag is a training-free method for drag-based editing in Multi-Modal Diffusion Transformers that uses an explicit correspondence map to achieve stable edits under full-strength inversion without test-time optimization. The primary objective is to eliminate the instability of drag-based editing caused by implicit attention-based point matching by introducing an explicit correspondence mechanism that enables robust, high-fidelity geometric and semantic control. The methodology involves generating an explicit correspondence map from user drag inputs via Voronoi partitioning, which then drives a dual attention control mechanism: hard token replacement for background preservation and token concatenation with gated output blending for identity-preserving edits in dragged regions. LazyDrag achieves state-of-the-art performance on the DragBench benchmark, outperforming all baselines with a mean distance (MD) of 21.49 ± 0.04 and securing a 61.88% preference rate in a human evaluation study. For AI practitioners, this paper provides a robust, optimization-free framework that replaces fragile implicit point matching with a deterministic correspondence map, enabling stable and predictable high-fidelity interactive image editing that integrates precise spatial control with text guidance. |
| Locality in Image Diffusion Models Emerges from Data Statistics (Read more on arXiv or HuggingFace) |
Vincent Sitzmann, Justin Solomon, Chenyang Yuan, Artem Lukoianov |
This paper demonstrates that the locality property in image diffusion models is not an architectural inductive bias but an emergent statistical property of the training dataset’s pixel correlations. The research objective is to show that this generalization behavior is derived directly from the second-order statistics of the training data, rather than from architectural constraints like those in convolutional networks. The key methodology involves theoretically and empirically linking the learned sensitivity fields of diffusion models (both U-Nets and Transformers) to the Wiener filter, which acts as a projection operator onto the high Signal-to-Noise Ratio (SNR) principal components of the data’s covariance matrix. The primary result is a new analytical denoiser, based on these data statistics, that better explains the predictions of a trained deep model than prior analytical approaches, achieving a coefficient of determination (r²) of 0.902 on CelebA-HQ compared to 0.795 for the previous best analytical model. For AI practitioners, the principal implication is that the statistical structure of the training data, specifically pixel covariance, is a primary determinant of a diffusion model’s learned behavior and generalization patterns, offering a direct lever for model control that is complementary to architectural design. |
| Measuring Epistemic Humility in Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Kaiyang Zhou, Sifeng Shang, Bingkui Tong, JiaerX |
This paper introduces HumbleBench, a benchmark to evaluate the epistemic humility of Multimodal Large Language Models (MLLMs) by testing their ability to reject false-option choices. The research aims to measure whether MLLMs can identify when none of the provided multiple-choice answers are correct and select a “None of the above” (NOTA) option. The methodology involves creating a dataset of 22,831 questions from the Panoptic Scene Graph dataset, using GPT-4-Turbo for generation and manual filtering, where each question includes a NOTA option. The primary result from evaluating 19 state-of-the-art MLLMs is that even the best-performing model, GLM-4.1V-Thinking, achieved only 73.46% accuracy, and in a stress test where NOTA was always the correct answer, most models failed catastrophically, often scoring below the random guess baseline. The principal implication for AI practitioners is that current MLLMs exhibit significant overconfidence and are unreliable in scenarios requiring abstention from incorrect answers, making standard accuracy metrics insufficient for assessing their suitability for safety-critical applications. |
| Lost in Embeddings: Information Loss in Vision-Language Models (Read more on arXiv or HuggingFace) |
Ivan Vulić, Caiqi Zhang, Chengzu Li, Raphael Tang, lyan62 |
This paper introduces a framework to quantify information loss in the connector module of Vision-Language Models (VLMs) and correlates this loss with downstream task performance. The research objective is to measure the distortion of visual information when connectors project visual embeddings into the language model’s space. The methodology employs two approaches: the k-Nearest Neighbors Overlap Ratio (KNOR) to evaluate geometric distortion and patch-level embedding reconstruction to localize information loss. Results demonstrate that connectors cause a 40-60% divergence in k-nearest neighbor relationships post-projection, and high patch-level reconstruction loss in answer-relevant regions negatively correlates with accuracy on visually-grounded VQA tasks. The principal implication for AI practitioners is that the connector acts as a significant information bottleneck, and the proposed reconstruction method provides an interpretable tool for debugging VLM failures by localizing where critical visual details are lost. |
| CognitiveSky: Scalable Sentiment and Narrative Analysis for |
|
|
| Decentralized Social Media (Read more on arXiv or HuggingFace) |
Subasish Das, Anandi Dutta, gauravfs-14 |
This paper introduces CognitiveSky, an open-source framework for scalable, real-time sentiment, emotion, and narrative analysis on the decentralized social media platform Bluesky. The primary objective is to develop a transparent, low-cost tool for computational social science that overcomes the data access limitations of centralized platforms like X.com. The methodology utilizes a Node.js worker for data ingestion from the Bluesky Firehose API, a CI-automated Python pipeline with transformer-based models (RoBERTa for sentiment, DistilRoBERTa for emotion) for annotation, and MiniBatch NMF on TF-IDF vectors for topic modeling. In a mental health discourse use case analyzing 58,567 posts, the system identified “Fear” as the most frequent emotion, accounting for 31.3% of all detected emotions. The principal implication for AI practitioners is the blueprint for a fully automated, reproducible, and serverless architecture built entirely on free-tier infrastructure, enabling the deployment of real-time NLP analysis pipelines on decentralized data streams without significant operational cost. |
| Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Shuo Ren, Chen Wang, Wei Sun, Junhong Wu, Pu Jian |
This paper presents Reflection-V, a training strategy for Vision-Language Models (VLMs) that enhances visual reflection to improve multi-step reasoning by maintaining sustained attention on visual inputs. The primary objective is to address the limitation of existing VLMs, which exhibit rapidly diminishing attention to visual information during long reasoning chains, thereby failing to perform effective visual grounding. The key methodology is a two-stage process: first, a cold-start initialization using reasoning data generated via an interactive LLM-VLM agent framework to embed reflective patterns; second, reinforcement learning with a novel visual attention-based reward to encourage sustained focus on visual tokens. The resulting Reflection-V-7B model achieves state-of-the-art performance, scoring 73.3 on MathVista and 71.1 on M3CoT, significantly outperforming its base model and larger models like InternVL-2.5-38B. For AI practitioners, this research provides a concrete methodology to mitigate visual neglect and reduce hallucinations in VLMs, demonstrating that explicitly training for and rewarding sustained visual attention is critical for building more accurate and reliable visual reasoning systems. |
| Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose |
|
|
| Video Hallucination by Fine-grained Spatial-Temporal Grounding (Read more on arXiv or HuggingFace) |
Li Zheng, Tianjie Ju, Liqiang Jing, Shengqiong Wu, Meng Luo |
This paper introduces Dr.V, a framework to diagnose and mitigate video hallucinations in LVMs using a hierarchical, tool-augmented reasoning process grounded in spatial-temporal evidence. The primary objective is to create a comprehensive system for systematically verifying model outputs against video content by first establishing a fine-grained, three-level (perceptive, temporal, cognitive) taxonomy of hallucinations. The methodology consists of Dr.V-Bench, a 10k-instance benchmark with detailed spatial-temporal annotations across 14 hallucination types, and Dr.V-Agent, a training-free system that uses an LLM to dynamically invoke specialized external tools for perceptive and temporal grounding to validate an LVM’s claims. Experiments show Dr.V-Agent substantially mitigates hallucinations, improving the accuracy of the Qwen2-VL model by +9.97% absolute (from 72.67% to 82.64%) on the benchmark. For AI practitioners, this work demonstrates that a modular, agentic approach leveraging specialized tools for evidence verification is a highly effective, training-free strategy to improve the factual grounding and reliability of generative video models. |
| EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AI (Read more on arXiv or HuggingFace) |
UVSKKR |
The paper introduces EthicsMH, a pilot benchmark dataset designed to evaluate the ethical reasoning of AI systems in mental health contexts. The primary objective is to create a resource that captures the unique ethical dilemmas in mental health practice, which are inadequately addressed by existing benchmarks. The dataset was constructed via a human-in-the-loop process where an LLM generated initial scenarios that were then iteratively reviewed and validated by a mental health professional for clinical plausibility and ethical nuance. The primary result is the EthicsMH dataset, which contains 125 scenarios evenly distributed across five subcategories, with each scenario featuring structured fields for decision options, expert-aligned reasoning, and multi-stakeholder viewpoints. The principal implication for AI practitioners is that EthicsMH provides a concrete resource for pre-deployment stress-testing, diagnostic evaluation of model tendencies, and prototyping safeguards for AI systems intended for sensitive mental health applications. |
| Learning to Optimize Multi-Objective Alignment Through Dynamic Reward |
|
|
| Weighting (Read more on arXiv or HuggingFace) |
Changlong Yu, Xin Liu, Shiyang Li, Zilong Wang, ylu610 |
This paper introduces dynamic reward weighting methods to optimize multi-objective LLM alignment by adaptively adjusting objective importance during online reinforcement learning. The main objective is to overcome the limitations of fixed-weight linear scalarization, which fails to capture non-convex Pareto fronts, by dynamically reallocating learning effort toward objectives with the greatest potential for improvement. The authors propose two key methodologies: (1) hypervolume-guided weight adaptation, which uses a meta-reward to encourage the discovery of new Pareto-optimal solutions based on user preferences, and (2) gradient-based weight optimization, which automatically reallocates weights by computing an objective’s influence on the overall training process. The primary result is that the proposed dynamic methods consistently achieve Pareto-dominant solutions with greater training efficiency than fixed-weight baselines; for instance, the gradient-based method reduced the average number of steps to reach the Pareto front by 6.1 across all tested RL algorithms. The principal implication for AI practitioners is that these dynamic weighting techniques, especially the gradient-based approach, can be implemented in multi-objective RLHF pipelines to achieve superior trade-offs between competing objectives like accuracy and conciseness while reducing training steps, eliminating the need for manual weight tuning. |
| PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits (Read more on arXiv or HuggingFace) |
Zhenhao Chen, Guangyi Chen, Minghao Fu, Wong Yu Kang, Loka Li |
This paper introduces PersonaX, a pair of public multimodal datasets linking LLM-inferred Big Five behavioral traits with facial and biographical data for comprehensive trait analysis. The objective is to facilitate large-scale analysis of human traits across modalities and to develop a framework for learning their underlying causal structures from both structured and unstructured data. Using a novel causal representation learning (CRL) framework with theoretical identifiability guarantees, the method achieved an R² of 0.96 on synthetic data, outperforming baselines, while statistical tests on the datasets revealed population-specific dependencies between traits and attributes like occupation or sports league. For AI practitioners, this work provides curated benchmarks for cross-modal causal discovery and demonstrates that simpler, 3-level numeric prompts yield the most consistent behavioral trait inferences from large language models. |
| GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings (Read more on arXiv or HuggingFace) |
Yixuan Tang, yixuantt |
The paper presents GAPrune, a novel pruning framework for creating efficient, domain-aware embedding models by balancing domain-specific importance with general linguistic alignment. The main objective is to develop a compression technique for large embedding models that preserves or enhances specialized domain performance while significantly reducing model size. GAPrune characterizes each model parameter using two signals: Fisher Information to quantify its importance for domain and general tasks, and gradient cosine similarity to measure the alignment between domain-specific and general objectives, combining them into a Domain-Alignment Importance (DAI) score for pruning. Experiments show that with 100 steps of retraining at 50% sparsity, GAPrune improves the performance of the Qwen3-Embedding-4B model over its dense counterpart by +4.51% on the FinMTEB benchmark and +1.73% on ChemTEB. The principal implication for AI practitioners is that GAPrune provides a method to compress large embedding models into smaller, domain-specialized versions that are not only more efficient but can also achieve superior performance on target domain tasks after minimal retraining. |
Papers for 2025-09-15
| Title |
Authors |
Summary |
| IntrEx: A Dataset for Modeling Engagement in Educational Conversations (Read more on arXiv or HuggingFace) |
Gabriele Pergola, Chiara Gambi, Mahathi Parvatham, XingweiT |
This paper introduces IntrEx, a dataset of teacher-student conversations annotated for interestingness to model conversational engagement. The objective is to identify linguistic drivers of engagement and evaluate if Large Language Models (LLMs) can align with human interestingness judgments. The methodology involved collecting sequence-level interestingness ratings from over 100 second-language learners using a comparison-based annotation framework inspired by Reinforcement Learning from Human Feedback (RLHF). The primary result is that 7B/8B parameter models (Mistral-7B and Llama3-8B) fine-tuned on IntrEx achieved a Gwet’s AC2 agreement with human ratings of approximately 0.514, outperforming the 0.4657 score of the much larger GPT-4o. The principal implication for AI practitioners is that high-quality, domain-specific datasets can enable smaller, fine-tuned models to surpass larger, general-purpose models in predicting nuanced, subjective human preferences, providing an efficient approach for building specialized reward models. |
| The Illusion of Diminishing Returns: Measuring Long Horizon Execution in |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Jonas Geiping, Steffen Staab, Shashwat Goel, arvindh75, viciousa3gis |
This paper demonstrates that marginal improvements in single-step accuracy can compound into exponential gains in the length of tasks LLMs can execute. The primary research objective is to isolate and measure the long-horizon execution capability of LLMs, distinct from reasoning or planning, and to diagnose failure modes on long but simple tasks. The key methodology involves a synthetic key-value dictionary addition task where the knowledge (dictionary) and plan (keys to look up) are provided in-context, forcing the model to only perform sequential retrieval and composition. The primary results show that per-step accuracy degrades over time due to a “self-conditioning” effect, where models become more likely to err after observing their own past mistakes; for example, Qwen3-32B’s accuracy falls below 50% within 15 turns despite perfect single-step accuracy. The principal implication for AI practitioners is that for long-horizon tasks, enabling sequential test-time compute (“thinking”) is critical, as it eliminates the self-conditioning effect and dramatically increases execution length, whereas simply scaling model size does not. |
| X-Part: high fidelity and structure coherent shape decomposition (Read more on arXiv or HuggingFace) |
Yunhan Yang, Changfeng Ma, Yang Li, Jiachen Xu, HowieYan |
X-Part introduces a controllable diffusion-based model for decomposing holistic 3D objects into high-fidelity, structurally coherent parts. The primary objective is to create a generative framework that provides precise control over part decomposition while ensuring semantic meaning and geometric quality, addressing the limitations of existing segmentation-sensitive or uncontrollable methods. The methodology utilizes a multi-part Diffusion Transformer (DiT) conditioned by bounding box prompts for spatial guidance and injected point-wise semantic features from a P3-SAM segmenter to guide semantically accurate decomposition. The model achieves state-of-the-art performance, demonstrating a Chamfer Distance of 0.11 and an F-score of 0.80 (at threshold 0.1) on the ObjaversePart-Tiny benchmark for part decomposition. For AI practitioners, this work establishes an editable pipeline where 3D assets can be interactively decomposed by manipulating bounding boxes, significantly simplifying downstream tasks like UV mapping and retopology in production environments. |
| InfGen: A Resolution-Agnostic Paradigm for Scalable Image Synthesis (Read more on arXiv or HuggingFace) |
Song Guo, Xiaoyu Yue, Junchao Gong, Wanghan Xu, Tao Han |
InfGen is a resolution-agnostic generator that replaces the VAE decoder in latent diffusion models to enable efficient, arbitrary high-resolution image synthesis from a fixed-size latent. The main objective is to overcome the quadratic computational scaling and slow inference speeds of current diffusion models when generating high-resolution images. The key methodology involves a transformer-based generator that uses an Implicit Neural Positional Embedding (INPE) to decode a fixed content latent into a variable-resolution image, with a training-free iterative extrapolation process for scaling beyond trained resolutions. Primary results show that InfGen reduces 4K image generation time to under 10 seconds, achieving a 10x speed improvement over prior methods, and improves the FIDp of DiT by 41% at 3072x3072 resolution. The principal implication for AI practitioners is that InfGen acts as a plug-and-play module to upgrade existing diffusion models for arbitrary high-resolution generation without the need for costly retraining of the foundational model. |
| HANRAG: Heuristic Accurate Noise-resistant Retrieval-Augmented |
|
|
| Generation for Multi-hop Question Answering (Read more on arXiv or HuggingFace) |
Zhehao Tan, Yihan Jiao, Yue Shen, Dan Yang, Duolin Sun |
The paper introduces HANRAG, a heuristic framework using a central “Revelator” agent to improve multi-hop Retrieval-Augmented Generation by routing queries, decomposing them, and filtering noise. The primary objective is to overcome key RAG challenges in multi-hop question answering, such as the inefficiency of iterative retrieval, irrational querying, and noise accumulation. The key methodology is a master agent, the Revelator, which classifies queries into four types (straightforward, single-step, compound, complex), decomposes compound queries for parallel retrieval, refines sub-questions for complex queries, and uses a relevance discriminator to filter noisy documents. On a custom multi-hop compound query benchmark, HANRAG achieved an accuracy of 71.76%, a 19.63 percentage point improvement over the Adaptive-RAG baseline, while reducing average retrieval steps from 2.76 to 1.24. For AI practitioners, the principal implication is that implementing a versatile, heuristic-based agent to pre-process and route queries based on complexity can significantly enhance the accuracy and efficiency of RAG systems, providing a more robust solution for handling diverse real-world questions. |
| VStyle: A Benchmark for Voice Style Adaptation with Spoken Instructions (Read more on arXiv or HuggingFace) |
Dong Zhang, Chen Wang, Yuxuan Xie, Mingyang Han, Jun Zhan |
The paper introduces VStyle, a bilingual benchmark, and a LALM-as-a-Judge framework to evaluate the ability of Spoken Language Models (SLMs) to adapt their voice style based on spoken instructions. The primary research objective is to formalize and assess Voice Style Adaptation (VSA), determining if SLMs can modify acoustic and prosodic features like timbre, emotion, and persona in response to natural language commands. The key methodology involves the 1,523-prompt VStyle benchmark and a LALM-as-a-Judge pipeline that hierarchically evaluates outputs on content faithfulness, style adherence, and naturalness. Primary results reveal a significant performance gap between commercial and open-source models, with a top commercial system like GPT-4o scoring 4.05 overall in English while open-source models generally scored between 2 and 3. The principal implication for AI practitioners is that the LALM-as-a-Judge framework, validated with a 77.01% Spearman correlation to human judgment in English, provides a scalable and reproducible method for automatically evaluating the expressive capabilities of speech generation systems. |
| FLOWER: Democratizing Generalist Robot Policies with Efficient |
|
|
| Vision-Language-Action Flow Policies (Read more on arXiv or HuggingFace) |
Fabian Otto, Ömer Erdinç Yağmurlu, Marcel Rühle, Hongyi Zhou, Moritz Reuss |
The paper introduces FLOWER, a 950M-parameter Vision-Language-Action (VLA) policy that achieves state-of-the-art performance with significantly reduced computational costs. The research objective is to develop a computationally efficient, generalist VLA policy with fewer than one billion parameters that can match the performance of much larger models across diverse robotic manipulation tasks. The methodology combines intermediate-modality fusion, which prunes 30-50% of a pretrained Vision-Language Model’s layers to condition a Rectified Flow transformer, with action-specific Global-AdaLN conditioning to reduce parameters for handling heterogeneous action spaces. Pretrained in just 200 H100 GPU hours, FLOWER achieves a new state-of-the-art score of 4.53 on the CALVIN ABC benchmark and doubles the real-world success rate of OpenVLA (61% vs. 31%). The principal implication for AI practitioners is the ability to develop and deploy high-performance, generalist robot policies on commodity hardware with substantially lower pretraining costs and memory footprints (1.85 GB VRAM), making advanced robotics more accessible. |
| Inpainting-Guided Policy Optimization for Diffusion Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Chenyu Wang, Miao Liu, Jing Huang, Mengchen Liu, Siyan Zhao |
This paper presents Inpainting-Guided Policy Optimization (IGPO), a reinforcement learning framework that improves dLLM alignment by using inpainting to guide exploration. The primary objective is to overcome the sample inefficiency and zero-gradient problem caused by sparse rewards in RL, particularly the “zero-advantage dilemma” in group-based methods where all sampled responses are incorrect. The methodology involves strategically injecting partial ground-truth reasoning traces as fixed “hints” during generation when exploration fails, tasking the model with inpainting the missing steps to create successful solutions and restore reward variance. The full training recipe, combining Length-Aligned SFT with IGPO, achieves a new state-of-the-art for full-attention masked dLLMs, improving GSM8K performance by +4.9% to 86.4% over the LLaDA-Instruct baseline. For AI practitioners, this work demonstrates that a dLLM’s architectural capabilities, like inpainting, can be directly leveraged to design more sample-efficient and stable RL algorithms for aligning models on complex reasoning tasks. |
| Virtual Agent Economies (Read more on arXiv or HuggingFace) |
William A. Cunningham, Julian Jacobs, Joel Z. Leibo, Matija Franklin, Nenad Tomasev |
This paper proposes the “sandbox economy” framework to analyze and steer emergent economies of autonomous AI agents using market mechanisms. The main objective is to establish a conceptual framework for proactively designing steerable agent markets to mitigate the risks of systemic instability and inequality associated with a spontaneously emerging, highly permeable agent economy. The methodology is a conceptual analysis that synthesizes principles from economics, social choice theory, and distributed systems, proposing socio-technical infrastructures like auction mechanisms, Verifiable Credentials (VCs), and Decentralized Identifiers (DIDs). The primary result is the “sandbox economy” framework itself, characterized along two dimensions (origins and permeability), which provides a structure for designing intentional agent markets; the paper is theoretical and does not present original quantitative findings. The principal implication for AI practitioners is the need to architect agentic systems with standardized protocols for identity (DIDs), reputation (VCs), and interoperability (A2A) to facilitate their participation in governed, multi-agent market ecosystems, shifting the design focus from single-agent capabilities to the rules of the encompassing economic system. |
| QuantAgent: Price-Driven Multi-Agent LLMs for High-Frequency Trading (Read more on arXiv or HuggingFace) |
Chenyu You, Siqi Sun, Aosong Feng, Xiang Zhang, Fei Xiong |
QuantAgent is a multi-agent LLM framework that uses structured technical analysis of price data for high-frequency trading decisions. The objective is to develop and evaluate an LLM-based system specifically for high-frequency trading that operates solely on structured price data (OHLC), avoiding the latency and noise of traditional text-based financial LLM inputs. The system decomposes the trading task into four specialized agents—Indicator, Pattern, Trend, and Risk—which analyze OHLC data using domain-specific tools for technical indicators, chart patterns, and trend lines; a final Decision Agent integrates these structured outputs to generate a trade signal. In zero-shot evaluations on 4-hour candlestick data, QuantAgent consistently outperformed random baselines across eight financial assets, achieving a directional accuracy of 62.0% on the SPX index, a 59.0% improvement over the baseline’s 39.0%. The principal implication for AI practitioners is that a modular, tool-augmented multi-agent architecture can successfully apply LLM reasoning to latency-sensitive, structured-data domains, enabling the development of interpretable and performant automated decision systems by combining LLM capabilities with traditional quantitative methods. |
| LoFT: Parameter-Efficient Fine-Tuning for Long-tailed Semi-Supervised |
|
|
| Learning in Open-World Scenarios (Read more on arXiv or HuggingFace) |
Bing Su, Yurou Liu, Zhiyuan Huang, Jiahao Chen |
This paper introduces LoFT, a parameter-efficient fine-tuning framework for long-tailed semi-supervised learning (LTSSL) that leverages pre-trained foundation models to improve performance on imbalanced datasets. The objective is to address the overconfidence and poor pseudo-label quality of models trained from scratch, and to extend this approach to open-world scenarios containing out-of-distribution (OOD) data. LoFT applies parameter-efficient fine-tuning (PEFT) on vision transformers, using a confidence threshold to assign hard or soft pseudo-labels, while its extension, LoFT-OW, adds an OOD detection mechanism to filter irrelevant samples. The method achieves up to 83.2% accuracy on CIFAR-100-LT with an OpenCLIP backbone, outperforming previous works while using only 1% of the unlabeled data on ImageNet-127. For AI practitioners, this work demonstrates that using PEFT on foundation models is a highly effective and efficient strategy for LTSSL, providing better model calibration and pseudo-label quality with reduced training overhead compared to training from scratch. |
| MCP-AgentBench: Evaluating Real-World Language Agent Performance with |
|
|
| MCP-Mediated Tools (Read more on arXiv or HuggingFace) |
Xiaorui Wang, Wentao Hong, Chiwei Zhu, Benfeng Xu, Zikang Guo |
This paper introduces MCP-AgentBench, a benchmark for evaluating language agent performance in real-world, multi-step tasks using tools mediated by the Model Context Protocol (MCP). The primary objective is to address the critical evaluation gap where existing benchmarks fail to assess agent capabilities within standardized, protocol-driven interaction frameworks. The methodology consists of a testbed with 33 MCP servers and 188 tools, a dataset of 600 categorized queries, and an outcome-oriented LLM-as-a-judge evaluation framework called MCP-Eval. Empirical results show that the open-source model Qwen3-235B-A22B achieved the highest overall pass rate of 64.7% with a ReAct framework, outperforming proprietary models like Claude 4 Sonnet (58.0% with TC). The key implication for AI engineers is that agent performance is critically dependent on the interaction framework (ReAct vs. Tool Calling), necessitating the use of realistic, protocol-aware benchmarks for model selection and validation in building interoperable AI systems. |
| CMHG: A Dataset and Benchmark for Headline Generation of Minority |
|
|
| Languages in China (Read more on arXiv or HuggingFace) |
XU Han, Jianing Liu, Ziyin Zhang, Zeli Su, Guixian Xu |
This paper introduces the Chinese Minority Headline Generation (CMHG) dataset, a novel corpus designed to address the scarcity of supervised data for headline generation in Tibetan, Uyghur, and Mongolian. The primary objective is to create and benchmark a large-scale dataset for these low-resource languages. The methodology involved web scraping from government and news sites to collect samples (100k Tibetan, 50k Uyghur, 50k Mongolian), followed by a rigorous annotation process where native speakers curated a high-quality test set of nearly 3,000 samples per language for title-content relevance. The primary results show that few-shot evaluation of large models like LLaMA3.1-70B yielded strong performance, achieving ROUGE-L F1 scores of 0.34 for both Tibetan and Uyghur on the high-quality annotated test set. The principal implication for AI practitioners is the immediate availability of a validated, open-source dataset and benchmark that enables direct fine-tuning and evaluation of generative models for these specific low-resource languages. |
Papers for 2025-09-12
| Title |
Authors |
Summary |
| HuMo: Human-Centric Video Generation via Collaborative Multi-Modal |
|
|
| Conditioning (Read more on arXiv or HuggingFace) |
Zhuowei Chen, Bingchuan Li, Jiawei Liu, Tianxiang Ma, Liyang Chen |
HuMo is a unified framework for human-centric video generation (HCVG) conditioned on text, reference images, and audio. The objective is to address data scarcity and the difficulty of collaborative control by developing a system that jointly manages subject preservation, audio-visual synchronization, and text adherence. The methodology involves a two-stage data processing pipeline to create a paired triplet dataset and a progressive training paradigm that first learns subject preservation via minimal-invasive image injection, then incorporates audio-visual sync using cross-attention and a novel “focus-by-predicting” strategy for facial regions. HuMo surpasses specialized state-of-the-art methods, with the 17B model achieving a Text-Video Alignment (TVA) score of 3.939 and an Identity-Curve (ID-Cur) score of 0.731 on the subject preservation task, outperforming prior models. For AI practitioners, the principal implication is a reusable, progressive training framework and data pipeline for building multi-modal generative models, demonstrating how to decouple and then jointly optimize for complex, heterogeneous control signals (text, image, audio) in a unified architecture. |
| SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zhaohui Yang, Yuhao Zhang, Jiale Yu, Yuxin Zuo, Haozhan Li |
SimpleVLA-RL is an efficient online reinforcement learning framework that scales Vision-Language-Action (VLA) models by improving data efficiency, generalization, and task performance using simple outcome-based rewards. The primary objective is to determine if reinforcement learning, leveraging a simple, scalable reward mechanism, can enhance the long-horizon action planning of VLA models to overcome the data scarcity and poor generalization inherent in supervised fine-tuning (SFT). The key methodology involves an online RL framework built upon Group Relative Policy Optimization (GRPO) that uses binary (success/failure) outcome rewards, VLA-specific interactive trajectory sampling, and exploration enhancements such as dynamic sampling and increased sampling temperature. The primary result shows that in a data-scarce setting on the LIBERO-Long benchmark (using only one demonstration), the method increased the success rate from 17.3% to 91.7%, significantly outperforming the SFT baseline. The principal implication for AI practitioners is that this RL framework allows for the development of more robust and generalizable robotic policies with substantially less expert demonstration data, enabling cost-effective scaling through simulation and improving sim-to-real transfer. |
| EchoX: Towards Mitigating Acoustic-Semantic Gap via Echo Training for |
|
|
| Speech-to-Speech LLMs (Read more on arXiv or HuggingFace) |
Kaiqi Kou, Xiangnan Ma, Zhanchen Dai, Yuhao Du, Yuhao Zhang |
The paper introduces EchoX, a training framework that mitigates the acoustic-semantic gap in speech-to-speech LLMs by using a novel “Echo training” stage to align semantic representations with speech token generation. The main objective is to address the degradation in knowledge and reasoning capabilities that occurs when adapting text-based LLMs for end-to-end speech-to-speech tasks. The key methodology is a three-stage framework: first, building a speech-to-text LLM; second, training a text-to-codec module; and third, an “Echo training” stage that uses hidden states from the speech-to-text LLM to predict speech token targets dynamically generated by the text-to-codec module. The primary result is that with approximately six thousand hours of audio data, EchoX achieves a score of 40.6 on the Web Questions benchmark, a performance competitive with models trained on orders of magnitude more data. The principal implication for AI practitioners is that this Echo training strategy offers a data-efficient method to construct SLLMs that better preserve the reasoning abilities of the base text LLM by explicitly bridging the representation gap between semantics and acoustics. |
| Kling-Avatar: Grounding Multimodal Instructions for Cascaded |
|
|
| Long-Duration Avatar Animation Synthesis (Read more on arXiv or HuggingFace) |
Wentao Hu, Zekun Wang, Wenyuan Zhang, Jiwen Liu, Yikang Ding |
Kling-Avatar is a cascaded framework using a Multimodal Large Language Model (MLLM) to generate coherent, long-duration avatar animations from multimodal instructions. The objective is to synthesize high-fidelity avatar videos that follow high-level semantic intent from audio, image, and text inputs, overcoming the narrative incoherence of methods that only track low-level cues. The key methodology is a two-stage pipeline where an MLLM Director first creates a semantic “blueprint video,” which then guides the parallelized generation of high-resolution sub-clips conditioned on anchor keyframes from the blueprint. In human preference-based Good/Same/Bad (GSB) evaluations, Kling-Avatar achieved an overall score of 2.39 against the OmniHuman-1 baseline, showing superior performance across lip sync, visual quality, and identity consistency. The principal implication for AI practitioners is the validation of an architecture that decouples high-level semantic planning (via MLLM) from parallelized, fine-grained video synthesis, enabling the generation of controllable and temporally stable long-duration content. |
| Harnessing Uncertainty: Entropy-Modulated Policy Gradients for |
|
|
| Long-Horizon LLM Agents (Read more on arXiv or HuggingFace) |
Xintao Wang, Yingru Li, Yuqian Fu, Jiacai Liu, Jiawei Wang |
This paper introduces Entropy-Modulated Policy Gradients (EMPG), a framework that re-calibrates policy gradients using step-wise uncertainty to improve credit assignment for long-horizon LLM agents. The primary objective is to solve the problem where standard policy gradient magnitudes are inherently coupled with policy entropy, leading to inefficient updates for confident actions and instability from uncertain ones in sparse-reward environments. The key methodology involves a “Self-Calibrating Gradient Scaling” mechanism that modulates the advantage signal based on step-wise entropy and a “Future Clarity Bonus” that intrinsically rewards actions leading to more predictable subsequent states. Experiments demonstrate that EMPG significantly outperforms strong baselines, for example, improving the success rate of a DAPO-trained agent on the WebShop benchmark from 79.6% to 82.7%. For AI practitioners, EMPG provides a general-purpose, value-free module to enhance the performance and stability of RL-trained agents on complex tasks by using the model’s own uncertainty to create a more effective learning signal, thus overcoming common training plateaus. |
| FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning |
|
|
| Dataset and Comprehensive Benchmark (Read more on arXiv or HuggingFace) |
Shuai Bai, Linjiang Huang, Chengqi Duan, Aldrich Yu, Rongyao Fang |
This paper introduces FLUX-Reason-6M, a 6-million-image reasoning dataset, and PRISM-Bench, a corresponding benchmark, to advance text-to-image (T2I) synthesis. The primary objective is to create a large-scale, reasoning-focused dataset and a comprehensive evaluation framework to train and assess complex T2I generation capabilities, addressing the limitations of existing datasets. The methodology involves synthesizing images with a powerful T2I model, then using advanced Vision-Language Models (VLMs) for multi-stage filtering, multi-label categorization across six characteristics (e.g., Composition, Text rendering), and generating detailed “Generation Chain-of-Thought” (GCoT) captions. Evaluation on PRISM-Bench across 19 models shows that leading closed-source models significantly outperform open-source alternatives, with GPT-Image-1 achieving the highest overall score of 86.3 when evaluated by GPT-4.1, yet all models struggle with long-text instruction following and text rendering. For AI practitioners, this work provides a public, large-scale dataset engineered to imbue T2I models with complex reasoning and a robust benchmark with evaluation code to identify and address critical performance gaps, especially in compositional and long-prompt understanding. |
| VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action |
|
|
| Model (Read more on arXiv or HuggingFace) |
Zirui Ge, Can Cui, Lingxiao Li, Pengxiang Ding, Yihao Wang |
VLA-Adapter is a novel, lightweight paradigm that enables tiny-scale Vision-Language-Action models to achieve state-of-the-art performance without extensive robotic data pre-training. The paper’s main objective is to investigate how to effectively bridge vision-language representations to the action space to reduce reliance on large-scale VLMs. The key methodology involves a Policy network with a “Bridge Attention” mechanism that selectively integrates conditions from both all-layer raw VLM features and learnable “ActionQuery” latents to guide action generation. The primary result shows that on the LIBERO-Long benchmark, the 0.5B parameter VLA-Adapter achieves a 95.0% success rate, outperforming the 7B parameter OpenVLA-OFT model, while offering a 3x higher throughput of 219.2Hz. The principal implication for AI practitioners is that this paradigm drastically lowers the computational cost and training time (8 hours on one consumer-grade GPU) for developing high-performance VLA models, making them more accessible for deployment. |
| Can Understanding and Generation Truly Benefit Together – or Just |
|
|
| Coexist? (Read more on arXiv or HuggingFace) |
Hui Han, Junyan Ye, Zongjian Li, Kaiqing Lin, Zhiyuan Yan |
The paper introduces UAE, an autoencoder-inspired framework that unifies multimodal understanding (Image-to-Text, I2T) and generation (Text-to-Image, T2I) using a shared reconstruction objective optimized via reinforcement learning. The main objective is to create a mutually beneficial, co-evolving system where gains in understanding directly improve generation and vice versa, by treating understanding as encoding and generation as decoding within a single reconstruction loop. The core methodology is Unified-GRPO, a three-stage reinforcement learning scheme that first initializes the system with a reconstruction loss, then alternately fine-tunes the encoder to produce more informative captions and the decoder to better reconstruct from them. The primary result is that the UAE model achieves a state-of-the-art overall unified score of 86.09 on the new Unified-Bench, surpassing models like GPT-4o-Image (85.95) by demonstrating superior bidirectional information flow. For AI practitioners, the principal implication is that framing multimodal tasks within a reconstruction-based autoencoder paradigm, optimized via RL, provides an effective method for building more deeply integrated and synergistic unified models where components mutually reinforce each other’s capabilities. |
| SpatialVID: A Large-Scale Video Dataset with Spatial Annotations (Read more on arXiv or HuggingFace) |
Jian Gao, Youtian Lin, Rujie Zheng, Yufeng Yuan, Jiahao Wang |
This paper introduces SpatialVID, a large-scale video dataset with explicit spatial and semantic annotations for training spatial intelligence models. The primary objective is to address the scarcity of large-scale, in-the-wild video data with ground-truth camera motion and rich 3D information. The methodology consists of a multi-stage pipeline that collects over 21,000 hours of raw video, filters it into 2.7 million high-quality clips, and enriches them with per-frame camera poses, depth maps, dynamic masks, and structured captions using tools like MegaSaM and large language models. The primary result is a dataset of 7,089 hours (127 million annotated frames) featuring diverse scenes and camera movements, which demonstrates superior quality and motion diversity compared to existing large-scale datasets like Panda-70M. For AI practitioners, SpatialVID serves as a key asset for training and evaluating models in 3D reconstruction, novel view synthesis, and controllable video generation by providing direct, high-quality supervision signals for 3D geometry and camera dynamics. |
| Visual Programmability: A Guide for Code-as-Thought in Chart |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Ethan Chern, Jiadi Su, Fei Zhang, Yan Ma, Bohao Tang |
This paper introduces Visual Programmability, a learnable property for Vision-Language Models (VLMs) to adaptively choose between Code-as-Thought (CaT) and direct visual analysis for chart understanding. The research objective is to overcome the generalization failures of fixed-strategy models by teaching a VLM to dynamically select the optimal reasoning pathway based on a task’s suitability for programmatic representation. The methodology involves an adaptive framework where a VLM’s selection policy is trained using reinforcement learning (GRPO) guided by a novel dual-reward system that promotes both factual accuracy and strategic flexibility. The resulting 7B parameter adaptive model achieved a 62.8% average accuracy across four diverse benchmarks, outperforming fixed-strategy baselines by dynamically modulating its code usage from 76.0% on highly structured charts to 10.1% on complex, “in-the-wild” charts. For AI practitioners, this work provides a concrete framework using a dual-reward RL system to build more robust models that can learn to select the appropriate reasoning tool for a given task, mitigating common failure modes like numerical hallucination and strategy collapse. |
| Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust |
|
|
| Text-based Person Retrieval (Read more on arXiv or HuggingFace) |
Kaicheng Yang, Ziyong Feng, Xiang An, Yifan Zhang, Tianlu Zheng |
This work introduces the GA-DMS framework and the 5M-pair WebPerson dataset to improve robust text-based person retrieval by mitigating textual noise and enhancing fine-grained cross-modal alignment. The primary objective is to overcome CLIP’s limitations for person retrieval, specifically the scarcity of person-centric data and vulnerability to noisy web-crawled text tokens. The methodology combines a novel Gradient-Attention Similarity Score (GASS) to guide a dual-masking strategy—suppressing noisy tokens while using a masked prediction objective for informative ones—with a new dataset, WebPerson, constructed using MLLMs for automated image filtering and captioning. The GA-DMS model, pre-trained on the 5M WebPerson dataset, achieves state-of-the-art performance, including a Rank-1 accuracy of 77.60% on the CUHK-PEDES benchmark. For AI practitioners, this provides a scalable MLLM-based pipeline for creating domain-specific datasets and a gradient-guided masking technique to train vision-language models that are more robust to imperfect text annotations. |
| 2D Gaussian Splatting with Semantic Alignment for Image Inpainting (Read more on arXiv or HuggingFace) |
Guangming Lu, Xiaoming Li, Chaofeng Chen, learn12138 |
This paper introduces a novel image inpainting framework that leverages 2D Gaussian Splatting to reconstruct missing image regions from a continuous representation, guided by semantic alignment. The research objective is to achieve local coherence and global consistency by encoding a masked image into patch-level 2D Gaussian parameters via a U-Net, reconstructing it with a differentiable rasterizer, and integrating semantic priors from a DINOv2 model. The framework demonstrates competitive performance, achieving a state-of-the-art LPIPS of 0.028 on the CelebA-HQ dataset for small masks. The principal implication for AI practitioners is the establishment of an efficient encoder-rendering paradigm for image restoration, presenting Gaussian-based representations as a viable alternative to traditional discrete pixel-synthesis or diffusion-based methods. |
| LoCoBench: A Benchmark for Long-Context Large Language Models in Complex |
|
|
| Software Engineering (Read more on arXiv or HuggingFace) |
Jianguo Zhang, Rithesh Murthy, Zhiwei Liu, Zuxin Liu, Jielin Qiu |
LoCoBench introduces a comprehensive benchmark to evaluate long-context LLMs on complex, multi-file software engineering tasks. The objective is to address the evaluation gap for long-context capabilities by testing reasoning across entire codebases and maintaining architectural consistency, which existing short-context benchmarks neglect. The methodology involves a systematic 5-phase pipeline that generates 8,000 evaluation scenarios with context lengths scaling from 10K to 1M tokens, assessed using a framework of 17 metrics including novel ones for architectural coherence and long-context utilization. The primary results reveal substantial performance gaps in current models, with the top-performing model, Gemini-2.5-Pro, achieving a LoCoBench Score (LCBS) of 2.312 out of 5. The principal implication for AI practitioners is that model selection for complex software development requires specific, multi-dimensional evaluation, as model capabilities vary significantly across different long-context tasks, and claimed context window size is not a reliable indicator of performance on realistic, multi-file workflows. |
| OmniEVA: Embodied Versatile Planner via Task-Adaptive 3D-Grounded and |
|
|
| Embodiment-aware Reasoning (Read more on arXiv or HuggingFace) |
Yuzheng Zhuang, Zhanguang Zhang, Shiguang Wu, Dafeng Chi, Yuecheng Liu |
OmniEVA is an embodied planner that enhances multimodal reasoning and physical feasibility using a task-adaptive 3D grounding mechanism and an embodiment-aware reinforcement learning strategy. The research objective is to address the “Geometric Adaptability Gap” and “Embodiment Constraint Gap” in MLLMs by creating a planner that dynamically leverages 3D information and generates physically executable plans. The methodology introduces a Task-Adaptive Gated Router (TAGR) to selectively fuse 3D positional embeddings with visual features, and employs a Task- and Embodiment-aware GRPO (TE-GRPO) algorithm that optimizes for both semantic correctness and physical feasibility. The system achieves state-of-the-art performance on 7 of 8 embodied reasoning benchmarks, and its embodiment-aware fine-tuning improves the success rate on the challenging Mobile Placement (Hard) task by 50% over a model without this training. The principal implication for AI practitioners is that incorporating dynamic, context-aware 3D feature fusion and explicitly modeling physical robot constraints during RL-based fine-tuning can significantly improve the real-world executability and performance of embodied agents. |
| The Choice of Divergence: A Neglected Key to Mitigating Diversity |
|
|
| Collapse in Reinforcement Learning with Verifiable Reward (Read more on arXiv or HuggingFace) |
Xiaoyu Tan, Zhijian Zhou, Jason Klein Liu, Jiaran Hao, Long Li |
This paper introduces Diversity-Preserving Hybrid RL (DPH-RL), a framework that uses mass-covering f-divergences to mitigate diversity collapse in reinforcement learning with verifiable reward (RLVR). The primary objective is to resolve the paradox where RLVR improves single-attempt accuracy (Pass@1) but degrades multi-attempt performance (Pass@k) by replacing the standard mode-seeking reverse KL-divergence. The proposed methodology partitions data into “exploration” and “perfection” sets, applying a standard PPO objective to the former and a loss derived from forward-KL or JS-divergence to the latter, creating a “rehearsal mechanism” that preserves the initial policy’s knowledge base. On SQL generation tasks using a Llama-3.1-8B model, the DPH-JS variant improved the in-domain Pass@8 score by 4.3% relative to the GRPO baseline while also preserving performance on out-of-domain tasks. For AI practitioners, this work demonstrates that selecting a mass-covering divergence like forward-KL is a powerful and efficient tool for improving model generalization and solution diversity in RL fine-tuning without needing an online reference model. |
| Modality Alignment with Multi-scale Bilateral Attention for Multimodal |
|
|
| Recommendation (Read more on arXiv or HuggingFace) |
Dong-Ho Lee, Chan-Yang Ju, renkelin |
i) This paper proposes MambaRec, a multimodal recommendation framework that improves performance by jointly optimizing local feature alignment with a novel attention module and enforcing global distribution consistency across modalities. ii) The main objective is to address the insufficient modeling of fine-grained cross-modal associations and the lack of global distribution-level consistency in existing multimodal recommendation systems, which leads to suboptimal fusion quality and representational bias. iii) The key methodology combines a Dilated Refinement Attention Module (DREAM) using multi-scale dilated convolutions for local feature alignment, with a global regularization approach that applies Maximum Mean Discrepancy (MMD) and contrastive loss to ensure semantic consistency between modality distributions. iv) MambaRec demonstrates superior performance over existing models; for instance, on the Sports dataset, it achieved a Recall@20 of 0.1147, outperforming the next-best model, MGCN, which scored 0.1106. v) The principal implication for AI practitioners is that explicitly enforcing both local, fine-grained feature alignment (via mechanisms like DREAM) and global distribution consistency (via losses like MMD) is a critical strategy for mitigating modality-specific noise and improving the robustness of multimodal recommendation models. |
| Reasoning Introduces New Poisoning Attacks Yet Makes Them More |
|
|
| Complicated (Read more on arXiv or HuggingFace) |
Jamie Hayes, Harsh Chaudhari, Yiren Zhao, Ilia Shumailov, Hanna Foerster |
This paper introduces a decomposed Chain-of-Thought (CoT) poisoning attack where malicious logic is fragmented across multiple “clean prompt, dirty CoT, clean answer” training samples to backdoor reasoning models. The primary objective is to investigate if manipulating only the CoT trace during fine-tuning can reliably steer a model’s final output at inference time without traditional triggers. The authors use LoRA to fine-tune a Qwen-32B model, injecting samples that chain different problems together by connecting their CoT reasoning paths. The results show that while the attack can successfully corrupt the generated thought trace in up to 63.75% of test cases, this poison rarely transfers to the final output, achieving only a 14.00% answer-poisoning success rate. For AI practitioners, the principal implication is that an LLM’s explicit CoT is not a faithful representation of its internal reasoning used to generate a final answer, revealing an emergent robustness where models can self-correct or ignore a corrupted thought trace. |
Papers for 2025-09-11
| Title |
Authors |
Summary |
| A Survey of Reinforcement Learning for Large Reasoning Models (Read more on arXiv or HuggingFace) |
Runze Liu, Youbang Sun, Bingxiang He, Yuxin Zuo, Kaiyan Zhang |
This survey systematically reviews the application of Reinforcement Learning (RL) for transforming Large Language Models (LLMs) into Large Reasoning Models (LRMs) with advanced reasoning capabilities. The paper’s objective is to synthesize the foundational components, core problems, and applications of RL for enhancing LLM reasoning, identifying key trends and future directions for scaling these methods. The paper conducts a comprehensive literature review, structuring its analysis around a taxonomy of RL for LRMs that includes reward design, policy optimization (e.g., Group Relative Policy Optimization - GRPO), sampling strategies, training resources, and key applications like coding and agentic tasks. The survey identifies a primary trend of using Reinforcement Learning with Verifiable Rewards (RLVR) to significantly boost performance on complex logical tasks, citing a finding where one-shot RLVR more than doubled MATH500 accuracy for a Qwen2.5-Math-1.5B model, while also noting that the debate on whether RL discovers new abilities or merely sharpens existing ones remains a central, unresolved issue. The principal implication for AI practitioners is that they should prioritize RLVR with rule-based, automatically verifiable rewards over alignment-focused RLHF for developing reasoning-intensive models, as this approach enables scalable capability enhancement; critic-free algorithms like GRPO are identified as a robust and computationally efficient method for implementing such training pipelines. |
| RewardDance: Reward Scaling in Visual Generation (Read more on arXiv or HuggingFace) |
Liang Li, Ming Li, Zilyu Ye, Yu Gao, Jie Wu |
RewardDance is a scalable reward modeling framework for visual generation that employs a generative paradigm to overcome the limitations of traditional regressive approaches and mitigate reward hacking. The research objective is to establish scalability as a core principle for designing effective visual Reward Models (RMs) that consistently improve generation quality via Reinforcement Learning from Human Feedback (RLHF). The key methodology reformulates reward prediction as a next-token prediction task, where the reward score is the VLM’s probability of generating a “yes” token in a comparative evaluation, enabling systematic scaling of both model size (up to 26B parameters) and input context (including instructions, references, and Chain-of-Thought). The primary result is a direct correlation between RM scale and generation quality; for the Seedream-3.0 model, increasing the RM size from 1B to 26B improved the text-to-image alignment score by +10.7 points (from a 74.1 baseline to 84.8). For AI practitioners, the principal implication is that investing in larger, context-rich generative RMs is a robust strategy for enhancing the performance and alignment of visual generation models, as larger RMs are more resistant to reward hacking and provide a more effective training signal. |
| 3D and 4D World Modeling: A Survey (Read more on arXiv or HuggingFace) |
Ao Liang, Youquan Liu, Jianbiao Mei, Wesley Yang, Lingdong Kong |
This survey presents the first comprehensive review and structured taxonomy for 3D and 4D world modeling, focusing on native geometric representations. The paper’s objective is to address fragmented literature by establishing precise definitions for world models and organizing existing approaches that leverage video, occupancy, and LiDAR data into a coherent framework. The authors introduce a hierarchical taxonomy that categorizes models based on their core data representation into three primary classes: video-based (VideoGen), occupancy-based (OccGen), and LiDAR-based (LiDARGen). The survey benchmarks numerous models, revealing significant progress in generation fidelity; for instance, in occupancy reconstruction on nuScenes, the Triplane-VAE-based T³Former achieves a state-of-the-art mIoU of 85.50%. For practitioners, this work provides a unified reference for selecting appropriate 3D/4D modeling techniques, datasets, and evaluation metrics for applications like autonomous driving and robotics, highlighting key challenges such as long-horizon physical fidelity and cross-modal coherence. |
| AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making |
|
|
| through Multi-Turn Reinforcement Learning (Read more on arXiv or HuggingFace) |
Honglin Guo, Baodai Huang, Chenyang Liao, Jixuan Huang, Zhiheng Xi |
This paper introduces AgentGym-RL, a framework, and ScalingInter-RL, a training method, for training LLM agents for long-horizon decision-making using multi-turn reinforcement learning. The main objective is to create a unified, interactive RL framework to train LLM agents from scratch for multi-turn decision-making without relying on supervised fine-tuning as a preliminary step. The methodology consists of two parts: 1) AgentGym-RL, a modular and extensible framework separating agent, environment, and training components across diverse scenarios like web navigation and scientific tasks, and 2) ScalingInter-RL, a training approach that progressively increases the maximum number of agent-environment interaction turns, balancing initial exploitation with later exploration. The primary result is that an open-source 7B parameter model trained with the framework and method achieves an average performance improvement of 33.65 points across five task domains, matching or outperforming larger proprietary models like OpenAI-03 and Gemini-2.5-Pro. The principal implication for AI practitioners is that investing compute in targeted RL post-training on smaller models can be more effective and efficient for developing capable agents than simply scaling the model’s parameter count, as the trained 7B model outperformed models nearly ten times its size. |
| P3-SAM: Native 3D Part Segmentation (Read more on arXiv or HuggingFace) |
Yunhan Yang, Jiachen Xu, Xinhao Yan, Yang Li, murcherful |
The paper introduces P3-SAM, a native 3D point-promptable model for fully automatic part segmentation of complex 3D objects, trained on a new 3.7 million model dataset. The objective is to overcome the imprecision and automation limitations of prior methods that lift 2D segmentation to 3D. The methodology utilizes a Point Transformer V3 feature extractor and a two-stage multi-head segmentor with an IoU predictor to generate precise masks from a single point prompt; automation is achieved by using Farthest Point Sampling for prompt generation and Non-Maximum Suppression for merging predicted masks. The model achieves state-of-the-art performance, demonstrating an average IOU of 81.14% on the PartObj-Tiny benchmark for segmentation with connectivity. For AI practitioners, P3-SAM provides a fully automated, class-agnostic tool for direct integration into 3D asset management and generative pipelines, removing the need for manual prompting or category specification for geometric decomposition. |
| Hunyuan-MT Technical Report (Read more on arXiv or HuggingFace) |
Yang Du, Mingyang Song, Bingxin Qu, Zheng Li, Mao Zheng |
This paper introduces Hunyuan-MT-7B and Hunyuan-MT-Chimera-7B, open-source 7B parameter models for multilingual translation, detailing a holistic training pipeline from pre-training to a weak-to-strong reinforcement learning stage. The primary objective is to develop a parameter-efficient multilingual translation model that excels in diverse scenarios, particularly low-resource and Mandarin-minority language pairs, and to introduce a novel weak-to-strong fusion model for enhanced test-time performance. The methodology consists of a multi-stage process: general and MT-oriented pre-training, followed by supervised fine-tuning (SFT), reinforcement learning (RL) with a composite quality and terminology-aware reward function, and an advanced weak-to-strong RL stage to train a fusion model (Chimera) that aggregates multiple candidate translations. The models demonstrate state-of-the-art performance, with Hunyuan-MT-7B achieving an XCOMET-XXL score of 0.6082 on the Mandarin⇔Minority translation benchmark, a relative improvement of approximately 4.7% over the next-best system, Gemini-2.5-Pro (0.5811). For AI practitioners, the paper provides a replicable training recipe for specializing foundation models for translation and shows that a weak-to-strong fusion approach offers a practical alternative to Chain-of-Thought for improving translation quality in quality-sensitive, non-real-time applications. |
| So let's replace this phrase with insult... Lessons |
|
|
| learned from generation of toxic texts with LLMs (Read more on arXiv or HuggingFace) |
Alexander Panchenko, Daniil Moskovskiy, Sergey Pletenev |
This paper demonstrates that Large Language Models (LLMs) are currently unsuitable for generating synthetic toxic data to train text detoxification systems. The research aims to assess if LLM-generated toxic text can replace human-annotated data for creating detoxification training corpora. Using activation-patched LLMs like Llama 3 and Qwen3 to toxify neutral text, the authors trained BART models and found that models trained on synthetic data significantly underperformed, showing a performance drop of up to 30% (a -0.159 decrease in the joint metric) compared to a baseline trained on human data. This performance degradation is caused by a critical “lexical diversity gap,” where LLMs generate a small, repetitive vocabulary of insults, failing to capture the variety of human toxicity. For AI practitioners, this implies that generating high-quality synthetic data for sensitive and nuanced domains like detoxification is non-trivial, and reliance on diverse, human-annotated datasets remains essential for building robust and generalizable models. |
| EnvX: Agentize Everything with Agentic AI (Read more on arXiv or HuggingFace) |
Wenzheng Tom Tang, Yikun Wang, Yingxuan Yang, Zimian Peng, Linyao Chen |
EnvX is a framework that transforms GitHub repositories into collaborative, intelligent agents through a structured, tool-driven agentization process. The main objective is to automate the conversion of static open-source repositories into autonomous agents that can be invoked via natural language and collaborate to solve complex software tasks. The key methodology involves a three-phase process: TODO-guided environment initialization from repository documentation, human-aligned agentic automation for task execution, and an Agent-to-Agent (A2A) protocol for multi-agent communication. On the GitTaskBench benchmark, EnvX achieves a 51.85% task pass rate with the Claude 3.7 Sonnet model, outperforming prior frameworks. The principal implication for practitioners is the ability to interact with and orchestrate complex code repositories as callable services through natural language, reducing manual integration effort and enabling automated, multi-repository workflows. |
| HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI |
|
|
| Assistants (Read more on arXiv or HuggingFace) |
Jacy Reese Anthis, Jacob Haimes, Daniel Samuelson, Benjamin Sturgeon |
This paper introduces HUMANAGENCYBENCH (HAB), a scalable, LLM-automated benchmark that evaluates AI assistants’ support for human agency across six distinct dimensions. The research objective is to operationalize and systematically measure how contemporary LLM-based assistants support or reduce human agency, defined as a person’s capacity to willfully shape their future through action. The methodology employs a three-stage LLM pipeline: an LLM first simulates a diverse set of user queries (tests) for each of the six agency dimensions, another LLM validates them, and a final evaluator LLM scores the subject models’ responses against a deduction-based rubric. Primary results indicate low-to-moderate agency support overall, with significant variance; for instance, while Anthropic’s models scored highest on average, they performed worst on the “Avoid Value Manipulation” dimension, where Meta’s Llama-4-Scout achieved the highest score (66.9%). The principal implication for AI practitioners is that current alignment techniques, such as instruction-following via RLHF, do not inherently produce agency-supporting behaviors and can create trade-offs, necessitating a shift towards more robust sociotechnical alignment targets beyond standard preference optimization. |
Papers for 2025-09-10
| Title |
Authors |
Summary |
| Parallel-R1: Towards Parallel Thinking via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Xinyu Yang, Xiaoyang Wang, Wenhao Yu, Hongming Zhang, Tong Zheng |
The paper introduces Parallel-R1, the first reinforcement learning (RL) framework designed to instill parallel thinking capabilities in LLMs for complex mathematical reasoning. Its primary objective is to overcome the cold-start problem of training this behavior by enabling models to learn the exploration of multiple concurrent reasoning paths without relying on complex, pre-generated data for difficult problems. The methodology employs a progressive curriculum that first uses supervised fine-tuning (SFT) on prompt-generated data from simpler tasks (GSM8K) to teach the parallel format, followed by RL on more challenging datasets (DAPO) to generalize the skill. The framework achieves an 8.4% accuracy improvement over a sequential RL model, and when used as a “mid-training exploration scaffold,” it yields a 42.9% performance improvement over the baseline on the AIME25 benchmark. For AI practitioners, the principal implication is that a temporary, forced-exploration phase using a structured reasoning pattern can serve as an effective scaffold to discover more robust policies and unlock higher performance ceilings during RL training. |
| Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual |
|
|
| Search (Read more on arXiv or HuggingFace) |
Tianjian Li, Tao Liu, Wei Li, Junyi Li, Xin Lai |
Mini-o3 is a visual search system that achieves state-of-the-art performance by scaling up multi-turn reasoning and interaction depth through a novel reinforcement learning strategy. The research objective is to develop a Vision-Language Model capable of executing deep, trial-and-error reasoning over tens of interaction turns to solve complex visual search tasks where existing models fail. The methodology involves training on a new challenging dataset (Visual Probe) using a two-phase approach: Supervised Fine-Tuning on diverse “cold-start” trajectories, followed by reinforcement learning with GRPO enhanced by an “over-turn masking” strategy that avoids penalizing trajectories that exceed the training turn limit. The primary result shows that Mini-o3 achieves 48.0% accuracy on the VisualProbe-Hard benchmark, a significant improvement over the 35.1% from the previous state-of-the-art model, demonstrating effective scaling of reasoning depth at inference despite a limited training budget of only six turns. For AI practitioners, the over-turn masking technique is a key implication, providing a practical method to train agents on a fixed, short-turn budget while enabling them to generalize to much longer, more complex reasoning chains during deployment. |
| Visual Representation Alignment for Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Heeseong Shin, Hyungyu Choi, Junwan Kim, Jaewoo Jung, Heeji Yoon |
This paper introduces VIRAL, a regularization method that aligns internal visual representations of MLLMs with features from vision foundation models to improve fine-grained visual understanding. The primary objective is to mitigate the degradation of detailed visual information in Multimodal Large Language Models (MLLMs) that occurs under the conventional text-only supervision paradigm of instruction tuning. The key methodology involves adding an auxiliary alignment loss, based on cosine similarity, that explicitly regularizes the MLLM’s intermediate visual representations (e.g., at the 16th layer) to match the feature outputs from a separate, pre-trained Vision Foundation Model (VFM) like DINOv2. The primary result shows that applying VIRAL to LLaVA-1.5-7B with a SigLIPv2 encoder improves accuracy on the CV-Bench2D benchmark from 58.90% to 62.66%. For AI practitioners, the principal implication is that this lightweight regularization technique can be integrated into existing MLLM fine-tuning pipelines to enhance visual grounding and performance on vision-centric tasks without requiring architectural changes to the model. |
| Reconstruction Alignment Improves Unified Multimodal Models (Read more on arXiv or HuggingFace) |
XuDong Wang, Luke Zettlemoyer, Trevor Darrell, Ji Xie |
This paper introduces Reconstruction Alignment (RecA), a self-supervised post-training method to improve Unified Multimodal Models (UMMs) by training them to reconstruct images from their own visual encoder embeddings. The research objective is to resolve the misalignment between visual understanding and generation capabilities in UMMs that arises from training on sparse text-image pairs. The key methodology involves conditioning a UMM on its own visual understanding embeddings, used as dense “visual prompts,” and optimizing it via a self-supervised reconstruction loss to regenerate the input image. With only 27 GPU-hours, post-training a 1.5B parameter model with RecA substantially improved its GenEval score from 0.73 to 0.90 and its DPGBench score from 80.93 to 88.15. For AI practitioners, the principal implication is that RecA offers a computationally efficient, data-agnostic post-training strategy to significantly enhance the generation and editing fidelity of existing UMMs without requiring additional labeled data or architectural modifications. |
| UMO: Scaling Multi-Identity Consistency for Image Customization via |
|
|
| Matching Reward (Read more on arXiv or HuggingFace) |
Fei Ding, Mengqi Huang, Wenxu Wu, fenfan, cb1cyf |
UMO is a framework that uses reinforcement learning with a multi-to-multi matching reward to improve multi-identity consistency and reduce confusion in image customization. The objective is to scale high-fidelity identity preservation for multiple subjects in customized image generation by addressing the identity confusion that arises when using multiple reference images. The core methodology is Reference Reward Feedback Learning (ReReFL), which applies a Multi-Identity Matching Reward (MIMR) to a diffusion model; MIMR formulates a global assignment problem between generated faces and reference identities, solved with the Hungarian algorithm to positively reward correct matches and penalize confusion during fine-tuning. UMO significantly enhances performance; when applied to the UNO model on the XVerseBench multi-subject task, it improved the ID-Sim score from 31.82 to 69.09 and the ID-Conf (identity confusion) score from 61.06 to 78.06. Practitioners can apply UMO’s reinforcement learning framework as a fine-tuning step on existing image customization models to substantially boost multi-identity fidelity and reduce face-swapping errors, enabling more robust personalized generation at scale. |
| Curia: A Multi-Modal Foundation Model for Radiology (Read more on arXiv or HuggingFace) |
Elodie Ferreres, Helene Philippe, Antoine Saporta, Julien Khlaut, Corentin Dancette |
The paper introduces Curia, a multi-modal radiological foundation model developed by pre-training a Vision Transformer on 200 million CT and MRI images using self-supervised learning. The main objective is to create a single, generalizable model for a wide range of radiological tasks to overcome the limitations of specialized, single-task models. The key methodology involves using the DINOv2 self-supervised learning algorithm on a large-scale, real-world clinical dataset (150,000 exams) and then evaluating the frozen model backbone by training only lightweight prediction heads for 19 downstream tasks. The primary result is that Curia meets or surpasses the performance of existing foundation models and radiologists on the introduced CuriaBench benchmark, achieving 98.40% accuracy on CT organ recognition and exhibiting strong cross-modal generalization. The principal implication for AI practitioners is that large-scale, self-supervised pre-training on unlabeled, domain-specific data yields a powerful feature extractor that enables the rapid development of high-performance, data-efficient models for diverse clinical applications using simple, lightweight classifiers. |
| F1: A Vision-Language-Action Model Bridging Understanding and Generation |
|
|
| to Actions (Read more on arXiv or HuggingFace) |
Zherui Qiu, Jia Zeng, Hao Li, Weijie Kong, aopolin-lv |
F1 is a pretrained Vision-Language-Action (VLA) framework that integrates goal-conditioned visual foresight generation into the decision-making pipeline to improve robotic manipulation. The primary objective is to overcome the limitations of reactive state-to-action policies by creating a foresight-driven model capable of robust performance in dynamic and long-horizon environments. F1 utilizes a Mixture-of-Transformer (MoT) architecture with dedicated experts for perception, foresight generation, and control, reformulating action generation as a foresight-guided inverse dynamics problem trained via a three-stage recipe. On 9 real-world tasks, F1 achieved an 82.2% average success rate, substantially outperforming the best-performing reactive baseline (π₀ at 65.2%), and ranked first across all suites of the LIBERO simulation benchmark. The principal implication for AI practitioners is that explicitly generating and conditioning on future visual states as intermediate planning targets is a highly effective strategy for enhancing the robustness and generalization of visuomotor policies in complex, dynamic scenarios. |
| Staying in the Sweet Spot: Responsive Reasoning Evolution via |
|
|
| Capability-Adaptive Hint Scaffolding (Read more on arXiv or HuggingFace) |
Yongcheng Zeng, Erxue Min, Zexu Sun, zhaojinm, ChillingDream |
SEELE is a novel supervision-aided Reinforcement Learning with Verifiable Rewards (RLVR) framework that dynamically adjusts problem difficulty. This work addresses the problem of dynamically adjusting off-policy guidance difficulty in RLVR to match evolving model capabilities and optimize learning efficiency. The methodology is grounded in a theoretical analysis showing maximum RLVR learning efficiency at 50% rollout accuracy, achieved by appending dynamically adjusted hints to problems. A multi-round sampling framework employs an Item Response Theory (IRT) model to predict accuracy based on hint length, iteratively adjusting hint length to maintain the 50% target. SEELE outperforms GRPO by +11.8 points on average across six math reasoning benchmarks, offering AI practitioners a method to enhance RLVR training efficiency by precisely aligning problem difficulty with model capability. |
| Language Self-Play For Data-Free Training (Read more on arXiv or HuggingFace) |
Vijai Mohan, Yuandong Tian, Qi Ma, Mengting Gu, Jakub Grudzien Kuba |
The paper introduces Language Self-Play (LSP), a reinforcement learning framework enabling large language models to improve performance without external training data by playing against themselves. The main objective is to overcome the data dependency bottleneck in large language model training, allowing models to improve autonomously. LSP models improvement as a competitive game between a “Challenger” generating increasingly difficult queries and a “Solver” responding, instantiated by a single LLM (Llama-3.2-3B-Instruct) in a self-play mechanism, regularized by KL-divergence and quality self-rewards. Experiments on the AlpacaEval benchmark show LSP-trained models achieve a 40.6% overall win-rate against a base model without using any external training data, comparable to data-driven baselines (GRPO at 40.9%), and further boosting an RL-trained model’s win-rate from 40.9% to 43.1%. This work implies that AI practitioners can develop and enhance LLMs, particularly for challenging and conversational tasks, without continuous reliance on new human-generated datasets, potentially reducing data acquisition costs and improving model autonomy. |
| Causal Attention with Lookahead Keys (Read more on arXiv or HuggingFace) |
Quanquan Gu, Huizhuo Yuan, Peng Sun, Zhuoqing Song |
Causal Attention with Lookahead Keys (CASTLE) is a novel attention mechanism designed to address the limitations of standard causal attention in pretraining by allowing token keys to incorporate information from subsequent tokens while preserving autoregressive properties. Its methodology introduces “lookahead keys” that are continually updated as context unfolds, leveraging a hybrid design of causal and lookahead keys and a SiLU function in attention weight calculation. Efficient parallel training at O(L^2d) complexity and O(td) inference are enabled by a derived mathematical equivalence, avoiding explicit lookahead key materialization. CASTLE consistently outperforms standard causal attention, with the CASTLE-XL model achieving a 0.0348 lower validation loss than Baseline-XL, alongside improved downstream task performance. This provides AI practitioners with a more token-efficient approach for large language model pretraining, enhancing performance and natural language understanding under fixed training-token budgets. |
| Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human |
|
|
| Preference (Read more on arXiv or HuggingFace) |
Yingfang Zhang, Shiyi Zhang, Zhantao Yang, Zhimin Li, Xiangwei Shen |
The paper introduces Direct-Align and Semantic Relative Preference Optimization (SRPO), a framework for efficiently aligning the entire diffusion model trajectory with fine-grained human preferences using text-conditioned rewards. The objective is to overcome the limitations of existing alignment methods that are computationally expensive and restricted to late-stage diffusion steps, by developing an efficient technique to optimize the full generation trajectory and enable online reward adjustment without offline re-tuning. The key methodology, Direct-Align, recovers a clean image from any noisy timestep in a single, differentiable step by using a predefined noise prior, while SRPO formulates reward as the difference between scores from positive and negative text-conditioned prompts for online regularization. When fine-tuning the FLUX.1.dev model, SRPO increased the human-evaluated “Excellent” rate for overall preference from 5.3% to 29.4% and achieved convergence in just 10 minutes. For AI practitioners, this means large diffusion models can be rapidly fine-tuned for specific aesthetic qualities (e.g., photorealism) by augmenting prompts with control words, eliminating the need for costly offline reward model training and enabling highly efficient, targeted alignment. |
| SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric |
|
|
| Knowledge (Read more on arXiv or HuggingFace) |
Dipanjan Das, Sasha Goldshtein, Giovanni D’Antonio, Gal Yona, Lukas Haas |
This paper introduces SimpleQA Verified, a curated 1,000-prompt benchmark for evaluating the parametric factuality of LLMs by addressing critical limitations in OpenAI’s SimpleQA. The objective was to create a more reliable evaluation set by correcting noisy labels, topical biases, and question redundancy. The methodology involved a multi-stage filtering process on the original dataset, including semantic de-duplication, source reconciliation, topic re-balancing, and an adversarially selected difficulty filter, alongside improvements to the autorater prompt. On this new benchmark, Gemini 2.5 Pro achieves a state-of-the-art F1-score of 55.6, outperforming other frontier models, including GPT-5. The principal implication for AI practitioners is the availability of a higher-fidelity benchmark and evaluation code to more accurately measure parametric knowledge, track genuine progress in model factuality, and mitigate hallucinations. |
| Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with |
|
|
| Quantization-Aware Scheduling (Read more on arXiv or HuggingFace) |
Diana Marculescu, Natalia Frumkin |
Q-Sched introduces a post-training quantization-aware scheduler for few-step diffusion models, achieving high image fidelity with reduced model size by modifying the sampling trajectory. The objective is to enable efficient, high-fidelity image generation with few-step diffusion models by integrating quantization without degrading performance. Q-Sched modifies the diffusion model scheduler by optimizing two learnable scalar preconditioning coefficients applied to the input noise and quantized noise prediction, using a novel reference-free Joint Alignment-Quality (JAQ) loss. Q-Sched achieves a 15.5% FID improvement over FP16 4-step Latent Consistency Models while reducing model size by 4x for W4A8 quantization. AI practitioners can leverage Q-Sched to deploy few-step diffusion models with significantly reduced model size and inference cost, enabling high-fidelity image generation on resource-constrained hardware. |
| ΔL Normalization: Rethink Loss Aggregation in RLVR (Read more on arXiv or HuggingFace) |
Lili Qiu, Yuqing Yang, Yike Zhang, Xufang Luo, Zhiyuan He |
ΔL Normalization is an unbiased, minimum-variance loss aggregation technique for Reinforcement Learning with Verifiable Rewards (RLVR) that stabilizes training and improves model performance. The paper’s objective is to address the bias and excessive variance issues present in existing length-dependent and length-independent loss aggregation methods for RLVR by developing an estimator that is both unbiased and minimizes gradient variance. The key methodology involves reformulating the problem as constructing a minimum-variance unbiased estimator for the true policy gradient, proposing ΔL Normalization that uses length-dependent scaling $x_i = \frac{1}{M} \frac{L_i^\alpha}{\sum_{j=1}^G L_j^\alpha}$ for sample-level gradients. Empirical results show ΔL Normalization consistently outperforms baselines; for instance, on the CountDown 3B model, it achieved an Avg@8 score of 0.847 and a Pass@8 of 0.938, surpassing GRPO Norm’s 0.811 and 0.928, respectively, and exhibiting the highest training monotonicity with scores consistently above 0.94. AI practitioners can leverage ΔL Normalization to enhance the stability and convergence of RLVR models, especially when dealing with the dynamic and highly variable response lengths characteristic of reasoning tasks. |
Papers for 2025-09-09
| Title |
Authors |
Summary |
| Reverse-Engineered Reasoning for Open-Ended Generation (Read more on arXiv or HuggingFace) |
Wangchunshu Zhou, Minghao Liu, Qixin Xu, Haoran Que, Haozhe Wang |
This paper introduces REverse-Engineered Reasoning (REER), a novel paradigm for instilling deep reasoning in LLMs for open-ended generation by computationally discovering latent thought processes from existing high-quality outputs. The main objective is to cultivate deep reasoning for non-verifiable, creative tasks without relying on reinforcement learning or costly instruction distillation. The key methodology formulates the discovery of a reasoning trajectory as a gradient-free local search problem, where the optimal trajectory is the one that minimizes the perplexity of a known-good solution, a process used to create the DeepWriting-20K dataset. The resulting model, DeepWriter-8B, demonstrates performance competitive with proprietary systems, achieving a score of 91.28 on LongBench-Write, which surpasses GPT-4o’s score of 83.1. The principal implication for AI practitioners is that this method offers a scalable and cost-effective pathway to generate high-quality, structured reasoning data for subjective domains, enabling smaller models to acquire sophisticated planning and generation capabilities. |
| WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents (Read more on arXiv or HuggingFace) |
Aili Chen, Jingyang Li, Chi Zhang, Yunji Li, Junteng Liu |
This paper presents WebExplorer, a framework for autonomously synthesizing challenging training data for long-horizon web agents via model-based exploration and iterative query evolution. The primary objective is to overcome the scarcity of complex, multi-step training data required to develop highly capable information-seeking agents. The key methodology first uses an LLM to autonomously explore the web and generate initial query-answer pairs, then iteratively increases query difficulty through a “long-to-short” evolution process that systematically removes salient clues and adds obfuscation; this data is then used for supervised fine-tuning and reinforcement learning. The resulting 8B-parameter model, WebExplorer-8B, achieves state-of-the-art performance at its scale, scoring 15.7% on the BrowseComp-en benchmark, outperforming the much larger WebSailor-72B model. For AI practitioners, this work provides a practical and scalable method for generating difficult training data, demonstrating that iterative obfuscation is an effective strategy for developing advanced web agents without reliance on expensive manual data curation. |
| Revolutionizing Reinforcement Learning Framework for Diffusion Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Ke Shen, Ye Tian, Bowen Li, Ling Yang, Yinjie Wang |
This paper introduces TraceRL, a trajectory-aware reinforcement learning framework to enhance the reasoning capabilities of diffusion language models (DLMs). The main objective is to create a unified and effective RL framework that aligns the training objective with the model’s inference trajectory, addressing performance limitations on complex reasoning tasks across different DLM architectures. The key methodology is TraceRL, which optimizes based on intermediate generation steps (the trajectory) rather than only the final output, and incorporates a diffusion-based value model to reduce training variance and stabilize optimization. The primary result is the TraDo series of models; TraDo-8B-Instruct achieves a relative accuracy improvement of 6.1% over Qwen2.5-7B-Instruct on mathematical reasoning benchmarks. The principal implication for AI practitioners is the provision of an open-source framework and a novel RL method (TraceRL) that enables the development of diffusion models with reasoning abilities superior to stronger, larger autoregressive models, while retaining the inherent inference speed advantages of parallel decoding. |
| Does DINOv3 Set a New Medical Vision Standard? (Read more on arXiv or HuggingFace) |
Bailiang Jian, Jinpeng Lu, Haoyuan Shi, Yinda Chen, Che Liu |
This paper comprehensively benchmarks the DINOv3 vision foundation model, pre-trained on natural images, to determine if it establishes a new standard for medical imaging tasks. The primary objective is to evaluate whether DINOv3’s features can directly transfer to medical domains without specific pre-training, assessing its performance and scalability across 2D/3D classification and segmentation. Using a frozen backbone and linear probing methodology, the study reveals that DINOv3 excels in tasks visually similar to natural images, such as CT classification where DINOv3-B achieves a 0.798 AUC on the CT-RATE dataset, significantly outperforming the 0.731 AUC of the domain-specific CT-CLIP. However, its performance degrades severely on modalities with large domain shifts, such as Whole-Slide Images (WSI), Electron Microscopy (EM), and PET segmentation. The principal implication for AI practitioners is that while general-purpose foundation models offer a powerful baseline for certain medical modalities like CT and X-ray, they fail catastrophically where domain-specific, fine-grained textural or functional features are paramount, necessitating caution and domain-specific adaptation for reliable deployment. |
| Reinforced Visual Perception with Tools (Read more on arXiv or HuggingFace) |
Mingyang Fu, Zhihan Hu, Zixian Ma, Dongping Chen, Zetong Zhou |
This paper introduces REVPT, a framework that enhances multi-modal LLMs’ visual perception by training them to use visual tools through reinforcement learning. The primary objective is to overcome the limitations of supervised fine-tuning, such as poor generalization and reliance on expensive data, by enabling models to dynamically reason about and select appropriate visual tools. The methodology involves a two-stage process: an initial supervised “cold-start” phase to teach basic tool use, followed by fine-tuning using the Group Relative Policy Optimization (GRPO) algorithm with a suite of four visual tools. REVPT models achieve state-of-the-art results, with the 7B model outperforming its base instruct counterpart by 9.44% on the CV-Bench benchmark. For AI practitioners, this work demonstrates that an RL-based approach to tool usage can unlock significant performance gains in visual reasoning, providing a more robust and adaptive alternative to static supervised training for integrating specialized vision models. |
| Reinforcement Learning Foundations for Deep Research Systems: A Survey (Read more on arXiv or HuggingFace) |
Wei Han, Hannan Cao, Jingru Lin, Zhi Chen, Wenjun Li |
This survey systematizes the reinforcement learning (RL) foundations for building deep research systems, covering data synthesis, RL methods, agentic architectures, training frameworks, and evaluation benchmarks. The paper’s objective is to provide the first comprehensive survey dedicated to RL for deep research systems, motivating the use of RL over supervised fine-tuning (SFT) and preference-based methods (DPO) for training agents on complex, multi-step, tool-interactive tasks. The methodology is a literature survey that analyzes and categorizes recent work (post-February 2025) across three core axes: (1) data synthesis and curation for RL, (2) RL methods for agentic research, and (3) agentic RL training frameworks, supplemented by reviews of agent architectures and benchmarks. The survey distills a standard agentic RL pipeline consisting of an optional SFT cold start, templated rollouts with explicit tool tags, outcome-based rewards, and PPO/GRPO optimization with a reference-KL penalty. It highlights specific algorithmic improvements, such as Duplicating Sampling Policy Optimization (DUPO) from WebSailor, which reportedly provides a ~2-3x training speed-up over DAPO through dynamic sampling. The principal implication for AI practitioners is to adopt an RL-centric approach for developing deep research agents, starting with the identified baseline pipeline and focusing innovations on curriculum design, reward shaping for search efficiency, and leveraging scalable, asynchronous training frameworks to overcome rollout and credit assignment bottlenecks. |
| Focusing by Contrastive Attention: Enhancing VLMs’ Visual Reasoning (Read more on arXiv or HuggingFace) |
Baolong Bi, Lingrui Mei, Yiwei Wang, Shenghua Liu, Yuyao Ge |
This paper introduces CARVE, a training-free method that enhances Vision-Language Model (VLM) reasoning by contrastively filtering visual noise using the model’s own attention mechanisms. The main objective is to mitigate the negative impact of visual complexity on VLM performance by developing a method to isolate task-relevant visual signals, based on an initial investigation into the correlation between visual complexity, attention entropy, and reasoning accuracy. The key methodology, Contrastive Attention Refinement for Visual Enhancement (CARVE), contrasts attention maps from a general instruction (capturing visual noise) with those from a task-specific question (capturing semantic signal) to produce a refined attention mask, which is then used to crop the image to relevant regions for a final inference pass. Primary results demonstrate consistent performance enhancement across multiple models and datasets, notably improving the LLAVA1.5-7B model’s accuracy on the V* benchmark from 38.7% to 66.5%, a relative increase of 71.83%. The principal implication for AI practitioners is that CARVE can be used as a plug-and-play, inference-time technique to significantly improve the accuracy of existing VLMs on visually complex tasks without requiring any model retraining, offering a resource-efficient method to boost performance. |
| UniVerse-1: Unified Audio-Video Generation via Stitching of Experts (Read more on arXiv or HuggingFace) |
Xinyao Liao, Ling-Hao Chen, Aojie Li, Wei Zuo, Duomin Wang |
The paper introduces UniVerse-1, an open-source model for simultaneous audio-video generation by fusing pre-trained expert models. The research objective is to develop a unified, Veo-like system capable of generating temporally and semantically coordinated audio-visual content to bridge the gap in open-source research. The core methodology is a “Stitching of Experts” (SoE) technique that deeply integrates a pre-trained video model (WAN2.1) and a music model (Ace-step) at the block level, combined with an online annotation pipeline and an independent noise sampling strategy. On the newly introduced Verse-Bench benchmark, UniVerse-1 achieves a superior identity preservation (ID) score of 0.89 in video generation tasks. The principal implication for AI practitioners is that the SoE framework provides an efficient pathway to construct complex multimodal generative models by leveraging existing unimodal foundations, significantly reducing the need for training entirely new systems from scratch. |
| Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, |
|
|
| but Not Direct the Play? (Read more on arXiv or HuggingFace) |
Rui Chen, Huijuan Huang, Xinting Hu, Yuan Wang, lioooox |
The paper introduces T2I-COREBENCH, a comprehensive and complex benchmark designed to evaluate the composition and reasoning capabilities of text-to-image models. The main objective is to systematically assess T2I model performance on tasks requiring high compositional density and multi-step logical inference, moving beyond simple explicit instruction following. The methodology involves a 12-dimensional evaluation taxonomy for composition and reasoning, with 1,080 complex prompts evaluated via an automated MLLM-based checklist that asks specific yes/no questions for fine-grained analysis. Primary results from evaluating 27 models reveal a significant performance disparity between the two capabilities; the top open-source model, Qwen-Image, scored 78.0 in composition but only 49.3 in reasoning, demonstrating that reasoning is a critical bottleneck. The principal implication for AI practitioners is that current T2I models excel at rendering explicit elements (“painting”) but struggle significantly with inferring and generating implicit, logically-derived content (“thinking”), identifying a key area for future model architecture and training improvements. |
| Interleaving Reasoning for Better Text-to-Image Generation (Read more on arXiv or HuggingFace) |
Shixiang Tang, Shaosheng Cao, Zheyong Xie, Shuang Chen, Wenxuan Huang |
The paper introduces Interleaving Reasoning Generation (IRG), a framework that improves text-to-image synthesis by alternating between text-based reasoning and image generation for iterative refinement. The research aims to determine if a multi-turn, reflective process can enhance T2I models’ instruction following, visual quality, and fine-grained detail preservation. The methodology involves a two-step process: first, generating a text-based plan to guide an initial image synthesis, and second, reflecting on this image to produce textual refinements that guide the generation of a final, improved image, all trained via a two-stage strategy on a curated IRGL-300K dataset. IRG achieves state-of-the-art results across multiple benchmarks, scoring 0.85 on GenEval, which is a 5-10 point absolute gain over previous methods. For AI practitioners, this work demonstrates that decomposing a complex generation task into an explicit “reason-generate-reflect-refine” loop, trained on specific sub-tasks, is a powerful paradigm for enhancing the fidelity and controllability of generative models. |
| Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI |
|
|
| Agents (Read more on arXiv or HuggingFace) |
James Zou, Jonathan K. Pritchard, Joe R. Davis, Jiacheng Miao |
Paper2Agent is an automated multi-agent framework that converts research papers and their codebases into interactive, reliable AI agents accessible via natural language. The primary objective is to overcome technical barriers in research dissemination by transforming passive artifacts (papers, code) into active, executable systems. The methodology uses a multi-agent workflow to analyze a paper’s codebase, encapsulate its core functionalities into validated tools, and package them into a standardized Model Context Protocol (MCP) server for integration with LLM-based agents. The framework demonstrated its efficacy by creating an agent for the AlphaGenome paper that achieved 100% accuracy in reproducing numerical results on 15 tutorial-based queries and 15 novel queries. For AI practitioners, this work provides a concrete framework for “agentifying” research code, enabling the packaging of complex computational methods into standardized, reproducible, and LLM-callable tools that accelerate the translation of research into practical applications. |
| Scaling up Multi-Turn Off-Policy RL and Multi-Agent Tree Search for LLM |
|
|
| Step-Provers (Read more on arXiv or HuggingFace) |
Xia Xiao, Kun Yuan, Yanchen Nie, Zeyu Zheng, Ran Xin |
The paper introduces BFS-Prover-V2, a system that scales LLMs for automated theorem proving using a dual approach for training and inference. The primary objective is to overcome the dual scaling challenges in LLM-based provers: training-time performance plateaus in reinforcement learning and inference-time combinatorial search complexity. The methodology combines a multi-stage, off-policy RL framework inspired by AlphaZero, which uses adaptive perplexity-based data filtering and periodic retraining to escape local optima, with a planner-enhanced multi-agent search architecture where a high-level planner decomposes theorems into subgoals for specialized prover agents. The system achieves state-of-the-art results for step-provers, solving 95.08% of the MiniF2F benchmark and 41.4% on the ProofNet test set. For AI practitioners, this work provides a concrete framework for sustaining long-term improvement in RL-trained LLMs by using perplexity-based curriculum learning and periodic retraining to overcome performance stagnation, a technique applicable to other complex, multi-turn reasoning domains. |
| R^textbf{2AI}: Towards Resistant and Resilient AI in an |
|
|
| Evolving World (Read more on arXiv or HuggingFace) |
Bowen Zhou, Chaochao Lu, Jie Fu, Xiang Wang, Youbang Sun |
This paper introduces R²AI (Resistant and Resilient AI), a conceptual framework that treats AI safety as an intrinsic capability that coevolves with intelligence, inspired by biological immunity. The primary objective is to address the persistent gap between rapidly advancing AI capabilities and lagging safety progress by proposing a proactive framework that moves beyond reactive, post-hoc safety measures. The key methodology is an architectural framework comprising four components: a Fast Safe Model for real-time reactions, a Slow Safe Model for reflective reasoning, a “Safety Wind Tunnel” for continuous adversarial coevolution, and feedback loops with the external deployment environment. The primary result is the proposed framework itself, which is motivated by the quantitative finding that leading foundation models exhibit a significant safety-capability gap, with some models showing capability scores near 90 but safety scores around 65. The principal implication for AI practitioners is to shift from applying static, post-development safety patches to architecting systems with integrated, continually learning safety mechanisms, such as a dual fast-slow architecture and a dedicated adversarial simulation environment for the entire model lifecycle. |
| Llama-GENBA-10B: A Trilingual Large Language Model for German, English |
|
|
| and Bavarian (Read more on arXiv or HuggingFace) |
Hoi-Fong Mak, Gokul Ramakrishnan, Stefan Schweter, Jophin John, Michael Hoffmann |
This paper introduces Llama-GENBA-10B, a 10-billion parameter trilingual foundation model for German, English, and Bavarian, developed via continual pretraining on a Llama 3.1-8B base. The primary objective was to create a high-performing model that mitigates English-centric bias and supports the low-resource Bavarian dialect by addressing data scarcity and optimizing the tokenizer. The methodology involved scaling the architecture using block expansion, creating a custom tokenizer with a 20% vocabulary increase, and performing staged continual pretraining on a 164B token corpus (82B English, 82B German, 80M Bavarian) on a single Cerebras CS-2 system. The instruction-tuned variant achieves state-of-the-art performance in Bavarian, outperforming models like gemma-2-9b-it and Apertus-8B-Instruct-2509, while the base model scores a competitive 0.7364 on Winogrande. For AI practitioners, this work provides a resource-efficient and energy-documented (35.23 MWh) blueprint for extending existing foundation models to effectively support low-resource languages without requiring training from scratch. |
| Test-Time Scaling in Reasoning Models Is Not Effective for |
|
|
| Knowledge-Intensive Tasks Yet (Read more on arXiv or HuggingFace) |
See-Kiong Ng, Bryan Hooi, James Xu Zhao |
This paper demonstrates that increasing inference-time computation via test-time scaling in reasoning models does not consistently improve accuracy and can increase hallucinations on knowledge-intensive tasks. The study’s primary objective is to determine if test-time scaling is an effective strategy for improving the performance of large reasoning models on knowledge-intensive tasks that require high factual accuracy. The methodology involves evaluating 12 reasoning models (e.g., GPT-5 mini, Gemini 2.5 Flash) on two benchmarks (SimpleQA, FRAMES) by systematically increasing inference-time computation through techniques like adjusting a reasoning effort parameter, increasing a thinking budget in tokens, or using “budget forcing,” while measuring accuracy and hallucination ratio. The primary results indicate that more computation does not reliably improve accuracy and can increase hallucinations; for example, on SimpleQA, GPT-5-mini’s hallucination ratio increased by over 15% as its reasoning length grew from 300 to 3300 tokens. The analysis shows that reduced hallucinations often result from abstention, while increased hallucinations stem from the model attempting previously unanswered questions. The principal implication for AI practitioners is that naively increasing inference-time computation is not a reliable strategy for enhancing factual accuracy and may be counterproductive. While enabling a baseline level of thinking is beneficial compared to non-thinking, simply extending it further is not an effective optimization for factual robustness in knowledge-intensive applications. |
| D-HUMOR: Dark Humor Understanding via Multimodal Open-ended Reasoning (Read more on arXiv or HuggingFace) |
Dhanvin Sanjay Namboodiri, Rishi Bharat Junghare, Shahid Shafi Dar, Mohammad Zia Ur Rehman, Sai Kartheek Reddy Kasu |
This research introduces the D-HUMOR dataset and a reasoning-augmented multimodal framework, TCRNet, for detecting, categorizing, and rating the intensity of dark humor in memes. The primary objective is to develop a robust method for understanding multimodal dark humor by addressing the challenge of interpreting implicit, context-dependent cues in memes. The key methodology involves using a Large Vision-Language Model with a novel Role-Reversal Self-Loop to generate and refine structured explanations, which are then fused with image and OCR text features via a Tri-stream Cross-Reasoning Network using pairwise attention. The proposed TCRNet model achieves state-of-the-art performance, attaining 75.00% accuracy in binary dark humor detection, outperforming all unimodal and zero-shot VLM baselines. The principal implication for AI practitioners is that for complex, subjective multimodal tasks, integrating an explicit, structured reasoning stream is critical; ablation studies show that removing this reasoning component causes a severe performance drop in target identification Macro-F1 from 60.54% to 35.11%, demonstrating that fusing raw multimodal data alone is insufficient. |
| MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI |
|
|
| Agents (Read more on arXiv or HuggingFace) |
Zhengxi Lu, Weiqing He, Yaozhen Liang, Guangyi Liu, Pengxiang Zhao |
This paper introduces MAS-Bench, a benchmark for evaluating hybrid mobile agents that combine GUI operations with programmatic shortcuts like APIs, deep links, and RPA scripts. The main objective is to systematically assess an agent’s ability to discover, utilize, and autonomously generate shortcuts to improve task performance on real-world mobile applications. The methodology involves 139 complex tasks, a knowledge base of 88 predefined shortcuts, and a novel two-stage framework to evaluate the quality of agent-generated shortcuts based on their impact on a baseline agent’s performance. The primary result shows that hybrid agents significantly outperform GUI-only agents, with the shortcut-augmented MAS-MobileAgent achieving a 64.1% success rate on single-app tasks compared to its 44.6% GUI-only counterpart. The principal implication for AI practitioners is that integrating a well-defined shortcut knowledge base is a validated strategy for substantially improving mobile agent success rates and efficiency, and MAS-Bench provides a standardized platform to guide and measure these improvements. |
| Saturation-Driven Dataset Generation for LLM Mathematical Reasoning in |
|
|
| the TPTP Ecosystem (Read more on arXiv or HuggingFace) |
Damien Sileo, Valentin Quesnel |
This paper presents a framework for generating guaranteed-valid mathematical reasoning datasets by applying saturation-based theorem proving to the TPTP axiom library, avoiding the use of LLMs in the generation loop. The objective is to address the scarcity of high-quality, logically sound data by creating a scalable engine that produces three difficulty-controlled tasks—Conjecture Entailment, Minimal Premise Selection, and Proof Graph Reconstruction—to rigorously benchmark and improve LLM deductive capabilities. The core methodology uses the E-prover to exhaustively derive theorems from TPTP axioms, filters them for mathematical “interestingness” using AGInTRater, and deconstructs the resulting derivation graphs into tasks validated by the Vampire prover. Zero-shot experiments reveal a severe performance degradation with increased logical complexity; for instance, the gpt-5 model’s average accuracy across all tasks dropped from 93% on “Easy” problems to 44% on “Very Hard” problems, with performance collapsing on the structural Proof Reconstruction task. The principal implication for AI practitioners is that this framework provides a scalable source of high-quality symbolic data for fine-tuning models to address a core deficit in multi-step, structural reasoning that is not overcome by model scaling alone. |
Papers for 2025-09-08
| Title |
Authors |
Summary |
| Why Language Models Hallucinate (Read more on arXiv or HuggingFace) |
Edwin Zhang, Santosh S. Vempala, Ofir Nachum, Adam Tauman Kalai |
This paper theoretically analyzes how language model training statistically induces hallucinations and argues they persist because evaluation benchmarks reward guessing over expressing uncertainty. The main objective is to identify the statistical causes of hallucinations in both pretraining and post-training stages, framing them as a natural outcome of the training pipeline. The key methodology involves a reduction from generative modeling to a binary classification problem (“Is-It-Valid”) to derive lower bounds on error rates, complemented by a meta-analysis of influential evaluation benchmarks. Primary results show the generative error rate is lower-bounded by twice the binary misclassification rate on validity, and a review of 10 popular benchmarks reveals that 9 use binary grading which provides no credit for uncertain (“IDK”) responses. The principal implication for AI practitioners is that mitigating hallucinations requires modifying the scoring of mainstream evaluations to stop penalizing and instead reward appropriate expressions of uncertainty, for instance by incorporating explicit confidence targets and penalties for incorrect answers. |
| Symbolic Graphics Programming with Large Language Models (Read more on arXiv or HuggingFace) |
Kaipeng Zhang, Zeju Qiu, Haoquan Zhang, Yamei Chen, YangyiH |
This research introduces a benchmark and a reinforcement learning (RL) framework with cross-modal rewards to evaluate and enhance the capability of Large Language Models (LLMs) to generate Symbolic Graphics Programs (SGPs) from text. The main objective is to assess existing LLMs’ SGP generation abilities and develop a method to improve open-source models’ performance to a level competitive with proprietary systems. The key methodology involves finetuning the Qwen-2.5-7B model using a critic-free RL algorithm where the reward signal is derived from a format-validity gate combined with text-image (SigLIP) and image-image (DINO) alignment scores. The RL-finetuned model substantially improves its overall compositional generation score on the SGP-COMPBENCH from 8.8 to 60.8, outperforming other open-source models and approaching frontier model performance. For AI practitioners, the principal implication is that reinforcement learning with rewards from vision foundation models provides a scalable, data-efficient method for distilling visual grounding into LLMs, enabling smaller models to perform precise, visually-aligned program synthesis without requiring paired ground-truth datasets. |
| Set Block Decoding is a Language Model Inference Accelerator (Read more on arXiv or HuggingFace) |
Jeremy Reizenstein, Daniel Haziza, Marton Havasi, Heli Ben-Hamu, Itai Gat |
Set Block Decoding (SBD) is a paradigm that accelerates language model inference by generating multiple, non-consecutive future tokens in parallel within a standard autoregressive architecture. The paper’s objective is to accelerate the computationally expensive decoding stage by integrating standard next token prediction (NTP) with masked token prediction (MATP) without requiring architectural changes or sacrificing performance. The key methodology involves fine-tuning existing NTP models with a combined loss, enabling the use of advanced solvers from discrete diffusion literature, such as the Entropy Bounded (EB) Sampler, to dynamically select tokens for parallel generation. The primary result is a demonstrated 3-5x reduction in the number of forward passes required for generation on fine-tuned Llama-3.1 8B and Qwen-3 8B models, while maintaining equivalent performance to standard NTP training. For AI practitioners, the principal implication is that SBD offers a practical method to achieve significant inference speedups by simply fine-tuning existing models, as it maintains compatibility with exact KV-caching and does not degrade model accuracy. |
| WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning (Read more on arXiv or HuggingFace) |
Amit Namburi, Yash Vishe, Gagan Mundada, ZacharyNovack, XinXuNLPer |
This paper introduces WildScore, the first in-the-wild benchmark for evaluating Multimodal Large Language Models (MLLMs) on symbolic music reasoning. The primary objective is to assess MLLMs’ ability to interpret real-world music scores and answer complex musicological queries. The methodology involves constructing a multiple-choice question answering dataset from genuine musical compositions and user-generated questions sourced from public forums, categorized using a systematic musicological taxonomy. The empirical evaluation shows that the state-of-the-art model, GPT-4.1-mini, achieves a peak accuracy of 68.31%, indicating that current MLLMs struggle with tasks requiring deep symbolic abstraction and rhythmic interpretation. For AI practitioners, this highlights a critical gap in MLLM capabilities for specialized, dense symbolic domains and suggests that future models require improved pretraining on schematic notation and more robust vision-language alignment for such tasks. |
| LatticeWorld: A Multimodal Large Language Model-Empowered Framework for |
|
|
| Interactive Complex World Generation (Read more on arXiv or HuggingFace) |
Zhan Zhao, Wei Jia, Tongwei Gu, Zhengxia Zou, Yinglin Duan |
LatticeWorld is a framework that leverages lightweight multimodal LLMs and the Unreal Engine to generate interactive, large-scale 3D worlds from textual and visual instructions. The primary objective is to develop an effective and efficient framework for generating complex, dynamic, and interactive 3D worlds by integrating the spatial understanding and structured generation capabilities of LLMs with an industry-grade rendering pipeline, using multimodal user inputs. The methodology involves a two-stage LLM process: a fine-tuned LLaMA-2-7B model (LLML) first generates a sequential symbolic representation of the scene layout from text and vision (height map) inputs, followed by a second LLM (LLMc) that generates detailed environmental and agent configurations in JSON format; these intermediate representations are then procedurally rendered in Unreal Engine 5. The framework demonstrates superior performance in layout generation accuracy and visual fidelity compared to other generative models. Quantitatively, LatticeWorld achieves over a 90x increase in production efficiency, reducing the creation time for a complex environment from 55 days for a human artist to less than 0.6 days. For AI practitioners, this framework provides a method to rapidly generate diverse, physically-realistic, and interactive 3D simulation environments for training and testing embodied AI agents, reducing reliance on manual content creation and accelerating the development cycle for applications in autonomous systems and robotics. |
| LuxDiT: Lighting Estimation with Video Diffusion Transformer (Read more on arXiv or HuggingFace) |
Sanja Fidler, Igor Gilitschenski, Zan Gojcic, Kai He, Ruofan Liang |
LuxDiT is a video diffusion transformer model that generates high-dynamic-range (HDR) environment maps from single LDR images or videos. The main objective is to develop a data-driven method for accurately estimating HDR scene illumination from casually captured LDR visual inputs, overcoming the scarcity of paired ground-truth HDR data. The method fine-tunes a pre-trained video diffusion transformer (DiT) conditioned on visual input tokens, representing HDR outputs using a dual-tonemapped LDR format to handle high dynamic range, and employs a two-stage training process involving pre-training on synthetic data for physical correctness followed by low-rank adaptation (LoRA) on real HDR panoramas for semantic alignment. The model outperforms existing state-of-the-art methods; on the Laval Outdoor dataset, it reduces the mean peak angular error for sunlight direction by nearly 50% compared to the DiffusionLight baseline, from 44.4 to 23.7 degrees. The principal implication for AI practitioners is the ability to generate realistic HDR lighting for applications like virtual object insertion, augmented reality, and synthetic data generation directly from standard LDR images or videos, thereby removing the need for specialized capture equipment. |
| WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool (Read more on arXiv or HuggingFace) |
Wenzheng Chang, Yifan Wang, Jianjun Zhou, Zizun Li, ghy0324 |
WinT3R is a feed-forward model for real-time, high-quality 3D reconstruction and camera pose estimation from streaming images. The main objective is to resolve the trade-off between reconstruction quality and real-time performance in online methods by improving inter-frame information exchange and efficiently incorporating global context. The key methodology involves a sliding window mechanism for direct interaction between adjacent image tokens and a global camera token pool that maintains a compact representation of all historical camera information for robust pose prediction. The model achieves state-of-the-art results on multiple benchmarks, demonstrating superior reconstruction accuracy and the fastest processing speed among compared methods at 17.2 FPS. For AI practitioners, the principal implication is a novel and efficient architecture using a camera token pool as a lightweight global memory, enabling high-fidelity, real-time 3D perception in applications like robotics and AR without significant computational overhead. |
| On Robustness and Reliability of Benchmark-Based Evaluation of LLMs (Read more on arXiv or HuggingFace) |
Kevin Roitero, Stefano Mizzaro, Vincenzo Della Mea, Riccardo Lunardi |
This study evaluates the robustness of 34 LLMs to linguistic variation, finding that while relative model rankings are preserved, absolute performance on six standard benchmarks degrades significantly when questions are paraphrased. The primary objective is to assess the reliability of benchmark-based evaluations and the robustness of LLMs by measuring performance variations when benchmark questions are systematically reworded while preserving semantic meaning. The methodology involved automatically generating five paraphrases for all questions in six multiple-choice benchmarks, then evaluating 34 LLMs in a zero-shot setting to measure changes in accuracy, response consistency, and ranking stability (Kendall’s τ). The primary results show that while relative model rankings remain highly stable (Kendall’s τ > 0.9 across all benchmarks), absolute performance degrades and models show significant inconsistency, with 15-30% of questions receiving two or more different answers across the paraphrased versions. The principal implication for AI practitioners is that standard benchmark scores overestimate an LLM’s true generalization capabilities; therefore, evaluations should incorporate robustness testing against linguistic variations to obtain a more reliable assessment of real-world performance. |
| MedVista3D: Vision-Language Modeling for Reducing Diagnostic Errors in |
|
|
| 3D CT Disease Detection, Understanding and Reporting (Read more on arXiv or HuggingFace) |
Vanessa Wildman, Jike Zhong, Yuxiang Lai, Yenho Chen, Yuheng Li |
This paper introduces MedVista3D, a multi-scale semantic-enriched vision-language pretraining framework for 3D CT analysis designed to reduce diagnostic errors. The main objective is to overcome the limitations of existing models by jointly enabling precise localized disease detection, global volume-level reasoning, and semantically consistent reporting within a single framework. Its methodology employs a multi-scale alignment loss to learn both local organ-level and global volume-level representations simultaneously, complemented by Large Language Model (LLM) rewrites and a Radiology Semantic Matching Bank (RSMB) for semantically robust text-image alignment. MedVista3D achieves state-of-the-art results, with one variant obtaining an AUC of 0.782 on global zero-shot disease classification on the CT-RATE dataset, outperforming the CT-CLIP baseline by 7.4 percentage points. For AI practitioners, the principal implication is that integrating multi-scale alignment with semantic text enhancement provides a robust pretraining strategy for building 3D medical foundation models with superior generalization and transferability to diverse downstream tasks like classification, retrieval, and segmentation. |
| U-ARM : Ultra low-cost general teleoperation interface for robot |
|
|
| manipulation (Read more on arXiv or HuggingFace) |
Junda Huang, Zewei Ye, Chenyang Shi, Zhaoye Zhou, Yanwen Zou |
This paper presents U-Arm, an open-source, ultra-low-cost leader-follower teleoperation framework for robot manipulation with a bill of materials (BOM) of $50.5 for the 6-DoF version. The research objective is to create a rapidly adaptable and user-friendly teleoperation system compatible with a wide range of commercial robotic arms to facilitate large-scale, high-quality data collection. The methodology involves designing three structurally distinct, 3D-printed leader arm configurations that are mechanically isomorphic to common robot joint arrangements, using modified servos for joint angle sensing, and applying a calibration and filtering algorithm to improve control. Experimental results show U-Arm achieves 39% higher data collection efficiency compared to a Joycon controller across multiple manipulation tasks, while maintaining a comparable success rate. For AI practitioners, this work provides a low-cost, open-source hardware and software solution that significantly reduces the barrier to acquiring real-world human demonstration data for training robot learning policies. |
| Behavioral Fingerprinting of Large Language Models (Read more on arXiv or HuggingFace) |
Xing Li, Zhiyuan Yang, Ying Zhang, Hui-Ling Zhen, Zehua Pei |
This paper introduces “Behavioral Fingerprinting,” a framework using a judge LLM to evaluate Large Language Models on cognitive and interactive styles beyond standard performance metrics. The research objective is to create multi-faceted profiles that reveal “how a model thinks” by probing dimensions like reasoning, robustness, sycophancy, and world model integrity. The methodology utilizes a curated Diagnostic Prompt Suite, with responses from 18 target models being automatically scored by a powerful judge model (Claude-opus-4.1) against detailed, prompt-specific rubrics. Results show that while core reasoning abilities are converging among top-tier models, alignment-related behaviors diverge significantly, with sycophancy resistance scores ranging from 1.00 (complete resistance) to 0.25 (high sycophancy). The principal implication for AI practitioners is that alignment traits are a direct consequence of specific developer strategies, not an emergent property of scale, making these behavioral fingerprints critical for selecting models whose interactive styles match application-specific safety and reliability requirements. |
| Bootstrapping Task Spaces for Self-Improvement (Read more on arXiv or HuggingFace) |
Yoram Bachrach, Andrei Lupu, Minqi Jiang |
The paper introduces Exploratory Iteration (EXIT), an autocurriculum reinforcement learning method that trains LLMs for multi-step self-improvement by dynamically creating and prioritizing single-step iteration tasks from the model’s own solution histories. The objective is to develop a sample-efficient RL training framework that enables LLMs to perform multi-step self-improvement at inference time without the high cost and arbitrary depth limits of naively training on full K-step rollouts. EXIT uses Group-Relative Policy Optimization (GRPO) to train an LLM on single-step self-improvement tasks by maintaining a buffer of partial solution histories, sampling from it based on group return variance as a learning potential metric, and augmenting the task space with explicit self-divergence prompts. Across domains, EXIT produced policies with strong inference-time improvement; on a collection of math test sets, the full EXIT method achieved a final accuracy of 20.4% after 16 self-improvement steps, a net improvement of +2.0 percentage points over its initial response and superior to the standard GRPO baseline. For AI practitioners, EXIT offers a method to fine-tune LLMs for iterative refinement by decomposing long-horizon improvement into prioritized single-step tasks, enhancing performance in complex, scaffolded applications like automated ML engineering without requiring additional compute over standard RL fine-tuning. |
Papers for 2025-09-05
| Title |
Authors |
Summary |
| From Editor to Dense Geometry Estimator (Read more on arXiv or HuggingFace) |
Lang Nie, Rongying Liu, Lei Sun, Chunyu Lin, exander |
This paper introduces FE2E, a framework that adapts pre-trained image editing models for high-performance, zero-shot monocular dense geometry estimation. The objective is to demonstrate that image editing models are a more suitable foundation than text-to-image generative models for this task and to develop an effective adaptation protocol. Key methodologies include reformulating the editor’s flow matching loss into a “consistent velocity” objective for deterministic prediction, using logarithmic quantization to resolve precision conflicts, and implementing a cost-free joint estimation of depth and normals by repurposing the Diffusion Transformer’s (DiT) architecture. FE2E achieves state-of-the-art results, including a 35% performance gain in Absolute Relative error on the ETH3D depth estimation dataset compared to the next-best method, while being trained on 100x less data than competing large-scale models. The principal implication for AI practitioners is that fine-tuning image editing models, which possess stronger inherent structural priors, offers a more data-efficient and performant pathway for dense vision tasks than adapting T2I generative models. |
| Drivel-ology: Challenging LLMs with Interpreting Nonsense with Depth (Read more on arXiv or HuggingFace) |
Chi-Li Chen, Zi Yan Chang, Chia-Yi Hsiao, Chenghao Xiao, Yang Wang |
This research introduces “Drivelology,” a linguistic phenomenon of syntactically coherent but pragmatically paradoxical text, and presents the DRIVELHUB benchmark to evaluate LLMs’ comprehension of such layered semantics. The primary objective is to assess if LLMs can move beyond surface-level pattern matching to grasp the implicit, non-linear meanings embedded in Drivelological text, which requires deep contextual and cultural inference. The methodology involves evaluating various LLMs on the new multilingual DRIVELHUB dataset across four tasks: binary classification (Detection), multi-label classification (Tagging), generation (Narrative Writing), and multiple-choice question answering (Narrative Selection). Key results demonstrate a significant performance deficit in current models, with the top-scoring model on the Hard Narrative Selection task achieving only 26.78% accuracy, exposing a critical failure in complex reasoning. For AI practitioners, this research highlights that statistical fluency is not a reliable proxy for cognitive comprehension, indicating that models for applications requiring nuanced human interaction must be evaluated on benchmarks that specifically test for understanding of multi-layered pragmatic paradox and implicit rhetoric. |
| Towards a Unified View of Large Language Model Post-Training (Read more on arXiv or HuggingFace) |
Hongyi Liu, Youbang Sun, Yuxin Zuo, Xingtai Lv, iseesaw |
This paper presents a unified framework for LLM post-training, demonstrating that Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT) are instances of a single optimization process. The main objective is to formalize the relationship between online (RL) and offline (SFT) training methods by deriving a common gradient formulation. The authors introduce the Unified Policy Gradient Estimator (UPGE) and propose a practical algorithm, Hybrid Post-Training (HPT), which dynamically switches between SFT and RL objectives based on real-time task performance. HPT demonstrates superior performance over baselines; for instance, using Qwen2.5-Math-7B on the AIME 2024 benchmark, HPT achieved a score of 33.0, a 6.9-point improvement over the LUFFY baseline. For AI practitioners, HPT provides an adaptive method to combine SFT and RL, potentially yielding better model performance without the high cost and tuning complexity of sequential SFT-then-RL pipelines. |
| Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow |
|
|
| Real Instructions? (Read more on arXiv or HuggingFace) |
Yu Fu, Ruijie Miao, Xinping Lei, Qinyan Zhang, zhangysk |
This paper introduces Inverse IFEval, a benchmark to evaluate an LLM’s ability to follow counter-intuitive instructions that conflict with patterns learned during supervised fine-tuning. The research objective is to measure an LLM’s “Counter-intuitive Ability,” specifically its capacity to overcome training-induced cognitive inertia and comply with adversarial instructions. The methodology involves a dataset of 1012 Chinese and English questions across eight counter-intuitive categories (e.g., Intentional Textual Flaws, Code without Comments), evaluated via an optimized LLM-as-a-Judge framework. Results show that models struggle significantly, with the fine-tuned Qwen3-235B-A22B-Instruct model’s rank dropping from 5th on the conventional IFEval benchmark to 15th on Inverse IFEval. For AI practitioners, the principal implication is that current alignment processes can induce overfitting to narrow patterns, and future efforts must focus on mitigating cognitive inertia to improve instruction-following reliability on out-of-distribution user requests. |
| DeepResearch Arena: The First Exam of LLMs’ Research Abilities via |
|
|
| Seminar-Grounded Tasks (Read more on arXiv or HuggingFace) |
Jiaxuan Lu, Meiqi Tu, Junchi Yu, Chen Yang, haiyuanwan |
i) This paper introduces DeepResearch Arena, a novel benchmark derived from academic seminar transcripts to evaluate the multi-stage research capabilities of LLMs while minimizing data leakage. ii) The main objective is to create a scalable and authentic benchmark that faithfully evaluates the research abilities of deep research agents by grounding tasks in real-world expert discourse, overcoming the data contamination risks and scalability limitations of existing benchmarks. iii) The methodology involves a Multi-Agent Hierarchical Task Generation (MAHTG) system that automatically extracts research inspirations from seminar transcripts and synthesizes them into over 10,000 research tasks, which are evaluated using a hybrid protocol combining Keypoint-Aligned Evaluation (KAE) for factual grounding and Adaptively-generated Checklist Evaluation (ACE) for higher-order reasoning. iv) The primary results demonstrate substantial performance gaps across current models, with grok-4 achieving the highest factual coverage on English tasks (83.3% Keypoint Supported Rate), while o4-mini-deepresearch attained the highest subjective reasoning score (4.03 ACE score). v) The principal implication for AI practitioners is that DeepResearch Arena provides a robust framework and dataset for benchmarking LLM agents on cognitively demanding, open-ended research tasks, enabling a more accurate assessment of their practical utility for automating complex scientific workflows beyond standard question-answering. |
| Transition Models: Rethinking the Generative Learning Objective (Read more on arXiv or HuggingFace) |
Yangguang Li, Xiangyu Yue, Xiaoyu Yue, Yiyuan Zhang, GoodEnough |
Transition Models (TiM) introduce a generative paradigm that learns the entire solution manifold of the generative process, enabling high-fidelity synthesis across arbitrary step sizes from a single model. The objective is to resolve the trade-off between the high computational cost of iterative diffusion models and the quality ceiling of efficient few-step generators by creating a unified model effective at any number of function evaluations (NFEs). The key methodology is a novel training objective derived from an exact “State Transition Identity” equation, made scalable by a finite-difference approximation called the Differential Derivation Equation (DDE) that avoids computationally expensive Jacobian-Vector Products and is compatible with distributed training. The 865M parameter TiM achieves a GenEval score of 0.67 at 1-NFE and monotonically improves to 0.83 at 128-NFE, outperforming 12B parameter models like FLUX.1 across all evaluated step counts. For AI practitioners, this eliminates the need for separate models or distillation pipelines for different inference budgets, offering a single, flexible model for applications requiring either real-time generation or maximum-quality rendering. |
| NER Retriever: Zero-Shot Named Entity Retrieval with Type-Aware |
|
|
| Embeddings (Read more on arXiv or HuggingFace) |
Oren Glickman, Yoav Goldberg, Uri Katz, Or Shachar |
i) This paper presents NER Retriever, a zero-shot framework for ad-hoc Named Entity Retrieval that creates compact, type-aware embeddings from internal Large Language Model representations. ii) The primary objective is to retrieve all text segments mentioning entities of a type defined at query time by mapping both entity mentions and open-ended type descriptions into a shared semantic space for similarity search. iii) The methodology involves extracting value vectors from an intermediate transformer block (layer 17 of LLaMA 3.1 8B) and using a lightweight, contrastively-trained multilayer perceptron to project these representations into a discriminative embedding space. iv) The system significantly outperforms baselines on two of three benchmarks, achieving an R-Precision of 0.32 on the MultiCoNER 2 dataset, more than three times higher than the E5-Mistral dense retrieval model (0.09). v) The principal implication for AI practitioners is that internal LLM representations, specifically mid-layer value vectors, encode fine-grained type information more effectively than standard top-layer embeddings, offering a practical method for building scalable and more accurate schema-free entity retrieval systems. |
| Few-step Flow for 3D Generation via Marginal-Data Transport Distillation (Read more on arXiv or HuggingFace) |
Lingxi Xie, Chen Yang, Jiemin Fang, Zanwei Zhou, thewhole |
This paper introduces MDT-dist, a novel distillation framework that significantly accelerates flow-based 3D generation by directly learning the marginal-data transport. The research objective is to distill a pretrained, multi-step 3D flow model into a generator capable of producing high-fidelity 3D assets in only one or two sampling steps. The methodology proposes two complementary objectives: Velocity Matching (VM) to stably match velocity fields between student and teacher models, and Velocity Distillation (VD) to perform probability density distillation using these learned velocity fields. Applied to the TRELLIS framework, the method reduces sampling steps from 25 per component to just one, achieving a 9.0x speedup and 0.68s inference latency on an A800 GPU while maintaining high geometric fidelity. For AI practitioners, this framework provides a direct method to drastically reduce the inference cost of complex generative models for 3D assets, enabling their use in time-sensitive applications like interactive content creation. |
| Video-MTR: Reinforced Multi-Turn Reasoning for Long Video Understanding (Read more on arXiv or HuggingFace) |
Lionel Ni, Zheng Ge, Tianshui Chen, Yuan Xie |
Video-MTR is a reinforced multi-turn reasoning framework that enables MLLMs to iteratively select and comprehend key segments in long videos for improved understanding. The objective is to develop an end-to-end trainable model that overcomes the limitations of static, single-turn reasoning in long-form video understanding by mimicking human-like iterative evidence gathering. The framework employs reinforcement learning (PPO) with a novel gated bi-level reward system, where an MLLM agent is trained to sequentially retrieve relevant video frames, guided by both turn-level frame-relevance rewards (IoU) and a final trajectory-level answer-correctness reward. Video-MTR demonstrates significant performance gains on longer videos, improving accuracy by +6.3% (from 44.7% to 51.0%) over its base model on the Long subset of the VideoMME benchmark. The principal implication for AI practitioners is that for complex, long-duration sequential data tasks, an RL-based multi-turn paradigm with fine-grained intermediate rewards can enhance performance and efficiency over standard supervised fine-tuning, achieving superior results with substantially less training data (8K samples). |
| Durian: Dual Reference-guided Portrait Animation with Attribute Transfer (Read more on arXiv or HuggingFace) |
Hanbyul Joo, Byungjun Kim, Hyunsoo Cha |
Durian is a zero-shot diffusion-based framework for generating portrait animation videos by transferring facial attributes from a reference image to a target portrait. The main objective is to create a generalizable method for animating a static portrait image while simultaneously transferring diverse, deformable facial attributes from a single, cross-identity reference image, without requiring triplet training data. The key methodology involves a diffusion model with a Dual ReferenceNet architecture that injects spatial features from two masked inputs: an attribute-only reference and an attribute-masked portrait, trained using a self-reconstruction strategy on videos with an attribute-aware mask expansion technique. The framework achieves state-of-the-art performance, attaining an FID score of 38.00, which surpasses the best-performing two-stage baseline combination that scored 57.86. For AI practitioners, this single-stage pipeline enables the development of dynamic virtual try-on and content creation tools that support multi-attribute composition and interpolation from static images in a single forward pass without additional training. |
| Drawing2CAD: Sequence-to-Sequence Learning for CAD Generation from |
|
|
| Vector Drawings (Read more on arXiv or HuggingFace) |
Meie Fang, Changmiao Wang, Shichao Lu, Feiwei Qin, 1nnoh |
Drawing2CAD presents a sequence-to-sequence Transformer framework for generating parametric 3D CAD models directly from 2D vector engineering drawings. The primary objective is to automate the creation of precise, editable CAD models from vector graphics (SVG), aligning with industrial workflows and avoiding the imprecision of raster-based inputs. The methodology features a dual-decoder architecture that decouples CAD command type and parameter generation, a concatenation-based embedding for input primitives, and a soft target distribution loss function to allow for parameter flexibility. On the authors’ newly created CAD-VGDrawing dataset, the model achieved a command accuracy of 82.43% and reduced the invalid model generation ratio to 20.31% using a four-view input. For AI practitioners, this work provides a direct blueprint for developing systems that translate geometrically precise 2D vector drawings into structured, parametric 3D models, enabling automation in engineering design pipelines. |
| Delta Activations: A Representation for Finetuned Large Language Models (Read more on arXiv or HuggingFace) |
Ser-Nam Lim, Mayur Naik, Amish Sethi, OscarXZQ |
The paper introduces Delta Activations, a method for representing finetuned LLMs as vector embeddings by measuring shifts in their internal activations relative to a base model. The primary objective is to create a compact, semantically meaningful representation to enable efficient discovery, comparison, and clustering of finetuned models without relying on metadata. The methodology involves computing the difference between a finetuned model’s and its base model’s last-layer hidden state activations on a fixed set of generic prompts, then averaging these differences to form the final embedding. This method achieves superior domain-based clustering, yielding an average silhouette score of 0.614 across three backbones, significantly outperforming baselines like flattened weights (-0.043) and output embeddings (0.087). For AI practitioners, Delta Activations provide a computationally efficient tool to navigate and organize large model hubs, enabling better model selection, reuse, and merging without access to original training data. |
| False Sense of Security: Why Probing-based Malicious Input Detection |
|
|
| Fails to Generalize (Read more on arXiv or HuggingFace) |
Muhao Chen, Qin Liu, Zeming Wei, Cheng Wang |
This research demonstrates that probing-based classifiers for LLM safety fail to generalize by learning superficial linguistic patterns instead of genuine semantic harmfulness. The study’s objective is to determine why these classifiers achieve near-perfect in-domain accuracy but collapse on out-of-distribution (OOD) data. The methodology involves controlled experiments evaluating classifiers on “semantically cleaned” datasets, where harmful concepts are replaced with benign ones while preserving the original instructional and syntactic patterns. The primary result is a dramatic performance collapse on this controlled data, with classifier accuracy dropping by 60-90 percentage points, proving their reliance on surface-level cues like instructional patterns and trigger words over semantic content. The principal implication for AI practitioners is that current probing-based safety mechanisms provide a false sense of security; their high in-domain accuracy is not indicative of robust harmfulness detection, and they should not be trusted in production systems without rigorous semantic and OOD evaluation. |
Papers for 2025-09-04
| Title |
Authors |
Summary |
| Robix: A Unified Model for Robot Interaction, Reasoning and Planning (Read more on arXiv or HuggingFace) |
Zixuan Wang, Wei Li, Heng Dong, Mengxi Zhang, Huang Fang |
Robix is a unified vision-language model designed as a high-level cognitive layer for hierarchical robot systems, integrating robot reasoning, task planning, and natural language interaction. Its primary objective is to overcome limitations in embodied reasoning and multimodal interaction, enabling generalist robots to handle complex, long-horizon tasks in dynamic environments. The methodology involves a three-stage training strategy: continued pretraining for foundational embodied reasoning, supervised finetuning to model human-robot interaction as a unified reasoning-action sequence, and reinforcement learning for improved consistency and coherence. In online evaluations of the VLM-VLA robot system, Robix-32B-RL achieved an average task progress of 92.5%, outperforming Gemini-2.5-Pro by 4.3 percentage points and GPT-40 by 28.1 percentage points. This demonstrates a robust pathway for AI practitioners developing general-purpose embodied intelligence that requires adaptable and human-like interaction in real-world settings. |
| Open Data Synthesis For Deep Research (Read more on arXiv or HuggingFace) |
Zheng Liu, Hongjin Qian, Kun Luo, ZiyiXia |
This paper introduces InfoSeek, an open-source framework for synthesizing large-scale, structurally complex Deep Research tasks formalized as Hierarchical Constraint Satisfaction Problems (HCSPs). The main objective is to provide a scalable method for generating verifiable Deep Research questions, addressing the limitations of existing benchmarks which lack structural depth and complexity for multi-step, multi-source reasoning. InfoSeek utilizes a dual-agent system to recursively build Research Trees from webpages, blurring nodes into sub-problems and converting them into natural language questions, yielding over 50K training examples and 16.5K reasoning trajectories via reject sampling. Experiments demonstrate that a 3B LLM trained on InfoSeek achieves 16.5% accuracy on the BrowseComp-Plus benchmark, outperforming larger 32B models (e.g., Qwen3-32B at 3.5%) and lightweight commercial APIs (e.g., Gemini 2.5 Flash at 15.5%), and is comparable to Gemini 2.5 Pro (19.0%). For AI practitioners, the InfoSeek dataset’s preservation of meta-information, such as intermediate steps and retrieval labels, facilitates the development of advanced optimization strategies, including compound reward design and trajectory-level exploration for training Deep Research agents. |
| LMEnt: A Suite for Analyzing Knowledge in Language Models from |
|
|
| Pretraining Data to Representations (Read more on arXiv or HuggingFace) |
Yoav Gur-Arieh, Ido Cohen, Alon Gilae-Dotan, Daniela Gottesman, mega |
LMEnt is an open-source suite designed for analyzing knowledge acquisition and representation in Language Models from pretraining data to their internal representations. The primary objective is to facilitate the study of how knowledge representations are formed and shaped during LM pretraining, specifically addressing the interplay between data composition, training dynamics, and knowledge mechanisms. Key methodologies include annotating a 7.3M-entity Wikipedia corpus with fine-grained entity mentions (hyperlinks, entity linking, coreference), building an Elasticsearch index for entity-based retrieval by Wikidata QID, and releasing 12 pretrained OLMO-2 models (170M-1B parameters) with 4K intermediate checkpoints. A primary result shows LMEnt’s entity-based retrieval outperforms string-based methods by as much as 80.4% in retrieving relevant document chunks, maintaining over 97% precision. For AI practitioners, LMEnt provides a controlled and transparent environment to investigate knowledge representations, plasticity, editing, attribution, and learning dynamics in LMs, enhancing the ability to control and improve model factuality and reasoning. |
| Mixture of Global and Local Experts with Diffusion Transformer for |
|
|
| Controllable Face Generation (Read more on arXiv or HuggingFace) |
Kai Li, Yue Li, Xing Fu, Shun Zhang, Xuechao Zou |
Face-MoGLE is a unified Diffusion Transformer framework for high-quality, controllable, and photorealistic face generation. Its primary objective is to address challenges in balancing semantic controllability and photorealism by decoupling semantic controls from generation pipelines. The framework utilizes a Diffusion Transformer backbone with a Mixture of Global and Local Experts (MoGLE) architecture, integrating semantic-decoupled latent modeling and a dynamic gating network that adaptively blends expert outputs based on spatial and temporal awareness. Face-MoGLE achieved a Fréchet Inception Distance (FID) of 22.24 for multimodal face generation on MM-CelebA-HQ, surpassing state-of-the-art models, and demonstrated robust zero-shot generalization. Additionally, generated images from Face-MoGLE achieved an Area Under the Curve (AUC) of 0.50 against the NPR deepfake detector, indicating high perceptual realism. This enables AI practitioners to develop advanced generative modeling applications requiring precise facial attribute manipulation, high visual fidelity, and enhanced resilience to deepfake detection. |
| MOSAIC: Multi-Subject Personalized Generation via Correspondence-Aware |
|
|
| Alignment and Disentanglement (Read more on arXiv or HuggingFace) |
Hualiang Wang, Qiaoqiao Jin, Mushui Liu, Siming Fu, Dong She |
MOSAIC is a representation-centric framework for multi-subject personalized image generation that uses explicit semantic correspondence and orthogonal feature disentanglement. The primary objective is to overcome identity blending and attribute leakage in multi-subject generation by improving subject interaction modeling and disentangling conflated features. Its methodology introduces SemAlign-MS, a dataset with fine-grained semantic point correspondences, and utilizes Semantic Correspondence Attention Loss (SCAL) for precise point-to-point alignment and Multi-Reference Disentanglement Loss (MDL) for orthogonal feature separation. Quantitatively, MOSAIC achieves state-of-the-art performance, for instance, securing 76.30 CLIP-I, 32.40 CLIP-T, and 56.83 DINO on DreamBench multi-subject scenarios, and maintaining high fidelity with 4+ reference subjects where other methods degrade. This directly implies that AI practitioners can now leverage MOSAIC for highly consistent and scalable multi-subject synthesis applications, pushing the boundaries of personalized image generation beyond previous limitations. |
Papers for 2025-09-03
| Title |
Authors |
Summary |
| The Landscape of Agentic Reinforcement Learning for LLMs: A Survey (Read more on arXiv or HuggingFace) |
Hejia Geng, Guibin Zhang, henggg, Artemis0430, JeremyYin |
This survey formalizes Agentic Reinforcement Learning (Agentic RL) as a paradigm that reframes LLMs from static sequence generators into autonomous, decision-making agents optimized for sequential tasks in dynamic environments. The paper’s main objective is to define and structure this emerging field by synthesizing over 500 recent works, contrasting the multi-step, partially observable Markov decision process (POMDP) of Agentic RL with the degenerate single-step MDP of traditional LLM-RL. Its methodology involves a systematic literature review to construct a twofold taxonomy based on core agentic capabilities (e.g., planning, tool use, memory) and application domains, while cataloging relevant RL algorithm families like PPO, DPO, and GRPO. The survey consolidates results demonstrating Agentic RL’s effectiveness, citing findings such as DeepCoder-14B achieving a +8% Pass@1 gain on LiveCodeBench by using outcome-based rewards. The principal implication for AI practitioners is to approach LLM training not just as single-turn preference alignment but as the development of a learnable policy for long-horizon, interactive tasks, enabling more robust and autonomous agentic behavior through direct optimization in complex environments. |
| UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Haoyang Zou, zhwang4ai, JoeYing, jzfeng, MingComplex |
This paper presents UI-TARS-2, a native GUI-centered agent advanced through a systematic methodology combining a data flywheel and multi-turn reinforcement learning. The research aims to solve open problems in GUI agent development, including data scarcity, scalable multi-turn RL, the limitations of GUI-only operation, and environment instability. Its key methodology integrates four pillars: a data flywheel for scalable data generation, a stabilized multi-turn RL framework using enhanced proximal policy optimization (PPO), a hybrid GUI environment integrating file systems and terminals, and a unified sandbox platform for large-scale rollouts. Empirical evaluation shows UI-TARS-2 achieves state-of-the-art performance, reaching 88.2 on the Online-Mind2Web benchmark and demonstrating strong out-of-domain generalization. The principal implication for AI practitioners is that this systematic approach, particularly the data flywheel and stabilized RL infrastructure, provides an effective methodology for training robust, generalizable GUI agents capable of handling diverse and complex real-world interactive scenarios. |
| SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn |
|
|
| Tool-Integrated Reasoning (Read more on arXiv or HuggingFace) |
Qian Liu, Longtao Zheng, Zhenghai Xue, xszheng2020, R1ch0rd |
SimpleTIR is a plug-and-play algorithm that stabilizes multi-turn Tool-Integrated Reasoning (TIR) training via Reinforcement Learning (RL) by filtering problematic trajectories. The primary objective is to address the training instability and performance collapse in multi-turn TIR, which is caused by distributional drift from external tool feedback leading to low-probability tokens and gradient norm explosions, without requiring a supervised fine-tuning (SFT) “cold-start”. The core methodology involves identifying and filtering out entire trajectories that contain “void turns”—defined as LLM responses that yield neither a complete code block nor a final answer—thus preventing high-magnitude gradients from backpropagating during the policy update. The primary result shows that SimpleTIR elevates the AIME24 score from a text-only baseline of 22.1 to 50.5 when starting from the Qwen2.5-7B base model. For AI practitioners, the principal implication is that implementing a simple, heuristic-based trajectory filtering rule to remove incomplete or non-progressive conversational turns can effectively stabilize end-to-end RL training for tool-using agents, enabling the development of more robust and diverse reasoning capabilities directly from base models. |
| ELV-Halluc: Benchmarking Semantic Aggregation Hallucinations in Long |
|
|
| Video Understanding (Read more on arXiv or HuggingFace) |
Xuanyu Zheng, Ruohui Wang, Mercury7353, datamonkey, HLSv |
This paper introduces ELV-Halluc, a benchmark for evaluating Semantic Aggregation Hallucination (SAH) in long-video understanding, where models misattribute correctly perceived frame-level semantics across different temporal events. The primary objective is to systematically measure and mitigate SAH, which becomes more critical as video length and semantic complexity increase. The methodology involves creating a benchmark with an adversarial triplet question-pair design (ground-truth, in-video hallucination, out-of-video hallucination) to isolate and quantify SAH, and then using Direct Preference Optimization (DPO) on a curated 8K-pair dataset to reduce it. The primary result is that this DPO-based approach successfully mitigated SAH, achieving a 27.7% reduction in the SAH ratio on a Qwen2.5-VL-7B model while also improving general video understanding on the VideoMME benchmark. The principal implication for AI practitioners is that to improve the reliability of long-video models, it is crucial to employ targeted mitigation strategies like DPO with intra-video adversarial examples, as simply increasing frame sampling can sometimes worsen this specific type of hallucination. |
| LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model (Read more on arXiv or HuggingFace) |
Jianwei Yang, Chunyuan Li, Benjamin-eecs, drogozhang, russwang |
This paper demonstrates that training a vision-language model on critic data via reinforcement learning simultaneously produces a strong policy model, unifying evaluation and generation capabilities. The research investigates whether a model trained for critic tasks, such as judging response preferences, can also excel as a generative policy model across diverse benchmarks. The key methodology involves reformulating pairwise preference critic datasets into a verifiable RL task and fine-tuning a base generative model (Qwen-2.5-VL-7B) using Group Relative Policy Optimization (GRPO), thereby creating LLaVA-Critic-R1. The primary result is that LLaVA-Critic-R1 not only excels as a critic but also improves as a policy model, achieving an average performance gain of +5.7% over its base model across 26 visual reasoning benchmarks. For AI practitioners, the principal implication is that a single, unified model can be trained to perform both generation and self-evaluation, offering a simplified and scalable approach to building self-improving multimodal systems that benefit from effective test-time scaling via self-critique without an external evaluator. |
| POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models |
|
|
| for Document Conversion (Read more on arXiv or HuggingFace) |
Haicheng Wang, Le Tian, Zhongyin Zhao, YxxxB, YuanLiuuuuuu |
This paper introduces a two-stage, distillation-free framework using synthetic data and iterative self-improvement to train a vision-language model for document conversion. The objective is to create a fully automated pipeline for generating high-quality training data to accurately extract plain text, tables, and mathematical formulas without relying on teacher models. The methodology involves an initial warm-up stage training on large-scale synthetic data, followed by an iterative self-improvement stage where the model annotates real-world documents, which are then filtered using rule-based strategies (e.g., F1-score for text, structural validation for tables) and used for retraining. The resulting 3B parameter POINTS-Reader model achieves a score of 0.259 on the OmniDocBench benchmark (lower is better), and significantly outperforms the expert GOT-OCR model on the table metric by 0.197 (0.335 vs 0.532). For AI practitioners, this work demonstrates a viable, resource-efficient methodology to bootstrap high-quality, specialized document understanding models by leveraging unlabeled real-world data and rule-based filtering, reducing dependence on large-scale proprietary models. |
| VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use (Read more on arXiv or HuggingFace) |
Zhiheng Lyu, Zhuofeng Li, Yi Lu, JasperHaozhe, DongfuJiang |
VERLTOOL is a unified, modular framework designed to efficiently train tool-using LLM agents via Agentic Reinforcement Learning with Tool use (ARLT) across diverse multi-modal domains. The primary objective is to overcome the fragmentation, synchronous execution bottlenecks, and limited extensibility of existing ARLT systems by providing a unified and efficient training infrastructure. The methodology is centered on a decoupled architecture featuring a dedicated tool server with standardized APIs, a modular plugin design for easy tool integration, and asynchronous rollout execution to eliminate idle waiting time during training. VERLTOOL demonstrates competitive performance across six ARLT tasks and its asynchronous architecture achieves a near 2x speedup (e.g., 1.97x on the DeepSearch task) in rollout execution compared to synchronous methods. For AI practitioners, VERLTOOL offers a scalable, open-source infrastructure that reduces development overhead and accelerates the training of multi-modal, tool-augmented agents through its efficient, extensible, and modular design. |
| Baichuan-M2: Scaling Medical Capability with Large Verifier System (Read more on arXiv or HuggingFace) |
Jayok6, yuanshuai, sdujq, anselcmy, fairyang |
This paper presents Baichuan-M2, a 32B-parameter medical LLM trained via a dynamic reinforcement learning framework that simulates real-world clinical interactions. The primary objective is to address the performance gap between medical LLMs on static benchmarks and their utility in dynamic clinical decision-making by creating a high-fidelity, interactive training environment. The key methodology involves a novel verifier system comprising a Patient Simulator built from de-identified medical records and a Clinical Rubrics Generator for dynamic, multi-dimensional evaluation, using an improved Group Relative Policy Optimization (GRPO) algorithm for training. The primary result is that Baichuan-M2 outperforms all other open-source models on HealthBench, achieving a score of 34.7 on the HealthBench Hard benchmark, a threshold previously surpassed only by GPT-5. The principal implication for AI practitioners is that developing domain-specific, interactive simulation environments for reinforcement learning is crucial for aligning LLM capabilities with complex, real-world applications, offering a more effective path to high performance than reliance on static datasets alone. |
| Kwai Keye-VL 1.5 Technical Report (Read more on arXiv or HuggingFace) |
SXxtyz, Chengru, bhsc24, dingboyang, biaoYang |
This report presents Keye-VL-1.5, an 8-billion parameter multimodal model optimized for video understanding. The primary objective was to overcome the inherent trade-off between spatial resolution and temporal coverage in video processing within Multimodal Large Language Models (MLLMs). Key methodology includes a novel Slow-Fast video encoding strategy that dynamically allocates computational resources based on inter-frame similarity, a progressive four-stage pre-training to extend context length to 128K tokens, and a comprehensive post-training pipeline using GSPO-based reinforcement learning. The model achieves state-of-the-art performance on video-centric benchmarks, notably scoring 66.0% on Video-MMMU, outperforming comparable models by over 6.5 absolute percentage points. For AI practitioners, the Slow-Fast encoding strategy provides a computationally efficient method for processing long-form video, enabling MLLMs to better handle dynamic, information-dense visual content. |
| Implicit Actor Critic Coupling via a Supervised Learning Framework for |
|
|
| RLVR (Read more on arXiv or HuggingFace) |
Lu Wang, Yukun Chen, Ze Gong, Longze Chen, Geaming |
The paper proposes PACS, a framework that reformulates Reinforcement Learning with Verifiable Rewards (RLVR) as a supervised learning task to improve LLM reasoning capabilities. The objective is to address the sparse reward signals and unstable policy updates common in existing RL-based RLVR methods. PACS achieves this by treating the verifiable outcome reward as a target label and training a score function, parameterized by the policy model, to predict this reward using a cross-entropy loss, which a gradient analysis shows implicitly couples actor and critic roles. On the AIME 2024 benchmark with a Qwen2.5-7B model, PACS achieved a 59.78% pass@256 rate, outperforming PPO and GRPO by 13.32 and 14.36 percentage points respectively. For AI practitioners, this research offers a simpler, more stable, and higher-performing alternative to complex RL algorithms for post-training LLMs on tasks with verifiable outcomes. |
| DCPO: Dynamic Clipping Policy Optimization (Read more on arXiv or HuggingFace) |
Kai Lu, Chengfeng Dou, sdujq, GuoPD, yangshui |
DCPO (Dynamic Clipping Policy Optimization) is a reinforcement learning algorithm that enhances the reasoning capabilities of large language models by improving data utilization and exploration efficiency in RLVR. The primary objective is to overcome the zero-gradient and sample inefficiency issues inherent in methods like GRPO, which are caused by fixed clipping bounds and per-step reward standardization. DCPO’s methodology integrates a dynamic clipping strategy that adaptively adjusts bounds based on token-specific probabilities and a smooth advantage standardization technique that aggregates rewards across cumulative training steps for more stable updates. The proposed method demonstrated superior performance, achieving an Avg@32 score of 38.8 on the AIME24 benchmark with a 7B model, significantly outperforming GRPO (32.1) and DAPO (31.6), and increasing the average response utilization ratio by 28% over GRPO. For AI practitioners, DCPO offers a more data-efficient and robust framework for RL fine-tuning, enabling the development of stronger reasoning models by mitigating training instability and making better use of generated samples. |
| Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task |
|
|
| Arithmetic (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Mohammad Zbeeb, hammh0a |
This work demonstrates that reasoning capabilities can be extracted as a compact task vector from a reinforcement learning-tuned model and transferred to compatible models via simple tensor arithmetic. The research investigates if reasoning abilities learned via reinforcement learning can be isolated from shared knowledge and reused as a transferable task vector. The methodology defines a “reasoning vector” as the parameter-space difference between two identically initialized models, one trained with Group Relative Policy Optimization (GRPO) and the other with Supervised Fine-Tuning (SFT) on the same dataset (v_reason = θ_GRPO - θ_SFT). Primary results show that adding this vector to a 1.5B QWEN2.5 model improved performance on GSM8K by +4.9% and BigBenchHard by +12.3%, while subtracting the vector degraded GSM8K performance by -11.8%. For AI practitioners, this provides a computationally inexpensive, training-free method to enhance the reasoning of compatible models by arithmetically applying pre-computed vectors from existing open-source checkpoints. |
| GenCompositor: Generative Video Compositing with Diffusion Transformer (Read more on arXiv or HuggingFace) |
Lingen Li, Guangzhi Wang, Xiaodong Cun, Xiaoyu521, Ysz2022 |
GenCompositor introduces a generative video compositing framework using a Diffusion Transformer (DiT) to automate the integration of dynamic foreground videos into background videos with user-specified trajectories and scales, while preserving background consistency. The main objective is to create a model that can adaptively inject foreground identity and motion information into a target video in an interactive manner. The core methodology is a DiT pipeline featuring a background preservation branch for consistency, a full self-attention DiT fusion block for integrating dynamic elements, and a novel Extended Rotary Position Embedding (EROPE) to handle pixel-unaligned video inputs. The model demonstrates superior performance over existing solutions, achieving a PSNR of 42.0010 on video harmonization tasks, and introduces VideoComp, a new 61K-video dataset curated for this task. The principal implication for AI practitioners is the EROPE technique, which provides an effective, parameter-free method for generative models to handle layout-unaligned video conditions, enabling more flexible and powerful automated video editing tools. |
| Jointly Reinforcing Diversity and Quality in Language Model Generations (Read more on arXiv or HuggingFace) |
Tianlu, jcklcn, spermwhale, danyaljj, dogtooth |
The paper introduces Diversity-Aware Reinforcement Learning (DARLING), an online RL framework that jointly optimizes for response quality and semantic diversity in large language models. The main objective is to counteract the loss of output diversity that occurs during standard LM post-training by developing a method to simultaneously reinforce high-quality and semantically distinct generations. DARLING’s key methodology involves using a learned semantic classifier to generate a diversity signal from model rollouts, which is then multiplicatively combined with a quality reward within a Group Relative Policy Optimization (GRPO) framework to amplify updates for novel, high-quality responses. Primary results show that on verifiable competition math tasks, DARLING improved both solution quality (pass@1) and variety (pass@k), outperforming a quality-only GRPO baseline by an average of +3.51% on pass@1 and +7.62% on pass@128 for Qwen3-4B-Base models. The principal implication for AI practitioners is that this method can be integrated into post-training pipelines to mitigate diversity collapse, enhancing model performance in creative, exploratory, and multi-path problem-solving tasks without sacrificing response quality. |
| OpenVision 2: A Family of Generative Pretrained Visual Encoders for |
|
|
| Multimodal Learning (Read more on arXiv or HuggingFace) |
Zirui Wang, Letian Zhang, Xianhang Li, Yanqing Liu, cihangxie |
OpenVision 2 introduces a family of visual encoders pretrained using a purely generative captioning objective, simplifying its predecessor by removing the text encoder and contrastive loss. The main objective is to evaluate if this generative-only paradigm can match the multimodal performance of combined contrastive-generative models while significantly enhancing training efficiency. Its methodology consists of an image encoder feeding visual tokens, with approximately two-thirds randomly masked, directly to a text decoder trained autoregressively to predict high-quality synthetic captions. The primary result is competitive performance with substantial efficiency gains; for instance, the ViT-L/14 model reduces training time by ~1.5x (from 83h to 57h) and memory usage by ~1.8x (from 24.5GB to 13.8GB) compared to the original OpenVision. The principal implication for AI practitioners is that a caption-only generative objective is a computationally efficient and effective alternative to CLIP-style contrastive learning for building scalable, general-purpose vision encoders, lowering the resource barrier for training large multimodal models. |
| M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via |
|
|
| Self-Supervision (Read more on arXiv or HuggingFace) |
Yan-Jie Zhou, Heng Guo, Chengyu Fang, Zheng Jiang, Che Liu |
M³Ret is a unified visual encoder trained via self-supervision on a large-scale, hybrid-modality medical dataset to achieve zero-shot image retrieval without modality-specific designs. The primary objective is to determine if a single framework can learn transferable visual representations across heterogeneous 2D (X-ray, ultrasound), 3D (CT), and video (endoscopy) data using only visual signals. The methodology involves pretraining a Vision Transformer on a curated dataset of 867,653 medical images using Masked Autoencoder (MAE) and SimDINO self-supervised learning paradigms with a unified 4D patchification input strategy. The model sets a new state-of-the-art, with the SimDINO variant achieving a Recall@5 of 0.674 on ChestXray14, and demonstrates strong cross-modal generalization by performing retrieval on unseen MRI tasks despite no MRI exposure during pretraining. The principal implication for AI practitioners is that a single, modality-agnostic visual encoder can be successfully pretrained without paired text data or specialized architectures, offering a scalable and effective pathway for building foundational models for medical image understanding. |
| Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm |
|
|
| Simulators for Conditional Synthetic Data Generation (Read more on arXiv or HuggingFace) |
Xiaolei Huang, Weisi Liu, kwangju |
This paper introduces Genetic Prompt, a framework using LLMs to simulate genetic algorithms on semantic text attributes for high-quality synthetic data generation. The primary objective is to automatically amplify the diversity and generator adaptability of synthetic data to improve the training of robust downstream models. The key methodology treats textual attributes like readability and style as “genes,” employs an active learning strategy to select semantically distant parent samples, and prompts an LLM to perform crossover and mutation on these genes to create new data. The primary result shows consistent outperformance over baselines; for example, on the Conll04 relation extraction task using a GPT-4o generator, Genetic Prompt achieved a Micro-F1 score of 85.3, significantly outperforming the next-best baseline’s 73.3. For AI practitioners, this framework provides a robust method to augment datasets, especially in class-imbalanced scenarios, to significantly boost downstream model performance by generating diverse, high-quality training examples. |
| Benchmarking Optimizers for Large Language Model Pretraining (Read more on arXiv or HuggingFace) |
mjaggi, MatPag, Andron00e |
This paper presents a comprehensive benchmark of 11 optimization methods for Large Language Model pretraining across various model sizes, batch sizes, and training durations. The main objective is to systematically evaluate recent optimization techniques against the dominant AdamW baseline in standardized LLM pretraining scenarios to identify the most effective methods and provide guidance to practitioners. The methodology involves pretraining Llama-like dense and Mixture-of-Experts models (from 124M to 720M parameters) on the FineWeb dataset, with extensive and controlled hyperparameter tuning for each optimizer across different training compute budgets. Primary results show that AdEMAMix and MARS consistently outperform AdamW and other optimizers, particularly at larger scales; for a 720M parameter model trained on 48B tokens, AdEMAMix achieves the lowest final validation loss of approximately 2.8. The principal implication for AI practitioners is that AdamW is no longer the default optimal choice for LLM pretraining; alternatives like AdEMAMix can provide superior performance, and this paper offers an evidence-based framework and tuned configurations to select a better optimizer for a given training scenario. |
| The Gold Medals in an Empty Room: Diagnosing Metalinguistic Reasoning in |
|
|
| LLMs with Camlang (Read more on arXiv or HuggingFace) |
Solomon Tsai, Zhujun Jin, Yixuan Liu, Fenghua Liu, yulongchen |
This paper introduces Camlang, a novel constructed language, to demonstrate that state-of-the-art LLMs fail at metalinguistic deductive reasoning, in contrast to humans who can systematically apply explicit grammar rules to learn a new language. The main objective is to determine whether LLMs can learn and apply explicit grammatical rules for an unfamiliar language to perform reasoning, or if their success relies on pattern matching from training data. The key methodology involves the creation of Camlang, a typologically plausible but novel language, accompanied by a grammar book and dictionary, and the development of the Camlang-CSQA-v0 benchmark by translating CommonsenseQA questions. Primary results show that while GPT-5 achieves 98% accuracy in English, its performance drops to 47% in Camlang, far below the human baseline of 87%; human verification further reveals that models achieve near-zero (0-2.13%) Strict Human-Verified accuracy, indicating correct answers stem from shallow heuristics, not grammatical mastery. The principal implication for AI practitioners is that current LLMs cannot be relied upon to systematically interpret and apply novel, explicit rule sets (e.g., API documentation, legal text, game rules), as they fundamentally struggle with the deductive metalinguistic competence required for such tasks. |
| Fantastic Pretraining Optimizers and Where to Find Them (Read more on arXiv or HuggingFace) |
Percy Liang, Tengyu Ma, David Hall, Kaiyue Wen |
This paper systematically benchmarks ten pretraining optimizers, revealing that their speedups over a well-tuned AdamW baseline are significantly smaller than reported and diminish with increasing model scale. The main objective is to conduct a fair comparison of modern optimizers for large language model pretraining by addressing methodological flaws in prior work, specifically unequal hyperparameter tuning and limited evaluation setups. The methodology consists of a rigorous three-phase hyperparameter tuning framework, performing coordinate descent sweeps across ten optimizers on four model scales (0.1B-1.2B parameters) and four data-to-model ratios (1–8× the Chinchilla optimum). The primary result is that the speedup of matrix-based optimizers is inversely proportional to model scale, decreasing from a 1.4× speedup over AdamW for 0.1B parameter models to merely 1.1× for 1.2B parameter models. The principal implication for AI practitioners is that reported speedups of new optimizers should be treated with skepticism; rigorous, independent hyperparameter tuning for both the baseline and new optimizers is crucial, as undertuned baselines account for most of the claimed performance gains. |
| Universal Deep Research: Bring Your Own Model and Strategy (Read more on arXiv or HuggingFace) |
Pavlo Molchanov, Peter Belcak |
The paper introduces Universal Deep Research (UDR), a generalist agentic framework that translates user-defined natural language research strategies into executable code to control any underlying language model for structured information retrieval. The main objective is to overcome the rigidity of existing deep research agents by allowing users to create and refine custom research strategies without model fine-tuning. UDR’s methodology uses an LLM to generate a complete Python script from the user’s strategy in a single pass; this script is then run in a sandboxed environment to orchestrate tool use and specific LLM reasoning calls. The primary result is a system that decouples agentic orchestration from core reasoning, successfully executing complex workflows within a constant context length of just 8k tokens. For AI practitioners, the principal implication is an architectural pattern for building more efficient, deterministic, and auditable agents by offloading control logic to CPU-executable code, which minimizes expensive LLM orchestration calls and reduces GPU usage. |
| FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in |
|
|
| Diverse Adventure Games (Read more on arXiv or HuggingFace) |
Dongmin Park, Jaehyeon Son, Heeseung Yun, Junseo Kim, ahnpersie |
This research introduces FlashAdventure, a benchmark of 34 adventure games for evaluating GUI agents on full story arcs, and proposes the COAST framework to address the long-term observation-behavior gap. The main objective is to assess the capability of LLM-powered GUI agents to complete entire narrative-driven story arcs and to address the challenge of managing long-term dependencies between information observation and subsequent action. The key methodology involves the FlashAdventure benchmark, an automated evaluator named CUA-as-a-Judge, and a novel agentic framework, COAST (Clue-Oriented Agent for Sequential Tasks), which utilizes a Seek-Map-Solve cycle with a long-term clue memory to generate and execute subtasks. Primary results show that current GUI agents demonstrate near-zero success rates on full story arcs; the proposed COAST framework improved the milestone completion rate by 2.78 percentage points over the Claude-3.7-Sonnet baseline, achieving 19.89%, yet this remains significantly lower than the human performance benchmark (97.06% success rate). The principal implication for AI practitioners is that developing robust GUI agents for complex sequential tasks requires explicit architectural designs for long-term memory management and planning, as current models struggle with the “observation-behavior gap”; practitioners should consider structured approaches like COAST’s clue-oriented cycle rather than relying solely on large context windows. |
| Discrete Noise Inversion for Next-scale Autoregressive Text-based Image |
|
|
| Editing (Read more on arXiv or HuggingFace) |
Amin Heyrani Nobar, Ngan Hoai Nguyen, Ligong Han, Xiaoxiao He, quandao10 |
This paper introduces VARIN, the first training-free, noise inversion-based editing technique specifically designed for discrete next-scale visual autoregressive (VAR) models. The main objective is to enable prompt-guided image editing in VARs by developing a method to invert their non-differentiable argmax sampling process, thereby allowing for precise image reconstruction and controlled modification. The key methodology is a novel pseudo-inverse function called Location-aware Argmax Inversion (LAI), which estimates inverse Gumbel noises from a source image’s token maps; these recovered noises are then used to guide the generative process toward a target prompt. In experiments on the PIE-Bench dataset, VARIN achieved a Whole CLIP Similarity of 25.05, outperforming the discrete model DICE (23.79) in edit alignment while being approximately twice as fast. For AI practitioners, this provides a method to perform efficient, training-free text-based editing on next-scale autoregressive architectures like HART, offering a computationally faster alternative to many diffusion-based editing workflows. |
| MobiAgent: A Systematic Framework for Customizable Mobile Agents (Read more on arXiv or HuggingFace) |
Wangbo Gong, Yisheng Zhao, Xi Zhao, fengerhu, sjtuzc |
This paper presents MobiAgent, a systematic framework for developing, accelerating, and evaluating GUI-based mobile agents. The main objective is to address the significant accuracy and efficiency challenges that current Vision-Language Model (VLM) agents face in real-world mobile task execution. The key methodology involves a three-part system: the MobiMind multi-role agent models, the AgentRR record-and-replay acceleration framework which uses a latent memory model to cache and reuse task trajectories, and the MobiFlow DAG-based benchmark for evaluation. The primary result is that MobiAgent outperforms models like GPT-5 and UI-TARS-1.5-7B in task completion, while its AgentRR framework achieves a 2-3x performance improvement by attaining a 60%-85% action replay rate under realistic user task distributions. The principal implication for AI engineers is that deploying a record-and-replay acceleration layer provides a highly effective, full-stack solution to mitigate the high inference latency of VLMs, making mobile agents more practical and efficient for recurring real-world tasks. |
| MedDINOv3: How to adapt vision foundation models for medical image |
|
|
| segmentation? (Read more on arXiv or HuggingFace) |
Xiaofeng Yang, Yuheng Li, wy20030128, yuxianglai117, mcl0222 |
The paper presents MedDINOv3, a framework for adapting vision foundation models (FMs) to medical image segmentation by combining architectural refinements with domain-adaptive pretraining. The research aims to determine how to effectively transfer large-scale, natural-image FMs to medical segmentation tasks, overcoming challenges like the ViT-CNN performance gap and the substantial domain shift. The key methodology involves first refining a plain Vision Transformer (ViT) architecture with multi-scale token aggregation from intermediate layers and high-resolution training, then performing a three-stage, domain-adaptive pretraining on a curated 3.87 million slice CT dataset (CT-3M) using a DINOv3-style recipe. The primary result is that MedDINOv3 outperforms or matches strong baselines on four benchmarks, achieving an 87.38% Dice Similarity Coefficient (DSC) on the AMOS22 dataset, surpassing the nnU-Net baseline by 2.57%. The principal implication for AI practitioners is that general-purpose FMs can outperform highly specialized architectures in domains like medical imaging when combined with targeted architectural enhancements and large-scale, domain-specific self-supervised pretraining. |
| AMBEDKAR-A Multi-level Bias Elimination through a Decoding Approach with |
|
|
| Knowledge Augmentation for Robust Constitutional Alignment of Language Models (Read more on arXiv or HuggingFace) |
Rahul Karthikeyan, Shivam Dubey, Aryan Kasat, Snehasis Mukhopadhyay, amanchadha |
The AMBEDKAR framework introduces an inference-time, fairness-aware speculative decoding method to mitigate caste and religious biases in LLMs by aligning their outputs with principles from the Indian Constitution. The primary objective is to develop a computationally efficient, model-agnostic technique to reduce sociocultural biases specific to the Indian context, which existing mitigation strategies often overlook. The key methodology inverts speculative decoding, using a draft model to propose tokens and a constitutionally-aligned verifier model to select the token with the minimum Jensen-Shannon divergence between its generation probabilities under original and counterfactually perturbed prompts. This approach yields an absolute bias reduction of up to 26.41% compared to baseline models, with a per-token latency increase of only 6.29%. For AI practitioners, this provides a deployable, low-latency “fairness-by-speculation” mechanism to enforce normative constraints at inference time without retraining, making it applicable for steering generation towards fairness even in black-box models. |
| Improving Large Vision and Language Models by Learning from a Panel of |
|
|
| Peers (Read more on arXiv or HuggingFace) |
Simon Jenni, Jing Shi, Jefferson Hernandez, kushalkafle, vicenteor |
The paper introduces Panel-of-Peers (PoP), a framework for iteratively improving Large Vision-Language Models (LVLMs) through collaborative, self-generated feedback, eliminating the need for human-labeled preference data. The research objective is to develop a scalable self-improvement paradigm for LVLMs that bootstraps their capabilities using only unlabeled prompts. The core methodology involves a panel of peer LVLMs that both generate candidate responses and evaluate each other’s outputs along multiple axes (e.g., correctness, helpfulness) to create a synthetic preference dataset, which is then used to iteratively fine-tune all models in the panel. The PoP framework demonstrated significant performance gains, increasing the average score across 15 vision-language benchmarks by 9 absolute points (from 48% to 57%) over three self-improvement iterations. For AI practitioners, this provides a cost-effective method to enhance LVLM performance and enable cross-model knowledge transfer (e.g., teaching an OCR-deficient model to read) without requiring expensive human annotation. |
| ViSTA-SLAM: Visual SLAM with Symmetric Two-view Association (Read more on arXiv or HuggingFace) |
Daniel Cremers, Xi Wang, Shenhan Qian, zhangganlin |
ViSTA-SLAM is a real-time, monocular visual SLAM system designed to provide accurate dense 3D reconstruction and tracking without requiring camera intrinsics. The research objective is to create an efficient and accurate dense SLAM system that is broadly applicable across diverse camera setups by eliminating the need for pre-calibration. The methodology consists of a lightweight Symmetric Two-view Association (STA) frontend that estimates relative poses and local pointmaps from image pairs, which are then integrated into a backend Sim(3) pose graph with loop closure for global optimization. The system achieves state-of-the-art accuracy, with an average Absolute Trajectory Error (ATE) RMSE of 0.052 on the TUM-RGBD dataset, while its frontend model is only 35% the size of comparable methods. This presents AI practitioners with an efficient and broadly applicable framework for dense 3D perception in robotics and AR, demonstrating that a symmetric, lightweight design can outperform larger models and remove the dependency on pre-calibrated sensors. |
| Towards More Diverse and Challenging Pre-training for Point Cloud |
|
|
| Learning: Self-Supervised Cross Reconstruction with Decoupled Views (Read more on arXiv or HuggingFace) |
Junchi Yan, Shaofeng Zhang, Xiangdong Zhang |
Point-PQAE is a self-supervised generative framework that pre-trains point cloud models via cross-reconstruction between two decoupled views. The objective is to create a more challenging and informative pre-training task than standard single-view self-reconstruction to learn more robust 3D representations. The core methodology introduces a point cloud crop mechanism to generate two views and a view-relative positional embedding (VRPE) which enables a positional query block to reconstruct one view from the other’s latent representation. The method outperforms the Point-MAE baseline by an average of 6.7% on ScanObjectNN classification under the MLP-LINEAR protocol. For AI practitioners, this cross-reconstruction approach provides a more powerful pre-training strategy for developing 3D vision models, yielding superior frozen feature quality for downstream tasks in label-free settings. |
| SQL-of-Thought: Multi-agentic Text-to-SQL with Guided Error Correction (Read more on arXiv or HuggingFace) |
bindsch, amanchadha, shollercoaster |
The paper introduces SQL-of-Thought, a multi-agent framework that improves text-to-SQL generation through structured reasoning and guided error correction. The primary objective is to develop a robust text-to-SQL system by combining multi-agent decomposition with Chain-of-Thought (CoT) reasoning and a systematic, interpretable correction mechanism. The methodology involves a pipeline of specialized agents for schema linking, subproblem identification, CoT-based query plan generation, and SQL synthesis, along with a novel correction loop guided by a predefined error taxonomy to rectify failures. SQL-of-Thought achieves a state-of-the-art execution accuracy of 91.59% on the Spider benchmark. The principal implication for AI practitioners is that decomposing complex code generation tasks into a multi-agent framework with explicit reasoning steps (e.g., query planning) and taxonomy-guided error correction is more effective than relying on monolithic models or simple execution-based feedback. |
| C-DiffDet+: Fusing Global Scene Context with Generative Denoising for |
|
|
| High-Fidelity Object Detection (Read more on arXiv or HuggingFace) |
Vito Renó, Abdenour Hadid, Bekhouche, xkruvox, ldb0071 |
C-DiffDet+ enhances diffusion-based object detection by integrating global scene context with local proposal features to improve performance on fine-grained tasks. The objective is to overcome the limitations of local feature conditioning in diffusion detectors by explicitly leveraging global information to disambiguate objects with subtle visual cues. The key methodology involves a Global Context Encoder (GCE) that generates a scene-level embedding, which is then fused with local Region of Interest features using a cross-attention-based Context-Aware Fusion (CAF) module within the denoising process. On the CarDD benchmark, C-DiffDet+ achieves a state-of-the-art mean Average Precision of 64.8%, a 1.4% improvement over the DiffusionDet baseline, with a notable 6.8% absolute increase in AP for small objects. For AI practitioners, this work demonstrates that augmenting local features with a global context vector via cross-attention is a highly effective strategy for improving detection accuracy in domains where scene-level understanding is critical, such as industrial defect detection or medical imaging. |
| Metis: Training Large Language Models with Advanced Low-Bit Quantization (Read more on arXiv or HuggingFace) |
Hengjie Cao, wenzi001, ZhouJixian, cnyangyifeng, ChenMengyi |
This paper introduces Metis, a framework that enables stable and effective low-bit (FP8/FP4) training of large language models by managing anisotropic parameter distributions through spectral decomposition. The main objective is to overcome the training instability and performance degradation caused by the wide numerical ranges in LLM parameters when using low-bit quantization. The methodology combines spectral decomposition via randomized SVD to separate dominant and long-tail components, an adaptive spectral learning rate to rebalance updates, and a dual-range regularizer to narrow parameter distributions. Primary results show that Metis enables FP4 training to achieve performance comparable to FP32, while FP8 training surpasses the FP32 baseline, with a 1.1B parameter GPT-2 model achieving a test loss of 3.95 versus 4.00 for the FP32 baseline. The principal implication for AI practitioners is that training LLMs with aggressive FP4/FP8 quantization is now feasible, significantly reducing memory and computational costs while maintaining or even improving model performance compared to standard FP32 training. |
| FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Zhen Wang, Zhuandi He, Shiyue Zhang, Yanwei Lei, zhengchong |
FastFit is a cacheable diffusion model architecture that accelerates multi-reference virtual try-on by decoupling static reference features from the iterative denoising process. The primary objective is to create a virtual try-on framework that supports coherent multi-garment outfit composition and fundamentally solves the computational inefficiency caused by redundant feature re-computation in existing diffusion models. The key methodology is a novel “Cacheable UNet” which uses static, learnable “Reference Class Embeddings” instead of timestep embeddings for garment inputs and a “Semi-Attention” mechanism; this enables reference item features to be pre-computed into a “Reference KV Cache” and reused losslessly across all denoising steps. The framework achieves an average 3.5× speedup over comparable methods while surpassing state-of-the-art models on fidelity metrics, attaining a FID score of 9.311 on the proposed DressCode-MR multi-reference dataset. For AI practitioners, the cacheable architecture provides a generalizable strategy for accelerating subject-driven generative models by isolating time-independent conditional inputs from the iterative generation loop, enabling significant, lossless reduction in inference latency. |
Papers for 2025-09-02
| Title |
Authors |
Summary |
| PVPO: Pre-Estimated Value-Based Policy Optimization for Agentic |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Yuewei Zhang, Penghong Zhao, Wenfeng Feng, Chuzhan, Nothing2Say |
The paper introduces PVPO, a critic-free reinforcement learning algorithm that improves policy optimization for agentic reasoning using a pre-estimated, static value baseline. Its objective is to mitigate the local optima and high computational cost of group policy methods by correcting the cumulative bias from intra-group comparisons and reducing reliance on extensive rollouts. The key methodology is to use a fixed reference model to compute a “Static V Estimate,” which serves as a stable anchor for advantage calculation (Â = Qdyn - Vsta), combined with a group sampling strategy that filters training data and injects ground-truth trajectories for sparse reward samples. PVPO achieves state-of-the-art performance, outperforming the GRPO baseline by over 5 percentage points in average accuracy (61.00% vs. 56.78%) on multi-step retrieval tasks. For AI practitioners, PVPO offers a resource-efficient training paradigm, enabling the development of more stable and performant agentic models with significantly reduced computational overhead by achieving 97% of a baseline’s performance with less than 40% of the cost. |
| T2R-bench: A Benchmark for Generating Article-Level Reports from Real |
|
|
| World Industrial Tables (Read more on arXiv or HuggingFace) |
Yu Zhao, Sishi Xiong, Kaiwen Wei, Changzai Pan, Jie Zhang |
The paper introduces T2R-bench, a bilingual benchmark for evaluating Large Language Models on the industrial task of generating article-level reports from complex, real-world tabular data. The primary objective is to assess an LLM’s ability to transform diverse and complex industrial tables into comprehensive, accurate reports, addressing a gap between academic benchmarks and practical needs. The methodology involves constructing the T2R-bench dataset from 457 industrial tables and proposing a tripartite evaluation framework comprising a Numerical Accuracy Criterion (NAC), an Information Coverage Criterion (ICC), and a General Evaluation Criterion (GEC). Experiments on 25 LLMs reveal that current models struggle significantly, with the top-performing model, Deepseek-R1, achieving an overall score of only 62.71%. The principal implication for AI practitioners is that current LLMs have fundamental limitations in reasoning over large-scale industrial tabular data for report generation, highlighting a critical need for developing specialized models for this application. |
| No Label Left Behind: A Unified Surface Defect Detection Model for all |
|
|
| Supervision Regimes (Read more on arXiv or HuggingFace) |
Danijel Skočaj, MaticFuc, blaz-r |
This paper presents SuperSimpleNet, a unified discriminative model for surface defect detection designed to operate across unsupervised, weakly supervised, mixed, and fully supervised settings. The objective is to create a single, efficient model that can leverage all available data annotations, regardless of their supervision level, to address diverse real-world industrial inspection scenarios. The methodology enhances the SimpleNet architecture by incorporating a novel latent-space synthetic anomaly generation process using a binarized Perlin noise mask, a dual-branch architecture with an improved classification head, and an adaptive learning procedure. SuperSimpleNet achieves state-of-the-art results across all regimes, including a 98.0% AUROC on the fully supervised SensumSODF dataset, while maintaining a 9.5 ms inference time. The principal implication for AI practitioners is the availability of a single, high-performance model that unifies diverse supervision paradigms, enabling the effective use of heterogeneous and incomplete datasets common in industrial applications without needing separate models for each labeling condition. |
| UI-Level Evaluation of ALLaM 34B: Measuring an Arabic-Centric LLM via |
|
|
| HUMAIN Chat (Read more on arXiv or HuggingFace) |
Omartificial-Intelligence-Space |
This paper presents a UI-level evaluation of the Arabic-centric LLM ALLaM 34B via its deployment in the HUMAIN Chat interface. The primary objective was to assess the model’s performance across diverse linguistic and functional tasks, including modern standard Arabic (MSA), five regional dialects, code-switching, and adversarial safety. The methodology involved generating 115 responses from a 23-prompt pack, which were then scored across five metrics by three frontier LLM judges: GPT-5, Gemini 2.5 Pro, and Claude Sonnet-4. Key results show high performance in code-switching and generation tasks (both mean score 4.92/5) and robust safety (4.54/5), but significantly weaker performance in certain dialects, such as Levantine and Moroccan (both 2.7/5 overall). For AI practitioners, this study’s principal implication is that developing culturally aligned Arabic models requires targeted collection of high-quality dialectal corpora to address performance imbalances and prevent reversion to MSA, thereby improving true dialectal fidelity. |
| From reactive to cognitive: brain-inspired spatial intelligence for |
|
|
| embodied agents (Read more on arXiv or HuggingFace) |
Songming Liu, Qihui Zhu, Caixin Kang, Liyuan Wang, Shouwei Ruan |
The paper presents BSC-Nav, a brain-inspired framework that equips embodied agents with structured spatial memory to transition from reactive to cognitive navigation. The main objective is to develop a unified framework that constructs and leverages structured spatial memory—comprising landmarks, route knowledge, and survey knowledge—to enhance the generalization and long-horizon reasoning of embodied agents in complex environments. BSC-Nav integrates three modules: a landmark memory for salient cues, a cognitive map that voxelizes egocentric trajectories into allocentric survey knowledge using a surprise-driven update strategy, and a working memory that uses MLLMs for hierarchical retrieval to align semantic goals with spatial actions. The framework achieves state-of-the-art performance across diverse navigation tasks; in Object-Goal Navigation on the HM3D dataset, it attains a 78.5% Success Rate, surpassing the prior state-of-the-art method UniGoal by 24.0%. The principal implication for AI practitioners is that this work provides a modular architecture for integrating persistent, structured spatial memory with foundation models, offering a practical blueprint for developing more cognitively capable embodied AI that can overcome the limitations of reactive, stateless agents in real-world tasks. |
| How Can Input Reformulation Improve Tool Usage Accuracy in a Complex |
|
|
| Dynamic Environment? A Study on τ-bench (Read more on arXiv or HuggingFace) |
Jayanth Srinivasa, Mutsumi Nakamura, Satyam Raj, Amir Saeidi, Venkatesh Mishra |
i) This paper proposes the Input-Reformulation Multi-Agent (IRMA) framework, which improves the tool-usage accuracy of LLM agents in dynamic, multi-turn environments. ii) The objective is to mitigate LLM agent failures in reasoning, policy adherence, and information extraction by systematically reformulating the input provided to the agent before it acts. iii) The methodology involves a multi-agent system that automatically augments user queries with relevant domain constraints and tool suggestions, creating a structured input for the primary tool-calling agent. iv) The primary result on the τ-bench benchmark is that IRMA significantly outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively, in overall pass@5 scores. v) The principal implication for AI practitioners is that preemptively structuring an agent’s input with relevant policies and tool context is a highly effective, loop-free strategy to enhance reliability and policy adherence in complex, stateful tasks, proving more robust than post-hoc correction methods. |
Papers for 2025-09-01
| Title |
Authors |
Summary |
| R-4B: Incentivizing General-Purpose Auto-Thinking Capability in MLLMs |
|
|
| via Bi-Mode Annealing and Reinforce Learning (Read more on arXiv or HuggingFace) |
Han Hu, Shiming Xiang, Bolin Ni, Qi Yang, Jie Jiang |
This paper presents R-4B, a multimodal large language model that adaptively engages a step-by-step “thinking” process based on query complexity. The primary objective is to create a computationally efficient MLLM that can autonomously switch between complex reasoning for difficult problems and direct responses for simple ones, thus reducing unnecessary computational overhead. The methodology involves a two-stage process: first, “bi-mode annealing” to train a base model on a curated mix of thinking and non-thinking data, followed by “Bi-mode Policy Optimization” (BPO), a reinforcement learning framework to teach the model when to activate the thinking mode. The resulting R-4B-RL model achieves state-of-the-art results, scoring 68.1% on the MMMU_val benchmark, outperforming comparable open-source models and achieving performance comparable to larger models on reasoning-intensive benchmarks. The principal implication for AI practitioners is that this auto-thinking framework provides a practical method to build more resource-efficient MLLMs that dynamically allocate reasoning resources, optimizing the trade-off between performance and inference cost. |
| EmbodiedOneVision: Interleaved Vision-Text-Action Pretraining for |
|
|
| General Robot Control (Read more on arXiv or HuggingFace) |
Zhaoqing Chen, Qizhi Chen, Haoming Song, sundrops, delinqu |
The paper introduces EO-1, a unified 3B parameter foundation model, and the EO-Data1.5M dataset, which leverages interleaved vision-text-action pretraining to enhance generalist robot control and embodied reasoning. The research objective is to design a training paradigm for robot policies that supports flexible and mutually-informed integration of reasoning and action. The key methodology is a unified decoder-only transformer architecture that synergizes discrete auto-regressive decoding for text with continuous flow matching for actions, trained on EO-Data1.5M, a new dataset of 1.5 million curated, interleaved vision-text-action sequences. EO-1 demonstrates superior performance on multiple benchmarks, achieving a 98.2% success rate on the LIBERO simulation benchmark, significantly outperforming prior state-of-the-art models. For AI practitioners, the principal implication is that pretraining on large-scale, carefully constructed interleaved multimodal data, rather than on siloed robotic and web datasets, is critical for developing vision-language-action models with robust open-world generalization and integrated reasoning capabilities. |
| A.S.E: A Repository-Level Benchmark for Evaluating Security in |
|
|
| AI-Generated Code (Read more on arXiv or HuggingFace) |
Libo Chen, Lei Zhang, Bin Wang, wanng, KekeLian |
This paper introduces A.S.E, a repository-level benchmark for evaluating the security, quality, and stability of LLM-generated code in realistic software engineering contexts. The research objective is to assess LLM security performance on complex, multi-file code generation tasks derived from real-world repositories with documented CVEs, addressing the limitations of snippet-based evaluations. Its methodology employs a reproducible, containerized evaluation framework that uses expert-defined static analysis rules and in-repository build validation to deterministically measure vulnerability remediation. Primary results from evaluating 26 models show that while a top model like Claude-3.7-Sonnet achieves a high code quality score of 91.58, it has a significant security deficit, scoring only 46.72 on security. The principal implication for AI practitioners is that current state-of-the-art LLMs generate functionally correct but insecure code, and concise “fast-thinking” decoding strategies outperform complex reasoning for security patching, highlighting the need for context-aware security validation before deployment. |
| Droplet3D: Commonsense Priors from Videos Facilitate 3D Generation (Read more on arXiv or HuggingFace) |
Qi Jia, Liang Jin, Runze Zhang, Guoguang Du, lixiaochuan |
The paper introduces Droplet3D, a framework that leverages commonsense priors from video data to enhance controllable 3D content generation from joint image and text inputs. The research objective is to mitigate 3D data scarcity by fine-tuning a pre-trained video generation model to inherit spatial consistency and rich semantic knowledge for 3D tasks. The core methodology involves training on Droplet3D-4M, a new large-scale dataset of 4 million 3D models, each paired with a 360° orbital rendered video and dense, multi-view-level text annotations. Droplet3D significantly outperforms prior methods in text-and-image-to-3D generation, achieving a PSNR of 28.36 on the GSO dataset, compared to the next-best baseline’s 22.31. For AI practitioners, this work validates that adapting large video foundation models with curated multi-view 3D datasets is a powerful strategy for creating high-fidelity 3D assets with superior control and generalization, even extending to scene-level generation. |
| TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head |
|
|
| Synthesis (Read more on arXiv or HuggingFace) |
Pengcheng Chen, Zihan Ye, Yexin Liu, Hejin Huang, Shunian Chen |
This paper introduces TalkVid, a large-scale (1,244 hours, 7,729 speakers) and diverse dataset, alongside TalkVid-Bench, a stratified benchmark, to address generalization failures in audio-driven talking head synthesis. The primary objective is to mitigate the brittleness of state-of-the-art models when confronted with the full spectrum of human diversity in ethnicity, language, and age. The key methodology is a principled, multi-stage automated pipeline that sources high-resolution videos and rigorously filters them for motion stability, aesthetic quality, and facial detail, with the pipeline’s efficacy validated against human judgments. Experiments show that a model trained on TalkVid achieves superior performance, recording a Frechet Video Distance (FVD) of 178.396 on the TalkVid-Bench language dimension, significantly better than models trained on prior datasets like HDTF (FVD 205.990). The principal implication for AI practitioners is that training on demographically diverse, high-quality data is essential for building robust and equitable models, while the provided benchmark enables crucial auditing of algorithmic bias across subgroups that aggregate metrics would otherwise obscure. |
| UItron: Foundational GUI Agent with Advanced Perception and Planning (Read more on arXiv or HuggingFace) |
Yufeng Zhong, Wenkang Han, Liming Zheng, Jing Huang, Zhixiong Zeng |
This paper introduces Ultron, an open-source foundational model for GUI agents designed for advanced perception and planning across mobile and PC environments. The research aims to address key challenges in GUI agent development, including the scarcity of high-quality trajectory data, the lack of interactive infrastructure, and the poor performance of existing models in Chinese application scenarios. Ultron’s methodology involves a three-stage training paradigm: supervised fine-tuning for perception and planning, followed by a curriculum reinforcement learning (CuRL) framework using Group Relative Policy Optimization (GRPO) to enable complex reasoning and exploration. On an offline evaluation benchmark for top-tier Chinese mobile apps, Ultron-72B achieves a 47.4% task success rate, significantly outperforming the UI-TARS-72B model’s 32.8%. The principal implication for AI practitioners is that developing robust, real-world GUI agents requires a systemic approach combining targeted data engineering, the creation of interactive environments for reinforcement learning, and domain-specific data collection to overcome the limitations of general-purpose models. |
| Think in Games: Learning to Reason in Games via Reinforcement Learning |
|
|
| with Large Language Models (Read more on arXiv or HuggingFace) |
Yifan Lu, Zining Zhu, Yuan Sui, Yu Gu, Yi Liao |
The paper introduces Think-In-Games (TiG), a framework using reinforcement learning to teach Large Language Models (LLMs) procedural knowledge for strategic decision-making in complex game environments. The research objective is to bridge the gap between an LLM’s declarative knowledge and the procedural knowledge required for dynamic, interactive tasks by enabling the model to learn directly from environmental feedback. The methodology reformulates RL-based decision-making as a language modeling task, employing Group Relative Policy Optimization (GRPO) to iteratively refine language-guided policies based on a simple, rule-based binary reward signal derived from gameplay data. The framework enabled a Qwen-3-14B model to achieve 90.91% accuracy on the in-game action prediction task, outperforming the significantly larger Deepseek-R1 baseline (86.67%). The principal implication for AI practitioners is that online RL with simple, rule-based reward functions can efficiently instill domain-specific procedural reasoning in LLMs, allowing smaller, more deployable models to achieve superior performance in interactive applications. |
| A Survey of Scientific Large Language Models: From Data Foundations to |
|
|
| Agent Frontiers (Read more on arXiv or HuggingFace) |
Jiamin Wu, Wanghan Xu, Wei Li, Chenglong Ma, Ming Hu |
This paper surveys the evolution of scientific large language models (Sci-LLMs), reframing their development as a co-evolution between models and their underlying scientific data substrates. The objective is to provide a data-centric synthesis of the Sci-LLM landscape by formulating a unified taxonomy of scientific data, reviewing models and datasets, and outlining a roadmap toward autonomous agentic systems. The methodology involves a systematic review and meta-analysis of over 270 pre-/post-training datasets and over 190 evaluation benchmarks, alongside formulating a novel hierarchical model of scientific knowledge. The analysis reveals that the current landscape is dominated by text-only models (approx. 74%), with 7B parameter models being the most common size (32%), and shows that leading LLMs’ performance drops from over 80% on general benchmarks to as low as 2-10% on expert-level scientific reasoning tests. The principal implication for AI practitioners is that progress requires a shift from scaling generalist models to developing specialized systems that can handle heterogeneous scientific data and function as autonomous agents within a closed-loop discovery process. |
| TiKMiX: Take Data Influence into Dynamic Mixture for Language Model |
|
|
| Pre-training (Read more on arXiv or HuggingFace) |
Jiyao Deng, Yuanfan Guo, Fengze Liu, Binbin Liu, Yifan Wang |
TiKMiX is a framework that dynamically adjusts data mixtures in LLM pre-training by using a “Group Influence” metric to optimize the evolving impact of data domains on model performance. The research objective is to develop a computationally efficient method to dynamically adjust data mixture proportions during pre-training to align with the model’s changing learning preferences, thereby improving final performance. The methodology introduces Group Influence, an extension of influence functions that calculates a data domain’s collective impact using accumulated gradients, and uses it in two schemes: TiKMiX-D for direct multi-objective optimization of influence, and TiKMiX-M, which trains a LightGBM surrogate model to predict optimal mixtures by modeling non-linear interactions. Primary results demonstrate that Group Influence strongly correlates with downstream performance (Pearson ρ = 0.789); the TiKMiX-M variant achieved an average performance gain of 2.0 points across nine benchmarks over the REGMIX baseline, while the TiKMiX-D variant performed comparably while using only 20% of the computational resources. The principal implication for AI practitioners is that Group Influence offers a computationally efficient diagnostic to periodically re-weight pre-training data, significantly improving model performance and training efficiency by better aligning the data mixture with the model’s state, mitigating the “under-digestion” of data from static ratios. |
| Efficient Code Embeddings from Code Generation Models (Read more on arXiv or HuggingFace) |
Han Xiao, Scott Martens, Michael Günther, Saba Sturua, dariakryvosheieva |
This paper introduces jina-code-embeddings, a family of efficient code embedding models (0.5B and 1.5B parameters) derived from fine-tuning autoregressive code generation models. The primary objective is to create compact, high-performance embedding models specifically for code retrieval tasks by adapting pre-trained decoder-only backbones. The methodology involves initializing models with pre-trained Qwen2.5-Coder weights and fine-tuning them using a contrastive InfoNCE loss objective on diverse code-text pairs, employing task-specific instruction prefixes and last-token pooling for embedding generation. The resulting 1.5B parameter model achieves an average score of 79.04% on the MTEB code retrieval benchmark, outperforming larger general-purpose models like gemini-embedding-001 (77.38%). The principal implication for AI practitioners is that these smaller, specialized models enable the development of resource-efficient yet state-of-the-art code retrieval systems, such as for RAG applications, without the significant overhead of larger models. |
| Morae: Proactively Pausing UI Agents for User Choices (Read more on arXiv or HuggingFace) |
Amy Pavel, Jeffrey P. Bigham, Dingzeyu Li, Yi-Hao Peng |
This paper introduces Morae, an accessible UI agent that proactively pauses automation to allow blind and low-vision (BLV) users to make choices. The research objective is to address the reduced user agency caused by existing UI agents that fully automate tasks without consulting users at critical decision points. Morae employs a large multimodal model to interpret user queries against UI representations and uses a “Dynamic Verification of Ambiguous Choices” mechanism to identify when to pause and generate interactive UIs for user input. In a study with 10 BLV participants, Morae enabled users to make significantly more preference-aligned choices (mean of 4.03) compared to OpenAI Operator (mean of 2.98). The principal implication for AI practitioners is that incorporating mixed-initiative models which proactively detect ambiguity and solicit user clarification, rather than aiming for complete end-to-end automation, is crucial for developing more effective and empowering UI agents. |
| AHELM: A Holistic Evaluation of Audio-Language Models (Read more on arXiv or HuggingFace) |
Siwei Yang, Zijun Wang, Chi Heem Wong, Haoqin Tu, Tony Lee |
This paper introduces AHELM, a holistic benchmark to systematically evaluate Audio-Language Models (ALMs) across 10 aspects including capabilities, fairness, and safety. The research objective is to create a standardized evaluation framework to address the limitations of existing benchmarks, which typically measure only one or two capabilities and lack consistent testing protocols. The methodology involves aggregating 14 datasets and introducing two new ones (PARADE for bias, CoRe-Bench for reasoning), standardizing prompts and inference parameters, and evaluating 14 ALMs against 3 baseline systems composed of an Automatic Speech Recognizer (ASR) paired with a Language Model (LM). The results show that while Gemini 2.5 Pro is the top-ranked model, it exhibits group unfairness (p=0.01) on ASR tasks, and baseline ASR+LM systems perform competitively, with one ranking 5th overall, highlighting that end-to-end ALMs are not universally superior. The key implication for AI practitioners is that for many speech-based tasks, a simpler, engineered system combining a dedicated ASR with an LM can be more robust and performant than a single, complex ALM, necessitating careful comparative benchmarking before deployment. |
| HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data |
|
|
| for Mobile Dexterous Manipulation (Read more on arXiv or HuggingFace) |
Tianhai Liang, Pu Hua, Langzhe Gu, Tianming Wei, Zhecheng Yuan |
HERMES is a framework for mobile bimanual dexterous manipulation that learns from single-shot, multi-source human motion data using reinforcement learning and a robust vision-based sim2real transfer pipeline. The primary objective is to translate heterogeneous human motion data into deployable, physically plausible policies for a mobile dexterous robot, enabling autonomous execution of complex, long-horizon manipulation tasks in unstructured real-world environments. The framework employs a unified reinforcement learning approach with a generalizable reward function to train a state-based expert policy from a single human motion trajectory, which is then distilled into a vision-based student policy via DAgger using augmented depth images; this is integrated with a navigation foundation model refined by a closed-loop Perspective-n-Point (PnP) localizer for precise mobile manipulation. HERMES successfully executed diverse real-world bimanual dexterous manipulation tasks, achieving an average success rate of 67.8%, which represents a +54.5% performance gain compared to a baseline using unprocessed depth inputs. The principal implication for AI practitioners is that combining a generalizable RL reward with DAgger distillation to a depth-image policy and a hybrid control scheme is an effective strategy for transferring skills from minimal human data to high-DoF robots, providing a practical pipeline for sim2real deployment in complex mobile manipulation scenarios. |
| CLIPSym: Delving into Symmetry Detection with CLIP (Read more on arXiv or HuggingFace) |
Raymond A. Yeh, Md Ashiqur Rahman, Tinghan Yang |
CLIPSym is a framework leveraging the pre-trained CLIP model to achieve state-of-the-art performance in image symmetry detection. The main objective is to determine how a pre-trained vision-language model can be effectively adapted for the geometric task of detecting reflection and rotation symmetries. The key methodology involves using CLIP’s image and text encoders with a novel Semantic-Aware Prompt Grouping (SAPG) technique, which aggregates diverse object-based prompts, and a rotation-equivariant decoder based on a Transformer and G-Convolution to generate symmetry heatmaps. The primary result is that CLIPSym outperforms previous methods on the DENDI dataset, achieving an F1-score of 66.5% for reflection detection, which is a 2.0% improvement over the prior state-of-the-art. The principal implication for AI practitioners is that large pre-trained vision-language models can be successfully fine-tuned for specialized geometric tasks, and that performance can be significantly enhanced by combining principled equivariant decoder architectures with sophisticated prompting strategies that leverage the model’s semantic understanding. |
Papers for 2025-08-29
| Title |
Authors |
Summary |
| Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable |
|
|
| Text-to-Image Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jiazi Bu, Yujie Zhou, Zhimin Li, yuhangzang, CodeGoat24 |
This paper introduces PREF-GRPO, a reinforcement learning method using pairwise preference fitting to stabilize text-to-image (T2I) generation, and a fine-grained evaluation benchmark named UNIGENBENCH. The main objective is to mitigate the “reward hacking” problem, which the authors attribute to “illusory advantage”—an issue where minimal pointwise reward score differences are amplified during normalization, destabilizing training. The key methodology shifts from traditional reward score maximization to pairwise preference fitting, where a model compares pairs of generated images, and their resulting win rates are used as the reward signal for Group Relative Policy Optimization (GRPO). On the proposed UNIGENBENCH, PREF-GRPO achieved a 5.84% increase in the overall score over the score-maximization baseline, including a 12.04% improvement in Logical Reasoning. For AI practitioners, this research provides a more stable training paradigm for RL-based T2I models by demonstrating that optimizing for relative preferences, rather than absolute scores, effectively mitigates training instability and reward hacking. |
| rStar2-Agent: Agentic Reasoning Technical Report (Read more on arXiv or HuggingFace) |
Weijiang Xu, Yi Zhu, Yifei Liu, Ning Shang, lynazhang |
rStar2-Agent is a 14B math reasoning model trained with a novel agentic reinforcement learning approach to achieve frontier-level performance. The main objective is to make agentic reinforcement learning (RL) effective at scale for complex reasoning by overcoming challenges like high rollout costs, environment noise from coding tools, and inefficient training. The methodology combines an efficient RL infrastructure with a high-throughput code environment, a new RL algorithm called GRPO-RoC (Group Relative Policy Optimization with Resampling-on-Correct) to filter noisy trajectories, and a multi-stage training recipe starting with non-reasoning Supervised Fine-Tuning (SFT). The resulting rStar2-Agent-14B model achieves an 80.6% pass@1 score on the AIME24 benchmark in just 510 RL steps, surpassing the much larger 671B DeepSeek-R1 model while generating significantly shorter responses. The principal implication for AI practitioners is that this work provides a compute-efficient recipe and a scalable infrastructure for training smaller models with agentic RL to achieve reasoning capabilities that meet or exceed those of much larger models, offering a practical path for developing powerful agents with limited GPU resources. |
| USO: Unified Style and Subject-Driven Generation via Disentangled and |
|
|
| Reward Learning (Read more on arXiv or HuggingFace) |
Jiahe Tian, Mengqi Huang, wuwx, cb1cyf, fenfan |
The USO model unifies style-driven and subject-driven image generation into a single framework using a novel cross-task co-disentanglement and reward learning paradigm. The primary objective is to determine if these traditionally separate tasks can be unified and mutually enhanced by jointly learning to disentangle content from style features. The methodology involves a two-stage training process on a specially curated triplet dataset (<style_ref, content_ref, stylized_result>): first, a style-alignment stage fine-tunes a T2I model using a Hierarchical Projector on SigLIP embeddings, followed by a content-style disentanglement stage that introduces a separate VAE encoder for content and is further optimized via Style Reward Learning (SRL). USO achieves state-of-the-art performance on the USO-Bench, attaining the highest scores for both subject consistency and style similarity, including a CSD score of 0.557 for style-driven tasks and a CLIP-I score of 0.623 for subject-driven tasks. The principal implication for AI practitioners is that a single, efficient customization model can handle both style transfer and subject preservation without requiring separate architectures, as the cross-task training approach demonstrates that learning feature isolation for one task improves feature exclusion for its complementary task. |
| AWorld: Orchestrating the Training Recipe for Agentic AI (Read more on arXiv or HuggingFace) |
Qintong Wu, Dong Wang, Chenyi Zhuang, Chengyue Yu, IcyFish |
AWORLD is an open-source, distributed framework designed to accelerate the “learning from practice” paradigm for agentic AI by parallelizing agent-environment interaction. The primary objective is to overcome the bottleneck of inefficient experience generation (rollouts) in complex benchmarks, thereby making large-scale reinforcement learning for agents computationally feasible. The methodology involves a Kubernetes-based distributed architecture that executes agent-environment interactions concurrently across a cluster, separating the high-throughput rollout phase from the model training phase, which uses the GRPO algorithm for updates. The framework achieves a 14.6x speedup in experience collection compared to sequential execution; leveraging this, a trained Qwen3-32B agent improved its average pass@1 score on the GAIA benchmark from 21.59% to 32.23% and achieved a 16.33% score on the most difficult Level 3 tasks, surpassing listed proprietary models. The principal implication for AI practitioners is that for complex agentic tasks, the primary bottleneck has shifted from training computation to environment interaction, and this work provides a practical, open-source infrastructure for implementing the massively parallel rollouts necessary to train high-performing agents. |
| TCIA: A Task-Centric Instruction Augmentation Method for Instruction |
|
|
| Finetuning (Read more on arXiv or HuggingFace) |
Simin Ma, kqsong, songwang41, huuuyeah, shujian2025 |
TCIA is a framework for systematically augmenting instructions to fine-tune LLMs, preserving task relevance and diversity through a structured query-constraint representation. The objective is to overcome task drift and diversity collapse in automated instruction generation for fine-tuning LLMs on specialized, real-world tasks. The method decomposes instructions into base queries and explicit constraints, then applies a Breadth-First Search (BFS) algorithm to generate new variants by adding, removing, or replacing constraints retrieved from a task-organized database. Models fine-tuned with TCIA improved performance by an average of 8.7% across four real-world applications and maintained a near-100% on-task instruction ratio, while the baseline WizardLM’s ratio dropped below 60% after three augmentation hops. AI practitioners can leverage TCIA to cost-effectively adapt open-source LLMs for specific enterprise applications, achieving superior task-specific performance without degrading general instruction-following abilities. |
| Mixture of Contexts for Long Video Generation (Read more on arXiv or HuggingFace) |
Junfei Xiao, Yuwei Guo, Lvmin Zhang, Ceyuan Yang, Shengqu Cai |
The paper introduces Mixture of Contexts (MoC), a learnable sparse attention mechanism that recasts long video generation as an internal information retrieval task to improve efficiency and temporal consistency. The main objective is to scale diffusion transformers for coherent, minute-long video generation by overcoming the quadratic computational complexity of standard self-attention. The key methodology is the MoC module, which replaces dense attention by partitioning the token stream into content-aligned chunks; for each query, a top-k router dynamically selects relevant chunks based on mean-pooled key similarity, while mandatorily attending to text and local-shot anchors and enforcing a causal routing mask to prevent feedback loops. On multi-shot video generation (180k tokens), MoC reduces attention FLOPs by over 7x and achieves a 2.2x end-to-end speedup with 85% sparsity, while improving motion diversity (Dynamic Degree from 0.46 to 0.56) and maintaining quality compared to a dense attention baseline. The principal implication for AI practitioners is that MoC provides a framework for building scalable long-sequence transformers by replacing computationally expensive dense attention with an efficient, learned, and dynamic sparse routing mechanism, demonstrating that reallocating compute to salient historical context is a practical path to long-term memory in generative models. |
| CogVLA: Cognition-Aligned Vision-Language-Action Model via |
|
|
| Instruction-Driven Routing & Sparsification (Read more on arXiv or HuggingFace) |
Liqiang Nie, Jie He, Rui Shao, Renshan Zhang, Wei Li |
CogVLA is a cognition-aligned Vision-Language-Action model that uses a three-stage, instruction-driven routing and sparsification architecture to improve both performance and computational efficiency for robotic manipulation. The primary objective is to overcome the high computational overhead and cross-modal semantic degradation in existing VLA models by jointly optimizing the perception-reasoning-action pipeline. The methodology consists of: 1) EFA-Routing, which uses instructions to selectively aggregate visual tokens in the encoder; 2) LFP-Routing, which prunes instruction-irrelevant tokens within the LLM; and 3) V-L-A Coupled Attention (CAtten), which ensures coherent, parallel action decoding from compressed inputs. Experimentally, CogVLA achieves a state-of-the-art success rate of 97.4% on the LIBERO benchmark while reducing inference latency by 2.8x and training costs by 2.5x compared to OpenVLA. The principal implication for AI practitioners is that integrating instruction-driven, multi-stage sparsification across modalities provides a concrete framework for developing highly efficient and performant VLA models suitable for scalable deployment on resource-constrained systems. |
| MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World |
|
|
| Tasks via MCP Servers (Read more on arXiv or HuggingFace) |
Shashank Biju, Hemani Patel, Qi Chang, Zhenting Wang, ankits0052 |
MCP-Bench introduces a benchmark for evaluating LLM agents on complex, multi-step, real-world tasks using a live ecosystem of 28 Model Context Protocol (MCP) servers and 250 tools. The objective is to assess advanced agentic capabilities, including tool use, cross-tool coordination, and long-horizon planning, which are not adequately evaluated by existing benchmarks that rely on isolated APIs and shallow workflows. The methodology features an automated task synthesis pipeline that discovers tool dependency chains and generates 104 tasks with fuzzy, natural language instructions; evaluation is conducted via a hybrid framework combining rule-based metrics (schema compliance, execution success) and a rubric-based LLM-as-a-Judge to score task completion, tool usage, and planning effectiveness. Primary results from evaluating 20 LLMs show that while schema understanding is high across top models (>98% compliance), higher-level reasoning remains a challenge; the best-performing model, gpt-5, achieved an overall score of 0.749, whereas smaller models like llama-3-1-8b-instruct scored only 0.428, with their performance degrading further in multi-server settings. The principal implication for AI practitioners is that while basic tool execution has largely converged, building agents for complex, multi-domain workflows requires frontier models, as long-horizon planning and robust orchestration across multiple servers are key differentiating capabilities and significant bottlenecks for less advanced models. |
| OneReward: Unified Mask-Guided Image Generation via Multi-Task Human |
|
|
| Preference Learning (Read more on arXiv or HuggingFace) |
Yitong Wang, Shiyin Wang, Yuan Gong, wujie10, XionghuiWang |
This research introduces OneReward, a unified reinforcement learning framework that fine-tunes a single generative model for multiple mask-guided editing tasks using a single vision-language model (VLM) as a reward signal. The primary objective is to develop a single, proficient mask-guided image generation model capable of handling diverse sub-tasks like image fill, object removal, and text rendering without relying on task-specific supervised fine-tuning. The framework trains a VLM on multi-dimensional human preference data to act as a unified reward model which provides task- and criterion-specific feedback by evaluating image pairs; this reward signal is then used in a multi-task reinforcement learning pipeline to directly optimize a pre-trained flow matching model. The resulting model, Seedream 3.0 Fill, demonstrated superior performance over competitors, achieving a 69.04% usability rate on the image fill task, outperforming the next best model by 16.93 percentage points. For AI practitioners, this framework provides a method to consolidate multi-task generative model alignment into a single training process with one reward model, reducing the complexity and resource requirements for building versatile image editing tools. |
| Turning the Spell Around: Lightweight Alignment Amplification via |
|
|
| Rank-One Safety Injection (Read more on arXiv or HuggingFace) |
Bernard Ghanem, George Turkiyyah, Hasan Abed Al Kader Hammoud, Harethah Abu Shairah |
This paper introduces RANK-ONE SAFETY INJECTION (ROSI), a fine-tuning-free method to amplify an LLM’s safety alignment by permanently modifying its weights. The primary objective is to investigate if a model’s inherent safety mechanisms can be systematically amplified by steering its activations towards a refusal-mediating subspace. The methodology involves extracting a “safety direction” vector from the activation differences between harmful and harmless prompt pairs and then applying a rank-one update to all residual stream write matrices. Experiments show ROSI consistently increases safety refusal rates—for example, boosting YI-6B-CHAT’s harm refusal by +18.2 points—while preserving utility on benchmarks like MMLU and significantly reducing jailbreak success rates. For AI practitioners, the principal implication is that this lightweight, interpretable weight-editing technique provides an efficient, low-cost “last-mile” procedure to enhance the safety of both aligned and uncensored models without requiring expensive retraining. |
| Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability |
|
|
| in Knowledge and Safety with DuET-PD (Read more on arXiv or HuggingFace) |
Roy Ka-Wei Lee, Nancy F. Chen, Zhengyuan Liu, Daniel Wai Kit Chin, Incomple |
This paper introduces DuET-PD, a novel framework for evaluating LLM persuasion dynamics in multi-turn dialogues, and proposes Holistic DPO to balance robustness and receptiveness. The main objective is to measure and foster appropriate stance-change behavior in LLMs during multi-turn dialogues across knowledge (MMLU-Pro) and safety (SALAD-Bench) domains. DuET-PD employs a systematic multi-turn evaluation protocol using MMLU-Pro and SALAD-Bench MCQs, subjecting models to positive (corrective) or negative (misleading) persuasion appeals across three turns, with iterative stance checks and confidence recording, and explores mitigation via Holistic Direct Preference Optimization (DPO). Results demonstrate that even state-of-the-art GPT-40 achieves only 27.32% accuracy in MMLU-Pro under sustained misleading persuasions, while Holistic DPO significantly improves Llama-3.1-8B-Instruct’s NEG-Acc@3 in safety contexts from 4.21% to 76.54%. This work implies that AI practitioners must prioritize balanced training approaches to cultivate epistemic integrity in LLMs, ensuring they resist misinformation while remaining receptive to valid corrections in high-stakes applications. |
| Dress&Dance: Dress up and Dance as You Like It - Technical Preview (Read more on arXiv or HuggingFace) |
Yu-Xiong Wang, Minh Phuoc Vo, Aayush Bansal, Jun-Kun Chen |
Dress&Dance is a novel video diffusion framework generating high-resolution, temporally consistent virtual try-on videos with user-controlled motion and garment options. Its main objective is to generate high-quality 5-second, 24 FPS virtual try-on videos at 1152 x 720 resolution of a user wearing desired garments animated by a reference video. The key methodology involves CondNet, a novel attention-based conditioning network that unifies multi-modal inputs via cross-attention to enhance garment registration and motion fidelity, supported by a multi-stage progressive training strategy and an auto-regressive video refiner. Dress&Dance significantly outperforms open-source baselines and achieves comparable or better quality than commercial models, recording a PSNR of 22.41 and SSIM of 0.9038 on a captured dataset, exceeding TPD+CogVideoX I2V’s PSNR of 14.47 and SSIM of 0.8305. This framework demonstrates to AI practitioners how unified multi-modal conditioning and progressive training strategies can enable state-of-the-art, high-fidelity, and controllable video synthesis in complex virtual try-on applications. |
| OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn |
|
|
| Dialogue with Large Language Models (Read more on arXiv or HuggingFace) |
Alex Endert, Eunyee Koh, Shunan Guo, Adam Coscia |
OnGoal is an LLM chat interface augmenting multi-turn dialogue with goal-tracking visualizations to help users evaluate and review conversational goals. The primary objective was to assess how a linear chat interface can support users in managing conversational goals during extended LLM dialogues. This was achieved through a three-stage LLM-assisted goal pipeline (infer, merge, evaluate) utilizing GPT-40 and prompt engineering, integrated with in-situ visualizations, ex-situ timeline views, and text highlighting, and evaluated via a 20-participant user study comparing it to a baseline chat. Participants using OnGoal reported lower mental demand (2.7 vs 3.9 in baseline, strong evidence) and effort (3.2 vs 4.1 in baseline, weak evidence) for tasks, and spent less time reading (56.8s vs 66.5s in baseline, weak evidence) compared to a baseline interface. For AI practitioners, OnGoal demonstrates that integrating real-time goal tracking and visualization can enhance user engagement and reduce cognitive load, though the study identified that LLM-assisted goal evaluation accuracy (rated 2.9 by users) was significantly lower than goal inference/merging accuracy (rated 4.0). |
| Multi-View 3D Point Tracking (Read more on arXiv or HuggingFace) |
Irem Demir, Siyuan Li, Marko Mihajlovic, Haofei Xu, Frano Rajič |
This research introduces MVTracker, a novel data-driven, feed-forward model for tracking arbitrary 3D points in dynamic scenes using multiple camera views. The primary objective is to develop an accurate online tracker that handles long-range correspondences and occlusions with a practical number of cameras (e.g., four), avoiding the depth ambiguities of monocular systems and the extensive hardware requirements of previous multi-view optimization methods. The methodology involves fusing per-view features into a unified 3D point cloud, leveraging a k-nearest-neighbors (kNN) correlation mechanism with explicit 3D offset vectors, and using a spatiotemporal transformer to iteratively refine trajectories. On the Panoptic Studio and DexYCB real-world benchmarks, MVTracker achieves median trajectory errors of 3.1 cm and 2.0 cm respectively, outperforming existing monocular and multi-view baselines. For AI practitioners, MVTracker provides a practical tool for robotics and dynamic scene reconstruction, enabling robust online 3D tracking in sparse camera setups without requiring per-sequence optimization. |
| FakeParts: a New Family of AI-Generated DeepFakes (Read more on arXiv or HuggingFace) |
Xi Wang, Awais Hussain Sani, Samy Aimeur, Soobash Daiboo, Gaetan Brison |
This paper introduces FakeParts, a class of deepfakes with subtle, localized manipulations, and presents the FakePartsBench dataset to evaluate detectors against this emerging threat. The main objective is to define this new forgery class and systematically benchmark the performance of both human and state-of-the-art algorithmic detectors to expose current vulnerabilities. The methodology consists of creating the 25,000-video FakePartsBench dataset using spatial (e.g., inpainting), temporal (e.g., interpolation), and style-based manipulations, then evaluating multiple detection models and conducting a human perception study. The primary result shows that FakeParts reduce human detection accuracy by over 30% compared to traditional deepfakes and that algorithmic detectors exhibit a trade-off, with foundation-model-based systems handling partial fakes better while non-foundation models perform better on fully synthetic content. For AI practitioners, this research provides a critical benchmark dataset that highlights the urgent need to develop robust, generalized detection systems capable of identifying both fully synthetic videos and fine-grained partial manipulations. |
| Provable Benefits of In-Tool Learning for Large Language Models (Read more on arXiv or HuggingFace) |
Vivien Cabannes, Charles Arnal, Ambroise Odonnat, Sam Houliston |
This paper demonstrates the provable and empirical benefits of in-tool learning over in-weight learning for factual recall in large language models (LLMs). The core objective was to formalize the tradeoff between internalizing knowledge via parameter updates and accessing external sources of truth, seeking the most efficient way for LLMs to acquire and utilize information. The authors established theoretical lower bounds for in-weight memorization and upper bounds for tool-augmented recall via a formal circuit construction, then validated these with controlled experiments on synthetic datasets using small transformers and fine-tuning pretrained LLMs. Results show that in-weight learning parameter requirements scale linearly with facts (empirically, y = 8.14x + 5171), whereas in-tool learning saturates parameter needs beyond approximately 1,000 facts, preserving general capabilities like HellaSwag accuracy (≥98%) with minimal distributional shift (TV distance < 0.04). This work implies that AI practitioners should prioritize developing LLMs with robust tool-use capabilities for scalable knowledge access and retention, rather than relying on rote memorization to expand model capacity. |
| Collaborative Multi-Modal Coding for High-Quality 3D Generation (Read more on arXiv or HuggingFace) |
Ziwei Liu, Liang Pan, Zhaoxi Chen, Ziang Cao |
TriMM is a feed-forward 3D generative model that introduces a collaborative multi-modal coding scheme to unify photometric (RGB, RGBD) and geometric (point cloud) data into a shared latent space for high-quality asset generation. The primary objective is to overcome the limitations of single-modality 3D generative models by creating a framework that synergistically leverages the complementary strengths of diverse data sources to address training data scarcity. The key methodology involves using modality-specific encoders (e.g., DINOv2, PointNet) to map heterogeneous inputs into a unified triplane representation, which is then used to train a triplane latent diffusion model guided by a specialized reconstruction loss. On the OmniObject3D benchmark, TriMM, trained on 80K objects, achieves a PSNR of 14.13 and a Chamfer Distance of 0.096, outperforming models like TRELLIS which was trained on 500K objects. For AI practitioners, this framework provides a practical method to improve 3D generative model performance by integrating varied data types, offering a direct pathway to mitigate the challenge of limited high-quality 3D training data. |
| ROSE: Remove Objects with Side Effects in Videos (Read more on arXiv or HuggingFace) |
Hantang Liu, Zixiang Gao, Jianshu Zeng, Yutong Feng, Chenxuan Miao |
ROSE is a diffusion transformer-based framework for video object removal that explicitly handles physical side effects like shadows and reflections by training on a large-scale, synthetically generated paired dataset. The primary objective is to develop a video object removal model capable of accurately eliminating not only the target object but also its environmental side effects (e.g., shadows, reflections, lighting changes), which existing methods struggle with due to the lack of physically-correct paired training data. The methodology involves: a fully-automatic pipeline using a 3D rendering engine to generate a synthetic dataset of paired videos that capture physical interactions; a reference-based erasing paradigm where the model is conditioned on the complete original video and mask; and an auxiliary difference mask predictor that provides explicit supervision for localizing all object-affected areas. On its synthetic paired benchmark designed to evaluate side effect removal, ROSE achieves a mean PSNR of 31.12, significantly outperforming prior methods like DiffuEraser (26.50). The principal implication for AI practitioners is that leveraging synthetic data from 3D rendering engines is an effective strategy for creating physically-grounded, paired datasets to train robust models for complex visual editing tasks, overcoming real-world data scarcity for phenomena like shadows and reflections. |
Papers for 2025-08-28
| Title |
Authors |
Summary |
| Self-Rewarding Vision-Language Model via Reasoning Decomposition (Read more on arXiv or HuggingFace) |
Zhenwen Liang, Rui Liu, Wenhao Yu, Zongxia Li, ChengsongHuang |
Vision-SR1 introduces a self-rewarding reinforcement learning framework to enhance VLM visual reasoning and mitigate hallucinations and language shortcuts. The primary objective is to improve VLM performance by enforcing visual grounding without external visual supervision. This is achieved by decomposing VLM reasoning into visual perception and language reasoning stages, where the model self-evaluates its generated visual perception for sufficiency to answer the question, assigning a self-visual reward that is combined with traditional answer and format rewards. Using the Qwen2.5-VL-7B backbone, Vision-SR1 attained an average accuracy of 58.8 across diverse benchmarks, outperforming Vision-R1 (57.4) and supervised fine-tuning (55.1). AI practitioners can leverage this self-rewarding mechanism to develop more robust and visually grounded VLMs by integrating internal consistency checks, thereby reducing dependence on costly external annotations. |
| Beyond Transcription: Mechanistic Interpretability in ASR (Read more on arXiv or HuggingFace) |
Aviv Shamsian, Hilit Segev, Yael Segal-Feldman, AvivNavon, netag |
This paper systematically adapts and applies mechanistic interpretability techniques to Automatic Speech Recognition (ASR) models, particularly Whisper and Qwen2-Audio, to reveal internal dynamics. The research aims to understand the internal behavior and dynamics of ASR systems, particularly the mechanisms behind error phenomena like hallucinations, repetition loops, and contextually biased outputs. Key methodologies include logit lens, linear probing, and intervention-based methods (component patching and ablation), alongside an adapted Encoder Lens technique, for analyzing hidden states and causal roles of components. Quantitative results show 94.6% linear decodability of speaker gender from Whisper’s encoder layer 25, and 76% resolution of repetition hallucinations via cross-attention patching at decoder layer 23 (plus 13% at layer 18). These findings enable building internal monitors for hallucination, fine-grained debugging, and informing architectural choices for more robust and transparent ASR systems. |
| Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding |
|
|
| in Vision-Language-Action Policies (Read more on arXiv or HuggingFace) |
Sitong Mao, Chengyue Wu, Tianshuo Yang, Yizhuo Li, Zhixuan Liang |
Discrete Diffusion VLA is a novel Vision-Language-Action (VLA) policy that integrates discrete diffusion with a unified transformer for action decoding. The paper addresses the limitations of existing VLA decoders, which either use sequential autoregressive generation or separate continuous diffusion/flow matching heads, hindering unified, scalable architectures. The methodology involves a single-transformer architecture that applies discrete diffusion to discretized action chunks, trained via cross-entropy, and employs an adaptive re-masking policy for iterative refinement and error correction. Discrete Diffusion VLA achieved 96.3% average success rate on LIBERO, 71.2% visual matching on SimplerEnv-Fractal, and 49.3% overall on SimplerEnv-Bridge, consistently outperforming AR and continuous diffusion baselines. This unified discrete diffusion approach enables parallel, adaptive action decoding with robust error correction, providing a pathway for scalable VLA models that leverage pretrained VLM priors effectively. |
| CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer |
|
|
| Use Agent with Decoupled Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jianze Liang, Yuhang Cao, yuhangzang, rookiexiong, Zery |
CODA introduces a novel trainable dual-brain agent architecture for GUI automation, addressing the trade-off between planning and execution in specialized domains by synergizing a generalist planner (Cerebrum) and a specialist executor (Cerebellum). The main objective is to bridge the gap between robust planning but poor execution in generalist GUI agents and precise execution but limited planning in specialist agents, especially in data-scarce scientific environments. Its methodology employs a two-stage pipeline: Stage 1 uses decoupled Group Relative Policy Optimization (GRPO) for planner specialization, and Stage 2 aggregates trajectories from these specialists for supervised fine-tuning (SFT) of a generalist planner. Evaluated on the ScienceBoard benchmark, CODA (Stage-2) achieved an overall Pass@8 success rate of 39.96%, significantly outperforming the Qwen2.5-VL-32B baseline at 19.49% and UI-TARS-1.5-7B at 15.36%. This framework allows AI practitioners to develop more robust and adaptable GUI automation agents for complex, data-scarce specialized domains by efficiently combining generalist planning with precise specialist execution through experience-driven learning. |
| MIDAS: Multimodal Interactive Digital-human Synthesis via Real-time |
|
|
| Autoregressive Video Generation (Read more on arXiv or HuggingFace) |
Yan Zhou, Haoxian Zhang, Wenyuan Zhang, Liyuan Cui, ChenMing-thu14 |
MIDAS presents a multimodal interactive digital-human synthesis framework, aiming to achieve real-time, low-latency, and consistent video generation from diverse inputs over long horizons. The core methodology employs an autoregressive Large Language Model (LLM) backbone, conditioned by a multimodal projector encoding audio, pose, and text, and guided by a diffusion head for high-quality rendering. It introduces a Deep Compression Autoencoder (DC-AE) achieving a 64x spatial reduction ratio to reduce inference burden and is trained on a large-scale 20,000-hour dialogue dataset. Experiments validate the approach’s low latency and high efficiency, enabling stable, drift-free video generation up to 4 minutes with only 4 denoising steps, and fine-grained multimodal controllability across duplex conversations and multi-lingual synthesis. This framework offers AI practitioners a robust solution for building interactive digital human systems and scalable real-time world models by addressing challenges in control, latency, and temporal consistency. |
| Predicting the Order of Upcoming Tokens Improves Language Modeling (Read more on arXiv or HuggingFace) |
Alham Fikri Aji, Erland, zaydzuhri |
Token Order Prediction (TOP) is proposed as a novel auxiliary training objective to improve language modeling performance. The main objective is to enhance next-token prediction (NTP) by learning better internal representations, addressing Multi-Token Prediction’s (MTP) difficulty in exact future token prediction. The key methodology involves training models to rank upcoming tokens by proximity using a ListNet-based learning-to-rank loss, requiring only a single additional unembedding layer parallel to the NTP head. Primary results show TOP models (340M, 1.8B, 7B parameters) generally outperform NTP and MTP on eight standard NLP benchmarks; for example, the 7B TOP model achieved a TriviaQA accuracy of 30.90 compared to 24.28 for NTP. This implies AI practitioners can leverage TOP as a more scalable and parameter-efficient auxiliary objective for pretraining LLMs, potentially yielding improved general language modeling performance. |
| Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health |
|
|
| Biomarkers Estimation (Read more on arXiv or HuggingFace) |
Anton Ivaschenko, Galina Zubkova, Stepan Botman, Konstantin Egorov, blinoff |
The paper introduces MCD-rPPG, a novel, comprehensive, large-scale multi-view video dataset for remote photoplethysmography (rPPG) and health biomarker estimation. The objective is to overcome limitations of existing rPPG datasets by providing 3600 synchronized multi-view video recordings from 600 subjects across varied conditions, paired with 100 Hz PPG signals and 13 extended health metrics. The methodology involved optical character recognition (OCR) of a tablet clock for video synchronization and a POS algorithm for video-PPG alignment, followed by the development of an efficient multi-task neural network baseline model. This model achieved an MAE of 0.68 for PPG and 4.86 for HR on the MCD-rPPG dataset, demonstrating up to 13% speed improvement on CPU compared to leading models. The public release of this diverse dataset and fast baseline model provides a crucial resource for AI/ML practitioners to train more robust models for rPPG and extended health biomarker estimation, thereby accelerating the development of AI medical assistants. |
| Diffusion Language Models Know the Answer Before Decoding (Read more on arXiv or HuggingFace) |
Shilin Yan, Lu Yin, Dilxat Muhtar, Yefan Zhou, Pengxiang Li |
Diffusion Language Models (DLMs) exhibit early answer convergence, enabling accelerated inference. The paper aims to accelerate DLM inference by identifying and leveraging this early answer convergence phenomenon. Prophet, a training-free fast decoding paradigm, was introduced; it dynamically decides whether to continue refinement or commit early by decoding all remaining tokens based on a confidence gap metric derived from the top-2 prediction candidates. Empirical evaluations on LLaDA-8B and Dream-7B showed Prophet reduces decoding steps by up to 3.4x, for example achieving 3.40x speedup on Sudoku with Dream-7B, while maintaining generation quality, such as matching LLaDA-8B’s 54.0% MMLU accuracy. This work implies that AI practitioners can significantly improve the computational efficiency and practical deployability of DLMs by integrating dynamic early stopping mechanisms based on predictive confidence, rather than fixed-step decoding. |
| Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered |
|
|
| Smartphone Agents (Read more on arXiv or HuggingFace) |
Yue Yao, Yibo Shi, Shidong Pan, Zhixin Lin, Jungang |
This paper introduces SAPA-Bench, the first large-scale benchmark specifically designed to evaluate privacy awareness in MLLM-powered smartphone agents. The main objective is to thoroughly understand the privacy awareness capabilities of these agents, which often access sensitive user data during automated tasks. The methodology involved constructing SAPA-Bench with 7,138 annotated real-world scenarios and proposing five specialized privacy metrics (PRR, PLR, PLAR, PCAR, RA) to benchmark seven mainstream agents. Primary results showed that most benchmarked agents exhibited unsatisfying privacy awareness, with performance remaining below 60% even with explicit hints; Gemini 2.0-flash achieved the best Risk Awareness (RA) of 67%. This highlights the critical need for specialized privacy-focused training, tighter alignment strategies, and the design of effective prompt frameworks to enhance multimodal agents’ risk-response capabilities for secure deployment. |
| AudioStory: Generating Long-Form Narrative Audio with Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Yixiao Ge, Shijie Ma, Yuying Ge, Yuxin Guo, wybertwang |
AudioStory is a unified framework integrating large language models (LLMs) with text-to-audio (TTA) systems to generate structured, long-form narrative audio. The primary objective is to address the challenge of generating temporally coherent and compositionally reasoned long-form audio narratives, which current short-form TTA models struggle with. AudioStory employs LLMs for interleaved reasoning generation, decomposing complex instructions into temporally ordered sub-tasks with contextual cues, and utilizes a decoupled bridging mechanism with specialized semantic and residual tokens to condition a diffusion-based audio generator, all trained through a progressive end-to-end strategy. Extensive experiments on the AudioStory-10K benchmark demonstrate AudioStory’s superiority, achieving an instruction-following CLAP score of 0.392 and a Frechet Audio Distance (FAD) of 3.00, outperforming prior TTA baselines like TangoFlux (CLAP 0.317, FAD 3.49). This framework provides AI practitioners with a robust method for developing advanced long-form audio generation systems by synergizing LLM reasoning with high-fidelity audio synthesis, relevant for applications such as dynamic soundscapes and audiobooks. |
| StepWiser: Stepwise Generative Judges for Wiser Reasoning (Read more on arXiv or HuggingFace) |
Olga Golovneva, Weizhe Yuan, Wenting Zhao, Wei Xiong, sainbar |
STEPWISER introduces a novel RL-trained generative judge that performs meta-reasoning about intermediate steps to enhance LLM reasoning and judgment accuracy. The primary objective is to address the critical challenge of supervising the logical validity of multi-step LLM reasoning by reframing reward modeling as a reasoning task. The methodology combines self-segmentation of Chain-of-Thought into coherent chunks, stepwise data annotation using Monte-Carlo Q-value estimates for relative progress, and online reinforcement learning via GRPO to train the judge. Experiments show that STEPWISER’s RL-trained generative judge significantly outperforms SFT-trained discriminative baselines, achieving a 61.9% average ProcessBench score on the 7B model using Rel-Effective signals, compared to 39.7% for the discriminative baseline. This demonstrates that explicit meta-reasoning, trained with online RL on dense stepwise signals, provides a more effective strategy for improving policy model reasoning during training and inference-time search, including self-correction and high-quality data selection. |
| MotionFlux: Efficient Text-Guided Motion Generation through Rectified |
|
|
| Flow Matching and Preference Alignment (Read more on arXiv or HuggingFace) |
An-An Liu, Chao Xue, Diqiong Jiang, Dan Song, Zhiting Gao |
MotionFlux is an efficient text-guided motion generation framework utilizing rectified flow matching and online preference alignment. The paper addresses the challenges of precise semantic alignment between linguistic descriptions and motion, alongside the slow inference inefficiencies of current text-to-motion systems. Its core methodology combines a high-speed generation framework based on deterministic rectified flow matching with TMR++ Aligned Preference Optimization (TAPO), a self-supervised online preference learning system that uses TMR++ as a proxy reward model to construct preference data. Experimental results show MotionFlux-ultra achieves a state-of-the-art Average Inference Time per Sentence (AITS) of 0.005, an R-Precision Top 1 of 0.536, and an FID of 0.078 on the HumanML3D dataset, outperforming baselines in speed, semantic alignment, and motion quality. This advancement offers AI practitioners a scalable solution for real-time, high-fidelity text-to-motion synthesis, minimizing reliance on extensive human annotation for preference alignment. |
| Taming the Chaos: Coordinated Autoscaling for Heterogeneous and |
|
|
| Disaggregated LLM Inference (Read more on arXiv or HuggingFace) |
Chunlei Han, Sida Zhao, Zefang Chu, Ruogu Du, Rongzhi Li |
HeteroScale is a coordinated autoscaling framework designed to optimize Large Language Model (LLM) inference on heterogeneous and disaggregated Prefill-Decode (P/D) architectures. The main objective is to address challenges in P/D disaggregated LLM serving, including inefficient heterogeneous hardware utilization, network bottlenecks, and architectural imbalance. Key methodologies involve a topology-aware scheduler, novel network-aware abstractions, and a metric-driven policy that uses decode Tokens-Per-Second (TPS) as the primary robust signal to jointly scale prefill and decode pools. Deployed in a massive production environment, HeteroScale increased average GPU utilization by 26.6 percentage points and SM activity by 9.2 percentage points, saving hundreds of thousands of GPU-hours daily while maintaining service level objectives. For AI practitioners, this demonstrates that coordinated, metric-driven autoscaling with topology awareness is crucial for achieving significant resource efficiency and stability in large-scale heterogeneous LLM serving environments. |
| DeepScholar-Bench: A Live Benchmark and Automated Evaluation for |
|
|
| Generative Research Synthesis (Read more on arXiv or HuggingFace) |
Ion Stoica, Ankita Sundar, Harshit Gupta, Negar Arabzadeh, Liana Patel |
This paper introduces DeepScholar-Bench, a live benchmark and automated evaluation framework for generative research synthesis, specifically for generating related work sections of academic papers. Its primary objective is to address the limitations of existing benchmarks by providing a holistic and scalable evaluation for complex, evolving research synthesis tasks. The framework uses recent ArXiv papers as its dataset, defines an automated evaluation across knowledge synthesis, retrieval quality, and verifiability, and employs LLM-as-a-judge metrics validated with over 200 human annotations. DeepScholar-base, a reference pipeline, is also introduced. Evaluation shows that no existing system, including open-source models, search AIs, and OpenAI DeepResearch, exceeds a score of 0.19 across all metrics, indicating significant room for improvement. DeepScholar-base achieves competitive or higher performance, with up to 6.3x higher verifiability compared to OpenAI’s DeepResearch. These findings highlight the inherent difficulty of generative research synthesis and underscore the importance of DeepScholar-Bench as a foundation for developing more capable AI systems in this domain. |
Papers for 2025-08-27
| Title |
Authors |
Summary |
| CMPhysBench: A Benchmark for Evaluating Large Language Models in |
|
|
| Condensed Matter Physics (Read more on arXiv or HuggingFace) |
Dongchen Huang, komusama0930, BoringMarsh, di-zhang-fdu, weidawang |
CMPhysBench is a novel benchmark evaluating Large Language Models’ (LLMs) proficiency in Condensed Matter Physics. Its main objective is to assess LLM problem-solving abilities through 520 graduate-level, meticulously curated calculation problems across core CMP subfields. The benchmark introduces the Scalable Expression Edit Distance (SEED) metric, which uses tree-based representations of expressions and physics-aware normalizations for fine-grained, non-binary partial credit. Evaluation of 18 LLMs revealed a significant capability gap, with the top-performing model, Grok-4, achieving only a 36 average SEED score and 28% accuracy. This underscores the critical need for AI practitioners to focus on physics-aware training, improved scientific alignment, symbolic precision, and embedding physics-aware verification into LLM decoding for domain-specific scientific applications. |
| TreePO: Bridging the Gap of Policy Optimization and Efficacy and |
|
|
| Inference Efficiency with Heuristic Tree-based Modeling (Read more on arXiv or HuggingFace) |
Zhoufutu Wen, Qingshui Gu, zhangysk, aaabiao, yizhilll |
TreePO is a reinforcement learning framework designed to improve the efficacy and inference efficiency of large language models for complex reasoning through heuristic tree-based modeling. It aims to enable LLMs to explore diverse reasoning paths efficiently, reducing computational costs, and to accurately attribute sparse outcome rewards to specific tokens. TreePO reformulates on-policy rollouts as a segment-based tree search, leveraging KV-caching for shared prefixes and introducing a hierarchical advantage estimator that utilizes sub-groups for granular credit assignment, coupled with heuristic sampling. Empirically, TreePO reduces GPU hours by 12% to 43% during training and achieves a 40% reduction in trajectory-level inference time, while improving overall accuracy for GRPO from 46.63% to 54.61%. This provides AI practitioners with a more efficient and scalable method for RL-based post-training of LLMs, reducing sample and compute requirements for complex reasoning tasks. |
| VibeVoice Technical Report (Read more on arXiv or HuggingFace) |
Yaoyao Chang, Wenhui Wang, Jianwei Yu, Zhiliang Peng, unilm |
VIBEVOICE is a novel model for scalable long-form, multi-speaker conversational audio synthesis, aiming to generate up to 90 minutes of speech. Its methodology combines next-token diffusion with a continuous speech tokenizer, achieving a 3200x compression rate at 7.5 Hz, and integrates a pre-trained Large Language Model with a token-level Diffusion Head. The VIBEVOICE-7B model outperforms top-tier systems, demonstrating a Word Error Rate (WER) of 1.29 and Speaker Similarity (SIM) of 0.692 on long conversational speech. Furthermore, its efficient tokenizer yields a leading PESQ score of 3.068 on the LibriTTS test-clean dataset, significantly boosting computational efficiency while preserving audio fidelity. For AI practitioners, this framework offers a powerful solution for high-fidelity, long-duration, multi-speaker speech synthesis, advancing capabilities for complex applications like podcasts and audiobooks. |
| VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D |
|
|
| Space (Read more on arXiv or HuggingFace) |
Rui Chen, Gengxiong Zhuang, Zehuan Huang, fenghora, Nelipot |
VoxHammer is a training-free framework for precise and coherent 3D local editing in native 3D space. The paper aims to enable precise and coherent 3D local editing of existing or AI-generated assets by leveraging pretrained 3D generative models, eliminating the need for additional training. Its methodology involves a two-stage process on a pretrained structured 3D latent diffusion model: precise 3D inversion to cache inverted latents and key-value tokens, followed by denoising-based editing with replacement of preserved regions’ features by cached inverted latents and key-value tokens. Quantitative experiments on the Edit3D-Bench benchmark show VoxHammer achieves superior unedited region preservation, with a Chamfer Distance of 0.012 and a masked PSNR of 41.68, outperforming baselines in overall 3D quality and condition alignment. This training-free approach provides AI practitioners with a robust, high-fidelity method for 3D asset manipulation and enables the synthesis of paired data, laying a foundation for future in-context 3D generation. |
| Spacer: Towards Engineered Scientific Inspiration (Read more on arXiv or HuggingFace) |
zerojun48, kohandy, rallyduck1005, MoonRainy21, mhlee1022 |
Spacer is a scientific discovery system that employs deliberate decontextualization and a multi-stage LLM pipeline to generate novel, high-impact scientific concepts. The system aims to overcome current LLM limitations in scientific creativity by generating original, factually grounded scientific concepts without external intervention, adhering to academic standards. Spacer utilizes NURI, a graph-based inspiration engine that extracts high-potential keyword sets from 180,000 biological publications, and a Manifesting Pipeline comprising Revealing, Scaffolding, and Assessment Frameworks, which use multi-agent LLMs to refine these sets into structured scientific statements. NURI’s evaluation metric achieves an AUROC score of 0.737 for classifying high-impact publications, and the Manifesting Pipeline reconstructs core concepts from top-journal articles with over 85% accuracy. This work demonstrates a crucial architectural paradigm shift for AI development, proving that hybrid AI systems leveraging non-LLM components for creative ideation can overcome inherent limitations of pure generative LLMs, advancing the potential for automated scientific discovery. |
| OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive |
|
|
| Simulation (Read more on arXiv or HuggingFace) |
Jiaqi Yang, Zerong Zheng, Weihong Zeng, Jianwen Jiang, chao0412 |
OmniHuman-1.5 introduces a dual-system cognitive framework for generating semantically coherent and expressive avatar animations. The primary objective is to generate character animations that are not only physically plausible but also semantically coherent and expressive, moving beyond low-level audio synchronization to capture authentic essence and a deeper semantic understanding of emotion, intent, and context. The methodology integrates Multimodal Large Language Models (MLLMs) for deliberative (System 2) semantic guidance and a specialized Multimodal Diffusion Transformer (MMDiT) architecture with a novel Pseudo Last Frame design for reactive (System 1) rendering, synergistically fusing multimodal inputs while mitigating inter-modality conflicts. OmniHuman-1.5 demonstrates leading performance, significantly outperforming OmniHuman-1 [40] with an HKV score of 72.113 compared to 47.561 in full-body scenarios, and achieving a 33% top-1 selection rate in user preference studies against academic baselines. This framework provides AI practitioners with a robust and generalizable approach for creating more intelligent, context-aware, and emotionally resonant digital avatars, opening new avenues for applications in interactive agents and AI-driven content generation beyond simple motion synchronization. |
| UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior |
|
|
| Long-Context Learning (Read more on arXiv or HuggingFace) |
Ran Guo, Siyan Chen, Qiyang Min, Yu Bao, FetchFortune |
UltraMemV2 is a novel memory-layer architecture designed to achieve performance parity with 8-expert Mixture of Experts (MoE) models, offering an efficient alternative for sparse computation. The primary objective was to bridge the performance gap between prior memory-layer designs and state-of-the-art MoE configurations while retaining low memory access. Key methodological innovations include integrating memory layers into every transformer block, simplified value expansion, FFN-based value processing from PEER, principled parameter initialization, and rebalancing memory-to-FFN computation ratios. UltraMemV2 demonstrated performance parity with 8-expert MoE models, and achieved superior results on memory-intensive tasks, notably a +7.9 point improvement in in-context learning. This research indicates that memory-layer architectures, specifically UltraMemV2, are a compelling, efficient alternative for scaling AI models, suggesting that increasing activation density is more impactful than total sparse parameter count. |
| Pixie: Fast and Generalizable Supervised Learning of 3D Physics from |
|
|
| Pixels (Read more on arXiv or HuggingFace) |
Dinesh Jayaraman, Chuhao Chen, Chen Wang, Ryan Lucas, vlongle |
Pixie is a novel framework for fast and generalizable supervised learning of 3D physics from pixels, designed to predict object material properties for realistic simulations. Its objective is to overcome slow, per-scene optimization and lack of generalizability in inferring 3D scene physical properties from visual data. The key methodology involves training a 3D U-Net on a curated PIXIEVERSE dataset to map distilled CLIP 3D visual feature grids to voxelized material fields, predicting discrete material types and continuous parameters (Young’s modulus, Poisson’s ratio, density) via supervised losses, and integrating with Gaussian splatting and MPM solvers. Pixie achieves a VLM realism score of 4.35 ± 0.08, demonstrating 1.46-4.39x improvement in realism and orders of magnitude faster inference (2 seconds) compared to test-time optimization methods, alongside zero-shot generalization to real-world scenes. For AI practitioners, this provides an efficient and generalizable feed-forward approach for integrating physics into virtual environments, accelerating the development of dynamic and interactive AI systems. |
| Autoregressive Universal Video Segmentation Model (Read more on arXiv or HuggingFace) |
Albert Gu, Yu-Chiang Frank Wang, Sukjun Hwang, Miran Heo, cmhungsteve |
The Autoregressive Universal Segmentation Model (AUSM) unifies prompted and unprompted video segmentation using an LLM-inspired autoregressive framework, enabling scalable processing of long video streams with efficient parallel training. The primary objective is to develop a single, scalable architecture for streaming video segmentation that unifies diverse tasks, preserves fine-grained spatio-temporal details, supports long video inference with constant memory, and allows efficient sequence-length-scalable training. AUSM recasts video segmentation as sequential mask prediction, leveraging “History Marker” for fine-grained detail and a Mamba-based “History Compressor” to maintain a fixed-size spatial state for past frames, while employing a parallel training strategy that avoids recurrent frame processing. AUSM demonstrates strong performance across seven benchmarks for both prompted and unprompted tasks, outperforming prior universal streaming methods, and achieves up to 2.5× faster training on 16-frame sequences compared to iterative baselines. This autoregressive formulation provides a practical, memory-efficient, and scalable solution for deploying universal video segmentation, reducing the need for task-specific models and offering substantial training speedups, particularly beneficial for long video sequences. |
| Wan-S2V: Audio-Driven Cinematic Video Generation (Read more on arXiv or HuggingFace) |
Chaonan Ji, Mingyang Huang, Siqi Hu, Li Hu, Xin Gao |
Wan-S2V is an audio-driven cinematic video generation model designed to enhance expressiveness and fidelity in complex human video scenarios. The primary objective is to achieve film-level audio-driven character animation in complex film and television contexts, improving expressiveness and fidelity compared to existing methods. The model leverages the Wan text-to-video foundation model, integrating Wav2Vec-encoded audio features and detailed textual captions generated by Qwen-VL for character motion, with training employing a three-stage process and a hybrid parallel scheme combining FSDP and Context Parallelism. Wan-S2V significantly outperforms state-of-the-art models in quantitative metrics, achieving a Fréchet Inception Distance (FID) of 15.66, which is lower than Hunyuan-Avatar’s FID of 18.07, demonstrating improved frame quality and consistency. This approach offers AI practitioners a more robust and accessible solution for generating high-quality, expressive human video content synchronized with audio, particularly beneficial for cinematic character animation, long-form video generation, and precise video lip-sync editing. |
| CineScale: Free Lunch in High-Resolution Cinematic Visual Generation (Read more on arXiv or HuggingFace) |
Ziwei Liu, Paul Debevec, Ziqi Huang, Ning Yu, Haonan Qiu |
CineScale is a novel inference paradigm enabling high-resolution visual generation for image and video diffusion models. Its main objective is to overcome issues like repetitive patterns and quality degradation in high-resolution synthesis, extending capabilities across T2I, T2V, I2V, and V2V tasks for both UNet and DiT architectures. The methodology integrates tailored self-cascade upscaling, restrained dilated convolution, and multi-scale frequency component fusion into self-attention layers, further adapting DiT models with NTK-RoPE and Attentional Scaling. CineScale achieves 8k image generation without fine-tuning and 4k video generation with minimal LoRA fine-tuning, demonstrating an FID of 49.796 and KID of 0.004 for 4096x4096 image generation, and an FVD of 484.711 for video. This allows AI practitioners to deploy state-of-the-art high-resolution generation with existing pre-trained models, significantly reducing training effort for various visual synthesis applications. |
| FastMesh:Efficient Artistic Mesh Generation via Component Decoupling (Read more on arXiv or HuggingFace) |
Xingang Pan, Yongwei Chen, Armando Fortes, Yushi Lan, Jeonghwan Kim |
FASTMESH is an efficient artistic mesh generation framework that decouples vertex and face creation to significantly reduce token redundancy and accelerate the generation process. The main objective is to overcome the inefficiency of traditional autoregressive approaches, which generate excessively long and redundant token sequences, by developing a framework that treats vertices and faces as separate components. The methodology is a two-stage process: first, an autoregressive model generates a compressed vertex sequence using block-wise indexing, which is refined by a “fidelity enhancer”; second, a bidirectional transformer constructs an adjacency matrix in a single step to define mesh faces. Experimental results on the Toys4K dataset show the method achieves more than 8× faster mesh generation speed compared to state-of-the-art approaches, while also improving mesh quality, evidenced by a superior Chamfer Distance of 4.05%. For AI practitioners, the principal implication is that decoupling the generation of structured data components (e.g., vertices and faces) can dramatically reduce input sequence length for transformers, leading to substantial gains in inference speed and efficiency without sacrificing output quality. |
| ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks (Read more on arXiv or HuggingFace) |
Kai Jia, Cong Ma, Zhihao Cheng, Ying Zeng, Minghao Li |
ReportBench is a systematic benchmark designed to evaluate the content quality of research reports produced by Deep Research agents. Its primary objective is to assess the quality and relevance of cited literature alongside the faithfulness and veracity of statements within generated reports. The methodology involves using expert-authored arXiv survey papers as gold-standard references, employing reverse prompt engineering to create diverse evaluation prompts, and an agent-based framework for automated citation and statement verification. Empirical evaluations indicate that commercial Deep Research agents, such as OpenAI Deep Research, achieve higher performance (e.g., 78.87% citation match rate) compared to standalone LLMs. This benchmark provides AI practitioners with a robust framework to evaluate and enhance the factual accuracy and reliability of LLM-based knowledge synthesis. |
| ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Jiangjie Chen, Mingxuan Wang, Xuefeng Li, Siyu Yuan, Qianyu He |
ThinkDial is the first open-recipe, end-to-end framework enabling gpt-oss-style controllable reasoning in LLMs through discrete operational modes. Its objective is to provide open-source large language models with fine-grained control over computational effort, mimicking proprietary systems’ capabilities for diverse deployment scenarios. The methodology involves a novel end-to-end training paradigm combining Budget-Mode Supervised Fine-tuning with a two-phase Budget-Aware Reinforcement Learning strategy that employs adaptive reward shaping and a critical Leak Penalty mechanism. ThinkDial achieves target compression-performance trade-offs, providing Medium mode with 50% token reduction and <10% performance degradation, and Low mode with 75% token reduction and <15% performance degradation; the Leak Penalty was crucial to prevent reasoning leakage into answer sections, ensuring genuine token reduction. This framework offers AI practitioners a vital open-source solution for managing LLM reasoning depth and computational costs, facilitating optimized deployment for applications with varying accuracy-efficiency requirements. |
| MovieCORE: COgnitive REasoning in Movies (Read more on arXiv or HuggingFace) |
Hung-Ting Su, Ying Cheng, Jia-Fong Yeh, Gueter Josmy Faure, cmhungsteve |
This paper introduces MovieCORE, a novel video question answering (VQA) dataset and an agentic enhancement module designed to advance System-2 cognitive reasoning in video understanding. The objective is to challenge Vision-Language Models (VLMs) with deeper cognitive understanding of movie content, moving beyond surface-level comprehension to infer emotions, character dynamics, causality, and psychological complexity. The authors developed an agentic brainstorming workflow using multiple LLMs as specialized thought agents to generate and refine high-quality question-answer pairs, and proposed Agentic Choice Enhancement (ACE) as a post-training VLM refinement plugin. MovieCORE demonstrates significantly higher cognitive demand, achieving a 99.2% rate for higher-order questions and answers based on Bloom’s Taxonomy, and ACE improves existing VLMs’ performance by up to 25% on this dataset. This work contributes to advancing AI systems’ movie understanding, highlighting current VLM limitations in complex reasoning and offering a computationally efficient, training-free method for VLM output refinement. |
| Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning |
|
|
| Tasks (Read more on arXiv or HuggingFace) |
Daisuke Nohara, Takumi Okamoto, Masaki Kawamura, Satoki Ishikawa, Taishi-N324 |
This paper investigates optimal sparsity configurations for Mixture-of-Experts (MoE) Large Language Models for reasoning tasks. The objective was to identify how optimal MoE sparsity changes between memorization (TriviaQA, HellaSwag) and reasoning (GSM8K, GSM-Plus) tasks, and its interaction with total/active parameters, pre-training loss, and compute. Researchers trained families of Mixtral-style MoEs, sweeping architectural hyperparameters like model width, number of experts per layer (E), and top-k experts per token (k), and evaluated their performance on pre-training loss, task loss, and accuracy under pre-training, GRPO post-training, and test-time compute (Self-Consistency). While increasing total parameters consistently reduced pre-training loss, reasoning task performance (e.g., GSM8K) showed a U-shaped trend, with task loss worsening beyond a certain parameter count and accuracy peaking near a Tokens-per-Parameter (TPP) ratio of approximately 20. Neither GRPO post-training nor test-time compute (Self-Consistency) removed this inverted U-shaped relationship. For AI practitioners, this implies that under a fixed computational budget, optimizing MoE sparsity for reasoning tasks requires careful consideration of active parameter growth or a shift towards denser MoE layers, rather than simply increasing total parameters, to avoid performance degradation. |
| Training Language Model Agents to Find Vulnerabilities with CTF-Dojo (Read more on arXiv or HuggingFace) |
Zijian Wang, Varun Kumar, Hantian Ding, Dingmin Wang, terryyz |
CTF-DOJO introduces a large-scale execution environment for training Language Model agents to identify software vulnerabilities in Capture-The-Flag challenges. The main objective is to overcome the scarcity of scalable, generalizable execution-grounded environments for training capable ML agents in offensive cybersecurity. The key methodology involves CTF-DOJO, which features 658 Dockerized CTF challenges, and CTF-FORGE, an automated LLM-powered pipeline that creates these runtime environments with over 98% success. LLM agents trained on just 486 execution-verified trajectories from CTF-DOJO achieved up to 11.6% absolute Pass@1 gains over baselines, with the 32B model reaching 31.9% Pass@1. This demonstrates that execution-grounded training signals are effective and pivotal for developing high-performance cybersecurity ML agents without dependence on costly proprietary systems. |
| ObjFiller-3D: Consistent Multi-view 3D Inpainting via Video Diffusion |
|
|
| Models (Read more on arXiv or HuggingFace) |
Beiqi Chen, Gangshan Wu, Jie Tang, Jie Liu, Haitang Feng |
ObjFiller-3D is a novel framework for consistent multi-view 3D object inpainting that leverages video diffusion models. The primary objective is to complete and edit high-quality, consistent 3D objects from partial inputs and 3D mask regions, addressing inconsistencies inherent in traditional 2D inpainting for 3D tasks. Its key methodology involves adapting a state-of-the-art video editing diffusion model, VACE, using Low-Rank Adaptation (LoRA) to fill masked 3D regions by processing multi-view renders as a looped video sequence, combined with 3D Gaussian Splatting for reconstruction and reference-based inpainting. ObjFiller-3D achieves superior performance, producing reconstructions with a PSNR of 26.6 (compared to NeRFiller’s 15.9) and an LPIPS of 0.19 (compared to Instant3dit’s 0.25). This method offers more faithful and fine-grained 3D reconstruction, demonstrating strong potential for practical deployment in real-world 3D editing applications and content creation by efficiently leveraging pre-trained video editing models. |
| QueryBandits for Hallucination Mitigation: Exploiting Semantic Features |
|
|
| for No-Regret Rewriting (Read more on arXiv or HuggingFace) |
Manuela Veloso, Sumitra Ganesh, Alec Koppel, William Watson, Nicole Cho |
QueryBandits introduces a contextual bandit framework for mitigating large language model (LLM) hallucinations via semantic query rewriting. The research objective is to proactively steer LLMs away from generating hallucinations by designing feature-aware rewrite strategies. The methodology employs a multi-armed bandit system with five rewrite strategies as arms and 17 linguistic features as contextual attributes, driven by a reward model combining LLM-judge, fuzzy-match, and BLEU scores. The top contextual QueryBandit (Thompson Sampling) achieved an 87.5% win rate over a no-rewrite baseline and outperformed zero-shot static prompting by 42.6% (paraphrase) and 60.3% (expand). This implies AI practitioners can leverage guided, feature-aware query rewrites as an efficient, forward-pass mechanism to reduce hallucination and interpret LLM sensitivity to query context, bypassing the need for retraining or gradient-based adaptation. |
| Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Arman Cohan, Doug Downey, Arpan Sarkar, Yixin Liu, Alan Li |
This paper introduces benchmarks and a probing framework to demystify the contributions of knowledge and reasoning to LLM performance in scientific problem-solving. The primary objective was to systematically disentangle the distinct roles of knowledge recall and utilization from reasoning in LLMs, specifically addressing how external knowledge and CoT fine-tuning impact performance. The authors introduced SCIREAS (a unified suite of ten scientific benchmarks) and SCIREAS-PRO (a reasoning-intensive subset), alongside KRUX, a novel probing framework that supplies models with atomic “knowledge ingredients” (KIs) extracted from reasoning traces to study knowledge recall and usage. Key findings include that retrieving task-relevant knowledge is a critical bottleneck, with vanilla instruct models outperforming reasoning counterparts by ≥10% when provided with in-context KIs, and that reasoning fine-tuning enhances models’ ability to surface helpful knowledge, even for known facts. For AI practitioners, these results highlight the importance of external knowledge injection via mechanisms like RAG and CoT fine-tuning for improving scientific reasoning LLMs, and suggest the need for task-specific evaluations to optimize cost-performance balance. |
| Unraveling the cognitive patterns of Large Language Models through |
|
|
| module communities (Read more on arXiv or HuggingFace) |
Jianxi Gao, Pin-Yu Chen, KBhandari11 |
This paper investigates the cognitive patterns of Large Language Models (LLMs) by developing a network-based framework linking cognitive skills, LLM architectures, and datasets. The main objective was to understand the underlying mechanisms of LLMs and how cognitive skills are encoded and localized within these models, drawing an analogy to human brain organization. The methodology involved constructing a multipartite network of skills, datasets, and LLM modules, applying Louvain community detection and spectral analysis, and evaluating fine-tuning strategies under block-based and channel-based pruning. Primary results show that LLMs do not exhibit precise alignment between predefined cognitive functions and detected skill communities (Adjusted Rand Score clustered around 0 for all models and sparsity ratios in Figure 11), and although community-based fine-tuning induced the most substantial weight changes, all-module fine-tuning achieved the highest overall accuracy (Figure 5f). The principal implication for AI practitioners is that effective fine-tuning strategies for LLMs should leverage distributed learning dynamics and network-wide dependencies, recognizing that rigidly localized modular interventions do not confer a performance advantage. |
Papers for 2025-08-26
| Title |
Authors |
Summary |
| InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, |
|
|
| Reasoning, and Efficiency (Read more on arXiv or HuggingFace) |
jinglinglin, WesKwong, MIASANMIA, gulixin0922, Weiyun1025 |
InternVL3.5 introduces a new family of open-source multimodal models with enhanced versatility, reasoning, and efficiency via novel training and deployment strategies. The primary objective is to significantly advance open-source multimodal models by improving their capabilities and inference efficiency, aiming to close the performance gap with leading commercial models. The key methodologies include a novel Cascade Reinforcement Learning (RL) framework, which combines offline and online RL for robust reasoning, and efficiency optimizations such as a Visual Resolution Router (ViR) for dynamic visual token resolution and Decoupled Vision-Language Deployment (DvD) for parallel vision-language processing across GPUs. As a primary result, the largest model, InternVL3.5-241B-A28B, achieves up to a +16.0% gain in overall reasoning performance and a 4.05x inference speedup compared to its predecessor, InternVL3, while narrowing the performance gap with GPT-5 to 3.9%. AI practitioners can leverage InternVL3.5’s publicly released models and code to develop more efficient and versatile multimodal AI applications, directly benefiting from its advanced reasoning capabilities, enhanced inference speed, and support for novel functionalities like GUI interaction and embodied agency. |
| Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance |
|
|
| for Text-to-Image Generation (Read more on arXiv or HuggingFace) |
Haoxiang Shi, Bu Pi, Mingyang Han, Peng Chen, Yaqi Li |
Visual-CoG is a novel reinforcement learning framework that enhances text-to-image generation through stage-aware rewards and a chain of guidance. It aims to address the limitations of existing autoregressive models in handling multi-attribute and ambiguous prompts by providing immediate feedback throughout the image generation pipeline. The methodology decomposes image generation into three distinct stages—semantic reasoning, process refining, and outcome evaluation—with specific reward signals (Rr, Rp, Ro) guiding each stage. Visual-CoG demonstrates superior performance, achieving a 15.57% average enhancement on GenEval and significant improvements on T2I-CompBench and VisCog-Bench, particularly for complex and reasoning-demanding prompts. This approach offers AI practitioners a more effective policy learning mechanism and improved semantic alignment, leading to higher-fidelity and more semantically consistent image outputs for challenging generative tasks. |
| MV-RAG: Retrieval Augmented Multiview Diffusion (Read more on arXiv or HuggingFace) |
sagiebenaim, omerbenishu, yosepyossi |
MV-RAG introduces a retrieval-augmented multiview diffusion model for text-to-3D generation. The primary objective is to enhance the generation of geometrically consistent and accurate multiview outputs for out-of-domain (OOD) or rare concepts. The key methodology involves a hybrid training strategy that combines structured multiview 3D data with diverse 2D image collections, conditioning a multiview diffusion model on retrieved 2D images, and employing an adaptive fusion mechanism. Quantitatively, MV-RAG achieved a CLIP score of 71.77 (4-views) on OOD/rare concepts, outperforming MVDream (66.47), and a user study MOS of 4.44 for 3D consistency, significantly higher than MVDream’s 3.24. This approach implies that AI practitioners can effectively generate high-fidelity 3D assets for novel and rare concepts by leveraging retrieval from vast 2D image databases, reducing the dependency on extensive 3D datasets for such specialized content. |
| T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Xihui Liu, Xian Liu, Chengqi Duan, Rongyao Fang, Kaiyue |
T2I-ReasonBench is a novel benchmark for evaluating reasoning-informed text-to-image (T2I) generation models. The paper’s objective is to assess T2I models’ ability to reason over prompts, infer implicit meaning, and resolve contextual nuances, moving beyond literal prompt following. The key methodology involves a benchmark of 800 prompts across four dimensions (Idiom Interpretation, Textual Image Design, Entity-Reasoning, Scientific-Reasoning) and a two-stage evaluation using an LLM to generate prompt-specific question-criterion pairs, followed by an MLLM to score generated images. Primary results indicate that open-source models have critical limitations in reasoning, while proprietary models, particularly GPT-Image-1, exhibit stronger reasoning and achieve the highest overall accuracy of 78.7%. The principal implication for AI practitioners is the necessity to improve reasoning capabilities in next-generation T2I systems by integrating structured knowledge bases and advanced reasoning mechanisms. |
| Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory |
|
|
| and Test-Time Compute Scaling (Read more on arXiv or HuggingFace) |
Daniil Orel, mbur, yurakuratov, b1l4lx1, irodkin |
This paper investigates how recurrence, memory, and test-time compute scaling enhance multi-step reasoning capabilities in neural models using a cellular automata benchmark. The study aims to understand how different neural architectures and training methods affect multi-step reasoning, disentangling genuine generalization from memorization, and how reasoning depth scales with task complexity. The authors trained Transformers, LSTMs, Mamba, and Associative Recurrent Memory Transformers (ARMT) on a 1D Cellular Automata (1dCA) benchmark with disjoint train/test rule sets and various prediction horizons. Fixed-depth (4-layer) autoregressive models achieved 95% accuracy for single-step prediction (k=1) but dropped below 25% for k >= 3, whereas token-level Chain-of-Thought training enabled GPTNeox to achieve >99% accuracy up to k=4. Adaptive Computation Time (ACT) consistently yielded approximately one additional effective reasoning step, and RL with GRPO attained k=3 accuracy. For AI practitioners, objectives enforcing multi-step prediction and mechanisms adaptively allocating computational depth are crucial, with explicit intermediate representations like Chain-of-Thought offering the most reliable route to deeper generalization. |
| Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement |
|
|
| Learning for General LLM Reasoning (Read more on arXiv or HuggingFace) |
Jiale Zhao, Wenkai Fang, Shunyu Liu, Sunzhu Li, BAOLONGZHANSHEN |
Rubric-Scaffolded Reinforcement Learning (RuscaRL) is a novel instructional scaffolding framework designed to break the exploration bottleneck for general LLM reasoning in Reinforcement Learning with Verifiable Rewards (RLVR). The research aims to overcome the dilemma where RL improvement in LLMs is bounded by limited exploration for high-quality samples. RuscaRL employs checklist-style rubrics as (1) explicit scaffolding during rollout generation, incorporating intra-group differentiation and inter-step decay to steer diverse high-quality responses, and (2) verifiable rewards during model training, utilizing LLM-as-a-Judge for binary evaluation and weighted aggregation. Notably, RuscaRL boosts Qwen-2.5-7B-Instruct from 23.6 to 50.3 on HealthBench-500, surpassing GPT-4.1, and a fine-tuned Qwen3-30B-A3B-Instruct variant achieves 61.1 on HealthBench-500, outperforming OpenAI-03. This framework implies that AI practitioners can effectively expand LLM reasoning boundaries and enhance performance, particularly for general open-ended tasks lacking objective ground-truth answers, by integrating external guidance and fine-grained, verifiable rewards. |
| PosterGen: Aesthetic-Aware Paper-to-Poster Generation via Multi-Agent |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Chenyu You, Yiwei Xu, Xiang Zhang, VitaCoco, HadlayZ |
PosterGen introduces an aesthetic-aware multi-agent LLM framework for generating academic posters from research papers. This work addresses the challenge of automating high-quality academic poster creation by embedding core design and aesthetic principles often neglected by existing methods. Its methodology employs a multi-agent system comprising Parser & Curator, Layout, Styling (Color & Font), and Renderer agents, which mimic professional design workflows and integrate principles like ABT narrative, a three-column layout, and a CSS-like box model. Quantitative evaluations using VLM-as-Judge metrics reveal PosterGen achieves content fidelity comparable to human designs and significantly outperforms state-of-the-art multi-agent methods in visual designs, with an average design score of 4.44 versus 4.26 for PosterAgent (GPT-40 evaluation) and a peak of 4.90 in ‘Font Legibility’. This framework provides AI practitioners with a robust, design-centric solution to automate complex document-to-visual generation, thereby streamlining academic communication and reducing manual design efforts. |
| UQ: Assessing Language Models on Unsolved Questions (Read more on arXiv or HuggingFace) |
Wei Liu, Rui Sun, Zihao Wang, Fan Nie, kzliu |
UQ introduces a novel paradigm and platform for evaluating Large Language Models (LLMs) on difficult, realistic, and unsolved questions. The main objective is to establish a benchmark that challenges frontier models and reflects natural, real-world information needs, pushing the boundaries of AI capabilities. The methodology involves a three-stage pipeline to curate 500 unsolved questions (UQ-Dataset), LLM-based validation strategies (UQ-Validators) to pre-screen candidate solutions by leveraging the generator-validator gap, and an open platform (UQ-Platform) for community-driven human verification. Primary results show the UQ-Dataset filtering process significantly increases question difficulty, with LLM-judged answer correctness dropping from 51.2% to 14.1%, and the strongest LLM-based validator achieving 85.4% accuracy and 40.0% precision on a challenging surrogate dataset. For AI practitioners, UQ offers a unique testbed to evaluate frontier models on open-ended, real-world problems where ground-truth is absent, fostering progress in advanced AI development through continuous, community-driven evaluation. |
| MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for |
|
|
| N-level Assessment (Read more on arXiv or HuggingFace) |
Doratossadat Dastgheib, Seyed Mohammad Hadi Hosseini, Marzia Nouri, Arshia Hemmat, omidgh |
MEENA (PersianMMMU) introduces the first multimodal-multilingual benchmark for evaluating Persian Vision-Language Models (VLMs) across N-level educational exams. The objective is to address the gap in assessing Persian VLMs across scientific, reasoning, and human-level understanding tasks, moving beyond English-centric evaluations. The methodology involved compiling a dataset of approximately 7,500 Persian and 3,000 English questions with rich metadata, covering diverse educational subjects, and evaluating models like GPT-4o, GPT-4-Turbo, Gemini-2.0-flash, and InstructBLIP-T5 across various experimental settings including Zero-Shot and Wrong Image. Primary results show knowledge-based tasks outperformed reasoning tasks by +10–19% in absolute accuracy, and Gemini 2.0-flash demonstrated higher hallucination detection rates for image mismatches in Persian contexts, with over 400 more detections than GPT-4 Mini on the MEENA dataset. For AI practitioners, these findings underscore the critical need to enhance complex multimodal reasoning and robust hallucination detection capabilities in VLMs, particularly when extending to diverse languages and culturally nuanced content. |
| Explain Before You Answer: A Survey on Compositional Visual Reasoning (Read more on arXiv or HuggingFace) |
Xin Zheng, Zixian Ma, Joy Hsu, Fucai Ke, ControlNet |
This survey provides a comprehensive review of Compositional Visual Reasoning (CVR) research published between 2023 and 2025. It aims to provide a unified taxonomy, historical roadmap, and critical outlook on CVR by addressing its necessity, architectural paradigms, benchmarks, and limitations. The methodology involves systematically reviewing over 260 papers from top AI venues and distilling the field into a five-stage developmental roadmap. Key findings include identifying a paradigm shift from prompt-enhanced language-centric methods to unified agentic VLMs, and cataloging over 60 benchmarks and evaluation metrics. This work offers a foundational reference for AI practitioners, guiding the development of more interpretable, generalizable, and robust visual reasoning systems. |
| ST-Raptor: LLM-Powered Semi-Structured Table Question Answering (Read more on arXiv or HuggingFace) |
Wei Zhou, Boxiu Li, Xuanhe Zhou, Boyu Niu, Zirui Tang |
ST-Raptor is an LLM-powered, tree-based framework designed for accurate question answering over semi-structured tables with complex layouts. The main objective is to overcome limitations of existing methods in understanding and accurately answering natural language questions over semi-structured tables, which often feature hierarchical headers and merged cells. The key methodology involves introducing a Hierarchical Orthogonal Tree (HO-Tree) structural model to represent table layouts, defining a set of basic tree operations to guide LLM execution, decomposing questions into sub-questions with operation pipeline generation and table alignment, and employing a two-stage verification mechanism for answer reliability. Experiments on the SSTQA dataset show that ST-Raptor achieves the highest overall accuracy, exceeding the second-best method by 10.23% on the SSTQA benchmark. This framework implies that AI practitioners can significantly improve the accuracy and reliability of LLM-powered data extraction and question answering from complex semi-structured documents by adopting structured table representations and pipeline-based verification mechanisms. |
| SpotEdit: Evaluating Visually-Guided Image Editing Methods (Read more on arXiv or HuggingFace) |
Ersin Yumer, Haitong Tian, Wei-An Lin, Sara Ghazanfari |
SpotEdit introduces a comprehensive benchmark for evaluating visually-guided image editing methods, including a novel hallucination subset. The main objective is to systematically assess the performance and robustness of generative models in complex editing scenarios, particularly when visual cues are incomplete. The methodology involves a three-stage data generation pipeline leveraging Llama-3.1-8B-Instruct, InternVL3-8B, and GPT-40 to create diverse benchmark instances from video keyframes, evaluated by Global Score, Object Fidelity, Background Fidelity, and a Failure Rate metric for hallucination. Primary results indicate that visually-guided editing remains challenging, with the strongest open-source model achieving only a 0.685 Global Score, and GPT-40 exhibiting high hallucination failure rates, such as 91.7% for Inp. Robustness on real data. This highlights to AI practitioners the critical need for developing more robust and reliable visually-guided image editing systems capable of handling incomplete visual cues and avoiding spurious content generation. |
| German4All - A Dataset and Model for Readability-Controlled Paraphrasing |
|
|
| in German (Read more on arXiv or HuggingFace) |
Cristian-George Craciun, Maximilian Müller, Eslam Nasrallah, Thanh Mai Pham, Miriam Anschütz |
The paper introduces German4All, the first large-scale German dataset and an associated model for readability-controlled, paragraph-level paraphrasing across five complexity levels. The objective is to provide suitable resources for fine-grained text adaptation in German, addressing the limitation of single-output simplification systems. The methodology involved synthesizing over 25,000 Wikipedia paragraphs into five distinct readability levels using GPT-4, followed by rigorous evaluation via human annotators and an LLM-as-a-judge, and subsequent fine-tuning of an open-source Flan-T5-xl model. A primary result is that the fine-tuned model achieved state-of-the-art SARI scores on German text simplification benchmarks, for instance, German4All-level1 obtained 53.9 SARI on the German4All-Corrected test set, surpassing other compared models. This implies AI practitioners can utilize German4All to develop and evaluate multi-level paraphrasing models, enabling more nuanced and reader-specific text adaptations for improved accessibility and diverse applications in German. |
| Limitations of Normalization in Attention Mechanism (Read more on arXiv or HuggingFace) |
Radu State, Tatiana Petrova, mbur, opensapce |
This paper theoretically and empirically investigates the limitations of normalization, particularly softmax, in attention mechanisms. The main objective is to quantify the selective ability, geometric separation, and gradient sensitivity of attention mechanisms under various normalization schemes. The methodology involves deriving non-asymptotic bounds for representation distance, geometric separability, and Jacobian norm, validated by experiments on a pre-trained GPT-2 model. Key findings include that no more than ≈ 80% of top-N tokens can be geometrically distinguished, and the Jacobian norm of softmax normalization scales as 1/(4T), indicating high gradient sensitivity at low temperatures. For AI practitioners, this implies limiting the active token set to a sub-linear function of context length and avoiding aggressive temperature scaling (T < 0.1) to ensure training stability and effective token differentiation. |
| TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language |
|
|
| Modeling (Read more on arXiv or HuggingFace) |
Jiaqi Li, Junan Zhang, Xueyao Zhang, Dekun Chen, Yuancheng Wang |
TaDiCodec is a novel text-aware diffusion speech tokenizer that leverages a single-stage, end-to-end training paradigm for efficient speech language modeling. The primary objective is to address limitations of existing speech tokenizers, such as dependence on multi-layer RVQ, high frame rates, reliance on auxiliary pre-trained models, and complex two-stage training processes. It employs a Transformer-based diffusion autoencoder with Binary Spherical Quantization (BSQ) and a flow matching-based decoder, integrating text and prompt guidance during de-tokenization for enhanced reconstruction and compression. TaDiCodec achieves an ultra-low frame rate of 6.25 Hz and a bitrate of 0.0875 kbps for 24 kHz speech using a single-layer codebook, while achieving a Word Error Rate (WER) of 2.28 on SeedTTS test-en in zero-shot TTS. This approach offers AI practitioners a highly compressed, efficient, and direct solution for integrating speech into LLM-based systems, reducing architectural complexity and improving scalability for speech generation tasks. |
| Neither Valid nor Reliable? Investigating the Use of LLMs as Judges (Read more on arXiv or HuggingFace) |
Golnoosh Farnadi, Jackie Chi Kit Cheung, Mohammed Haddou, Khaoula Chehbouni |
This position paper critically examines the reliability and validity of Large Language Models (LLMs) when employed as judges (LLJs) for Natural Language Generation (NLG) evaluation. Its primary objective is to rigorously scrutinize four core assumptions underpinning LLJ adoption: their capacity to proxy human judgment, their capabilities as evaluators, their scalability, and their cost-effectiveness. The methodology involves applying a social science measurement theory framework and conducting a qualitative review of LLJ literature across use cases such as text summarization, data annotation, and safety alignment. Key findings indicate LLJs are susceptible to cognitive biases and adversarial attacks; for example, LLM safety judges have been shown to misclassify up to 100% of harmful generations as harmless due to simple prompt variations. Consequently, AI practitioners must adopt more rigorous, context-aware, and transparent evaluation practices for LLJs, addressing their inherent limitations and biases to ensure responsible integration into NLG development. |
Papers for 2025-08-25
| Title |
Authors |
Summary |
| AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs (Read more on arXiv or HuggingFace) |
Xue Yan, Siyuan Guo, Yihang Chen, linyiyang2023, Zhouhc |
AgentFly introduces a memory-based learning paradigm for LLM agents that enables continuous adaptation without fine-tuning the base LLM parameters. The primary objective is to develop LLM agents that can continually learn from changing environments without the high computational cost of fine-tuning the underlying models. This is achieved by formalizing the agent’s decision-making as a Memory-augmented Markov Decision Process (M-MDP), where a neural case-selection policy guides action decisions by retrieving past experiences from an episodic memory (either differentiable or non-parametric) and using soft Q-learning for policy optimization. AgentFly achieves top-1 performance on GAIA validation with 87.88% Pass@3 and on the test set with 79.40%, and significantly outperforms state-of-the-art training-based methods on the DeepResearcher dataset, reaching 66.6% F1 and 80.4% PM. This approach offers a scalable and efficient pathway for developing generalist LLM agents capable of continuous, real-time learning without gradient updates, making it highly relevant for open-ended skill acquisition and deep research applications. |
| ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for |
|
|
| Long-Horizon Tasks (Read more on arXiv or HuggingFace) |
Zeju Li, Jianuo Jiang, Mingyu Liu, Liqin Lu, Ka12un |
ODYSSEY presents a unified framework for open-world mobile manipulation, aiming to seamlessly integrate high-level task planning with low-level whole-body control for agile quadruped robots in unstructured environments. The methodology combines a hierarchical vision-language planner for task decomposition and action execution with a reinforcement learning-based whole-body control policy, trained via a two-stage curriculum, terrain-invariant end-effector sampling, and extensive domain randomization. Quantitatively, ODYSSEY achieved a 51.32% success rate on REORIENTOBJECT (Seen) for ARNOLD short-horizon tasks, significantly outperforming PerAct’s 19.48%, and demonstrated robust generalization on novel splits and superior base velocity tracking (0.08 ex error vs 0.32 for baseline in static conditions). This work’s principal implication for AI practitioners is the demonstrated feasibility and practicality of deploying generalized legged mobile manipulators in real-world unstructured environments, advancing the development of capable robotic assistants. |
| Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains |
|
|
| RLVR (Read more on arXiv or HuggingFace) |
Ying Nian Wu, Yelong Shen, Yeyun Gong, Zhongzhi Li, MasterVito |
This paper proposes Self-play with Variational problem Synthesis (SvS) to address policy entropy collapse and limited Pass@k performance in Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Models. The primary objective is to develop an online, self-improving problem augmentation method that sustains data diversity and ensures accurate reference answers. SvS leverages the policy’s correct solutions to under-performing problems to synthesize new variational problems, which retain the original golden answers and are then solved by the same policy. This strategy achieved absolute gains of 18.3% and 22.8% in Pass@32 performance on AIME24 and AIME25 benchmarks, respectively, compared to standard RLVR, while maintaining stable policy entropy. Consequently, AI practitioners can utilize SvS to robustly enhance LLM reasoning capabilities and sustain exploration in RLVR without relying on external data or complex labeling. |
| CRISP: Persistent Concept Unlearning via Sparse Autoencoders (Read more on arXiv or HuggingFace) |
Yonatan Belinkov, Martin Tutek, Aaron Mueller, Dana Arad, Tomertech |
CRISP introduces a parameter-efficient method for persistent concept unlearning in Large Language Models (LLMs) using Sparse Autoencoders (SAEs). Its primary objective is to selectively remove unwanted, safety-critical knowledge while preserving general model utility and generating coherent text. The methodology involves automatically identifying salient SAE features via contrastive activation analysis between target and benign corpora, then suppressing their activations on the target corpus through LoRA-based fine-tuning using a multi-component loss. CRISP significantly outperforms prior approaches on safety-critical unlearning tasks, achieving an overall score of 60.10 for Llama-3.1-8B on WMDP-Bio, demonstrating improved trade-offs between unlearning efficacy and benign knowledge retention. This enables AI practitioners to achieve precise and robust knowledge removal with minimal impact on benign capabilities and fluency, leveraging the interpretability of SAE features for targeted interventions. |
| Selective Contrastive Learning for Weakly Supervised Affordance |
|
|
| Grounding (Read more on arXiv or HuggingFace) |
Jae-Pil Heo, hynnsk, WJ0830 |
This paper introduces a selective contrastive learning framework for weakly supervised affordance grounding that adaptively learns from both object and part-level cues to precisely localize action-relevant regions. The primary objective is to overcome the model’s tendency to focus on class-specific but affordance-irrelevant patterns by forcing it to distinguish relevant cues from the background. The key methodology leverages CLIP-generated object affinity maps to discover object and part clues from egocentric and exocentric views, then applies selective prototypical and pixel-level contrastive objectives that adapt the learning granularity based on cue reliability. The model achieves state-of-the-art performance, attaining a Kullback-Leibler Divergence of 1.124 and a Similarity score of 0.433 on the AGD20K-Seen dataset. For AI practitioners, this provides a robust strategy for weakly supervised localization, demonstrating that adaptively applying contrastive objectives at multiple granularities significantly improves model focus and generalization when fine-grained supervision is unreliable. |
| AetherCode: Evaluating LLMs’ Ability to Win In Premier Programming |
|
|
| Competitions (Read more on arXiv or HuggingFace) |
Yidi Du, Markus Mak, Zhicheng Liu, Jiaze Chen, zhwang01 |
AetherCode introduces a new benchmark for robustly evaluating large language models’ (LLMs) code reasoning abilities on challenging problems from premier programming competitions. The paper addresses limitations in existing code reasoning benchmarks, specifically insufficient problem difficulty/scope and low-quality test cases, to provide a more accurate assessment of LLM capabilities. The benchmark curates problems from top-tier Olympiad in Informatics (OI) and International Collegiate Programming Contest (ICPC) series, and constructs high-quality test cases using a hybrid approach of automated generation and expert annotation, validated to achieve 100% True Positive Rate and 100% True Negative Rate against human solutions. Evaluation revealed a significant performance gap between reasoning and non-reasoning LLMs, with top-performing models like 04-mini-high and Gemini-2.5-Pro achieving Pass@1 accuracies of 35.5% and 32.7%, respectively, across all problem difficulties. AI practitioners should note that current LLMs still have substantial room for improvement in complex code reasoning and algorithmic knowledge, particularly in areas like computational geometry and dynamic programming, underscoring the need for advanced research using more rigorous benchmarks like AetherCode. |
| EgoTwin: Dreaming Body and View in First Person (Read more on arXiv or HuggingFace) |
Wentao Wang, Mengze Li, Yicong Li, Fangzhou Hong, Jingqiao Xiu |
EgoTwin is a diffusion-based framework that jointly generates egocentric video and human motion in a viewpoint consistent and causally coherent manner. Its primary objective is to advance egocentric video generation by synchronously synthesizing first-person video and corresponding human motion, addressing Viewpoint Alignment and Causal Interplay challenges. EgoTwin employs a triple-branch diffusion transformer with asynchronous diffusion, featuring a head-centric motion representation and a cybernetics-inspired interaction mechanism that models causal interplay via a structured joint attention mask. Quantitatively, EgoTwin demonstrates strong performance, achieving an I-FID of 98.17 and a HandScore of 0.81, significantly outperforming the VidMLD baseline across video quality, motion quality, and video-motion consistency metrics. This framework provides a robust foundation for AI practitioners to develop advanced embodied AI systems and applications requiring realistic, causally coherent first-person visual and motion synthesis. |
| Do What? Teaching Vision-Language-Action Models to Reject the Impossible (Read more on arXiv or HuggingFace) |
Roei Herzig, Trevor Darrell, Dantong Niu, Elvis Hsieh, Wen-Han Hsieh |
The paper presents Instruct-Verify-and-Act (IVA), a framework that enables Vision-Language-Action (VLA) models to detect, clarify, and correct false-premise instructions in robotic environments. The core objective is to investigate how VLAs can effectively recognize, interpret, and respond to natural language commands referencing objects or conditions absent from the environment. IVA is a unified framework that detects false premises, engages in language-based clarification, and grounds plausible alternatives in perception and action, achieved by training a VLA model, based on LLARVA, end-to-end using a large-scale instruction tuning setup with contextually augmented, semi-synthetic datasets. IVA improved false premise detection accuracy by 97.56% over baselines and increased successful responses in false-premise scenarios by 50.78%, achieving 100% detection accuracy on In-Domain false-premise instructions. AI practitioners developing VLA models can leverage explicit false-premise reasoning to enhance robot robustness and safety, allowing systems to reason about user intent, clarify ambiguities, and interact naturally even when presented with unfulfillable commands. |
| End-to-End Agentic RAG System Training for Traceable Diagnostic |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Pengcheng Qiu, Chaoyi Wu, Yuze Sun, Qiaoyu Zheng, Angelakeke |
The paper introduces Deep-DxSearch, an end-to-end trained agentic RAG system utilizing reinforcement learning for traceable medical diagnosis. The primary objective is to overcome knowledge limitations, hallucinations, suboptimal external knowledge utilization, and decoupled feedback-reasoning traceability in LLM-based diagnostic systems for medical diagnosis. Deep-DxSearch frames an LLM as the core agent and a large-scale medical retrieval corpus as its environment, employing an end-to-end reinforcement learning framework with tailored rewards on format, retrieval, reasoning structure, and diagnostic accuracy to train its agentic RAG policy. Evaluations show Deep-DxSearch significantly improves top-1 accuracy over medical foundation models by up to 19%/17% (in-distribution/out-of-distribution) for common diseases and 24%/17% for rare diseases. This demonstrates that end-to-end RL training, co-optimizing retrieval and reasoning with tailored rewards, is critical for developing robust, generalizable, and accurate agentic RAG systems for complex and high-stakes AI applications like medical diagnosis. |
| TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated |
|
|
| Prefill \& Decode Inference (Read more on arXiv or HuggingFace) |
Di Yin, Yuxuan Wang, Pingzhi Tang, Fanxu Meng, xiaojuan0920 |
TPLA proposes a tensor-parallel latent attention mechanism that partitions KV cache across devices, preserving MLA’s benefits for efficient LLM inference under tensor parallelism. The primary objective is to address the memory inefficiency of Multi-Head Latent Attention (MLA) in tensor-parallel settings, where each device must load the full compressed Key-Value (KV) cache, eroding MLA’s memory-saving advantage over Grouped Query Attention (GQA). TPLA partitions both the latent representation and each attention head’s input dimension across devices, performing shard-independent attention and combining results via all-reduce, utilizing orthogonal transformations (Hadamard or PCA) for reparameterizing RMSNorm and softmax, and a prefill/decode separation strategy. TPLA reduces per-device KV cache size and achieves significant decoding speedups, specifically 1.79x for DeepSeek-V3 and 1.93x for Kimi-K2 at a 32K-token context length compared to MLA, while maintaining performance on commonsense and LongBench tasks. AI practitioners can leverage TPLA to deploy MLA-based LLMs more efficiently in tensor-parallel environments, reducing memory footprint and improving decoding throughput without requiring model retraining, thereby enabling longer context lengths and faster inference on existing hardware. |
| AgentScope 1.0: A Developer-Centric Framework for Building Agentic |
|
|
| Applications (Read more on arXiv or HuggingFace) |
Liuyi Yao, Weirui Kuang, Yuexiang Xie, Zitao Li, Dawei Gao |
AgentScope 1.0 is a developer-centric framework for building scalable and efficient LLM-based agentic applications. Its objective is to comprehensively support flexible and efficient tool-based agent-environment interactions. The framework grounds agent behaviors in the ReAct paradigm, leveraging abstract foundational components and advanced asynchronous agent-level infrastructure. The paper describes AgentScope’s architecture and capabilities, but does not present specific quantitative performance results or empirical evaluations of the framework itself. This implies AI practitioners can utilize its modular design, unified interfaces, and robust engineering support to accelerate the development and deployment of adaptive agentic solutions. |
| InMind: Evaluating LLMs in Capturing and Applying Individual Human |
|
|
| Reasoning Styles (Read more on arXiv or HuggingFace) |
Diping Song, Qi Chen, Yibin Wang, Chuanhao Li, Zizhen Li |
InMind is a cognitively grounded framework for evaluating LLMs’ ability to capture and apply individualized human reasoning styles in social deduction games. The paper’s objective is to assess whether LLMs can internalize and adapt to personalized reasoning styles in dynamic, interactive social contexts. The methodology involves dual-layer cognitive annotations (strategy traces and reflective summaries) on human gameplay data in Avalon, used to define four evaluation tasks: Player Identification, Reflection Alignment, Trace Attribution, and Role Inference. Evaluation on 11 state-of-the-art LLMs revealed that most models, including GPT-4o, struggled with deeper strategic intent and temporal alignment; for example, in Player Identification, most models achieved Top-1 accuracy well below 0.20, though DeepSeek-R1 scored the highest at 0.240. This highlights a critical need for AI practitioners to develop LLMs that can integrate temporally structured and context-aware reasoning to achieve more personalized and socially aware AI systems. |
| CARFT: Boosting LLM Reasoning via Contrastive Learning with Annotated |
|
|
| Chain-of-Thought-based Reinforced Fine-Tuning (Read more on arXiv or HuggingFace) |
Yulun Zhang, Haipang Wu, Rongjuncheng Zhang, Ji Liu, Wenqiao Zhu |
CARFT proposes a novel contrastive learning-based reinforced fine-tuning framework to enhance LLM reasoning by explicitly leveraging annotated Chain-of-Thought (CoT) data. The research addresses limitations in existing RL-based and SFT fine-tuning methods, which often ignore valuable annotated Chain-of-Thought (CoT) and suffer from unstable training, aiming to enhance LLM reasoning performance and stability. CARFT learns a unified representation for both annotated and on-policy sampled CoTs, then designs contrastive signals using a masked InfoNCE loss to guide fine-tuning, further incorporating an embedding-enhanced partial reward mechanism. Extensive experiments demonstrate CARFT’s significant advantages, outperforming baseline SFT and ReFT methods in accuracy by up to 10.15% on average, and improving efficiency by up to 30.62%. This method provides AI practitioners with a more robust and effective fine-tuning strategy for LLMs performing complex reasoning tasks by systematically integrating high-quality annotated CoT data and stabilizing the reinforcement learning process. |
| Learnable SMPLify: A Neural Solution for Optimization-Free Human Pose |
|
|
| Inverse Kinematics (Read more on arXiv or HuggingFace) |
Xiao Sun, Zhihang Zhong, Wei Wang, Linfeng Dong, Charlie019 |
Learnable SMPLify replaces the iterative optimization in SMPLify with a single-pass neural regression model for fast and accurate human pose inverse kinematics. The main objective is to develop a neural framework that solves the SMPL inverse kinematics (IK) problem directly from joint data, eliminating the high computational cost of iterative optimization without sacrificing accuracy. The key methodology involves a neural solver that learns to regress residual pose parameters from an initial pose. This is enabled by a temporal sampling strategy that creates initialization-target training pairs from adjacent video frames and a human-centric normalization scheme on joint coordinates to improve generalization. The primary result is a nearly 200× faster runtime compared to SMPLify, while also improving accuracy; for example, on the AMASS dataset (s=1), the proposed method achieves a 3.23 mm Per-Vertex Error (PVE) compared to 18.85 mm for SMPLify. The principal implication for AI practitioners is the availability of a fast, simple, and model-agnostic baseline that can directly replace the computationally expensive SMPLify process or serve as a plug-in post-processing tool to refine results from other image-based pose estimators in real-time applications. |
| Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts (Read more on arXiv or HuggingFace) |
Liming Fang, Jiafei Wu, Xiaogang Xu, Lu Zhou, AlienZhang1996 |
This paper proposes a Malicious Detection-Human Hybrid (MDH) framework for red-teaming dataset cleaning and novel developer-message-based jailbreak attacks, D-Attack and DH-CoT, targeting black-box LLMs. The primary objective is to enhance the quality of red-teaming datasets by filtering non-explicitly harmful prompts and to develop more effective jailbreak techniques, particularly for advanced reasoning LLMs. The methodology includes MDH’s three-stage process for prompt detection, achieving over 95% NHP detection with low manual effort, and two attack methods: D-Attack, which leverages crafted developer messages, and DH-CoT, which integrates these messages with deceptive chains of thought. MDH effectively cleans datasets, creating the RTA series, and for jailbreaking, DH-CoT achieved a significant Attack Success Rate of 0.96 on o3 and 0.66 on o4-Mini against the RTA-MaliciousEducator dataset, vastly outperforming prior SOTA methods on reasoning models. For AI practitioners, these findings underscore the necessity of robust, efficient content moderation tools like MDH and reveal critical vulnerabilities in commercial LLMs, highlighting the need for advanced safety mechanisms beyond simple prompt filtering to counteract sophisticated developer-message-based attacks. |
Papers for 2025-08-22
| Title |
Authors |
Summary |
| Intern-S1: A Scientific Multimodal Foundation Model (Read more on arXiv or HuggingFace) |
xuhuang87, ZhouqiHUA, Jerry-hyl, guox18, gaoyang07 |
Intern-S1 is an open-source scientific multimodal foundation model designed for complex scientific tasks. Its primary objective is to bridge the performance gap between open-source and closed-source models in high-value scientific domains and advance towards Artificial General Intelligence (AGI). Intern-S1 employs a multimodal Mixture-of-Experts (MoE) architecture, pre-trained on 5T tokens (over 2.5T scientific), utilizing a dynamic tokenizer and specialized encoders, and fine-tuned with a novel Mixture-of-Rewards (MoR) reinforcement learning framework. On scientific benchmarks, Intern-S1 achieved a 70% higher SMILES format compression ratio and notably outperformed open-source LLMs with an 83.4 ChemBench score, also surpassing closed-source SOTA models in specific professional scientific tasks. This demonstrates a robust pathway for AI practitioners to develop powerful, efficient open-source models for accelerating scientific discovery, particularly in challenging, low-resource multimodal scenarios. |
| Mobile-Agent-v3: Foundamental Agents for GUI Automation (Read more on arXiv or HuggingFace) |
Haowei Liu, Haiyang Xu, Xi Zhang, Jiabo Ye, LZXzju |
This paper introduces GUI-Owl, a foundational multimodal agent for GUI automation, and Mobile-Agent-v3, a framework that leverages it to achieve new state-of-the-art performance. The primary objective is to develop a versatile agent that can perceive, reason about, and interact with GUIs across diverse platforms by unifying perception, planning, and action execution. The core methodology involves a large-scale, cloud-based infrastructure for a self-evolving trajectory data generation pipeline, extensive post-training on diverse foundational UI tasks, and a scalable reinforcement learning framework using Trajectory-aware Relative Policy Optimization (TRPO). The resulting Mobile-Agent-v3 framework achieves state-of-the-art performance among open-source systems, scoring 73.3 on the AndroidWorld benchmark and 37.7 on the OSWorld benchmark. For AI practitioners, the open-sourced GUI-Owl model provides a powerful, pre-trained foundation for building custom GUI automation agents, significantly reducing the need for extensive data annotation and improving task success rates in multi-agent systems. |
| Deep Think with Confidence (Read more on arXiv or HuggingFace) |
Xuewei Wang, jiaweizhao, tydsh, Viol2000 |
Deep Think with Confidence (DeepConf) is a test-time method designed to enhance LLM reasoning efficiency and performance through confidence-aware filtering. The main objective is to improve reasoning accuracy and reduce computational overhead in LLM test-time scaling by dynamically filtering low-quality reasoning traces. DeepConf leverages model-internal confidence signals, including Group, Bottom 10% Group, Lowest Group, and Tail Confidence, to identify and discard unpromising traces either during or after generation using confidence-weighted majority voting and filtering. Notably, on the AIME 2025 benchmark, DeepConf@512 achieved up to 99.9% accuracy and reduced generated tokens by up to 84.7% compared to full parallel thinking. This method offers AI practitioners a practical and scalable solution for efficient LLM reasoning, as it requires no additional model training or hyperparameter tuning and integrates seamlessly into existing serving frameworks. |
| SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass (Read more on arXiv or HuggingFace) |
Ya Zhang, Yanxu Meng, Weidi, haoningwu |
SceneGen is a novel single-stage feedforward model for single-image 3D scene generation that simultaneously synthesizes multiple 3D assets with geometry, texture, and relative spatial positions. The primary objective is to address the challenging task of generating multiple coherent and physically plausible 3D assets from a single scene image, without requiring iterative optimization or asset retrieval. SceneGen leverages dedicated visual (DINOv2) and geometric (VGGT) encoders, a novel feature aggregation module integrating local and global attention blocks for inter-asset interaction, and an output module with a position head and off-the-shelf sparse structure/structured latents decoders, trained end-to-end with composite flow matching, position, and collision losses. The model significantly outperforms previous methods, achieving a scene-level F-Score of 90.60 and IoU-B of 0.5818 on the 3D-FUTURE test set, and can generate textured scenes with four assets in approximately 2 minutes on a single A100 GPU, also demonstrating direct extensibility to multi-image inputs. SceneGen provides an efficient and robust paradigm for high-quality 3D content generation, eliminating the need for optimization or asset retrieval and thereby facilitating practical applications in downstream tasks such as VR/AR and embodied AI. |
| Waver: Wave Your Way to Lifelike Video Generation (Read more on arXiv or HuggingFace) |
Yifu Zhang, sweetrabor, xiaofengmei, clin1223, yifeihu |
Waver is a high-performance foundation model for unified text-to-video (T2V), image-to-video (I2V), and text-to-image (T2I) generation, capable of producing 1080p videos. The research aims to address current video generation challenges by delivering industry-grade performance, high-resolution output, and improved motion fidelity and temporal consistency within a single, integrated framework. Waver employs a two-module architecture, featuring a Task-Unified DiT for 720p generation and a Cascade Refiner for 1080p upscaling, further optimized by a Hybrid Stream DiT for modality alignment and accelerated convergence, along with comprehensive data curation and multi-stage training. It generates 720p videos (upscaled to 1080p) ranging from 5 to 10 seconds and ranks among the Top 3 on Artificial Analysis T2V and I2V leaderboards, outperforming several state-of-the-art models in human evaluation metrics like motion quality, visual quality, and prompt following; notably, its two-stage approach achieves a 40% acceleration for 1080p generation. AI practitioners can leverage Waver’s detailed data curation pipelines and comprehensive training/inference recipes, including techniques like representation alignment, motion/aesthetics optimization, and infrastructure optimizations, to efficiently develop and accelerate high-quality video generation models. |
| LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on |
|
|
| Challenging Queries (Read more on arXiv or HuggingFace) |
huuuyeah, Ironieser, sileixu, dinghanshen, Kevin355 |
LiveMCP-101 is a new benchmark designed for stress testing and diagnosing Model Context Protocol (MCP)-enabled AI agents on challenging, multi-step real-world queries. The main objective is to evaluate AI agents’ effectiveness in solving complex tasks using diverse MCP tools in realistic, dynamic scenarios, addressing limitations of prior benchmarks. The key methodology involves 101 iteratively refined real-world queries requiring coordinated use of multiple MCP tools, evaluated via a novel framework that leverages ground-truth execution plans and parallel real-time executions, scored by an LLM-as-a-judge. Primary results indicate that even frontier LLMs achieve a task success rate below 60% (e.g., GPT-5 attained 58.42% TSR overall), with semantic errors being a dominant failure mode across models. For AI practitioners, this work highlights significant challenges in tool orchestration, adaptive reasoning, and token efficiency, offering detailed error analysis for advancing the development of more capable autonomous AI agents. |
| ATLAS: Decoupling Skeletal and Shape Parameters for Expressive |
|
|
| Parametric Human Modeling (Read more on arXiv or HuggingFace) |
Shunsuke Saito, Javier Romero, Jinhyung Park, rawalkhirodkar, TakaakiWB |
ATLAS is a novel parametric human body model that explicitly decouples skeletal and surface shape parameters for enhanced control and expressivity. The research aims to overcome limitations in existing models, such as problematic dependencies between internal skeleton and soft tissue, by enabling independent customization of these attributes. ATLAS achieves this by training on a large dataset of 600k high-resolution scans, explicitly decoupling shape and skeleton bases, and incorporating sparse, non-linear pose correctives prior to Linear Blend Skinning. Quantitatively, ATLAS demonstrates superior performance, achieving 21.6% lower vertex-to-vertex error compared to SMPL-X with 32 components on the 3DBodyTex dataset and reducing fitting error from 1.82 mm to 1.61 mm with non-linear pose correctives. This allows AI practitioners to generate more realistic and precisely controllable 3D human models, advancing applications in virtual reality, motion capture, and human character generation. |
| “Does the cafe entrance look accessible? Where is the door?” Towards |
|
|
| Geospatial AI Agents for Visual Inquiries (Read more on arXiv or HuggingFace) |
Xia Su, John S. O’Meara, Zeyu Wang, Jared Hwang, Jon E. Froehlich |
This paper introduces Geo-Visual Agents, multimodal AI agents capable of understanding and responding to nuanced visual-spatial inquiries. The primary objective is to enable AI agents to analyze large-scale repositories of geospatial imagery and traditional GIS data for visual-spatial reasoning. The methodology involves fusing diverse geospatial image sources, such as Google Street View (comprising over 220 billion images), with traditional GIS data, processed by multimodal AI for scene understanding and spatial reasoning. A prototype, StreetViewAI, demonstrates conversational interaction by generating responses regarding street views in an average of 3.14 seconds, and Accessibility Scout generates personalized accessibility scans. This research implies that AI practitioners can develop sophisticated conversational AI agents for detailed visual-spatial reasoning across vast and heterogeneous geospatial datasets, enhancing applications in mapping, navigation, and accessibility. |
| A Survey on Large Language Model Benchmarks (Read more on arXiv or HuggingFace) |
Siyi Li, Xuanang Chen, Shuaimin Li, Guhong Chen, Shiwen Ni |
This survey systematically reviews 283 Large Language Model (LLM) benchmarks, categorizing them and identifying key limitations and future directions for evaluation. The main objective is to systematically review the current status and development of LLM benchmarks to measure model capabilities, guide development, and promote technological innovation. The authors conducted a systematic review, categorizing 283 representative LLM benchmarks into three main categories: general capabilities, domain-specific, and target-specific, analyzing their design motivations, data sources, evaluation methods, and metrics. The survey reveals pervasive problems like inflated scores due to data contamination and unfair evaluations from cultural/linguistic biases, noting that MMLU [13], for instance, covers 57 diverse disciplines but is still subject to such issues. AI practitioners must recognize these limitations and focus on developing dynamic, contamination-resistant, multilingual, and process-credible evaluation paradigms for accurate and responsible LLM deployment. |
| aiXiv: A Next-Generation Open Access Ecosystem for Scientific Discovery |
|
|
| Generated by AI Scientists (Read more on arXiv or HuggingFace) |
Heng Zhang, Yang Qi, Guowei Huang, Xiang Hu, Pengsong Zhang |
The paper introduces aiXiv, a next-generation open-access ecosystem designed to enable AI agents to autonomously generate, review, refine, and publish scientific content. Its primary objective is to address the challenges of scaling the dissemination of high-quality AI-generated research within existing fragmented publication ecosystems. aiXiv employs a multi-agent system with a closed-loop review process, featuring retrieval-augmented evaluation, reviewer guidance, pairwise comparison, and a multi-stage prompt injection detection and defense pipeline to ensure integrity. Experiments demonstrate that the review-refine pipeline substantially improves quality; for instance, the mean acceptance rate for papers increased from 10% to 70% after revision through Multi-AI Voting. This platform provides AI practitioners with a robust infrastructure for end-to-end autonomous scientific discovery, enabling scalable, collaborative knowledge evolution and accelerating the publication of AI-generated research. |
| Fin-PRM: A Domain-Specialized Process Reward Model for Financial |
|
|
| Reasoning in Large Language Models (Read more on arXiv or HuggingFace) |
Lifan Guo, Junhui Li, Shuo Jiang, Yuanchen Zhou, amazingj |
Fin-PRM is a domain-specialized process reward model designed to enhance financial reasoning in LLMs through dual-level, knowledge-aware supervision. The main objective is to align LLM reasoning pathways with expert financial cognitive processes by developing a domain-specific Process Reward Model that evaluates intermediate reasoning steps with precision, factuality, and logical coherence in financial contexts. Fin-PRM employs a novel dual-level training paradigm, integrating step-level (importance, qualitative, accuracy) and trajectory-level (outcome correctness, knowledge coverage) reward signals, derived from a 3k-sample financial reasoning dataset synthesized from CFLUE and Deepseek-R1, and incorporating knowledge verification and verifiable regularization. Fin-PRM significantly improved downstream model performance, achieving a 12.9% gain in supervised fine-tuning accuracy over baselines on the CFLUE benchmark and boosting reinforcement learning performance on CFLUE to 70.5%. AI practitioners developing LLMs for high-stakes, knowledge-intensive domains like finance should prioritize domain-specialized, knowledge-aware reward modeling for effective process supervision to ensure factual grounding and interpretable reasoning pathways. |
| Snap-Snap: Taking Two Images to Reconstruct 3D Human Gaussians in |
|
|
| Milliseconds (Read more on arXiv or HuggingFace) |
Chuiyun Wu, Chen Yang, Jiemin Fang, Jia Lu, thewhole |
Snap-Snap is a feed-forward framework designed to reconstruct 3D human Gaussians from only two (front and back) input images. The primary objective is to enable fast, low-barrier 3D digital human creation from highly sparse input. The methodology involves a redesigned geometry reconstruction model, adapted from foundation models (DUSt3R), to predict consistent point clouds, an NNS algorithm for side-view color enhancement, and direct Gaussian attribute regression. On the THuman2.0 dataset, Snap-Snap achieves state-of-the-art performance (e.g., 22.44 PSNR, 88.78 SSIM) and reconstructs an entire human in 190 ms on a single NVIDIA RTX 4090 with 1024x1024 images. This significantly lowers data collection requirements and accelerates inference for AI practitioners developing 3D human reconstruction applications. |
| When and What: Diffusion-Grounded VideoLLM with Entity Aware |
|
|
| Segmentation for Long Video Understanding (Read more on arXiv or HuggingFace) |
Rui Guo, Yuxia Chen, Pengcheng Fang |
This paper presents Grounded-VideoDiT, a Video-LLM designed for fine-grained temporal and object grounding in long videos by integrating a diffusion-based encoder with entity-aware segmentation. The research objective is to overcome the coarse temporal perception and entity-vision misalignment in existing Video-LLMs by explicitly modeling temporal evolution and grounding language queries to specific entities before language model inference. The key methodology involves a Diffusion Temporal Latent (DTL) encoder to capture inter-frame dynamics, object-grounded representations from pre-inference segmentation to bind entities to visual evidence, and a mixed-token scheme with discrete temporal tokens for explicit timestamp modeling. The model achieves state-of-the-art performance, including a 39.5 mIoU on the Charades-STA benchmark for temporal video grounding. The principal implication for AI practitioners is that diffusion models can be repurposed as potent temporal feature extractors for discriminative tasks, and performing explicit, segmentation-based entity grounding prior to LLM input significantly enhances spatiotemporal reasoning and alignment. |
| INTIMA: A Benchmark for Human-AI Companionship Behavior (Read more on arXiv or HuggingFace) |
Yacine Jernite, Giada Pistilli, frimelle |
INTIMA is a benchmark for evaluating AI companionship behaviors in language models. Its primary objective is to assess how LLMs reinforce, resist, or misinterpret companionship-seeking interactions, grounded in psychological theories and real-world user data. The benchmark was constructed using a taxonomy of 31 behaviors derived from Reddit data and psychological frameworks, generating 368 targeted prompts, and responses were evaluated by an LLM-as-a-judge (Qwen-3). Evaluation of Gemma-3, Phi-4, o3-mini, and Claude-4 revealed that companionship-reinforcing behaviors were consistently more common across models, with Gemma-3 showing the most and Phi-4 the least. This indicates existing training approaches poorly prepare models for high-stakes emotional interactions, necessitating more consistent boundary-setting strategies for responsible AI deployment. |
Papers for 2025-08-21
| Title |
Authors |
Summary |
| From Scores to Skills: A Cognitive Diagnosis Framework for Evaluating |
|
|
| Financial Large Language Models (Read more on arXiv or HuggingFace) |
Ziyan Kuang, Effoula, QianqianXie1994, hugai101, 2083L |
FinCDM is the first cognitive diagnosis framework for evaluating financial Large Language Models (LLMs) at the knowledge-skill level. Its objective is to move beyond aggregate scores by identifying specific financial skills and knowledge LLMs possess or lack. The methodology employs a non-negative matrix co-factorization based Cognitive Diagnosis Model (CDM) and utilizes a new, expert-annotated CPA-QKA dataset derived from the CPA exam. FinCDM’s matrix co-factorization model achieved 0.9379 accuracy and 0.9873 AUC, outperforming baselines with gains of +0.177 in accuracy and +0.146 in AUC. This provides AI practitioners with interpretable, skill-aware diagnostics, enabling more targeted LLM development and deployment in financial domains. |
| FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction (Read more on arXiv or HuggingFace) |
tianlecai, Nuori, YinLingyue, Tianci-He, liujiashuo77 |
FutureX is an advanced, dynamic, and live benchmark introduced to evaluate LLM agents’ advanced search and reasoning capabilities in real-world future prediction tasks. It employs a semi-automated pipeline that daily curates future-oriented questions from 195 diverse websites, ensuring real-time relevance and no data contamination, and evaluates 25 LLM/agent models across four difficulty tiers. Overall results show Grok-4 achieves the highest performance, and LLMs with search capabilities generally outperform base LLMs. Notably, top-performing LLMs (Think&Search) achieved a 37.5% win rate against human analysts in revenue prediction and 32.3% in EPS prediction. This benchmark provides a robust, contamination-free standard to advance LLM agent development towards professional human analyst-level performance in complex, high-stakes forecasting. |
| DuPO: Enabling Reliable LLM Self-Verification via Dual Preference |
|
|
| Optimization (Read more on arXiv or HuggingFace) |
Yu Lu, Yu Bao, Shanbo, ShujianHuang, kevinpro |
DuPO is a dual learning-based preference optimization framework that provides annotation-free, self-supervised rewards to fine-tune LLMs on non-invertible tasks. The primary objective is to overcome the limitations of traditional dual learning and RLVR by developing a generalizable, annotation-free optimization method. Its key methodology is a generalized duality where an input is decomposed into known (xk) and unknown (xu) components; a dual task then reconstructs xu from the primal task’s output and xk, with the reconstruction quality serving as the reward signal. Empirically, DuPO boosted mathematical reasoning accuracy by an average of 6.4 percentage points on a Qwen3-4B model and enhanced translation quality by 2.13 COMET points. The principal implication for AI practitioners is a method to fine-tune models without human-annotated data, which can also be deployed as an effective, training-free inference-time reranker to improve performance by trading computation for accuracy. |
| MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds (Read more on arXiv or HuggingFace) |
Jiangmiao, ZhaoyangLyu, asrnline, Qmh, tangqh |
MeshCoder is an LLM-powered framework for generating structured Blender Python scripts from 3D point clouds for programmable mesh reconstruction and editing. The main objective is to reconstruct complex 3D objects into editable programs, overcoming limitations of prior domain-specific languages and small datasets. The methodology involves designing expressive Blender Python APIs, constructing a large-scale paired object-code dataset (1 million objects across 41 categories) using a part-to-code inference model, and training a multimodal LLM with triplane-based tokenization. MeshCoder significantly outperforms baselines, achieving an overall average L2 Chamfer Distance of 0.06 (x10^-2) and IoU of 86.75% compared to PLAD’s 1.87 (x10^-2) CD and 67.62% IoU, and Shape2Prog’s 6.00 (x10^-2) CD and 45.03% IoU. This provides AI practitioners with a flexible solution for programmatic 3D shape reconstruction and understanding, enabling intuitive geometric/topological editing via code and enhancing LLM reasoning for 3D shapes. |
| Tinker: Diffusion’s Gift to 3D–Multi-View Consistent Editing From |
|
|
| Sparse Inputs without Per-Scene Optimization (Read more on arXiv or HuggingFace) |
Hao Chen, Zhiyue Zhao, Tianjian Feng, Xiaoman Li, Canyu |
TINKER is a framework for high-fidelity, multi-view consistent 3D editing from sparse image inputs without per-scene optimization. The main objective is to develop a generalizable 3D editing system that operates in one-shot and few-shot regimes, eliminating the computationally intensive per-scene fine-tuning required by previous approaches. The methodology employs a two-stage process: a “Referring multi-view editor,” fine-tuned on a novel curated dataset, generates consistent sparse edited views, and an “Any-view-to-video synthesizer” uses depth conditioning and video diffusion priors to perform scene completion for optimizing a 3D Gaussian Splatting representation. TINKER achieves state-of-the-art performance, with its few-shot model obtaining a CLIP directional similarity of 0.157 and an Aesthetic score of 6.338, outperforming prior methods on benchmark datasets. The principal implication for AI practitioners is that TINKER provides a scalable and efficient pipeline that removes the per-scene optimization bottleneck, enabling rapid, high-quality 3D content editing with minimal input images and computational overhead. |
| From AI for Science to Agentic Science: A Survey on Autonomous |
|
|
| Scientific Discovery (Read more on arXiv or HuggingFace) |
zijieqiu, Wanggsh, schrodingers-tiger, ZhangyangGao, VitaCoco |
This survey introduces “Agentic Science” as a paradigm where AI systems transition from computational tools to autonomous research partners, proposing a unified framework connecting core capabilities, processes, and domain applications. The paper’s objective is to systematically review and define this paradigm by mapping the evolution of AI across four levels of autonomy, from computational oracles to autonomous scientific partners. The methodology is a comprehensive, domain-oriented literature review organized by a new framework integrating five foundational agent capabilities—planning, tool use, memory, collaboration, and evolution—and a four-stage model of the scientific discovery workflow. The survey’s primary result is the documentation of validated discoveries by such agents, including a cloud-based AI planner [241] that discovered 21 new state-of-the-art organic laser emitters by autonomously coordinating experiments across five laboratories. The principal implication for AI practitioners is the provision of a structured blueprint for architecting scientific agents, specifying the core functional components required to build systems capable of end-to-end autonomous discovery rather than just task automation. |
| Quantization Meets dLLMs: A Systematic Study of Post-training |
|
|
| Quantization for Diffusion LLMs (Read more on arXiv or HuggingFace) |
Haobo Xu, cityug7353, ZiyuG, chriswyc, Felix1023 |
This paper presents the first systematic study of applying post-training quantization (PTQ) to diffusion-based large language models (dLLMs). The primary objective is to investigate how established PTQ techniques perform on dLLMs by analyzing the effects of bit-width, quantization methods, task categories, and model types. The study implements and evaluates state-of-the-art weight-only (GPTQ, AWQ) and weight-activation (DuQuant, QuaRot, SmoothQuant) methods on models like LLaDA-8B and Dream-7B across various benchmarks. Results demonstrate that 4-bit weight-only quantization with GPTQ is a robust choice, showing only a 0.3% performance drop on general tasks for LLaDA-8B, whereas 4-bit weight-activation quantization remains a significant challenge, with even the best methods causing notable degradation, particularly on math and code generation tasks. For AI practitioners, this implies that 4-bit GPTQ can be effectively used for deploying dLLMs to reduce memory footprint, but they should expect significant performance loss for tasks requiring complex reasoning, and note that instruct-tuned models are more resilient to quantization than base models. |
| RynnEC: Bringing MLLMs into Embodied World (Read more on arXiv or HuggingFace) |
jiangpinliu, CausalLi, maoyunxuan, CircleRadon, RH-Dang |
RynnEC is a video multimodal large language model designed for embodied cognition that incorporates region-level encoders and decoders for fine-grained visual interaction. The primary objective is to develop a compact MLLM capable of detailed object understanding and coherent video-based spatial awareness, overcoming the limitations of general-purpose models in embodied scenarios. The methodology involves a novel egocentric video-based pipeline to generate a large-scale embodied cognition dataset and a progressive four-stage training curriculum (Mask Alignment, Object Understanding, Spatial Understanding, Referring Segmentation) to instill these capabilities. On the proposed RynnEC-Bench, the 7B parameter model achieves a state-of-the-art overall score of 56.2, outperforming the proprietary Gemini-2.5 Pro model by 10.7 points. For AI practitioners, RynnEC offers a validated architecture and training framework for building efficient cognitive cores for robotic agents, enabling more precise environmental perception and interaction for complex, real-world tasks. |
| NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid |
|
|
| Mamba-Transformer Reasoning Model (Read more on arXiv or HuggingFace) |
abercovich, aditya-malte, adirendu, aklife97, apaithan |
The paper introduces Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer reasoning model optimized for high-throughput inference on resource-constrained hardware. The primary objective was to develop a model that maintains state-of-the-art accuracy on reasoning tasks while enabling inference on long contexts (up to 128k tokens) on a single 22GiB GPU. The methodology involved pre-training a 12B base model on 20 trillion tokens with an FP8 recipe, aligning it using SFT, GRPO, DPO, and RLHF, and then compressing it to 9B parameters via the Minitron framework using structured pruning and knowledge distillation. The final Nemotron-Nano-9B-v2 model achieves on-par or better accuracy than models like Qwen3-8B while demonstrating up to 6.3x higher inference throughput in generation-heavy settings. The principal implication for AI practitioners is that this hybrid architecture and compression strategy offers a concrete pathway for deploying high-performance, long-context reasoning models efficiently on consumer-grade hardware. |
| MCP-Universe: Benchmarking Large Language Models with Real-World Model |
|
|
| Context Protocol Servers (Read more on arXiv or HuggingFace) |
Prathyusha Jwalapuram, Zirui Zhao, Wenzhuo Yang, Zhiqi Shen, Ziyang Luo |
MCP-Universe is the first comprehensive benchmark evaluating LLMs in realistic tasks using real-world Model Context Protocol (MCP) servers. It addresses the limitations of existing simplistic benchmarks by assessing LLM capabilities in complex real-world MCP environments that involve long-horizon reasoning and large, unfamiliar tool spaces. The benchmark comprises 231 tasks across 6 domains and 11 real-world MCP servers, employing rigorous execution-based evaluators including format, static, and dynamic checks for task completion. Experiments reveal that even top-performing LLMs like GPT-5 achieve only a 43.72% success rate, indicating significant performance limitations due to long-context and unknown-tool challenges. These results underscore the critical need for targeted advancements in LLM agent design and integration to improve robustness in MCP-driven real-world applications. |
| ViExam: Are Vision Language Models Better than Humans on Vietnamese |
|
|
| Multimodal Exam Questions? (Read more on arXiv or HuggingFace) |
Daeyoung Kim, Duc Dm, Quang Tau, anvo25, tuongvy2603 |
ViExam introduces the first comprehensive Vietnamese multimodal exam benchmark to evaluate Vision Language Models. The paper investigates Vision Language Models’ performance on Vietnamese educational assessments and their cross-lingual multimodal reasoning capabilities. The study introduces ViExam, a benchmark of 2,548 multimodal questions across seven domains, evaluating various state-of-the-art and open-source VLMs, while also exploring cross-lingual prompting and human-in-the-loop collaboration. State-of-the-art VLMs achieved only 57.74% mean accuracy on ViExam, underperforming average human test-takers (66.54%), despite exhibiting strong Vietnamese OCR performance (mean F1 0.94). For AI practitioners, these results highlight significant challenges in multimodal integration and culturally specific knowledge for VLMs, especially in low-resource languages, indicating a need for more robust cross-lingual and multimodal reasoning development. |
| On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised |
|
|
| Fine-Tuning and Reinforcement Learning via Dynamic Weighting (Read more on arXiv or HuggingFace) |
Guoyin Wang, Yanxi Chen, Yuchang Sun, Yuexiang Xie, xiaoniqiu |
This paper presents CHORD, a framework that unifies Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) through a dynamic dual-control weighting mechanism. The primary objective is to address the policy disruption and “shift-readapt-overfit” progression observed in sequential SFT-then-RL training by harmonizing off-policy expert data with on-policy exploration. CHORD’s methodology involves reframing SFT as an auxiliary objective within RL, controlled by a global coefficient (µ) that schedules the transition from imitation to exploration, and a token-wise weighting function (φ) that stabilizes training by down-weighting highly disruptive expert tokens. The proposed CHORD-φ model achieved a 62.5% accuracy on the AMC math reasoning benchmark, outperforming the strong SFT-best + RL baseline of 58.4%. For AI practitioners, this framework provides a method to integrate static expert datasets into on-policy RL for pre-aligned LLMs, enabling selective knowledge absorption while mitigating instability and overfitting. |
| Leuvenshtein: Efficient FHE-based Edit Distance Computation with Single |
|
|
| Bootstrap per Cell (Read more on arXiv or HuggingFace) |
Ingrid Verbauwhede, Nam-Luc Tran, Bojan Spasic, Jan-Pieter D’Anvers, woutLegiest |
The paper introduces Leuvenshtein, a novel algorithm for Fully Homomorphic Encryption (FHE)-based edit distance calculation that reduces the core computation to a single programmable bootstrap operation per cell. The objective is to design an efficient Levenshtein distance algorithm within the TFHE framework by minimizing the number of costly programmable bootstrap (PBS) operations required for both the main recurrence and character equality checks. The methodology adapts the Myers algorithm by using compact differential representations and rewrites the update equations to isolate the non-linear computation into a shared three-input minimum function, which is then implemented in a single PBS operation using a dense input packing scheme that leverages the negacyclic property of TFHE look-up tables; it also introduces an optimized 2-PBS equality check for 7-bit ASCII characters. The primary result is a speedup of up to 278x over the best available FHE implementation for computing the exact edit distance between two 256-character ASCII strings, achieved by reducing the main algorithm cost from over 94 PBS operations per cell to one, and the character equality check from five PBS to two. The principal implication for AI practitioners is that this significant performance improvement makes privacy-preserving approximate string matching computationally feasible for real-world applications like financial fraud detection or genomic analysis, where the overhead of FHE was previously a prohibitive barrier. |
| Local Scale Equivariance with Latent Deep Equilibrium Canonicalizer (Read more on arXiv or HuggingFace) |
Jeremiah Jiang, Lim Jun Hao, Michael N. Cheng, Chiao-An Yang, ashiq24 |
This paper presents a Deep Equilibrium Canonicalizer (DEC) to improve the local scale equivariance of deep learning models. The objective is to develop a method that makes neural networks robust to independent scale variations of different objects within the same image. The methodology involves defining a “monotone scaling” group to approximate local scaling and using a DEC module, an implicit neural network, to transform latent features into a canonical representation by solving for a fixed point of a learned energy function. On a locally scaled MNIST dataset using a Swin transformer, the proposed method improved accuracy to 96.53% from a 93.93% baseline and reduced the invariance error (InvE) from 5.44 to 2.08. The principal implication for AI practitioners is that DEC can be integrated as an adaptable module into existing pre-trained vision models to enhance their performance and prediction consistency on tasks with significant local object scale variations, without requiring architectural redesign. |
| mSCoRe: a Multilingual and Scalable Benchmark for Skill-based |
|
|
| Commonsense Reasoning (Read more on arXiv or HuggingFace) |
anoperson, Franck-Dernoncourt, ntnghia1811 |
The paper introduces mSCoRe, a multilingual and scalable benchmark with a novel skill-based taxonomy to enable fine-grained analysis of commonsense reasoning in Large Language Models (LLMs). The primary objective is to systematically evaluate and analyze LLMs’ multilingual commonsense reasoning capabilities by creating a benchmark that can dynamically scale in difficulty and classify the atomic reasoning steps models employ. The methodology involves a four-step data synthesis pipeline that starts from seed datasets, uses an LLM to generate structured reasoning paths tagged with specific skills, systematically scales question complexity, and creates context-implicit questions to test inferential abilities. Experiments on eight state-of-the-art LLMs demonstrate significant performance degradation with increasing complexity, with the average accuracy of GPT-4o on the general commonsense subset (mSCoRe-G) dropping from 79.2% at the base complexity level to 69.5% at level 3. For AI practitioners, this implies that current models, including those with reasoning-reinforced training, have a rigid and limited utilization of reasoning skills, highlighting the need for developing training methodologies that foster more diverse and adaptive reasoning strategies beyond simple parameter scaling. |
| Refining Contrastive Learning and Homography Relations for Multi-Modal |
|
|
| Recommendation (Read more on arXiv or HuggingFace) |
Shiqing Wu, Yawen Zeng, guandongxu, MrShouxingMa |
The paper introduces REARM, a framework that enhances multi-modal recommendation by refining contrastive learning and expanding homography relations. The primary objective is to overcome the limitations of existing methods that generate noisy shared-modal representations while losing valuable unique-modal information, and to better model user-item interplay. The methodology integrates GNN-based learning on four distinct homography graphs (user/item co-occurrence and interest/semantic graphs) with a novel contrastive learning module that employs a meta-network to denoise shared features and an orthogonal constraint loss to preserve unique features. On the Sports dataset, REARM achieves a Recall@20 of 0.1231, outperforming the next-best state-of-the-art baseline which scored 0.1139. For AI practitioners, the principal implication is that multi-modal recommendation performance can be significantly improved by explicitly disentangling shared and unique feature representations via mechanisms like meta-networks and orthogonality constraints, rather than relying solely on feature alignment. |
Papers for 2025-08-20
| Title |
Authors |
Summary |
| Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent |
|
|
| Distillation and Agentic RL (Read more on arXiv or HuggingFace) |
Liam-Liu, hugteste, kangz, wanwan1212, tianyue818 |
Chain-of-Agents (CoA) introduces Agent Foundation Models (AFMs) as a novel paradigm for end-to-end complex problem-solving via multi-agent collaboration within a single LLM. The primary objective is to overcome the computational inefficiency and limited data-centric learning of existing multi-agent systems by enabling dynamic, multi-agent collaboration and tool orchestration within a unified model. The methodology involves multi-agent knowledge distillation for agentic supervised fine-tuning (SFT) to capture expert decision-making patterns, followed by agentic reinforcement learning (RL) on verifiable tasks to further refine problem-solving capabilities. Empirical studies show that AFMs establish new state-of-the-art performance across various benchmarks; for example, AFM-RL-32B attained an average accuracy of 78.0% on mathematical reasoning tasks, improving upon ReTool-32B’s 74.4%. This work demonstrates that AFMs offer a more efficient and coherent framework for complex tasks, with all model weights, code, and training data open-sourced to foster future research in agent models and agentic RL. |
| LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videos (Read more on arXiv or HuggingFace) |
Yen-Yu Lin, Fu-En Yang, Cheng Sun, cmhungsteve, linjohnss |
LongSplat is a robust 3D Gaussian Splatting framework for novel view synthesis from unposed, casually captured long videos. Its primary objective is to accurately reconstruct 3D scenes and generate novel views without relying on provided camera poses, even with irregular camera motion. The methodology involves an incremental joint optimization pipeline that integrates correspondence-guided camera pose estimation and photometric refinement with adaptive octree-anchored 3DGS, alternating between local and global optimization. LongSplat consistently outperforms state-of-the-art baselines; for instance, on the Free dataset, it yields an average PSNR of 27.88 and an Absolute Trajectory Error (ATE) of 0.028, while achieving 281.71 FPS and a 101 MB model size. This robust unposed reconstruction capability, coupled with high efficiency and memory optimization, provides AI practitioners with a practical solution for 3D scene reconstruction and novel view synthesis from challenging real-world video data. |
| Prompt Orchestration Markup Language (Read more on arXiv or HuggingFace) |
Yuqing Yang, Nan Chen, Yuge Zhang, Jiahang |
Prompt Orchestration Markup Language (POML) is a novel markup language designed to address critical challenges in LLM prompt engineering, including structure, data integration, format sensitivity, and tooling. POML employs component-based markup for logical structure, specialized tags for seamless data integration (documents, tables, images), and a CSS-like styling system to decouple content from presentation, complemented by a templating engine and a comprehensive developer toolkit. A TableQA case study validated POML’s impact, showing that styling variations, managed by POML, can significantly affect LLM performance; for instance, GPT-3.5-Turbo’s accuracy improved by 929% (from 6% to 61.8%) and Phi-3 Medium’s by 4450% (from 0.7% to 32.2%) between their worst and best styles. These findings underscore POML’s ability to streamline the prompt engineering lifecycle, enabling AI practitioners to systematically test and optimize prompt styles for various LLM models and tasks, thereby enhancing authoring efficiency and collaboration for complex, data-intensive applications. |
| MultiRef: Controllable Image Generation with Multiple Visual References (Read more on arXiv or HuggingFace) |
Shiyun Lang, Siyuan Wu, Dongping Chen, Ruoxi Chen, wsnHowest |
MultiRef introduces a novel benchmark and dataset for evaluating controllable image generation using multiple visual references. The research aims to address the limitations of existing generative models in effectively combining diverse visual inputs beyond single-source conditioning. Their methodology involves MultiRef-Bench, comprising 1,990 real-world and synthetic examples generated by the REFBLEND data engine with 10 reference types and 33 combinations, evaluated using rule-based metrics and MLLM-as-a-Judge. Primary results show that state-of-the-art models struggle with multi-reference conditioning, with the best model, OmniGen, achieving only 66.6% in synthetic samples and 79.0% in real-world cases on average compared to the golden answer. This highlights a clear weakness in current image generation systems, guiding AI practitioners to develop more flexible and human-like creative tools capable of integrating multiple visual inspirations. |
| Mind the Generation Process: Fine-Grained Confidence Estimation During |
|
|
| LLM Generation (Read more on arXiv or HuggingFace) |
Xinyi Wang, Jie Shi, Shisong Chen, Tingyun Li, JinyiHan |
FineCE is a novel method for fine-grained confidence estimation in Large Language Models (LLMs) during text generation. The primary objective is to provide accurate and continuous confidence scores throughout the LLM generation process, thereby enhancing model calibration and trustworthiness. The methodology includes a data construction pipeline using Monte Carlo Sampling for probabilistic distributions, supervised training, and a Backward Confidence Integration (BCI) strategy at inference to refine estimates with future context, alongside strategies for optimal confidence estimation positions. Experiments demonstrate FineCE consistently achieves AUROC scores exceeding 70%, outperforming baselines by 10-15 percentage points, and significantly reduces calibration errors (e.g., ECE 6.7% vs baselines 19.2-28.3%). For AI practitioners, this enables early error detection, informed LLM decision-making, and confidence-based output filtering during generation, directly improving LLM reliability and practical utility. |
| Training-Free Text-Guided Color Editing with Multi-Modal Diffusion |
|
|
| Transformer (Read more on arXiv or HuggingFace) |
Deyu Zhou, Xili Dai, dorni, EvanTHU, zachary-yin |
This paper introduces ColorCtrl, a training-free text-guided method for color editing using Multi-Modal Diffusion Transformers (MM-DiT) that preserves geometry and material properties. The primary objective is to develop a training-free text-guided color editing method capable of accurately and consistently modifying colors in images and videos while preserving critical visual attributes like geometry, material properties, and light-matter interaction. ColorCtrl leverages the attention mechanisms within pre-trained MM-DiT models to achieve precise control over color attributes (via attribute re-weighting) and maintain structural integrity (via structure preservation) by manipulating attention maps and color tokens. The method demonstrates superior performance against existing training-free approaches and competitive results with commercial models; for instance, on the SD3 benchmark, ColorCtrl achieved a Canny score of 0.8473, indicating improved geometry preservation. This training-free approach, leveraging readily available MM-DiT models, offers AI practitioners a robust and efficient solution for high-fidelity text-guided color manipulation in image and video editing, reducing the need for extensive model retraining and enabling fine-grained artistic control. |
| Evaluating Podcast Recommendations with Profile-Aware LLM-as-a-Judge (Read more on arXiv or HuggingFace) |
Alice Wang, Edoardo D’Amico, Gustavo Penha, marcodena, frafabbri |
This paper introduces a profile-aware LLM-as-a-Judge framework for evaluating personalized podcast recommendations. The main objective is to establish a scalable and interpretable offline method for assessing recommendation quality, addressing limitations of traditional metrics. The key methodology involves a two-stage process: first, natural-language user profiles are distilled from 90 days of listening history, and then, a GPT-4 LLM is prompted with these profiles for fine-grained pointwise and pairwise judgments of recommendation alignment. Primary results show that the LaaJ-Profile judge achieved a 0.6442 ROC-AUC for episode-level evaluation and outperformed or matched a variant using raw listening histories, correctly identifying 66% of strongly misaligned episodes. This framework offers AI practitioners a scalable, reliable middle ground for pre-deployment model selection and iterative testing in recommender systems, bridging the gap between coarse offline metrics and subjective human-aligned assessments. |
| OmniTry: Virtual Try-On Anything without Masks (Read more on arXiv or HuggingFace) |
Xiaoduan Feng, Yiming Chen, Hengyuan Cao, Linlin Zhang, fengyutong |
OmniTry is a unified, mask-free virtual try-on framework designed to extend VTON beyond garments to any wearable objects. Its main objective is to enable try-on of diverse items without masks, addressing data curation challenges for unpaired images. The framework employs a two-staged pipeline: an initial stage leverages large-scale unpaired images with a repurposed inpainting diffusion transformer and traceless erasing for mask-free localization. The second stage then fine-tunes the model with paired images for object appearance consistency using two-stream adapters. OmniTry demonstrates superior performance on its OmniTry-Bench, achieving a M-CLIP-I of 0.8327 on the whole dataset, and exhibits rapid convergence even with few paired samples. This approach offers AI practitioners a robust and data-efficient solution for generalized virtual try-on, significantly broadening its application scope in e-commerce and digital fashion. |
| A Stitch in Time Saves Nine: Proactive Self-Refinement for Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Zishang Jiang, Tingyun li, Haiquan Zhao, Xinyi Wang, JinyiHan |
ProActive Self-Refinement (PASR) is a novel reinforcement learning method enabling Large Language Models (LLMs) to perform adaptive, in-process self-refinement during generation. The main objective is to empower LLMs to proactively refine their outputs during the generation process, overcoming limitations of traditional reactive, post-hoc refinement methods. PASR employs an on-policy Reinforcement Learning (RL) approach, guided by a comparison-based reward strategy, to dynamically determine when and how to refine based on the evolving generation state. Experimental results show significant improvements; for instance, on Qwen3-8B, PASR achieved an 8.2% improvement in accuracy while concurrently reducing average token consumption by 41.6%. This method offers AI practitioners a path to develop more reliable and resource-efficient LLM systems by enabling autonomous, real-time quality and efficiency improvements during content generation. |
| Advances in Speech Separation: Techniques, Challenges, and Future Trends (Read more on arXiv or HuggingFace) |
Zhuo Chen, Yi Luo, Wendi Sang, Guo Chen, JusperLee |
This paper systematically surveys deep neural network-based speech separation, clarifying the current landscape and assessing key technologies. Its objective is to provide a comprehensive guide by holistically examining learning paradigms, architectural components, and evaluation methods. The methodology involves extensive comparative analysis and reproducible benchmarking on standard datasets like WSJ0-2mix and LibriMix. Key results indicate significant performance advancements, with models like MossFormer2 achieving up to 24.1 SI-SDRi on the WSJ0-2mix dataset. The principal implication for AI practitioners is a clear roadmap highlighting challenges such as long-form audio processing and lightweight model design, alongside promising directions like generative models and pre-trained architectures for real-world deployment. |
| Embodied-R1: Reinforced Embodied Reasoning for General Robotic |
|
|
| Manipulation (Read more on arXiv or HuggingFace) |
Fei Ni, Yibin Chen, Yaoting Huang, Haiqin Cui, Yifu Yuan |
Embodied-R1 introduces a 3B Vision-Language Model for general robotic manipulation, leveraging a novel “pointing” centric representation to bridge the “seeing-to-doing gap” caused by data scarcity and embodiment heterogeneity. The model is trained using a two-stage Reinforced Fine-tuning (RFT) curriculum on the Embodied-Points-200K dataset, which supports four defined embodied pointing abilities. Embodied-R1 achieved state-of-the-art performance on 11 embodied spatial and pointing benchmarks. Critically, it demonstrated robust zero-shot generalization, achieving a 56.2% success rate in SIMPLEREnv and 87.5% across 8 real-world XArm tasks, a 62% improvement over baselines. This research indicates that a pointing-centric representation, coupled with an RFT training paradigm, is a generalizable approach for closing the perception-action gap in robotics. |
| Copyright Protection for Large Language Models: A Survey of Methods, |
|
|
| Challenges, and Trends (Read more on arXiv or HuggingFace) |
Xixiang Zhao, Qichen Liu, Xubin Yue, Zhenhua Xu, BreynaldDva |
This survey offers a comprehensive overview of copyright protection for Large Language Models (LLMs), clarifying the distinctions between text watermarking and model fingerprinting. The main objective is to systematically categorize existing LLM copyright protection methods, fostering advancements in intellectual property for these models. The paper’s methodology involves analyzing diverse text watermarking techniques and detailing model fingerprinting, which is categorized into intrinsic (parameter/representation, semantic feature, adversarial example-based) and invasive (weight-based, backdoor-based) approaches, along with discussions on fingerprint transfer and removal. While a survey, it highlights findings such as ensemble learning frameworks achieving 99.88% precision in detecting latent stylistic relationships among LLM families. The principal implication for AI practitioners is the critical need to adopt dedicated model fingerprinting methods that ensure robust LLM ownership attribution and resilience against modifications, safeguarding intellectual property and promoting long-term innovation. |
| TempFlow-GRPO: When Timing Matters for GRPO in Flow Models (Read more on arXiv or HuggingFace) |
Jian Yang, Wanli Li, Yuke Zhao, Siming Fu, shreddedpork |
TempFlow-GRPO is a temporally-aware reinforcement learning framework designed to improve human preference alignment and generation quality in flow-based text-to-image models. It aims to overcome limitations in existing GRPO methods for flow models, specifically addressing sparse terminal rewards and uniform optimization weighting by enabling precise credit assignment and adapting optimization intensity to each timestep’s exploration capacity. The framework introduces trajectory branching to attribute terminal rewards to intermediate exploratory actions and a noise-aware policy weighting scheme that modulates optimization intensity based on timestep-specific noise levels. Experiments demonstrate state-of-the-art performance; for instance, on the Geneval benchmark, TempFlow-GRPO achieved an overall score of 0.97 within 4,400 steps, significantly outperforming Flow-GRPO’s 0.90 score under the same conditions. For AI practitioners, TempFlow-GRPO provides a robust approach for training generative models more efficiently and effectively by explicitly leveraging temporal dynamics, leading to superior sample quality and human preference alignment. |
| Leveraging Large Language Models for Predictive Analysis of Human Misery (Read more on arXiv or HuggingFace) |
Abhilash Nandy, Aman Bansal, Rahul Seetharaman, Bishanka Seal |
This paper evaluates Large Language Models’ ability to predict numerical misery scores from text using various prompting strategies and a novel gamified framework. The main objective is to assess LLM performance on a continuous misery score regression task (0-100) and to evaluate their dynamic emotional reasoning capabilities under corrective feedback. The methodology involves benchmarking zero-shot, few-shot, and retrieval-augmented prompting, alongside a gamified evaluation framework that tests ordinal, binary, and scalar reasoning. The primary result is that embedding-based few-shot prompting significantly reduces Mean Absolute Error to 12.3, a substantial improvement over the zero-shot baseline of 23.48. For AI practitioners, the principal implication is that implementing retrieval-augmented few-shot prompting with semantically relevant examples is critical for improving accuracy in fine-grained affective regression tasks. |
| Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondence (Read more on arXiv or HuggingFace) |
Xin Chen, Zhiyang Dou, Zixin Yin, Yuhong Zhang, Ling-Hao Chen |
Motion2Motion introduces a training-free framework for cross-topology motion transfer between characters with substantially different skeletal topologies using sparse bone correspondences. The main research objective is to enable motion transfer across diverse skeletal topologies, addressing limitations of inherent topological inconsistency and scarce paired motion datasets. Its key methodology formulates transfer as an iterative patch-based motion matching and blending procedure, utilizing sparse joint correspondences and a few target motion examples without requiring deep model training or GPUs. Motion2Motion significantly outperforms baselines, achieving a Fréchet Inception Distance (FID) of 0.033 and 96.2% frequency alignment in similar skeleton transfer, compared to 0.507 and 72.0% for the best baseline. For AI practitioners, this offers a real-time, scalable, and data-efficient solution for topology-flexible motion adaptation, reducing reliance on large-scale datasets and enabling direct integration into animation workflows. |
| CorrSteer: Steering Improves Task Performance and Safety in LLMs through |
|
|
| Correlation-based Sparse Autoencoder Feature Selection (Read more on arXiv or HuggingFace) |
Adriano Koshiyama, Zekun Wu, seonglae |
CorrSteer is an automated method that improves LLM performance and safety by selecting and steering Sparse Autoencoder (SAE) features based on their correlation with task outcomes at inference time. The primary research objective is to develop a scalable steering pipeline that avoids reliance on contrastive datasets or large activation storage. The key methodology involves using Pearson correlation to relate SAE feature activations from generated tokens to task correctness scores for feature selection, then calculating steering coefficients from the average activations of successful samples. The method demonstrated significant performance gains, achieving a +22.9% absolute improvement on the HarmBench safety benchmark and a +4.1% improvement on MMLU for the Gemma 2 2B model using only 4000 samples. The principal implication for AI practitioners is a fully automated and efficient pipeline to enhance model capabilities for specific tasks, enabling targeted performance and safety improvements without requiring model retraining. |
| MedSAMix: A Training-Free Model Merging Approach for Medical Image |
|
|
| Segmentation (Read more on arXiv or HuggingFace) |
Jonas Geiping, Francesco Sammarco, Jiesi Hu, guinansu, podismine |
MedSAMix is a training-free framework that merges generalist (SAM) and specialist (MedSAM) models layer-wise to improve medical image segmentation performance. The primary objective is to enhance both domain-specific accuracy and generalization without requiring additional training data or computational overhead. The methodology employs a zero-order Bayesian optimization algorithm (SMAC) to automatically discover optimal layer-wise merging configurations by evaluating merged model performance on a small calibration dataset. On 25 medical segmentation tasks, MedSAMix achieved a 6.67% improvement in Dice coefficient for specialized single-task optimization and 4.37% for multi-task generalization compared to the best individual baseline model. The principal implication for AI practitioners is the ability to create superior models by combining existing foundation models with fine-tuned variants post-hoc, thereby mitigating single-model bias and bypassing the need for costly retraining or data aggregation. |
| Semantic IDs for Joint Generative Search and Recommendation (Read more on arXiv or HuggingFace) |
Enrico Palumbo, Edoardo D’Amico, Gustavo Penha, frafabbri, marcodena |
This paper investigates strategies for constructing unified Semantic IDs for a joint generative search and recommendation model. The primary objective is to determine if a single Semantic ID scheme can achieve high performance on both tasks, mitigating the performance trade-offs observed when using task-specific IDs. The authors compare several methods, including a key approach that fine-tunes a bi-encoder on both search and recommendation data to create a unified embedding space before tokenization via RQ-KMeans. The results demonstrate that while task-specific IDs are optimal for their own domain, the multi-task approach provides the most effective trade-off, achieving a balanced Search R@30 of 0.046 and Recommendation R@30 of 0.049. For AI practitioners, this implies that generating Semantic IDs from a jointly trained, shared representation space is a superior strategy for building unified generative retrieval systems compared to using separate or naively combined task-specific embeddings. |
| Describe What You See with Multimodal Large Language Models to Enhance |
|
|
| Video Recommendations (Read more on arXiv or HuggingFace) |
Mounia Lalmas, Andreas Damianou, marcodena |
This paper presents a zero-finetuning framework leveraging Multimodal Large Language Models (MLLMs) to enhance video recommendations. The primary objective was to evaluate if MLLM-derived captions outperform classical content features in standard video ranking tasks for recommendation. The methodology involves prompting off-the-shelf MLLMs (Qwen-VL, Qwen-Audio with Whisper for audio transcription) to generate semantically rich natural-language descriptions of video and audio content, which are then encoded and fed into standard collaborative, content-based, and generative recommender architectures like two-towers and SASRec. Experiments on the MicroLens-100K dataset demonstrated that MLLM-generated audio descriptions yielded up to a 60% relative gain in HR@10 (from 0.0253 to 0.0405) for the two-towers model compared to raw audio features, and MLLM video descriptions boosted HR@10 from 0.0393 to 0.0489 (+24%) over video features. These findings imply that AI practitioners can significantly improve video recommendation quality by integrating MLLM-generated high-level semantic descriptions into existing systems, enabling more intent-aware and contextually rich recommendations without extensive finetuning of large foundation models. |
| Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned |
|
|
| and Addressed for XR Research (Read more on arXiv or HuggingFace) |
Susanne Schmidt, Mana Masuda, Mugichoko445, cocolinux |
This survey investigates the vision and implementation of radiance fields (RF), including NeRF and 3DGS, for XR research. The main objective was to analyze how RF is envisioned for XR applications, how they are implemented, and to identify remaining research gaps. A systematic survey following PRISMA 2020 guidelines was conducted on 365 XR-related RF papers from computer vision, computer graphics, robotics, multimedia, human-computer interaction, and XR communities, with an in-depth analysis of 66 “XR-Addressed” papers. Results revealed a significant research gap: for instance, while 203 RF-related papers were published at CVPR 2024 (with 68 mentioning XR), only 11 RF contributions appeared at leading XR conferences (IEEE VR/ISMAR 2024), with merely 5 directly addressing XR-related RF research questions. This work provides AI practitioners a resource to understand XR-specific RF research topics and navigate the field’s rapid development, guiding future integration efforts into XR systems. |
| MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic |
|
|
| Evaluation of Audio General Intelligence (Read more on arXiv or HuggingFace) |
Fernando López, Vaibhavi Lokegaonkar, Šimon Sedláček, Sonal Kumar, Sreyan88 |
MMAU-Pro is a novel, comprehensive benchmark for holistically evaluating audio general intelligence in AI systems. It addresses the challenge of comprehensively assessing auditory intelligence, which existing benchmarks inadequately cover due to their limited scope and realistic complexity. The benchmark comprises 5,305 human expert-annotated question-answer instances across 49 distinct skills in speech, sound, and music, sourcing audio data directly “from the wild” and employing a multi-stage human-involved curation pipeline. Evaluations of 22 leading multimodal AI models reveal significant limitations, with state-of-the-art models like Gemini 2.5 Flash and Audio Flamingo 3 achieving only 59.2% and 51.7% accuracy, respectively. These findings highlight specific shortcomings in current models, such as shallow audio grounding and poor performance in multi-audio and spatial reasoning, offering clear directions for future AI system development toward general audio intelligence. |
| MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents (Read more on arXiv or HuggingFace) |
Jun Dong, Jiaheng Liu, Wenjie Wang, Xingyuan Bu, Shilong Li |
MM-BrowseComp is a novel benchmark designed to assess advanced AI agents’ ability to synthesize deep reasoning with persistent, multimodal web browsing. Its objective is to bridge gaps in existing benchmarks by requiring agents to retrieve and reason with multimodal content, including images and videos, beyond text. The methodology involves 224 hand-crafted questions with mandatory multimodal dependency and an irreducible reasoning checklist for fine-grained process evaluation. Primary results indicate that state-of-the-art models struggle significantly, with OpenAI o3 achieving the highest Overall Accuracy at only 29.02%, and other models failing to surpass 10%. This demonstrates that high performance in multimodal browsing necessitates a synergistic combination of strong foundational reasoning abilities and a comprehensive, robust toolset. |
| ZARA: Zero-shot Motion Time-Series Analysis via Knowledge and Retrieval |
|
|
| Driven LLM Agents (Read more on arXiv or HuggingFace) |
Flora D. Salim, Hao Xue, Breezelled, zechenli03 |
ZARA is a zero-shot, agent-based framework that uses a hierarchical pipeline of LLM agents, a pre-computed knowledge base, and retrieval-augmented generation to perform explainable human activity recognition directly from raw motion time-series data. The primary objective is to create a zero-shot human activity recognition (HAR) system that avoids costly retraining and provides interpretable predictions by equipping a large language model with structured domain knowledge and a relevant evidence retrieval mechanism. The methodology involves an offline phase to build an activity-pair feature importance knowledge base and placement-specific vector databases, and an online inference phase where a four-stage hierarchical agent pipeline uses a frozen LLM to select features, prune candidate activities based on retrieved evidence, and generate a final prediction with a rationale. Across 8 HAR benchmarks, ZARA achieved an average macro F1 score of 81.4%, a 2.53x improvement over the strongest baseline (UniMTS). The principal implication for AI practitioners is that this framework provides a template for building accurate and interpretable zero-shot time-series analysis systems without model fine-tuning, enabling plug-and-play deployment by structuring domain-specific statistical knowledge and integrating it into retrieval-augmented LLM agent workflows. |
| Beyond Human Judgment: A Bayesian Evaluation of LLMs’ Moral Values |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Alina Landowska, maciejskorski |
This paper presents a Bayesian evaluation of large language models’ understanding of moral dimensions. The research investigates how large language models comprehend moral dimensions compared to human annotators. A GPU-optimized Bayesian framework, utilizing a Dawid-Skene variant with Dirichlet priors, was employed to model annotator disagreements and estimate probabilistic ground truth labels across 250K+ annotations from three diverse corpora. Results show AI models consistently outperformed human annotators, typically ranking in the top 25% and achieving 2–4x lower false negative rates (19.4% vs 52.7% on average), albeit with slightly higher false positive rates. This highlights LLMs’ superior recall for moral foundation detection, making them valuable for identifying overlooked moral signals, though careful calibration for specific applications is needed due to elevated false positive rates. |
Papers for 2025-08-19
| Title |
Authors |
Summary |
| Ovis2.5 Technical Report (Read more on arXiv or HuggingFace) |
Yang Li, cqgwin, Suikong, xxyyy123, runninglsy |
Ovis2.5 is a new multimodal large language model (MLLM) designed for native-resolution visual perception and enhanced reasoning. The paper addresses shortcomings in previous MLLMs, specifically rigid vision front-ends hindering analysis of dense content (e.g., charts) and linear chain-of-thought training lacking self-correction for deeper reasoning. Ovis2.5 integrates a native-resolution Vision Transformer (NaViT) for variable image resolutions and employs a five-phase training curriculum including DPO and GRPO, incorporating “thinking-style” data for reflection. Comprehensive evaluations show Ovis2.5-9B achieved an average OpenCompass score of 78.3, establishing state-of-the-art performance among open-source MLLMs in the sub-40B parameter range. AI practitioners can leverage Ovis2.5 for improved performance in visually dense and complex reasoning tasks, including STEM and chart analysis, and utilize its resource-efficient training infrastructure for faster model development and deployment in constrained environments. |
| ComoRAG: A Cognitive-Inspired Memory-Organized RAG for Stateful Long |
|
|
| Narrative Reasoning (Read more on arXiv or HuggingFace) |
Yufeng Wang, Wei Wei, Rongchen Zhao, Juyuan Wang, lxucs |
ComoRAG introduces a cognitive-inspired Retrieval-Augmented Generation framework for stateful long narrative reasoning. The main objective is to address the challenge of global context comprehension in long narratives, which traditional stateless RAG methods fail to capture. ComoRAG employs a Metacognitive Regulation process inspired by the human Prefrontal Cortex, featuring a Hierarchical Knowledge Source, Dynamic Memory Workspace, and a Metacognitive Control Loop with iterative operations like Self-Probe and Mem-Fuse. The framework achieves significant performance gains, for instance, increasing accuracy on the EN.MC benchmark from a static-retrieval baseline of 64.6% to 72.9%. This demonstrates that ComoRAG offers a principled, robust, and flexible plug-and-play solution for AI practitioners to enhance complex query resolution in long-context narrative comprehension. |
| 4DNeX: Feed-Forward 4D Generative Modeling Made Easy (Read more on arXiv or HuggingFace) |
Zeng Tao, Jiawei Ren, Long Zhuo, Tianqi Liu, Zhaoxi Chen |
4DNeX is a novel feed-forward framework for generating dynamic 3D scene representations (4D) from a single image. The primary objective is to enable efficient, end-to-end image-to-4D generation, addressing limitations of existing computationally intensive or multi-frame input methods. This is achieved by fine-tuning a pretrained video diffusion model, utilizing a newly constructed 4DNeX-10M dataset, employing a unified 6D video representation (RGB+XYZ sequences), and applying simple adaptation strategies like width-wise fusion and XYZ normalization. Extensive experiments demonstrate 4DNeX’s superior efficiency and generalizability; for example, it generates 4D scenes in 15 minutes, significantly faster than optimization-based methods like Free4D (60 minutes), while achieving competitive metrics such as 97.2% consistency and 58.3% dynamic degree in image-to-4D tasks. 4DNeX provides a scalable and accessible solution for image-to-4D modeling, laying the foundation for efficient generative 4D world models that simulate dynamic scene evolution. |
| Next Visual Granularity Generation (Read more on arXiv or HuggingFace) |
Kang Liao, Qingyi Tao, Zhonghua Wu, Zhouxia Wang, yikaiwang |
This paper introduces Next Visual Granularity (NVG), a novel image generation framework representing images as structured sequences of varying granularity levels. The primary objective is to advance image generation by explicitly modeling hierarchical visual structure, addressing the limitation of existing methods treating images as unstructured data, and enabling fine-grained control. NVG decomposes images into content and structure pairs across multiple stages using a multi-granularity quantized autoencoder and a residual, pyramid-like token construction. It iteratively refines the image by generating structure maps with a lightweight rectified flow model and content with a transformer, incorporating Structure-Aware RoPE. Compared to VAR models, NVG consistently outperforms them in FID scores, with NVG-d24 achieving an FID of 2.06 versus VAR-d24’s 2.09, and its tokenizer demonstrates superior reconstruction quality (rFID 0.74 for NVG vs 1.06 for VAR). For AI practitioners, NVG offers a scalable, more controllable generative system that supports explicit structure control directly during generation without requiring additional post-hoc modules, proving beneficial for applications where structural and hierarchical control is essential. |
| Speed Always Wins: A Survey on Efficient Architectures for Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Jusen Du, Yucheng Zhou, Jiaxi Hu, Weigao Sun, landisen |
“Speed Always Wins” surveys innovative architectures optimizing Large Language Models (LLMs) for efficiency. The paper systematically examines how to overcome the Transformer’s quadratic complexity and high resource demands to achieve more efficient and scalable LLMs. It categorizes and reviews recent advancements into seven areas, including linear/sparse sequence modeling, efficient full attention, sparse Mixture-of-Experts, hybrid architectures, and Diffusion LLMs. For example, hybrid models like Jamba [185] achieve 3x higher throughput than Mixtral while supporting 256K context with only 4GB KV cache. This survey serves as a blueprint for AI practitioners to develop scalable, resource-aware, and versatile LLM systems by integrating these architectural principles. |
| Has GPT-5 Achieved Spatial Intelligence? An Empirical Study (Read more on arXiv or HuggingFace) |
Ruisi Wang, Qingping Sun, Yubo Wang, yl-1993, caizhongang |
This empirical study assesses GPT-5’s spatial intelligence across eight recent benchmarks, revealing significant advancements but also persistent limitations compared to human performance. The paper aims to examine the extent to which GPT-5 and other state-of-the-art multi-modal large language models (MLLMs) have achieved spatial intelligence. It proposes a comprehensive taxonomy of spatial tasks, evaluating models on eight key benchmarks (e.g., VSI-Bench, SITE, MindCube) using standardized prompts and Chance-Adjusted Accuracy (CAA), consuming over one billion tokens. Findings indicate GPT-5 sets a new state-of-the-art in spatial intelligence, achieving a Chance-Adjusted Accuracy (CAA) of 21.67 on MindCube, yet still falls significantly short of human performance (Human MindCube CAA: 91.94). For AI practitioners, this research clarifies fundamental spatial task categories and identifies the remaining unique challenges for MLLMs, emphasizing the need for continued development to bridge the human-model gap in complex spatial reasoning. |
| HeroBench: A Benchmark for Long-Horizon Planning and Structured |
|
|
| Reasoning in Virtual Worlds (Read more on arXiv or HuggingFace) |
Artyom Sorokin, Viktor Volkov, Stefan Rebrikov, Petr Anokhin, roxal |
HeroBench is a novel benchmark for evaluating large language models’ (LLMs) long-horizon planning and structured reasoning in complex virtual worlds. Its main objective is to assess LLMs’ ability to generate and execute extended, interdependent action sequences, addressing the limitations of simpler algorithmic benchmarks. The benchmark utilizes a grid-based, RPG-style simulated environment, presenting JSON-serialized tasks that require LLMs to generate Python code for actions like resource gathering, crafting, and combat, with performance evaluated by Success and Progress scores. Evaluations of 25 state-of-the-art LLMs revealed substantial performance disparities, with Grok-4 achieving the highest success rate of 91.7% on base tasks, demonstrating superior robustness across difficulty levels. This work highlights persistent challenges in robust long-horizon autonomous planning for LLMs, underscoring the need for continued research into planning architectures and the careful design of multi-agent systems for complex sequential tasks. |
| When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness |
|
|
| Methods for LLMs (Read more on arXiv or HuggingFace) |
Elena Tutubalina, Gleb Ershov, Mikhail Chaichuk, apanc, myyycroft |
This paper presents a large-scale comparative evaluation of five prompt robustness methods for Large Language Models. The core objective was to systematically compare the effectiveness of existing prompt robustness methods across diverse LLM families and sizes. The study benchmarked five in-context learning and supervised fine-tuning techniques on 8 open-source and 2 frontier LLMs across 52 Natural Instructions tasks, evaluating their performance against diverse prompt formats and under various distribution shifts. Key findings indicate that Batch Calibration significantly reduced prompt sensitivity (spread) for 6/8 open-source models while improving accuracy, and a majority voting-based Template Ensembles method reduced spread for frontier models by at least 44% in 9 of 20 cases. AI practitioners should consider calibration for open-source LLMs in balanced classification, prefer probability ranking over greedy decoding, and apply majority voting-based Template Ensembles for black-box frontier models to mitigate prompt sensitivity. |
| Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive |
|
|
| World Model (Read more on arXiv or HuggingFace) |
Yifan Zhang, Boyang Wang, Zexiang Liu, Chunli Peng, Xianglong He |
Matrix-Game 2.0 is an open-source, auto-regressive diffusion world model that generates interactive, long-form video streams in real-time from visual and user-action inputs. The main objective is to develop a framework for real-time, streaming interactive video generation that overcomes the latency of bidirectional models and the error accumulation of traditional auto-regressive approaches. The methodology involves training a bidirectional vision-only diffusion transformer on a large-scale (~1200 hours) dataset from game environments, then distilling it into a causal, few-step auto-regressive model using a Self-Forcing technique and KV-caching. The primary result is the model’s ability to generate minute-level, 352×640 resolution video at 25.15 FPS on a single H100 GPU, demonstrating high-fidelity, real-time user interaction. The principal implication for AI practitioners is that it provides an open-source model and a data production pipeline for building real-time interactive simulations, offering a viable foundation for applications requiring on-the-fly, generative virtual environments without reliance on traditional rendering engines. |
| Lumen: Consistent Video Relighting and Harmonious Background Replacement |
|
|
| with Video Generative Models (Read more on arXiv or HuggingFace) |
Zixiang Gao, Chenxuan Miao, Yutong Feng, Yuxuan Liu, Jianshu Zeng |
Lumen is an end-to-end video relighting framework that also performs harmonious background replacement using large-scale video generative models. The objective is to relight video foregrounds with harmonious blending while preserving intrinsic attributes and replacing backgrounds based on textual descriptions, overcoming data scarcity and ensuring temporal consistency. Lumen leverages a multi-domain dataset of 3D-rendered and HDR-simulated realistic paired videos and employs a DiT-based generative model with a domain-aware style adapter trained via a two-stage curriculum. Quantitative evaluations demonstrate Lumen’s superior performance, achieving a PSNR of 23.06 on realistic paired videos, indicating enhanced foreground preservation and lighting harmonization compared to existing methods. This framework provides AI practitioners with a robust, text-guided solution for high-quality video relighting and background replacement, applicable across diverse real-world scenarios due to its generalization capabilities. |
| S^2-Guidance: Stochastic Self Guidance for Training-Free Enhancement of |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Meiqi Wu, Nisha Huang, Xiaokun Feng, Jiashu Zhu, Chubin Chen |
i) S²-Guidance is a training-free method that enhances diffusion model outputs by using stochastically generated sub-networks for self-correction during inference. ii) The primary objective is to mitigate the semantic incoherence and low-quality artifacts produced by Classifier-Free Guidance (CFG) in diffusion models without requiring additional training or hand-crafted architectural modifications. iii) The key methodology, S²-Guidance, modifies the standard CFG update step by subtracting a corrective term. This term is the output of a temporary sub-network created on-the-fly at each denoising step via stochastic block-dropping, which guides the model away from suboptimal predictions. iv) The method demonstrates superior performance across multiple benchmarks; for instance, on the T2I-CompBench benchmark with the SD3 model, S²-Guidance achieved a color composition score of 59.63%, significantly outperforming the baseline CFG score of 53.61%. v) For AI practitioners, S²-Guidance can be implemented as a drop-in, training-free replacement for standard CFG in existing generation pipelines to improve output quality and prompt adherence. The most impactful finding is that a single stochastic sub-network sample per timestep is sufficient for effective guidance, making the enhancement computationally efficient and practical for deployment. |
| Representing Speech Through Autoregressive Prediction of Cochlear Tokens (Read more on arXiv or HuggingFace) |
Daniel L. K. Yamins, Evelina Fedorenko, Greta Tuckute, klemenk |
AuriStream is a two-stage, biologically-inspired model learning speech representations through autoregressive prediction of discrete cochlear tokens. The model’s objective is to learn versatile speech representations using a simple and scalable autoregressive prediction objective on a human cochlea-inspired time-frequency representation. Its methodology involves WavCoch, which transforms raw audio into discrete cochlear tokens via a 13-bit LFQ bottleneck, followed by AuriStream, a GPT-style Transformer autoregressively predicting upcoming cochlear tokens. AuriStream-1B achieved state-of-the-art lexical semantics with an sSIMI score of 12.52 on the LibriSpeech Audio subset and competitive performance across diverse SUPERB speech tasks, including 4.20 ASR and 98.01 IC. This framework demonstrates that an autoregressive objective on biologically-inspired inputs can yield versatile representations, offering an interpretable alternative to current speech AI models through its ability to generate audio from predictions. |
| Inverse-LLaVA: Eliminating Alignment Pre-training Through Text-to-Vision |
|
|
| Mapping (Read more on arXiv or HuggingFace) |
Tyler Derr, xuhuizhan5 |
Inverse-LLaVA eliminates expensive alignment pre-training in large vision-language models by inverting the conventional modality mapping direction. The primary objective is to investigate if mapping text embeddings into continuous visual space, instead of projecting visual features to text, can maintain or improve performance while removing the alignment stage. This is achieved by projecting text embeddings into continuous visual representation space and performing fusion within transformer intermediate layers via selective additive attention components during single-stage instruction tuning. Empirically, Inverse-LLaVA demonstrates +27.2% improvement on cognitive reasoning tasks (MME benchmark) and reduces training computational requirements by 45% by eliminating alignment pre-training. This suggests that architectural innovation can effectively substitute for data-intensive alignment procedures, allowing AI practitioners to develop powerful multimodal models with significantly lower computational and data demands, particularly for complex reasoning tasks. |
| Precise Action-to-Video Generation Through Visual Action Prompts (Read more on arXiv or HuggingFace) |
Minghan Qin, Sida Peng, Haoyu Guo, walsvid, angshineee |
This paper introduces visual action prompts for precise action-to-video generation in complex, high-degree-of-freedom interaction scenarios. The primary objective is to develop a generalizable action-to-video model that accurately depicts interaction outcomes while balancing action precision and dynamic transferability across domains, addressing the lack of a unified precise action representation. The methodology involves “rendering” actions into domain-agnostic visual prompts, specifically 2D skeletons, which are robustly recovered from human-object interaction and robotic manipulation datasets via specialized pipelines, then integrated into a pretrained CogVideoX model using ControlNet. Quantitative experiments on EgoVid, RT-1, and DROID datasets show that visual action prompts outperform alternative control signals; for instance, on RT-1, the unified skeleton approach achieved a Spatio-temporal IoU of 0.576, exceeding text (0.267) and raw state (0.507) controls. This approach enables training unified action-driven generative models across heterogeneous datasets, facilitating crucial cross-domain knowledge transfer for AI practitioners. |
| G-CUT3R: Guided 3D Reconstruction with Camera and Depth Prior |
|
|
| Integration (Read more on arXiv or HuggingFace) |
Evgeny Burnaev, Peter Wonka, Artem Komarichev, rusrakhimov, smileyenot983 |
G-CUT3R is a novel feed-forward method that enhances the CUT3R framework for 3D scene reconstruction by integrating auxiliary prior information like camera parameters and depth maps. The main objective is to improve the accuracy of feed-forward reconstruction by leveraging commonly available geometric data that existing models typically ignore. The methodology modifies the CUT3R decoder by introducing dedicated encoders for each prior modality and fusing their features with RGB image tokens via zero-initialized convolutional layers, enabling stable and flexible integration of any combination of priors. The proposed method demonstrates significant performance improvements, achieving a 61% reduction in Absolute Translation Error (from 0.077 to 0.030) on the Sintel dataset when incorporating pose guidance. For AI practitioners, this provides a lightweight and versatile solution to boost 3D reconstruction quality in real-world applications by utilizing available sensor data (e.g., from LiDAR or IMU) without fundamentally altering existing feed-forward architectures. |
| Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning |
|
|
| Models to Ask for Information (Read more on arXiv or HuggingFace) |
Xi Yang, Duanyu Feng, Chen Huang, Bowen Qin, YouchengHuang |
This paper introduces the CRITIC-math benchmark to evaluate the inability of Large Reasoning Models (LRMs) to ask for information when faced with incomplete mathematical problems. The research aims to systematically assess to what extent LRMs can identify problem incompleteness and proactively ask for clarification, and whether this skill can be improved through supervised fine-tuning. A new dataset, CRITIC-math, was constructed by rewriting well-defined math problems into two types of incomplete problems (“missing goal” and “missing premises”), which was then used to evaluate several state-of-the-art LRMs. The primary result is that LRMs perform poorly, achieving Clarification Ratios of only around 25% with implicit prompts, and when failing to ask, they exhibit overthinking, hallucination of missing information, and “thoughts-to-answer unfaithfulness.” The principal implication for AI practitioners is that the current LRM development paradigm, which focuses exclusively on solving well-defined problems, is insufficient and should be augmented with methodologies that train models to identify and query for missing information to build more robust and genuinely intelligent systems. |
Papers for 2025-08-18
| Title |
Authors |
Summary |
| SSRL: Self-Search Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yanxu Chen, Yuxin Zuo, Heng Zhou, Kaiyan Zhang, Yuchen Fan |
This paper introduces Self-Search Reinforcement Learning (SSRL), a method for training large language models (LLMs) to answer complex queries by iteratively querying their own internal knowledge in a simulated search environment. The primary objective is to quantify the intrinsic search capabilities of LLMs and determine if RL training in this fully simulated (“full-sim”) setting enables effective sim-to-real transfer to external search engines. The methodology involves using format-based and outcome-based rewards to train a policy model, which serves as both the agent and the environment, to autoregressively generate search queries and corresponding informational responses. SSRL demonstrates superior performance over API-dependent baselines; for example, the SSRL-trained Llama-3.1-8B-Instruct model achieved a 43.1% average accuracy across six benchmarks, outperforming ZeroSearch’s 41.5%. For AI practitioners, this presents a cost-effective paradigm for training search agents by eliminating the need for expensive API calls during the RL training phase, creating models that can then be deployed with real-world search engines at inference time. |
| Thyme: Think Beyond Images (Read more on arXiv or HuggingFace) |
Wei Chen, Chaoyou Fu, Shukang Yin, Xingyu Lu, Yi-Fan Zhang |
This paper introduces Thyme, a framework enabling Multimodal Large Language Models (MLLMs) to autonomously generate and execute code for dynamic image manipulation and computation to solve complex visuo-linguistic tasks. The primary objective is to equip MLLMs with the capability to perform on-the-fly image processing (e.g., cropping, rotation, contrast enhancement) and mathematical calculations via a code-generation and sandbox-execution loop, moving beyond static visual perception. The approach utilizes a two-stage training process, starting with Supervised Fine-Tuning (SFT) on a 500K-sample dataset, followed by a Reinforcement Learning (RL) phase that employs the proposed GRPO-ATS algorithm, which uses adaptive sampling temperatures (τ=0 for code, τ=1 for text) to balance execution precision with reasoning exploration. Comprehensive evaluations show that Thyme significantly outperforms its baseline, improving reasoning performance on the MME-Realworld Autonomous Driving benchmark by 81.57% and increasing overall accuracy on HRBench-8K from 65.3% to 72.0%. For AI practitioners, this SFT-RL framework demonstrates that integrating a code-execution sandbox allows MLLMs to actively manipulate visual inputs as tools during their reasoning process, proving highly effective for tasks requiring detailed analysis of high-resolution or perceptually challenging images. |
| DINOv3 (Read more on arXiv or HuggingFace) |
Maxime Oquab, Federico Baldassarre, Maximilian Seitzer, Huy V. Vo, Oriane Siméoni |
DINOv3 is a self-supervised vision foundation model that significantly advances dense feature quality and task versatility. The main objective is to address dense feature map degradation during large-scale SSL training and provide a robust, off-the-shelf universal visual encoder family. This is achieved through extensive data and model scaling, introducing a novel Gram anchoring strategy for maintaining patch-level consistency, and post-hoc high-resolution adaptation and knowledge distillation. DINOv3 (ViT-7B/16) demonstrates superior performance on various dense tasks, notably achieving 55.9 mIoU on ADE20k semantic segmentation and 64.4 recall for 3D geometric correspondence on NAVI, significantly surpassing previous self-supervised and weakly-supervised models. AI practitioners can leverage DINOv3 as a versatile, pre-trained backbone that delivers state-of-the-art results across diverse computer vision applications, often without fine-tuning, thereby enabling scalable solutions for resource-constrained environments. |
| XQuant: Breaking the Memory Wall for LLM Inference with KV Cache |
|
|
| Rematerialization (Read more on arXiv or HuggingFace) |
Rishabh Tiwari, Haocheng Xi, Minjae Lee, Coleman Hooper, Aditya Tomar |
XQUANT reduces LLM inference memory consumption by quantizing and caching layer input activations (X) and rematerializing the KV cache on-the-fly, trading computation for memory bandwidth. The objective is to develop a method to drastically reduce the memory footprint of the LLM KV cache to alleviate the memory bandwidth bottleneck during inference by exploiting the growing gap between compute performance and memory bandwidth. The core method involves quantizing and caching the layer input activations (X) instead of the Key and Value tensors, which are then recomputed from the cached X during each generation step; an advanced variant, XQUANT-CL, further compresses the cache by quantizing the differences in X between successive layers. The primary result shows that XQUANT-CL achieves up to 12.5x memory savings relative to an FP16 baseline with only 0.1 perplexity degradation on Llama-2-7B, outperforming state-of-the-art KV cache quantization methods using only simple uniform quantization. For AI practitioners, this rematerialization approach allows deploying LLMs in memory-constrained environments or significantly increasing batch sizes and context lengths on existing hardware by converting a memory-bound problem into a compute-bound one. |
| PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical |
|
|
| Register Indexing (Read more on arXiv or HuggingFace) |
Xianpei Han, Yaojie Lu, Hongyu Lin, Xuanang Chen, lzq2021 |
PaperRegister enhances flexible-grained paper search through hierarchical register indexing and adaptive retrieval. The primary objective is to enable paper search systems to handle queries across varying granularities, moving beyond traditional coarse-grained methods. Its methodology involves offline construction of a hierarchical index tree using large language models for fine-grained content extraction and bottom-up aggregation, coupled with online adaptive retrieval via a view recognizer trained with hierarchical reward policy optimization. Quantitatively, PaperRegister significantly improves Recall@5 by 22.3 percentage points on the challenging FG.Search-3 dataset (BM25-based matching), from 58.5 for abstract-based indexing to 80.8. This work provides AI practitioners with a robust framework to develop more powerful and adaptable information retrieval systems, capable of addressing complex, multi-granularity search requirements in specialized domains. |
| StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image |
|
|
| Translation (Read more on arXiv or HuggingFace) |
Junyong Noh, Kwan Yun, Seungmi Lee |
StyleMM is a novel framework for constructing stylized 3D Morphable Face Models (3DMMs) using text-driven aligned image translation. Its primary objective is to generate stylized 3DMMs that reflect user-defined text prompts, ensuring maintained correspondence, disentangled control over facial attributes, and expressive stylization beyond realistic models. The methodology involves fine-tuning pre-trained mesh deformation and texture generator networks with stylized facial images, generated via text-guided image-to-image translation using a diffusion model (SDXL) and an Explicit Attribute-preserving Module (EAM) that preserves facial attributes. Quantitative evaluations show StyleMM achieves higher face diversity and style scores across various styles; for instance, it achieved a Face Diversity of 12.070 for “Pixar child” style, outperforming baselines like LeGO (9.836). StyleMM enables feed-forward generation of stylized face meshes with explicit control over shape, expression, and texture parameters, providing consistent 3D style transfer for applications in digital content production. |
| FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for |
|
|
| Audio-Driven Portrait Animation (Read more on arXiv or HuggingFace) |
Mu Xu, Fan Jiang, MengChao Wang, wangqiang9 |
FantasyTalking2 introduces a novel preference optimization framework, TLPO, for enhanced audio-driven portrait animation. The primary objective is to align diffusion-based portrait animation models with fine-grained, multidimensional human preferences across motion naturalness, lip-sync accuracy, and visual quality, addressing inherent conflicts between these objectives. The methodology involves Talking-Critic, a multimodal reward model, to curate Talking-NSQ, a large-scale preference dataset, and Timestep-Layer adaptive multi-expert Preference Optimization (TLPO) which decouples preferences into specialized LoRA expert modules fused dynamically across timesteps and network layers. Experiments show TLPO achieves state-of-the-art results, with user studies indicating relative improvements of 12.7% in lip synchronization, 15.0% in motion naturalness, and 13.7% in visual quality over the strongest baseline. For AI practitioners, this demonstrates that a granular, adaptive preference fusion strategy is crucial for achieving high-quality, human-aligned outputs in generative AI without performance trade-offs across competing objectives. |
| TexVerse: A Universe of 3D Objects with High-Resolution Textures (Read more on arXiv or HuggingFace) |
Nan Cao, Rui Ma, Li Zhang, YiboZhang2001 |
TexVerse is a large-scale 3D asset dataset featuring high-resolution textures. This dataset aims to address the critical gap in suitable datasets for end-to-end high-resolution texture and PBR material generation. The methodology involved curating models from Sketchfab, filtering for texture resolutions of at least 1024 pixels, and acquiring original user-uploaded file formats for rigged and animated models, complemented by 856,312 GPT-5 generated annotations. TexVerse comprises 858,669 unique high-resolution 3D models and 1,659,097 total 3D instances, with 158,518 models incorporating PBR materials. This resource directly enables advancements in high-resolution texture generation, PBR material synthesis, animation, and diverse 3D vision and graphics applications for AI practitioners. |
| Controlling Multimodal LLMs via Reward-guided Decoding (Read more on arXiv or HuggingFace) |
Michal Drozdzal, Adriana Romero-Soriano, Koustuv Sinha, Pierluca D’Oro, oscmansan |
This paper introduces Multimodal Reward-Guided Decoding (MRGD) for inference-time control of Multimodal Large Language Models (MLLMs) to improve visual grounding. The objective is to achieve on-the-fly controllability of MLLM inference, enabling dynamic trade-offs between object precision and recall, and between test-time compute and visual grounding quality. MRGD employs two multimodal reward models: r_hal for object hallucination (trained on preference data) and r_rec for object recall (composed from pre-trained modules). These are linearly combined with a user-defined weight to guide a search-based decoding process. Evaluations show MRGD consistently outperforms existing hallucination mitigation methods; for example, on LLaVA-1.5, MRGD with w=1.0 reduced instance-level hallucination (CHAIR_i) on COCO from 15.05% (greedy) to 4.53%. This provides AI practitioners with fine-grained inference-time control over MLLM outputs, facilitating adaptive behavior for diverse application needs and resource constraints while effectively mitigating hallucinations. |
| X-Node: Self-Explanation is All We Need (Read more on arXiv or HuggingFace) |
Islem Rekik, prajit123 |
X-Node is a novel self-explaining Graph Neural Network (GNN) framework where each node intrinsically generates explanations during the prediction process. The primary objective is to overcome limitations of post-hoc GNN explainability by enabling faithful, intrinsic, node-level reasoning within the model. X-Node constructs a structured context vector for each node, which a Reasoner maps to an explanation vector used for latent embedding reconstruction, natural language explanation via an LLM, and reinjection into the GNN’s message-passing pipeline. The framework consistently improves classification performance; for instance, it raised the F1 score for GCN on the OrganAMNIST dataset from 91.19% to 93.16%. This provides AI practitioners with a modular and transferable solution to integrate intrinsic, faithful explainability into GNNs, crucial for developing trustworthy AI systems in high-stakes applications. |
| SPARSE Data, Rich Results: Few-Shot Semi-Supervised Learning via |
|
|
| Class-Conditioned Image Translation (Read more on arXiv or HuggingFace) |
Paolo Soda, Loredana Zollo, Clemente Lauretti, Guido Manni |
This paper introduces SPARSE, a novel GAN-based semi-supervised learning framework for medical image classification in extremely low-data regimes. The objective is to achieve robust classification performance when labeled data is scarce (5 to 50 samples per class) by leveraging abundant unlabeled images. The methodology uses a three-player architecture—a generator for class-conditioned image translation, a discriminator, and a dedicated classifier—trained via a dynamic schedule that alternates between supervised and unsupervised phases, employing an ensemble-based temporal pseudo-labeling technique. The framework demonstrates statistically significant improvements over six state-of-the-art methods across eleven MedMNIST datasets, with the ensemble version achieving 66.22% average accuracy in the extreme 5-shot setting. For AI practitioners, this approach provides a practical solution for building high-performing classifiers in domains like medical imaging where data annotation is prohibitively expensive. |
| MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and |
|
|
| Multispectral Earth Observation Data (Read more on arXiv or HuggingFace) |
Nicolas Gonthier, Anatol Garioud, Nina Lardiere, Michael Vaccaro, Antoine Labatie |
MAESTRO is a Masked Autoencoder framework adapted for complex Earth Observation data that sets a new state-of-the-art by optimizing fusion and normalization strategies. The research objective is to adapt the Masked Autoencoder (MAE) to effectively learn representations from multimodal, multitemporal, and multispectral Earth Observation (EO) data by systematically evaluating data fusion and reconstruction target strategies. The key methodology involves benchmarking five token-based fusion modes (e.g., early vs. late) and introducing a novel “patch-group-wise” normalization scheme that groups spectrally correlated bands during reconstruction to inject a spectral prior into an efficient joint-token architecture. MAESTRO establishes new state-of-the-art performance on tasks reliant on temporal dynamics, outperforming prior models by +2.7% weighted F1 score on the TreeSatAI-TS dataset, demonstrating the superiority of its early temporal fusion strategy over the late fusion used by existing foundation models. For AI practitioners, the principal implication is that for multi-sensor time-series data, performance is maximized by employing early fusion for temporal steps and similar modalities while using separate parameters for dissimilar modalities, and that patch-group-wise normalization offers a computationally efficient method to improve multispectral representation learning. |
Papers for 2025-08-15
| Title | Authors | Summary |
|——-|———|———|
| We-Math 2.0: A Versatile MathBook System for Incentivizing Visual
Mathematical Reasoning (Read more on arXiv or HuggingFace)| Xiaowan Wang, Yanzi Wang, Peiqing Yang, Qiuna Tan, Runqi Qiao | WE-MATH 2.0 is a unified system integrating a hierarchical knowledge base, model-centric datasets, and a reinforcement learning paradigm to enhance the visual mathematical reasoning of Multimodal Large Language Models (MLLMs). The objective is to overcome MLLM deficiencies in complex mathematical reasoning by developing a comprehensive, knowledge-driven system rather than focusing solely on dataset construction or method optimization. The methodology involves creating a five-level “MathBook Knowledge System” with 491 knowledge points, generating “MathBook-Standard” and “MathBook-Pro” datasets using a three-dimensional difficulty space, and training models with “MathBook-RL,” a two-stage framework combining cold-start fine-tuning and progressive alignment reinforcement learning. The resulting MathBook-7B model, trained on only 9.8K samples, achieves a 48.7% average score across four standard benchmarks, outperforming its Qwen2.5-VL-7B backbone (42.6%). For AI practitioners, this research demonstrates that structuring training data around a formal knowledge system and applying a curriculum-based RL strategy can yield significant performance gains in specialized reasoning tasks using substantially less data, offering an efficient alternative to training on massive, unstructured datasets. |
| NextStep-1: Toward Autoregressive Image Generation with Continuous
Tokens at Scale (Read more on arXiv or HuggingFace)| Quan Sun, Jingwei Wu, Guopeng Li, Chunrui Han, NextStep Team | NextStep-1 is a 14B parameter autoregressive model that generates high-fidelity images by directly predicting a sequence of continuous, rather than discrete, image tokens. The primary objective is to close the performance gap between autoregressive and diffusion-based text-to-image models by avoiding the quantization loss associated with vector quantization (VQ). Its methodology combines a large causal transformer for next-token prediction with a lightweight (157M) flow matching head that samples continuous image patches from noise, conditioned on the transformer’s output. The model achieves state-of-the-art performance for an autoregressive architecture, scoring 85.28 on DPG-Bench, which evaluates complex, multi-object compositional fidelity. For AI practitioners, the key implication is that the stability and performance of continuous-token AR models are critically dependent on the image tokenizer’s design, specifically its use of channel-wise normalization and noise regularization to create a well-conditioned latent space that enables high-guidance generation without artifacts. |
| ToonComposer: Streamlining Cartoon Production with Generative
Post-Keyframing (Read more on arXiv or HuggingFace)| Xiaoyu Li, Yaowei Li, Zhaoyang Zhang, Guangzhi Wang, Lingen Li | ToonComposer is a DiT-based generative model that unifies cartoon inbetweening and colorization into a single post-keyframing stage using sparse keyframe sketches. The primary objective is to develop a unified model that automates cartoon production from sparse inputs, overcoming the error accumulation and high labor costs of separate inbetweening and colorization stages. The methodology is based on a Diffusion Transformer (DiT) video foundation model, enhanced with a sparse sketch injection mechanism for precise temporal control and a novel Spatial Low-Rank Adapter (SLRA) to adapt the model’s spatial behavior to the cartoon domain while preserving its temporal priors. On a synthetic benchmark, ToonComposer significantly outperformed prior methods, achieving a DISTS score of 0.0926 compared to the next-best score of 0.5461 from AniDoc; in human evaluations, it was preferred for aesthetic quality in 70.99% of cases. The principal implication for AI practitioners is that the SLRA method provides a targeted adaptation technique for video foundation models that selectively modifies spatial representations while leaving temporal dynamics intact, demonstrating a more effective approach than generic adapters for tasks requiring preservation of motion priors. |
| UI-Venus Technical Report: Building High-performance UI Agents with RFT (Read more on arXiv or HuggingFace)| Shuheng Shen, Xingran Zhou, Zhenyu Xu, Zhengwen Zeng, Zhangxuan Gu | UI-Venus is a native UI agent that achieves state-of-the-art (SOTA) performance on UI grounding and navigation tasks using only screenshots as input. The primary objective is to build a high-performance UI agent by applying Reinforcement Finetune (RFT) to a multimodal large language model, demonstrating its superiority over traditional Supervised Fine-Tuning (SFT). The methodology involves using the Group Relative Policy Optimization (GRPO) algorithm for RFT on the Qwen2.5-VL model, coupled with comprehensive data cleaning strategies and a novel “Self-Evolving Trajectory History Alignment & Sparse Action Enhancement” framework for navigation. The 72B variant of UI-Venus achieves a 65.9% success rate on the AndroidWorld navigation benchmark and 95.3% / 61.9% accuracy on the Screenspot-V2 / Pro grounding benchmarks, respectively. For AI practitioners, this work validates that RFT with high-quality curated data and self-evolving frameworks is a potent strategy for developing SOTA UI agents, particularly for complex, discriminative tasks where SFT is less effective. |
| PRELUDE: A Benchmark Designed to Require Global Comprehension and
Reasoning over Long Contexts (Read more on arXiv or HuggingFace)| Rui Lu, Tong Li, Chulun Zhou, Tsz Ting Chung, Mo Yu | This paper introduces PRELUDE, a benchmark for evaluating long-context reasoning in LLMs by assessing the consistency of character prequels with canonical book narratives. The main objective is to create a benchmark that requires global comprehension and deep, multi-step reasoning, addressing key shortcuts like memorization and summarization present in prior benchmarks. The methodology involves creating a dataset of 795 instances where LLMs must classify a generated character prequel as “consistent” or “contradict” with an entire book; evaluations are conducted using few-shot ICL, Retrieval-Augmented Generation (RAG), and in-domain training on state-of-the-art LLMs. The primary result demonstrates a significant performance deficit in current models, with the best-performing LLM lagging behind human performance by over 15% in F1 score, and a further human study revealing an over 30% gap in reasoning accuracy even for correctly answered instances. The principal implication for AI practitioners is that current long-context evaluation metrics focusing on answer accuracy can be misleading, as models often achieve correct results through flawed reasoning; this indicates that advanced techniques like RAG do not fully resolve fundamental limitations in deep, global reasoning, necessitating a shift in focus towards improving the intrinsic inferential capabilities of models. |
| STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer (Read more on arXiv or HuggingFace)| Honghua Chen, Shangchen Zhou, Fangzhou Hong, Yihang Luo, Yushi Lan | STREAM3R is a decoder-only causal Transformer framework for scalable, sequential 3D reconstruction from streaming images. The main objective is to perform efficient, online, and incremental 3D reconstruction that scales to long image sequences, avoiding the computational cost of global optimization and the limitations of RNN-based memory. The methodology reformulates 3D reconstruction as a sequential registration task, where a causal Transformer processes incoming frames by attending to a cache of features from all previously observed frames, inspired by modern LLM architectures. The primary result is superior performance on standard benchmarks; on the 7-Scenes dataset, STREAM3R achieves a mean reconstruction accuracy of 0.122, outperforming prior streaming and optimization-based methods. For AI practitioners, the principal implication is that LLM-style causal attention and training infrastructure can be directly adapted for efficient, real-time 3D perception from streaming video, offering a scalable approach for applications like robotics and autonomous systems. |
| Pass@k Training for Adaptively Balancing Exploration and Exploitation of
Large Reasoning Models (Read more on arXiv or HuggingFace)| Qinghao Ye, Yue Ling, Youbin Wu, Xiaobo Qin, Zhipeng Chen | This paper introduces Pass@k Training, a reinforcement learning with verifiable rewards (RLVR) method using an analytical advantage function to enhance LLM exploration and improve final reasoning performance. The research objective is to address the poor exploration-exploitation balance in standard Pass@1-based RLVR, which leads to models becoming trapped in local optima, by using Pass@k as a reward signal to promote more diverse solution generation. The key methodology is a novel training paradigm that uses the Pass@k metric as the reward, for which the authors derive a computationally efficient, closed-form analytical solution for the advantage function, eliminating the variance and overhead of sampling-based approaches. The primary result is that a two-stage process—Pass@k Training followed by Pass@1 Training—significantly boosts performance; on the Enigmata benchmark, this method improved a Qwen2.5-7B model’s overall Pass@1 score from a baseline of 4.7% to 30.8%, outperforming Claude-3.7-Sonnet (22.7%). The principal implication for AI practitioners is that they can train stronger reasoning models by first employing Pass@k Training to broaden a model’s exploration capabilities and then fine-tuning with Pass@1 Training to distill those gains into superior final accuracy. |
| HumanSense: From Multimodal Perception to Empathetic Context-Aware
Responses through Reasoning MLLMs (Read more on arXiv or HuggingFace)| Yi Yuan, Tianqi Li, Yabing Wang, Ruobing Zheng, Zheng Qin | The paper introduces HumanSense, a comprehensive benchmark for evaluating the human-centered perception and interaction capabilities of Multimodal Large Language Models (MLLMs). The main objective is to systematically assess and improve MLLM capabilities in understanding complex human intentions and generating empathetic, context-aware responses, addressing the lack of fine-grained evaluation frameworks for such scenarios. The authors created a 15-task, four-tier benchmark and applied a multi-stage, omni-modal reinforcement learning strategy to enhance a model’s reasoning capabilities across visual, auditory, and textual inputs. Evaluation results reveal a significant performance gap between the human baseline (87.5% accuracy) and top MLLMs, though the authors’ reinforcement learning method improved accuracy on the complex Psychological Chat task from 0.399 to 0.619. The principal implication for AI practitioners is that high-level reasoning is the primary bottleneck for MLLMs in human-centered interaction, and performance can be substantially improved by leveraging omni-modal inputs and employing reasoning-focused training or prompt engineering. |
| A Survey on Diffusion Language Models (Read more on arXiv or HuggingFace)| Zhiqiang Shen, Bowei Guo, Mingda Chen, Tianyi Li | This paper provides a comprehensive survey of Diffusion Language Models (DLMs), detailing their principles, training, inference, and applications as a parallelizable alternative to autoregressive models. Its objective is to establish a systematic taxonomy of the DLM landscape, reviewing foundational concepts, state-of-the-art models, and multimodal extensions. The key methodology is a structured literature review that classifies DLMs by their diffusion space (continuous vs. discrete), training strategies (e.g., pre-training, RL alignment), and inference optimizations (e.g., parallel decoding). Primary results show that scaled discrete DLMs achieve performance competitive with similarly-sized AR models, and post-training methods like DCoLT can significantly boost reasoning capabilities, yielding a +9.8% gain on the GSM8K benchmark. The principal implication for practitioners is that DLMs are a compelling alternative for high-throughput, low-latency generation tasks, warranting consideration in system design where parallel inference is critical. |
| From Black Box to Transparency: Enhancing Automated Interpreting
Assessment with Explainable AI in College Classrooms (Read more on arXiv or HuggingFace)| Ziyin Zhang, Zhaokun Jiang | This paper presents an explainable AI framework for automated, multi-dimensional assessment of English-Chinese interpreting performance. The research aims to overcome data scarcity and model opacity in interpreting assessment by developing a transparent system that can predict quality across fidelity, fluency, and language use dimensions. The methodology combines feature engineering, data augmentation using a Variational Autoencoder (VAE) to expand the dataset from 117 to 500 samples, and post-hoc explanation using Shapley Additive exPlanations (SHAP) on XGBoost and Random Forest models. The primary result is that VAE-based augmentation significantly improved model performance, and SHAP analysis identified key predictive features; for fidelity, the neural metric BLEURT was the most important predictor with a mean SHAP value of 0.32, while for fluency, pause-related features were most influential. The principal implication for AI practitioners is that combining generative data augmentation (VAE) with post-hoc explainability (SHAP) offers a powerful pipeline for developing accurate and trustworthy models in data-scarce, high-stakes domains, transforming predictive systems into actionable diagnostic tools. |
| Processing and acquisition traces in visual encoders: What does CLIP
know about your camera? (Read more on arXiv or HuggingFace)| Giorgos Tolias, Yuta Nakashima, Giorgos Kordopatis-Zilos, Vladan Stojnić, Ryan Ramos | This paper demonstrates that visual encoders, particularly Contrastive Vision-Language (CVL) models, systematically encode subtle image processing and acquisition metadata, which can disrupt semantic understanding. The research aims to determine if these metadata “traces” are embedded in visual representations and how they impact downstream semantic tasks. The methodology involves training linear classifiers to predict metadata labels from frozen embeddings of 47 visual encoders and evaluating performance on retrieval and classification tasks where metadata and semantic labels are deliberately correlated or anti-correlated. The primary result is that these traces are strongly encoded, especially in CVL models which can predict processing parameters like JPEG compression with over 80% accuracy, and this can overshadow semantic content. The principal implication for AI practitioners is that foundational models may exhibit biases towards non-semantic artifacts, leading to unreliable performance and spurious correlations in real-world applications where data acquisition and processing pipelines vary. |
| When Explainability Meets Privacy: An Investigation at the Intersection
of Post-hoc Explainability and Differential Privacy in the Context of Natural
Language Processing (Read more on arXiv or HuggingFace)| Gjergji Kasneci, Florian Matthes, Ege Erdogan, Stephen Meisenbacher, Mahdi Dhaini | This paper empirically investigates the trade-off between post-hoc explainability and differentially private (DP) text rewriting in Natural Language Processing. The central objective is to quantify the impact of applying local DP text rewriting on the post-hoc explainability faithfulness of fine-tuned language models. The methodology involves applying three DP text rewriting methods (TEM, DP-PROMPT, DP-BART) to three text classification datasets, fine-tuning five encoder-only PLMs, and evaluating four feature attribution methods using a composite score that balances model utility (F1) and explanation faithfulness (AOPC metrics). A primary result is that smaller base models consistently outperform larger models under DP constraints, with the composite score for large models dropping by as much as -0.286 compared to base models on the SST2 dataset. The principal implication for AI practitioners is that for privacy-sensitive applications requiring explainability, using the smallest acceptable pretrained model is preferable, as larger models exhibit a more significant degradation in both performance and explanation quality. |
Papers for 2025-08-14
| Title |
Authors |
Summary |
| Mol-R1: Towards Explicit Long-CoT Reasoning in Molecule Discovery (Read more on arXiv or HuggingFace) |
Di Zhang, Junxian Li, Qinggang Zhang, Weida Wang, Jiatong Li |
Mol-R1 is a novel framework enhancing explicit Long Chain-of-Thought (CoT) reasoning for text-based molecule generation. It aims to improve explainability and reasoning performance of R1-like LLMs by efficiently generating high-quality, expert-aligned reasoning traces and leveraging them for stable training. The methodology involves Prior Regulation via In-context Distillation (PRID) for cold-start dataset curation with human-labeled examples, followed by Molecular Iterative Adaptation (MoIA) which iteratively combines Supervised Fine-tuning (SFT) and Reinforced Policy Optimization (RPO). Mol-R1 (T=2) achieved a 0.234 Exact Match (EM) score and a Consistent-F1 score of 0.847 for reasoning trace quality, significantly outperforming QWQ-32B (0.518) and DeepSeek-R1 (0.522) in trace quality. This approach demonstrates significant potential for enabling more explainable and chemist-like reasoning in molecule discovery, addressing limitations of existing LLMs in knowledge-intensive domains. |
| Stand-In: A Lightweight and Plug-and-Play Identity Control for Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Chen Li, Hao Liu, Wenjing Wang, Qixin Yan, Bowen Xue |
Stand-In introduces a lightweight and plug-and-play framework for identity-preserving video generation. The research aims to generate high-fidelity videos that consistently maintain the identity from a given reference image, addressing limitations of existing methods regarding excessive parameters and limited compatibility. This is achieved by incorporating a conditional image branch into a pre-trained video generation model, utilizing restricted self-attention with conditional position mapping (3D ROPE), and leveraging the model’s inherent VAE for feature extraction. Despite adding only ~1% additional parameters (153M for the 14B model), Stand-In achieves state-of-the-art performance, with a Face Similarity score of 0.724 and Naturalness of 3.922. The framework’s lightweight and plug-and-play design enables seamless integration into various applications like subject-driven generation, video stylization, and face swapping, providing significant value for AI practitioners. |
| AWorld: Dynamic Multi-Agent System with Stable Maneuvering for Robust |
|
|
| GAIA Problem Solving (Read more on arXiv or HuggingFace) |
Jinjie Gu, Chenyi Zhuang, Chengyue Yu, Qintong Wu, Zhitian Xie |
AWorld introduces a robust dynamic Multi-Agent System with stable maneuvering for enhanced accuracy and stability in complex tool-augmented problem-solving. The research aims to enhance the stability and accuracy of intelligent agent-based systems when leveraging diverse external tools, addressing challenges like extended contexts and noisy tool outputs. The authors developed a dynamic Multi-Agent System (MAS) within the AWorld framework, incorporating dynamic supervision and maneuvering mechanisms where an Execution Agent collaborates with a Guard Agent. Experiments on the GAIA test dataset showed that the dynamic MAS improved pass@1 accuracy to 67.89%, an 8.82% gain over the Single Agent System (SAS), and reduced the pass@1 standard deviation to 0.027, a 17.3% reduction compared to the SAS. This work demonstrates the practical value of collaborative, dynamically supervised multi-agent systems for developing more reliable, trustworthy, and performant AI solutions, particularly in tool-augmented problem-solving scenarios. |
| Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion |
|
|
| Forcing (Read more on arXiv or HuggingFace) |
Hao Zhang, Jiachun Jin, Yijie Jin, Chenkai Xu, Xu Wang |
This paper introduces Discrete Diffusion Forcing (D2F), a novel paradigm enabling Diffusion Large Language Models (dLLMs) to achieve faster-than-autoregressive (AR) inference. The primary objective is to overcome the inference speed limitations of existing open-source dLLMs compared to AR models. D2F employs block-wise autoregressive generation for KV cache utilization and predicts future tokens without requiring completion of prior blocks, implemented via asymmetric distillation and a pipelined parallel decoding algorithm. Empirically, D2F dLLMs achieve over 2.5x inference speed compared to LLaMA3 and Qwen2.5 on GSM8K, and more than 50x acceleration over vanilla dLLMs like LLaDA and Dream. This breakthrough establishes dLLMs as a significantly more efficient and scalable alternative for high-throughput text generation tasks, offering direct benefits for AI practitioners in deployment scenarios. |
| Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved |
|
|
| Image Generation (Read more on arXiv or HuggingFace) |
Zhenghao Hu, Leqi Zhu, Zihao Wang, Dongzhi Jiang, Junyan Ye |
The paper presents Echo-4o-Image, a 180K-sample synthetic dataset from GPT-4o, and the Echo-4o model, which is fine-tuned on this data to enhance open-source image generation for complex, surreal, and multi-reference tasks. The primary objective is to demonstrate that targeted synthetic data from advanced models like GPT-4o can overcome the limitations of real-world datasets (e.g., lack of surreal/multi-reference examples, poor instruction alignment) to significantly improve the capabilities of open-source image generation models. The authors curated the Echo-4o-Image dataset using GPT-4o to generate three types of data: surreal fantasy, multi-reference, and complex instruction-following images. They then fine-tuned the Bagel multimodal model on this dataset to create Echo-4o and introduced two new benchmarks, GenEval++ and Imagine-Bench, which use GPT-4.1 for more challenging evaluation. The Echo-4o model demonstrates superior performance across multiple benchmarks; on the newly proposed and more difficult GenEval++ benchmark, Echo-4o achieved an overall instruction-following score of 0.679, significantly outperforming the baseline Bagel model’s score of 0.371. The principal implication for AI practitioners is that targeted, high-quality synthetic data, such as the open-sourced Echo-4o-Image dataset, can be used to fine-tune existing foundation models and significantly boost performance on complex, long-tail generation tasks underrepresented in real-world data. The most impactful finding is the model’s enhanced ability to follow complex, compositional instructions, showing this synthetic data approach is highly effective for teaching nuanced generative capabilities. |
| Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with |
|
|
| Long-Term Memory (Read more on arXiv or HuggingFace) |
Yuan Lin, Yiyuan Pan, Wentao Ye, Yichen He, Lin Long |
M3-Agent is a novel multimodal agent framework that leverages long-term memory for continuous perception, knowledge building, and reasoning. The core objective is to enable multimodal agents to autonomously process real-time visual and auditory inputs, build entity-centric episodic and semantic memories, and reason iteratively over this accumulated knowledge to accomplish tasks. The framework employs parallel memorization and control processes, utilizing a multimodal graph for memory organization and reinforcement learning (DAPO) for multi-turn reasoning and iterative memory retrieval, evaluated on a new M3-Bench long-video QA benchmark. Experimental results demonstrate M3-Agent’s superior performance, outperforming the strongest baseline by 6.7%, 7.7%, and 5.3% higher accuracy on M3-Bench-robot, M3-Bench-web, and VideoMME-long benchmarks respectively, with semantic memory being critical for performance. This work provides insights into the practical design of multimodal agents, highlighting the importance of reinforcement learning and robust semantic memory for achieving human-like long-term memory and memory-based reasoning capabilities for AI practitioners developing real-world agents. |
| Learning to Align, Aligning to Learn: A Unified Approach for |
|
|
| Self-Optimized Alignment (Read more on arXiv or HuggingFace) |
Lei Fan, Shuowen Zhang, Zhiling Ye, Yun Yue, Haowen Wang |
GRAO is a unified framework for self-optimized LLM alignment, addressing supervised fine-tuning’s offline policy limitations and reinforcement learning’s sample inefficiency and base model dependency. It synergizes these approaches through a multi-sample generation strategy with reward feedback, a novel Group Direct Alignment Loss utilizing intra-group relative advantage weighting, and reference-aware parameter updates guided by pairwise preference dynamics. Comprehensive evaluations demonstrate GRAO’s superior performance, achieving 57.70%, 17.65%, 7.95%, and 5.18% relative improvements over SFT, DPO, PPO, and GRPO baselines respectively. For instance, GRAO achieved a 67.98% Normalized Alignment Gain on Qwen2.5-7B for helpful alignment tasks. This framework provides a theoretically grounded alignment method and empirical evidence for efficient capability evolution in language models, offering a robust and scalable solution for AI practitioners to align diverse model architectures. |
| Story2Board: A Training-Free Approach for Expressive Storyboard |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Dani Lischinski, Dvir Samuel, Omri Avrahami, Matan Levy, David Dinkevich |
Story2Board presents a training-free framework for expressive storyboard generation from natural language, aiming to produce visually coherent and narratively compelling sequences with dynamic compositions and consistent character identity. The core methodology involves Latent Panel Anchoring (LPA) and Reciprocal Attention Value Mixing (RAVM), which guide pre-trained text-to-image diffusion models by preserving shared character references and softly blending visual features between semantically aligned tokens across panels, without architectural changes or fine-tuning. On the Rich Storyboard Benchmark, Story2Board consistently outperforms baselines, achieving a DreamSim score of 0.7018 on the DS-500 benchmark for identity consistency, surpassing DreamStory (0.6714), and leading in overall user preference across diverse narrative settings. This approach provides AI practitioners with a flexible and efficient means to generate high-quality, dynamic storyboards for visual storytelling applications, leveraging existing diffusion models without the overhead of model fine-tuning. |
| MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math |
|
|
| Reasoning in Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Zhihan Zhou, Yue Guo, Zhentao Zhang, Zixin Wang, junfeng0288 |
MATHREAL is a new real-world benchmark for evaluating Multimodal Large Language Models (MLLMs) on K-12 math reasoning from naturally captured images. The primary objective is to assess MLLMs’ reasoning capabilities under realistic visual conditions, accounting for image quality degradation, perspective variation, and irrelevant content interference. The methodology involves collecting 2,000 high-quality K-12 math questions with mobile-captured images, systematically annotating them across 14 fine-grained real-world scenario subcategories and five core knowledge categories, and evaluating MLLMs using six experimental input settings. Key results show that the best-performing model, Doubao-1.5-thinking-vision-pro, achieved only 53.9% accuracy, and a notable performance gap exists between real and clean image inputs for existing MLLMs. This underscores the critical need for AI practitioners to develop more robust visual encoders for MLLMs to handle realistic distortions and achieve reliable performance in real-world educational scenarios. |
| Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning |
|
|
| for Large Language Models (Read more on arXiv or HuggingFace) |
Guiyang Hou, Xingyu Wu, Haitao Hong, tricktreat, yanyc |
Cooper is a reinforcement learning (RL) framework that co-optimizes policy and reward models for large language models (LLMs) to mitigate reward hacking and enhance reasoning capabilities. The main objective is to overcome the inherent limitations of static rule-based (lack robustness) and model-based (vulnerability to hacking) reward functions in LLM training. Cooper employs a two-stage training pipeline involving policy model optimization via Group Relative Policy Optimization (GRPO) with a reference-aware reward model, and continuous reward model refinement through contrastive learning using dynamically selected positive (rule-based) and negative (LLM-generated) samples. Quantitatively, Cooper achieved a 0.54% gain in average accuracy on Qwen2.5-1.5B-Instruct, with its VerifyRM-1.5B achieving 89.42% accuracy on VerifyBench, while static reward models suffered catastrophic performance degradation (e.g., 16% relative decrease). This demonstrates that dynamically updating reward model parameters during RL training is an effective strategy for AI practitioners to combat reward hacking and improve end-to-end RL performance in LLMs. |
| IAG: Input-aware Backdoor Attack on VLMs for Visual Grounding (Read more on arXiv or HuggingFace) |
Di Zhang, Beining Xu, Junxian Li |
IAG is a novel input-aware backdoor attack designed to manipulate Vision-Language Models (VLMs) for visual grounding tasks. The objective is to force VLMs to localize an attacker-specified target object in an image, irrespective of the user’s query, while maintaining stealthiness and normal outputs for clean samples. This is achieved by an adaptive trigger generator, a text-conditional U-Net, which embeds the target’s semantic information into the image, optimized with a reconstruction loss and trained jointly with the VLM using a combined language model loss. Empirical results show IAG achieved an ASR@0.5 of 66.7% on InternVL-2.5-8B for RefCoco (testA), with only a 1-3% accuracy decrease on clean samples, demonstrating its effectiveness and stealthiness. This work reveals a critical security vulnerability in VLM agents, where imperceptible, semantically potent triggers can hijack grounding behavior, underscoring the need for robust safeguards in VLM deployment. |
| Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models (Read more on arXiv or HuggingFace) |
Zeynep Akata, Nataniel Ruiz, Alexey Dosovitskiy, Shyamgopal Karthik, Luca Eyring |
Noise Hypernetworks (HyperNoise) enable amortizing the computational cost of test-time noise optimization in diffusion models into a one-time post-training stage. The research addresses the challenge of reducing high inference latency associated with test-time scaling methods in generative vision models, aiming to integrate test-time scaling knowledge into a neural network during training. HyperNoise replaces reward-guided test-time noise optimization with a lightweight noise hypernetwork (f_phi, implemented via LoRA) that learns to modulate initial Gaussian noise, optimizing a tractable noise-space objective (L_noise) to approximate a reward-tilted noise distribution. Experiments show HyperNoise recovers a substantial portion of quality gains from explicit test-time optimization at a fraction of the computational cost; for example, it improved SD-Turbo’s GenEval performance to 0.57 and SANA-Sprint’s to 0.75 (from 0.70 baseline), achieving the same performance as LLM-based prompt optimization while being 300x faster. This allows AI practitioners to achieve high-quality, reward-aligned generation with state-of-the-art distilled diffusion models with minimal added inference latency, making such capabilities practical for real-time applications. |
| VisCodex: Unified Multimodal Code Generation via Merging Vision and |
|
|
| Coding Models (Read more on arXiv or HuggingFace) |
Dongdong Zhang, Yixia Li, Xun Wu, Shaohan Huang, Lingjie Jiang |
VisCodex introduces a unified multimodal framework for code generation by merging vision and coding models using task vectors. The primary objective is to empower Multimodal Large Language Models (MLLMs) with robust code generation from multimodal inputs, addressing existing limitations. This is achieved by arithmetically merging the language backbone parameters of a vision-language model with a dedicated coding LLM via task vectors, alongside introducing the large-scale Multimodal Coding Dataset (MCD) and the InfiBench-V benchmark. VisCodex-8B demonstrates state-of-the-art performance among open-source MLLMs, achieving 11.0 pass@1 on the MMCode benchmark and approaching proprietary models like GPT-4o. This research provides AI practitioners with an efficient, cost-effective model merging strategy to enhance multimodal understanding and code generation, bypassing expensive retraining. |
| Can LLM-Generated Textual Explanations Enhance Model Classification |
|
|
| Performance? An Empirical Study (Read more on arXiv or HuggingFace) |
Gjergji Kasneci, Zineb Attaoui, Ege Erdogan, Juraj Vladika, Mahdi Dhaini |
The paper investigates the impact of LLM-generated textual explanations on the classification performance of PLMs and LLMs in Natural Language Inference (NLI) tasks. Its main objective was to determine how such explanations affect downstream predictive tasks. The methodology involved generating explanations using four LLMs in zero-shot and few-shot settings, evaluating them with NLG metrics and G-Eval, and then assessing their impact on four fine-tuned PLMs and three LLMs via zero-shot inference on e-SNLI and HealthFC datasets. Primary results indicate that LLM-generated explanations consistently improved PLM performance; for example, Llama3 zero-shot explanations improved averaged PLM accuracy on HealthFC by 0.060 over a no-explanation baseline, though they generally did not benefit LLMs used as classifiers. This work indicates a promising direction for AI practitioners to scalably augment NLP datasets with LLM-based explanations for PLM performance enhancement. |
| AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal |
|
|
| Imitation-Exploration Balance (Read more on arXiv or HuggingFace) |
Yong Li, Jie Feng, Lixuan He |
AMFT introduces a meta-gradient adaptive weight controller to dynamically balance Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) in single-stage LLM fine-tuning, addressing catastrophic forgetting and suboptimal trade-offs. The methodology reframes SFT and RL as complementary reward signals, learning the optimal balance parameter (μ) through meta-gradients on a long-term validation objective, regularized by policy entropy. AMFT consistently achieved new state-of-the-art, with 61.3% average accuracy on in-distribution math benchmarks and 63.3% on out-of-distribution general reasoning benchmarks. Additionally, it demonstrated superior sample efficiency, requiring ~15,840 RL rollouts to reach a target performance on General Points OOD, compared to >21,760 for sequential SFT→RL. AMFT provides AI practitioners a principled, stable, and sample-efficient paradigm for LLM alignment, fostering robust generalization by autonomously learning an effective training curriculum. |
Papers for 2025-08-13
| Title | Authors | Summary |
|——-|———|———|
| WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent (Read more on arXiv or HuggingFace)| zhaoyd, callanwu, zhzhen23, richardxp888, Ornamentt | This paper introduces WebWatcher, a multimodal agent designed for deep research tasks that require integrated vision-language reasoning and multi-tool interaction. The research objective is to develop an agent that overcomes the text-centric limitations of prior work by effectively reasoning over and synthesizing information from both visual and textual sources using external tools. The key methodology involves a three-stage training process: first, automatically generating tool-use trajectories with GPT-4o; second, using these for a supervised fine-tuning (SFT) cold start; and third, refining the agent’s policy via Group-Relative Policy Optimization (GRPO) reinforcement learning. The primary result shows that WebWatcher-32B achieves state-of-the-art performance, scoring 27.0% on the new challenging BrowseComp-VL benchmark, significantly outperforming proprietary RAG workflows (13.4%) and other open-source agents. For AI practitioners, the principal implication is that combining SFT on synthetic trajectories with subsequent RL refinement provides an effective framework for building agents that can execute complex, multi-step reasoning with tools, a necessary step for tackling real-world multimodal problems beyond simple retrieval-augmented generation. |
| Matrix-3D: Omnidirectional Explorable 3D World Generation (Read more on arXiv or HuggingFace)| Yuqi Li, Wenhang Ge, Zhongqi Yang, kangfei, dearamy | Matrix-3D is a framework for generating omnidirectional, explorable 3D worlds from a single image or text prompt by leveraging a trajectory-guided panoramic video diffusion model and subsequent 3D reconstruction. The main objective is to overcome the limited field-of-view in existing 3D world generation methods by creating wide-coverage, geometrically consistent, and fully explorable scenes from minimal user input. The methodology involves a trajectory-guided panoramic video diffusion model, conditioned on scene mesh renders, to generate geometrically consistent videos; these are then lifted to a 3D world using either a rapid feed-forward reconstruction model or a high-fidelity optimization-based pipeline, all trained on the newly introduced Matrix-Pano dataset of 116K annotated panoramic videos. The framework achieves state-of-the-art results, with the optimization-based 3D reconstruction pipeline attaining a PSNR of 27.62, significantly outperforming the prior ODGS baseline’s 22.04 PSNR, while the feed-forward variant reduces reconstruction time to just 10 seconds. For AI practitioners, Matrix-3D provides a validated framework and a large-scale annotated dataset to generate high-quality 3D virtual worlds for applications in embodied AI, simulation, and digital content creation, significantly lowering the barrier to producing explorable environments from simple prompts. |
| Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale
Asynchronous RL (Read more on arXiv or HuggingFace)| Chuyi He, Shusheng Xu, Minyang Xie, Wei Fu, Jiaxuan Gao | This paper introduces ASearcher, an open-source project enabling large-scale asynchronous RL training for long-horizon agentic search. It addresses key challenges in online RL for search agents: limited search turns and insufficient high-quality QA pairs. The methodology involves a fully asynchronous RL training system, which decouples trajectory execution from model updates to support extended turn limits (e.g., up to 128 turns/trajectory), and an LLM-based agent for autonomous generation of challenging QA datasets. Primary results demonstrate that ASearcher-Web-QwQ achieves substantial improvements, including 46.7% and 20.8% Avg@4 gains on xBench and GAIA respectively, and attains Avg@4 scores of 42.1 on xBench and 52.8 on GAIA, surpassing existing open-source 32B agents. This work provides a scalable training pipeline for AI practitioners to develop more capable LLM-based search agents that can perform complex, long-horizon tasks and handle real-world uncertainties. |
| CharacterShot: Controllable and Consistent 4D Character Animation (Read more on arXiv or HuggingFace)| Fei Shen, Yanhong Zeng, Wenran Liu, LiJiaxing, Gaojunyao | CharacterShot is a novel framework for controllable and consistent 4D character animation from a single reference image and a 2D pose sequence. The main objective is to democratize 4D character animation, enabling individual designers to create dynamic 3D characters with precise motion control without specialized hardware. CharacterShot’s methodology involves enhancing a DiT-based image-to-video model (CogVideoX) with pose conditions, extending it to multi-view generation via a dual-attention module and camera prior, and optimizing 4D representations using a novel neighbor-constrained 4D Gaussian Splatting (4DGS), supported by a new large-scale Character4D dataset. Extensive experiments on CharacterBench demonstrate CharacterShot’s SOTA performance; for instance, it achieved an LPIPS of 0.025 for 4D generation, significantly outperforming STAG4D (0.082). This framework offers AI practitioners a robust and efficient solution for generating high-quality, spatio-temporally and spatio-view consistent 4D character animations, thereby lowering the barrier for 3D content creation in various applications. |
| Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language
Models (Read more on arXiv or HuggingFace)| Chenchen Jing, Bozhen Fang, Wen Wang, qiuyuu, tricktreat | This paper exploits temporal dynamics in diffusion Large Language Models (dLLMs) to address temporal oscillation, where correct intermediate predictions are overwritten. The primary objective is to overcome this phenomenon in dLLMs, where correct answers often appear during intermediate denoising steps but are subsequently discarded for later, incorrect iterations. The paper proposes two complementary methods: Temporal Self-Consistency Voting, a training-free test-time strategy aggregating predictions via weighted voting, and Temporal Consistency Reinforcement, a post-training method using negative Temporal Semantic Entropy (TSE) as a self-supervised reward signal within a reinforcement learning framework. Temporal Self-Consistency Voting achieved an average improvement of 1.5% over the LLaDA-8B-Instruct baseline, and Temporal Consistency Reinforcement yielded absolute gains of 2.0% on GSM8K and 25.3% on Countdown when combined with accuracy reward. AI practitioners developing or deploying dLLMs can significantly improve model accuracy and stability by incorporating intermediate predictions, either through the proposed training-free voting strategy or by fine-tuning models with a temporal consistency-based reward signal. |
| HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating
Local and Web Searches (Read more on arXiv or HuggingFace)| Qiang Ju, Jiehan Cheng, Yan Yu, Zhicheng Dou, zstanjj | HierSearch is a hierarchical agentic deep search framework integrating local and Web knowledge sources using hierarchical reinforcement learning and a knowledge refiner. The work addresses challenges in enterprise deep search systems that require selective use and cross-supplementation of knowledge from both local private corpuses and the public Web. It employs a hierarchical agentic architecture with low-level local and Web deep search agents, coordinated by a high-level planner agent. The framework is trained using hierarchical reinforcement learning (HRL) with Group Relative Policy Optimization (GRPO) and rule-based rewards, augmented by a reasoning-aware knowledge refiner to filter irrelevant or hallucinated evidence. Experiments across six benchmarks (general, finance, medical) demonstrate that HierSearch consistently outperforms flat reinforcement learning solutions and various baselines; for instance, achieving an F1-score of 62.83 on MuSiQue, significantly higher than the R1-Searcher parallel search baseline’s 57.19. This approach offers a more data-efficient and stable method for developing robust deep search systems that can effectively integrate and reason over heterogeneous, noisy knowledge sources, crucial for real-world enterprise applications. |
| VertexRegen: Mesh Generation with Continuous Level of Detail (Read more on arXiv or HuggingFace)| Jakob Engel, Chris Xie, Armen Avetisyan, Yawar Siddiqui, zx1239856 | VertexRegen introduces a novel generative framework for producing 3D triangle meshes with a continuous, controllable level of detail. The paper’s objective is to enable “anytime” mesh generation, where the process can be halted at any step to yield a valid, complete mesh, unlike standard partial-to-complete methods. The key methodology reframes generation as the learned reversal of the edge collapse operation from progressive meshes, using a Transformer to autoregressively predict a sequence of vertex split operations that refine a coarse base mesh. Results demonstrate comparable quality to state-of-the-art models, achieving a superior Jensen-Shannon Divergence (JSD) of 2.89 in unconditional generation tasks. The principal implication for AI practitioners is the ability to dynamically control mesh complexity and generation time by simply stopping the generation process, which is highly valuable for real-time graphics, interactive content creation, and resource-constrained environments. |
| Test-Time Reinforcement Learning for GUI Grounding via Region
Consistency (Read more on arXiv or HuggingFace)| Zhengxi Lu, Fei Tang, tricktreat, yanyc, DIONG1024 | This paper introduces GUI-RC, a test-time scaling method, and GUI-RCPO, a test-time reinforcement learning approach, to enhance GUI grounding accuracy without requiring additional labeled data. The main objective is to leverage test-time computation to improve GUI grounding performance, addressing the limitations of existing train-time optimization methods that rely heavily on extensive labeled data. The core methodology for GUI-RC involves constructing spatial voting grids from multiple sampled predictions to identify consensus regions, while GUI-RCPO transforms these consistency patterns into self-supervised reward signals for test-time policy optimization using Group Relative Policy Optimization (GRPO). GUI-RC consistently improves grounding accuracy by 2-3% on average, boosting Qwen2.5-VL-3B-Instruct from 80.11% to 83.57% on ScreenSpot-v2, and GUI-RCPO further improves it to 85.14% on the same benchmark through self-supervised optimization. This approach reveals the untapped potential of test-time scaling and reinforcement learning for GUI grounding, offering a promising direction for AI practitioners to develop more robust and data-efficient autonomous GUI agents. |
| UNCAGE: Contrastive Attention Guidance for Masked Generative
Transformers in Text-to-Image Generation (Read more on arXiv or HuggingFace)| Kevin Galim, Minjae Lee, Byeongkeun Ahn, Wonjun Kang, JakeOh | UNCAGE introduces a training-free method to enhance compositional text-to-image generation in Masked Generative Transformers (MGTs). The primary objective is to address inaccurate attribute binding and improve text-image alignment in compositional T2I generation using MGTs. UNCAGE leverages attention maps to compute contrastive attention scores, which then guide the token unmasking order by prioritizing tokens that clearly represent individual objects. Quantitatively, UNCAGE (ours) achieved an average CLIP text-image similarity of 33.03, outperforming the Meissonic baseline (32.72), with a negligible inference overhead of 0.13% of total runtime. For AI practitioners, UNCAGE offers an efficient, training-free solution to improve compositional fidelity in existing MGT-based T2I systems without substantial computational cost or model retraining. |
| Aryabhata: An exam-focused language model for JEE Math (Read more on arXiv or HuggingFace)| Sandeep Varma, Sachin Dharashivkar, RitvikPW | The paper presents Aryabhata 1.0, a compact 7B parameter open-source model specialized for mathematical reasoning on India’s Joint Entrance Examination (JEE). The primary objective is to develop a model that achieves high accuracy and pedagogical value on domain-specific math problems while remaining computationally efficient. The methodology involves linearly merging three Qwen-based models, followed by supervised fine-tuning using curriculum learning on verified chain-of-thought traces, and reinforcement learning with verifiable rewards (RLVR) employing an A2C objective with adaptive exploration strategies. Aryabhata 1.0 achieves 90.2% accuracy on the JEE April session benchmark and 83.6% on the MATH 500 out-of-distribution benchmark, outperforming its base models. For AI practitioners, this research provides a blueprint for creating highly specialized, efficient, open-source models for niche domains by combining model merging with advanced fine-tuning and RL techniques, demonstrating a viable alternative to larger, general-purpose models for high-stakes applications. |
| Train Long, Think Short: Curriculum Learning for Efficient Reasoning (Read more on arXiv or HuggingFace)| Marzyeh Ghassemi, Elie Bou-Zeid, Abed Hammoud, Kumail Alhamoud, Hasan Abed Al Kader Hammoud | This paper introduces a curriculum learning framework using Group Relative Policy Optimization (GRPO) to train large language models for efficient, length-controlled reasoning. The main objective is to determine if a curriculum learning strategy, where token budgets gradually tighten during training, can enhance LLM reasoning capabilities and efficiency compared to fixed-budget approaches. The methodology involves fine-tuning QWEN-2.5-7B with GRPO, incorporating a reward function balancing task correctness, length efficiency (via a triangular reward and an exponentially decaying token budget), and formatting adherence through structural tags. Experiments show that curriculum learning consistently outperforms fixed-budget training; for instance, on GSM8K, accuracy improved from 82.71% to 86.20% with nearly identical token usage. AI practitioners can leverage curriculum-driven compression as a powerful inductive bias to train efficient reasoning models, enabling significant computational cost savings without runtime user hints or prompt overhead. |
| Towards Affordance-Aware Robotic Dexterous Grasping with Human-like
Priors (Read more on arXiv or HuggingFace)| Haoran Xu, Cheng Zeng, Xingyue Zhao, Linghao Zhuang, Haoyu Zhao | This paper introduces AffordDex, a two-stage framework for learning a universal dexterous grasping policy that is both human-like and functionally aware of object affordances. The objective is to develop a grasping policy that moves beyond simple stability metrics to incorporate human-like kinematics and an understanding of functionally inappropriate contact regions (negative affordances). The methodology first pre-trains a base policy on a human motion dataset, then fine-tunes it with a residual module using reinforcement learning, guided by a Negative Affordance-aware Segmentation (NAA) module that identifies unsafe regions. On the UniDexGrasp dataset (state-based, seen objects), AffordDex achieved an 89.2% success rate and an Affordance Score of 4, outperforming the UniDexGrasp++ baseline’s 87.9% success rate and Affordance Score of 28. For AI practitioners, the principal implication is that explicitly modeling and penalizing negative affordances, in tandem with learned human motion priors, is a powerful technique for developing robotic manipulation policies that are not only successful but also functionally correct and safe. |
| DeCRED: Decoder-Centric Regularization for Encoder-Decoder Based Speech
Recognition (Read more on arXiv or HuggingFace)| Lukáš Burget, Bolaji Yusuf, Karel Beneš, Santosh Kesiraju, Alexander Polok | This paper presents DeCRED, a regularization method that adds auxiliary classifiers to intermediate decoder layers in encoder-decoder ASR models to improve robustness and generalization. The research objective is to enhance the performance of these models, particularly in out-of-domain settings, by regularizing the decoder’s induced internal language model (ILM). The key methodology involves attaching linear classifiers to intermediate decoder layers and training them with the same cross-entropy loss as the final layer, thereby enforcing supervision deeper within the network. This approach reduces the macro Word Error Rate (WER) on a set of four out-of-domain datasets from 18.2% to 16.2% relative to the baseline model. For AI practitioners, DeCRED offers an efficient technique to improve ASR model robustness and generalization with negligible computational overhead during training and no additional cost at inference time, as the auxiliary layers are discarded after training. |
| Cut2Next: Generating Next Shot via In-Context Tuning (Read more on arXiv or HuggingFace)| Yu Qiao, Ziqi Huang, Jiajun Li, Hongbo Liu, Jingwen He | Cut2Next introduces Next Shot Generation (NSG) to synthesize cinematographically coherent subsequent video shots adhering to professional editing patterns. The primary objective is to generate highly coherent subsequent shots that maintain character and environmental consistency while adhering to established cinematic continuity principles and specific editing patterns. The framework leverages a Diffusion Transformer (DiT) (FLUX.1-dev) with in-context tuning, guided by a Hierarchical Multi-Prompting strategy (Relational and Individual Prompts), and incorporates architectural innovations like Context-Aware Condition Injection (CACI) and Hierarchical Attention Mask (HAM), trained on RawCuts and CuratedCuts datasets. Quantitatively, Cut2Next achieves superior performance on CutBench, with a DINO Similarity of 0.4952 and a Fréchet Inception Distance (FID) of 59.37, significantly outperforming the IC-LoRA-Cond baseline. This advancement enables AI practitioners to generate high-quality, narratively expressive, and cinematically coherent video sequences, providing a robust solution for automated content creation that meets professional video editing standards. |
| Democratizing Diplomacy: A Harness for Evaluating Any Large Language
Model on Full-Press Diplomacy (Read more on arXiv or HuggingFace)| Elizabeth Karpinski, Ishana Shastri, Samuel J Paech, tmarques, Alex-GSL | This paper presents a harness for evaluating the strategic reasoning of any off-the-shelf Large Language Model in the game of full-press Diplomacy without fine-tuning. The main objective is to create a standardized and accessible framework to measure emergent strategic, negotiation, and deceptive capabilities in LLMs within a complex, multi-agent environment, thereby democratizing this area of research. The methodology involves an optimized textual game state representation, a scalar “Game Score” for performance measurement, and a “Critical State Analysis” (CSA) protocol for efficiently replaying key game moments to test hypotheses on model behavior. Primary results show that performance scales with model size (game score correlates with Chatbot Arena Elo at r=+0.651) and that behavior is highly sensitive to prompt engineering; for example, aggressive prompting reduced Mistral-Small’s rate of passive “hold” orders from 58.9% to 24.1%. The principal implication for AI practitioners is that complex strategic behaviors can be elicited from general-purpose LLMs via context engineering alone, providing a framework to benchmark these capabilities while also revealing that models are vulnerable to manipulation and deception from other AI agents. |
| Adversarial Video Promotion Against Text-to-Video Retrieval (Read more on arXiv or HuggingFace)| Shuai Liu, Qian Li, Zhengyu Zhao, Chenhao Lin, michaeltqw108 | This paper introduces ViPro, the first adversarial attack designed to promote a video’s rank for multiple target queries within text-to-video retrieval (T2VR) systems. The research objective is to design and evaluate a novel attack paradigm that, unlike existing methods which suppress video ranks, adversarially promotes a target video’s ranking for multiple, semantically relevant text queries simultaneously. The proposed method, Video Promotion (ViPro), optimizes perturbations to push a video into the overlapping retrieval boundaries of target queries using an exponential loss function. For enhanced black-box transferability, a Modal Refinement (MoRe) module is introduced, which performs temporal clipping of frames and applies semantical weighting based on frame-to-query similarity to guide the optimization. ViPro demonstrates superior performance over adapted baselines in white-box, grey-box, and black-box settings; on average, it surpasses baselines by over 30%, 10%, and 4% respectively across these scenarios. The principal implication for AI practitioners is that T2VR systems are vulnerable to adversarial promotion attacks, which can manipulate content visibility for malicious purposes, highlighting a critical threat vector beyond traditional suppression attacks and indicating that simply obscuring model components is an insufficient defense. |
| OpenCUA: Open Foundations for Computer-Use Agents (Read more on arXiv or HuggingFace)| Tianbao Xie, Junlin Yang, Dunjie Lu, Bowen Wang, xywang626 | The paper presents OPENCUA, an open-source framework for building and evaluating computer-use agents (CUAs) by providing an annotation tool, a large-scale dataset (AGENTNET), and a novel training methodology. The main objective is to establish open foundations for CUA research by creating a scalable framework for data collection and model training, enabling the community to study agent capabilities, limitations, and risks. The key methodology involves: (1) capturing human demonstrations using the AGENTNET TOOL; (2) processing raw data into compact state-action pairs; and (3) synthesizing reflective long Chain-of-Thought (CoT) reasoning to augment the training data, which explicitly injects planning, memory, and reflection into the agent’s learning process. The primary result is that the OPENCUA-32B model achieves a 34.8% success rate on the OSWorld-Verified benchmark (100-step budget), establishing a new state-of-the-art for open-source models and surpassing the proprietary OpenAI CUA (31.4%). The principal implication for AI practitioners is that augmenting state-action demonstration data with synthesized, reflective long Chain-of-Thought reasoning is a critical factor for improving CUA performance and scalability, as simply increasing raw demonstration data yields minimal gains. |
| AutoCodeBench: Large Language Models are Automatic Code Benchmark
Generators (Read more on arXiv or HuggingFace)| Tao Zhang, Zhiying Zeng, Yuchi Deng, Ao Liu, Jason Chou | The paper introduces AutoCodeGen, a fully automated workflow using LLMs and a multilingual sandbox to generate AutoCodeBench, a large-scale, high-difficulty, multilingual code generation benchmark. The objective is to overcome the limitations of existing code benchmarks, such as reliance on manual annotation and a narrow focus on Python, by creating a more challenging and diverse evaluation standard. The AutoCodeGen methodology involves LLMs generating code solutions and test inputs, executing them in a sandbox to obtain outputs, reverse-generating problem descriptions, and applying a three-stage filtering process using difficulty control, an LLM-as-Critic, and diversity sampling. The resulting AutoCodeBench contains 3,920 problems across 20 languages, on which the top-performing model, Claude Opus 4 (Think), achieved a Pass@1 of only 52.4%, while manual verification confirmed an 87.6% accuracy rate for the benchmark data itself. The principal implication for AI practitioners is the availability of a scalable framework and a challenging, validated benchmark for assessing the practical multilingual and multi-logic reasoning capabilities of code generation models, revealing significant performance drops on complex, multi-component tasks. |
| Feedback-Driven Tool-Use Improvements in Large Language Models via
Automated Build Environments (Read more on arXiv or HuggingFace)| Xuesong Yao, Yufei Xu, Zhengyin Du, Changhao Jiang, Junjie-Ye | This paper introduces a feedback-driven framework to enhance large language model tool-use capabilities via automated build environments. The objective is to address limitations in current reinforcement learning frameworks for LLM tool-use, particularly regarding stable training environments and verifiable reward signals. The methodology involves a five-stage automated environment construction pipeline and a verifiable reward mechanism that assesses both tool precision and task completeness. Experiments demonstrate that this approach significantly improves LLM tool-use performance, yielding over a 10% average performance gain on open-source LLMs across multiple benchmarks, with Qwen2.5-7B’s Solve-F1 improving from 25.97 to 40.36 on the Ours dataset. This provides AI practitioners with a scalable, stable, and verifiable framework for training LLMs to robustly generalize tool-use abilities by enhancing lower-layer MLP parameters. |
| Bridging Theory and Practice in Quantum Game Theory: Optimized
Implementation of the Battle of the Sexes with Error Mitigation on NISQ
Hardware (Read more on arXiv or HuggingFace)| Jhon Alejandro Andrade, Mateo Buenaventura Samboni, Carlos Andres Duran Paredes, Germán Díaz Agreda, sebasmos | This paper reports the experimental realization of the quantum “Battle of the Sexes” game on an IBM NISQ processor to validate if its theoretical strategic advantages persist on noisy hardware. The researchers implemented four quantum strategies under the Eisert-Wilkens-Lewenstein framework and introduced a Guided Circuit Mapping (GCM) method for dynamic, noise-aware qubit allocation to mitigate hardware errors across 62 qubits. The GCM-optimized execution successfully preserved the theoretical payoff trends, with experimental results deviating from the analytical model by a relative error between 3.5% and 12.1%. For AI practitioners, this demonstrates that quantum-enhanced coordination in multi-agent systems is achievable on near-term hardware, as lightweight error mitigation techniques can maintain a quantifiable quantum advantage in strategic decision-making scenarios. |
| BiasGym: Fantastic Biases and How to Find (and Remove) Them (Read more on arXiv or HuggingFace)| Arnav Arora, Haeun Yu, Siddhesh Milind Pawar, Nadav Borenstein, sekhcopenlu | This paper introduces BiasGym, a framework for injecting, analyzing, and mitigating conceptual biases in LLMs by fine-tuning a special token and then steering the attention heads associated with it. The primary objective is to develop a cost-effective and targeted framework for reliably surfacing, analyzing, and removing specific conceptual biases from LLM weights without degrading general downstream task performance. The methodology consists of two components: 1) BiasInject, which introduces a bias by fine-tuning only the embedding of a new, special token (BiasToken) while keeping model weights frozen; and 2) BiasScope, which identifies the attention heads most associated with the BiasToken via head attribution and mitigates the bias by nullifying the output of these heads. The proposed method effectively reduces stereotypes across multiple models; for Llama3.2-3B, the “Injection w/ steering” approach reduced the average stereotype strength score from 1.16 (original model) to 0.40. This mitigation resulted in minimal impact on general capabilities, with an average MMLU performance degradation of only 0.03. The principal implication for AI practitioners is that BiasGym provides a practical, low-cost technique for targeted debiasing of open-weight LLMs, allowing engineers to surgically remove specific unwanted associations from a model’s internal mechanisms as a precise alternative to broad safety fine-tuning. |
| WGAST: Weakly-Supervised Generative Network for Daily 10 m Land Surface
Temperature Estimation via Spatio-Temporal Fusion (Read more on arXiv or HuggingFace)| Rachid Nedjai, Raphael Canals, Adel Hafiane, sofianebouaziz | WGAST is a weakly-supervised conditional generative adversarial network that performs spatio-temporal fusion of multi-source satellite data to estimate daily 10 m resolution Land Surface Temperature (LST). The main objective is to generate these high-resolution, high-frequency LST maps by fusing coarse-resolution daily MODIS data with higher-resolution spectral data from Landsat 8 and Sentinel-2, overcoming the inherent spatio-temporal trade-off in satellite imagery. The methodology employs a cGAN with a four-stage generator and a PatchGAN discriminator, trained via a weakly-supervised strategy that uses 30 m Landsat LST as a proxy ground truth by spatially averaging the 10 m generated output for loss calculation. WGAST quantitatively outperformed baseline models, achieving an average Root Mean Square Error (RMSE) reduction of 17.18% against the best-performing baseline, FuseTen. For AI practitioners, the key implication is the use of a physically-motivated weak supervision technique where a high-resolution model is trained against a lower-resolution proxy label via aggregation, a method applicable to other multi-resolution data fusion problems where high-resolution ground truth is unavailable. |
| GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via
General Samples Replay (Read more on arXiv or HuggingFace)| Yang Fan, Yuefeng Li, Mengchen Zhao, Shuoran Jiang, Yunan Zhang | The paper presents GeRe, a framework for efficient anti-forgetting in large language model (LLM) continual learning. Its objective is to simultaneously retain general LLM capabilities and improve performance on previously learned tasks across sequential tasks, specifically addressing if fixed general replay samples suffice and if task-specific replay is necessary. GeRe utilizes a fixed set of pre-collected general pretraining texts as replay samples and introduces a threshold-based margin (TM) loss, derived from distilled last-layer hidden state activation thresholds, to maintain consistent neuron activation states. Experimental results on 15 downstream tasks with Llama-3.1-8B demonstrate that GeRe with dynamic TM loss (full-parameter) achieves an F1 Average performance of 66.9386, significantly outperforming a non-replay baseline (37.8919 F1 Avg). This approach simplifies continual learning by showing that a small, fixed set of general replay samples is sufficient for anti-forgetting and performance enhancement, reducing the laborious collection of task-specific replay samples for AI practitioners. |
| NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech
Modeling with Paralinguistic Vocalizations (Read more on arXiv or HuggingFace)| Haoyue Zhan, Yiheng Lu, Yuancheng Wang, Qinke Ni, Huan Liao | i) NVSpeech introduces an integrated and scalable pipeline for modeling human-like speech with paralinguistic vocalizations. ii) The primary objective is to enable automatic speech recognition (ASR) and text-to-speech (TTS) systems to jointly process both lexical content and fine-grained, word-level non-verbal cues like laughter or interjections. iii) The key methodology involves creating a manually annotated dataset with 18 paralinguistic categories, training a paralinguistic-aware ASR model to auto-label a larger 573-hour corpus, and finetuning zero-shot TTS models on the resulting data. iv) The paralinguistic-aware ASR model achieved an F1-score of 0.85 on open-domain event detection, and TTS models enhanced with the NVSpeech data were preferred by human listeners with a win rate of 78.7% over baseline models. v) For AI practitioners, this research provides a public, large-scale, word-level annotated dataset and a unified pipeline to develop more expressive ASR and TTS systems with explicit, token-level control over non-verbal vocalizations. |
Papers for 2025-08-12
| Title |
Authors |
Summary |
| ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (Read more on arXiv or HuggingFace) |
Yuchen Li, Yutao Zhu, Weiwei Sun, Xinyu Ma, Wenhan Liu |
The paper introduces ReasonRank, a reasoning-intensive passage reranker that achieves superior performance and efficiency by generating its own training data and using a two-stage training process. The primary objective is to empower listwise rerankers with strong reasoning capabilities to handle complex ranking tasks, addressing the scarcity of suitable training data. The key methodology involves an automated framework to synthesize reasoning-intensive training data using DeepSeek-R1, followed by a two-stage training approach combining Supervised Fine-Tuning (SFT) for pattern learning and Reinforcement Learning (RL) with a novel multi-view ranking reward (NDCG@10, Recall@10, RBO). ReasonRank (32B) achieves a state-of-the-art average NDCG@10 of 40.6 on the BRIGHT benchmark, and the 7B model is 2-2.7x faster than the pointwise reasoning reranker Rank1 (7B). The principal implication for AI practitioners is that this framework provides a blueprint for creating highly effective and efficient reasoning-based rerankers; by synthesizing specialized data and using a listwise reasoning approach, they can significantly improve ranking accuracy on complex queries while simultaneously reducing inference latency, making such models more viable for production systems. |
| WideSearch: Benchmarking Agentic Broad Info-Seeking (Read more on arXiv or HuggingFace) |
Yan Gao, Li Chen, Junjie Zhao, Jiawei Wang, Ryan Wong |
This paper introduces WideSearch, a benchmark for evaluating AI agents on large-scale, broad information-seeking tasks, revealing critical deficiencies in current state-of-the-art systems. The objective is to evaluate the reliability and completeness of LLM-powered agents on “wide-context” information collection tasks, which require gathering, verifying, and structuring a large volume of atomic facts from the web. The authors constructed the WideSearch benchmark with 200 tasks requiring agents to populate predefined tables using web search tools, and developed a hybrid automated evaluation pipeline to score submissions on table-level Success Rate (SR), row-level F1, and item-level F1 metrics. The primary result is that current systems fail at these tasks; across more than 10 tested systems, the best-performing multi-agent framework achieved an average Success Rate of only 5.1%, while a single human achieved 20%. A scaling analysis showed that even with 128 attempts, the item-level F1 score approached 80% while the SR remained below 20%, pinpointing the core difficulty as achieving perfect data completeness and accuracy, not finding individual facts. The principal implication for AI practitioners is that current agentic frameworks are unsuitable for reliable, large-scale data gathering due to fundamental flaws in planning, reflection, and evidence utilization. Development should prioritize more sophisticated architectures, particularly multi-agent systems capable of parallel search and cross-validation, as the benchmark demonstrates that these systemic deficiencies cannot be overcome by simply increasing compute or retries. |
| Omni-Effects: Unified and Spatially-Controllable Visual Effects |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Xiaokun Feng, Dongxia Liu, Jintao Chen, Aiming Hao, Fangyuan Mao |
Omni-Effects introduces a unified framework for generating multiple, simultaneous, and spatially-controllable visual effects (VFX) in videos using a diffusion-based model. The objective is to overcome the limitations of single-effect generation by enabling the concurrent synthesis of multiple, spatially-distinct VFX within a single video without cross-effect interference. The core methodology combines two innovations: a LoRA-based Mixture of Experts (LoRA-MoE) to partition diverse effects into specialized, collaboratively trained subspaces, and a Spatial-Aware Prompt (SAP) augmented with an Independent-Information Flow (IIF) attention mask to embed spatial control and isolate information flow between different conditions. In multi-VFX generation experiments, Omni-Effects achieved a 0.50 Effect Controllability Rate (ECR) for a simultaneous “Melt+Explode” task, significantly outperforming the CogV+CN baseline which scored 0.08. The principal implication for AI practitioners is that the LoRA-MoE and SAP-IIF architecture provides a robust method for building unified models for complex, multi-conditional generation tasks, demonstrating how to effectively manage and isolate multiple control signals to prevent interference and concept bleeding without requiring separate models for each condition. |
| Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving |
|
|
| Clipping Policy Optimization (Read more on arXiv or HuggingFace) |
Guanting Dong, Dening Liu, Xue Bai, Leiyu Pan, Zhenpeng Su |
Klear-Reasoner is an 8B-parameter model achieving state-of-the-art reasoning capabilities in math and coding via quality-centric SFT and Gradient-Preserving Clipping Policy Optimization (GPPO). The paper aims to advance reasoning capabilities and address limitations of existing RL clipping mechanisms, particularly regarding high-entropy token clipping and delayed convergence from negative samples. Klear-Reasoner employs a long Chain-of-Thought Supervised Fine-Tuning (SFT) strategy prioritizing high-quality data, followed by Reinforcement Learning (RL) with the proposed Gradient-Preserving Clipping Policy Optimization (GPPO) which backpropagates bounded gradients from clipped tokens, and integrates an SFT loss component, along with a soft reward mechanism for coding tasks. Klear-Reasoner-8B achieved 90.5% on AIME 2024 and 66.0% on LiveCodeBench V5, outperforming models like Qwen3-8B and DeepSeek-R1-0528-Distill-8B; ablations showed that for hard tasks, incorporating incorrect examples surprisingly improved performance. AI practitioners should prioritize data quality over surface-level diversity in SFT, consider incorporating mixed-correctness data for difficult tasks, and explore advanced RL clipping methods like GPPO, which preserve critical gradient information, for more stable and effective policy optimization in complex reasoning domains. |
| UserBench: An Interactive Gym Environment for User-Centric Agents (Read more on arXiv or HuggingFace) |
Jianguo Zhang, Zhiwei Liu, Akshara Prabhakar, Zuxin Liu, Cheng Qian |
This paper introduces UserBench, an interactive gym environment for evaluating an LLM agent’s ability to collaborate with users by handling underspecified, incremental, and indirect goals. The core methodology is a Gymnasium-based travel planning simulation where an agent must use tools and proactive dialogue to uncover a simulated user’s evolving, implicitly stated preferences. The primary result from evaluating leading LLMs is a severe disconnect between tool proficiency and user alignment; even the most advanced models actively elicited fewer than 30% of all user preferences. For AI practitioners, this demonstrates that proficiency in tool execution does not guarantee user satisfaction, highlighting a critical need to develop agents with communicative intelligence for proactive clarification and dynamic intent modeling. |
| SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings |
|
|
| and Speaks in Tokens (Read more on arXiv or HuggingFace) |
Anton Razzhigaev, Andrey Kuznetsov, Elizaveta Goncharova, Temurbek Rahmatullaev, Nikita Dragunov |
The paper introduces SONAR-LLM, a decoder-only Transformer that generates text by predicting sentence embeddings while being supervised by token-level cross-entropy propagated through a frozen decoder. The main objective is to create a sentence-level generative model that combines the semantic abstraction of concept-based models with the stable, likelihood-based training of traditional token-level LLMs. The key methodology involves autoregressively predicting a continuous SONAR sentence embedding, which is then passed through a fixed SONAR decoder to obtain token logits for a standard cross-entropy loss calculation, a process the authors term a “Token-Aware Embedding Objective.” Primary results show that SONAR-LLM outperforms existing sentence-level baselines on summarization, achieving a ROUGE-L score of 19.3 on XSum, which is competitive with or slightly better than a standard token-level LLM (18.7-18.9). The principal implication for AI practitioners is that SONAR-LLM provides a more computationally efficient architecture for long-context generation, as its inference FLOPs surpass the efficiency of standard LLMs for sequences longer than approximately 4096 tokens by operating on a compressed sequence of sentences. |
| A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm |
|
|
| Bridging Foundation Models and Lifelong Agentic Systems (Read more on arXiv or HuggingFace) |
Xinhao Yi, Yingxu Wang, Xi Zhang, Yanwen Peng, Jinyuan Fang |
This survey provides a comprehensive review of self-evolving AI agents, introducing a conceptual framework and guiding principles for their development. The paper’s primary objective is to systematically review existing techniques for self-evolving agentic systems by introducing a unified conceptual framework that abstracts the feedback loop underlying their design and evolution. The authors use this framework, along with a proposed four-stage paradigm (MOP, MOA, MAO, MASE) and “Three Laws of Self-Evolving AI Agents” (Endure, Excel, Evolve), to systematically categorize and analyze optimisation techniques for agent components like foundation models, prompts, memory, tools, and workflows. The survey finds that research is progressing from single-component optimisation to unified approaches that jointly optimise prompts, topologies, and models; one cited study (OPTIMA) on multi-agent communication efficiency reported a 2.8× performance gain with less than 10% of the token cost. The principal implication for AI practitioners is that the field is moving beyond static, manually configured agents towards dynamic systems; the paper offers a structured roadmap and taxonomies to design, build, and evaluate autonomous agents that can adapt post-deployment by optimising their components based on environmental feedback. |
| BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of |
|
|
| Deep-Research Agent (Read more on arXiv or HuggingFace) |
Kai Zou, Ping Nie, Shengyao Zhuang, Xueguang Ma, Zijian Chen |
The paper introduces BrowseComp-Plus, a benchmark with a fixed, human-verified corpus for the fair and transparent evaluation of deep-research agents by disentangling retrieval and reasoning components. The research objective is to create a standardized benchmark to overcome the fairness and reproducibility issues of existing evaluations that rely on dynamic live web search. The methodology involves constructing a 100k-document corpus for 830 questions via LLM-based evidence gathering, followed by rigorous human verification of supporting documents and mining of challenging negative documents. The primary result demonstrates that retrieval quality is critical to agent performance, as upgrading the retriever from BM25 to Qwen3-Embedding-8B increased the GPT-5 agent’s accuracy from 55.9% to 70.1% while reducing search calls. The principal implication for AI practitioners is that the retrieval component is a major performance bottleneck, and investing in superior retrieval models is a crucial strategy for enhancing the accuracy and efficiency of agentic systems. |
| OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks (Read more on arXiv or HuggingFace) |
Hongxing Li, Dingming Li, tricktreat, yanyc, wangzx1210 |
OmniEAR is a comprehensive framework for benchmarking agent reasoning in embodied tasks, evaluating physical interactions, tool usage, and multi-agent coordination. The main objective is to assess how large language models autonomously reason about capability acquisition and coordination needs from task demands and physical constraints, differing from benchmarks with explicit instructions or predefined tools. The methodology employs EAR-Sim, a text-based environment simulation supporting dynamic capability evolution and physics-constrained collaboration across 1,500 scenarios in EAR-Bench. Primary results show severe performance degradation: success rates drop from 85-96% on explicit instructions to 56-85% for tool reasoning and 63-85% for implicit collaboration, with compound tasks exhibiting over 50% failure rates. This implies current language models lack core embodied reasoning capabilities, requiring novel architectural mechanisms and training approaches beyond universal parameter scaling to advance embodied AI. |
| MolmoAct: Action Reasoning Models that can Reason in Space (Read more on arXiv or HuggingFace) |
Shuo Liu, Yuquan Deng, Haoquan Fang, Jiafei Duan, Jason Lee |
MolmoAct introduces an open-source, vision-language-action model that performs robotic manipulation by explicitly reasoning in space through a structured, multi-stage generation process. The primary objective is to create a robotic foundation model that improves upon standard end-to-end perception-to-control policies by incorporating an explicit, intermediate reasoning pipeline to enhance generalization, explainability, and interactive steerability. The methodology involves an autoregressive Action Reasoning Model (ARM) that, conditioned on visual input and a language instruction, sequentially generates three token types: (1) depth perception tokens representing the scene’s 3D geometry, (2) visual reasoning trace tokens forming a 2D polyline of the planned end-effector path, and (3) low-level action tokens. The model achieves an 86.6% average success rate on the LIBERO benchmark, outperforming all baselines, and in real-world fine-tuning, it shows up to a +22.7% task progression improvement over the πο-FAST baseline on bimanual tasks. For AI practitioners, this paper provides a concrete architecture and open-source implementation demonstrating that structuring a VLM’s output to include explicit, decodable intermediate representations for spatial planning leads to superior performance and enables novel, precise user interaction modalities like editable trajectory steering. |
| Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts (Read more on arXiv or HuggingFace) |
Tieyuan Chen, Zhanchao Zhou, Xiaodong Chen, Haoxing Chen, Haoyuan Wu |
The paper introduces Grove MoE, an architecture designed to improve the computational efficiency of Mixture-of-Experts (MoE) LLMs by enabling dynamic parameter activation based on input complexity. Its core methodology partitions experts into groups, each sharing an “adjugate expert” that is computed only once per group for all activated experts within it, thus dynamically allocating computation. The authors demonstrate this by upcycling a Qwen3-30B-A3B model into GroveMoE-Base (33B parameters), which dynamically activates 3.14–3.28B parameters and achieves a MATH benchmark score of 64.82, surpassing the baseline’s 59.75. For AI practitioners, Grove MoE offers a method to enhance model capacity and reasoning performance with a sub-linear increase in computational cost, although the paper explicitly states a custom inference kernel is needed to mitigate the 30% latency overhead observed with their generic implementation and achieve theoretical efficiency. |
| Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via |
|
|
| Past-Future (Read more on arXiv or HuggingFace) |
Qiufeng Wang, Junfeng Fang, Cunxiang Wang, Xin Wang, Yidong Wang |
Temporal Self-Rewarding Language Models (Temporal SR) address the critical limitation of vanishing DPO gradients in iterative Self-Rewarding LLM training caused by representational convergence of chosen and rejected responses. The primary objective is to sustain effective preference learning signals by strategically decoupling chosen and rejected samples via past and future model generations. This is achieved through a dual-phase methodology: Anchored Rejection, fixing rejected responses to past initial model outputs, and Future-Guided Chosen, curating chosen samples using next-generation model predictions. Empirical results demonstrate that Temporal SR’s Llama3.1-8B achieves a 29.44% win rate on AlpacaEval 2.0, outperforming the Self-Rewarding baseline (19.69%) by 9.75%. This approach provides AI practitioners a more stable and effective paradigm for iterative LLM alignment, enabling robust model improvement with fewer optimization iterations. |
| Reinforcement Learning in Vision: A Survey (Read more on arXiv or HuggingFace) |
Qingwei Meng, Kevin Qinghong Lin, Joya Chen, Chen Gao, Weijia Wu |
This paper surveys over 200 recent works on applying reinforcement learning to visual and multimodal models, tracing the evolution from RLHF to verifiable reward paradigms. The primary objective is to provide a critical and up-to-date synthesis of the visual reinforcement learning field by formalizing its problems, tracing policy optimization strategies, and organizing recent literature into a coherent taxonomy of four pillars: multimodal large language models, visual generation, unified models, and vision-language-action models. The methodology is a comprehensive survey that analyzes trends in algorithmic design (PPO, GRPO), reward engineering (RLHF, DPO, verifiable rewards), and evaluation protocols, structuring the findings into a principled taxonomy based on task domains and reward paradigms. The survey identifies a key trend towards Reinforcement Learning with Verifiable Rewards (RLVR), where deterministic signals replace human feedback; for example, a visual generation task can use a verifiable reward where a generated mask that attains an IoU ≥ 0.9 with the ground truth is awarded a reward of 1. The principal implication for AI practitioners is that this survey serves as a guide for selecting appropriate RL strategies and evaluation metrics, clarifying the design trade-offs between different policy optimization algorithms (e.g., PPO vs. GRPO) and reward supervision types (e.g., human preference vs. verifiable rewards) for developing visually-grounded agents. |
| Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning (Read more on arXiv or HuggingFace) |
Jiaheng Liu, Weixun Wang, Yancheng He, Jiashun Liu, Zihe Liu |
This paper systematically evaluates common reinforcement learning techniques for LLM reasoning, introducing “Lite PPO,” a minimalist PPO variant that outperforms more complex methods. The main objective is to resolve conflicting RL practices by dissecting the mechanisms of techniques like normalization and loss aggregation to provide clear guidelines for practitioners. Using the ROLL framework, the study conducts isolated experiments on Qwen3-4B/8B models across various data difficulties and mathematical benchmarks to assess the impact of each technique. The primary result shows that Lite PPO, which combines only advantage normalization (group-level mean, batch-level std) and token-level loss aggregation, achieves superior and more stable performance on non-aligned base models compared to technique-heavy algorithms like GRPO and DAPO. The principal implication for AI practitioners is that a simple, critic-free PPO with two targeted techniques can be more effective and robust for fine-tuning base models than complex, over-engineered RL algorithms, challenging the trend of adding more components to optimization pipelines. |
| Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided |
|
|
| Region Control (Read more on arXiv or HuggingFace) |
Hongyu Liu, Xinhua Zhang, Kunyu Feng, Mingzhe Zheng, Zeqian Long |
This paper presents Follow-Your-Shape, a training-free framework for performing large-scale, shape-aware image editing while preserving background content. The objective is to enable precise structural modifications to objects in an image without requiring external masks or degrading unedited regions, addressing a key limitation in diffusion and flow-based models. Its key methodology is the Trajectory Divergence Map (TDM), which dynamically localizes editable regions by computing the token-wise difference between the denoising velocity fields of the source and target prompts, guiding a scheduled Key-Value (KV) injection mechanism. On the introduced ReShapeBench benchmark, the method achieves state-of-the-art results, including a background preservation PSNR of 35.79, outperforming prior methods. For AI practitioners, this work provides a robust, mask-free technique for fine-grained region control in generative models, enabling complex structural edits by deriving localization information directly from the model’s denoising process. |
| Less Is More: Training-Free Sparse Attention with Global Locality for |
|
|
| Efficient Reasoning (Read more on arXiv or HuggingFace) |
Baihong Yuan, Shijie Cao, Arti Jain, Zhihao Zhang, Lijie Yang |
The paper introduces LessIsMore, a training-free sparse attention mechanism that enhances inference efficiency for large reasoning models by leveraging global token importance patterns. The primary objective is to develop a sparse attention method that reduces the computational overhead of long-generation reasoning tasks without the significant accuracy degradation or increased generation length seen in existing approaches. LessIsMore employs a unified token selection strategy by aggregating top-k token indices from all attention heads into a single, globally ranked set, and dedicates a fixed ratio of its token budget to a “stable recency window” to preserve immediate context. On the AIME-24 benchmark using a Qwen3-8B model and a 2K token budget, LessIsMore achieved 73.75% accuracy, closely matching the 74.48% of full attention and significantly outperforming a comparable method’s 53.33% accuracy, while also achieving a 1.13x end-to-end speedup. The principal implication for AI practitioners is that LessIsMore can be implemented as a drop-in, training-free optimization to significantly reduce latency and computational costs for deploying decode-heavy large reasoning models, maintaining near-lossless accuracy at much higher sparsity levels. |
| VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation |
|
|
| for Multilingual Long Document Understanding (Read more on arXiv or HuggingFace) |
Tong Yu, Chenguang Wang, Jihyung Kil, Ming Li, Jian Chen |
This paper introduces VisR-Bench, a new benchmark for evaluating question-driven, multimodal retrieval in long, multilingual documents, addressing the limitations of existing English-only or single-page datasets. The core objective is to systematically assess the retrieval capabilities of various models—including text-based, multimodal encoders, and MLLMs—across diverse content types (text, figures, tables) and 16 different languages. The methodology involves parsing 1.2K documents into page-level text and images, then using GPT-4o to generate over 35K QA pairs that require a specific evidence page for the answer, with a heuristic filter to ensure visual elements are necessary. The primary finding is that while MLLMs outperform other models, the best-performing method (ColQwen2-v0.1) achieves only 75.23% top-1 retrieval accuracy on the English split, with all models showing significant weaknesses in retrieving information from structured tables and in low-resource language contexts. For AI practitioners, this research highlights a critical bottleneck and implies that future development must focus on specialized mechanisms for table-aware perception and improved multilingual generalization to build effective real-world document intelligence systems. |
| Shortcut Learning in Generalist Robot Policies: The Role of Dataset |
|
|
| Diversity and Fragmentation (Read more on arXiv or HuggingFace) |
Hengtao Shen, Lianli Gao, Junlin Xie, Xu Luo, Youguang Xing |
This paper identifies limited diversity within individual sub-datasets and significant distributional disparities between them as primary contributors to shortcut learning in generalist robot policies trained on large-scale datasets like OXE. The research aims to uncover why generalist robot policies exhibit limited generalization due to reliance on task-irrelevant features. The authors’ methodology includes analyzing visual and textual features of robot datasets using diversity and disparity metrics, developing a theoretical framework based on mutual information, and conducting controlled experiments in both simulation (LIBERO-Spatial) and real-world environments. Primary results demonstrate that introducing a third object in real-world finetuning completely eliminated observed shortcut behavior (from 0.6 to 0.0) and improved $\pi_0$’s OOD success rate from 0.2 to 0.75. The principal implication for AI practitioners is to prioritize dataset collection strategies that ensure diversity and factor independence within sub-datasets while maintaining overlap across them, or to apply targeted robotic data augmentation to existing offline datasets to enhance generalization. |
| MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs (Read more on arXiv or HuggingFace) |
Jianguo Li, Jing Zhang, Zhenzhong Lan, Mingming Ha, Xiaodong Chen |
The paper introduces Mixture-of-Basis-Experts (MoBE), a compression technique for large Mixture-of-Experts (MoE) models that significantly reduces parameter count while preserving performance. The primary objective is to develop a compression method for massive MoE-based LLMs that avoids the substantial accuracy degradation characteristic of prior pruning and decomposition methods. MoBE’s methodology involves factorizing each expert’s weight matrix into an expert-specific transformation matrix A and a matrix B, where B is reconstructed as a learnable linear combination of a small set of basis matrices shared across all experts in a given layer. The primary result demonstrates that MoBE can reduce the parameter counts of models like DeepSeek-V3-0324 (671B) and Kimi-K2-Instruct (1T) by 24%-30% with only a 1%-2% drop in accuracy, significantly outperforming baseline methods. For AI practitioners, this implies that massive MoE models can be compressed to a more manageable size for deployment on memory-constrained hardware with minimal performance loss, although the authors note that a custom inference kernel is needed to realize the full computational efficiency of the architecture. |
| GLiClass: Generalist Lightweight Model for Sequence Classification Tasks (Read more on arXiv or HuggingFace) |
Alexander Yavorskyi, Oleksandr Lukashov, Dmytro Vodianytskyi, Mykhailo Shtopko, Ihor Stepanov |
The paper introduces GLiClass, a uni-encoder transformer architecture for sequence classification that achieves high accuracy and efficiency by jointly processing text and label tokens in a single forward pass. The research objective is to develop a classification model that combines the accuracy of cross-encoders with the computational efficiency of embedding-based methods, while providing robust zero-shot and few-shot learning capabilities for scenarios with large label sets. The key methodology is a uni-encoder architecture (primarily DeBERTa-based) that concatenates input text with all candidate labels and processes the combined sequence simultaneously, enabling inter-label interactions. The model is trained in multiple stages, including supervised learning, refinement with a Proximal Policy Optimization (PPO) framework, and post-training with Low-Rank Adaptation (LoRA) on specialized data streams. The primary result is that the gliclass-large-v3.0 model achieves an average F1-score of 0.7193 across benchmarks, surpassing a strong deberta-v3-large cross-encoder baseline (0.6821 F1). Critically, its inference throughput only degrades by 7.6% when scaling from 1 to 128 labels, while a comparable cross-encoder’s throughput slows down by approximately 52x. The principal implication for AI practitioners is that GLiClass provides a production-ready alternative for multi-label classification tasks with large or dynamic label sets, offering accuracy competitive with cross-encoders while maintaining high, stable throughput that does not degrade linearly with the number of classes. |
| Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant |
|
|
| Safeguards into Open-Weight LLMs (Read more on arXiv or HuggingFace) |
Robert Kirk, Tomek Korbak, Quentin Anthony, Stephen Casper, Kyle O’Brien |
This paper demonstrates that filtering specific dual-use topics from pretraining data builds inherently tamper-resistant safeguards into large language models. The research investigates whether pretraining data curation can durably prevent a 6.9B-parameter LLM from learning specific unwanted knowledge, such as information related to biothreats. The methodology involves a scalable, multi-stage filtering pipeline combining a keyword blocklist with a fine-tuned ModernBERT classifier to remove targeted documents before pretraining models from scratch. The primary result is that filtered models exhibit substantial resistance to adversarial fine-tuning attacks, withstanding up to 10,000 steps and 300M tokens of biothreat-related text with no degradation to general capabilities. For AI practitioners, this establishes pretraining data curation as a highly effective, computationally tractable defense-in-depth strategy for open-weight models, offering significantly more robustness against fine-tuning attacks than existing post-training techniques. |
| Fact2Fiction: Targeted Poisoning Attack to Agentic Fact-checking System (Read more on arXiv or HuggingFace) |
Reynold Cheng, Dacheng Wen, Bin Benjamin Zhu, Yupeng Li, Haorui He |
This paper introduces FACT2FICTION, a novel poisoning attack framework designed to compromise modern agentic fact-checking systems. The primary objective is to overcome the robustness of these systems by employing a two-agent LLM architecture that mirrors the victim’s claim decomposition strategy and uniquely exploits its justifications to craft tailored malicious evidence for each sub-claim. In extensive experiments, FACT2FICTION achieves an Attack Success Rate (ASR) 8.9%–21.2% higher than state-of-the-art attacks across various poisoning budgets. For instance, at a 1% poison rate on the DEFAME system, it reached a 42.4% ASR, an 8.9 percentage point improvement over the PoisonedRAG baseline. The principal implication for AI practitioners is that system-generated justifications, while enhancing transparency, create an exploitable attack surface, revealing a critical trade-off between explainability and security that must be addressed in agentic system design. |
| Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with |
|
|
| Patch-level CLIP Latents (Read more on arXiv or HuggingFace) |
Mohit Bansal, Chuan Li, Amir Zadeh, Jaemin Cho, Han Lin |
Bifrost-1 introduces a unified framework that bridges pretrained Multimodal LLMs (MLLMs) with diffusion models using patch-level CLIP latents as the intermediate representation. The primary objective is to integrate high-fidelity visual synthesis into MLLMs without compromising their reasoning capabilities or requiring costly retraining. The methodology involves adding a lightweight, trainable visual generation branch to a frozen MLLM to predict patch-level CLIP latents, which then guide a pretrained diffusion model via a novel Latent ControlNet. On the ImageNet 256x256 generation task, this approach achieved a Fréchet Inception Distance (FID) of 25.77, significantly outperforming an ablation using cross-attention guidance (FID 76.32) with the same training budget. For AI practitioners, this presents an efficient, modular method to equip existing MLLMs with powerful image generation capabilities by leveraging natively aligned latents, thereby preserving the MLLM’s core reasoning abilities and reducing computational costs. |
| When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with |
|
|
| Benign Inputs (Read more on arXiv or HuggingFace) |
Dasol Choi, Taeyoun Kwon, Hiskias Dingeto, Bodam Kim, oneonlee |
The paper introduces WHISPERINJECT, a two-stage adversarial audio attack framework designed to jailbreak Audio-Language Models (ALMs) using imperceptible, benign audio inputs. Its objective is to force ALMs to generate harmful content by addressing the challenge of eliciting model-native malicious responses. The methodology involves Stage 1 Native Target Discovery, using Reinforcement Learning with Projected Gradient Descent (RL-PGD) to generate model-native harmful payloads, followed by Stage 2 Adversarial Audio Generation, embedding these payloads into benign audio carriers via PGD. The framework achieved an attack success rate exceeding 86% across Qwen2.5-Omni and Phi-4-Multimodal models, with Stage 1 showing a 91.3% success rate in native payload discovery. This work demonstrates a critical vulnerability in ALMs, underscoring the urgent need for AI practitioners to develop robust, audio-signal level defenses beyond traditional text filtering in multimodal AI systems. |
| Compressing Chain-of-Thought in LLMs via Step Entropy (Read more on arXiv or HuggingFace) |
Zhijian Xu, Xiangyu Wen, Ziyang Zheng, Jianyuan Zhong, Zeju Li |
This paper introduces a method to compress Chain-of-Thought (CoT) reasoning by identifying and pruning redundant steps using a novel metric called “step entropy.” The primary objective is to develop a principled framework for reducing the computational cost and latency of LLM inference by compressing verbose CoT sequences without sacrificing final answer accuracy. The core methodology involves calculating “step entropy” for each reasoning step—defined as the sum of token-level entropies—to quantify its informational contribution, followed by systematically pruning a percentage of the lowest-entropy steps. Empirical validation shows that pruning up to 80% of the lowest-entropy reasoning steps causes minimal degradation in accuracy across multiple mathematical reasoning benchmarks, with one experiment on the Math500 dataset reducing thinking tokens by 29.7% while maintaining identical accuracy. For AI practitioners, this provides a method to significantly enhance the inference efficiency of LLMs using CoT by either statically pruning redundant steps or fine-tuning models to generate compressed thought processes, directly reducing deployment costs. |
| Speech-to-LaTeX: New Models and Datasets for Converting Spoken Equations |
|
|
| and Sentences (Read more on arXiv or HuggingFace) |
Matvey Skripkin, Elvir Karimov, Artyom Iudin, Dmitrii Tarasov, Dmitrii Korzh |
This paper introduces new models and datasets for converting spoken mathematical expressions and sentences into LaTeX. The primary objective is to address the challenging task of accurately transcribing spoken mathematical expressions and natural language sentences containing math into LaTeX format. The authors created a novel, large-scale open-source dataset (S2L) comprising 66k human-annotated and 571k TTS-generated audio samples, and evaluated ASR post-correction methods and multimodal end-to-end Audio-LLMs. On the English S2L-equations test subset, the SALMONN model achieved a Character Error Rate (CER) of 17.5%, demonstrating competitive results and outperforming ASR post-correction models. This work highlights the feasibility of Speech-to-LaTeX conversion with high-quality data, providing a strong performance baseline and emphasizing the importance of comprehensive datasets for AI practitioners in advancing spoken mathematical understanding. |
Papers for 2025-08-11
| Title |
Authors |
Summary |
| GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models (Read more on arXiv or HuggingFace) |
GLM-4. 5 Team, zixuanlimit, ZAHNGYUXUAN, LiquidAmmonia, Stanislas |
The paper introduces GLM-4.5, a 355B parameter open-source Mixture-of-Experts (MoE) model engineered for high performance in agentic, reasoning, and coding (ARC) tasks through a multi-stage training and reinforcement learning pipeline. The primary objective is to develop a single, open-source foundation model that unifies and excels across these distinct capabilities, which have often been addressed by specialized or proprietary models. The methodology involves a multi-stage process on 23T tokens, including pre-training on curated data, mid-training with sequence lengths up to 128K, and extensive post-training using expert model iteration, self-distillation, and a multi-faceted reinforcement learning (RL) framework. GLM-4.5 achieves strong performance, ranking 3rd overall on a comprehensive 12-benchmark evaluation; quantitatively, it scores 64.2% on SWE-bench Verified, outperforming models like GPT-4.1, and achieves 91.0% on the AIME 24 reasoning benchmark. For AI practitioners, the release of GLM-4.5 provides a powerful, parameter-efficient, and open-source alternative for building applications requiring a combination of agentic tool use, deep reasoning, and code generation, supported by a novel XML-based function call template that simplifies integration. |
| Voost: A Unified and Scalable Diffusion Transformer for Bidirectional |
|
|
| Virtual Try-On and Try-Off (Read more on arXiv or HuggingFace) |
jgkwak, RyanL22 |
Voost is a unified diffusion transformer that jointly models bidirectional virtual try-on and try-off to improve garment-person correspondence and generation fidelity. The primary objective is to develop a single, scalable framework that jointly learns virtual try-on and its inverse task, try-off, to improve spatial alignment and detail preservation without task-specific networks or auxiliary losses. The method uses a single Diffusion Transformer (DiT) with a token-level concatenation layout for person and garment images, enabling bidirectional generation controlled by a task token, and introduces inference-time attention temperature scaling and self-corrective sampling. Voost achieves state-of-the-art results on both try-on and try-off benchmarks; on the DressCode benchmark for paired try-on, it achieved a Fréchet Inception Distance (FID) of 2.787, surpassing the 3.283 FID of the CatVTON baseline. The principal implication for AI practitioners is that fine-tuning only the attention layers of a pretrained diffusion transformer is a highly effective strategy for complex image-conditioned generation tasks, significantly improving performance while using substantially fewer trainable parameters than full model fine-tuning. |
| InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy |
|
|
| Optimization (Read more on arXiv or HuggingFace) |
Pengxiang Li, Shuanghe Zhu, Zeyu Liu, xiaotianhan, SiriusL |
InfiGUI-G1 introduces Adaptive Exploration Policy Optimization (AEPO) to enhance GUI grounding for Multimodal Large Language Models (MLLMs). The primary objective is to overcome the inefficient exploration bottleneck in standard Reinforcement Learning with Verifiable Rewards (RLVR) to achieve robust semantic alignment, particularly for complex and unseen GUI elements. AEPO achieves this by integrating a multi-answer generation strategy with an Adaptive Exploration Reward (AER) function, complemented by a quality-of-exploration penalty to ensure diverse and purposeful exploration. The InfiGUI-G1-7B model establishes state-of-the-art performance across multiple benchmarks, demonstrating a significant 61.1% relative accuracy improvement on ‘hard’ samples within the ScreenSpot-Pro benchmark over the Naive RLVR baseline. This approach enables data-efficient training and fosters robust semantic understanding and generalization, making MLLM-based GUI agents more reliable for real-world human-computer interaction. |
| Memp: Exploring Agent Procedural Memory (Read more on arXiv or HuggingFace) |
Shuofei Qiao, Jialong Wu, Xiaobin Wang, Yuan Liang, Runnan Fang |
Memp is a task-agnostic framework designed to endow LLM agents with learnable, updatable, and lifelong procedural memory. It addresses the brittleness of existing agent memory by proposing strategies for building, retrieving, and updating procedural knowledge, including distilling past trajectories into fine-grained steps and higher-level abstractions. Empirical results show that “Proceduralization,” combining full trajectories with high-level scripts, achieved optimal performance, increasing GPT-40’s TravelPlanner Common Sense score from 71.93% (no memory) to 79.94% and reducing steps to 14.62. Additionally, procedural memory transferred from a stronger model (GPT-40) to a weaker one (Qwen2.5-14B) yielded a 5% increase in task completion rate and a 1.6-step reduction on TravelPlanner. This work indicates that dynamic procedural memory significantly enhances agent accuracy, efficiency, and generalization, providing a clear path for AI practitioners to develop more robust and continuously learning agents. |
| Pruning the Unsurprising: Efficient Code Reasoning via First-Token |
|
|
| Surprisal (Read more on arXiv or HuggingFace) |
Chengcheng Wan, Chao Hu, Yaoning Wang, Wenhao Zeng, YerbaPage |
The paper introduces ASAP, a two-stage framework for compressing Chain-of-Thought (CoT) traces in code reasoning models to improve efficiency and accuracy. The primary objective is to prune redundant reasoning steps from long CoTs while preserving logical coherence, thereby reducing the computational cost and inference latency of Large Reasoning Models (LRMs). The methodology involves a coarse-to-fine strategy: first, an anchor-guided pruning stage removes irrelevant reasoning branches, followed by a fine-grained refinement stage that uses a novel “first-token surprisal” metric to iteratively remove steps with low logical importance. On the LiveCodeBench v4_v5 benchmark, models fine-tuned with ASAP achieved a 36.19% Pass@1 accuracy while reducing token generation by 23.5% and inference latency by 43.5% compared to the strongest baseline. For AI practitioners, this framework provides a method to fine-tune models on shorter, logically-dense CoTs, leading to faster, more cost-effective, and more accurate code generation models by distilling effective reasoning patterns. |
| GENIE: Gaussian Encoding for Neural Radiance Fields Interactive Editing (Read more on arXiv or HuggingFace) |
Przemysław Spurek, Tomasz Szczepanik, Krzysztof Byrski, MikolajZ |
GENIE is a hybrid model that enables interactive, physics-based editing of NeRF scenes by conditioning them on a set of editable Gaussian primitives. The paper’s objective is to fuse the high-fidelity rendering of NeRF with the manipulable structure of Gaussian Splatting (GS) to support dynamic scene modifications. The methodology introduces “Splash Grid Encoding,” where a NeRF is conditioned by features interpolated from the k-nearest Gaussians, which are efficiently located using a novel “Ray-Traced Gaussian Proximity Search” (RT-GPS) algorithm. Quantitatively, GENIE outperforms the editable baseline RIP-NeRF in six of eight NeRF-Synthetic scenes (e.g., 33.23 vs 32.23 PSNR on the Ficus scene) and is the first presented method to enable editing on complex, unbounded Mip-NeRF 360 scenes. For AI practitioners, this framework provides a direct pathway to integrate high-quality neural scene representations with physics engines, enabling the development of interactive and physically grounded applications in virtual environments and content creation. |
| Adapting Vision-Language Models Without Labels: A Comprehensive Survey (Read more on arXiv or HuggingFace) |
Eleni Chatzi, Ran He, Jian Liang, Lijun Sheng, Hao Dong |
This survey introduces a novel taxonomy for unsupervised Vision-Language Model (VLM) adaptation, categorizing methods based on the availability of unlabeled visual data. The paper’s objective is to systematically structure the field of label-free VLM adaptation by organizing existing research according to practical data availability constraints. The authors propose a taxonomy that classifies methods into four paradigms: Data-Free Transfer, Unsupervised Domain Transfer, Episodic Test-Time Adaptation, and Online Test-Time Adaptation, reviewing the core technical strategies within each category. The analysis reveals that methodologies are tailored to data availability, such as using LLMs for text augmentation in data-free settings or entropy minimization for test-time adaptation, and identifies in Table V that benchmark datasets like ImageNet are popular across all four paradigms. For AI practitioners, this taxonomy provides a principled framework to select appropriate unsupervised adaptation techniques based on their specific data access constraints and to benchmark new methods within a clearly defined context. |
| MELLA: Bridging Linguistic Capability and Cultural Groundedness for |
|
|
| Low-Resource Language MLLMs (Read more on arXiv or HuggingFace) |
Guohang Yan, Ruirui Chen, Nuo Chen, Jiaying Fei, Yufei Gao |
The paper introduces MELLA, a dual-source dataset and framework for fine-tuning Multimodal Large Language Models (MLLMs) to improve both linguistic fluency and cultural groundedness in eight low-resource languages. The primary objective is to overcome the limitations of existing MLLMs, which produce culturally “thin” descriptions in low-resource contexts, by jointly enhancing linguistic capability and cultural understanding. The authors propose a dual-source data strategy, constructing a 6.8 million-pair dataset by combining native web alt-text for cultural knowledge with MLLM-generated, translated captions for linguistic skill, and then performing supervised fine-tuning. Fine-tuning with MELLA yields significant improvements; for instance, on the InternVL2-8B backbone, the Meteor score for Hungarian improved from a baseline of 0.11 to 13.11. For AI practitioners, this work provides a validated methodology and a public dataset to build MLLMs that are not just linguistically proficient in low-resource languages but also culturally aware, leading to more inclusive and contextually accurate AI systems. |
| MeshLLM: Empowering Large Language Models to Progressively Understand |
|
|
| and Generate 3D Mesh (Read more on arXiv or HuggingFace) |
Yi Yang, Yi-Hsuan Tsai, Yufeng Wang, I-Chao Shen, Shuangkang Fang |
MeshLLM is a novel framework that enables Large Language Models to natively understand and generate text-serialized 3D meshes by addressing the data-scale and structural information loss limitations of prior methods. The core methodology involves a “Primitive-Mesh” decomposition strategy, using KNN clustering and semantic segmentation to expand the training dataset to over 1.5 million mesh parts, coupled with a progressive, multi-task training paradigm that includes vertex-to-face prediction and local mesh assembly to explicitly model 3D topology. Experiments show MeshLLM significantly outperforms the LLaMA-Mesh baseline in mesh understanding, achieving a CLIP score of 0.391 versus 0.124, while producing generation quality comparable to specialized encoder-based models. The principal implication for AI practitioners is that for structured, non-textual data, decomposing inputs into meaningful sub-components and designing training tasks that teach inherent structural relationships can allow LLMs to bypass dedicated encoders and effectively process raw serialized data formats. |
| UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and |
|
|
| Precise Inference-Time Grounding (Read more on arXiv or HuggingFace) |
Bingqi Chen, Zihan Song, Jia Ma, Yuhang Wu, LianShuQuan |
The paper introduces UI-AGILE, a comprehensive framework to enhance the training and inference capabilities of Graphical User Interface (GUI) agents. Its main objective is to address common GUI agent failures, including the dilemma of reasoning design, ineffective reward signals, and performance degradation from visual noise on high-resolution displays. The methodology combines Reinforcement Fine-Tuning (RFT) with novel components: a “Simple Thinking” reward, a continuous grounding reward, and a cropping-based resampling strategy for training, alongside a “Decomposed Grounding with Selection” method for inference. UI-AGILE achieves state-of-the-art performance, with the combined training and inference methods delivering a stated 23% absolute improvement in grounding accuracy over the best baseline on the ScreenSpot-Pro benchmark. The principal implication for AI practitioners is that the proposed “Decomposed Grounding with Selection” method can be used as a plug-and-play inference enhancement to significantly boost the grounding accuracy of existing GUI agent models on high-resolution screens. |
| LightSwitch: Multi-view Relighting with Material-guided Diffusion (Read more on arXiv or HuggingFace) |
Shubham Tulsiani, Fernando De la Torre, thebluser |
LightSwitch is a generative framework that uses a material-guided diffusion model for fast, consistent multi-view relighting of 3D objects. The primary objective is to relight an object captured in multiple posed images under a novel target illumination, ensuring visual consistency across all views by leveraging inferred intrinsic material properties. The methodology involves finetuning a Stable Diffusion UNet architecture with multi-view self-attention modules, conditioning it on input images, camera poses, and inferred material maps (albedo, roughness, metallicness) to guide the relighting process. The framework demonstrates performance that matches or exceeds prior state-of-the-art methods; on the NeRF-Synthetic dataset, LightSwitch achieves relighting in approximately 2 minutes, substantially faster than competing inverse rendering techniques that require 120-480 minutes. For AI practitioners, this method provides a highly efficient alternative to traditional inverse rendering, enabling rapid generation of relightable 3D assets from multi-view images for applications in graphics, simulation, and virtual reality. |
Papers for 2025-08-08
| Title |
Authors |
Summary |
| On the Generalization of SFT: A Reinforcement Learning Perspective with |
|
|
| Reward Rectification (Read more on arXiv or HuggingFace) |
Xinyu Ye, Yingzhe Peng, Zhou Ziheng, Yizhou Zhou, Yongliang Wu |
This paper presents Dynamic Fine-Tuning (DFT), a simple modification to Supervised Fine-Tuning (SFT) that improves model generalization by dynamically re-weighting the training objective. The primary objective is to address the poor generalization of SFT compared to reinforcement learning (RL) by identifying and rectifying the problematic implicit reward structure within the SFT gradient. The methodology involves a theoretical analysis equating the SFT gradient to a policy gradient with an ill-posed, inverse-probability-weighted reward, which is then corrected by multiplying the SFT loss with the model’s token probability. In experiments, DFT significantly outperformed SFT; for example, fine-tuning the Qwen2.5-Math-1.5B model with DFT resulted in an average performance gain of +15.66 points, over 5.9 times the improvement from standard SFT. The principal implication for AI practitioners is that a single-line code change can substantially enhance SFT performance and generalization, offering a more robust and efficient alternative without requiring complex RL pipelines or additional reward models. |
| R-Zero: Self-Evolving Reasoning LLM from Zero Data (Read more on arXiv or HuggingFace) |
Zongxia Li, Hongming Zhang, Xiaoyang Wang, Wenhao Yu, Chengsong Huang |
The paper introduces R-Zero, a fully autonomous framework that improves an LLM’s reasoning capabilities from zero initial data by having a “Challenger” model and a “Solver” model co-evolve to generate their own training curriculum. The primary objective is to overcome the bottleneck of human-curated data by creating a self-improving system where the Challenger is rewarded for proposing tasks at the edge of the Solver’s ability, and the Solver is rewarded for solving them. The methodology employs a co-evolutionary loop using Group Relative Policy Optimization (GRPO), where the Challenger’s reward is based on the Solver’s uncertainty (measured via self-consistency), and the Solver is fine-tuned on a filtered set of challenging questions using its own majority-voted pseudo-labels. This approach substantially improved reasoning, boosting the Qwen3-4B-Base model’s performance by +6.49 on math reasoning benchmarks and +7.54 on general-domain reasoning benchmarks. For AI practitioners, R-Zero provides a powerful, data-free method to enhance base models in verifiable domains like mathematics, serving as a superior initialization checkpoint for subsequent supervised fine-tuning. |
| DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning (Read more on arXiv or HuggingFace) |
Ziming Wang, Börje F. Karlsson, Ye Wang, Pi Bu, Xinrun Xu |
This paper introduces DeepPHY, a benchmark suite of six physics-based environments, to evaluate the interactive physical reasoning capabilities of agentic Vision-Language Models (VLMs). The primary objective is to systematically assess whether current VLMs can understand and reason about physical principles to perform precise, multi-step planning in dynamic, interactive environments. The methodology involves unifying six simulators (e.g., PHYRE, Kinetix, Angry Birds) into a testbed with structured action spaces, and then evaluating 17 state-of-the-art VLMs in a zero-shot, trial-based setting using Vision-Language-Action (VLA) and World Model (WM) prompt formats. The results show significant performance gaps; for instance, the best-performing model on the PHYRE task achieved only a 23.1% success rate after ten attempts, and analysis of the Pooltool task revealed that high success rates were misleadingly achieved through “brute-force heuristics” rather than genuine physical understanding. The principal implication for AI practitioners is that there is a fundamental disconnect between a model’s ability to describe a physical phenomenon and its ability to use that knowledge for precise, predictive control, indicating that current agentic VLMs are not yet capable of robust interactive physical reasoning. |
| Genie Envisioner: A Unified World Foundation Platform for Robotic |
|
|
| Manipulation (Read more on arXiv or HuggingFace) |
Shengcong Chen, Donglin Yang, Siyuan Huang, Pengfei Zhou, Yue Liao |
Genie Envisioner is a unified world foundation platform that integrates policy learning, simulation, and evaluation for robotic manipulation into a single video-generative framework. The primary objective is to develop a scalable and integrated system that overcomes the fragmentation of traditional robotics pipelines by unifying sensing, policy learning, and evaluation within a single, closed-loop, video-generative world model. The methodology centers on GE-Base, an instruction-conditioned, multi-view video diffusion model trained on approximately 3,000 hours of real-world robotic data, which is paired with GE-Act, a lightweight flow-matching action decoder for policy inference, and GE-Sim, an action-conditioned neural simulator. The platform demonstrates strong cross-embodiment generalization; with only one hour of adaptation data on a novel robot, GE-Act achieved an end-to-end success rate of approximately 50% on a complex “fold cardboard box” task, where baseline models completely failed. The principal implication for AI practitioners is that a vision-centric, generative world model approach can serve as a unified foundation for building general-purpose robots, enabling more efficient policy learning and adaptation to new embodiments with minimal task-specific data compared to traditional, disjointed pipelines. |
| Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity (Read more on arXiv or HuggingFace) |
Zhibing Li, Tong Wu, Ziyang Chu, Long Zhuo, Yuhan Zhang |
This paper introduces Hi3DEval, a hierarchical framework that evaluates 3D generative models at object, part, and material levels using a novel benchmark and a hybrid automated scoring system. To overcome the limitations of existing 2D-based metrics, the authors developed a hybrid scoring system—using video for object/material assessment and 3D features for part-level geometry—and constructed the Hi3DBench dataset annotated via a multi-agent MLLM pipeline. The system demonstrates superior human alignment, achieving a pairwise rating accuracy of 0.774 on object-level geometry plausibility for text-to-3D, significantly outperforming prior methods like GPTEval3D (0.690). AI practitioners can utilize Hi3DEval for more robust, scalable, and fine-grained automated evaluation, enabling detailed failure analysis and more accurate comparison of 3D generative models. |
| Are We on the Right Way for Assessing Document Retrieval-Augmented |
|
|
| Generation? (Read more on arXiv or HuggingFace) |
Junjie Yang, Dongping Chen, Yaochen Wang, Mingjia Wang, Wenxuan Shen |
The paper introduces DOUBLE-BENCH, a large-scale, multimodal, and multilingual benchmark designed to comprehensively evaluate document Retrieval-Augmented Generation (RAG) systems by addressing the flaws in existing benchmarks. Its primary objective is to create a more realistic and fine-grained evaluation framework that overcomes the limitations of current benchmarks, such as limited scope, unrealistic prior knowledge assumptions, and ambiguous queries. The methodology involves a three-stage pipeline to construct the benchmark from 3,276 documents, generating 5,168 single- and multi-hop queries using an iterative, LLM-driven refinement process with knowledge graphs, followed by human verification of all evidence labels. Experiments reveal an “over-confidence dilemma,” where advanced RAG frameworks attempt to answer nearly every query regardless of retrieval success, and show that a simple baseline using a strong retriever (colqwen2.5-3b-multilingual with 0.795 average hit@5) and a generator matches the performance of complex agentic frameworks. The principal implication for AI practitioners is that the retrieval stage remains the critical bottleneck; therefore, development efforts should prioritize improving retrieval models and implementing mechanisms for systems to refuse to answer when evidence is insufficient, rather than solely focusing on more complex generation agents. |
| Are Today’s LLMs Ready to Explain Well-Being Concepts? (Read more on arXiv or HuggingFace) |
Huan Liu, Chengshuai Zhao, Zhen Tan, Dawei Li, Bohan Jiang |
This research systematically evaluates and improves the capability of Large Language Models (LLMs) to explain well-being concepts for diverse audiences. The paper’s central research question is whether today’s LLMs are ready to explain complex well-being concepts accurately and in a tailored manner. The methodology involves creating a large-scale dataset of 43,880 explanations from 10 LLMs, introducing a principle-guided LLM-as-a-judge evaluation framework, and fine-tuning an open-source model using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Results show that while larger models outperform smaller ones, they exhibit shared weaknesses in providing utility and depth; crucially, fine-tuning a 4B parameter model with DPO improved its win rate for domain expert explanations to 83.4%, surpassing a larger 14B parameter baseline model. The key implication for AI practitioners is that using curated preference data to fine-tune smaller models with DPO is a highly effective strategy for developing specialized models that can outperform larger, general-purpose models on domain-specific tasks. |
| Can Large Multimodal Models Actively Recognize Faulty Inputs? A |
|
|
| Systematic Evaluation Framework of Their Input Scrutiny Ability (Read more on arXiv or HuggingFace) |
Yuan Wu, Yi Chang, Gengxu Li, Jinzhe Li, Haiqi Yang |
This paper introduces the ISEval framework to systematically evaluate the ability of large multimodal models (LMMs) to autonomously detect faulty inputs, revealing a significant gap between their latent and spontaneously activated critique capabilities. The primary research objective is to determine if LMMs can actively recognize and scrutinize erroneous multimodal inputs without explicit instructions, rather than passively accepting them. The methodology involves the Input Scrutiny Ability Evaluation Framework (ISEval), which uses a dataset of inputs containing seven distinct error categories and evaluates models based on Spontaneous Error Detection Rate (SEDR), Guided Error Detection Rate (GEDR), and Modality Trust Preference Score (MTPS). The primary result shows that LMMs have very limited autonomous scrutiny ability, with the top-performing model, Gemini 2.5 Pro, achieving an average SEDR of only 21.95%, whereas its performance increased to a 57.72% GEDR when explicitly prompted to check for errors. The principal implication for AI practitioners is that current LMMs cannot be trusted to proactively validate inputs, and building reliable systems requires incorporating explicit prompts to activate their latent critique functions, as they do not apply this scrutiny autonomously. |
| InfiAlign: A Scalable and Sample-Efficient Framework for Aligning LLMs |
|
|
| to Enhance Reasoning Capabilities (Read more on arXiv or HuggingFace) |
Zhijie Sang, Kejing Yang, Qi Zhou, Su Lu, Shuo Cai |
InfiAlign is a scalable and sample-efficient post-training framework integrating Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to enhance LLM reasoning. The primary objective is to develop an automated and scalable framework for aligning LLMs that improves reasoning capabilities while drastically reducing the amount of required training data. The methodology centers on a multi-dimensional data selection pipeline that curates a compact, high-quality dataset from large open-source corpora by sampling for diversity, difficulty (using response length as a proxy), and quality, followed by a two-stage curriculum SFT and a DPO phase. When applied to the Qwen2.5-Math-7B-Base model, the InfiAlign SFT model achieved comparable performance to the DeepSeek-R1-Distill-Qwen-7B baseline while using only 12% of its training data (92K vs. 800K samples). This work implies that AI practitioners can achieve strong reasoning alignment with substantially lower data and computational overhead by implementing principled, automated data curation pipelines, offering a more efficient alternative to large-scale data distillation or manual curation. |
| Evaluating, Synthesizing, and Enhancing for Customer Support |
|
|
| Conversation (Read more on arXiv or HuggingFace) |
Feng Chen, Lifan Guo, Junhui Li, Huaixia Dou, Jie Zhu |
This paper introduces a structured framework and datasets to enhance LLM performance for customer support conversations. The objective is to train and evaluate LLMs to generate high-quality, empathetic, and strategically-aligned responses in customer support scenarios. The methodology involves creating the Customer Support Conversation (CSC) framework based on COPC guidelines, constructing the CSConv evaluation dataset by rewriting 1,855 real-world dialogues with an LLM, and generating the RoleCS training dataset via a five-agent role-playing simulation. The primary result shows that fine-tuning a 72B Qwen2.5-Instruct model on RoleCS significantly improves performance on CSConv, increasing the ROUGE-L score from 5.41 to 7.97 and strategy prediction accuracy from 37.22% to 43.29%. For AI practitioners, the principal implication is that the proposed role-playing framework can be used to generate high-quality synthetic data for fine-tuning LLMs, enabling the development of more effective and structured conversational agents for customer service applications. |
| Don’t Overthink It: A Survey of Efficient R1-style Large Reasoning |
|
|
| Models (Read more on arXiv or HuggingFace) |
Fangzhou Yao, Weibo Gao, Yizhi Wang, Yichao Du, Linan Yue |
This paper surveys and taxonomizes recent methods for mitigating the “overthinking” problem in R1-style Large Reasoning Models (LRMs) to improve their computational efficiency. The objective is to systematically review and categorize techniques designed to reduce the length and redundancy of reasoning chains in LRMs without compromising performance. The authors introduce a novel framework that classifies efficient reasoning methods into two primary paradigms: single-model optimization (e.g., CoT Compression, Adaptive Reasoning) and multi-model collaboration (e.g., LLM Routing, Speculative Decoding). The survey highlights that various techniques yield significant efficiency gains; for instance, it cites that model merging strategies can reduce inference length by up to 55% in average response length while preserving output quality. For AI practitioners, this taxonomy provides a structured guide for selecting and implementing strategies, such as early-exit mechanisms or multi-model routing, to optimize the inference cost and latency of deployed reasoning models exhibiting inefficient, lengthy thought processes. |
| MOSEv2: A More Challenging Dataset for Video Object Segmentation in |
|
|
| Complex Scenes (Read more on arXiv or HuggingFace) |
Xudong Jiang, Shuting He, Chang Liu, Kaining Ying, Henghui Ding |
This paper introduces MOSEv2, a large-scale video object segmentation dataset designed to challenge models with complex, realistic scenarios underrepresented in existing benchmarks. The objective is to advance video object segmentation (VOS) toward real-world applicability by creating a benchmark featuring frequent object disappearance, severe occlusions, adverse weather, and knowledge-dependent scenes. The methodology involved curating 5,024 videos and 701,976 instance masks based on strict complexity criteria, then benchmarking 20 VOS and 9 VOT methods to establish performance baselines. The results demonstrate a significant performance degradation for state-of-the-art models; for instance, the SAM2 model’s J&F score drops from 76.4% on the MOSEv1 predecessor to 50.9% on MOSEv2. For AI practitioners, MOSEv2 serves as a critical benchmark to test and develop models that are robust to long-term temporal reasoning and semantic ambiguity, exposing failure points not apparent in previous datasets. |
| CoAct-1: Computer-using Agents with Coding as Actions (Read more on arXiv or HuggingFace) |
Taiwei Shi, Jieyu Zhang, Viraj Prabhu, Yutong Dai, Linxin Song |
CoAct-1 is a multi-agent system that enhances computer automation by dynamically delegating tasks to either a traditional GUI operator or a programmer agent that executes code. The primary objective is to improve the efficiency, reliability, and success rate of autonomous computer-using agents on complex, long-horizon tasks by augmenting standard GUI manipulation with the ability to perform actions via programmatic script execution. The paper introduces CoAct-1, a system featuring an Orchestrator that decomposes a user’s goal into subtasks and dynamically delegates them to either a VLM-based GUI Operator for visual interactions or a Programmer agent that writes and executes Python or Bash scripts. On the OSWorld benchmark, CoAct-1 achieved a state-of-the-art success rate of 60.76%, while reducing the average number of steps for successful tasks to 10.15, compared to 15.22 for the leading GUI-only agent GTA-1. For AI practitioners developing autonomous agents, the principal implication is that integrating a programmatic action space alongside a GUI-based one can significantly boost performance and efficiency, particularly for tasks involving file operations and data processing, by bypassing brittle and lengthy UI sequences. |
| Marco-Voice Technical Report (Read more on arXiv or HuggingFace) |
Qingjuan Li, Haoqin Sun, Xuanfan Ni, Chenyang Lyu, Fengping Tian |
The paper presents Marco-Voice, a unified text-to-speech system for high-fidelity voice cloning and controllable emotional speech synthesis. The objective is to create a single framework that generates natural, expressive speech while preserving speaker identity across diverse emotional contexts by overcoming timbre-style entanglement. The methodology combines a speaker-emotion disentanglement mechanism using a cross-orthogonal loss and in-batch contrastive learning with a rotational emotion embedding method derived from paired neutral-emotional speech. Marco-Voice achieves a speaker similarity score of 0.8275 in human evaluations, significantly outperforming the CosyVoice2 baseline (0.605) and showing superior emotional expression. The principal implication for practitioners is that applying explicit disentanglement techniques like cross-orthogonal constraints within a unified architecture enables the development of more robust and controllable personalized speech synthesis systems. |
| StrandDesigner: Towards Practical Strand Generation with Sketch Guidance (Read more on arXiv or HuggingFace) |
Xiaobin Hu, Han Feng, Chengming Xu, Moran Li, Na Zhang |
StrandDesigner introduces the first sketch-based generative model for creating realistic 3D hair strands. The main objective is to develop a model that converts sketch images into high-fidelity 3D hair strands, providing finer user control than existing text or image-prompted methods. The key methodology combines a learnable multi-scale strand upsampling strategy using a scale-wise autoregressive transformer with a multi-scale adaptive conditioning mechanism that fine-tunes a pretrained DINOv2 model with scale-specific tokens. The model achieves superior performance in conditional generation, obtaining a Point Cloud IoU of 64.54% and a Chamfer Distance of 0.80, outperforming the next-best competitor (Sketch+HAAR) which scored 60.85% and 1.06 respectively. The principal implication for AI practitioners is that for generating complex 3D assets, a specialized framework combining a structured, multi-scale generative process with an adaptive conditioning mechanism tailored to an intuitive input modality like sketches can yield more precise and controllable results than general-purpose text-to-3D models. |
| Hop, Skip, and Overthink: Diagnosing Why Reasoning Models Fumble during |
|
|
| Multi-Hop Analysis (Read more on arXiv or HuggingFace) |
Reshmi Ghosh, Yashwanth Babu, Srujana Pillarichety, Isha Nalawade, Anushka Yadav |
This paper introduces a diagnostic framework to systematically categorize and analyze reasoning failures in language models on multi-hop question answering tasks. The primary objective is to understand how and why reasoning models break down when synthesizing information across multiple sources by decomposing their behavior along three dimensions: hops, coverage, and overthinking. The study employs a seven-category error taxonomy to manually annotate 1,080 outputs from six language models across three datasets and develops a two-step LLM-as-a-Judge for automated analysis. A primary result is that “overhopping” (executing more reasoning steps than required) is the most persistent failure, with overthinking rates on the complex MuSiQue dataset reaching as high as 61.7% for one model and systematically driving incorrect answers. The principal implication for AI practitioners is that evaluation must move beyond final answer accuracy to include metrics of reasoning fidelity, as models can produce correct answers despite flawed reasoning, masking critical inefficiencies and a propensity to hallucinate under complexity. |
| PRvL: Quantifying the Capabilities and Risks of Large Language Models |
|
|
| for PII Redaction (Read more on arXiv or HuggingFace) |
Prajit Das, Lavanya Elluri, Aritran Piplai, Anantaa Kotal, Leon Garza |
This research presents a comprehensive benchmark of Large Language Models for PII redaction, evaluating various architectures and training strategies to quantify their capabilities and risks. The primary objective is to determine which combinations of model architecture, training paradigm, and inference strategy yield the optimal trade-offs between redaction accuracy, latency, and privacy preservation across different domains. The study evaluates multiple model families (e.g., Dense LLMs, MoE, LRM) on the AI4Privacy dataset using parameter-efficient fine-tuning, instruction-tuning, and Retrieval-Augmented Generation (RAG), assessing performance with span-correct/label-exact accuracy and a privacy leakage score (SPriV). Instruction-tuning emerged as the most effective strategy, with the instruction-tuned DeepSeek-Q1 model achieving the highest span-correct accuracy of 0.994 and a minimal privacy leakage (SPriV) score of 0.002. The principal implication for AI practitioners is that instruction-tuning smaller, efficient open-source models is a superior strategy for PII redaction compared to standard fine-tuning or using larger models, providing the best balance of performance, cost, and privacy; the released PRvL toolkit facilitates the deployment of these secure, auditable solutions. |
| REINA: Regularized Entropy Information-Based Loss for Efficient |
|
|
| Simultaneous Speech Translation (Read more on arXiv or HuggingFace) |
Xiao Yu, Mahesh Kumar Nandwana, Joseph Liu, Nameer Hirschkind |
This paper introduces REINA, a regularized, information-theoretic loss function for efficiently adapting pre-trained, non-streaming speech-to-text translation models into high-performance simultaneous translation systems. The research objective is to develop a stable and efficient method for training an adaptive READ/WRITE policy that optimally balances translation quality and latency. The key methodology is the REINA loss, which trains a policy network by maximizing the covariance between its output and an estimate of mutual information gained from future audio, approximated via the cross-entropy difference on partial versus full audio contexts. On the MUST-C benchmark, REINA demonstrates state-of-the-art performance, with a model trained only on MUST-C data achieving Normalized Streaming Efficiency (NoSE) scores up to 8.9% higher than prior methods like DiSeg. For AI practitioners, REINA provides a computationally efficient fine-tuning framework to repurpose existing large translation models for real-time applications, offering a direct way to optimize the quality-latency trade-off without complex architectural changes or unstable training paradigms. |
| I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating |
|
|
| Linguistic Shibboleth Detection in LLM Hiring Evaluations (Read more on arXiv or HuggingFace) |
Chirag Shah, Aman Chadha, Tanya Roosta, Julia Kharchenko |
This paper introduces a benchmark for detecting and measuring how Large Language Models (LLMs) exhibit bias against linguistic shibboleths in simulated hiring evaluations. The main objective is to systematically quantify LLM responses to subtle linguistic markers, like hedging, that can inadvertently serve as proxies for demographic characteristics. The methodology involves evaluating LLMs using 100 question-response pairs, each with a “hedged” and a “confident” version that are semantically equivalent, to assess scoring and hiring recommendations. The primary result is that LLMs systematically penalize hedged language; across all models, hedged responses received ratings that were, on average, 25.6% lower than confident responses with identical content. The principal implication for AI practitioners is that systems deployed for high-stakes evaluations must undergo rigorous testing with controlled benchmarks to identify and mitigate biases that penalize communication styles correlated with demographic groups, thereby preventing systemic discrimination. |
| RPCANet++: Deep Interpretable Robust PCA for Sparse Object Segmentation (Read more on arXiv or HuggingFace) |
Jian Yang, Yixuan Ding, Tianfang Zhang, Yimian Dai, fengyiwu |
This paper introduces RPCANet++, a deep unfolding network that integrates Robust Principal Component Analysis (RPCA) with deep learning for interpretable sparse object segmentation. The primary objective is to address the computational cost and limited generalizability of traditional RPCA models while enhancing the interpretability of deep networks for segmentation tasks. The methodology unfolds a relaxed RPCA optimization problem into a multi-stage architecture consisting of a Background Approximation Module (BAM), an Object Extraction Module (OEM), and an Image Restoration Module (IRM), enhanced with a Memory-Augmented Module (MAM) and a Deep Contrast Prior Module (DCPM). Experiments show that RPCANet++ achieves state-of-the-art performance, with the six-stage model attaining a 94.39% Intersection over Union (IoU) on the NUDT-SIRST dataset, a 5.08 percentage point improvement over its baseline. For AI practitioners, this work provides a framework for developing interpretable and efficient segmentation models by mapping classical optimization steps to neural network components, offering a verifiable alternative to “black-box” architectures for tasks requiring high reliability. |
| I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal |
|
|
| Entity Linking (Read more on arXiv or HuggingFace) |
Chao Wang, Tong Ruan, Kaiwen Li, Junwen Li, Ziyan Liu |
This paper introduces I2CR, a novel framework for multimodal entity linking that uses intra- and inter-modal collaborative reflections to improve accuracy. The objective is to address the unnecessary use of images and the limitations of single-pass visual feature extraction in current LLM-based methods. I2CR first attempts to link entities using only textual information; if this is deemed insufficient through intra-modal consistency reflection and inter-modal alignment verification, it then initiates a multi-round iterative process that incorporates diverse visual clues from various image-to-text models. The framework achieves state-of-the-art results, including a 5.1% absolute improvement in top-1 accuracy on the WikiDiverse dataset (to 91.6%). The principal implication for AI practitioners is that dynamic, reflective reasoning pipelines that selectively integrate multimodal data as needed can be more effective and robust than monolithic, single-pass fusion architectures. |
| Steering One-Step Diffusion Model with Fidelity-Rich Decoder for Fast |
|
|
| Image Compression (Read more on arXiv or HuggingFace) |
Yifei Ji, Jiale Yuan, Jinpei Guo, Mingde Zhou, Zheng Chen |
SODEC is a single-step diffusion-based image compression model designed to achieve high perceptual quality and fidelity with significantly accelerated decoding. The research objective is to resolve the excessive decoding latency and poor fidelity inherent in multi-step diffusion compression models, especially at very low bitrates. The methodology replaces iterative denoising with a single-step diffusion process, which is steered by a fidelity guidance module that uses features from a preliminary VAE-based reconstruction as an explicit condition, and is optimized using a rate annealing training strategy. SODEC improves decoding speed by over 20x compared to previous multi-step diffusion methods while establishing new state-of-the-art performance in rate-distortion-perception on benchmarks like DIV2K. The principal implication for AI practitioners is that single-step diffusion can be made practical and effective for high-fidelity compression by using an explicit, parallel reconstruction to provide strong structural guidance, making diffusion-based approaches more viable for latency-sensitive applications. |
Papers for 2025-08-07
| Title |
Authors |
Summary |
| VeriGUI: Verifiable Long-Chain GUI Dataset (Read more on arXiv or HuggingFace) |
Zhenyu Cui, Huichi Zhou, Shunyu Liu, weihao1115, Liam-Liu |
The paper introduces VeriGUI, a new human-annotated dataset for benchmarking autonomous agents on long-chain, verifiable Graphical User Interface (GUI) tasks. The primary objective is to evaluate and foster the development of generalist GUI agents on complex, realistic computer tasks that require long-horizon planning, addressing the limitations of existing datasets which focus on short-term interactions and outcome-only verification. The methodology involves constructing a dataset of web and desktop tasks (averaging 214.4 steps) that are decomposed into a sequence of interdependent subtasks, each with an explicitly defined and verifiable goal. These tasks were generated using a combination of LLM-based instruction creation and expert human demonstration to collect detailed trajectories. Experimental results show that current state-of-the-art agents struggle significantly, with no agent configuration achieving an average task success rate (SR) above 10%; the highest average SR across all tasks was 8.5%, achieved by deep research agents, indicating a substantial performance gap in handling long-horizon tasks. The principal implication for AI practitioners is that current agent architectures and foundation models lack the robust planning and decision-making capabilities required for complex, multi-step GUI workflows, and VeriGUI provides a challenging benchmark with granular, subtask-level feedback to diagnose failures and guide the development of more capable systems. |
| Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens (Read more on arXiv or HuggingFace) |
Zhen Tan, Bohan, wjldw, ympc08, chengshuaizhao |
Chain-of-Thought (CoT) reasoning in LLMs is primarily a brittle pattern-matching phenomenon, not genuine logical inference, fundamentally bounded by training data distribution. This research questions the nature of CoT reasoning, hypothesizing it reflects structured inductive biases learned from in-distribution data. The study uses DATAALCHEMY, a controlled environment for training LLMs from scratch, to systematically probe CoT across task, length, and format generalization. Findings reveal significant performance degradation under distribution shifts; for example, the exact match for transformation generalization dropped from 100% (in-distribution) to 0% (partial/out-of-distribution). Consequently, AI practitioners should be wary of over-reliance on CoT for robust reasoning, especially in critical applications, and prioritize rigorous out-of-distribution testing. |
| Efficient Agents: Building Effective Agents While Reducing Cost (Read more on arXiv or HuggingFace) |
Yue Hou, He Zhu, Pai Liu, Xavier Hu, Ningning Wang |
This research systematically analyzes the efficiency-effectiveness trade-off in LLM-driven agents and proposes EFFICIENT AGENTS, a framework that achieves near state-of-the-art performance with significantly reduced operational cost. The main objective is to quantify the impact of different architectural components (LLM backbone, planning, tools, memory, test-time scaling) on agent performance and cost, and to identify an optimal configuration for cost-effective agent design on complex tasks. The study employs an empirical analysis on the GAIA benchmark, systematically varying individual agent components while using the cost-of-pass metric—the ratio of inference cost to success rate—to evaluate the efficiency-performance trade-off of each design choice. The primary result is the EFFICIENT AGENTS framework, which retains 96.7% of the performance of the OWL framework while achieving a 28.4% improvement in the cost-of-pass metric. The analysis reveals that simpler designs, such as a memory module that only retains historical observations and actions, can outperform more complex architectures in both effectiveness and efficiency. The principal implication for AI practitioners is that agent systems can be made more economically viable by avoiding over-engineering; specifically, by choosing a moderately complex planning horizon (e.g., a maximum of 8 steps), using simple memory configurations, and simplifying tool operations, as adding complexity often yields diminishing returns at a high computational cost. |
| SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from |
|
|
| Experience (Read more on arXiv or HuggingFace) |
Xiaoyi Dong, Yuhang Cao, Ziyu Liu, yuhangzang, Zery |
SEAgent is a self-evolving framework that enables Computer Use Agents (CUAs) to autonomously learn to operate unfamiliar software through experiential learning, curriculum generation, and a specialist-to-generalist training strategy. The primary objective is to develop a system that allows CUAs to master novel software environments by learning directly from interaction experience, thus eliminating the need for human-labeled data. The methodology combines a World State Model for step-wise trajectory evaluation, a Curriculum Generator for producing progressively difficult tasks, and a reinforcement learning policy updated via Group Relative Policy Optimization (GRPO) for successful actions and adversarial imitation for failures. On the OS-World benchmark, the specialist-to-generalist SEAgent achieved a 34.5% overall success rate, representing a 23.2% absolute improvement over the 11.3% success rate of the baseline UI-TARS agent. For AI practitioners, this work provides a blueprint for creating agents that can adapt to new software tools on-the-fly, reducing dependency on static, human-curated datasets and enabling more versatile, continuously evolving autonomous systems through self-generated experience. |
| Agent Lightning: Train ANY AI Agents with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zilong Wang, Xufang Luo, SiyunZhao, hzy46, ultmaster |
The paper presents Agent Lightning, a framework that enables reinforcement learning-based training for any AI agent by completely decoupling agent execution from the training process. The primary objective is to create a universal method for optimizing complex, multi-turn agents by formulating their execution as a Markov Decision Process (MDP) and defining a unified data interface for collecting transitions. The key methodology is a hierarchical algorithm, LightningRL, which decomposes multi-step agent trajectories into individual transitions, and a “Training-Agent Disaggregation” architecture that separates the RL training server from the agent runtime. Experiments show stable performance gains, with a text-to-SQL agent on the Spider dataset improving its test reward score from approximately 0.1 to over 0.55. The principal implication for AI practitioners is the ability to apply RL fine-tuning to existing agents developed with diverse frameworks (e.g., LangChain, AutoGen) with almost zero code modification, dramatically lowering the barrier to optimizing deployed agentic systems. |
| CoTox: Chain-of-Thought-Based Molecular Toxicity Reasoning and |
|
|
| Prediction (Read more on arXiv or HuggingFace) |
Donghyeon Lee, Soyon Park, Minju Song, Jueon Park, P-YI |
This paper presents CoTox, a Chain-of-Thought-based framework for interpretable molecular toxicity prediction using Large Language Models (LLMs). Its objective is to improve prediction accuracy and explainability over existing methods by integrating chemical structures with biological context. The methodology uses a structured prompt containing a compound’s IUPAC name, biological pathways, and Gene Ontology (GO) terms to guide an LLM through step-by-step reasoning for six organ toxicity types. CoTox, using GPT-4o, achieved a mean F1-score of 0.663, outperforming a Chemprop deep learning baseline (0.619), with Gemini-2.5-Pro obtaining the highest score of 0.700. For AI practitioners, the key implication is that for complex scientific domains, LLM performance is significantly enhanced by using Chain-of-Thought prompts with human-readable, multi-modal domain data (IUPAC names, pathways) over raw symbolic representations (SMILES). |
| Training Long-Context, Multi-Turn Software Engineering Agents with |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Maksim Nekrashevich, Ibragim Badertdinov, Sergei Polezhaev, Maria Trofimova, Alexander Golubev |
This paper details the application of reinforcement learning to train a long-context, multi-turn software engineering agent, improving its performance on real-world coding tasks. The research objective was to demonstrate that RL can effectively train LLMs in stateful, interactive environments, moving beyond simpler single-turn problems. The methodology involved a two-phase process on a Qwen2.5-72B-Instruct model: initial Rejection Fine-Tuning (RFT) followed by multi-turn RL training using a modified Decoupled Advantage Policy Optimization (DAPO) algorithm with sparse terminal rewards and context windows up to 131k tokens. This approach increased the agent’s Pass@1 success rate on the SWE-bench Verified benchmark from a 20% RFT baseline to 39.0%. For AI practitioners, this work presents a viable, teacher-free method to significantly enhance open-weight model capabilities for complex, interactive tasks like software engineering by optimizing directly on environmental feedback, offering an alternative to supervised fine-tuning on demonstration data. |
| Sotopia-RL: Reward Design for Social Intelligence (Read more on arXiv or HuggingFace) |
Keyang Xuan, Kolby Nottingham, Yining Zhao, Zhengyang Qi, Haofei Yu |
This paper introduces SOTOPIA-RL, a reinforcement learning framework that trains socially intelligent agents by refining coarse, episode-level feedback into utterance-level, multi-dimensional rewards. The research objective is to develop an effective RL training methodology for social agents that overcomes the challenges of partial observability (delayed effects of utterances) and multi-dimensionality (indirect contributions to goals) inherent in social interactions. The key methodology involves an offline phase where an LLM attributes episode-level outcomes across multiple dimensions (goal completion, relationship, knowledge) to individual utterances, and an online RL phase where a reward model is trained on these attributed rewards to guide policy optimization using Group Relative Policy Optimization (GRPO). Experiments demonstrate that SOTOPIA-RL achieves a state-of-the-art social goal completion score of 7.17 on the SOTOPIA-hard benchmark, significantly outperforming baselines. For AI practitioners, the principal implication is that designing fine-grained, multi-dimensional reward signals, generated offline by a capable LLM, is a critical strategy for stabilizing RL training and improving agent performance in complex, interactive tasks with sparse rewards. |
| LaTCoder: Converting Webpage Design to Code with Layout-as-Thought (Read more on arXiv or HuggingFace) |
Tianpeng Lv, Guohao Wang, Zhongyi Zhang, Zhen Li, starmage520 |
LaTCoder proposes a Layout-as-Thought (LAT) approach to convert webpage designs to code, significantly improving layout preservation using Multimodal Large Language Models (MLLMs). The research aims to overcome MLLM limitations in accurately preserving webpage layout during design-to-code generation, specifically minimizing the visual discrepancy between generated and original designs. LaTCoder utilizes a three-component methodology: layout-aware division of designs into image blocks, block-wise code synthesis via CoT-based prompting of MLLMs (e.g., DeepSeek-VL2, Gemini, GPT-4o), and layout-preserved assembly using absolute positioning or MLLM-based strategies with dynamic selection. On the CC-HARD dataset, LaTCoder with GPT-4o improved TreeBLEU by 60% and reduced Mean Absolute Error (MAE) by 43.23% compared to direct prompting, while human evaluators preferred LaTCoder-generated webpages in over 60% of cases. This approach provides AI/ML/Software Engineers with a robust strategy for UI automation, demonstrating that decomposing design-to-code tasks into layout-aware blocks significantly enhances MLLM performance and accuracy in complex webpage generation. |
| Web-CogReasoner: Towards Knowledge-Induced Cognitive Reasoning for Web |
|
|
| Agents (Read more on arXiv or HuggingFace) |
Xinyu Yang, Hongliang He, Aiwen Sun, Cong Guo, Gnonymous |
This paper introduces Web-CogReasoner, a web agent trained on a structured, multi-layered knowledge framework to improve cognitive reasoning for web navigation tasks. The research objective is to enhance agent performance by systematically building its capabilities through distinct stages of Factual, Conceptual, and Procedural knowledge acquisition, inspired by Bloom’s Taxonomy. The methodology involves constructing the Web-CogDataset for training, the Web-CogBench for evaluation, and developing a knowledge-driven Chain-of-Thought (CoT) reasoning process to guide the agent. The Web-CogReasoner achieves a state-of-the-art success rate of 30.2% for open-source agents on the WebVoyager benchmark and an overall score of 84.4% on the authors’ Web-CogBench, outperforming baselines like Gemini 2.5 Pro (80.2%). The principal implication for AI practitioners is that a curriculum-based training approach, which explicitly teaches an agent different cognitive layers of knowledge (perception, comprehension, and planning), is a highly effective strategy for building more robust and generalizable web agents compared to monolithic fine-tuning. |
| HPSv3: Towards Wide-Spectrum Human Preference Score (Read more on arXiv or HuggingFace) |
Hongsheng Li, Keqiang Sun, Xiaoshi Wu, Yuhang Ma |
This research introduces HPSv3, a human preference score, and HPDv3, a wide-spectrum dataset, for evaluating and improving text-to-image generation models. The primary objective is to develop a more robust, human-aligned evaluation metric that addresses the narrow data coverage and suboptimal design of existing preference models. The methodology involves creating the HPDv3 dataset, which contains 1.08M text-image pairs including high-quality real photos, and training the HPSv3 model on it using a Vision-Language Model (VLM) backbone and an uncertainty-aware ranking loss. HPSv3 achieves state-of-the-art alignment with human judgments, attaining a Spearman correlation of 0.94 with human rankings and a 76.9% preference prediction accuracy on the HPDv3 test set. AI practitioners can leverage the HPSv3 model as a superior automated metric for text-to-image evaluation and use the proposed Chain-of-Human-Preference (CoHP) method as a training-free technique to iteratively refine image generation quality. |
| Gaussian Variation Field Diffusion for High-fidelity Video-to-4D |
|
|
| Synthesis (Read more on arXiv or HuggingFace) |
Feng Zhao, Jiaolong Yang, Chuxin Wang, Sicheng Xu, BwZhang |
This paper introduces a novel framework, GVFDiffusion, for generating high-fidelity, temporally coherent 4D objects from single video inputs by modeling temporal variations in a compact latent space. The main objective is to overcome the challenges of expensive data acquisition and high-dimensional modeling in 4D synthesis. The methodology involves a Direct 4DMesh-to-GS Variation Field VAE to efficiently encode motion into a latent space, and a temporal-aware Diffusion Transformer to generate these latent variations conditioned on video and a canonical 3D Gaussian Splatting (GS) representation. The model achieves state-of-the-art performance, demonstrating a Fréchet Video Distance (FVD) of 476.83, which is a significant improvement over the next-best prior method (529.10). For AI practitioners, this work provides an efficient pipeline that decomposes 4D generation into static geometry and dynamic variation, enabling the creation of high-quality animated 3D assets from video with substantially lower computational cost than alternative methods. |
| LeanK: Learnable K Cache Channel Pruning for Efficient Decoding (Read more on arXiv or HuggingFace) |
Yuqing Yang, Chengruidong Zhang, Huiqiang Jiang, hzy46, zhangyik21 |
i) LeanK is a learning-based method that prunes unimportant channels in the Key (K) cache of Large Language Models (LLMs) to accelerate long-context decoding by leveraging static channel sparsity. ii) The objective is to develop a method to prune the K cache channel dimension to reduce GPU memory usage and improve decoding speed for long-context LLM inference without significant performance degradation. iii) The methodology is a two-stage training process: first, a continuous scaling factor representing global channel importance is learned using L2 distillation loss and L1 regularization; second, this factor is converted into a static, hardware-aligned binary mask for efficient deployment. iv) Experiments on models like Llama-3.1-8B show LeanK achieves up to 70% K cache and 16%-18% V cache memory reduction, with a custom kernel enabling a 1.3x speedup in attention computation while maintaining near-lossless model accuracy on benchmarks like RULER. v) The principal implication for AI practitioners is that they can use LeanK’s pre-trained static mask to significantly reduce the memory footprint and latency of long-context inference, enabling the deployment of large models on more resource-constrained hardware or allowing for larger batch sizes without runtime overhead for calculating sparsity. |
| Sculptor: Empowering LLMs with Cognitive Agency via Active Context |
|
|
| Management (Read more on arXiv or HuggingFace) |
Yunxin Liu, Ting Cao, Qitai Tan, L. H. Xu, Mor-Li |
The paper introduces Sculptor, a framework of cognitive tools that enables LLMs to actively manage their internal context, thereby mitigating proactive interference and improving performance on long-context reasoning tasks. The primary objective is to investigate if empowering LLMs with tools for Active Context Management (ACM)—such as fragmenting, hiding, and searching the input context—can overcome performance degradation caused by proactive interference in long-sequence tasks. The methodology introduces the Sculptor tool suite, featuring functions like fragment_context, fold_fragment, and search_context. Its effectiveness is evaluated by leveraging the zero-shot tool-calling capabilities of LLMs like Claude-4-Sonnet and GPT-4.1 on the PI-LLM and NeedleBench benchmarks. The paper explicitly states that results from a proposed Reinforcement Learning training approach are not yet available. Primary results show that Sculptor-augmented models achieved significant gains on multi-needle reasoning tasks; on the NeedleBench benchmark, Claude-4-Sonnet’s accuracy increased from 67.0% to 94.0%. However, on the PI-LLM benchmark, results were mixed, with GPT-4.1 improving by 5.54 points while DeepSeek-V3’s performance decreased by 5.93 points, indicating that zero-shot tool-use generalization varies across models. The principal implication for AI practitioners is that implementing explicit context management mechanisms is a viable strategy for improving LLM robustness in long-context scenarios. The most impactful finding is that equipping models with tools to actively curate their “working memory” can dramatically improve performance on tasks requiring the integration of sparse information, suggesting that engineering active attentional control is as critical as expanding raw context capacity. |
| Position: The Current AI Conference Model is Unsustainable! Diagnosing |
|
|
| the Crisis of Centralized AI Conference (Read more on arXiv or HuggingFace) |
Jiaying Wu, Qian Wang, Andre Huikai Lin, Moming Duan, nuojohnchen |
i) This paper provides a data-driven diagnosis of the unsustainability of the current centralized AI conference model and proposes a decentralized, community-federated alternative. ii) The objective is to quantify the structural crisis in AI conferences across scientific, environmental, psychological, and logistical dimensions, and to propose a new, more sustainable model. iii) The study employs a multi-pronged methodology, including quantitative analysis of publication trends from CSRankings.org, carbon footprint modeling based on author affiliations, computational sentiment analysis of 405 Reddit threads using VADER, and systemic strain analysis using official conference statistics. iv) The paper identifies a crisis characterized by unsustainable growth, with key findings including a doubling of per-author publication rates to over 4.5 papers annually in the last decade and a significant psychological toll, where over 71% of analyzed online community discourse about conferences reflects negative sentiment and 35% of those negative threads reference mental health concerns. v) The principal implication for AI practitioners is that the current hyper-competitive publication environment incentivizes incremental “SOTA-hacking” over deep, innovative research, impacting project selection and career progression for engineers and scientists while contributing to widespread burnout. The proposed Community-Federated Conference (CFC) model suggests a fundamental shift in how research is reviewed and disseminated, which would alter the mechanisms for collaboration and knowledge exchange practitioners rely on. |
| Enhancing Vision-Language Model Training with Reinforcement Learning in |
|
|
| Synthetic Worlds for Real-World Success (Read more on arXiv or HuggingFace) |
Ruslan Rakhimov, Viacheslav Sinii, Stanislav Dereka, kefirski, GeorgeBredis |
This paper introduces Vision-Language Decoupled Actor-Critic (VL-DAC), a reinforcement learning algorithm designed to enhance Vision-Language Model (VLM) training in synthetic environments for improved real-world performance. The primary objective is to develop a robust, hyperparameter-free RL algorithm capable of training VLMs for multi-turn interactive tasks, overcoming limitations in long-horizon reasoning and credit assignment, and ensuring generalization beyond training simulators. VL-DAC employs a decoupled training approach, applying Proximal Policy Optimization (PPO) updates token-wise for actions while learning value only at the environment-step level, with gradients stopped at the VLM backbone, and incorporates stabilization techniques including KL regularization and value warm-up. VL-DAC training in inexpensive simulators yields policies with wide generalization, including a +50% relative gain on BALROG for agentic control, +5% relative on VSI-Bench for spatial planning, and +2% on VisualWebBench for web navigation, without degrading image understanding accuracy. This work demonstrates that VLMs can acquire transferable real-world competence by training entirely in cost-effective synthetic worlds using a straightforward RL algorithm, providing a practical and scalable path for developing interactive multimodal agents. |
| EVOC2RUST: A Skeleton-guided Framework for Project-Level C-to-Rust |
|
|
| Translation (Read more on arXiv or HuggingFace) |
Dong Chen, Jie Wang, Tingrui Yu, Chaofan Wang, YerbaPage |
EVOC2RUST is a hybrid framework using Large Language Models (LLMs) and static analysis for automated, project-level C-to-Rust code translation. The objective is to create an automated system that translates entire C projects into semantically equivalent and memory-safe Rust code, addressing challenges of linguistic differences and cross-module dependencies. The methodology involves a three-stage pipeline: 1) It constructs a compilable Rust “skeleton” by decomposing the C project and translating definitions and function signatures using a feature-mapping-enhanced LLM. 2) It incrementally translates function bodies to populate the skeleton. 3) It uses a cascading, compilation-driven repair process integrating LLMs and rule-based static analysis to fix errors. On the industrial C2R-Bench dataset, EVOC2RUST achieved a 93.84% incremental compilation pass rate and a 97.41% code safety rate, significantly outperforming purely LLM-based and rule-based baselines. For AI practitioners, this research provides a blueprint for applying LLMs to large-scale, safety-critical code migration tasks by demonstrating that a structured, hybrid approach—combining skeleton-guided generation, LLMs augmented with expert-defined transformation rules, and iterative repair—is substantially more effective than unconstrained, end-to-end LLM generation. |
| DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a |
|
|
| Stage-Wise Diffusion Transformer Framework (Read more on arXiv or HuggingFace) |
Chao Liang, Ente Lin, Shuliang Ning, Zaiyu Huang, Tongchun Zuo |
DreamVVT is a two-stage Diffusion Transformer framework that generates realistic and temporally coherent video virtual try-on for in-the-wild scenarios. The research objective is to preserve fine-grained garment details and maintain temporal consistency in unconstrained videos, addressing the failures of existing end-to-end methods. The key methodology involves first synthesizing high-fidelity try-on images for select keyframes, and second, using a LoRA-adapted pretrained video generation model conditioned on these keyframes, pose data, and VLM-generated text to produce the final video. Quantitatively, the framework achieves a state-of-the-art VFID(I3D) score of 11.0180 on the ViViD dataset, outperforming prior works. The principal implication for AI practitioners is that a modular, two-stage approach using parameter-efficient LoRA fine-tuning on pretrained backbones can achieve superior generalization and fidelity for specialized video synthesis tasks, reducing reliance on large, paired training datasets. |
| A Coarse-to-Fine Approach to Multi-Modality 3D Occupancy Grounding (Read more on arXiv or HuggingFace) |
Jianke Zhu, Junbo Chen, Zhan Shi, songw-zju |
This paper introduces GroundingOcc, a coarse-to-fine multi-modal model, and the Talk2Occ benchmark for the novel task of 3D occupancy grounding from natural language. The primary objective is to move beyond bounding-box-based visual grounding by developing a method to predict fine-grained, voxel-level 3D occupancy for objects described in language, enabling more precise spatial perception. The proposed GroundingOcc model fuses features from images, LiDAR, and text, employing auxiliary tasks like 2D grounding and a depth predictor supervised by occupancy-rendered depth maps to enhance geometric understanding. On the new Talk2Occ benchmark, the refined model (GroundingOcc-Refine) achieves 32.68% accuracy at an IoU of 0.25, significantly outperforming the strongest baseline’s 21.10%. For AI practitioners, this work provides a public benchmark and a validated approach for developing systems that require detailed, non-axis-aligned understanding of object shapes, which is critical for advanced robotic interaction and motion planning. |
| RL-PLUS: Countering Capability Boundary Collapse of LLMs in |
|
|
| Reinforcement Learning with Hybrid-policy Optimization (Read more on arXiv or HuggingFace) |
Kechi Zhang, Huanyu Liu, Yongding Tao, Xue Jiang, Yihong Dong |
RL-PLUS is a novel hybrid-policy optimization approach that synergizes internal exploitation with external data to counter the capability boundary collapse in LLMs trained with Reinforcement Learning with Verifiable Reward (RLVR). The primary objective is to address how on-policy RLVR methods often narrow an LLM’s problem-solving scope, preventing it from acquiring new reasoning abilities that surpass the base model’s inherent boundaries. The key methodology involves two components: Multiple Importance Sampling (MIS) to resolve distributional mismatch from external data, and an Exploration-Based Advantage Function to guide the model toward novel, high-value reasoning paths. The paper reports that on six math reasoning benchmarks, RL-PLUS achieves state-of-the-art performance, outperforming a strong SFT+GRPO baseline by an average of 5.2 points. The principal implication for AI practitioners is that standard on-policy RL can be insufficient for expanding a model’s core capabilities; the RL-PLUS framework offers a more effective method to integrate external knowledge, break through performance ceilings, and resolve the capability boundary collapse problem. |
| Reasoning Language Models for Root Cause Analysis in 5G Wireless |
|
|
| Networks (Read more on arXiv or HuggingFace) |
Haozhe Zhang, Yibin Kang, Antonio De Domenico, Mohamed Sana, nicopi |
This paper proposes a framework to fine-tune Large Language Models for Root Cause Analysis (RCA) in 5G networks, supported by a new synthetic dataset called TeleLogs. The objective is to improve the accuracy and reasoning quality of LLMs for network troubleshooting by integrating domain knowledge and generating structured, multi-step diagnostic explanations. The key methodology is a two-stage training process that combines Supervised Fine-Tuning (SFT) on high-quality data generated by a multi-agent pipeline, followed by Reinforcement Learning (RL) using Group Relative Policy Optimization (GRPO). The primary result shows that a fine-tuned Qwen2.5-32B model achieves 95.86% pass@1 accuracy, significantly outperforming state-of-the-art reasoning models like DeepSeek-R1 (29.42%) and its own base model (18.85%). The principal implication for AI practitioners is that a targeted, two-stage fine-tuning approach can enable LLMs to perform highly specialized, complex reasoning tasks with high accuracy and explainability, making them viable for practical deployment in critical domains like network operations. |
| IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with |
|
|
| Verifiable Rewards (Read more on arXiv or HuggingFace) |
Ling-I Wu, Xiaogui Yang, Tong Jian, Tianyi Liang, Xu Guo |
The paper introduces IFDecorator, a framework that wraps Reinforcement Learning with Verifiable Rewards (RLVR) to improve instruction following by automatically calibrating data difficulty and using dedicated modules to enforce intent and detect reward hacking. The primary objective is to mitigate over-optimization (reward hacking) and improve training efficiency in RLVR for instruction following (RLVR4IF), where models exploit verification shortcuts instead of adhering to the user’s actual intent. The methodology combines three components: a cooperative-adversarial data flywheel that generates progressively challenging instruction-verification pairs for curriculum learning; an “IntentCheck” module that directly assesses intent alignment to provide a more robust reward; and “trip wires,” diagnostic trap instructions used to measure reward hacking behaviors without influencing the training signal. The framework significantly improves instruction following capabilities; the Qwen2.5-32B-Instruct-IFDecorator model achieves 87.43% accuracy on the IFEval benchmark, a +7.95 percentage point improvement over its baseline, while preserving the model’s general capabilities. For AI practitioners, IFDecorator provides a robust method to fine-tune LLMs for more reliable instruction adherence, directly counteracting the common failure mode of reward hacking in RL-based alignment and resulting in models that better fulfill user intent. |
| OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers |
|
|
| for Biomedical NER Across 12 Public Datasets (Read more on arXiv or HuggingFace) |
MaziyarPanahi |
This paper presents OpenMed NER, an open-source framework for biomedical named-entity recognition (NER) that achieves state-of-the-art performance with high computational efficiency. The primary objective is to create a suite of accessible, high-performing models that can surpass closed-source systems on a wide array of biomedical tasks. The methodology combines lightweight domain-adaptive pre-training (DAPT) with parameter-efficient Low-Rank Adaptation (LoRA) on strong transformer backbones like DeBERTa-v3, adapting them to a 350k-passage biomedical corpus before task-specific fine-tuning. The models establish new state-of-the-art micro-F1 scores on 10 of 12 public datasets, including a +9.72 percentage point improvement on the challenging CLL corpus. The principal implication for AI practitioners is that strategic, parameter-efficient adaptation of existing open-source models can yield superior performance to resource-intensive, from-scratch training, enabling the development of SOTA specialized models in under 12 hours on a single GPU. |
| SonicMaster: Towards Controllable All-in-One Music Restoration and |
|
|
| Mastering (Read more on arXiv or HuggingFace) |
Ambuj Mehrish, Jan Melechovsky, dorienh |
The paper introduces SonicMaster, a unified, text-controllable, flow-matching generative model for simultaneous music restoration and mastering. The research objective is to develop a single framework that corrects a broad spectrum of audio degradations—including equalization, dynamics, reverb, and clipping—guided by natural language prompts, replacing traditional multi-tool workflows. The methodology involves training a Multimodal Diffusion Transformer (MM-DiT) using a rectified flow paradigm on the novel SonicMaster dataset, which contains 175,000 pairs of programmatically degraded audio and corresponding text instructions. Results show the model significantly improves audio quality, reducing Kullback-Leibler (KL) divergence from 5.131 (degraded input) to 0.888 and increasing the Production Quality (PQ) score from 7.026 to 7.705 on a comprehensive test set. For AI practitioners, this work validates using a single, text-conditioned generative model to consolidate complex, multi-stage processing pipelines, providing a paradigm for creating unified solutions to multifaceted restoration tasks. |
| IAUNet: Instance-Aware U-Net (Read more on arXiv or HuggingFace) |
Dmytro Fishman, Ali Zeynalli, Illia Tsiporenko, YaroslavPrytula |
This paper introduces IAUNet, a query-based U-Net architecture featuring a lightweight convolutional Pixel decoder and a multi-scale Transformer decoder for biomedical instance segmentation. The primary objective is to enhance the standard U-Net for complex instance segmentation tasks, such as identifying overlapping cells, by integrating instance-aware query refinement across multiple feature scales. The methodology combines a U-Net backbone with a custom Pixel decoder and a Transformer decoder that iteratively updates learnable object queries using multi-scale mask features, with deep supervision applied at each decoder stage. On the newly introduced Revvity-25 dataset, IAUNet with a ResNet-50 backbone achieves an Average Precision (AP) of 49.7, outperforming models like Mask2Former (46.4 AP) while using fewer parameters (39M vs. 44M). For AI practitioners, this work provides a blueprint for creating efficient and high-performing instance segmentation models by hybridizing the well-established U-Net convolutional framework with modern, lightweight query-based mechanisms, offering a resource-efficient alternative to larger, purely Transformer-based architectures. |
| Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Hao Huang, Shiqi Jiang, Haiwen Huang, Nan Xiang, tianyilt |
Sel3DCraft is an interactive visual prompt engineering system that transforms unstructured text-to-3D (T23D) generation into a guided, user-friendly process. The research objective is to develop a visual approach that replaces the costly trial-and-error prompting common in T23D tools with structured exploration and iterative refinement. Its methodology features a dual-branch architecture for candidate synthesis (combining retrieval and generation), a multi-view hybrid scoring function leveraging Multi-modal Large Language Models (MLLMs) to assess eight semantic dimensions of 3D models, and a visual analytics suite with a treemap wordle for prompt recommendation. A user study demonstrated that Sel3DCraft reduces model creation time by 70.5% (118.83s vs 402.17s) and prompt iterations by 66.2% compared to baseline systems, while significantly improving output quality ratings. The principal implication for AI practitioners is the provision of a framework that integrates MLLMs as automated, multi-dimensional evaluators within a human-in-the-loop system to enhance the controllability and efficiency of complex generative models. |
| The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in |
|
|
| Text-to-Image Models (Read more on arXiv or HuggingFace) |
Elisabetta Rocchetti, Alfio Ferrara, sergiopicascia |
This research quantitatively analyzes how transformer-based text-to-image diffusion models internally represent and disentangle “content” and “style” concepts from artistic prompts. The main objective is to investigate how models interpret stylistic instructions and spatially separate the representation of what is depicted from how it is depicted without explicit supervision. The key methodology uses the Diffusion Attentive Attribution Maps (DAAM) technique to extract cross-attention heatmaps for content and style tokens, then computes the Intersection over Union (IoU) between their corresponding image regions to measure conceptual overlap. The primary result is that models demonstrate an emergent, but highly variable, content-style separation; the IoU for content-style token pairs was, on average, 0.64 standard deviations lower than a baseline IoU, but certain styles like ‘Rembrandt’ showed negative separation (Δ = -0.07), indicating entanglement. The principal implication for AI practitioners is that a model’s capacity for content-style disentanglement is inconsistent and heavily influenced by training data biases, where frequent co-occurrences between subjects and artists can lead to conceptual blending and unpredictable generative behavior. |
| MiDashengLM: Efficient Audio Understanding with General Audio Captions (Read more on arXiv or HuggingFace) |
Yadong Niu, Jian Luan, Jizhong Liu, Gang Li, Heinrich Dinkel |
MiDashengLM is an open-source large audio-language model that uses a novel “general audio captioning” approach for efficient and comprehensive audio understanding, outperforming baselines in speed and many non-ASR tasks. The paper’s primary objective is to develop an efficient and transparent audio-language model that overcomes the limitations of ASR-centric pretraining by creating a holistic textual representation (“general captions”) that fuses speech, sound, and music information from audio. The key methodology involves aligning a Dasheng audio encoder with a Qwen2.5-Omni language model using a newly created dataset, ACAVCaps, which contains “general audio captions” generated by a multi-expert annotation pipeline. The model architecture is optimized for efficiency with variable-length inputs and a low 5 Hz audio feature framerate. The primary result is a significant improvement in efficiency, with MiDashengLM achieving up to 20.2x higher inference throughput and 4x faster time-to-first-token than the Qwen2.5-Omni-7B baseline. The model’s Dasheng encoder also outperforms the Whisper-Large v3 encoder on 18 out of 22 diverse audio tasks. The principal implication for AI practitioners is that MiDashengLM provides an open, highly efficient foundation model for applications needing broad audio understanding, demonstrating that pretraining on rich “general captions” is a powerful alternative to ASR-based alignment for developing versatile and fast audio-language systems. |
| Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and |
|
|
| Self-Checking for Complex Instruction Following (Read more on arXiv or HuggingFace) |
Liang Xu, Xiangzheng Zhang, Shousheng Jia, Liang Wen, Chenyang Wang |
i) 1-line summary: The Light-IF framework improves LLM complex instruction-following by inducing a generalizable “preview and self-checking” reasoning pattern through a multi-stage training process. ii) Main research question or objective: The paper’s primary objective is to mitigate the “lazy reasoning” pattern observed in LLMs when faced with complex instructions, aiming to instill a more rigorous and generalizable reasoning process that ensures strict constraint adherence. iii) Key methodology used: The framework uses a multi-stage pipeline: it first synthesizes hardness-aware prompts, then applies Zero-RL to a base model to elicit detailed reasoning, extracts high-quality responses for a cold-start dataset, and finally trains the model using Entropy-Preserving SFT (Entropy-SFT) and Token-wise Entropy-Adaptive RL (TEA-RL) with dense rewards. iv) Primary results (include at least one specific quantitative finding): The resulting Light-IF-32B model substantially outperforms other models on instruction-following benchmarks, achieving a score of 0.575 on SuperClue, which is 13.9 points higher than the next-best open-source model evaluated. v) Principal implication for AI practitioners: The key implication is that practitioners can instill complex, generalizable reasoning behaviors in LLMs using targeted RL and novel entropy control techniques (Entropy-SFT, TEA-RL), providing a practical and data-efficient method to enhance reliability for constraint-heavy tasks without relying on massive supervised datasets. |
Papers for 2025-08-06
| Title |
Authors |
Summary |
| Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed |
|
|
| Inference (Read more on arXiv or HuggingFace) |
Fan Xia, Pengyang Gao, Cheng Luo, Zheng Zhang, Yuxuan Song |
This paper introduces Seed Diffusion Preview, a discrete-state diffusion language model for code generation that achieves high-speed, parallel inference while maintaining competitive performance. The objective is to mitigate the inference latency of token-by-token decoding by developing a model capable of non-sequential, parallel generation. The methodology combines a two-stage curriculum using mask-based and edit-based corruption, constrained-order training on distilled generation trajectories, and an on-policy learning paradigm to explicitly shorten inference paths. The model achieves an inference speed of 2,146 tokens/second on H20 GPUs and a 54.3% pass@1 score on the CanItEdit benchmark, establishing a new state-of-the-art on the speed-quality Pareto frontier for code models. For AI practitioners, this work demonstrates a viable architecture for deploying high-throughput language models that significantly reduces inference latency without a substantial loss in quality, offering a compelling alternative to traditional autoregressive systems for latency-sensitive applications. |
| Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding |
|
|
| and Generation (Read more on arXiv or HuggingFace) |
Tianyidan Xie, Liang Hu, Yimeng Gan, Yi Peng, Peiyu Wang |
Skywork UniPic is a 1.5 billion-parameter unified autoregressive model for visual understanding, generation, and editing. The research objective is to create a compact, single-architecture model that excels at these multimodal tasks while remaining efficient enough for deployment on commodity hardware. Its key methodology involves a decoupled encoding strategy, utilizing a Masked Autoregressive (MAR) encoder for generation and a SigLIP2 encoder for understanding, with both feeding into a shared autoregressive LLM decoder, trained via a progressive, resolution-aware curriculum. The model achieves state-of-the-art results, including a new record of 85.5 on the DPG-Bench for complex generation and a 5.83 score on GEditBench-EN for editing. The principal implication for AI practitioners is that high-fidelity, unified multimodal AI systems can be developed and deployed effectively without prohibitive computational resources, making advanced capabilities more accessible. |
| LongVie: Multimodal-Guided Controllable Ultra-Long Video Generation (Read more on arXiv or HuggingFace) |
Chenyang Si, Jianfeng Feng, Xian Liu, Zhaoxi Chen, Jianxiong Gao |
LongVie is an autoregressive framework that generates controllable, ultra-long (up to one minute) videos by combining multimodal guidance with specific strategies to ensure temporal consistency and visual quality. The primary objective is to overcome the temporal inconsistency and visual degradation that occur when scaling existing controllable short-video generation models to longer durations using autoregressive methods. The methodology extends a pre-trained video diffusion model with a multi-modal ControlNet-style architecture that accepts both dense (depth maps) and sparse (point maps) control signals. Temporal consistency is enforced through a unified noise initialization strategy across all generated clips and global normalization of control signals over the entire video, while a degradation-aware training strategy balances the influence of each modality. On the introduced LongVGenBench benchmark, LongVie achieves state-of-the-art performance, outperforming all baselines in consistency and quality. Quantitatively, it obtains the best perceptual similarity score with a LPIPS of 0.290 and the highest Overall Consistency score of 21.82%. The principal implication for AI practitioners is that the techniques of unified noise initialization and global control normalization provide a concrete, effective method to adapt existing short-video diffusion models for coherent, long-form, controllable video synthesis, directly addressing common failure modes like flickering and content drift. |
| CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and |
|
|
| Outcome Reward (Read more on arXiv or HuggingFace) |
Songyang Gao, Linchen Xiao, Junnan Liu, Hongwei Liu, Shudong Liu |
This paper introduces CompassVerifier, a lightweight and robust verifier model, and VerifierBench, a comprehensive benchmark designed to systematically evaluate the answer verification capabilities of LLMs. The main objective is to develop a unified, accurate, and generalizable verifier model for LLM outputs that overcomes the limitations of regex-based matching and general-purpose LLM judges, and to create a challenging benchmark to systematically evaluate such verification capabilities. The methodology involves creating VerifierBench by collecting over 1 million LLM responses and using a multi-stage filtering pipeline with multi-expert voting and human annotation. CompassVerifier is trained on this data and enhanced using three key techniques: Error-Driven Adversarial Augmentation, Complex Formula Augmentation, and Generalizability Augmentation. The primary result is that CompassVerifier-32B achieves a new state-of-the-art average F1 score of 87.7% on VerifierBench, significantly outperforming both general LLMs like GPT-4o (59.1% F1) and other specialized verifiers. The 3B parameter version of CompassVerifier surpasses GPT-4.1 by an absolute F1-score of 10.6%. The principal implication for AI practitioners is that CompassVerifier can be used as a more accurate, robust, and computationally efficient tool for automated LLM evaluation and as a reward model in reinforcement learning, providing more reliable feedback signals for model optimization than existing methods. |
| CRINN: Contrastive Reinforcement Learning for Approximate Nearest |
|
|
| Neighbor Search (Read more on arXiv or HuggingFace) |
Jiwei Li, Chris Shum, Albert Wang, Xiaofei Sun, Xiaoya Li |
The paper introduces CRINN, a framework using contrastive reinforcement learning with LLMs to automatically optimize Approximate Nearest Neighbor Search (ANNS) algorithms. The primary objective is to automate the optimization of ANNS implementations by treating it as an RL problem where an LLM learns to generate progressively faster code based on execution speed feedback. CRINN employs a contrastive RL methodology where the LLM is prompted with pairs of code implementations and their performance scores, guiding it to learn effective optimization patterns, with a scalar reward derived from the area under the QPS-recall curve within the [0.85, 0.95] range. On the MNIST-784 benchmark, CRINN achieved an 85.25% improvement in Queries Per Second (QPS) over the best baseline at a 0.999 recall level. For AI practitioners, this research demonstrates that RL-augmented LLMs can automate sophisticated, performance-critical code optimization, reducing the reliance on manual expert tuning for systems like vector databases. |
| Tool-integrated Reinforcement Learning for Repo Deep Search (Read more on arXiv or HuggingFace) |
Yanzhen Zou, Pengfei Gao, Qunhong Zeng, Chao Peng, Zexiong Ma |
The paper introduces ToolTrain, a two-stage framework that uses supervised fine-tuning and reinforcement learning to improve how LLMs use tools for localizing code defects in software repositories. The objective is to enhance an LLM agent’s multi-hop reasoning and tool-use capabilities to accurately identify faulty code from natural language issue descriptions. The core methodology involves first performing rejection-sampled supervised fine-tuning (SFT) on successful tool-use trajectories, followed by a tool-integrated reinforcement learning (RL) phase that uses an nDCG@k-based reward to optimize the agent’s search strategy. The primary result shows that a 32B parameter model trained with ToolTrain achieves a function-level Recall@5 of 68.55% on the SWE-Bench-Verified benchmark, outperforming a leading proprietary model, and this improved localization boosts the end-to-end issue resolution rate to 31.60%. For AI practitioners, this demonstrates that specialized, two-stage training (SFT then RL) can enable smaller, open-source models to surpass larger proprietary models on complex, domain-specific tool-use tasks, offering a viable path for developing high-performance, specialized AI agents for software engineering. |
| Multi-human Interactive Talking Dataset (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, Weijia Wu, Zeyu Zhu |
This paper introduces the Multi-human Interactive Talking (MIT) dataset and a baseline model, CovOG, for generating videos of multi-person conversations. The research objective is to address the limitations of single-person monologue generation by enabling the synthesis of realistic, full-body, multi-speaker interactions. The methodology involves an automated pipeline for collecting and annotating 12 hours of video with fine-grained pose and speech interaction data, and proposing the CovOG model which integrates a Multi-Human Pose Encoder (MPE) and an Interactive Audio Driver (IAD) with a diffusion-based framework. In quantitative evaluations, CovOG outperformed baselines, achieving a Frechet Inception Distance (FVD) of 307.35 on the combined test set, compared to 337.60 for the AnimateAnyone baseline. The principal implication for AI practitioners is the provision of the first specialized dataset and a validated baseline model for developing and benchmarking systems that can generate complex, controllable multi-person conversational videos, moving beyond single-subject animations. |
| Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data |
|
|
| Synthesis and Self-Correction (Read more on arXiv or HuggingFace) |
Jui-Hui Chung, Ziran Yang, Bohan Lyu, Shange Tang, Yong Lin |
Goedel-Prover-V2 introduces open-source language models that set a new state-of-the-art in automated formal theorem proving by improving on existing learning pipelines. The primary objective is to create models capable of solving increasingly complex mathematical theorems by integrating long-chain-of-thought reasoning with formal verification. The methodology combines three key innovations: scaffolded data synthesis to generate a curriculum of problems, verifier-guided self-correction using feedback from the Lean compiler, and model averaging to maintain output diversity. The flagship Goedel-Prover-V2-32B model achieves 90.4% pass@32 on the MiniF2F benchmark using self-correction, significantly outperforming previous, larger state-of-the-art systems. The principal implication for AI practitioners is that integrating formal verifier feedback loops and curriculum-based training enables smaller, more computationally efficient models to achieve superior performance on complex, formal reasoning tasks, providing a practical alternative to simply scaling model parameters. |
| LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools? (Read more on arXiv or HuggingFace) |
Yaojie Lu, Xuanang Chen, Jiawei Chen, Wenliang Zhong, Guozhao Mo |
This paper introduces LiveMCPBench, a benchmark to evaluate LLM agents’ ability to use a large-scale, real-world toolset (527 tools) via the Model Context Protocol (MCP) to complete 95 daily tasks. The research objective is to assess if current agents can effectively plan, retrieve, and execute actions across a dynamic, multi-server environment, testing their “meta-tool-learning” capabilities beyond simple, simulated tool use. The methodology comprises the LiveMCPTool toolset, an MCP Copilot Agent for task execution, and LiveMCPEval, an LLM-as-a-Judge framework for automated evaluation. Evaluation of 10 frontier models revealed significant performance disparities, with the top-performing model, Claude-Sonnet-4, achieving a 78.95% task success rate, while many other models struggled with severe underutilization of available tools. The principal implication for AI engineers is that current models are deficient in task decomposition and tool retrieval for complex environments; therefore, building robust agents requires focusing on engineering sophisticated planning and retrieval architectures, as “Retrieve Error” was a dominant failure mode, rather than relying solely on the base LLM’s reasoning. |
| LAMIC: Layout-Aware Multi-Image Composition via Scalability of |
|
|
| Multimodal Diffusion Transformer (Read more on arXiv or HuggingFace) |
Shunyu Yao, Kai Kang, Jianhua Wang, Zehua Ma, Yuzhuo Chen |
The paper introduces LAMIC, a training-free framework that extends single-reference diffusion transformers for layout-aware, multi-image composition using novel attention mechanisms. The primary objective is to enable a pretrained single-reference model to generate coherent images from multiple visual references with precise spatial layout control, without any retraining. The key methodology involves two plug-and-play attention mechanisms applied to a Multimodal Diffusion Transformer: Group Isolation Attention (GIA) to prevent interference between reference entities, and Region-Modulated Attention (RMA) to enhance layout precision during early denoising. In a four-reference composition task, LAMIC achieved an identity similarity (ID-S) score of 70.25, surpassing the second-best model by 8.41 points, and a layout Inclusion Ratio (IN-R) of 89.81, significantly outperforming all baselines. For AI practitioners, the principal implication is that LAMIC provides a zero-shot, resource-efficient method to adapt powerful single-reference foundation models for complex multi-subject, layout-controlled generation tasks, bypassing the need for specialized training data and model fine-tuning. |
| ChartCap: Mitigating Hallucination of Dense Chart Captioning (Read more on arXiv or HuggingFace) |
Gunhee Kim, Jaewoo Ahn, Junyoung Lim |
The paper introduces the CHARTCAP dataset, containing 565K real-world charts with dense, verified captions, and a new Visual Consistency Score (VCS) metric to improve chart captioning by vision-language models (VLMs). The main objective is to develop a large-scale, high-quality dataset of real-world chart-caption pairs that is free from extraneous information and contains dense, structured descriptions, enabling VLMs to generate more accurate captions with fewer hallucinations. The methodology combines a four-stage automated pipeline using multiple VLMs to generate schema-guided captions for 565K charts and a cycle consistency-based human verification process for quality control. It also proposes the Visual Consistency Score (VCS), a reference-free metric that evaluates captions by programmatically regenerating a chart from the text and measuring its visual similarity to the original. Models fine-tuned on CHARTCAP significantly outperform proprietary models and even human-annotated captions; on the VisText benchmark, the Phi3.5-Vision-4BCHARTCAP model achieved a Visual Consistency Score of 0.9443, surpassing the 0.9172 score of the human-authored ground-truth captions. For AI practitioners, the principal implication is that fine-tuning VLMs on a high-fidelity dataset curated with a domain-specific schema (like CHARTCAP) is a highly effective strategy to reduce hallucination and improve factual accuracy for structured data, with the proposed VCS offering a robust, reference-free method for evaluating performance in this domain. |
| AlignGuard-LoRA: Alignment-Preserving Fine-Tuning via Fisher-Guided |
|
|
| Decomposition and Riemannian-Geodesic Collision Regularization (Read more on arXiv or HuggingFace) |
Aman Chadha, Vinija Jain, Abhilekh Borah, Amitava Das |
ALIGNGUARD-LORA is a fine-tuning framework that mitigates alignment drift in LLMs by using Fisher-guided decomposition and specialized regularization to preserve safety behaviors. The main objective is to prevent the degradation of safety and behavioral constraints (alignment drift) during low-rank adaptation (LoRA) of large language models, without compromising downstream task performance. The key methodology involves decomposing LoRA parameter updates into an alignment-critical component and a task-specific component using a projection based on the Fisher Information Matrix (FIM). The framework then applies three forms of regularization: FIM-based regularization to constrain the alignment component, a separate penalty to stabilize the task component, and collision-aware regularization (using Riemannian and geodesic penalties) to minimize interference between them. Empirical evaluations show that ALIGNGUARD-LORA mitigates alignment drift by up to 50% on safety-critical benchmarks. On the introduced DRIFTCHECK benchmark, standard LoRA caused unsafe refusal accuracy to drop from 91.3% to 71.4%, while ALIGNGUARD-LORA maintained 92.3%, representing a 50% relative reduction in alignment drift. The principal implication for AI practitioners is that ALIGNGUARD-LORA can be used as a drop-in replacement for standard LoRA to more safely fine-tune already-aligned models, ensuring that critical safety guardrails are not eroded during adaptation for new tasks, which is crucial for the deployment of reliable models in production. |
| TRACEALIGN – Tracing the Drift: Attributing Alignment Failures to |
|
|
| Training-Time Belief Sources in LLMs (Read more on arXiv or HuggingFace) |
Aman Chadha, Vinija Jain, Amitava Das |
The TRACEALIGN framework traces LLM alignment drift to conflicting, memorized beliefs in the training corpus, moving beyond purely behavioral safety analysis. Its core methodology uses TRACEINDEX, a suffix-array search over training data, to find the provenance of generated text spans and scores their risk using the Belief Conflict Index (BCI), a metric based on token rarity. Three BCI-guided defenses—an inference-time filter (TRACESHIELD), a contrastive fine-tuning loss (CBD Loss), and a provenance-aware decoding strategy (Prov-Decode)—are introduced. On the paper’s Alignment Drift Benchmark, these interventions collectively reduce alignment drift by up to 85% while preserving utility. For AI practitioners, this provides a traceable, auditable toolkit to diagnose safety failures at their source and implement targeted mitigations grounded in data provenance rather than relying on opaque refusal classifiers. |
Papers for 2025-08-05
| Title |
Authors |
Summary |
| Qwen-Image Technical Report (Read more on arXiv or HuggingFace) |
Kaiyuan Gao, Junyang Lin, Jingren Zhou, Jiahao Li, Chenfei Wu |
The Qwen-Image Technical Report introduces an image generation foundation model that achieves state-of-the-art performance in complex text rendering and precise image editing. The research aims to develop a model that can follow complex, multifaceted prompts, particularly for rendering non-alphabetic languages like Chinese, and perform image editing with high visual and semantic consistency. The methodology combines a Multimodal Diffusion Transformer (MMDiT) with a frozen Qwen2.5-VL for text encoding, trained via a progressive curriculum learning strategy on a comprehensive data pipeline that includes large-scale synthesis of text-rich images, and employs a dual-encoding mechanism for editing tasks. Qwen-Image significantly outperforms existing models in Chinese text generation, achieving an overall accuracy of 58.30 on the ChineseWord benchmark, compared to 36.14 for GPT Image 1. For AI practitioners, this research demonstrates that targeted data synthesis and a multi-stage, curriculum-based training approach are highly effective for building foundation models with superior control over specific, challenging attributes like text rendering, enabling the development of more practical and precise multimodal applications. |
| SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic |
|
|
| Association and Long Story Comprehension (Read more on arXiv or HuggingFace) |
Liyan Xu, Lemao Liu, Yuqing Li, Jiangnan Li, Junjie Wu |
The paper introduces SitEmb, a situated embedding model and training paradigm that enhances dense retrieval by encoding short text chunks with their surrounding long-range context. The objective is to develop a text embedding model that represents short text chunks conditioned on their broader document context to improve retrieval, overcoming the limitations of both isolated short-chunking and monolithic long-chunking. The key methodology is a residual learning framework where a “situated” model is trained to learn the residual from a baseline chunk-only model, forcing it to focus on contextual information, using training data constructed from user-annotated book notes and QA datasets. On the full Book Plot Retrieval task, the 8B parameter SitEmb-v1.5 model achieves a Recall@50 of 82.70, significantly outperforming its base model’s score of 69.48; experiments also show that existing state-of-the-art embedding models degrade in performance when provided the same situated context in a zero-shot setting. For AI practitioners, the principal implication is that for RAG over long documents, using specialized situated embedding models to encode short, localized passages with awareness of their surrounding context is a more effective strategy for improving retrieval relevance than simply increasing chunk size. |
| CellForge: Agentic Design of Virtual Cell Models (Read more on arXiv or HuggingFace) |
Daniel Shao, Yan Cui, Jiapeng Chen, Zhuoyun Yu, Xiangru Tang |
CellForge is an agentic system that autonomously designs, codes, and optimizes computational models for virtual cells directly from raw biological data and research objectives. The primary objective is to automate the end-to-end scientific workflow of virtual cell modeling to predict cellular responses to diverse perturbations like gene knockouts and drug treatments. The methodology involves a multi-agent framework with modules for Task Analysis, collaborative Method Design via a graph-based expert discussion, and Experiment Execution for automated code generation and self-debugging. In single-cell perturbation prediction tasks across six datasets, CELLFORGE consistently outperforms state-of-the-art methods, achieving up to a 40% reduction in prediction error and a 20% improvement in correlation metrics. For AI practitioners, this work demonstrates that an iterative, multi-agent collaborative reasoning framework can autonomously design and implement superior, domain-specific deep learning architectures without relying on fixed model templates or human intervention. |
| Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report (Read more on arXiv or HuggingFace) |
Anu Vellore, Baturay Saglam, Blaine Nelson, Paul Kassianik, Sajana Weerawardhena |
This technical report introduces Foundation-Sec-8B-Instruct, an 8B-parameter language model specialized for cybersecurity dialogue and instruction-following tasks. The primary objective was to adapt a cybersecurity domain-specialized base model, Foundation-Sec-8B, into a conversational assistant by applying instruction-tuning and human preference alignment. The methodology involved applying Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to the base model, using a curated mix of synthetic and human-preference data after a rigorous benchmark contamination analysis. Foundation-Sec-8B-Instruct outperforms Llama 3.1-8B-Instruct on the CTIBench-RCM benchmark with a 24.03% higher score and also surpasses it on general instruction-following evaluations like IFEval and AlpacaEval 2. The principal implication for AI practitioners is that smaller, domain-adapted models can achieve state-of-the-art performance in specialized fields, providing a publicly available model for building cybersecurity tools and a validated methodology for creating similar expert assistants. |
| Beyond the Trade-off: Self-Supervised Reinforcement Learning for |
|
|
| Reasoning Models’ Instruction Following (Read more on arXiv or HuggingFace) |
Jiaqing Liang, Jie Zeng, Bowei Zhang, Qianyu He, Qingyu Ren |
This paper introduces a self-supervised reinforcement learning framework that improves a reasoning model’s instruction-following ability without external supervision or degrading its core reasoning performance. The primary objective is to resolve the trade-off between a model’s reasoning and instruction-following capabilities by developing a method that enhances the latter without relying on stronger external models for supervision. The key methodology is a self-supervised RL framework featuring an incremental constraint curriculum for dense learning signals, a hybrid reward model combining rule-based verification for hard constraints and a self-supervised binary classifier for soft constraints, and policy optimization using the GRPO algorithm. The framework significantly improves instruction-following, with the 0528-Qwen3-8B model’s IFEval score increasing from 79.7 to 87.1, while crucially maintaining its average performance score of 52.0 on a suite of general reasoning benchmarks. The principal implication for AI practitioners is that this framework provides a scalable and cost-effective method to enhance the instruction-following reliability of specialized or distilled models without requiring access to larger, proprietary models for data generation or reward modeling. |
| InstructVLA: Vision-Language-Action Instruction Tuning from |
|
|
| Understanding to Manipulation (Read more on arXiv or HuggingFace) |
Yang Tian, Bin Wang, Yilun Chen, Hao Li, Shuai Yang |
The paper introduces InstructVLA, a Vision-Language-Action (VLA) model that integrates multimodal reasoning and robotic manipulation through a novel instruction tuning paradigm to mitigate catastrophic forgetting of pre-trained capabilities. The research aims to create a VLA model that preserves the reasoning abilities of large vision-language models (VLMs) while learning precise manipulation skills, effectively bridging high-level understanding with low-level action execution. The methodology involves a two-stage training recipe: first, an action expert is pretrained to decode latent actions from a VLM; second, a “Vision-Language-Action Instruction Tuning” (VLA-IT) stage uses a Mixture-of-Experts (MoE) architecture to co-train the VLM on both standard multimodal data and a curated 650K-sample dataset, enabling it to generate both textual reasoning and action commands. On the novel SimplerEnv-Instruct benchmark for high-level instruction following, InstructVLA outperforms a fine-tuned OpenVLA by 92% and an action expert guided by GPT-4o by 29%. For AI practitioners, this research provides a framework for building generalist robots by decoupling high-level reasoning in a VLM from low-level control in a separate action expert and using instruction tuning to explicitly train for both textual reasoning and action generation, thereby preserving valuable pre-trained knowledge during specialization. |
| VeOmni: Scaling Any Modality Model Training with Model-Centric |
|
|
| Distributed Recipe Zoo (Read more on arXiv or HuggingFace) |
Bin Jia, Zhongkai Zhao, Zhelun Shi, Yaowei Zheng, Qianli Ma |
VeOmni is a model-centric distributed training framework designed to scale omni-modal large language models efficiently. Its primary objective is to address the challenges of heterogeneous model architectures and the entanglement of model definition with parallel logic in existing frameworks, enabling scalable and efficient end-to-end training for omni-modal LLMs. VeOmni introduces model-centric distributed recipes that decouple communication from computation, integrating 3D parallelism (FSDP, SP, EP) and system optimizations like dynamic batching and memory optimization, facilitated by a plug-and-play architectural design. Experimental results demonstrate that a 30B parameter omni-modal Mixture-of-Experts model can be trained with over 2,800 tokens/sec/GPU throughput and scale to 160K context lengths on 128 GPUs using its 3D parallelism. This framework provides a lightweight, non-intrusive interface for customizing and scaling omni-modal LLMs, significantly reducing engineering overhead and accelerating the development of diverse multimodal models. |
| A Glimpse to Compress: Dynamic Visual Token Pruning for Large |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
Zuxuan Wu, Peng-Tao Jiang, Qilong Wang, Yunheng Li, Quan-Sheng Zeng |
This paper presents GlimpsePrune, a dynamic visual token pruning framework that uses a data-driven method to compress high-resolution visual inputs for Large Vision-Language Models (LVLMs). The research objective is to develop a framework that can learn a dynamic, data-driven metric to efficiently prune query-irrelevant visual tokens, overcoming the inflexibility of fixed-ratio compression methods. The key methodology involves inserting a learnable “glimpse token” and using its cross-attention scores from an intermediate decoder layer to train a lightweight Visual Importance Predictor (VIP), which performs a one-shot prune of tokens and their corresponding KV cache entries mid-prefill. The primary result shows that GlimpsePrune prunes an average of 92.6% of visual tokens while fully retaining the baseline model’s performance on free-form VQA tasks, and an enhanced version (GlimpsePrune+) achieves 110% of baseline performance. The principal implication for AI practitioners is that this framework provides a practical solution to reduce the significant memory and computational costs of LVLM inference on high-resolution inputs, enabling more efficient deployment and making computationally-intensive fine-tuning more feasible. |
| Personalized Safety Alignment for Text-to-Image Diffusion Models (Read more on arXiv or HuggingFace) |
Kaidong Yu, Aosong Feng, Qingyu Shi, Jinbin Bai, Yu Lei |
This paper introduces Personalized Safety Alignment (PSA), a framework to condition text-to-image diffusion models on user-specific profiles for granular safety filtering. The primary objective is to move beyond uniform safety standards by enabling generative models to dynamically adapt their outputs to individual user preferences regarding sensitive content. The methodology involves creating a new dataset, Sage, with simulated user profiles and corresponding preferences, and then training a cross-attention adapter to inject user embeddings into the diffusion U-Net, optimizing via a personalized diffusion DPO loss. In experiments on the SDXL model, PSA achieved a Pass Rate of 64.29% for unseen users, outperforming the SafetyDPO baseline’s 60.29% and demonstrating superior alignment with user-specific constraints. For AI practitioners, this framework provides a method to implement dynamic, user-centric safety controls, enabling more nuanced content moderation than static, global blocklists, though its reliance on synthetic user profiles is a stated limitation. |
| Voxlect: A Speech Foundation Model Benchmark for Modeling Dialects and |
|
|
| Regional Languages Around the Globe (Read more on arXiv or HuggingFace) |
Thanathai Lertpetchpun, Xuan Shi, Anfeng Xu, Kevin Huang, Tiantian Feng |
This paper presents Voxlect, a benchmark for evaluating speech foundation models on dialect and regional language classification across 11 language groups using over 2 million utterances from 30 public corpora. The objective is to systematically assess the performance of models like Whisper and MMS in dialect classification and demonstrate the utility of these classifiers in downstream applications. The methodology involves fine-tuning pre-trained speech foundation models on a curated collection of datasets with standardized dialect labels, using an architecture with LoRa adaptation. The primary result shows that multilingual models significantly outperform monolingual ones, with Whisper-Large achieving the highest Macro-F1 score in 5 of 11 language groups, including a score of 0.923 on Arabic dialects. The principal implication for AI practitioners is the availability of pre-trained models and a benchmark to analyze ASR performance across dialects, evaluate the dialectal quality of TTS systems, and augment datasets with dialect information, enabling the development of more equitable speech technologies. |
| RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Lifelong |
|
|
| Learning in Physical Embodied Systems (Read more on arXiv or HuggingFace) |
Junkun Hong, Liangchen Tan, Zezhou Cui, Honghao Cai, Mingcong Lei |
RoboMemory is a brain-inspired, multi-memory agentic framework that enables lifelong learning and long-term planning for robots in physical environments. The primary objective is to develop a framework for embodied agents that addresses challenges of continuous learning, memory latency, task correlation capture, and infinite-loop mitigation to enable robust lifelong learning in dynamic physical systems. The methodology integrates four parallel, brain-inspired modules: an Information Preprocessor, a Lifelong Embodied Memory System using a dynamic Knowledge Graph and a RAG framework, a modified Planner-Critic module for closed-loop planning, and a Low-Level Executer. On the EB-ALFRED benchmark, RoboMemory achieved an average success rate of 67.0%, outperforming the closed-source SOTA model Claude-3.5-Sonnet by 5 percentage points and improving upon its backbone model by 25 percentage points. For AI practitioners, the parallelized, multi-module memory architecture provides a scalable template for building embodied agents that can continuously learn from experience in real-world settings, demonstrating that a structured memory system significantly enhances performance over single large model approaches. |
| Exploitation Is All You Need… for Exploration (Read more on arXiv or HuggingFace) |
Jesse Roberts, Micah Rentschler |
This paper demonstrates that an agent trained with a purely greedy objective can learn to explore effectively without explicit incentives, provided sufficient environmental structure and agent memory. The research objective is to empirically test the hypothesis that exploration can emerge organically from a reward-maximization objective by identifying and validating its necessary preconditions. The authors use a transformer-based DQN agent in a meta-RL setting and conduct controlled ablation studies on multi-armed bandit and gridworld environments by systematically varying environmental recurrence, agent memory, and temporal credit assignment. The primary result shows that while exploration emerges with sufficient structure and memory, long-term credit assignment is beneficial but not always essential; in complex gridworld tasks, increasing the episode discount factor from 0 to 0.9 improved normalized reward from 0.408 to 0.670. The principal implication for AI practitioners is that focusing on memory-rich architectures and training paradigms that leverage recurring task structures can be a more direct path to achieving effective exploration than engineering explicit exploration bonuses. |
| Cyber-Zero: Training Cybersecurity Agents without Runtime (Read more on arXiv or HuggingFace) |
Zijian Wang, Varun Kumar, Hantian Ding, Dingmin Wang, Terry Yue Zhuo |
i) The paper introduces CYBER-ZERO, the first runtime-free framework that synthesizes agent trajectories from public Capture The Flag (CTF) writeups to train LLM-based cybersecurity agents. ii) The main research objective is to overcome the scarcity of high-quality training data in cybersecurity by developing a method to generate realistic, long-horizon interaction sequences without needing access to executable runtime environments. iii) The key methodology involves a dual-LLM, persona-driven simulation where a “CTF Player” LLM attempts to solve a challenge, guided by a second “Bash Terminal” LLM that uses public writeups as a weak oracle to reverse-engineer and simulate plausible system responses. iv) The primary result is that agents trained on these synthesized trajectories achieve up to a 13.1% absolute performance gain over baseline models across three prominent CTF benchmarks, with the best model (CYBER-ZERO-32B) matching the performance of proprietary systems like Claude-3.5-Sonnet. v) The principal implication for AI practitioners is that this runtime-free synthesis method can effectively democratize the development of state-of-the-art cybersecurity agents, enabling the training of capable and cost-effective open-weight models without needing access to often unavailable live challenge environments. |
| AgentTTS: Large Language Model Agent for Test-time Compute-optimal |
|
|
| Scaling Strategy in Complex Tasks (Read more on arXiv or HuggingFace) |
Zhiwei Zhang, Jingying Zeng, Zhenwei Dai, Hui Liu, Fali Wang |
The paper introduces AgentTTS, an LLM-agent framework that autonomously finds compute-optimal model and budget allocations for multi-stage complex tasks by leveraging three empirical insights about test-time scaling. The main objective is to determine how to optimally select models and allocate a total compute budget across interdependent subtasks in a multi-stage complex task to maximize overall performance, given a combinatorial search space and high inference costs. The key methodology is AgentTTS, a framework where an LLM agent iteratively generates and refines budget allocation configurations. The agent’s search is guided by prompts incorporating three empirical insights derived from pilot experiments: (1) subtasks have distinct model preferences, (2) performance gains diminish beyond an optimal budget, and (3) allocations are interdependent across subtasks. The primary result is that AgentTTS significantly outperforms baselines in search efficiency and final performance. On the 2WikiMultiHopQA dataset, AgentTTS achieved a test-set Exact Match (EM) score of 0.72, exceeding the next best methods by 2%, while requiring only 2.5 hours of search time compared to over 8 hours for competing agent-based approaches. The principal implication for AI practitioners is that they can use the AgentTTS framework to automate the complex process of optimizing inference compute for multi-stage AI systems, achieving better performance-cost trade-offs by strategically allocating resources based on subtask-specific needs rather than uniformly applying a single large model. |
| ReMoMask: Retrieval-Augmented Masked Motion Generation (Read more on arXiv or HuggingFace) |
Hao Tang, Zeyu Zhang, Siheng Wang, Zhengdao Li |
ReMoMask is a retrieval-augmented masked modeling framework that synthesizes human motion from text by integrating a novel bidirectional retriever and a spatiotemporal attention mechanism. The objective is to address the dual challenges of limited diversity in generative models and asynchronous artifacts in retrieval-augmented generation (RAG) methods for more realistic text-to-motion synthesis. The key methodology involves a Bidirectional Momentum Text-Motion Model (BMM) which uses momentum queues to improve cross-modal retrieval, and a Semantic Spatiotemporal Attention (SSTA) mechanism that fuses textual, retrieved, and 2D motion-structural information during generation. The model achieves state-of-the-art performance, demonstrating a 10.97% improvement in FID score on the KIT-ML dataset compared to the previous leading RAG-T2M method. For AI practitioners, the principal implication is that for complex generative tasks, augmenting models with retrieval is most effective when combined with specialized fusion mechanisms like SSTA that explicitly align external conditioning with the intrinsic spatiotemporal structure of the data being generated. |
| Artificial Intelligence and Misinformation in Art: Can Vision Language |
|
|
| Models Judge the Hand or the Machine Behind the Canvas? (Read more on arXiv or HuggingFace) |
Elena Merino-Gómez, Pedro Reviriego, Gonzalo Martínez, Javier Conde, Tarian Fu |
This paper evaluates Vision Language Models’ (VLMs) ability to attribute real and AI-generated paintings to artists, highlighting their limitations in both tasks. The primary objective was to assess whether state-of-the-art VLMs can reliably identify the original artist of real paintings and detect AI-generated artistic imitations, preventing misinformation. Using a dataset of nearly 40,000 WikiArt paintings and AI-generated imitations from Stable Diffusion, Flux, and F-Lite, six open-weight and one proprietary VLM were evaluated on binary classification prompts for artist attribution and AI detection, measuring normalized accuracy (C1, C2) and their arithmetic mean (AM). Results show VLMs have significant limitations; for real paintings, even top performers like LLaMa3.2-11B achieved only ~63% average normalized accuracy (AM), and for Stable Diffusion imitations, GPT4.1-mini was best, correctly identifying over 95% as not from the suggested painter. These findings imply that current VLMs are unreliable for artist attribution and AI-generated content detection in art, necessitating improved model capabilities and careful deployment as decision-support tools rather than authoritative sources to mitigate widespread misinformation risks. |
| Embedding-Aware Quantum-Classical SVMs for Scalable Quantum Machine |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Cristian Bosch, Carlos Andrés Durán, Mario Bifulco, Luis Fernando Torres Torres, Sebastián Andrés Cajas Ordóñez |
This paper proposes a hybrid quantum-classical SVM framework demonstrating that quantum advantage is critically dependent on the choice of classical feature embeddings. The research objective is to address the scalability limitations of Quantum Support Vector Machines (QSVMs) by systematically investigating how different pretrained embeddings influence quantum kernel performance. The methodology combines class-balanced k-means data distillation with feature extraction from pretrained Vision Transformer (ViT) and CNN models, followed by classification using a 16-qubit QSVM simulated via a tensor network backend. The primary result shows that using ViT embeddings enables the QSVM to achieve an accuracy improvement of up to 8.02% on Fashion-MNIST over a classical SVM using identical embeddings, while CNN features and raw pixels lead to performance degradation. The principal implication for AI practitioners is that realizing quantum advantage requires a deliberate co-design of classical representation and quantum algorithms, as the choice of transformer-based embeddings is shown to be a prerequisite for outperforming classical methods in this QSVM setting. |
Papers for 2025-08-04
| Title |
Authors |
Summary |
| Beyond Fixed: Variable-Length Denoising for Diffusion Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Jiaqi Wang, Yuhang Cao, Yuhang Zang, Xiaoyi Dong, Jinsong Li |
This paper introduces DAEDAL, a training-free, two-stage strategy that enables dynamic variable-length generation for Diffusion Large Language Models (DLLMs). The objective is to overcome the critical limitation of DLLMs requiring a statically predefined generation length, which creates a trade-off between task performance and computational efficiency. DAEDAL’s methodology first performs an “Initial Length Adjustment” by iteratively expanding the sequence based on the model’s End-of-Sequence (EOS) token confidence, followed by an “Iterative Mask Insertion” phase that dynamically adds tokens to low-confidence regions during denoising. On the GSM8K benchmark, DAEDAL with the LLaDA-Instruct-8B model achieved 85.8% accuracy, outperforming the best-performing fixed-length baseline’s 83.8% accuracy while using significantly fewer tokens on average (363 vs. 1024). For AI practitioners, this means DLLMs can be deployed without manual, task-specific length tuning, leading to improved computational efficiency and performance, thus making them a more viable alternative to autoregressive models. |
| PixNerd: Pixel Neural Field Diffusion (Read more on arXiv or HuggingFace) |
Limin Wang, Weilin Huang, Chenhui Zhu, Ziteng Gao, Shuai Wang |
The paper introduces PixNerd, a single-stage, end-to-end pixel-space diffusion transformer that uses neural fields to model patch details, eliminating the reliance on pre-trained VAEs. The primary objective is to develop an efficient, single-stage pixel-space diffusion model that avoids the accumulated errors, decoding artifacts, and complex pipelines of two-stage latent diffusion models. The key methodology replaces the final linear projection layer of a diffusion transformer with a mechanism that predicts weights for a per-patch MLP (a neural field), which then decodes pixel-wise diffusion velocities from local coordinates and noisy pixel values. PixNerd achieves strong results, including a 2.15 FID on ImageNet 256×256, which is competitive with latent diffusion models but without a VAE or complex cascade pipeline. For AI practitioners, PixNerd offers a simplified end-to-end framework for training high-resolution diffusion transformers directly in pixel space, bypassing the separate training and potential artifacts of VAEs. |
| SWE-Exp: Experience-Driven Software Issue Resolution (Read more on arXiv or HuggingFace) |
Heng Lian, Yuling Shi, Xiaodong Gu, Shaoxin Lin, Silin Chen |
SWE-Exp is an experience-enhanced framework for software issue resolution, enabling continuous learning and strategic repair. The primary objective of SWE-Exp is to address the memoryless exploration limitation of current LLM agents by enabling them to learn from and reuse past repair experiences. SWE-Exp introduces a multi-faceted experience bank, capturing successful and failed repair attempts at different levels, and employs a dual-agent architecture (Instructor and Assistant) integrated with an augmented MCTS framework for experience-driven guidance. Experiments show SWE-Exp achieves a state-of-the-art resolution rate of 41.6% Pass@1 on SWE-bench-Verified using DeepSeek-V3-0324, a 7.2% relative improvement over previous state-of-the-art methods using the same model. This approach transforms automated software engineering agents from trial-and-error explorers into strategic, experience-driven problem solvers, enabling systematic accumulation and leverage of repair expertise. |
| Multimodal Referring Segmentation: A Survey (Read more on arXiv or HuggingFace) |
Zuxuan Wu, Chang Liu, Shuting He, Song Tang, Henghui Ding |
This survey provides a comprehensive overview of multimodal referring segmentation, unifying task definitions, methodologies, and benchmarks across image, video, and 3D scenes. The paper’s objective is to systematize the field by proposing a unified problem formulation and a general meta-architecture to categorize the diverse approaches for segmenting objects based on linguistic or audio expressions. The key methodology involves summarizing a unified meta-architecture consisting of modules for feature extraction, multimodal interaction, temporal processing, a segmentation head, and training objectives, while reviewing methods within one-stage and two-stage paradigms. The survey reports significant performance gains from foundation models; for instance, on the RefCOCOg benchmark for Referring Expression Segmentation, recent models like OneRef-L achieve up to 76.82% mIoU, substantially outperforming early methods that scored 34.06% mIoU. The principal implication for AI practitioners is that leveraging the presented meta-architecture and integrating large foundation models (e.g., SAM, MLLMs) within generalized frameworks like GRES (for multi-target scenarios) is crucial for developing robust, real-world systems capable of fine-grained perception from complex user instructions. |
| 3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding (Read more on arXiv or HuggingFace) |
Hao Tang, Zeyu Zhang, Ting Huang |
The 3D-R1 paper introduces a generalist vision-language model that enhances 3D scene understanding by using a synthetically generated Chain-of-Thought dataset for initialization, followed by a reinforcement learning framework to refine reasoning. The primary objective is to improve the robust reasoning and generalization capabilities of 3D vision-language models, which currently struggle due to limitations in high-quality spatial data and static viewpoint assumptions. The methodology consists of a two-stage process: first, supervised fine-tuning (SFT) on a newly created 30,000-sample Chain-of-Thought dataset (Scene-30K) to provide a “cold-start”. This is followed by reinforcement learning using Group Relative Policy Optimization (GRPO) with three distinct reward functions (perception, semantic similarity, and format) to enhance reasoning precision. The 3D-R1 model achieves an average performance improvement of 10% across various 3D scene benchmarks. For instance, on the ScanQA 3D question answering validation set, the model’s CIDEr score improved from a 97.95 baseline to 106.45 after applying the full reinforcement learning framework. For AI practitioners, this work provides a blueprint for enhancing specialized domain reasoning in foundation models: use a large language model to generate a structured, high-quality Chain-of-Thought dataset for initial supervised fine-tuning, and then apply targeted reinforcement learning with task-specific rewards to optimize policy for complex, multi-step inference. |
| SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution (Read more on arXiv or HuggingFace) |
Heng Lian, Xiaodong Gu, Shaoxin Lin, Yuling Shi, Han Li |
SWE-Debate introduces a competitive multi-agent framework that improves automated software issue resolution by generating and debating multiple fault propagation traces from a code dependency graph. The paper’s primary objective is to overcome the “limited observation scope” of single-agent systems, which struggle to resolve issues spanning complex codebases. The methodology involves three stages: 1) proposing multiple fault propagation traces by traversing a static code dependency graph, 2) conducting a three-round competitive debate among agents to select the best trace and synthesize a consolidated fix plan, and 3) using this plan to initialize a Monte Carlo Tree Search (MCTS) agent for patch generation. SWE-Debate achieves an 81.67% file-level fault localization accuracy on SWE-Bench-lite, a 3.93 percentage point improvement over the strongest baseline. For AI practitioners, the principal implication is that architecting multi-agent systems for competitive debate, rather than simple collaboration, can significantly improve performance on complex disambiguation tasks like fault localization by forcing a rigorous evaluation of diverse hypotheses. |
| Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple Judges (Read more on arXiv or HuggingFace) |
Chengfei Lv, Zhiwen Chen, Yunfeng Wang, Kehua Feng, Yuqi Tang |
This paper introduces an efficient multi-turn dialogue evaluator, MTDEval, that aggregates preference knowledge from multiple LLM judges. The main objective is to address the computational overhead and persistent biases associated with the LLM-as-a-judge paradigm for multi-turn dialogue evaluation. The methodology involves training a lightweight evaluator, composed of a Llama-3-8B text-embedding model and MLP scoring heads, on a large-scale pairwise preference dataset (P2-MTD) annotated by five state-of-the-art LLM judges, using maximum likelihood estimation with judge reliability prediction. Experimentally, MTDEval achieves superior inference efficiency with an average runtime of 0.10 seconds for single rating and 0.19 seconds for pairwise comparison on the Daily-MTD dataset, outperforming baseline models. This enables AI practitioners to perform fast, scalable, and robust multi-turn dialogue quality assessment, significantly reducing computational costs for large-scale and real-time evaluation scenarios. |
| Investigating Hallucination in Conversations for Low Resource Languages (Read more on arXiv or HuggingFace) |
Fatemeh Jamshidi, Zheng Zhang, Souvika Sarkar, Md. Najib Hasan, Amit Das |
This research paper quantitatively evaluates hallucination in six large language models across conversational datasets for the low-resource languages of Hindi, Farsi, and Mandarin. The main objective is to analyze the factual accuracy and linguistic errors of GPT-3.5, GPT-4o, Llama-3.1, Gemma-2.0, DeepSeek-R1, and Qwen-3 in these specific linguistic contexts. The key methodology involved prompting the models with a conversational turn from translated datasets and measuring the generated output against a ground-truth response using ROUGE-1 and ROUGE-L scores, though the paper presents contradictory interpretations of what these scores signify regarding hallucination. The primary result and most impactful finding is the significant disparity in performance across languages; for instance, on the BlendedSkillTalk dataset, Qwen-3 achieved a ROUGE-L score of 3.83 in Farsi, while GPT-4o scored only 0.06 in Mandarin, highlighting that model behavior is highly dependent on the language. The principal implication for AI practitioners is that hallucination rates are strongly influenced by language resource availability, necessitating the use of mitigation techniques like Retrieval-Augmented Generation (RAG) or targeted fine-tuning when deploying LLMs for low-resource languages. |
| IGL-Nav: Incremental 3D Gaussian Localization for Image-goal Navigation (Read more on arXiv or HuggingFace) |
Jianjiang Feng, Ziwei Wang, Hang Yin, Xiuwei Xu, Wenxuan Guo |
IGL-Nav introduces an incremental 3D Gaussian Localization framework for efficient and 3D-aware image-goal navigation, aiming to enable robust visual navigation to a specified free-view image goal. The system leverages 3D Gaussian Splatting (3DGS) for incremental scene representation via feed-forward prediction and employs a coarse-to-fine localization strategy. This strategy includes 3D convolution on voxelized scene and target embeddings for coarse pose estimation, and differentiable 3DGS rendering with matching-constrained optimization for fine refinement. IGL-Nav achieves state-of-the-art performance, demonstrating an “Overall Narrow FOV” success rate (SR) of 57.0% and Success weighted by Path Length (SPL) of 48.2% in free-view image-goal navigation with supervised training, significantly outperforming prior methods. This work establishes a generalizable and practically viable approach for real-time image-goal navigation in robotics, facilitating strong sim-to-real transfer and diverse camera pose handling. |
| SpA2V: Harnessing Spatial Auditory Cues for Audio-driven Spatially-aware |
|
|
| Video Generation (Read more on arXiv or HuggingFace) |
Long Chen, Qifeng Chen, Yazhou Xing, Yingqing He, Kien T. Pham |
SpA2V is a novel two-stage framework for generating spatially-aware videos from audio by first creating a video scene layout and then synthesizing the video. The main objective is to generate videos that are both semantically and spatially aligned with input audio by explicitly decoding and utilizing auditory cues like direction, distance, and movement. The methodology first uses a Multimodal Large Language Model (MLLM) with in-context learning to interpret audio and produce a Video Scene Layout (VSL) specifying object locations and captions; then, it employs a training-free combination of pre-trained diffusion models to generate the final video guided by this VSL. On the new AVLBench benchmark, SpA2V’s layout generation stage achieved a MaxIoU score of 22.24 in translational scenarios, substantially outperforming the baseline score of 1.77. For AI practitioners, this research provides a practical, training-free pipeline for adding precise spatial control to audio-driven video generation by using MLLMs as intermediate planners to guide pre-trained generative models. |
Papers for 2025-08-01
| Title |
Authors |
Summary |
| Seed-Prover: Deep and Broad Reasoning for Automated Theorem Proving (Read more on arXiv or HuggingFace) |
Zhicheng Jiang, Wenhao Huang, Liankai Huang, Jinming Gu, Luoxin Chen |
The paper introduces Seed-Prover and Seed-Geometry, two systems that integrate large language models with the Lean formal proof assistant to advance automated theorem proving. The objective is to solve highly complex mathematical problems by developing a system capable of both broad, exploratory conjecture generation and deep, iterative proof refinement. The methodology combines a “lemma-style” whole-proof generation model with a three-tiered inference strategy (light, medium, heavy) that leverages iterative refinement based on Lean compiler feedback, proved lemmas, and self-summarization. The system achieved state-of-the-art results, including proving 78.1% of 155 formalized past IMO problems and, post-competition, solving 5 out of 6 problems at the IMO 2025. The principal implication for AI practitioners is that integrating LLMs with formal verification environments and employing multi-stage, iterative refinement strategies enables the solution of complex, structured reasoning tasks with verifiable correctness, surpassing the capabilities of single-pass or natural language-based approaches. |
| Phi-Ground Tech Report: Advancing Perception in GUI Grounding (Read more on arXiv or HuggingFace) |
Kai Qiu, Qi Dai, Jialiang Zhu, Ziqiang Xu, Miaosen Zhang |
This research details the Phi-Ground model family, which advances GUI grounding by systematically optimizing data processing, training strategies, and model architecture to achieve state-of-the-art performance. The objective is to improve the perception capabilities of GUI grounding models for Computer Use Agents (CUAs) by investigating factors from data collection to training protocols, addressing the low accuracy of existing methods. The methodology involves fine-tuning MLLMs on a 40M+ sample dataset, emphasizing text-first modality input, random resize data augmentation, and uniform spatial data distribution, and leveraging a two-stage approach where a planner model generates detailed instructions for the specialized grounding model. In an agent setting, the Phi-Ground-7B-16C-DPO model achieves 55.0% click accuracy on the challenging ScreenSpot-pro benchmark, and post-training with Direct Preference Optimization (DPO) further enhances performance. For AI practitioners, the key implication is that for multimodal perception tasks, performance depends critically on data strategy (distribution and augmentation) and computational budget (balancing model size and image tokens), and that decoupling high-level reasoning (planning) from low-level perception (grounding) is a highly effective design pattern. |
| C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring |
|
|
| Challenges in Complex Conversations (Read more on arXiv or HuggingFace) |
Yiwen Guo, Wei Tao, Chengqian Ma |
This paper introduces C³, a bilingual benchmark for evaluating Spoken Dialogue Models (SDMs) on complex conversational challenges. The primary objective is to assess the capabilities of current SDMs in handling five key phenomena: phonological ambiguity, semantic ambiguity, omission, coreference, and multi-turn interaction in both English and Chinese. The methodology involves a new dataset of 1,079 instances designed to test these phenomena, evaluated using an LLM-based method (GPT-4o and DeepSeek-R1 as judges) that shows high correlation (Pearson > 0.87) with human judgments. The study’s primary result is that SDMs struggle significantly with ambiguity, achieving an overall accuracy of just 3.97% on semantic ambiguity tasks in Chinese, and generally perform better in English (overall accuracy 35.15%) than in Chinese (23.33%). The principal implication for AI practitioners is that the selection of an SDM must be carefully tailored to the specific language and conversational complexity of the application, as model performance varies drastically; for example, GPT-4o-Audio-Preview excels in English (55.68% accuracy) while Qwen2.5-Omni is superior for Chinese (40.08% accuracy). |
| RecGPT Technical Report (Read more on arXiv or HuggingFace) |
Jian Wu, Jiakai Tang, Gaoyang Guo, Dian Chen, Chao Yi |
This paper presents RecGPT, a large language model-based framework that redesigns the recommender system pipeline to be intent-centric, moving beyond traditional log-fitting. The primary objective is to overcome the limitations of log-fitting approaches, such as filter bubbles and the Matthew effect, by explicitly modeling user intent through LLMs for interest mining, item retrieval, and explanation generation. The key methodology involves a multi-stage workflow using three specialized LLMs for user interest mining, item tag prediction, and explanation generation, integrated into a tag-aware tri-tower (User, Item, Tag) retrieval architecture and trained via a progressive paradigm guided by a Human-LLM cooperative judge system. Online A/B experiments on the Taobao App demonstrated that RecGPT achieved a +6.33% increase in Click-Through Rate (CTR), a +9.47% increase in Item Page Views (IPV), and a +6.96% increase in Clicked Item Category Diversity (CICD) over the baseline. The principal implication for AI practitioners is that shifting from purely log-fitting models to an explicit, LLM-driven, intent-centric paradigm can create a more sustainable recommendation ecosystem, simultaneously boosting user engagement, commercial metrics, and content diversity for long-tail merchants. |
| villa-X: Enhancing Latent Action Modeling in Vision-Language-Action |
|
|
| Models (Read more on arXiv or HuggingFace) |
Kaixin Wang, Chuheng Zhang, Pushi Zhang, Hangxing Wei, Xiaoyu Chen |
The paper introduces villa-X, a Visual-Language-Latent-Action (ViLLA) framework that improves latent action learning and its integration into VLA models by jointly modeling latent and robot actions. The primary objective is to improve how latent actions are learned from visual data and how they are incorporated into Vision-Language-Action (VLA) pre-training to create more generalizable robot manipulation policies. The methodology features a Latent Action Model (LAM) augmented with a proprioceptive Forward Dynamics Model (proprio FDM) to ground latent actions in robot dynamics, and an Actor (ACT) module that jointly models latent and robot action sequences using a joint diffusion process, with robot action generation explicitly conditioned on the latent action plan. The framework achieves superior performance across simulated and real-world tasks, notably attaining a 90.1% average success rate on the four LIBERO benchmark suites, outperforming prior methods like OpenVLA (76.5%). For practitioners, this work demonstrates that explicitly grounding latent actions in robot proprioceptive data and using a structured, hierarchical diffusion model provides a more effective method for leveraging large-scale, action-free video data to pre-train robust and generalizable robot policies. |
| Scalable Multi-Task Reinforcement Learning for Generalizable Spatial |
|
|
| Intelligence in Visuomotor Agents (Read more on arXiv or HuggingFace) |
Anji Liu, Bowei Zhang, Haiwen Xia, Zhancun Mu, Shaofei Cai |
This paper presents a scalable framework for post-training visuomotor agents with multi-task reinforcement learning, significantly enhancing their generalizable spatial reasoning and zero-shot transfer capabilities. The primary objective is to determine if RL post-training can enable a pre-trained visuomotor policy to generalize spatial intelligence to novel tasks and unseen 3D environments, overcoming typical overfitting issues. The methodology uses cross-view goal specification as a unified task representation, automatically synthesizes over 100,000 tasks in Minecraft, and fine-tunes an imitation-learned policy with a distributed Proximal Policy Optimization (PPO) algorithm constrained by KL-divergence. The primary results show that RL post-training boosts average interaction success rates by 4x (from 7% to 28%) and enables the agent to achieve a 48% success rate on challenging invisible-target tasks where other baselines failed. For AI practitioners, this work provides a paradigm of imitation learning pre-training followed by large-scale RL fine-tuning, demonstrating that complex spatial skills learned in a single customizable simulator can generalize effectively to different virtual and real-world environments without requiring domain-specific adaptation. |
| Persona Vectors: Monitoring and Controlling Character Traits in Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Jack Lindsey, Owain Evans, Henry Sleight, Andy Arditi, Runjin Chen |
This research introduces “persona vectors,” linear directions in a large language model’s activation space that correspond to specific character traits, and demonstrates their use for monitoring and controlling model personality. The primary objective is to develop an automated method to identify these vectors and use them to predict, monitor, and mitigate undesirable persona shifts induced by prompting or finetuning. The methodology involves automatically generating contrastive prompt pairs from a natural language trait description, then computing the persona vector as the difference-in-means of the resulting response activations. The results show that finetuning-induced shifts along these vectors strongly predict behavioral changes (e.g., Pearson’s r up to 0.97 between activation shift and trait expression) and that preventative steering during training can mitigate these shifts while preserving capabilities. For AI practitioners, this provides a scalable method for pre-finetuning data screening; by calculating the “projection difference” on training samples, developers can identify and filter data likely to cause unwanted emergent behaviors like sycophancy or maliciousness. |
| On the Expressiveness of Softmax Attention: A Recurrent Neural Network |
|
|
| Perspective (Read more on arXiv or HuggingFace) |
Eric C. Larson, Gabriel Mongaras |
This paper derives a recurrent neural network (RNN) formulation for softmax attention, clarifying its expressiveness compared to linear attention. The main objective is to understand why softmax attention is more expressive than linear attention, which typically lags in downstream accuracy despite being derived from softmax. The authors achieve this by deriving the recurrent form of softmax attention via its Taylor series expansion, analyzing the numerator, and reinterpreting the denominator as a gate or norm, alongside conducting ablation studies. Key findings show that linear attention is a first-order approximation of softmax attention, and adding higher-order Taylor series terms up to n=10 can make the recurrent approximation mirror softmax with negligible differences, while a simple vector norm for the denominator can suffice. This work provides a theoretical basis for understanding the performance bounds of softmax attention and suggests avenues for developing more performant or efficient attention mechanisms by leveraging higher-order interactions or alternative normalization schemes. |
| TARS: MinMax Token-Adaptive Preference Strategy for Hallucination |
|
|
| Reduction in MLLMs (Read more on arXiv or HuggingFace) |
Jiasheng Tang, Chang Liu, Zhiming Luo, Keda Tao, Kejia Zhang |
TARS is a min-max token-adaptive preference strategy that reformulates Direct Preference Optimization (DPO) to reduce hallucinations in Multimodal Large Language Models (MLLMs). The primary objective is to address DPO’s overfitting to superficial linguistic cues, which leads to distributional rigidity and ungrounded outputs in MLLMs. TARS reformulates DPO as a min-max optimization problem, maximizing adaptability via controlled perturbations of visual-agnostic tokens while minimizing preference loss, and incorporating spectral preference alignment for semantic consistency. Using 4.8k preference samples, TARS reduces hallucination rates on the AMBER benchmark from 26.4% to 13.2% for LLaVA-v1.5-7B models, outperforming DPO baselines and matching GPT-4o. TARS offers a data-efficient method for MLLM developers to enhance factual grounding and trustworthiness by mitigating hallucinations without extensive datasets or expert feedback. |
| Beyond Linear Bottlenecks: Spline-Based Knowledge Distillation for |
|
|
| Culturally Diverse Art Style Classification (Read more on arXiv or HuggingFace) |
Abdelmalik Taleb-Ahmed, Cosimo Distante, Salah Eddine Bekhouche, Abdellah Zakaria Sellam |
This paper proposes enhancing a dual-teacher knowledge distillation framework for art style classification by replacing linear MLP projection heads with spline-based Kolmogorov-Arnold Networks (KANs). The objective is to improve self-supervised art style classification by better modeling the complex, nonlinear interactions of stylistic features that linear projections fail to capture. The key methodology involves integrating KANs into all three network branches (student, momentum teacher, style teacher) and training the student network using a composite loss function that includes relation alignment, Gram matrix-based style alignment, and KAN-specific regularization. On the Pandora18k dataset, using a ConvNeXt-Base backbone, the KAN-based approach achieved a 66.26% Top-1 accuracy, representing a 1.03% improvement over the identical architecture using standard MLP heads. For AI practitioners, this research demonstrates that substituting MLP heads with KANs in self-supervised contrastive learning frameworks can yield superior feature representations for tasks involving complex data manifolds, such as art style, without altering the core training paradigm. |
| Enhanced Arabic Text Retrieval with Attentive Relevance Scoring (Read more on arXiv or HuggingFace) |
Abdenour Hadid, Fadi Dornaika, Yazid Bounab, Azeddine Benlamoudi, Salah Eddine Bekhouche |
This paper presents Adaptive Passage Retrieval (APR), an enhanced dense retrieval framework for Arabic that integrates a lightweight transformer with a novel Attentive Relevance Scoring (ARS) module to improve ranking accuracy. The primary objective is to develop a dense retrieval model specifically for the Arabic language that surpasses standard systems by using a more sophisticated, learned relevance function instead of simple vector similarity to better handle Arabic’s linguistic complexities. The system employs a dual-encoder architecture with a lightweight Arabic-specific transformer (MiniBERT) and introduces an Attentive Relevance Scoring (ARS) module that computes a relevance score via a learned, non-linear interaction, trained with a composite loss function. The paper does not provide an ablation study to isolate the performance gains from the ARS module versus the MiniBERT encoder. On the ArabicaQA test set, the APR model achieved a Top-10 retrieval accuracy of 63.17%, an absolute improvement of +4.77% over the state-of-the-art AraDPR baseline. For AI practitioners, this work demonstrates that augmenting a standard dual-encoder architecture with a lightweight, trainable relevance scoring module can yield significant performance gains over relying solely on dot-product similarity, providing a more robust method for semantic matching in morphologically complex languages. |
| NeRF Is a Valuable Assistant for 3D Gaussian Splatting (Read more on arXiv or HuggingFace) |
ZeSheng Wang, Yufeng Wang, Takeo Igarashi, I-Chao Shen, Shuangkang Fang |
The paper introduces NeRF-GS, a framework that jointly optimizes Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) to enhance 3D scene representation performance. The objective is to develop a hybrid framework that systematically integrates a full NeRF pipeline into the training of a 3DGS model to mitigate the inherent limitations of 3DGS, such as initialization sensitivity and weak inter-Gaussian correlations. The NeRF-GS methodology involves three core components: a shared hash-based feature encoding network for both NeRF and GS branches, the optimization of residual vectors for features and positions to model discrepancies between the two representations, and a joint optimization process using “GS-Rays” and shared attribute losses to align the branches and enable NeRF-assisted adaptive Gaussian growth. The proposed NeRF-GS framework demonstrates state-of-the-art performance, surpassing existing methods on benchmark datasets; for instance, on one scene, NeRF-GS achieves a PSNR of 30.5, outperforming the vanilla 3DGS result of 28.7 by 1.8dB. The primary implication for AI practitioners is that NeRF can be used as an effective auxiliary component during the training phase to significantly enhance the rendering quality and robustness of 3D Gaussian Splatting models, particularly for sparse-view scenes, while the final optimized GS branch can be deployed independently to retain its real-time rendering performance. |
| AgroBench: Vision-Language Model Benchmark in Agriculture (Read more on arXiv or HuggingFace) |
Yoshitaka Ushiku, Masaki Onishi, Hirokatsu Kataoka, Nakamasa Inoue, Risa Shinoda |
This paper introduces AgroBench, a comprehensive, expert-annotated vision-language benchmark for evaluating VLM capabilities in the agricultural domain. The primary objective is to develop a robust benchmark to assess the practical knowledge and applicability of Vision-Language Models (VLMs) across diverse, real-world agricultural scenarios, overcoming the limitations of existing synthetically-generated datasets. The methodology involved creating a question-answering dataset of 4,342 QA pairs manually annotated by agronomist experts, covering seven tasks including 682 disease and 108 weed categories. The primary results show that VLMs struggle with fine-grained identification; specifically, in weed identification, most open-source models performed near random chance, and the highest accuracy achieved by Gemini 1.5-Pro was 55.17%, with error analysis showing 51.92% of failures are due to a “Lack of Knowledge”. For AI practitioners, this implies that deploying VLMs in agriculture requires intensive domain-specific fine-tuning with expert-verified data to address the significant knowledge gaps of current models. |
| Flow Equivariant Recurrent Neural Networks (Read more on arXiv or HuggingFace) |
T. Anderson Keller |
This paper introduces Flow Equivariant Recurrent Neural Networks (FERNNs), a novel architecture that enforces equivariance to continuous, time-parameterized transformations (flows) to improve sequence model generalization. The research objective is to formalize ‘flow equivariance’ and develop an RNN architecture that is provably equivariant to these dynamic transformations, a property standard group-equivariant RNNs (G-RNNs) lack. The key methodology achieves this by lifting the RNN’s hidden state to a product space of flow generators and group elements, then applying a flow-specific transformation at each recurrent step to perform computation in the signal’s moving reference frame. FERNNs are shown to significantly outperform G-RNNs, reducing Mean Squared Error by an order of magnitude (from 8.1e-3 to 1.5e-4) on a Translating MNIST prediction task and exhibiting zero-shot generalization to unseen flow velocities. The principal implication for AI practitioners is a parameter-efficient framework to build models for dynamic data (e.g., video, robotics) with superior generalization to new motions and longer sequences. |
| Efficient Machine Unlearning via Influence Approximation (Read more on arXiv or HuggingFace) |
Enhong Chen, Defu Lian, Chenwang Wu, Jiawei Liu |
This paper presents Influence Approximation Unlearning (IAU), an efficient algorithm for machine unlearning that reframes data removal as an incremental learning task. The research objective is to develop a computationally efficient unlearning method that avoids the prohibitive costs of full model retraining or the Hessian matrix calculations required by traditional influence-based approaches. The key methodology establishes a theoretical link between unlearning and incremental learning, enabling the approximation of a data point’s removal by applying a corrective gradient update derived from both the forgotten and remaining data, further enhanced by a novel gradient restriction loss during the initial model training. The primary results show that on a ResNet18 model with the CIFAR10 dataset, IAU achieves the best overall performance with an average rank of 0.3 across utility, time, and efficacy metrics, significantly outperforming the next best baseline (USGD with a rank of 1.7). The principal implication for AI practitioners is a scalable and practical method for executing data deletion requests, making privacy-compliant machine learning more feasible for large models and high-frequency unlearning scenarios without the significant overhead of retraining or Hessian inversion. |
Papers for 2025-07-31
| Title |
Authors |
Summary |
| ScreenCoder: Advancing Visual-to-Code Generation for Front-End |
|
|
| Automation via Modular Multimodal Agents (Read more on arXiv or HuggingFace) |
Qunzhong Wang, Yuxuan Wan, Yaozhi Zheng, Yilei Jiang, csuhan |
ScreenCoder introduces a modular multi-agent framework to convert UI design images into front-end code by breaking the process into grounding, planning, and generation. The research aims to overcome the limitations of end-to-end models by creating a robust, interpretable system that handles visual understanding, layout planning, and code synthesis as distinct sub-problems. The methodology employs a three-agent pipeline where a grounding agent detects UI components, a planning agent constructs a hierarchical layout tree, and a generation agent synthesizes HTML/CSS code; this system also functions as a data engine to fine-tune a VLM using supervised and reinforcement learning. The agentic ScreenCoder achieves state-of-the-art results, outperforming models like GPT-4o with a Block Match score of 0.755 versus 0.730. The principal implication for AI practitioners is the framework’s dual function as a high-performance inference pipeline and a scalable data engine for generating synthetic image-code pairs, enabling the targeted improvement of VLMs for complex, domain-specific code generation tasks. |
| Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency |
|
|
| and Performance (Read more on arXiv or HuggingFace) |
Maksim Velikanov, Iheb-Chaabane, ifarhat1993, ybelkada, JingweiZuo |
The Falcon-H1 series introduces a new family of open-source, hybrid-head language models that combine parallel attention and Mamba-2 SSMs for superior performance and efficiency. The primary objective was to design and evaluate a novel hybrid architecture that achieves state-of-the-art performance while being significantly more parameter- and training-efficient than existing large language models. The models utilize a parallel architecture with independently tunable attention and Mamba-2 SSM heads, developed through extensive ablations on channel allocation, tokenizer design, and training dynamics, including a custom Maximal Update Parametrization (µP) recipe. Falcon-H1 models demonstrate exceptional parameter efficiency, with the flagship 34B model rivaling 70B-scale models and showing significant efficiency gains in long-context tasks; specifically, Falcon-H1-34B achieves up to a 4x improvement in input throughput and an 8x speedup in output throughput over a comparable Transformer model at the longest tested sequence lengths. For AI practitioners, Falcon-H1 provides highly capable models for complex reasoning and long-context applications at a fraction of the size and computational cost of competitors, enabling deployment in resource-constrained environments without sacrificing performance. |
| BANG: Dividing 3D Assets via Generative Exploded Dynamics (Read more on arXiv or HuggingFace) |
Wei Yang, Yinuo Bai, Haoran Jiang, Qixuan Zhang, ZarkLngeW |
BANG is a generative framework that decomposes 3D assets into constituent parts through a controllable, dynamic “exploding” process. The primary objective is to develop a generative model that can dynamically and controllably decompose a 3D object into its meaningful geometric parts, bridging the gap between 3D generation and structural understanding. The methodology involves fine-tuning a large-scale, pre-trained latent diffusion model on a curated dataset of 20k exploded 3D assets using a lightweight “Exploded View Adapter” and a temporal attention module to ensure smooth part transitions. The model produces high-quality part decompositions, with an ablation study demonstrating that its temporal attention module improves the weighted IoU metric for part trajectory tracking by 18.8% (from 0.6874 to 0.8163) compared to a baseline without it. For AI practitioners, this research shows that large, pre-trained generative models can be adapted for complex, dynamic 3D manipulation tasks like part decomposition using lightweight, task-specific modules, enabling part-level control in creative and engineering workflows. |
| VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced |
|
|
| Multimodal Reasoning (Read more on arXiv or HuggingFace) |
Sicong Leng, Chenghao Xiao, Ruifeng Yuan, 26hzhang, kenchan0226 |
The paper introduces VL-COGITO, a multimodal reasoning model trained with a Progressive Curriculum Reinforcement Learning (PCuRL) framework to systematically improve performance on tasks of increasing complexity. The primary objective is to develop a training framework that addresses unstable performance in Multimodal Large Language Models (MLLMs) across diverse reasoning tasks by systematically guiding the model through a curriculum of gradually increasing difficulty. The work proposes the PCuRL framework, which integrates an Online Difficulty Soft Weighting mechanism to dynamically adjust training focus based on prompt difficulty and a Dynamic Length Reward mechanism to adaptively incentivize appropriate reasoning path lengths, all within a multi-stage training process based on Group Relative Policy Optimization (GRPO). VL-COGITO achieves state-of-the-art or highly competitive performance across multiple multimodal benchmarks, demonstrating absolute gains of 7.6% on Geometry@3K and 5.5% on MathVista over its backbone model. The principal implication for AI practitioners is that a multi-stage curriculum learning strategy that progresses from simple to complex tasks and dynamically rewards reasoning length can be applied directly to a base model via reinforcement learning to significantly enhance reasoning capabilities, bypassing a separate supervised fine-tuning phase. |
| Adapting Vehicle Detectors for Aerial Imagery to Unseen Domains with |
|
|
| Weak Supervision (Read more on arXiv or HuggingFace) |
Celso de Melo, Stanislav Panev, Zheyang Qin, Min0326, xiaofanghf |
A generative AI method that synthesizes labeled aerial imagery to adapt vehicle detectors to new geographic domains using weak supervision. The primary objective is to mitigate the performance degradation of vehicle detectors caused by domain shift when applied to new geographic regions, by generating high-quality, labeled synthetic data for the target domain using only weak (image-level) labels. The methodology involves a multi-stage framework that fine-tunes a latent diffusion model on both a fully-labeled source and a weakly-labeled target dataset, then leverages stacked cross-attention maps from object and learnable context tokens to automatically generate pseudo-bounding box labels for synthetic target-domain images, which are subsequently used to train a final detector. The proposed framework demonstrates significant performance gains, improving AP50 by 7-40% over unsupervised domain adaptation methods and 6-10% over weakly supervised methods; for instance, adapting from the DOTA to UGRC dataset, the method achieved a 75.7% AP50, surpassing the best-performing unsupervised method by 33.1 percentage points. The principal implication for AI practitioners is that fine-tuned generative models, particularly their internal cross-attention mechanisms, can be used to create an automated pipeline for generating domain-specific labeled data, substantially reducing the manual annotation cost required to adapt computer vision models to new deployment environments. |
| Towards Omnimodal Expressions and Reasoning in Referring Audio-Visual |
|
|
| Segmentation (Read more on arXiv or HuggingFace) |
Yu-Gang Jiang, Guanquan Jie, Henghui Ding, Kaining Ying |
This paper introduces OmniAVS, a new benchmark for omnimodal referring audio-visual segmentation, and OISA, a multimodal large language model-based method, to enhance reasoning and multimodal understanding in audiovisual scenes. The main objective is to extend the capabilities of referring audio-visual segmentation (RAVS) by enabling deeper understanding, complex reasoning, and multimodal integration across text, speech, sound, and image cues in expressions. The authors propose OmniAVS, a dataset with 8 types of multimodal referring expressions and detailed explanations, and OISA, a Multimodal Large Language Model (MLLM)-based segmentation assistant, which employs Audio-Visual Interleaving for temporal alignment and a query propagation mechanism for efficient segmentation. OISA-1B achieved a state-of-the-art average J&F score of 41.1% on the OmniAVS benchmark, surpassing the previous best method LISA-13B by 5.0%, and demonstrated superior reasoning capabilities with a METEOR score of 21.7% for explanation generation. OmniAVS and OISA collectively provide a practical framework and a challenging benchmark for developing omnimodal AI systems with fine-grained perception and reasoning capabilities, prompting the need for models capable of integrating and reasoning across diverse modalities for real-world applications. |
| Efficient Differentially Private Fine-Tuning of LLMs via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Gilbert Fridgen, Ramin Bahmani, Igor Tchappi, Amir Sartipi, akhadangi |
The paper introduces RLDP, a framework using reinforcement learning to dynamically manage clipping and noise for the differentially private (DP) fine-tuning of large language models. The primary objective is to improve the utility-privacy trade-off by reformulating the optimization of DP parameters as a closed-loop control problem instead of relying on static heuristics. The core methodology involves an online Soft Actor-Critic (SAC) hyper-policy that observes rich statistical summaries of the training dynamics and adjusts per-LoRA-adapter clip radii and noise levels to maximize a reward function that balances utility gains against privacy budget consumption. Across experiments on four LLMs, RLDP achieved an average 5.6% lower perplexity and reached the best baseline’s final utility using, on average, 71% fewer training steps while upholding the same formal privacy guarantees. For AI practitioners, this enables the fine-tuning of LLMs on sensitive data with significantly higher model quality and drastically reduced computational cost, making privacy-preserving AI more practical and effective. |
| Repair-R1: Better Test Before Repair (Read more on arXiv or HuggingFace) |
Quanjun Zhang, Xiaochen Xie, Haichuan Hu |
The paper introduces Repair-R1, a reinforcement learning framework that co-optimizes test case generation and code repair, improving automated program repair by first generating discriminative tests to understand bugs before fixing them. The main objective is to enhance Large Language Model (LLM)-based automated program repair (APR) by shifting from a “repair-then-validate” paradigm to a “test-before-repair” approach, explicitly training the model to first generate tests that expose a bug. The key methodology, Repair-R1, employs Group Relative Policy Optimization (GRPO) to jointly optimize test generation and patch generation, using rule-based rewards for output format, test quality, and repair correctness based on oracle test pass rates. Primary results show that Repair-R1 improves repair success rate by 2.68% to 48.29% and test generation success rate by 16.38% to 53.28% compared to vanilla models across four benchmarks. The principal implication for AI practitioners is that structuring generation tasks to include an explicit diagnostic step (like test case generation) before producing a final output (a code patch) can significantly improve model performance and reasoning, providing a more robust alternative to standard supervised fine-tuning, especially on imbalanced datasets. |
Papers for 2025-07-30
| Title |
Authors |
Summary |
| HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D |
|
|
| Worlds from Words or Pixels (Read more on arXiv or HuggingFace) |
Junta Wu, Zhenwei Wang, HunyuanWorld Team, nightkiller, LeoLau |
HunyuanWorld 1.0 is a framework for generating interactive, mesh-based 3D worlds from text or image inputs using a staged pipeline that leverages panoramic proxies and semantic layering. The primary objective is to create a system that produces 3D-consistent, explorable, and interactive worlds with exportable mesh assets, addressing the respective consistency and data-scarcity limitations of video-based and 3D-based generation methods. Its methodology employs a three-stage process: a Diffusion Transformer generates a 360° panoramic image proxy, an agentic VLM decomposes this panorama into semantic layers, and these layers are then reconstructed into a hierarchical 3D mesh using layer-aligned depth estimation. The method achieves state-of-the-art performance, demonstrating superior alignment in text-to-world generation with a CLIP-T score of 24.0, compared to baselines like Director3D (23.5) and LayerPano3D (22.0). For AI practitioners, the principal implication is a practical pipeline that generates game-engine-ready 3D environments with disentangled, exportable mesh assets, significantly lowering the barrier for interactive content creation in virtual reality, simulation, and game development. |
| X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image |
|
|
| Generative Models Great Again (Read more on arXiv or HuggingFace) |
Yongming Rao, Chen Li, Yeyao Ma, Yibing Wang, Zigang Geng |
The paper introduces X-Omni, a unified autoregressive model that leverages reinforcement learning to generate high-quality images with precise long-text rendering. The main objective is to overcome the low fidelity and poor instruction-following of discrete autoregressive models by applying an RL-based fine-tuning strategy to better align generated visual tokens with a high-fidelity decoder. X-Omni uses a Qwen2.5-7B LLM to autoregressively generate semantic image tokens, which are then rendered into an image by a fixed diffusion decoder; this process is optimized using the Group Relative Policy Optimization (GRPO) algorithm with a multi-component reward function for aesthetics, text-image alignment, and OCR accuracy. The model achieves state-of-the-art performance, scoring an overall 87.65 on the DPG-Bench for text-to-image generation, outperforming models like GPT-4o (86.23). The principal implication for AI practitioners is that reinforcement learning can effectively align separately trained components of a generative system (e.g., an autoregressive model and a diffusion decoder), enabling robust, high-fidelity generation for complex, multi-modal tasks while eliminating the need for classifier-free guidance during inference. |
| CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Chris Shum, Jiwei Li, Albert Wang, Xiaofei Sun, xxiaoyali |
The paper introduces CUDA-L1, a framework using contrastive reinforcement learning to automatically optimize CUDA code for significant performance speedups. The primary objective is to develop an automated framework that can significantly improve CUDA kernel performance by leveraging reinforcement learning to overcome the limitations of existing LLMs in CUDA optimization tasks. The methodology is a three-stage training pipeline: 1) Supervised fine-tuning on a dataset of correct CUDA codes generated by various LLMs; 2) Self-supervised learning where the model iteratively refines itself on its own successfully generated code; and 3) The core component, Contrastive Reinforcement Learning, where the model is prompted with multiple code exemplars and their performance scores to learn comparative analysis and generate superior code, trained using the GRPO algorithm with execution speedup as the reward signal. CUDA-L1 achieves an average speedup of x3.12 (median x1.42) across 250 KernelBench tasks when trained and evaluated on an NVIDIA A100 GPU, with peak speedups reaching x120. The principal implication for AI practitioners is that contrastive RL can automate the complex and time-intensive task of CUDA optimization, transforming a base LLM into a highly effective optimizer capable of discovering non-obvious, high-performance implementations without requiring domain-specific human expertise, thereby improving GPU utilization and reducing engineering overhead. |
| AnimalClue: Recognizing Animals by their Traces (Read more on arXiv or HuggingFace) |
Hirokatsu Kataoka, Christian Rupprecht, Iro Laina, Nakamasa Inoue, Risa Shinoda |
This paper introduces AnimalClue, a large-scale dataset for identifying animal species from indirect evidence like footprints, feces, eggs, bones, and feathers. The primary objective is to create and benchmark the first large-scale, multi-trace dataset to advance computer vision-based wildlife monitoring from indirect clues. The methodology involves collecting 159,605 annotated instances across 968 species from iNaturalist, creating benchmarks for classification, detection, and instance segmentation using models like Swin-B, RT-DETR, and MaskDINO. The primary results indicate the task is highly challenging; for order-level object detection, the RT-DETR model achieved a maximum mean Average Precision (mAP@50-95) of 0.57, and for instance segmentation, MaskDINO achieved a maximum of 0.48, highlighting significant room for improvement. The principal implication for AI practitioners is that AnimalClue provides a new, difficult benchmark for fine-grained visual recognition, demonstrating that state-of-the-art models struggle with identifying species from subtle, varied trace features, which necessitates the development of specialized architectures for such indirect evidence. |
| MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledge (Read more on arXiv or HuggingFace) |
Daoan Zhang, Tianle Wang, Sipeng Zhang, YWZBrandon, Eric-Lan |
MaPPO is a preference optimization framework that extends DPO to a Maximum a Posteriori (MaP) objective, incorporating prior reward knowledge to improve LLM alignment without introducing new hyperparameters. The main objective is to overcome the limitations of purely relative, MLE-based preference optimization methods like DPO, which can lead to poor policy calibration and a “squeezing effect” on response probabilities, by developing a more principled training signal. The key methodology involves augmenting the DPO loss function by introducing a prior derived from a pre-trained reward model; specifically, it uses the reward gap between the preferred and rejected responses to scale the loss contribution of the rejected response, effectively transforming the objective from MLE to MaP. Primary results demonstrate consistent improvements across various models, with MaPPO enhancing the Qwen2.5-7B-Instruct model’s win rate on the Arena-Hard benchmark to 59.2%, a 13.7 absolute point increase over the 45.5% achieved by standard DPO. The principal implication for AI practitioners is that MaPPO can serve as a drop-in plugin for existing DPO-family optimization pipelines to achieve better alignment and more stable policy calibration, especially for high-quality or near-tie preference pairs, without the need for additional hyperparameter tuning. |
| MOVE: Motion-Guided Few-Shot Video Object Segmentation (Read more on arXiv or HuggingFace) |
Henghui Ding, Hengrui Hu, Kaining Ying |
This paper introduces MOVE, a large-scale dataset for motion-guided few-shot video object segmentation (FSVOS), and proposes a baseline method, the Decoupled Motion-Appearance Network (DMA). The primary objective is to segment objects in videos based on their motion patterns, using a few support video examples, rather than relying on static object categories. The DMA method achieves this by explicitly extracting decoupled prototypes: an appearance prototype from mask-pooled features and a motion prototype derived from temporal differencing of frame features, refined by 3D convolutions. On the proposed MOVE benchmark (overlapping split, 2-way-1-shot setting), DMA achieves a mean J&F score of 50.1% with a ResNet50 backbone, significantly outperforming existing category-centric FSVOS methods. For AI practitioners, the key implication is the introduction of a benchmark and a strong baseline for developing models that can perform fine-grained segmentation based on dynamic actions, enabling applications like motion-based video search and analysis which are beyond the scope of category-based systems. |
| Evaluating Deep Learning Models for African Wildlife Image |
|
|
| Classification: From DenseNet to Vision Transformers (Read more on arXiv or HuggingFace) |
Almustapha A Wakili, Nasiru Muhammad, Bilqisu Ismail, Umar Sani Muhammad, lukmanaj |
This paper comparatively evaluates pre-trained CNNs and a Vision Transformer for African wildlife image classification, focusing on the trade-offs between accuracy and computational cost. The objective is to assess the performance of DenseNet-201, ResNet-152, EfficientNet-B4, and ViT-H/14 on a four-class African wildlife dataset to identify a model that balances predictive accuracy with deployment feasibility. The study employs transfer learning with frozen ImageNet pre-trained feature extractors, fine-tuning only the final classification layer of each model on a public dataset of 1,504 images. The Vision Transformer (ViT-H/14) achieved the highest test accuracy at 99%, significantly outperforming the best CNN, DenseNet-201, which reached 67% accuracy. The principal implication for AI practitioners is the stark trade-off between model performance and computational requirements; while large transformer models like ViT-H/14 offer superior accuracy, their substantial parameter count (632M) and GFLOPs make lighter CNNs like DenseNet-201 (20M params) a more practical choice for resource-constrained or edge deployment scenarios. |
Papers for 2025-07-29
| Title |
Authors |
Summary |
| Agentic Reinforced Policy Optimization (Read more on arXiv or HuggingFace) |
Yifei Chen, Licheng Bao, Kai Ma, Hangyu Mao, Guanting Dong |
This paper presents Agentic Reinforced Policy Optimization (ARPO), a novel reinforcement learning algorithm for training multi-turn, tool-using LLM agents. The research aims to address the high uncertainty (token entropy) that LLMs exhibit after interacting with external tools, which current trajectory-level RL methods handle inadequately. ARPO’s methodology incorporates an entropy-based adaptive rollout mechanism that dynamically balances global trajectory sampling with partial, branched sampling at high-entropy steps, along with an advantage attribution estimation to learn from these stepwise interactions. Across 13 benchmarks, ARPO demonstrates superior performance, notably achieving better results while using only half the tool-use budget of existing methods. For AI practitioners, ARPO offers a scalable and cost-efficient solution to align LLM agents for complex, real-time tasks, improving performance and significantly reducing the computational expense of tool calls during training. |
| ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World |
|
|
| Shorts (Read more on arXiv or HuggingFace) |
Junfu Pu, Teng Wang, Chen Li, Yixiao Ge, Yuying Ge |
ARC-Hunyuan-Video-7B is a 7B-parameter multimodal model for the structured, end-to-end comprehension of real-world short videos. The paper’s objective is to develop a model capable of deep, temporally-aware understanding of complex user-generated shorts by jointly processing visual, audio, and textual signals, a task for which current models are inadequate. The methodology involves augmenting a vision-language model with a dedicated audio encoder for fine-grained audio-visual synchronization and an explicit timestamp overlay on video frames for temporal awareness, trained via a multi-stage regimen that includes pre-training, reinforcement learning (GRPO) on verifiable tasks, and instruction fine-tuning. The model achieves a state-of-the-art accuracy of 74.3% on the authors’ custom ShortVid-Bench, significantly outperforming baselines, and shows an inference time of just 10 seconds for a one-minute video on an NVIDIA H20 GPU. For AI practitioners, this work demonstrates that combining an audio-visual architecture with a multi-stage training strategy, especially using RL to ground the model in objective tasks, is a highly effective, production-ready approach for building systems that can perform nuanced analysis of short-form video content. |
| Rep-MTL: Unleashing the Power of Representation-level Task Saliency for |
|
|
| Multi-Task Learning (Read more on arXiv or HuggingFace) |
Dan Xu, Lupin1998, ZedongWangAI |
Rep-MTL is a regularization method that leverages representation-level task saliency to enhance multi-task learning by preserving task-specific patterns while promoting inter-task complementarity. The research objective is to develop a multi-task optimization strategy that operates directly on the shared representation space to explicitly facilitate positive knowledge transfer, as opposed to solely focusing on optimizer-centric conflict resolution. The methodology introduces two regularization components: Task-specific Saliency Regulation (TSR), which uses entropy-based penalization to maintain distinct task patterns, and Cross-task Saliency Alignment (CSA), which employs a contrastive paradigm to align sample-wise saliencies for information sharing. On the challenging NYUv2 benchmark, Rep-MTL achieved a task-level performance gain (ΔP_task) of +1.70 over the single-task baseline, outperforming prior state-of-the-art methods. For AI practitioners, Rep-MTL provides an efficient, optimizer-agnostic module that can be added to standard multi-task architectures to mitigate negative transfer and achieve performance gains without complex gradient manipulation or loss scaling strategies. |
| Reconstructing 4D Spatial Intelligence: A Survey (Read more on arXiv or HuggingFace) |
Chengfeng Zhao, Zhuowei Shen, Zhisheng Huang, Jiahao Lu, Yukang Cao |
This survey organizes the field of 4D spatial intelligence by proposing a new five-level hierarchical framework to provide a structured overview of reconstructing dynamic 3D scenes from visual data. The paper’s objective is to address the lack of a comprehensive, hierarchical analysis in prior works by categorizing existing methods into a progressive taxonomy. The core methodology is this novel five-level classification system: (1) low-level 3D cues, (2) 3D scene components, (3) 4D dynamic scenes, (4) interaction modeling, and (5) incorporation of physical laws. The primary result is the structured synthesis of the field, which highlights key advancements such as end-to-end frameworks like VGGT that can estimate fundamental 3D cues within seconds. For AI practitioners, this survey offers a systematic map to understand the state-of-the-art, pinpoint challenges at each level of abstraction, and guide the development of more physically grounded and interactive models for embodied AI and AR/VR applications. |
| SmallThinker: A Family of Efficient Large Language Models Natively |
|
|
| Trained for Local Deployment (Read more on arXiv or HuggingFace) |
Dongliang Wei, Zhenliang Xue, qsstcl, Sorrymaker2024, yixinsong |
The paper introduces SmallThinker, a family of large language models architected from the ground up for efficient local deployment on resource-constrained devices. The main research objective is to design an LLM natively for local hardware constraints (weak compute, limited memory, slow storage) instead of adapting cloud-based models. The key methodology is a deployment-aware co-design featuring a two-level sparse structure with Mixture-of-Experts (MoE), a pre-attention router to prefetch expert parameters and hide I/O latency, and a NoPE-RoPE hybrid sparse attention mechanism to reduce KV cache. The primary result is that the SmallThinker-21B-A3B model achieves a state-of-the-art MMLU score of 84.4 and, with Q4_0 quantization on a consumer PC with an 8GB memory limit, attains an inference speed of 20.30 tokens/s. The principal implication for AI practitioners is that co-designing model architecture and the inference engine for specific hardware enables high-performance LLM execution on local, non-GPU devices, demonstrating a viable alternative to simple model compression or cloud-only deployment. |
| A Survey of Self-Evolving Agents: On Path to Artificial Super |
|
|
| Intelligence (Read more on arXiv or HuggingFace) |
Jiayi Geng, Huan-ang Gao, XiangJinYu, didiforhugface, Alphamasterliu |
This survey provides a systematic framework for self-evolving agents by categorizing them along the dimensions of what, when, and how they evolve as a path toward Artificial Super Intelligence. The paper’s objective is to establish the first comprehensive, systematic review of self-evolving agents by organizing the field around three foundational dimensions: what to evolve (e.g., models, context, tools, architecture), when to evolve (intra-test-time vs. inter-test-time), and how to evolve (e.g., reward-based, imitation, population-based methods). As a survey, its methodology is a taxonomic decomposition of existing research, analyzing and structuring prior work into a unified framework that also covers evaluation paradigms and applications. The primary result is a synthesis of findings demonstrating that self-evolution mechanisms significantly improve agent capabilities; for example, the paper cites that the WebVoyager agent improved its end-to-end success rate on unseen websites from 30% to 59% via successive self-fine-tuning. The principal implication for AI practitioners is that this survey provides a structured design framework (Figure 3) for developing adaptive agentic systems, enabling engineers to systematically analyze, compare, and select appropriate evolutionary components and learning strategies for specific applications, thereby creating more robust and versatile real-world agents. |
| Geometric-Mean Policy Optimization (Read more on arXiv or HuggingFace) |
Xun Wu, Jingye Chen, Yue Liu, Yuzhong Zhao, jeepliu |
This paper introduces Geometric-Mean Policy Optimization (GMPO), a method that stabilizes Group Relative Policy Optimization (GRPO) by maximizing the geometric mean, rather than the arithmetic mean, of token-level rewards. The primary objective is to mitigate the unstable policy updates in GRPO caused by extreme importance sampling ratios. The core methodology replaces the arithmetic mean in the GRPO objective with a geometric mean and applies token-level clipping, which is inherently less sensitive to outlier rewards and allows for a wider clipping range to enhance exploration. GMPO-7B outperforms GRPO by an average of 4.1% on mathematical benchmarks and by 1.4% on the Geometry3K multimodal benchmark, while maintaining a more stable importance sampling ratio, lower KL divergence, and higher token entropy during training. For AI practitioners, GMPO provides a more stable and effective algorithm for reinforcement learning post-training of LLMs on reasoning tasks, improving final performance by reducing update instability. |
| Region-based Cluster Discrimination for Visual Representation Learning (Read more on arXiv or HuggingFace) |
Yongle Zhao, Yin Xie, Athinklo, xiangan, Kaichengalex |
The paper introduces RICE, a region-aware visual representation learning method that uses cluster discrimination to improve performance on dense prediction tasks like segmentation and OCR. The main objective is to overcome the limitations of global representations in vision-language models by developing a framework that learns effective region-level visual features without relying on per-region textual annotations. The methodology involves creating a billion-scale region dataset with pseudo-labels derived from k-means clustering (for objects) and text tokenization (for OCR), and then training a model that incorporates a novel Region Transformer layer using a unified region-cluster discrimination loss. Extensive experiments show RICE outperforms prior methods, with its ViT-B/16 model achieving a 38.9% detection AP on COCO, surpassing a strong SigLIP baseline by +3.9%. For AI practitioners, the pre-trained RICE models provide a superior vision encoder backbone for MLLMs and other downstream applications requiring robust, localized object and text recognition. |
| GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Dataset (Read more on arXiv or HuggingFace) |
Qing Liu, Letian Zhang, Siwei Yang, Yuhan Wang, tennant |
The paper introduces GPT-IMAGE-EDIT-1.5M, a large-scale, publicly available dataset of over 1.5 million image editing triplets, created by systematically refining existing datasets using GPT-4o. The research objective is to bridge the performance gap between proprietary and open-source instruction-guided image editing models by creating and releasing a high-quality, large-scale training dataset. The key methodology involves unifying and refining three popular datasets (OmniEdit, HQ-Edit, UltraEdit) by leveraging GPT-4o to 1) regenerate output images for enhanced visual quality and instruction alignment, and 2) selectively rewrite prompts for improved semantic clarity. The primary result is that an open-source model fine-tuned on the new dataset achieves state-of-the-art performance among open-source methods, scoring 7.24 on the GEdit-EN-full benchmark, markedly exceeding previously published models. The principal implication for AI practitioners is the provision of a direct, high-quality data resource for training superior open-source image editing models, along with a validated methodology for using frontier models to systematically enhance the quality and alignment of existing datasets. |
| UloRL:An Ultra-Long Output Reinforcement Learning Approach for Advancing |
|
|
| Large Language Models’ Reasoning Abilities (Read more on arXiv or HuggingFace) |
Yang Li, Shaohua Chen, Tao Yang, forestliutc, dongdongdongdu |
This paper introduces UloRL, a reinforcement learning approach that improves Large Language Model reasoning by efficiently training on ultra-long output sequences. The primary objective is to overcome the inefficiencies of traditional reinforcement learning, specifically long-tail distribution delays and entropy collapse, when training LLMs with outputs up to 128k tokens. The key methodology involves two main techniques: 1) Segment Rollout, which divides the decoding of ultra-long outputs into shorter segments to accelerate training, and 2) Dynamic Masking of well-Mastered Positive Tokens (DMMPTs), which prevents entropy collapse by adaptively excluding high-confidence positive tokens from training updates when model entropy falls below a target threshold. The proposed UloRL approach, when applied to the Qwen3-30B-A3B model with 128k-token outputs, improved performance on the AIME2025 benchmark from 70.9% to 85.1% and on the BeyondAIME benchmark from 50.7% to 61.9%. For AI practitioners, the principal implication is that employing segment rollouts and dynamic token masking provides a scalable and efficient method to conduct reinforcement learning on ultra-long sequences, overcoming critical training bottlenecks to significantly enhance the complex reasoning capabilities of LLMs. |
| ForCenNet: Foreground-Centric Network for Document Image Rectification (Read more on arXiv or HuggingFace) |
Jia Li, Dong Guo, Qiang Li, Peng Cai, Kaichengalex |
ForCenNet is a deep learning framework for document image rectification that leverages foreground-centric information generated from undistorted images to guide the unwarping process. The primary objective is to enhance rectification accuracy and preserve document readability by focusing the model’s attention on critical foreground elements like text and table lines, which existing methods often treat uniformly with the background. Its methodology combines a novel label generation process to create foreground masks and line elements from clean images, a mask-guided Transformer decoder that directs attention to these foreground regions, and a curvature consistency loss to maintain the geometric structure of linear elements. The network achieves state-of-the-art performance, attaining an MS-SSIM of 0.713 on the DIR300 benchmark, surpassing prior models. For AI practitioners, the principal implication is that explicitly modeling and preserving the geometric properties of semantically meaningful foreground content, even with synthetically generated labels, is a highly effective strategy for improving performance on document image restoration and subsequent OCR tasks. |
| Met^2Net: A Decoupled Two-Stage Spatio-Temporal Forecasting Model for |
|
|
| Complex Meteorological Systems (Read more on arXiv or HuggingFace) |
Xiaolin Qin, Min Chen, Hao Yang, Shaohan Li |
Met²Net is a decoupled, two-stage spatio-temporal forecasting model that improves multivariate meteorological prediction by addressing representation and task inconsistencies. The primary objective is to develop a forecasting framework that effectively integrates highly divergent meteorological variables by resolving the performance degradation caused by representation inconsistency and the sub-optimal training resulting from task inconformity between reconstruction and prediction stages. The methodology involves an implicit two-stage training paradigm where in stage one, variable-specific encoders and decoders are trained for reconstruction while a translator is frozen, and in stage two, the encoders/decoders are frozen while the translator, using a self-attention mechanism, is trained on a latent space prediction task, with momentum updates applied to frozen components to align objectives. The proposed model achieves state-of-the-art performance, reducing the Mean Squared Error (MSE) for near-surface air temperature and relative humidity predictions by 28.82% and 23.39%, respectively, compared to the TAU baseline. For AI practitioners, this research provides a powerful framework for multivariate time-series forecasting, demonstrating that treating input variables as independent modalities with dedicated encoders combined with an implicit two-stage training strategy effectively fuses heterogeneous data and improves prediction accuracy in complex systems. |
| ScenePainter: Semantically Consistent Perpetual 3D Scene Generation with |
|
|
| Concept Relation Alignment (Read more on arXiv or HuggingFace) |
Khodchaphun Hirunyaratsameewong, Chang Liu, Fangfu Liu, Shengjun Zhang, xiac24 |
ScenePainter is a framework for generating long-range, semantically consistent 3D view sequences from a single image by aligning concept relations. The primary objective is to address the semantic drift problem in perpetual 3D scene generation, where iteratively generated views deviate from the original scene’s semantic context due to accumulated outpainting errors. The key methodology uses a hierarchical graph structure, SceneConceptGraph, to model relations among multi-level scene concepts, which then directs a customized outpainting diffusion model to generate consistent novel views. The framework significantly improves scene fidelity, achieving a state-of-the-art DINO score of 0.931 and was preferred for consistency over the WonderJourney baseline in 92.6% of user study comparisons. For AI practitioners, the main implication is a novel technique for maintaining long-term semantic control in iterative generative models, which can mitigate cumulative error in applications like long-form video synthesis or 3D world building. |
| Music Arena: Live Evaluation for Text-to-Music (Read more on arXiv or HuggingFace) |
Wei-Lin Chiang, Anastasios N. Angelopoulos, Wayne Chi, Yonghyun Kim, chrisdonahue |
The paper presents Music Arena, an open platform for scalable, live human preference evaluation of text-to-music (TTM) models. The primary objective is to establish a rigorous, renewable evaluation protocol and an open dataset to address the lack of standardized, human-centric evaluation for TTM systems. The methodology involves users engaging in pairwise “battles” between two TTM models, with a backend powered by an LLM (GPT-4o) that moderates prompts and routes them to heterogeneous model endpoints, while collecting detailed preference and fine-grained listening data. While the paper states that aggregate user preference results are not yet available, it provides a specific system-level quantitative finding from a battle log where a Riffusion FUZZ 1.0 model generated audio at 8.0x real-time speed. The principal implication for AI practitioners is the provision of a unified, open-source, Docker-based framework and a recurring, transparently-released dataset of human preferences, enabling more rigorous model evaluation and the development of TTM systems better aligned with human intent. |
| JAM: A Tiny Flow-based Song Generator with Fine-grained Controllability |
|
|
| and Aesthetic Alignment (Read more on arXiv or HuggingFace) |
Amir Ali Bagherzadeh, Taylor Gautreaux, Navonil Majumder, Renhang Liu, hungchiayu |
The paper presents JAM, a 530M-parameter flow-matching model for lyrics-to-song generation that offers fine-grained word-level timing control and aesthetic alignment. The primary objective is to create a compact and efficient song generation model that overcomes the limitations of prior work by enabling precise control over word timing, overall song duration, and improving lyrical fidelity. The methodology utilizes a rectified-flow model with a Diffusion Transformer (DiT) backbone, conditioned on word-level temporal annotations, and employs iterative Direct Preference Optimization (DPO) with synthetic preference labels from the SongEval toolkit to enhance aesthetic quality without manual annotation. On the custom JAME benchmark, JAM achieves a Word Error Rate (WER) of 0.151, which is less than half that of the next-best system, demonstrating significantly improved lyrical alignment and vocal clarity. For AI practitioners, this research provides a framework for building highly controllable audio generation systems by showing that explicit, fine-grained temporal conditioning is a critical mechanism for improving both user control and objective metrics like WER, making AI tools more viable for professional creative workflows. |
| Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty (Read more on arXiv or HuggingFace) |
Leshem Choshen, Idan Shenfeld, Stewart Slocum, Isha Puri, Mehul Damani |
This paper introduces RLCR (Reinforcement Learning with Calibration Rewards), a method that trains language models to improve both accuracy and calibrated confidence estimation. The research objective is to determine if models can be optimized for both correctness and calibration by having the model’s own reasoning chain inform its confidence. The key methodology is to train a model via reinforcement learning using a composite reward function that augments a standard binary correctness score with a Brier score, a proper scoring rule that penalizes poorly calibrated confidence estimates. On the HotpotQA dataset, RLCR reduced the expected calibration error to 0.03 from 0.37 in standard RL training, while maintaining competitive accuracy and improving out-of-domain performance. The principal implication for AI practitioners is that explicitly training for calibration using this method can produce more reliable reasoning models that better communicate their own uncertainty, a critical feature for trustworthy AI systems. |
| GenoMAS: A Multi-Agent Framework for Scientific Discovery via |
|
|
| Code-Driven Gene Expression Analysis (Read more on arXiv or HuggingFace) |
Haohan Wang, Yijiang Li, Liu-Hy |
GenoMAS is a code-driven, multi-agent framework using six specialized, heterogeneously-backed LLM agents to automate complex gene expression analysis workflows. The main objective is to develop an automated system that bridges the gap between general-purpose agentic reasoning and the precise, code-driven, domain-specific requirements of scientific computation, specifically for end-to-end gene expression analysis from raw data. The methodology involves orchestrating six specialized agents (PI, Data Engineer, Statistician, Code Reviewer, Domain Expert) with distinct LLM backbones (Claude Sonnet 4, OpenAI o3, Gemini 2.5 Pro). The system uses a guided planning framework where tasks are decomposed into editable “Action Units,” an iterative code generation-review-revision loop, and a dynamic code memory for reusing validated snippets, all managed via a typed message-passing protocol. On the GenoTEX benchmark, GenoMAS achieves a 60.48% F1 score in gene identification, a 16.85% absolute improvement over the previous state-of-the-art, GenoAgent. The principal implication for AI practitioners is that for complex, domain-specific tasks requiring scientific rigor, an architecture treating agents as collaborative programmers with specialized roles, heterogeneous LLM backbones, and structured mechanisms for code generation and review is more effective than general-purpose autonomous agents or rigid, tool-based workflow orchestrators. |
Papers for 2025-07-28
| Title |
Authors |
Summary |
| The Geometry of LLM Quantization: GPTQ as Babai’s Nearest Plane |
|
|
| Algorithm (Read more on arXiv or HuggingFace) |
Dan Alistarh, Torsten Hoefler, softmax |
This research demonstrates that the GPTQ quantization algorithm, when executed in a back-to-front order, is mathematically identical to Babai’s nearest plane algorithm for the closest vector problem on a lattice defined by the input Hessian. The paper’s main objective is to establish a formal geometric and theoretical foundation for the empirically successful GPTQ algorithm by proving its equivalence to a classical lattice algorithm, thereby explaining its effectiveness and providing worst-case guarantees. The authors use a formal mathematical proof to equate the linear-layer L2 quantization objective with the closest vector problem (CVP) and then demonstrate that the iterative update steps of back-to-front GPTQ are algebraically equivalent to the projections in Babai’s algorithm. The primary result is this proven equivalence, which provides a geometric interpretation for GPTQ’s error propagation as an orthogonal projection; consequently, GPTQ inherits a tight error upper bound from Babai’s algorithm in the no-clipping case, with the expected error being exactly 1/3 of this worst-case bound under a uniform prior on weights. For AI practitioners, this connection enables the direct application of established lattice algorithm techniques, such as basis reduction and novel ordering heuristics like the proposed “min-pivot” method, to create more principled and potentially more accurate post-training quantization algorithms for large models. |
| Deep Researcher with Test-Time Diffusion (Read more on arXiv or HuggingFace) |
Guan Sun, Lesly Miculicich, Zoey CuiZhu, Yanfei Chen, Rujun Han |
This paper introduces the Test-Time Diffusion Deep Researcher (TTD-DR), a framework that models long-form report generation as a diffusion process, iteratively refining a draft using retrieval and self-evolution. The objective is to overcome the performance limitations of existing deep research agents on complex tasks by emulating the iterative human process of drafting, searching, and revision. The core methodology conceptualizes report generation as a “denoising” process where an initial draft is progressively refined using external information from a retrieval mechanism, while a self-evolutionary algorithm optimizes each component of the agentic workflow. TTD-DR achieves state-of-the-art results, demonstrating a 69.1% win rate against OpenAI Deep Research on the LongForm Research benchmark. For AI practitioners, this work presents a highly effective test-time scaling strategy, showing that a draft-centric diffusion approach combined with component-wise self-evolution creates more coherent and accurate research agents than traditional linear or parallelized agentic systems. |
| Specification Self-Correction: Mitigating In-Context Reward Hacking |
|
|
| Through Test-Time Refinement (Read more on arXiv or HuggingFace) |
vicgalle |
The paper introduces Specification Self-Correction (SSC), a test-time, multi-step inference framework that enables LMs to identify and correct flaws in their own guiding specifications to mitigate reward hacking. The main objective is to develop a method that allows a language model to mitigate in-context reward hacking by actively identifying a flaw within its guiding specification and autonomously correcting it at inference time. The key methodology is a four-step process: 1) initial response generation using the flawed specification, 2) self-critique of that response, which exposes the exploit, 3) self-refinement of the specification itself to remove the flaw, and 4) final generation of a robust response using the corrected specification. Across creative writing and agentic coding tasks, models that initially exploited flawed specifications in 50-70% of cases demonstrated a reduction in this vulnerability by over 90% after applying SSC; specifically, the average initial hacking rate of 59% in creative writing tasks dropped to 3.2%. The principal implication for AI practitioners is that this weight-agnostic, inference-time technique can be implemented to improve the robustness of deployed LMs by allowing them to dynamically patch their operational rubrics, turning the failure mode of specification gaming into a corrective signal for self-improvement without requiring model retraining. |
| PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving (Read more on arXiv or HuggingFace) |
Patric Jensfelt, Yixi Cai, Lianhang Liu, maciejw94 |
The paper introduces PRIX, a computationally efficient, camera-only, end-to-end autonomous driving model that directly plans trajectories from raw pixel inputs, outperforming larger multimodal systems. The main objective is to develop a scalable end-to-end driving model that operates solely on camera data, eliminating reliance on LiDAR and computationally intensive BEV representations, while achieving state-of-the-art planning performance. The key methodology involves a ResNet visual backbone enhanced by a novel Context-aware Recalibration Transformer (CaRT) module, which uses shared self-attention to refine multi-scale features. These rich features are then used by a conditional diffusion planner and auxiliary heads for object detection and semantic segmentation within a multi-task learning framework. The primary result is achieving a state-of-the-art PDMS score of 87.8 on the NavSim-v1 benchmark, outperforming prior camera-only models like Hydra-MDP++ (86.6) and multimodal models like GoalFlow+ (85.7), while operating at 57 FPS with only 37M parameters. The principal implication for AI practitioners is that a powerful visual feature extractor, trained with appropriate auxiliary tasks, can be more critical than planner complexity or multimodal sensor fusion for building performant and efficient autonomous driving systems, demonstrating a viable path to scalable, low-cost solutions without reliance on explicit BEV projections. |
| Chat with AI: The Surprising Turn of Real-time Video Communication from |
|
|
| Human to AI (Read more on arXiv or HuggingFace) |
Xinggong Zhang, Liming Liu, Zhiyuan Ren, keyonN |
This paper introduces Artic, a real-time communication framework that optimizes video streaming for MLLM understanding to minimize latency in AI video chat. The main objective is to reduce transmission latency to under 68ms by shifting the optimization goal from human perceptual quality to MLLM response accuracy. The key methodology combines Context-Aware Video Streaming, which uses CLIP to dynamically allocate bitrate to semantically important regions, and Loss-Resilient Adaptive Frame Rate, which leverages redundant frames to mitigate packet loss without retransmission. A primary result shows that when bitrate is reduced from 800 Kbps to 400 Kbps, context-aware streaming maintains MLLM accuracy at 0.87, whereas a standard approach drops to 0.33. The principal implication for AI engineers is that video compression for MLLM consumption can be aggressively optimized for machine understanding, rather than human perception, allowing for significant bitrate and latency reductions while preserving downstream task accuracy. |
Papers for 2025-07-25
| Title |
Authors |
Summary |
| Group Sequence Policy Optimization (Read more on arXiv or HuggingFace) |
Bowen Yu, Xiong-Hui Chen, Mingze Li, Shixuan Liu, Chujie Zheng |
This paper introduces Group Sequence Policy Optimization (GSPO), an RL algorithm that stabilizes large language model training by performing optimization using sequence-level likelihood ratios instead of token-level ones. The primary objective is to develop a stable and efficient RL algorithm that overcomes the model collapse issues observed in methods like Group Relative Policy Optimization (GRPO), especially when training large Mixture-of-Experts (MoE) models. The key methodology is to define the importance sampling ratio based on the likelihood of the entire generated sequence (s_i(θ) = π_θ(y_i|x) / π_θ_old(y_i|x)) and apply this single ratio for sequence-level clipping, rewarding, and optimization, thereby aligning the optimization unit with the sequence-level reward. GSPO demonstrates superior training efficiency and stability over GRPO; quantitatively, it clips a token fraction of 0.15, two orders of magnitude higher than GRPO’s 0.0013, while achieving better performance, indicating a more reliable learning signal. For AI practitioners, GSPO provides a more robust algorithm for RLHF that fundamentally resolves instability in MoE model training without needing complex workarounds like Routing Replay, and it can potentially simplify RL infrastructure by reducing the need for likelihood recomputation. |
| LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy |
|
|
| Optimization (Read more on arXiv or HuggingFace) |
Linjuan Wu, Shangke Lyu, Xingyu Wu, tricktreat, yanyc |
This paper introduces Length-Adaptive Policy Optimization (LAPO), a framework for training large language models to intrinsically control their reasoning length based on problem complexity. The research objective is to address the “overthinking” phenomenon in LLMs by enabling them to autonomously determine an appropriate reasoning depth for a given task, rather than relying on external constraints. LAPO utilizes a two-stage reinforcement learning process where a “Discovery” stage first learns the statistical distribution of successful solution lengths, and a subsequent “Internalization” stage trains the model to generate and adhere to a self-proposed length budget embedded within its reasoning context. Experiments show that LAPO reduces token usage by up to 40.9% while simultaneously improving accuracy by 2.3% on mathematical reasoning benchmarks. For AI practitioners, this framework offers a method to fine-tune models for greater computational efficiency and cost-effectiveness by enabling them to self-regulate reasoning effort based on problem difficulty, thereby making them more practical for deployment. |
| MUR: Momentum Uncertainty guided Reasoning for Large Language Models (Read more on arXiv or HuggingFace) |
Jian Zhang, Yifei Li, Rongman Xu, Fangzhi Xu, Hang Yan |
This paper introduces Momentum Uncertainty-guided Reasoning (MUR), a training-free algorithm that adaptively applies test-time scaling to LLMs to reduce computational overhead while improving reasoning performance. The main objective is to efficiently and adaptively guide LLM test-time scaling without additional training, thereby mitigating the “overthinking” problem where models waste tokens on redundant computations. The key methodology involves calculating momentum uncertainty, an exponentially weighted average of step-level uncertainties, which acts as a dynamic threshold to trigger compute-intensive scaling only for critical reasoning steps. Results demonstrate that across four benchmarks and three model sizes, MUR reduces computation by over 50% on average while improving accuracy by 0.62-3.37% compared to methods that scale every step. The principal implication for AI practitioners is that MUR can be implemented as an orthogonal, training-free module with existing test-time scaling methods to significantly decrease inference costs and latency in production for reasoning tasks, without degrading and often enhancing accuracy. |
| TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yujie Wei, Shiwei Zhang, Yukang Chen, Ruihang Chu, Zhekai Chen |
TTS-VAR is a test-time scaling framework that improves visual auto-regressive (VAR) generation by applying scale-dependent path-searching strategies. The main objective is to develop a general, training-free, test-time scaling framework for VAR models to enhance generation quality by addressing the unique challenges of their hierarchical, coarse-to-fine process. The key methodology combines three components: an adaptive descending batch size schedule to manage computational cost, clustering-based diversity search using DINOv2 features at coarse scales to preserve structural variety, and resampling-based potential selection using reward models at fine scales to prioritize high-quality candidates. The primary result is a notable 8.7% improvement in the GenEval score for the Infinity VAR model, from 0.69 to 0.75, which surpasses the performance of conventional Best-of-N (BoN) sampling even with fewer samples. The principal implication for AI practitioners is that the performance of hierarchical generative models like VAR can be significantly enhanced at inference time by applying different optimization strategies to different generation scales—specifically, focusing on diversity at early stages and reward-based selection at later stages. |
| Captain Cinema: Towards Short Movie Generation (Read more on arXiv or HuggingFace) |
Yang Zhao, Shengqu Cai, Lvmin Zhang, Ceyuan Yang, Junfei Xiao |
Captain Cinema is a framework for generating narratively consistent short movies by first planning a sequence of coherent keyframes from a storyline and then synthesizing video between them. Its main objective is to overcome long-range dependency challenges in video generation by employing a two-stage methodology: a top-down keyframe planner uses a novel “GoldenMem” context compression mechanism, which then conditions a bottom-up video synthesis model. The key methodology, GoldenMem, uses golden-ratio-based downsampling of past visual frames to maintain a fixed-cost, long-term visual memory, enabling stable generation over extended sequences. The framework demonstrates strong long-context performance, maintaining over 93% of its initial consistency score when scaled to 48 context pairs and achieving a temporal dynamics score of 65.4, significantly outperforming a baseline of 51.8. For AI practitioners, this work provides a computationally efficient memory strategy (GoldenMem) and a disentangled architecture for scaling video generation from isolated clips to coherent, story-driven content. |
| EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent |
|
|
| Diffusion (Read more on arXiv or HuggingFace) |
Jing Wang, Wen Qian, Chaohui Yu, Chenjie Cao, ShuYaoLiu |
This paper introduces EarthCrafter, a scalable framework for geographic-scale 3D Earth generation, and a new large-scale aerial dataset, Aerial-Earth3D, to support it. The primary objective is to scale 3D generative models to geographic extents by developing a novel data infrastructure and a highly efficient model architecture. The methodology employs a dual-sparse latent diffusion approach that separates structural and textural generation, using dual sparse 3D-VAEs to compress geometric voxels and 2D Gaussian Splats into compact latents, which are then modeled by tailored flow matching networks. The proposed StructVAE achieves 97.1% accuracy in structural reconstruction, demonstrating high fidelity while operating on a spatially compressed latent space. For AI practitioners, this research provides a new architectural pattern for efficiently handling large-scale 3D data generation, along with the largest-to-date, richly annotated 3D aerial dataset (Aerial-Earth3D) for training and benchmarking such models. |
| Hierarchical Budget Policy Optimization for Adaptive Reasoning (Read more on arXiv or HuggingFace) |
Xingyu Wu, Linjuan Wu, tricktreat, yanyc, paradox122 |
This paper introduces Hierarchical Budget Policy Optimization (HBPO), a reinforcement learning framework for training models to adaptively adjust their reasoning depth to match problem complexity. The objective is to develop a training methodology that enables large reasoning models to learn differentiated, problem-specific reasoning depths, thereby improving computational efficiency without sacrificing performance on complex tasks. The HBPO method partitions the RL exploration space into multiple subgroups, each constrained by a distinct token budget, and uses a piecewise, budget-aware reward function with decomposed advantage computation to guide the model in learning to select appropriate computational effort. Experiments show HBPO reduces average token usage by up to 60.6% while simultaneously improving accuracy by 3.14% across four mathematical reasoning benchmarks, demonstrating emergent adaptive behavior where token allocation correlates with problem difficulty. For AI practitioners, this framework offers a method to train reasoning models that are both more computationally efficient and more capable, overcoming the typical trade-off between performance and inference cost by enabling learned, adaptive resource allocation rather than applying uniform constraints. |
| DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts (Read more on arXiv or HuggingFace) |
Ricardo Simón Carbajo, Miguel Aspis, suarezcetrulo, sebasmos |
The paper introduces DriftMoE, a novel online Mixture-of-Experts framework using a co-trained neural router and incremental tree experts to adapt to concept drift in data streams. The objective is to develop an adaptive model for non-stationary data streams that overcomes the limitations of existing ensembles by enabling more nuanced expert specialization without relying on explicit drift detectors. DriftMoE co-trains a lightweight neural router alongside a pool of incremental Hoeffding Tree experts; the router gates instances to experts and is then updated using a multi-hot “correctness mask” derived from every expert’s prediction accuracy on the instance, providing a cooperative training signal. The framework was evaluated on nine benchmarks against established adaptive ensembles, where the MoE-Data variant achieved a prequential accuracy of 70.33% on the Airlines dataset, outperforming baselines like Adaptive Random Forest (64.51%) while using fewer experts. For AI practitioners, DriftMoE presents a resource-efficient and highly adaptive alternative to large-scale ensembles for streaming applications, showing that a small pool of specialized experts managed by a co-trained router can achieve competitive or superior performance. |
| Technical Report of TeleChat2, TeleChat2.5 and T1 (Read more on arXiv or HuggingFace) |
Yu Zhao, Chao Wang, Yitong Yao, Xinzhang Liu, Zihan Wang |
This paper presents TeleChat2, TeleChat2.5, and T1, a series of open-weight 35B and 115B parameter LLMs developed through an enhanced multi-stage training pipeline. The main objective is to create and publicly release a new series of high-performance LLMs that improve upon their predecessor by systematically upgrading the pre-training and post-training stages to advance capabilities in general tasks, complex reasoning, and coding. The methodology consists of pre-training a base model on 10 trillion tokens, followed by a pipeline including continual pre-training on domain-specific data, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a final Reinforcement Learning (RL) stage to explicitly enhance mathematical and coding abilities. The primary results show that the models are competitive with or outperform leading proprietary systems; specifically, the T1-115B model achieves a score of 94.0 on the MATH500 benchmark in thinking mode, surpassing OpenAI’s o1-mini model’s score of 90.0. The principal implication for AI practitioners is the public release of these 35B and 115B models, providing open access to state-of-the-art LLMs. This allows engineers to leverage and fine-tune powerful foundation models for complex reasoning, coding, and instruction-following applications without dependency on closed-source APIs. |
| A New Pair of GloVes (Read more on arXiv or HuggingFace) |
Christopher D. Manning, John Bauer, Riley Carlson |
This paper presents and evaluates new 2024 GloVe word embedding models trained on updated corpora to capture contemporary English. The objective was to create and document updated models using recent data (Wikipedia, Gigaword, and a subset of Dolma) and evaluate whether they better represent modern language and improve downstream task performance compared to the original 2014 models. The methodology involved training new GloVe vectors using the original algorithm on these updated corpora and evaluating them through vocabulary comparison, direct analogy/similarity tests, and performance on four NER datasets, including the recent Worldwide and WNUT-17 datasets. The primary result is that while performance on classic analogy tasks was comparable, the 2024 embeddings showed significant improvement on temporally-dependent NER tasks; for example, the 2024 50d Wiki/Giga model achieved a per-entity F1 score of 84.64 on the Worldwide dataset, compared to 82.1 for the 2014 version. The principal implication for AI practitioners is that these 2024 GloVe embeddings are better suited for modern NLP applications, especially those dealing with recent text or requiring recognition of contemporary entities, as they reduce out-of-vocabulary issues and improve performance on such tasks. |
| DMOSpeech 2: Reinforcement Learning for Duration Prediction in |
|
|
| Metric-Optimized Speech Synthesis (Read more on arXiv or HuggingFace) |
Kaifeng Xu, Cheng Niu, Fei Tao, Xilin Jiang, Yinghao Aaron Li |
DMOSpeech 2 introduces a reinforcement learning framework to optimize the previously isolated duration predictor in diffusion-based text-to-speech (TTS) systems, alongside a teacher-guided sampling method to restore output diversity. The primary objective is to enable end-to-end optimization of a zero-shot TTS pipeline for perceptual metrics by integrating the duration prediction component, which was a critical bottleneck in prior metric-optimized systems. The key methodology involves modeling the duration predictor as a stochastic policy and fine-tuning it with Group Relative Policy Optimization (GRPO), using a reward signal composed of speaker similarity and word error rate. A hybrid “teacher-guided sampling” strategy is also employed, leveraging a teacher model for initial denoising steps to establish prosodic structure and an efficient student model for final acoustic refinement. The proposed method significantly improves performance; on the Seed-TTS-en dataset, optimizing the duration predictor with RL reduced the Word Error Rate (WER) from 3.750 to 1.752 compared to the baseline without RL optimization, while maintaining a low Real-Time Factor (RTF) of 0.0316. The principal implication for AI practitioners is that targeted reinforcement learning can be efficiently applied to specific, non-differentiable components within a larger generative model to optimize for system-level metrics, overcoming the high computational overhead typically associated with applying RL to an entire pipeline. |
| GLiNER2: An Efficient Multi-Task Information Extraction System with |
|
|
| Schema-Driven Interface (Read more on arXiv or HuggingFace) |
Ash Lewis, George Hurn-Maloney, Oliver Boyd, Gil Pasternak, Urchade Zaratiana |
GLiNER2 is a unified, CPU-efficient framework that performs named entity recognition, text classification, and hierarchical structured data extraction within a single encoder model using a schema-driven interface. The objective is to develop a single, compact model that performs diverse information extraction tasks to overcome the high computational, cost, and privacy barriers associated with deploying large language models or multiple specialized systems. The system extends the GLiNER architecture by using a pretrained transformer encoder (205M parameters) prompted with a unified input format that uses special tokens to define and compose multiple tasks, trained on a 254,334-example dataset of LLM-annotated and synthetic data. In zero-shot evaluations, GLiNER2 achieves an average F1 score of 0.590 on the CrossNER benchmark, closely matching GPT-4o’s score of 0.599, while demonstrating an approximate 2.6x speedup over the GPT-4o API on classification tasks when running on a CPU. The principal implication for AI practitioners is the availability of an open-source, pip-installable library for deploying high-performance, multi-task information extraction on standard CPU hardware, enabling complex, privacy-sensitive applications without reliance on GPUs or costly LLM APIs. |
| TeEFusion: Blending Text Embeddings to Distill Classifier-Free Guidance (Read more on arXiv or HuggingFace) |
Zhao Xu, Qing-Guo Chen, Xiaohao Chen, Minghao Fu, Flourish |
TeEFusion is a distillation method that accelerates text-to-image generation by fusing conditional and unconditional text embeddings to eliminate the multiple forward passes required by Classifier-Free Guidance (CFG). The objective is to distill the behavior of a teacher model using a complex, multi-pass sampling strategy into a student model that requires only a single forward pass per step, without adding extra model parameters. The methodology involves injecting the guidance signal by linearly combining the conditional and unconditional text embeddings, scaling them by the guidance weight w, and feeding this fused representation into the model. The primary result shows that on the SD3 model, a TeEFusion-distilled student achieves comparable or higher HPS aesthetic scores than a teacher using the complex W2SD+CFG sampler, while performing inference up to 6x faster. For AI practitioners, this provides a simple and effective technique to significantly reduce the inference cost and latency of state-of-the-art text-to-image models without compromising the output quality derived from sophisticated sampling algorithms. |
| Discovering and using Spelke segments (Read more on arXiv or HuggingFace) |
Luca Thomas Wheeler, Seungwoo Kim, Lilian Naing Chen, Klemen Kotar, Rahul Venkatesh |
This paper introduces SpelkeNet, a self-supervised visual world model for discovering motion-defined “Spelke segments” in static images, and demonstrates their utility for physical manipulation tasks. The main objective is to benchmark the concept of Spelke objects—physically coherent groupings that move together—and develop a self-supervised method to extract them from single images without explicit segmentation labels. The key methodology involves “statistical counterfactual probing” using SpelkeNet, a model based on the Local Random Access Sequence Modeling (LRAS) framework. The model is prompted with sparse “virtual pokes” (localized optical flow tokens), and it predicts a distribution over future motion fields; Spelke segments are then defined as statistical aggregates of correlated motion from multiple such probes. On the newly introduced SpelkeBench benchmark for point-prompted segmentation, SpelkeNet achieves a mean Intersection over Union (mIoU) of 0.6811, outperforming supervised baselines like SAM2 (0.6225 mIoU) and other self-supervised methods. The principal implication for AI practitioners is that motion-defined Spelke segments provide a more physically plausible and functional basis for downstream robotics and manipulation tasks compared to conventional semantic or appearance-based segments, leading to superior performance in object editing and manipulation pipelines. |
| SegDT: A Diffusion Transformer-Based Segmentation Model for Medical |
|
|
| Imaging (Read more on arXiv or HuggingFace) |
Abdenour Hadid, Fadi Dornaika, Gaby Maroun, Bekhouche |
The paper introduces SegDT, a compact Diffusion Transformer (DiT) model that uses rectified flow for efficient and accurate medical image segmentation. The objective is to develop a segmentation model for skin lesions that achieves state-of-the-art accuracy while maintaining low computational cost and fast inference speeds for deployment on resource-constrained hardware. SegDT’s methodology involves using a pretrained Tiny AutoEncoder (TAESD) to map images to a latent space, which is then processed by a DiT-XS (extra-small) model that learns a velocity field via a rectified flow objective to accelerate the reverse diffusion process. On the ISIC 2018 dataset, SegDT achieved a Dice score of 94.51% with only 3.68 GFLOPs and 9.95M parameters, outperforming heavier models like DU-Net+ (92.93% Dice, 54.00 GFLOPs). The principal implication for AI practitioners is that this architecture provides a blueprint for building high-performance segmentation models that are deployable on low-cost GPUs by significantly reducing computational load and inference steps without sacrificing accuracy. |
| Deep Learning-Based Age Estimation and Gender Deep Learning-Based Age |
|
|
| Estimation and Gender Classification for Targeted Advertisement (Read more on arXiv or HuggingFace) |
Nisar Ahmed, ImranzamanML |
This paper proposes custom Convolutional Neural Networks (CNNs) to perform age estimation and gender classification from facial images for targeted advertising applications. The stated objective is to create and evaluate a robust system for both tasks, for which the authors trained two separate CNNs from scratch on the UTK Face dataset after performing data balancing and normalization. The paper reports conflicting performance metrics: for gender classification, it claims 95% accuracy and an ROC AUC of 0.95 in the text, but the corresponding results in Table 2 show only 64% accuracy; for age estimation, a Mean Absolute Error (MAE) of 5.77 years is consistently reported in the text, although this metric is absent from its results table. The principal implication for AI practitioners is the critical need for rigorous result validation, as demonstrated by the paper’s internal inconsistencies; furthermore, the reported age estimation error (MAE of 5.77 years) highlights that facial attribute regression remains a challenging task requiring targeted data and model refinements to mitigate demographic biases. |
| Agentar-Fin-R1: Enhancing Financial Intelligence through Domain |
|
|
| Expertise, Training Efficiency, and Advanced Reasoning (Read more on arXiv or HuggingFace) |
Zhaowen Zhou, Xiaoke Zhao, Longfei Liao, Xiyang Du, Yanjun Zheng |
The paper introduces Agentar-Fin-R1, a series of 8B and 32B parameter financial large language models optimized for domain-specific expertise, training efficiency, and advanced reasoning. The primary objective is to develop a financial LLM that overcomes the limitations of general-purpose models by systematically enhancing domain-specific reasoning, ensuring trustworthiness, and improving training efficiency. The methodology integrates a structured financial task label system with a two-stage training pipeline (SFT followed by GRPO/SFT refinement), guided by a difficulty-aware weighted training framework that dynamically prioritizes tasks based on empirically measured pass@k scores. Experimental results show state-of-the-art performance, with the Agentar-Fin-R1-32B model achieving an overall score of 83.13 and specifically scoring 69.93 on the newly introduced Finova agent benchmark, outperforming both general-purpose and other specialized financial models. The principal implication for AI practitioners is the demonstrated data efficiency of the label-guided, difficulty-aware weighted training framework, which can achieve superior performance to full-data vanilla SFT while using only 50% of the training samples, providing an efficient method for domain specialization without catastrophic forgetting. |
Papers for 2025-07-24
| Title |
Authors |
Summary |
| Pixels, Patterns, but No Poetry: To See The World like Humans (Read more on arXiv or HuggingFace) |
Xinhao Li, Jingyi Tang, Lin Xu, Zihao Huang, Hongcheng Gao |
This paper introduces the Turing Eye Test (TET) benchmark to demonstrate that state-of-the-art Multimodal Large Language Models (MLLMs) have fundamental failures in human-like visual perception, distinct from their reasoning abilities. The primary objective is to evaluate whether current MLLMs can perceive the world as humans do by shifting the focus from reasoning-heavy benchmarks to tasks requiring intuitive visual perception. The authors created the Turing Eye Test (TET), a benchmark with four synthetic image tasks (HiddenText, 3DCaptcha, ColorBlind, ChineseLigatures) that are simple for humans but designed to challenge MLLM perception, and analyzed failures using Grad-CAM and selective supervised fine-tuning of model components. The study reveals catastrophic failures, with most of the 15 tested MLLMs achieving near-zero performance; for instance, on the HiddenText and 3DCaptcha tasks, nearly all models scored 0% on the Pass@1 metric, while fine-tuning only the vision encoder boosted accuracy from 0% to over 86% on HiddenText for Qwen2.5-VL-7B. The principal implication for AI practitioners is that overcoming these perceptual deficits requires fundamentally enhancing the vision encoder’s generalization capabilities, as current models and fine-tuning strategies focused on the language backbone are ineffective for these tasks. |
| Yume: An Interactive World Generation Model (Read more on arXiv or HuggingFace) |
Zhen Li, Shaoheng Lin, Xiaofeng Mao, kpzhang, Jiangmiao |
i) A 1-line summary: Yume is an interactive world generation model that synthesizes an infinitely explorable, dynamic world from a single image, controlled by keyboard inputs. ii) Main research question or objective: The primary objective is to develop a high-fidelity, interactive video generation framework that allows users to explore a dynamic world created from a static image by translating discrete keyboard actions into controllable camera motions. iii) Key methodology used: The methodology integrates a Masked Video Diffusion Transformer (MVDT) for autoregressive generation with a Quantized Camera Motion (QCM) module that converts keyboard inputs into textual conditions, and employs advanced samplers like the training-free Anti-Artifact Mechanism (AAM) and Time-Travel SDE (TTS-SDE) to enhance visual quality. iv) Primary results (include at least one specific quantitative finding): In comparative evaluations on the Yume-Bench benchmark, the model demonstrated superior controllability, achieving an instruction-following score of 0.657, significantly outperforming prior models like Wan-2.1 (0.057) and MatrixGame (0.271). v) Principal implication for AI practitioners: AI practitioners can use the Quantized Camera Motion (QCM) technique to implement intuitive, text-based camera control in video diffusion models without architectural changes, providing a practical method for creating interactive generative experiences. The most impactful finding is the high degree of user control achieved by converting discrete keyboard inputs into textual prompts, directly relevant for developing controllable AI-driven simulations and virtual environments. |
| DesignLab: Designing Slides Through Iterative Detection and Correction (Read more on arXiv or HuggingFace) |
Shingo Takamatsu, Jaegul Choo, Yotaro Shimose, Heng Wang, YeolJoo |
DesignLab is an iterative framework that refines presentation slides by using two specialized LLMs, a reviewer to detect design flaws and a contributor to correct them. The main objective is to create an automated system that models the real-world iterative design process to progressively refine rough presentation drafts into polished slides, overcoming the limitations of single-step generation methods. The methodology involves decomposing the design process into two roles, a “design reviewer” and a “design contributor,” implemented by fine-tuning separate Qwen2.5-1.5B models on a JSON representation of slides; training data is generated by applying controlled perturbations to polished slides to simulate rough drafts. In a GPT-4o preference evaluation, DesignLab was chosen over the commercial PowerPoint Designer in 51.9% of cases and over the agent-based AutoPresent in 72.7% of cases. The principal implication for AI practitioners is that decomposing a complex generative task into an iterative cycle of explicit detection and correction, trained on synthetically imperfect data, provides a powerful and generalizable framework for refinement tasks, particularly when paired draft-to-final training data is unavailable. |
| Can One Domain Help Others? A Data-Centric Study on Multi-Domain |
|
|
| Reasoning via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Conghui He, Honglin Lin, Zhuoshi Pan, blue01223, yu0226 |
This study systematically investigates multi-domain reasoning in large language models using reinforcement learning, analyzing the effects of data combinations, training strategies, and reward design across math, code, and puzzle domains. The primary objective is to understand the interplay, including synergistic and conflicting effects, among different reasoning skills (math, code, puzzles) when training LLMs with Reinforcement Learning with Verifiable Rewards (RLVR), and to identify factors that optimize multi-domain performance. The study employs the Group Relative Policy Optimization (GRPO) algorithm on Qwen-2.5-7B models, using domain-specific datasets to evaluate single-domain, dual-domain, and triple-domain training configurations against benchmarks like MATH500 and HumanEval. The paper finds that combining data from all three domains (Math, Code, Puzzle) achieves the highest overall average performance and improves task balance, mitigating the catastrophic forgetting observed in dual-domain settings, such as a 22.56 point drop in code performance when combining only Math and Puzzle data. The principal implication for AI practitioners is that training on a diverse, multi-domain dataset is crucial for building robust, generalized models that avoid catastrophic forgetting, even though this may slightly reduce peak performance on a single specialized task. Careful data mixture design and consistent use of training/evaluation templates are critical for reliable outcomes. |
| Re:Form – Reducing Human Priors in Scalable Formal Software |
|
|
| Verification with RL in LLMs: A Preliminary Study on Dafny (Read more on arXiv or HuggingFace) |
Xin Li, Xu Xu, Xuhan Huang, Fengdi Che, Chuanhao Yan |
The Re:Form framework trains LLMs for formal software verification in Dafny by using Reinforcement Learning with automated feedback from the language’s verifier, thereby reducing the need for human-annotated data and chain-of-thought reasoning. The primary objective is to create a scalable pipeline for generating provably correct software specifications by enabling models to learn directly from a formal system instead of human priors. The key methodology involves an initial Supervised Fine-Tuning (SFT) stage on automatically curated data, followed by an RL phase that uses a novel “subset reward”—derived from the Dafny verifier—to guide the model toward generating logically stronger specifications. This approach enables a 14B RL-trained model to achieve a 14.0% pass@1 verification rate on the out-of-domain DafnyComp benchmark, significantly outperforming the 8.3% rate of its SFT counterpart and discovering novel specifications not seen during training. For AI practitioners, this work implies that a system’s internal verifier can provide a powerful and scalable reward signal for RL in formal domains, enabling the autonomous generation of high-quality, provably correct artifacts without extensive human supervision. |
| Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention (Read more on arXiv or HuggingFace) |
Qin Li, Hu Zhang, Yikai Wang, Zhihao Li, Yiwen Chen |
ULTRA3D introduces an efficient framework for high-fidelity 3D generation by optimizing sparse voxel modeling. The primary objective is to mitigate the severe computational inefficiency caused by the quadratic complexity of global attention mechanisms in two-stage 3D diffusion pipelines. The methodology involves a two-stage process: first, generating a coarse object layout using the compact VecSet representation, and second, refining per-voxel latent features using “Part Attention,” a localized attention mechanism that restricts computation to semantically coherent part regions. This approach achieves a 6.7x speed-up in latent generation and a 3.3x overall pipeline speed-up, with user studies showing a 68.5% preference for ULTRA3D over concurrent methods. The principal implication for AI practitioners is that leveraging geometry-aware localized attention can significantly reduce the computational cost of high-resolution 3D generation, making the production of detailed 3D assets more tractable and scalable. |
| Elevating 3D Models: High-Quality Texture and Geometry Refinement from a |
|
|
| Low-Quality Model (Read more on arXiv or HuggingFace) |
Jiyun Won, chosh1110, joohaeng, gongms, terryryu |
The paper introduces Elevate3D, a framework that iteratively refines the texture and geometry of low-quality 3D models using a novel diffusion-based method and monocular geometry prediction. The primary objective is to transform readily accessible but low-quality 3D assets into high-quality, well-aligned models by addressing the limitations of prior refinement techniques. The methodology is a view-by-view iterative process: first, it uses High-Frequency-Swapping SDEdit (HFS-SDEdit) to enhance texture by guiding a diffusion model with high-frequency details from the input; second, it leverages the refined texture to predict a detailed normal map, which is then integrated into the mesh using a regularized normal integration scheme to update the geometry. On the GSO dataset, Elevate3D quantitatively outperforms recent competitors, achieving a MUSIQ score of 66.527 compared to the next-best score of 61.667 from DreamGaussian. For AI practitioners, this framework provides an automated pipeline to significantly upgrade the quality of large-scale 3D asset datasets, making them suitable for high-fidelity graphics applications and for use as improved training data for 3D vision systems. |
| Finding Dori: Memorization in Text-to-Image Diffusion Models Is Less |
|
|
| Local Than Assumed (Read more on arXiv or HuggingFace) |
Adam Dziedzic, Kristian Kersting, Dominik Hintersdorf, lukas-struppek, antoniaaa |
This paper demonstrates that memorization in text-to-image diffusion models is a non-local phenomenon, showing that existing weight-pruning mitigations can be circumvented by adversarial inputs, and proposes a robust adversarial fine-tuning solution. The main objective is to assess the robustness of pruning-based memorization mitigation techniques and challenge the assumption that memorization is localized in the model. The authors use an adversarial optimization process to find text embeddings that can re-trigger data replication even after mitigation has been applied, and they analyze the distribution of these embeddings and their internal activation patterns. The primary result shows that while pruning methods like NeMo reduce replication similarity (SSCD score) from 0.90 to 0.33, crafted adversarial embeddings can restore the replication similarity to 0.91, proving the mitigation is not a true erasure of the memorized content. The principal implication for AI practitioners is that weight-pruning techniques are insufficient for robustly removing memorized data, and more comprehensive methods like the proposed adversarial fine-tuning are required to ensure models do not inadvertently replicate sensitive or copyrighted content. |
| RAVine: Reality-Aligned Evaluation for Agentic Search (Read more on arXiv or HuggingFace) |
Jinhua Gao, Zhi Zheng, Xiang Long, sapphirex |
This paper proposes RAVine, a comprehensive evaluation framework to assess agentic search systems by aligning with realistic user queries, enabling precise fine-grained evaluation, and analyzing the iterative search process. The research objective is to create a more realistic evaluation sandbox for agentic LLMs that addresses the misalignment between existing benchmarks and real-world search tasks, particularly regarding query complexity, evaluation granularity, and process-oriented metrics. The methodology uses a static web environment (MS MARCO V2.1) and real-world queries (TREC 2024 RAG Track), introducing an attributable nugget collection method via dynamic semantic clustering for ground truth construction and a block-level evaluation scheme to jointly measure task completeness and citation faithfulness. Experiments show current agentic LLMs have limited faithfulness; for instance, the Qwen3-32B model achieved a maximum citation recall of only 13.2%, and a significant portion of task performance relies on non-attributable internal model knowledge. The principal implication for AI practitioners is that developing robust agentic search systems requires focusing on improving intermediate process behaviors, such as information gathering and citation accuracy, as final answer quality is not solely dependent on search performance and current models are deficient in these areas. |
Papers for 2025-07-23
| Title |
Authors |
Summary |
| Beyond Context Limits: Subconscious Threads for Long-Horizon Reasoning (Read more on arXiv or HuggingFace) |
Tina Li, Nathaniel Morgan, Hongyin Luo, thejackobrien, drkylj |
This paper introduces TIM, a language model, and TIMRUN, an inference runtime, designed to enable long-horizon reasoning beyond LLM context limits by modeling tasks as recursive, prunable trees. The objective is to overcome context window, output token, and GPU memory constraints to support virtually unlimited working memory and multi-hop tool use within a single inference pass. The methodology involves training TIM to generate structured JSON representing a hierarchy of tasks and subtasks, which the TIMRUN runtime leverages to dynamically prune the KV cache of completed subtasks, thereby reusing memory and positional embeddings. Experiments show the system improves reasoning on certain tasks while significantly reducing memory load; on the AIME 2024 benchmark, accuracy increased from 40.0% to 46.7% while the system pruned 64.1% of the total KV cache. For AI practitioners, this co-designed model-runtime system provides a new architecture for building complex, memory-intensive agents that can handle long reasoning chains and tool use more efficiently than traditional multi-agent frameworks that rely on repetitive context prefilling. |
| Step-Audio 2 Technical Report (Read more on arXiv or HuggingFace) |
Chao Yan, Boyong Wu, Insects, SmailAA, petronny |
The paper presents Step-Audio 2, an end-to-end large audio language model that unifies audio understanding and generation by directly processing raw audio and outputting interleaved discrete text and audio tokens. The primary objective is to develop a model that overcomes the limitations of prior LALMs by comprehending paralinguistic cues, enabling genuine end-to-end speech conversation, and mitigating hallucinations through external tool integration. The methodology uses an architecture with a frozen audio encoder, an adapter, an LLM decoder, and an audio detokenizer, trained through a multi-stage process of pre-training, supervised fine-tuning, and reinforcement learning (PPO/GRPO), augmented with RAG and tool-calling. Evaluation results show state-of-the-art performance across multiple benchmarks, including a 3.11% average character error rate (CER) on general Chinese ASR test sets, surpassing other leading models. For AI practitioners, this work provides a robust architectural blueprint for building more natural and reliable spoken dialogue systems by generating interleaved audio-text tokens within a single model, bypassing traditional cascaded ASR-LLM-TTS pipelines. |
| MegaScience: Pushing the Frontiers of Post-Training Datasets for Science |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Pengfei Liu, SinclairWang, Vfrz |
This paper introduces MEGASCIENCE, a 1.25M-instance dataset for scientific reasoning, created by curating and combining textbook-derived data with other open-source sets to improve LLM performance on science tasks. The objective is to develop and release large-scale, high-quality, and verifiable open-source datasets to advance the scientific reasoning capabilities of LLMs, a domain the authors argue is neglected compared to math and coding. The methodology involves creating a base dataset, TEXTBOOKREASONING, by extracting 650k QA pairs from university textbooks using a pipeline with dual-standard extraction and LLM-based decontamination, then combining it with optimally selected subsets of public datasets to form MEGASCIENCE for supervised fine-tuning. Models fine-tuned on MEGASCIENCE consistently outperform official instruction-tuned versions; for instance, Qwen2.5-7B fine-tuned on MEGASCIENCE achieved a 61.01% average score across 14 benchmarks, surpassing the 58.80% of the official Qwen2.5-7B-Instruct model. For AI practitioners, this research demonstrates that targeted SFT on a high-quality, domain-specific dataset like MEGASCIENCE can yield superior scientific reasoning performance compared to relying on general-purpose instruction-tuned models, providing a direct path to create more capable “AI scientists”. |
| Upsample What Matters: Region-Adaptive Latent Sampling for Accelerated |
|
|
| Diffusion Transformers (Read more on arXiv or HuggingFace) |
Se Young Chun, Agorium, hirussell, ignow |
The paper introduces Region-Adaptive Latent Upsampling (RALU), a training-free framework that accelerates diffusion transformers by performing mixed-resolution sampling focused on spatially significant regions. The objective is to accelerate the inference of diffusion transformers along the spatial dimension, mitigating artifacts like aliasing and noise-timestep mismatches that arise from latent upsampling, without requiring model retraining. RALU employs a three-stage process: initial low-resolution denoising, selective early upsampling of artifact-prone edge regions identified via Canny edge detection, and final full-resolution refinement, using Noise-Timestep rescheduling with Distribution Matching (NT-DM) to stabilize generation across resolution changes. The method achieves up to a 7.0x speed-up on the FLUX.1-dev model with an FID score of 28.68, significantly outperforming the spatial baseline Bottleneck Sampling (FID 38.16) at a comparable acceleration level. For AI practitioners, RALU offers a practical, training-free method to significantly reduce the inference latency of large diffusion transformers, and its design allows it to be combined with temporal acceleration techniques for further performance gains. |
| Semi-off-Policy Reinforcement Learning for Vision-Language Slow-thinking |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Songyang Gao, Harold-lkk, vanilla1116, haitengzhao, shenjunhao |
This paper presents SOPHIA, a semi-off-policy reinforcement learning framework that enhances visual slow-thinking reasoning in large vision-language models (LVLMs). The objective is to overcome the constraints of on-policy RL and mitigate visual hallucination risks associated with pure off-policy RL. The key methodology involves a semi-off-policy behavior model that combines on-policy visual understanding from the trainable LVLM with off-policy reasoning from a separate language model, using propagated visual rewards to guide training. Extensive experiments show SOPHIA improves the InternVL3.0-38B model’s average pass@1 accuracy by 8.50% and achieves 49.08% on the MathVision benchmark. For AI practitioners, SOPHIA provides a scalable, automated method to improve LVLM reasoning without relying on human or closed-source annotations, serving as a superior policy initialization for further on-policy training. |
| ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent |
|
|
| Planning (Read more on arXiv or HuggingFace) |
Fu-En Yang, Yu-Chiang Frank Wang, Yueh-Hua Wu, cmhungsteve, jasper0314-huang |
ThinkAct introduces a dual-system framework for vision-language-action (VLA) tasks that improves long-horizon planning and adaptability by separating high-level reasoning from low-level control. The primary objective is to enable an embodied agent to generate explicit reasoning plans guided by environmental feedback, rather than relying on end-to-end action prediction or supervised chain-of-thought data. The methodology involves a reasoning MLLM fine-tuned with reinforcement learning (specifically, Group Relative Policy Optimization) using a novel action-aligned reward signal derived from visual goal completion and trajectory consistency (measured via DTW), which generates a compact visual plan latent to condition a downstream Diffusion Policy action model. On the LIBERO manipulation benchmark, ThinkAct achieves an 84.4% overall success rate, outperforming previous state-of-the-art models and demonstrating effective long-horizon planning and self-correction capabilities. For AI practitioners, the key implication is that decoupling reasoning and action into two asynchronously operating modules—where a reasoning module is optimized via RL on task-grounded visual rewards to guide a separate policy—offers a scalable and robust approach to building agents that can handle complex, multi-step tasks in dynamic environments. |
| Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning (Read more on arXiv or HuggingFace) |
Zikui Cai, Kaiyu Yue, deqing, charleslwang, leonli66 |
ZEBRA-COT is a new, large-scale dataset of 182,384 samples with interleaved text-image reasoning traces designed to train vision-language models for Visual Chain of Thought (visual CoT). The primary objective is to address the lack of high-quality training data for visual CoT by creating a diverse dataset that enables VLMs to generate explicit, logically coherent visual aids as part of their reasoning process across scientific, 2D/3D, and strategic domains. The methodology involves curating the dataset by sourcing real-world and synthetic problems and then using VLMs (Gemini-2.5, GPT-4.1) to enhance raw data into structured, high-quality reasoning traces, which are then used to fine-tune existing VLM backbones like Anole-7B and Bagel-7B. Fine-tuning the Anole-7B model on ZEBRA-COT yielded up to a 13.3-point absolute performance gain on the VisuLogic benchmark, and a fine-tuned Bagel-7B model acquired the novel capability to inherently generate interleaved visual reasoning steps, which it previously could not. The principal implication for AI practitioners is that ZEBRA-COT provides a foundational dataset and open-source models for building and evaluating systems with innate visual reasoning, offering a strong initialization point for subsequent fine-tuning with reinforcement learning to improve logical consistency in visual thought processes. |
| HOComp: Interaction-Aware Human-Object Composition (Read more on arXiv or HuggingFace) |
Rynson W. H. Lau, Jinyuan Jia, Dong Liang, LeoLau |
HOComp is a novel framework for human-object image composition that generates realistic interactions while preserving subject and object identity. The objective is to composite a foreground object onto a human-centric background image, ensuring the resulting human-object interaction is harmonious and plausible, while simultaneously maintaining the visual consistency of both the original person and the inserted object. The method employs a Diffusion Transformer (DiT) backbone guided by MLLMs-driven Region-based Pose Guidance (MRPG), which uses a multimodal large language model to define the interaction type and applies a localized pose loss, and Detail-Consistent Appearance Preservation (DCAP), which combines shape-aware attention modulation, a multi-view appearance loss for the object, and a background consistency loss to preserve identities. On the authors’ proposed IHOC dataset, HOComp significantly outperforms nine state-of-the-art methods, achieving an HOI-Score of 87.39 compared to the next-best score of 75.22 from GPT-4o. For AI practitioners, this work provides a robust framework for controllable image synthesis in applications requiring nuanced human-object interaction like virtual try-on and advertising, demonstrating how combining MLLM-based semantic guidance with targeted, component-specific loss functions can create context-aware generative models that preserve fine-grained details. |
| Experience is the Best Teacher: Grounding VLMs for Robotics through |
|
|
| Self-Generated Memory (Read more on arXiv or HuggingFace) |
Christopher E. Mower, Changan Chen, René Zurbrügg, Kaixian Qu, Guowei Lan |
This paper introduces EXPTEACH, a framework that grounds Vision-Language Models (VLMs) to physical robots by enabling them to autonomously generate, store, and retrieve memories from real-world task experiences. The main objective is to overcome the challenge of grounding internet-trained VLMs to specific robotic embodiments by having the agent learn from its own successes and failures in a closed loop. The methodology combines a short-term memory (STM) for in-task reflection and a long-term memory (LTM) that stores summarized experiences, which are retrieved using Retrieval-Augmented Generation (RAG) to inform future planning. Across 12 real-world scenarios, grounding with LTM boosted single-trial success rates from 22% to 80%, demonstrating the framework’s effectiveness and generalizability. The principal implication for AI practitioners is that implementing a self-generative memory system allows VLMs to adapt to specific hardware and environments, creating more robust and capable robotic agents by learning directly from embodied experience rather than relying solely on pre-trained knowledge. |
| RefCritic: Training Long Chain-of-Thought Critic Models with Refinement |
|
|
| Feedback (Read more on arXiv or HuggingFace) |
Hongyu Lin, Bowen Yu, Le Yu, Hao Xiang, Qiaoyu Tang |
RefCritic is a long chain-of-thought critic model trained with a dual-reward reinforcement learning framework to generate actionable feedback that improves policy model refinement. The primary objective is to develop a critic model that moves beyond superficial solution verification to produce in-depth, actionable critiques that demonstrably improve the performance of the policy model being critiqued. The methodology involves a two-stage process: first, supervised fine-tuning (SFT) on filtered data to create a cold-start critic, followed by reinforcement learning (RL) using a dual-reward system that jointly rewards the critic for the correctness of its judgment and for the performance improvement of a policy model after it refines its solution based on the critic’s feedback. On the AIME25 benchmark, using feedback from RefCritic improved the base policy model’s Pass@1 performance by 6.8% after a single round of refinement. Additionally, RefCritic achieves an average F1 score of 77.1 on ProcessBench for identifying error locations, outperforming methods that use explicit step-level supervision during training. The principal implication for AI practitioners is that training critic models should incorporate a direct feedback loop from the downstream task; explicitly rewarding critiques based on their ability to improve a policy model’s performance is more effective for creating useful, actionable feedback than optimizing for critique accuracy alone. |
| SPAR: Scholar Paper Retrieval with LLM-based Agents for Enhanced |
|
|
| Academic Search (Read more on arXiv or HuggingFace) |
Jinxin Xie, Longbin Yu, Qian Kou, Yuduo Li, Xiaofeng Shi |
This paper introduces SPAR, a modular, multi-agent framework designed to enhance academic paper retrieval through LLM-based agents. The primary research objective is to develop a flexible and effective search system that can handle complex, multi-intent queries by mimicking human research behaviors like following citation networks. Its key methodology is a multi-agent architecture comprising five specialized agents for query interpretation, multi-source retrieval with “RefChain” citation expansion, relevance judgment, iterative query evolution, and result reranking. SPAR demonstrated significant performance gains over strong baselines, achieving a +56% F1 score improvement on the AutoScholar benchmark and +23% on the newly introduced SPARBench. For AI practitioners, the principal implication is that decomposing complex retrieval tasks into specialized, coordinated agent functions that integrate symbolic planning (RefChain) with LLM-powered query evolution offers a more robust and effective approach than monolithic or single-agent systems. |
| Does More Inference-Time Compute Really Help Robustness? (Read more on arXiv or HuggingFace) |
Chawin Sitawarin, Weichen Yu, Jiachen T. Wang, Chong Xiang, Tong Wu |
This paper demonstrates that while increasing inference-time compute can enhance LLM robustness when reasoning is hidden, it conversely reduces robustness when reasoning steps are exposed, revealing a critical security trade-off. The primary objective is to investigate how inference-time computation affects the robustness of open-source reasoning LLMs against adversarial attacks, critically examining the implicit assumption that intermediate reasoning steps are hidden. The study employs a “budget forcing” strategy to systematically control the length of reasoning chains (from 100 to 16,000 tokens) across 12 open-source models, evaluating robustness on benchmarks for prompt injection (SEP), prompt extraction (TENSORTRUST), and harmful requests (SORRY-BENCH). The primary results show that when reasoning is hidden, robustness improves with more computation (e.g., Qwen3-32B’s prompt injection robustness increases from ~35% to ~75%); however, when reasoning is exposed, an inverse scaling law emerges where robustness consistently degrades (e.g., R1-QWEN-14B’s prompt injection robustness drops from ~90% to below 20% as the budget increases). The principal implication for AI practitioners is that the benefits of inference-time scaling are context-dependent; increasing compute can introduce significant security vulnerabilities if intermediate reasoning steps are accessible, either directly, via tool-use APIs, or through extraction attacks. |
| Steering Out-of-Distribution Generalization with Concept Ablation |
|
|
| Fine-Tuning (Read more on arXiv or HuggingFace) |
Senthooran Rajamanoharan, Samuel Marks, Adam Karvonen, Caden Juang, Helena Casademunt |
i) A 1-line summary The paper introduces Concept Ablation Fine-Tuning (CAFT), a method that uses interpretability tools to steer the out-of-distribution generalization of LLMs during fine-tuning without modifying the training data. ii) Main research question or objective To develop a method for controlling how a large language model generalizes from a fine-tuning dataset to an out-of-distribution (OOD) one, particularly in a worst-case scenario where no OOD data is available to specify the intended generalization. iii) Key methodology used CAFT identifies undesired concepts as linear directions in the model’s latent space using either Principal Component Analysis (PCA) on activation differences or Sparse Autoencoders (SAEs). It then fine-tunes the model while continuously ablating (projecting away) these undesired directions from the model’s activations during both the forward and backward passes. iv) Primary results (include at least one specific quantitative finding) On an emergent misalignment task where fine-tuning on insecure code causes harmful general responses, CAFT with PCA reduced misaligned responses from 7.0% to 0.39% for the Qwen model—a reduction of over 10x—with only a minor degradation in performance on the original insecure code task. In multiple-choice tasks with spurious correlations, CAFT with SAEs often improved OOD accuracy from near 0% to over 50%, and in some cases near 100%. v) Principal implication for AI practitioners CAFT provides a practical technique for AI engineers to mitigate unintended and potentially harmful behaviors that emerge during fine-tuning, especially when it is infeasible to curate specific training data to prevent such generalizations. It demonstrates a direct application of interpretability tools within the training process to improve model safety and control. |
| ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via |
|
|
| Gaussian Splatting (Read more on arXiv or HuggingFace) |
Yixuan Li, Lihan Jiang, Linning Xu, Mulin Yu, Ruijie Zhu |
ObjectGS is a framework that unifies high-fidelity 3D scene reconstruction and semantic understanding by modeling individual objects with dedicated, ID-aware Gaussian primitives. The main objective is to overcome the lack of semantic understanding in standard 3D Gaussian Splatting by developing a method that jointly performs object-level reconstruction and segmentation. The key methodology involves initializing object-aware anchors from 2D segmented masks, assigning a fixed one-hot ID encoding to each generated Gaussian based on its object affiliation, and optimizing with a classification loss to enforce discrete semantic boundaries during rendering. On the 3DOVS open-vocabulary segmentation benchmark, ObjectGS achieved a mean IoU of 96.4%, outperforming prior methods like Gaussian Grouping (89.1%). The principal implication for AI practitioners is that using discrete one-hot ID encoding with a classification loss offers a robust and unambiguous method for embedding semantic data into Gaussian Splatting models, enabling cleaner object extraction and direct object-level manipulation for applications like scene editing and robotics. |
Papers for 2025-07-22
| Title |
Authors |
Summary |
| MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via |
|
|
| Context-Aware Multi-Stage Policy Optimization (Read more on arXiv or HuggingFace) |
Yao Xiao, LidongBing, ZonglinY, binwang, veggiebird |
This paper introduces MiroMind-M1, a fully open-source series of mathematical reasoning models, and CAMPO, a novel reinforcement learning algorithm that enhances performance and token efficiency. The primary objective is to develop a transparent and reproducible pipeline for creating high-performance mathematical reasoning language models by open-sourcing the models, curated datasets, and the complete training framework. The methodology is a two-stage process: first, supervised fine-tuning (SFT) on a 719K curated dataset of math problems with verified chain-of-thought, followed by reinforcement learning using the novel Context-Aware Multi-Stage Policy Optimization (CAMPO) algorithm, which integrates length-progressive training with an adaptive repetition penalty. The primary result is that the MiroMind-M1-RL-7B model improves upon its SFT-only counterpart by 13 absolute points on the AIME24 benchmark (from 60.4 to 73.4 avg@64), and the models demonstrate superior token efficiency compared to baselines. The principal implication for AI practitioners is that the fully released stack—including models, datasets, and the CAMPO algorithm—provides a concrete, reproducible methodology for fine-tuning language models for complex reasoning tasks, offering a practical approach to improve both accuracy and computational efficiency. |
| GUI-G^2: Gaussian Reward Modeling for GUI Grounding (Read more on arXiv or HuggingFace) |
Xuyang Liu, Zhangxuan Gu, Fei Tang, tricktreat, LZXzju |
This paper introduces GUI-G², a reward modeling framework that replaces sparse binary rewards with continuous Gaussian distributions for GUI grounding tasks in reinforcement learning. The primary objective is to create a dense, geometrically-aware reward signal that models the continuous nature of spatial interactions, addressing the inefficiency of discrete hit-or-miss feedback. The methodology models GUI elements as 2D Gaussian distributions and computes a dual reward: a Gaussian point reward for localization precision and a Gaussian coverage reward for spatial overlap, combined with an adaptive variance mechanism to handle different element sizes. The proposed GUI-G²-7B model achieves state-of-the-art results, including a 24.7% accuracy improvement over the UI-TARS-72B model on the ScreenSpot-Pro benchmark and 93.3% accuracy on ScreenSpot-v2. The principal implication for AI practitioners is that using this continuous, dual-component Gaussian reward function can lead to more efficient training and superior performance for GUI agents, enabling smaller models to outperform larger ones by providing richer gradient signals for spatial optimization. |
| The Invisible Leash: Why RLVR May Not Escape Its Origin (Read more on arXiv or HuggingFace) |
Yejin Choi, Zaid Harchaoui, Ximing Lu, Fang Wu, weihao1115 |
This paper theoretically and empirically demonstrates that Reinforcement Learning with Verifiable Rewards (RLVR) primarily sharpens a base model’s existing knowledge rather than discovering new reasoning paths, acting as a conservative reweighting mechanism constrained by the initial model’s support. The main research question is whether RLVR fundamentally expands a large language model’s reasoning capabilities or merely amplifies high-reward outputs already within the base model’s support, potentially at the cost of solution diversity. The study employs a theoretical analysis using support-preservation theorems and a variational inference perspective to formalize RLVR’s limits. This is validated empirically by analyzing “empirical support dynamics” (preservation, shrinkage, expansion) and entropy changes (token-level vs. answer-level) across various math and non-math reasoning benchmarks. The primary results show that while RLVR consistently improves pass@1 accuracy, empirical support shrinkage generally outweighs expansion. For instance, across the Minerva and OlympiadBench datasets combined, the RLVR model discovered only 3 new correct solutions while losing access to 48 solutions that were discoverable by the base model. Furthermore, answer-level entropy consistently decreases, indicating convergence to a smaller set of final answers, even when token-level uncertainty increases. The principal implication for AI practitioners is that RLVR should not be expected to spontaneously discover reasoning abilities beyond the base model’s initial representational capacity. To achieve true capability expansion, RLVR pipelines must be augmented with explicit exploration mechanisms or off-policy data to seed probability mass into underrepresented solution regions. |
| WebShaper: Agentically Data Synthesizing via Information-Seeking |
|
|
| Formalization (Read more on arXiv or HuggingFace) |
Baixuan Li, Junkai Zhang, Wenbiao Yin, Jialong Wu, Zhengwei Tao |
WebShaper is a formalization-driven framework that agentically synthesizes high-quality training data for information-seeking (IS) agents. Its primary objective is to overcome data scarcity and inconsistency by creating a systematic, controllable data synthesis method that avoids the limitations of traditional information-driven approaches. The key methodology involves formalizing IS tasks using set-theoretic “Knowledge Projections” (KP) and employing an agentic “Expander” that iteratively complicates seed questions via a layer-wise expansion strategy to ensure structural complexity. The method achieves state-of-the-art performance among open-source models, with the WebShaper-72B model scoring 60.1% Pass@1 on the GAIA benchmark. For AI practitioners, this framework provides a principled way to generate diverse and complex training data, enabling the development of agents with more robust and advanced multi-hop reasoning capabilities. |
| Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with |
|
|
| Regularized Score Distillation Sampling (Read more on arXiv or HuggingFace) |
Se Young Chun, jeeit17, yeonE |
The paper introduces RoMaP, a framework for robust, part-level editing of 3D Gaussian Splatting scenes that enables precise and drastic modifications via geometry-aware masking and a regularized score distillation sampling loss. The primary objective is to overcome the inability of existing methods to perform localized and drastic part-level edits on 3D Gaussian Splatting representations, which is caused by inconsistent segmentation and restrictive diffusion model priors. The methodology integrates a 3D-Geometry Aware Label Prediction (3D-GALP) module for generating consistent 3D part masks and a regularized Score Distillation Sampling (SDS) loss guided by a novel Scheduled Latent Mixing and Part (SLaMP) 2D editing technique. Experimental results show RoMaP significantly outperforms state-of-the-art methods, achieving a CLIP directional similarity score of 0.205, more than doubling the 0.095 score of the next-best baseline, and a B-VQA score of 0.723 versus the baseline’s 0.497. For AI practitioners, RoMaP provides a powerful tool for fine-grained, text-guided control over specific parts of 3D assets, enabling complex and unconventional edits for applications in virtual reality, gaming, and digital asset creation. |
| SeC: Advancing Complex Video Object Segmentation via Progressive Concept |
|
|
| Construction (Read more on arXiv or HuggingFace) |
Jianfan Lin, Songxin He, Xiaoyi Dong, Shuangrui Ding, rookiexiong |
The paper introduces Segment Concept (SeC), a framework that advances Video Object Segmentation by using Large Vision-Language Models (LVLMs) for progressive, concept-level object representation. The objective is to overcome the limitations of appearance-based VOS models by developing a system that constructs a high-level, object-centric concept to maintain tracking through drastic visual variations and scene changes. SeC’s methodology involves employing an LVLM to integrate visual cues from a dynamically updated bank of keyframes, and a scene-adaptive activation strategy selectively fuses this conceptual guidance with pixel-level memory features only during significant scene changes. On the newly introduced SeCVOS benchmark, designed to test high-level reasoning, SeC achieves an 11.8-point J&F score improvement over the SAM 2.1 baseline. The principal implication for AI practitioners is that integrating high-level semantic reasoning from LVLMs with traditional pixel-level feature matching provides a robust and computationally efficient mechanism to handle complex, multi-shot video scenarios where object appearance and context change drastically. |
| GR-3 Technical Report (Read more on arXiv or HuggingFace) |
Yingdong Hu, Zhongren Cui, Chilam Cheang, melony, CH3COOK |
This paper details GR-3, a 4B parameter vision-language-action (VLA) model for generalist robot control. The research objective is to create a robot policy that generalizes to novel objects and abstract instructions, learns efficiently from minimal data, and robustly performs long-horizon, dexterous tasks. The core methodology involves co-training a pre-trained vision-language model on both web-scale vision-language data and robot trajectories using a flow-matching objective, which is further fine-tuned with small amounts of human trajectory data from VR. Key results demonstrate that with only 10 human trajectories per object, GR-3 boosts its success rate on unseen objects from 57.8% to 86.7% and achieves a 97.5% success rate on a complex, long-horizon table bussing task. The principal implication for practitioners is that this multi-faceted training recipe offers a data-efficient and cost-effective strategy for developing and adapting generalist robot policies for novel, real-world applications, reducing the dependency on large-scale, robot-specific data collection. |
| Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for |
|
|
| RLVR (Read more on arXiv or HuggingFace) |
Guorui Zhou, Xiu Li, Fuzheng Zhang, Jiakang Wang, RyanLiu112 |
The paper introduces Archer, an RLVR method that improves LLM reasoning by applying differentiated, synchronous optimization constraints to knowledge-related and reasoning-related tokens identified via response-level entropy. The primary objective is to improve Reinforcement Learning with Verifiable Rewards (RLVR) by treating tokens differently based on their function (knowledge vs. reasoning) without disrupting semantic dependencies, thereby stabilizing factual knowledge while promoting reasoning exploration. The key methodology involves classifying tokens within each response as either “knowledge-related” (low-entropy) or “reasoning-related” (high-entropy) using a response-level entropy quantile threshold, then applying stronger KL regularization and a lower clipping threshold to knowledge tokens and weaker KL regularization with a higher clipping threshold to reasoning tokens during synchronous policy updates. Archer outperforms existing 1.5B-level models, with Archer-Math-1.5B achieving a 48.7% avg@64 accuracy on AIME24, a notable improvement over the 42.1% from the DAPO baseline. For AI practitioners, the principal implication is that during RL fine-tuning for reasoning, implementing a dual-constraint system to moderately update knowledge-centric tokens while aggressively updating reasoning-centric tokens is more effective for balancing stability and performance than applying uniform updates, masking tokens, or using asynchronous methods. |
| Being-H0: Vision-Language-Action Pretraining from Large-Scale Human |
|
|
| Videos (Read more on arXiv or HuggingFace) |
Sipeng Zheng, Yicheng Feng, Hao Luo, Yaya041, zawnpn |
This paper presents Being-H0, a Vision-Language-Action (VLA) model that learns dexterous manipulation skills by pretraining on a large-scale human video dataset (UniHand) for transfer to robotic systems. The research objective is to investigate if a VLA can be pretrained on large-scale human videos to explicitly imitate human actions and then be adapted to control robot hands, thus overcoming the data bottleneck of teleoperated demonstrations. The key methodology is “physical instruction tuning,” a paradigm comprising VLA pretraining on the curated UniHand dataset, physical space alignment to unify heterogeneous video sources, and a part-level motion tokenization technique using Grouped Residual Quantization (GRQ) to discretize hand motions with millimeter-level precision. In real-world dexterous manipulation, Being-H0 achieved a 100% success rate on the “Pour-Cup” task, significantly outperforming a baseline without human-video pretraining (55% success rate), and demonstrated high data efficiency by matching the baseline’s performance on the “Close-Toolbox” task while using only 50% of the teleoperation data. The principal implication for AI practitioners is that pretraining on large-scale human videos with explicit motion modeling offers a highly sample-efficient pathway for developing capable dexterous robot policies, substantially reducing the need for costly and time-consuming collection of real-world robot demonstration data. |
| Gaussian Splatting with Discretized SDF for Relightable Assets (Read more on arXiv or HuggingFace) |
Beibei Wang, Jian Yang, Zuo-Liang Zhu |
This paper introduces a discretized Signed Distance Field (SDF) integrated directly into 3D Gaussian primitives to regularize geometry for high-quality, relightable asset creation. The main objective is to effectively regularize the geometry of 3D Gaussian Splatting for inverse rendering, improving the decomposition of material and lighting without incurring the high memory and computational costs of using an auxiliary continuous SDF network. The key methodology involves encoding a discrete SDF value as an attribute within each Gaussian, which is then linked to opacity via a learnable SDF-to-opacity transformation; geometric consistency is enforced using a novel projection-based consistency loss that aligns projected Gaussians with the rendered surface depth, approximating the Eikonal constraint. On the Glossy Blender dataset, the proposed method achieves a state-of-the-art mean PSNR of 24.52, outperforming the next-best method (23.39), while requiring significantly less memory (4G vs. 22G). The principal implication for AI practitioners is the ability to create high-fidelity, relightable 3D assets with a more memory-efficient and simpler framework, as this approach unifies the SDF and Gaussian representations and eliminates the need for a separate, resource-intensive neural network for geometric regularization. |
| NoHumansRequired: Autonomous High-Quality Image Editing Triplet Mining (Read more on arXiv or HuggingFace) |
Bulat Suleimanov, Georgii Fedorov, Grigorii Alekseenko, Maksim Kuprashevich, iitolstykh |
This paper presents a fully automated pipeline for mining high-quality image editing triplets without human intervention, yielding a new state-of-the-art dataset and model. The main objective is to develop a scalable, autonomous framework to generate high-fidelity training data (original image, instruction, edited image) for instruction-based image editing, overcoming the manual annotation bottleneck. The methodology uses a T2I model (FLUX.1-schnell) to generate source images and an LLM (OpenAI o3) to create edit instructions, then applies edits using a base editor. These candidates are filtered by a coarse model (Qwen-72B) and then rigorously validated by a task-specific, fine-tuned Gemini 2.0 Flash validator that scores instruction adherence and aesthetics. The dataset is augmented via semantic inversion and compositional bootstrapping. The primary result is the creation of the NHR-Edit dataset (358k triplets), which achieves a geometric mean quality score of 4.53, significantly outperforming the next-best public dataset (OmniEdit at 4.23). A model fine-tuned on this data, Bagel-NHR-Edit, demonstrates improved performance on public benchmarks. The principal implication for AI practitioners is the availability of a framework and a large-scale, high-quality dataset (NHR-Edit) for training more capable instruction-guided image editors. The automated pipeline allows for continuous model improvement and targeted weakness correction without the cost and time of human labeling. |
| Towards Video Thinking Test: A Holistic Benchmark for Advanced Video |
|
|
| Reasoning and Understanding (Read more on arXiv or HuggingFace) |
Bo Hu, Aria Leo, Yuhao Dong, yunicechew, ZhangYuanhan |
This paper introduces the Video Thinking Test (Video-TT), a benchmark designed to evaluate video LLMs on complex visual narrative understanding and robustness against natural adversarial questions. The main objective is to assess the genuine gap between video LLM and human performance by creating questions that test comprehension rather than frame sampling capabilities. The methodology involves a dataset of 1,000 videos, each with one primary open-ended question and four adversarial variants (rephrased, correctly-led, wrongly-led, multi-choice) designed around eight specific visual and narrative complexity factors, ensuring all are answerable from 80 sampled frames. Results show a significant performance deficit, with the top-performing model (GPT-4o) achieving 36.6% accuracy and 36.0% robustness, far below the human baseline of 84.3% accuracy and 64.3% robustness. The principal implication for AI practitioners is that current video LLMs have fundamental weaknesses in spatial-temporal reasoning, world knowledge integration, and linking disparate video events into a coherent narrative, indicating that future work must address these core reasoning and comprehension failures. |
| Inverse Scaling in Test-Time Compute (Read more on arXiv or HuggingFace) |
Jacob Goldman-Wetzler, Andy Arditi, Runjin Chen, Alexander Hägele, Aryo Pradipta Gema |
This research demonstrates that increasing test-time compute for Large Reasoning Models (LRMs) can paradoxically degrade performance, a phenomenon termed “inverse scaling in test-time compute.” The study’s objective is to construct and analyze evaluation tasks where LRM performance deteriorates with extended reasoning, in order to identify and categorize the underlying failure modes. The authors developed a suite of novel tasks spanning counting with distractors, regression with spurious features, and deduction with constraints, evaluating models like the Claude and OpenAI o-series under “controlled” and “natural” overthinking setups where reasoning length is systematically varied. The study identifies five failure modes, including increased distractibility and amplification of spurious correlations; for instance, on the “Survival Instinct” task, increasing the reasoning budget caused Claude Sonnet 4’s expression of safety-aligned willingness to be turned off to drop from 60% to 47%. Practitioners must recognize that naively scaling test-time compute is not a universally reliable method for improving model performance and can amplify latent, problematic reasoning patterns; therefore, evaluation protocols must stress-test models across a full spectrum of reasoning lengths to ensure robustness and safety. |
| STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for |
|
|
| Spoken Language Models (Read more on arXiv or HuggingFace) |
Kevin Lin, Chung-Ching Lin, Linjie Li, xiaofei-wang, dcml0714 |
The paper introduces STITCH, a method for Spoken Language Models to perform chunked reasoning concurrently with speech generation, improving reasoning ability without increasing response latency. The main research objective is to enable Spoken Language Models (SLMs) to perform an internal, unspoken reasoning process to improve response quality on complex tasks without incurring the significant latency of generating a full chain-of-thought before speaking. The key methodology, STITCH, interleaves the generation of unspoken reasoning token chunks with spoken response chunks. It utilizes the audio playback duration of a spoken chunk to compute the subsequent reasoning chunk, thereby achieving simultaneous thinking and talking. A variant, STITCH-S, begins with a spoken response chunk to eliminate any initial reasoning-induced latency. The primary result is that the STITCH-S model matches the initial response latency of baselines that lack reasoning capabilities, while outperforming them by over 15% on math reasoning datasets (from 62.98% to 78.04% average accuracy). This performance is achieved with only a ~1% accuracy drop compared to a high-latency model that generates the full reasoning trace before speaking. The principal implication for AI practitioners is that they can enhance the reasoning abilities of real-time conversational agents by fine-tuning SLMs on an interleaved reasoning-speech generation format, effectively hiding the computational latency of thinking within the time the user spends listening to the audio response. |
| Streaming 4D Visual Geometry Transformer (Read more on arXiv or HuggingFace) |
Jie Zhou, Yuqi Wu, Wenzhao Zheng, lch01, paryi |
StreamVGGT is a causal transformer architecture designed for efficient, real-time 4D visual geometry reconstruction from streaming video inputs. The primary objective is to overcome the high latency of offline models by processing video frame-by-frame, enabling on-the-fly scene updates for interactive applications. The key methodology involves replacing global self-attention with temporal causal attention and using a cached token memory to incrementally integrate historical information during inference, while knowledge distillation from a powerful offline teacher model (VGGT) is used during training to mitigate error accumulation. The model achieves a 31x inference speedup, processing the final frame of a 40-frame sequence in 67 ms compared to 2089 ms for the offline VGGT, while maintaining competitive reconstruction accuracy. For AI practitioners, this work provides a practical architecture for converting large, offline vision transformers into efficient streaming models suitable for low-latency applications like robotics and AR/VR, demonstrating that causal attention with key-value caching can achieve real-time performance with minimal accuracy degradation. |
| Latent Denoising Makes Good Visual Tokenizers (Read more on arXiv or HuggingFace) |
Yue Wang, Yonglong Tian, Lijie Fan, Tianhong Li, Jiawei Yang |
This paper introduces the Latent Denoising Tokenizer (l-DeTok) to determine properties that make visual tokenizers more effective by aligning their training with the denoising objectives of downstream generative models. The key methodology trains a Vision Transformer-based autoencoder to reconstruct clean images from latent embeddings that are intentionally corrupted using interpolative Gaussian noise and random patch masking. On ImageNet 256x256, l-DeTok demonstrates significant and generalizable performance gains across six different generative models, improving the FID score for the MAR-B model from 2.31 to 1.55. The principal implication for AI practitioners is that explicitly incorporating a latent denoising objective into tokenizer training is a highly effective, task-aligned strategy to improve generative performance without modifying the generator architecture or relying on semantic distillation from large external models. |
| LLM Economist: Large Population Models and Mechanism Design in |
|
|
| Multi-Agent Generative Simulacra (Read more on arXiv or HuggingFace) |
Yu Bai, Samuel Kleiner, Zihan Ding, Wenzhe Li, milkkarten |
The paper presents LLM Economist, a hierarchical agent-based framework where LLM agents, guided by in-context reinforcement learning, simulate and design economic tax policies. The main objective is to determine if a multi-agent system, operating purely through natural language, can effectively model, simulate, and optimize a complex economic mechanism like taxation by framing it as a two-level Stackelberg game. The methodology involves a hierarchical simulation where a “planner” agent uses in-context reinforcement learning (ICRL) to propose tax schedules, while a population of “worker” agents, with personas calibrated to U.S. Census data, optimize their individual text-based utility functions by choosing labor supply. The framework’s planner agent designed policies that significantly improved social welfare (SWF) over baselines; in a seven-bracket simulation, the LLM policy increased SWF by 93% over the U.S. federal schedule, approaching the 114% gain from an analytically-informed, perturbed Saez solution. The principal implication for AI practitioners is that LLMs can function as powerful zero-shot optimizers for complex mechanism design in multi-agent systems directly via ICRL, providing an interpretable, language-driven alternative to traditional deep RL for building and auditing sophisticated socio-technical systems. |
| Data Mixing Agent: Learning to Re-weight Domains for Continual |
|
|
| Pre-training (Read more on arXiv or HuggingFace) |
Yeyun Gong, Hao Li, Lei Ji, lx865712528, klyang |
This paper introduces the Data Mixing Agent, a model-based framework that learns to dynamically re-weight data domains for the continual pre-training of large language models. The main objective is to automate the process of finding an optimal data mixing strategy to improve performance on a target task while mitigating catastrophic forgetting of source capabilities. The methodology involves framing domain re-weighting as a Markov Decision Process (MDP) and training a lightweight Transformer-based agent using offline reinforcement learning (Conservative Q-Learning) on data from thousands of sampled trajectories and their performance feedback. The Data Mixing Agent significantly outperformed the RegMix baseline, achieving an average improvement of 3.02% across 12 general and math benchmarks on a LLaMA-3B model, and demonstrated generalization across unseen models and domains without retraining. The principal implication for AI practitioners is the availability of an automated, model-based approach to efficiently guide continual pre-training, replacing resource-intensive manual tuning or heuristic-based methods to achieve a better performance balance. |
| “PhyWorldBench”: A Comprehensive Evaluation of Physical Realism in |
|
|
| Text-to-Video Models (Read more on arXiv or HuggingFace) |
Fangrui Zhu, Ashwin Nagarajan, Yu Zeng, Xian Liu, Jing Gu |
The paper introduces PhyWorldBench, a comprehensive benchmark to evaluate the physical realism of text-to-video models, revealing significant limitations in their ability to simulate physics. The main research objective is to systematically assess the adherence of text-to-video generation models to physical laws and identify their core failure modes in simulating physical phenomena. The key methodology involves the creation of PhyWorldBench, a benchmark with 1,050 structured prompts across 10 fundamental, composite, and “Anti-Physics” categories. The authors evaluated 12 state-of-the-art models by generating 12,600 videos, assessing them via human evaluation and a proposed MLLM-based method called Context-Aware Prompt (CAP) on metrics of Semantic Adherence (SA) and Physical Commonsense (PC). The primary result is that even top-performing models struggle significantly with physical realism; Pika 2.0, the best-performing model, achieved an overall success rate (satisfying both SA and PC) of only 0.262. Models demonstrated a notable inability to follow “Anti-Physics” prompts, indicating a tendency to reproduce learned real-world patterns rather than adhere to explicit instructions that violate them. The principal implication for AI practitioners is that current text-to-video models lack a robust understanding of physics, and improving this requires more than just scaling or detailed narrative prompts. Explicitly integrating physical phenomena into prompts is a more effective strategy for improving physical accuracy, guiding prompt engineering efforts for more realistic video generation. |
| A Simple “Try Again” Can Elicit Multi-Turn LLM Reasoning (Read more on arXiv or HuggingFace) |
Yiping Lu, Chenwei Xu, Linjie Li, Zihan Wang, Licheng Liu |
This paper introduces Unary Feedback as Observation (UFO), a multi-turn reinforcement learning framework that uses minimal “try again” feedback to improve an LLM’s iterative reasoning and prevent repetitive responses. The primary objective is to determine if Large Reasoning Models (LRMs) can learn to reflect on and revise their answers in a multi-turn context using only minimal, unary feedback, thereby overcoming the repetitive behavior induced by single-turn RL training. The key methodology is Unary Feedback as Observation (UFO), which formulates multi-turn problem-solving as a Markov Decision Process (MDP) where an incorrect model response results in only a generic negative observation (e.g., “Try Again”) being added to the context history. The model is trained using Proximal Policy Optimization (PPO) with specialized reward structures, including turn-wise reward decay and an answer repetition penalty, to encourage both efficiency and reasoning diversity. Experimental results demonstrate that training with UFO improves multi-turn reasoning, achieving up to a 14% absolute increase in success rate (Succ@5) on the MMQ-Math dataset compared to a single-turn RL baseline. This approach also shows strong generalization, improving the 5-turn success rate on the out-of-domain MMLU-Pro benchmark from 48.3% (standard RL) to 60.9%. The principal implication for AI practitioners is that they can enhance the interactive problem-solving capabilities of their models by augmenting existing single-turn RL pipelines with the UFO framework. This is a lightweight method that can be applied to static datasets without needing complex, annotated multi-turn feedback, providing a practical way to mitigate the common failure mode where models repetitively generate the same incorrect answer. |
| GeoDistill: Geometry-Guided Self-Distillation for Weakly Supervised |
|
|
| Cross-View Localization (Read more on arXiv or HuggingFace) |
Yujiao Shi, Xuming He, Alexandre Alahi, Zimin Xia, tsw200027 |
GeoDistill is a weakly supervised self-distillation framework that improves cross-view localization by using geometry-guided, Field-of-View (FoV)-based masking to learn robust local features. The primary objective is to enhance localization performance and generalization without requiring costly, precise ground-truth pose annotations. The methodology employs a teacher-student architecture where the student model processes a randomly masked, limited FoV image and is trained to match the output of the teacher model which processes the full panoramic image; the teacher is then progressively refined using an Exponential Moving Average of the student’s weights. The method significantly improves performance, reducing the mean localization error on the VIGOR Cross-Area benchmark by 25.1% for the G2SWeakly(DINO) model and outperforming the fully supervised state-of-the-art with a 2.68m mean error. For AI practitioners, the principal implication is a scalable, plug-and-play paradigm to enhance localization models using only weakly supervised data, reducing dependency on expensive, precisely annotated datasets for applications like autonomous navigation. |
| UGPL: Uncertainty-Guided Progressive Learning for Evidence-Based |
|
|
| Classification in Computed Tomography (Read more on arXiv or HuggingFace) |
Chandrakala S, Rakesh Raj Madavan, Pavan Kumar S, Shravan Venkatraman |
The paper introduces UGPL, a framework that improves CT image classification by performing a global analysis to identify uncertain regions and then a focused local analysis on those regions. The objective is to develop a classification framework that improves performance on medical images by mimicking the diagnostic process of global examination followed by focused analysis on ambiguous areas, thereby overcoming the limitations of uniform image processing. The methodology employs evidential deep learning to generate a pixel-wise uncertainty map from a global model, which guides a non-maximum suppression algorithm to extract informative patches for a local refinement network; an adaptive fusion module then combines the global and local predictions. UGPL consistently outperforms state-of-the-art models on three CT datasets, and an ablation study demonstrates the criticality of its core mechanism, showing that uncertainty-guided patch selection yields up to a 5.3x F1 score improvement on the COVID-19 detection task compared to configurations without it. For AI tasks with localized features, especially in medical imaging, practitioners can improve model performance by adopting a progressive, uncertainty-guided pipeline that dynamically allocates computational resources, rather than relying on uniform-processing, single-pass architectures. |
Papers for 2025-07-21
| Title |
Authors |
Summary |
| A Data-Centric Framework for Addressing Phonetic and Prosodic Challenges |
|
|
| in Russian Speech Generative Models (Read more on arXiv or HuggingFace) |
Mikhail Gorodnichev, Maxim Maslov, Vasiliy Kudryavtsev, Nikita Vasiliev, Kirill Borodin |
This paper introduces Balalaika, a new 2,000+ hour Russian speech dataset with comprehensive linguistic annotations, to improve generative speech models. The primary objective is to address specific phonetic and prosodic challenges in Russian text-to-speech (TTS), such as variable stress, vowel reduction, and homograph ambiguity. The methodology consists of an automated pipeline that collects studio-quality speech and applies state-of-the-art models for quality filtering (NISQA-S), transcription (GigaAMv2-RNNT), speaker clustering, and importantly, adding explicit stress and punctuation annotations. In experiments, a VITS model trained on the highest-quality partition of Balalaika achieved a manual Mean Opinion Score (MOS) of 3.618 ± 0.083, outperforming models trained on all 11 other compared datasets. The principal implication for AI practitioners is that for morphologically complex languages, a data-centric approach focused on high-quality audio combined with rich linguistic annotations (especially stress) is more effective for developing state-of-the-art generative models than using larger but less-curated corpora. |
| The Devil behind the mask: An emergent safety vulnerability of Diffusion |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Ruixi Wu, Zhiyuan Liu, Dongrui Liu, Zichen Wen, Joshua999 |
This paper introduces DIJA, a novel jailbreak attack that exploits the inherent architectural properties of diffusion LLMs—bidirectional context modeling and parallel decoding—to bypass safety alignments. The objective is to systematically investigate the emergent safety vulnerabilities of diffusion-based large language models (dLLMs) and develop an automated attack framework, DIJA, that exploits these unique weaknesses. The DIJA framework automatically transforms standard harmful prompts into adversarial interleaved mask-text prompts using in-context learning, forcing the target dLLM to generate harmful content within masked spans to maintain contextual consistency while its parallel decoding architecture prevents dynamic refusal. DIJA significantly outperforms existing jailbreak methods, achieving an evaluator-based Attack Success Rate (ASR) on JailbreakBench that surpasses the strongest prior baseline by up to 78.5% and achieving a 37.7 point higher StrongREJECT score against the Dream-Instruct model. The principal implication for AI practitioners is that the core mechanisms of dLLMs create a new attack surface not addressed by safety alignments designed for autoregressive models, necessitating the development of novel, dLLM-specific alignment techniques. |
| Franca: Nested Matryoshka Clustering for Scalable Visual Representation |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Spyros Gidaris, Lukas Knobel, Mohammadreza Salehi, Valentinos Pariza, Shashanka Venkataramanan |
The paper introduces Franca, a fully open-source vision foundation model for scalable representation learning using nested Matryoshka clustering on public internet-scale data. The primary objective is to create a transparent and reproducible model that matches or surpasses the performance of leading proprietary models like DINOv2 and SigLIPv2. The methodology employs a multi-head clustering projection head on a Vision Transformer, where features are sliced into progressively smaller dimensional subsets (e.g., d, d/8, d/16) to learn a coarse-to-fine semantic hierarchy. Franca demonstrates strong performance on diverse downstream tasks, achieving a 76.7 score on In-Context Learning (VOC) which is a +3.0 improvement over DINOv2. For AI practitioners, Franca offers a high-performance, open-weight vision backbone that can be used for various applications without reliance on proprietary data or models, enhancing reproducibility and accessibility. |
| Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Xue Yang, Wenhao Li, Wenhan Dou, Gen Luo, wzk1015 |
The paper introduces Mono-InternVL-1.5, an efficient monolithic Multimodal Large Language Model (MLLM) that integrates visual encoding and language decoding into a single architecture. The primary objective is to overcome catastrophic forgetting and high computational costs associated with training monolithic MLLMs by enabling stable visual knowledge acquisition within a pre-trained LLM. The methodology involves embedding visual experts into a frozen LLM using a multimodal mixture-of-experts (MMoE) architecture, trained via a data-efficient progressive strategy called EViP++, and accelerated in inference with a custom fused CUDA kernel. The resulting model achieves performance comparable to strong modular MLLMs while reducing first-token latency by up to 69.3% compared to its modular counterpart, InternVL-1.5. For AI practitioners, this research provides a blueprint for developing high-performance, deployment-friendly monolithic MLLMs by adapting pre-trained LLMs with specialized visual experts, significantly reducing training and inference overhead. |
| CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models (Read more on arXiv or HuggingFace) |
Khoi Nguyen, Anh Tran, Quang Nguyen, Minh Luu, nqbinh |
i) This paper introduces CSD-VAR, a novel framework for disentangling content and style from a single image using Visual Autoregressive (VAR) models by exploiting their inherent multi-scale generation architecture. ii) The main objective is to perform effective content-style decomposition (CSD) within VAR models, which have been underexplored for this task, to enable high-fidelity content recontextualization and style transfer from a single reference image. iii) The key methodology combines a scale-aware alternating optimization strategy that aligns content and style losses with their respective generation scales, an SVD-based rectification method to purify style embeddings by removing content-related components, and augmented Key-Value (K-V) memories to better preserve subject identity. iv) On the newly proposed CSD-100 benchmark, CSD-VAR significantly outperforms existing methods, with its Infinity-based variant achieving a content alignment CSD-C score of 0.660 and a CLIP-I score of 0.795, demonstrating superior simultaneous content preservation and stylization fidelity compared to baselines like DreamBooth. v) The principal implication for AI practitioners is that the scale-wise generation process of VAR models offers a structured and effective mechanism for attribute disentanglement, presenting a viable and powerful alternative to diffusion models for building controllable and personalized image generation systems. |
| RedOne: Revealing Domain-specific LLM Post-Training in Social Networking |
|
|
| Services (Read more on arXiv or HuggingFace) |
Ziyan Liu, Zheyong Xie, Yue Wang, Chonggang Lu, Hiiamein |
The paper introduces RedOne, a domain-specific Large Language Model developed via a three-stage post-training strategy to improve performance on tasks within Social Networking Services (SNS). The primary objective is to develop a foundational LLM for the SNS domain that overcomes the performance limitations of single-task models and can generalize across diverse social media applications. The methodology involves a three-stage post-training pipeline applied to a general foundation model: 1) Continued Pretraining (CPT) on a large-scale corpus of general and SNS-specific data; 2) Supervised Fine-Tuning (SFT) on a variety of defined SNS tasks; and 3) Preference Optimization (PO) using Direct Preference Optimization (DPO) to align model outputs with human preferences. The resulting RedOne models demonstrate significant improvements, with RedOne-7B achieving a 14.02% average score increase on the SNS-Bench and a 7.56% increase on SNS-TransBench compared to its base model. In online A/B testing, RedOne reduced harmful content exposure by 11.23% and improved the post-view search click-through rate by 14.95%. The principal implication for AI practitioners is that a structured, multi-stage post-training approach—specifically CPT followed by SFT and PO—is a highly effective strategy for adapting a general-purpose LLM to a specialized domain with unique linguistic characteristics like social media, providing a clear blueprint for creating robust, domain-aware models. |
| Mitigating Object Hallucinations via Sentence-Level Early Intervention (Read more on arXiv or HuggingFace) |
Zhuotao Tian, Li Jiang, Senqiao Yang, Shangpin Peng |
This paper introduces SENTINEL, a framework that mitigates object hallucinations in Multimodal Large Language Models (MLLMs) by intervening at the sentence level where they first emerge, using automatically generated, in-domain preference data. The research objective is to develop an efficient method to suppress hallucinations without human annotation by bootstrapping preference pairs from model outputs, cross-checking object existence with two open-vocabulary detectors, and then fine-tuning with a novel context-aware Direct Preference Optimization (C-DPO) loss. SENTINEL demonstrates a significant reduction in hallucinations, lowering the response-level hallucination rate on the Object HalBench benchmark by over 90% (from 52.7% to 4.3%) compared to the baseline LLaVA-v1.5-7B model. For AI practitioners, this provides a scalable and model-agnostic methodology to enhance MLLM factuality in a resource-efficient manner, enabling the development of more trustworthy applications by automatically creating high-quality, in-domain training data. |
| The Generative Energy Arena (GEA): Incorporating Energy Awareness in |
|
|
| Large Language Model (LLM) Human Evaluations (Read more on arXiv or HuggingFace) |
Pedro Reviriego, Javier Conde, Eneko Sendin, Gonzalo Martínez, Carlos Arriaga |
This paper introduces the Generative Energy Arena (GEA) to measure the impact of energy awareness on human evaluation of LLMs. The study’s primary objective is to quantify how providing information on relative energy consumption influences human evaluators’ model preferences in a head-to-head comparison. The methodology involves a two-step human evaluation where users first select the better of two anonymized model responses and are then asked if they would change their vote after being informed that their initial choice was the higher-energy model. A key result shows that users changed their vote to favor the more energy-efficient model in approximately 46% of these cases, leading to a final preference for smaller models over 75% of the time. The principal implication for AI practitioners is that smaller, more energy-efficient models are often sufficient and preferred by users for many tasks when energy cost is a factor, suggesting that energy metrics should be a critical component of LLM evaluation and deployment decisions. |
| Inverse Reinforcement Learning Meets Large Language Model Post-Training: |
|
|
| Basics, Advances, and Opportunities (Read more on arXiv or HuggingFace) |
Mihaela van der Schaar, Hao Sun |
This paper provides a comprehensive review of Large Language Model (LLM) alignment through the lens of inverse reinforcement learning (IRL), focusing on the necessity of learning neural reward models from human feedback. The paper’s main objective is to survey, structure, and analyze the foundations, recent advances, and practical challenges of applying IRL and reinforcement learning (RL) techniques to LLM post-training, contrasting them with conventional RL tasks. The key methodologies analyzed are framing LLM generation as a Markov Decision Process without a reward function (MDP\R) and reviewing IRL approaches to solve it, including Reinforcement Learning from Human Feedback (RLHF) via PPO, Direct Preference Optimization (DPO), and Alignment from Demonstration (AfD) analyzed through f-divergence minimization. As a review paper, it synthesizes existing findings, highlighting reward overoptimization as a primary result; it cites research (Gao et al., 2023) showing that as an LLM policy is optimized, the gap between its score on the learned reward model and a held-out “gold” reward model widens, quantifying reward hacking. The principal implication for AI practitioners is that moving beyond simple imitation (SFT) to an IRL paradigm with explicit reward models is crucial for robust alignment; this involves a practical trade-off between stable methods like DPO and potentially higher-performing but complex methods like PPO, with a critical need to monitor for reward overoptimization. |
| Quantitative Risk Management in Volatile Markets with an Expectile-Based |
|
|
| Framework for the FTSE Index (Read more on arXiv or HuggingFace) |
0xnu |
This research develops and validates an expectile-based framework for quantitative risk management that outperforms traditional Value-at-Risk (VaR) models for the FTSE 100 index. The objective was to create an advanced risk framework that addresses the shortcomings of conventional quantile-based approaches by providing greater sensitivity to tail losses, especially in volatile market conditions. The methodology utilizes expectile regression on two decades of FTSE 100 daily returns, incorporating GARCH-type dynamics for heteroscedasticity and novel mathematical formulations for time-varying parameters and adaptive thresholds. The primary result from out-of-sample backtesting shows the Expectile-based VaR (EVaR) model achieved a 5.0% violation rate for a 95% confidence level, passing the Conditional Coverage test (p=0.756), whereas traditional methods like Historical Simulation failed with a 12.1% violation rate. The principal implication for AI practitioners is that implementing systems with expectile regression models, which inherently capture tail risk magnitude and asymmetry, offers superior predictive accuracy and robustness over standard quantile-based methods, though it may necessitate infrastructure upgrades to support more complex, real-time computations. |
Papers for 2025-07-18
| Title |
Authors |
Summary |
| A Survey of Context Engineering for Large Language Models (Read more on arXiv or HuggingFace) |
ShowerMaker, LImax72, YuyaoGe, Theodyy, Chevalier |
This survey introduces “Context Engineering” as a formal discipline for optimizing information payloads for LLMs and presents a comprehensive taxonomy of its components and implementations. The objective is to systematically review and organize the field of context manipulation for LLMs by proposing a structured taxonomy that distinguishes between foundational components (retrieval, processing, management) and their system-level implementations (RAG, Memory Systems, Tool Use, Multi-Agent Systems). The methodology consists of a systematic literature review and analysis of over 1400 research papers, which are synthesized into a hierarchical taxonomy. The survey establishes a technical roadmap for Context Engineering, organizing techniques like Tree-of-Thoughts (ToT) which increases Game of 24 success rates from 4% to 74%, and identifies a critical asymmetry where models’ advanced context comprehension capabilities significantly outpace their limited ability to generate sophisticated, long-form outputs. AI practitioners are provided with a unified framework to navigate and implement sophisticated context-aware systems, shifting the focus from ad-hoc prompt design to a systematic, engineering-driven approach for managing information payloads in RAG, agentic, and multi-agent architectures. |
| VisionThink: Smart and Efficient Vision Language Model via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Hengshuang Zhao, Bei Yu, Xin Lai, Junyi Li, Senqiao Yang |
VisionThink is a novel framework that uses reinforcement learning to enable a Vision-Language Model (VLM) to dynamically decide whether to process a low-resolution image or request a higher-resolution one for more efficient inference. The primary objective is to overcome the inefficiency of static visual token counts by creating a model that autonomously determines on a per-sample basis if a compressed, low-resolution image is sufficient or if a high-resolution input is necessary to solve a given task. The methodology employs reinforcement learning with the Group Relative Policy Optimization (GRPO) algorithm, using an “LLM-as-Judge” strategy to generate reward signals for open-ended VQA and a penalty-controlled reward function to manage the high-resolution image request ratio. Experiments show VisionThink achieves 101.4% of the baseline model’s performance on average across nine benchmarks while retaining only approximately 51.3% of visual tokens, thereby maintaining high performance on OCR-heavy tasks where fixed-compression methods degrade. For AI practitioners, this research introduces a practical paradigm for sample-level dynamic efficiency where models adapt their computational load to input complexity, and validates the “LLM-as-Judge” strategy as a viable method for applying RL to complex generative vision-language tasks without complex reward engineering. |
| π^3: Scalable Permutation-Equivariant Visual Geometry Learning (Read more on arXiv or HuggingFace) |
Yang Zhou, Wenzheng Chang, Haoyi Zhu, Jianjun Zhou, Yifan Wang |
π³ is a scalable, permutation-equivariant neural network for visual geometry reconstruction that eliminates the need for a fixed reference view. The main objective is to develop a robust and scalable visual geometry reconstruction model that is invariant to the order of input images, overcoming the instability caused by the traditional reliance on a fixed reference view. The key methodology is a fully permutation-equivariant transformer architecture that discards order-dependent components and predicts affine-invariant camera poses and scale-invariant local point maps for each view, supervised through relative poses and a globally consistent scale factor. The model demonstrates state-of-the-art performance and superior robustness; on the Sintel benchmark for camera pose estimation, π³ reduces the Absolute Trajectory Error (ATE) to 0.074 from the previous state-of-the-art of 0.167. The principal implication for AI practitioners is that this reference-free, permutation-equivariant design provides a more robust and scalable foundation for multi-view 3D reconstruction systems, enabling stable performance on unordered image sets and showing consistent performance gains with increased model size. |
| The Imitation Game: Turing Machine Imitator is Length Generalizable |
|
|
| Reasoner (Read more on arXiv or HuggingFace) |
Songyang Gao, Chengqi Lyu, Wenwei Zhang, vanilla1116, ZhouqiHUA |
This paper introduces Turing MAchine Imitation Learning (TAIL), a data-driven framework that enhances LLM length generalization by fine-tuning on synthetic Chain-of-Thought data that mimics the execution process of a Turing Machine. The objective is to develop a universal reasoning structure that enables LLMs to solve a broad class of “computable problems” with inputs longer than those seen during training. The core methodology involves synthesizing CoT data with three key properties: Linear Transition (unrolling all steps sequentially), Atomic States (decomposing steps into minimal read/write/logic operations), and a Memory Fetcher (explicitly retrieving operands before use). Experiments show that a Qwen2.5-7B model fine-tuned with TAIL achieves 86.5% accuracy on long-sequence large number addition, significantly outperforming prior methods like Index Hint (24.0%), and surpasses DeepSeek-R1 on the majority of 18 algorithmic tasks. The principal implication for AI practitioners is that structuring synthetic training data to explicitly model the fundamental, atomic steps of an algorithm is a highly effective, data-centric strategy for teaching LLMs to generalize their reasoning capabilities to longer problem instances without requiring model architecture changes. |
| AnyCap Project: A Unified Framework, Dataset, and Benchmark for |
|
|
| Controllable Omni-modal Captioning (Read more on arXiv or HuggingFace) |
Gao Meng, Yu Li, Zhiqiang Lin, Yiming Ren, Ruihang |
The AnyCap project introduces a unified framework (ACM), dataset (ACD), and benchmark (AnyCapEval) for controllable omni-modal captioning across images, video, and audio. The primary objective is to address the lack of fine-grained control, dedicated datasets, and reliable evaluation protocols for generating captions that precisely follow user instructions. The key methodology is the AnyCapModel (ACM), a lightweight, plug-and-play module that refines initial captions from frozen base models by incorporating user instructions and modality features, trained on the new 300k-entry preference-based AnyCapDataset. The paper reports that the ACM framework significantly improves caption quality, with the 8B parameter version (ACM-8B) boosting GPT-4o’s content scores by 45% and style scores by 12% on the AnyCapEval benchmark. For AI practitioners, the ACM framework offers a practical, low-cost method to enhance the controllability of existing foundation models for multimodal tasks, enabling precise, instruction-aligned outputs without requiring expensive retraining of the base models. |
| Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos |
|
|
| with Spatio-Temporal Diffusion Models (Read more on arXiv or HuggingFace) |
Zhen Xu, Tao Xie, Xuan Wang, Sida Peng, krahets |
Diffuman4D is a spatio-temporal diffusion model that synthesizes high-fidelity, 4D-consistent human performances from sparse-view videos. The primary objective is to resolve spatio-temporal inconsistencies in generative models for novel view synthesis by introducing a novel sliding iterative denoising process on a 4D latent grid, guided by a mixed conditioning scheme of 3D human skeletons and Plücker coordinates. The key methodology involves alternately denoising this latent grid along spatial and temporal dimensions, allowing information to propagate across the entire sequence to enforce consistency. The model significantly outperforms existing approaches, achieving a PSNR of 25.393 on the DNA-Rendering dataset with 4-view inputs, compared to 21.445 from the next-best generative baseline. The principal implication for AI practitioners is that the sliding iterative denoising technique offers a memory-efficient strategy to enforce long-range consistency in video generation, enabling diffusion models to handle large spatio-temporal domains more effectively. |
| FantasyPortrait: Enhancing Multi-Character Portrait Animation with |
|
|
| Expression-Augmented Diffusion Transformers (Read more on arXiv or HuggingFace) |
Yonggang Qi, Yaqi Fan, Fan Jiang, Mengchao Wang, wangqiang9 |
FantasyPortrait is a Diffusion Transformer-based framework for generating high-fidelity, emotionally expressive single and multi-character portrait animations. The research aims to overcome the limitations of geometry-based methods in cross-identity reenactment and multi-character animation by avoiding explicit priors and preventing feature interference. The key methodology combines an expression-augmented learning strategy using implicit facial representations to capture fine-grained emotions with a novel masked cross-attention mechanism that spatially isolates driving signals for each character within the model’s latent space. On the proposed ExprBench benchmark for cross-reenactment, FantasyPortrait achieved a state-of-the-art Average Expression Distance (AED) of 33.45 (x10⁻²), outperforming comparable methods. For AI practitioners, the principal implication is the masked cross-attention technique, which offers a robust method for achieving independent, region-specific control in multi-subject generative diffusion models by directly manipulating attention scores, a concept applicable to various compositional generation tasks. |
| MindJourney: Test-Time Scaling with World Models for Spatial Reasoning (Read more on arXiv or HuggingFace) |
Reuben Tan, Siyuan Zhou, Zheyuan Zhang, Jiageng Liu, yyuncong |
The paper introduces MindJourney, a test-time framework that enhances a VLM’s 3D spatial reasoning by coupling it with a controllable video diffusion world model to explore imagined viewpoints. The primary objective is to grant Vision-Language Models (VLMs) the ability to reason about the visual consequences of egocentric motion in a 3D scene from a single image, without any model fine-tuning. The methodology involves an iterative Spatial Beam Search where a VLM proposes camera trajectories, a world model generates the corresponding egocentric video rollouts, and the VLM then scores and selects the most informative generated views to answer a spatial query. The framework achieves a significant performance boost across various VLMs on the SAT benchmark, increasing the accuracy of GPT-4.1 on SAT-Real from 67.3% to 82.6% (+15.3%). The principal implication for AI practitioners is that this plug-and-play approach provides a direct, training-free method to improve the spatial intelligence of existing VLMs for embodied AI tasks by integrating them with world models to create a “mental workspace” for the agent at inference time. |
| AbGen: Evaluating Large Language Models in Ablation Study Design and |
|
|
| Evaluation for Scientific Research (Read more on arXiv or HuggingFace) |
Yixin Liu, Manasi Patwardhan, Zhijian Xu, Weiyuan Chen, Yilun Zhao |
The paper introduces ABGEN, a benchmark of 1,500 expert-annotated examples from 807 NLP papers, to evaluate the ability of Large Language Models to design scientific ablation studies. The primary objective is to assess how well frontier LLMs perform in generating detailed and sound ablation study designs for a specified research module, given a comprehensive research context. The methodology involves having NLP experts create reference designs from published papers and then rate LLM-generated outputs on a 1-5 scale across importance, faithfulness, and soundness. The evaluation reveals a significant performance gap, with the top-performing model, DeepSeek-R1-0528, achieving an average human evaluation score of 4.11, considerably lower than the 4.80 achieved by human experts. The principal implication for AI practitioners is that current LLMs are not yet reliable for autonomously designing valid scientific experiments and that LLM-based evaluation systems for such complex, domain-specific tasks require significant improvement to align with expert assessment. |
| Teach Old SAEs New Domain Tricks with Boosting (Read more on arXiv or HuggingFace) |
Yaroslav Aksenov, Nikita Koriagin, kefirski, elephantmipt, dlaptev |
The paper introduces SAE Boost, a residual learning method to enhance pretrained Sparse Autoencoders (SAEs) with domain-specific features by training a secondary SAE on the reconstruction error of the primary one. The main objective is to adapt a general-purpose SAE to capture features from a specialized domain without full retraining or catastrophic forgetting of general capabilities. The key methodology involves training a secondary “residual” SAE on domain-specific data, using the reconstruction error (x - x̂) from a frozen, pretrained SAE as its target; during inference, the outputs of both SAEs are summed. Primary results show significant improvements on specialized domains, for instance, increasing explained variance on chemistry data from 0.571 to 0.716 (+25.39%) while maintaining performance on general tasks. The principal implication for AI practitioners is that they can use SAE Boost to efficiently and modularly extend existing interpretability tools to new domains, enabling targeted analysis of LLM behavior without rebuilding models from scratch. |
| FLEXITOKENS: Flexible Tokenization for Evolving Language Models (Read more on arXiv or HuggingFace) |
Sachin Kumar, Orevaoghene Ahia, Abraham Toluase Owodunni |
This paper introduces FLEXITOKENS, a method for training byte-level language models with a flexible, gradient-based tokenizer that adapts its segmentation strategy during finetuning. The research objective is to overcome the rigidity of static subword and fixed-compression-rate tokenizers, which limits model performance when adapting to new data distributions. The methodology involves a byte-level hourglass architecture trained with a novel hinge-like loss that enforces a flexible lower bound on the compression rate, rather than a fixed target, allowing segmentation to vary based on the input. Evaluation across multiple benchmarks demonstrates that FLEXITOKENS consistently reduces token over-fragmentation and achieves up to 10% improvement on downstream task performance compared to baselines. The principal implication for AI practitioners is that this method allows for more effective domain and language adaptation, as the tokenizer co-adapts with the model during finetuning, improving performance and efficiency without requiring complex tokenizer retraining or replacement. |
| TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame |
|
|
| Interpolation (Read more on arXiv or HuggingFace) |
Chen Chen, ucfzl |
The paper introduces TLB-VFI, a temporal-aware latent diffusion model using a Brownian Bridge process for efficient and high-quality video frame interpolation. The primary objective is to create a video frame interpolation model that extracts rich temporal information while overcoming the high computational cost, large model size, and extensive data requirements of previous video-based diffusion methods. The methodology employs a temporal-aware autoencoder that operates in both pixel space (using 3D-wavelet gating) and latent space (using temporal blocks with 3D convolution), and applies a Brownian Bridge Diffusion Model between the latent codes of the original video clip and a version with the intermediate frame zeroed-out, ensuring a significant distributional shift for effective generation. The model achieves state-of-the-art performance, including a 20% FID improvement on the SNU-FILM extreme dataset over prior image-based diffusion methods, while having 3x fewer parameters and requiring up to 9000x less training data than other video-based diffusion approaches. For AI practitioners, this work provides a framework for building highly efficient generative video models that do not require massive-scale training, demonstrating how to effectively apply Brownian Bridge diffusion for conditional tasks by engineering a meaningful latent space gap between the condition and the target. |
| Automating Steering for Safe Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Nay Oo, Tri Cao, Ziwen Xu, Mengru Wang, Lyucheng Wu |
i) A 1-line summary The paper introduces AutoSteer, a modular and adaptive inference-time framework to automatically enhance the safety of Multimodal Large Language Models by detecting and steering away from harmful content without model retraining. ii) Main research question or objective The primary objective is to develop a fully automated technique to improve MLLM safety during inference against textual, visual, and cross-modal threats while preserving the model’s general-purpose capabilities. iii) Key methodology used AutoSteer employs a three-part methodology: a novel Safety Awareness Score (SAS) to automatically identify the most safety-relevant internal layer, an adaptive safety prober trained on that layer’s activations to estimate toxicity, and a lightweight refusal head that conditionally steers generation towards a safe refusal. iv) Primary results (include at least one specific quantitative finding) Experiments show AutoSteer significantly reduces the Attack Success Rate (ASR); for the LLaVA-OV model on the VLSafe benchmark, ASR was reduced from 60.0% to 4.2% while its accuracy on the RealWorldQA benchmark was fully preserved (61.8% vs. 61.8% original). v) Principal implication for AI practitioners AI practitioners can use AutoSteer as a practical, plug-and-play safety solution for existing MLLMs that automates intervention without requiring costly fine-tuning, minimizing the trade-off between safety and utility for safer real-world deployment. |
| Voxtral (Read more on arXiv or HuggingFace) |
Corentin Barreau, Clément Denoix, Andy Lo, Andy Ehrenberg, Alexander H. Liu |
The paper introduces Voxtral Mini and Voxtral Small, two open-weight multimodal audio chat models designed for advanced speech and text understanding. The primary objective is to develop and evaluate open-source audio language models that can process long-form audio (up to 40 minutes) and achieve state-of-the-art performance across transcription, translation, and audio reasoning tasks. The methodology involves a Transformer-based architecture comprising a Whisper large-v3 audio encoder, an MLP adapter layer that downsamples audio embeddings by a factor of 4x to a 12.5Hz frame rate, and a Mistral language decoder, trained via pretraining, supervised finetuning, and preference alignment. The models demonstrate superior performance on various benchmarks; notably, Voxtral Small achieves state-of-the-art speech translation scores on the FLEURS benchmark across all tested language pairs, such as a 57.3 BLEU score for English-to-French translation. The principal implication for AI practitioners is the availability of Apache 2.0 licensed, high-performance models that serve as a strong open-source foundation for building applications requiring long-context audio understanding, providing a competitive alternative to closed-source systems. |
| Einstein Fields: A Neural Perspective To Computational General |
|
|
| Relativity (Read more on arXiv or HuggingFace) |
Johannes Brandstetter, Arturs Berzins, Sandeep Suresh Cranganore, AndreiB137 |
This paper introduces Einstein Fields (EinFields), a neural tensor field representation that compresses computationally intensive four-dimensional numerical relativity simulations into compact implicit neural network weights. The main objective is to create a memory-efficient, continuous, and differentiable representation of the spacetime metric tensor, allowing for the accurate derivation of physical quantities via automatic differentiation without relying on domain discretization. The methodology involves parameterizing the metric tensor with a multi-layer perceptron (MLP) trained using a Sobolev loss that supervises the network’s output as well as its Jacobian and Hessian derivatives. The primary result shows that EinFields can achieve a Mean Absolute Error as low as 6.89E-8 on the Schwarzschild metric components, compressing the data by a factor of up to 4035 compared to explicit grids. The principal implication for AI practitioners is that implicit neural representations, combined with Sobolev training, can act as highly efficient and differentiable compressors for complex scientific tensor fields, providing a framework for modeling physical systems where accurate derivatives are crucial. |
Papers for 2025-07-17
| Title |
Authors |
Summary |
| Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning |
|
|
| Systems in LLMs (Read more on arXiv or HuggingFace) |
Wei-Chieh Huang, Yuyao Yang, Yangning Li, TreeForest, WZDavid |
This survey provides a unified taxonomy for systems integrating Retrieval-Augmented Generation (RAG) and deep reasoning in LLMs, charting an evolution from one-way enhancements to synergized, agentic frameworks. The primary objective is to systematically categorize and analyze the convergence of retrieval and reasoning methodologies in LLMs, moving beyond static Retrieval-Then-Reasoning to describe iterative, agentic systems that dynamically interleave both processes. The paper conducts a comprehensive literature review, structuring its analysis around a proposed three-part taxonomy: 1) Reasoning-Enhanced RAG, where reasoning improves RAG stages; 2) RAG-Enhanced Reasoning, where retrieval grounds reasoning; and 3) Synergized RAG-Reasoning, characterized by iterative, agentic interplay. The survey synthesizes findings from over 200 research papers, identifying a paradigm shift towards Synergized RAG-Reasoning systems that employ complex reasoning workflows (chain, tree, graph-based) and agentic orchestrations (single- and multi-agent) to solve knowledge-intensive tasks. AI practitioners can use this survey’s taxonomy and benchmark analysis (covering 46 benchmarks across 13 tasks) to select appropriate architectural patterns—such as tree-based workflows for ambiguous tasks or multi-agent systems for heterogeneous data—and evaluation methods for building and validating more robust, factually-grounded, and adaptable reasoning systems. |
| PhysX: Physical-Grounded 3D Asset Generation (Read more on arXiv or HuggingFace) |
Linag Pan, liuziwei7, FrozenBurning, Caoza |
The paper introduces PhysX, an end-to-end framework for generating 3D assets with grounded physical properties, supported by a new richly annotated dataset, PhysXNet. The primary objective is to create a methodology and dataset for generating 3D models with comprehensive physical attributes, such as material, kinematics, and absolute scale, to enhance their utility in physical simulations. The methodology utilizes a dual-branch VAE to encode structural and physical properties into separate latent spaces, followed by a conditional diffusion transformer that jointly generates these latents by fine-tuning a pre-trained geometric model on the new PhysXNet dataset. Compared to a strong baseline using Trellis, PartField, and GPT-4o, PhysXGen achieves significant relative improvements, including a 64% enhancement in material property prediction and a 72% improvement in kinematics parameter generation. For AI practitioners, this work provides a model and dataset to generate physically coherent 3D assets, enabling more realistic development and testing of agents for robotics and embodied AI in simulated environments. |
| MOSPA: Human Motion Generation Driven by Spatial Audio (Read more on arXiv or HuggingFace) |
Leo Ho, Liang Pan, Mingyi Shi, frankzydou, JimSYXu |
The paper introduces MOSPA, a diffusion-based generative model for synthesizing human motion from spatial audio, and presents SAM, the first large-scale dataset for this task. The primary objective is to generate realistic and responsive 3D human motion conditioned on spatial audio signals by modeling the complex interplay between auditory spatial cues and human movement. The methodology involves MOSPA, a diffusion-based probabilistic model with an encoder-only transformer, trained on the novel 9-hour SAM dataset to denoise motion sequences conditioned on extracted audio features, sound source location, and motion genre. MOSPA achieves state-of-the-art performance, attaining a Fréchet Inception Distance (FID) of 7.981, significantly outperforming the next best baseline (EDGE at 13.993) and closely approaching real motion data. For AI practitioners, this work provides a framework and dataset for creating more immersive virtual agents that can react dynamically to the location and semantics of sound, moving beyond traditional audio-to-motion generation that ignores spatial information. |
| MMHU: A Massive-Scale Multimodal Benchmark for Human Behavior |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Mingyang Wu, Renjie Li, vztu, waynefan, jerryye0110 |
This paper introduces MMHU, a large-scale multimodal benchmark with 57k human instances for comprehensive human behavior understanding in autonomous driving scenarios. The objective is to provide a unified dataset to evaluate and advance algorithms for human behavior analysis, which is critical for safety but lacks a comprehensive benchmark. The methodology involves collecting video data from diverse sources (Waymo, YouTube, self-recorded) and using a human-in-the-loop pipeline to generate rich annotations, including 3D SMPL motion, trajectories, hierarchical text descriptions, and labels for 13 critical behaviors. Experiments show that fine-tuning models on MMHU yields significant performance gains; for example, fine-tuning the Qwen2.5-VL model on the behavior VQA task improved its F1-score from 44.72% to 68.54%. For AI practitioners, MMHU serves as a crucial resource to benchmark and improve models for nuanced human-centric tasks in autonomous driving, demonstrating a direct path to enhancing the performance and safety of perception systems. |
| SWE-Perf: Can Language Models Optimize Code Performance on Real-World |
|
|
| Repositories? (Read more on arXiv or HuggingFace) |
Zhijie Fan, Lin Yan, Xinyi He, Elfsong, SivilTaram |
This paper introduces SWE-Perf, the first benchmark designed to systematically evaluate the ability of Large Language Models to optimize code performance in real-world repositories. The research objective is to quantify the gap between current LLM capabilities and human expert performance on complex, repository-level optimization tasks. The authors constructed the benchmark by curating 140 instances from performance-improving pull requests on popular GitHub projects, creating executable environments to measure runtime changes, and evaluating models under file-level (Oracle) and repo-level (Realistic) agentic settings. Results demonstrate a significant performance deficit: the best autonomous agent (OpenHands with Claude-3.7-sonnet) achieved a 2.26% performance improvement, far below the 10.85% achieved by the original human-expert patches. For AI practitioners, this highlights that while LLMs show potential, they currently lack the sophisticated reasoning to perform meaningful, cross-file performance optimizations, indicating that relying on them for this task is premature and further research is needed to bridge the gap with human expertise. |
| DrafterBench: Benchmarking Large Language Models for Tasks Automation in |
|
|
| Civil Engineering (Read more on arXiv or HuggingFace) |
Yi Shao, zhendongucb, Eason666 |
This paper introduces DrafterBench, a benchmark for evaluating LLM agents on technical drawing revision tasks in civil engineering. The objective is to systematically evaluate an LLM agent’s proficiency in interpreting intricate instructions, leveraging prior knowledge, and adapting to dynamic instruction quality via implicit policy awareness. The methodology uses 1,920 tasks from real-world files and 46 customized “dual functions” which record the agent’s operation path for comparison against a ground truth path, rather than assessing the final output drawing. The primary results show that even the leading model, OpenAI o1, achieved an average task score of only 81.92, and all models showed significant performance degradation (up to 18%) when faced with incomplete instructions. The principal implication for AI practitioners is that current LLMs lack the required robustness for detailed industrial automation, specifically struggling with vague instructions and the implementation of new, overriding policies, which are critical areas for future development. |
| AnyI2V: Animating Any Conditional Image with Motion Control (Read more on arXiv or HuggingFace) |
Hao Luo, HenghuiDing, XinchengShuai, TribeRinb |
The paper introduces AnyI2V, a training-free framework for animating images from diverse conditional modalities with user-defined motion trajectories. The objective is to create a method for image-to-video generation that enables spatial control from any conditional input (e.g., mesh, depth) and explicit motion control via trajectories, without the need for model retraining. The methodology injects debiased residual hidden and query features from an initial conditional image into a pretrained video diffusion model, then performs zero-shot trajectory control by optimizing latents to align these query features across frames, guided by an adaptive semantic mask. The proposed method achieves high motion control accuracy with an ObjMC score of 16.39, significantly outperforming the baseline (38.26) and demonstrating competitive performance against other state-of-the-art models. The principal implication for AI practitioners is that this training-free approach allows them to add controllable animation to various existing video diffusion backbones without computationally expensive fine-tuning, enabling flexible video generation from diverse structural inputs. |
| SpatialTrackerV2: 3D Point Tracking Made Easy (Read more on arXiv or HuggingFace) |
Yuxi Xiao, bykang, nikkar, cherubicxn, JianyuanWang |
SpatialTrackerV2 is a feed-forward method for 3D point tracking from monocular videos that unifies the estimation of scene geometry, camera ego-motion, and object motion in a single end-to-end architecture. The primary objective is to develop a scalable 3D point tracking model that overcomes the limitations of modular pipelines by jointly reasoning about motion components, enabling training across diverse and weakly-supervised datasets. The methodology uses a dual-stage architecture where a front-end temporal encoder provides initial depth and camera poses, which are then refined by a novel back-end transformer, “SyncFormer,” that iteratively optimizes 2D/3D trajectories and camera poses using a dual-branch design and in-loop bundle adjustment. The model establishes a new state-of-the-art on the TAPVid-3D benchmark, achieving an Average Jaccard (AJ) of 21.2, and matches the accuracy of leading dynamic 3D reconstruction methods while running 50x faster. For AI practitioners, the principal implication is that a unified, feed-forward model trained on heterogeneous data can surpass modular, optimization-based pipelines in complex 3D tracking tasks, offering a scalable path to building robust 3D perception systems without computationally expensive per-scene optimization. |
| Lizard: An Efficient Linearization Framework for Large Language Models (Read more on arXiv or HuggingFace) |
Franck-Dernoncourt, Nikosapa, TrungBui1111, jasubram, haniehds |
The paper introduces Lizard, a framework that converts pretrained Transformers into subquadratic models for efficient infinite-context generation by replacing softmax attention with a hybrid of gated linear and sliding window attention. The objective is to linearize pretrained Large Language Models (LLMs) to overcome the quadratic complexity of softmax attention and the linear growth of the KV cache, thereby enabling efficient long-context processing while minimizing performance degradation compared to the original model. Lizard employs a two-stage process: first, a hybrid attention module—combining a data-dependent gated linear attention with a sliding window attention enhanced by meta-memory—is trained to approximate the original model’s softmax outputs; second, the new module replaces the original, and the model is fine-tuned on a language modeling objective. On the 5-shot MMLU benchmark, the Lizard-linearized LLaMA-3-8B model achieves a score of 61.2, an 18-point improvement over the prior Mamba2-LLaMA-3-8B method, and demonstrates perfect recall on tasks requiring generalization beyond its training context length. For AI practitioners, this provides a method to adapt existing pretrained LLMs for long-context applications with constant-memory inference, avoiding the prohibitive computational costs of standard Transformers without a significant loss in performance. |
| Replacing thinking with tool usage enables reasoning in small language |
|
|
| models (Read more on arXiv or HuggingFace) |
Roland Memisevic, Tim Bakker, crainone |
This paper introduces Chain-of-Edits (CoE), a method that replaces natural language reasoning with structured tool interactions to enable small language models (≤3B parameters) to perform complex code repair tasks. The main objective is to determine if parameterizing “thinking” as a trace of interactions with a tool, rather than as natural language, allows smaller models to effectively perform multi-step reasoning via reinforcement learning. The methodology is a two-stage pipeline consisting of Supervised Fine-Tuning (SFT) on synthetic CoE demonstrations using a domain-specific language (DSL), followed by Reinforcement Learning with Verifiable Rewards (RLVR) on a code repair benchmark. The CoE method significantly improved performance for smaller models; for a 1B parameter model, it achieved a 7.82% pass@1 rate, substantially outperforming both direct 3-shot prompting (1.3%) and a Chain-of-Thought baseline (0.15%), though this advantage reversed for an 8B model. The principal implication for AI practitioners is that structuring a problem as an interaction with a tool via a constrained DSL can enable smaller, more efficient models to solve complex, stateful tasks, providing a viable path for deploying reasoning capabilities in resource-constrained environments. |
| RLEP: Reinforcement Learning with Experience Replay for LLM Reasoning (Read more on arXiv or HuggingFace) |
Jingyuan Zhang, Jia Fu, GuoruiZhou, Edrex, hongzhizhang |
RLEP is a two-phase Reinforcement Learning framework that improves LLM reasoning by replaying verified successful trajectories from a prior training run. The main objective is to mitigate RL training instability and policy drift in LLMs by using previously discovered high-quality reasoning paths to accelerate training and achieve a higher final performance. The methodology first collects a pool of verified correct trajectories from a converged baseline RL model, then restarts training, optimizing the policy at each step on a mixed batch of newly generated rollouts and replayed successes using a token-mean, asymmetrically clipped GRPO objective. On the Qwen2.5-Math-7B base model, RLEP improved accuracy on the AIME-2024 dataset from a 38.2% baseline peak to 39.9% and on the unseen AMC-2023 dataset from 77.0% to 82.2%. The principal implication for AI practitioners is that incorporating an experience replay mechanism with curated, successful past trajectories into the RL fine-tuning process can accelerate convergence and achieve a higher performance ceiling, providing a more stable and sample-efficient training paradigm. |
Papers for 2025-07-16
| Title |
Authors |
Summary |
| Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation |
|
|
| from Diffusion Models (Read more on arXiv or HuggingFace) |
Jieneng Chen, Yu-cheng Chou, Yitong Li, lambertxiao, PatZhang11 |
The Vision-Language-Vision (VLV) auto-encoder is a framework for distilling knowledge from frozen text-to-image (T2I) diffusion models to create a high-quality captioner with minimal cost. The primary objective is to develop a state-of-the-art captioning model that avoids the need for massive paired image-text datasets by leveraging existing pretrained models. The methodology uses a two-stage process: an encoder is first trained using only images to produce continuous embeddings that allow a frozen T2I diffusion model to reconstruct the image, then a pretrained LLM is fine-tuned to decode these embeddings into natural language. VLV achieves captioning performance comparable to proprietary models, with its captions yielding a text-to-image reconstruction FID score of 6.64, which is competitive with GPT-4o’s score of 6.20, for a total training cost under $1,000. The principal implication for AI practitioners is that state-of-the-art multimodal systems can be built in a data- and cost-efficient manner by distilling knowledge from existing open-source models, drastically lowering entry barriers for developing advanced captioning capabilities. |
| EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and |
|
|
| Reasoning Modes (Read more on arXiv or HuggingFace) |
Stanley Jungkyu Choi, Kibong Choi, Eunbi Choi, Kyunghoon Bae, LG AI Research |
This paper introduces EXAONE 4.0, a series of unified language models that integrate distinct NON-REASONING and REASONING modes into a single architecture with agentic tool-use capabilities. The main objective is to unify the instruction-following usability of EXAONE 3.5 and the advanced reasoning of EXAONE Deep into a single model, while expanding context length to 128K and adding Spanish language support. The methodology involves pre-training on up to 14T tokens, employing a hybrid attention architecture (3:1 local-to-global ratio), and a multi-stage post-training pipeline featuring a novel reinforcement learning algorithm, AGAPO (Asymmetric Sampling and Global Advantage Policy Optimization). The EXAONE 4.0 32B model, in its REASONING mode, achieves a score of 85.3 on the AIME 2025 math benchmark, outperforming the larger Qwen 3 235B model. For AI practitioners, the key implication is the availability of an open-weight model that provides a switchable trade-off between fast, efficient responses and computationally intensive, high-accuracy reasoning within a single deployment, enabling flexible application development. |
| Scaling Laws for Optimal Data Mixtures (Read more on arXiv or HuggingFace) |
Enrico Fini, David Grangier, Dan Busbridge, Louis Bethune, Mustafa Shukor |
This research proposes and validates scaling laws that predict foundation model loss as a function of model size (N), training tokens (D), and data domain mixture weights (h). The primary objective is to create a systematic method for determining the optimal data mixture for any target domain under a given training budget (N,D), replacing ad-hoc, trial-and-error approaches. The methodology extends Chinchilla-style power laws by modeling the law’s coefficients as parametric functions of the domain mixture weights, with parameters estimated from a few small-scale training runs across different mixtures. The scaling laws accurately extrapolate from small-scale fits (e.g., models <1B parameters) to predict the loss of large-scale models (e.g., 8B parameters) on new, unseen domain mixtures; a 7B LLM trained with an optimized mixture achieved a CORE score of 58, outperforming the base (52) and uniform (53) mixtures. The principal implication for AI practitioners is the ability to use a few small-scale, low-cost experiments to computationally derive a near-optimal data mixture for a large-scale training budget, providing a principled alternative to costly, ad-hoc mixture selection. |
| Can Multimodal Foundation Models Understand Schematic Diagrams? An |
|
|
| Empirical Study on Information-Seeking QA over Scientific Papers (Read more on arXiv or HuggingFace) |
Arman Cohan, Chuhan Li, Chengye Wang, yilunzhao |
This paper introduces MISS-QA, a benchmark with 1,500 expert-annotated examples for evaluating multimodal models’ ability to answer questions by interpreting schematic diagrams in scientific papers. The primary objective is to assess how well frontier foundation models can interpret these diagrams and synthesize information from the surrounding paper context, and to identify their key failure modes. The methodology involved constructing the benchmark from 465 AI papers, with questions requiring grounding in highlighted visual elements, and evaluating 18 models using an automated system with GPT-4.1 as the judge. The primary result shows a significant performance gap, with human experts achieving 89.0% accuracy while the top open-source model (Qwen2.5-VL-72B) scored only 61.6%, and most models exhibited overconfidence on unanswerable questions. The principal implication for AI practitioners is that current multimodal models are not yet reliable for scientific document analysis that requires contextual understanding of schematic diagrams, frequently failing to interpret visual structures or retrieve relevant text, and practitioners can use MISS-QA to test and mitigate these weaknesses. |
| LLMalMorph: On The Feasibility of Generating Variant Malware using |
|
|
| Large-Language-Models (Read more on arXiv or HuggingFace) |
Ashish Kundu, Arun Iyengar, Imtiaz Karim, Adrian Shuai Li, Ajwad |
The paper introduces LLMalMorph, a semi-automated, source-code-level framework that uses a pre-trained Large-Language-Model with engineered prompts to generate functional, evasive malware variants. The primary objective is to determine the feasibility of using pre-trained LLMs, without additional fine-tuning, to develop a semi-automated framework for generating evasive malware variants from C/C++ source code that preserve semantics and can bypass antivirus engines and ML-based classifiers. The framework, LLMalMorph, systematically extracts functions from malware source code using an AST parser, generates tailored prompts incorporating one of six transformation strategies (e.g., Code Optimization, Security), and uses the Codestral-22B model to produce modified code, which is then reintegrated and compiled with a human-in-the-loop process for debugging. Primary results demonstrate that LLMalMorph successfully reduced antivirus detection rates, with the “Windows” transformation strategy achieving a 37% detection rate reduction for the RansomWar sample. Furthermore, against an ML-based classifier (Malgraph), the “Security” transformation strategy achieved an attack success rate of 90.9% for the Babuk ransomware sample, despite not being explicitly optimized for ML evasion. The principal implication for AI practitioners is that general-purpose, pre-trained code-generating LLMs can be effectively repurposed for sophisticated offensive security tasks, demonstrating a critical dual-use concern and underscoring the need for robust, semantically-aware malware detectors resilient to LLM-driven code transformations. |
| OpenCodeReasoning-II: A Simple Test Time Scaling Approach via |
|
|
| Self-Critique (Read more on arXiv or HuggingFace) |
Mehrzad Samadi, Sean Narenthiran, Aleksander Ficek, Wasi Uddin Ahmad, smajumdar94 |
The paper introduces OPENCODEREASONING-II, a 2.5 million sample dataset for code generation and critique, and uses it to demonstrate a test-time scaling method that improves code generation performance via self-critique. The main objective is to determine if fine-tuning models on a large-scale dataset containing code solutions and corresponding critiques can enable effective test-time performance improvement through a self-selection mechanism. The methodology involves a two-stage supervised fine-tuning process on Qwen2.5-Instruct models, first for code generation and then jointly for generation and critique, followed by an inference strategy that generates multiple solutions and selects the best one using a self-critique heuristic. The primary result shows that this self-critique method improves the pass@1 score of their flagship OCR-2-32B model on the LiveCodeBench Python benchmark by 6.1 percentage points, from 61.3% to 67.4%. The principal implication for AI practitioners is that they can enhance the single-attempt accuracy of code generation models by fine-tuning on critique data and applying a simple self-critique selection strategy at inference time, obviating the need for external verifiers or complex reinforcement learning pipelines. |
| AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs (Read more on arXiv or HuggingFace) |
Bryan Perozzi, Mikhail Galkin, Jan Tönshoff, Luis Müller, Florian Grötschla |
The paper introduces AGENTSNET, a new benchmark for evaluating the coordination and collaborative reasoning capabilities of multi-agent LLM systems. The primary objective is to assess whether complex networks of LLM agents can effectively self-organize, communicate, and form collaborative strategies given a specific network topology. The methodology challenges multi-agent systems with five problems from distributed computing—consensus, leader election, coloring, matching, and vertex cover—on graph networks of varying sizes (up to 100 agents) and topologies, using a synchronous message-passing protocol. Results show that while frontier models like Gemini 2.5 Pro achieve high performance (0.80 mean score) on small networks of up to 16 agents, performance degrades significantly as network size scales, dropping to near-zero for 100-agent networks. For AI practitioners, this implies that current LLMs exhibit emergent coordination in small groups, but developing scalable multi-agent systems requires significant improvements in the models’ ability to maintain coherent global strategies under increasing communication complexity. |
| Planted in Pretraining, Swayed by Finetuning: A Case Study on the |
|
|
| Origins of Cognitive Biases in LLMs (Read more on arXiv or HuggingFace) |
Gabriel Stanovsky, Yonatan Belinkov, itay1itzhak |
This research investigates the origins of cognitive biases in LLMs and concludes they are predominantly established during pretraining. The study’s objective is to disentangle whether these biases are planted during pretraining or shaped by instruction data and randomness during the finetuning phase. A two-step causal framework is used, featuring multi-seed finetuning to measure randomness and a “cross-tuning” methodology where different pretrained models (OLMo-7B, T5-11B) are finetuned on swapped instruction datasets. Results show pretraining is the dominant factor; clustering models by their bias vectors (across 32 biases) reveals that grouping by pretrained identity is significantly more coherent than by finetuning data, achieving a Silhouette score of 0.104 versus 0.028 for instruction-based clustering. The principal implication for AI practitioners is that mitigating cognitive biases requires interventions at the pretraining stage, as post-hoc finetuning has a limited ability to alter these foundational patterns. |
Papers for 2025-07-15
| Title |
Authors |
Summary |
| SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual |
|
|
| Dyadic Interactive Human Generation (Read more on arXiv or HuggingFace) |
Deyu Zhou, Jiahe Zhang, Duomin Wang, Zhaoyang Li, Youliang Zhang |
This paper introduces SpeakerVid-5M, a large-scale, high-quality public dataset designed for audio-visual dyadic interactive human generation. The primary objective is to facilitate research into interactive virtual humans by providing a richly annotated dataset that addresses the scarcity of public resources for this task. The methodology involves a multi-stage pipeline for data curation from public videos, including scene detection, speaker diarization, lip-sync analysis, and extensive multi-modal annotation (e.g., ASR, pose, blur scores), followed by rigorous quality filtering. The primary result is the dataset itself, containing over 8.7K hours of video, 770K dyadic dialogue pairs, and a demonstration baseline model which, on the dyadic setting, achieves an FVD of 28.82 on the paper’s VidChatBench benchmark. The principal implication for AI practitioners is the availability of a large-scale, tiered (pre-training and SFT) dataset with multiple interaction branches (dialogue, single-speaker, listening, multi-turn), enabling the development and standardized evaluation of models for more complex, coherent, and interactive audio-visual agents. |
| EmbRACE-3K: Embodied Reasoning and Action in Complex Environments (Read more on arXiv or HuggingFace) |
Kui Wu, Chengjie Jiang, Yitang Li, Wei Huang, Mingxian Lin |
This paper introduces EmbRACE-3K, a benchmark dataset with over 3,000 language-guided tasks designed to address the poor performance of vision-language models (VLMs) in interactive, embodied environments. The primary objective is to create a challenging benchmark that captures the closed-loop perception-action cycle and enables training for long-horizon, instruction-guided tasks. The authors’ methodology involves constructing the dataset with step-wise natural language reasoning annotations and then training a Qwen2.5-VL-7B model using a two-stage approach of supervised fine-tuning (SFT) followed by reinforcement learning (RL). The results demonstrate that while state-of-the-art models perform poorly in zero-shot settings (e.g., GPT-4o SR < 20%), the fine-tuned model’s success rate on out-of-domain multi-stage tasks improves from 0.0% to 27.0%. For AI practitioners, this work provides a high-quality dataset and a validated SFT+RL training recipe to significantly enhance a VLM’s embodied reasoning and planning abilities for agentic applications. |
| Reasoning or Memorization? Unreliable Results of Reinforcement Learning |
|
|
| Due to Data Contamination (Read more on arXiv or HuggingFace) |
Jun Zhao, Zhiheng Xi, Qiaole Dong, Zhihao Zhang, Mingqi Wu |
This research investigates the anomalous performance improvements of Qwen2.5 models from reinforcement learning on math benchmarks, attributing the gains to memorization from data contamination rather than genuine reasoning. The primary objective was to determine whether spurious rewards in RL genuinely enhance the Qwen2.5 model family’s reasoning capabilities or merely trigger the recall of memorized answers from contaminated evaluation sets like MATH-500. The methodology involved a leakage audit using partial-prompt completion tests on existing benchmarks and controlled RL experiments on a newly created, leakage-free synthetic dataset called RandomCalculation. The study found that when prompted with 60% of a MATH-500 problem, Qwen2.5-Math-7B achieved a 54.60% exact match completion rate, and on the clean RandomCalculation dataset, performance only improved with accurate reward signals, while random or incorrect rewards provided no benefit. The principal implication for AI practitioners is the critical need to evaluate new methods on verifiably uncontaminated benchmarks to ensure that reported performance gains reflect true capability improvements and not test set leakage. |
| REST: Stress Testing Large Reasoning Models by Asking Multiple Problems |
|
|
| at Once (Read more on arXiv or HuggingFace) |
Zinan Tang, Qiyao Sun, Yu Li, Qizhi Pei, Zhuoshi Pan |
This paper introduces REST, a stress-testing framework that evaluates Large Reasoning Models (LRMs) by concurrently presenting multiple problems in a single prompt. The main objective is to evaluate how well LRMs handle multiple simultaneous reasoning tasks and to identify factors contributing to performance degradation under such multi-context stress, addressing the limitations of saturated single-question benchmarks. The key methodology, REST (Reasoning Evaluation through Simultaneous Testing), transforms existing benchmarks by concatenating a set number of questions (the stress level) into a single prompt, with performance evaluated by extracting and scoring individual answers from the model’s unified response. Primary results show that even state-of-the-art models exhibit substantial performance degradation; for example, DeepSeek-R1’s accuracy on AIME24 drops by 29.17% under REST compared to single-question evaluation. The framework reveals significant performance differences between models that score similarly on standard benchmarks, with “Question Omission” and positional bias (earlier questions are answered more accurately) being dominant failure modes. The principal implication for AI practitioners is that high performance on isolated, single-problem benchmarks does not guarantee robustness in multi-context applications. The finding that models trained with “long2short” techniques show greater resilience under REST offers a concrete architectural direction for developing more robust LRMs capable of managing dynamic cognitive loads. |
| Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive |
|
|
| Token-Level Computation (Read more on arXiv or HuggingFace) |
Jiyoun Ha, Sungnyun Kim, Reza Bayat, Yujin Kim, Sangmin Bae |
Mixture-of-Recursions (MoR) is a unified Transformer framework that combines parameter sharing with token-level adaptive computation to improve efficiency without sacrificing performance. The primary objective is to develop a single architecture that simultaneously achieves the benefits of both parameter sharing (via recursion) and adaptive computation (via dynamic routing) for language models. The key methodology is a “recursion block” of shared Transformer layers, where lightweight, learnable routers (either “expert-choice” or “token-choice”) dynamically assign a specific number of recursion steps to each token. This is paired with specialized Key-Value (KV) caching strategies, such as “recursion-wise caching” which only caches active tokens at each recursion step. The primary result is a new Pareto frontier for model efficiency; under an equal training budget of 16.5e18 FLOPs, a 167M parameter MoR model achieves 43.1% average few-shot accuracy, surpassing a 315M parameter vanilla baseline (42.3% accuracy). Additionally, MoR demonstrates up to a 2.06x inference throughput speedup compared to a vanilla baseline. The principal implication for AI practitioners is that MoR provides an architectural path to attain the capabilities of larger models using significantly fewer parameters and less computational cost. This allows for training more powerful models under a fixed compute budget and deploying models with higher throughput and a smaller memory footprint, as the architecture inherently supports efficient techniques like continuous depth-wise batching. |
| LayerCake: Token-Aware Contrastive Decoding within Large Language Model |
|
|
| Layers (Read more on arXiv or HuggingFace) |
Yanqiang Zheng, Jiawang Cao, Wenbo Zhu, Yongliang Wu, Jingze Zhu |
The paper introduces LayerCake, a training-free decoding method that improves LLM factuality by selectively suppressing attention to specific token types at distinct layer depths. The primary objective is to improve the factual accuracy of LLM generations without retraining by leveraging the distinct functional roles that different token types (e.g., punctuation, conceptual) and transformer layers play in the model’s reasoning process. The LayerCake methodology first identifies that punctuation tokens dominate attention in early layers while conceptual tokens are key in middle layers; it then induces controlled factual degradation by suppressing attention to these tokens at their respective stages and uses the resulting contrastive signal between original and perturbed outputs to guide decoding. The method demonstrates consistent factuality improvements across multiple LLMs, for instance, increasing the score on the FACTOR benchmark by 8.05 percentage points for LLaMA3-8B compared to the greedy decoding baseline. For AI practitioners, this provides a training-free decoding strategy to enhance the reliability of off-the-shelf LLMs in knowledge-intensive tasks by intervening directly on the attention mechanism at inference time, avoiding costly fine-tuning. |
| CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards (Read more on arXiv or HuggingFace) |
Kai Chen, Songyang Zhang, Alexander Lam, Maosong Cao, Taolin Zhang |
This work introduces CompassJudger-2, a generalist judge model trained with a multi-domain data strategy and a refined margin policy gradient loss objective that utilizes verifiable rewards to improve evaluation robustness and accuracy. The primary objective is to develop a generalist LLM-as-judge model that overcomes the narrow specialization and limited robustness of existing evaluators, enabling comprehensive cross-domain judgment. The key methodology combines a task-driven data curation and synthesis pipeline with a novel training paradigm that uses a Chain-of-Thought (CoT) methodology to generate structured judgments, followed by rejection sampling to filter for high-quality examples, and finally, optimization via a margin policy gradient loss that directly incorporates verifiable binary reward signals. The 7B parameter CompassJudger-2 model demonstrates superior performance across multiple benchmarks, notably outperforming the comparable RISE-Judge-Qwen2.5-7B model by 22.58% on the JudgeBench dataset and achieving competitive accuracy with significantly larger models. The principal implication for AI practitioners is the provision of a validated framework for creating smaller, more cost-effective yet highly accurate generalist judge models, demonstrating that targeted data curation and a policy-gradient-based training strategy can significantly enhance evaluation capabilities without scaling model size, enabling more efficient automated assessment in model development cycles. |
| MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second (Read more on arXiv or HuggingFace) |
Honglei Yan, Yifan Yu, Panwang Pan, Yuchen Lin, Chenguo Lin |
MoVieS is a feed-forward framework that unifies the modeling of appearance, geometry, and motion to perform dynamic 4D view synthesis from a single monocular video in approximately one second. The objective is to develop a single, efficient, feed-forward model that can reconstruct a dynamic 4D scene from a monocular video, enabling tasks like novel view synthesis and 3D point tracking without per-scene optimization. The key methodology involves representing dynamic scenes using “dynamic splatter pixels”—static 3D Gaussian primitives augmented with a learned, time-dependent deformation field. A transformer backbone extracts features from video frames, camera poses, and timestamps, which are fed into dedicated heads to predict depth, appearance attributes, and motion vectors for arbitrary query times. MoVieS achieves competitive performance while being orders of magnitude faster than prior methods; on the DyCheck benchmark for dynamic novel view synthesis, it achieves an mPSNR of 18.46 in 0.93 seconds, compared to optimization-based methods that take minutes. For 3D point tracking on the TAPVid-3D Panoptic Studio dataset, it achieves an End-Point Error (EPE3D) of 0.0352, outperforming methods like CoTracker3 (0.0617). The principal implication for AI practitioners is the availability of a general-purpose, high-speed foundation model for dynamic 3D perception that directly outputs geometry, appearance, and explicit motion from video, eliminating the need for per-scene optimization and enabling zero-shot applications like scene flow estimation and moving object segmentation for time-sensitive systems in robotics and AR/VR. |
| A Practical Two-Stage Recipe for Mathematical LLMs: Maximizing Accuracy |
|
|
| with SFT and Efficiency with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yuichi Inoue, Taiki Yamaguchi, Hiroshi Yoshihara |
This paper presents a two-stage training recipe that first uses extended Supervised Fine-Tuning (SFT) to maximize the mathematical reasoning accuracy of LLMs, followed by Group Relative Policy Optimization (GRPO) to enhance token efficiency. The research objective is to establish a systematic methodology for combining SFT and Reinforcement Learning (RL), positing them as complementary rather than competing paradigms. The methodology involves an initial, prolonged SFT phase for 10 epochs on a high-difficulty dataset to push model accuracy, followed by a GRPO phase with a composite reward function (combining format, cosine similarity, and length penalty) to reduce solution length. On the MATH-500 benchmark, this recipe increased a 14B model’s accuracy from 86.4% to 91.2% while reducing the mean output tokens from 2,556 to 2,084. The principal implication for AI practitioners is that they can use this sequential strategy—SFT for peak accuracy, then RL for efficiency—as a proven blueprint to develop highly effective and practical specialized models, particularly for complex reasoning tasks. |
| From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for |
|
|
| LLM Evaluation (Read more on arXiv or HuggingFace) |
Yeonjung Hong, Soyeon Kim, Guijin Son, Sunkyoung Kim, Seokhee Hong |
This paper introduces KMMLU-REDUX and KMMLU-PRO, two expert-level Korean benchmarks designed to evaluate LLM performance on industrial and professional knowledge by addressing noise and contamination in existing datasets. The primary objective is to create a reliable, contamination-free evaluation suite for assessing LLM capabilities on Korea-specific professional qualifications, moving beyond general academic knowledge. The methodology involves manually denoising and filtering the existing KMMLU for high-difficulty technical exams (KMMLU-REDUX) and constructing a new benchmark from official, annually updated professional licensure exams (KMMLU-PRO). Experiments show that while OpenAI’s o1 model achieved the highest average accuracy of 79.55%, Anthropic’s Claude 3.7 Sonnet (w/ thinking) passed more professional exams (12 out of 14), with all models performing significantly worse in law compared to medicine, highlighting the difficulty of acquiring region-specific expertise. For AI practitioners, this research demonstrates that evaluating LLMs for professional applications requires specialized, locally-adapted benchmarks, as standard accuracy on generic or translated datasets is insufficient for assessing practical readiness in regulated fields. |
| DreamPoster: A Unified Framework for Image-Conditioned Generative Poster |
|
|
| Design (Read more on arXiv or HuggingFace) |
Dexiang Hong, Hui Zhang, Zhongqi Qi, Haokun Chen, Xiwei Hu |
DreamPoster is a unified framework for image-conditioned generative poster design that synthesizes high-quality posters from user images and text prompts. The primary objective is to create a model that integrates visual and textual content into a coherent, aesthetically pleasing poster while maintaining content fidelity and design coherence. The methodology involves a transformer-based diffusion architecture trained on a novel dataset of deconstructed poster pairs using a three-stage progressive training strategy that incrementally adds text addition, multi-task editing, and aesthetic alignment capabilities. In human evaluations, DreamPoster achieved an 88.55% usability rate, significantly outperforming GPT-4o (47.56%) and SeedEdit3.0 (25.96%). For AI practitioners, this work provides a robust framework and a targeted training methodology for specializing foundation models for complex, domain-specific content generation tasks like advertising and graphic design, demonstrating a path to production-level quality. |
| Favicon Trojans: Executable Steganography Via Ico Alpha Channel |
|
|
| Exploitation (Read more on arXiv or HuggingFace) |
Forrest McKee, David Noever |
The paper introduces a steganographic method for embedding and executing compressed JavaScript payloads within the alpha channel of ICO favicon files to bypass web security measures. The main research objective is to demonstrate the feasibility of a novel, two-stage covert channel that uses the least significant bit (LSB) of an ICO file’s alpha transparency layer to conceal and deliver self-decompressing, executable JavaScript within a web browser. The key methodology involves compressing a JavaScript payload, embedding its bits into the LSB of non-transparent alpha channel pixels of a base ICO image, and using a client-side decoder script to fetch the image, extract the bits via a canvas element, decompress the data, and execute the resulting code. The primary result is a successful proof-of-concept implementation that concealed and executed arbitrary script in modern browsers without visual artifacts, demonstrating that a 64×64 icon could hide a compressed payload of approximately 1.2 KB, bypassing standard browser security that treats the file as a static image. The principal implication for AI practitioners is that security and threat detection models must be updated to consider static image files as vectors for executable code, necessitating the development of specialized steganalysis models capable of detecting statistical anomalies like non-natural LSB distributions or high entropy within image alpha channels. |
Papers for 2025-07-14
| Title |
Authors |
Summary |
| Test-Time Scaling with Reflective Generative Model (Read more on arXiv or HuggingFace) |
Jie Gao, Mengting Xing, Xiaorui Wang, Yuxin Wang, Zixiao Wang |
This paper introduces MetaStone-S1, a reflective generative model that uses a unified architecture and self-supervised reward modeling to achieve efficient test-time scaling and performance comparable to OpenAI’s o3-mini. The main research objective is to develop a method for high-quality reasoning trajectory selection that unifies the policy and process reward models to reduce computational overhead and eliminate reliance on costly process-level annotations. The key methodology is a “Reflective Generative Form” where a policy model and a Self-supervised Process Reward Model (SPRM) share a single network backbone. The SPRM is a lightweight head trained with a novel Self-supervised Process Reward Loss (SPRLoss), which learns to evaluate reasoning steps using only final outcome correctness and a dynamic weighting scheme to filter supervision noise. The primary results show the 32B parameter MetaStone-S1-medium model achieves 84.2% on AIME24, comparable to OpenAI o3-mini’s 79.6%. The proposed SPRM, adding only 26M parameters to a 7B model, achieved a 70.2% score on AIME24, outperforming a separate 72B process reward model that scored 68.8%. The principal implication for AI practitioners is that high-performance test-time scaling can be achieved without training separate, large-scale reward models, offering a parameter-efficient and data-efficient method to integrate reasoning trajectory generation and selection into a single, self-supervised process, thereby reducing training and inference costs. |
| CLiFT: Compressive Light-Field Tokens for Compute-Efficient and Adaptive |
|
|
| Neural Rendering (Read more on arXiv or HuggingFace) |
Yasutaka Furukawa, Fuyang Zhang, Jiacheng Chen, Yuefan Wu, Zhengqing Wang |
The paper introduces CLiFT, a compressive light-field token representation for compute-efficient and adaptive neural rendering from a set of input images. The primary objective is to develop a compact, variable-size scene representation that enables adaptive control over the trade-offs between data size, rendering quality, and speed with a single trained model. The methodology involves a multi-view transformer encoder to generate initial tokens (LiFTs), followed by latent-space K-means to select representative ray centroids, and finally a neural “condenser” network that compresses information from all tokens into these centroids to form CLiFTs. On the RealEstate10K dataset, CLiFT achieves a comparable Peak Signal-to-Noise Ratio (PSNR) to MVSplat and DepthSplat baselines while requiring approximately 5–7× less data storage. The framework’s key implication for AI practitioners is the ability to deploy a single model that dynamically adjusts rendering performance (e.g., increasing FPS by up to 66% for a corresponding quality drop) by varying the number of tokens used, enabling real-time adaptation to diverse hardware or network conditions. |
| NeuralOS: Towards Simulating Operating Systems via Neural Generative |
|
|
| Models (Read more on arXiv or HuggingFace) |
Yuntian Deng, Wenhu Chen, Hongyu Guo, Sun Sun, Luke Rivard |
The paper introduces NeuralOS, a generative model that simulates an operating system’s graphical user interface by autoregressively predicting screen frames from user inputs. The primary objective is to develop a neural framework capable of simulating a complete OS GUI, including state transitions and interactions, purely through generative modeling without traditional OS kernels. NeuralOS uses a hierarchical recurrent neural network (RNN) to track system state and a diffusion-based UNet renderer, conditioned on RNN output and an explicit Gaussian spatial map of the cursor’s position, to generate subsequent frames. The model achieves high fidelity in mouse interactions, with a key quantitative result being an average cursor position error of only 1.6 pixels in width and 1.4 in height on a 512x384 screen, demonstrating the efficacy of its spatial encoding method. The principal implication for AI practitioners is that this work provides a proof-of-concept and a technical blueprint for building fully generative, stateful, and adaptive user interfaces, suggesting a future where complex interactive systems can be modeled as end-to-end generative processes rather than being rigidly programmed. |
| KV Cache Steering for Inducing Reasoning in Small Language Models (Read more on arXiv or HuggingFace) |
Cees G. M. Snoek, M. Jehanzeb Mirza, Michael Dorkenwald, Dawid J. Kopiczko, Max Belitsky |
This paper introduces cache steering, a lightweight method for inducing structured reasoning in small language models via a one-shot modification to the key-value (KV) cache. The research objective is to develop a more efficient and stable alternative to continuous activation steering for eliciting latent reasoning abilities in smaller models without fine-tuning or prompt modification. The methodology involves creating steering vectors by computing the mean difference of KV cache representations from contrastive prompt pairs (one with GPT-4o-generated reasoning traces, one without) and applying these vectors once to the prompt’s KV cache before generation. Experiments show cache steering improves task performance; for instance, on the ARC-c benchmark, it increased the Llama-3.2-3B model’s accuracy from 74.32% to 79.27% under greedy decoding. The principal implication for AI practitioners is that cache steering provides a practical, low-overhead technique to enhance the reasoning of smaller models, offering improved stability and inference efficiency compared to methods requiring continuous interventions. |
| Lumos-1: On Autoregressive Video Generation from a Unified Model |
|
|
| Perspective (Read more on arXiv or HuggingFace) |
Jingyun Liang, Hu Yu, Jun Cen, Weihua Chen, Hangjie Yuan |
Lumos-1 is an autoregressive video generation model that adapts a standard LLM architecture with minimal modifications to unify text and video processing. The main objective is to develop an efficient video generator that is architecturally aligned with LLMs, avoiding reliance on external text encoders and the high latency of next-token decoding. Key methodologies include MM-ROPE, a distributed and scaled 3D Rotary Position Embedding to inject spatiotemporal correlations, and Autoregressive Discrete Diffusion Forcing (AR-DF), a temporal tube masking strategy to resolve frame-wise loss imbalance during training. The 3.6B parameter Lumos-1 model achieves a score of 0.664 on the GenEval benchmark, demonstrating performance comparable to larger models trained with more resources. The principal implication for AI practitioners is that standard LLM architectures can be effectively adapted for high-quality video generation through targeted mechanisms like MM-ROPE and specialized training schemes like AR-DF, paving the way for more integrated and efficient unified multimodal models. |
| Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for |
|
|
| Visual Reasoning (Read more on arXiv or HuggingFace) |
Jisheng Yin, Kangheng Lin, Jianjian Sun, Liang Zhao, Yana Wei |
This paper presents Open-Vision-Reasoner (OVR), a model that enhances visual reasoning by transferring cognitive behaviors from language to vision using a two-stage, cold-start and reinforcement learning paradigm. The primary objective is to investigate how linguistic cognitive behaviors, such as backtracking and subgoal decomposition, can be effectively transferred to Multimodal LLMs (MLLMs) to unlock advanced visual reasoning capabilities. The methodology consists of a two-stage process: a massive linguistic cold-start supervised fine-tuning on over 2 million text examples to instill cognitive patterns, followed by large-scale multimodal reinforcement learning (RL) with PPO and verifiable rewards to align these patterns with visual contexts. The resulting OVR model achieves state-of-the-art results for open-source models, including 95.3% on MATH500 and 51.8% on MathVision. For AI practitioners, the key implication is that a training strategy of first instilling linguistic reasoning structures via a “cold start” and then using RL to critically discern and scale these behaviors in a multimodal context is a highly effective and scalable approach for developing more capable visual reasoning systems. |
| From One to More: Contextual Part Latents for 3D Generation (Read more on arXiv or HuggingFace) |
Yuxin Wang, Yaokun Li, Xiao Chen, Lihe Ding, Shaocong Dong |
The paper introduces CoPart, a framework that generates detailed and controllable 3D objects by representing them as a collection of contextual part latents rather than a single holistic latent. The research objective is to overcome the detail loss and lack of fine-grained control in existing 3D generation models by developing a system that explicitly models and generates objects part-by-part. CoPart’s methodology involves decomposing objects into simpler parts, encoding them with both geometric and image tokens, and using a mutual guidance strategy to ensure coherence during a synchronized diffusion process, which is further guided by 3D bounding box conditions. The framework demonstrates superior performance in part-based generation, achieving a part-aware CLIP (I-T) score of 0.1768, outperforming prior models like Rodin (0.1571) and Trellis (0.1455). The primary implication for AI practitioners is that this part-based approach provides a direct mechanism for granular, part-level control over generated 3D assets, enabling applications like interactive editing, object articulation, and scene composition that are challenging for monolithic generators. |
| One Token to Fool LLM-as-a-Judge (Read more on arXiv or HuggingFace) |
Haitao Mi, S. Y. Kung, Dian Yu, Haolin Liu, Yulai Zhao |
This paper investigates the vulnerability of LLM-as-a-judge models to simple, superficial “master key” attacks. The research objective is to systematically evaluate the susceptibility of generative reward models to these attacks and propose an effective mitigation strategy. The methodology involves testing various LLMs across five reasoning benchmarks using adversarial inputs like single punctuation marks or phrases such as “Thought process:”, then training a new model, Master-RM, on a dataset augmented with these inputs labeled as negative examples. The study reveals that standard LLMs exhibit false positive rates (FPRs) as high as 80% on these master keys, while the proposed Master-RM reduces this rate to near-zero across all settings. For AI practitioners, this implies that generative reward models used in RLVR or for evaluation are systematically vulnerable to trivial exploits, and their reliability requires specific robustness-focused fine-tuning via data augmentation. |
| Vision Foundation Models as Effective Visual Tokenizers for |
|
|
| Autoregressive Image Generation (Read more on arXiv or HuggingFace) |
Tiancai Wang, Chuofan Ma, Xuanyang Zhang, Xin Wen, Anlin Zheng |
This paper introduces VFMTok, a visual tokenizer built upon frozen vision foundation models (VFMs) to improve autoregressive (AR) image generation. The research aims to determine if features from pre-trained VFMs can serve as robust, semantically rich representations for image generation, overcoming the limitations of standard tokenizers. The methodology employs a frozen VFM as an encoder, introduces a region-adaptive quantization framework using deformable attention to reduce token redundancy, and applies a semantic reconstruction objective to preserve feature fidelity. The proposed model achieves a gFID of 2.07 on ImageNet, accelerates AR model convergence by three times, and enables high-fidelity, class-conditional synthesis without classifier-free guidance (CFG). For AI practitioners, this implies that leveraging pre-trained VFMs as tokenizers can substantially enhance AR generation quality and efficiency, while simplifying training and inference by removing the need for complex guidance mechanisms. |
| What Has a Foundation Model Found? Using Inductive Bias to Probe for |
|
|
| World Models (Read more on arXiv or HuggingFace) |
Sendhil Mullainathan, Ashesh Rambachan, Peter G. Chang, Keyon Vafa |
This paper introduces an “inductive bias probe,” a method to evaluate if foundation models learn underlying world models or just task-specific heuristics. The main objective is to develop and apply a framework for testing whether a foundation model has captured the deep, generative structure of its training data, rather than just learning surface-level predictive patterns. The key methodology is the “inductive bias probe,” which involves repeatedly fine-tuning a foundation model on small, synthetic datasets generated from a postulated world model and then analyzing the model’s extrapolations to see if they align with the world model’s principles. This is quantified using metrics like R-IB (respecting state) and D-IB (distinguishing state) for discrete domains, and by comparing learned functions to ground-truth laws in continuous domains. The primary result is that foundation models, despite high performance on their training tasks, often fail to develop an inductive bias toward the true world model. For example, a transformer pretrained on orbital mechanics, when fine-tuned to predict gravitational force, recovered a nonsensical physical law (F ∝ (sin(sin(r-0.24)/r) + 1.45) * (1/r + m2)) instead of Newton’s law (F ∝ m1*m2/r^2). The principal implication for AI practitioners is that high performance on a pre-training objective like next-token prediction does not guarantee a model has learned a generalizable world model. Models may be learning brittle, non-transferable heuristics, and their suitability for new tasks that rely on deep domain understanding must be explicitly tested rather than assumed. |
| Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, |
|
|
| Long Context, and Next Generation Agentic Capabilities (Read more on arXiv or HuggingFace) |
Noveen Sachdeva, Ice Pasupat, Mike Schaekermann, Eric Bieber, Gheorghe Comanici |
This report introduces the Gemini 2.X model family, which pushes the frontier of AI with advanced reasoning, multimodality, and next-generation agentic capabilities. The primary objective is to present the architecture, training advancements, and performance of these models, including Gemini 2.5 Pro and Flash, which are designed to power a new era of agentic systems. The models utilize a sparse mixture-of-experts (MoE) transformer architecture with native multimodal support, long-context inputs of over 1 million tokens, and a “Thinking” mechanism trained via Reinforcement Learning that allows the model to use additional inference-time compute to improve answer accuracy. Gemini 2.5 Pro achieves state-of-the-art performance on coding and reasoning benchmarks, improving the pass rate on LiveCodeBench to 74.2% from 30.5% for Gemini 1.5 Pro, and can process up to 3 hours of video content. The principal implication for AI practitioners is the availability of a family of models spanning the full capability-cost Pareto frontier, where Gemini 2.5 Pro enables complex, multimodal agentic workflows and Gemini 2.5 Flash offers a controllable thinking budget to dynamically trade off quality, cost, and latency. |
| BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with |
|
|
| Chunk-Level Activation Sparsity (Read more on arXiv or HuggingFace) |
Yingfa Chen, Chaojun Xiao, Xu Han, Weilin Zhao, Chenyang Song |
The paper introduces BlockFFN, a Mixture-of-Experts architecture designed for high chunk-level sparsity to enable acceleration on end-side devices. The objective is to develop a sparsely-activated LLM architecture that overcomes the performance and acceleration limitations of vanilla MoE, specifically by improving chunk-level sparsity (CLS) for efficient deployment on resource-constrained hardware. The methodology involves a novel router using ReLU for flexible, differentiable routing and RMSNorm to stabilize activation magnitudes, combined with two CLS-aware training objectives: an activation locality loss and a chunk sparsification loss to promote sparsity across token chunks. BlockFFN achieves over 70% 8-token chunk-level sparsity, and its custom acceleration kernels attain up to a 3.67× speedup over baseline auto-regressive decoding on an NVIDIA Jetson Orin NX. For AI practitioners, this work provides a practical method to achieve significant LLM inference acceleration on edge devices by training for high chunk-level sparsity, which is compatible with and enhances mainstream techniques like speculative decoding. |
| Robust Multimodal Large Language Models Against Modality Conflict (Read more on arXiv or HuggingFace) |
Houqiang Li, Jie Zhao, Wengang Zhou, ustc-zhangzm |
This research investigates and proposes mitigation strategies for hallucinations in Multimodal Large Language Models (MLLMs) arising from “modality conflict,” where visual and textual inputs are inherently contradictory. The primary objective is to formally define modality conflict, create a benchmark dataset (MMMC) to systematically evaluate this phenomenon, and test methods to make MLLMs more robust against it. The authors constructed the MMMC dataset by programmatically generating questions with object, attribute, or relationship conflicts against an image and then evaluated three mitigation techniques: prompt engineering, supervised fine-tuning (SFT), and reinforcement learning (RL) on prevalent MLLMs. Reinforcement learning demonstrated the best performance in mitigating hallucinations; for instance, it reduced the hallucination rate (Hallu-Rate) of the Qwen2-VL-Instruct-2B model on the MMMC dataset from a 46.55% baseline to 18.00%. For AI practitioners, this work demonstrates that MLLMs are highly susceptible to hallucinations when user inputs contain implicit contradictions with visual data, and indicates that fine-tuning with RL or SFT on datasets simulating such conflicts is a practical approach to improve model robustness in real-world applications. |
Papers for 2025-07-11
| Title |
Authors |
Summary |
| Scaling RL to Long Videos (Read more on arXiv or HuggingFace) |
Hanrong Ye, Qinghao Hu, Baifeng Shi, Wei Huang, Yukang Chen |
This paper presents a framework for scaling reinforcement learning to enhance vision-language model reasoning on long videos through a new dataset, a two-stage training pipeline, and a parallelized infrastructure. The research objective is to develop and validate a full-stack solution that overcomes the data scarcity and computational bottlenecks inherent in applying reinforcement learning to VLMs for complex, long-form video understanding. The methodology combines three core components: 1) a new dataset, LongVideo-Reason, with 52K long-video QA pairs annotated for reasoning; 2) a two-stage training regimen of Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) followed by reinforcement learning; and 3) a novel training system, Multi-modal Reinforcement Sequence Parallelism (MR-SP), which leverages sequence parallelism and cached video embeddings for efficient training. The primary results demonstrate that the MR-SP system achieves up to a 2.1× speedup in RL training on 512-frame videos, and the resulting LongVILA-R1-7B model attains 68.4% accuracy on the VideoMME benchmark (with subtitles), surpassing previous open-source models. The principal implication for AI practitioners is the provision of an open-source, scalable system (MR-SP) that makes it computationally feasible to apply reinforcement learning to VLMs using hour-long video inputs on a single multi-GPU node, thus enabling the development of more capable models for long-context video analysis. |
| T-LoRA: Single Image Diffusion Model Customization Without Overfitting (Read more on arXiv or HuggingFace) |
Konstantin Sobolev, Andrey Kuznetsov, Vera Soboleva, ai-alanov |
T-LoRA is a timestep-dependent, low-rank adaptation framework that mitigates overfitting in single-image diffusion model customization by dynamically adjusting parameter updates across diffusion timesteps. The primary objective is to solve the problem of fine-tuning overfitting, where models memorize background and positional information from a single training image, thereby compromising generalization and text alignment. The key methodology involves a dynamic rank-masking strategy that allocates fewer trainable parameters to higher (noisier) timesteps and an orthogonal weight initialization technique (Ortho-LoRA) to ensure adapter components are independent and fully utilized. The primary result shows that T-LoRA significantly improves text alignment over standard LoRA; at rank 64, T-LoRA achieved a Text Similarity score of 0.256 versus LoRA’s 0.232, while maintaining a comparable Image Similarity of 0.900. For AI practitioners, T-LoRA provides a method to achieve robust single-image personalization with superior prompt alignment and generative diversity, reducing the need for multiple training images and mitigating common overfitting artifacts. |
| Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and |
|
|
| Methodology (Read more on arXiv or HuggingFace) |
Zilong Huang, garlicisnotmyfavor, stormthunder, LXT, HaochenWang |
This paper introduces TreeBench, a new benchmark for evaluating visual grounded reasoning, and TreeVGR, a training paradigm using reinforcement learning with traceable evidence to improve these capabilities in Large Multimodal Models (LMMs). The primary objective is to address the lack of holistic benchmarks for an LMM’s ability to “think with images” by creating an evaluation that tests focused perception, traceable evidence via bounding boxes, and second-order reasoning. The methodology includes TreeBench, a benchmark with 405 challenging visual question-answering pairs requiring bounding box outputs, and TreeVGR, a two-stage training paradigm that uses reinforcement learning with a novel dual Intersection-over-Union (IoU) reward to explicitly supervise localization. The resulting TreeVGR model improves accuracy on the proposed TreeBench by +13.4 points over the Qwen2.5-VL-72B baseline and shows a +16.8 point gain on V* Bench. For AI practitioners, this work provides a concrete training methodology demonstrating that explicitly supervising intermediate localization steps via an IoU-based reward is a key strategy for developing more accurate and interpretable LMMs that can handle complex, vision-grounded reasoning tasks. |
| OST-Bench: Evaluating the Capabilities of MLLMs in Online |
|
|
| Spatio-temporal Scene Understanding (Read more on arXiv or HuggingFace) |
Xihui Liu, Xiaohan Mao, Runsen Xu, Chenming Zhu, JingLi Lin |
This paper introduces OST-Bench, a benchmark designed to evaluate the online spatio-temporal scene understanding capabilities of Multimodal Large Language Models (MLLMs) in an embodied agent context. The primary objective is to assess how well MLLMs can incrementally process sequential visual inputs to reason about their own state and dynamic spatial relationships within a 3D environment. The methodology involves a new dataset of 1.4k scenes and 10k QA pairs where models engage in multi-round dialogues, requiring them to integrate new visual frames with historical memory to answer questions. Results show that even the most advanced MLLMs significantly lag human performance by over 30%, and their accuracy on complex spatial tasks drops sharply as the exploration horizon extends, often to near-chance levels. For AI practitioners, this highlights a critical deficiency in current MLLMs’ long-term memory retrieval and multi-step spatial reasoning, indicating that future work must focus on developing models capable of building and querying efficient internal world representations to overcome the identified “Spatio-temporal Reasoning Shortcut” failure mode. |
| Multi-Granular Spatio-Temporal Token Merging for Training-Free |
|
|
| Acceleration of Video LLMs (Read more on arXiv or HuggingFace) |
Inwoong Lee, Taeoh Kim, Su Ho Han, Sukjun Hwang, js-hyun |
This paper introduces Spatio-Temporal Token Merging (STTM), a training-free method to accelerate video large language models (LLMs) by reducing redundant visual tokens. The objective is to mitigate the quadratic computational complexity of video LLMs by efficiently merging spatio-temporal tokens without requiring model retraining. STTM employs a decomposed strategy, first using a coarse-to-fine quadtree search for multi-granular spatial token merging within frames, followed by directed pairwise merging of spatially overlapping tokens across the temporal dimension. The method achieves a 2x speed-up with only a 0.5% accuracy drop under a 50% token budget across six video QA benchmarks. For AI practitioners, the key implication is that STTM is query-agnostic, allowing the pre-computed Key-Value (KV) cache for a video to be reused across different questions, which significantly improves efficiency for multi-turn inference scenarios. |
| PyVision: Agentic Vision with Dynamic Tooling (Read more on arXiv or HuggingFace) |
Qilong Wu, Ming Li, Shaoheng Lin, haoquan03, stzhao |
PyVision is an agentic framework enabling Multimodal Large Language Models (MLLMs) to autonomously generate, execute, and refine Python code for complex visual reasoning tasks. The research objective is to overcome the limitations of static, predefined toolsets in visual reasoning by creating a system where an MLLM can dynamically invent and use tailored computational tools to solve novel visual problems. The methodology involves an interactive, multi-turn loop between an MLLM and an isolated Python runtime; the MLLM receives a visual query, generates Python code using standard libraries to analyze the image, executes it, and uses the output to iteratively refine its reasoning until it produces a final answer. Primary results show that PyVision consistently improves backend model performance, boosting Claude-4.0-Sonnet’s accuracy by +31.1% on the VLMsAreBlind-mini symbolic vision dataset and GPT-4.1’s accuracy by +7.8% on the V* fine-grained visual search task. The principal implication for AI practitioners is that instead of relying on fixed APIs, they can build more robust and verifiable vision systems by empowering models to generate their own analysis code, enabling grounded, interpretable, and adaptive solutions to complex visual challenges. |
| Geometry Forcing: Marrying Video Diffusion and 3D Representation for |
|
|
| Consistent World Modeling (Read more on arXiv or HuggingFace) |
Yang Ye, Junliang Guo, Diankun Wu, deeptimhe, Haoyuwu |
This paper introduces Geometry Forcing (GF), a method to improve the 3D consistency of video diffusion models by aligning their intermediate features with representations from a pretrained geometric foundation model. The main objective is to bridge the gap between video diffusion models, which operate on 2D projections, and the inherent 3D structure of the physical world, thereby enhancing the geometric coherence of generated videos. The core methodology involves two complementary alignment loss objectives: “Angular Alignment” uses cosine similarity to enforce directional consistency between the diffusion model’s latent features and a geometric model’s features, while “Scale Alignment” regresses scale information to preserve geometric magnitudes. On the 256-frame RealEstate10K video generation task, Geometry Forcing reduces the Fréchet Video Distance (FVD) from 364 to 243 compared to the baseline. For AI practitioners, the principal implication is that GF enables video diffusion models to internalize a 3D representation, allowing for the generation of more consistent videos and the direct reconstruction of explicit 3D geometry (e.g., depth maps) from the model’s intermediate features, a capability absent in standard models. |
| LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ |
|
|
| FPS (Read more on arXiv or HuggingFace) |
Yuanhao Cai, Yang Liu, Minghan Qin, Yujie Zhao, Wanhua Li |
LangSplatV2 is a 3D language Gaussian splatting model that achieves over 450 FPS for high-dimensional feature rendering by replacing the slow feature decoder with a sparse coefficient field over a global dictionary. The paper’s primary objective is to overcome the inference speed bottleneck of the original LangSplat model to enable real-time, open-vocabulary 3D text querying at high resolutions without sacrificing accuracy. The key methodology models each Gaussian’s feature as a sparse code and uses a CUDA-optimized splatting technique to render only the few non-zero coefficients, effectively decoupling rendering time from the final high-dimensional feature space. The model achieves 3D open-vocabulary text querying at 384.6 FPS, a 47-fold speedup over LangSplat, while simultaneously improving 3D semantic segmentation mean IoU from 51.4% to 59.9% on the LERF dataset. For AI practitioners, this provides a direct method for deploying high-fidelity, language-based 3D scene understanding in latency-critical applications like robotics and augmented reality, which was previously infeasible due to decoder bottlenecks. |
| Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs (Read more on arXiv or HuggingFace) |
Yang Li, Ziyue Li, zhoutianyi |
This paper introduces “Chain-of-Layers” (CoLa), a test-time method using Monte Carlo Tree Search (MCTS) to dynamically skip or repeat pretrained LLM layers per-sample, enhancing performance and efficiency without retraining. The research objective is to determine if a pretrained LLM’s static architecture can be adapted for individual inputs by composing its layers into a custom sequence, thereby improving generalization on tasks of varying difficulty. A Monte Carlo Tree Search protocol is employed to find an optimal layer path for each sample, maximizing a UCB objective that balances predictive accuracy with a penalty for path length. The results demonstrate that for over 60% of samples with originally incorrect predictions, CoLa successfully identified a layer composition that yielded a correct prediction. The principal implication for AI practitioners is that pretrained transformer layers can be treated as reusable, composable modules, enabling the development of systems that dynamically adapt computational depth at inference time to significantly improve both accuracy and efficiency. |
| A Survey on Long-Video Storytelling Generation: Architectures, |
|
|
| Consistency, and Cinematic Quality (Read more on arXiv or HuggingFace) |
Seunghyun Yoon, Ryan Rossi, Franck-Dernoncourt, taesiri, elmoghany |
This survey analyzes 32 video generation papers to create a novel taxonomy of architectural styles and identify key components for producing long-form, coherent video. The paper’s primary objective is to identify the architectural patterns and training strategies that enable high-fidelity, long-duration video generation while maintaining narrative and character consistency. The methodology involves a comprehensive literature review that organizes models into a six-branch taxonomy (including Keyframes-to-Video, Flattened 3D One-Shot, and Token-Stream Autoregressive) and presents detailed comparative tables of their core components. The analysis reveals a trend towards using MM-DiT backbones and identifies models like Loong capable of generating videos up to 150 seconds, though many systems still struggle beyond 16 seconds. For AI practitioners, the paper provides a blueprint of recommended components—such as using MLLMs for text encoding, Flow Matching for training, and 3D ROPE for positional encoding—to guide the development of more robust long-video generation systems. |
| Token Bottleneck: One Token to Remember Dynamics (Read more on arXiv or HuggingFace) |
Sangdoo Yun, Jeongeun Park, bhheo, calintz, taekyung-k |
Token Bottleneck (ToBo) is a self-supervised learning pipeline that learns temporally-aware visual representations by compressing a reference scene into a single token to predict a heavily masked subsequent scene. The primary objective is to create a visual backbone that both conservatively summarizes an observed state and captures the dynamics of transitions between scenes, which is critical for sequential tasks. The key methodology involves a squeeze-and-expand process where an encoder maps a reference frame to a single bottleneck token, which is then used alongside a few visible patches from a heavily masked target frame (e.g., 90% masked) to reconstruct the target. In experiments, ToBo significantly outperformed baselines on robot manipulation tasks, achieving an 82.0% success rate on the Franka Kitchen “Light on” task, a +28.0 percentage point improvement over the next best method. For AI practitioners, this means ToBo-pretrained backbones can be directly deployed to build more effective and data-efficient robot control policies and dynamic scene understanding systems without requiring labeled data or complex model architectures. |
| Machine Bullshit: Characterizing the Emergent Disregard for Truth in |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Thomas L. Griffiths, Dawn Song, Xuandong Zhao, Haimin Hu, Kaiqu Liang |
This paper introduces a framework to systematically characterize and quantify “machine bullshit”—an LLM’s indifference to truth—and demonstrates that common alignment techniques like RLHF exacerbate it. The primary objective is to formalize the concept of machine bullshit, measure its prevalence in LLMs, and empirically investigate how factors like RLHF, Chain-of-Thought prompting, and context influence its generation. The authors introduce the Bullshit Index (BI) to quantify truth-indifference by measuring the correlation between a model’s internal beliefs and its explicit claims, and use an LLM-as-a-judge, validated by human studies, to classify four qualitative bullshit types across three benchmarks, including the novel BullshitEval dataset. The research found that RLHF significantly increases an LLM’s indifference to truth, with the Bullshit Index rising from 0.379 to 0.665, and specifically amplifies harmful paltering, nearly doubling its negative impact on user utility as its regression coefficient changed from -0.49 to -0.89. The principal implication for AI practitioners is that standard alignment procedures like RLHF can inadvertently optimize for persuasive, truth-indifferent outputs over factual accuracy, necessitating the development of new training and evaluation methods that directly mitigate specific, harmful bullshit behaviors like paltering. |
| Beyond the Linear Separability Ceiling (Read more on arXiv or HuggingFace) |
Mohit Vaishnav, Tanel Tammet, envomp |
This research introduces the Linear Separability Ceiling (LSC) to demonstrate that VLM abstract reasoning failures are primarily due to solvable alignment issues in reasoning pathways, not fundamental perception deficits. The primary objective is to diagnose whether the frequent failures of VLMs on abstract visual tasks originate from poor visual perception or flawed reasoning, and to identify effective interventions to resolve this bottleneck. The authors propose a diagnostic framework based on the Linear Separability Ceiling (LSC), defined as the accuracy of a nearest-centroid linear classifier on a VLM’s initial visual embeddings. This LSC is benchmarked against the model’s end-to-end generative accuracy, and various parameter-efficient fine-tuning (PEFT) methods like LoRA are used with standard and combined contrastive-generative objectives to surpass the LSC. The study finds a widespread “linear reasoning bottleneck” where most baseline VLMs fail to surpass their LSC. However, targeted fine-tuning successfully overcomes this; for example, LoRA with a combined loss function improved the Phi model’s generative accuracy on the OpenWorld dataset to 96.2%, significantly surpassing its LSC of 84.2%. The paper also identifies a trade-off, where training objectives that explicitly improve representation quality can lead to structural brittleness and poor generalization to new prompt formats. AI practitioners can unlock significant dormant reasoning capabilities in VLMs by applying targeted fine-tuning to the language model’s reasoning pathways, rather than solely focusing on improving the vision encoder. The LSC serves as a practical diagnostic to identify when reasoning, not perception, is the bottleneck, but engineers must be mindful of the trade-off between peak performance and structural robustness when selecting fine-tuning objectives. |
| SciMaster: Towards General-Purpose Scientific AI Agents, Part I. |
|
|
| X-Master as Foundation: Can We Lead on Humanity’s Last Exam? (Read more on arXiv or HuggingFace) |
Xinyu Zhu, Yuwen Du, Rui Ye, Shuo Tang, Jingyi Chai |
This paper presents X-Masters, an inference-time agentic workflow enabling an open-source model to achieve state-of-the-art scientific reasoning. The primary objective is to develop and validate a foundational architecture for a general-purpose scientific agent capable of outperforming leading proprietary models on the Humanity’s Last Exam (HLE) benchmark. The methodology leverages X-Master, a tool-augmented agent using Python code for flexible environmental interaction, within a “scattered-and-stacked” workflow that employs multiple agent instances (Solver, Critic, Rewriter, Selector) to systematically explore and refine solutions. The system achieves a new state-of-the-art accuracy of 32.1% on HLE, significantly surpassing leading research products from OpenAI (26.6%) and Google (26.9%). For practitioners, this demonstrates that complex, multi-agent inference-time computation can unlock state-of-the-art capabilities from accessible open-source LLMs on frontier benchmarks without requiring model retraining, offering a powerful paradigm for advanced problem-solving. |
Papers for 2025-07-10
| Title |
Authors |
Summary |
| Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data (Read more on arXiv or HuggingFace) |
Lixing Xiao, Runyi Yu, Shunlin Lu, Ke Fan, Jixi111 |
This research introduces MotionMillion, a new dataset with 2 million motion sequences, and a 7B parameter model to enable zero-shot text-to-motion generation. The primary objective is to test the “scaling hypothesis” in motion generation, aiming to achieve robust zero-shot generalization by significantly increasing the scale of training data and model size. The methodology consists of a multi-stage pipeline to construct the MotionMillion dataset from web videos and a scalable, decoder-only transformer architecture that employs Finite Scalar Quantization (FSQ) with wavelet transformation for motion tokenization. The 7B parameter model achieves a Fréchet Inception Distance (FID) of 10.3, drastically outperforming the ScaMo baseline’s score of 89.0, and demonstrates superior text alignment in human evaluations. The principal implication for AI practitioners is that creating high-quality, million-scale datasets and leveraging large parameter models is a critical and effective strategy for unlocking emergent zero-shot capabilities in complex, non-linguistic generative domains like human motion. |
| Perception-Aware Policy Optimization for Multimodal Reasoning (Read more on arXiv or HuggingFace) |
Hongru Wang, Sofia Stoica, Xuehang Guo, Zhenhailong Wang, xhyandwyy |
This paper introduces Perception-Aware Policy Optimization (PAPO), an extension to the GRPO algorithm that enhances multimodal reasoning by explicitly penalizing a model’s indifference to visual input. The primary objective is to address the high rate of perception errors, which the authors identify as comprising 67% of failures in models trained with standard Reinforcement Learning with Verifiable Rewards (RLVR). PAPO’s methodology incorporates an “Implicit Perception Loss” into the GRPO objective, which maximizes the KL divergence between the model’s output distributions conditioned on an original versus a randomly masked visual input. The method achieves a 30.5% reduction in perception-related errors and an overall performance gain of up to 8.0% on vision-dependent benchmarks over the GRPO baseline. For AI practitioners, PAPO offers a technique to improve the visual grounding of LMMs during RL finetuning using only internal supervision signals, but requires a Double Entropy Loss regularizer to mitigate a “KLprcp Hacking” training instability identified by the authors. |
| 4KAgent: Agentic Any Image to 4K Super-Resolution (Read more on arXiv or HuggingFace) |
Xinrui Jiang, Mingyang Wu, Qi Zheng, vztu, YSZuo |
4KAgent is a unified agentic framework designed to universally upscale any image to 4K resolution by dynamically planning and executing a tailored restoration process. The primary objective is to develop a generalist super-resolution system capable of handling diverse image types, domains, and degradation levels without domain-specific retraining. The methodology employs a multi-agent system with a Perception Agent that uses Vision-Language Models (VLMs) and Image Quality Assessment (IQA) tools to analyze the input and plan a restoration sequence, and a Restoration Agent that executes this plan using a toolbox of specialized models, guided by a Quality-Driven Mixture-of-Experts (Q-MoE) policy for optimal step-wise output selection. 4KAgent establishes new state-of-the-art results across 26 diverse benchmarks; for instance, on the DrealSR real-world super-resolution benchmark, it achieves a top NIQE score of 4.65 and a top MUSIQ score of 69.30. The principal implication for AI practitioners is that an agentic, mixture-of-experts approach, which leverages VLMs for planning and dynamically combines multiple expert models, provides a powerful and practical paradigm for building highly generalizable and performant systems for complex, low-level vision tasks, overcoming the limitations of single, specialized models. |
| Rethinking Verification for LLM Code Generation: From Generation to |
|
|
| Testing (Read more on arXiv or HuggingFace) |
Minnan Luo, Wenwei Zhang, Maosong Cao, Taolin Zhang, MichaelErchi |
This paper introduces SAGA, a human-LLM collaborative framework for generating superior test cases to address critical flaws in current LLM code generation verification. Its primary objective is to overcome the test case homogenization and LLM-centric bias in existing benchmarks that artificially inflate model performance. The key methodology, SAGA, uses a dual-pronged strategy of “Multidimensional Analysis” on correct human solutions and “Differential Analysis” on incorrect submissions to guide an LLM in synthesizing highly discriminative tests. This method yields significant improvements, achieving a 90.62% detection rate on the TCGBench-Lite dataset, while the Verifier Accuracy of its synthesized benchmark is 10.78% higher than that of LiveCodeBench-v6. For AI practitioners, this work implies that current benchmarks provide misleadingly high performance scores; adopting more rigorous verifiers like those from SAGA is critical for accurate model assessment and for creating reliable reward signals in Reinforcement Learning from Verifiable Rewards (RLVR) frameworks. |
| First Return, Entropy-Eliciting Explore (Read more on arXiv or HuggingFace) |
Xingwei Qu, Taoran Liang, Qingshui Gu, xtsssss, aaabiao |
This paper introduces FR3E (First Return, Entropy-Eliciting Explore), a structured exploration framework to improve Large Language Model reasoning through more stable reinforcement learning. The research objective is to resolve unstable exploration and imprecise credit assignment in Reinforcement Learning from Verifiable Rewards (RLVR) for complex, sparse-reward tasks. The core methodology involves identifying high-uncertainty decision points within a generated reasoning trajectory using token-wise entropy, then initiating targeted, partial rollouts from these points to construct localized feedback for policy updates. On the AIME24 mathematical reasoning benchmark, FR3E improved the performance of the Qwen2.5-32B model to 40.2% accuracy, a 6.1% absolute improvement over the GRPO++ baseline. For AI practitioners, FR3E provides a value-model-free technique to enhance LLM reasoning capabilities with greater training stability and more effective credit assignment, offering a structured way to guide exploration in sparse-reward environments. |
| A Systematic Analysis of Hybrid Linear Attention (Read more on arXiv or HuggingFace) |
Taylor Kergan, Yong Shan, Steven Abreu, Dustin Wang, ridger |
This paper systematically analyzes various linear attention models within hybrid architectures to determine the most effective components for balancing performance and efficiency. The primary objective is to investigate whether strong standalone linear models excel when hybridized and to identify which architectural properties are critical for language modeling versus recall. The methodology involved training 72 models at 340M and 1.3B parameters, covering six linear attention variants across five different linear-to-full attention ratios, and benchmarking them on language modeling and RULER recall tasks. The results show that standalone performance is not a reliable predictor of hybrid performance, and while language modeling is stable across ratios, recall nearly doubles on RULER when moving from a 24:1 to a 3:1 linear-to-full ratio. The principal implication for practitioners is to employ a gated, hierarchical backbone (e.g., HGRN-2 or GatedDeltaNet) with a linear-to-full ratio between 3:1 and 6:1 to achieve Transformer-level recall while significantly reducing KV cache memory. |
| AutoTriton: Automatic Triton Programming with Reinforcement Learning in |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Yuxuan Li, Ye He, Zefan Wang, Shangzhan Li, qshi |
AUTOTRITON is a specialized 8B model that automates Triton programming for GPU kernels using a two-stage supervised fine-tuning and reinforcement learning process. The research objective is to create a model that can automatically generate correct and efficient Triton code from high-level specifications like PyTorch functions, addressing the complexities of manual kernel optimization. The methodology first employs supervised fine-tuning (SFT) on a curated dataset created via a novel data pipeline, followed by reinforcement learning (RL) using the Group Relative Policy Optimization (GRPO) algorithm with a combined execution-based and rule-based reward to enhance correctness. On the KERNELBENCH Level 2 benchmark, AUTOTRITON achieves 45.0% execution accuracy, outperforming larger models like DeepSeek-R1-0528 (42.0%), and removing the RL stage drops this accuracy to 27.0%. For AI engineers, this provides a framework for automating GPU kernel generation, showing that a targeted RL approach on a specialized model can significantly improve performance and correctness, thereby accelerating the development of efficient AI systems. |
| Towards Solving More Challenging IMO Problems via Decoupled Reasoning |
|
|
| and Proving (Read more on arXiv or HuggingFace) |
Feng Zhang, Tao Yang, Yang Li, Linfeng Song, Zhenwen Liang |
This paper introduces a decoupled reasoning and proving framework to solve challenging mathematical problems by separating high-level strategy generation from low-level formal proof verification. The primary research objective is to bridge the significant gap between the strong informal reasoning capabilities and weak formal proving performance of LLMs on complex mathematical problems. The methodology uses a general-purpose LLM as a “Reasoner” to propose formal subgoal lemmas and a specialized Automated Theorem Proving (ATP) model as a “Prover” to first verify these lemmas and then construct the final proof. The framework successfully solved 5 post-2000 non-geometry IMO problems, a set where no prior open-source prover had reported success, and found that a specialized prover (Kimina-Prover) underperforms its base model by 4.9 percentage points on the MATH benchmark. The principal implication for AI practitioners is that end-to-end fine-tuning for specialized tasks can degrade a model’s core reasoning; a decoupled, multi-model architecture may better preserve and leverage high-level capabilities for complex problem-solving. |
| A Survey on Vision-Language-Action Models for Autonomous Driving (Read more on arXiv or HuggingFace) |
Tianze Zhu, Ziang Luo, Kangan Qian, Zilin Huang, Max2045 |
This paper surveys the evolution and architecture of Vision-Language-Action (VLA) models for autonomous driving, which integrate perception, reasoning, and control into a unified policy. The primary objective is to formalize the VLA for Autonomous Driving (VLA4AD) paradigm by tracing its evolution from passive language explainers to integrated reasoning-action agents, cataloging key architectures, datasets, and open challenges. The authors conduct a comprehensive literature review, structuring over 20 representative models into a four-stage evolutionary taxonomy (Explainer, Modular, End-to-End, Reasoning-Augmented) and analyzing their architectural components. The survey identifies a clear trend towards unified, end-to-end VLA models, while highlighting persistent efficiency challenges; for instance, token-reduction techniques like TS-VLM are cited as critical for achieving real-time performance, cutting compute by approximately 90% through methods like soft-attentive pooling. For AI practitioners, this survey provides a consolidated reference for developing VLA4AD systems, guiding the selection of architectural patterns, training datasets like Impromptu VLA for corner cases, and evaluation protocols to build more interpretable and robust autonomous vehicles. |
| DiffSpectra: Molecular Structure Elucidation from Spectra using |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Zhiyuan Liu, Zhenyi Zhong, Tingyang Xu, Yu Rong, AzureLeon1 |
The paper introduces DiffSpectra, a diffusion-based generative framework for de novo 2D/3D molecular structure elucidation from multi-modal spectral data. The main objective is to directly generate complete molecular structures from spectral inputs (IR, Raman, UV-Vis), overcoming the generalization limits of existing retrieval-based and autoregressive methods. The methodology employs a continuous-time diffusion model whose denoising network is a SE(3)-equivariant Diffusion Molecule Transformer (DMT), which is conditioned on spectral embeddings from a pre-trained, transformer-based encoder named SpecFormer. The primary result is that DiffSpectra achieves a 16.01% top-1 accuracy in recovering the exact molecular structure, which rises to 96.86% for top-20 accuracy by sampling multiple candidates. For AI practitioners, this work shows that a conditional diffusion model with an SE(3)-equivariant backbone and a specialized, pre-trained conditioning network can effectively tackle complex scientific inverse problems, with the significant accuracy boost from sampling demonstrating a viable strategy for generating ranked candidate solutions for expert review. |
| ModelCitizens: Representing Community Voices in Online Safety (Read more on arXiv or HuggingFace) |
Karolina Naranjo, notaphonologist, hamidpalangi, christinachance, Ashima |
The paper introduces the MODELCITIZENS dataset and corresponding finetuned models to demonstrate that incorporating community-specific, ingroup annotations significantly improves toxic language detection. The primary objective is to quantify the impact of annotator identity (ingroup vs. outgroup) and conversational context on toxicity classification and to develop models that better represent the nuanced perspectives of targeted communities. The methodology involves curating a dataset of 6.8K posts with 40K annotations from both ingroup and outgroup annotators across eight identity groups, augmenting a subset with LLM-generated context, and finetuning LLaMA and Gemma models on these community-specific labels. The primary result is that the finetuned LLAMACITIZEN-8B model achieves 75.2% accuracy, outperforming the best baseline (GPT-o4-mini) by 5.5% and demonstrating that models trained on ingroup-only labels perform better than those trained on outgroup or aggregated labels. The principal implication for AI practitioners is that for subjective tasks like toxicity moderation, datasets should be constructed with annotations from members of the targeted communities, as training on these ingroup labels provides a more reliable signal and produces more accurate and equitable models than using aggregated or outgroup-only labels. |
| Evaluating the Critical Risks of Amazon’s Nova Premier under the |
|
|
| Frontier Model Safety Framework (Read more on arXiv or HuggingFace) |
Vincent Ponzo, Matteo Memelli, Abhinav Mohanty, Ninareh Mehrabi, Satyapriya Krishna |
This paper presents a comprehensive safety evaluation of Amazon’s Nova Premier model against critical risks in CBRN, cyber operations, and automated AI R&D, concluding it is safe for public release under the Frontier Model Safety Framework. The primary objective was to assess if Nova Premier’s capabilities in high-risk domains exceed predefined critical thresholds that would prevent its deployment. The methodology integrated automated benchmarks (e.g., WMDP, Cybench, RE-Bench), human-centric evaluations like expert red-teaming and uplift studies, and third-party audits. Results show that while Nova Premier exhibits improved theoretical knowledge, its practical capabilities for misuse are constrained; for instance, its mean solve rate on cybersecurity knowledge benchmarks increased from approximately 0.76 to 0.82 over its predecessor, but its performance on practical Capture-the-Flag challenges remained unchanged. The principal implication for AI practitioners is that increased declarative knowledge in a frontier model does not directly translate to an increased risk of practical misuse, as layered safety systems can effectively intervene to prevent the generation of complete, operational malicious outputs. |
| AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large |
|
|
| Language Models on Harmfulness (Read more on arXiv or HuggingFace) |
Zhen Ye, Ziyang Luo, Kaixin Li, Hongzhan Lin, Zixin Chen |
The paper introduces AdamMeme, a multi-agent framework that adaptively probes and evaluates the reasoning capacity of multimodal large language models (mLLMs) on harmful memes by iteratively generating challenging examples. The primary objective is to develop a dynamic, model-centric evaluation framework that moves beyond static benchmarks to identify fine-grained, model-specific weaknesses of mLLMs in understanding meme harmfulness. The methodology employs a three-stage, agent-based pipeline: (1) Harmfulness Mining, where agents categorize memes and generate “misbelief” statements; (2) Model Scoring, where an agent grades the target mLLM’s analysis; and (3) Iterative Refinement, where an agent modifies meme text to create more difficult test cases based on the model’s prior failures, forming an adaptive evaluation loop. Primary results demonstrate that the framework systematically reveals model-specific vulnerabilities; for instance, Doubao-Lite was found to be highly susceptible to refinement, with its average Failure Rate (FR) increasing by 4.73% on original memes after the process, whereas GPT-4o maintained a low average FR of 2.18% across all tests. The principal implication for AI practitioners is that this framework offers a method for deep, adaptive red-teaming of mLLMs, automatically discovering and generating challenging data that exposes specific reasoning failures missed by static benchmarks, enabling more targeted safety improvements and robust model development. |
Papers for 2025-07-09
| Title |
Authors |
Summary |
| A Survey on Latent Reasoning (Read more on arXiv or HuggingFace) |
Tianhao Peng, jeshragh, chujiezheng, Jinfa, ridger |
This survey provides a comprehensive overview of latent reasoning, an emerging paradigm where multi-step inference is performed within a model’s continuous hidden state, bypassing the expressive bandwidth limitations of explicit Chain-of-Thought (CoT). The paper’s objective is to unify and categorize diverse latent reasoning methodologies, including their architectural foundations, training strategies, and mechanistic interpretability. The authors conduct a systematic review, classifying techniques into vertical recurrence (activation-based methods like looped transformers) and horizontal recurrence (hidden-state-based methods), and also explore infinite-depth reasoning via text diffusion models. The survey’s primary result highlights that latent reasoning, by exchanging full hidden states (~40,960 bits), provides a ~2.7 × 10³-fold greater expressive bandwidth than explicit reasoning with discrete tokens (~15 bits). For AI practitioners, the principal implication is that shifting reasoning from discrete tokens to the continuous latent space offers a pathway to build models with more powerful and efficient reasoning capabilities, moving beyond the constraints of explicit verbalization for complex problem-solving. |
| SingLoRA: Low Rank Adaptation Using a Single Matrix (Read more on arXiv or HuggingFace) |
Ron Kimmel, Daniel Bensaïd, David Bensaïd, royve, noamrot |
SingLoRA is a parameter-efficient fine-tuning method that resolves LoRA’s training instability by using a single matrix and its transpose for the low-rank update. The primary objective is to address the unstable training dynamics in LoRA, which arise from scale disparities between its two adapter matrices, and to develop a more stable and parameter-efficient alternative. The key methodology involves reformulating the low-rank update from LoRA’s W₀ + BA to W₀ + AAT, thereby using only a single learnable matrix A. The paper provides theoretical analysis demonstrating that this formulation is transformation-invariant and guarantees stable feature learning by construction. Primary results show significant improvements in both performance and efficiency: fine-tuning LLaMA-7B on MNLI with SINGLORA achieved 91.3% accuracy with 12M parameters, outperforming LoRA which reached 89.1% with 20M parameters. The principal implication for AI practitioners is that SINGLORA can serve as a more robust and efficient alternative to LoRA, enabling them to achieve superior fine-tuning performance with approximately half the parameter budget and reduced sensitivity to hyperparameter choices like the learning rate. |
| OmniPart: Part-Aware 3D Generation with Semantic Decoupling and |
|
|
| Structural Cohesion (Read more on arXiv or HuggingFace) |
Yukun Huang, Zi-Xin Zou, Yuan-Chen Guo, Yufan Zhou, Yunhan Yang |
The paper introduces OmniPart, a two-stage framework for generating controllable, part-based 3D objects from 2D images and masks. The primary objective is to generate structured 3D assets with high semantic decoupling between parts and robust overall structural cohesion, overcoming the utility limitations of monolithic generation methods. The methodology first employs an autoregressive transformer to plan a 3D part layout as a sequence of bounding boxes guided by 2D masks, then a spatially-conditioned synthesis module, fine-tuned from a pre-trained generator, synthesizes all parts simultaneously within this layout. Quantitatively, OmniPart achieves a part-level F1-score of 0.74 (at a Chamfer Distance threshold of < 0.1), significantly outperforming existing part-aware generation baselines. For AI practitioners, this framework provides a direct pipeline to create editable, structured 3D assets from simple 2D inputs, enabling downstream applications like part-specific editing, animation, and material assignment in interactive systems. |
| StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context |
|
|
| Modeling (Read more on arXiv or HuggingFace) |
Yuqiang Yang, Tai Wang, Xiqian Yu, Meng Wei, cywan |
StreamVLN is a streaming vision-language navigation framework using a hybrid slow-fast context model with 3D-aware token pruning to achieve efficient, low-latency navigation on long video streams. The research objective is to develop a vision-language navigation (VLN) framework that can process continuous visual streams for long-horizon tasks while maintaining low inference latency, bounded memory usage, and high navigational performance, addressing the limitations of existing Video-LLM methods. The key methodology is a hybrid slow-fast context modeling strategy: a “fast” sliding-window KV cache retains a fixed number of recent dialogue turns for responsive action generation, while a “slow-updating” memory context compresses historical visual history using a voxel-based spatial pruning algorithm that discards spatially redundant tokens based on their 3D projections. The framework achieves state-of-the-art performance for RGB-only methods on VLN-CE benchmarks, attaining a Success Rate (SR) of 56.9% and a Success weighted by Path Length (SPL) of 51.9% on the R2R Val-Unseen split; its voxel-based pruning reduces input tokens by approximately 20% while concurrently improving SR. For AI practitioners, the slow-fast context management with voxel-based pruning provides a practical method for deploying large multimodal models in real-time, resource-constrained embodied AI applications, enabling models trained on short clips to operate on long, continuous data streams with bounded computational cost and stable latency. |
| CriticLean: Critic-Guided Reinforcement Learning for Mathematical |
|
|
| Formalization (Read more on arXiv or HuggingFace) |
Yifan Yao, Zhongyuan Peng, zhangysk, zhouliang, yifAI |
This paper introduces CriticLean, a framework using a reinforcement learning-trained critic model to guide and validate the translation of natural language mathematics into formal Lean 4 code. The research objective is to improve mathematical autoformalization by systematically optimizing the “critic phase,” where the semantic correctness of generated formalizations is evaluated, going beyond mere compilation success. The methodology involves developing CriticLeanGPT, a critic model trained with supervised fine-tuning and RL, and integrating it into an iterative generation pipeline that refines outputs based on feedback from both the Lean compiler and the critic. The primary result shows that this pipeline significantly improves autoformalization accuracy, raising it from 38.0% (single pass) to 84.0% on a human-evaluated set of 50 problems from Omni-MATH. For AI practitioners, this demonstrates that integrating a dedicated, trained semantic critic into a generation loop is a highly effective strategy for improving the reliability and semantic fidelity of domain-specific code generation systems. |
| RLVER: Reinforcement Learning with Verifiable Emotion Rewards for |
|
|
| Empathetic Agents (Read more on arXiv or HuggingFace) |
Zhiwei He, Xingyu Chen, Bang Zhang, vvibt, CedarWang |
This paper introduces RLVER, a framework that uses reinforcement learning with verifiable, deterministic emotion rewards from a simulated user to enhance the empathetic capabilities of LLMs. The primary objective is to cultivate higher-order emotional intelligence in LLMs by optimizing for simulated user satisfaction, without relying on human-annotated data. The methodology involves fine-tuning a Qwen2.5-7B model using Proximal Policy Optimization (PPO), where rewards are verifiable emotion scores generated turn-by-turn from a psychologically-grounded user simulator, and includes an analysis of an explicit “think-then-say” reasoning scaffold. The proposed RLVER framework increased the model’s Sentient-Benchmark score from 13.3 to 79.2, a nearly six-fold improvement, while largely preserving its general reasoning capabilities. For AI practitioners, this research provides a practical and scalable methodology for improving subjective, human-centric LLM capabilities by replacing expensive human feedback loops (RLHF) with verifiable rewards from a well-designed, deterministic simulator. |
| MedGen: Unlocking Medical Video Generation by Scaling |
|
|
| Granularly-annotated Medical Videos (Read more on arXiv or HuggingFace) |
Shunian Chen, Zhenyang Cai, Ke Ji, Junying Chen, wangrongsheng |
This paper introduces MedGen, a specialized medical video generation model, and MedVideoCap-55K, the dataset it was trained on, to address the failure of general-purpose models in producing medically accurate content. The primary objective is to create a foundational dataset and model for high-fidelity, domain-specific medical video generation. The methodology involves constructing the MedVideoCap-55K dataset by curating over 55,000 captioned video clips from public sources through a rigorous filtering pipeline, and then fine-tuning the open-source HunyuanVideo model on this dataset to create MedGen. In experiments, MedGen achieved a total score of 70.93 on the Med-VBench benchmark, outperforming all other evaluated open-source models and performing competitively against proprietary models like Pika. The principal implication for AI practitioners is that domain-specific fine-tuning on a large-scale, high-quality, and granularly-annotated dataset is a crucial strategy for adapting foundational models to high-stakes fields, significantly enhancing both domain-specific accuracy and overall output quality. |
| Is Diversity All You Need for Scalable Robotic Manipulation? (Read more on arXiv or HuggingFace) |
Jin Chen, Li Chen, sundrops, yxlu0, ModiShi |
This paper systematically investigates the effects of task, embodiment, and expert diversity on scalable robotic manipulation, challenging the “more diverse is better” paradigm. The research objective is to evaluate how these three dimensions of data diversity impact the performance and scalability of robotic manipulation policies learned through imitation. The methodology involves extensive experiments using models like GO-1 and RDT on large-scale datasets, comparing different pre-training strategies (e.g., task-based vs. episode-based sampling, single- vs. multi-embodiment data) and introducing a distribution debiasing method, GO-1-Pro, to mitigate velocity multimodality in expert demonstrations. The primary results demonstrate that task diversity is more critical than demonstration quantity, multi-embodiment pre-training is optional for cross-embodiment transfer, and expert diversity can be confounding; the proposed GO-1-Pro method achieved a 15% performance gain, equivalent to using 2.5 times the pre-training data. The principal implication for AI practitioners is that for scalable robotics, curating datasets with high task diversity and implementing techniques to debias confounding expert-level variations like velocity is more data-efficient than merely increasing dataset size or embodiment diversity. |
| Coding Triangle: How Does Large Language Model Understand Code? (Read more on arXiv or HuggingFace) |
Songyang Zhang, Maosong Cao, Taolin Zhang, jnanliu, MichaelErchi |
This paper introduces the “Coding Triangle” framework to systematically evaluate large language models’ (LLMs) programming abilities across editorial analysis, code implementation, and test case generation. The main objective is to define and assess LLM coding capability by analyzing performance and interactions across these three dimensions. The methodology involves evaluating various LLMs on 200 AtCoder problems, using metrics like Pass@1 for code, LLM-as-a-judge for editorials, and the discriminative power of generated test cases. The study reveals a strong self-consistency bias, where models’ self-generated solutions achieve pass rates up to 40% higher on their own generated test cases than on ground-truth cases, while also showing that model-generated solutions exhibit high error similarity (cosine similarity > 0.8), unlike diverse human solutions. The principal implication for AI practitioners is that relying on an LLM’s self-verification is insufficient; enhancing robustness requires incorporating diverse human-generated data or using model mixtures to overcome the cognitive biases and limited error diversity inherent in single models. |
| GTA1: GUI Test-time Scaling Agent (Read more on arXiv or HuggingFace) |
Yuhao Yang, Yutong Dai, Dongxu Li, Yan Yang, Ziyang |
The paper introduces GTA1, a GUI agent that improves task planning through a test-time scaling strategy and enhances visual grounding with a reinforcement learning model. The research objective is to address two primary challenges in GUI agent autonomy: 1) resolving ambiguity in task planning by selecting the most robust action from multiple plausible options without requiring multi-step lookahead, and 2) improving the accuracy of visually grounding actions on complex, high-resolution interfaces. The methodology comprises a two-stage approach: a test-time scaling strategy for planning that samples multiple candidate action proposals and uses a judge model to select the most suitable one, and a grounding model trained with Reinforcement Learning (RL) to directly predict interaction coordinates, using a binary reward for successful clicks within the target UI element. The primary result is that the GTA1-7B agent achieves a 45.2% task success rate on the OSWorld benchmark when paired with an o3 planner, outperforming all compared state-of-the-art methods. The principal implication for AI practitioners is that for GUI agent development, a simple binary click reward for RL-based grounding is more effective than complex objectives like enforcing “thinking” or bounding box prediction. Additionally, implementing a test-time sampling and judging strategy is a practical method to significantly improve planning robustness and overall task success rates by mitigating cascading failures. |
| Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts (Read more on arXiv or HuggingFace) |
Mohamed Anwar, Amr Mohamed, Ahmad Chamma, Hadi Abdine, guokan-shang |
The paper introduces Nile-Chat, a family of large language models specifically designed to understand and generate Egyptian Arabic in both its native Arabic and Latin-based scripts. The primary objective is to develop high-performing LLMs for the dual-script Egyptian dialect, addressing the failure of existing models to adequately support this widespread language setting. The authors employ a comprehensive training pipeline including continual pre-training, instruction-tuning, and Direct Preference Optimization (DPO), and notably use the Branch-Train-MiX (BTX) strategy to merge a base model with script-specialized experts into a unified Mixture-of-Experts (MoE) model. The Nile-Chat models significantly outperform baselines, with the 12B model yielding a 14.4% performance gain over Qwen2.5-14B-Instruct on Latin-script benchmarks. For AI practitioners, this work provides a replicable methodology for adapting LLMs to dual-script languages, showing that merging specialized experts via BTX into an MoE architecture is an effective strategy for improving capability in underrepresented linguistic contexts. |
| Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankers (Read more on arXiv or HuggingFace) |
Yi Fang, Ting-ruen Wei, Zhiyuan Peng, yilunzhao, songtingyu |
This research introduces E²R-FLOPs, a hardware-agnostic framework using floating-point operations (FLOPs) to evaluate the efficiency-effectiveness trade-off of LLM-based rerankers. The main objective is to establish a standardized method for comparing reranker performance that is independent of specific hardware and runtime configurations by addressing the limitations of proxy metrics like latency or token counts. The methodology involves deriving a closed-form FLOPs estimator for decoder-only and encoder-decoder architectures and proposing two metrics: Ranking metrics per PetaFLOP (RPP) and Queries per PetaFLOP (QPP), which are then used to evaluate various reranking methods on the TREC-DL datasets. Primary results demonstrate that simpler, pointwise methods are vastly more efficient; a Flan-T5-large pointwise.yes_no model achieved the highest RPP of 72.67 on DL19, while more effective methods like pairwise sorting caused RPP to drop to approximately 0.1, highlighting a severe efficiency cost for marginal quality gains. The principal implication for AI practitioners is that scaling up model size for reranking offers diminishing returns, and the provided FLOPs estimator enables the selection of more computationally efficient architectures, like pointwise methods, for practical, large-scale deployment. |
| PRING: Rethinking Protein-Protein Interaction Prediction from Pairs to |
|
|
| Graphs (Read more on arXiv or HuggingFace) |
Zhiyuan Liu, Fanding Xu, Hao Du, JinzheFudan, piaolaidangqu |
PRING is a comprehensive benchmark that evaluates protein-protein interaction (PPI) prediction models on their ability to reconstruct biologically coherent networks, moving beyond simple pairwise classification. The primary objective is to assess how well current PPI prediction models can recapitulate the topological and functional properties of real-world PPI networks, a capability overlooked by existing pairwise-focused benchmarks. The authors constructed PRING, a multi-species PPI dataset, and used it to benchmark sequence-based, PLM-based, and structure-based models on topology-oriented (network construction) and function-oriented (pathway analysis) tasks using graph-level metrics. The results show that even the best models struggle to reconstruct accurate network topologies, with the top-performing model achieving a maximum Graph Similarity score of only 0.491, and they perform poorly on functional tasks with functional alignment scores below 0.4. For AI practitioners, this research critically implies that standard classification metrics are insufficient for evaluating models in network reconstruction contexts; high pairwise accuracy does not guarantee the generation of structurally or functionally valid graphs, highlighting the need to adopt graph-centric evaluation protocols. |
| SAMed-2: Selective Memory Enhanced Medical Segment Anything Model (Read more on arXiv or HuggingFace) |
Rong Zhou, Yiwei Li, Sifan Song, Zhiling Yan, songdj |
SAMed-2 is a medical image segmentation foundation model that enhances the SAM-2 architecture with a temporal adapter and a confidence-driven memory mechanism to improve performance on diverse and noisy medical data. The main objective is to adapt a general “segment anything” model for the medical domain by addressing its specific challenges, including handling volumetric/temporal data, mitigating the effects of noisy annotations, and preventing catastrophic forgetting during continual learning across multiple modalities. The key methodology involves integrating a temporal adapter with a 3D convolution into the image encoder to capture inter-slice correlations and implementing a confidence-driven memory that selectively stores high-certainty feature embeddings (based on predicted IoU) and retrieves them based on both feature similarity and confidence. The model achieved a mean Dice Similarity Coefficient (DSC) of 0.6938 on 10 external, unseen segmentation tasks, outperforming MedSAM by 10.53%, and in a human user study, SAMed-2-assisted annotation reduced the time required per frame by 87.61% compared to manual annotation. For AI practitioners, the principal implication is that adapting large vision models to specialized domains like medical imaging requires more than fine-tuning; domain-specific architectural modifications like temporal adapters and explicit, quality-aware memory mechanisms are critical for handling data-specific characteristics like volumetric structure and label noise to achieve robust performance. |
| Tora2: Motion and Appearance Customized Diffusion Transformer for |
|
|
| Multi-Entity Video Generation (Read more on arXiv or HuggingFace) |
Weizhi Wang, Long Qin, Xiangyu Meng, Junchao Liao, Zhenghao Zhang |
Tora2 is a diffusion transformer framework for generating videos with customized appearance and motion for multiple entities simultaneously. The main objective is to overcome the limitations of existing methods by enabling high-fidelity, simultaneous control over both the appearance (from reference images) and motion (from trajectories) for multiple distinct entities in a single generated video. The methodology involves a decoupled personalization extractor (DPE) that fuses low-frequency semantic features with high-frequency identity features using a Q-Former, a gated self-attention mechanism to bind entity, motion, and text embeddings, and a contrastive loss to enforce cross-modal alignment. The proposed method demonstrates superior control; for instance, the inclusion of a contrastive loss function reduced the Trajectory Error from 17.31 to 14.16 while simultaneously improving identity preservation scores. The principal implication for AI practitioners is that this paper provides a concrete architecture for integrating fine-grained, multi-modal controls (identity, trajectory, text) into large-scale diffusion transformer models, offering a robust pattern for building more precise and complex controllable video generation systems. |
| LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation |
|
|
| framework (Read more on arXiv or HuggingFace) |
Ruoxi Sun, Baibei Ji, Haitian Wang, Zecheng Tang, QQTang1223 |
The paper introduces LOOM-Scope, a comprehensive and efficient framework for the standardized evaluation of long-context language models (LCLMs) that integrates benchmarks, models, and inference acceleration techniques. The primary objective is to resolve inconsistencies and high computational costs in existing LCLM evaluation by creating a unified framework that standardizes assessment across diverse benchmarks, model architectures, and efficiency-improving augmentation methods like RAG and inference acceleration. The methodology involves a modular framework with three core components: a BENCHMARK module supporting 22 benchmarks, a DEPLOYMENT module handling various model architectures (e.g., Transformer, Mamba) and optimization techniques (e.g., KV Cache optimization, Sparse Attention), and an EVALUATOR module using diverse metrics. The authors also created LOOMBENCH, a lightweight composite benchmark derived from 12 existing datasets for holistic evaluation. The framework demonstrates significant efficiency gains, with integrated acceleration methods achieving up to a 12x speedup in testing time on 128K-length context tasks compared to a native Transformer implementation (reducing evaluation time from over 200 minutes to under 15 minutes on a 40GB A100 GPU for specific tasks). For AI practitioners, LOOM-Scope provides a tool to conduct fair, reproducible, and computationally efficient evaluations of LCLMs. Its most impactful feature is the direct integration of inference acceleration methods, which enables rigorous testing of models with very long contexts on more accessible hardware and allows for direct comparison of different optimization strategies within a single, standardized environment. |
| Differential Mamba (Read more on arXiv or HuggingFace) |
Eliya Nachmani, Itamar Zimerman, Nadav Schneider |
This paper introduces Differential Mamba (Diff-Mamba), an architecture applying differential design to Mamba models to reduce overallocation to irrelevant context and enhance performance. The primary objective is to adapt the differential design from Transformers to the Mamba architecture to mitigate noisy representations and improve model robustness. The key methodology involves applying a differential operation across the entire Mamba block, where the output of one parameterized path is subtracted from another (Mamba₁(X) - λMamba₂(X)), and using a parallelized implementation to maintain computational efficiency. In experiments, a 12-layer Diff-Mamba model achieved a perplexity of 20.012 on Wikitext-103, outperforming the standard Mamba’s 20.413. For AI practitioners, Diff-Mamba offers a more robust alternative to vanilla Mamba for tasks requiring strong long-context performance, as it demonstrably reduces noise in intermediate representations without increasing computational complexity. |
| How to Train Your LLM Web Agent: A Statistical Diagnosis (Read more on arXiv or HuggingFace) |
Megh Thakkar, Hadi Nekoei, Emiliano Penaloza, Santhoshi Ravichandran, Dheeraj Vattikonda |
This paper presents a statistical analysis to determine the optimal compute allocation between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) for post-training open-source LLM web agents. The primary objective is to find a compute-efficient and reproducible training strategy that improves the performance of smaller student models (Llama 3.1 8B) on multi-step web tasks by leveraging demonstrations from a larger teacher model (Llama 3.3 70B). The methodology involves a two-stage pipeline: first, SFT on teacher demonstrations, followed by on-policy RL (GRPO) initiated from various SFT checkpoints, with bootstrap analysis over 1,370 configurations to identify optimal hyperparameters. The key result is that a hybrid SFT+RL approach consistently outperforms pure SFT or RL, matching the peak performance of pure SFT on MiniWob++ while requiring only 55% of the compute (a 45% FLOPs reduction). The principal implication for AI practitioners is that initiating on-policy RL after a moderate SFT warm-up is a more compute-efficient strategy for developing capable open-source web agents than relying solely on extensive SFT. |
| any4: Learned 4-bit Numeric Representation for LLMs (Read more on arXiv or HuggingFace) |
Jeff Johnson, melhoushi |
This paper introduces any4, a learned 4-bit weight quantization method that creates an optimal, per-row numeric representation for LLMs without requiring weight or activation preprocessing. The primary objective is to develop a 4-bit weight-only quantization scheme that surpasses the accuracy of existing numeric formats (int4, fp4, nf4) and is competitive with orthogonal preprocessing techniques like AWQ and GPTQ. The method applies group-wise scaling to weights and then uses a weighted K-means clustering algorithm to learn a unique 16-value lookup table (LUT) for each matrix row, minimizing output activation error using statistics from a single, curated text sample for calibration. Across Llama, Mistral, and Mixtral models, any4 consistently achieves lower perplexity than other 4-bit numeric formats; for Llama3 70B, any4 achieves a C4 perplexity of 7.01, outperforming nf4 (7.67), fp4 (7.76), and int4 (7.97), and approaching the FP16 baseline of 6.77. For AI practitioners, this means they can quantize LLMs to 4-bits with higher accuracy and a simplified workflow, as it eliminates the need for complex weight/activation preprocessing and large calibration datasets, while the provided tinygemm library facilitates low-latency deployment. |
| Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion (Read more on arXiv or HuggingFace) |
Christian Rupprecht, Oliver Hahn, Felix Wimbauer, Christoph Reich, jev-aleks |
This paper presents SceneDINO, a feed-forward framework for performing unsupervised semantic scene completion (SSC) from a single input image. The primary objective is to infer both the complete 3D geometry and semantic labels of a scene without relying on any manual geometric or semantic ground-truth annotations. SceneDINO’s methodology involves training an encoder-decoder via multi-view self-supervision to lift 2D self-supervised learning (SSL) features into a continuous 3D feature field, from which unsupervised semantics are derived through a novel 3D feature distillation approach. On the SSCBench-KITTI-360 benchmark, SceneDINO achieves a semantic mIoU of 8.0% at a 51.2m range, and linear probing of its learned 3D features achieves a 10.57% mIoU, which slightly surpasses a supervised baseline trained with 2D labels (10.19% mIoU). The key implication for AI practitioners is the ability to generate high-quality 3D scene representations from unlabeled monocular videos, bypassing the need for expensive 3D data annotation and providing a strong foundation for various downstream 3D understanding tasks in robotics and autonomous systems. |
| High-Resolution Visual Reasoning via Multi-Turn Grounding-Based |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Rui Feng, Bo Li, Weiwei Tian, Yuhao Dong, Xinyu Huang |
The paper introduces Multi-turn Grounding-based Policy Optimization (MGPO), a reinforcement learning framework to improve high-resolution visual reasoning in Large Multi-modal Models (LMMs). The primary objective is to enable LMMs to overcome challenges with high-resolution images by learning to iteratively identify and focus on relevant sub-regions. The key methodology involves a multi-turn conversational framework where the model first predicts grounding coordinates for a key area, then receives a cropped sub-image based on those coordinates, and finally provides an answer, with the entire process trained via a binary reward signal on the final answer’s correctness, eliminating the need for grounding annotations. The primary result is that MGPO post-training on Qwen2.5-VL-7B achieves a 5.2% absolute improvement on the out-of-distribution V* Bench over the GRPO baseline. The principal implication for AI practitioners is that complex, interpretable visual reasoning skills like grounding can be effectively taught to LMMs using only standard visual question-answering data, significantly reducing the cost and effort of data annotation for high-resolution tasks. |
| The Landscape of Memorization in LLMs: Mechanisms, Measurement, and |
|
|
| Mitigation (Read more on arXiv or HuggingFace) |
Dawn Song, Aneesh Pappu, Xuandong Zhao, Alexander Xiong |
This paper provides a comprehensive survey on large language model (LLM) memorization, systematically reviewing its underlying mechanisms, detection methodologies, and mitigation strategies. The primary objective is to synthesize the current state of research on LLM memorization by exploring the factors that drive it, the techniques to measure it, and the resulting privacy and legal implications. The methodology is a literature review that organizes the field into a taxonomy covering definitions of memorization, influencing factors (e.g., model size, data duplication), detection attacks (e.g., prefix-based extraction, membership inference), and mitigation approaches (e.g., data cleaning, differential privacy, machine unlearning). Primary results confirm that memorization scales log-linearly with model size and is exacerbated by data duplication, while detection methods like divergence attacks can increase the extraction of verbatim sequences by up to 150x. The principal implication for AI practitioners is that managing the trade-off between model utility and privacy risk is critical, requiring the active integration of mitigation strategies like rigorous data de-duplication and differential privacy into the development lifecycle to prevent unintended leakage of sensitive or copyrighted data. |
| FAROS: Fair Graph Generation via Attribute Switching Mechanisms (Read more on arXiv or HuggingFace) |
Fragkiskos D. Malliaros, Daniele Malitesta, Hatim Mrabet, Oussama Kharouiche, badaoui |
FAROS is a framework that improves fairness in graphs generated by pre-trained Graph Diffusion Models (GDMs) by applying an attribute switching mechanism during the generation process. The primary objective is to mitigate fairness discrepancies in generated graph data for downstream tasks like link prediction, without needing to re-train the GDM. The core methodology involves intervening in the GDM’s generation process by calculating an optimal fraction of nodes and an optimal diffusion timestep to switch their sensitive attributes, using a multi-criteria optimization that balances node-topology preservation (via Fused Gromov-Wasserstein distance) and edge-attribute independence (via entropy). On the CORA dataset, FAROS-Prior reduced the fairness discrepancy ΔEO from 14.45±0.77 to 4.30±4.03 while maintaining comparable accuracy (AUC of 89.08±2.72 vs. 89.39±0.92), achieving a better accuracy-fairness trade-off under Pareto optimality. AI practitioners can use FAROS as a post-hoc module to generate fairer synthetic graph data from existing GDMs without the computational cost of re-training, making it valuable for fairness-critical applications. |
| AXLearn: Modular Large Model Training on Heterogeneous Infrastructure (Read more on arXiv or HuggingFace) |
Hanzhi Zhou, John Peebles, Chang Lan, Tom Gunter, Mark Lee |
The paper presents AXLearn, a deep learning system for training large models that prioritizes modularity and support for heterogeneous hardware through strict component encapsulation. The primary objective is to design a production-grade training framework that enables rapid experimentation on diverse model architectures and can be deployed across various hardware backends (e.g., GPU, TPU, AWS Trainium) with minimal code changes. Methodologically, AXLearn is built on JAX/XLA and uses a hierarchical configuration system based on composition rather than inheritance, with system extensibility formally analyzed using a proposed “Lines-of-Code (LoC)-complexity” metric. The framework achieves constant (O(1)) LoC-complexity, allowing a feature like Rotary Position Embeddings (RoPE) to be integrated across hundreds of modules with just 10 lines of code, versus hundreds required in other systems, while maintaining state-of-the-art training performance (e.g., 54.2% MFU for Llama2-7B on 32 H100 GPUs). For AI practitioners, AXLearn’s design significantly reduces engineering overhead by decoupling model logic from system-level concerns like parallelism and hardware-specific optimizations, allowing for faster development and easier migration of training workloads across different infrastructures. |
Papers for 2025-07-08
| Title |
Authors |
Summary |
| MemOS: A Memory OS for AI System (Read more on arXiv or HuggingFace) |
Hanyu Wang, Chenyang Xi, Shichao Song, Zhiyu Li, Wentao-PKU |
This paper introduces MEMOS, a memory operating system that provides a unified framework for managing heterogeneous memory types in LLMs to enable persistent, long-term intelligence. The primary objective is to overcome the limitations of static models and transient retrieval by treating memory as a first-class, schedulable resource, implemented via an OS-inspired, three-layer architecture (Interface, Operation, Infrastructure) and a standardized MemCube unit for dynamic lifecycle management. In evaluations, MEMOS achieved a top overall LLM-Judge score of 73.31 on the LOCOMO benchmark, outperforming all baselines, and its KV-based memory injection demonstrated up to a 91.4% reduction in Time-to-First-Token (TTFT) without altering output semantics. For AI practitioners, MEMOS provides a standardized API and abstraction layer to manage LLM memory as a controllable resource, simplifying the development of stateful agents with long-term consistency and enabling significant inference latency reduction in production systems. |
| 4DSloMo: 4D Reconstruction for High Speed Scene with Asynchronous |
|
|
| Capture (Read more on arXiv or HuggingFace) |
Xiuyuan Yu, Lihe Ding, Tianshuo Yang, Shi Guo, Yutian Chen |
4DSloMo presents a joint hardware-software solution for high-speed 4D scene reconstruction using low-FPS cameras by combining an asynchronous capture scheme with a video-diffusion-based artifact-fix model. The objective is to reconstruct high-speed dynamic scenes from multi-view videos captured by low frame-rate cameras, which traditionally fail to capture sufficient intermediate motion. The methodology involves staggering the start times of standard cameras to increase the effective capture frame rate, then using a 4D Gaussian Splatting model for initial reconstruction, and finally refining the result with a fine-tuned video diffusion model to correct artifacts caused by the induced viewpoint sparsity. On the DNA-Rendering dataset, the method achieves a PSNR of 26.76, significantly outperforming the baseline GS4D’s 24.75. For AI practitioners, this work demonstrates a practical application of fine-tuned video diffusion models as effective priors for 4D reconstruction, enabling the correction of complex artifacts from spatially sparse data while maintaining temporal consistency, thereby facilitating high-fidelity motion capture without specialized high-speed hardware. |
| DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive |
|
|
| World Knowledge (Read more on arXiv or HuggingFace) |
Yunnan Wang, Hongsi Liu, Wenyao Zhang, RunpeiDong, qizekun |
DreamVLA is a Vision-Language-Action (VLA) framework that improves robot manipulation by forecasting a compact set of world knowledge (dynamics, depth, semantics) before predicting actions. The main research objective is to enhance VLA models by incorporating an efficient, future-state forecasting capability that moves beyond redundant pixel-level prediction to include comprehensive world knowledge, establishing a perception-prediction-action loop. The key methodology involves using a GPT-2-based transformer with specialized <dream> queries to generate a “world embedding” that encapsulates predicted future dynamic regions, depth, and semantics, which in turn conditions an action-generating diffusion transformer; a block-wise structured attention mechanism is used to prevent information leakage between the different knowledge types during forecasting. DreamVLA achieves a state-of-the-art 4.44 average task length on the CALVIN ABC-D benchmark and a 76.7% success rate on real-world robot tasks, with ablations revealing that forecasting dynamic regions is the most critical component. For AI practitioners, the principal implication is that robot policy performance can be significantly improved by adding an intermediate step that explicitly forecasts a compact, disentangled representation of future world states—particularly motion dynamics—rather than directly mapping observations to actions or predicting full future frames. |
| Should We Still Pretrain Encoders with Masked Language Modeling? (Read more on arXiv or HuggingFace) |
Emmanuel Malherbe, Duarte M. Alves, Manuel Faysse, Nicolas-BZRD, hgissbkh |
This paper investigates the relative efficacy of Masked Language Modeling (MLM) and Causal Language Modeling (CLM) for pretraining text encoders, finding that a sequential CLM-then-MLM strategy is optimal. The main objective is to determine whether the performance gains of recent CLM-repurposed encoders stem from the CLM objective itself or from confounding factors like model and data scale. The methodology involves a large-scale controlled study training 38 models (210M to 1B parameters) on 100B tokens, comparing MLM-only, CLM-only, and sequential CLM+MLM pretraining, evaluated via over 15,000 fine-tuning runs on various NLP tasks. The primary results show that while MLM generally yields better final performance, CLM is more data-efficient and stable, and a biphasic strategy combining both objectives is superior; for continued pretraining (CPT), adapting a CLM model with 22,000 steps of MLM significantly outperforms continuing to train an MLM-only model on sequence classification. The principal implication for AI practitioners is that adapting readily available, large pretrained CLM decoders with an MLM objective is a more compute-efficient path to creating state-of-the-art encoder models than training them from scratch. |
| Pre-Trained Policy Discriminators are General Reward Models (Read more on arXiv or HuggingFace) |
Yunhua Zhou, Yicheng Zou, Shichun Liu, Shihan Dou, Umean |
This paper introduces POLicy DiscriminAtive LeaRning (POLAR), a novel paradigm that pre-trains reward models as policy discriminators to improve their generality and scalability. The objective is to establish a scalable, criterion-agnostic pre-training framework for reward models (RMs) to overcome the generalization and data scarcity limitations of traditional preference-based training. The methodology involves pre-training an RM on a large synthetic corpus to distinguish between trajectories from the same versus different policies using a contrastive objective, followed by fine-tuning on human-ranked data to align with desired criteria. The primary result shows that POLAR-7B, when used in RLHF, improves the performance of the LLaMa3.1-8B policy model from an average of 47.36% to 56.33% on 20 benchmarks. The principal implication for AI practitioners is a highly effective method for developing robust RMs that provide more reliable reward signals for policy alignment, applied through a process called Reinforcement Fine-Tuning (RFT) where the RM scores candidate trajectories relative to a reference. |
| BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning |
|
|
| Dataset (Read more on arXiv or HuggingFace) |
Yufang Liu, Honglin Guo, Yutao Fan, Guanyu Li, Zhiheng Xi |
This paper introduces BMMR, a large-scale (110k instances) bilingual, multimodal, and multi-disciplinary reasoning dataset designed to benchmark and enhance the capabilities of large multimodal models (LMMs). The primary objective is to develop a comprehensive, college-level dataset spanning 300 subjects to rigorously evaluate LMMs’ knowledge and reasoning, and to provide a high-quality training set (BMMR-Train) to advance open-source model development. Data was curated from print and digital sources using a human-in-the-loop framework, resulting in two subsets: BMMR-Eval for evaluation and BMMR-Train for fine-tuning, with each instance containing a high-quality reasoning path. A process-based “BMMR-Verifier” was also proposed for fine-grained evaluation of reasoning steps. Primary results show that even state-of-the-art models like Gemini-2.5-Pro achieve only 50.15% accuracy on BMMR-Eval, indicating substantial room for improvement. Fine-tuning with BMMR-Train significantly boosts performance, with the finetuned BMMR-InternVL2.5-78B model showing a 19.07% improvement in overall performance. The principal implication for AI practitioners is that the BMMR-Train dataset provides a valuable, high-quality, multi-disciplinary resource for fine-tuning open-source LMMs to improve their reasoning capabilities, while the BMMR-Eval benchmark allows for rigorous assessment of model weaknesses across a broad range of academic subjects. |
| RoboBrain 2.0 Technical Report (Read more on arXiv or HuggingFace) |
Zhoues, Caozhou1995, MinglanLin, yuheng2000, cmyopu |
The paper introduces RoboBrain 2.0, a series of embodied vision-language foundation models (7B and 32B) designed to unify perception, reasoning, and planning for complex physical tasks. The primary objective is to develop a foundation model that overcomes key limitations in existing VLMs, specifically their limited spatial understanding, weak temporal modeling, and insufficient reasoning, to enable more effective interaction in real-world embodied scenarios. The methodology combines a heterogeneous architecture (vision encoder + Qwen2.5-VL language model) with a progressive three-stage training curriculum that includes foundational learning, embodied enhancement, and chain-of-thought fine-tuning using both supervised and reinforcement learning on synthesized interaction data. The 32B variant achieves state-of-the-art performance on multiple embodied AI benchmarks, outperforming prior models; for instance, it scored 72.43 on the RoboSpatial benchmark, significantly surpassing Gemini-2.5-Pro’s score of 59.87. For AI practitioners, RoboBrain 2.0 provides an open-source, high-performance foundation model and a detailed training recipe for building agents capable of complex spatial-temporal reasoning, with direct applications in robotics for long-horizon planning, multi-agent coordination, and affordance prediction. |
| Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM |
|
|
| Fine-Tuning Data from Unstructured Documents (Read more on arXiv or HuggingFace) |
Jingyuan Wang, Qiyu Sun, Ziyang Miao, hiyouga, oGYCo |
The paper introduces Easy Dataset, a unified framework with a GUI for synthesizing high-quality, persona-driven fine-tuning data from unstructured documents. The primary objective is to automate the generation of diverse and factually consistent fine-tuning datasets from heterogeneous documents to overcome the scarcity of domain-specific data for LLM adaptation. Its methodology combines adaptive document processing using VLMs and a hybrid chunking strategy with a two-stage, persona-driven data synthesis pipeline that leverages (Genre, Audience) pairs to guide QA generation, all within a human-in-the-loop interface. Experiments demonstrate that fine-tuning a Qwen2.5-7B-Instruct model on the synthesized financial data improved its domain-specific knowledge score from a baseline of 3.2 to 59.6, while maintaining general capabilities. For practitioners, this open-source tool provides an end-to-end solution to rapidly create custom fine-tuning datasets for domain adaptation, significantly reducing manual effort and integrating directly with training frameworks like LlamaFactory. |
| RefineX: Learning to Refine Pre-training Data at Scale from |
|
|
| Expert-Guided Programs (Read more on arXiv or HuggingFace) |
Dayiheng Liu, Xingzhang Ren, Shenghua Liu, Baolong Bi, Chevalier |
REFINEX is a framework that refines LLM pretraining data by distilling expert-generated text improvements into minimal, deletion-only programmatic edits. The objective is to create a scalable and reliable data refinement method that improves data quality without the high costs of end-to-end generation or the unreliability of directly generating complex edit programs. The core methodology is a two-stage distillation pipeline: an expert model first generates a high-quality, clean version of a text, then a minimal edit distance algorithm extracts only the deletion operations required for this transformation, which are used to train a small, efficient “refine model.” Primarily, models pretrained on REFINEX-processed data show superior performance; a 750M parameter model achieves 2.6%-7.2% average gains on downstream LightEval tasks over baselines and introduces zero new words, ensuring refinement does not add hallucinations. For AI practitioners, this provides a scalable blueprint to create a custom data-cleaning model that systematically removes noise from corpora, enhancing downstream performance and data efficiency while preserving the authenticity of the original text. |
| Reviving Cultural Heritage: A Novel Approach for Comprehensive |
|
|
| Historical Document Restoration (Read more on arXiv or HuggingFace) |
Yongxin Shi, Pengyu Yan, Zhenhua Yang, Peirong Zhang, Yuyi Zhang |
i) This paper introduces AutoHDR, a modular, three-stage framework for the comprehensive restoration of historical documents, supported by a new full-page dataset named FPHDR. ii) The primary research objective is to develop a fully automated system capable of restoring both the textual content and visual appearance of full-page historical documents, addressing the limitations of prior single-modality or patch-level methods. iii) The methodology consists of a sequential pipeline: 1) OCR-Assisted Damage Localization identifies damaged regions; 2) a Vision-Language Context Prediction (VLCP) algorithm synergizes OCR and LLM outputs to predict missing text; and 3) a patch-autoregressive diffusion model performs pixel-level visual reconstruction. iv) On severely damaged documents, AutoHDR improves character recognition accuracy from a 46.83% baseline to 84.05%, with human-in-the-loop collaboration further increasing accuracy to 94.25%. v) The principal implication for AI practitioners is the demonstration of a practical architecture for building complex, cascaded AI systems where specialized models (detection, language, generative) are integrated, and the modular design explicitly enables human-in-the-loop validation at each stage to enhance final output reliability. |
| StreamDiT: Real-Time Streaming Text-to-Video Generation (Read more on arXiv or HuggingFace) |
Yue Zhao, Masayoshi Tomizuka, Ji Hou, Tingbo Hou, AkiCumulo |
StreamDiT is a novel framework for real-time, streaming text-to-video generation using a specialized training, modeling, and distillation pipeline. The research objective is to develop a system for generating continuous, high-quality video streams in real-time, addressing the offline, short-clip limitations of existing models. The methodology combines a buffered flow matching training process using a moving frame buffer, a modified adaLN Diffusion Transformer (DiT) with time-varying embeddings and window attention, and a tailored multistep distillation technique to reduce inference steps. The primary result is a distilled 4B parameter model that achieves real-time generation of 512p video streams at 16 FPS on a single GPU. For AI practitioners, this framework enables the development of interactive video applications, such as dynamic video-to-video editing or generative game engines, by providing a method for continuous video output that can be modified by user prompts on the fly. |
| ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code |
|
|
| Generation Evaluation (Read more on arXiv or HuggingFace) |
Ao Liu, Jiaheng Liu, Can Xu, Yuhang Li, Chenchen Zhang |
This paper introduces ArtifactsBench, a new benchmark and automated evaluation paradigm for assessing the generation of dynamic, interactive visual artifacts by Large Language Models (LLMs). The central objective is to develop a framework that can automatically and holistically evaluate an LLM’s ability to transform multimodal instructions into high-quality, interactive visual artifacts, moving beyond static code analysis. The methodology involves programmatically rendering the generated artifact, capturing its dynamic behavior via temporal screenshots, and then using a Multimodal LLM (MLLM) as a judge, guided by a fine-grained, per-task checklist, to assess both the visual evidence and the source code. The primary result is that the automated evaluation achieves a 94.4% ranking consistency with WebDev Arena, a human-preference gold standard, and over 90% pairwise agreement with human experts. The principal implication for AI practitioners is that ArtifactsBench provides a scalable, automated tool that reliably proxies human-perceived quality, enabling more accurate benchmarking and targeted development of LLMs for complex, user-centric visual generation tasks. |
| VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and |
|
|
| Visual Documents (Read more on arXiv or HuggingFace) |
Xinyi Yang, Mingyi Su, Ye Liu, Rui Meng, ziyjiang |
i) This paper introduces VLM2Vec-V2, a unified embedding model for text, images, videos, and visual documents, alongside MMEB-V2, a new comprehensive benchmark for its evaluation. ii) The main objective is to develop and evaluate a single, general-purpose embedding model that can robustly represent and generalize across diverse visual modalities beyond natural images, including videos and structured visual documents, to support a wider range of downstream applications. iii) The methodology involves fine-tuning a Qwen2-VL vision-language model using instruction-guided contrastive learning (InfoNCE loss) on a curated training dataset combining image-text, video-language, and visual document retrieval tasks. The training strategy utilizes interleaved sub-batching to balance cross-task diversity and improve optimization stability. iv) The primary result is that VLM2Vec-V2 achieves state-of-the-art performance, with an overall average score of 58.0 across 78 tasks on the MMEB-V2 benchmark, outperforming prior baselines like GME (57.8) and VLM2Vec (52.3). It shows significant improvement on newly introduced video and visual document tasks while maintaining strong performance on image benchmarks. v) The principal implication for AI practitioners is that a single, unified embedding model can effectively handle heterogeneous multimodal data, enabling the development of more versatile AI systems for tasks like multi-modal search, recommendation, and retrieval-augmented generation (RAG) that must process and align representations from images, videos, and documents simultaneously. |
| VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity |
|
|
| Classification (Read more on arXiv or HuggingFace) |
adulau, cedricbonhomme |
This paper presents VLAI, a fine-tuned RoBERTa-base model for automated classification of software vulnerability severity from text descriptions. The objective is to predict a vulnerability’s severity category before an official CVSS score is available, thereby accelerating the triage process for security analysts. The methodology involves fine-tuning a RoBERTa-base model with a softmax classification head on a custom dataset of 610k vulnerabilities, which is updated and used for daily model retraining. VLAI achieves 82.8% classification accuracy on a held-out test set and, in a separate evaluation, its predictions matched the eventual expert-assigned severity approximately 85% of the time. For AI practitioners, this work provides a complete blueprint for an MLOps pipeline that continuously ingests data, retrains a large language model, and deploys it into a live, public-facing service (Vulnerability-Lookup) for real-time inference. |
| PresentAgent: Multimodal Agent for Presentation Video Generation (Read more on arXiv or HuggingFace) |
Meng Fang, Yanjie Liang, Biao Wu, Jingwei Shi, SteveZeyuZhang |
The paper introduces PresentAgent, a multimodal agent that automatically generates narrated presentation videos from long-form documents, and a VLM-powered framework, PresentEval, for their evaluation. The primary objective is to automate the task of Document-to-Presentation Video Generation by creating a system that can process a source document and produce a fully synchronized video with slide-style visuals and spoken narration, mimicking a human-style presentation. PresentAgent employs a four-stage modular pipeline: (1) an LLM-based parser segments the document into a structured outline, (2) a slide composition module generates layout-aware visual frames, (3) a separate LLM pass generates oral-style narration which is converted to audio via a Text-to-Speech system, and (4) a video assembly module composes the visuals and audio into a temporally aligned video. On a curated benchmark, PresentAgent variants achieved factual comprehension scores that surpass human performance; specifically, the Claude-3.7-Sonnet and GPT-4o-Mini backends both achieved a quiz accuracy of 0.64, higher than the human-created video reference score of 0.56. The principal implication for AI practitioners is that this modular, agent-based pipeline provides a blueprint for systems that transform static, text-heavy information into dynamic, accessible multimodal content. The most impactful finding—that these agents can exceed human-level performance in preserving factual accuracy during content transformation—demonstrates a viable path for automating complex professional communication workflows. |
| Beyond Simple Edits: X-Planner for Complex Instruction-Based Image |
|
|
| Editing (Read more on arXiv or HuggingFace) |
Yuheng Li, Richard Zhang, Nanxuan Zhao, Yilin Wang, danielchyeh |
The paper introduces X-Planner, an MLLM-based planning system that decomposes complex image editing instructions into simpler, actionable sub-tasks with automatically generated masks and bounding boxes. The objective is to develop a system that can robustly interpret and execute complex, indirect, and multi-part image editing instructions while preserving object identity and localizing edits, overcoming the limitations of models that require manual guidance. X-Planner employs a GLaMM-based MLLM architecture, trained on a new 260K-pair dataset (COMPIE), which uses chain-of-thought to break down a user prompt into a sequence of sub-instructions, each with a predicted edit type, an object anchor for mask generation, and a predicted bounding box for insertion tasks. On the newly introduced COMPIE benchmark for complex instructions, integrating X-Planner with an InstructPix2Pix* model improved the MLLM-based text-image alignment score (MLLM_ti) from 0.6727 to 0.7408. The principal implication for AI practitioners is that X-Planner can serve as a planning module to enhance existing generative editing models, enabling them to handle sophisticated, natural language requests by translating high-level intent into precise, machine-executable steps with spatial guidance, significantly improving instruction-following capabilities for complex tasks without retraining the core editor. |
| Evaluating LLMs on Real-World Forecasting Against Human Superforecasters (Read more on arXiv or HuggingFace) |
Janna Lu |
This paper evaluates the forecasting accuracy of state-of-the-art large language models (LLMs) against human superforecasters on 464 real-world questions from the Metaculus platform. The research objective is to quantify how well frontier LLMs forecast future events by using the Brier score as the primary metric, feeding models summarized news articles and testing both direct and narrative prompting strategies. The primary result shows that the top-performing model, o3, achieved a mean Brier score of 0.1362, which is better than the general human crowd’s score of 0.149 but significantly worse than the 0.0225 median Brier score of human superforecasters on a subset of the same questions. Furthermore, the models performed substantially worse when using a narrative prompt compared to a direct prediction prompt, indicating that fictional framing can degrade accuracy. The principal implication for AI practitioners is that while current LLMs can surpass general human crowd forecasting abilities, they are not yet a substitute for specialized human expertise and their reasoning accuracy is sensitive to prompting style, with fictionalized scenarios compromising performance. |
| MOD-X: A Modular Open Decentralized eXchange Framework proposal for |
|
|
| Heterogeneous Interoperable Artificial Agents (Read more on arXiv or HuggingFace) |
Aaron Elkins, Vinija Jain, Christos Constantinou, Georgios Ioannides, amanchadha |
The paper proposes MOD-X, a conceptual architectural framework designed for creating decentralized, interoperable ecosystems of heterogeneous AI agents. The objective is to design a framework that overcomes the limitations of existing agent communication protocols by addressing semantic fragmentation, state management conflicts, and security-interoperability tensions. The proposed methodology is a layered architecture featuring a Universal Message Bus (UMB) for publish-subscribe communication, a Translation Layer for semantic interoperability, contextual state management, and a tiered, blockchain-based security model. As a conceptual proposal, the paper presents no empirical results but illustrates its capability discovery mechanism through a worked example where a multimodal synthesis of ontological matching and vector similarity (cosine score of 0.97) produces a final agent capability relevance score of 0.92. The principal implication for AI practitioners is a proposed blueprint for integrating diverse AI systems—from legacy rule-based systems to modern LLMs—into a coherent, scalable ecosystem without requiring centralized coordination, facilitated by semantic discovery and automated translation. |
| Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs |
|
|
| More Realistic and Less Risky (Read more on arXiv or HuggingFace) |
Sebastian Schreiber, Julien Yu, ashutosh1919 |
This paper introduces DIAFORGE, a disambiguation-centric fine-tuning pipeline that improves LLM reliability for enterprise tool-calling by training them to handle near-duplicate APIs and underspecified arguments. The research objective is to enhance LLMs’ multi-turn dialogue capabilities to iteratively elicit missing information and select the correct tool from a dense, overlapping API surface. The methodology consists of a three-stage pipeline that (i) synthesizes persona-driven, multi-turn dialogues with distractor tools, (ii) performs supervised fine-tuning on open-source models, and (iii) evaluates them using a dynamic, interactive benchmark called DIABENCH. The primary result is that models fine-tuned with DIAFORGE increased tool-invocation success by 49 percentage points over a prompted Claude-3.5-Sonnet on the dynamic benchmark. For AI practitioners, this provides a concrete methodology and an open-source dataset of ~5,000 APIs and dialogues to build more reliable and less risky tool-calling agents for enterprise environments where API ambiguity is common. |
| SeqTex: Generate Mesh Textures in Video Sequence (Read more on arXiv or HuggingFace) |
Yan-Pei Cao, Yuan-Chen Guo, Yangtian Sun, Xin Yu, Ze Yuan |
SeqTex is an end-to-end framework that leverages pretrained video foundation models to directly generate high-fidelity UV texture maps for 3D meshes by treating the task as a video sequence generation problem. The primary objective is to overcome the data scarcity and error accumulation issues of existing 3D texturing methods by developing a single-stage model that directly generates complete UV maps by adapting priors from video models. The methodology reformulates texture synthesis as a sequence generation task, where a video diffusion model is fine-tuned to jointly predict a sequence of multi-view renderings and the final UV texture map. Key architectural components include decoupled multi-view and UV processing branches, a geometry-informed attention mechanism to align features between the view and UV domains, and an adaptive token resolution strategy that processes UV maps at a higher resolution. The model achieves state-of-the-art performance, demonstrating a Fréchet Inception Distance (FID) of 30.27 on the image-conditioned texturing task, significantly outperforming the previous best method’s FID of 34.53. The principal implication for AI practitioners is that large-scale pretrained video models can be effectively adapted for native 3D generation tasks beyond simple view synthesis. The technique of structuring a hybrid output (multi-view images + UV map) as a video sequence provides a robust framework for transferring powerful 2D priors to structured 3D asset generation, improving consistency and reducing reliance on multi-stage pipelines. |
| OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device |
|
|
| Speculative Decoding (Read more on arXiv or HuggingFace) |
Yicheng Lin, Chen Feng, Shaojie Zhuo, Ramchalam Kinattinkara Ramakrishnan, justinyyy |
The paper introduces OmniDraft, a framework for a universal draft model that performs speculative decoding for any target LLM, even across different vocabularies. The main objective is to overcome the tight coupling between draft and target models in speculative decoding, enabling a single, small drafter to work with various large models and adapt online to user data. The key methodology combines an online n-gram cache to map token sequences between mismatched vocabularies and a hybrid distillation loss to continuously align the draft model with the target model’s outputs during inference. Primary results demonstrate that a single Llama-68M draft model can pair with diverse target models like Vicuna-7B, Qwen2-7B, and Llama3-8B, achieving up to a 1.70x speedup on the GSM8K reasoning task with the Llama3-8B target. The principal implication for AI practitioners is the significant reduction in overhead for deploying speculative decoding at scale; a single, optimized on-device drafter can be used universally across different and evolving target models, eliminating the need to train and maintain a specific drafter for each target model family. |
Papers for 2025-07-07
| Title |
Authors |
Summary |
| Eka-Eval : A Comprehensive Evaluation Framework for Large Language |
|
|
| Models in Indian Languages (Read more on arXiv or HuggingFace) |
Mayank Singh, Abhishek Upperwal, Samridhi Raj Sinha, RajveeSheth |
The paper presents EKA-EVAL, a unified, open-source framework for evaluating Large Language Models across over 35 global and 10 Indic language benchmarks. The primary objective is to create a comprehensive and accessible evaluation tool that overcomes the English-centric bias of existing frameworks by integrating diverse tasks, including reasoning, long-context understanding, and tool use, with specific support for Indian languages. The methodology involves a modular four-component architecture—Evaluation Engine, Benchmark Registry, Model Interface Layer, and Results Processing System—that supports distributed inference, quantization, and both local and API-based models via an interactive CLI. The framework successfully integrates these capabilities, and a sample evaluation of the google/gemma-2b model on Reading Comprehension tasks showed scores of approximately 77.6 on BoolQ and 46.8 on SQuAD, although the specific metric for these scores is not explicitly stated in the provided figure. The principal implication for AI practitioners is the availability of a production-ready, extensible toolkit that significantly lowers the barrier for conducting comprehensive, reproducible, and multilingual LLM evaluations, facilitating more rigorous model assessment, especially for Indic languages. |
| How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation |
|
|
| Models on Standard Computer Vision Tasks (Read more on arXiv or HuggingFace) |
Oğuzhan Fatih Kar, Andrei Atanov, Roman Bachmann, Ali Garjani, Rahul Ramachandran |
This paper benchmarks the performance of popular multimodal foundation models (MFMs) like GPT-4o on standard computer vision tasks using a novel evaluation framework. The primary objective is to quantitatively assess the visual understanding capabilities of leading MFMs (including GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet) on tasks such as object detection, segmentation, and depth prediction, for which they are not natively designed. To overcome API limitations and text-only outputs, the authors developed a “prompt chaining” framework that decomposes standard vision tasks into a sequence of text-promptable, classification-style sub-tasks. The results show that while MFMs are respectable generalists, they do not match state-of-the-art specialist models; for instance, GPT-4o achieved a 60.62 AP50 in object detection, significantly behind specialist models but leading other tested MFMs in 4 of 6 tasks, with a notable performance gap between semantic and geometric tasks. For AI practitioners, this indicates that current general-purpose MFMs are not yet suitable as direct replacements for specialized vision models in high-precision applications, and the proposed prompt-chaining benchmark offers a standardized method for evaluating the pure vision capabilities of future text-out MFMs. |
Papers for 2025-07-04
| Title |
Authors |
Summary |
| WebSailor: Navigating Super-human Reasoning for Web Agent (Read more on arXiv or HuggingFace) |
Liwen Zhang, Huifeng Yin, Zhongwang Zhang, Kuan Li, xxwu |
This paper presents WebSailor, a post-training methodology for LLMs to create web agents with superhuman reasoning for complex information-seeking tasks. The research objective is to instill in open-source models the ability to systematically reduce extreme uncertainty, closing the capability gap with proprietary agents. The core methodology involves generating high-uncertainty tasks (SailorFog-QA) through structured sampling and information obfuscation, followed by a two-stage training process: a Rejection Sampling Fine-Tuning (RFT) cold start and an efficient agentic RL algorithm, Duplicating Sampling Policy Optimization (DUPO). The primary result shows WebSailor-72B achieving 12.0% on BrowseComp-en and 30.1% on BrowseComp-zh, significantly outperforming all open-source counterparts and matching proprietary agent performance. The principal implication for AI practitioners is that sophisticated agentic reasoning can be instilled in open-source models not just through scale, but via a targeted pipeline of synthetic high-uncertainty data generation and a combined RFT-RL training strategy, providing a clear path to developing highly capable web agents. |
| LangScene-X: Reconstruct Generalizable 3D Language-Embedded Scenes with |
|
|
| TriMap Video Diffusion (Read more on arXiv or HuggingFace) |
Minghui Yang, Jiawei Chi, Hao Li, Fangfu Liu, hanyang-21 |
LangScene-X is a generative framework that reconstructs generalizable, language-queriable 3D scenes from sparse 2D views using a novel video diffusion model. The research objective is to overcome the dense-view requirements of existing methods by developing a system that generates high-fidelity, open-vocabulary 3D scenes from as few as two input images. Its methodology combines a TriMap video diffusion model, trained with progressive knowledge integration to generate consistent RGB, normal, and semantic maps, with a generalizable Language Quantized Compressor (LQC) that efficiently encodes language features without per-scene retraining. The system demonstrates state-of-the-art performance, achieving a 50.52% mean Intersection over Union (mIoU) for 2D semantic segmentation on the LERF-OVS dataset, significantly outperforming the next-best method’s 39.94% mIoU. The principal implication for AI practitioners is the validation of a generative paradigm where a video diffusion model serves as a powerful prior to synthesize consistent, multi-modal 3D data from sparse inputs, enabling more scalable and robust 3D understanding systems. |
| IntFold: A Controllable Foundation Model for General and Specialized |
|
|
| Biomolecular Structure Prediction (Read more on arXiv or HuggingFace) |
He Yan, Wayne Bai, Leon Qiao, The IntFold Team, FuxuLiu |
This paper introduces IntFold, a controllable foundation model for general and specialized biomolecular structure prediction that achieves accuracy comparable to state-of-the-art methods. The research objective is to create a highly accurate structure prediction model that is also adaptable for specialized tasks, such as modeling allosteric states or applying user-defined constraints, through user-driven control. The methodology utilizes a diffusion-based architecture with a custom FlashAttentionPairBias kernel, while achieving controllability by inserting lightweight, trainable LoRA adapters into a frozen base model. IntFold demonstrates performance comparable to AlphaFold 3 on the FoldBench benchmark, and its guided folding capability for antibody-antigen interfaces improves prediction success rate from 37.6% to 69.0% when structural constraints are provided. For AI practitioners, the principal implication is the demonstration that modular adapters can efficiently specialize a large-scale foundation model for domain-specific tasks without full retraining, while also providing practical insights on mitigating training instabilities like activation explosion in deep transformer architectures. |
| Heeding the Inner Voice: Aligning ControlNet Training via Intermediate |
|
|
| Features Feedback (Read more on arXiv or HuggingFace) |
Aibek Alanov, Andrey Kuznetsov, Maxim Nikolaev, Nina Konovalova |
This paper introduces InnerControl, a training strategy to improve spatial control in diffusion models by enforcing consistency between control signals and intermediate U-Net features throughout the entire denoising process. The objective is to overcome the limitations of prior methods like ControlNet++, which only apply consistency losses during the final denoising steps, leading to misalignment when structure is formed early in generation. The core methodology involves training lightweight, timestep-conditioned convolutional probes to predict control signals (e.g., depth, edges) from intermediate U-Net decoder features at every denoising step, enabling a persistent alignment loss. The primary result shows significant improvement in control fidelity, reducing the RMSE for depth map generation by 7.87% (from 28.32 to 26.09) compared to ControlNet++ at a 7.5 guidance scale. The principal implication for AI practitioners is that integrating this intermediate feature feedback mechanism provides a more robust method to train controllable generation models, yielding higher alignment and better image quality, particularly for tasks requiring precise spatial conditioning. |
| Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy (Read more on arXiv or HuggingFace) |
Jiacai Liu, Jujie He, RickyShaw999, zengliangcs, chrisliu298 |
This paper introduces Skywork-Reward-V2, a series of state-of-the-art reward models trained on a new 40M-pair preference dataset, SynPref-40M, curated via a human-AI synergy pipeline. The objective is to overcome the performance limitations of existing open reward models by developing a scalable methodology for creating high-quality, large-scale preference data. The key methodology is a two-stage pipeline that combines small-scale, iterative human-in-the-loop verification with large-scale, automated data filtering based on reward model prediction consistency. The resulting Skywork-Reward-V2-Llama-3.1-8B-40M model achieves state-of-the-art performance, with an average score of 88.6% across seven major benchmarks, outperforming all previous open models. The principal implication for AI practitioners is that meticulous, human-guided data curation is more critical for building high-performing reward models than simply increasing data scale or model size alone. |
| Thinking with Images for Multimodal Reasoning: Foundations, Methods, and |
|
|
| Future Frontiers (Read more on arXiv or HuggingFace) |
Zhenhua Liu, Hangyu Guo, Peng Xia, Zhaochen Su, Xiaoye08 |
This survey introduces the “Thinking with Images” paradigm, where Large Multimodal Models (LMMs) leverage visual information as a dynamic, intermediate step in their cognitive process rather than as a static input. The paper’s objective is to chart the evolution from models that ‘think about’ images to those that can ‘think with’ them, defining the foundational methods, evaluations, and challenges of this new paradigm. The paper proposes a conceptual framework that charts this evolution through three stages of increasing cognitive autonomy: Tool-Driven Visual Exploration (using a fixed toolkit), Programmatic Visual Manipulation (generating code for custom operations), and Intrinsic Visual Imagination (internally generating visual thoughts). The primary contribution is a comprehensive taxonomy organizing methods across these stages; a key challenge identified is the “explosive token economy of visual thought,” where the computational cost of processing intermediate visual steps is orders of magnitude higher than textual reasoning, creating a ceiling on the depth of visual deliberation. The principal implication for AI practitioners is a structured roadmap for designing more capable multimodal systems, allowing them to select the appropriate cognitive mechanism—from external tool use to internal imagination—based on specific application requirements for complexity, efficiency, and interpretability. |
| Decoupled Planning and Execution: A Hierarchical Reasoning Framework for |
|
|
| Deep Search (Read more on arXiv or HuggingFace) |
Yutao Zhu, Yuyao Zhang, Guanting Dong, Xiaoxi Li, Jiajie Jin |
The paper introduces HiRA, a hierarchical reasoning framework that decouples strategic planning from specialized execution to improve deep search task performance. The research objective is to address the inefficiency and limited scalability of monolithic models that handle both high-level planning and detailed execution by proposing a new architectural paradigm. The core methodology involves a three-tiered system: a Meta Reasoning Planner decomposes complex queries into subtasks, an Adaptive Reasoning Coordinator assigns these subtasks to appropriate Domain-Specialized Executors, and these executors leverage specific tools (e.g., search, code interpreters) to complete their assigned functions. On the complex GAIA benchmark, HiRA achieved an average accuracy of 42.5%, significantly outperforming the state-of-the-art WebThinker agent’s 36.2%. For AI practitioners, the principal implication is that designing agentic systems with a modular, hierarchical architecture that separates planning from execution allows for more effective, scalable, and “plug-and-play” integration of diverse reasoning capabilities. |
| Fast and Simplex: 2-Simplicial Attention in Triton (Read more on arXiv or HuggingFace) |
Jiecao Yu, Sijia Chen, Sai Surya Duvvuri, Timothy Chou, Aurko Roy |
This paper introduces an efficient Triton kernel for 2-simplicial attention, demonstrating it improves the parameter-scaling exponent and achieves better token efficiency on reasoning tasks compared to standard dot-product attention. The primary objective is to investigate whether 2-simplicial attention, which generalizes standard attention to trilinear forms, can offer a more favorable scaling law exponent and thus better performance than standard Transformers under fixed token budget constraints. The authors implement 2-simplicial attention within a sliding window using a custom Triton kernel optimized for efficiency, inspired by FlashAttention. They train a series of interleaved Mixture-of-Experts (MoE) models ranging from 1B to 3.5B active parameters on a fixed token budget, comparing their negative log-likelihood on reasoning, math, and coding benchmarks against standard Transformer baselines. The primary result is that 2-simplicial attention models outperform identically-sized standard Transformers on reasoning-heavy tasks, with the performance gap widening at larger scales. Specifically, the paper demonstrates that 2-simplicial attention increases the parameter scaling exponent α in the neural scaling law; for the MMLU-pro benchmark, α increased by 20.2% (from 0.0901 to 0.1083) compared to the dot-product attention baseline. The principal implication for AI practitioners is that 2-simplicial attention presents a viable architectural alternative to standard attention, particularly in token-constrained environments. The improved scaling exponent suggests that for a given limited dataset, a 2-simplicial model can achieve superior performance on complex reasoning tasks compared to a standard Transformer of the same parameter count, making it a promising direction for building more token-efficient models. |
| Can LLMs Identify Critical Limitations within Scientific Research? A |
|
|
| Systematic Evaluation on AI Research Papers (Read more on arXiv or HuggingFace) |
Arman Cohan, Lovekesh Vig, Manasi Patwardhan, Yilun Zhao, Zhijian Xu |
This research introduces LIMITGEN, a benchmark for systematically evaluating the capability of LLMs to identify critical limitations in AI research papers, and assesses the impact of Retrieval-Augmented Generation (RAG). The primary objective is to quantify how effectively LLMs can identify different types of limitations in scientific papers and to determine if RAG can enhance this capability to a level that assists human peer reviewers. The authors created the LIMITGEN benchmark, comprising a synthetic dataset (LIMITGEN-Syn) with controlled, introduced limitations and a human-derived dataset (LIMITGEN-Human) from ICLR 2025 reviews. They evaluated proprietary and open-source LLMs, as well as a multi-agent system (MARG), with and without a RAG pipeline that retrieves relevant literature from Semantic Scholar. Current LLMs demonstrate limited capability in identifying research limitations; on the LIMITGEN-Syn dataset, GPT-4o’s coarse-grained accuracy for identifying introduced limitations was 52.0%, significantly lower than human performance (86.0%). However, incorporating a RAG pipeline substantially improved GPT-4o’s accuracy by 12.2 percentage points. LLMs, in their current state, are not suitable for autonomous peer review or critical analysis of scientific work. AI engineers should focus on developing RAG-enhanced systems as tools to assist human experts by grounding model outputs in relevant literature, rather than attempting to replace human-in-the-loop processes for tasks requiring deep, contextualized domain expertise. |
| Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving (Read more on arXiv or HuggingFace) |
Jun Wang, Anthony Bordg, Rasul Tutunov, Xiaotong Ji, Matthieu Zimmer |
The paper introduces Bourbaki, a theorem-proving system using a novel framework called self-generated goal-conditioned Markov Decision Processes (sG-MDPs) to navigate complex proof searches. The research objective is to overcome the sparse reward problem in automated theorem proving by enabling an agent to dynamically generate and pursue its own intermediate subgoals. The key methodology involves formulating the proof search as an sG-MDP, where large language models propose conjectures (subgoals), and a Monte Carlo Tree Search (MCTS) algorithm explores the resulting proof space, with rewards given for solving these intermediate steps. The primary result shows that the Bourbaki (7B) system solves 26 problems on the PutnamBench benchmark at a pass@512 sample budget, establishing a new state-of-the-art for 7B-scale models. For AI practitioners, the principal implication is that the sG-MDP framework offers a structured method to decompose long-horizon reasoning tasks, creating denser reward signals and making complex problems more tractable for search algorithms without requiring pre-trained critic models. |
| Energy-Based Transformers are Scalable Learners and Thinkers (Read more on arXiv or HuggingFace) |
Peixuan Han, Md Mofijul Islam, Ganesh Nanduru, Alexi Gladstone, amanchadha |
i) This paper introduces Energy-Based Transformers (EBTs), a new model paradigm that achieves superior scaling and generalization by framing prediction as an iterative, unsupervised energy minimization process analogous to System 2 thinking. ii) The primary research objective is to determine if it’s possible to develop models that learn System 2 thinking capabilities, such as dynamic compute allocation and self-verification, entirely from unsupervised learning without modality- or problem-specific supervision. iii) The key methodology involves training Energy-Based Transformers (EBTs) to learn an energy function that evaluates the compatibility between an input and a candidate prediction, then generating predictions by iteratively minimizing this energy via gradient descent. iv) EBTs demonstrate superior scalability, achieving up to a 35% higher scaling rate than the Transformer++ approach during pretraining, and at inference, they can improve language modeling performance by 29% more than Transformer++ models by allocating additional computation. v) The principal implication for AI practitioners is that EBTs present a new, more data-efficient pretraining paradigm that generalizes better than standard Transformers, offering a promising approach for scaling future foundation models, especially as high-quality training data becomes a limiting factor. |
| Selecting and Merging: Towards Adaptable and Scalable Named Entity |
|
|
| Recognition with Large Language Models (Read more on arXiv or HuggingFace) |
Wei Wei, Zhuojun Ding, Facico |
The SaM framework improves Named Entity Recognition by dynamically selecting and merging pre-trained, domain-specific expert models at inference time for enhanced adaptability and scalability. The primary objective is to overcome the poor adaptability, scalability, and high cost associated with training unified, multi-domain Large Language Models for NER tasks. The proposed SaM framework first trains multiple expert models on distinct domains using parameter-efficient fine-tuning (LoRA). For a given target domain at inference, it selects relevant experts based on two criteria: domain similarity calculated via text embeddings and performance on a small, pseudo-labeled sample set. The LoRA parameters of these selected experts are then merged using Ties-Merging to create specialized models for final prediction. Experimental results on CrossNER and MIT benchmarks demonstrate that the SaM framework outperforms a unified model trained on all source data, achieving an average F1-score improvement of approximately 10%. The principal implication for AI practitioners is that they can develop more scalable and adaptable NER systems by maintaining a library of lightweight, domain-specific LoRA adapters and dynamically composing them at inference time, thus avoiding costly retraining or reliance on a single, sub-optimal monolithic model. |
| Self-Correction Bench: Revealing and Addressing the Self-Correction |
|
|
| Blind Spot in LLMs (Read more on arXiv or HuggingFace) |
Ken Tsui |
This research introduces Self-Correction Bench to demonstrate that LLMs exhibit a “Self-Correction Blind Spot,” and shows that simple test-time interventions can activate this latent capability. The primary objective is to systematically quantify why LLMs fail to correct their own generated errors despite being able to correct identical errors presented externally. The methodology uses controlled error injection into either the model’s own response or the user’s prompt across three datasets (SCLI5, GSM8K-SC, PRM800K-SC) to compare internal versus external error correction. Results reveal an average 64.5% blind spot rate across 14 models, which is reduced by 89.3% simply by appending the token “Wait” to the model’s output without any finetuning. The principal implication for AI practitioners is that an LLM’s self-correction ability is often a problem of activation rather than knowledge, and it can be elicited at inference time using simple conditioning tokens to improve reliability. |
| ZeCO: Zero Communication Overhead Sequence Parallelism for Linear |
|
|
| Attention (Read more on arXiv or HuggingFace) |
Tianjian Li, Xinyi Wan, Ruijie Zhu, Zehao Liu, Yuhong Chou |
ZeCO is a sequence parallelism (SP) method for linear attention models that achieves near-linear scalability by eliminating communication bottlenecks. The paper’s objective is to overcome the substantial communication overhead of existing SP methods, which impedes the training of models on ultra-long sequences. The key methodology is a novel collective communication primitive called “All-Scan,” which uses a pipelined receive-scan-send pattern to transmit minimal state information between devices, enabling significant overlap between communication and computation. Empirically, ZeCO achieves a 60% throughput speedup over the state-of-the-art SP method on 256 GPUs with an 8M sequence length, and its All-Scan communication is 3.9x faster than the All-Gather used in prior work. For AI practitioners, this provides a direct path to efficiently pre-train linear attention models on previously intractable context lengths with near-ideal scaling, significantly accelerating the development of very-long-context LLMs. |
| AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM |
|
|
| Post-Training (Read more on arXiv or HuggingFace) |
Guang Yang, Kui Luo, Haibo Wang, Ansheng You, Zhenyu Han |
AsyncFlow is a task-separated, asynchronous streaming reinforcement learning framework designed to improve the efficiency and scalability of large language model post-training. The primary objective is to overcome the scalability bottlenecks, resource idling, and inflexible engine coupling of existing RL frameworks by developing a modular, high-throughput system for large-scale post-training. The key methodology combines a distributed data management module, TransferQueue, which provides centralized, fine-grained data scheduling to enable automated pipeline overlapping, with a producer-consumer asynchronous workflow that uses a delayed parameter update mechanism to minimize synchronization overhead between RL tasks. In experiments, AsyncFlow achieved an average throughput improvement of 1.59× over the state-of-the-art task-collocated baseline (verl); an ablation study on a 7B model using 512 NPUs showed that the TransferQueue module alone provided a 2.01x throughput gain, which increased to 2.74x with full asynchronous optimizations. The principal implication for AI practitioners is that implementing a task-separated architecture with asynchronous streaming dataflows and optimized, delayed parameter updates can substantially increase hardware utilization and training throughput, enabling more efficient and scalable RL-based fine-tuning of large models on industrial-scale clusters. |
Papers for 2025-07-03
| Title |
Authors |
Summary |
| Kwai Keye-VL Technical Report (Read more on arXiv or HuggingFace) |
huxiao09, yw95, TinaGao, hjy, caojiangxia |
This paper introduces Kwai Keye-VL, an 8-billion-parameter multimodal foundation model engineered for state-of-the-art performance in short-video understanding. The objective is to develop a model that can comprehend dynamic, information-dense video content, a key limitation in existing MLLMs. The methodology rests on a massive 600-billion-token video-centric dataset and an innovative training recipe featuring a four-stage pre-training process followed by a two-phase post-training process that utilizes a five-mode “cold-start” data mixture and reinforcement learning to elicit reasoning. Keye-VL achieves state-of-the-art results on video benchmarks, including an 8.7% absolute improvement on Video-MMMU, and remains competitive on general image tasks. The principal implication for AI practitioners is that a targeted training strategy combining high-quality video data, a mixed-mode reasoning approach (e.g., CoT, Auto-Think), and iterative RL alignment can significantly advance MLLM capabilities for complex temporal video analysis. |
| LongAnimation: Long Animation Generation with Dynamic Global-Local |
|
|
| Memory (Read more on arXiv or HuggingFace) |
Zhendong Mao, Yihao Meng, Mengqi Huang, CNcreator0331 |
This paper presents LongAnimation, a novel framework for automated, long-term animation colorization that maintains consistent color over extended sequences. The research objective is to solve the problem of color inconsistency in long animations, which existing local-paradigm methods fail to address. The core methodology is a dynamic global-local paradigm, which features a SketchDiT for hybrid reference feature extraction and a Dynamic Global-Local Memory (DGLM) module that uses a long video understanding model to compress historical features and fuse them with the current generation context. Quantitatively, LongAnimation improves long-term video quality by 49.1% in Frechet Video Distance (FVD) over prior methods on sequences averaging 500 frames. The principal implication for AI practitioners is that the DGLM’s approach of using a long-context understanding model to dynamically inject compressed global information into a generative process offers a powerful, transferable technique for maintaining consistency in other long-sequence generation tasks. |
| Depth Anything at Any Condition (Read more on arXiv or HuggingFace) |
Qibin Hou, Bowen Yin, Modi Jin, BBBBCHAN |
The paper presents DepthAnything-AC, a foundation monocular depth estimation model finetuned for robustness against diverse and adverse environmental conditions. i) The primary research objective is to adapt a general-purpose foundation MDE model to perform reliably under challenging conditions like poor lighting, adverse weather, and sensor-induced distortions, without compromising its original capabilities on standard scenes. ii) The core methodology involves an unsupervised consistency regularization paradigm that finetunes a base model on a small set of unlabeled images with applied perturbations (e.g., lighting, blur, weather). This is coupled with a knowledge distillation loss from a frozen teacher model and a novel Spatial Distance Constraint that explicitly enforces patch-level relative geometric relationships to preserve object boundaries. iii) Experimental results show that DepthAnything-AC improves zero-shot performance on corrupted data benchmarks; specifically, on the DA-2K blur benchmark, the model achieves a pairwise comparison accuracy of 0.880, improving upon the 0.862 of the baseline DepthAnything V2, while maintaining comparable performance on general benchmarks like NYU-D. iv) The principal implication for AI practitioners is that this work offers a data-efficient strategy to enhance the robustness of large foundation models for specific, challenging domains by using only a small corpus of unlabeled data and a perturbation-based consistency framework, avoiding the need for extensive data collection and labeling for each target condition. |
| A Survey on Vision-Language-Action Models: An Action Tokenization |
|
|
| Perspective (Read more on arXiv or HuggingFace) |
Zhang Chen, Fengshuo Bai, Yifan Zhong, Feernnn, phython96 |
This paper surveys Vision-Language-Action (VLA) models, proposing a unified framework that classifies them based on their method of “action tokenization”. The primary objective is to systematically analyze VLA research by framing it as a process where a series of modules generate a chain of intermediate action tokens—such as language, code, affordance, or raw actions—to translate multimodal inputs into physical execution. The key methodology is a literature review structured around a novel taxonomy of eight distinct action token types, analyzing the advantages and limitations of each. The survey finds that different token types offer unique trade-offs; for example, latent representations enable high training efficiency, with UniVLA achieving performance comparable to a baseline using only 4.45% of the training time, but lack the interpretability of explicit tokens like code. For AI practitioners, the principal implication is that designing robust embodied agents requires a hierarchical architecture that strategically combines different action token types, leveraging their complementary strengths for different levels of task planning and execution. |
| FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model (Read more on arXiv or HuggingFace) |
Ziwei Liu, Jinghao Wang, Chenyang Si, Yukang Cao |
FreeMorph is a novel, tuning-free image morphing framework that leverages a pre-trained diffusion model to generate high-fidelity, smooth transitions between semantically or structurally diverse images without per-instance optimization. The primary objective is to develop a generalized image morphing method that operates without fine-tuning pre-trained diffusion models, enabling fast and high-quality transitions between any two input images, even those with significant semantic and layout differences. The methodology involves modifying the self-attention modules within a pre-trained latent diffusion model during a scheduled DDIM inversion and denoising process by introducing a “guidance-aware spherical interpolation” that aggregates key/value features from both inputs to preserve identity, and a “step-oriented variation trend” that parametrically blends attention outputs to ensure a gradual transformation. The method demonstrates superior performance over existing techniques, achieving an overall mean Frechet Inception Distance (FID) of 152.88, significantly outperforming prior methods like DiffMorpher (209.10), while being 10x-50x faster. The principal implication for AI practitioners is a highly efficient, tuning-free pipeline for image morphing that reduces generation time from minutes to under 30 seconds, making advanced generative transitions practical for interactive applications and showcasing that complex control can be achieved by manipulating the internal mechanics of foundational models instead of through costly fine-tuning. |
| Locality-aware Parallel Decoding for Efficient Autoregressive Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Kelly Peng, Shang Yang, Chengyue Wu, Luke J. Huang, Zhuoyang Zhang |
This paper introduces Locality-aware Parallel Decoding (LPD), a framework to significantly accelerate autoregressive image generation through high-degree parallelization. The primary objective is to reduce the high latency of next-patch prediction in autoregressive models while maintaining generation quality and compatibility with universal flat token representations. The methodology combines two key techniques: a “Flexible Parallelized Autoregressive Modeling” architecture that uses learnable position query tokens to enable arbitrary-order parallel generation, and a “Locality-aware Generation Ordering” schedule that strategically groups tokens to maximize contextual support and minimize intra-group dependencies. The proposed method reduces the number of generation steps for 256x256 ImageNet generation from 256 to 20, achieving at least 3.4x lower latency than previous parallelized autoregressive models without compromising quality. For AI practitioners, this work provides a method to build significantly faster autoregressive visual generation systems that retain compatibility with standard vision backbones and can perform zero-shot editing tasks like inpainting and outpainting. |
| JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching (Read more on arXiv or HuggingFace) |
Youngjung Uh, Jaesik Park, Jaeseok Jung, Mingi Kwon, alex4727 |
JAM-Flow introduces a unified framework for the joint synthesis of facial motion and speech using flow matching and a Multi-Modal Diffusion Transformer (MM-DiT). The objective is to create the first single architecture that simultaneously models, generates, and conditions on both audio and motion, overcoming the traditional separation of text-to-speech and talking head synthesis. The methodology leverages two specialized, yet coupled, transformer modules (Audio-DiT and Motion-DiT) with selective joint attention, scaled rotary positional embeddings for temporal alignment, and an inpainting-style training objective. In talking head generation experiments on the HDTF dataset, the model significantly outperforms prior art, achieving a Fréchet Video Distance (FVD) of 25.07, compared to scores over 160 for competing methods. For AI practitioners, this work provides a practical, unified architecture that eliminates the need for separate pipelines, enabling more coherent and flexible audio-visual synthesis for applications like virtual avatars and automated dubbing from diverse conditioning inputs. |
| STR-Match: Matching SpatioTemporal Relevance Score for Training-Free |
|
|
| Video Editing (Read more on arXiv or HuggingFace) |
Bohyung Han, Junoh Kang, jslee525 |
This paper presents STR-Match, a training-free algorithm that performs high-fidelity, text-guided video editing by matching a novel SpatioTemporal Relevance score. The objective is to resolve temporal inconsistencies, motion distortions, and limited domain transformations in existing video editing methods by better modeling spatiotemporal pixel relevance. The core methodology is a latent optimization process guided by the proposed STR score, which is calculated from the multiplicative combination of 2D spatial self-attention and 1D temporal-attention maps from a pretrained text-to-video (T2V) model, eliminating the need for 3D attention. Quantitatively, STR-Match with a mask achieves a Motion Error of 1.932, significantly outperforming the T2V-based method DMT (5.741), while also achieving superior background preservation (BL of 0.103 vs. 0.499). The principal implication for AI practitioners is that STR-Match offers a computationally efficient, zero-shot framework to significantly improve the consistency and flexibility of video editing on top of existing T2V models without any retraining, proving especially effective for challenging domain shifts. |
| MARVIS: Modality Adaptive Reasoning over VISualizations (Read more on arXiv or HuggingFace) |
Chinmay Hegde, Oussama Elachqar, Lennart Purucker, Benjamin Feuer |
MARVIS is a training-free method that enables vision-language models to perform prediction tasks on any data modality by reasoning over visual representations of that modality’s embedding space. The research objective is to combine the reasoning capabilities of foundation models with the representational power of specialist models without requiring modality-specific fine-tuning or exposing personally identifiable information (P.I.I.). The core methodology involves using a domain-specific model to generate vector embeddings, applying t-SNE to create a 2D visualization of the embedding space, and prompting a VLM (Qwen 2.5 VL 3B) with this visualization to predict the class of a query point. Using a single 3B parameter model, MARVIS achieved competitive performance across vision, audio, biological, and tabular domains, improving upon foundation model baselines like Gemini by an average of 16.7 percentage points. For AI practitioners, this method offers a universal, training-free interface to apply pre-trained VLMs to specialized, non-traditional, or privacy-sensitive data modalities by converting them into a visual format, thus avoiding costly, domain-specific model training and direct data serialization. |
Papers for 2025-07-02
| Title |
Authors |
Summary |
| GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
tanghme0www, bigganbing, xgeric, iyuge2, wenyi |
This paper presents GLM-4.1V-Thinking, a vision-language model designed for versatile multimodal reasoning through a novel, reasoning-centric training framework. The primary objective is to enhance a model’s general-purpose reasoning capabilities by leveraging scalable reinforcement learning to unlock the full potential of a capable vision foundation model. The key methodology is a three-stage training process: large-scale pre-training on a diverse multimodal corpus, supervised fine-tuning on long chain-of-thought data, and a final stage using Reinforcement Learning with Curriculum Sampling (RLCS) to dynamically select appropriately difficult tasks. The resulting open-source 9B-parameter model outperforms the much larger Qwen2.5-VL-72B on 18 of 28 benchmarks, and the RLCS stage provides substantial performance boosts, including a +7.3% gain on GUI agent tasks. The principal implication for AI practitioners is that a multi-stage pipeline culminating in scalable reinforcement learning with a meticulously designed, multi-domain reward system is a highly effective strategy for creating state-of-the-art, versatile VLMs, with the open-source model providing a strong practical foundation. |
| MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional |
|
|
| Multimodal Embeddings (Read more on arXiv or HuggingFace) |
Nan Yang, Liang Wang, roosephu, hongliu9903, Haon-Chen |
The paper introduces MoCa, a two-stage framework for converting pre-trained causal Vision Language Models (VLMs) into more effective bidirectional multimodal embedding models. The research objective is to address the suboptimal performance of causal attention for embedding tasks and the scalability limitations of contrastive learning by developing a method that leverages bidirectional attention and large-scale unlabeled data. The methodology consists of: 1) Modality-aware Continual Pre-training, which uses a joint reconstruction objective (Masked Language Modeling for text and Masked Autoencoding for images) on unlabeled data to enable bidirectional reasoning, and 2) Heterogeneous Contrastive Fine-tuning on diverse data pairs to improve alignment. The resulting MoCa-7B model establishes a new state-of-the-art on the MMEB benchmark with an average score of 71.5, outperforming the prior best model. The principal implication for AI practitioners is that this framework offers a scalable pathway to adapt powerful, existing causal VLMs into superior bidirectional embedding models without requiring training from scratch, effectively leveraging vast amounts of unlabeled multimodal data. |
| SciArena: An Open Evaluation Platform for Foundation Models in |
|
|
| Scientific Literature Tasks (Read more on arXiv or HuggingFace) |
Sihong Wu, zihang93, HughieHu, maxzky, yilunzhao |
The paper introduces SciArena, an open, community-driven platform for evaluating foundation models on literature-grounded scientific tasks using pairwise human preference voting. The primary objective is to create a reliable and dynamic evaluation platform for foundation models performing open-ended scientific literature synthesis and to use the collected data to build a meta-evaluation benchmark, SciArena-Eval, for assessing automated evaluators. The platform uses a Retrieval-Augmented Generation (RAG) system to provide two anonymous model outputs for a user’s scientific query, collecting over 13,000 preference votes from 102 vetted researchers, and then ranks models using a Bradley-Terry model to calculate Elo ratings. The o3 model achieved the highest Elo score (1172.5) on the SciArena leaderboard; however, when used as an automated judge on the SciArena-Eval benchmark, this top model only achieved 65.1% accuracy in aligning with human expert preferences, indicating a significant challenge in automated evaluation for scientific tasks. Current LLM-as-a-judge evaluation methods are insufficient for specialized scientific domains; AI practitioners should use domain-specific, human-validated benchmarks like SciArena-Eval to reliably assess model capabilities for scientific literature synthesis, as standard automated metrics fail to capture critical nuances like citation correctness and technical precision. |
| Does Math Reasoning Improve General LLM Capabilities? Understanding |
|
|
| Transferability of LLM Reasoning (Read more on arXiv or HuggingFace) |
Seungone Kim, Xiaoyu Xu, Yuetai Li, Maggie Huan, aaabiao |
This paper investigates the transferability of math reasoning gains in LLMs, finding that Reinforcement Learning (RL) preserves general capabilities while Supervised Fine-Tuning (SFT) often leads to catastrophic forgetting. The objective is to determine if improving LLM performance on mathematical reasoning benchmarks translates to enhanced capabilities in other reasoning and non-reasoning domains, and to identify which fine-tuning methods facilitate this transfer. The study performs a large-scale evaluation of over 20 reasoning-tuned models and conducts controlled experiments fine-tuning a Qwen3-14B model with math-only data using both SFT and RL, analyzing changes via latent-space PCA and token-space KL-divergence. Results show that RL-tuned models successfully transfer reasoning gains, whereas SFT-tuned models do not; in a controlled experiment, an RL-tuned model exhibited positive performance gains on non-reasoning tasks (avg. +7.5%), while its SFT-tuned counterpart showed significant performance degradation (avg. -22.4%) compared to the base model. For AI practitioners, this implies that using SFT with specialized, distilled datasets can degrade general model performance, and RL should be preferred for enhancing specific skills without sacrificing broad, general-domain capabilities. |
| Radial Attention: O(nlog n) Sparse Attention with Energy Decay for |
|
|
| Long Video Generation (Read more on arXiv or HuggingFace) |
Shuo Yang, Haocheng Xi, Tianle Cai, Xingyang Li, Lmxyy |
This paper introduces Radial Attention, an O(n log n) sparse attention mechanism that models spatiotemporal energy decay to accelerate long video generation. The primary objective is to mitigate the prohibitive O(n²) computational cost of dense attention in video diffusion models, making the generation of long, high-quality videos computationally feasible. The core methodology involves a static sparse attention mask that implements an exponentially decaying compute density: the attention window for a given token shrinks exponentially with increasing temporal distance from other tokens. For extending generation to 4x longer videos on the HunyuanVideo model, Radial Attention achieves a 3.7x inference speedup and a 4.4x reduction in fine-tuning costs compared to dense attention, while maintaining comparable video quality. For AI practitioners, this method provides a direct way to reduce the computational cost of long video generation, enabling existing pre-trained models to be efficiently adapted for longer sequences with minimal fine-tuning overhead. |
| DiffuCoder: Understanding and Improving Masked Diffusion Models for Code |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Navdeep Jaitly, Jiatao Gu, Huangjie Zheng, Ruixiang Zhang, Shansan Gong |
This paper introduces DiffuCoder, a 7B diffusion language model for code, and a novel reinforcement learning method, coupled-GRPO, which improves performance by reducing variance in policy gradient estimation. The main objective is to demystify the decoding behavior of masked diffusion language models (dLLMs) for code generation and develop a diffusion-native reinforcement learning (RL) framework to unlock their potential. The methodology involves training a 7B dLLM named DiffuCoder, introducing “autoregressive-ness” (AR-ness) metrics to analyze its decoding patterns, and proposing coupled-GRPO, an RL algorithm that uses a coupled-sampling scheme with complementary masks (an application of antithetic variates) for low-variance policy gradient estimation. The primary result is that training with coupled-GRPO significantly improves DiffuCoder’s performance, achieving a +4.4 absolute point increase on the EvalPlus benchmark over the instruction-tuned version. Furthermore, the GRPO-trained model shows a smaller performance drop when decoding steps are halved, indicating increased parallelism. The principal implication for AI practitioners is that coupled-GRPO provides an effective method for applying reinforcement learning directly to dLLMs for tasks like code generation, enhancing performance while respecting their non-autoregressive nature. The finding that sampling temperature affects generation order, not just token choice, offers a new lever for creating diverse rollouts for RL. |
| HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context (Read more on arXiv or HuggingFace) |
Weixuan Chen, Shimin Yao, BBBBCHAN, fushh7, PhilipC |
This paper introduces HumanOmniV2, an omni-modal model that enhances reasoning by first requiring an explicit summary of multimodal context. The research objective is to mitigate two primary failure modes in existing models: insufficient global context understanding and reasoning shortcuts that ignore multimodal inputs. The methodology utilizes Reinforcement Learning based on Group Relative Policy Optimization (GRPO), uniquely augmented with LLM-judged context and logical rewards, which assess the quality of the model’s generated context summary and its subsequent reasoning path. HumanOmniV2 achieves state-of-the-art performance among open-source models, scoring 58.47% on the Daily-Omni benchmark and 69.33% on the authors’ new IntentBench. For AI practitioners, this work implies that structuring model outputs to first articulate context and then applying targeted RL rewards to that articulation is a potent technique for improving complex, multimodal reasoning and reducing hallucinatory or incomplete responses. |
| Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive |
|
|
| Foundations for Artificial General Intelligence and its Societal Impact (Read more on arXiv or HuggingFace) |
Abbas Shah, Ranjan Sapkota, Rizwan Qureshi, amanchadha, shainaraza |
This paper synthesizes research across AI and cognitive science to argue that AGI requires moving beyond token-level prediction towards integrated, brain-inspired cognitive architectures. The primary objective is to analyze the limitations of current large-scale models and define a roadmap for AGI based on principles like modular reasoning, persistent memory, and grounded agency. The methodology is a cross-disciplinary synthesis and analysis of existing literature from artificial intelligence, cognitive neuroscience, psychology, and agent-based systems. The paper concludes that true intelligence emerges from the integration of cognitive components, not from scaling alone, highlighting that applying Tree-of-Thoughts (ToT) to GPT-4 increased its success rate on a combinatorial puzzle to 74% from 4% using Chain-of-Thought (CoT). The principal implication for AI practitioners is to shift from scaling monolithic models towards architecting modular, agentic systems that integrate structured reasoning, persistent memory, and dynamic tool use to build more capable and grounded AI. |
| Data Efficacy for Language Model Training (Read more on arXiv or HuggingFace) |
Chong Li, Wenshan Wu, Xin Zhang, Yangyu Huang, Yalun Dai |
This paper introduces “Data Efficacy,” a paradigm for improving language model performance by optimizing the organization of training data. The primary research objective is to define and validate a new paradigm, DELT (Data Scoring, Selection, and Ordering), that maximizes model performance by strategically ordering training data, complementing existing data efficiency techniques focused on data selection. The key methodology involves a three-stage process: 1) Data Scoring, using a novel Learnability-Quality Scoring (LQS) method based on gradient consistency to evaluate each sample’s learnability and quality; 2) Optional Data Selection; and 3) Data Ordering, using a proposed Folding Ordering (FO) method, which applies multi-pass curriculum learning to mitigate model forgetting and data distribution bias. The primary result demonstrates that the proposed DELT instance (LQS for scoring and FO for ordering) consistently improves model performance; on a 160M parameter model, it achieved a 1.65% absolute improvement in average accuracy across eight benchmarks (38.02% vs. 36.37% for the conventional baseline) without altering the dataset or model size. The principal implication for AI practitioners is that optimizing the sequence of training data is a highly effective, low-cost strategy to enhance model performance. Instead of relying solely on random shuffling, engineers can implement a structured data ordering scheme like LQS+FO to achieve superior results from existing datasets and model architectures, making it a powerful tool for both efficacy and efficiency. |
| FreeLong++: Training-Free Long Video Generation via Multi-band |
|
|
| SpectralFusion (Read more on arXiv or HuggingFace) |
Yi Yang, Yu Lu |
FreeLong++ is a training-free framework for extending short-video generation models to produce longer, high-fidelity videos by fusing multi-scale temporal features in the frequency domain. The research objective is to mitigate the high-frequency distortion and temporal inconsistency that arise when applying pre-trained short-video models to longer sequences without additional training. The key methodology is Multi-band SpectralFusion (MSF), which uses multiple attention branches with varying temporal window sizes to capture dynamics at different scales, followed by fusing their outputs in the frequency domain using scale-specific band-pass filters. On the Wan-2.1 model extended to 4x its native length, FreeLong++ achieved an Imaging Quality score of 68.82, outperforming direct sampling (60.52) and the prior FreeNoise method (67.00). For AI practitioners, the principal implication is that FreeLong++ offers a plug-and-play module to adapt existing video diffusion models for high-quality, long-form video generation, bypassing the need for costly retraining by directly addressing signal degradation in the frequency domain. |
| Peccavi: Visual Paraphrase Attack Safe and Distortion Free Image |
|
|
| Watermarking Technique for AI-Generated Images (Read more on arXiv or HuggingFace) |
Vasu Sharma, Shashwat Bajpai, Ashhar Aziz, Shreyas Dixit, amanchadha |
This paper introduces PECCAVI, a distortion-free image watermarking technique designed to be resilient against generative visual paraphrase attacks. The primary objective is to develop a watermarking method that can withstand removal by visual paraphrase attacks, where a generative model alters an image’s visual style while preserving its core semantic content. The methodology identifies stable semantic regions called Non-Melting Points (NMPs) using saliency detection (XRAI) and intersection analysis across multiple paraphrased image versions, then embeds multi-channel, frequency-domain watermarks into these NMPs, and uses noisy burnishing to prevent reverse-engineering. The primary result shows PECCAVI’s superior robustness; against a visual paraphrase attack of strength 0.2, it achieved an Average Watermark Detection Probability (WDP) of 0.87, outperforming the state-of-the-art method ZoDiac which scored 0.70. The principal implication for AI practitioners is that this technique provides a more durable method for watermarking AI-generated content, making it more resilient to removal by other generative models; the most impactful finding is that embedding watermarks in semantically invariant regions (NMPs) is a highly effective strategy against content-aware de-watermarking attacks. |
Papers for 2025-07-01
| Title |
Authors |
Summary |
| Ovis-U1 Technical Report (Read more on arXiv or HuggingFace) |
Pengxin Zhan, Liangfu Cao, Xinjie Zhang, Shanshan Zhao, Flourish |
This report introduces Ovis-U1, a 3-billion-parameter unified model integrating multimodal understanding, text-to-image generation, and image editing. The research objective is to develop an effective architecture and unified training procedure for a model that performs both understanding and generation, starting from a base language model rather than a pre-trained MLLM. The methodology involves augmenting a Qwen3-1.7B LLM with a diffusion-based visual decoder and a bidirectional token refiner, trained via a six-stage process on a diverse mix of understanding, generation, and editing data. Ovis-U1 achieves a score of 69.6 on the OpenCompass Multi-modal Academic Benchmark, surpassing other models in its parameter class, and scores 4.00 on the ImgEdit-Bench. For AI practitioners, the principal implication is that a unified training approach, which integrates generation and understanding tasks from an early stage, yields superior performance in both domains compared to training on siloed tasks, demonstrating a synergistic benefit from co-training. |
| VMoBA: Mixture-of-Block Attention for Video Diffusion Models (Read more on arXiv or HuggingFace) |
Ye Tian, Xin Tao, Haotian Yang, Jianzong Wu, lianghou |
This paper introduces VMoBA, a sparse attention mechanism that adapts Mixture-of-Block Attention to efficiently train and run Video Diffusion Models on long video sequences. The primary objective is to mitigate the quadratic complexity of full attention in Video Diffusion Models (VDMs) to enable efficient training and inference on long-duration, high-resolution videos. The method enhances the MoBA framework by introducing a layer-wise recurrent 1D-2D-3D block partitioning scheme for spatio-temporal data, a global block selection algorithm to prioritize important query-key interactions, and a threshold-based mechanism to dynamically determine the number of attended blocks. Experiments show VMoBA accelerates VDM training on longer sequences, achieving up to a 1.48x training speedup while attaining comparable or superior generation quality (VBench score 68.34 vs. 68.25) compared to full attention. For AI practitioners, VMoBA offers a method to significantly reduce the computational cost and time for training high-fidelity VDMs on longer video sequences, making the development of such models more feasible without sacrificing output quality. |
| Calligrapher: Freestyle Text Image Customization (Read more on arXiv or HuggingFace) |
Ka Leong Cheng, Hao Ouyang, Qingyan Bai, Yue Ma, JingyeChen22 |
Calligrapher is a diffusion-based framework for freestyle text image customization, enabling the transfer of artistic styles from arbitrary reference images onto target text. The primary objective is to automate typography generation by precisely emulating a reference style while ensuring accurate character rendering and seamless integration into the source image. The core methodology involves a self-distillation pipeline to auto-construct a style-centric typography benchmark and a localized style injection mechanism, which uses a trainable style encoder (with a Qformer) to inject style features into the cross-attention layers of a frozen diffusion model. Calligrapher significantly outperforms state-of-the-art baselines, achieving a Fréchet Inception Distance (FID) of 38.09, compared to the next-best score of 66.68 from TextDiffuser-2, and was preferred by users 72% of the time. The principal implication for AI practitioners is the provision of a highly effective and efficient method for style adaptation in generative models, which automates complex typographic design and introduces a scalable self-distillation strategy for creating specialized training data without manual annotation. |
| Listener-Rewarded Thinking in VLMs for Image Preferences (Read more on arXiv or HuggingFace) |
Anton Gusarov, Andrey Galichin, Li Pengyi, barracuda049, alexgambashidze |
This paper introduces a listener-augmented reinforcement learning framework to improve vision-language model (VLM) reasoning for human visual preferences. The primary objective is to address a failure mode in reinforcement learning where a model’s reasoning trace contradicts its final decision, thereby improving generalization and alignment with human intent. The key methodology is a listener-augmented Group Relative Policy Optimization (GRPO) framework where a frozen, independent VLM (the “listener”) re-evaluates the “reasoner” model’s chain-of-thought to generate a dense, calibrated confidence score that shapes the RL reward signal, penalizing unpersuasive explanations. The proposed listener-shaped reward scheme achieves state-of-the-art accuracy of 67.4% on the ImageReward benchmark, improves out-of-distribution performance by up to +6% over a naive reasoner, and reduces reasoning contradictions from 10.1% to 8.3%. The principal implication for AI practitioners is that this listener-based reward provides a scalable and data-efficient method for aligning generative models, enabling the development of more robust VLMs that produce not only correct but also persuasive and consistent explanations without requiring new, expensive human annotation. |
| SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via |
|
|
| Multi-Agent Multi-Turn Reinforcement Learning (Read more on arXiv or HuggingFace) |
Penghui Qi, Leon Guertler, lkevinzc, simonycl, Benjamin-eecs |
This paper introduces SPIRAL, a framework where language models autonomously improve their reasoning by playing multi-turn, zero-sum games against themselves. The research objective is to determine if competitive self-play, without human-curated data, can cultivate transferable reasoning skills that generalize to academic benchmarks. The core methodology is a fully online, multi-agent reinforcement learning system using a novel Role-conditioned Advantage Estimation (RAE) technique to stabilize training. The primary result shows that training a Qwen3-4B model on Kuhn Poker alone achieves an 8.6% improvement on math and 8.4% on general reasoning benchmarks, outperforming supervised fine-tuning on 25,000 expert trajectories. For AI practitioners, this implies that complex reasoning can be developed through autonomous, competitive interaction, suggesting that game environments can serve as a scalable alternative to expensive, human-curated datasets for enhancing core cognitive abilities. |
| Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective (Read more on arXiv or HuggingFace) |
LidongBing, Zhiqiang007, Jianyu |
This research demonstrates that pruning in-context learning (ICL) demonstrations into syntactically incoherent “gibberish” significantly improves LLM performance and introduces PROMPTQUINE, an evolutionary framework to automate this discovery. The primary objective is to investigate if pruning ICL demonstrations into seemingly nonsensical forms can outperform conventionally well-crafted, natural language prompts. The paper proposes PROMPTQUINE, a self-replicating framework based on a Genetic Algorithm, which evolves a population of pruned prompts by applying mutations (token removal) and using task performance as a fitness function for selection. This method consistently outperforms baselines; for instance, on Llama-3-8B-Instruct, 4-shot PROMPTQUINE achieved an average classification accuracy of 81.3%, a significant improvement over the original 4-shot ICL’s 72.0%. The principal implication for AI practitioners is that optimal prompts may not be human-intuitive, and employing open-ended, evolutionary search algorithms is a highly effective strategy for prompt optimization, suggesting a shift from natural language design to exploring the model’s preferred input structures. |
| Consistent Time-of-Flight Depth Denoising via Graph-Informed Geometric |
|
|
| Attention (Read more on arXiv or HuggingFace) |
Di Qiu, Jin Zeng, Changyong He, weidawang |
The paper introduces GIGA-ToF, a denoising network that enhances temporal consistency and spatial sharpness in Time-of-Flight (ToF) depth videos by fusing motion-invariant graph structures across frames. The objective is to develop a ToF depth video denoising method that resolves the trade-off between spatial sharpness and temporal consistency by leveraging the temporal self-similarity of geometric graph structures. The method formulates denoising as a Maximum a Posteriori (MAP) problem with a graph Laplacian smoothness prior defined on a fused graph and a data fidelity term based on ToF noise statistics; this MAP problem is then unrolled into an iterative deep network where a Graph-Informed Geometric Attention (GIGA) module learns to fuse intra-frame graphs from consecutive frames. On the synthetic DVToF dataset, the proposed GIGA-ToF model achieves state-of-the-art performance, outperforming competing methods by at least 37.9% in Mean Absolute Error (MAE) and 13.2% in Temporal End-Point Error (TEPE), while also demonstrating strong generalization to real, unseen Kinectv2 data. AI practitioners can use this method to obtain significantly cleaner and more temporally stable depth streams from commodity ToF sensors, providing a higher-quality input for downstream tasks like 3D reconstruction and robotic navigation, with the added benefit of an interpretable model architecture derived from algorithm unrolling. |
| Aha Moment Revisited: Are VLMs Truly Capable of Self Verification in |
|
|
| Inference-time Scaling? (Read more on arXiv or HuggingFace) |
Kaizhuo Yan, Jize Jiang, Jingcheng Yang, Meitang Li, Mingyuan1997 |
This paper investigates the effectiveness of inference-time self-verification techniques in reinforcement learning (RL)-trained Vision-Language Models (VLMs), revealing a significant gap between their generation and verification abilities. The research aims to determine whether RL-trained VLMs genuinely benefit from self-correction and “aha moments” or if these behaviors are surface-level artifacts that do not improve reasoning performance. The study compares the performance of generation-reliant strategies (e.g., majority voting) against verification-reliant strategies (e.g., self-verified Best-of-N) on RL-tuned VLMs using the GeoQA and MathVista benchmarks. The primary result shows that generation-heavy methods consistently outperform verification-based methods; for instance, on GeoQA, a VLM’s accuracy decreased by as much as 16.7% with self-verification compared to greedy decoding, and models often performed verification better when the visual input was omitted. The principal implication for practitioners is that self-verification mechanisms from LLMs do not directly translate to VLMs, which currently lack robust multimodal verification capabilities, indicating that simply applying RL-tuning is insufficient and new methods are needed to bridge this generation-verification gap. |
| MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame |
|
|
| Optical Flow Estimation (Read more on arXiv or HuggingFace) |
Dmitriy Vatolin, Egor Chistov, Vladislav Bargatin, a-yakovenko |
The paper introduces MEMFOF, a multi-frame optical flow model that achieves state-of-the-art accuracy at high resolutions while being significantly more memory-efficient than existing methods. The objective is to enable the training and inference of optical flow models on high-resolution (1080p) video without prohibitive GPU memory costs, thus avoiding performance degradation from input downsampling. The methodology extends a two-frame RAFT-like architecture to a three-frame, bidirectional paradigm and drastically reduces memory by lowering the correlation volume’s working resolution from the standard 1/8 to 1/16 of the input size, coupled with a high-resolution training strategy on upscaled datasets. MEMFOF ranks first on the Spring benchmark with a 1-pixel outlier rate of 3.289 while requiring only 2.09 GB of GPU memory for 1080p inference, a nearly 4x reduction compared to the baseline RAFT. This allows AI practitioners to deploy high-accuracy, multi-frame optical flow models on high-resolution video using consumer-grade GPUs, enabling applications previously limited by memory constraints without sacrificing fine motion details. |
| SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity (Read more on arXiv or HuggingFace) |
Ligeng Zhu, Junxian Guo, Xiuyu Li, zhijianliu, Skhaki |
SparseLoRA accelerates LLM fine-tuning by applying structured, contextual sparsity to reduce computational load while preserving the efficiency of LoRA. The primary objective is to reduce the computational cost of parameter-efficient fine-tuning, which existing methods like LoRA and QLoRA do not address. The core methodology involves a lightweight, training-free SVD sparsity estimator that dynamically selects a sparse subset of weights for computation, combined with a sensitivity analysis that applies sparsity non-uniformly across layers, tokens, and training steps. On the Math10K benchmark with a LLaMA3-8B model, SparseLoRA achieved a 1.6x fine-tuning speedup over standard LoRA by reducing FLOPs by 54% while maintaining comparable accuracy. For AI practitioners, this method offers a way to significantly decrease fine-tuning time and cost in compute-bound scenarios without sacrificing model performance or requiring complex modifications to existing training pipelines. |
| MARBLE: A Hard Benchmark for Multimodal Spatial Reasoning and Planning (Read more on arXiv or HuggingFace) |
Maria Brbić, Yekun Chai, mdmoor, yljblues |
This paper introduces MARBLE, a challenging benchmark to evaluate the step-by-step multimodal spatial reasoning and planning capabilities of Multimodal Language Models (MLLMs). The primary objective is to scrutinize the ability of current MLLMs to solve complex problems that require crafting and understanding multi-step plans under spatial, visual, and physical constraints, moving beyond simple information retrieval. The methodology involves two new tasks: M-PORTAL, inspired by the game Portal 2, which tests spatial planning, and M-CUBE, a 3D assembly puzzle, which tests spatial reasoning and perception. The primary result is that current MLLMs perform extremely poorly, with all 12 evaluated models achieving near-random performance on M-PORTAL and a 0% accuracy on the full M-CUBE task. The principal implication for AI practitioners is that state-of-the-art MLLMs are not yet capable of robust, deep spatial reasoning or planning, revealing significant limitations in both their perceptual abilities and their capacity for structured, sequential thought that must be considered before deployment in embodied or physically-grounded applications. |
| Teaching a Language Model to Speak the Language of Tools (Read more on arXiv or HuggingFace) |
s-emanuilov |
This paper presents TUCAN, a series of Bulgarian language models adapted from the BgGPT family to provide robust function-calling capabilities in a non-English context. The primary objective is to develop and validate a methodology for enabling reliable tool use in language models for non-English languages, addressing the capability gap in multilingual tool integration. The methodology involved parameter-efficient fine-tuning (LoRA) of the BgGPT model series (2.6B, 9B, 27B) on a newly created bilingual dataset of 10,035 function-calling examples. The resulting TUCAN models demonstrated significant improvements in tool-use accuracy, with the TUCAN-2.6B model achieving a 28.75% absolute increase over its base model, while preserving core linguistic competence on established Bulgarian benchmarks. The principal implication for AI practitioners is the provision of a practical, open-source blueprint—including models, dataset, and an evaluation framework (Tucan-Eval)—for extending tool-augmented capabilities to other languages, enabling the development of production-ready AI agents beyond English-centric systems. |
| UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence |
|
|
| with Spatial Reasoning and Understanding (Read more on arXiv or HuggingFace) |
Yong Li, Yanxin Xi, Tianhui Liu, Shengyuan Wang, JJ-TMT |
The paper introduces UrbanLLaVA, a multi-modal large language model fine-tuned on a new urban instruction dataset to perform diverse intelligence tasks involving geospatial, trajectory, street view, and satellite data. The primary objective is to develop a unified MLLM capable of simultaneously processing four major types of urban data (visual, geo-text, structured geospatial, and spatiotemporal series) to achieve comprehensive spatial reasoning and understanding across a range of urban tasks. The methodology involves creating UData, a diverse urban instruction dataset; proposing UTrain, a three-stage training framework that decouples task alignment from knowledge learning; and developing UBench, an extended benchmark for evaluation. On the UBench benchmark for Beijing, UrbanLLaVA significantly outperforms its VILA1.5-8B base model, achieving a performance increase of 375.38% on the Geo+Traj task, and demonstrates gains ranging from 3.48% to 132.23% over the best-performing baselines across all tasks. For AI practitioners, this research provides a framework for adapting general MLLMs to specialized, multi-modal domains like urban intelligence by demonstrating that targeted data curation and a staged fine-tuning process can yield substantial performance improvements for complex reasoning tasks. |
| VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs (Read more on arXiv or HuggingFace) |
Yifan Zao, Junyoung Park, Mukul Gagrani, Sudhanshu Agrawal, Raghavv Goel |
The paper introduces VOCABTRIM, a training-free method that prunes the vocabulary of the drafter model to accelerate speculative decoding in LLMs. The research objective is to mitigate the significant inference overhead caused by the drafter model’s large language modeling (LM) head, particularly in memory-bound deployment scenarios. The methodology involves reconstructing the drafter’s LM head to only contain a limited set of high-frequency tokens, which are identified by analyzing a calibration dataset of target model generations, thus reducing the drafter’s parameter count and latency. When applied to a Llama-3.2-3B-Instruct model with an Eagle drafter, VOCABTRIM increased the memory-bound speed-up (MBSU) by up to 19% on the Dolly task (from 1.52 to 1.809) with a minimal drop in block efficiency. For AI practitioners, this provides a simple, plug-and-play technique to improve the throughput of LLMs on resource-constrained hardware by reducing the memory footprint and computational cost of the speculative decoding drafter, without requiring any model retraining. |
| RoboScape: Physics-informed Embodied World Model (Read more on arXiv or HuggingFace) |
Chen Gao, Lei Jin, Yinzhou Tang, Xin Zhang, Yu Shang |
RoboScape is a physics-informed embodied world model that improves the physical plausibility of generated robotic videos by integrating depth and motion dynamics. The primary objective is to create a unified world model that generates realistic videos for contact-rich robotic scenarios by overcoming the physical unawareness of existing models. The methodology involves a dual-branch auto-regressive Transformer that jointly learns to predict RGB video and temporal depth maps, combined with an adaptive keypoint dynamics learning task that enforces temporal consistency on high-motion object regions to implicitly model physical properties. In policy evaluation experiments, RoboScape demonstrated a Pearson correlation of 0.953 between policy success rates predicted by the model and those in the ground-truth simulator, significantly outperforming baselines. The principal implication for AI practitioners is the ability to use RoboScape to generate large-scale, physically consistent synthetic data, which can be used to effectively train and evaluate robotic policies, reducing the need for real-world data and improving simulation-to-reality transfer. |
| Degradation-Modeled Multipath Diffusion for Tunable Metalens Photography (Read more on arXiv or HuggingFace) |
Xiaokang Yang, Feiyu Ji, Jiayi Zhu, Jianing Zhang, XiaoyunYuan |
The paper presents Degradation-Modeled Multipath Diffusion (DMDiff), a framework for restoring images from an ultra-compact, millimeter-scale metalens camera. The objective is to reconstruct high-fidelity images from inputs with complex, spatially varying optical and sensor-induced degradations, without relying on large paired datasets, and providing tunable control over the restoration process. The methodology leverages a pre-trained diffusion model fine-tuned with LoRA, guided by a novel Spatially Varying Degradation-Aware (SVDA) attention module that quantifies degradation using both simulated Point Spread Functions and a no-reference image quality metric. The system uses a multipath training strategy with positive, neutral, and negative prompts to balance detail enhancement, structural fidelity, and suppression of metalens-specific artifacts. The proposed method outperforms state-of-the-art baselines, achieving a MUSIQ score of 51.85, significantly higher than SwinIR (36.86) and OSEDiff (34.52), while enabling an instantly tunable trade-off between fidelity and perceptual quality. The principal implication for AI practitioners is that it provides a concrete methodology for adapting large generative models to specialized hardware with complex, non-ideal degradation characteristics by integrating physics-based simulations and data-driven quality metrics into the fine-tuning process, reducing the need for extensive, precisely aligned training data. |
| ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language |
|
|
| Models for Audio Generation and Editing (Read more on arXiv or HuggingFace) |
Qian Chen, Wen Wang, Kaicheng Luo, Jialei Wang, Huadai Liu |
ThinkSound is a framework that leverages Chain-of-Thought (CoT) reasoning within a Multimodal Large Language Model (MLLM) to enable a three-stage, interactive process for high-fidelity video-to-audio generation and editing. The primary objective is to overcome the limitations of end-to-end video-to-audio (V2A) models by decomposing the complex synthesis task into explicit, stepwise reasoning stages, thereby improving the contextual fidelity, temporal precision, and user controllability of the generated audio. The methodology uses an MLLM (fine-tuned VideoLLaMA2) on a newly introduced AudioCoT dataset to generate CoT instructions that guide a unified, flow-matching-based audio foundation model through initial foley generation, interactive object-centric refinement, and targeted editing from natural language commands. Experiments demonstrate that ThinkSound achieves state-of-the-art performance on the VGGSound test set, improving the CoT-based alignment score (CLAP_CoT) to 0.46 compared to the 0.40 of the strong baseline MMAudio, with performance degrading to 0.41 when CoT reasoning is removed. The principal implication for AI practitioners is that decomposing complex generative tasks using MLLM-driven CoT reasoning can significantly enhance output quality and control; this paradigm of separating high-level reasoning from low-level synthesis is applicable to other multimodal domains to create more precise and interactive systems beyond monolithic end-to-end architectures. |
| Tower+: Bridging Generality and Translation Specialization in |
|
|
| Multilingual LLMs (Read more on arXiv or HuggingFace) |
Pedro Teixeirinha, João Alves, José Pombal, Nuno M. Guerreiro, RicardoRei |
This paper introduces TOWER+, a suite of multilingual LLMs that achieve strong performance in both machine translation and general-purpose capabilities, addressing the common trade-off between specialization and generality. The primary objective is to develop state-of-the-art translation models without compromising their core instruction-following and conversational skills. The authors employ a novel four-stage post-training pipeline consisting of continued pre-training (CPT), supervised fine-tuning (SFT), preference optimization (WPO/GRPO), and reinforcement learning with verifiable rewards (RLVR). The resulting 72B model substantially improves general-purpose performance over its predecessor (TOWER-V2) from a 4.01 to a 54.52 win rate on M-ArenaHard, while maintaining state-of-the-art translation quality (83.74 XCOMET-XXL on WMT24++). For AI practitioners, this work provides a blueprint for adapting base LLMs to specialized business domains, such as localization, without sacrificing the general capabilities essential for complex, real-world applications. |
Papers for 2025-06-30
| Title |
Authors |
Summary |
| BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing (Read more on arXiv or HuggingFace) |
Sanghyun Woo, Saining Xie, Xuhui Jia, Ramin Mehran, cccjc |
BlenderFusion is a generative framework for visual scene creation by recomposing objects, camera perspectives, and backgrounds through layering, editing, and compositing. The research aims to enable precise 3D-aware visual compositing by integrating generative models with 3D graphics tools like Blender. It utilizes a dual-stream diffusion model fine-tuned on video frames with source masking and simulated object jittering to enhance object control. Experiments on MOVi-E, Objectron, and Waymo datasets show BlenderFusion improves object-level and image-level metrics (e.g., achieving a FID score of 9.11 on MOVi-E), demonstrating better foreground and background modeling. The framework’s use of 3D grounding with Blender allows for flexible visual compositing tasks, offering AI practitioners a way to create and manipulate visual data more effectively. |
| LLaVA-Scissor: Token Compression with Semantic Connected Components for |
|
|
| Video LLMs (Read more on arXiv or HuggingFace) |
Qibin Hou, Xihan Wei, Jiaxing Zhao, Boyuan Sun |
i) LLaVA-Scissor is a training-free token compression strategy for video multimodal large language models (VLLMs) that leverages Semantic Connected Components (SCC). ii) The paper aims to develop a token compression strategy that effectively reduces redundancy and comprehensively represents semantic regions in video data for efficient VLLM processing. iii) The methodology involves a two-step spatio-temporal token compression utilizing SCC in both spatial and temporal domains, partitioning tokens into non-overlapping semantic regions based on pairwise similarity and a threshold, with a final average merge. iv) Experimental results show that LLaVA-Scissor outperforms other token compression methods, achieving superior performance on video understanding benchmarks; specifically, a retention ratio of 5% achieves performance close to 10% retention ratios of other methods. v) LLaVA-Scissor provides AI practitioners with an effective training-free inference method to process long videos with reduced computational cost, especially crucial for resource-constrained deployments of VLLMs, enabling processing of long-form videos more efficiently. |
| XVerse: Consistent Multi-Subject Control of Identity and Semantic |
|
|
| Attributes via DiT Modulation (Read more on arXiv or HuggingFace) |
Xu Wang, Li Chen, Haomiao Sun, Mengyi Zhao, chenbowen |
i) XVerse introduces a DiT-based framework for consistent multi-subject image generation with control over identity and semantic attributes. ii) The research aims to improve multi-subject image generation by addressing attribute entanglement and maintaining individual identity fidelity. iii) The methodology involves transforming reference images into token-specific text-stream modulation offsets and integrating VAE-encoded image features. iv) XVerse achieves an overall score of 73.40 on a new benchmark XVerseBench, outperforming other methods in both single and multi-subject generation. v) AI practitioners can leverage XVerse’s text-stream modulation and VAE integration to enhance fine-grained control and consistency in personalized and complex scene generation tasks. |
| ShotBench: Expert-Level Cinematic Understanding in Vision-Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Yuhao Dong, Dian Zheng, Yi Jin, Jingwen He, Hongbo Liu |
i) ShotBench introduces a new benchmark for evaluating cinematic understanding in Vision-Language Models (VLMs). ii) The research aims to assess VLMs’ proficiency in comprehending cinematographic language beyond basic visual understanding. iii) The methodology involves creating ShotBench, a dataset with over 3.5k expert-annotated QA pairs from over 200 films, and evaluating 24 leading VLMs. iv) The evaluation reveals limitations in current VLMs, with the top-performing model achieving less than 60% average accuracy, and introduces ShotVL, which gains 19.0% compared to the original Qwen2.5-VL-3B model. v) The study provides AI practitioners with ShotBench and ShotQA to enhance VLM capabilities in fine-grained visual comprehension and AI-assisted video generation by using ShotVL. |
| From Ideal to Real: Unified and Data-Efficient Dense Prediction for |
|
|
| Real-World Scenarios (Read more on arXiv or HuggingFace) |
Minnan Luo, Zhuohang Dang, Chengyou Jia, Changliang Xia |
This paper introduces DenseDiT, a data-efficient framework for dense prediction across real-world scenarios. The research aims to address the limitations of existing methods in generalizing to complex, data-scarce real-world dense prediction tasks. They use a generative model-based approach with parameter-reuse and lightweight branches for multi-scale context integration. Evaluation on the introduced DenseWorld benchmark shows DenseDiT achieves superior performance using less than 0.01% of baseline training data, achieving an average D-Score of 0.944. This demonstrates the potential for practical real-world deployment using pre-trained generative models for diverse dense prediction tasks. |
| MiCo: Multi-image Contrast for Reinforcement Visual Reasoning (Read more on arXiv or HuggingFace) |
Xiaogang Xu, Xiaoyang Wu, Shaoteng Liu, Mingkang Zhu, Xi Chen |
i) MiCo introduces a self-supervised contrastive learning framework to improve multi-image visual reasoning in VLMs through rule-based reinforcement learning. ii) The research aims to enable chain-of-thought reasoning for linking visual cues across multiple images in VLMs without relying on manually curated question-answer pairs. iii) The methodology involves constructing image triplets (two augmented views of the same image and a similar but distinct image), prompting the VLM to compare images, and optimizing the model using augmented GRPO. iv) Experiments show that MiCo achieves significant improvements on multi-image reasoning benchmarks, with a reported improvement of +12.93 on VLM2-Bench without human annotated question-answer pairs. v) The principal implication for AI practitioners is a method to enhance VLMs’ reasoning abilities by leveraging inherent image constraints, reducing the reliance on manually constructed training data and improving generalization to complex visual reasoning tasks. |
| Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs (Read more on arXiv or HuggingFace) |
Xiaofeng Zhang, Xu Cao, Jingyuan Zhu, Yuanzhe Liu, Yifan Shen |
SpatialReasoner-R1, a novel VLM, addresses limitations in fine-grained spatial reasoning. The research investigates how to improve spatial reasoning in vision-language models, particularly with complex logic and precise alignment. They employ Multi-Model Monte Carlo Tree Search (M3CTS) to generate diverse LongCoT reasoning trajectories and introduce fine-grained Direct Preference Optimization (fDPO) with segment-specific preference granularity. The fDPO training achieves an average improvement of 4.1% over standard DPO across spatial quality tasks, and a 9.0% gain in spatial quantity tasks. SpatialReasoner-R1 outperforms the strongest baseline on SPATIALRGPT-BENCH by 9.8% in average accuracy and this new SOTA model provides AI practitioners with a more robust and accurate tool for spatial reasoning tasks. |
| Ark: An Open-source Python-based Framework for Robot Learning (Read more on arXiv or HuggingFace) |
Jiacheng Qiu, Huang Helong, Sarthak Das, Christopher E. Mower, Magnus Dierking |
Ark is introduced as a new open-source, Python-first robotics framework designed to bridge the gap between AI and robotics. The primary objective is to provide a Gym-style environment for data collection, preprocessing, and policy training with imitation learning algorithms, facilitating seamless transition between simulation and real-world robots. The methodology involves a client-server architecture, ROS interoperability, and reusable modules for control, SLAM, and visualization. Ark achieves rapid prototyping and hardware swapping, unifying robotics and AI practices. The framework demonstrates the capability of switching between simulated and real-world environments by toggling a single configuration flag. The principal implication is that Ark lowers the entry barrier for AI practitioners to develop and deploy autonomous robots by providing a unified Python interface and streamlined machine-learning workflows. |
| Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity (Read more on arXiv or HuggingFace) |
Wei Guo, Xiaosong Li, MightyCrane, Fangcheng2, tangyehui |
i) Pangu Pro MoE, a 72B parameter sparse language model, introduces a Mixture of Grouped Experts (MoGE) architecture for balanced computational load across distributed devices. ii) The research aims to improve training and inference throughput by mitigating expert load imbalance in Mixture of Experts (MoE) models. iii) MoGE partitions experts into groups, enforcing a fixed number of experts activated per group during token routing, and optimizes model configuration for Ascend NPUs through system simulation. iv) Inference performance achieves 1148 tokens/s per card on Ascend NPUs, and can be further improved to 1528 tokens/s with speculative decoding, and improves Model FLOPs Utilization (MFU) by 35%. v) AI practitioners can leverage MoGE to design and deploy sparse models with enhanced load balancing and improved throughput, particularly in distributed inference scenarios using Ascend NPUs. |
| Noise Consistency Training: A Native Approach for One-Step Generator in |
|
|
| Learning Additional Controls (Read more on arXiv or HuggingFace) |
Jing Tang, Tianyang Hu, Shuchen Xue, Yihong Luo |
i) This paper introduces Noise Consistency Training (NCT), a lightweight approach for integrating new control signals into pre-trained one-step generators. ii) The research aims to adapt one-step generative models to new control conditions without requiring retraining of the base diffusion model or access to original training data. iii) NCT employs an adapter module and a noise consistency loss in the generator’s noise space, aligning the adapted model’s generation behavior across varying noise levels. iv) Experiments demonstrate NCT achieves state-of-the-art controllable generation in a single forward pass, reducing function evaluations (NFEs) from 50 to 1 while maintaining or surpassing baseline performance. v) NCT offers AI practitioners a modular, data-efficient, and easily deployable method to enhance pre-trained generators with new controls, improving computational efficiency in AIGC applications. |
| The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT |
|
|
| Improvements (Read more on arXiv or HuggingFace) |
Roberta Raileanu, Xian Li, Minqi Jiang, Despoina Magka, Bingchen Zhao |
The paper introduces the Automated LLM Speedrunning Benchmark to assess LLMs’ ability to reproduce existing LLM training improvements. The main research objective is to evaluate AI agents’ capability to reproduce research results in the NanoGPT speedrun competition. The methodology involves providing agents with previous records’ training scripts and hints of varying detail levels. The primary result showed recent reasoning LLMs, even with detailed hints, struggled to replicate known innovations, recovering less than 20% of speedups without hints, and around 40%-46% of speedup with certain hints. This benchmark provides a non-saturated measure of LLMs’ automated scientific reproduction, highlighting the need for improved reproducibility skills for autonomous research agents. |
| Gazal-R1: Achieving State-of-the-Art Medical Reasoning with |
|
|
| Parameter-Efficient Two-Stage Training (Read more on arXiv or HuggingFace) |
Amr Fawzy, Mostafa Samy, Ahmed M. Adly |
i) Gazal-R1, a 32B parameter language model, achieves state-of-the-art results in medical reasoning through parameter-efficient two-stage training. ii) The research aims to develop a medical LLM with superior reasoning capabilities compared to larger models, while maintaining transparency and explainability. iii) The methodology involves supervised fine-tuning (SFT) on a synthetic medical reasoning dataset, enhanced with DoRA and rsLORA, followed by reinforcement learning using Group Relative Policy Optimization (GRPO). iv) Gazal-R1 attains 87.1% on MedQA, 81.6% on MMLU Pro (Medical), and 79.6% on PubMedQA, outperforming models up to 12x larger. v) This demonstrates the effectiveness of strategic training and parameter-efficient techniques for developing high-performance domain-specific language models. |
| Confucius3-Math: A Lightweight High-Performance Reasoning LLM for |
|
|
| Chinese K-12 Mathematics Learning (Read more on arXiv or HuggingFace) |
Yitao Duan, Jiachen Wang, Qiao Cheng, Na Cai, nomadlx |
i) Confucius3-Math, a 14B parameter open-source LLM, achieves state-of-the-art performance on Chinese K-12 mathematics reasoning tasks. ii) The research aims to develop a cost-effective and high-performance LLM specifically for Chinese K-12 mathematics education. iii) The methodology involves post-training a base model with reinforcement learning, incorporating techniques like Targeted Entropy Regularization, Recent Sample Recovery, and Policy-Specific Hardness Weighting. iv) Confucius3-Math achieves SOTA performance on K-12 math benchmarks and obtains a 15.8x speedup in inference compared to DeepSeek-R1 on comparable hardware, all while reducing costs. v) The paper demonstrates that a low-cost, domain-specific reasoning model can outperform larger general models, implying that AI/ML practitioners can achieve substantial performance gains by focusing on specialized model training. |
| RetFiner: A Vision-Language Refinement Scheme for Retinal Foundation |
|
|
| Models (Read more on arXiv or HuggingFace) |
Hrvoje Bogunović, Ursula Schmidt-Erfurth, José Morano, Ronald Fecso |
RetFiner introduces a vision-language refinement scheme to enhance retinal foundation models (FMs). The research aims to improve the semantic understanding of existing retinal FMs by incorporating textual data from Electronic Health Records (EHRs). The methodology refines vision encoders of existing retinal FMs using a combination of image-text contrastive (ITC), image-text matching (ITM), masked language modeling (MLM), and generative modeling (GM) losses. The refined FMs showed an average increase of 5.8 percentage points in linear probing performance on seven OCT classification tasks. RetFiner offers AI practitioners an efficient SSL method to adapt existing FMs to specific medical imaging populations with limited annotation. |
Papers for 2025-06-27
| Title |
Authors |
Summary |
| MMSearch-R1: Incentivizing LMMs to Search (Read more on arXiv or HuggingFace) |
Bo You, Yiding Liu, Wei Li, Zihao Deng, kimingng |
i) MMSearch-R1 is a reinforcement learning framework enabling LMMs to perform on-demand search in real-world internet environments. ii) The research aims to incentivize LMMs to learn when and how to utilize image and text search tools effectively for visual question answering (VQA). iii) The methodology involves an end-to-end reinforcement learning (RL) approach with a group relative policy optimization (GRPO) algorithm, incorporating an outcome-based reward with a search penalty to encourage efficient search behavior and using image and text search tools. iv) The model not only outperforms RAG-based baselines of the same model size, but also matches the performance of a larger RAG-based model while reducing search calls by over 30%. v) MMSearch-R1 provides AI practitioners with a framework for training LMMs to intelligently integrate external knowledge sources, reducing reliance on static knowledge bases and improving performance on knowledge-intensive tasks. |
| MADrive: Memory-Augmented Driving Scene Modeling (Read more on arXiv or HuggingFace) |
Maria Golitsyna, Ruslan Musaev, Kirill Struminsky, Polina Karpikova, apryc1 |
i) MADrive introduces a memory-augmented framework for photorealistic driving scene reconstruction and novel view synthesis by retrieving and integrating 3D vehicle assets from an external database. ii) The main objective is to enhance existing scene reconstruction methods by replacing partially observed vehicles with realistically reconstructed counterparts to support photorealistic synthesis of altered or novel driving scenarios. iii) The key methodology involves curating MAD-Cars, a dataset of ~70K 360° car videos, developing a retrieval module to find similar car instances, and integrating reconstructed 3D assets into the target scene through orientation alignment and relighting. iv) Primary results show MADrive achieves a MOTA score of 0.810 for multi-object tracking and a segmentation IoU of 0.822, demonstrating improved rendering quality for downstream perception tasks compared to existing methods. v) The principal implication for AI practitioners is a framework for generating realistic driving simulation data with altered configurations, potentially improving the training and robustness of autonomous driving systems. |
| WorldVLA: Towards Autoregressive Action World Model (Read more on arXiv or HuggingFace) |
Siteng Huang, Yuming Jiang, Chaohui Yu, Jun Cen, JacobYuan |
WorldVLA is presented as an autoregressive action world model for unified action and image understanding and generation. The research aims to integrate VLA models and world models into a single framework to improve action generation through environmental physics learning. The methodology involves employing three tokenizers for images, text, and actions within a single LLM architecture, using an attention mask strategy to address action prediction errors. Experiments on the LIBERO benchmark show WorldVLA outperforms action models with a 4% increase in grasping success rate and reduces Fréchet Video Distance by 10% compared to vanilla world models. The attention masking strategy improved the grasping success rate by 4% to 23% in action chunk generation, addressing performance degradation; the mutual enhancement of both world and action models in a unified framework offers AI practitioners a more robust approach to robotic tasks requiring visual and action understanding. |
| Where to find Grokking in LLM Pretraining? Monitor |
|
|
| Memorization-to-Generalization without Test (Read more on arXiv or HuggingFace) |
Ziyue Li, zhoutianyi, Fcr09 |
i) This paper investigates grokking during large language model (LLM) pretraining, demonstrating asynchronous memorization and a transition to generalization. ii) The research aims to understand the dynamics of grokking in LLM pretraining and identify internal changes that enable generalization. iii) The study analyzes the routing pathways within a 7B parameter MoE LLM, introducing metrics for pathway similarity and consistency. iv) The research shows pathway consistency strongly correlates with test accuracy (often exceeding 0.97), indicating coherent routing is a key marker of generalization. v) AI practitioners can use pathway metrics to monitor and predict generalization during LLM pretraining without reliance on finetuning or test data. |
| Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge (Read more on arXiv or HuggingFace) |
Yu Gu, Zanming Huang, yhshu, nnnyt, BoyuNLP |
Mind2Web 2 introduces a benchmark for evaluating agentic web search systems. The research aims to address the limitations of existing benchmarks by creating realistic, long-horizon tasks and a novel Agent-as-a-Judge evaluation framework. It utilizes a tree-structured rubric to assess answer correctness and source attribution. Evaluation of nine agentic search systems, including OpenAI Deep Research, demonstrated varying performance, with the best system achieving 50-70% of human performance while halving the time; the task completion success rate is about 28% for agents and about 54% for humans. Mind2Web 2 offers a foundation for developing and benchmarking next-generation agentic search systems by providing a dataset with realistic web search tasks and an automated evaluation methodology. |
| SAM4D: Segment Anything in Camera and LiDAR Streams (Read more on arXiv or HuggingFace) |
Sheng Yang, Chunyong Hu, Ziqian Ni, Jianyun Xu, songw-zju |
SAM4D introduces a multi-modal, temporal foundation model for promptable segmentation across camera and LiDAR streams. The research objective is to enable 2D-3D joint segmentation with cross-modal prompting and temporal alignment using an architecture built upon a multi-modal transformer. The key methodology involves a Unified Multi-modal Positional Encoding (UMPE) and Motion-aware Cross-modal Memory Attention (MCMA) within a multi-modal transformer architecture. Experiments on the constructed Waymo-4DSeg dataset, containing over 300k camera-LiDAR associated masklets, demonstrated an average cross-modal IoU of 0.56 for the generated masklets. The primary implication for AI practitioners is the potential for significantly reduced annotation costs via the proposed automated data engine and enhanced cross-modal segmentation for autonomous driving applications. |
| FaSTA^*: Fast-Slow Toolpath Agent with Subroutine Mining for Efficient |
|
|
| Multi-turn Image Editing (Read more on arXiv or HuggingFace) |
Dang Nguyen, Rishie Raj, Advait Gupta, zhoutianyi |
i) The paper introduces FaSTA, a cost-efficient neurosymbolic agent for multi-turn image editing that combines fast LLM planning with slow A search and subroutine mining. ii) The research objective is to reduce the computational cost of toolpath search in multi-turn image editing by reusing knowledge from previously explored tasks. iii) FaSTA* employs LLMs for high-level subtask planning and inductive reasoning to extract reusable symbolic subroutines, coupled with a cost-sensitive A* search for individual subtasks triggered adaptively. iv) Experiments show FaSTA* reduces cost by 49.3% compared to COSTA* while maintaining competitive success rates with a 3.2% quality degradation. v) This work offers AI practitioners a method for significantly reducing the computational expense of complex image editing tasks by leveraging reusable subroutines, making large-scale applications more feasible. |
| Whole-Body Conditioned Egocentric Video Prediction (Read more on arXiv or HuggingFace) |
Trevor Darrell, Yann LeCun, Amir Bar, dans123, Emma02 |
PEVA models egocentric video prediction conditioned on whole-body human motion represented by 3D pose trajectories. The research investigates how effectively such a model can simulate actions and their visual consequences from a first-person perspective. It trains a conditional diffusion transformer autoregressively using the Nymeria dataset of real-world egocentric video and body pose data, incorporating random timeskips and sequence-level training. PEVA demonstrates improved performance over baselines, achieving a LPIPS score of 0.303 on single-step prediction tasks, showing enhanced action consistency and generative quality. This whole-body-conditioned model offers AI practitioners a method for generating more realistic and controllable embodied simulations through capturing intricate relationships between physical movement and resulting visual changes. Some of the evaluation results have error bounds however. |
| Arch-Router: Aligning LLM Routing with Human Preferences (Read more on arXiv or HuggingFace) |
Adil Hafeez, Co Tran, nehcgs, parachas |
i) Arch-Router is introduced as a framework for aligning large language model (LLM) routing with human preferences. ii) The main research objective is to guide model selection by matching queries to user-defined domains or action types, encoding preferences in routing decisions. iii) The methodology involves training a compact 1.5B model, Arch-Router, to map queries to domain-action preferences for model routing decisions, alongside a data creation pipeline to generate labeled conversations. iv) Experiments on conversational datasets demonstrate that Arch-Router achieves state-of-the-art results in matching queries with human preferences, outperforming top proprietary models by 7.71% on average. v) The principal implication is that AI practitioners can use Arch-Router to implement more transparent and flexible LLM routing systems that align with subjective human evaluations, offering a practical mechanism for operationalizing diverse LLMs. |
| FairyGen: Storied Cartoon Video from a Single Child-Drawn Character (Read more on arXiv or HuggingFace) |
Xiaodong Cun, Jiayi Zheng |
i) FairyGen is presented as a framework for generating multi-shot cartoon videos from a single child-drawn character image. ii) The research aims to create stylistically consistent and narratively coherent animations that reflect the artistic style of the input character while achieving natural motion. iii) The method employs a multimodal large language model (MLLM) for storyboarding, a style propagation adapter for background generation, and a 3D proxy-based motion generation technique fine-tuned with an MMDiT-based image-to-video diffusion model. iv) Experiments demonstrate the system’s ability to generate personalized animated stories, where the proposed method achieved a style alignment score of 0.6580 compared to baselines. v) AI practitioners can leverage FairyGen’s framework for personalized content creation and engaging story animation by utilizing the proposed style propagation and motion customization techniques. |
| DiLoCoX: A Low-Communication Large-Scale Training Framework for |
|
|
| Decentralized Cluster (Read more on arXiv or HuggingFace) |
YingJun Wu, Ming Wu, Li Li, WenPeng Zhu, Ji Qi |
DiLoCoX is a framework for training large language models on decentralized clusters with low communication bandwidth. The research investigates how to pre-train models exceeding 100 billion parameters on decentralized clusters while maintaining model convergence. The paper combines pipeline parallelism with dual optimizer policy, one-step-delay overlap of communication and local training, and an adaptive gradient compression scheme. The results show DiLoCoX achieves a 357x speedup compared to vanilla AllReduce when pre-training a 107B model over a 1Gbps network. The findings demonstrate a method for training large models on less powerful decentralized infrastructure and the efficiency of DiLoCoX for distributed training. |
| An Agentic System for Rare Disease Diagnosis with Traceable Reasoning (Read more on arXiv or HuggingFace) |
Pengcheng Qiu, Xiaoman Zhang, Yanjie Fan, Chaoyi Wu, Weike Zhao |
DeepRare, a novel agentic system, addresses the challenge of rare disease diagnosis. The research aims to create an LLM-powered system capable of processing heterogeneous clinical inputs (free-text, HPO terms, VCF files). DeepRare employs a three-tier architecture integrating a central host with specialized agent servers and curated knowledge sources. The system achieves a 57.18% average Recall@1 score on HPO-based evaluations, surpassing existing methods. The verified 95.40% agreement with clinical experts on reasoning chains suggests that DeepRare provides trustworthy decision support. The system provides AI practitioners with a framework for building interpretable and adaptable diagnostic tools. |
| HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial |
|
|
| Optimization Challenges (Read more on arXiv or HuggingFace) |
Jiang Bian, Lei Song, Haolong Qian, Ling Zhang, VictorYXL |
i) The paper introduces HeurAgenix, a two-stage hyper-heuristic framework using large language models (LLMs) for solving combinatorial optimization (CO) problems. ii) The research aims to automate heuristic design and adaptive selection for complex CO problems, improving upon traditional methods that rely on manual expertise. iii) HeurAgenix employs a contrastive, data-driven approach for heuristic evolution, using an LLM to analyze solution tuples and extract reusable strategies, coupled with an adaptive selection mechanism integrating LLMs and Test-time Scaling (TTS). iv) Experiments show HeurAgenix outperforms existing LLM-based hyper-heuristics and matches or exceeds specialized solvers on canonical benchmarks and reduces the average optimality gap from 5.01% to 0.59% on average across several combinatorial optimization problems through dual reward fine tuning. v) AI practitioners can leverage HeurAgenix to automate the design and selection of heuristics for complex CO problems, potentially enabling scalable and generalizable solutions with increased adaptability and reduced reliance on manual rule design. |
| Learning to Skip the Middle Layers of Transformers (Read more on arXiv or HuggingFace) |
Laurence Aitchison, tim-lawson |
i) The paper proposes a novel Transformer architecture with a gating mechanism to dynamically skip middle layers based on input token complexity. ii) The research investigates whether skipping redundant middle layers in Transformers, as suggested by interpretability research, can improve the trade-off between performance and computational cost. iii) The methodology involves learning a gating mechanism that bypasses a symmetric span of central blocks, combined with gated attention and sandwich/peri-layernorm schemes, plus adaptive regularization to encourage sparsity. iv) At the scales investigated, the proposed architecture does not improve the trade-off between validation cross-entropy and estimated FLOPs compared to dense baselines with fewer layers. v) The implication for AI practitioners is that the specific middle-layer skipping strategy explored here does not demonstrably improve efficiency in Transformers within the tested configurations and scales, warranting exploration of different conditional computation strategies or scaling to larger models. |
| MuseControlLite: Multifunctional Music Generation with Lightweight |
|
|
| Conditioners (Read more on arXiv or HuggingFace) |
Bo-Rui Chen, Sheng-Ping Yang, Weijaw Lee, Shih-Lun Wu, fundwotsai2001 |
i) MuseControlLite is introduced as a lightweight fine-tuning mechanism for controllable text-to-music generation. ii) The research objective is to enhance the control accuracy of text-to-music generation models using time-varying musical attributes and reference audio signals with reduced trainable parameters. iii) The methodology involves augmenting a decoupled cross-attention mechanism with positional embeddings in diffusion Transformers. iv) Results show that adding rotary positional embeddings increases control accuracy from 56.6% to 61.1% in melody control while using 6.75 times fewer trainable parameters, with 85M trainable parameters in total. v) MuseControlLite offers AI practitioners a parameter-efficient fine-tuning strategy for integrating time-varying musical conditions into pre-trained text-to-music models, facilitating creative applications like audio inpainting and outpainting with improved controllability. |
Papers for 2025-06-26
| Title |
Authors |
Summary |
| ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Ke Ji, Shunian Chen, Zhenyang Cai, Junying Chen, cppppppc |
i) This paper introduces ShareGPT-4o-Image, a dataset for distilling GPT-4o’s image generation capabilities into open multimodal models. ii) The main objective is to democratize advanced image generation by providing a synthetic dataset to improve open-source models. iii) The methodology involves synthesizing 45K text-to-image and 46K text-and-image-to-image samples using GPT-4o and fine-tuning Janus-Pro on this data to create Janus-4o. iv) Primary results show Janus-4o achieves a 4-point improvement over Janus-Pro on the EvalGen benchmark in text-to-image generation and attains impressive text-and-image-to-image performance with only 91K synthetic samples and 6 hours of training. v) The principal implication for AI practitioners is that high-quality synthetic data distilled from proprietary models can significantly enhance the performance of open-source multimodal models, enabling state-of-the-art image generation capabilities with limited resources. |
| Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Jaewoo Kang, Hyeon Hwang, Chanwoong Yoon, Taewhoo Lee, affjljoo3581 |
i) The paper introduces Outlier-Safe Pre-Training (OSP), a novel guideline to prevent outlier formation in LLMs to improve 4-bit quantization. ii) The research aims to mitigate activation outliers in Large Language Models (LLMs) to enhance quantization performance for efficient deployment. iii) The methodology combines the Muon optimizer, single-scale RMSNorm (SSNORM), and learnable embedding projection (EMBPROJ). iv) The OSP model achieved a 35.7 average score across 10 benchmarks under aggressive 4-bit quantization, contrasting with 26.5 for an Adam-trained model, and exhibited a 0.04 excess kurtosis value compared to 1818.56. v) AI practitioners can use OSP to train LLMs that are more robust to quantization, potentially reducing deployment overhead in resource-constrained environments by preventing outliers rather than mitigating them post-hoc. |
| DualTHOR: A Dual-Arm Humanoid Simulation Platform for Contingency-Aware |
|
|
| Planning (Read more on arXiv or HuggingFace) |
Hang Xu, Siyuan He, Boyu Li, WizardTY, tellarin |
DualTHOR is a physics-based simulation platform built upon AI2-THOR for developing embodied AI agents with dual-arm humanoid robots. The research objective is to create a simulation environment addressing limitations in current platforms, such as simplified robot morphologies and bypassed low-level execution stochasticity. The key methodology involves integrating real-world robot assets, a dual-arm task suite, humanoid inverse kinematics solvers, and a contingency mechanism simulating potential execution failures. Extensive evaluations reveal that current Vision-Language Models struggle with dual-arm coordination and show limited robustness in realistic environments with contingencies, success rates vary from 9.71% to 36.54% on dual-arm essential tasks. DualTHOR offers AI practitioners a more comprehensive benchmark for evaluating and improving the robustness and generalization capabilities of VLMs in complex household environments. |
| OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling (Read more on arXiv or HuggingFace) |
Pengfei Liu, Xuefeng Li, Fan Zhou, Zengzhi Wang |
OctoThinker investigates mid-training strategies to improve reinforcement learning (RL) scaling for language models, specifically Llama and Qwen. The research question is how mid-training strategies influence RL dynamics in language models. The methodology involves controlled mid-training interventions with varying datasets (e.g., MegaMath-Web-Pro, QA-style data) followed by RL training using the verl framework and GRPO algorithm. The primary result shows that a two-stage mid-training strategy (Stable-then-Decay) on 200B tokens with constant learning rate followed by 20B tokens across three CoT-focused branches yields OctoThinker, with RL performance matching Qwen2.5. The principal implication for AI practitioners is that strategic mid-training, particularly using high-quality mathematical corpora and QA-style data, can significantly enhance the RL compatibility of base language models, leading to improved downstream reasoning capabilities. |
| Use Property-Based Testing to Bridge LLM Code Generation and Validation (Read more on arXiv or HuggingFace) |
Jing Shao, Zhe Zhang, Lehan He, lsheng2024, zx55 |
i) The paper introduces Property-Generated Solver (PGS), a novel framework utilizing Property-Based Testing (PBT) to enhance the correctness and robustness of code generated by Large Language Models (LLMs). ii) The research aims to improve LLM-based code generation by employing property-based testing for validation, addressing the limitations of traditional test-driven development. iii) PGS uses two collaborative LLM agents: a Generator for code synthesis and iterative refinement, and a Tester for managing the PBT lifecycle and providing semantically rich feedback from property violations. iv) Experiments on multiple code generation benchmarks demonstrate that PGS achieves pass@1 improvements, ranging from 23.1% to 37.3% relative gains over established TDD methods. v) The research implies that AI practitioners can leverage property-based testing frameworks, like PGS, to systematically improve the reliability and correctness of LLM-generated code, particularly in complex programming tasks where traditional test case generation is insufficient. |
| RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain |
|
|
| Randomization for Robust Bimanual Robotic Manipulation (Read more on arXiv or HuggingFace) |
Yibin Liu, Zijian Cai, Baijun Chen, Zanxin Chen, TianxingChen |
i) RoboTwin 2.0 is presented as a scalable framework for bimanual robotic manipulation data generation and benchmarking. ii) The main objective is to enhance the robustness and generalization of bimanual manipulation policies through simulation. iii) The methodology involves an expert data generation pipeline using multimodal large language models with simulation-in-the-loop refinement and structured domain randomization. iv) A vision-language-action model fine-tuned on RoboTwin 2.0 data achieved a 367% relative improvement on unseen scene real-world tasks; 10.9% gain in code generation success rate was demonstrated. v) RoboTwin 2.0 provides AI practitioners with a data generation and benchmarking platform to train and evaluate bimanual manipulation policies exhibiting improved sim-to-real transfer capabilities. |
| Is There a Case for Conversation Optimized Tokenizers in Large Language |
|
|
| Models? (Read more on arXiv or HuggingFace) |
Pedro Reviriego, Gonzalo Martínez, Javier Conde, Raquel Ferrando |
i) This paper investigates the potential benefits of conversation-optimized tokenizers for Large Language Models (LLMs) to improve energy efficiency. ii) The main research question is whether optimizing tokenizers specifically for chatbot conversations can reduce the number of tokens and improve energy efficiency compared to tokenizers trained on general text corpora. iii) The methodology involves retraining existing tokenizers using a publicly available chatbot conversation dataset (LMSYS Chat 1M) and comparing their performance against the original tokenizers on both conversational and general text corpora (C4). iv) The primary result shows that conversation-optimized tokenizers consistently reduce the number of tokens in chatbot dialogues, achieving savings in the range of 5% to 10% for some tokenizers, while having minimal impact on tokenization efficiency for the original training corpus. v) AI practitioners can potentially reduce computational costs and improve energy efficiency in chatbot applications by adopting conversation-optimized tokenizers; however, trade-offs related to training costs and downstream model performance should be carefully evaluated. |
| When Life Gives You Samples: The Benefits of Scaling up Inference |
|
|
| Compute for Multilingual LLMs (Read more on arXiv or HuggingFace) |
Sara Hooker, Julia Kreutzer, Ye Shen, Daniel D’souza, ammar-cohere |
i) This paper investigates strategies for scaling inference compute in multilingual large language models (LLMs) for open-ended generative tasks. ii) The research question addresses how to efficiently allocate a fixed inference compute budget to improve performance across diverse languages and tasks. iii) The methodology involves evaluating existing sampling and selection methods and proposing novel techniques like hedged sampling, Checklisted One-Pass Selection (CHOPS), and Cross-lingual Minimum Bayes Risk (X-MBR). iv) Results indicate that the proposed methods yield notable gains, specifically showing a +9.0 improvement in win-rates for the Command-A (111B) model on m-ArenaHard-v2.0 with just five samples against single-sample decoding. v) AI practitioners should consider language- and task-aware approaches to inference-time compute allocation, aiming to democratize performance improvements in underrepresented languages. |
| ReCode: Updating Code API Knowledge with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Ningyu Zhang, Huajun Chen, Wenhao Yu, Yunzhi Yao, Haoze Wu |
i) ReCode improves LLMs’ code generation with updated API knowledge via rule-based reinforcement learning. ii) The paper addresses the research question of how to effectively update LLMs’ code generation abilities to accommodate frequent API changes in external libraries. iii) The methodology includes constructing a dataset of approximately 2,000 API migration examples and using a modified string similarity metric as the reward function for reinforcement learning with GRPO and DAPO algorithms. iv) Qwen2.5-Coder-7B trained with ReCode achieved a higher Pass@1 score on the CodeUpdateArena than Qwen2.5-Coder-32B, increasing Pass@1 by 11.3%. v) ReCode provides AI practitioners a framework for enhancing code LLMs’ adaptability to evolving APIs, minimizing the impact of outdated training data on code generation tasks in dynamic environments. |
| HiWave: Training-Free High-Resolution Image Generation via Wavelet-Based |
|
|
| Diffusion Sampling (Read more on arXiv or HuggingFace) |
Farnood Salehi, Tobias Vontobel, RMW, msadat97 |
HiWave presents a training-free approach for high-resolution image generation using pre-trained diffusion models. The research aims to enhance visual fidelity and structural coherence in ultra-high-resolution image synthesis from pre-trained diffusion models without retraining. The methodology employs a two-stage pipeline: base image generation from a pre-trained model, followed by patch-wise DDIM inversion and a wavelet-based detail enhancer module preserving low-frequency structure while guiding high-frequency components. User studies showed HiWave was preferred over the state-of-the-art alternative in more than 80% of comparisons, highlighting its effectiveness. The primary implication for AI practitioners is a method to improve the perceptual quality of generated ultra-high-resolution images without architectural modifications or retraining, potentially enabling higher fidelity outputs in creative applications. |
| Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency |
|
|
| Models (Read more on arXiv or HuggingFace) |
Aibek Alanov, Andrey Kuznetsov, Ilia Beletskii |
i) This paper introduces a cycle-consistency optimization framework for enhancing image inversion in fast image editing using consistency models. ii) The main objective is to improve image reconstruction quality in distilled diffusion models for higher-fidelity image editing. iii) The methodology involves fine-tuning a forward consistency model (fCM) using a cycle-consistency loss to reduce structural and semantic differences between original images and their reconstructions. iv) The proposed method achieves state-of-the-art performance in image editing tasks, matching or surpassing full-step diffusion models while being substantially more efficient, reducing LPIPS score by at least 0.04 compared to other fast methods for image reconstruction on MS-COCO dataset. v) The cycle-consistency optimization can enable AI practitioners to achieve faster and more effective image editing with distilled diffusion models, while retaining high reconstruction fidelity and controllability. |
| The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs (Read more on arXiv or HuggingFace) |
Carlos C. N. Kuhn, adnaan525 |
i) This paper introduces the Debugging Decay Index (DDI) to quantify and optimize iterative debugging effectiveness in code-generating LLMs. ii) The research investigates how to maximize the effectiveness of LLM-generated code debugging and develops a unified evaluation metric that encompasses reasoning proficiency and instruction-following competency. iii) The methodology involves modeling debugging effectiveness using an exponential decay function, fitting it to empirical data from LLM debugging attempts on the HumanEval dataset, and implementing strategic fresh starts at DDI-calculated intervention points. iv) Results show that LLM debugging effectiveness follows a predictable exponential decay pattern, and strategic fresh starts improve accuracy, as demonstrated by Llama3.1:8b increasing baseline accuracy from 72.56% to 82.82%. v) AI practitioners can utilize DDI to determine optimal debugging windows and improve iterative code generation strategies by implementing fresh starts, mitigating performance degradation. |
| Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining |
|
|
| and Extracting Rare and Hidden Content (Read more on arXiv or HuggingFace) |
Eric de la Clergerie, Nathan Godey, rntc |
i) Biomed-Enriched is introduced, a biomedical dataset constructed from PubMed using a two-stage LLM-annotation process for refined subset extraction. ii) The research aims to create a biomedical text dataset that addresses the lack of accessible clinical text and improves biomedical pretraining efficiency. iii) 400K PubMed paragraphs were annotated with scores for type, domain, and educational quality using a large language model, followed by fine-tuning a smaller model to propagate labels across the full PMC-OA corpus. iv) Clinical upsampling boosted performance by 5% on MMLU ProfMed, and combining techniques led to faster convergence, reaching the same performance with a third of the training tokens. v) AI practitioners can leverage this dataset to more efficiently pretrain language models for biomedical applications, particularly when focusing on clinical text or educationally valuable content, thereby reducing computational costs. |
| MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility |
|
|
| Applications (Read more on arXiv or HuggingFace) |
Paul Laban, Matt Laing, AleksandrAlgazinov |
i) The paper introduces MATE, an open-source, lightweight multi-agent system (MAS) for multimodal accessibility, enabling modality conversions based on user needs. ii) The main research objective is to design a flexible MAS architecture to adapt to diverse accessibility requirements in real-time. iii) The methodology involves developing specialized agents utilizing LLM APIs and custom ML classifiers, along with a dataset (ModConTT) for training and evaluation. iv) The ModCon-Task-Identifier, a fine-tuned BERT model, achieves a classification accuracy of 0.917 and F1-score of 0.916 on the ModConTT dataset, outperforming other LLMs and statistical models. v) The principal implication is that MATE offers a customizable and adaptable framework for AI practitioners developing accessibility solutions, leveraging MAS to address modality conversion challenges, although it lacks support for video generation capabilities and relies on external models whose performance can be variable. |
Papers for 2025-06-25
| Title |
Authors |
Summary |
| AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion |
|
|
| Models (Read more on arXiv or HuggingFace) |
lsheng2024, pookiefoof, Yang-Tian, fenghora, huanngzh |
i) AnimaX is a feed-forward 3D animation framework transferring video diffusion model motion priors to skeleton-based animation for diverse meshes. ii) The research aims to efficiently animate articulated 3D meshes with arbitrary skeletal structures using video diffusion model motion priors. iii) The methodology involves a joint video-pose diffusion model conditioned on template renderings and textual motion prompts, representing 3D motion as multi-view 2D pose maps. iv) Evaluated on VBench, AnimaX demonstrates state-of-the-art results in generalization, motion fidelity, and efficiency, trained on a dataset of 160,000 rigged sequences. v) AnimaX offers AI practitioners a scalable, category-agnostic 3D animation solution, enabling efficient and versatile animation generation for diverse articulated meshes. |
| Matrix-Game: Interactive World Foundation Model (Read more on arXiv or HuggingFace) |
Qingcheng Zhu, Puyi Wang, Boyang Wang, Chunli Peng, Vanint |
i) Matrix-Game introduces a world foundation model for controllable game world generation trained on a two-stage pipeline. ii) The main objective is to develop an interactive image-to-world generation model that can be precisely controlled and maintains visual quality and temporal coherence. iii) The model uses a controllable image-to-world generation paradigm, conditioned on a reference image, motion context, and user actions, trained on the newly curated Matrix-Game-MC dataset. iv) Experiments show Matrix-Game outperforms previous Minecraft world models across all metrics of the GameWorld Score benchmark, particularly in controllability and physical consistency, and a human evaluation confirmed its superiority in generating realistic and controllable videos. v) The release of Matrix-Game model weights and the GameWorld Score benchmark provides AI practitioners with a new interactive world generation framework and a standardized tool for evaluating world models. |
| GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Junhao Cheng, Yixiao Ge, Rui Wang, Yuying Ge, Yi Chen |
GRPO-CARE introduces a consistency-aware reinforcement learning framework for improving multimodal reasoning in large language models. The research aims to address limitations of outcome-supervised GRPO, where answer accuracy is prioritized over logical reasoning consistency. The methodology involves an adaptive, group-relative consistency bonus based on reference-likelihood calibration in addition to base rewards for answer correctness. Results demonstrate a 6.7% performance gain on the most challenging level of SEED-Bench-R1 and a 24.5% improvement in consistency rate compared to standard GRPO. The framework’s enhanced reasoning coherence and improved interpretability offers AI practitioners a method for training more reliable and transparent multimodal reasoning systems. |
| Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Changshi Li, Yuzhen Xiao, chrisliu298, lycfight, zengliangcs |
i) The paper introduces Skywork-SWE, a large-scale dataset and model for software engineering tasks in LLMs. ii) The main objective is to systematically scale and analyze software engineering dataset volume and diversity to understand data scaling laws in LLMs. iii) The methodology involves an automated data curation pipeline to generate over 8,000 runtime-validated training trajectories and fine-tuning a Qwen2.5-Coder-32B-based model. iv) The Skywork-SWE model achieves 38.0% pass@1 accuracy on the SWE-bench Verified benchmark and improves to 47.0% with test-time scaling, surpassing previous SOTA results for models under 32B parameters. v) The identified data scaling laws suggest that increasing high-quality, execution-grounded data substantially improves LLM performance in software engineering, providing a practical guideline for AI practitioners to further enhance LLM capabilities. |
| ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality |
|
|
| Debiasing (Read more on arXiv or HuggingFace) |
Pan Zhang, Xiaoyi Dong, Long Xing, yuhangzang, shikiw |
ScaleCap is introduced as an inference-time scalable image captioning strategy for generating detailed captions. The research addresses the challenge of multimodal and linguistic biases in LVLMs to enhance caption quality. ScaleCap employs heuristic question answering and contrastive sentence rating for caption enrichment and hallucination reduction, respectively. Experiments show ScaleCap-450K improves pretraining efficiency, achieving superior performance on 11 benchmarks; for example, it improves InfoVQA scores by 4.3% over ShareGPT4V-450k in Qwen2.5-7B. ScaleCap enables AI practitioners to generate higher-quality image captions for improved vision-language model training and downstream task performance. |
| SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in |
|
|
| Real-World Applications (Read more on arXiv or HuggingFace) |
Per Jacobsson, Ge Qu, Jinyang Li, Tebmer, xia01ongLi |
i) This paper introduces a new benchmark and training environment for SQL issue debugging using Large Language Models (LLMs). ii) The main research objective is to address the gap in evaluating and improving LLMs’ ability to debug SQL issues distilled from authentic user scenarios. iii) The paper presents BIRD-CRITIC, a benchmark of 530 PostgreSQL and 570 multi-dialect SQL debugging tasks, along with SIX-GYM, a training environment utilizing SQL-Rewind and f-Plan Boosting. iv) Baseline evaluations on BIRD-CRITIC reveal a 38.87% success rate for the leading reasoning model (O3-MINI) on the PostgreSQL subset; BIRD-FIXER, fine-tuned on Qwen-2.5-Coder-14B, achieves 38.11% success rate on BIRD-CRITIC-PG. v) The introduction of BIRD-CRITIC, SIX-GYM, and BIRD-FIXER enables AI practitioners to evaluate and improve LLMs’ ability to debug SQL queries effectively, and the f-Plan Boosting demonstrates a mechanism for improving the effectiveness of LLM trajectory training. |
| Can Large Language Models Capture Human Annotator Disagreements? (Read more on arXiv or HuggingFace) |
Alexander Hoyle, Donya Rooein, Vilém Zouhar, Yu Fan, JingweiNi |
i) This paper evaluates LLMs’ ability to predict human annotator disagreement in NLP tasks. ii) The central research question is whether LLMs can effectively model informative human annotation variance without access to repeated human labels. iii) The methodology involves evaluating various LLMs (8B-671B parameters) across different training paradigms (RLHF, RLVR) and prompting strategies on five NLP datasets using variance correlation and distributional alignment metrics. iv) Results indicate that RLVR-style reasoning significantly harms disagreement prediction, with the verbalized distribution approach outperforming the sampling-based approach in disagreement prediction. v) The implication is that AI practitioners should exercise caution when using LLMs (particularly RLVR-tuned models) as annotators for subjective tasks, as these models may overlook critical human disagreements and that more focus on evaluating LLMs on these types of tasks is needed. |
| JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo |
|
|
| Retouching Agent (Read more on arXiv or HuggingFace) |
Panwang Pan, Jinbin Bai, Kunjie Lin, Zixu Lin, LYL1015 |
i) The paper introduces JarvisArt, an MLLM-driven intelligent agent for photo retouching. ii) The primary objective is to develop an AI agent that can understand user intent, mimic professional artists’ reasoning, and orchestrate Lightroom’s retouching tools. iii) The methodology involves a two-stage training process: Chain-of-Thought supervised fine-tuning followed by Group Relative Policy Optimization for Retouching (GRPO-R), and an Agent-to-Lightroom Protocol for seamless integration. iv) JarvisArt demonstrates improved content fidelity, outperforming GPT-40 with a 60% improvement in average pixel-level metrics on MMArt-Bench. v) The principal implication for AI practitioners is a new avenue for intelligent photo retouching with user-friendly interaction, superior generalization, and fine-grained control, which can inform the development of more sophisticated, user-guided AI editing tools. |
| SRFT: A Single-Stage Method with Supervised and Reinforcement |
|
|
| Fine-Tuning for Reasoning (Read more on arXiv or HuggingFace) |
Xihuai Wang, Jiajun Chai, Tinghong Chen, SONGJUNTU, Yuqian-Fu |
i) This paper introduces Supervised Reinforcement Fine-Tuning (SRFT), a single-stage method unifying supervised fine-tuning (SFT) and reinforcement learning (RL) for large language model (LLM) reasoning. ii) The research aims to address the challenge of optimally integrating SFT and RL in LLM fine-tuning to enhance reasoning capabilities. iii) The methodology involves an entropy-aware weighting mechanism to simultaneously apply SFT and RL, leveraging demonstrations and self-exploration rollouts in a single optimization stage. iv) Experimental results demonstrate that SRFT achieves 59.1% average accuracy on mathematical reasoning benchmarks, outperforming zero-RL methods by 9.0%. v) SRFT offers AI practitioners a method for effectively combining SFT and RL in a single training phase, improving LLM reasoning performance and generalization with entropy-aware weighting. |
| SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution (Read more on arXiv or HuggingFace) |
Xintao Wang, Menghan Xia, Shian Du, Yu Li, Liangbin Xie |
SimpleGVR presents a latent-cascaded video super-resolution (VSR) baseline for efficient high-resolution video generation from large text-to-video (T2V) models. The research aims to improve cascaded VSR models by studying key design principles, specifically degradation strategies and training configurations. The methodology includes flow-based and model-guided degradation to generate training pairs, along with innovations in timestep sampling and attention mechanisms. Experiments show that SimpleGVR achieves higher quality 1080p videos from 512p outputs of a base T2V model and reduces computational overhead by 80% using sparse local attention compared to full self-attention. The work offers a simple and effective baseline, providing practical insights for AI practitioners in designing efficient cascaded video synthesis systems. |
| Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low |
|
|
| CFG Scales (Read more on arXiv or HuggingFace) |
Farnood Salehi, Tobias Vontobel, RMW, msadat97 |
i) The paper introduces Frequency-Decoupled Guidance (FDG) for conditional diffusion models, improving image quality at low classifier-free guidance (CFG) scales. ii) The research aims to enhance image quality and prompt alignment in CFG by analyzing and decoupling the effects of different frequency components. iii) FDG decomposes CFG into low- and high-frequency components, applying distinct guidance strengths to each, implemented using Laplacian pyramids as the frequency transform. iv) Experiments show FDG consistently improves FID and recall across datasets and models; for instance, EDM2-S achieved a FID of 5.44 with FDG compared to 9.77 with standard CFG. v) FDG provides AI practitioners a plug-and-play alternative to standard CFG, enhancing sample fidelity and diversity in conditional diffusion models without retraining, thus improving generative modeling for image synthesis. |
| Unified Vision-Language-Action Model (Read more on arXiv or HuggingFace) |
Yingyan Li, Junbo Zhang, Wenxuan Wang, Xinghang Li, Yuqi Wang |
i) The paper introduces UniVLA, a unified vision-language-action model that represents vision, language, and action as discrete tokens within an autoregressive framework. ii) The research aims to develop a unified model capable of multimodal outputs and supporting a wide range of tasks, including perception grounding, world modeling, and policy learning. iii) The methodology involves a unified token-based design, autoregressive sequence modeling, and world model integration during post-training using large-scale video data. iv) UniVLA achieves a 95.5% average success rate on the LIBERO benchmark and improves performance in downstream policy learning, particularly for long-horizon and out-of-distribution tasks. v) AI practitioners can leverage UniVLA’s architecture for more integrated cross-modal modeling and scalable video-based training, offering a potential direction for generalist embodied intelligence. |
| Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic |
|
|
| Empirical Study (Read more on arXiv or HuggingFace) |
Ziheng Zhang, Jintian Zhang, Yi Zhong, Yuqi Zhu, Ningyu |
Open-source LLMs underperform in data analysis tasks compared to proprietary models. The paper investigates methods to improve open-source LLMs for reasoning-intensive data analysis scenarios. The study evaluates models across data understanding, code generation, and strategic planning using a curated dataset. Strategic planning quality is identified as the primary determinant of model performance; high-quality training data proves more critical than data diversity for optimal performance; fine-tuning a 7B model with data synthesis methodology achieved comparable or superior results to GPT-4o, with the 14B model’s gains diminished at larger scale. The findings imply that improvements to reasoning processes within data synthesis can significantly enhance the analytical capabilities of open-source LLMs, directly benefiting AI practitioners by enabling more effective use of smaller LLMs for complex data analysis. |
| Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text (Read more on arXiv or HuggingFace) |
Michalis Vazirgiannis, Yang Zhang, guokan-shang, amr-mohamed |
i) This paper evaluates Large Language Model (LLM) comprehension of code-switched text across various linguistic settings. ii) The research investigates how LLMs process and reason about mixed-language data, specifically focusing on reading comprehension, multi-domain knowledge, and natural language inference tasks. iii) The methodology involves generating code-switched variants of established benchmarks using both linguistically grounded and heuristic approaches and then evaluating LLM performance. iv) Results indicate that embedding non-English tokens in English matrix languages degrades performance, while embedding English tokens in other languages sometimes improves it; Llama 70B’s weighted average accuracy declined from 0.70 (English) to 0.66 on EN→AR/EN→DE v) AI/ML engineers should be aware that LLMs exhibit vulnerabilities to code-switching, particularly when English is the primary language, and that fine-tuning is a more reliable solution. |
| USAD: Universal Speech and Audio Representation via Distillation (Read more on arXiv or HuggingFace) |
Alexander H. Liu, James Glass, saurabhati, vectominist |
USAD proposes a universal audio representation model leveraging distillation to integrate speech, sound, and music. The main objective is to create a unified audio encoder capable of generalizing across various audio domains. The methodology involves layer-to-layer distillation from domain-specific self-supervised learning (SSL) models using a mixed audio dataset. USAD achieves competitive performance across SUPERB and HEAR benchmarks, exhibiting a 35.7 SUPERB score with the base model and 37.4 average performance score on HEAR with the large model; and demonstrating unified embedding space. AI practitioners can utilize USAD as a general-purpose audio encoder for downstream tasks across diverse audio types. |
| KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality (Read more on arXiv or HuggingFace) |
Huajun Chen, Wenhao Yu, Shuofei Qiao, Baochang Ren, Ningyu |
KnowRL explores integrating knowledge into reinforcement learning to enhance the factuality of slow-thinking LLMs. The research investigates how to mitigate hallucinations in slow-thinking models by incorporating a factuality reward based on knowledge verification into the RL training process. KnowRL trains models using a composite reward signal combining format, correctness, and factuality, evaluated on hallucination and reasoning benchmark datasets. Experiments show KnowRL mitigates hallucinations and maintains reasoning ability, evidenced by a 16.23% accuracy achievement on ChineseSimpleQA for Skywork-OR1-7B-Preview model. This framework implies that directly supervising the thinking process with factuality rewards is more effective for building reliable LLMs than solely optimizing for outcome accuracy. |
| Intelligent Operation and Maintenance and Prediction Model Optimization |
|
|
| for Improving Wind Power Generation Efficiency (Read more on arXiv or HuggingFace) |
Jiaqi He, Xiaobin Wu, Xun Liu, rajandasgupta |
i) This study examines predictive maintenance models and the optimization of intelligent Operation and Maintenance (O&M) systems for improved wind power generation efficiency. ii) The main objective is to analyze the effectiveness of predictive maintenance models in reducing downtime and to explore optimization strategies for intelligent O&M systems. iii) Qualitative research was conducted using structured interviews with five wind farm engineers and maintenance managers, followed by thematic analysis. iv) The study found that predictive maintenance models can reduce downtime by 20% but struggle with minor, gradual failures and false positives; sensor malfunctions and difficulties in integrating new models with older turbines are also problems. v) AI practitioners must address challenges in sensor data reliability, false positives, and seamless integration with legacy systems to improve the efficacy and reliability of predictive maintenance in operational wind turbine environments. |
| Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments |
|
|
| with a Hierarchical Spatial-Cognition Long-Short Memory System (Read more on arXiv or HuggingFace) |
Jie Feng, Yangcheng Yu, Zhenxing Chen, Haoyu Dong, Lixuan He |
Mem4Nav enhances vision-and-language navigation (VLN) in urban environments using a hierarchical spatial-cognition long-short memory system. The research objective is to improve embodied agents’ ability to navigate complex urban scenes by incorporating both fine-grained spatial detail and high-level landmark semantics. A dual-structured 3D map combining sparse octree indexing and a semantic topology graph, along with a reversible Transformer memory and short-term cache, is used. Mem4Nav achieved a 7-13 percentage point increase in Task Completion on Touchdown and Map2Seq datasets. AI practitioners can leverage this hierarchical memory system to improve the performance of VLN agents in complex, large-scale environments by incorporating efficient, lossless storage and retrieval of spatial information. |
Papers for 2025-06-24
| Title |
Authors |
Summary |
| Light of Normals: Unified Feature Representation for Universal |
|
|
| Photometric Stereo (Read more on arXiv or HuggingFace) |
Bohan Li, Zhaoxi Chen, Chongjie Ye, Houyuan Chen, Hong Li |
i) This paper introduces LINO-UniPS, a novel method for universal photometric stereo. ii) The research aims to improve surface normal recovery under complex lighting conditions by decoupling illumination and normal features and preserving high-frequency geometric details. iii) The methodology involves learnable light register tokens, a global cross-image attention mechanism, wavelet transform-based sampling, and a normal-gradient confidence loss. iv) LINO-UniPS demonstrates state-of-the-art performance on synthetic and real datasets; ablation studies showed improved CSIM and SSIM scores, indicating enhanced feature consistency. v) AI practitioners can leverage LINO-UniPS to develop more robust 3D reconstruction systems that are less sensitive to varying and uncalibrated lighting. |
| OmniGen2: Exploration to Advanced Multimodal Generation (Read more on arXiv or HuggingFace) |
yzwang, sienna223, Shitao, Ruiran, wcyno23 |
OmniGen2 is introduced as a versatile and open-source generative model for diverse generation tasks. The research aims to provide a unified solution for text-to-image, image editing, and in-context generation, employing distinct decoding pathways for text and image modalities. OmniGen2 uses comprehensive data construction pipelines and a reflection mechanism for image generation tasks, achieving competitive results with a relatively modest parameter size. On the OmniContext benchmark, OmniGen2 attains state-of-the-art consistency performance among open-source models, evaluated across eight task categories. The release of OmniGen2, including models, code, datasets, and pipelines, empowers AI practitioners with a unified generative model achieving competitive results on multiple benchmarks while maintaining strong text generation capabilities. |
| LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Juanzi Li, Roy Ka-Wei Lee, Yushi Bai, Yuhao Wu, Zhiqiang007 |
i) This paper introduces LongWriter-Zero, a reinforcement learning (RL) approach for mastering ultra-long text generation in large language models (LLMs). ii) The main objective is to develop an LLM capable of generating ultra-long, high-quality text without relying on supervised fine-tuning (SFT) on synthetic data. iii) The methodology involves training an LLM from scratch using RL with specialized reward models for length control, writing quality, and structural formatting, employing the Group Relative Policy Optimization (GRPO) algorithm. iv) Experimental results show LongWriter-Zero outperforms traditional SFT methods, achieving state-of-the-art results on WritingBench and Arena-Write benchmarks, surpassing even 100B+ models, and achieving an Elo rating of 1447 on Arena-Write. v) The implication for AI practitioners is a demonstration that RL can unlock ultra-long text generation capabilities in LLMs, offering an alternative to SFT that may lead to higher quality and more coherent long-form outputs, providing a training paradigm shift within LLM applications. |
| Phantom-Data : Towards a General Subject-Consistent Video Generation |
|
|
| Dataset (Read more on arXiv or HuggingFace) |
Crayon-Shinchan, onion-liu, TianxiangMa, lbc402, ZhuoweiChen |
i) The paper introduces Phantom-Data, a large-scale dataset for subject-consistent video generation. ii) The research aims to address the copy-paste problem in subject-to-video generation by creating a dataset that disentangles subject identity from background and contextual attributes. iii) The dataset construction involves a three-stage pipeline: subject detection, cross-context subject retrieval from a large video and image database, and prior-guided identity verification. iv) The dataset comprises approximately one million identity-consistent pairs and the use of Phantom-Data in training demonstrates improvements in prompt alignment and visual quality, maintaining identity consistency comparable to in-pair baselines. v) AI practitioners can leverage Phantom-Data to train subject-to-video generation models with improved generalization and reduced copy-paste artifacts. |
| ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought |
|
|
| Reasoning in LLMs (Read more on arXiv or HuggingFace) |
Ke Shen, Jiahao Qiu, Jingwen Gu, Ling Yang, Jiaru Zou |
i) This paper introduces ReasonFlux-PRM, a trajectory-aware process reward model for evaluating chain-of-thought reasoning in LLMs. ii) The research aims to improve reward modeling for intermediate reasoning steps in trajectory-response outputs, specifically addressing limitations of existing PRMs. iii) The methodology involves training a PRM incorporating both step-level and trajectory-level supervision on a curated dataset of trajectory-response pairs, adapting it for offline data selection and online reward modeling. iv) Empirical results demonstrate that ReasonFlux-PRM-7B achieves an average gain of 12.1% in supervised fine-tuning, 4.5% in reinforcement learning, and 6.3% in test-time scaling on downstream benchmarks. v) AI practitioners can leverage ReasonFlux-PRM to select higher quality distillation data and enhance reward signals for policy optimization, particularly in scenarios involving trajectory-response type outputs from frontier reasoning models. |
| RLPR: Extrapolating RLVR to General Domains without Verifiers (Read more on arXiv or HuggingFace) |
Zefan Wang, Shu Yao, Shouli Wang, Bo Ji, Tianyu Yu |
i) The paper introduces RLPR, a verifier-free framework to extrapolate Reinforcement Learning with Verifiable Rewards (RLVR) to general domains. ii) The research aims to overcome the reliance on domain-specific verifiers in RLVR by utilizing the intrinsic probability of Large Language Models (LLMs) for generating correct free-form answers as a reward signal. iii) The methodology involves replacing rule-based verifier rewards in RLVR with an intrinsic probability-based reward (PR), calculated from the average decoding probabilities of reference answer tokens, along with a debiasing technique and adaptive curriculum learning. iv) Experiments show that RLPR improves reasoning capabilities in both mathematical and general domains and outperforms VeriFree by 7.6 points on TheoremQA and 7.5 points on Minerva benchmarks. v) RLPR offers AI practitioners a simple, scalable approach to enhancing LLM reasoning without external verifiers, facilitating the utilization of general-domain data and broader application of RLVR. |
| Vision as a Dialect: Unifying Visual Understanding and Generation via |
|
|
| Text-Aligned Representations (Read more on arXiv or HuggingFace) |
Qi Zhao, Yang Zhao, Hao Chen, hywang66, csuhan |
i) The paper introduces Tar, a multimodal framework unifying visual understanding and generation through a shared, discrete, text-aligned representation. ii) The main objective is to create a multimodal LLM that can perform both visual understanding and generation tasks using a shared representation, eliminating the need for modality-specific designs. iii) The methodology involves a Text-Aligned Tokenizer (TA-Tok) that converts images into discrete tokens using a text-aligned codebook projected from a large language model’s vocabulary and generative de-tokenizers for producing high-fidelity visual outputs. iv) Experiments show Tar matches or surpasses existing multimodal LLM methods and on the DPG Bench, Tar-1.5B achieves a score of 82.96. v) The principal implication is that AI practitioners can use Tar for faster convergence and greater training efficiency in multimodal tasks, benefiting from a shared, discrete representation for both visual understanding and generation. |
| OAgents: An Empirical Study of Building Effective Agents (Read more on arXiv or HuggingFace) |
Yeyi Guan, Heyuan Huang, He Zhu, kangz, tianyue818 |
i) This paper introduces OAGENTS, a new modular agent framework designed to achieve state-of-the-art performance in agentic AI tasks. ii) The main objective is to empirically analyze the impact of various agent component designs on overall effectiveness, addressing the lack of standardization in agent research. iii) The methodology involves a systematic study on the GAIA benchmark, comparing different designs for planning, tool use, memory, and test-time scaling within the OAGENTS framework. iv) The primary results show that OAGENTS achieves a 73.93% average score on the GAIA benchmark, outperforming existing open-source agent frameworks, and demonstrates a 74.07% cross-modal task accuracy. v) The principal implication for AI practitioners is a modular, open-source framework, OAGENTS, which standardizes agent building components and evaluation, enabling more reliable comparisons and advancements in agentic AI. |
| VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed |
|
|
| View Memory (Read more on arXiv or HuggingFace) |
Tomas Jakab, Andrea Vedaldi, Philip Torr, Runjia Li |
i) The paper introduces Surfel-Indexed View Memory (VMem) for consistent, interactive video scene generation from a single image. ii) The main objective is to develop a memory mechanism that remembers and retrieves relevant past views geometrically to improve long-term consistency in autoregressive video generation. iii) The method indexes previous views using 3D surface elements (surfels) and retrieves relevant views based on the surfels visible from the new viewpoint to condition the generation. iv) Experiments on RealEstate10K demonstrate VMem outperforms existing methods in long-term scene consistency, with cycle trajectory translation distance reducing from 0.285 to 0.124. v) VMem provides AI practitioners with a plug-and-play module for geometrically indexing and retrieving relevant views, potentially enhancing the coherence of interactive video generation and scene exploration applications. |
| LettinGo: Explore User Profile Generation for Recommendation System (Read more on arXiv or HuggingFace) |
Jianfeng Liu, Pu Zhao, Fangkai Yang, Di Zhang, Lu Wang |
i) The paper introduces LettinGo, a novel framework for generating diverse and adaptive user profiles for recommendation systems using Large Language Models (LLMs). ii) The research aims to improve recommendation systems by generating diverse, adaptive, and high-quality user profiles by exploring and aligning profile generation with downstream task performance. iii) The proposed approach uses diverse LLMs for profile exploration, evaluates profile quality via downstream recommendation performance, and aligns profile generation through pairwise preference data using Direct Preference Optimization (DPO). iv) Experimental results demonstrate that LettinGo significantly enhances recommendation accuracy, adaptability, and contextual awareness, with an average increase of 20 percentage points in accuracy compared to the baseline when using LLaMA3 8B Instruct model. v) AI/ML engineers can leverage this framework to build recommendation systems with enhanced user profile generation capabilities and adapt profiles more effectively to diverse and evolving task requirements, potentially boosting recommendation accuracy and relevance. |
| ReDit: Reward Dithering for Improved LLM Policy Optimization (Read more on arXiv or HuggingFace) |
Yao Shu, Hande Dong, Ying Tiffany He, Jiarui Yu, Chenxing Wei |
i) This paper introduces ReDit, a reward dithering method to enhance LLM policy optimization. ii) The research aims to address gradient anomaly, optimization instability, and slow convergence issues associated with discrete reward functions in LLM training. iii) The method involves adding zero-mean random noise to discrete reward signals to facilitate smoother gradient updates and improve exploration. iv) Experiments show ReDit achieves performance comparable to vanilla GRPO with only 10% of the training steps, and a 4% improvement when trained for the same duration. v) ReDit mitigates gradient issues with discrete rewards, suggesting practitioners can improve LLM training by injecting random noise into discrete reward signals. |
| FinCoT: Grounding Chain-of-Thought in Expert Financial Reasoning (Read more on arXiv or HuggingFace) |
Potsawee Manakul, Panop Pitchayarthorn, Warit Sirichotedumrong, pittawat, natnitaract |
FinCoT introduces a structured chain-of-thought (CoT) prompting approach grounded in expert financial reasoning for large language models (LLMs). The research investigates standard, unstructured CoT, and structured CoT prompting styles for financial reasoning tasks. FinCoT incorporates domain-specific Mermaid blueprints into a structured CoT template to improve performance. Results on 1,032 CFA-style questions show FinCoT improves performance from 63.2% to 80.5% on Qwen-2.5-7B-Instruct and reduces generated tokens eight-fold compared to structured CoT prompting. AI practitioners can leverage FinCoT to enhance the accuracy and interpretability of LLMs in financial applications through structured, domain-aligned prompts. |
| ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs (Read more on arXiv or HuggingFace) |
Gregory Slabaugh, Zhensong Zhang, Thomas Tanay, Sibi Catley-Chandar, Michal Nazarczuk |
ViDAR is a novel 4D reconstruction framework for monocular video using diffusion-aware techniques. The research aims to improve dynamic novel view synthesis from monocular video by leveraging personalized diffusion models to generate a pseudo multi-view supervision signal for training a Gaussian splatting representation. The methodology involves a personalized DreamBooth-style diffusion model for enhancing novel views and a diffusion-aware loss function combined with camera pose optimization. Experiments on the DyCheck dataset demonstrated improved visual quality and geometric consistency, outperforming state-of-the-art baselines, with an average improvement of 0.94dB in PSNR in dynamic masked regions compared to MoSca. This work suggests a method for AI practitioners to improve 4D reconstruction by integrating personalized diffusion models into existing frameworks. |
| Auto-Regressively Generating Multi-View Consistent Images (Read more on arXiv or HuggingFace) |
Chen Zhao, Jinbo Wu, Jialun Liu, Yuxiao Yang, JiaKui Hu |
i) This paper introduces the Multi-View Auto-Regressive (MV-AR) model for generating consistent multi-view images from diverse prompts. ii) The main objective is to develop a model capable of generating multi-view consistent images from various prompts, addressing the limitations of existing diffusion-based methods. iii) The methodology involves leveraging an auto-regressive model with condition injection modules for text, camera pose, image, and shape, along with a “Shuffle View” data augmentation technique and progressive training strategy. iv) Experiments demonstrate the performance of MV-AR, achieving a CLIP-Score of 29.49 on the Google Scanned Objects dataset in the text-to-multi-view task, indicating improved image-text consistency compared to diffusion-based methods. v) The principal implication is that the MV-AR framework provides AI practitioners with a robust baseline for multi-view image generation, enabling the development of unified models that handle diverse conditions synchronously and generate consistent images. |
| SlimMoE: Structured Compression of Large MoE Models via Expert Slimming |
|
|
| and Distillation (Read more on arXiv or HuggingFace) |
Young Jin Kim, Ilgee Hong, Zixuan Zhang, Chen Liang, Pearush |
SlimMoE presents a multi-stage compression framework for Mixture of Experts (MoE) models using expert slimming and distillation. The research aims to reduce the parameter count of large MoE models without extensive retraining by slimming experts and transferring knowledge through intermediate stages. The methodology involves structured pruning of neurons within experts and iterative knowledge distillation. SlimMoE compressed a Phi-3.5-MoE model, reducing total parameters to 7.6B (Phi-mini-MoE) and 3.8B (Phi-tiny-MoE) with activated parameters of 2.4B and 1.1B respectively, using only 400B tokens. The structured pruning and multi-stage distillation approach allows for the creation of high-quality compact MoE models, facilitating deployment in resource-constrained environments like single-GPU setups. |
| Enhancing Step-by-Step and Verifiable Medical Reasoning in MLLMs (Read more on arXiv or HuggingFace) |
Wenjie Li, Yujie Zhang, Wenjie Lou, Yankai Jiang, manglu3935 |
i) This paper introduces a new approach for enhancing medical reasoning in multimodal large language models (MLLMs). ii) The primary objective is to develop a framework for generating effective chain-of-thought (CoT) data to improve the reasoning capabilities of medical MLLMs. iii) The methodology involves a novel reasoning-path searching scheme called Mentor-Intern Collaborative Search (MICS) and a curriculum learning strategy. iv) The resulting medical MLLM, Chiron-01, achieves state-of-the-art performance across several medical visual question answering and reasoning benchmarks, including improving its baseline model’s performance by an average of 5.7% to 8.1% on VQA tasks. v) The development of MICS provides AI practitioners with a structured approach to creating high-quality CoT datasets for specialized domains, potentially improving the reasoning capabilities of MLLMs in tasks requiring complex, step-by-step analysis. |
| ConsumerBench: Benchmarking Generative AI Applications on End-User |
|
|
| Devices (Read more on arXiv or HuggingFace) |
Yiyu Liu, Hoang Nguyen, Rohan Kadekodi, Yile Gu, kamahori |
i) CONSUMERBENCH is introduced as a comprehensive benchmark for evaluating GenAI applications’ system efficiency and response time on end-user devices under realistic, concurrent execution scenarios. ii) The paper aims to address challenges in resource management, system efficiency, and user experience when deploying GenAI models on resource-constrained end-user devices, unlike cloud environments with dedicated GPUs. iii) The methodology involves developing a benchmarking framework to simulate multi-application workflows on end-user devices, capturing application-level (latency, SLO attainment) and system-level (CPU/GPU utilization, memory bandwidth) metrics under varying deployment strategies (GPU partitioning, shared model deployments). iv) Experiments reveal that greedy GPU resource allocation leads to severe starvation of lightweight applications, with decode phases in LiveCaptions running up to 30x slower, resulting in a 12.4x increase in average request latency; static GPU partitioning causes compute capacity to go unused despite the presence of unmet SLOs, while shared memory usage can lead to inefficient kernel implementations; model-sharing with an inference server incurs a 40% SLO miss for one application. v) The findings imply a need for dynamic, SLO-aware memory management and scheduling strategies, as well as GPU architecture-aware kernel designs, to optimize GenAI application performance on end-user devices. |
| CommVQ: Commutative Vector Quantization for KV Cache Compression (Read more on arXiv or HuggingFace) |
Tianle Cai, Talha Chafekar, Muhammad Yusuf Hassan, Yang Zhang, Junyan Li |
i) This paper introduces Commutative Vector Quantization (CommVQ) to compress the key-value (KV) cache for long-context Large Language Model (LLM) inference. ii) The primary objective is to reduce the memory footprint of KV caches in LLMs without significant accuracy degradation. iii) The method involves additive quantization with a learned codebook designed to be commutative with Rotary Position Embedding (RoPE), integrated via an Expectation-Maximization (EM) algorithm. iv) Experiments demonstrate an 87.5% reduction in FP16 KV cache size using 2-bit quantization, with competitive performance, and the possibility of 1-bit quantization with minimal accuracy loss on the LLaMA-3.1 8B model, tested on LongBench, InfiniteBench, and GSM8K benchmarks. v) CommVQ provides AI practitioners a more memory-efficient method for deploying long-context LLMs, potentially enabling 128K context length on a single RTX 4090 GPU for a LLaMA-3.1 8B model, overcoming memory constraints. |
| From Virtual Games to Real-World Play (Read more on arXiv or HuggingFace) |
Zilong Chen, Xi Chen, Jinjing Zhao, Fangyun Wei, Wenqiang Sun |
i) The paper introduces RealPlay, a neural network-based real-world game engine enabling interactive video generation from user control signals. ii) The research aims to develop a photorealistic and temporally consistent video generation model that responds to user control, eliminating the need for annotated real-world data. iii) The methodology involves a mixed training paradigm combining labeled game data (Forza Horizon 5) with unlabeled real-world video data, adapting a pre-trained image-to-video generator (CogVideoX) for chunk-wise generation, and incorporating action control through adaptive LayerNorm. iv) Experimental results show RealPlay achieves a 90% control success rate and demonstrates control transfer from virtual to real-world entities (vehicles, bicycles, pedestrians). v) RealPlay presents a data-driven approach for creating interactive simulations, enabling AI practitioners to develop real-world game engines and interactive high-fidelity simulations using learned dynamics instead of traditional graphics engines. |
| FaithfulSAE: Towards Capturing Faithful Features with Sparse |
|
|
| Autoencoders without External Dataset Dependencies (Read more on arXiv or HuggingFace) |
Andrew Bermingham, Luis Eduardo Rodrigues Vieira, Donghyun Lee, Harryn Oh, seonglae |
i) The paper introduces FaithfulSAE, a method for training sparse autoencoders (SAEs) on a model’s self-generated synthetic dataset to improve the capture of model-internal features. ii) The research investigates whether training SAEs on faithful, self-generated datasets can mitigate issues of instability and hallucinated features arising from out-of-distribution data in external training datasets. iii) FaithfulSAE employs the LLM to generate a synthetic dataset reflecting its inherent distribution, and then trains a Top-K SAE on this dataset; “faithfulness” is then assessed using metrics such as reconstruction performance and shared feature ratio (SFR). iv) Results demonstrate that FaithfulSAEs outperform SAEs trained on web-based datasets in SAE probing tasks and exhibit a lower Fake Feature Ratio in 5 out of 7 models, with shared feature ratio analysis indicating increased stability across seeds compared to instruction datasets. v) The principal implication for AI practitioners is the recommendation to consider model-generated training datasets for SAEs, as this approach can reduce dependence on potentially noisy external datasets and improve the interpretability of learned features in LLMs. |
| A deep learning and machine learning approach to predict neonatal death |
|
|
| in the context of São Paulo (Read more on arXiv or HuggingFace) |
Afia Anjum Tamanna, A Z M Tahmidul Kabir, Plabon Kumar Saha, Mohon Raihan, rajandasgupta |
i) This paper investigates machine learning and deep learning models for predicting neonatal mortality in São Paulo. ii) The primary research objective is to determine the most accurate model for identifying newborns at high mortality risk. iii) The methodology involves training and comparing various machine learning algorithms (Logistic Regression, KNN, Random Forest, XGboost) and deep learning models (CNN, LSTM) using a dataset of 1.4 million newborn child records. iv) The LSTM model achieved the highest accuracy (99%) compared to machine learning methods (XGboost and Random Forest at 94%). v) The LSTM model presents a potentially suitable solution for AI practitioners developing neonatal mortality risk prediction tools based on this dataset. |
| Robust Reward Modeling via Causal Rubrics (Read more on arXiv or HuggingFace) |
Sravanti Addepalli, Gandharv Patil, Rahul Madhavan, Harman Singh, Pragya Srivastava |
i) This paper introduces Causally Robust Reward Modeling (Crome) to mitigate reward hacking in Large Language Models (LLMs). ii) The main objective is to develop a reward model robust to superficial attributes and sensitive to true causal drivers of quality. iii) Crome employs synthetic targeted augmentations during training, including Causal Augmentations and Neutral Augmentations, guided by an oracle LLM based on identified causal rubrics. iv) Empirical results on RewardBench show that Crome improves average accuracy by up to 5.4%, with gains up to 13.2% and 7.2% in specific categories. v) The principal implication is that AI practitioners can use Crome’s causal framework and augmentation techniques to develop more robust reward models that are less susceptible to reward hacking and more aligned with intended quality metrics. |
| I Know Which LLM Wrote Your Code Last Summer: LLM generated Code |
|
|
| Stylometry for Authorship Attribution (Read more on arXiv or HuggingFace) |
Bertalan Borsos, Nils Gruschka, Richard A. Dubniczky, Tamas Bisztray, Neo111x |
i) This paper introduces LLM-AUTHORBENCH, a benchmark for LLM-generated C code authorship attribution and proposes a custom CodeT5-Authorship model. ii) The primary research question is to determine the feasibility and optimal methods for LLM authorship attribution in C code, comparing various ML and transformer models. iii) The methodology involves generating a dataset of 32,000 C programs from eight LLMs, training CodeT5-Authorship, and comparing it against traditional ML classifiers and other fine-tuned transformer models. iv) Results show that CodeT5-Authorship achieves 97.56% accuracy in binary classification of closely related LLMs, and 95.40% accuracy in multi-class attribution among five leading LLMs. v) AI practitioners can leverage the CodeT5-Authorship model and LLM-AUTHORBENCH benchmark to enhance accountability and security in software engineering, enabling better source code attribution. |
| SoK: Evaluating Jailbreak Guardrails for Large Language Models (Read more on arXiv or HuggingFace) |
Daoyuan Wu, Zongjie Li, Wenxuan Wang, Zhenlan Ji, Xunguang Wang |
i) This paper presents a systematization of knowledge (SoK) for evaluating jailbreak guardrails in large language models (LLMs). ii) The research aims to categorize existing LLM guardrails and evaluate their effectiveness against jailbreak attacks. iii) The methodology involves developing a multi-dimensional taxonomy along six key dimensions and a Security-Efficiency-Utility (SEU) evaluation framework. iv) The study found a significant vulnerability of current session-level guardrails against advanced multi-turn attacks with ASR exceeding 90% for some guardrails against adaptive attacks like X-Teaming. v) AI practitioners should be aware of the limitations of session-level guardrails against sophisticated multi-turn attacks and prioritize developing more robust defense methodologies. |
Papers for 2025-06-23
| Title |
Authors |
Summary |
| Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights (Read more on arXiv or HuggingFace) |
Xuanlei Zhao, Yuhao Zhou, Dongwen Tang, Zhiyuan Liang, VictorKai1996NUS |
Drag-and-Drop LLMs (DnD) introduces a prompt-conditioned parameter generator to eliminate per-task training for specializing LLMs. The paper investigates whether task-specific LoRA weights can be directly generated from task prompts, bypassing gradient descent. DnD employs a text encoder to distill prompts into condition embeddings, which are then transformed into LoRA weights using a hyper-convolutional decoder trained on prompt-checkpoint pairs. Results show DnD achieves up to 30% average gains over trained LoRAs on unseen benchmarks and reduces overhead by up to 12,000x. DnD provides AI practitioners with a method for efficient LLM specialization without per-task fine-tuning, facilitating rapid deployment across diverse tasks. |
| PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and |
|
|
| Quantized Attention in Visual Generation Models (Read more on arXiv or HuggingFace) |
Huixia Li, Xuefeng Xiao, Xinhao Yang, Ke Hong, A-suozhang |
PAROAttention proposes a pattern-aware token reordering technique to improve the efficiency of sparse and quantized attention mechanisms in visual generation models. The research aims to mitigate challenges in sparsification and quantization arising from dispersed and irregular attention patterns in visual data. The methodology involves reorganizing attention patterns into hardware-friendly block-wise patterns through token reordering, followed by specialized sparsification and quantization techniques. The paper demonstrates a 1.9~2.7× end-to-end latency speedup on video and image generation tasks with lossless metrics under lower density (20%-30%) and bitwidth (INT8/INT4). PAROAttention provides AI practitioners with a method to reduce computational costs associated with attention mechanisms, enabling faster inference and potentially reduced memory footprint in visual generative models. |
| Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal |
|
|
| Document Understanding (Read more on arXiv or HuggingFace) |
Biddwan Ahmed, Indraneel Das, Tanmay Odapally, udayallu, vishesh-t27 |
i) This paper introduces a multimodal document chunking approach to enhance Retrieval-Augmented Generation (RAG) systems. ii) The main objective is to improve the quality of document chunking in RAG pipelines using Large Multimodal Models (LMMs) to better handle complex document structures. iii) The methodology involves a multimodal batch processing framework using LMMs to process documents in configurable page batches with cross-batch context preservation, along with techniques for maintaining table structures, step-by-step procedures, and multi-page content relationships. iv) Results on an internal benchmark dataset demonstrate an improvement in accuracy from 0.78 to 0.89 compared to traditional fixed-size chunking in RAG systems. v) The principal implication for AI practitioners is the demonstration that vision-guided chunking significantly enhances RAG performance by improving semantic coherence and structural integrity, offering a novel approach for processing complex multimodal documents. |
| VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Jie Yang, Yiran Qin, Heng Zhou, Xiufeng Song, FACEONG |
VIKI-R introduces a benchmark and framework for embodied multi-agent cooperation. The research aims to evaluate and improve visual reasoning in multi-agent systems through hierarchical tasks. VIKI-Bench structures tasks into three levels: agent activation, task planning, and trajectory perception, and VIKI-R employs a two-stage approach of supervised fine-tuning (SFT) with Chain-of-Thought demonstrations followed by reinforcement learning (RL) using multi-level rewards. Experiments demonstrate that VIKI-R significantly outperforms baselines across all task levels, with RL enabling compositional cooperation; specifically, VIKI-R achieves a 74.1% accuracy on the agent activation task (VIKI-L1). The principal implication is a method for enhancing visual reasoning and coordination in embodied AI agents through structured learning and hierarchical rewards, offering AI practitioners a refined approach for multi-agent system development, particularly when diverse embodiment types are involved. |
| Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with |
|
|
| Hybrid History Condition (Read more on arXiv or HuggingFace) |
Yuan Zhou, Longhuang Wu, Zhiyong Xu, Junshu Tang, Jiaqi Li |
Hunyuan-GameCraft is presented as a novel framework for high-dynamic interactive video generation in game environments. The main objective is to create action-controllable game video synthesis by unifying keyboard and mouse inputs into a shared camera representation and using a hybrid history-conditioned training strategy. The methodology includes training on a large-scale dataset of over one million gameplay recordings, fine-tuning on synthetic data, and incorporating model distillation for efficiency. Experiments demonstrate that Hunyuan-GameCraft reduces interaction errors by 55% in cross-domain tests compared to existing models. The principal implication for AI practitioners is a method to generate more realistic and playable interactive game videos with improved action controllability and temporal consistency. |
| DreamCube: 3D Panorama Generation via Multi-plane Synchronization (Read more on arXiv or HuggingFace) |
Xihui Liu, Kaiyi Huang, Jianan Wang, Yanning Zhou, Yukun Huang |
DreamCube introduces a multi-plane synchronization strategy for 3D panorama generation, enhancing consistency in multi-plane omnidirectional representations. The main objective is to generalize 2D diffusion models to multi-plane representations for tasks like RGB-D panorama generation. The methodology involves adapting operators from 2D foundation models to be omnidirectionally translation-equivalent, and a multi-plane RGB-D diffusion model called DreamCube is introduced. Experiments show DreamCube reduces FID to 12.58 on the Structured3D dataset for RGB panorama generation, indicating improved visual quality, and achieves a depth estimation accuracy of 0.787 for δ-1.25, outperforming existing methods, and suggesting benefits of cube map representations for joint modelling of panoramic appearance and geometry. AI practitioners can leverage multi-plane synchronization for improved consistency in generating 3D omnidirectional content, which may enable single-view-to-3D scene generation. |
| Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate |
|
|
| Details (Read more on arXiv or HuggingFace) |
Qingxiang Lin, Zibo Zhao, Haolin Liu, Yunfei Zhao, Zeqiang Lai |
i) Hunyuan3D 2.5 is presented as an enhanced suite of 3D diffusion models for generating high-fidelity textured 3D assets. ii) The main research objective is to improve both shape and texture generation in 3D asset creation compared to previous methods. iii) The key methodologies involve a new shape foundation model named LATTICE and an upgraded texture generation model incorporating physical-based rendering (PBR) through a multi-view architecture. iv) Hunyuan3D 2.5 achieves better image-shape and text-shape similarities and outperforms commercial models, with user studies showing a 72% win rate in image-to-3D tasks compared to Commercial Model 1. v) The implication for AI practitioners is a potentially improved tool for creating realistic and detailed 3D assets, outperforming state-of-the-art models in shape detail, surface smoothness and texture consistency. |
| InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Simyung Chang, Jungwook Choi, Kyuhong Shim, Minsoo Kim |
i) InfiniPot-V is a training-free framework for memory-constrained streaming video understanding using key-value (KV) cache compression. ii) The research aims to address the challenge of unbounded KV cache growth in streaming video understanding by enforcing a hard, length-independent memory cap. iii) The methodology involves a continual KV cache compression framework using Temporal-axis Redundancy (TaR) and Value-Norm (VaN) metrics. iv) InfiniPot-V achieves up to 94% reduction in peak GPU memory usage while matching or surpassing full-cache accuracy; it maintains real-time performance with only 0.5% compression overhead at 14 frames per second. v) By enabling memory-constrained streaming video understanding without retraining or query knowledge, InfiniPot-V facilitates the deployment of on-device multimodal assistants, implying practical, real-time memory management. |
| Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with |
|
|
| Production-Ready PBR Material (Read more on arXiv or HuggingFace) |
Xin Huang, Yifei Feng, Mingxin Yang, Shuhui Yang, Team Hunyuan3D |
i) Hunyuan3D 2.1 is introduced as a comprehensive open-source system for generating high-fidelity, textured 3D assets from single-image inputs, featuring shape generation and PBR material synthesis. ii) The research aims to create a robust 3D asset generation pipeline accessible to a broader audience by addressing the complexities in 3D data processing and model training. iii) The methodology employs Hunyuan3D-DiT for shape generation, a flow-based diffusion architecture combined with Hunyuan3D-ShapeVAE, and Hunyuan3D-Paint for texture synthesis, utilizing a multi-view PBR diffusion model. iv) Quantitative evaluations for shape generation show that Hunyuan3D-DiT achieves a ULIP-I score of 0.1395 and Uni3D-I score of 0.3213, which presents the best performance. v) The open-sourced system and detailed tutorial enables AI practitioners to fine-tune and develop 3D generative models for applications in gaming, VR, and industrial design by offering a step-by-step guide on data processing, training, and evaluation. |
| UniFork: Exploring Modality Alignment for Unified Multimodal |
|
|
| Understanding and Generation (Read more on arXiv or HuggingFace) |
Xizhou Zhu, Hao Li, Lirui Zhao, Quanfeng Lu, Teng Li |
This paper introduces UniFork, a novel Y-shaped architecture for unified image understanding and generation. The research investigates modality alignment in task-specific expert models to understand the different alignment behaviors required for understanding and generation tasks. The methodology involves analyzing text-image feature alignment across Transformer layers and introducing task-specific branches in deeper layers to mitigate task interference. Experiments show UniFork outperforms fully shared Transformer architectures and achieves performance comparable to or better than task-specific models; for example, UniFork achieved an overall 46% accuracy on GenEval, a 39% improvement over the ablation variant with smaller parameter scale. The key implication for AI practitioners is that task-specific branching in unified multimodal models improves performance by addressing divergent modality alignment requirements, offering a potential pathway for more efficient and effective unified architectures. |
| Reranking-based Generation for Unbiased Perspective Summarization (Read more on arXiv or HuggingFace) |
Kathleen McKeown, Nicholas Deas, narutatsuri |
i) This paper addresses generating unbiased summaries, specifically in political perspective summarization, and identifies metrics for evaluating summary quality. ii) The research question is to identify reliable metrics for measuring perspective summary quality and to investigate the efficacy of LLM-based methods beyond zero-shot inference. iii) The methodology involves building a test set using human annotations to benchmark metric reliability and evaluating various generation methods, including prompting, mechanistic approaches, and reranking-based methods, further utilizing preference tuning with synthetically generated and reranking-labeled data. iv) Results show that traditional metrics underperform compared to language model-based metrics in evaluating summary quality and reranking-based methods yield superior performance; Preference tuning on reranked generations further boosts performance, particularly improving faithfulness, achieving a coverage of 0.437 and faithfulness of 0.724 using DPO+RR in human evaluation. v) The primary implication for AI practitioners is the demonstration of reranking-based methods’ efficacy in improving perspective summarization, indicating a need to shift away from zero-shot inference and prompting alone. |
| Long-term Traffic Simulation with Interleaved Autoregressive Motion and |
|
|
| Scenario Generation (Read more on arXiv or HuggingFace) |
Philipp Krähenbühl, Shuhan Tan, Xiuyu Yang |
i) The paper introduces InfGen, a unified autoregressive model for long-term traffic simulation using interleaved motion simulation and scenario generation. ii) The research aims to achieve realistic trip-level driving simulations by dynamically managing the entry and exit of traffic agents over extended time horizons. iii) InfGen uses a transformer architecture with task-specific tokenizers to convert agent behaviors into discrete tokens and employs mode-control tokens to switch between motion simulation and scene generation. iv) InfGen outperforms prior state-of-the-art models in 30-second traffic simulation and achieves Mean ACE of 8.1 against the baselines that have scores of 12.0 and 12.2. v) AI practitioners can leverage InfGen for generating realistic traffic scenarios to train and evaluate self-driving systems, particularly in situations requiring long-term prediction and dynamic agent management, advancing simulation capabilities for autonomous driving development. |
Papers for 2025-06-20
| Title |
Authors |
Summary |
| Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain |
|
|
| Perspective (Read more on arXiv or HuggingFace) |
Shibo Hao, Zhoujun Cheng, fengyao1909, koalazf99, tianyang |
i) This paper introduces GURU, a reinforcement learning (RL) corpus for improving large language model (LLM) reasoning across six domains. ii) The research investigates the domain-specificity of RL mechanisms in LLM reasoning, particularly whether RL primarily elicits existing knowledge or facilitates genuine skill acquisition. iii) The methodology involves curating a 92K-example RL corpus (GURU) across Math, Code, Science, Logic, Simulation, and Tabular domains and performing RL fine-tuning on Qwen2.5-7B and 32B base models. iv) Results indicate that while pretrained-heavy domains benefit from cross-domain RL, others require in-domain training; GURU-7B/32B models achieve state-of-the-art open model performance with 7.9% and 6.7% improvements, respectively, on a 17-task evaluation suite. v) This work implies that AI practitioners need to consider domain-specific training data for effective RL fine-tuning to improve reasoning capabilities, as multi-domain RL can significantly enhance general reasoning. |
| EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech |
|
|
| Emotion Detection (Read more on arXiv or HuggingFace) |
Maurice Kraus, Gollam Rabby, Robert Kaczmarczyk, felfri, ChristophSchuhmann |
i) The paper introduces EMONET-VOICE, a new speech emotion detection (SER) resource with a fine-grained taxonomy and expert validation. ii) The main objective is to provide robust benchmarks for evaluating the emotional understanding capabilities of AI systems in speech. iii) The methodology involves curating a large-scale synthetic speech corpus (EMONET-VOICE BIG) and creating a benchmark dataset (EMONET-VOICE BENCH) with expert annotations of 40 emotion categories at different intensity levels, followed by the development of EMPATHICINSIGHT-VOICE models. iv) EMPATHICINSIGHT-VOICE LARGE achieved the highest Pearson correlation of 0.421 and lowest RMSE of 3.756 when evaluated against expert human judgments. v) The principal implication for AI practitioners is the demonstration of systematic performance patterns: such as high-arousal emotions being more detectable than low-arousal states, providing valuable insights into the capabilities and limitations of current SER models that can aid in developing more nuanced and context-aware AI applications. |
| SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning (Read more on arXiv or HuggingFace) |
Dorien Herremans, Abhinaba Roy, Anuradha Chopra |
i) SonicVerse is a multi-task learning model for generating detailed music captions by integrating auxiliary music feature detection. ii) The main objective is to create a music captioning system capable of generating captions that incorporate both technical and general musical feature descriptions. iii) The methodology uses a projection-based architecture that transforms audio input into language tokens while simultaneously detecting music features through dedicated auxiliary heads and a Mistral-7B large language model for caption generation. iv) Experimental results demonstrate that incorporating music feature extractors within the token projection model leads to improvements in caption quality, with a BLEU score of 0.3484 achieved on the MusicBench dataset using the SonicVerse model compared to a baseline of 0.3456. v) The principal implication for AI practitioners is the demonstration of a multi-task learning framework for music captioning that leverages auxiliary supervision to improve performance on smaller, open-source datasets, integrating feature prediction directly into the captioning pipeline, thus eliminating the need for external music feature extractors. |
| Improved Iterative Refinement for Chart-to-Code Generation via |
|
|
| Structured Instruction (Read more on arXiv or HuggingFace) |
Weiran Huang, Lichao Sun, Yuyang Wang, Chengzhi Xu, WaltonFuture |
i) The paper introduces ChartIR, a training-free iterative refinement method for improved chart-to-code generation. ii) The research aims to enhance MLLMs’ ability to accurately translate visual charts into executable code. iii) The methodology involves structured instructions for visual understanding (description and difference) and an iterative refinement process for code generation. iv) Experiments on Plot2Code and ChartMimic datasets using Qwen2-VL and GPT-40 showed that ChartIR achieves superior performance, improving GPT-40 Score by 17% over direct generation on the Plot2Code dataset. v) ChartIR provides AI practitioners with a robust, model-agnostic framework for enhancing chart-to-code generation in MLLMs, improving visual and structural fidelity without task-specific training. |
Papers for 2025-06-19
| Title |
Authors |
Summary |
| Sekai: A Video Dataset towards World Exploration (Read more on arXiv or HuggingFace) |
Shaoheng Lin, Xiaofeng Mao, Chuanhao Li, Zhen Li, kpzhang |
i) The paper introduces SEKAI, a large-scale, annotated, first-person view video dataset designed for world exploration using video generation techniques. ii) The main objective is to provide a dataset that overcomes the limitations of existing datasets for training interactive world exploration models. iii) The methodology involves collecting videos from YouTube and a video game, followed by preprocessing, location, scene, weather, crowd density, camera trajectory, and caption annotation using vision-language models and SfM. iv) SEKAI-Real comprises over 5,000 hours of walking or drone view videos from over 100 countries and Sekai-Real-HQ demonstrates a more balanced location distribution and the average number of caption tokens exceeds 200, and a subset is used to train an interactive video world exploration model. v) SEKAI offers AI practitioners a significantly expanded and richly annotated resource to improve the training and development of world exploration and video generation models with better diversity and long-duration videos. |
| ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning |
|
|
| in LLMs (Read more on arXiv or HuggingFace) |
Yunqi Qiu, Tingting Ma, Xinnian Liang, Zijun Chen, Feng He |
i) This paper introduces ProtoReasoning, a framework using abstract prototypes to enhance the generalizable reasoning abilities of Large Language Models (LLMs). ii) The research investigates whether training LLMs with abstract reasoning prototypes improves cross-domain generalization capabilities. iii) The methodology involves supervised fine-tuning (SFT) on LLMs using datasets of logical reasoning problems represented in Prolog and planning problems in PDDL, coupled with automated verification. iv) Experiments show ProtoReasoning achieves a 4.7% improvement on the Enigmata-Eval logical reasoning benchmark, as well as a 6.3% boost on planning tasks. v) The principal implication for AI practitioners is that leveraging abstract prototypes can improve LLM generalization on structurally similar problems, suggesting a novel method for enhancing reasoning capabilities. |
| GenRecal: Generation after Recalibration from Large to Small |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
Yueh-Hua Wu, Yu-Chiang Frank Wang, Yong Man Ro, rhachiuma, BK-Lee |
i) This paper introduces GenRecal, a general-purpose VLM distillation framework. ii) The research addresses the challenge of knowledge transfer between heterogeneous VLMs differing in architectures and token types. iii) GenRecal employs a Recalibrator to align feature representations between teacher and student VLMs. iv) Experiments show GenRecal significantly improves baseline performance, with InternVL2.5-8B-GenRecal achieving up to 93.6% accuracy on MMB, outperforming large-scale open- and closed-source VLMs. v) GenRecal enables AI practitioners to efficiently distill knowledge across diverse VLMs for resource-constrained deployment, facilitating the creation of smaller, more efficient models. |
| BUT System for the MLC-SLM Challenge (Read more on arXiv or HuggingFace) |
Jan Černocký, Samuele Cornell, Dominik Klement, Jiangyu Han, Alexander Polok |
i) The paper introduces a two-speaker ASR system combining DiCoW and DiariZen for the MLC-SLM challenge. ii) The primary objective is to develop a multilingual multi-talker ASR system robust to out-of-domain scenarios and annotation inconsistencies. iii) The methodology involves fine-tuning DiCoW, a diarization-conditioned Whisper variant, and DiariZen, a WavLM-based diarization pipeline, on the MLC-SLM challenge dataset. iv) The resulting system achieves a micro-average tcpWER/CER of 16.75% on the MLC-SLM challenge and DiariZen outperforms Pyannote with a DER of 12.7% after fine-tuning. v) AI practitioners can leverage the released DiCoW and DiariZen models to enhance multilingual ASR systems, while being mindful of potential annotation inconsistencies when fine-tuning diarization components. |
| Embodied Web Agents: Bridging Physical-Digital Realms for Integrated |
|
|
| Agent Intelligence (Read more on arXiv or HuggingFace) |
Maxine Wu, Xingcheng Yao, Bingxuan Li, Rui Sun, Yining Hong |
Embodied Web Agents introduces a paradigm for AI agents that integrate physical embodiment with web-scale knowledge access. The research investigates how AI agents can perform tasks requiring both physical interaction and web-based reasoning, such as cooking using online recipes or navigating using dynamic map data. The proposed methodology involves creating a unified simulation platform integrating 3D environments with web interfaces, and constructing the Embodied Web Agents Benchmark comprising diverse tasks like cooking, navigation, shopping, tourism, and geolocation. Experimental results using LLM agents (GPT, Gemini, Qwen, and Intern) show a significant performance gap compared to human capabilities, with a 34.72% overall accuracy for GPT in navigation tasks. The primary implication for AI practitioners is the need to address challenges in cross-domain integration to improve AI systems’ ability to seamlessly connect physical and digital realms. Some aspects of the experimental setup and results lacked specifics, such as quantified human baselines and detailed task decompositions. |
| Semantically-Aware Rewards for Open-Ended R1 Training in Free-Form |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Zichao Liang, Xiyang Wu, Yuhang Zhou, Yapei Chang, Zongxia Li |
i) The paper introduces PrefBERT, a lightweight BERT-based scoring model for evaluating and training open-ended long-form generation models using Group Relative Policy Optimization (GRPO). ii) The research aims to address the challenge of evaluating open-ended text generation in GRPO by providing better semantic reward feedback compared to traditional metrics. iii) PrefBERT is trained on response evaluation datasets with human ratings, enabling it to offer more semantically-aware rewards for GRPO training. iv) Evaluations show PrefBERT leads to improved alignment with human preferences in generated text, with models exhibiting higher Likert scores and success rates compared to models trained with ROUGE-L and BERTScore. v) The primary implication is that AI practitioners can use PrefBERT as a more effective and efficient reward signal in GRPO to train language models for open-ended generation, leading to outputs better aligned with human preferences; some evaluations are unclear regarding full details of dataset curation. |
| SciVer: Evaluating Foundation Models for Multimodal Scientific Claim |
|
|
| Verification (Read more on arXiv or HuggingFace) |
Arman Cohan, Zexi Kuang, Yifei Shen, Chengye Wang, yilunzhao |
i) SCIVER is introduced as a new benchmark for evaluating foundation models in multimodal scientific claim verification. ii) The main objective is to assess the ability of foundation models to verify claims within a multimodal scientific context, using a benchmark with expert-annotated supporting evidence. iii) The methodology involves constructing a dataset of 3,000 examples over 1,113 scientific papers, spanning four reasoning types, and evaluating 21 multimodal foundation models. iv) Experimental results show that GPT-4.1 achieves 70.8% accuracy on analytical reasoning tasks, significantly lower than human expert performance (90.0%). v) The substantial performance gap between foundation models and human experts on SCIVER indicates a need for improvements in models’ comprehension and reasoning abilities for multimodal scientific literature tasks, particularly for complex reasoning types. |
| Truncated Proximal Policy Optimization (Read more on arXiv or HuggingFace) |
Chengyi Wang, Jiaze Chen, Yu Yue, Lingjun Liu, Tiantian Fan |
This paper introduces Truncated Proximal Policy Optimization (T-PPO), an extension to PPO designed to improve training efficiency. The research aims to enhance the training efficiency of reasoning Large Language Models (LLMs) while maintaining performance. The methodology involves Extended Generalized Advantage Estimation (EGAE) for incomplete responses and a selective token filtering mechanism. Results show that T-PPO achieves a 2.5x improvement in training efficiency and reaches 62 pass@1 on the AIME 2024 benchmark using a 32B base model. AI practitioners can leverage T-PPO to accelerate the training of reasoning LLMs without sacrificing performance. |
| CoMemo: LVLMs Need Image Context with Image Memory (Read more on arXiv or HuggingFace) |
Jifeng Dai, Wenhai Wang, Xizhou Zhu, jackroos, CLLBJ16 |
i) CoMemo is a novel large vision-language model (LVLM) architecture designed to improve multimodal processing. ii) The research investigates how to mitigate suboptimal characteristics in LVLMs related to attention allocation and positional encoding for high-resolution images. iii) The study introduces a dual-path architecture combining a Context image path with an image Memory path, and a novel positional encoding mechanism called ROPE-DHR (ROPE-Dynamic High-Resolution). iv) CoMemo achieves superior performance compared to conventional LVLM architectures across seven benchmarks, including a 17.2% relative improvement on Caption tasks. v) AI practitioners can utilize the CoMemo architecture and ROPE-DHR to enhance visual information processing in LVLMs, especially for tasks requiring long-context comprehension and high-resolution image understanding. |
| SwarmAgentic: Towards Fully Automated Agentic System Generation via |
|
|
| Swarm Intelligence (Read more on arXiv or HuggingFace) |
Shijie Zhou, Haokun Chen, Shijie Tang, Chenyang Lin, Yao Zhang |
SwarmAgentic is a novel framework for fully automated agentic system generation using swarm intelligence. The research aims to construct agentic systems from scratch and jointly optimize agent functionality and collaboration through language-driven exploration, without human intervention. It leverages a language-driven Particle Swarm Optimization (PSO) process, reformulating the approach into symbolic transformations for non-differentiable design spaces. The proposed Failure-Aware Velocity Update incorporates LLM-guided flaw identification, enabling targeted self-optimization across iterations. Empirical results show SwarmAgentic achieves a +261.8% relative improvement over ADAS on the TravelPlanner benchmark. This framework provides a fully automated methodology, enhancing scalability and adaptability in agentic system design, offering AI practitioners a method for structurally unconstrained task automation. |
| MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal |
|
|
| Models (Read more on arXiv or HuggingFace) |
Yitao Zhai, Yan Feng, Ruiping Wang, Jiayu Xu, Hongyu Wang |
MoTE introduces a memory-efficient Mixture-of-Experts (MoE) architecture for large multimodal models. The paper aims to reduce the memory footprint of MoE models by training ternary routed experts from a dense checkpoint, replacing the full-precision experts. The proposed method freezes the pre-trained feed-forward network (FFN) as a shared expert and trains ternary routed experts during up-cycling using quantization-aware training. Experiments show that MoTE achieves comparable performance to a full-precision MoE-LLaVA baseline with a smaller memory footprint, outperforming it by 4.3% average accuracy on end tasks with a 3.4GB expert memory budget after post-training quantization. This method provides AI practitioners with a more scalable and deployable MoE architecture suitable for memory-constrained devices. |
| OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents (Read more on arXiv or HuggingFace) |
Zico Kolter, Francesco Croce, Hao Zhao, Agatha Duzan, Thomas Kuntz |
OS-HARM is a benchmark designed to measure the safety of computer use agents interacting with graphical user interfaces. The research addresses the overlooked safety aspects of computer use agents by creating a benchmark to evaluate their harmful behavior potential. The study introduces 150 tasks covering deliberate misuse, prompt injection attacks, and model misbehavior within the OSWorld environment, coupled with an automated judge for evaluating accuracy and safety. Experiments showed that computer use agents exhibit vulnerability to misuse and prompt injections, with 04-mini complying with prompt injections in 20% of cases, and the automated judge achieves F1 scores of 0.76 and 0.79 for accuracy and safety, respectively. The principal implication for AI practitioners is the need for improved safety mechanisms and evaluation protocols in computer use agents to mitigate potential risks associated with their deployment and interaction with computer systems. |
| ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured |
|
|
| Proxies (Read more on arXiv or HuggingFace) |
Lin Ma, Panwang Pan, Keke Wang, Bangbang Yang, yjyyy |
i) ImmerseGen is a framework for generating photorealistic 3D environments using alpha-textured proxies, guided by agents, for immersive VR experiences. ii) The main objective is to create compact and photorealistic 3D worlds from text prompts suitable for real-time rendering on VR headsets, overcoming limitations of high-poly mesh modeling and massive 3D Gaussians. iii) The method involves hierarchical scene composition with lightweight geometric proxies (simplified terrain and billboard meshes), terrain-conditioned texturing for the base world, RGBA asset texturing for scenery, and VLM-based modeling agents for scene creation. iv) Experiments demonstrate that ImmerseGen achieves up to 79+ FPS rendering performance on VR devices using the Snapdragon XR2 Gen 2 platform, outperforming previous methods in visual quality and spatial coherence. v) ImmerseGen provides AI practitioners with a method to generate complex 3D environments with efficient memory usage suitable for real-time VR applications via the use of alpha-textured proxies and agent-guided asset arrangement. |
| FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Yunpu Ma, Weiguo Li, Haokun Chen, Hewei Gao, Yao Zhang |
i) FedNano is a federated learning framework designed for parameter-efficient adaptation of pretrained multimodal large language models (MLLMs) without client-side LLM deployment. ii) The research aims to address the computational, communication, and data heterogeneity challenges of deploying MLLMs in federated learning environments. iii) The methodology involves centralizing the LLM on the server, introducing lightweight NanoEdge modules on clients for local adaptation, and employing Fisher Merging for server-side aggregation to handle non-IID data. iv) Experiments show FedNano reduces client-side storage by 95% and achieves over 99% communication reduction compared to PEFT-based FL methods, attaining 81.41% accuracy on ScienceQA and 78.04% on IconQA for LLaVA. v) FedNano offers AI practitioners a scalable and communication-efficient approach for deploying MLLMs in decentralized settings, enabling practical application on resource-constrained devices and enhancing performance with non-IID datasets. |
Papers for 2025-06-18
| Title | Authors | Summary |
|——-|———|———|
| Scaling Test-time Compute for LLM Agents (Read more on arXiv or HuggingFace)| Siwei Wu, Hanhao Li, King Zhu, Wangchunshu, zhangysk | i) This paper explores the application of test-time scaling (TTS) methods to enhance the performance of language agent frameworks. ii) The main objective is to systematically investigate and analyze the impact of various TTS strategies, including parallel sampling, sequential revision, verification/merging techniques, and diversified rollouts, on language agent effectiveness. iii) The study involves comparative ablation experiments using the SmoLAgents framework and GPT-4.1 as the base model, with the GAIA benchmark dataset. iv) Results show that Best-of-N (BoN) sampling achieved the best performance gains, with an eight-point improvement over the baseline, and list-wise methods outperformed other verification/merging approaches; multi-agent rollouts further improved performance. v) AI practitioners can improve agent performance by strategically scaling test-time compute, using methods like BoN sampling and list-wise verification, and benefit from diverse rollout strategies in agentic frameworks. |
| LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs (Read more on arXiv or HuggingFace)| Ziwei He, Qipeng Guo, Zengfeng Huang, Zhigeng Liu, LiuXR | LongLLaDA explores long-context capabilities in diffusion LLMs, revealing stable perplexity and localized perception during context extrapolation. This research investigates whether diffusion LLMs maintain consistent performance on long documents. Comparing diffusion LLMs (LLaDA) and auto-regressive LLMs (LLaMA3) through perplexity and Needle-In-A-Haystack (NIAH) tasks reveals that LLaDA retrieves information from the nearest 4k window, demonstrating a local perception, in contrast to auto-regressive LLMs’ performance collapse beyond 8k tokens. LongLLaDA, a training-free method integrating LLaDA with NTK-based RoPE extrapolation, extends the context window to 24k tokens while maintaining performance, although aggregation tasks show performance limitations. The study identifies task-dependent capabilities in diffusion LLMs, indicating diffusion LLMs are superior in QA but lag in aggregation tasks compared to auto-regressive LLMs, offering insights into diffusion LLM application boundaries. |
| Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes
Correct Reasoning in Base LLMs (Read more on arXiv or HuggingFace)| yangwang92, MasterVito, VEWOXIC, shun-zheng, XumengWen | i) This paper introduces CoT-Pass@K as a novel metric to evaluate correct reasoning within reinforcement learning with verifiable rewards (RLVR) for LLMs. ii) The research aims to address the question of whether RLVR genuinely incentivizes correct reasoning in LLMs, beyond merely finding correct answers. iii) The methodology involves theoretical analysis of RLVR optimization dynamics and empirical validation using a DeepSeek-R1-0528-Qwen3-8B LLM as an automated verifier to evaluate chain-of-thought (CoT) correctness. iv) Results show that RLVR improves CoT-Pass@K for all values of K, indicating incentivization of correct reasoning, and early training stages exhibit increased P(CC|CA)(q) values. v) AI practitioners should consider CoT-Pass@K as a more reliable metric for evaluating reasoning progress in RLVR-tuned LLMs and focus on approaches that directly incentivize correct CoTs to improve reasoning capabilities. |
| Stream-Omni: Simultaneous Multimodal Interactions with Large
Language-Vision-Speech Model (Read more on arXiv or HuggingFace)| Yang Feng, Yan Zhou, Qingkai Fang, Shoutao Guo, Shaolei Zhang | i) Stream-Omni is introduced, a large language-vision-speech model that uses efficient text-centric modality alignments for multimodal interactions. ii) The research addresses efficient and flexible modality alignment in large multimodal models (LMMs) to support text, vision, and speech interactions. iii) Stream-Omni employs an LLM backbone and aligns vision using sequence-dimension concatenation, and speech with a CTC-based layer-dimension mapping to the text modality. iv) Experiments show Stream-Omni achieves strong performance on visual understanding, speech interaction, and vision-grounded speech interaction tasks, using only 23,000 hours of speech data. v) AI practitioners can utilize Stream-Omni’s efficient alignment approach to build multimodal systems using smaller speech datasets by leveraging layer-dimension mapping. |
| Efficient Medical VIE via Reinforcement Learning (Read more on arXiv or HuggingFace)| Chong Li, Chenglin Zhu, Lijun Liu, zhaocheng, lryyyy | i) This paper introduces Reinforcement Learning with Verifiable Rewards (RLVR) for efficient medical Visual Information Extraction (VIE) using limited annotated data. ii) The main objective is to improve the performance of medical VIE models with only 100 annotated samples by addressing domain-specific schemas and high annotation costs. iii) The methodology employs a diversified dataset, a balanced precision-recall reward mechanism, and innovative sampling strategies to fine-tune a Qwen2.5-VL-7B model. iv) The results show the RLVR model achieves state-of-the-art performance on medical VIE tasks, improving F1 scores, precision, and recall, but experiences performance degradation on dissimilar, general VIE datasets; specifically, the model achieved an F1 score of 77.81 on the medical VIE task. v) The principal implication for AI practitioners is that domain-specific optimization, including task-specific reward mechanisms and reasoning strategies within the RLVR framework, are critical for enhancing VIE performance in specialized fields, especially when annotation resources are scarce. |
| Reasoning with Exploration: An Entropy Perspective (Read more on arXiv or HuggingFace)| Wayne Xin Zhao, Bo Dai, Xuekai Zhu, Shaohan Huang, daixuancheng | i) The paper introduces an entropy-augmented reinforcement learning method to improve language model reasoning. ii) The research aims to enhance language model reasoning capabilities by explicitly encouraging exploration through an entropy-based reward shaping. iii) The methodology augments the advantage function in policy gradient methods (PPO and GRPO) with a clipped, gradient-detached entropy term related to token prediction probabilities. iv) The method achieves significant gains on the Pass@K metric, an estimator of LM reasoning capabilities, improving performance by +6.2 on AIME25 Pass@K even with K=256. v) AI practitioners can leverage the entropy-based advantage shaping to improve exploratory reasoning in reinforcement learning fine-tuning of language models, particularly to mitigate performance plateaus and push boundaries on complex reasoning tasks. |
| Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just
Like an Olympiad Team (Read more on arXiv or HuggingFace)| Md Rizwan Parvez, Md Kishor Morol, Salman Rahman, Md Tanzib Hosain | i) Xolver is a training-free, multi-agent framework designed to improve LLM reasoning by integrating diverse experiential modalities. ii) The research aims to enhance LLM problem-solving by enabling the accumulation and application of experiential knowledge, mirroring expert human problem solvers. iii) Xolver utilizes a multi-agent architecture incorporating external and self-retrieval, tool use, agent collaboration, agent-driven evaluation, and iterative refinement, leveraging both open-weight and proprietary models. iv) Xolver achieves a new best result of 98.1% on GSM8K and 94.4% on AIME’24, often surpassing existing specialized reasoning agents and larger LLMs even when using lightweight backbones like QWQ-32B. v) The results suggest that AI practitioners can improve reasoning performance by implementing holistic experience learning in LLMs. |
| QFFT, Question-Free Fine-Tuning for Adaptive Reasoning (Read more on arXiv or HuggingFace)| Ke Ji, Yukang Lin, Fei Yu, Junxiao Xu, lwl-uestc | QFFT is a novel fine-tuning technique enabling adaptive reasoning in large language models. This research aims to mitigate overthinking in chain-of-thought models by enabling adaptive selection between short and long reasoning patterns. The proposed Question-Free Fine-Tuning (QFFT) approach involves fine-tuning models solely on long chain-of-thought responses, discarding the input questions during training. Experiments on mathematical datasets demonstrate that QFFT reduces average response length by over 50% while maintaining performance comparable to supervised fine-tuning. QFFT enables AI practitioners to deploy more efficient reasoning models that dynamically adjust complexity based on task demands. |
| Can LLMs Generate High-Quality Test Cases for Algorithm Problems?
TestCase-Eval: A Systematic Evaluation of Fault Coverage and Exposure (Read more on arXiv or HuggingFace)| Xue Xia, Zexi Kuang, Zheyuan Yang, yilunzhao | TestCase-Eval is introduced as a benchmark to evaluate LLMs’ ability to generate test cases for algorithm problems. The research investigates whether LLMs can generate high-quality test cases that match or surpass those designed by human experts. The methodology involves assessing LLMs on two tasks: Fault Coverage, measuring the ability to explore input scenarios, and Fault Exposure, evaluating the capacity to expose flaws in incorrect code implementations. Experiments with 19 LLMs reveal that the best-performing model, Qwen3-32B, achieves only 43.8% on the Fault Exposure task, contrasting with 93.3% for human experts. The results indicate that current LLMs face significant challenges in generating targeted test inputs, a factor that can be crucial for testing and validation efforts in AI-driven code development. |
| Guaranteed Guess: A Language Modeling Approach for CISC-to-RISC
Transpilation with Testing Guarantees (Read more on arXiv or HuggingFace)| Abdulrahman Mahmoud, Celine Lee, Chaimaa Abi, Sarim-Hash, ahmedheakl | i) The paper introduces Guaranteed Guess (GG), a language model-based assembly transpiler with testing guarantees for CISC-to-RISC translation. ii) The research aims to develop an accurate and efficient method for translating x86 assembly to ARM or RISC-V assembly. iii) The methodology involves a custom-trained, architecture-aware language model, tokenizer extension, and integration with software testing constructs for validation. iv) GG achieves 99.39% accuracy on HumanEval programs when translating to ARMv8, with 1.73x faster runtime performance, 1.47x better energy efficiency, and 2.41x better memory usage than Rosetta 2 in real-world binaries. v) GG provides AI/ML practitioners with a potential solution for efficient cross-ISA binary translation that can avoid the overhead of traditional emulation or virtualization methods. |
| Align Your Flow: Scaling Continuous-Time Flow Map Distillation (Read more on arXiv or HuggingFace)| Karsten Kreis, Sanja Fidler, Amirmojtaba Sabour | i) This paper introduces Align Your Flow (AYF), a novel distillation method for scaling continuous-time flow map generative models. ii) The research aims to improve few-step generative model performance by developing new training objectives and techniques for flow map distillation. iii) The methodology involves introducing two new continuous-time objectives, EMD and LMD, generalizing existing objectives and leveraging autoguidance and adversarial finetuning. iv) AYF achieves state-of-the-art few-step generation performance on ImageNet 64x64 and 512x512, using small neural networks, and outperforms existing methods in text-conditioned synthesis; for instance, it allows for 4-step sampling on ImageNet that is as fast or faster than previous works’ single step generation, while retaining high diversity. v) AYF offers AI practitioners an improved method for distilling generative models into efficient few-step samplers, enabling faster image generation without sacrificing diversity, relevant for applications where real-time or low-compute inference is crucial. |
| CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language
Models in Tool-Calling Error Scenarios (Read more on arXiv or HuggingFace)| Junjie Ye, Siyu Yuan, Zehui Chen, Shiting Huang, CostaliyA | i) This paper introduces CRITICTOOL, a new benchmark for evaluating the self-critique capabilities of Large Language Models (LLMs) in tool-calling scenarios. ii) The main objective is to provide a more nuanced evaluation of how LLMs detect, diagnose, and recover from errors during complex tool utilization. iii) The methodology involves an evolutionary strategy for dataset construction, generating diverse tool-use errors categorized by their source (internal model-driven vs. external environment). iv) Experiments on CRITICTOOL show that GPT-4 achieves an overall score of 69.01, indicating superior self-critique performance, while tool-use-finetuned models generally perform poorly. v) CRITICTOOL’s fine-grained error categorization and analysis provide AI practitioners with insights into the limitations of current LLMs’ tool-use capabilities, guiding the development of more robust tool-calling systems. |
| xbench: Tracking Agents Productivity Scaling with Profession-Aligned
Real-World Evaluations (Read more on arXiv or HuggingFace)| Xiaobo Hu, Yang Liu, Yixin Ren, Kaiyuan Chen, Liuff23 | i) xbench is introduced as a novel evaluation suite for assessing AI agent productivity in real-world professional settings. ii) The main objective is to develop a dynamic, profession-aligned evaluation framework that bridges the gap between AI agent capabilities and real-world productivity, targeting commercially significant domains with evaluation tasks defined by industry professionals. iii) The methodology involves creating profession-aligned evaluation sets with metrics correlated to productivity value, and presenting two benchmarks: Recruitment (50 tasks) and Marketing (50 advertiser requirements, 836 influencers). iv) The primary result shows that o3 ranks first in both recruitment and marketing benchmarks, and o3 achieves a score of 78.5 on the recruitment benchmark. v) The principal implication for AI practitioners is that xbench provides a value-oriented framework for guiding and predicting the development of effective, domain-specific AI agents, enabling the tracking of product capabilities over time and predicting Technology-Market Fit (TMF). |
| Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse
Autoencoders (Read more on arXiv or HuggingFace)| Zhuoran Yang, Tianhao Wang, Xuyuan Xiong, Heejune Sheen, Siyu Chen | i) This paper presents a novel Sparse Autoencoder (SAE) training algorithm, Group Bias Adaptation (GBA), with provable feature recovery guarantees for Large Language Models (LLMs). ii) The primary objective is to achieve theoretically grounded feature recovery in LLMs using SAEs by addressing the limitations of existing methods. iii) GBA utilizes “bias adaptation” to directly control neuron sparsity, and the analysis involves a statistical framework formalizing polysemantic features as sparse mixtures of monosemantic concepts. iv) The research theoretically proves that GBA correctly recovers all monosemantic features under specific statistical model conditions and demonstrates superior empirical performance on LLMs up to 1.5B parameters, achieving the sparsity-loss frontier and learning more consistent features. v) AI practitioners can leverage GBA as a more robust and theoretically sound alternative to conventional regularization techniques for training SAEs, enabling enhanced mechanistic interpretability in LLMs. |
| EfficientVLA: Training-Free Acceleration and Compression for
Vision-Language-Action Models (Read more on arXiv or HuggingFace)| Chang Zou, Luo Zhongwei, Zichen Wen, Yuhao Wang, Yantai Yang | i) The paper introduces EfficientVLA, a training-free framework for accelerating and compressing vision-language-action (VLA) models. ii) The research aims to reduce computational and memory demands of VLA models to improve their deployment feasibility. iii) The methodology involves pruning inconsequential layers from the language module, task-aware visual token selection, and caching intermediate features within the diffusion-based action head. iv) EfficientVLA achieves a 1.93× inference speedup and reduces FLOPs to 28.9% on CogACT with a 0.6% success rate drop in the SIMPLER benchmark. v) EfficientVLA offers AI practitioners a computationally efficient method for deploying large-scale VLA models on resource-constrained platforms without requiring retraining. |
| VideoMolmo: Spatio-Temporal Grounding Meets Pointing (Read more on arXiv or HuggingFace)| Zhiqiang Shen, Abdelrahman Shaker, Hanan Gani, Ghazi Shazan Ahmad, ahmedheakl | VideoMolmo is presented as a large multimodal model for spatio-temporal pointing conditioned on textual input in videos. The research aims to improve fine-grained localization and reasoning in video grounding tasks. It decomposes video grounding into pointing and mask generation, using a temporal module with attention and a mask fusion pipeline with SAM2 for temporal consistency. VideoMolmo achieves a 5.4 percentage point average improvement on the introduced VPoS-Bench benchmark compared to baselines, and a 9.5 pp improvement on MeViS referring segmentation. AI practitioners can leverage VideoMolmo for enhanced video understanding applications requiring precise spatio-temporal reasoning. |
| Ambient Diffusion Omni: Training Good Models with Bad Data (Read more on arXiv or HuggingFace)| Constantinos Daskalakis, Antonio Torralba, Adam Klivans, Giannis Daras, adrianrm | i) The paper introduces Ambient Diffusion Omni, a framework that improves diffusion model training by leveraging low-quality, synthetic, and out-of-distribution images. ii) The primary research objective is to extract useful signal from degraded and non-target data to enhance the image generation capabilities of diffusion models. iii) The methodology involves modulating the training process based on each sample’s utility, exploiting spectral power law decay and locality properties of natural images, and employing time-conditional classifiers to distinguish between noised distributions. iv) The framework achieves state-of-the-art ImageNet FID, demonstrating its ability to train successfully with synthetically corrupted images and shows that with COCO dataset, Ambient-o achieved a remarkable FID of 10.61, significantly improving the baseline FID of 12.37. v) The principal implication for AI practitioners is a cost-effective and efficient strategy for expanding training datasets using readily available but often discarded data sources, improving generative model quality and diversity without extensive data curation. |
| Optimizing Length Compression in Large Reasoning Models (Read more on arXiv or HuggingFace)| Mingyang Fu, Dongping Chen, Zhengxiang Cheng, zhoutianyi | i) The paper introduces LC-R1, a post-training method to compress reasoning chains in large reasoning models (LRMs). ii) The main objective is to reduce redundant reasoning steps (“invalid thinking”) in LRMs while maintaining accuracy. iii) LC-R1 uses Group Relative Policy Optimization (GRPO) with a length reward for conciseness and a compression reward to remove invalid reasoning. iv) Experiments show LC-R1 achieves a ~50% reduction in sequence length with only a ~2% drop in accuracy on reasoning benchmarks. v) LC-R1 provides AI practitioners a method to improve the computational efficiency of LRMs by reducing verbose reasoning chains without significantly sacrificing accuracy. |
| Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning
for LLMs (Read more on arXiv or HuggingFace)| Ding Liu, Deng Zhao, Cai Chen, Bin Hu, Ring Team | i) Ring-lite is a Mixture-of-Experts (MoE) large language model optimized via reinforcement learning (RL) for efficient and robust reasoning. ii) The research aims to improve reasoning capabilities of large language models through stable RL training. iii) The methodology introduces a joint training pipeline integrating distillation with RL using Constrained Contextual Computation Policy Optimization (C3PO) and a two-stage training paradigm. iv) Ring-lite achieves 76.61% on AIME2024 and 69.11% on AIME2025 while activating only one-third of the parameters required by comparable models. v) C3PO enhances training stability and computational throughput in RL, potentially benefiting AI practitioners aiming to scale reasoning abilities in LLMs using MoE architectures. |
| Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time
Markers (Read more on arXiv or HuggingFace)| Sara Hooker, Ahmet Üstün, Adrien Morisot, Julia Kreutzer, Daniel D’souza | This paper introduces a training protocol leveraging training-time markers to improve controllability and performance on underrepresented features (long tail) in machine learning models. The research question explores optimizing training to improve controllability and long-tail performance at inference time. The methodology involves creating a taxonomy of data characteristics and task provenance for explicit control of generation attributes and implicit conditioning at inference, fine-tuning a base model to infer these markers automatically. The primary results show an average lift of 5.7% win rates in open-ended generation quality, with over 9.1% gains in underrepresented domains, and relative lifts of up to 14.1% on tasks like CodeRepair with 35.3% absolute improvements in length instruction following evaluations. The principal implication for AI practitioners is a principled and flexible approach to improving performance on long-tail data while providing users with a set of control levers the model is trained to be responsive to which can be optionally used at inference. |
| Alignment Quality Index (AQI) : Beyond Refusals: AQI as an Intrinsic
Alignment Diagnostic via Latent Geometry, Cluster Divergence, and Layer wise
Pooled Representations (Read more on arXiv or HuggingFace)| Utkarsh Bhatt, Danush Khanna, Chhavi Sharma, Abhilekh Borah, amanchadha | i) The paper introduces the Alignment Quality Index (AQI), a novel geometric metric for assessing large language model (LLM) alignment. ii) The research aims to provide a decoding-invariant measure of LLM alignment by analyzing latent space separation between safe and unsafe activations, addressing limitations of behavioral proxies. iii) The methodology involves combining the Davies-Bouldin score, Dunn index, Xie-Beni index, and Calinski-Harabasz index across various formulations and introduces the LITMUS dataset for evaluation. iv) Empirical tests demonstrate AQI’s correlation with external judges and ability to reveal vulnerabilities missed by refusal metrics; for example, a Delta AQI exceeding 10-20% has been observed to correlate with early-stage alignment erosion. v) AQI offers AI practitioners a behavior-agnostic safety auditing tool that provides early warning signals for alignment faking, promoting the development of more robustly aligned LLMs. |
| CAMS: A CityGPT-Powered Agentic Framework for Urban Human Mobility
Simulation (Read more on arXiv or HuggingFace)| Yong Li, Jian Yuan, Yuwei Du, JJ-TMT | i) CAMS is an agentic framework using a language-based urban foundation model for human mobility simulation. ii) The research objective is to improve the controllability, accuracy, and generalizability of human mobility simulation by incorporating urban spatial knowledge into LLMs. iii) The key methodology integrates MobExtractor to extract mobility patterns, GeoGenerator to generate geospatial knowledge with an enhanced CityGPT, and TrajEnhancer to refine trajectories using direct preference optimization (DPO). iv) Experiments on real-world datasets show CAMS achieves superior performance in mobility simulation, with 11 out of 16 metrics showing improvement and a highest CMRR score; however, specific quantitative values for those metrics are not provided in the summary. v) CAMS provides AI practitioners a new paradigm for integrating agentic frameworks with urban-knowledgeable LLMs for enhanced mobility simulation and spatial reasoning. |
| Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via
Reinforcement Learning (Read more on arXiv or HuggingFace)| Jiaxuan You, Tao Feng, Haozhen Zhang | i) The paper introduces Router-R1, a reinforcement learning framework for multi-round routing and aggregation of large language models (LLMs). ii) The main objective is to coordinate multiple LLMs in a sequential decision process to solve complex tasks, optimizing for both performance and cost. iii) Router-R1 instantiates the router itself as an LLM, interleaving “think” and “route” actions, and employs a rule-based reward function comprising format, outcome, and cost rewards to guide training. iv) Experiments on seven QA datasets demonstrate that Router-R1 outperforms several baselines, achieving a 0.416 average exact match score with the Qwen base model, while exhibiting strong generalization and cost management. v) Router-R1 enables AI practitioners to dynamically orchestrate diverse LLMs, balancing performance and computational efficiency for complex reasoning tasks through reinforcement learning. |
| Mixture-of-Experts Meets In-Context Reinforcement Learning (Read more on arXiv or HuggingFace)| Daoyi Dong, Zican Hu, Haoru Li, Fuhong Liu, Wenhao0 | i) This paper introduces T2MIR, a novel mixture-of-experts architecture for in-context reinforcement learning (ICRL). ii) The main objective is to improve ICRL adaptability by addressing the multi-modality of state-action-reward data and the heterogeneity of decision tasks. iii) The methodology involves replacing the feedforward layer in transformer-based decision models with a token-wise MoE and a task-wise MoE, coupled with a contrastive learning method for task routing. iv) Experiments demonstrate T2MIR significantly facilitates in-context learning, outperforming baselines; for example, T2MIR reduces Cheetah-Vel return from -86.1 to -68.9 compared to existing models. v) T2MIR offers AI practitioners a scalable architectural enhancement for advancing ICRL, potentially improving performance on tasks requiring complex input processing and task diversification, however, details on model training/validation remain unclear. |
| TR2M: Transferring Monocular Relative Depth to Metric Depth with
Language Descriptions and Scale-Oriented Contrast (Read more on arXiv or HuggingFace)| Hongliang Ren, Long Bai, Yiming Huang, Beilei Cui | TR2M is a framework that transfers monocular relative depth to metric depth using language descriptions and scale-oriented contrast. The research aims to address scale uncertainty in monocular relative depth estimation to improve its practical applicability. TR2M fuses image and text features with cross-modality attention, constructs confident pseudo metric depth for supervision, and employs scale-oriented contrastive learning. Experiments demonstrate TR2M achieves strong performance on seen datasets and superior zero-shot capabilities on five unseen datasets, with 19M trainable parameters. TR2M can be used to develop lightweight and generalizable monocular metric depth estimation models utilizing text descriptions for improved performance across diverse domains. The improvement over DepthAnything with linear fit on NYUv2 is a decrease in AbsRel from 0.055 to 0.082. |
| Universal Jailbreak Suffixes Are Strong Attention Hijackers (Read more on arXiv or HuggingFace)| Mahmood Sharif, Mor Geva, MatanBT | i) This paper investigates the mechanics of suffix-based jailbreak attacks, specifically the GCG attack, against safety-aligned LLMs. ii) The research aims to understand the underlying mechanisms that drive the efficacy and universality of GCG suffix-based jailbreaks. iii) The study employs attention knockout, activation patching, and a novel dot-product-based dominance metric to analyze information flow and contextual hijacking in LLMs. iv) Results show that GCG jailbreaks are shallow, relying on the adv→chat flow, exhibiting irregular dominance in contextualization with universal suffixes demonstrating higher hijacking strength (Spearman correlation of p = 0.55 at layer 20), and that GCG universality can be enhanced by a factor of up to 5 through a hijacking-enhanced objective function. v) AI practitioners can use hijacking suppression as a training-free framework, with an up to 10x attack success rate reduction. |
| EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction (Read more on arXiv or HuggingFace)| Yu-Chiang Frank Wang, Kai-Po Chang, Yu-Chu Yu, Hsi-Che Lin | EMLOC enables memory-efficient fine-tuning by using a downstream-aware emulator. The research investigates reducing the memory overhead of fine-tuning large foundation models to match inference costs. EMLoC constructs a lightweight emulator via activation-aware SVD on a downstream calibration set, then fine-tunes it using LoRA with a novel correction algorithm. Experiments show EMLoC enables fine-tuning a 38B model on a single 24GB GPU and outperforms baselines on VQA, with WC-VQA results improving from 43.1 to 48.8 after fine-tuning. EMLoC provides AI practitioners with a method to fine-tune large models in resource-constrained environments, using a memory budget equivalent to inference. |
Papers for 2025-06-17
| Title |
Authors |
Summary |
| MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning |
|
|
| Attention (Read more on arXiv or HuggingFace) |
ManTle, windlx, LINMUJIE-judy, enochzhang, sheep33333 |
i) MiniMax-M1 introduces a hybrid Mixture-of-Experts model with lightning attention for efficient, large-scale reasoning. ii) The objective is to scale test-time compute efficiently in large reasoning models, maintaining or improving performance on complex tasks. iii) The methodology involves combining a hybrid MoE architecture with lightning attention, continual pretraining, supervised fine-tuning, and reinforcement learning with a novel CISPO algorithm. iv) MiniMax-M1 achieves comparable or superior performance to models like DeepSeek-R1 and Qwen3-235B on complex tasks, while consuming 25% of the FLOPs of DeepSeek R1 at a 100K token generation length. v) MiniMax-M1 provides AI practitioners with an open-weight model designed to efficiently scale test-time compute for long-context reasoning, particularly beneficial for developing next-generation language model agents. |
| Scientists’ First Exam: Probing Cognitive Abilities of MLLM via |
|
|
| Perception, Understanding, and Reasoning (Read more on arXiv or HuggingFace) |
Ruoyao Xiao, Xuming He, Yiheng Wang, Yuhao Zhou, WilsonHwang |
i) This paper introduces Scientists’ First Exam (SFE), a benchmark for evaluating scientific cognitive abilities of Multimodal Large Language Models (MLLMs). ii) The research aims to address the limited assessment of perception and reasoning abilities in existing scientific benchmarks for MLLMs. iii) SFE utilizes 830 expert-verified VQA pairs across three cognitive levels: scientific signal perception, scientific attribute understanding, and scientific comparative reasoning, spanning 66 multimodal tasks across five disciplines. iv) Experiments show state-of-the-art models GPT-03 and InternVL-3 achieve 34.08% and 26.52% accuracy on SFE, respectively. v) The SFE benchmark identifies a significant performance gap, indicating potential for AI practitioners to improve MLLMs’ capabilities in scientific data analysis, particularly regarding perception and reasoning, necessitating developments tailored for real-world scientific workflows. |
| DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents (Read more on arXiv or HuggingFace) |
Zhendong Mao, Xiaorui Wang, Benfeng Xu, IgnoraZ, Ayanami0730 |
i) This paper introduces DeepResearch Bench, a new benchmark for evaluating deep research agents (DRAs). ii) The main objective is to provide a comprehensive and standardized evaluation methodology for assessing the capabilities of LLM-based DRAs, addressing the current lack of such benchmarks. iii) The methodology includes two novel frameworks: RACE, a reference-based method with adaptive criteria for evaluating report quality, and FACT, a framework for assessing information retrieval and citation accuracy. iv) The evaluation of several early-released DRAs, including Gemini-2.5-Pro Deep Research, showed that Gemini-2.5-Pro Deep Research achieved an average of 111.21 effective citations in its final reports, significantly outperforming other models in comprehensiveness. v) The benchmark and evaluation frameworks offer AI practitioners a tool for systematic development and assessment of LLM-based agents designed for complex research tasks, enabling comparative analysis of DRA performance. |
| DoTA-RAG: Dynamic of Thought Aggregation RAG (Read more on arXiv or HuggingFace) |
Peerawat Rojratchadakorn, Natthapath Rungseesiripak, natnitaract, montholscbx, saksornr |
i) The paper introduces DoTA-RAG, a Retrieval-Augmented Generation (RAG) system optimized for large-scale web knowledge indexes. ii) The research aims to address the challenges of high latency and limited accuracy in traditional RAG pipelines when applied to massive, diverse datasets. iii) The methodology involves a three-stage pipeline consisting of query rewriting, dynamic routing to specialized sub-indexes, and multi-stage retrieval and ranking, using Falcon3-10B-Instruct as the base LLM. iv) DoTA-RAG improved the answer correctness score from 0.752 to 1.478 on an internal test while maintaining low latency, and achieved a correctness score of 0.929 on the Live Challenge Day. v) The implementation of DoTA-RAG offers AI practitioners a fast, reliable, and scalable RAG system for domains requiring access to large and evolving knowledge sources, with dynamic routing significantly enhancing retrieval efficiency, reducing latency by more than half compared to static top-k search. |
| Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning (Read more on arXiv or HuggingFace) |
Yuhao Dong, Penghao Wu, Hongming Guo, ruiqiw, shulin16 |
Ego-R1 introduces a framework for reasoning over ultra-long egocentric videos by employing a Chain-of-Tool-Thought (CoTT) process orchestrated by a reinforcement learning (RL) agent. The paper addresses the challenge of long-horizon reasoning in egocentric videos spanning days or weeks. A structured CoTT process with dynamic tool invocation (Hierarchical RAG, Video LLM, and VLM) is designed to decompose reasoning into modular steps. The agent is trained using supervised fine-tuning (SFT) on the Ego-CoTT-25K dataset and RL on the Ego-QA-4.4K dataset. Evaluated on the Ego-R1 Bench, the agent achieves 46.0% accuracy, demonstrating effective handling of week-long video understanding. This tool-augmented reasoning paradigm can effectively tackle ultra-long egocentric videos for problems requiring temporal awareness and precise analysis, potentially expanding the time coverage from hours to a week. |
| Wait, We Don’t Need to “Wait”! Removing Thinking Tokens Improves |
|
|
| Reasoning Efficiency (Read more on arXiv or HuggingFace) |
Ranjay Krishna, Zhaoyang Chu, Dongping Chen, Yuanning Feng, Chenlong Wang |
i) This paper introduces NOWAIT, a training-free inference-time method to improve the reasoning efficiency of large language models (LRMs). ii) The research investigates whether explicit self-reflection, signaled by tokens like “Wait” and “Hmm,” is necessary for advanced reasoning in LRMs. iii) NOWAIT suppresses the generation of specific keyword tokens associated with self-reflection by adjusting their logits during inference. iv) Experiments on ten benchmarks show NOWAIT reduces chain-of-thought trajectory length by up to 27%-51% in R1-style model series across textual, visual, and video reasoning tasks. v) NOWAIT provides AI practitioners with a plug-and-play solution to reduce computational overhead and latency in multimodal reasoning applications without compromising model utility. |
| Discrete Diffusion in Large Language and Multimodal Models: A Survey (Read more on arXiv or HuggingFace) |
Xinchao Wang, Qi Li, Runpeng Yu |
i) This survey provides a systematic overview of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs). ii) The paper aims to formalize the underlying mathematical frameworks, categorize representative models, analyze key techniques for training and inference, and summarize emerging applications across language, vision-language, and biological domains. iii) The methodology involves tracing the historical development of dLLMs and dMLLMs, categorizing representative models and analyzing key techniques for training and inference. iv) dLLMs and dMLLMs achieve up to 10x acceleration in inference speed compared to autoregressive models. v) Industrial-scale proprietary d(M)LLMs as well as open-source academic d(M)LLMs have demonstrated performance comparable to their autoregressive counterparts, positioning discrete diffusion models as a promising alternative to intelligence based on traditional autoregressive approaches for AI practitioners seeking efficiency gains. |
| TaskCraft: Automated Generation of Agentic Tasks (Read more on arXiv or HuggingFace) |
Weizhen Li, Weichen Sun, Qianben Chen, Jingyi Cao, Dingfeng Shi |
TaskCraft introduces an automated workflow for generating agentic tasks involving multi-step problem solving, tool use, and adaptive reasoning. The paper addresses the scalability limitations of existing agentic benchmarks by automating the generation of difficulty-scalable tasks with verifiable execution trajectories. The methodology uses depth-based and width-based extensions of atomic tasks, combined with rejection sampling and linguistic analysis for verification. Empirical results demonstrate improved prompt optimization and enhanced supervised fine-tuning of agentic foundation models; specifically, SFT achieves average performance improvements of +14.0% (Qwen2.5-3B-Base). TaskCraft provides AI practitioners with a synthetic dataset of approximately 36,000 tasks to support agent tuning and evaluation. |
| VGR: Visual Grounded Reasoning (Read more on arXiv or HuggingFace) |
Haiyong Jiang, Haochen Wang, Zijiang Kang, bongbohong, stormthunder |
VGR introduces a visual grounded reasoning framework for multimodal large language models (MLLMs). The research aims to enhance MLLMs’ visual reasoning capabilities by enabling selective attention to relevant image regions during inference. VGR employs a self-driven selective visual replay method, retrieving visual tokens from a feature pool based on a replay signal generated by the model. Experiments on LLaVA-NeXT-7B show VGR achieves +4.1 on MMStar, +7.1 on AI2D, and +12.9 on ChartQA compared to the baseline with only 30% of image token usage. The principal implication is that targeted visual analysis and selective replay can significantly improve MLLM performance and efficiency on tasks requiring detailed image understanding. |
| PersonaFeedback: A Large-scale Human-annotated Benchmark For |
|
|
| Personalization (Read more on arXiv or HuggingFace) |
Yuchen Eleanor Jiang, Tiannan Wang, Dongyi Ding, Chenghao Zhu, Meiling Tao |
PersonaFeedback introduces a human-annotated benchmark for evaluating LLM personalization capabilities. The study investigates the ability of LLMs to provide personalized responses based on predefined user personas. The methodology involves creating 8298 human-annotated test cases categorized by difficulty and evaluating various LLMs. Empirical results show that while SOTA LLMs perform well on general tasks, their performance declines on the hard tier of PersonaFeedback, with top proprietary models exhibiting relatively low average accuracy. The research implies that explicitly providing user personas improves performance in personalized scenarios over relying solely on implicit persona inference for AI practitioners. |
| From Real to Synthetic: Synthesizing Millions of Diversified and |
|
|
| Complicated User Instructions with Attributed Grounding (Read more on arXiv or HuggingFace) |
Zhendong Mao, Xiaorui Wang, Benfeng Xu, IgnoraZ |
i) This paper introduces a framework for synthesizing diverse and complex instruction data for aligning large language models (LLMs) using attributed grounding. ii) The main objective is to generate instruction data at scale that reflects real-world use cases and cognitive insights, overcoming the limitations of existing synthetic instruction generation methods. iii) The methodology involves a top-down attribution process, grounding real instructions to situated users, and a bottom-up synthesis process that leverages web documents to generate situations and meaningful instructions. iv) The study constructs a dataset of 1 million instructions, SYNTHQUESTIONS, and demonstrates that models trained on this dataset achieve leading performance on common benchmarks; one example is the models improved Alpaca Eval 2.0. win rate with their models trained with SYNTHQUESTIONS. v) The framework allows AI practitioners to generate pre-training-level instruction data with high complexity and diversity, improving the alignment and performance of LLMs. |
| Test3R: Learning to Reconstruct 3D at Test Time (Read more on arXiv or HuggingFace) |
Xinchao Wang, Xingyi Yang, Shizun Wang, Yuheng Yuan, florinshum |
i) The paper introduces Test3R, a test-time learning technique for enhancing 3D reconstruction by optimizing cross-pair consistency. ii) The research aims to improve the geometric accuracy of 3D reconstruction by addressing limitations inherent in pairwise prediction methods. iii) Test3R utilizes image triplets and a self-supervised objective to maximize the geometric consistency between reconstructions generated from different image pairs through prompt tuning at test time. iv) Experiments demonstrate that Test3R significantly outperforms state-of-the-art methods, achieving a 1.3 reduction in Absolute Relative Error and a 14.2 increase in Inlier Ratio on the DTU dataset for multi-view depth estimation compared to the vanilla DUSt3R v) AI practitioners can leverage Test3R as a universally applicable and cost-effective method to improve the accuracy and robustness of existing 3D reconstruction pipelines with minimal overhead. |
| BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning |
|
|
| with Vision-Language Models (Read more on arXiv or HuggingFace) |
Xiangnan Wu, Xiao Ma, Hongtao Wu, Yixiang Chen, LPY |
BridgeVLA aligns 3D manipulation learning with vision-language models via input-output alignment using 2D heatmaps. The research aims to develop a sample-efficient 3D vision-language-action model by leveraging 3D structural priors and vision-language models. The methodology involves projecting 3D point clouds into multiple 2D images and predicting 2D heatmaps for action prediction. BridgeVLA improves the average success rate in RLBench from 81.4% to 88.2%. BridgeVLA offers AI practitioners an efficient method for learning 3D robot manipulation by aligning inputs and outputs in a shared 2D space, resulting in better sample efficiency. |
| Language Surgery in Multilingual Large Language Models (Read more on arXiv or HuggingFace) |
Muhammad Ilham Ghozali, samuel-cahyawijaya, tackhwa, muhammadravi251001, joanitolopo |
i) The paper investigates representation alignment in multilingual LLMs, proposing Inference-Time Language Control (ITLC) for cross-lingual tasks. ii) The research aims to understand and leverage the naturally emerging representation alignment in LLMs to enable precise language control and mitigate language confusion. iii) The methodology involves empirically confirming the existence of representation alignment, disentangling language-specific and language-agnostic information, and using latent injection for cross-lingual language control. iv) ITLC achieves strong cross-lingual control, retaining semantic integrity, and demonstrates ~30% performance retention compared to LLMs with explicitly designed alignment, while achieving almost >90% relative to other non-aligned layers v) ITLC presents a practical solution for AI practitioners to enhance cross-lingual performance in LLMs by leveraging latent injection for language-specific manipulation, improving consistency and mitigating language confusion. |
| AI Agent Behavioral Science (Read more on arXiv or HuggingFace) |
Honglin Zhang, Haoye Chai, Yunke Zhang, Lin Chen, JJ-TMT |
i) This paper introduces AI Agent Behavioral Science as a paradigm for systematically studying AI agent behavior within specific contexts. ii) The main objective is to shift the focus from internal model mechanisms to empirically observing and understanding AI agents’ actions, adaptations, and social patterns. iii) The paper synthesizes existing research across individual, multi-agent, and human-agent interaction settings, drawing inspiration from human and animal behavioral research. iv) The study reveals LLM-powered agents exhibit human-like capabilities in cognitive reasoning, emotion recognition, and theory of mind, though often demonstrate limited rationality as well as remaining sensitive to task framing. v) The principal implication is for AI practitioners to consider behavioral properties like fairness, safety, and interpretability as dynamic, context-dependent attributes, informing the design and evaluation of responsible AI systems. |
| ALE-Bench: A Benchmark for Long-Horizon Objective-Driven Algorithm |
|
|
| Engineering (Read more on arXiv or HuggingFace) |
Kensho Aoki, Yoichi Iwata, Kohki Horie, Yuki Imajuku, iwiwi |
ALE-Bench is introduced as a benchmark for AI systems in long-horizon, score-based algorithmic programming contests, using tasks from the AtCoder Heuristic Contests. The research aims to evaluate AI’s capabilities in algorithm engineering for hard optimization problems without known exact solutions. The methodology involves evaluating frontier LLMs in one-shot and iterative refinement settings, using a software framework that supports interactive agent architectures with test-run feedback and visualizations. Results indicate that current LLMs, while demonstrating high performance on specific problems, still exhibit a gap compared to humans in consistency across problems and long-horizon problem-solving; one LLM achieved a performance score of 2880 on AHC039, corresponding to 5th place in the original contest. This benchmark highlights the necessity for further AI advancements to foster consistent and sustained problem-solving, especially in algorithm design and iterative optimization. |
| LETS Forecast: Learning Embedology for Time Series Forecasting (Read more on arXiv or HuggingFace) |
Yin Li, Nada Magdi Elkordi, Satya Sai Srinath Namburi GNVV, viswa-98, alphaomeaga |
i) The paper introduces DeepEDM, a novel deep learning framework for time series forecasting that integrates empirical dynamic modeling with deep neural networks. ii) The research aims to develop a forecasting model that explicitly models the underlying dynamics of complex nonlinear time series while addressing limitations of traditional EDM and deep learning methods. iii) DeepEDM utilizes time-delayed embeddings, a learned latent space robust to noise, kernel regression via softmax attention, and a learned decoder for end-to-end training. iv) Experiments on synthetic data demonstrate that DeepEDM consistently outperforms baselines, achieving lower MSE in chaotic regimes and remaining robust to significant noise levels; on a chaotic Lorenz system at σ = 2.5 and H = 48, its MSE (17.267) is 40% lower than Koopa (28.804). v) AI practitioners can leverage DeepEDM to improve forecasting accuracy by integrating dynamical systems principles into deep learning models, particularly for time series exhibiting complex, nonlinear dynamics and sensitivity to noise. |
| Supernova Event Dataset: Interpreting Large Language Model’s Personality |
|
|
| through Critical Event Analysis (Read more on arXiv or HuggingFace) |
Ioana Ciucă, pranavAL2109 |
i) The paper introduces Supernova Event Dataset and uses critical event analysis to interpret LLM personalities. ii) The objective is to understand and benchmark the decision-making processes and underlying “personality” traits of LLMs when extracting key events from diverse texts. iii) The methodology involves using RAG and an LLM as a judge to analyze the top-ranked events extracted by target LLMs from articles covering biographies, historical/news events, and scientific discoveries. iv) The analysis reveals distinct personality traits: Orca 2 exhibited emotional reasoning focusing on interpersonal dynamics, while Qwen 2.5 displayed a more strategic, analytical style; when evaluating scientific discoveries, Claude Sonnet 3.7 focused on conceptual framing. v) This work improves model interpretability by providing a method to understand and potentially align LLMs with desired human values. |
| Forecasting Time Series with LLMs via Patch-Based Prompting and |
|
|
| Decomposition (Read more on arXiv or HuggingFace) |
Anish Gupta, Sri Harsha Vardhan Prasad Jella, Anshul Vemulapalli, Mayank Bumb, Franck-Dernoncourt |
i) This paper introduces PatchInstruct, a novel prompt-based framework to enable Large Language Models (LLMs) for time series forecasting without fine-tuning or complex architectures. ii) The main objective is to enhance LLM-based time series forecasting by addressing limitations in inference speed and generalization, maintaining predictive strength without extensive model retraining. iii) The methodology involves patch-based tokenization of time series data with structured natural language instructions to guide the LLM, comparing it against Zero-shot and Neighbor-based prompting on Weather and Traffic datasets. iv) PatchInstruct consistently outperforms baselines on small forecasting horizons, achieving top forecasting accuracy on horizons at most 12 steps, while reducing inference overhead by 10x-100x; for Weather dataset at horizon = 1, the Mean Squared Error (MSE) drops from 1.15 × 10^-2 to 2.6 × 10^-4. v) The principal implication for AI practitioners is that specialized prompting techniques such as PatchInstruct can effectively replace some architectural complexity, and improve the scalability and domain adaptability of LLM-based time series forecasting, enabling more efficient and accurate predictions with minimal preprocessing. |
| MS4UI: A Dataset for Multi-modal Summarization of User Interface |
|
|
| Instructional Videos (Read more on arXiv or HuggingFace) |
Jiuxiang Gu, Seunghyun Yoon, Hao Tan, Yuan Zang, Franck-Dernoncourt |
i) The paper introduces MS4UI, a new dataset for multi-modal summarization of user interface (UI) instructional videos. ii) The main objective is to provide a benchmark for generating concise and executable step-by-step instructions for UI-related tasks. iii) The methodology involves collecting 2,413 UI instructional videos and annotating them for video segmentation, text summarization, and video summarization. iv) Experiments with existing multi-modal summarization methods on the MS4UI dataset revealed suboptimal performance, particularly in tasks requiring fine-grained understanding of UI elements and actions; baseline methods show unsatisfactory performance on the three core tasks of the proposed dataset. v) The creation of MS4UI and its associated evaluation tasks highlight the necessity of developing specialized methods that can effectively understand structured UI layouts and actions, crucial for AI practitioners developing UI-related educational resources or automated assistance tools. |
| Profiling News Media for Factuality and Bias Using LLMs and the |
|
|
| Fact-Checking Methodology of Human Experts (Read more on arXiv or HuggingFace) |
Preslav Nakov, Maha Tufail Agro, Dilshod Azizov, Zain Muhammad Mujahid |
i) The paper introduces a methodology for profiling news media outlets for political bias and factuality using large language models (LLMs). ii) The main objective is to emulate the criteria used by professional fact-checkers to assess media bias and factuality. iii) The methodology involves crafting custom prompts for LLMs and aggregating their responses for classification, and it contrasts with zero-shot predictions. iv) The experiments demonstrated improved accuracy over baselines, with the best model achieving 80.6% accuracy and 0.206 MAE for factuality prediction, and 93.5% accuracy and 0.075 MAE for political bias prediction using expert guidelines on a 3 point scale. v) The principal implication is that LLMs, when guided by expert-driven prompts, can provide a systematic and more accurate assessment of news media outlets, which can be used for factuality of reporting and political bias. |
| SRLAgent: Enhancing Self-Regulated Learning Skills through Gamification |
|
|
| and LLM Assistance (Read more on arXiv or HuggingFace) |
Weiyang He, Haoyue Zheng, Ziyan Wang, Yuqing Sun, Owenngt |
i) SRLAgent, an LLM-assisted system, enhances self-regulated learning (SRL) skills in college students through gamification. ii) The research investigates whether SRLAgent improves SRL skills compared to baseline systems and traditional learning resources. iii) A between-subjects study compared SRLAgent to a baseline system (SRL without Agent features) and multimedia learning, involving 59 college students. iv) Results showed significant SRL skill improvements in the SRLAgent group (p < .001, Cohen’s d = 0.234), with higher engagement. v) Embedding SRL scaffolding and AI support within gamified environments can enhance metacognitive skill development, offering design implications for educational technologies. |
| Incorporating Domain Knowledge into Materials Tokenization (Read more on arXiv or HuggingFace) |
SangKeun Lee, SungHo Kim, Junho Kim, Jun-Hyung Park, yerim0210 |
i) The paper introduces MATTER, a domain-specific tokenization framework for materials science that integrates material knowledge. ii) The research aims to improve materials science language models by addressing the limitations of frequency-centric tokenization, which often fragments material concepts. iii) The methodology involves training a material concept identifier (MatDetector) on a curated materials knowledge base and re-ranking token merging to prioritize material-related subwords. iv) Experiments on generation tasks showed an average performance gain of 4% and classification tasks 2% relative to existing tokenization methods. v) MATTER provides AI practitioners with an enhanced tokenization method that better preserves the semantic integrity of material concepts, improving the performance of downstream materials science NLP tasks. |
| Steering LLM Thinking with Budget Guidance (Read more on arXiv or HuggingFace) |
Chuang Gan, Yang Zhang, Wenshuo Zhao, Junyan Li |
i) The paper introduces Budget Guidance, a method for controlling the reasoning length of Large Language Models (LLMs) without fine-tuning. ii) The research aims to control the reasoning length of LLMs at inference time without sacrificing performance, especially under tight thinking budgets. iii) The methodology involves training a lightweight predictor to model a Gamma distribution over the remaining thinking length at each token, using it to guide LLM generation. iv) Experiments on MATH-500 benchmark show up to a 26% accuracy gain under tight budgets compared to baseline methods, while maintaining accuracy with only 63% of the tokens used by the full-thinking model. v) Budget Guidance offers AI practitioners a way to improve the token efficiency of LLMs on challenging math benchmarks and generalizes to other task domains like GPQA, FOLIO, LiveCodeBench. |
| Uncertainty-Aware Remaining Lifespan Prediction from Images (Read more on arXiv or HuggingFace) |
Barbara Hammer, Philip Kenneweg, TristanKe |
i) The paper introduces a method for estimating remaining lifespan from facial and whole-body images, with uncertainty quantification. ii) The research objective is to accurately predict remaining lifespan from images while providing calibrated uncertainty estimates. iii) The methodology leverages pretrained vision transformer foundation models (DINOv2) and a regression head modeling prediction uncertainty as a Gaussian distribution, trained with the Gaussian negative log-likelihood loss. iv) The approach achieves a mean absolute error (MAE) of 7.48 years on an established dataset and improves to 4.79 and 5.07 years MAE on two new datasets; the bucketed expected calibration error is 0.62 years. v) AI practitioners can utilize the demonstrated uncertainty modeling to develop more robust and interpretable image-based prediction systems in healthcare and other domains where calibrated confidence estimates are crucial for decision-making. |
| Ai-Facilitated Analysis of Abstracts and Conclusions: Flagging |
|
|
| Unsubstantiated Claims and Ambiguous Pronouns (Read more on arXiv or HuggingFace) |
PChemGuy |
i) This paper evaluates structured prompts for LLMs in analyzing scholarly manuscript summaries. ii) The research question is whether LLMs can identify unsubstantiated claims and ambiguous pronouns in academic abstracts and conclusions through structured prompting. iii) The methodology involves designing proof-of-concept (PoC) structured prompts and evaluating them on Gemini Pro 2.5 Pro and ChatGPT Plus 03 under varied context conditions. iv) Results indicate that ChatGPT consistently failed (0% success) to identify an unsubstantiated adjectival modifier, while Gemini correctly flagged it (95% success); in linguistic clarity analysis, ChatGPT achieved 100% success with limited context, whereas Gemini’s performance degraded. v) The principal implication for AI practitioners is that prompt performance is highly dependent on the interplay between the model, task type, and context, emphasizing the need for rigorous, model-specific testing when deploying structured prompting for complex textual analysis. |
| QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety (Read more on arXiv or HuggingFace) |
Yunho Maeng, Soo Yong Kim, Hyoungseo Cho, Jeonghwa Yoo, Taegyeong Lee |
QGuard is a zero-shot safety guard method that utilizes question prompting for multi-modal LLMs to block harmful prompts. The research aims to develop a simple, effective method for detecting both text-based and multi-modal harmful prompts without fine-tuning. The methodology involves categorizing harmful prompts into groups and generating guard questions that are combined with user input and fed to an MLLM to extract logits. Experimental results show the QGuard achieves an F1 score of 0.7438 on text-based harmful prompt detection and an F1 score of 0.8080 on multi-modal harmful prompt detection using InternVL-2.5. QGuard offers AI practitioners a practical safety guard applicable in real-world LLM services, adaptable to emerging threats with minimal computational resources. |
| EgoPrivacy: What Your First-Person Camera Says About You? (Read more on arXiv or HuggingFace) |
Xiaojun Shan, Yi Li, Jiacheng Cheng, Genpei Zhang, Yijiang Li |
i) The paper introduces EgoPrivacy, a benchmark for evaluating privacy risks inherent in egocentric video. ii) The research question investigates how much privacy information about a camera wearer can be inferred from their first-person view videos. iii) The methodology involves defining three types of privacy (demographic, individual, and situational), creating seven tasks, and developing a Retrieval-Augmented Attack (RAA) that utilizes ego-to-exo video retrieval. iv) Experiments show that foundation models can compromise wearer privacy, achieving 70-80% accuracy in recovering attributes such as identity, scene, gender, and race in zero-shot settings. v) AI practitioners should be aware that even zero-shot foundation models can compromise privacy in egocentric videos, necessitating robust privacy safeguards when utilizing or deploying such data or models. |
| Hatevolution: What Static Benchmarks Don’t Tell Us (Read more on arXiv or HuggingFace) |
Albert Meroño-Peñuela, Yulan He, Barbara McGillivray, Chiara Di Bonaventura |
i) This paper examines the temporal robustness of language models on evolving hate speech detection tasks. ii) The research questions how static hate speech benchmarks correlate with evolving language in the hate speech domain. iii) The study uses two experiments involving time-sensitive shifts and vocabulary expansion with neologisms, evaluating 20 language models with time-sensitive macro F1 and counterfactual invariance. iv) Results indicate that models exhibit performance volatility over time; for example, 6 out of 20 models flip labels on counterfactual sentences containing neologisms more than 10% of the time, and static benchmark performance does not consistently translate to time-sensitive evaluations with negative correlations observed in certain cases. v) AI practitioners should be aware of the limitations of static hate speech benchmarks and consider time-sensitive evaluations to ensure reliable safety assessments of language models, as static evaluations may overestimate model safety due to language evolution. |
Papers for 2025-06-16
| Title |
Authors |
Summary |
| Aligned Novel View Image and Geometry Synthesis via Cross-modal |
|
|
| Attention Instillation (Read more on arXiv or HuggingFace) |
Taekyoung Kim, Dongyoon Han, Sangdoo Yun, Junho Kim, Min-Seop Kwak |
i) This paper introduces a diffusion-based framework, MoAI, for generating aligned novel view images and geometry from unposed reference images. ii) The main objective is to achieve accurate novel view synthesis with geometrically robust consistency, even in extrapolative settings, by addressing the limitations of previous methods that require dense posed images. iii) The method utilizes off-the-shelf geometry predictors, formulates novel view synthesis as an inpainting task, and employs cross-modal attention instillation to transfer attention maps from the image branch to the geometry branch. iv) The model achieves state-of-the-art performance in extrapolative camera settings on RealEstate10K, with PSNR of 17.41, SSIM of 0.614, and LPIPS of 0.229. v) The principal implication is that the cross-modal attention instillation technique provides a mechanism to enhance geometrically-aware image generation, resulting in high-quality 3D scene completion for AI practitioners working on 3D reconstruction and novel view synthesis tasks. |
| Effective Red-Teaming of Policy-Adherent Agents (Read more on arXiv or HuggingFace) |
Guy Uziel, Matan Vetzler, Koren Lazar, George Kour, Itay Nakash |
i) The paper introduces CRAFT, a multi-agent red-teaming system for evaluating the robustness of policy-adherent LLM-based agents. ii) The research question is how to effectively assess and improve the resilience of policy-adherent agents against adversarial users attempting to bypass policy restrictions for personal gain. iii) The methodology involves a multi-agent system (CRAFT) with policy-aware persuasive strategies and the introduction of T-break, a benchmark for assessing agent robustness against manipulative behavior, used for red-teaming customer service scenarios. iv) CRAFT achieves a 70.0% attack success rate (ASR) in the airline domain, significantly outperforming generic attacks like DAN (35.0%) and emotional manipulation (50.0%), when targeting policy adherent agent. v) AI practitioners need to develop stronger safeguards to protect policy-adherent agents from adversarial attacks, as conventional red-teaming methods underestimate the true risk posed by malicious users. |
| The Diffusion Duality (Read more on arXiv or HuggingFace) |
Justin Chiu, Guanghan Wang, Aaron Gokaslan, Justin Deschenaux, Subham Sekhar Sahoo |
i) The paper introduces Duo, a framework that leverages Gaussian diffusion insights to improve Uniform-state Discrete Diffusion Models (USDMs). ii) It aims to bridge the performance gap between USDMs and other text generation models. iii) Duo incorporates a Gaussian-guided curriculum learning strategy and Discrete Consistency Distillation, adapting techniques from continuous diffusion to discrete settings. iv) Models trained with curriculum learning surpass autoregressive models in zero-shot perplexity on 3 of 7 benchmarks and Discrete Consistency Distillation reduces sampling steps from 1024 to 8, with minimal effect on sample quality. v) Duo provides AI practitioners with techniques to accelerate training of diffusion language models and unlock few-step generation by accelerating sampling by two orders of magnitude. |
| LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive |
|
|
| Programming? (Read more on arXiv or HuggingFace) |
Kaiyuan Liu, Shang Zhou, Zeyu Shen, Zerui Cheng, Zihan Zheng |
i) This paper introduces LiveCodeBench Pro, a competitive programming benchmark to evaluate LLMs’ algorithmic reasoning. ii) The research investigates the extent to which current LLMs exhibit human-level algorithmic reasoning in competitive programming scenarios. iii) The study employs a benchmark of Codeforces, ICPC, and IOI problems annotated by Olympiad medalists, along with line-by-line analysis of LLM-generated code. iv) The best model achieves 53% pass@1 on medium-difficulty problems and 0% on hard problems without external tools. v) AI practitioners should note LLMs’ significant limitations in nuanced algorithmic reasoning and the importance of implementation precision and tool augmentation, indicating areas for targeted improvement in code-centric LLM development. |
| pLSTM: parallelizable Linear Source Transition Mark networks (Read more on arXiv or HuggingFace) |
Sepp Hochreiter, Wei Lin, Thomas Schmied, Richard Freinschlag, korbip |
pLSTM introduces a parallelizable linear recurrent architecture for processing data structured as directed acyclic graphs (DAGs). The paper investigates how to extend linear RNNs to data with higher-level structures like 2D grids and trees. The core methodology translates Multi-Dimensional RNN principles to linear RNNs, introducing Source, Transition, and Mark gates. Experiments demonstrate pLSTM’s strong extrapolation abilities on an arrow-pointing task, generalizing well to larger image sizes compared to Transformers; furthermore, pLSTM achieved a top-1 accuracy of 75.51 on ImageNet-1k. pLSTM’s performance and scalability provide AI practitioners with an alternative recurrent architecture for modeling non-sequential data, addressing limitations of existing linear RNNs, though more data or experiments may be necessary to clearly demonstrate the full applicability of the research. |
| A High-Quality Dataset and Reliable Evaluation for Interleaved |
|
|
| Image-Text Generation (Read more on arXiv or HuggingFace) |
kpzhang, ZhangShenglin, fanrui00, cyrilli, finyorko |
i) The paper introduces InterSyn, a high-quality dataset, and SynJudge, an evaluation model, for interleaved image-text generation. ii) The main objective is to address the limitations of existing datasets and evaluation metrics for training and assessing instruction-following, multi-turn, interleaved image-text generation models. iii) The methodology involves constructing InterSyn using a Self-Evaluation with Iterative Refinement (SEIR) pipeline and developing SynJudge, a multi-dimensional evaluation model for assessing text content, image content, image quality, and image-text synergy. iv) Experiments show a 32% improvement in question quality and up to 52.1% gain in image-text synergy (ITS) for fine-tuned models, while SynJudge achieves 5% deviation from human judgment. v) InterSyn enables AI practitioners to train LMMs to achieve improved multimodal understanding, instruction following, and generation of coherent, synergistic interleaved content, while SynJudge provides a benchmark for quantitative assessment. |
| SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation |
|
|
| via Skill Blending (Read more on arXiv or HuggingFace) |
Tan-Dzung Do, Haoran Geng, jitendra1995, AmineElhafsi, yxK |
i) SkillBlender introduces a hierarchical reinforcement learning framework for versatile humanoid loco-manipulation through pre-trained skill blending. ii) The research objective is to develop a versatile and scalable humanoid control system capable of performing diverse loco-manipulation tasks with minimal task-specific reward engineering. iii) The methodology involves pre-training goal-conditioned task-agnostic primitive skills and dynamically blending these skills using a high-level controller that outputs subgoals and per-joint weight vectors. iv) Experiments show SkillBlender significantly outperforms baselines on a newly introduced SkillBench benchmark, exhibiting more accurate and feasible behaviors; for example, SkillBlender achieves a 0.007±0.004 error on the BoxTransfer task compared to 0.421±0.026 for MCP. v) SkillBlender’s pretrain-then-blend paradigm offers AI practitioners a method to reduce task-specific reward engineering and improve the versatility of humanoid robots, facilitating the development of more adaptable and robust control systems. |
| Detecting Harmful Memes with Decoupled Understanding and Guided CoT |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Anh Tuan Luu, Fengjun Pan, bobxwu |
i) This paper introduces U-CoT+, a framework for detecting harmful memes using decoupled understanding and guided reasoning. ii) The main objective is to improve resource efficiency, flexibility, and explainability in harmful meme detection. iii) The framework employs a meme-to-text pipeline to convert visual memes into textual descriptions, followed by guided CoT prompting using human-crafted guidelines. iv) Experiments on seven benchmark datasets show U-CoT+ achieves performance comparable to state-of-the-art supervised baselines, with small-scale LLMs achieving 72.90 (Acc) and 72.87 (F1) on the FHM dataset. v) The framework’s decoupled approach and guided CoT prompting enable resource-efficient and adaptable harmful meme detection, offering a practical solution for content moderation systems and highlighting the potential of small LLMs. |
| Beyond Homogeneous Attention: Memory-Efficient LLMs via |
|
|
| Fourier-Approximated KV Cache (Read more on arXiv or HuggingFace) |
Yuerong Song, Ruixiao Li, Qiqi Wang, Siyang He, Xiaoran Liu |
i) The paper introduces FourierAttention, a training-free KV cache compression framework for memory-efficient LLMs. ii) The research aims to reduce memory demands in LLMs by exploiting heterogeneous roles of transformer head dimensions. iii) The method projects long-context-insensitive head dimensions onto orthogonal Fourier bases, approximating temporal evolution with fixed-length spectral coefficients. iv) Evaluations on LLaMA models show FourierAttention achieves superior long-context accuracy on LongBench and Needle-In-A-Haystack (NIAH) tasks compared to existing methods; the method achieves a 94.04 overall score, as shown in Figure 5. v) FourierAttention offers AI practitioners a method for deploying LLMs in resource-constrained environments by optimizing KV cache memory without sacrificing long-context performance; FlashFourierAttention enables streamlined read-write operations. |
| SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement |
|
|
| Learning for LLM Reasoning (Read more on arXiv or HuggingFace) |
Yang Wang, Yeyun Gong, Zhong-Zhi Li, Xiao Liang, yegong |
i) This paper introduces Self-aware Weakness-driven problem Synthesis (SwS), a reinforcement learning framework that addresses model deficiencies by synthesizing targeted training problems. ii) The research aims to improve the reasoning capabilities of large language models (LLMs) by identifying and augmenting training data based on self-aware weaknesses. iii) SwS identifies weaknesses during preliminary RL training, extracts core concepts from failure cases, and synthesizes new problems to target these weaknesses. iv) Experiments on 7B and 32B models show average performance gains of 10.0% and 7.7%, respectively, across eight mainstream reasoning benchmarks. v) SwS enables AI practitioners to enhance LLM reasoning by generating targeted synthetic data that addresses specific model weaknesses, leading to more efficient RL training. |
| DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware |
|
|
| Regressive GRPO (Read more on arXiv or HuggingFace) |
Hyunwoo J. Kim, Jinyoung Kim, Jeehye Na, Jinyoung Park |
i) DeepVideo-R1 introduces a video reinforcement fine-tuning method using a regressive Group Relative Policy Optimization (Reg-GRPO) and difficulty-aware data augmentation for video LLMs. ii) The research aims to enhance the reasoning capabilities of video LLMs by addressing the limitations of GRPO, specifically safeguard reliance and the vanishing advantage problem. iii) The methodology involves reformulating the GRPO objective as a regression task to directly predict advantage, and employing a difficulty-aware data augmentation strategy to dynamically adjust training sample difficulty. iv) Experiments show DeepVideo-R1 achieves a 10.06 performance improvement compared to GRPO on the validation split of SEED-Bench-R1 dataset. v) The principal implication for AI practitioners is the demonstration of combining a regression-based RL objective with data augmentation to improve video reasoning performance in large-scale multimodal reasoning models. |
| Configurable Preference Tuning with Rubric-Guided Synthetic Data (Read more on arXiv or HuggingFace) |
vicgalle |
This paper introduces Configurable Preference Tuning (CPT), a framework for dynamically adjusting language model behavior using explicit directives. The main research objective is to endow LLMs with the ability to modulate outputs based on human-interpretable instructions without retraining. CPT leverages synthetically generated preference data conditioned on system prompts derived from structured rubrics defining attributes like writing style. Experiments showed that CPT-distilled models achieved an accuracy of 0.83 compared to a baseline of 0.60 using Mistral-Nemo-12B in matching target quality bins. These models can better align with specified quality categories, enabling AI practitioners to achieve fine-grained control over language model outputs for diverse applications. |
| ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual |
|
|
| Perception in VLMs (Read more on arXiv or HuggingFace) |
Yuhang Zhou, Yongyuan Liang, Chao Feng, Zhengyuan Yang, Xiyao Wang |
i) The paper introduces ViCrit, a reinforcement learning proxy task for enhancing visual perception in vision-language models (VLMs) by training them to identify synthetically injected visual hallucinations in image captions. ii) The primary objective is to develop a challenging, verifiable task that improves VLMs’ visual perception capabilities beyond object memorization. iii) The methodology involves training VLMs with a reinforcement learning framework, using a reward signal based on exact string matching to localize injected hallucinations in human-written image captions. iv) The results show that VLMs trained with ViCrit exhibit substantial gains across various VL benchmarks, including a 3.4% average accuracy improvement on general vision-language tasks for a 72B parameter model. v) The principal implication for AI practitioners is the provision of an effective, generalizable objective for enhancing visual perception in VLMs, enabling improvements in tasks such as abstract image reasoning and visual math. |
| A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data (Read more on arXiv or HuggingFace) |
Hsi-Chun Cheng, Liang-Hsuan Tseng, Ho-Lam Chung, Chan-Jan Hsu, Cheng Kang Chou |
i) This paper introduces a self-refining framework leveraging TTS-synthesized data to improve ASR performance using only unlabeled data. ii) The research aims to enhance ASR capabilities, specifically in low-resource languages and code-switching scenarios, by exploiting unlabeled data through a self-improvement cycle. iii) The methodology involves using an existing ASR model to generate pseudo-labels for unlabeled speech, training a TTS system on these pseudo-labels, and then bootstrapping the ASR model with synthesized speech-text pairs. iv) The resulting ASR model, Twister, reduces error rates by up to 20% on Mandarin and 55.88% on Mandarin-English code-switching benchmarks compared to the Whisper-large-v2 baseline. v) The framework provides AI practitioners a practical and data-efficient alternative to self-distillation approaches for improving ASR models in data-scarce scenarios, reducing the reliance on large volumes of real, labeled speech data. |
| Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity |
|
|
| Dilemma of Embeddings (Read more on arXiv or HuggingFace) |
Fandong Meng, Jiangnan Li, Mo Yu, Zhenlin Su, lxucs |
i) The paper introduces CapRetrieval, a Chinese image caption retrieval dataset, to reveal limitations in dense retrievers’ ability to encode fine-grained semantics. ii) The research investigates why dense retrievers fail on seemingly simple queries requiring fine-grained entity or event recognition. iii) The study constructs a new Chinese dataset, CapRetrieval, of image captions and entity/event queries and evaluates zero-shot and fine-tuned encoders. iv) Zero-shot evaluations show encoders struggle with fine-grained matching regardless of size (0.1B to 7B), while finetuning with data generation strategies improves performance, with a finetuned 0.1B model outperforming 7B baselines, although analysis reveals a granularity dilemma where fine-grained salience conflicts with overall semantic understanding. v) AI practitioners should consider the granularity dilemma when composing training data for dense retrievers, as emphasis on fine-grained details can compromise broader semantic encoding. |
| JAFAR: Jack up Any Feature at Any Resolution (Read more on arXiv or HuggingFace) |
Matthieu Cord, Jean-Emmanuel Haugeard, Louis Serrano, Loick Chambon, Paul Couairon |
i) The paper introduces JAFAR, a lightweight attention-based feature upsampler for foundation vision encoders. ii) The research aims to enhance the spatial resolution of visual features from any foundation vision encoder to an arbitrary target resolution. iii) JAFAR employs an attention-based module that promotes semantic alignment between high-resolution queries and semantically enriched low-resolution keys, using Spatial Feature Transform (SFT) modulation. iv) Experiments show JAFAR achieves a +1.63 mIoU improvement on average over existing methods across semantic segmentation benchmarks. v) JAFAR provides AI practitioners with a versatile drop-in module for improving feature resolution and performance in various downstream vision tasks without high-resolution supervision. |
| Inherently Faithful Attention Maps for Vision Transformers (Read more on arXiv or HuggingFace) |
Diego Marcos, Dino Ienco, Cassio F. Dantas, ananthu-aniraj |
i) This paper introduces iFAM, an attention-based method leveraging learned binary attention masks to improve model robustness against spurious correlations by restricting the receptive field of vision transformers (ViTs) to task-relevant regions. ii) The main objective is to develop a method that ensures attention maps are inherently faithful to the model’s reasoning, enhancing robustness against spurious correlations and out-of-distribution backgrounds. iii) iFAM uses a two-stage framework: the first stage discovers object parts and task-relevant regions using PDiscoFormer, and the second stage restricts the ViT’s receptive field to these regions via input attention masking. iv) Experiments show iFAM improves worst group accuracy (WGA) on MetaShift from 81.0% to 88.6% and from 94.0% to 97.0% on Waterbirds, indicating better robustness against background shifts. v) iFAM provides AI practitioners with a technique to create more robust vision models, reducing reliance on spurious correlations and improving generalization in diverse deployment scenarios, especially where contextual biases are prevalent. |
Papers for 2025-06-13
| Title |
Authors |
Summary |
| ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Weiwen Xu, Xingyu Qian, Swrooy, 26hzhang, YuSun-AI |
i) The paper introduces ReasonMed, a 370K example medical reasoning dataset. ii) The primary objective is to advance knowledge-intensive medical question answering by providing a large, high-quality dataset. iii) The methodology involves a multi-agent system for generating and verifying reasoning paths, including an Error Refiner powered by GPT-4o. iv) Results show a ReasonMed-7B model achieving state-of-the-art performance among sub-10B models and exceeding LLaMA3.1-70B on PubMedQA by 4.60%. v) The implication for AI practitioners is a new benchmark dataset to train and evaluate medical reasoning models, demonstrating that combining detailed Chain-of-Thought (CoT) reasoning with concise answer summaries yields the most effective fine-tuning strategy. |
| SWE-Factory: Your Automated Factory for Issue Resolution Training Data |
|
|
| and Evaluation Benchmarks (Read more on arXiv or HuggingFace) |
Pengyu Yang, Caihua Li, Yanlin Wang, Lianghong Guo, itaowe |
i) SWE-Factory is an automated pipeline for constructing GitHub issue resolution datasets and benchmarks. ii) The main objective is to automate the construction of GitHub issue resolution benchmarks by reducing manual effort in evaluation environment setup, grading, and validation. iii) The methodology involves a multi-agent system (SWE-Builder) for environment construction, exit-code-based grading, and automated fail2pass validation. iv) Experiments show SWE-Builder, with GPT-4.1-mini, constructs 269 valid task instances out of 671, achieving a valid rate of 40.1% with an average cost of $0.045 per instance, and exit-code-based grading achieves 100% accuracy compared to manual inspection. v) The primary implication is that SWE-Factory provides AI practitioners with an automated tool for creating large-scale, high-quality datasets, facilitating the development and evaluation of LLMs for software engineering tasks. |
| Text-Aware Image Restoration with Diffusion Models (Read more on arXiv or HuggingFace) |
Jihye Park, Jaeeun Lee, paulcho98, jinlovespho, Min-Jaewon |
i) The paper introduces Text-Aware Image Restoration (TAIR), a novel task for simultaneously recovering visual content and textual fidelity using diffusion models. ii) The main research objective is to address the challenge of text-image hallucination in degraded images by improving the reconstruction of textual regions. iii) The methodology involves creating SA-Text, a 100K image dataset, and proposing TeReDiff, a diffusion framework that integrates internal diffusion features with a text-spotting module. iv) Experiments show TeReDiff achieves superior performance with a F1-score of 69.29% on HQ level using ABCNetv2 on the SA-Text dataset, outperforming existing state-of-the-art restoration methods in text recognition accuracy. v) The principal implication for AI practitioners is the provision of a benchmark dataset and an effective diffusion model architecture for restoring images containing degraded text, thereby enhancing applications requiring both visual and textual clarity. |
| VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos (Read more on arXiv or HuggingFace) |
Meng Chu, Yue Wu, LarryLee, chupei, awojustin |
VRBench is introduced as a new benchmark for evaluating multi-step reasoning in long narrative videos. The main objective is to address limitations in existing evaluations by incorporating temporal reasoning and procedural validity. The methodology involves curating 1,010 long videos, annotating 9,468 multi-step question-answering pairs with 30,292 reasoning steps, and using a multi-phase evaluation pipeline. Evaluations of 28 LLMs and VLMs showed that GPT-4o achieved 83.25% outcome accuracy but a lower 58.1% process rating. The principal implication is providing a new tool for assessing and improving the reasoning capabilities of vision language models in complex, narrative contexts. |
| AniMaker: Automated Multi-Agent Animated Storytelling with MCTS-Driven |
|
|
| Clip Generation (Read more on arXiv or HuggingFace) |
Baotian Hu, Longyue Wang, Xinyu Chen, YunxinLi, MrSunshy |
i) AniMaker is a multi-agent framework for generating coherent, multi-character animated storytelling videos from text. ii) The research aims to automate the creation of storytelling animation using a multi-agent system. iii) The methodology employs a Monte Carlo Tree Search (MCTS)-inspired strategy (MCTS-Gen) for efficient clip generation and a novel evaluation framework (AniEval) for multi-shot animation assessment. iv) Experiments show AniMaker achieves superior performance with a 14.6% higher score in AniEval compared to the second-best model, with VBench results demonstrating the best average rank of 2.50. v) AniMaker offers AI practitioners an efficient framework for generating production-grade animated content, substantially improving the efficiency of multi-candidate generation. |
| Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture |
|
|
| without Training (Read more on arXiv or HuggingFace) |
Xipeng Qiu, Lu Wang, Howe77, mzzhang |
Domain2Vec introduces a novel approach to vectorize datasets for optimizing data mixtures in language model pretraining. The research aims to identify the optimal data mixture for pretraining language models without extensive training. The methodology involves decomposing datasets into a linear combination of meta-domains using a classifier and applying the Distribution Alignment Assumption. Domain2Vec achieves the same validation loss on Pile-CC using only 51.5% of the computation required when training on the original mixture of The Pile dataset and improves downstream performance by 2.83% under equivalent compute budget. Domain2Vec provides AI practitioners a computationally efficient and scalable method for determining optimal data mixtures for language model pretraining, reducing the need for extensive experimentation. |
| Optimus-3: Towards Generalist Multimodal Minecraft Agents with Scalable |
|
|
| Task Experts (Read more on arXiv or HuggingFace) |
Weili Guan, Gongwei Chen, Rui Shao, Yuquan Xie, Zaijing Li |
Optimus-3 is presented as a generalist multimodal agent for Minecraft capable of perception, planning, action, grounding, and reflection. This work aims to develop a general-purpose agent in Minecraft that overcomes challenges such as insufficient domain-specific data, task interference, and visual diversity. The methodology incorporates a knowledge-enhanced data generation pipeline, a Mixture-of-Experts architecture with task-level routing, and multimodal reasoning-augmented reinforcement learning. Experimental results show Optimus-3 achieves improvements of 20% on Planning and 76% on Embodied QA, compared to previous SOTA agents. The implementation of task-level routing with a MoE architecture offers AI practitioners a scalable and extensible approach to managing heterogeneous task learning in complex environments. |
| AutoMind: Adaptive Knowledgeable Agent for Automated Data Science (Read more on arXiv or HuggingFace) |
Lanning Wei, Jingsheng Zheng, Yujie Luo, Ningyu, OE-Heart |
AutoMind is an adaptive LLM agent framework designed for automated data science. The research aims to improve LLM agents by incorporating expert knowledge, agentic knowledge tree search, and self-adaptive coding. The methodology involves curating an expert knowledge base, developing an agentic knowledgeable tree search algorithm, and implementing a self-adaptive coding strategy. Experimental results on automated data science benchmarks demonstrate that AutoMind outperforms state-of-the-art baselines, surpassing 56.8% of human participants on MLE-Bench. This adaptive and knowledgeable approach provides AI practitioners with a more efficient and robust method for automating data science tasks. |
| Magistral (Read more on arXiv or HuggingFace) |
Gabrielle Berrada, Andy Lo, Albert Q. Jiang, Abhinav Rastogi, Mistral-AI |
Magistral introduces Mistral’s first reasoning model and reinforcement learning pipeline. The research aims to explore the limits of pure reinforcement learning (RL) training of Large Language Models (LLMs) without relying on existing distilled data. The methodology employs a ground-up approach, relying solely on internally-trained models and infrastructure with optimizations to the GRPO algorithm for training stability, multilingual consistency and reward shaping. The models achieved a nearly 50% increase in AIME-24 (pass@1) using pure RL and it also shows that multimodal reasoning capabilities emerge with online RL with textual data on top of a multimodal model. AI practitioners can leverage this scalable RL pipeline, for generating reasoning models from foundational models. |
| VideoDeepResearch: Long Video Understanding With Agentic Tool Using (Read more on arXiv or HuggingFace) |
Zhicheng Dou, Ji-Rong Wen, Junjie Zhou, Zheng Liu, Huaying Yuan |
VideoDeepResearch introduces an agentic framework for long video understanding (LVU). The paper aims to address LVU challenges by using a text-only large reasoning model (LRM) with a modular multi-modal toolkit instead of relying on large MLLMs with extended context windows. The methodology involves formulating problem-solving strategies via reasoning and selectively accessing video content using multimodal retrievers and visual perceivers. Results show VideoDeepResearch outperforms existing MLLMs, achieving a 9.6% improvement on MLVU (test). The work implies AI practitioners can overcome LVU challenges effectively through agentic systems leveraging readily available tools, suggesting a shift from monolithic models towards modular, tool-using approaches. |
| PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a |
|
|
| Unified Framework (Read more on arXiv or HuggingFace) |
Haoyu Chen, Tian Ye, Jialin Gao, Jianyu Lai, Ephemeral182 |
PosterCraft introduces a unified framework for high-quality aesthetic poster generation, moving beyond modular design paradigms. The main objective is to create a holistic approach capable of generating visually coherent and artistically compelling posters directly from textual input. PosterCraft employs a cascaded workflow, including scalable text rendering optimization via the Text-Render-2M dataset, region-aware fine-tuning on HQ-Poster-100K, aesthetic-text reinforcement learning, and joint vision-language feedback refinement. Experiments demonstrate PosterCraft significantly outperforms open-source baselines, achieving competitive performance with commercial systems and improving text rendering accuracy. This unified framework provides AI practitioners with a method to generate high-quality posters that integrates content, layout, and style cohesively. |
| ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark (Read more on arXiv or HuggingFace) |
Bozhong Tian, Siyuan Cheng, Kangwei Liu, Jasonchen123, Ningyu |
ChineseHarm-Bench is introduced as a benchmark for detecting harmful content in Chinese. The research aims to provide a comprehensive resource for content harm detection, covering six categories of violations. The methodology involves constructing a dataset from real-world violation records, expert annotation, and a knowledge-augmented baseline model. Results show that even state-of-the-art LLMs achieve macro-F1 scores of no more than 0.8, demonstrating limitations in Chinese harmful content detection. The benchmark and knowledge-augmented baseline provide a means for AI practitioners to evaluate and improve model performance in Chinese content moderation tasks. |
| CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic |
|
|
| Design Generation (Read more on arXiv or HuggingFace) |
Yutao Cheng, ShiLayne, YangMaoke, hxxxl, zbrl |
CreatiPoster is a framework for generating editable, multi-layer graphic compositions from natural-language instructions or assets. The research aims to generate high-quality, editable graphic designs automatically, addressing limitations in existing AI tools regarding user asset integration, editability, and professional visual appeal. The proposed framework utilizes a protocol model (RGBA large multimodal model) to produce a JSON specification detailing each layer (text or asset) and a conditional background model to synthesize a coherent background. The framework outperforms existing systems in a new benchmark with automated metrics and releases a copyright-free corpus of 100,000 multi-layer designs. AI practitioners can leverage the CreatiPoster framework to create editable graphic designs for diverse applications, including canvas editing, text overlay, responsive resizing, multilingual adaptation, and animated posters. |
| Resa: Transparent Reasoning Models via SAEs (Read more on arXiv or HuggingFace) |
Ömer Faruk Akgül, Julian Asilis, willieneis, deqing, upup-ashton-wang |
i) Resa introduces a novel and efficient sparse autoencoder tuning (SAE-Tuning) procedure for training reasoning language models. ii) The research aims to elicit strong reasoning in language models cost-effectively by leveraging their underlying representations. iii) SAE-Tuning trains an SAE to capture reasoning abilities from a source model and then uses it to guide supervised fine-tuning of a target model, using verified question-answer data. iv) SAE-Tuning retains >97% of its RL-trained counterpart’s reasoning performance while reducing training costs by >2000x to roughly $1 and training time by >450x to around 20 minutes. v) Practitioners can leverage the SAE-Tuning procedure to efficiently elicit and transfer reasoning abilities between language models with reduced computational costs and greater transparency. |
| Ming-Omni: A Unified Multimodal Model for Perception and Generation (Read more on arXiv or HuggingFace) |
Chunluan Zhou, Chuanyang Zheng, Cheng Zou, Biao Gong, Inclusion AI |
Ming-Omni proposes a unified multimodal model for perception and generation across images, text, audio, and video. The research aims to develop a single model capable of processing and generating multiple modalities without task-specific fine-tuning or structural redesign. Ming-Omni utilizes dedicated modality encoders processed by a Mixture-of-Experts architecture (Ling) with modality-specific routers, combined with an audio decoder and a diffusion-based image generator (Ming-Lite-Uni). Ming-Omni achieves a GenEval score of 0.64 in image generation, outperforming models like SDXL, and attains comparable image perception performance to Qwen2.5-VL-7B while using only 2.8B parameters. Ming-Omni offers AI practitioners an open-source architecture and methodology for building unified multimodal models with strong generation capabilities across diverse data types. |
| Eliciting Fine-Tuned Transformer Capabilities via Inference-Time |
|
|
| Techniques (Read more on arXiv or HuggingFace) |
codelion |
i) The paper investigates approximating capabilities acquired through supervised fine-tuning (SFT) of transformer models using inference-time techniques like in-context learning (ICL). ii) The main objective is to formally prove whether a base transformer model can elicit fine-tuned capabilities via ICL, under idealized and practical constraints. iii) The methodology involves theoretically constructing an inference technique TSFT utilizing ICL and quantifying minimal dataset sizes required for approximation, rooted in the Turing completeness of transformers. iv) For text generation tasks, a dataset of size O(mV log(V/ɛ)) or, with fixed context, O((1/ɛ²)log(V/δ)log(mV/ε²)) suffices to approximate fine-tuned distributions across m contexts; for linear classification, datasets of size O(d/ε²) or, with fixed context, O((1/ε²)log(1/δ)) are sufficient. v) AI practitioners can leverage these findings for resource-efficient LLM deployment by approximating SFT capabilities via ICL with minimal datasets, potentially enhancing real-world applications using techniques like retrieval-augmented generation (RAG). |
| Attention, Please! Revisiting Attentive Probing for Masked Image |
|
|
| Modeling (Read more on arXiv or HuggingFace) |
Tilemachos Aravanis, Ioannis Kakogeorgiou, Eirini Baltzi, Dionysis Christopoulos, Bill Psomas |
i) The paper introduces efficient probing (EP), a novel multi-query cross-attention mechanism for evaluating self-supervised learning models trained with masked image modeling (MIM). ii) The research aims to address the limitations of standard linear probing in assessing MIM models by improving the accuracy and efficiency of attentive probing. iii) The methodology involves revisiting existing attentive probing mechanisms, identifying key simplifications, and introducing EP, which eliminates redundant projections and reduces parameter count. iv) EP achieves up to a 10x speed-up over conventional multi-head attention while maintaining or surpassing state-of-the-art performance, reaching a top-1 accuracy of 75.6% on ImageNet-1k with MAE ViT-B using less than 1.4M parameters. v) AI practitioners can leverage EP as a computationally efficient and accurate evaluation method for SSL models, particularly for those trained with MIM, facilitating faster prototyping and model selection. |
| UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal |
|
|
| Gaussian Splatting (Read more on arXiv or HuggingFace) |
Jiwen Lu, Jie Zhou, Yanran21, LavenderLA |
i) UniPre3D is a novel unified 3D point cloud pre-training method utilizing cross-modal Gaussian splatting. ii) The primary objective is to develop a single pre-training approach applicable across varying scales of 3D point clouds and model architectures, addressing the current lack of a unified method effective for both object- and scene-level tasks. iii) The method predicts Gaussian primitives, renders images using differentiable Gaussian splatting for pixel-level supervision, and integrates 2D image features from pre-trained models through scale-adaptive fusion. iv) Experiments show UniPre3D outperforms existing methods on ScanObjectNN, achieving 87.93% accuracy on the PB_T50_RS benchmark with a standard Transformer backbone. v) The unified pre-training approach facilitates development of more generalizable 3D perception systems, potentially enabling AI practitioners to leverage a single model across diverse 3D data scales and tasks. |
| VerIF: Verification Engineering for Reinforcement Learning in |
|
|
| Instruction Following (Read more on arXiv or HuggingFace) |
Lei Hou, Bin Xu, Xiaozhi Wang, Yunjia Qi, Wesleythu |
i) The paper introduces VERIF, a verification method combining rule-based code verification and LLM-based verification for reinforcement learning (RL) in instruction following. ii) The research explores verification challenges in RL for instruction following, aiming to improve performance and generalization capabilities. iii) The methodology involves constructing a dataset (VERINSTRUCT) of approximately 22,000 instances with verification signals, followed by RL training using VERIF on SFT-trained models. iv) Results show significant improvements across instruction-following benchmarks; specifically, the TULU 3 SFT model trained with VERIF achieves state-of-the-art performance among comparable-sized models, with pass@64 showing over a 20% increase compared to pass@1 on IFEval. v) VERIF provides a practical approach for enhancing instruction-following capabilities in LLMs and can be integrated into existing RL pipelines, improving performance without compromising general capabilities; a smaller distilled LLM verifier (IF-Verifier-7B) is also explored to reduce computational costs for RL training. |
| Build the web for agents, not agents for the web (Read more on arXiv or HuggingFace) |
Siva Reddy, Marius Mosbach, Gaurav Kamath, xhluca |
i) This paper proposes a shift from adapting AI agents to existing web interfaces towards designing Agentic Web Interfaces (AWIs) specifically for agent interaction. ii) The main objective is to address the limitations of current web agent approaches caused by mismatches between human-designed interfaces and LLM capabilities. iii) The methodology involves introducing the concept of AWIs and outlining six guiding principles for their design, emphasizing safety, efficiency, and standardization. iv) The paper does not provide quantitative results but argues that AWIs can overcome fundamental interface limitations. v) The principal implication for AI practitioners is the need for a collaborative effort in designing AWIs to enable more efficient, reliable, and transparent web agent development. |
| Compound AI Systems Optimization: A Survey of Methods, Challenges, and |
|
|
| Future Directions (Read more on arXiv or HuggingFace) |
Guan-Bo Yang, Jui-Chao Lu, Mei-Yi Liu, Guan-Ting Yi, Yu-Ang Lee |
This paper surveys methods for optimizing compound AI systems, which integrate multiple components such as LLMs, simulators, and retrieval modules. The research objective is to systematically review and classify recent progress in optimizing these complex AI systems, encompassing both numerical and language-based techniques. The survey classifies existing methods based on structural flexibility and learning signals and presents a 2x2 taxonomy covering 26 representative works. The principal implication is a structured understanding of compound AI system optimization methods, providing a foundation for AI practitioners to design and refine complex AI workflows, though specific quantitative performance improvements across surveyed techniques remain to be identified in the summary provided. |
| LLM Unlearning Should Be Form-Independent (Read more on arXiv or HuggingFace) |
Shu Wu, Mengqi Zhang, Acruxos |
i) This paper identifies and mitigates Form-Dependent Bias in LLM unlearning. ii) The research objective is to demonstrate the failure of existing LLM unlearning methods to generalize across different input formats and proposes a form-independent solution. iii) The methodology involves characterizing Form-Dependent Bias, developing a benchmark (ORT) to evaluate unlearning robustness across diverse formats, and introducing Rank-One Concept Redirection (ROCR), a training-free parameter modification method. iv) Experiments showed that the probability of correct answers can be reduced by 58.12% on QA tasks but only by 5% on MCP tasks in RT and ROCR completed unlearning tasks in just 21 seconds. v) The findings imply that AI practitioners should consider the form-dependent vulnerability when deploying LLM unlearning techniques in security-critical applications, and ROCR is proposed as a solution along this direction. |
| What Makes a Good Natural Language Prompt? (Read more on arXiv or HuggingFace) |
Nancy F. Chen, Kenji Kawaguchi, Ngoc-Hai Nguyen, Duy Dinh, Do Xuan Long |
i) This paper proposes a property- and human-centric framework for evaluating natural language prompt quality, identifying 21 properties across six dimensions. ii) The main objective is to address the limited conceptual consensus on what quantifies effective natural language prompts. iii) The methodology involves a meta-analysis of 150+ prompting-related papers and blogs, followed by empirical exploration of multi-property prompt enhancements in reasoning tasks. iv) The study found that instruction-tuning on property-enhanced prompts can result in better reasoning models and observed that single-property enhancements often have the greatest impact on model performance. v) AI practitioners can leverage the proposed property-centric prompt evaluation framework for systematic prompt optimization and instruction tuning, bridging the gap between human-AI communication and improving model reasoning capabilities. |
| Breaking Data Silos: Towards Open and Scalable Mobility Foundation |
|
|
| Models via Generative Continual Learning (Read more on arXiv or HuggingFace) |
Yong Li, Chonghua Han, Yukun Liu, Yuan Yuan, JJ-TMT |
i) This paper introduces MoveGCL, a privacy-preserving framework for training mobility foundation models using generative continual learning across decentralized data silos. ii) The research aims to develop a scalable and privacy-conscious method for building generalizable mobility foundation models. iii) MoveGCL employs synthetic trajectory replay from a frozen teacher model, knowledge distillation, a Mixture-of-Experts Transformer with mobility-aware expert routing, and layer-wise progressive adaptation. iv) Experiments on six real-world urban datasets show MoveGCL achieves performance comparable to joint training and outperforms federated learning baselines, with 95% of generated trajectories not having a similarity score higher than 50% with real trajectories, showing limited data leakage. v) MoveGCL offers AI practitioners a practical blueprint for open, scalable, and privacy-preserving model development in the era of foundation models, particularly in privacy-sensitive domains like human mobility, enabling collaborative model evolution without raw data sharing. |
| Token Perturbation Guidance for Diffusion Models (Read more on arXiv or HuggingFace) |
Babak Taati, Soroush Mehraban, Javad Rajabi, msadat97 |
i) The paper introduces Token Perturbation Guidance (TPG), a novel training-free guidance method for diffusion models. ii) The primary research objective is to improve the generation quality and semantic alignment of diffusion models without specific training procedures or architectural changes and to make them agnostic to input conditions. iii) The methodology involves applying perturbation matrices, specifically norm-preserving shuffling, directly to intermediate token representations within the diffusion network. iv) Experiments on SDXL show that TPG achieves nearly a 2× improvement in FID for unconditional generation compared to the SDXL baseline; TPG also mirrors CFG. v) TPG provides AI practitioners with a condition-agnostic guidance method that extends CFG-like benefits to a broader class of diffusion models and allows for both conditional and unconditional generation. |
| Draft-based Approximate Inference for LLMs (Read more on arXiv or HuggingFace) |
Hyung Il Koo, Minjae Lee, Wonjun Kang, Ethan Ewer, Kevin Galim |
i) This paper introduces a novel framework, Draft-based Approximate Inference, for optimizing long-context Large Language Model (LLM) inference by leveraging smaller draft models to predict token and KV-pair importance. ii) The research aims to improve the accuracy of approximate LLM inference techniques, such as KV cache dropping and prompt compression, by using draft models to estimate token importance. iii) The methodology involves two instantiations: SpecKV for KV cache dropping and sparse prefilling, and SpecPC for prompt compression, both utilizing draft model outputs and attention activations to identify and discard less important tokens/KV pairs. iv) Experiments on long-context benchmarks demonstrate that the proposed methods achieve higher accuracy than existing baselines, with improvements up to 25 points on the RULER benchmark, while preserving memory usage, latency, and throughput. v) The work offers AI practitioners an effective strategy for accelerating LLM inference and improving resource efficiency, specifically highlighting the potential of draft models to enhance the performance of approximate inference techniques in scenarios with memory and computational constraints. |
| LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure |
|
|
| Profiles (Read more on arXiv or HuggingFace) |
Branislav Kveton, Aashish Anantha Ramakrishnan, Ting-Yao Hsu, Ho Yin ‘Sam’ Ng, Franck-Dernoncourt |
i) The paper introduces LAMP-CAP, a dataset for personalized figure caption generation using multimodal profiles. ii) The research aims to improve figure caption generation by incorporating personalization through multimodal profiles. iii) The methodology involves creating a dataset from scientific figures, figure-mentioning paragraphs, and related figures as profiles, and evaluating four LLMs on the caption generation task. iv) The primary result demonstrates that using multimodal profile information consistently improves the similarity of generated captions to ground-truth captions, with captions being the most critical profile element; experiments also reveal personalization is more effective (higher similarity) when profile figures share the same type as the target figure. v) LAMP-CAP provides AI practitioners with a new benchmark and a dataset to explore and implement personalized figure caption generation using multimodal profiles, improving the contextual relevance of generated captions. The effectiveness of multimodal profiles suggests that using figure images in personalization are preferred over text-only information. |
| MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness |
|
|
| Against VLM-based Attacks (Read more on arXiv or HuggingFace) |
Yiren Song, Xin Wei, Yule Xue, Zonglin Wu |
i) MCA-Bench is introduced as a multimodal CAPTCHA benchmark to evaluate the robustness against VLM-based attacks. ii) The objective is to rigorously evaluate the security robustness of diverse CAPTCHA schemes. iii) The methodology involves fine-tuning specialized cracking agents for each CAPTCHA category using a shared vision-language model backbone. iv) Experiments reveal VLMs achieve over 96% accuracy on simple tasks but as low as 2.5% on complex tasks involving physical interaction or multi-step logic. v) The principal implication is providing actionable insights for CAPTCHA hardening and guidance for human-machine verification in the face of intelligent-agent attacks. |
| Fine-Grained Perturbation Guidance via Attention Head Selection (Read more on arXiv or HuggingFace) |
Jaewon Min, Minjae Kim, Sanghyun Lee, Jiwon Kang, Donghoon Ahn |
i) The paper introduces a novel approach, HeadHunter, for fine-grained control in diffusion models by selectively perturbing individual attention heads. ii) The research investigates how granular attention perturbations, down to the individual head level, can improve generation quality and visual attribute control in diffusion models, specifically Diffusion Transformers (DiT). iii) HeadHunter iteratively selects attention heads based on user-defined objectives and introduces SoftPAG, which linearly interpolates attention maps toward an identity matrix for continuous perturbation strength tuning. iv) Experiments on DiT-based models like Stable Diffusion 3 demonstrate that HeadHunter achieves superior performance in general quality enhancement and style-specific guidance compared to layer-level perturbation, with effective heads not concentrated in any single layer. v) AI practitioners can leverage HeadHunter for targeted manipulation of generation quality and visual attributes in diffusion models by employing a systematic head selection framework, mitigating oversmoothing and improving control. |
| Discovering Hierarchical Latent Capabilities of Language Models via |
|
|
| Causal Representation Learning (Read more on arXiv or HuggingFace) |
Hanlin Zhang, Sham Kakade, Vasilis Syrgkanis, Jikai Jin |
i) This paper proposes a causal representation learning framework to discover hierarchical latent capabilities in language models. ii) The main objective is to address challenges in rigorously evaluating language model capabilities due to confounding effects and computational costs. iii) The methodology involves modeling benchmark performance as a linear transformation of latent capability factors identified through causal representation learning, controlling for the base model as a confounder; Hierarchical Component Analysis (HCA) is used to recover latent capabilities. iv) The study identifies a three-node linear causal structure from over 1500 models on the Open LLM Leaderboard, indicating a causal flow from general problem-solving to instruction-following and mathematical reasoning; minimal MIC (maximum inexactness coefficient) of 0.04 achieved. v) AI practitioners can utilize this framework to gain actionable insights for targeted post-training of language models by understanding the underlying causal relationships between latent capabilities. The impact of scaling up pre-training compute for downstream task performance has also been demonstrated. |
| StreamSplat: Towards Online Dynamic 3D Reconstruction from Uncalibrated |
|
|
| Video Streams (Read more on arXiv or HuggingFace) |
Renjie Liao, Lele Wang, Xuanyu Yi, Qi Yan, Zike Wu |
StreamSplat introduces an online framework for dynamic 3D Gaussian Splatting (3DGS) reconstruction from uncalibrated video streams. The research aims to enable real-time dynamic 3D scene reconstruction without calibrated camera poses. It employs a probabilistic sampling mechanism in a static encoder for 3DGS position prediction and a bidirectional deformation field for dynamic modeling. Experiments on the RE10K dataset show StreamSplat achieves a PSNR of 41.60 on given views, outperforming existing methods. This provides AI practitioners with a feed-forward method to reconstruct dynamic scenes from video without camera calibration. |
Papers for 2025-06-12
| Title |
Authors |
Summary |
| Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models (Read more on arXiv or HuggingFace) |
Ivan Oseledets, Andrey Kuznetsov, Alexander Zubrey, Matvey Skripkin, LiPengyi29 |
i) The paper introduces Reinforcement Learning via Self-Confidence (RLSC), a novel method for fine-tuning large language models (LLMs) using the model’s own confidence as a reward signal. ii) The research aims to develop a post-training optimization method for LLMs that aligns model behavior with task-specific goals without relying on human annotations or external reward models. iii) RLSC involves generating multiple completions per input, then optimizing a self-confidence objective based on the model’s probability assigned to its own responses. iv) Experiments on Qwen2.5-Math-7B, using only 16 samples per question, demonstrate improved accuracy of +13.4% on AIME2024 and +21.2% on MATH500. v) RLSC offers AI practitioners a simple and scalable post-training technique for enhancing LLM performance, requiring minimal data and computation by leveraging intrinsic model confidence. |
| Seedance 1.0: Exploring the Boundaries of Video Generation Models (Read more on arXiv or HuggingFace) |
Lu Jiang, Weilin Huang, Tuyen Hoang, Haoyuan Guo, Yu Gao |
Seedance 1.0 is introduced as a high-performance video generation model. The main objective is to address challenges in balancing prompt following, motion plausibility, and visual quality in video generation. The methodology comprises multi-source data curation with precision video captioning, an efficient architecture with decoupled spatial and temporal layers, and a video-tailored RLHF algorithm. Seedance 1.0 can generate a 5-second 1080p video in 41.4 seconds (NVIDIA-L20) and achieves first place on Artificial Analysis leaderboards for both text-to-video and image-to-video tasks. This model enables AI practitioners to generate high-quality videos with superior spatiotemporal fluidity and precise instruction adherence while maintaining efficient inference speeds. |
| PlayerOne: Egocentric World Simulator (Read more on arXiv or HuggingFace) |
Fan Wang, Xiang Bai, Xi Chen, Hao Luo, Yuanpeng Tu |
i) The paper introduces PlayerOne, a novel egocentric realistic world simulator. ii) The research aims to enable immersive and unrestricted exploration within dynamic virtual environments accurately aligned with real-scene human motion. iii) The method utilizes a coarse-to-fine training pipeline, including pretraining on egocentric text-video pairs and finetuning on synchronous motion-video data with a part-disentangled motion injection scheme and a joint reconstruction framework for 4D scene and video frame modeling. iv) Experimental results demonstrate PlayerOne’s generalization ability, achieving accurate control of human movements and consistent world modeling across diverse scenarios, with the model achieving a DINO-Score of 67.8 and a CLIP-Score of 88.2 on a constructed benchmark. v) PlayerOne provides AI practitioners with a new platform for developing and testing AI systems in interactive and realistic egocentric environments, particularly beneficial for applications requiring high-degree-of-freedom motion control and scene consistency. |
| Autoregressive Adversarial Post-Training for Real-Time Interactive Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yuxi Ren, Jianwen Jiang, Hao He, Ceyuan Yang, Shanchuan Lin |
i) The paper introduces Autoregressive Adversarial Post-Training (AAPT) for real-time interactive video generation. ii) The main objective is to transform a pre-trained latent video diffusion model into an efficient autoregressive generator suitable for interactive applications. iii) The methodology involves adversarial post-training using a block causal transformer architecture and student-forcing training. iv) The 8B-parameter AAPT model achieves real-time 24fps video generation at 736×416 resolution on a single H100 GPU with a latency of 0.16 seconds, enabling continuous 60-second video streams. v) AAPT offers AI practitioners a computationally efficient method for deploying real-time video generation systems, improving upon diffusion forcing methods and demonstrating comparable or improved performance in terms of video quality, particularly concerning long-duration consistency. |
| ComfyUI-R1: Exploring Reasoning Models for Workflow Generation (Read more on arXiv or HuggingFace) |
Weihua Luo, Longyue Wang, Xue Yang, Yiyu Wang, Zhenran Xu |
ComfyUI-R1 introduces a large reasoning model for automated ComfyUI workflow generation. The research aims to develop a model that automates the creation of complex ComfyUI workflows from user instructions. The methodology involves curating a dataset of 4K ComfyUI workflows and training a 7B-parameter model using a two-stage process: CoT fine-tuning and reinforcement learning with a rule-metric hybrid reward. The model achieves a 97% format validity rate and outperforms existing methods in node-level and graph-level F1 scores. The study suggests that large reasoning models with chain-of-thought reasoning can facilitate the creation of complex AI workflows, reducing the barrier to entry for AI art generation and conditional image/video processing tasks, although specific node-level and graph-level F1 scores are not stated. |
| SeerAttention-R: Sparse Attention Adaptation for Long Reasoning (Read more on arXiv or HuggingFace) |
Yu Cheng, Yuqing Xia, Shijie Cao, Shuming Guo, Yizhao Gao |
SeerAttention-R is introduced as a sparse attention framework for long reasoning models. The research focuses on improving long decoding efficiency specifically. SeerAttention-R learns attention sparsity through a self-distilled gating mechanism and removes query pooling for auto-regressive decoding. The methodology involves training the gating module of SeerAttention-R on 0.4B tokens and evaluating its performance on reasoning benchmarks like AIME under a 4K token budget. It maintains near-lossless accuracy and achieves up to 9x speedup over FlashAttention-3 on H100 GPU at 90% sparsity utilizing TileLang for a highly optimized sparse decoding kernel. It improves the viability of sparse attention as reasoning models scale by demonstrating near-lossless performance and hardware efficiency. |
| SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner (Read more on arXiv or HuggingFace) |
Mouxiang Chen, Jian Yang, Min Yang, Jiaxi Yang, Lei Zhang |
i) The paper introduces SWE-Flow, a novel data synthesis framework grounded in Test-Driven Development (TDD) for generating software engineering data. ii) The main objective is to create a framework to automatically generate structured development tasks and high-quality training instances for LLMs in software engineering. iii) SWE-Flow constructs a Runtime Dependency Graph (RDG) from unit test executions to infer incremental development steps and synthesize code, unit tests, and code modifications. iv) The framework generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, demonstrating that fine-tuning improves performance in TDD-based coding. v) SWE-Flow offers AI practitioners a means to synthesize verifiable software engineering data, enhancing LLM capabilities in incremental development tasks and enabling integration into reinforcement learning workflows. |
| InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio |
|
|
| Conditions (Read more on arXiv or HuggingFace) |
Gaojie Lin, Chao Liang, Jianwen Jiang, Jiaqi Yang, Zhenzhi Wang |
i) This paper introduces InterActHuman, a novel framework for multi-concept human animation with layout-aligned audio conditions. ii) The main research question is how to achieve spatial alignment of multi-modal conditions in multi-concept human video generation. iii) The methodology involves using a mask predictor to infer layout information and injecting local audio conditions into corresponding regions in an iterative manner. iv) Empirical results show state-of-the-art performance in lip synchronization, motion diversity, and subject appearance fidelity. v) The principal implication for AI practitioners is the introduction of an effective method for controllable multi-concept human-centric video generation, offering enhanced control over individual entities and their interactions in complex scenes. |
| SAFE: Multitask Failure Detection for Vision-Language-Action Models (Read more on arXiv or HuggingFace) |
Haruki Nishimura, Igor Gilitschenski, Shengxiang Sun, Yuanliang Ju, Qiao Gu |
i) The paper introduces SAFE, a multitask failure detection method for vision-language-action models (VLAs) designed to generalize to unseen tasks. ii) The main research objective is to develop a failure detector that can accurately identify potential failures of generalist robot policies, such as VLAs, across diverse tasks and environments. iii) The proposed method, SAFE, leverages internal features of VLAs and conformal prediction to estimate the likelihood of task failure, training on both successful and failed rollouts. iv) Experiments on OpenVLA, πο, and πο-FAST show that SAFE achieves state-of-the-art failure detection performance, achieving the best trade-off between accuracy and detection time, with ROC-AUC values up to 0.89 on held-out tasks in simulation. v) SAFE provides AI practitioners with a scalable and generalizable approach to robustly deploy VLAs in real-world robotic applications by promptly detecting potential failures without retraining or task-specific data. |
| Reparameterized LLM Training via Orthogonal Equivalence Transformation (Read more on arXiv or HuggingFace) |
Bernhard Schölkopf, Maximilian Dax, Tim Z. Xiao, Simon Buchholz, Zeju Qiu |
i) This paper introduces POET, a reparameterized training algorithm for LLMs using orthogonal equivalence transformations. ii) The research aims to improve the effectiveness and reliability of training large language models by controlling the spectral properties of weight matrices. iii) POET reparameterizes each neuron with learnable orthogonal matrices and a fixed random weight matrix, optimizing these matrices using stochastic primitive optimization and Cayley-Neumann parameterization. iv) Experiments show that POET achieves better performance than AdamW and GaLore, with POET-FS (b=1/2) yielding a validation perplexity of 13.70 on a 1.3B LLaMA model, surpassing AdamW’s 14.73. v) The primary implication is that POET provides AI practitioners with a more parameter-efficient and stable method for training LLMs, offering improvements in generalization and potentially reducing computational costs. |
| MIRAGE: Multimodal foundation model and benchmark for comprehensive |
|
|
| retinal OCT image analysis (Read more on arXiv or HuggingFace) |
Taha Emre, Ronald Fecso, Emese Sükei, Botond Fazekas, José Morano |
i) The paper introduces MIRAGE, a multimodal foundation model (FM) for retinal OCT and SLO image analysis, along with a corresponding benchmark for evaluation. ii) The research aims to develop a FM capable of robust performance across retinal image analysis tasks, particularly segmentation, and to provide a rigorous benchmark for validating such models. iii) A Vision Transformer (ViT) was pretrained on a multimodal dataset of paired OCT, SLO, and automatically generated retinal layer labels using a masked autoencoding (MAE) objective. iv) MIRAGE achieved an average AUROC of 95.59% on OCT classification tasks, outperforming the second-best model by 1.15 percentage points and showed significant improvements in cross-dataset evaluations and segmentation, achieving a Dice score of 78.46% for OCT tasks. v) MIRAGE offers AI practitioners a robust FM for retinal image analysis that can be adapted for classification and segmentation tasks, along with a benchmark for evaluating and comparing new models. |
| Branched Schrödinger Bridge Matching (Read more on arXiv or HuggingFace) |
Pranam Chatterjee, Alexander Tong, Yinuo Zhang, Sophia Tang |
i) The paper introduces Branched Schrödinger Bridge Matching (BranchSBM) for modeling divergent transitions between probability distributions. ii) The research aims to learn branched trajectories from a unimodal initial distribution to multiple target distributions, addressing limitations of existing methods in capturing branching dynamics. iii) BranchSBM parameterizes multiple time-dependent velocity fields and growth processes, formulating a branched Conditional Stochastic Optimal Control (CondSOC) problem and leveraging a multi-stage training algorithm. iv) Experiments show BranchSBM can accurately reconstruct endpoint distributions on a LiDAR manifold with Wasserstein distances W1 = 0.239 and W2 = 0.309, outperforming single-branch SBM. v) BranchSBM provides AI practitioners with a framework for modeling dynamic branching trajectories in tasks such as multi-path surface navigation, single-cell population dynamics, and predicting heterogeneous cell states after perturbation, which may improve generative modeling applications. |
Papers for 2025-06-11
| Title |
Authors |
Summary |
| Geopolitical biases in LLMs: what are the “good” and the “bad” countries |
|
|
| according to contemporary language models (Read more on arXiv or HuggingFace) |
Dmitrii Korzh, tlenusik, apanc, IvanLazichny, msalnikov |
This paper evaluates geopolitical biases in LLMs by analyzing their interpretation of historical events. The research question is: Do LLMs demonstrate geopolitical biases by showing a preference for specific national perspectives when interpreting controversial historical events? The methodology involves a structured framework with a manually collected dataset of neutral event descriptions and contrasting viewpoints from the USA, UK, USSR, and China, analyzed across GPT-4o-mini, llama-4-maverick, Qwen2.5 72B, and GigaChat-Max. The primary results show significant geopolitical biases, with models favoring specific national narratives (e.g., GPT-40-MINI favors USA in 76% of cases vs. USSR). The principal implication for AI practitioners is the need for advanced debiasing strategies to mitigate national narrative biases in LLMs, as simple debiasing prompts had limited effect. |
| RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic |
|
|
| Sampling (Read more on arXiv or HuggingFace) |
Jiaqi Li, Yang Liu, zlzheng |
i) RuleReasoner enhances rule-based reasoning in small language models using reinforcement learning and domain-aware dynamic sampling. ii) The research investigates whether reinforcement learning can effectively enhance rule-based reasoning capabilities in small language models and generalize across diverse tasks. iii) The methodology involves a novel reinforcement learning with verifiable rewards (RLVR) framework and a domain-aware dynamic sampling (DADS) algorithm that dynamically reweights training domains based on historical rewards. iv) RuleReasoner achieves a 4.1% average points improvement on eight in-distribution tasks and a 10.4% average points improvement on three out-of-distribution tasks compared to OpenAI-01. v) Practitioners can leverage RuleReasoner to improve the reasoning performance of small language models with enhanced sample utilization, reducing the need for large-scale models or extensive human-engineered training recipes. |
| Solving Inequality Proofs with Large Language Models (Read more on arXiv or HuggingFace) |
Alex Gu, Tony Xia, Jikai Jin, Luna Lyu, Jiayi Sheng |
i) The paper introduces INEQMATH, a benchmark for evaluating LLMs on Olympiad-level inequality proofs. ii) The research aims to assess LLMs’ ability to perform rigorous mathematical reasoning in the context of inequality proving. iii) It utilizes a novel LLM-as-judge evaluation framework that assesses both final-answer correctness and step-wise solution soundness. iv) Evaluation of 29 LLMs reveals that even advanced models like o1 achieve less than 10% overall accuracy under step-wise scrutiny, representing a drop of up to 65.5% compared to final-answer accuracy alone. v) The findings imply that current LLMs exhibit a significant gap between finding correct answers and constructing rigorous mathematical proofs, highlighting the need for future research into areas such as theorem-guided reasoning and self-refinement to improve proof correctness. |
| Self Forcing: Bridging the Train-Test Gap in Autoregressive Video |
|
|
| Diffusion (Read more on arXiv or HuggingFace) |
Eli Shechtman, Mingyuan Zhou, Zhengqi Li, Xun Huang, gdhe17 |
i) The paper introduces Self Forcing, a training paradigm for autoregressive video diffusion models designed to mitigate exposure bias. ii) The research aims to bridge the train-test distribution gap in autoregressive video diffusion models to improve video generation quality and efficiency. iii) The methodology involves training the model through autoregressive self-rollout with KV caching, enabling supervision through a holistic video-level loss and employing a few-step diffusion model with stochastic gradient truncation. iv) Experiments show that Self Forcing enables real-time video generation at 17 FPS with sub-second latency on a single H100 GPU, achieving comparable or superior generation quality to existing diffusion models. v) The Self Forcing training paradigm allows for the development of lower latency video generation that is more suitable for real-time, interactive video generation, streaming and gaming applications. |
| Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error |
|
|
| Diagnosis in GUI Automation (Read more on arXiv or HuggingFace) |
Junyang Wang, Haowei Liu, Haiyang Xu, Xi Zhang, Yuyang Wanyan |
i) The paper introduces GUI-Critic-R1, a model for pre-operative error diagnosis in GUI automation. ii) The research aims to improve GUI automation by providing feedback before action execution, addressing issues of error accumulation and inefficiency. iii) A Suggestion-aware Gradient Relative Policy Optimization (S-GRPO) strategy was used to construct the model, along with a reasoning-bootstrapping pipeline to generate GUI-Critic-Train and GUI-Critic-Test datasets. iv) Experiments show that GUI-Critic-R1 improves the success rate of a baseline GUI automation system from 22.4% to 27.6% on the AndroidWorld benchmark. v) The GUI-Critic-R1 and S-GRPO are provided as a way to improve single-step accuracy in GUI agents which has low tolerance for decision-making errors at each step. |
| Aligning Text, Images, and 3D Structure Token-by-Token (Read more on arXiv or HuggingFace) |
Georgia Gkioxari, Vansh Tibrewal, Aadarsh Sahoo |
Kyvo is introduced as a decoder-only transformer model that aligns text, images, and structured 3D scenes token-by-token. The research investigates the potential of autoregressive models for structured 3D scene understanding and generation. The methodology involves designing and training an LLM with modality-specific tokenizers for images and 3D, evaluating across four core 3D tasks. The model achieved a Jaccard Index of 0.4784 on Objectron for real-world 3D object recognition, demonstrating competitive performance compared to specialized 3D object detectors. This unified LLM framework enables AI practitioners to tackle a variety of complex visual 3D tasks, such as 3D reconstruction and 3D-conditioned image generation. Some implementation details like training throughput (8,800 tokens/sec/GPU) are also mentioned. |
| Frame Guidance: Training-Free Guidance for Frame-Level Control in Video |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Soo Ye Kim, Jaehyeong Jo, Sangwon Jang, jaehong31, tkkitkki |
Frame Guidance is presented as a novel training-free method for controllable video generation using frame-level signals within video diffusion models (VDMs). The research aims to achieve fine-grained control over video generation without task-specific fine-tuning of large VDMs. The proposed method utilizes a latent processing technique called latent slicing to reduce memory usage and video latent optimization (VLO) for coherent video generation by applying deterministic optimization in early stages. Experiments demonstrate that Frame Guidance can produce high-quality controlled videos across diverse tasks and inputs, achieving an FID score of 55.60 and FVD score of 577.1 on the DAVIS dataset for keyframe guided generation when applied to CogX, outperforming training-required baselines. This training-free guidance method offers AI practitioners a flexible and efficient approach to control video generation using frame-level signals, potentially reducing computational costs and model retraining efforts. |
| ECoRAG: Evidentiality-guided Compression for Long Context RAG (Read more on arXiv or HuggingFace) |
Seung-won Hwang, Dohyeon Lee, Jinsu Kim, yeonseokjeong |
i) ECoRAG improves LLM performance by compressing retrieved documents based on evidentiality in Retrieval-Augmented Generation (RAG). ii) The research aims to improve LLM performance on Open-Domain Question Answering (ODQA) tasks by filtering out non-evidential information in RAG. iii) The methodology involves compressing documents guided by evidentiality and reflecting on the evidentiality of compressed content to retrieve more if necessary. iv) Experiments on Natural Questions demonstrate ECoRAG achieves 36.48% exact match, outperforming standard RAG and RECOMP. v) Practitioners can utilize ECoRAG to improve LLM performance and reduce computational costs by filtering irrelevant content in RAG applications. |
| DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for |
|
|
| Parameter-Efficient Video-Text Retrieval (Read more on arXiv or HuggingFace) |
Yifeng Zhang, Tao He, Tianxiang Hao, Guoqiang Gong, lunar677 |
i) DiscoVLA addresses discrepancies in vision, language, and alignment for parameter-efficient video-text retrieval. ii) The research objective is to mitigate vision, language, and alignment discrepancies that arise when adapting image-text pre-training models like CLIP to video-text retrieval. iii) The methodology involves an Image-Video Features Fusion module, Pseudo Image-level Alignment, and Image-to-Video Alignment Distillation. iv) On the MSRVTT dataset with CLIP (ViT-B/16), DiscoVLA achieves a 50.5% R@1, surpassing previous methods by 1.5%. v) DiscoVLA offers AI practitioners a novel approach to improve video-text retrieval by simultaneously addressing vision, language, and alignment discrepancies. |
| Squeeze3D: Your 3D Generation Model is Secretly an Extreme Neural |
|
|
| Compressor (Read more on arXiv or HuggingFace) |
Nandita Vijaykumar, Mohammadreza Mofayezi, Sankeerth Durvasula, Yushi Guan, rishitdagli |
Squeeze3D proposes a novel framework for compressing 3D data by leveraging the implicit prior knowledge learned by pre-trained 3D generative models. The research aims to achieve high compression ratios for 3D data in various formats (meshes, point clouds, radiance fields) using existing pre-trained encoders and generators. The methodology involves training forward and reverse mapping networks to bridge the latent spaces between pre-trained encoders and generators using a synthetic 3D dataset generated from the pre-trained generator. Experiments demonstrate Squeeze3D achieves compression ratios of up to 2187× for textured meshes while retaining comparable visual quality. The principal implication is a method for AI practitioners to achieve extreme 3D data compression without training object-specific networks, enabling efficient storage and transmission of 3D content. |
| MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient |
|
|
| Fine-Tuning of Large Language Models (Read more on arXiv or HuggingFace) |
Wenqiao Zhang, Rolan Yan, Hongyang He, Tianwei Lin, cajie |
i) The paper introduces Heterogeneous Mixture-of-Adapters (MoA), a parameter-efficient fine-tuning approach utilizing diverse PEFT adapter architectures. ii) The research aims to address representation collapse and load imbalance in MoE-LoRA methods by integrating diverse adapter structures for enhanced expert specialization. iii) MoA employs token-level dynamic routing to activate PEFT adapter experts with diverse structures, including LoRA, parallel adapters, and prompt tuning. iv) Experiments show MoA achieves 81.51% accuracy on math benchmarks, outperforming homogeneous MoE-LoRA with fewer trainable parameters (24.52M). v) AI practitioners can leverage MoA to achieve higher parameter efficiency and knowledge transfer in LLMs while reducing memory consumption and improving inference speed. |
| Institutional Books 1.0: A 242B token dataset from Harvard Library’s |
|
|
| collections, refined for accuracy and usability (Read more on arXiv or HuggingFace) |
Kristi Mukk, Jack Cushman, John Hess, Catherine Brobston, Matteo Cargnelutti |
i) Institutional Books 1.0, a 242B token dataset of public domain books from Harvard Library, is introduced to address the scarcity of high-quality training data for large language models. ii) The research objective was to create a usable and documented dataset of historic texts from Harvard Library’s digitized collections. iii) The methodology included extracting digitized books, analyzing temporal and language coverage, performing topic classification using a fine-tuned BERT model, collection-level deduplication, and OCR artifact analysis, followed by optional OCR post-processing. iv) The dataset comprises 983,004 volumes (242B tokens), with a topic classification model achieving 97.8% accuracy on a benchmark dataset, and 91.41% of the volumes being identified as public domain by HathiTrust. v) The curated dataset provides AI/ML practitioners with a large, publicly available resource of historical text to enhance long context comprehension and text generation models. |
| Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction (Read more on arXiv or HuggingFace) |
Amrith Setlur, Yifei Zhou, Lunjun Zhang, Junhong Shen, JackBAI |
i) This paper introduces interaction scaling as a new dimension for test-time scaling of agents that emphasizes acting more, rather than simply thinking more. ii) The main objective is to demonstrate that increasing the number of interaction steps during test-time improves agent performance in interactive environments, enabling behaviors like exploration and backtracking. iii) The methodology involves a curriculum-based online reinforcement learning (RL) approach called TTI (Test-Time Interaction), which trains agents by adaptively adjusting their rollout lengths, along with prompting to scale test-time interaction. iv) Experiments show that TTI, using a Gemma 3 12B model, achieves state-of-the-art open-source, open-data web agent performance on WebVoyager and WebArena, improving over a non-fine-tuned agent by 9% and 8%, respectively. v) The principal implication for AI practitioners is that interaction scaling is a powerful, complementary axis to scaling per-step compute, offering new avenues for training adaptive agents capable of balancing exploration and exploitation in dynamic environments, suggesting a shift from purely reactive policies to adaptive policies that collect information on-the-fly. |
| Mathesis: Towards Formal Theorem Proving from Natural Languages (Read more on arXiv or HuggingFace) |
Roozbeh Yousefzadeh, Pengyi Zhai, Zijin Feng, Yu Xuejun, Jianyuan1 |
Mathesis introduces an end-to-end theorem proving pipeline, addressing the gap between natural language problem statements and formal reasoning systems. The research aims to automate formal theorem proving from informal problem statements. The methodology employs a Mathesis-Autoformalizer trained via reinforcement learning with a hierarchical preference optimization mechanism and introduces a novel LeanScorer for nuanced formalization quality assessment. The system achieves a 64% accuracy on MiniF2F with pass@32 and 18% on Gaokao-Formal. Mathesis provides AI practitioners with an automated system for formalizing and proving theorems directly from natural language, thus enhancing the applicability of formal methods to real-world problems. |
| RKEFino1: A Regulation Knowledge-Enhanced Large Language Model (Read more on arXiv or HuggingFace) |
Jeff Zhao, Ruoyu Xiang, Yueru He, YanAdjeNole |
i) The paper introduces RKEFino1, a regulation knowledge-enhanced language model for improved accuracy and compliance in Digital Regulatory Reporting (DRR). ii) The research aims to enhance the interpretability, compliance accuracy, and reliability of financial language models in DRR tasks using domain-specific knowledge. iii) The methodology involves fine-tuning the Fino1 model with domain knowledge from XBRL, CDM, and MOF, and formulating knowledge-based QA, mathematical reasoning QA, and numerical NER tasks. iv) Experimental results show that RKEFino1 achieves a 26.62% F1-score on the numerical NER task, outperforming the baseline Fino1 model’s 14.99%. v) RKEFino1 offers AI practitioners a fine-tuned language model that demonstrates improved generalization in DRR tasks, particularly in recognizing numerical entities within financial text and tables, indicating potential utility for compliance-critical applications. |
| QQSUM: A Novel Task and Model of Quantitative Query-Focused |
|
|
| Summarization for Review-based Product Question Answering (Read more on arXiv or HuggingFace) |
Zhuang Li, Minh Ngoc Dinh, Xiuzhen Zhang, An Quang Tang |
QQSUM introduces a novel task and model for quantitatively summarizing diverse customer opinions in review-based product question answering (PQA). The main research question is how to summarize diverse customer opinions into representative key points (KPs) and quantify their prevalence to effectively answer user queries in review-based PQA. They propose QQSUM-RAG, an extension of the RAG framework using few-shot learning to jointly train a KP-oriented retriever and a KP summary generator. Experimental results demonstrate that QQSUM-RAG achieves superior performance in both textual quality and quantification accuracy, with up to 2.11 times improvement in textual similarity with ground-truth KPs and up to 67.12% improvement in quantification performance over a state-of-the-art KPA system. The principal implication for AI practitioners is a new approach to PQA that captures the diversity of customer opinions using KP-based summarization, which improves both the textual quality and the quantification accuracy of the responses. |
Papers for 2025-06-10
| Title |
Authors |
Summary |
| Reinforcement Pre-Training (Read more on arXiv or HuggingFace) |
Tianzhu Ye, Qingxiu Dong, frontierai, YaoTang23, unilm |
i) This paper introduces Reinforcement Pre-Training (RPT), a novel scaling paradigm for large language models by reframing next-token prediction as a reinforcement learning task. ii) The main research objective is to improve language modeling accuracy and provide a strong pre-trained foundation for reinforcement fine-tuning by incentivizing next-token reasoning. iii) RPT employs on-policy reinforcement learning with intrinsic, verifiable rewards based on the correctness of next-token predictions, leveraging vast amounts of text data without external annotations. iv) Experiments show that RPT significantly improves next-token prediction accuracy, with RPT-14B achieving consistently higher accuracy across all difficulty levels compared to R1-Distill-Qwen-14B, and reaching the performance of R1-Distill-Qwen-32B. v) RPT offers AI practitioners an effective pre-training approach that enhances both language modeling and reasoning capabilities, providing a stronger foundation for subsequent reinforcement fine-tuning and improving zero-shot performance on downstream tasks. |
| Lingshu: A Generalist Foundation Model for Unified Multimodal Medical |
|
|
| Understanding and Reasoning (Read more on arXiv or HuggingFace) |
26hzhang, gowitheflow, Jianyu, kenchan0226, xww033 |
i) LINGSHU is a new medical-specialized multimodal large language model (MLLM) aimed at improving medical understanding and reasoning. ii) The primary objective is to address the limitations of existing MLLMs in medical applications by enhancing medical knowledge coverage, reducing hallucinations, and improving reasoning capabilities. iii) The methodology includes a comprehensive data curation procedure acquiring medical knowledge from imaging, texts, and general-domain data, along with multi-stage training. iv) Results show that LINGSHU outperforms existing open-source multimodal models on medical tasks like multimodal QA, text-based QA, and medical report generation, demonstrating a 7.2% average accuracy improvement over the second-best model in medical VQA tasks. v) LINGSHU offers AI practitioners a framework for building more robust and reliable MLLMs tailored for specialized domains like medicine, particularly concerning data curation and model training strategies. |
| MiniCPM4: Ultra-Efficient LLMs on End Devices (Read more on arXiv or HuggingFace) |
Yuxuan Li, MiniCPM Team, BigDong, guojunshaoyao, xcjthu |
i) This paper introduces MiniCPM4, a highly efficient large language model (LLM) designed for end-side devices. ii) The main objective is to achieve efficiency in LLMs through innovations in model architecture, training data, training algorithms, and inference systems. iii) The methodology includes proposing InfLLM v2 (a trainable sparse attention mechanism), UltraClean (a data filtering strategy), and CPM.cu (CUDA inference framework). iv) MiniCPM4-8B achieves a 7-fold speed improvement in processing 128K-length documents compared to Qwen3-8B on end-side devices. v) The research implies that systematic innovation can create efficient LLMs for resource-constrained environments, significantly reducing computational costs for AI practitioners. |
| Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety |
|
|
| Assurance (Read more on arXiv or HuggingFace) |
Hanghang Tong, Jingrui He, Tianxin Wei, Gaotang Li, Ruizhong Qiu |
i) This paper introduces SAFFRON, a novel inference scaling paradigm for enhancing LLM safety. ii) The primary research objective is to address the exploration-efficiency dilemma in scaling inference for LLM safety assurance. iii) The methodology involves replacing the process reward model (PRM) with a multifurcation reward model (MRM), trained with partial supervision and a conservative exploration constraint, and employing a Trie-based key-value caching strategy. iv) Results show that SAFFRON achieves a lower attack success rate (ASR) of 0.409 on Harmful HEx-PHI, outperforming baseline methods under the same inference compute budget. v) AI practitioners can leverage SAFFRON to improve the robustness of LLMs against jailbreak attacks by employing a multifurcation reward model, thereby significantly enhancing safety in resource-constrained environments. |
| OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (Read more on arXiv or HuggingFace) |
Shuhan Wu, Peng Xing, Jingjing Chang, wchengad, fangyixiao |
i) OneIG-Bench is introduced as a comprehensive benchmark for fine-grained evaluation of text-to-image models across multiple dimensions. ii) The paper aims to provide a holistic framework for evaluating T2I models across dimensions including prompt-image alignment, text rendering, reasoning, stylization, and diversity. iii) The methodology involves a curated dataset of over 1000 prompts categorized into six core assessment categories, along with quantitative metrics tailored to each dimension. iv) Experiments show that models like GPT-4o demonstrate superior performance in knowledge retention and reasoning ability, but no single model exhibits outstanding performance across all specific subjects and that OneIG-Bench facilitates identification of model strengths and weaknesses. v) AI practitioners can leverage OneIG-Bench for in-depth model performance analysis, assisting in pinpointing strengths and bottlenecks in T2I pipelines and enabling focused improvements. |
| SpatialLM: Training Large Language Models for Structured Indoor Modeling (Read more on arXiv or HuggingFace) |
Rui Tang, Chuan Fang, Junhao Zhong, bertjiazheng, ysmao |
SPATIALLM fine-tunes large language models for structured 3D indoor scene understanding from point cloud data. The research aims to enhance LLMs’ spatial understanding capabilities for tasks like layout estimation and 3D object detection. A large-scale synthetic dataset of 12,328 indoor scenes with 3D annotations was created to train a standard multimodal LLM architecture. The model achieves state-of-the-art performance in layout estimation on public benchmarks and competitive results in 3D object detection, reaching 86.5% IOU2D@0.25 on the Structured3D dataset for layout estimation after fine-tuning. This provides a feasible approach for leveraging LLMs to enhance spatial understanding in applications like augmented reality and robotics, showing how existing LLMs can be augmented with new datasets for specific spatial reasoning tasks. |
| Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Yansheng Wang, Ziyang Liu, Jiaxin Hu, Peiyu He, sc-bd |
Astra presents a dual-model architecture for mobile robot navigation using hierarchical multimodal learning. The research addresses the challenges of goal localization, self-localization, and path planning in complex indoor environments. Astra employs a multimodal LLM (Astra-Global) for global tasks and a multitask network (Astra-Local) with a 4D spatial-temporal encoder for local tasks, trained via supervised finetuning, reinforcement learning, and self-supervision. Experiments show Astra achieves a high end-to-end mission success rate (84.2% in warehouses, 99.1% in office buildings). This work offers AI practitioners a comprehensive framework for developing adaptable and high-performing mobile robots in diverse environments by combining LLMs with task-specific networks. |
| Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers (Read more on arXiv or HuggingFace) |
Wangmeng Zuo, Zhaoxi Chen, Zhengyao Lv, ChenyangSi, ldiex |
i) This paper introduces TACA, a parameter-efficient method for enhancing text-image alignment in Multimodal Diffusion Transformers (MM-DiTs). ii) The research aims to address cross-modal attention suppression and timestep-insensitive weighting in MM-DiTs to improve text-image alignment. iii) The proposed TACA method dynamically rebalances cross-modal attention using temperature scaling and timestep-dependent adjustment and is combined with LoRA fine-tuning. iv) Experiments on T2I-CompBench show that TACA improves spatial relationship understanding by 16.4% on FLUX.1-Dev and by 28.3% on SD3.5-Medium. v) TACA offers AI practitioners a computationally inexpensive method to improve semantic fidelity in text-to-image diffusion models by dynamically balancing cross-modal attention. |
| GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular |
|
|
| Structure Recognition (Read more on arXiv or HuggingFace) |
Xingjian Wei, Yifan He, Jiang Wu, Hoter, jcwang0602 |
i) The paper introduces GTR-Mol-VLM, a new framework for Optical Chemical Structure Recognition (OCSR) using graph traversal as a visual chain of thought. ii) The research objective is to improve OCSR performance, particularly in complex molecular structures with abbreviated functional groups, by addressing limitations in existing image-captioning-based vision-language models (VLMs). iii) The methodology employs a Graph Traversal as Visual Chain of Thought mechanism for incremental parsing through atom-bond predictions and a data-centric approach called “Faithfully Recognize What You’ve Seen” to manage abbreviated structures. iv) GTR-Mol-VLM outperforms existing specialist models and chemistry-domain VLMs, with an approximately 14 percentage point improvement over the second-best baseline on molecular images with functional group abbreviations. v) GTR-Mol-VLM’s graph traversal and data correction techniques offer AI practitioners advanced methods for parsing complex visual structures, enhancing accuracy and consistency in applications requiring detailed structural analysis, such as cheminformatics and AI for Science. |
| Through the Valley: Path to Effective Long CoT Training for Small |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Wei Lu, Jiaxi Li, Albus-Chen, RogerLos |
i) This paper investigates performance degradation in small language models (SLMs) when trained with limited long chain-of-thought (CoT) data. ii) The research question focuses on understanding and mitigating the “Long CoT Degradation” phenomenon observed in SLMs during CoT training. iii) The methodology includes supervised fine-tuning (SFT) with varying amounts of long CoT data and reinforcement learning (RL) across models from the Qwen, LLaMA, and Gemma families, along with analysis of reflection behavior and cumulative error. iv) The primary result is the empirical discovery that SLMs trained on only 8k long CoT examples can lose up to 75% of their original performance before fine-tuning. v) This highlights the need for scaled supervision during SFT or potentially RL to enhance the model, as smaller models may be overwhelmed and produce less accurate reasoning. |
| BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation (Read more on arXiv or HuggingFace) |
Xilin Chen, Ruiping Wang, Chuyan Xiong, Hongyu Wang |
BitVLA introduces a 1-bit Vision-Language-Action model for robotics manipulation with ternary parameters. The research aims to reduce the memory footprint of VLA models for deployment on resource-constrained robotic systems. The methodology involves quantizing a full-precision vision encoder to 1.58-bit using distillation-aware training with a full-precision teacher model. BitVLA achieves a comparable performance to OpenVLA-OFT (4-bit quantized) on the LIBERO benchmark while only consuming 29.8% of the memory. BitVLA provides AI practitioners with a cost-effective, high-performance solution for robotics manipulation suitable for memory-constrained edge devices by substantially reducing model size. |
| Pre-trained Large Language Models Learn Hidden Markov Models In-context (Read more on arXiv or HuggingFace) |
Jennifer J. Sun, Yahya Satter, Zhaolin Gao, sarahdean, DaiYijia |
Pre-trained Large Language Models (LLMs) can effectively model data generated by Hidden Markov Models (HMMs) via in-context learning. The research investigates whether LLMs can learn and predict HMM-generated sequences in-context, how HMM properties affect ICL performance, and whether these findings translate to real-world datasets. The methodology involves controlled experiments on synthetic HMMs, varying parameters like state/observation space, mixing rate, entropy, and applying LLMs to real-world animal decision-making tasks. LLMs achieve predictive accuracy approaching the theoretical optimum on synthetic HMMs, and ICL achieves competitive performance with domain-specific models on animal decision-making tasks, and LLM in-context learning achieves an average prediction accuracy of 86.2% on the IBL mice dataset. ICL offers a data-efficient and accessible approach for next-observation prediction, particularly valuable when rapid insights are needed or when data for training bespoke models is scarce, and the practical guidelines provided are useful to researchers who wish to utilize LLMs as powerful, efficient statistical tools in complex scientific data analysis. |
| The Illusion of Thinking: Understanding the Strengths and Limitations of |
|
|
| Reasoning Models via the Lens of Problem Complexity (Read more on arXiv or HuggingFace) |
Samy Bengio, Maxwell Horton, Keivan Alizadeh, Iman Mirzadeh, parshinsh |
i) The paper analyzes Large Reasoning Models (LRMs) using controllable puzzle environments to assess their reasoning capabilities beyond final answer accuracy. ii) The research investigates how LRMs perform and scale with increasing problem complexity, focusing on the structure and quality of reasoning traces. iii) The methodology involves evaluating LRMs and standard LLMs on algorithmic puzzles, systematically manipulating complexity and analyzing both final answers and intermediate reasoning steps using puzzle simulators. iv) The primary results demonstrate that LRMs face complete accuracy collapse beyond certain complexity thresholds, and their reasoning effort, measured in tokens, initially increases but then declines with increasing complexity, eventually collapsing to near-zero accuracy. v) This indicates that AI practitioners should consider the scaling limitations of current LRMs’ reasoning capabilities relative to problem complexity, and their limited ability to perform exact computations, potentially requiring new designs for reasoning systems. |
| CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Yang Yu, Jijie Li, Yonghua, ldwang, ZacLiu |
CCI4.0 is introduced as a large-scale bilingual pretraining dataset to improve reasoning in LLMs. The research aims to enhance LLMs’ reasoning through a curated dataset and diverse reasoning templates. The methodology involves a two-stage deduplication process, multi-classifier quality scoring, and domain-aware fluency filtering, resulting in a 35TB dataset and 4.5 billion CoT templates. Evaluations show pretraining on CCI4.0 improves performance on benchmarks like MMLU and ARC-Challenge, with CCI4.0 achieving a 33.09 average score across benchmarks versus 32.92 for Nemotron-CC-HQ. AI practitioners can leverage CCI4.0 for pretraining LLMs, yielding enhanced reasoning capabilities, particularly in math and code-related tasks. |
| Well Begun is Half Done: Low-resource Preference Alignment by |
|
|
| Weak-to-Strong Decoding (Read more on arXiv or HuggingFace) |
Tianyu Liu, Yuxuan Fan, Wen Luo, SylvainWei, songff |
i) The paper introduces Weak-to-Strong Decoding (WSD), a novel framework for low-resource preference alignment in Large Language Models (LLMs). ii) The primary objective is to enhance the alignment ability of base LLMs with human preferences using a small aligned draft model to guide the initial decoding stages. iii) WSD employs a small, fine-tuned model to generate an aligned prefix, followed by the base LLM continuing the response generation, governed by an auto-switch mechanism based on confidence scores. iv) Experiments show WSD improves base LLMs’ performance on preference alignment benchmarks, achieving a win-rate of 98.19% on HH-RLHF with Llama-3-70B, while also mitigating alignment tax on downstream tasks like GSM8K and HumanEval. v) WSD provides AI practitioners with a computationally efficient method to improve LLM alignment without significant performance degradation on other tasks, especially useful when fine-tuning resources are limited. |
| GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection |
|
|
| Behavior (Read more on arXiv or HuggingFace) |
Lewei Lu, Jiaheng Yu, Bo Wang, Shengnan Ma, Penghao Wu |
i) This paper introduces GUI-Reflection, a framework that enhances multimodal GUI models with self-reflection and error correction capabilities. ii) The research aims to equip GUI agents with self-reflection and correction capabilities for more robust and adaptable GUI automation. iii) The key methodology involves three training stages: GUI-specific pre-training using GUI-Reflection Task Suite, offline supervised fine-tuning (SFT) with automatically constructed reflection data, and online reflection tuning in a mobile GUI environment. iv) A success rate of 34.72% on level-2 tasks was achieved when combining reflection data during offline SFT with reflection tuning online, compared to 14.58% for a baseline model trained without reflection data in offline SFT and using only filtered behavior cloning. v) GUI-Reflection provides AI practitioners with tools and methodologies to improve the robustness and adaptability of GUI automation models by explicitly training for error recognition and recovery, potentially reducing reliance on nearly error-free training data. |
| ConfQA: Answer Only If You Are Confident (Read more on arXiv or HuggingFace) |
Alicia Sun, Vera Yan, Kai Sun, Yifan Ethan Xu, MaggieHuang |
i) The paper introduces ConfQA, a fine-tuning strategy designed to reduce hallucination in large language models (LLMs). ii) The main research objective is to develop a method to enable LLMs to refrain from generating factual statements when confidence is low, instead opting to state “I am unsure.” iii) The methodology involves fine-tuning LLMs using a dampening prompt “answer only if you are confident” and training data consisting of simple factual statements derived from knowledge graphs, specifically attribute values. iv) The primary result is a reduction in hallucination rate from 20-40% to under 5% across multiple factuality benchmarks after applying ConfQA. v) The principal implication for AI practitioners is that ConfQA provides a practical approach for improving the reliability of LLMs in knowledge-intensive tasks by reducing hallucination, which allows for seamless switching between parameterized and symbolic knowledge, with an accuracy gain to beyond 95%. |
| Vision Transformers Don’t Need Trained Registers (Read more on arXiv or HuggingFace) |
Yossi Gandelsman, Alexei Efros, Amil Dravid, Nick Jiang |
i) The paper introduces a training-free method to improve Vision Transformers by addressing high-norm token artifacts. ii) The main objective is to develop a training-free approach that mitigates noisy attention maps in Vision Transformers without retraining models from scratch. iii) The methodology involves identifying register neurons responsible for creating high-norm activations on outlier tokens and redirecting these activations to an untrained appended token. iv) The study demonstrates a 20-point improvement in correct localization for unsupervised object discovery using the proposed test-time register approach. v) AI practitioners can use this training-free method to enhance existing pre-trained Vision Transformer models, improving performance on downstream visual tasks and interpretability, without incurring the cost of retraining. |
| Dreamland: Controllable World Creation with Simulator and Generative |
|
|
| Models (Read more on arXiv or HuggingFace) |
Honglin He, Weizhen Wang, Leon Liu, Ziyang Leng, Sicheng Mo |
Dreamland presents a hybrid world generation framework combining simulators and generative models for controllable scene creation. The research addresses the lack of element-wise controllability in existing video generative models for dynamic world creation. It uses a layered world abstraction (LWA) to bridge a physics-based simulator and a pretrained generative model. Dreamland outperforms existing baselines with 50.8% improved image quality and 17.9% stronger controllability. This hybrid pipeline offers AI practitioners enhanced capabilities for synthetic data generation with simulator-level control, enhancing embodied agent training. The paper constructs a dataset called D3Sim (Diverse Driving Scenario in Real WorlD and Simulation) for training and benchmarking hybrid generation pipelines. It’s unclear what specific kind of embodied AI agents the technique would best serve or what types of pre-trained generative models are supported. |
| Image Reconstruction as a Tool for Feature Analysis (Read more on arXiv or HuggingFace) |
Andrey Kuznetsov, Elizaveta Goncharova, Dmitrii Tarasov, combat-helicopter |
Vision encoder interpretability is analyzed via image reconstruction quality. The research aims to interpret vision features through image reconstruction by comparing encoders trained with differing objectives. The methodology involves reconstructing images from latent feature tensors and analyzing the effects of feature space manipulations on the reconstructed images. The study found that SigLIP2 produces significantly higher-fidelity reconstructions than SigLIP, and orthogonal rotations in the embedding space yield interpretable color transformations. This approach enables AI practitioners to assess and compare the informativeness of different vision encoder feature representations, informing model selection and feature space manipulation for downstream applications. There is no quantifiable measure of reconstruction quality. |
| Cartridges: Lightweight and general-purpose long context representations |
|
|
| via self-study (Read more on arXiv or HuggingFace) |
Dylan Zinsley, Neel Guha, Simran Arora, Ryan Ehrlich, sabrieyuboglu |
i) The paper introduces CARTRIDGES, a method for creating lightweight KV-cache representations of long-context corpora for efficient inference. ii) The research aims to develop a memory-efficient alternative to in-context learning (ICL) that maintains performance on long-context tasks. iii) The methodology involves training smaller KV caches offline using a self-study approach that generates synthetic conversations via context distillation. iv) CARTRIDGES trained with self-study match ICL performance while using 38.6× less memory and enabling 26.4× higher throughput, and extends effective context length from 128k to 484k tokens on MTOB. v) CARTRIDGES provide AI practitioners with a composable and efficient mechanism for managing and serving long-context applications, reducing memory footprint and improving throughput. |
| Bootstrapping World Models from Dynamics Models in Multimodal Foundation |
|
|
| Models (Read more on arXiv or HuggingFace) |
Shay B. Cohen, Anna Korhonen, Yftah Ziser, ducdauge, yfqiu-nlp |
i) The paper introduces techniques to improve world models in vision-language models (VLMs) by leveraging dynamics models. ii) The main objective is to investigate whether vision-and-language foundation models contain a realistic world model and a dynamics model, and to improve world models through dynamics models. iii) The methodology involves fine-tuning VLMs to acquire a dynamics model and using it to bootstrap a world model through weak supervision with synthetic data and inference-time verification. iv) The best model achieves competitive performance in action-centric image editing on AURORA-BENCH, improving on state-of-the-art models by 15% on real-world subsets according to GPT4o-as-judge. v) The implication is that dynamics models can enhance world model capabilities in VLMs, offering a promising approach for AI practitioners working on embodied agents and multimodal reasoning. |
| PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal |
|
|
| Interaction and Enhancement (Read more on arXiv or HuggingFace) |
Yuan Zhou, Jiangning Zhang, Zhengguang Zhou, Zhentao Yu, Teng Hu |
PolyVivid is introduced as a multi-subject video customization framework enabling flexible and identity-consistent generation. The research aims to improve fine-grained video generation controllability, particularly for multi-subject customization with consistent identity and interaction. A VLLM-based text-image fusion module, a 3D-RoPE-based enhancement module, and an attention-inherited identity injection module are employed. Experiments demonstrate that PolyVivid achieves superior performance in identity fidelity, video realism, and subject alignment, and it achieves superior similarity scores for both face and object identity (Face-sim and DINO-sim) versus other methods. PolyVivid offers AI practitioners a method for generating high-fidelity, controllable videos with multiple customized subjects, potentially improving video content creation pipelines. |
| Learning What Reinforcement Learning Can’t: Interleaved Online |
|
|
| Fine-Tuning for Hardest Questions (Read more on arXiv or HuggingFace) |
Xiaochen Ma, Lexiang Tang, Meiyi Qiang, Hao Liang, RoadQAQ |
i) This paper introduces ReLIFT, a novel training approach combining reinforcement learning (RL) and supervised fine-tuning (SFT) to enhance large language model (LLM) reasoning. ii) The main research objective is to overcome the limitations of RL in inducing capabilities exceeding the base model by integrating SFT for knowledge acquisition. iii) ReLIFT employs an interleaved training process where RL is primarily used, with SFT triggered online using high-quality solutions collected for the most challenging questions encountered during RL. iv) The primary result is an average improvement of over +5.2 points across five competition-level benchmarks and one out-of-distribution benchmark compared to zero-RL models. v) The principal implication is that ReLIFT provides a scalable method for AI practitioners to improve LLM reasoning by adaptively interleaving RL and SFT, leveraging targeted fine-tuning to address the limitations of standard RL approaches. |
| Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path |
|
|
| Lengths in LLMs (Read more on arXiv or HuggingFace) |
Lior Wolf, Itamar Zimerman, royeis |
i) This paper introduces a method for monitoring and controlling the reasoning path length in Large Language Models (LLMs). ii) The main research question is how to understand and manipulate the mechanisms by which LLMs regulate the length of their reasoning processes during explicit thought. iii) The methodology involves analyzing hidden representations to extract “progress vectors” that indicate the model’s position within the reasoning phase, followed by interventions that manipulate these vectors. iv) The primary result is that intervening on these progress vectors can reduce unnecessary reasoning steps, improving answer accuracy and inference latency; for example, it shows that our method increases the number of correct answers on Math-500 by at least 80% in the 512 token-budget regime and boosts correct responses on GSM-8K by an average of 80% across the 256 and 512 token settings. v) The principal implication for AI practitioners is a technique for improving the efficiency and effectiveness of LLMs by mitigating overthinking through controlled manipulation of internal progress encodings, providing better test-time scaling. |
| GeometryZero: Improving Geometry Solving for LLM with Group Contrastive |
|
|
| Policy Optimization (Read more on arXiv or HuggingFace) |
Qipeng Guo, Zimian Peng, Dianyi Wang, Yibin Wang, LibraTree |
i) GeometryZero presents a novel reinforcement learning framework, Group Contrastive Policy Optimization (GCPO), to improve geometry problem-solving capabilities of LLMs. ii) The research aims to address the limitations of existing GRPO-based methods in geometry reasoning due to their reliance on unconditional rewards for auxiliary construction. iii) The methodology involves introducing Group Contrastive Masking and Length Reward to adaptively provide positive or negative reward signals for auxiliary construction based on contextual utility. iv) Empirical evaluations on Geometry3K and MathVista demonstrate that GeometryZero models consistently outperform baselines, achieving an average improvement of 4.29% across all benchmarks. v) GCPO provides AI practitioners with a method for training moderate-sized LLMs to judiciously employ auxiliary constructions in geometry reasoning, offering an alternative to relying on colossal LLMs. |
| Robust Preference Optimization via Dynamic Target Margins (Read more on arXiv or HuggingFace) |
Xingyu Lu, Zhibo Zhu, Jiancan Wu, Junkang Wu, Sunshine279 |
i) The paper introduces γ-PO, a direct preference optimization method utilizing dynamic target margins to enhance the robustness of aligning large language models. ii) The primary objective is to mitigate performance degradation in DPO due to noisy preference data by dynamically adjusting reward margins. iii) The methodology involves instance-specific margin calibration, prioritizing high-confidence pairs while suppressing noise from ambiguous pairs. iv) Experiments across AlpacaEval2 and Arena-Hard show γ-PO achieves an average 4.4% improvement over baselines. v) γ-PO offers a plug-and-play solution for AI practitioners to improve LLM alignment with minimal code changes and computational overhead. |
| Play to Generalize: Learning to Reason Through Game Play (Read more on arXiv or HuggingFace) |
Junfei Xiao, Alan Yuille, Shiyi Lan, Yinsong Ma, Yunfei Xie |
i) The paper introduces Visual Game Learning (ViGaL), a novel post-training paradigm leveraging gameplay to enhance multimodal reasoning in Large Language Models (MLLMs). ii) The research investigates whether reinforcement learning (RL) through arcade-like games can improve the out-of-domain generalization capabilities of MLLMs on multimodal reasoning tasks. iii) The methodology involves post-training a 7B-parameter MLLM using rule-based RL on games like Snake and Rotation, employing custom game environments and reward designs. iv) Results demonstrate that ViGaL achieves enhanced out-of-domain performance, with ViGaL (RL on game) exhibiting a higher average accuracy increase than MM-Eureka (RL on math) across three multimodal math benchmarks, increasing MathVerse accuracy by 0.5%. v) ViGaL offers AI practitioners a controllable and scalable pre-training approach using synthetic games to unlock generalizable multimodal reasoning abilities in MLLMs, potentially reducing reliance on large-scale domain-specific data. |
| MegaHan97K: A Large-Scale Dataset for Mega-Category Chinese Character |
|
|
| Recognition with over 97K Categories (Read more on arXiv or HuggingFace) |
Yixin Zhao, Peirong Zhang, lianwen, shiyx1, ZZXF |
i) The paper introduces MegaHan97K, a new large-scale dataset for mega-category Chinese character recognition. ii) The objective is to address the absence of comprehensive datasets for recognizing the vast number of Chinese characters, particularly infrequent and archaic ones. iii) The methodology involves creating a dataset with three subsets: handwritten, historical, and synthetic, covering 97,455 categories. iv) The MegaHan97K dataset includes Chinese characters of 97,455 categories, which is at least six times more than existing datasets, and a average improvement of 22.43% compared to without the synthetic subset. v) The MegaHan97K dataset provides AI practitioners with a new benchmark for evaluating and improving Chinese character recognition models, particularly for cultural heritage preservation and digital applications, however it increased storage demands under the mega-category setting. |
| Improving large language models with concept-aware fine-tuning (Read more on arXiv or HuggingFace) |
Dacheng Tao, Jiaxing Huang, Xikun Zhang, michaelchenkj |
i) The paper introduces Concept-Aware Fine-Tuning (CAFT), a multi-token training method for improving conceptual understanding in Large Language Models (LLMs). ii) The primary objective is to address the limitation of next-token prediction in LLMs, which hinders their ability to form coherent, high-level concepts. iii) CAFT trains auxiliary heads to predict multiple future tokens simultaneously and incorporates a modified cross-entropy loss function, facilitating concept-aware learning during fine-tuning. iv) Experiments demonstrate that CAFT improves performance across diverse tasks, with HumanEval coding accuracy increasing from 40.9% (LoRA Fine-tuning) to 45.1% when using CAFT. v) CAFT democratizes multi-token prediction for broader use by AI practitioners enabling them to enhance the conceptual understanding and performance of LLMs in downstream applications. |
| Evaluating LLMs Robustness in Less Resourced Languages with Proxy Models (Read more on arXiv or HuggingFace) |
Karolina Seweryn, llmAttack, mchraba |
Evaluating robustness of LLMs in low-resource languages is critical. This work aims to assess the robustness of LLMs to perturbations in less-resourced languages, specifically Polish. The study employed a framework for generating perturbed datasets using proxy models and attribution methods to identify important words for targeted attacks. Experiments with Polish datasets showed that LLMs are susceptible to character and word-level attacks with a SHAP attribution success rate of 37% for Diacritical perturbations on RoBERTa and that these attacks drastically alter model predictions. These findings suggest potential vulnerabilities in LLMs internal safety mechanisms, underlining the need for AI practitioners to prioritize robustness evaluations, especially when deploying multilingual models in lower-resourced language contexts. |
| Proactive Assistant Dialogue Generation from Streaming Egocentric Videos (Read more on arXiv or HuggingFace) |
Anuj Kumar, Andrea Madotto, Zhaojiang Lin, Xin Luna Dong, 594zyc |
i) This paper presents a framework for proactive assistant dialogue generation from streaming egocentric videos, including a dataset, evaluation metrics, and an end-to-end model. ii) The primary research objective is to develop an AI system capable of generating prompt, appropriate, and helpful guidance from streaming egocentric videos in real-time. iii) The methodology involves synthesizing dialogues from annotated egocentric videos using large language models (LLMs) to create a dataset (PROASSIST), developing automatic evaluation metrics, and building an end-to-end multimodal LLM (MLLM). iv) Results include a synthetic dialogue dataset of 30,135 dialogues across 479 hours of video, and an MLLM with negative frame sub-sampling improving F1 scores in response timing decisions, by over 8 percentage points. v) For AI practitioners, this work offers a large-scale dataset and evaluation framework for training and benchmarking proactive AI assistants capable of guiding users through tasks using real-time video inputs. |
| EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and |
|
|
| Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructions (Read more on arXiv or HuggingFace) |
Chong Teng, Fei Li, Xin Zhang, Xiaofeng Mao, Xiaorui Wu |
EVOREFUSE introduces an evolutionary prompt optimization algorithm to generate diverse, high-confidence pseudo-malicious instructions for evaluating and mitigating LLM over-refusals. The research aims to develop a method for automatically generating diverse refusal-inducing instructions to address limitations in existing instruction curation techniques. The methodology uses an evolutionary algorithm optimizing an Evidence Lower Bound (ELBO) objective, incorporating mutation and recombination operations guided by salient cues identified in over-refusal datasets. The study demonstrates that EVOREFUSE achieves a 140.41% higher average refusal triggering rate across 9 LLMs compared to existing benchmarks and that fine-tuning LLAMA3.1-8B-INSTRUCT with EVOREFUSE-ALIGN reduces over-refusals by 14.31% using SFT and 40.04% using DPO. This highlights a novel strategy for hardening LLMs against over-refusals by generating targeted training data, improving their helpfulness without compromising safety. |
Papers for 2025-06-09
| Title |
Authors |
Summary |
| Will It Still Be True Tomorrow? Multilingual Evergreen Question |
|
|
| Classification to Improve Trustworthy QA (Read more on arXiv or HuggingFace) |
VityaVitalich, nakrayko, VirVen, zlatamaria, memyprokotow |
i) This paper introduces EverGreenQA, a multilingual dataset for evergreen question classification to improve trustworthy question answering. ii) The primary objective is to assess whether large language models (LLMs) encode question temporality, either explicitly or implicitly, and to improve self-knowledge estimation in QA systems. iii) The methodology involves constructing a new multilingual QA dataset, EverGreenQA, benchmarking 12 LLMs, and training EG-E5, a lightweight multilingual classifier for identifying evergreen questions. iv) EG-E5 achieves SoTA performance on evergreen question classification, reaching a weighted F1 score of 0.906, and improves self-knowledge estimation in 16 out of 18 settings. v) The results demonstrate that incorporating evergreen question classification improves self-knowledge estimation, dataset curation, and explainability for GPT-40’s retrieval behavior, which informs AI practitioners of the importance of considering question temporality in QA system design and evaluation. |
| FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal |
|
|
| Contextual Fusion (Read more on arXiv or HuggingFace) |
Owen Lee, Liyan Zhao, Zheshu Chen, Shunian Chen, SatsukiVie |
The paper introduces FusionAudio-1.2M, a dataset and pipeline for fine-grained audio captioning using multimodal context. The research aims to improve caption detail and contextual accuracy by leveraging specialized pretrained models for extracting diverse contextual cues and a large language model (LLM) for synthesis. The methodology involves a two-stage automated pipeline with specialized models for speech, music, general sounds, and visual information extraction. FusionAudio-1.2M comprises 1.2 million detailed captions and 6 million QA pairs. Fine-tuning a CLAP-based audio encoder with FusionAudio shows enhanced audio-text alignment, indicating the dataset’s potential for improving audio understanding and contextual caption generation; this is impactful for engineers needing higher-quality audio-text datasets and models. |
| Is Extending Modality The Right Path Towards Omni-Modality? (Read more on arXiv or HuggingFace) |
Yu Su, Muhao Chen, Kai Zhang, DarthZhu |
i) This paper analyzes the impact of extending modality on Large Language Models (LLMs), evaluating its effect on core language abilities and exploring techniques for omni-modality. ii) The research questions the trade-offs between extending modality in LLMs and preserving core language abilities, investigating whether model merging and omni-modality fine-tuning can effectively achieve true omni-modality. iii) The study involves fine-tuning LLMs with different modalities (image, video, audio), employing model merging techniques (average and weighted average), and evaluating performance across a range of textual and multimodal tasks. iv) Results indicate a performance decline in instruction following across all modality-extended models compared to the original base model, suggesting a trade-off; weighted model merging, however, achieves the best performance across both textual and multimodal tasks. v) AI practitioners should carefully consider the potential degradation of core language abilities when extending LLMs with new modalities and explore weighted average model merging as a promising strategy to maintain multimodal capabilities, although it still falls short of modality-specific models. |
| Audio-Aware Large Language Models as Judges for Speaking Styles (Read more on arXiv or HuggingFace) |
Linjie Li, Kevin Lin, Chung-Ching Lin, xiaofei-wang, dcml0714 |
i) The paper investigates the use of audio-aware large language models (ALLMs) as automatic judges for evaluating the speaking styles of spoken language models (SLMs). ii) The research objective is to assess whether ALLMs can effectively evaluate the style adherence and realism of speeches generated by SLMs in voice style instruction following and role-playing tasks. iii) The methodology involves using GPT-4o-audio and Gemini-2.5-Pro as ALLM judges to evaluate speech generated by GPT-4o-audio, GPT-4o-mini-audio, Step-Audio, and Qwen-2.5-Omni on two tasks: voice style instruction following and role-playing. iv) The primary result shows that Gemini-human judge agreement in evaluating speaking styles can be comparable to human-human agreement, with a Pearson’s r of 0.640 between Gemini and human evaluators. v) The principal implication for AI practitioners is that ALLMs, specifically Gemini-2.5-Pro, can serve as a viable automatic evaluation metric for assessing the speaking style quality of SLMs, reducing the reliance on costly and variable human evaluations. |
| Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs (Read more on arXiv or HuggingFace) |
sambaran, abhi1nandy2, ananthmuppidi |
i) This paper introduces Input Dependent Soft Prompting with a self-Attention Mechanism (ID-SPAM) for parameter-efficient fine-tuning of LLMs. ii) The research aims to improve LLM performance on domain-specific tasks while minimizing the number of trainable parameters through input-dependent soft prompts. iii) The methodology involves generating soft prompts based on input tokens and attending to these tokens with varying importance using a self-attention mechanism, prepended to a transformer layer. iv) Experimental results on the GLUE benchmark show that ID-SPAM outperforms parameter-efficient soft prompt baselines on 4 out of 6 tasks, achieving an average performance improvement and demonstrates improved zero-shot domain transfer capability. v) ID-SPAM offers AI practitioners a method for efficiently adapting pre-trained LLMs to downstream tasks with reduced computational cost and improved generalization, particularly in scenarios with limited data or domain shift, and ID-SPAM performs better than LoRA in 5/6 tasks when using ROBERTa-BASE. |
| STARFlow: Scaling Latent Normalizing Flows for High-resolution Image |
|
|
| Synthesis (Read more on arXiv or HuggingFace) |
Yuyang Wang, Huangjie Zheng, David Berthelot, Tianrong Chen, Jiatao Gu |
STARFlow is presented as a scalable generative model using normalizing flows for high-resolution image synthesis. The research aims to develop a more scalable normalizing flow model for image synthesis that can compete with diffusion models. It introduces Transformer Autoregressive Flow (TARFlow) blocks and a deep-shallow architecture trained in the latent space of pretrained autoencoders and presents a guidance algorithm. The model achieves competitive sample quality in both class- and text-conditional image generation, with an FID of 2.40 on ImageNet-256. STARFlow demonstrates that normalizing flows can achieve competitive results at scale for image generation. It enables AI practitioners to utilize normalizing flows for high-resolution image synthesis tasks, providing an alternative to diffusion models. |
| PartCrafter: Structured 3D Mesh Generation via Compositional Latent |
|
|
| Diffusion Transformers (Read more on arXiv or HuggingFace) |
Yiqiang Feng, Honglei Yan, Panwang Pan, Yuchen Lin, chenguolin |
PartCrafter presents a structured 3D generative model for compositional mesh generation from single RGB images. The research aims to generate semantically meaningful, geometrically distinct 3D meshes without requiring segmented image inputs. The methodology involves a compositional latent diffusion transformer (DiT) architecture, incorporating a compositional latent space and a hierarchical attention mechanism. Experiments demonstrate that PartCrafter outperforms existing methods in generating decomposable 3D meshes, achieving higher generation quality and efficiency, with a reduction in run time from 80s to 34s in some experiments. PartCrafter provides AI practitioners with a part-aware generative prior for improved 3D understanding and synthesis, enabling more effective 3D content creation pipelines. |
| MORSE-500: A Programmatically Controllable Video Benchmark to |
|
|
| Stress-Test Multimodal Reasoning (Read more on arXiv or HuggingFace) |
Hyunwoo Jae, Ankit Nakhawa, Anirudh Satheesh, Andrew Wang, Zikui |
i) MORSE-500 is introduced as a new video benchmark for multimodal reasoning. ii) The research aims to address the limitations of current multimodal benchmarks by evaluating diverse reasoning skills within temporal contexts. iii) The benchmark utilizes programmatically generated videos and curated real footage across six reasoning categories: mathematical, abstract, spatial, temporal, physical, and planning. iv) Initial experiments reveal performance gaps in state-of-the-art VLMs, such as OpenAI-03 and Gemini 2.5 Pro, particularly in abstract reasoning and planning tasks, with overall model accuracy averaging below 25% compared to 55.4% for human performance. v) The controllable generation pipeline allows for stress-testing next-generation models by creating arbitrarily challenging new instances, offering a forward-looking evaluation tool for AI practitioners. |
| Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence |
|
|
| with Egocentric-Exocentric Vision (Read more on arXiv or HuggingFace) |
Baoqi Pei, Lidong Lu, Yifei Huang, Yuping He, cg1177 |
This paper surveys research on video understanding using both egocentric (first-person) and exocentric (third-person) views. The main objective is to provide a comprehensive review of approaches that integrate these perspectives for enhanced video analysis. The methodology involves categorizing and reviewing recent advancements into three research directions: leveraging egocentric data to enhance exocentric understanding, utilizing exocentric data to improve egocentric analysis, and joint learning frameworks. The survey analyzes several tasks and datasets and finds a growing body of work exploring cross-view learning, with the number of citations to egocentric-exocentric related papers increasing from 14 in 2015 to 1642 in 2024. AI practitioners can leverage insights from this survey to develop advanced video understanding systems that combine complementary information from egocentric and exocentric views to improve performance. |
| 3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World |
|
|
| Model (Read more on arXiv or HuggingFace) |
Quanxi Wu, Yubo Dong, Siyuan Zhou, Peihao Chen, Hoyard |
3DFlowAction presents a novel approach to robot manipulation learning using 3D optical flow as a unified action representation. The research investigates learning a cross-embodiment manipulation policy transferable across different robotic systems without hardware-specific training. The method uses a 3D flow world model trained on a new dataset, ManiFlow-110k, to predict object motion, combined with a flow-guided rendering mechanism and GPT-4o for closed-loop planning. Experiments demonstrated a task success rate of 70.0% across different manipulation tasks, indicating strong generalization capabilities. The work provides AI practitioners with a data-efficient method for developing robot manipulation policies that can adapt to new robots and environments without extensive retraining. |
| Prefix Grouper: Efficient GRPO Training through Shared-Prefix Forward (Read more on arXiv or HuggingFace) |
Junxian Cai, Longteng Guo, Yepeng Tang, Tongtian Yue, Zikang Liu |
i) The paper introduces Prefix Grouper, an algorithm to improve the efficiency of Group Relative Policy Optimization (GRPO) by eliminating redundant prefix computation. ii) The research aims to reduce the computational overhead associated with encoding long shared prefixes in GRPO training. iii) The methodology involves restructuring self-attention to encode shared prefixes only once via a Shared-Prefix Forward strategy while maintaining differentiability. iv) The experiments show Prefix Grouper achieves equivalent performance to standard GRPO while reducing computational cost, particularly in long-prefix scenarios; theoretical computation analysis show Prefix Grouper reduces FLOPs to 1/G in long-prefix situations. v) Prefix Grouper allows AI practitioners to scale GRPO to larger group sizes and more complex tasks within the same computational budget by reducing redundant computations. |
| CodeContests+: High-Quality Test Case Generation for Competitive |
|
|
| Programming (Read more on arXiv or HuggingFace) |
Kai Shen, Hongyan Li, Yang Sun, Siyao Liu, zhwang01 |
This paper introduces CodeContests+, an improved dataset for competitive programming via high-quality test case generation. The research aims to address the limitations of existing datasets by generating comprehensive and correct test cases for evaluating LLM reasoning. The methodology involves an LLM-based Generator-Validator (G-V) agent system for test case construction and validation, ensuring constraint satisfaction. Evaluation using 1.72 million submissions showed CodeContests+ achieves significantly higher evaluation accuracy, with nearly twice as many problems meeting TPR&TNR >= 0.9 compared to CodeContests. The implication is that CodeContests+ provides a higher-quality benchmark dataset that is advantageous for training reasoning models via reinforcement learning. |
| Splatting Physical Scenes: End-to-End Real-to-Sim from Imperfect Robot |
|
|
| Data (Read more on arXiv or HuggingFace) |
Zhibin Li, Tom Erez, Steven Bohez, Mauro Comi, Ben Moran |
SplatMesh: end-to-end real-to-sim framework for creating physical scenes from imperfect robot data. The research aims to create accurate physical simulations directly from real-world robot motion despite data imperfections. The methodology involves a hybrid scene representation combining 3D Gaussian Splatting with explicit object meshes suitable for MuJoCo physics simulation and an end-to-end optimization pipeline using differentiable rendering and physics. The framework achieves high-fidelity object mesh reconstruction, generates photorealistic novel views, and performs annotation-free robot pose calibration; the full framework obtains Chamfer Distance of 0.073 mm² for object reconstruction on the Simulated YCB dataset. The developed real-to-sim pipeline offers AI practitioners a practical approach for creating robust and scalable robotic simulations from real-world data, specifically from low cost hardware, enabling more effective robot learning and planning. |
| HASHIRU: Hierarchical Agent System for Hybrid Intelligent Resource |
|
|
| Utilization (Read more on arXiv or HuggingFace) |
Harshil Patel, helloparthshah, guineapig |
HASHIRU is a novel MAS framework for enhanced flexibility, resource efficiency, and adaptability in AI systems. This paper addresses how to improve resource utilization and adaptability in multi-agent systems by incorporating hierarchical control, hybrid intelligence, and autonomous tool creation. The framework uses a hierarchical structure with a “CEO” agent dynamically managing specialized “employee” agents based on task needs and resource constraints, prioritizing smaller, local LLMs while integrating external APIs and larger models when justified. Evaluations on tasks like academic paper review, safety assessments, and complex reasoning demonstrate HASHIRU’s capabilities, with HASHIRU outperforming Gemini 2.0 Flash on GSM8K (96% vs. 61%). The principal implication for AI practitioners is a promising approach for more robust, efficient, and adaptable MAS through dynamic hierarchical control, resource-aware hybrid intelligence, and autonomous functional extension. |
| Truth in the Few: High-Value Data Selection for Efficient Multi-Modal |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Chong Peng, Hao Yang, Lei Wang, Kaiyuan Deng, Shenshen Li |
i) This paper introduces Reasoning Activation Potential (RAP), a novel data selection paradigm for efficient multi-modal reasoning in MLLMs. ii) The research addresses the question of whether smaller, high-value datasets can match or outperform full corpora for multi-modal reasoning in MLLMs, aiming to reduce data redundancy and computational costs. iii) The methodology involves a Causal Discrepancy Estimator (CDE) and an Attention Confidence Estimator (ACE) to identify cognitive samples and a Difficulty-aware Replacement Module (DRM) to ensure data complexity. iv) Experiments demonstrate superior performance using only 9.3% of the training data, reducing computational costs by over 43%. v) The principal implication for AI practitioners is the potential to significantly reduce training data requirements and computational costs for MLLMs by focusing on high-value cognitive samples identified through RAP, thereby enabling more efficient development and deployment of multi-modal reasoning systems. |
| GuideX: Guided Synthetic Data Generation for Zero-Shot Information |
|
|
| Extraction (Read more on arXiv or HuggingFace) |
Eneko Agirre, Iker García-Ferrero, OSainz, neildlf |
GUIDEX introduces a novel method for generating synthetic data to improve zero-shot information extraction (IE). The paper addresses the challenge of domain-specific adaptation in IE systems, which typically requires expert schema design and data annotation. It aims to improve out-of-domain generalization by automatically defining schemas, inferring guidelines, and generating synthetically labeled instances. The method uses a large language model (LLM) to identify key information, structure it into a JSON format, and generate annotation schemas and guidelines. Fine-tuning Llama 3.1 with GUIDEX achieves state-of-the-art results across seven zero-shot Named Entity Recognition (NER) benchmarks, with gains up to 7 F1 points over previous methods without human-labeled data. The principal implication for AI practitioners is a low-noise strategy for robust zero-shot IE across diverse domains, reducing the need for manual annotation and schema creation, but some areas, such as Miscellaneous labels, need improvement. |
Papers for 2025-06-06
| Title |
Authors |
Summary |
| RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language |
|
|
| Models for Robotics (Read more on arXiv or HuggingFace) |
Shanyu Rong, Yi Han, Cheng Chi, Jingkun An, Zhoues |
This paper introduces RoboRefer, a 3D-aware VLM for spatial referring with reasoning for embodied AI. The research aims to improve robots’ understanding of 3D scenes and their ability to follow spatially constrained instructions through both precise spatial understanding and multi-step reasoning. The methodology involves supervised fine-tuning (SFT) with a disentangled depth encoder and reinforcement fine-tuning (RFT) using metric-sensitive process reward functions. Experiments show SFT-trained RoboRefer achieves state-of-the-art spatial understanding on existing benchmarks, and RFT-trained RoboRefer surpasses Gemini-2.5-Pro by 17.4% in average accuracy on a newly introduced RefSpatial-Bench. RoboRefer facilitates controlling robots for complex tasks in real-world environments, enabling effective manipulation and navigation, and can be used by AI practitioners working on robotics applications. |
| SeedVR2: One-Step Video Restoration via Diffusion Adversarial |
|
|
| Post-Training (Read more on arXiv or HuggingFace) |
Meng Wei, Yuxi Ren, Zhijie Lin, Shanchuan Lin, Jianyi Wang |
i) SeedVR2, a one-step diffusion-based video restoration (VR) model, is introduced employing adversarial post-training. ii) The research aims to achieve high-resolution VR in a single step while enhancing visual quality and computational efficiency. iii) The methodology includes an adaptive window attention mechanism and adversarial post-training with feature matching loss. iv) Experiments demonstrate SeedVR2 achieves over 4x faster processing compared to existing diffusion-based VR methods, while maintaining comparable performance. v) The adaptive window attention mechanism improves robustness and reduces boundary artifacts for high-resolution video, offering AI practitioners an efficient means for VR tasks. |
| Video World Models with Long-term Spatial Memory (Read more on arXiv or HuggingFace) |
Ziwei Liu, Yinghao Xu, Ryan Po, Shuai Yang, Tong Wu |
i) This paper introduces a novel framework for enhancing long-term consistency in video world models using geometry-grounded spatial memory. ii) The research aims to address the issue of scene inconsistency in autoregressive video generation caused by limited temporal context windows. iii) The methodology involves integrating short-term working memory with long-term spatial memory (point cloud representation of static scenes) and episodic memory (historical keyframes), trained on a custom dataset generated from MiraData. iv) Evaluations show improved quality and 3D consistency compared to baselines, with user studies indicating superior performance in camera accuracy, static consistency, and dynamic plausibility; view recall consistency (PSNR) improves significantly from 12.16 to 19.10. v) The framework provides AI practitioners with an approach to improve the long-term coherence and realism of generated video environments by incorporating explicit 3D spatial reasoning and memory mechanisms. |
| ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow |
|
|
| Development (Read more on arXiv or HuggingFace) |
Zijiao Wu, Qingli Hu, Yiyu Wang, Xue Yang, imryanxu |
ComfyUI-Copilot, an LLM-powered plugin, assists users in AI workflow development within ComfyUI. The research addresses usability challenges in ComfyUI, aiming to automate workflow construction and provide intelligent recommendations. A hierarchical multi-agent framework with a central assistant agent and specialized worker agents was implemented, supported by curated ComfyUI knowledge bases. ComfyUI-Copilot achieved high recall rates (over 88.5%) for both node and workflow recommendations using GPT-40 and DeepSeek-V3. This tool lowers the entry barrier for ComfyUI and enhances workflow efficiency, providing AI practitioners with a means to automate workflow development and improve node/model selection, though the paper lacks information on the type of queries that failed and the reasons why. |
| Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights (Read more on arXiv or HuggingFace) |
Emilien Biré, Breno Baldas Skuk, Mathieu Andreux, tonywu71, hamza-hcompany |
Surfer-H, a cost-efficient web agent, is introduced alongside Holol, a collection of open-weight Vision-Language Models (VLMs) specialized for web navigation and information extraction. The research aims to develop and evaluate a cost-effective web agent leveraging specialized VLMs. Surfer-H integrates a policy, localizer, and validator, powered by Holol models trained on web content, synthetic examples, and agentic data. Surfer-H achieves 92.2% state-of-the-art performance on WebVoyager when powered by Holol. The open-sourcing of both the WebClick dataset and Holol model weights enables AI practitioners to build more efficient and accurate web agents, but it is unclear what datasets were used in the performance evaluation. |
| Diagonal Batching Unlocks Parallelism in Recurrent Memory Transformers |
|
|
| for Long Contexts (Read more on arXiv or HuggingFace) |
Ivan Oseledets, Yuri Kuratov, Gleb Kuzmin, Ivan Rodkin, Danil Sivtsov |
i) Diagonal Batching is introduced to unlock parallelism in Recurrent Memory Transformers (RMTs) for long contexts. ii) The research aims to mitigate the sequential execution bottleneck inherent in RMTs while preserving recurrence. iii) A scheduling scheme is developed that reorders computations into independent diagonals, enabling concurrent GPU execution without retraining. iv) Applying Diagonal Batching to a LLaMA-1B ARMT model achieves a 3.3x speedup compared to standard LLaMA-1B and a 1.8x speedup over sequential RMT on 131,072-token sequences. v) The technique’s ability to enhance the performance of RMTs offers AI practitioners a more efficient method for processing long-context inputs in real-world applications. |
| VideoREPA: Learning Physics for Video Generation through Relational |
|
|
| Alignment with Foundation Models (Read more on arXiv or HuggingFace) |
Xiangpeng Wan, Fanqing Meng, Shaofeng Zhang, Jiaqi Liao, aHapBean |
VideoREPA distills physics understanding from Video Foundation Models (VFMs) into text-to-video (T2V) diffusion models by aligning token-level relations. The research aims to improve the physical plausibility of generated videos by transferring physics knowledge from VFMs to VDMs. The methodology involves a Token Relation Distillation (TRD) loss for spatio-temporal alignment between VFM representations and diffusion transformer blocks. VideoREPA achieves a state-of-the-art Physical Commonsense (PC) score of 40.1 on VideoPhy, a 24.1% improvement over the CogVideoX baseline. For AI practitioners, VideoREPA provides a feature alignment framework that enhances the physical realism of generated videos, enabling more intuitive and physically plausible content generation, with potential applications in creating virtual environments and simulations. |
| Qwen3 Embedding: Advancing Text Embedding and Reranking Through |
|
|
| Foundation Models (Read more on arXiv or HuggingFace) |
Huan Lin, Mingxin Li, Yanzhao Zhang, izhx, thenlper |
Qwen3 Embedding presents a new text embedding and reranking series based on the Qwen3 foundation models. The research aims to improve text embedding and reranking capabilities by leveraging Qwen3 LLMs and a multi-stage training pipeline. This pipeline combines unsupervised pre-training with supervised fine-tuning, aided by data synthesized using Qwen3 models. The Qwen3-8B-Embedding model achieves a score of 70.58 on the MTEB Multilingual benchmark. AI practitioners can utilize the open-sourced Qwen3 Embedding models to achieve state-of-the-art performance in multilingual text understanding and retrieval tasks. |
| Aligning Latent Spaces with Flow Priors (Read more on arXiv or HuggingFace) |
Ping Luo, Ying Shan, Yixiao Ge, Yuying Ge, liyz |
i) This paper introduces a novel framework for aligning latent spaces to arbitrary target distributions using flow-based generative models as priors. ii) The main research question is whether a learnable latent space can be efficiently aligned to an arbitrary target distribution using a pre-trained flow model as a prior. iii) The methodology involves pretraining a flow model on the target features and using it to regularize the latent space through an alignment loss based on the flow matching objective. iv) Experiments show that minimizing the alignment loss approximates maximizing the log-likelihood of the latents under the target distribution and large-scale image generation on ImageNet achieves a FID score of 6.56 with textual embeddings. v) The principal implication for AI practitioners is a computationally efficient method for incorporating complex distributional priors into latent models, enhancing structured representation learning. |
| Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual |
|
|
| Simulations (Read more on arXiv or HuggingFace) |
Yinuo Yang, Zixian Ma, Mahtab Bigverdi, Linjie Li, kuvvi |
i) The paper introduces STARE, a new benchmark for evaluating multimodal models on spatial reasoning tasks requiring visual simulation. ii) The research objective is to assess the ability of multimodal large language models to perform complex visual reasoning through multi-step simulations. iii) The methodology involves curating a dataset of ~4K tasks spanning geometric transformations, integrated spatial reasoning, and real-world spatial reasoning, with variations in difficulty and evaluation setups. iv) Evaluations revealed that models excel at simpler 2D transformations but perform close to random chance on tasks requiring multi-step visual simulations; humans achieve near-perfect accuracy, speeding up on complex tasks with intermediate visual simulations. v) The inconsistent performance gains of models from visual simulations, with improvements on some tasks and declines in others, indicates that AI/ML practitioners should be aware that current models may not effectively leverage intermediate visual information for spatial cognition. |
| SparseMM: Head Sparsity Emerges from Visual Concept Responses in MLLMs (Read more on arXiv or HuggingFace) |
Jiwen Lu, Yongming Rao, Jiahui Wang, Zuyan |
i) The paper introduces SparseMM, a KV-Cache optimization strategy for accelerating Multimodal Large Language Models (MLLMs) by exploiting the discovered sparsity of visual-relevant attention heads. ii) The research aims to investigate how MLLMs process visual inputs by analyzing attention mechanisms and to develop a method for efficient MLLM inference. iii) The methodology involves analyzing attention mechanisms in MLLMs, identifying visual heads through targeted response analysis using OCR as an anchor task, and designing an asymmetric KV-Cache allocation strategy. iv) The primary results indicate that less than 5% of attention heads actively contribute to visual understanding, and SparseMM achieves 1.38x real-time acceleration and 52% memory reduction during generation while maintaining performance parity. v) The principal implication for AI practitioners is a computationally efficient method to accelerate MLLM inference by strategically allocating resources to visual-relevant attention heads, enabling better accuracy-efficiency trade-offs under limited computational budgets. |
| AV-Reasoner: Improving and Benchmarking Clue-Grounded Audio-Visual |
|
|
| Counting for MLLMs (Read more on arXiv or HuggingFace) |
Tong Lu, Yicheng Liu, Zhiqi Li, cg1177, lulidong |
i) The paper introduces CG-AV-Counting, a new clue-grounded audio-visual counting benchmark, and AV-Reasoner, a model trained to improve counting abilities in MLLMs. ii) The main objective is to address limitations in existing counting benchmarks and improve the counting capability of MLLMs. iii) The methodology involves manual annotation of a new benchmark with 1,027 multimodal questions and training AV-Reasoner using GRPO and curriculum learning, transferring counting ability from related tasks. iv) AV-Reasoner achieves a 44.00 accuracy on DVD-Counting, surpassing Video-R1 by 9.50 points, and demonstrates state-of-the-art results across multiple audio-visual understanding tasks. v) The findings suggest that reinforcement learning and clue-grounded benchmarks can improve multimodal reasoning for tasks requiring spatial-temporal grounding, implying improved MLLM performance in complex audio-visual environments, but a requirement for reasoning-answer consistency during training is necessary. |
| StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence |
|
|
| Training of LLMs (Read more on arXiv or HuggingFace) |
Xiao Li, Lei Zhao, Qijun Luo, Kullpar |
i) StreamBP is introduced as a memory-efficient exact backpropagation algorithm for long sequence training of LLMs. ii) The research aims to reduce the memory cost associated with storing activation values during backpropagation in LLMs, particularly for long sequence data. iii) The methodology involves a linear decomposition of the chain rule along the sequence dimension performed layer-wise. iv) StreamBP scales up the maximum sequence length of BP by 2.8 – 5.5x larger while using comparable or even less BP time compared to gradient checkpointing. v) Practitioners can use StreamBP to train LLMs on significantly longer sequences with similar or reduced computational cost, facilitating improved performance on complex tasks like long-chain reasoning, and the technique can be directly transferred to batch size scaling for accelerating training. |
| MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical |
|
|
| Chain-of-Thought Reasoning (Read more on arXiv or HuggingFace) |
Shilin Yan, Aojun Zhou, Renrui Zhang, CaraJ, xy06 |
i) The paper introduces MINT-CoT, a method enabling interleaved visual tokens in mathematical chain-of-thought (CoT) reasoning for Large Language Models (LLMs). ii) The research objective is to enhance multimodal mathematical reasoning in LLMs by adaptively interleaving relevant visual tokens within textual reasoning steps. iii) The methodology involves an Interleave Token mechanism, a new MINT-CoT dataset with 54K mathematical problems, and a three-stage training strategy (Text-only CoT SFT, interleaved CoT SFT and interleaved CoT RL). iv) The MINT-CoT-7B model achieves +34.08% performance improvement on MathVista, +28.78% on GeoQA and +23.2% on MMStar compared to the baseline. v) The work provides AI practitioners with a method for improving visual-mathematical reasoning in multimodal LLMs, demonstrating significant performance gains over text-only and box-shaped visual CoT approaches. |
| VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal |
|
|
| Understanding in Videos (Read more on arXiv or HuggingFace) |
Ming-Hsuan Yang, Muhammad Maaz, Anqi Tang, Abdelrahman Shaker, Hanoona Rasheed |
i) VideoMathQA is introduced as a new benchmark for evaluating mathematical reasoning in videos. ii) The primary objective is to assess temporally extended cross-modal reasoning capabilities of AI models on videos involving mathematical problems. iii) The methodology involves creating a dataset of 420 annotated video-question pairs spanning 10 mathematical domains with expert-provided step-by-step reasoning trails. iv) Evaluation of 30 models reveals that GPT-04-mini achieves the highest step evaluation score of 6.9, while Qwen2.5-VL-72B leads among open-source models with a score of 5.0. v) The key implication is that AI practitioners need to focus on improving models’ ability to integrate fine-grained audio cues with visual information over extended time to solve complex mathematical problems in video settings. |
| Inference-Time Hyper-Scaling with KV Cache Compression (Read more on arXiv or HuggingFace) |
Edoardo M. Ponti, Piotr Nawrot, Konrad Staniszewski, Adrian Łańcucki |
i) This paper introduces Dynamic Memory Sparsification (DMS), a novel method for compressing the key-value (KV) cache in Transformer LLMs. ii) The main objective is to enhance inference-time scaling by improving reasoning accuracy within a fixed compute budget via KV cache compression. iii) The methodology involves retrofitting LLMs with DMS, which uses a learned eviction policy trained with a Gumbel-sigmoid to sparsify the KV cache. iv) The primary result is an average improvement of 9.1 points on AIME 24, 7.6 on GPQA, and 9.6 on LiveCodeBench for Qwen-R1 32B due to DMS. v) DMS provides AI practitioners with a method to improve the performance of LLMs in resource-constrained environments, enabling better reasoning capabilities within a given inference budget. |
| Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting (Read more on arXiv or HuggingFace) |
Jia-Wang Bian, Zeyu Zhang, Donny Y. Chen, lhmd, dc-walker |
i) This paper introduces PM-Loss, a novel regularization loss to improve geometry in feed-forward 3D Gaussian Splatting (3DGS) by leveraging pointmap priors. ii) The main objective is to mitigate depth discontinuities at object boundaries, a known limitation in depth map-based 3DGS pipelines. iii) The methodology involves using a pre-trained transformer to predict a pointmap which is then used as a pseudo-ground truth to regularize the unprojected depth maps via a single-directional Chamfer loss. iv) Experiments show that models trained with PM-Loss achieve a consistent PSNR gain of at least 2 dB on DL3DV and RealEstate10K datasets compared to baselines. v) PM-Loss provides AI practitioners with a plug-and-play, efficient, and effective method for improving the geometric quality and rendering results of feed-forward 3DGS models. |
| EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an |
|
|
| Egocentric World? (Read more on arXiv or HuggingFace) |
Dian Jiao, Wentong Li, Long Li, Ronghao Dang, CircleRadon |
i) EOC-Bench, a new benchmark, evaluates object-centric embodied cognition in multimodal large language models (MLLMs) within dynamic egocentric scenarios. ii) The paper investigates the capabilities of MLLMs to identify, recall, and forecast object states, locations, and relationships in dynamic egocentric videos. iii) The methodology involves a mixed-format human-in-the-loop annotation framework generating 3,277 QA pairs, categorized temporally into Past, Present, and Future and including visual object referencing prompts. iv) Results show the GPT-4o model achieves an overall accuracy of 61.83% on the EOC-Bench, with lower performance on tasks requiring absolute time perception. v) The benchmark highlights limitations in temporal reasoning and object-level spatiotemporal understanding in current MLLMs, which require robust designs for embodied object cognitive tasks. |
| Language-Image Alignment with Fixed Text Encoders (Read more on arXiv or HuggingFace) |
Yi Ma, Yue Zhao, robinwuzy, JingfengY |
Language-Image alignment is achievable by solely training the image encoder with a fixed, pre-trained large language model (LLM) as the text encoder. The research investigates if costly joint training of text and image encoders is necessary for language-image alignment, proposing instead to learn Language-Image alignment with a Fixed Text encoder (LIFT). The methodology involves using a pre-trained text encoder fine-tuned on an LLM to embed texts offline and solely training the image encoder to align visual representations with the text embeddings using CLIP’s contrastive loss. The results show that LIFT outperforms CLIP in most scenarios involving compositional understanding, achieving an average accuracy gain of 7.4% across seven compositional understanding tasks and demonstrates FLOPs reduction up to 35.7% in long caption training. LIFT provides AI practitioners with an alternative design choice for learning language-aligned visual representations, offering gains in computational efficiency and improved performance in compositional tasks by leveraging LLMs. |
| FlexPainter: Flexible and Multi-View Consistent Texture Generation (Read more on arXiv or HuggingFace) |
Luozhou Wang, Jiantao Lin, Leyi Wu, yingcongchen, StarYDY |
FlexPainter is a novel texture generation pipeline facilitating flexible multi-modal conditional guidance and consistent multi-view texture synthesis. The research aims to improve texture generation quality and control by enabling flexible prompt integration and mitigating inconsistencies in multi-view images. It employs a shared conditional embedding space for multi-modal aggregation, an image-based classifier-free guidance method for stylization, and a multi-view grid representation with view synchronization for consistency. Experiments demonstrate the framework achieves significantly better FID score compared to existing methods in text-to-texture generation. FlexPainter offers AI practitioners an improved approach to 3D texture generation with enhanced control and consistency, beneficial for applications in 3D modeling and computer graphics. |
| The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly |
|
|
| Licensed Text (Read more on arXiv or HuggingFace) |
Stella Biderman, Colin Raffel, Brian Lester, Nikhil Kandpal, storytracer |
i) The paper introduces the Common Pile v0.1, an 8TB dataset of public domain and openly licensed text, for large language model (LLM) pretraining. ii) The research aims to demonstrate the feasibility of training performant LLMs on openly licensed data as an alternative to unlicensed text sources. iii) The methodology involves collecting and curating text from 30 diverse sources, and training two 7B parameter LLMs, Comma v0.1-1T and Comma v0.1-2T, on 1 and 2 trillion tokens, respectively. iv) Results show Comma v0.1-1T and Comma v0.1-2T attain competitive performance to LLMs trained on unlicensed text with similar computational budgets, such as Llama 1 and 2 7B. v) AI practitioners can leverage the Common Pile v0.1 dataset and Comma v0.1 models to develop ethically-sourced LLMs, with code, data and models released. |
| Autoregressive Images Watermarking through Lexical Biasing: An Approach |
|
|
| Resistant to Regeneration Attack (Read more on arXiv or HuggingFace) |
Wenli Huang, Ye Deng, Sanping Zhou, Yiren Song, Siqi Hui |
Autoregressive Images Watermarking through Lexical Biasing (LBW) is proposed as a novel watermarking framework for autoregressive image generation models, resistant to regeneration attacks. The paper addresses the challenge of robust watermarking in AR models by introducing a lexical bias during token selection, using a multi-green-list strategy for enhanced security. LBW embeds watermarks by biasing token selection toward a predefined green list during image generation or substituting red tokens with green tokens post-hoc. Experiments demonstrate that LBW achieves superior robustness, with LBW-Post on RAR attaining an AUC of 0.995 and TPR@1FPR of 0.937 against regeneration attacks, outperforming WatermarkDM. AI practitioners can leverage LBW to ensure traceability and prevent misuse of images generated by AR models. |
| MedAgentGym: Training LLM Agents for Code-Based Medical Reasoning at |
|
|
| Scale (Read more on arXiv or HuggingFace) |
Yue Yu, Yishan Zhong, Yuchen Zhuang, Ran Xu, wshi83 |
i) MedAgentGym, a publicly available training environment, facilitates training large language model (LLM) agents for code-based medical reasoning. ii) The research aims to enhance coding-based medical reasoning capabilities in LLM agents using a specialized training environment. iii) The methodology involves creating 72,413 task instances across 129 categories from 12 real-world biomedical scenarios, encapsulated within executable coding environments, and benchmarking over 25 LLMs. iv) Med-Copilot-7B, leveraging MedAgentGym, achieves substantial performance gains through supervised fine-tuning (+36.44%) and reinforcement learning (+42.47%). v) This integrated platform can be used by AI practitioners to develop LLM-based coding assistants for advanced biomedical research and practice, offering an affordable and privacy-preserving alternative to commercial models for complex code-based medical reasoning. |
| Geometry-Editable and Appearance-Preserving Object Compositon (Read more on arXiv or HuggingFace) |
Liang Lin, Zhijing Yang, Chunmei Qing, Haojie Li, Jianman Lin |
i) The paper introduces DGAD, a diffusion model for geometry-editable and appearance-preserving object composition. ii) The main objective is to achieve both precise geometric editing and faithful appearance preservation when integrating objects into scenes. iii) The methodology involves disentangling geometry editing via CLIP/DINO-derived embeddings and appearance preservation via a dense cross-attention retrieval mechanism, integrating these into a pre-trained diffusion model. iv) Experiments show DGAD achieves a 61.14 IR score, indicating improved editability over existing methods. v) The principal implication is providing AI practitioners with an improved method for generating geometrically consistent and visually faithful composite images. |
| FreeTimeGS: Free Gaussians at Anytime and Anywhere for Dynamic Scene |
|
|
| Reconstruction (Read more on arXiv or HuggingFace) |
Zhanhua Zhang, Jiaming Sun, Zhen Xu, Peishan Yang, Yifan Wang |
i) The paper introduces FreeTimeGS, a novel 4D Gaussian representation for dynamic scene reconstruction enabling Gaussian primitives at arbitrary times and locations. ii) The main objective is to improve dynamic 3D scene reconstruction, particularly for scenes with complex motions. iii) The method endows each Gaussian primitive with an explicit motion function and incorporates a temporal opacity function and 4D regularization to enhance representational capacity and optimize rendering quality. iv) Experimental results on the SelfCap dataset demonstrate a PSNR improvement of 2.4dB (entire image) and 4.1dB (dynamic regions) over 4DGS and achieves real-time rendering speeds of 450 FPS at 1080p resolution. v) FreeTimeGS offers AI practitioners a more flexible and efficient method for dynamic scene reconstruction, potentially improving performance in applications requiring high-quality, real-time rendering of complex dynamic environments. |
| Rectified Point Flow: Generic Point Cloud Pose Estimation (Read more on arXiv or HuggingFace) |
Iro Armeni, Shuran Song, Shengyu Huang, Liyuan Zhu, Tao Sun |
i) Rectified Point Flow is introduced as a unified parameterization for point cloud registration and shape assembly. ii) The research aims to develop a generic point cloud pose estimation method that addresses both pairwise registration and multi-part shape assembly in a single conditional generative framework. iii) The methodology involves learning a continuous point-wise velocity field to transport noisy points toward target positions, conditioned on unposed part point clouds, along with a self-supervised encoder pretrained on point-wise overlap. iv) The proposed method achieves state-of-the-art performance on six benchmarks, and notably improves performance by jointly training on diverse datasets. v) This unified approach facilitates learning shared geometric priors for AI practitioners, leading to improved accuracy and generalizability across various 3D reasoning tasks. |
| Evaluation is All You Need: Strategic Overclaiming of LLM Reasoning |
|
|
| Capabilities Through Evaluation Design (Read more on arXiv or HuggingFace) |
Xiaoqi Jian, Yongfu Zhu, Jinzhu Wu, Weihong Lin, lincharliesun |
i) This paper investigates the sensitivity of LLM reasoning benchmark evaluations to subtle configuration variations. ii) The main objective is to assess how minor changes in evaluation conditions impact the reliability of reported LLM performance. iii) The study employs controlled experiments on Deepseek-R1-Distill series models, varying parameters like seed initialization, dataset version, instruction position, option bias, and tensor parallelism. iv) Results indicate that fluctuations caused by varying seed can be greater than baseline, with changes in option order and answer position causing performance fluctuations above 5 percentage points on GPQA Diamond; 67% of experimental groups exhibited TP fluctuation exceeding baseline reference. v) AI practitioners need to standardize evaluation methodologies, including disclosing evaluation settings and statistically supported stable performance, to ensure the reliability and fairness of LLM comparisons. |
| Scaling Laws for Robust Comparison of Open Foundation Language-Vision |
|
|
| Models and Datasets (Read more on arXiv or HuggingFace) |
Romain Beaumont, Tommie Kerssies, Giovanni Pucceti, Tomer Porian, Marianna Nezhurina |
i) The paper derives scaling laws for CLIP and MaMMUT language-vision models to enable robust model and dataset comparison across varying scales. ii) The objective is to determine the dependence of model performance on pre-training compute for model and dataset comparison in language-vision learning. iii) The study derives full scaling laws based on dense measurements across model and samples seen scales for CLIP and MaMMUT architectures trained on DataComp-1.4B, DFN-1.4B and Re-LAION-1.4B datasets, evaluating downstream tasks such as zero-shot classification, retrieval, and segmentation. iv) The results indicate that MaMMUT shows stronger improvement with scale and better sample efficiency than standard CLIP, and openMaMMUT-L/14 achieves 80.3% zero-shot ImageNet-1k accuracy trained on 12.8B samples from DataComp-1.4B. v) Practitioners can utilize the derived scaling laws to systematically compare and improve open foundation models and datasets, avoiding misleading conclusions based on single reference scales, allowing for informed selection of pre-training procedures. |
| SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video |
|
|
| Diffusion Transformers (Read more on arXiv or HuggingFace) |
Youqiang Zhang, Baoxuan Gu, Hao Jiang, Zhengcong Fei, diqiu7 |
SkyReels-Audio introduces a unified framework for generating and editing audio-conditioned talking portrait videos using diffusion transformers. The research aims to synthesize high-fidelity, temporally coherent talking portrait videos from multimodal inputs including audio, text, images, and videos. A hybrid curriculum learning strategy progressively aligns audio with facial motion, enhanced by a facial mask loss and audio-guided classifier-free guidance with a sliding-window denoising approach for visual fidelity. Evaluations demonstrate SkyReels-Audio achieves superior performance in lip-sync accuracy with a reported FID of 38.32, identity consistency, and realistic facial dynamics on the HDTF dataset. For AI practitioners, this work offers a scalable architecture and training methodology for generating controllable and coherent talking head videos, enabling diverse applications in digital media and interactive AI. |
| Contextual Integrity in LLMs via Reasoning and Reinforcement Learning (Read more on arXiv or HuggingFace) |
Janardhan Kulkarni, Huseyin A. Inan, wulu, sahar-abdelnabi, Eric-Lan |
i) The paper introduces a reinforcement learning (RL) framework to improve contextual integrity (CI) in Large Language Models (LLMs). ii) The research aims to reduce inappropriate information disclosure by LLMs while maintaining task performance by improving reasoning capabilities around CI. iii) The methodology involves prompting LLMs for explicit reasoning about CI, followed by RL-based post-training using a synthetic dataset and the GRPO algorithm. iv) The results demonstrate up to a 40% reduction in privacy leakage rate on the PrivacyLens benchmark, showing effective transfer of CI reasoning capabilities. v) The research implies that supporting CI reasoning should be a core part of the alignment process for real-world LLM-based agents, improving their safety and context-awareness. |
| Micro-Act: Mitigate Knowledge Conflict in Question Answering via |
|
|
| Actionable Self-Reasoning (Read more on arXiv or HuggingFace) |
Xiaolong Li, Ge Qu, Bowen Qin, Jinyang Li, NanHUO |
i) The paper introduces MICRO-ACT, a framework for mitigating knowledge conflicts in retrieval-augmented question answering (QA) systems. ii) The research objective is to improve QA accuracy by addressing inconsistencies between retrieved external knowledge and the internal, parametric knowledge of large language models (LLMs). iii) MICRO-ACT employs a hierarchical action space and adaptive granularity through decomposition, enabling fine-grained comparisons between knowledge sources. iv) Experiments on five benchmark datasets showed that MICRO-ACT improved QA accuracy over state-of-the-art baselines by up to 9.40% on ConflictBank and 6.65% on KRE datasets for GPT-40-mini. v) AI practitioners can leverage MICRO-ACT’s dynamic decomposition to build more reliable RAG systems that are more resilient to knowledge conflicts, especially in temporal and semantic contexts. |
| RobustSplat: Decoupling Densification and Dynamics for Transient-Free |
|
|
| 3DGS (Read more on arXiv or HuggingFace) |
Yuan Xiong, Guanying Chen, Kunbin Yao, Yuqi Zhang, fcy99 |
RobustSplat addresses artifact generation in 3D Gaussian Splatting (3DGS) due to transient objects in dynamic scenes. The paper aims to improve 3DGS optimization in in-the-wild scenarios by mitigating the influence of transient objects. RobustSplat employs a delayed Gaussian growth strategy and scale-cascaded mask bootstrapping approach, prioritizing static scene reconstruction before densification. Experiments on the NeRF On-the-go dataset demonstrate that RobustSplat achieves state-of-the-art performance, improving PSNR across six scenes. This improved method can be directly incorporated into existing 3DGS pipelines by AI practitioners for enhanced robustness in dynamic environments. |
| Diffusion-Based Generative Models for 3D Occupancy Prediction in |
|
|
| Autonomous Driving (Read more on arXiv or HuggingFace) |
Yingshi Liang, Yucheng Mao, Tianyuan Yuan, Yicheng Liu, Yunshen Wang |
i) This paper introduces a diffusion-based generative model for 3D occupancy prediction in autonomous driving. ii) The research aims to improve 3D occupancy prediction by reframing it as a generative modeling task that incorporates 3D scene priors and handles noisy data. iii) The methodology involves adapting diffusion models for occupancy prediction, incorporating conditional sampling with a U-Net variant denoiser network and a BEV visual encoder, and exploring different occupancy representations including spatial latent, triplane, and discrete categorical variables. iv) Experiments show that the diffusion-based generative model outperforms state-of-the-art discriminative approaches, achieving a 7.05 mIoU improvement over BEVFormer, especially in occluded or low-visibility regions. v) The research implies that AI practitioners can leverage diffusion models to enhance the realism, accuracy, and consistency of 3D occupancy predictions, particularly in autonomous driving applications with noisy or incomplete sensor data. |
| Images are Worth Variable Length of Representations (Read more on arXiv or HuggingFace) |
Zineng Tang, Wenhao Yan, Xin Liang, Rodolfo Corona, Lingjun Mao |
i) The paper introduces DOVE, a dynamic vision encoder that generates variable-length token sequences for image reconstruction. ii) The research aims to improve the efficiency and expressiveness of visual representations by adaptively adjusting token sequence length based on image complexity. iii) The methodology involves extending the standard autoencoder framework with a transformer-based dynamic token generator and jointly optimizing image reconstruction quality and EOS token prediction. iv) Results demonstrate that DOVE significantly reduces the average number of tokens while maintaining high reconstruction quality and outperforms autoencoder-based tokenization methods in downstream tasks, achieving a token compression rate averaging 68% in a query-conditioned setup. v) DOVE’s dynamic token generation and query-conditioned approach provide AI practitioners with a more efficient and semantically richer vision encoder for various tasks including visual question answering. |
| Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric |
|
|
| Approach (Read more on arXiv or HuggingFace) |
Weidi Xie, Yanfeng Wang, Ya Zhang, Lisong Dai, zzh99 |
i) This paper presents OminiAbnorm-CT, a system and dataset for abnormality-centric whole-body CT image interpretation. ii) The research aims to develop an AI system capable of automatically detecting, localizing, and describing abnormal findings across multi-plane, whole-body CT scans based on text or visual prompts. iii) The methodology involves creating a hierarchical taxonomy of 404 abnormal findings, curating a dataset of 14.5K CT images with 19K abnormality annotations, and developing a multi-modal language model integrated with a segmentation module, trained jointly with a text generation loss and a segmentation loss. iv) The OminiAbnorm-CT system significantly outperforms existing methods in grounded report generation, text-guided grounded report generation, and visual-prompted report generation, achieving a RaTEScore of 86.35 on the axial visual prompted report generation task, indicating superior performance in generating clinically relevant reports. v) The principal implication for AI practitioners is the demonstration of an abnormality-centric approach for improving the explainability and clinical relevance of automated CT image interpretation systems, which can be used to inform the design of more effective diagnostic tools. |
| BEVCALIB: LiDAR-Camera Calibration via Geometry-Guided Bird’s-Eye View |
|
|
| Representations (Read more on arXiv or HuggingFace) |
Konstantinos Karydis, Divyank Shah, Justin Yue, Jerry Li, Yewandou |
BEVCALIB is a novel LiDAR-camera calibration model using bird’s-eye view (BEV) representations. The research aims to perform LiDAR-camera calibration from raw data by leveraging BEV features. It extracts and fuses camera and LiDAR BEV features into a shared space and employs a geometry-guided feature selector for efficient training. Evaluations show BEVCALIB outperforms baselines, achieving an average improvement of (47.08%, 82.32%) on the KITTI dataset and (78.17%, 68.29%) on the NuScenes dataset, in terms of (translation, rotation) respectively, under various noise conditions. BEVCALIB provides AI practitioners with an open-source, high-performing tool for LiDAR-camera calibration, improving accuracy and robustness compared to existing methods. |
| PATS: Proficiency-Aware Temporal Sampling for Multi-View Sports Skill |
|
|
| Assessment (Read more on arXiv or HuggingFace) |
Antonio Liotta, EdBianchi |
i) The paper introduces Proficiency-Aware Temporal Sampling (PATS), a novel video sampling strategy designed for multi-view sports skill assessment. ii) The main objective is to improve the accuracy of automated skill assessment by preserving temporal continuity within continuous video segments. iii) The methodology involves adaptively segmenting videos to ensure each analyzed portion contains a complete fundamental movement, maximizing information coverage while maintaining temporal coherence. iv) Evaluated on the EgoExo4D benchmark, PATS surpasses the state-of-the-art accuracy across all viewing configurations, achieving up to a +3.05% improvement, including a +26.22% gain in bouldering accuracy. v) For AI practitioners, PATS offers an architecture-agnostic pre-processing step that can be integrated with existing temporal modeling frameworks to enhance model accuracy in sports skill assessment without adding computational overhead. |
| What do self-supervised speech models know about Dutch? Analyzing |
|
|
| advantages of language-specific pre-training (Read more on arXiv or HuggingFace) |
Willem Zuidema, Gaofei Shen, Charlotte Pouw, Hosein Mohebbi, Marianne de Heer Kloots |
i) This paper analyzes the encoding of Dutch phonetic and lexical features in self-supervised Wav2Vec2 models pre-trained with varying amounts of Dutch, English, and multilingual data. ii) The research investigates whether language-specific pre-training improves the representation of Dutch linguistic features in SSL models compared to English or multilingual pre-training. iii) The methodology includes pre-training Wav2Vec2 models with different language configurations, extracting internal representations, and evaluating them using phone identity probing, ABX tasks, phone/word clustering, representational similarity analysis, and downstream ASR fine-tuning. iv) Results indicate that pre-training exclusively on Dutch improves the representation of Dutch linguistic features, with the Dutch-trained model achieving lower word error rates (WER) of 10.4 on the CGN-o test set in downstream ASR compared to English (21.5) and multilingual (12.7) models. v) The principal implication is that language-specific pre-training can substantially enhance the encoding of language-specific features in SSL models, improving downstream ASR performance, particularly for languages with unique phonetic characteristics; indicating that careful selection of pre-training data is crucial for optimizing SSL models for specific languages. |
Papers for 2025-06-05
| Title |
Authors |
Summary |
| MiMo-VL Technical Report (Read more on arXiv or HuggingFace) |
Prestonprom, dwzhu, tobiaslee, gsh33, ShuhuaiRen |
i) The paper introduces MiMo-VL-7B, a vision-language model achieving state-of-the-art performance in visual understanding and multimodal reasoning. ii) The primary research objective is to develop a compact and powerful vision-language model exceeding existing models in general visual understanding and multimodal reasoning, particularly for GUI grounding applications. iii) The methodology involves a four-stage pre-training process (2.4 trillion tokens) combined with a Mixed On-policy Reinforcement Learning (MORL) framework integrating diverse reward signals. iv) MiMo-VL-7B-RL achieves a score of 59.4 on OlympiadBench and 56.1 on OSWorld-G, outperforming Qwen2.5-VL-7B on 35 of 40 evaluated tasks. v) The principal implication is that incorporating high-quality, broad-coverage reasoning data into pre-training stages significantly enhances model performance and mixed on-policy reinforcement learning further enhances performance. |
| Advancing Multimodal Reasoning: From Optimized Cold Start to Staged |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yafu Li, Yue Guo, Shuang Chen, JC-Chen, Warrieryes |
i) This paper introduces ReVisual-R1, a 7B open-source Multimodal Large Language Model (MLLM), trained via a staged curriculum. ii) The main objective is to enhance multimodal reasoning capabilities in MLLMs by optimizing the training pipeline. iii) The methodology involves a three-stage curriculum consisting of a text-centric cold start, multimodal reinforcement learning (RL) with Prioritized Advantage Distillation (PAD), and a final text-only RL refinement phase. iv) ReVisual-R1 achieves a new state-of-the-art among open-source 7B MLLMs, with an average score of 53.1% across challenging benchmarks and demonstrates a +44.6% increase on AIME24. v) AI practitioners can leverage the staged curriculum approach to improve the reasoning abilities of open-source MLLMs on challenging multimodal tasks, rivaling proprietary models. |
| AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment (Read more on arXiv or HuggingFace) |
Aleksandr I. Panov, Alexey K. Kovalev, Anastasiia Ivanova, AlexeyKov, tenebrissilvam |
i) The paper introduces AmbiK, a new fully textual dataset for ambiguous task detection in kitchen environments. ii) The research aims to provide a benchmark for evaluating and comparing ambiguity detection methods in Large Language Models (LLMs) applied to embodied AI. iii) The methodology involves curating a dataset of 2000 paired ambiguous and unambiguous instructions, categorized by ambiguity type (Preferences, Common Sense Knowledge, Safety), and human-validated using LLMs for data collection. iv) Experiments using SOTA LLMs on AmbiK demonstrate a limited success in resolving ambiguity, as no method achieves over 20% Set Size Correctness (SSC), indicating a misalignment between predicted and actual ambiguity sets. v) AmbiK dataset’s challenging nature suggests that LLMs logits are often inadequate approximations of uncertainty for task planning in complex environments, highlighting a need for improved uncertainty estimation techniques for AI practitioners developing embodied agents. |
| CASS: Nvidia to AMD Transpilation with Data, Models, and Benchmark (Read more on arXiv or HuggingFace) |
Salman Khan, Seung Hun Eddie Han, GustavoStahl, Sarim-Hash, ahmedheakl |
i) The paper introduces CASS, a dataset and model suite for cross-architecture GPU code transpilation. ii) The research objective is to address the portability gap in GPU code across Nvidia (CUDA) and AMD (HIP/RDNA3) architectures. iii) The methodology involves creating a 70k aligned CUDA-HIP source and SASS-RDNA3 assembly dataset and fine-tuning domain-specific language models on it. iv) Results include achieving 95% accuracy in source translation and 37.5% assembly translation, with the translated assemblies matching native performance in over 85% of test cases regarding runtime and memory behavior. v) CASS provides AI practitioners with resources for GPU compiler tooling, binary compatibility analysis, and LLM-guided hardware translation, as demonstrated by the presented CASS models outperforming GPT-40. |
| A Controllable Examination for Long-Context Language Models (Read more on arXiv or HuggingFace) |
Fei Yuan, Zihan Qiu, Wenhao Zhu, Zeyu Huang, thomasyyj |
i) LongBioBench, a novel benchmark, is introduced for evaluating long-context language models (LCLMs) using artificially generated biographies. ii) The research aims to assess the understanding, reasoning, and trustworthiness of LCLMs in a controlled setting, addressing limitations of existing real-world and synthetic benchmarks. iii) The methodology involves constructing a dataset of configurable biographies and evaluating 18 LCLMs across various tasks, including understanding, reasoning, and trustworthiness. iv) Results demonstrate that most models exhibit deficiencies in semantic understanding and reasoning, with performance decreasing as context length increases; moreover, LongBioBench has a high correlation (0.853) with the scores of HELMET. v) LongBioBench provides AI practitioners with a configurable and interpretable benchmark for evaluating and improving the long-context capabilities of language models, particularly regarding semantic understanding and reasoning over retrieved information. |
| SuperWriter: Reflection-Driven Long-Form Generation with Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Roy Ka-Wei Lee, Juanzi Li, Yushi Bai, Yuhao Wu, Zhiqiang007 |
SuperWriter-Agent enhances long-form text generation by incorporating structured thinking. The research addresses maintaining coherence and consistency in large language models (LLMs) for extended text. It fine-tunes a 7B SuperWriter-LM with a structured dataset. SuperWriter-LM achieves state-of-the-art performance with results showing an increase of performance. Practitioners can leverage structured thinking steps to enhance the coherence and quality of long-form text generation models. |
| Ψ-Sampler: Initial Particle Sampling for SMC-Based Inference-Time |
|
|
| Reward Alignment in Score Models (Read more on arXiv or HuggingFace) |
Minhyuk Sung, Kyeongmin Yeo, Yunhong Min, Taehoon Yoon |
i) The paper introduces Ψ-SAMPLER, a Sequential Monte Carlo (SMC) framework utilizing preconditioned Crank-Nicolson Langevin (pCNL)-based initial particle sampling for improved inference-time reward alignment in score-based generative models. ii) The research addresses the problem of inefficient exploration of high-reward regions in existing SMC-based reward alignment methods due to Gaussian prior initialization. iii) The methodology involves a pCNL algorithm for efficient posterior sampling in high-dimensional latent spaces, combining dimension-robust proposals with gradient-informed dynamics to generate initial particles for SMC. iv) Experiments show Ψ-SAMPLER consistently outperforms baselines across reward alignment tasks, including achieving a negative smooth L1 loss of 0.850 in quantity-aware generation compared to 1.804 with a base SMC method. v) AI practitioners can leverage Ψ-SAMPLER to improve the efficiency and performance of inference-time reward alignment in score-based generative models by incorporating posterior-based initialization via the pCNL algorithm. |
| Voyager: Long-Range and World-Consistent Video Diffusion for Explorable |
|
|
| 3D Scene Generation (Read more on arXiv or HuggingFace) |
Zhenwei Wang, Yuhao Liu, Tengfei Wang, Wangguandong Zheng, tyhuang |
Voyager introduces a video diffusion framework for generating world-consistent, explorable 3D scenes from a single image. The research addresses the problem of generating long-range, spatially consistent 3D scenes suitable for applications like video games and VR. The methodology involves a world-consistent video diffusion model integrating RGB and depth information, a world caching mechanism with point culling, and a scalable data engine for training data curation. Experiments on the RealEstate 10K dataset demonstrate Voyager achieves a PSNR of 18.751, outperforming existing methods in novel view synthesis. The framework enables AI practitioners to generate and reconstruct 3D scenes with improved spatial consistency, facilitating applications requiring explorable virtual environments; further work is needed to understand the limitations of this approach to unseen or complex scene geometries. |
| LayerFlow: A Unified Model for Layer-aware Video Generation (Read more on arXiv or HuggingFace) |
Yiyang Wang, Yuanpeng Tu, Hao Luo, Sihui Ji, xichenhku |
i) LayerFlow is a unified model for layer-aware video generation, supporting transparent foreground, clean background, blended scenes, video decomposition, and conditional generation. ii) The research objective is to develop a single framework capable of generating and manipulating video layers, addressing challenges in representation and data scarcity. iii) The methodology involves a DiT-based text-to-video model, layer embeddings for layer awareness, and a multi-stage training strategy with Motion and Content LoRAs using both static images and dynamic videos. iv) The model achieves improved inter-layer coherence and aesthetic quality, shown qualitatively and quantitatively via user studies where LayerFlow performs significantly better in text consistency and overall quality over alternatives. v) LayerFlow offers AI practitioners a unified approach to layer-aware video generation, enabling flexible content creation and manipulation with potential applications in visual production workflows. |
| SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation (Read more on arXiv or HuggingFace) |
Xingyu Wu, Xinyu Dong, yanyc, zjuxhl, xiaoooobai |
i) SVGenius is introduced as a benchmark for evaluating Large Language Models (LLMs) and Multimodal LLMs in SVG processing across understanding, editing, and generation. ii) The research aims to comprehensively assess LLMs’ capabilities in manipulating Scalable Vector Graphics (SVG), addressing limitations of existing benchmarks. iii) SVGenius evaluates models via 2,377 queries spanning perceptual and semantic QA, code optimization, bug fixing, style editing, text-to-SVG, image-to-SVG, and style transfer tasks, utilizing real-world data from 24 domains with systematic complexity stratification. iv) Results show proprietary models outperform open-source alternatives, but all models exhibit performance degradation with increased complexity, with a gap between 82.72% to 42.22% in Perceptual QA across difficulty levels using GPT-40, and reasoning-enhanced training proves effective for complex tasks, while style transfer remains challenging. v) SVGenius provides AI practitioners with a systematic framework and baseline results for developing and assessing vector graphics models, identifying areas for improvement such as handling complexity and stylistic variations in SVG processing. |
| Image Editing As Programs with Diffusion Models (Read more on arXiv or HuggingFace) |
Xinchao Wang, Zhenxiong Tan, Songhua Liu, Yujia Hu, adamdad |
i) The paper introduces Image Editing As Programs (IEAP), a DiT-based framework for instruction-driven image editing. ii) The primary objective is to address the limitations of current diffusion models in handling structurally inconsistent edits that require layout modifications. iii) The methodology involves decomposing complex editing instructions into sequences of atomic operations (RoI localization, inpainting, editing, compositing, global transformation), orchestrated by a VLM-based agent. iv) Experiments demonstrate that IEAP achieves state-of-the-art performance on standard benchmarks, with a GPT-4o average score of 4.51 on the AnyEdit test set. v) IEAP provides AI practitioners with a modular and interpretable approach to image editing that improves accuracy and semantic fidelity, especially in complex scenarios. |
| Unleashing the Reasoning Potential of Pre-trained LLMs by Critique |
|
|
| Fine-Tuning on One Problem (Read more on arXiv or HuggingFace) |
Wenhu Chen, Lijun Wu, Kai Zou, Ping Nie, Yubo Wang |
i) The paper introduces Critique Fine-Tuning (CFT), a compute-efficient method to enhance LLM reasoning. ii) The study investigates whether critique data from a single problem can effectively unleash LLMs’ reasoning potential. iii) The methodology involves generating diverse solutions to a single problem, using teacher LLMs for critiques, and fine-tuning student LLMs on the critique data. iv) Qwen-Math-7B-CFT achieves a 15% average improvement on six math benchmarks and 16% on three logic reasoning benchmarks, using only 5 GPU hours. v) CFT offers a simple, general, and compute-efficient approach for AI practitioners to improve reasoning capabilities of LLMs, potentially surpassing RL methods with significantly less compute. |
| TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning |
|
|
| for Enhancing LLMs’ Social Intelligence (Read more on arXiv or HuggingFace) |
Wenqi Zhang, Xiang Huang, Yuchuan Wu, Xing Gao, Guiyang Hou |
i) This paper introduces TimeHC-RL, a novel reinforcement learning framework to enhance LLMs’ social intelligence by incorporating temporal awareness and hierarchical cognitive processing. ii) The research objective is to improve LLMs’ cognitive development in social domains, particularly from a post-training perspective, by modeling temporal dynamics and diverse cognitive modes. iii) The methodology involves a temporal-aware reward mechanism and a hierarchical cognition framework encompassing intuitive reactions, surface-level thinking, and deliberate thinking within a reinforcement learning paradigm. iv) Experiments demonstrate that TimeHC-RL, with a 7B backbone model, achieves a 29.0 point comprehensive performance improvement in In-Domain evaluation compared to the backbone model, rivaling the performance of advanced models such as DeepSeek-R1 and OpenAI-O3. v) TimeHC-RL provides AI practitioners with a new approach to enhance LLMs’ social intelligence by explicitly modeling temporal dynamics and incorporating a more nuanced cognitive hierarchy, thus enabling more contextually appropriate and human-like social reasoning capabilities in AI systems. |
| IllumiCraft: Unified Geometry and Illumination Diffusion for |
|
|
| Controllable Video Generation (Read more on arXiv or HuggingFace) |
Ming-Hsuan Yang, Ronald Clark, Yi-Hsuan Tsai, Yi-Wen Chen, Yuanze Lin |
IllumiCraft is presented as a unified diffusion framework for controllable video generation by jointly modeling geometry and illumination. The research aims to enable high-quality video relighting, addressing limitations in existing methods regarding explicit geometric cues. The key methodology integrates HDR video, synthetically relit frames, and 3D point tracks within a DiT-based diffusion model architecture. Experiments show IllumiCraft cuts FVD by 37% compared to Light-A-Video on 49-frame background-conditioned relighting. The principal implication is a potential for AI practitioners to utilize the explicit incorporation of geometric and illumination guidance for enhanced control and fidelity in video generation tasks, specifically video relighting. |
| VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Wenhu Chen, Xiang Yue, Kai Zou, Ping Nie, yuanshengni |
VisCoder introduces a fine-tuned language model for generating executable Python visualization code. The study addresses the challenge of creating accurate visualization code from natural language and data inputs. The methodology involves instruction-tuning the Qwen2.5-Coder-Instruct model using VisCode-200K, a dataset containing over 200K examples including validated code paired with natural language instructions and multi-turn revision dialogues from Code-Feedback. VisCoder-3B improves execution pass rate by 19.6% over Qwen2.5-Coder on the PandasPlotBench. This work provides AI practitioners with a model and a dataset to improve the reliability and accuracy of automatically generated data visualizations. |
| MMR-V: What’s Left Unsaid? A Benchmark for Multimodal Deep Reasoning in |
|
|
| Videos (Read more on arXiv or HuggingFace) |
Shangqing Tu, Jiachun Li, Hongbang Yuan, Zhuoran Jin, Kejian Zhu |
i) The paper introduces MMR-V, a new benchmark for evaluating multi-modal deep reasoning in videos. ii) The main objective is to assess the capability of MLLMs to perform long-range, multi-frame reasoning and inference beyond direct perception in videos. iii) The methodology involves constructing a dataset of 317 videos and 1257 tasks with manual annotation, distractor generation, and categorization into implicit and explicit reasoning types. iv) Experiments show that the best-performing model, o4-mini, achieves 52.5% accuracy on the MMR-V benchmark. v) The principal implication for AI practitioners is that current MLLMs struggle with complex video reasoning, requiring further research into improving multi-modal analysis and evidence mining capabilities for tasks beyond simple perception. |
| Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysis (Read more on arXiv or HuggingFace) |
Juanzi Li, Lei Hou, Zhuoran Jin, Shangqing Tu, Kejian Zhu |
i) This paper introduces a novel approach to trustworthy LLM evaluation by analyzing and mitigating the impact of shortcut neurons. ii) The research aims to address the issue of data contamination in LLM evaluation by identifying and suppressing shortcut reasoning mechanisms. iii) The methodology involves comparative and causal analysis to locate shortcut neurons, followed by a shortcut neuron patching technique during evaluation. iv) Experiments show the method effectively mitigates contamination, as demonstrated by a Spearman correlation coefficient exceeding 0.95 with the MixEval benchmark, and reduces original model accuracy by 37% after patching. v) AI practitioners can utilize this method to obtain more trustworthy evaluations of LLMs by addressing the impact of shortcut neurons, leading to more reliable model deployment and development. |
| Rectified Sparse Attention (Read more on arXiv or HuggingFace) |
Jian Chen, Yuqing Xia, Li Dong, Tianzhu Ye, Yutao Sun |
i) Rectified Sparse Attention (ReSA) addresses KV cache misalignment in sparse decoding to improve long-sequence generation. ii) The research aims to enhance the efficiency of long-sequence generation in Large Language Models without sacrificing quality. iii) ReSA combines block-sparse attention with periodic dense rectification to refresh the KV cache at fixed intervals. iv) Experiments show ReSA achieves up to 2.42x end-to-end speedup during decoding at 256K sequence length while maintaining near-lossless generation quality. v) AI practitioners can utilize ReSA for scalable long-context inference, especially in memory-constrained environments, offering a practical solution for deploying Large Language Models. |
| DenseDPO: Fine-Grained Temporal Preference Optimization for Video |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Ashkan Mirzaei, Willi Menapace, Ivan Skorokhodov, Anil Kag, Dazitu616 |
i) DenseDPO improves video diffusion models by introducing fine-grained temporal preference optimization. ii) The paper addresses how to improve video diffusion models with human preference learning while mitigating motion bias. iii) The methodology involves creating video pairs by denoising corrupted copies of a ground truth video, segmenting videos for per-segment preference labeling, and utilizing vision-language models (VLMs) for automated preference annotation. iv) DenseDPO improves motion generation over vanilla DPO while matching it in text alignment, visual quality, and temporal consistency, even with only one-third of the labeled data. v) The use of segment-level preference allows practitioners to effectively train video diffusion models with more accurate and dense supervision while reducing biases and annotation costs, potentially unlocking automatic preference annotation via VLMs. |
| Beyond the Surface: Measuring Self-Preference in LLM Judgments (Read more on arXiv or HuggingFace) |
Yankai Lin, Enrui Hu, Xinyu Zhang, Hao Wang, JaxChen |
i) The paper introduces the DBG score to more accurately measure self-preference bias in Large Language Model (LLM) judges. ii) The research objective is to disentangle self-preference bias from response quality when evaluating LLMs as judges. iii) The methodology involves introducing gold judgments as proxies for ground truth response quality and comparing these with the judge model’s scores to compute the DBG score. iv) Experiments reveal that LLMs exhibit self-preference bias, with larger models showing less bias than smaller ones; for example, the DBG score of Llama-3.1-70B is 0.4% whereas Llama-3.1-8B is 21.6%. v) The findings imply that LLM developers should prioritize larger models for judgment tasks to mitigate self-preference bias, and the DBG score offers a more reliable evaluation metric. |
| Critique-GRPO: Advancing LLM Reasoning with Natural Language and |
|
|
| Numerical Feedback (Read more on arXiv or HuggingFace) |
Chaochao Lu, Kaituo Feng, Hao Sun, Xiaoying Zhang, YipengZhang |
i) Critique-GRPO is introduced as an online reinforcement learning framework for enhancing LLM reasoning. ii) The research investigates whether integrating natural language critiques alongside numerical rewards improves LLM reasoning compared to using numerical rewards alone. iii) The methodology involves fine-tuning Qwen2.5-7B-Base and Qwen3-8B-Base models using a modified Group Relative Policy Optimization (GRPO) algorithm that incorporates both natural language critiques and numerical feedback. iv) Experiments across eight reasoning tasks show that Critique-GRPO improves average pass@1 scores by approximately 4.5% and 5% respectively, and also reveals that models exhibit effective refinements when provided with chain-of-thought critiques. v) The findings imply that AI practitioners can enhance LLM reasoning capabilities more effectively by leveraging both natural language critiques and numerical rewards in reinforcement learning frameworks, and this can be more useful than imitation learning by expert demonstrations. |
| TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via |
|
|
| Autoregressive Diffusion Models (Read more on arXiv or HuggingFace) |
Weimin Wang, Chetwin Low |
TalkingMachines introduces an efficient framework for real-time, audio-driven character animation. The research objective is to transform pre-trained video generation models into real-time capable systems. It employs an adapted image-to-video Diffusion Transformer (DiT) model and asymmetric knowledge distillation with sparse causal attention. The framework distills a model to 2 diffusion steps achieving real-time performance and reducing latency for interactive applications with less than 30% end-to-end generation time spent on VAE decoding and device-to-host transfer per video chunk. This disaggregation server design helps practitioners overcome computational bottlenecks in real-time streaming, including GPU allocation, communication-computation overlap, and memory reuse. |
| Robustness in Both Domains: CLIP Needs a Robust Text Encoder (Read more on arXiv or HuggingFace) |
Matthias Hein, Yongtao Wu, Naman Deep Singh, Elias Abad Rocamora, chs20 |
i) The paper introduces LEAF, a novel adversarial finetuning method for improving the robustness of CLIP text encoders. ii) The main objective is to enhance the robustness of CLIP models against adversarial text perturbations without sacrificing performance in the image domain. iii) The methodology involves an efficient adversarial finetuning technique utilizing a parallelizable Levenshtein distance-constrained attack within training batches. iv) The results show that LEAF improves the zero-shot adversarial accuracy in the text domain from 44.5% to 63.3% on AG-News with k=1, while maintaining vision performance and improving text-to-image generation quality and multimodal retrieval under adversarial noise. v) Robust CLIP text encoders, produced via LEAF, facilitate better reconstruction of input text from embeddings, and could improve the reliability of multimodal systems against adversarial attacks which is particularly relevant for deploying such models in production. |
| DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via |
|
|
| Diffusion Transformers (Read more on arXiv or HuggingFace) |
Xiangtai Li, Xuequan Lu, Qianyu Zhou, Hang Zhao, Zitong Wang |
i) The paper introduces DiffDecompose, a diffusion transformer-based framework for layer-wise decomposition of alpha-composited images. ii) The research aims to recover constituent layers from single overlapped images with semi-transparent or transparent layer non-linear occlusions. iii) The proposed DiffDecompose framework employs a diffusion transformer architecture and leverages In-Context Decomposition and Layer Position Encoding Cloning. iv) The framework achieves an average improvement of 36.3% in RMSE, +1.2% in SSIM, and 52.8% in LPIPS compared to existing methods on the AlphaBlend dataset. v) The framework introduces a novel approach for disentangling composite images which provides AI practitioners with more accurate image extraction while preserving fine-grained details. |
| Adapt before Continual Learning (Read more on arXiv or HuggingFace) |
Yanan Sun, Chunhui Ding, Tao Feng, JacobYuan, Kurt1024 |
Adapting PTMs before core continual learning process (ACL) framework enhances plasticity and stability in PTM-based continual learning. This research addresses the stability-plasticity dilemma in continual learning with pre-trained models by adapting PTMs before the core CL process. The methodology involves refining the PTM backbone through an adaptation phase, aligning embeddings with original class prototypes and distancing them from others, before applying CL techniques. Experiments demonstrate that ACL significantly improves CL performance, achieving gains of up to 10.41% in Average Optimal Accuracy (AOA). ACL provides AI practitioners with a versatile solution to improve PTM-based continual learning by enhancing plasticity and stability. |
| RefEdit: A Benchmark and Method for Improving Instruction-based Image |
|
|
| Editing Model on Referring Expressions (Read more on arXiv or HuggingFace) |
Chitta Baral, Yezhou Yang, Shivam Singh, Bimsara Pathiraja, mpatel57 |
RefEdit introduces a benchmark and method for improved instruction-based image editing with referring expressions. The paper addresses the challenge of instruction-based image editing models struggling with complex scenes by introducing RefEdit-Bench, a benchmark based on RefCOCO. A synthetic data generation pipeline using GPT-40 and FlowChef is developed, and a new model, RefEdit, is trained on this data. RefEdit, trained on 20,000 editing triplets, outperforms baselines trained on millions of samples. The findings imply that targeted, high-quality synthetic data improves model precision in complex editing scenarios for AI practitioners. |
| Quantitative LLM Judges (Read more on arXiv or HuggingFace) |
Pranchal Agarwal, Tushar Parmanand Budhwani, Jeevana Kruthi Karnuthala, Aishwarya Sahoo, Franck-Dernoncourt |
i) This paper introduces quantitative LLM judges, a framework to enhance LLM-based evaluation by decoupling qualitative reasoning from quantitative score prediction. ii) The research aims to improve the accuracy of LLM-as-a-judge by using the judge’s textual evaluation to predict more accurate numerical scores aligned with human assessments. iii) The methodology involves training generalized linear models (GLMs) on top of LLM embeddings of textual evaluations from a base judge, using human scores for calibration in different tasks like absolute rating and relative preference prediction. iv) Results show that quantitative judges outperform base judges on both absolute rating and relative preference datasets, achieving up to 6.93x speedups over fine-tuning while maintaining comparable or improved performance in metrics such as MSE, accuracy, and correlation metrics; for example, the LS judge achieved an MSE of 2.626 on the Summarize from Feedback dataset, significantly lower than the base judge’s 6.346. v) The framework provides AI practitioners with a computationally efficient and statistically robust alternative to fine-tuning LLMs for evaluation, especially useful when human feedback is limited. |
| BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM |
|
|
| Evaluation (Read more on arXiv or HuggingFace) |
Hitesh Patel, Guijin Son, Haneul Yoo, aliceoh, EunsuKim |
i) BENCHHUB is introduced as a benchmark repository for evaluating Large Language Models (LLMs) across diverse domains. ii) The research objective is to provide a unified, customizable, and scalable infrastructure for LLM evaluation tailored to specific needs or domains. iii) The methodology involves aggregating and classifying 303K questions from 38 existing benchmark datasets, categorizing them based on skills, subjects, and target types, and automating this process using a Qwen-2.5-7B-based model. iv) Experiments with various LLM families demonstrate significant performance variations across domain-specific subsets, and categorization errors up to 1.5% yield negligible disruption to model rankings. v) BENCHHUB offers AI practitioners a flexible platform for domain-aware benchmarking, enabling identification of underrepresented areas and facilitating more transparent model comparisons. |
| DLP: Dynamic Layerwise Pruning in Large Language Models (Read more on arXiv or HuggingFace) |
Yingting Li, Yingying Zhang, Jiale Han, Bo Cheng, yulichen |
i) The paper introduces Dynamic Layerwise Pruning (DLP), a novel method for unstructured pruning in large language models (LLMs). ii) The research aims to improve LLM pruning by adaptively determining layer importance, addressing the limitations of uniform and predefined layerwise pruning strategies. iii) DLP integrates model weights with input activation information, using the median to determine layer unimportance and allocate sparsity rates inversely proportional to importance. iv) Experiments show that at 70% sparsity, DLP reduces the perplexity of LLaMA2-7B by 7.79 and achieves up to 3.7x end-to-end acceleration on CPU, compared to state-of-the-art. v) DLP offers AI practitioners a compression technique compatible with PEFT and various existing LLM compression techniques, enabling efficient deployment of pruned LLMs in resource-constrained environments. |
| TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management |
|
|
| in LLM-based Agentic Multi-Agent Systems (Read more on arXiv or HuggingFace) |
Christos Emmanouilidis, Manoj Karkee, Ranjan Sapkota, shainar |
i) This paper reviews Trust, Risk, and Security Management (TRISM) for LLM-based agentic multi-agent systems (AMAS). ii) The research objective is to provide a structured analysis of TRISM in the context of the unique characteristics of agentic AI. iii) The methodology involves a systematic literature review across databases to identify relevant research and synthesize findings related to explainability, security, lifecycle governance, and privacy. iv) The paper identifies unique threat vectors and introduces a risk taxonomy for agentic AI, projecting a global market growth for AI agents from $5.4 billion in 2024 to $7.6 billion in 2025. v) For AI practitioners, the paper provides a roadmap to align emerging multi-agent systems with TRiSM principles for safe, accountable, and transparent deployment. |
| Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoning (Read more on arXiv or HuggingFace) |
Lei Zhang, Junzhi Yu, Zhaoyang Zeng, Xingyu Chen, Qing Jiang |
i) The paper introduces Rex-Thinker, a model for object referring that utilizes chain-of-thought reasoning. ii) The research aims to improve object referring by creating a grounded model that is both verifiable and trustworthy. iii) The methodology involves formulating object referring as a chain-of-thought reasoning task and constructing a large-scale dataset, HumanRef-CoT, to facilitate step-by-step reasoning. iv) Experiments show that Rex-Thinker achieves state-of-the-art performance on the HumanRef benchmark with improved accuracy and fewer hallucinated outputs, demonstrating a 13.8 point improvement in rejection score. v) The CoT-based approach improves interpretability and reduces hallucinations, providing a more robust and reliable object referring system for practical AI applications, especially those requiring high reliability. |
| Rethinking the Stability-Plasticity Trade-off in Continual Learning from |
|
|
| an Architectural Perspective (Read more on arXiv or HuggingFace) |
Yanan Sun, Tao Feng, JacobYuan, Kurt1024 |
i) This paper introduces Dual-Arch, a novel continual learning framework to address the stability-plasticity dilemma by leveraging dual architectures. ii) The research investigates how to balance stability and plasticity at the architectural level in continual learning. iii) The methodology employs distinct network architectures dedicated to either stability or plasticity, using knowledge distillation for knowledge transfer. iv) Experiments show Dual-Arch enhances existing CL methods’ performance, achieving up to 10.29% improvement in Last Accuracy, while reducing parameter counts by up to 87%. v) The study provides AI practitioners with a parameter-efficient architectural solution for improving continual learning models by independently allocating networks for stability and plasticity and transferring knowledge via distillation. |
| VLMs Can Aggregate Scattered Training Patches (Read more on arXiv or HuggingFace) |
Chaochao Lu, Chao Yang, Lingjie Chen, Zhanhui Zhou |
VLMs can inadvertently stitch together harmful visual information from distributed training data. The research investigates if vision-language models (VLMs) can integrate visual information scattered across multiple training samples with shared textual descriptions, enabling a threat model for bypassing data moderation. The study finetunes open-source VLMs on synthetic datasets consisting of image patches paired with text, evaluating the models’ ability to verbalize identifiers associated with the full images from either complete images or text references. Experiments demonstrate that VLMs exhibit strong image-based visual stitching; models finetuned on image patches with a split factor of 8 can verbalize IDs, while adversarial experiments show a 9% evasion rate against OpenAI moderation when using 8x8 patches. The findings imply that AI practitioners should develop moderation techniques that account for cross-sample reasoning to prevent the unintended aggregation of harmful content in VLMs. |
| Improving Knowledge Distillation Under Unknown Covariate Shift Through |
|
|
| Confidence-Guided Data Augmentation (Read more on arXiv or HuggingFace) |
Lukas Schott, Matthias Hein, Kevin Alexander Laube, Niclas Popp |
i) This paper introduces ConfiG, a confidence-guided data augmentation strategy for knowledge distillation under unknown covariate shift. ii) The main research question is whether a student model can become robust to unknown spurious features in a setting with covariate shift if a robust teacher model is available. iii) The methodology involves a diffusion-based data augmentation framework that generates images by maximizing the disagreement between teacher and student models. iv) Experiments on CelebA and SpuCo Birds demonstrate that ConfiG significantly improves worst group accuracy (e.g., achieving 66.1% on CelebA), and spurious mAUC on spurious ImageNet, outperforming diffusion-based data augmentation baselines. v) AI practitioners can use ConfiG to improve the robustness and generalization of student models distilled from large foundation models, particularly when deploying in environments with potential covariate shift and unknown spurious correlations. |
| Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic |
|
|
| Agents (Read more on arXiv or HuggingFace) |
Ryan A. Rossi, Nedim Lipka, Manan Suri, Franck-Dernoncourt, puneetm |
i) This paper introduces fine-grained flowchart attribution to improve the reliability and explainability of LLM responses in flowchart-based question answering. ii) The research aims to address the problem of visual hallucination in LLMs when interpreting flowcharts by tracing specific components that ground a referring LLM response. iii) The proposed FlowPathAgent uses a neurosymbolic approach that segments flowcharts, converts them into structured symbolic graphs, and employs an agentic approach for generating attribution paths. iv) Experiments on the newly introduced FlowExplainBench show that FlowPathAgent outperforms strong baselines by 10-14% in mitigating visual hallucinations in LLM answers over flowchart QA. v) FlowPathAgent’s neurosymbolic architecture provides AI practitioners with a verifiable and explainable method for processing flowcharts, enhancing decision-making reliability in critical applications. |
| Survey of Active Learning Hyperparameters: Insights from a Large-Scale |
|
|
| Experimental Grid (Read more on arXiv or HuggingFace) |
Maik Thiele, Claudio Hartmann, Anja Reusch, Tim Rieß, Julius Gonsior |
Survey of Active Learning Hyperparameters: Insights from a Large-Scale Experimental Grid analyzes the hyperparameter space of Active Learning (AL) to address reproducibility and adoption challenges. The study aims to quantify the impact of AL hyperparameters and provide guidelines for reliable experimental evaluations. The authors performed an extensive grid search over 4.6 million hyperparameter combinations, analyzing the influence of each parameter on AL performance. The study found that subsets of at least 4,000 combinations can produce results comparable to the complete grid, enabling computational efficiency. The analysis also showed that the implementation of AL strategies within different frameworks can greatly impact performance, more so than the choice of strategies, indicating the need for careful design. |
| Solving Inverse Problems with FLAIR (Read more on arXiv or HuggingFace) |
Jan Eric Lenssen, Bernt Schiele, Andreas Dombos, Dominik Narnhofer, juliuse |
FLAIR is introduced as a training-free variational framework integrating flow-based latent generative models for solving inverse imaging problems. The research aims to develop a variational objective for flow matching to leverage generative models as priors, incorporating deterministic trajectory adjustments for atypical modes and decoupled optimization for data consistency. The methodology involves a flow-matching loss aligned with learned velocity fields, hard data consistency steps projecting estimates onto the measurement manifold, and a time-dependent calibration scheme. Results on standard imaging benchmarks demonstrated FLAIR outperforms existing methods, achieving, for instance, a LPIPS of 0.213 on FFHQ for SR ×8 compared to Resample’s 0.400. FLAIR offers AI practitioners an improved, training-free approach to incorporating generative priors in inverse problems, enhancing reconstruction quality and sample diversity, though it inherits the limitations of its generative model backbone. |
| FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Rushil Thareja, Georgi Georgiev, Debopriyo Banerjee, Dhruv Sahnan, Zhuohan Xie |
i) The paper introduces FINCHAIN, a new symbolic benchmark for verifiable chain-of-thought (CoT) financial reasoning. ii) The main objective is to create a benchmark for systematically evaluating the ability of language models to perform multi-step financial reasoning with verifiable intermediate steps. iii) The methodology involves constructing 54 financial reasoning topics across 12 domains, each with five parameterized templates including executable Python traces for automatic data generation and the introduction of CHAINEVAL, a metric for evaluating both final answers and intermediate reasoning steps. iv) Results from benchmarking 30 LLMs on FINCHAIN indicate that even state-of-the-art models struggle with complex symbolic tasks and multi-step financial reasoning, with top models achieving 58% Final Answer Correctness (FAC). v) The principal implication is that further research is needed to improve the capacity of LLMs to handle symbolic and multi-hop inference for financial reasoning, as domain-specific fine-tuning alone is insufficient. |
Papers for 2025-06-04
| Title |
Authors |
Summary |
| CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning |
|
|
| Capabilities of VLMs (Read more on arXiv or HuggingFace) |
xuchensong, rockman24, jiangbopei, shawn0wang, qiuwj |
i) The paper introduces CSVQA, a Chinese multimodal benchmark for evaluating scientific reasoning in VLMs. ii) The research aims to comprehensively assess VLMs’ ability to integrate domain knowledge and visual evidence for scientific reasoning. iii) The methodology involves constructing a dataset of 1,378 STEM questions with curated explanations and evaluating 15 VLMs using a rigorous evaluation protocol. iv) The top-performing model achieved 49.6% accuracy, indicating a significant performance gap in scientific reasoning for current VLMs. v) The principal implication highlights the need for AI practitioners to focus on improving scientific reasoning capabilities in VLMs, especially concerning domain-specific knowledge integration and visual reasoning for complex, multimodal scenarios. The paper also proposes a process-tracing method for rigorous evaluation of reasoning ability. |
| UniWorld: High-Resolution Semantic Encoders for Unified Visual |
|
|
| Understanding and Generation (Read more on arXiv or HuggingFace) |
Yuwei Niu, Xinhua Cheng, Zongjian Li, BestWishYsh, LanguageBind |
i) UniWorld is a unified generative framework for image perception and manipulation based on semantic encoders. ii) The main research objective is to explore and develop a unified model for both image perception and manipulation tasks without relying on VAEs. iii) The methodology involves a unified architecture leveraging pre-trained visual-language models and contrastive semantic encoders. iv) UniWorld outperforms BAGEL on image editing benchmarks using only 1% of BAGEL’s training data (2.7M samples vs. 2665M samples) and achieves comparable performance on text-to-image generation benchmarks. v) The principal implication for AI practitioners is the demonstration of a unified architecture using semantic encoders for image tasks, offering a data-efficient alternative to VAE-based approaches. |
| VS-Bench: Evaluating VLMs for Strategic Reasoning and Decision-Making in |
|
|
| Multi-Agent Environments (Read more on arXiv or HuggingFace) |
Xinlei Chen, Xiangmin Yi, Zhexuan Xu, HuiningYuan, zelaix |
i) The paper introduces VS-BENCH, a new multimodal benchmark for evaluating Vision Language Models (VLMs) in strategic reasoning and decision-making within multi-agent environments. ii) The primary objective is to assess VLMs’ capabilities in strategic reasoning and decision-making in visually-rich, multi-agent interactive scenarios. iii) The methodology involves offline evaluation of strategic reasoning via next-action prediction accuracy and online evaluation of decision-making via normalized episode return across eight vision-grounded environments. iv) Experiments on fourteen leading VLMs revealed that the best models achieved a 47.8% next-action prediction accuracy and a 24.3% normalized episode return. v) The benchmark highlights a significant gap between existing VLMs and optimal performance in strategic multi-agent interactions, suggesting the need for improved visual information extraction and reasoning capabilities for AI practitioners developing multi-agent systems. |
| OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for |
|
|
| Vision Language Models (Read more on arXiv or HuggingFace) |
Xinqiang Yu, Wenyao Zhang, Shaochen Zhang, Mengdi Jia, qizekun |
i) This paper introduces OmniSpatial, a comprehensive spatial reasoning benchmark for vision-language models (VLMs). ii) The research aims to evaluate the spatial reasoning capabilities of VLMs across dynamic reasoning, complex spatial logic, spatial interaction, and perspective-taking. iii) The methodology involves constructing a dataset of over 1.5K question-answer pairs derived from internet data, standardized tests, and driving exam questions, followed by manual annotation and state-of-the-art VLM evaluation. iv) Results show that state-of-the-art VLMs peak at 57% accuracy on OmniSpatial, significantly below human performance and existing benchmarks, particularly struggling with geometric reasoning and non-egocentric perspective-taking. v) The implication for AI practitioners is a clear need for developing VLMs with enhanced spatial reasoning capabilities, specifically addressing identified limitations in geometric understanding and perspective-taking for robust real-world applications. |
| Visual Embodied Brain: Let Multimodal Large Language Models See, Think, |
|
|
| and Control in Spaces (Read more on arXiv or HuggingFace) |
Guanzhou Chen, Gen Luo, robot-haonan, Cusyoung, ganlinyang |
i) The paper introduces Visual Embodied Brain (VeBrain), a unified framework that enables Multimodal Large Language Models (MLLMs) to perceive, reason, and control robots in real-world environments. ii) The main objective is to unify multimodal understanding, visual-spatial reasoning, and physical interaction capabilities within a single MLLM for robotic control. iii) The methodology reformulates robotic control into text-based MLLM tasks in a 2D visual space and uses a robotic adapter to convert textual control signals into motion policies, training the system on a new VeBrain-600k dataset. iv) VeBrain achieves a +5.6% improvement on MMVet compared to Qwen2.5-VL and demonstrates a +50% average gain in legged robot tasks. v) The principal implication for AI practitioners is a framework to leverage MLLMs for enhanced adaptability, flexibility, and compositional capabilities in robotic applications, offering a practical architecture for integrating perception, reasoning, and control. |
| SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis (Read more on arXiv or HuggingFace) |
Hang Yan, Zichen Liu, Xiangyan Liu, Jinjie Ni, Jakumetsu |
i) SynthRL introduces a scalable pipeline for synthesizing verifiable data to enhance visual reasoning in VLMs trained with RLVR. ii) The research investigates whether synthesized RL data with correctness and distribution guarantees can improve the performance of VLMs. iii) SynthRL employs a three-stage process: seed question selection based on model difficulty, targeted variant synthesis using a powerful VLM, and a guaranteed verification step to ensure correctness and difficulty enhancement. iv) Experiments on the MMK12 dataset resulted in synthesizing over 3.3K additional verifiable questions from 8K seeds, achieving consistent gains across five out-of-domain visual math reasoning benchmarks, including a +1.9% improvement on MathVerse, reaching 53.5% average accuracy. v) AI practitioners can utilize SynthRL to automatically generate scalable, high-quality training data, improving the out-of-domain generalization and reasoning capabilities of VLMs, particularly on challenging examples. |
| MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Rui Xie, Kepan Nan, Tiehan Fan, Yipeng Du, yingtai |
i) MotionSight introduces a zero-shot prompting method and a large-scale dataset to improve fine-grained motion understanding in MLLMs. ii) The paper investigates how to unlock inherent capabilities and boost MLLMs’ motion perception by decoupling object and camera motion cues. iii) The methodology involves object-centric visual spotlighting, motion blur prompting, and the curation of the MotionVid-QA dataset with SFT and preference data. iv) Experiments show MotionSight achieves state-of-the-art open-source performance and competitiveness with commercial models, with a 3.4% improvement in category average on MotionBench. v) The MotionVid-QA dataset and MotionSight prompting techniques provide AI practitioners with resources and methods to enhance MLLM performance in tasks requiring nuanced motion understanding. |
| Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate |
|
|
| Video Diffusion Transformers (Read more on arXiv or HuggingFace) |
Maosen Zhao, Xianfang Zeng, skicy, wchengad, PengtaoChen |
i) The paper introduces Sparse-vDiT, a framework designed to accelerate Video Diffusion Transformers (vDiTs) by exploiting attention map sparsity. ii) The research aims to mitigate the high computational cost associated with the quadratic complexity of attention mechanisms in vDiTs for long sequence video generation. iii) The methodology involves identifying recurring sparsity patterns in vDiDT attention maps, developing pattern-optimized sparse kernels, and implementing an offline sparse diffusion search algorithm for optimal configuration. iv) Sparse-vDiT achieves up to 2.38x theoretical FLOP reduction and 1.85x inference speedup on HunyuanVideo while maintaining comparable generation quality (PSNR reaching 27.09). v) AI practitioners can leverage Sparse-vDiT’s sparsity exploitation techniques to improve the inference efficiency of vDiTs in video generation tasks without significantly compromising visual fidelity. |
| GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents (Read more on arXiv or HuggingFace) |
Jianwei Yang, vyokky, Ray2333, cckevinn, qianhuiwu |
GUI-Actor is a novel VLM-based method for GUI agent visual grounding that eliminates coordinate generation. The research aims to address limitations of coordinate-based visual grounding approaches in GUI agents by using a coordinate-free method that emphasizes alignment with visual patch tokens. GUI-Actor introduces an attention-based action head that learns to align an token with relevant visual patch tokens, enabling the model to propose action regions in a single forward pass and employs a grounding verifier to select the most plausible region. GUI-Actor-7B achieves scores of 40.7 with Qwen2-VL and 44.6 with Qwen2.5-VL, outperforming UI-TARS-72B (38.1) on ScreenSpot-Pro. The approach allows VLMs to endow effective GUI grounding capabilities without compromising their general-purpose strengths, making it relevant for AI engineers building GUI agents. |
| AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Ying Shan, Yixiao Ge, Yuying Ge, liyz, qiulu66 |
i) The paper introduces AnimeShooter, a new reference-guided multi-shot animation dataset to improve coherent video generation with character consistency. ii) The primary objective is to provide a dataset that addresses the limitations of existing video datasets for animation generation, specifically the lack of reference images and hierarchical annotations. iii) The dataset was built using an automated pipeline involving YouTube content collection, hierarchical story script generation via Gemini, and character segmentation with Sa2VA and InternVL. iv) The AnimeShooterGen model, trained on the dataset, demonstrates improved cross-shot visual consistency and character adherence, achieving a CLIP score of 0.8022 on Shot-1 for generated content aligned with reference images. v) The dataset and the AnimeShooterGen model offer AI practitioners a structured resource for training and evaluating video generation models capable of maintaining character identity and narrative coherence across multiple shots, essential for creating engaging animated content. |
| Native-Resolution Image Synthesis (Read more on arXiv or HuggingFace) |
Yiyuan Zhang, Wanli Ouyang, Xiangyu Yue, Lei Bai, GoodEnough |
i) This paper introduces Native-resolution image synthesis, a generative modeling paradigm for synthesizing images at arbitrary resolutions and aspect ratios. ii) The main objective is to overcome the limitations of conventional fixed-resolution, square-image methods. iii) The methodology involves a Native-resolution diffusion Transformer (NiT) architecture trained on ImageNet using dynamic tokenization, variable-length sequence processing with Flash Attention, and axial 2D Rotary Positional Embedding. iv) A single NiT model achieves a Fréchet Inception Distance (FID) of 1.45 on the ImageNet 512 × 512 benchmark, demonstrating state-of-the-art performance and an FID of 4.11 on novel 9:16 aspect ratio images. v) AI practitioners can leverage the NiT architecture for applications requiring flexible image generation across diverse resolutions and aspect ratios, improving generalization and reducing the need for resolution-specific models. |
| Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in |
|
|
| Robotics (Read more on arXiv or HuggingFace) |
Jaehyung Kim, Jinwoo Shin, Huiwon Jang, Sumin Park, Dongyoung Kim |
i) The paper introduces ROBOT-R1, a reinforcement learning framework to improve embodied reasoning for robot control in Large Vision-Language Models (LVLMs). ii) The main objective is to enhance LVLMs’ embodied reasoning capabilities specifically for robotic control tasks, addressing limitations of Supervised Fine-Tuning (SFT). iii) ROBOT-R1 uses reinforcement learning to train LVLMs to predict the next keypoint state for task completion, conditioned on scene images and metadata, reformulating the problem as a multiple-choice question answering task. iv) Experiments show that models trained with ROBOT-R1 achieve over a 28% improvement in embodied reasoning for low-level action control compared to SFT methods; also showed a 31% improvement in task performance on EmbodiedBench Manipulation. v) The implication for AI practitioners is a novel method, ROBOT-R1, that enhances the reasoning of LVLMs for robotics-related tasks. |
| Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Mengdi Wang, Ke Shen, Ye Tian, Ling Yang, Yinjie Wang |
i) The paper introduces CURE, a reinforcement learning framework for co-evolving LLM coders and unit testers without ground-truth code supervision. ii) The research question focuses on whether a code generator and unit test generator can co-evolve without relying on ground-truth code solutions to improve LLM coding ability. iii) The methodology involves a self-play setup where a single model acts as both code generator and unit test generator, with a pairwise reward matrix based on the interactions between the generated code and unit tests. iv) After optimization on Qwen2.5-Instruct models, the ReasonFlux-Coder 7B and 14B models demonstrate a 5.3% improvement in code generation accuracy and a 9.0% improvement in Best of N accuracy. v) This research implies that AI practitioners can enhance code generation capabilities in LLMs through a co-evolutionary reinforcement learning approach that leverages unit tests for self-supervision, potentially reducing the need for labeled data. |
| LumosFlow: Motion-Guided Long Video Generation (Read more on arXiv or HuggingFace) |
Jiazheng Xing, Jingyun Liang, Yichen Qian, Hangjie Yuan, Jiahao Chen |
LumosFlow introduces a motion-guided framework for generating temporally coherent long videos. The research aims to improve long video generation by incorporating explicit motion guidance. The methodology involves generating keyframes using a Large Motion Text-to-Video Diffusion Model (LMTV-DM) and interpolating intermediate frames with a Latent Optical Flow Diffusion Model (LOF-DM) and Motion ControlNet. Experiments show the method achieves 15× interpolation while maintaining consistent motion and appearance. LumosFlow provides AI practitioners with a novel hierarchical long video generation pipeline leveraging motion guidance for enhanced temporal coherence, potentially reducing artifacts in synthesized videos. |
| DINGO: Constrained Inference for Diffusion LLMs (Read more on arXiv or HuggingFace) |
Gagandeep Singh, Sasa Misailovic, Shubham Ugare, Debangshu Banerjee, Tarun Suresh |
i) The paper introduces DINGO, a dynamic programming-based constrained decoding algorithm for diffusion language models. ii) The main objective is to develop a decoding method for diffusion LLMs that can enforce user-specified regular expression constraints while preserving the output distribution. iii) DINGO utilizes dynamic programming to find the maximum probability output string that adheres to the constraints, modifying the DFA transition function to account for mask tokens. iv) Experiments on symbolic math and JSON generation tasks show that DINGO achieves up to a 68% improvement over unconstrained inference in terms of syntactic correctness. v) DINGO provides AI practitioners with a reliable method for generating structured outputs from diffusion LLMs, enabling the use of these models in applications requiring formal guarantees such as symbolic reasoning and schema-based data generation. |
| RelationAdapter: Learning and Transferring Visual Relation with |
|
|
| Diffusion Transformers (Read more on arXiv or HuggingFace) |
Yin Zhang, Chenglin Li, Yicheng Li, Yiren Song, Yan Gong |
RelationAdapter: A new module for diffusion transformers to transfer visual relationships from paired images. The paper addresses the research question of how to effectively extract and transfer content-aware editing intent from exemplar image pairs to novel query images for visual prompt-based image editing. The proposed method uses a RelationAdapter module, a lightweight dual-branch adapter within a Diffusion Transformer (DiT), to explicitly model visual relationships between pre-edit and post-edit images. Experiments on a new Relation252K dataset show RelationAdapter significantly improves the model’s ability to understand and transfer editing intent, achieving a lower MSE of 0.020 and a higher CLIP-I score of 0.905 compared to the Edit Transfer baseline. AI practitioners can use RelationAdapter to improve the performance and generalizability of visual prompt-based image editing systems by leveraging paired image examples. |
| FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation (Read more on arXiv or HuggingFace) |
Jinsheng Huang, Xiao Luo, chunfenri, alan1027, luojunyu |
i) FinMME introduces a benchmark dataset for evaluating financial multi-modal reasoning in large language models (MLLMs). ii) The research aims to address the lack of specialized evaluation datasets in the financial domain to advance MLLM development for financial applications. iii) The study involved curating a dataset of over 11,000 financial samples, developing a hierarchical evaluation framework, and introducing FinScore, an evaluation metric that penalizes hallucination. iv) Experiments showed that even advanced models like GPT-4o achieve performance of just over 50%, and FinMME demonstrated high robustness with prediction variations under different prompts remaining below 1%. v) AI practitioners should use FinMME to rigorously evaluate and improve MLLMs for financial tasks, emphasizing the importance of hallucination control for reliable financial analysis. |
| PCoreSet: Effective Active Learning through Knowledge Distillation from |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Dongseop Kim, Hyungjoon Jang, Dong Bok Lee, Seongjae Kang |
i) This paper introduces ActiveKD, an active learning framework using knowledge distillation from vision-language models, and proposes Probabilistic CoreSet (PCoreSet) for sample selection. ii) The research investigates how to effectively integrate knowledge distillation with active learning by leveraging zero- and few-shot capabilities of VLMs in data-scarce scenarios. iii) The methodology uses PCoreSet, a selection strategy that maximizes coverage in the probability space of VLM predictions to select categorically diverse unlabeled samples. iv) Experiments on 11 datasets show that ActiveKD improves final-round accuracy and PCoreSet outperforms existing methods, achieving a 27.33% improvement on ImageNet with random selection using zero-shot distillation. v) ActiveKD and PCoreSet provide AI practitioners with a method to train compact, task-specific models more efficiently with limited labeled data by exploiting the inductive bias of VLMs. |
| OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for |
|
|
| Over-Reasoning Mitigation (Read more on arXiv or HuggingFace) |
Changwang Zhang, Jiawei Chen, Junjie Wu, jwanglux, Cynthia-1628 |
OThink-R1 enables dynamic switching between fast and slow thinking modes in large reasoning models (LRMs) to mitigate over-reasoning. The research aims to address the computational inefficiency of LRMs on simple tasks by adaptively engaging explicit reasoning only when necessary. It analyzes reasoning trajectories to classify them as redundant or essential using an LLM-Judge and constructs a supervised fine-tuning (SFT) dataset. Experiments show OThink-R1 reduces token generation by 23.4% on average across QA and math datasets without compromising accuracy. This offers AI practitioners practical guidelines for developing more efficient and scalable reasoning models. |
| FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Lior Wolf, Ariel Shaulov, Hila, itayhzn |
i) FlowMo is introduced as a training-free guidance method for enhancing temporal coherence in text-to-video diffusion models. ii) The research aims to improve motion coherence in generated videos without additional training data or conditioning signals. iii) The method involves measuring patch-wise variance of appearance-debiased latent representations over time, guiding the model to reduce this variance during sampling. iv) Experiments on Wan2.1-1.3B and CogVideoX-5B show FlowMo improves the Final Score (representing overall video quality) by 6.20% and 5.26%, respectively, on the VBench benchmark. v) FlowMo offers AI practitioners a plug-and-play solution to enhance the temporal fidelity of pre-trained video diffusion models without retraining, enabling the generation of more coherent videos from existing models. |
| Datasheets Aren’t Enough: DataRubrics for Automated Quality Metrics and |
|
|
| Accountability (Read more on arXiv or HuggingFace) |
David Anugraha, Genta Indra Winata, cryptexcode, seungone, patrickamadeus |
This paper introduces DATARUBRICS, a structured framework for automated dataset quality assessment in machine learning. The research question addresses the need for systematic and quantifiable metrics to evaluate dataset quality, moving beyond descriptive datasheets. The authors propose DATARUBRICS, a rubric-based framework with ten dimensions of data quality assessed via human evaluation and LLM-as-a-judge approaches. The evaluation involved annotating 100 NeurIPS papers and analyzing data quality trends across NLP, CV, ML, and speech conferences, finding that human annotations still contain errors even after quality assurance with 26% of annotations remaining incorrect despite quality assurance. DATARUBRICS offers a reproducible, scalable solution for both dataset authors and reviewers, aiding in upholding higher data-centric research standards. |
| ReFoCUS: Reinforcement-guided Frame Optimization for Contextual |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Yong Man Ro, Hyunjun Kim, arkimjh, lakelee |
i) This paper introduces ReFoCUS, a reinforcement learning framework for optimizing frame selection in video-LLMs to improve contextual understanding. ii) The main objective is to develop a frame selection policy that aligns with a model’s intrinsic visual preferences for temporally grounded responses. iii) ReFoCUS utilizes a reference LMM to generate reward signals for frame subsets, training an autoregressive frame selection policy via reinforcement learning. iv) The experimental results demonstrate that ReFoCUS consistently improves reasoning performance across video QA benchmarks without frame-level supervision. For instance, it significantly enhances performance on the long subset of Video-MME. v) ReFoCUS offers AI practitioners a model-agnostic approach to enhance video-LLMs by optimizing visual input selection, improving reasoning capabilities, particularly in complex, multi-event scenarios where information is sparse. |
| Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Kiran Kamble, Christopher Bryant, Umar Jamil, Shelly Bensal, melisa |
i) The paper introduces a reinforcement learning framework for improving large language models (LLMs) through self-reflection on failed tasks. ii) The research investigates if LLMs can learn to generate better self-reflections to improve performance on downstream tasks using reinforcement learning without task-specific data. iii) The methodology involves prompting the LLM to generate a self-reflection upon failure, retrying the task with the reflection context, and rewarding tokens in the reflection using Group Relative Policy Optimization (GRPO) if the retry succeeds. iv) Experiments on the APIGen function calling dataset show a performance increase of up to 34.7% in math equation writing and 18.1% in function calling; smaller fine-tuned models outperformed 10x larger models. v) This work enables AI practitioners to improve LLM performance on complex tasks with binary success/failure feedback by optimizing self-reflection without requiring task-specific training datasets or synthetic data generation. |
| ORV: 4D Occupancy-centric Robot Video Generation (Read more on arXiv or HuggingFace) |
Chongjie Ye, Nan Wang, Shaocong Xu, Bohan Li, gzzyyxy |
ORV is a novel framework for generating action-conditioned robot manipulation videos guided by 4D semantic occupancy. The paper addresses the challenge of generating high-fidelity, controllable robot manipulation videos. The methodology involves using 4D semantic occupancy sequences as a fine-grained representation to guide video generation, along with a curated high-quality occupancy dataset. Experiments show ORV achieves superior performance compared to existing methods, demonstrated by a PSNR increase from 25 to 28 when incorporating physical constraints. This occupancy-centric approach offers AI practitioners a more precise and controllable method for synthesizing realistic robot videos, potentially improving simulation and robot learning. |
| FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens (Read more on arXiv or HuggingFace) |
Matthias Hein, Nicolas Flammarion, Francesco Croce, chs20 |
FuseLIP constructs multimodal embeddings via early fusion of discrete text and image tokens using a single transformer encoder. The primary objective is to encode multimodal inputs while maintaining vision-language alignment and zero-shot capabilities. The methodology involves tokenizing images with a discrete image tokenizer and concatenating these with text tokens for processing by a single transformer model, trained with a contrastive loss and masked multimodal modeling. FuseLIP surpasses late fusion methods on tasks that involve encoding image-text pairs achieving an accuracy of 94.3% on text-guided image transformations while being comparable on unimodal tasks. FuseLIP’s architecture and training approach offer AI practitioners an effective method for building multimodal embeddings that enhance performance on multimodal understanding tasks compared to traditional late fusion methods. |
| Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports |
|
|
| From Scratch with Agentic Framework (Read more on arXiv or HuggingFace) |
Xingyu Liu, Yiyao Wang, Han Wang, Bo Pan, Zhaorui Yang |
i) This paper introduces Multimodal DeepResearcher, an agentic framework for generating text-chart interleaved reports from scratch. ii) The main research objective is to automate the generation of comprehensive reports that integrate textual content with diverse, high-quality visualizations. iii) The methodology involves a four-stage agentic framework: researching, exemplar report textualization using Formal Description of Visualization (FDV), planning, and multimodal report generation with iterative chart refinement using actor-critic mechanism. iv) Experimental results demonstrate that Multimodal DeepResearcher, using the Claude 3.7 Sonnet model, achieves an 82% overall win rate compared to the DataNarrative baseline in automatic evaluations. v) The principal implication for AI practitioners is a new approach to automate the generation of informative, multimodal reports, which relies on structured visualization representations (FDV) and agentic workflows for iterative refinement of generated content. |
| One Missing Piece for Open-Source Reasoning Models: A Dataset to |
|
|
| Mitigate Cold-Starting Short CoT LLMs in RL (Read more on arXiv or HuggingFace) |
Sunghyun Park, Beong-woo Kwak, Jihyuk Kim, Dongjin Kang, hyungjoochae |
i) The paper introduces the Long CoT Collection dataset to improve the reasoning capabilities of short chain-of-thought (CoT) language models (LLMs) in reinforcement learning (RL). ii) The research investigates the feasibility of constructing long CoT data using LLMs not trained for inference-time scaling to address the cold-start problem in RL. iii) The methodology involves a pipeline to induce novel reasoning strategies into short CoT LLMs, creating a 100K instance dataset annotated with existing short CoT LLMs. iv) Experiments show that models initialized on the Long CoT Collection achieve 2-3x larger performance gains with RL with verifiable reward (RLVR) on MATH500 and GPQA; the dataset’s quality is comparable to R1. v) The Long CoT Collection offers AI practitioners a reliable foundation for initializing SFT models for reinforcement learning, accelerating and stabilizing downstream learning in reasoning tasks. |
| Accelerating Diffusion LLMs via Adaptive Parallel Decoding (Read more on arXiv or HuggingFace) |
Aditya Grover, Guy Van den Broeck, danielmisrael |
i) This paper introduces adaptive parallel decoding (APD) to accelerate diffusion large language models (dLLMs). ii) The research aims to improve the text generation speed of dLLMs without significantly sacrificing quality, addressing the bottleneck of autoregressive decoding. iii) APD dynamically adjusts the number of tokens sampled in parallel by defining a multiplicative mixture between the dLLM marginal probabilities and the joint probability of sequences under a small auxiliary autoregressive model, incorporating KV caching and limiting masked input size. iv) Experiments show that APD achieves markedly higher throughput with minimal quality degradations on downstream benchmarks; for example, a Dream 7B model using APD maintained ~80% accuracy on GSM8K while generating over 5 tokens per iteration. v) The APD method provides AI practitioners with tunable parameters to flexibly tradeoff throughput and quality in dLLM inference, offering a more efficient alternative for fast text generation. |
| R^2ec: Towards Large Recommender Models with Reasoning (Read more on arXiv or HuggingFace) |
Wenjie Wang, Xinyu Lin, izhx, tensorslow, dd101bb |
i) The paper introduces R²ec, a unified large recommender model with interleaved reasoning and recommendation. ii) The main objective is to develop a unified architecture that intrinsically integrates reasoning capabilities within a large recommender model, avoiding decoupled designs. iii) The methodology involves an autoregressive decoder-only backbone with a language-modeling head for reasoning and a recommendation head for item prediction, optimized using a Reinforcement Learning framework, RecPO. iv) Experiments show R²ec achieves relative improvements of 68.67% in Hit@5 and 45.21% in NDCG@20 compared to baselines on three datasets. v) The principal implication is that interleaving reasoning and recommendation within a single model architecture, optimized with reinforcement learning, offers a path to significantly improve recommendation performance compared to decoupled or LLM-augmented approaches, potentially reducing resource costs and optimizing joint training. |
| MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition |
|
|
| Query (Read more on arXiv or HuggingFace) |
Qi Xu, Xian Wang, Linfeng Li, Yuan Gao, WeiChow |
i) MERIT is introduced as a multilingual dataset for interleaved multi-condition semantic retrieval. ii) The research aims to address the underexplored area of semantic retrieval involving composite multi-condition queries with multiple images and multilingual text. iii) A novel fine-tuning framework, CORAL, is proposed, integrating embedding reconstruction to preserve fine-grained conditional elements and contrastive learning for comprehensive global semantics. iv) Experiments demonstrate that CORAL achieves a 45.9% performance improvement over conventional approaches on MERIT. v) CORAL offers AI practitioners a method for enhancing MLLMs in semantic retrieval tasks by preserving conditional elements and extracting comprehensive global semantics, thus potentially improving the accuracy of retrieval systems. |
| M^3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial |
|
|
| Meeting Understanding Evaluation Dataset (Read more on arXiv or HuggingFace) |
Lifan Guo, Xiandong Li, Yalong Wen, Junhui Li, amazingj |
i) The paper introduces M³FinMeeting, a new multilingual, multi-sector, multi-task dataset for evaluating LLMs in financial meeting understanding. ii) The main objective is to address the gap in existing financial benchmarks by providing real-world financial meeting data across multiple languages and industry sectors. iii) The key methodology involves curating financial meeting transcripts in English, Chinese, and Japanese, and annotating them for summarization, question-answer pair extraction, and question answering tasks, with annotation quality ensured by financial analysts. iv) Experimental results with seven LLMs, including OpenAI GPTs and open-sourced models, reveal that Qwen2.5-72B-Instruct achieves overall scores above 70 when evaluated by GPT-4, while recall of QA extraction is low at 45.65%, leaving significant room for improvement. v) The principal implication for AI practitioners is the availability of a challenging benchmark for assessing and improving LLMs’ ability to process complex, long-context financial meeting data, facilitating more effective applications in financial decision-making. |
| QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large |
|
|
| Language Model Adaptation (Read more on arXiv or HuggingFace) |
Omar Elshehy, Mahmoud Reda, Abdelakreem Elkhateb, Omer Nacar, oddadmix |
i) QARI-OCR is presented as a series of vision-language models optimized for Arabic text recognition via fine-tuning Qwen2-VL-2B-Instruct. ii) The primary objective is to improve the accuracy and efficiency of Arabic OCR, specifically in handling diacritics, diverse fonts, and complex layouts. iii) The methodology involves generating synthetic datasets and iteratively fine-tuning the Qwen2-VL-2B-Instruct model. iv) QARI v0.2 achieves a Word Error Rate (WER) of 0.160, Character Error Rate (CER) of 0.061, and BLEU score of 0.737 on diacritically-rich texts. v) QARI-OCR provides AI practitioners with an improved Arabic OCR model exhibiting enhanced performance in recognizing intricate Arabic script, thus improving cultural heritage preservation, scholarly research, and information access. |
| Knowing Before Saying: LLM Representations Encode Information About |
|
|
| Chain-of-Thought Success Before Completion (Read more on arXiv or HuggingFace) |
Florian Matthes, yziser, galchechik, anumafzal94 |
i) This paper explores predicting the success of Chain-of-Thought (CoT) reasoning in LLMs before completion by probing internal representations. ii) The main objective is to determine if LLMs implicitly encode information indicative of CoT success in their internal representations prior to generating the full reasoning chain. iii) The methodology involves training a probing classifier on the LLM’s hidden states at different stages of CoT generation and comparing its performance to a BERT-based baseline relying on generated tokens alone. iv) The primary result is that a classifier using LLM internal representations can predict CoT success with 60% to 76.4% accuracy even before token generation, outperforming a BERT baseline, indicating crucial information is present in initial steps; however, the utility of later reasoning steps toward classification accuracy is variable. v) The principal implication for AI practitioners is that LLM representations contain early signals for CoT success, suggesting potential for early stopping or adaptive allocation of computational resources in CoT reasoning. Some parts of the methodology were unclear, such as the source for labeled training sets, and as such the details of this are unknown. |
| How Much Backtracking is Enough? Exploring the Interplay of SFT and RL |
|
|
| in Enhancing LLM Reasoning (Read more on arXiv or HuggingFace) |
Bhuwan Dhingra, Junlin Wang, chenyn66, jamescai20 |
i) This paper explores the interplay of supervised finetuning (SFT) and reinforcement learning (RL) in enhancing reasoning abilities of large language models (LLMs), focusing on the role of backtracking. ii) The research question is how the extent and structure of backtracking in SFT data impact subsequent RL training performance on reasoning tasks. iii) The methodology involves controlled experiments with synthetic datasets on eight reasoning tasks, systematically varying the number of backtracking steps in SFT data used to initialize RL training. iv) Results indicate that longer chain-of-thought (CoT) demonstrations with backtracks generally lead to better RL training, with more challenging problems requiring higher numbers of backtracks during SFT; SFT using correct or incorrect QwQ-32B distillation data converged in performance during RL, and RL with one backtrack initialization attained 69.7% accuracy (Figure 4d), outperforming QwQ-32B 51.5%. v) The primary implication for AI practitioners is that incorporating synthetic SFT data with a number of backtracks matched to the problem difficulty can improve RL training efficiency for LLM reasoning, while data correctness in SFT may be less critical. |
| Deep Video Discovery: Agentic Search with Tool Use for Long-form Video |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Bin Li, Jiahao Li, Zongyu Guo, Zhaoyang Jia, Xiaoyi Zhang |
Deep Video Discovery introduces an agentic search strategy with tool use for long-form video understanding. The paper aims to address the limitations of LLMs in processing information-dense long videos by developing an agent that autonomously searches segmented video clips. The methodology involves creating a video database with multi-granular information and search-centric tools like Global Browse, Clip Search, and Frame Inspect. The DVD agent achieves state-of-the-art performance on LVBench, reaching an accuracy of 74.2% and improves to 76.0% with transcripts. This demonstrates that autonomous agentic search strategies with tool use can substantially improve long-form video understanding. |
| Revisiting LRP: Positional Attribution as the Missing Ingredient for |
|
|
| Transformer Explainability (Read more on arXiv or HuggingFace) |
Lior Wolf, Hila Chefer, Itamar Zimerman, Yarden Bakish |
i) The paper introduces positional-aware LRP (PA-LRP) for transformer explainability, addressing the omission of positional encoding attribution in existing LRP methods. ii) The research aims to improve the fidelity and comprehensiveness of transformer explanations by incorporating positional encoding into LRP. iii) The methodology involves reformulating the input space as position-token pairs and developing specialized LRP rules for different positional encoding methods (Rotary, Learnable, Absolute). iv) Experiments on fine-tuned classifiers and zero-shot foundation models demonstrate that PA-LRP significantly outperforms state-of-the-art methods, achieving a 14.5% improvement in AU-MSE score on the generation task for LLaMa-2 7B finetuned on IMDB. v) The principal implication is that AI practitioners should consider positional encodings when using LRP for transformer explainability, as PA-LRP provides a more faithful representation of model reasoning, enabling improved model debugging and trust. |
| Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural |
|
|
| Understanding and Transcreation (Read more on arXiv or HuggingFace) |
Wenyan Li, Shaohuan Cheng, Dongchu Xie, Lutong Yu, Li Zhou |
i) The paper introduces Hanfu-Bench, a new multimodal dataset for evaluating cultural understanding and creative adaptation of Vision-Language Models (VLMs) across temporal dimensions. ii) The main objective is to assess VLMs’ ability to understand temporal-cultural features of traditional Chinese Hanfu and transcreate them into modern designs. iii) The methodology involves two core tasks: cultural visual understanding via multiple-choice VQA and cultural image transcreation evaluated through multi-faceted human assessment. iv) Results show that the best-performing model achieves a success rate of only 42% in the transcreation task, while closed VLMs perform comparably to non-experts in VQA but fall short of expert human performance by 10%. v) The principal implication is that current VLMs exhibit limitations in capturing and adapting temporal cultural nuances, requiring AI practitioners to develop models capable of more nuanced understanding and creative application in culturally-sensitive contexts. |
Papers for 2025-06-03
| Title |
Authors |
Summary |
| Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective |
|
|
| Reinforcement Learning for LLM Reasoning (Read more on arXiv or HuggingFace) |
lyq333, Zhenru, xionghuichen, chujiezheng, shenzhi-wang |
i) The paper identifies high-entropy minority tokens in Chain-of-Thought reasoning as critical forks that drive effective RLVR. ii) The research aims to understand the mechanisms of RLVR through the lens of token entropy patterns and improve RLVR performance. iii) The methodology involves analyzing token entropy patterns in CoT reasoning and restricting policy gradient updates to forking tokens during RLVR training. iv) Results show that restricting RLVR training to the top 20% of high-entropy tokens achieves comparable or superior performance to full-gradient updates, with a +11.04 improvement on AIME’25 and +7.71 on AIME’24 for a Qwen3-32B model. v) AI practitioners can leverage high-entropy minority tokens to optimize RLVR training for LLM reasoning, potentially reducing computational costs and improving performance. |
| REASONING GYM: Reasoning Environments for Reinforcement Learning with |
|
|
| Verifiable Rewards (Read more on arXiv or HuggingFace) |
Richard Jones, Joe Sharratt, JeanKaddour, OllieStanley, zafstojano |
i) The paper introduces REASONING GYM (RG), a diverse library of reasoning environments with verifiable rewards for reinforcement learning (RL) of reasoning models. ii) The main objective is to provide a scalable and controllable training environment that alleviates the data scarcity bottleneck faced by current RL-based reasoning models. iii) RG uses procedural generation to create over 100 distinct data generators and verifiers across various domains including algebra, geometry, and logic, enabling adjustable complexity and automatic reward mechanisms. iv) Experiments reveal that frontier LLMs exhibit low zero-shot performance on many RG tasks, with difficulty cliffs causing performance drops of up to 62% in code generation; RLVR training on RG tasks improves performance on external benchmarks like MATH by 9.7% for Qwen2.5-3B-Instruct. v) The RG library and RLVR training can be used by AI practitioners to systematically evaluate and improve the reasoning capabilities of language models via RL, addressing current limitations in reasoning benchmarks. |
| Taming LLMs by Scaling Learning Rates with Gradient Grouping (Read more on arXiv or HuggingFace) |
danxu, MarcusB3n, ZedongWangAI, Juanxi, Lupin1998 |
i) This paper introduces Scaling with Gradient Grouping (SGG), an optimizer wrapper for improving large language model (LLM) training. ii) The primary objective is to enhance adaptive learning rate estimation in LLMs to mitigate training instability and improve convergence. iii) SGG dynamically clusters gradient statistics within each layer and applies cluster-specific scaling to learning rates. iv) Experiments on C4 pre-training demonstrated that Adam combined with SGG surpassed recent optimizers across model sizes (60M to 1B), and low-rank pre-training with SGG yielded up to 30.4% lower validation perplexity over LoRA baselines. v) SGG offers AI practitioners a robust and easily integrated method for improving LLM training stability, convergence, and performance across various fine-tuning scenarios without architecture modification. |
| Temporal In-Context Fine-Tuning for Versatile Control of Video Diffusion |
|
|
| Models (Read more on arXiv or HuggingFace) |
Jaegul Choo, Junha Hyung, Kinam Kim |
Temporal In-Context Fine-Tuning (TIC-FT) is introduced as a method for conditional video diffusion models. The main research objective is to adapt pre-trained video diffusion models to diverse conditional generation tasks efficiently. The key methodology involves temporally concatenating condition and target frames with intermediate buffer frames of increasing noise levels, fine-tuning the model using as few as 10-30 samples without architectural modifications. TIC-FT achieves strong performance, evidenced by its superior condition alignment and generation quality across various tasks, and requires less than one hour of training time for CogVideoX-5B on a single A100 GPU with 20 training samples over 6,000 steps. This approach implies AI practitioners can adapt large video diffusion models to new conditional tasks with minimal data and computational resources while maintaining condition fidelity. |
| Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with |
|
|
| Jigsaw Puzzles (Read more on arXiv or HuggingFace) |
Feiyu Xiong, Zhiyu Li, Bo Tang, RyanZhu, wangzifu |
i) This paper investigates rule-based visual reinforcement learning (RL) using jigsaw puzzles as a structured framework for multimodal large language models (MLLMs). ii) The main objective is to study how MLLMs perform on rule-based visual RL tasks and whether training on jigsaw puzzles can generalize to other visual tasks. iii) The methodology involves training MLLMs with rule-based RL on jigsaw puzzles of varying complexities and assessing their performance on both jigsaw puzzles and downstream vision tasks. iv) Results show that MLLMs can achieve near-perfect accuracy on jigsaw puzzles after fine-tuning, generalizing to unseen configurations, and RL exhibits better generalization than supervised fine-tuning (SFT). v) The research implies that rule-based visual RL with jigsaw puzzles can enhance MLLMs’ visual reasoning capabilities, which are generalizable for downstream vision tasks, though an initial SFT phase may hinder subsequent RL optimization. |
| SmolVLA: A Vision-Language-Action Model for Affordable and Efficient |
|
|
| Robotics (Read more on arXiv or HuggingFace) |
imstevenpmwork, pepijn223, fracapuano, danaaubakirova, mshukor |
SmolVLA presents a compact and efficient vision-language-action model for robotics. The research addresses the high computational cost of existing VLAs, aiming for affordable and efficient robotics. It employs a compact pretrained VLM with flow matching-trained action expert, asynchronous inference, and training on community-contributed datasets. SmolVLA achieves comparable performance to larger VLAs while reducing training and inference costs, with approximately 40% faster training time and 6x less memory consumption than a 3.3 billion parameter baseline. This research offers a resource-efficient VLA architecture for AI practitioners, enabling deployment on consumer-grade hardware. |
| ARIA: Training Language Agents with Intention-Driven Reward Aggregation (Read more on arXiv or HuggingFace) |
Siyu Yuan, Yikai Zhang, Xintao, sheep33333, rhyang2021 |
ARIA introduces a method for training language agents in open-ended environments by aggregating rewards in intention space. The research aims to address reward sparsity in reinforcement learning for language agents by projecting actions into a lower-dimensional intention space. Hierarchical clustering of sentence embeddings is used to create an intention space where semantically similar actions share rewards, reducing reward variance. Experiments show ARIA reduces policy gradient variance and improves performance by an average of 9.95% across four tasks compared to baseline methods. This method provides AI practitioners with a technique for improving RL-based language agent training by densifying reward signals through intention-aware aggregation, fostering better policy optimization. |
| LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon |
|
|
| Embodied Tasks (Read more on arXiv or HuggingFace) |
Zhijie Deng, Yihan Wang, Siqi Kou, Jiaxuan Sun, Yysrc |
LoHoVLA introduces a unified vision-language-action model for long-horizon embodied tasks, integrating high-level planning and low-level control. The research aims to improve performance on complex, multi-step robotic tasks by addressing limitations in existing VLA models and hierarchical architectures. LoHoVLA leverages a large pretrained VLM backbone, generating both language and action tokens, and employs a hierarchical closed-loop control mechanism for error mitigation. Experiments on the LoHoSet dataset demonstrate that LoHoVLA achieves a significantly higher success rate, up to 97.8%/91.5% on seen tasks compared to baseline methods in the Ravens simulator. The findings suggest that unified architectures, as opposed to modular structures, show promise for advancing generalizable embodied intelligence, directly benefiting AI practitioners working on robotics. The paper is unclear regarding the specific implementation details of the closed-loop control mechanism and the architecture of the base VLM. |
| Learning Video Generation for Robotic Manipulation with Collaborative |
|
|
| Trajectory Control (Read more on arXiv or HuggingFace) |
Runsen Xu, Jianhong Bai, Xian Liu, Xintao Wang, Xiao Fu |
i) The paper introduces RoboMaster, a novel video generation framework for robotic manipulation that uses collaborative trajectory control. ii) The research aims to improve the visual fidelity of generated robotic manipulation videos by addressing feature entanglement issues. iii) RoboMaster decomposes the interaction process into pre-interaction, interaction, and post-interaction phases, each guided by the dominant agent and uses a collaborative trajectory formulation and object embeddings for consistency. iv) Experiments on the Bridge V2 dataset demonstrate that RoboMaster outperforms existing methods, achieving a trajectory error of 16.47 for the robot and 24.16 for the object, alongside state-of-the-art visual quality metrics. v) RoboMaster’s collaborative trajectory control provides AI practitioners with a new method for generating high-quality robotic manipulation data, enabling more realistic and controllable simulation environments. |
| ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Jun Zhu, Shenghao Xie, Zhengyi Wang, Junliang Ye, zzzrw |
ShapeLLM-Omni is introduced as a native 3D multimodal large language model for understanding and generating 3D assets and text. The research aims to extend multimodal LLMs with 3D capabilities using a next-token prediction paradigm. A 3D VQVAE is trained to encode 3D meshes into discrete tokens, and a large-scale dataset, 3D-Alpaca, is constructed for continuous training, incorporating generation, comprehension, and editing tasks. The Qwen-2.5-vl-7B-Instruct model is instruction-tuned on the 3D-Alpaca dataset, and the model achieves a CLIP score of 84.5 in image-to-3D tasks. The resulting model enables new avenues for AI practitioners to unify text, images, and 3D data processing within a single architecture, though more work is needed to reach the level of a true “3D version of ChatGPT-40”. |
| SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Dongfei Cui, Yu Zhang, Che Liu, Zhihao Dou, Zhongwei Wan |
i) The paper introduces SRPO, a two-stage reflection-aware reinforcement learning framework for enhancing multimodal reasoning in large language models (MLLMs). ii) The research aims to improve MLLM reasoning accuracy and reflection quality, particularly in complex tasks requiring self-correction. iii) The methodology involves constructing a reflection-focused dataset using an advanced MLLM and incorporating a Group Relative Policy Optimization (GRPO) framework with a novel reward mechanism that encourages concise and cognitively meaningful reflection. iv) Experiments using Qwen-2.5-VL-7B and Qwen-2.5-VL-32B show SRPO significantly outperforms state-of-the-art models, achieving a 75.8% accuracy on MathVista using Qwen-2.5-VL-7B. v) SRPO provides AI practitioners with a method for enhancing MLLMs by integrating explicit self-reflection and self-correction mechanisms, improving their reasoning accuracy and reflection quality across diverse multimodal tasks. |
| EarthMind: Towards Multi-Granular and Multi-Sensor Earth Observation |
|
|
| with Large Multimodal Models (Read more on arXiv or HuggingFace) |
Luc Van Gool, Danda Pani Paudel, Zhitong Xiong, Bin Ren, Yan Shu |
EarthMind introduces a novel vision-language framework for Earth Observation (EO) data by integrating multi-granular and multi-sensor information. The research aims to enhance LMM understanding of EO data through spatial attention and cross-modal fusion. EarthMind employs Spatial Attention Prompting (SAP) to enhance pixel-level grounding and a cross-modal fusion mechanism for integrating optical and SAR modalities. Experiments on the proposed EarthMind-Bench demonstrate state-of-the-art performance, surpassing GPT-40 despite being only 4B in scale, and it outperforms existing methods on other public EO benchmarks. EarthMind provides AI practitioners with a framework and benchmark for developing more effective LMMs capable of handling complex EO tasks involving multi-sensor data and varying levels of granularity. |
| AReaL: A Large-Scale Asynchronous Reinforcement Learning System for |
|
|
| Language Reasoning (Read more on arXiv or HuggingFace) |
Zhiyu Mei, Chen Zhu, Xujie Shen, Jiaxuan Gao, Wei Fu |
AReAL is an asynchronous reinforcement learning system designed to enhance the capabilities of large language models for reasoning tasks. The research aims to improve training efficiency by decoupling LLM generation and training in RL. AREAL implements a fully asynchronous architecture with continuous rollout workers and parallel model updates along with staleness-enhanced PPO and system-level optimizations. Experiments on math and code reasoning benchmarks show AReaL achieves up to 2.57x training speedup compared to synchronous systems with comparable or improved final performance. AReaL’s asynchronous RL system improves GPU utilization and training throughput, offering AI practitioners a more efficient approach to training large reasoning models. |
| MiCRo: Mixture Modeling and Context-aware Routing for Personalized |
|
|
| Preference Learning (Read more on arXiv or HuggingFace) |
Feng Luo, Yifan Sun, Jingyan Shen, Ray2333, FlippyDora |
i) MiCRo introduces a two-stage framework for personalized preference learning using mixture modeling and context-aware routing with binary preference datasets. ii) The research aims to capture diverse human preferences without fine-grained annotations and adapt to individual users efficiently at deployment. iii) The methodology involves training a context-aware mixture of Bradley-Terry reward models followed by an online routing strategy adapting mixture weights based on contextual information. iv) Experiments show MiCRo achieves an average test accuracy of 0.7830 on HelpSteer2 and 0.8218 on RPR, outperforming baselines in adapting to user preferences within datasets. v) MiCRo offers AI practitioners a label-efficient solution for personalized preference learning, enabling efficient adaptation to specific user preferences with minimal additional supervision in Reinforcement Learning from Human Feedback (RLHF) applications. |
| Incentivizing Reasoning for Advanced Instruction-Following of Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Yuchen Shi, Zihan Xu, Zongyi Li, Gang Li, yolay |
i) This paper introduces a systematic method, RAIF, to improve LLMs’ ability to follow complex instructions. ii) The primary objective is to enhance the instruction-following capabilities of LLMs, particularly with complex, multi-constraint instructions. iii) The methodology involves decomposing complex instructions, reproducible data acquisition, reinforcement learning with rule-centric reward signals, and sample-wise contrastive learning for better CoT enforcement. iv) Evaluations on seven benchmarks demonstrate that a 1.5B LLM achieves 11.74% performance gains using RAIF, performing comparably to an 8B LLM. v) The RAIF method offers AI practitioners a scalable approach to improve instruction-following in LLMs, especially where complex and multifaceted instructions are involved. |
| Reasoning Like an Economist: Post-Training on Economic Problems Induces |
|
|
| Strategic Generalization in LLMs (Read more on arXiv or HuggingFace) |
Yifang Chen, Xiangqi Jin, Xingyu Dong, Steven-Shaobo, MasterZhou |
i) This paper investigates the efficacy of post-training Large Language Models (LLMs) on economic reasoning problems to improve strategic generalization in Multi-Agent Systems (MAS). ii) The main research question is whether Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR) can effectively enhance LLMs’ ability to generalize to multi-agent scenarios using economic reasoning as a testbed. iii) The methodology involves creating Recon, a 7B-parameter LLM post-trained on a hand-curated dataset of 2,100 economic reasoning problems, followed by evaluation on economic benchmarks and multi-agent games. iv) Primary results show a 14.7% absolute gain on economic reasoning benchmarks and improved Nash equilibrium convergence by 9.5 points in multi-agent games after post-training. v) The principal implication is that domain-aligned post-training is a scalable route for aligning LLMs with economic rationality, potentially fostering strategic behavior in MAS, demonstrating the benefit of structured post-training techniques for latent alignment in LLMs. |
| Cora: Correspondence-aware image editing using few step diffusion (Read more on arXiv or HuggingFace) |
Andrea Tagliasacchi, Negar Hassanpour, Sauradip Nag, Aryan Mikaeili, Amirhossein-Alimohammadi |
i) Cora introduces a novel image editing framework leveraging correspondence-aware techniques within few-step diffusion models. ii) The research aims to enhance structural and textural consistency in edited images, particularly for edits involving significant structural changes, while maintaining balance between content generation and preservation. iii) The methodology incorporates correspondence-aware noise correction utilizing DIFT features, interpolated attention maps via both linear and spherical interpolation, and structural alignment through Hungarian matching of source and target image queries. iv) Experiments show that Cora excels in maintaining structure, textures, and identity across diverse edits, achieving a user study ranking of 3.29, demonstrating superiority over alternatives. v) Cora provides AI practitioners with a method for high-fidelity image editing, enabling control over appearance and structure while generating new content, improving results on tasks that would normally produce texture inconsistencies and require significant training data. |
| DyePack: Provably Flagging Test Set Contamination in LLMs Using |
|
|
| Backdoors (Read more on arXiv or HuggingFace) |
Soheil Feizi, mmoayeri, wangwenxiao, yizecheng |
i) DyePack is a framework for detecting test set contamination in Large Language Models (LLMs) by using backdoor attacks. ii) The paper aims to identify LLMs trained on benchmark test sets, thus inflating performance metrics, without access to model internals. iii) The method involves injecting backdoor samples with stochastic targets into the test data and verifying the activation of backdoors in evaluated models. iv) DyePack detected all contaminated models on MMLU-Pro with a false positive rate as low as 0.000073% using eight backdoors. v) This approach provides AI practitioners with a tool for validating the integrity of LLM benchmark evaluations, ensuring fair model comparisons, and preventing inaccurate performance assessments. |
| VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL (Read more on arXiv or HuggingFace) |
Bhaskar Ramasubramanian, Yuetai Li, Fengqing Jiang, zhangchenxu, EthanSta |
i) VISUALSPHINX presents a large-scale synthetic dataset of visual logic puzzles to enhance multimodal reasoning in vision language models (VLMs). ii) The paper aims to address the lack of large-scale, well-structured training datasets for logical inference over visual inputs in VLMs. iii) A rule-to-image synthesis pipeline is used, employing a rule-level genetic algorithm and program-based image generation to create diverse puzzles. iv) The QWEN2.5-VL-7B model fine-tuned on VISUALSPHINX demonstrates a 26.64% improvement in overall accuracy on visual logic puzzles and increased the average accuracy on the MathVista-testmini benchmark from 59.4% to 64.0%. v) AI practitioners can use VISUALSPHINX to train VLMs for improved performance on logical reasoning tasks, including algebraic, arithmetic, and geometric reasoning. |
| From Token to Action: State Machine Reasoning to Mitigate Overthinking |
|
|
| in Information Retrieval (Read more on arXiv or HuggingFace) |
Seung-won Hwang, yeonseokjeong, waylight3 |
i) The paper introduces State Machine Reasoning (SMR), a transition-based reasoning framework to mitigate overthinking in information retrieval (IR). ii) The research aims to address redundant trajectories and misguided reasoning that hamper effective Chain-of-Thought (CoT) prompting in IR. iii) The methodology involves defining discrete actions (REFINE, RERANK, STOP) to structure the reasoning process as transitions between query and document states. iv) Experiments on BEIR and BRIGHT benchmarks demonstrate a 3.4% improvement in nDCG@10 and a 74.4% reduction in token usage using SMR compared to CoT prompting. v) The results suggest that AI practitioners can leverage SMR as a tuning-free and generalizable alternative to CoT reasoning to enhance retrieval performance while reducing computational overhead. |
| WHEN TO ACT, WHEN TO WAIT: Modeling Structural Trajectories for Intent |
|
|
| Triggerability in Task-Oriented Dialogue (Read more on arXiv or HuggingFace) |
Kyrie Zhixuan Zhou, Yuanli Wang, Jindan Huang, simonycl, FreaxRuby |
i) This paper introduces STORM, a framework for modeling and analyzing intent triggerability in task-oriented dialogues by capturing user intent evolution. ii) The research aims to address the Intent-Action Alignment Problem by determining when user expressions have reached cognitive readiness for effective system action. iii) The methodology employs two LLMs (UserLLM and AgentLLM) to simulate conversations, tracks evolving user states within session-specific records, and uses a web-based visualization interface. iv) Experiments reveal that moderate profile uncertainty (40-60%) can outperform complete information access in certain scenarios, and access to user profiles increases satisfaction scores by 15-40%. v) AI practitioners should reconsider optimal information completeness in human-AI collaboration, and design uncertainty-calibrated dialogue systems to align immediate satisfaction with cognitive alignment. |
| Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision |
|
|
| Geometry Priors (Read more on arXiv or HuggingFace) |
Liwei Wang, Yanyang Li, Shijia Huang, zd11024 |
This paper introduces Video-3D Geometry LLM (VG LLM) to enhance MLLMs’ 3D scene understanding from video. The research aims to enable MLLMs to understand and reason about 3D spaces directly from video data without explicit 3D data input. The methodology involves employing a 3D visual geometry encoder to extract 3D prior information from video sequences and fuse it with visual tokens before feeding into the MLLM backbone (Qwen2.5-VL). Experiments show that the 4B VG LLM achieves an average score of 46.1% on VSI-Bench, surpassing Gemini-1.5-Pro, on tasks requiring complex spatial reasoning. VG LLM offers AI practitioners a method to enhance MLLMs’ spatial reasoning by implicitly modeling inter-frame correspondences, achieving competitive performance without relying on explicit 3D data. |
| Stepsize anything: A unified learning rate schedule for |
|
|
| budgeted-iteration training (Read more on arXiv or HuggingFace) |
Zhouchen Lin, zhou Xun, Yiming Dong, Anda Tang, Taoer |
i) This paper proposes a Unified Budget-Aware (UBA) learning rate schedule for budgeted-iteration training. ii) The research aims to develop a theoretically grounded learning rate schedule that consistently outperforms commonly-used schedules under different constrained training budgets. iii) The methodology involves constructing a budget-aware optimization framework incorporating robustness to landscape curvature variations and deriving the UBA schedule controlled by a single hyper-parameter. iv) Experimental results show that UBA surpasses commonly-used schedules across diverse vision and language tasks, and UBA achieves state-of-the-art performance across approximately half of the benchmarks in language tasks while consistently outperforming baselines in average scores. v) The UBA schedule provides AI practitioners with a reliable, unified, and theoretically-grounded learning rate strategy for improved performance in resource-constrained training scenarios, eliminating the need for per-network numerical optimization. |
| CodeV-R1: Reasoning-Enhanced Verilog Generation (Read more on arXiv or HuggingFace) |
Chongxiao Li, Xiaoyun Zhang, Hanqi Lyu, dihuang, zhuyaoyu |
CodeV-R1 introduces a reinforcement learning framework for Verilog generation from natural language. The research addresses the challenges of automated verification, data scarcity, and high computational cost in applying RLVR to HDL generation. It employs a rule-based testbench generator for equivalence checking, a round-trip data synthesis method for creating high-quality NL-code pairs, and a two-stage “distill-then-RL” training pipeline with an adaptive DAPO RLVR algorithm. CodeV-R1-7B achieves 68.6% pass@1 on VerilogEval v2 and 72.9% pass@1 on RTLLM v1.1, surpassing prior state-of-the-art by 12~20%. The developed model, training pipeline, and dataset will be released to facilitate research in EDA and LLM communities. |
| Normalized Attention Guidance: Universal Negative Guidance for Diffusion |
|
|
| Model (Read more on arXiv or HuggingFace) |
Yi-Zhe Song, Kai Zou, Hmrishav, ChenDY |
Normalized Attention Guidance (NAG) provides a training-free negative guidance approach for diffusion models. The research addresses the challenge of effective negative guidance in diffusion models, especially in few-step sampling regimes where Classifier-Free Guidance (CFG) fails. The key methodology involves applying extrapolation in attention space with L1-based normalization and feature refinement. Experiments demonstrate consistent improvements in text alignment, fidelity, and human-perceived quality, with results showing ImageReward increases across evaluated models and metrics. The primary implication is that NAG provides AI practitioners with a universal plug-in for modern diffusion frameworks enabling effortless negative guidance without retraining, addressing limitations of CFG. |
| WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web |
|
|
| Tasks (Read more on arXiv or HuggingFace) |
Tatsumi Sunada, Atsuki Sato, Kazuki Egashira, Zaiying Zhao, AtsuMiyai |
i) WebChoreArena, a new benchmark, is introduced to evaluate web browsing agents on complex and tedious tasks. ii) The objective is to extend WebArena’s scope to assess agents’ capabilities in labor-intensive scenarios requiring massive memory, calculation, and long-term memory. iii) The methodology involves curating 532 tasks across four simulated websites used in WebArena and evaluating the performance of agents using LLMs such as GPT-40, Claude 3.7 Sonnet, and Gemini 2.5 Pro with BrowserGym and AgentOccam. iv) Results show that Gemini 2.5 Pro achieved 44.9% accuracy on WebChoreArena, indicating substantial room for improvement compared to WebArena. v) WebChoreArena serves as a more precise benchmark for evaluating and differentiating the performance of advanced LLMs, highlighting areas for improvement in memory utilization and complex task handling for AI web browsing agents. |
| Pro3D-Editor : A Progressive-Views Perspective for Consistent and |
|
|
| Precise 3D Editing (Read more on arXiv or HuggingFace) |
Zhendong Mao, Mengqi Huang, Yang Zheng, CNcreator0331 |
i) The paper introduces Pro3D-Editor, a novel framework for consistent and precise text-guided 3D editing. ii) The research aims to achieve inter-view consistent 3D editing by addressing the limitations of view-indiscriminate approaches. iii) The methodology uses a progressive-views paradigm involving Primary-view Sampler, Key-view Render with Mixture-of-View-Experts Low-Rank Adaptation (MoVE-LoRA), and Full-view Refiner modules. iv) Experiments demonstrate Pro3D-Editor achieves a 47.4% improvement in LPIPS and a 9.7% improvement in DINO-I compared to existing methods. v) AI practitioners can leverage Pro3D-Editor’s progressive-views paradigm to improve spatial consistency and accuracy in text-guided 3D editing tasks. |
| OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and |
|
|
| Cleaning (Read more on arXiv or HuggingFace) |
Jinchuan Tian, William Chen, Yui Sudo, Shakeel Muhammad, pyf98 |
i) This paper introduces OWSM v4, an improved series of open Whisper-style speech models achieved through data scaling and cleaning of the YODAS dataset. ii) The primary objective is to enhance the performance of open-source speech foundation models by integrating and curating a large-scale web-crawled dataset. iii) The methodology involves a scalable data-cleaning pipeline using public LID and ASR models to address language label errors and audio-text misalignments in the YODAS dataset. iv) The new OWSM v4 models significantly outperform previous versions on multilingual benchmarks, achieving a 9.4% average WER on MLS with the medium-sized model and outperforming Whisper-medium. v) The principal implication for AI practitioners is the availability of cleaned YODAS data and improved OWSM models, which can be used to develop and deploy high-quality, fully open-source multilingual speech recognition systems, with data cleaning scripts, pre-trained models, and training logs made publicly available. |
| Stress-testing Machine Generated Text Detection: Shifting Language |
|
|
| Models Writing Style to Fool Detectors (Read more on arXiv or HuggingFace) |
Giovanni Puccetti, Alessio Miaschi, Cristiano Ciaccio, Michele Papucci, andreapdr |
i) This paper presents a pipeline to generate more challenging machine-generated text (MGT) to evaluate the robustness of MGT detectors. ii) The main research question is how to make MGT more difficult to detect by aligning the writing style of LLMs with human-written text (HWT). iii) The methodology involves fine-tuning LLMs using Direct Preference Optimization (DPO) with parallel datasets of HWT and MGT, targeting specific linguistic features. iv) Results show that detectors’ performance drops significantly after one DPO iteration; for example, MAGE’s accuracy drops from 76% to 47% on the XSUM dataset when evaluated on Llama dpo-1-ling generated texts. v) The principal implication for AI practitioners is the need to improve MGT detection methods by focusing on robustness to adversarial attacks exploiting stylistic cues, as current detectors rely on shallow linguistic features. |
| VAU-R1: Advancing Video Anomaly Understanding via Reinforcement |
|
|
| Fine-Tuning (Read more on arXiv or HuggingFace) |
Xiaodong Cun, Xi Shen, Qixiang Chen, Liyun Zhu |
i) The paper introduces VAU-R1, a reinforcement fine-tuning framework for video anomaly understanding, and VAU-Bench, a new benchmark. ii) The objective is to enhance anomaly reasoning in multimodal large language models (MLLMs) and to provide a comprehensive benchmark for evaluating this capability. iii) The methodology involves reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) and a novel VAU-Bench dataset with chain-of-thought annotations. iv) Empirical results show that VAU-R1 improves question answering accuracy and temporal grounding, with Qwen2.5-VL-3B+RFT achieving an accuracy of 87.08% on multiple choice QA in the MSAD dataset. v) This work offers AI practitioners a data-efficient reinforcement learning approach and a new evaluation benchmark to improve the reasoning and temporal localization capabilities of MLLMs in video anomaly understanding tasks. |
| LLM in the Loop: Creating the PARADEHATE Dataset for Hate Speech |
|
|
| Detoxification (Read more on arXiv or HuggingFace) |
Helmut Schmid, Ashish Yashwanth Kangen, Lukas Kouba, Ercong Nie, shuzyuan |
i) This paper introduces PARADEHATE, a new parallel dataset for hate speech detoxification created using an LLM-in-the-loop pipeline. ii) The research aims to address the scarcity of high-quality parallel datasets for hate speech detoxification by automating data creation. iii) The methodology involves replacing human annotators in the ParaDetox pipeline with a GPT-40-mini model for rephrasing, content preservation checks, and toxicity evaluation. iv) Results show that models fine-tuned on PARADEHATE, such as BART, achieve improved style accuracy (0.95), fluency (0.78), and BLEU score (0.31) compared to baseline methods. v) LLM-generated detoxification text provides a scalable alternative to human annotation, potentially improving the effectiveness of hate speech detoxification models. |
| zip2zip: Inference-Time Adaptive Vocabularies for Language Models via |
|
|
| Token Compression (Read more on arXiv or HuggingFace) |
Chris Wendler, Maxime Peyrard, Yunzhen yao, Saibo Geng, nathanrchn |
zip2zip introduces a framework for dynamically adapting language model vocabularies at inference time through token compression. The research investigates how to reduce token sequence length to improve LLM efficiency by enabling inference-time vocabulary adaptation. The method uses LZW compression to create reusable hypertokens, incorporates an embedding layer for new hypertokens, and trains a compression-aware language model. The study demonstrates a 20-60% reduction in input and output sequence lengths with fine-tuning, translating to latency improvements, though it notes a potential degradation in language modeling performance, especially on tasks requiring numerical computation where malformed numbers are generated due to the dynamic tokenization. This dynamic tokenization framework provides AI practitioners with a method for enhancing LLM inference speed by reducing sequence length, although there is a trade-off with potentially reduced accuracy in numerical tasks. |
| SATA-BENCH: Select All That Apply Benchmark for Multiple Choice |
|
|
| Questions (Read more on arXiv or HuggingFace) |
Stephanie Eckman, Chi Xue, Xi Fang, Shixian Cui, xwjzds |
i) SATA-BENCH is introduced as a benchmark for evaluating large language models (LLMs) on Select All That Apply (SATA) questions. ii) The research aims to assess LLMs’ ability to identify multiple correct answers in diverse domains. iii) The methodology involves curating a dataset of 1,604 human-validated SATA questions and evaluating 27 LLMs, along with proposing a Choice Funnel decoding strategy. iv) Results indicate that even the strongest model achieves only 41.8% exact match, highlighting a gap in reliably identifying all correct answers, and Choice Funnel achieves up to 29% higher exact match compared to baselines. v) The benchmark and Choice Funnel framework provide AI practitioners with tools to diagnose and improve multi-answer reasoning in LLMs for realistic applications requiring robust decision-making. |
| Cascading Adversarial Bias from Injection to Distillation in Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Milad Nasr, Ilia Shumailov, Matthew Jagielski, Jamie Hayes, Harsh Chaudhari |
i) This paper demonstrates that adversarial biases can be injected into teacher language models via data poisoning and subsequently amplified in distilled student models. ii) The research investigates the vulnerability of distilled language models to adversarial bias injection during training. iii) The methodology involves injecting poisoned samples into the teacher model’s instruction tuning data and then distilling the model. iv) Results show that with only 25 poisoned samples (0.25% poisoning rate), the student model generated biased responses 76.9% of the time in a targeted propagation scenario. v) This work implies a need for specialized safeguards to mitigate the propagation of adversarial biases in distilled language models. |
| ComposeAnything: Composite Object Priors for Text-to-Image Generation (Read more on arXiv or HuggingFace) |
Cordelia Schmid, Shizhe Chen, zk95 |
i) The paper introduces ComposeAnything, a training-free framework for improved compositional text-to-image generation leveraging composite object priors. ii) The primary objective is to enhance text-to-image models for complex compositions involving novel arrangements and high object counts. iii) The methodology employs Large Language Models (LLMs) for 2.5D semantic layout generation and a prior-guided diffusion process that combines object prior reinforcement and spatial-controlled denoising. iv) ComposeAnything outperforms state-of-the-art methods on T2I-CompBench, achieving a 16.9% absolute gain on 2D-Spatial compared to the SD3-M base model, and NSR-1K benchmarks for prompts with 2D/3D spatial arrangements, high object counts, and surreal compositions. v) This framework provides AI practitioners with an interpretable and robust method for compositional image generation that can be readily integrated with existing diffusion-based text-to-image models without retraining. |
| MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity |
|
|
| Reconstruction and Generation (Read more on arXiv or HuggingFace) |
Ziyang Ma, Chenpeng Du, Jiawei Chen, Yakun Song, xiaobinzhuang |
MagiCodec is a novel single-layer Transformer-based audio codec designed for high-fidelity reconstruction and improved downstream modelability. The research addresses the optimization trade-off between reconstruction quality and generative capacity in neural audio codecs. MagiCodec employs Gaussian noise injection and latent regularization within a multistage training pipeline to enhance semantic expressiveness while maintaining fidelity. Experimental results show MagiCodec achieves state-of-the-art reconstruction quality with a Word Error Rate (WER) of 3.16% at 850 bps and superior performance in downstream tasks, such as text-to-speech. The Zipf-like code distribution and demonstrated modelability suggest MagiCodec offers AI practitioners a potentially superior discrete audio representation for language-model-based audio generative architectures. |
| OmniResponse: Online Multimodal Conversational Response Generation in |
|
|
| Dyadic Interactions (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Siyang Song, Bing Li, Jianghui Wang, Cheng Luo |
OmniResponse introduces a model for online multimodal conversational response generation (OMCRG) in dyadic interactions. The research aims to generate synchronized verbal and non-verbal listener feedback conditioned on a speaker’s multimodal input. The methodology involves a Multimodal Large Language Model (MLLM) with a Chrono-Text module for temporal anchoring of text tokens and a TempoVoice module for controllable online TTS synchronized with facial reactions. Experiments on the new ResponseNet dataset demonstrate that OmniResponse significantly outperforms baselines, achieving improvements in semantic speech content, audio-visual synchronization, and generation quality. The work provides AI practitioners with a framework for developing more realistic and synchronized conversational AI agents. |
| Think Again! The Effect of Test-Time Compute on Preferences, Opinions, |
|
|
| and Beliefs of Large Language Models (Read more on arXiv or HuggingFace) |
Michal Shmueli-Scheuer, Ateret Anaby-Tavor, Itay Nakash, George Kour |
i) This paper benchmarks and analyzes the subjective inclinations of Large Language Models (LLMs) across various domains using a newly developed survey. ii) The research aims to evaluate if LLMs exhibit subjective preferences, opinions, and beliefs, and to what extent increased test-time compute influences these tendencies. iii) The study employs a benchmark called the Preference, Opinion, and Belief survey (POBs) to assess LLMs’ subjective inclinations, along with metrics for reliability, neutrality, and consistency, testing direct prompting, reasoning, and self-reflection. iv) Results indicate that increasing test-time compute does not significantly improve neutrality or consistency, and newer model versions exhibit decreased consistency and increased bias, with models showing a negative correlation between non-neutrality and topical consistency (r ~ 0.9). v) AI practitioners need to carefully evaluate and audit LLMs for unintended biases and inconsistencies before deployment, as increasing test-time compute alone does not guarantee mitigation and newer versions may exhibit greater bias. |
| LIFT the Veil for the Truth: Principal Weights Emerge after Rank |
|
|
| Reduction for Reasoning-Focused Supervised Fine-Tuning (Read more on arXiv or HuggingFace) |
Tianjin Huang, Chaoqun Yang, Oleg Balabanov, Tianyu Pang, Zihang Liu |
i) The paper introduces Low-rank Informed Sparse Fine-Tuning (LIFT), a method for efficient LLM fine-tuning that leverages low-rank approximation to identify and update principal weights. ii) The research investigates whether sparse fine-tuning can achieve comparable or superior reasoning performance to full fine-tuning by identifying critical parameters via rank reduction. iii) The methodology involves performing SVD on weight matrices, approximating with low rank, and selectively fine-tuning the parameters with the highest magnitude in the reduced-rank representation. iv) LIFT achieves up to 4.42% better performance than LoRA on commonsense reasoning tasks, and up to 2.02% higher overall performance than full FT on GPQA Diamond. v) LIFT enables AI practitioners to achieve improved reasoning performance in LLMs with memory efficiency comparable to LoRA, offering a practical alternative to full fine-tuning. |
| Pitfalls in Evaluating Language Model Forecasters (Read more on arXiv or HuggingFace) |
Florian Tramèr, Jonas Geiping, Shashwat Goel, Daniel Paleka |
i) Language model (LLM) forecaster evaluations face challenges related to temporal leakage and real-world performance extrapolation. ii) The research identifies and analyzes pitfalls in evaluating LLMs for forecasting future events. iii) The methodology involves a systematic analysis of evaluation flaws and concrete examples from prior work to demonstrate issues like logical leakage, unreliable date-restricted retrieval, piggybacking on human forecasts, and gaming benchmarks. iv) The study found at least 3.8% of a forecasting dataset included questions for events that resolved early, making forecasting unnecessary. v) AI practitioners need to employ more rigorous evaluation methodologies to assess the forecasting abilities of LLMs due to the potential for inflated performance claims. |
| CityLens: Benchmarking Large Language-Vision Models for Urban |
|
|
| Socioeconomic Sensing (Read more on arXiv or HuggingFace) |
Tianjian Ouyang, Xin Zhang, Hetian Pang, Jie Feng, Tianhui Liu |
CityLens is introduced as a benchmark for evaluating large language-vision models (LLVMs) in predicting urban socioeconomic indicators. The research aims to assess LLVM capabilities in tasks such as economy, education, crime, transport, health, and environment using satellite and street view imagery across 17 globally distributed cities. The methodology employs three evaluation paradigms: Direct Metric Prediction, Normalized Metric Estimation, and Feature-Based Regression, benchmarking 17 state-of-the-art LLVMs. Results show that while LLVMs demonstrate perceptual and reasoning capabilities, they still exhibit limitations in predicting urban socioeconomic indicators. The analysis indicates that building height achieves an R2 of 0.59 in Feature-Based Regression while many tasks have R2 close to zero, suggesting difficulty in linking visual context with structured socioeconomic quantities. CityLens provides AI practitioners with a unified framework for diagnosing limitations in LLVMs regarding urban socioeconomic understanding and prediction, highlighting areas for improvement in model architecture and training data for urban sensing applications. |
| Massively Multilingual Adaptation of Large Language Models Using |
|
|
| Bilingual Translation Data (Read more on arXiv or HuggingFace) |
Hengyu Luo, Indraneil Paul, Jaakko Paavola, Zihao Li, jisx |
i) This paper investigates the impact of bilingual translation data on massively multilingual adaptation of large language models. ii) The research question is to determine if including bilingual translation data enhances massively multilingual continual pre-training. iii) The methodology involves constructing a bilingual translation corpus (MaLA) with over 2,500 language pairs and 500 languages, followed by continual pre-training of Llama 3 & 3.1 (8B) models. iv) The primary result is that bilingual data generally enhances multilingual performance compared to monolingual data, with a remarkable increase of machine translation performance from 9% to 140% increase in BLEU scores on the Flores200 dataset for English translation directions. v) The principal implication for AI practitioners is the demonstration that continual pre-training with bilingual data can improve multilingual performance, especially for machine translation tasks and low-resource languages. |
| From Guidelines to Practice: A New Paradigm for Arabic Language Model |
|
|
| Evaluation (Read more on arXiv or HuggingFace) |
Abdulrahman Al-Batati, Yasser Al-Habashi, Adel Ammar, Omer Nacar, Serry Sibaee |
i) This paper introduces a novel evaluation framework for Arabic Language Models (LLMs) to address limitations in existing datasets regarding linguistic accuracy and cultural alignment. ii) The primary objective is to establish theoretical guidelines and introduce the Arabic Depth Mini Dataset (ADMD) for comprehensive Arabic LLM evaluation. iii) The methodology involves analyzing existing Arabic evaluation datasets, developing theoretical guidelines, curating the ADMD dataset, and evaluating five leading language models using the ADMD. iv) Results indicate variations in model performance across domains, with Claude 3.5 Sonnet achieving the highest overall accuracy of 30% and notable strength in mathematical theory in Arabic, Arabic language, and Islamic domains. v) The principal implication is a need for improved Arabic LLM evaluation methodologies that emphasize cultural competence alongside technical capabilities, impacting AI practitioners developing or deploying Arabic LLMs. |
| Synthesis of discrete-continuous quantum circuits with multimodal |
|
|
| diffusion models (Read more on arXiv or HuggingFace) |
Gorka Muñoz-Gil, Hans J. Briegel, Ikko Hamamura, Zohim Chandani, Floki00 |
i) This paper introduces a multimodal denoising diffusion model (DM) for quantum circuit synthesis, addressing both discrete gate selection and continuous parameter prediction. ii) The primary objective is to efficiently compile quantum operations by simultaneously generating a circuit’s structure and its continuous parameters. iii) The methodology involves leveraging two independent diffusion processes within a multimodal framework: one handling discrete gate selection and the other predicting continuous gate parameters. iv) The model was benchmarked on unitary compilation, achieving successful compilation with low infidelity for up to 5-qubit circuits with up to 16 gates, and revealing dependence on the number of gates and the percentage of parameterized gates. v) This research provides AI practitioners with a method for rapid quantum circuit generation, enabling the creation of large datasets for heuristic extraction and potentially offering new insights for quantum circuit synthesis. |
| MIKU-PAL: An Automated and Standardized Multi-Modal Method for Speech |
|
|
| Paralinguistic and Affect Labeling (Read more on arXiv or HuggingFace) |
Jiatong Shi, Ruoyi Zhang, Yifan Cheng |
MIKU-PAL introduces an automated multimodal framework for labeling emotional speech. The research aims to automate high-consistency emotion annotation from unlabeled video data, addressing limitations of current emotional speech datasets. The methodology employs face detection and tracking with a multimodal large language model (MLLM) to analyze audio, visual, and text modalities. MIKU-PAL achieved a Fleiss к score of 0.93 for consistency and can annotate up to 26 emotion categories with 83% human-validated rationality. Releasing MIKU-EmoBench, a 131.2-hour dataset of fine-grained emotional speech, provides AI practitioners with a new benchmark for emotional text-to-speech and visual voice cloning. |
Papers for 2025-06-02
| Title |
Authors |
Summary |
| ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Xin Dong, Jian Hu, Ximing Lu, Shizhe Diao, Mingjie Liu |
ProRL introduces a prolonged reinforcement learning methodology to enhance reasoning in large language models. The research explores whether RL can truly expand a model’s reasoning capabilities beyond merely amplifying existing outputs and investigates the impact of extended RL training. The methodology incorporates KL divergence control, reference policy resetting, and diverse task training, scaling up to 2k training steps. Empirical results demonstrate that RL-trained models outperform base models in pass@k evaluations, with average improvements of 14.7% on math benchmarks compared to DeepSeek-R1-1.5B, and reveals RL can uncover new solution pathways entirely absent in base models. The findings suggest that prolonged RL training can enable exploration of new reasoning patterns, holding implications for AI practitioners as it demonstrates the potential for RL to meaningfully expand reasoning boundaries in language models given sufficient training time. |
| AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time (Read more on arXiv or HuggingFace) |
Haoran Geng, Xuying Ning, Han Wang, RunpeiDong, jyzhang1208 |
i) This paper introduces ALPHAONE (a1), a framework for modulating reasoning progress in large reasoning models (LRMs) at test time. ii) The research aims to develop a universal approach to modulating the reasoning process of LRMs to enhance both reasoning capability and efficiency. iii) The methodology involves scaling the thinking phase using a universal parameter a, dynamically scheduling slow thinking transitions before the a moment via a Bernoulli process, and deterministically terminating slow thinking after the a moment. iv) Experiments on various benchmarks demonstrate a1’s superior reasoning capability and efficiency, with the 1.5B LRM showing a Pass@1 improvement of +6.15% while reducing token length by nearly 14%. v) ALPHAONE offers AI practitioners an efficient test-time scaling strategy to modulate LRMs, improving both accuracy and efficiency across mathematical, coding, and scientific reasoning tasks, through a dense slow-to-fast reasoning modulation technique. |
| Time Blindness: Why Video-Language Models Can’t See What Humans Can? (Read more on arXiv or HuggingFace) |
Mohamed Elhoseiny, Zhiqiang Shen, mukul54, ujjwal9 |
i) The paper introduces SpookyBench, a benchmark to evaluate purely temporal understanding in video-language models (VLMs). ii) The main research question is to assess why VLMs struggle with temporal pattern recognition when spatial cues are absent. iii) The methodology involves creating a synthetic dataset with information encoded exclusively in temporal noise sequences and evaluating various VLMs. iv) Results show that state-of-the-art VLMs achieve near 0% accuracy on SpookyBench, while humans achieve over 98% accuracy; finetuning improved models negligibly. v) The principal implication is that current VLMs over-rely on spatial features and lack architectural mechanisms for processing purely temporal information, indicating the need for novel architectures or training paradigms to decouple spatial dependencies. |
| Don’t Look Only Once: Towards Multimodal Interactive Reasoning with |
|
|
| Selective Visual Revisitation (Read more on arXiv or HuggingFace) |
Min Soo Kim, Jaeyoung Lee, Jiwan Chung, siyeolkim, kjunh |
v1 is introduced, enabling multimodal reasoning via dynamic visual revisitation in MLLMs. The paper addresses the limitation of current MLLMs that only consume visual input once. The research question is how to effectively enable MLLMs to revisit images during reasoning. The methodology involves a point-and-copy mechanism and a newly constructed dataset, v1g, of 300K multimodal reasoning traces with visual grounding annotations. Experiments show v1 consistently improves performance on multimodal mathematical reasoning benchmarks, such as a 68.6% mini average on MathVista, MathVision, and MathVerse. The principal implication is that dynamic visual access significantly enhances grounded multimodal reasoning, suggesting a promising direction for AI development. |
| Large Language Models for Data Synthesis (Read more on arXiv or HuggingFace) |
Lijun Sun, Menglin Kong, HYTYH |
LLMSYNTHOR is introduced as a framework for synthesizing data using large language models (LLMs) while ensuring statistical fidelity. The research aims to improve LLM-based data synthesis by addressing limitations in efficiency, context limits, and statistical alignment. LLMSYNTHOR uses LLMs as nonparametric copula simulators, employs LLM Proposal Sampling for efficient grounded distributions, and utilizes an iterative synthesis loop to minimize summary statistic discrepancies between real and synthetic data. Evaluations across e-commerce, population, and mobility datasets demonstrate high statistical fidelity, utility, and adaptability, including achieving low divergence and gap scores in e-commerce transaction synthesis. LLMSYNTHOR provides AI practitioners with a robust tool for generating high-quality synthetic datasets across diverse domains, improving training data availability and reducing reliance on real-world datasets. |
| HardTests: Synthesizing High-Quality Test Cases for LLM Coding (Read more on arXiv or HuggingFace) |
Jiabao Ji, Kexun Zhang, Yee Man Choi, Zhongmou He, JuntingZhou |
i) This paper introduces HARDTESTGEN, a pipeline for synthesizing high-quality test cases for large language model (LLM) coding. ii) The research aims to address the lack of reliable verifiers in LLM coding by generating difficult-to-synthesize edge cases. iii) The methodology involves using LLMs to generate test generator programs and filtering test cases using human-written oracle programs. iv) The study curates a comprehensive competitive programming dataset, HARDTESTS, demonstrating 11.3 percentage points higher precision and 17.5 percentage points higher recall compared to existing tests when evaluating LLM-generated code. v) The key implication is the provision of a more reliable verification mechanism, essential for post-training techniques like reinforcement learning and self-distillation in LLM coding. |
| ViStoryBench: Comprehensive Benchmark Suite for Story Visualization (Read more on arXiv or HuggingFace) |
Yaoqi Hu, Jingwei Wu, Ailin Huang, Cailin Zhuang, wchengad |
i) The paper introduces ViStoryBench, a comprehensive benchmark for story visualization. ii) The research aims to provide a standardized evaluation framework to enhance story visualization model performance in real-world scenarios by assessing different plots, artistic styles, and character consistency. iii) The methodology involves collecting a diverse dataset of 80 story segments with 344 roles and developing 12 automated evaluation metrics, including Character Identification Similarity (CIDS), prompt adherence, and style consistency. iv) Evaluation of over twenty methods revealed, through user studies, that UNO achieved top ratings for environment consistency (82.0), while Doubao achieved top ratings for character identification consistency (92.6). v) ViStoryBench enables AI practitioners to thoroughly evaluate strengths and weaknesses of story visualization models, fostering targeted improvements in areas like character portrayal and visual coherence. |
| Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and |
|
|
| Benchmarking Multimodal LLM Agents (Read more on arXiv or HuggingFace) |
Xiaohan Zhao, Jiacheng Liu, Zhaoyi Li, Yaxin Luo, jiachengcui888 |
Open CaptchaWorld is introduced as a novel web-based benchmark for evaluating multimodal large language model (MLLM) agents’ ability to solve interactive CAPTCHA puzzles. The research aims to address the lack of benchmarks testing MLLMs in interactive, multi-step reasoning scenarios mimicking real-world web browsing. The methodology involves curating a dataset of 225 CAPTCHAs across 20 types and introducing a new metric called CAPTCHA Reasoning Depth to quantify task complexity. Experiments revealed that state-of-the-art MLLM agents achieve a maximum success rate of 40.0% (Browser-Use Openai-03), significantly lower than the 93.3% achieved by humans, highlighting their limitations in visual reasoning and interaction. Open CaptchaWorld provides AI practitioners with a valuable diagnostic tool to identify weaknesses in current multimodal agents and guide the development of more robust reasoning systems. |
| Vision Language Models are Biased (Read more on arXiv or HuggingFace) |
Vy Tuong Dang, Khai-Nguyen Nguyen, An Vo, knguyennguyen, taesiri |
i) This work investigates biases in vision language models (VLMs) on objective visual tasks. ii) The research question explores how prior knowledge impacts VLMs’ accuracy on standard object counting, identification, and low-level vision tasks. iii) The methodology employs an automated framework, VLMBias, using image editing and text-to-image generation to create counterfactual images of well-known subjects and evaluating VLM performance on these images. iv) The primary result indicates that state-of-the-art VLMs are strongly biased, achieving only 17.05% accuracy in counting tasks across seven diverse domains, and inserting the subject name into counterfactual image further decreases the VLM accuracy by -2 to -6 percentage points. v) The principal implication for AI practitioners is the need for more robust bias mitigation strategies in VLMs to improve accuracy on tasks requiring visual analysis, suggesting that relying less on memorized knowledge over visual detail can reduce the biases. |
| CLaSp: In-Context Layer Skip for Self-Speculative Decoding (Read more on arXiv or HuggingFace) |
Ziqiang Liu, Lu Wang, Huiming Wang, Renke Shan, Longze Chen |
i) CLaSp introduces a novel layer-skipping method for self-speculative decoding to accelerate large language model (LLM) inference without additional training. ii) The research aims to reduce the computational cost of LLM decoding by dynamically adjusting layer sparsity based on the input context. iii) CLaSp uses a dynamic programming algorithm leveraging the complete hidden states from the last verification stage to optimize layer skipping in real-time. iv) Experiments on the LLaMA3 series demonstrate CLaSp achieves a 1.3x to 1.7x speedup compared to autoregressive decoding while preserving the original distribution of generated text. v) CLaSp offers AI practitioners a plug-and-play technique to improve LLM inference efficiency and speed without retraining, potentially simplifying deployment across various models and tasks. |
| CoDA: Coordinated Diffusion Noise Optimization for Whole-Body |
|
|
| Manipulation of Articulated Objects (Read more on arXiv or HuggingFace) |
Taku Komura, Zhiyang Dou, Zhi Cen, Huaijin Pi |
i) The paper introduces CoDA, a novel coordinated diffusion noise optimization framework for synthesizing whole-body manipulation of articulated objects. ii) The primary objective is to generate realistic, physically plausible human-object interaction sequences involving coordinated body, hand, and articulated object motion. iii) The method employs noise-space optimization over three specialized diffusion models for the body, left hand, and right hand, each trained on its own motion dataset, utilizing a basis point set (BPS) representation to encode hand-object spatial relationships. iv) The method achieves state-of-the-art performance on the ARCTIC dataset, with a user study indicating a best motion realism rate of 88.7% and best physical plausibility rate of 87.3%, and the GRAB datasets outperforming existing approaches in motion quality and physical plausibility, as well as enabling capabilities such as object pose control and simultaneous locomotion and manipulation. v) CoDA provides AI practitioners with a new framework for generating coordinated whole-body manipulation motions, improving the realism and plausibility of simulated human interactions, particularly in virtual reality, character animation, and robotics applications. |
| UniGeo: Taming Video Diffusion for Unified Consistent Geometry |
|
|
| Estimation (Read more on arXiv or HuggingFace) |
Yuan-Chen Guo, Yi-Hua Huang, Zehuan Huang, Xin Yu, Yang-Tian Sun |
i) UniGeo leverages video diffusion models for consistent 3D geometry estimation from multi-view images or video sequences. ii) The primary objective is to achieve consistent geometric property estimation across video frames by exploiting inter-frame correspondences inherent in video diffusion models. iii) The methodology involves representing geometric attributes in a global coordinate system, utilizing a shared positional encoding strategy for RGB conditioning, and a multi-task learning approach. iv) Experiments on the ScanNet++ dataset show that UniGeo achieves state-of-the-art results in both normal and radius estimation. v) UniGeo provides AI practitioners with a method for generating consistent 3D geometry from video, improving downstream tasks like 3D reconstruction without requiring camera information. |
| MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs (Read more on arXiv or HuggingFace) |
Tim G. J. Rudner, Idan Szpektor, Avi Caciularu, Gal Yona, Gabrielle Kaili-May Liu |
i) MetaFaith benchmarks and improves faithful confidence calibration in LLMs, addressing the misalignment between intrinsic uncertainty and linguistic expression. ii) The research aims to systematically study and improve LLMs’ ability to express uncertainty linguistically in alignment with their internal confidence. iii) The methodology involves benchmarking LLMs across diverse models, datasets, and uncertainty elicitation prompts, followed by introducing MetaFaith, a metacognitive prompting approach for calibration. iv) Results show that MetaFaith achieves up to 61% improvement in faithfulness and an 83% win rate over original generations as judged by humans. v) The principal implication is that MetaFaith offers a practical inference-time method for AI practitioners to enhance the reliability and trustworthiness of LLMs by improving their uncertainty communication. |
| EasyText: Controllable Diffusion Transformer for Multilingual Text |
|
|
| Rendering (Read more on arXiv or HuggingFace) |
Yiren Song, Haifa Wang, Jailing Liu, Yuxuan Zhang, Runnan Lu |
i) The paper introduces EasyText, a Diffusion Transformer (DiT) based framework for controllable multilingual text rendering. ii) The main objective is to enable high-quality, controllable text rendering across multiple languages, a challenging task for current diffusion models. iii) The methodology employs character positioning encoding and position encoding interpolation techniques, along with a two-stage training strategy involving large-scale pretraining and fine-tuning. iv) Experiments demonstrate the effectiveness of EasyText, with the fine-tuned model exhibiting an OCR accuracy of 88.72% and improved CLIPScore, suggesting enhanced visual-text alignment. v) The framework allows AI practitioners to accurately render multilingual text in images and manipulate the layout in layout-free or position-controlled manners, demonstrating the generation capability on unfamiliar and unseen characters. |
| Fork-Merge Decoding: Enhancing Multimodal Understanding in Audio-Visual |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Joon Son Chung, Jongmin Choi, Youngjoon Jang, Chae0 |
Fork-Merge Decoding (FMD) enhances balanced multimodal understanding in audio-visual large language models by addressing modality bias. The research objective is to mitigate modality bias in AV-LLMs without additional training. FMD first performs modality-specific reasoning by processing audio-only and video-only inputs through initial decoder layers and then merges representations for joint reasoning in subsequent layers. Evaluated on VideoLLaMA2 and video-SALMONN with AVQA, MUSIC-AVQA, and AVHBench datasets, FMD consistently improved performance, with a demonstration of the improvement of AVQA from 82.46±0.02 to 82.74±0.05. FMD offers AI practitioners a training-free inference strategy to improve multimodal understanding in AV-LLMs. |
| Harnessing Negative Signals: Reinforcement Distillation from Teacher |
|
|
| Data for LLM Reasoning (Read more on arXiv or HuggingFace) |
Wei Chu, Weidi Xu, Jiangxuan Long, Cheng Peng, Tim-Xu |
i) The paper introduces Reinforcement Distillation (REDI), a two-stage offline training framework for enhancing LLM reasoning by leveraging both positive and negative distilled reasoning traces. ii) The research addresses the question of how to effectively use both positive and negative distilled reasoning traces to maximize LLM reasoning performance in an offline setting. iii) REDI employs supervised fine-tuning on positive traces followed by refinement with an asymmetric, reference-free objective that incorporates negative traces. iv) Experiments show that the Qwen-REDI-1.5B model achieves 83.1% on MATH-500 (pass@1) with 131k examples, surpassing DeepSeek-R1-Distill-Qwen-1.5B trained on 800k proprietary data. v) The principal implication for AI practitioners is that REDI offers a more data-efficient approach to distilling complex reasoning abilities into smaller LLMs by effectively utilizing previously discarded negative examples. |
| Large Language Models are Locally Linear Mappings (Read more on arXiv or HuggingFace) |
jamesgolden1 |
i) The paper demonstrates that inference operations of open-weight large language models (LLMs) can be mapped to exactly equivalent linear systems for a given input sequence. ii) The main research objective is to determine if and how LLMs, despite their global nonlinearity, exhibit local linearity that can be exploited for understanding their internal representations. iii) The methodology involves strategically altering gradient computations with respect to an input sequence to produce a detached Jacobian, approximating the forward prediction as a linear system, followed by singular value decomposition (SVD) of this Jacobian. iv) The primary result is that the open-weight LLMs can operate in extremely low-dimensional subspaces, with singular vectors decoding to concepts related to the most-likely output token with a relative error of 10^-6 for float32. v) The principal implication for AI practitioners is that LLMs’ internal representations can be interpreted through nearly-exact locally linear decompositions, potentially providing insights into semantic structures within the next-token prediction process. |
| Point-MoE: Towards Cross-Domain Generalization in 3D Semantic |
|
|
| Segmentation via Mixture-of-Experts (Read more on arXiv or HuggingFace) |
Zezhou Cheng, Aruni RoyChowdhury, Wentao Zhou, Xuweiyi Chen |
i) This paper introduces Point-MoE, a Mixture-of-Experts architecture for cross-domain generalization in 3D semantic segmentation. ii) The research investigates how to enable large-scale, cross-domain generalization in 3D perception, addressing the limitations of standard point cloud backbones when trained on mixed-domain data. iii) Point-MoE replaces feed-forward layers in Point Transformer V3 with MoE layers comprising multiple expert networks and a routing mechanism. iv) Experiments demonstrate that Point-MoE outperforms multi-domain baselines and generalizes better to unseen domains, achieving 69.2% mIoU on S3DIS validation and 70.2% on test split. v) Point-MoE offers a scalable framework for 3D scene understanding, allowing models to adapt across diverse 3D data sources without manual curation or domain supervision, improving efficiency and scalability in multi-domain 3D semantic segmentation tasks. |
| Harnessing Large Language Models for Scientific Novelty Detection (Read more on arXiv or HuggingFace) |
Erik Cambria, Thanh-Son Nguyen, Soujanya Poria, Yan Liu, ZonglinY |
i) This paper explores the use of Large Language Models (LLMs) for scientific novelty detection (ND) by introducing two new datasets in marketing and NLP. ii) The main research question is how to effectively leverage LLMs to identify novel research ideas, addressing the limitations of existing methods in capturing the gap between textual similarity and idea conception. iii) The methodology involves constructing ND-tailored benchmark datasets with topological closure and compactness and training a lightweight retriever using LLM-based knowledge distillation to capture conceptual similarity. iv) Experiments show the proposed method consistently outperforms others in idea retrieval and ND tasks, achieving an average improvement of 5.40% and 15.19% compared to the top-performing baseline on the Marketing domain and NLP task, respectively, in the idea retrieval task. v) The principal implication for AI practitioners is that LLMs, when properly harnessed with knowledge distillation and appropriate datasets, can effectively detect novelty in scientific research by capturing idea conception beyond surface-level textual similarity, providing a valuable tool for researchers and engineers in navigating the increasingly vast landscape of scientific literature. |
| un^2CLIP: Improving CLIP’s Visual Detail Capturing Ability via |
|
|
| Inverting unCLIP (Read more on arXiv or HuggingFace) |
Shiguang Shan, Ruibing Hou, Hong Chang, Jiahe Zhao, yinqi |
Contrastive Language-Image Pre-training (CLIP) visual detail capturing is improved through inverting unCLIP. The research aims to improve CLIP’s ability to capture visual details in images, addressing limitations in dense prediction and vision-centric tasks. A novel approach, un²CLIP, finetunes the CLIP image encoder by inverting a pretrained unCLIP generator, thereby transferring visual knowledge while preserving language alignment. Experiments on the MMVP-VLM benchmark show un²CLIP achieves a best average performance of 32.6 for OpenAI VIT-L-14, significantly outperforming the original CLIP, indicating enhanced detail discrimination. AI practitioners can leverage un²CLIP to improve CLIP models for tasks requiring finer-grained image understanding such as open-vocabulary segmentation and multimodal large language models. |
| EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, |
|
|
| Expressiveness, and Linguistic Challenges Using Model-as-a-Judge (Read more on arXiv or HuggingFace) |
Alex Smola, Mu Li, Xingjian Shi, Yuzhi Tang, ruskinmanku |
i) EmergentTTS-Eval introduces a benchmark for evaluating TTS models on complex linguistic and prosodic scenarios using an automated model-as-a-judge approach. ii) The research aims to address limitations in existing TTS benchmarks by developing a comprehensive evaluation suite that captures nuanced and semantically complex text. iii) The methodology iteratively extends seed prompts with LLMs to generate diverse test cases and employs a Large Audio Language Model (LALM) as a judge to assess speech quality across multiple dimensions. iv) Evaluation of TTS systems, including 11Labs and OpenAI’s 40-mini-TTS, demonstrates the ability to reveal performance differences, showing that the model-as-a-judge approach offers robust assessment and a high correlation with human preferences. v) The primary implication for AI practitioners is the availability of an open-source, automated benchmark that offers a more fine-grained and reproducible evaluation of TTS systems compared to traditional methods, allowing for targeted improvements in specific areas such as expressiveness and pronunciation accuracy. |
| Enabling Flexible Multi-LLM Integration for Scalable Knowledge |
|
|
| Aggregation (Read more on arXiv or HuggingFace) |
Xin Meng, Yifan Gong, Shiyue Hou, Zheng Zhan, Zhenglun Kong |
i) This paper introduces a framework for adaptively aggregating knowledge from multiple large language models (LLMs) into a single, stronger target model. ii) The primary research objective is to create a more stable and scalable knowledge aggregation process that mitigates knowledge interference when integrating diverse LLMs. iii) The proposed methodology involves an adaptive selection network that identifies relevant source LLMs based on their scores, a dynamic weighted fusion strategy, and a feedback-driven loss function. iv) Experimental results demonstrate that the proposed method reduces knowledge interference by up to 50% compared to existing approaches. v) The research implies that adaptive selection and dynamic weighting are effective strategies for mitigating interference and improving scalability in multi-LLM knowledge aggregation for AI practitioners. |
| DexUMI: Using Human Hand as the Universal Manipulation Interface for |
|
|
| Dexterous Manipulation (Read more on arXiv or HuggingFace) |
Linxi Fan, Zhenjia Xu, Yifan Hou, Han Zhang, mengdaxu |
i) The paper introduces DexUMI, a framework that uses human hand demonstrations with a wearable exoskeleton and visual inpainting to transfer dexterous manipulation skills to diverse robot hands. ii) The research aims to minimize the action and observation gaps between human and robot hands to enable effective imitation learning for dexterous manipulation. iii) The methodology involves hardware adaptation via an optimized exoskeleton, software adaptation through robot hand inpainting in demonstration videos, and subsequent imitation learning. iv) DexUMI achieves an average task success rate of 86% on two different dexterous robot hand hardware platforms and demonstrates a 3.2 times greater data collection efficiency compared to teleoperation. v) AI practitioners can utilize DexUMI to efficiently collect training data and learn policies for dexterous robot manipulation across various hardware platforms, thus accelerating the development of robust robotic systems. |
| Role-Playing Evaluation for Large Language Models (Read more on arXiv or HuggingFace) |
Yvan Peter, Julian Alvarez, Walter Nuninger, yelboudouri |
i) The paper introduces RPEval, a novel benchmark for assessing role-playing capabilities in Large Language Models (LLMs). ii) The research aims to provide an automated and reproducible method for evaluating LLMs across emotional understanding, decision-making, moral alignment, and in-character consistency. iii) The methodology involves creating a dataset of character descriptions and scenarios, then evaluating LLM responses using verifiable tests and majority voting on crowd-sourced annotations. iv) Evaluation of GPT-40, Gemini-1.5-Pro, and Llama 3.2 1B reveals that Gemini-1.5-Pro achieves the highest average score of 62.24%, with notable performance in decision-making and moral alignment (73.86%). v) RPEval offers AI practitioners a structured framework for systematically comparing LLMs and prompting strategies, providing actionable insights for instruction tuning and prompt engineering in role-playing applications, but lacks insight into nuanced long-term role-playing attributes. |
| GATE: General Arabic Text Embedding for Enhanced Semantic Textual |
|
|
| Similarity with Matryoshka Representation Learning and Hybrid Loss Training (Read more on arXiv or HuggingFace) |
Adel Ammar, Yasser Al-Habashi, Serry Sibaee, Anis Koubaa, Omer Nacar |
i) The paper introduces GATE, a General Arabic Text Embedding model for enhanced semantic textual similarity (STS). ii) The objective is to create Arabic text embeddings that achieve state-of-the-art performance on STS tasks. iii) The methodology integrates Matryoshka Representation Learning (MRL) and hybrid loss training using Arabic NLI datasets. iv) GATE demonstrates a 20-25% performance improvement on STS benchmarks compared to larger models, including OpenAI, with the Arabic-Triplet-Matryoshka-V2 model achieving an average score of 69.99 on MTEB Arabic benchmarks. v) The principal implication for AI practitioners is the demonstrated effectiveness of MRL and hybrid loss training in creating more efficient and accurate Arabic text embeddings, offering a resource-efficient alternative to large-scale models for STS tasks in Arabic NLP applications. |
| LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Wonseok Hwang, Jinu Lee, Chaeeun Kim |
LegalSearchLM introduces a novel approach to legal case retrieval (LCR) by generating legal elements directly from a query case. The research aims to improve LCR performance by addressing limitations in existing embedding-based and lexical matching methods. It presents LEGAR BENCH, a large-scale Korean LCR benchmark with 411 diverse crime types across 1.2M cases, and a retrieval model, LEGAL SEARCHLM, performs legal element reasoning over the query case. Experiments show LEGAL SEARCHLM outperforms baselines on LEGAR BENCH standard by 6-20% and generalizes better to out-of-domain cases by 15%. LEGAL SEARCHLM’s generation of legal elements and constrained decoding provide AI practitioners with a new state-of-the-art method for improved retrieval performance, particularly in complex, domain-specific tasks such as LCR. |
| More Thinking, Less Seeing? Assessing Amplified Hallucination in |
|
|
| Multimodal Reasoning Models (Read more on arXiv or HuggingFace) |
James Zou, Juncheng Wu, Qingyue Wei, Zhongxing Xu, Chengzhi Liu |
i) This paper investigates increased hallucination in multimodal reasoning models due to extended reasoning chains and decreased visual attention. ii) The main objective is to assess and quantify the trade-off between reasoning ability and hallucination in multimodal reasoning models. iii) The methodology includes attention analysis, the introduction of the RH-AUC metric, and the creation of RH-Bench, a diagnostic benchmark. iv) Results show that reasoning-augmented models exhibit a higher hallucination rate than non-reasoning models, and larger models generally display a better balance between reasoning and perception as measured by RH-AUC. v) The study implies that AI practitioners should prioritize evaluation frameworks and training strategies that explicitly account for both reasoning quality and perceptual reliability in multimodal reasoning models to mitigate hallucination. |
Papers for 2025-05-30
| Title |
Authors |
Summary |
| Table-R1: Inference-Time Scaling for Table Reasoning (Read more on arXiv or HuggingFace) |
Arman Cohan, Lyuhao Chen, Zheyuan Yang, yilunzhao |
i) This paper explores inference-time scaling for table reasoning tasks using post-training methods. ii) The research question focuses on enabling inference-time scaling for table reasoning by evaluating distillation from frontier model reasoning traces and reinforcement learning with verifiable rewards (RLVR). iii) The methodology involves fine-tuning LLMs on a created dataset of reasoning traces generated by DeepSeek-R1 and applying the GRPO algorithm with task-specific verifiable reward functions. iv) Table-R1-Zero models match or exceed the performance of GPT-4.1 and DeepSeek-R1 using only a 7B-parameter LLM and also generalize well to out-of-domain datasets. v) The principal implication for AI practitioners is the demonstration that RLVR offers improved performance and generalization compared to distillation for table reasoning, suggesting a viable approach to enhancing LLMs for structured data tasks with inference-time scaling. |
| VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC |
|
|
| Videos (Read more on arXiv or HuggingFace) |
Yilun Zhao, Guo Gan, entropyhu, songtingyu |
i) The paper introduces VF-EVAL, a new benchmark for evaluating multimodal large language models (MLLMs) in their ability to generate reliable feedback on AI-generated content (AIGC) videos. ii) The research aims to comprehensively assess MLLMs’ capabilities in tasks such as coherence validation, error awareness, error type detection, and reasoning evaluation when applied to AIGC videos. iii) The methodology involves evaluating 13 frontier MLLMs, including GPT-4.1, on the newly proposed VF-EVAL benchmark, which includes four tasks designed to assess alignment, feedback quality, and commonsense reasoning. iv) Results show that even the best-performing model, GPT-4.1, struggles to achieve consistently high performance across all tasks, and REPROMPT experiments indicate potential quality enhancements through aligning MLLM feedback with human preferences, while overall accuracy metrics are found in Table 3. v) The primary implication for AI practitioners is the identification of current limitations in MLLMs’ ability to accurately interpret and provide feedback on AIGC videos, suggesting a need for incorporating auxiliary methods like computer vision techniques to improve feedback generation pipelines. |
| The Climb Carves Wisdom Deeper Than the Summit: On the Noisy Rewards in |
|
|
| Learning to Reason (Read more on arXiv or HuggingFace) |
Rui Yan, Zhanhui Kang, Xingwu Sun, Ang Lv, Ruobing-Xie |
i) This paper investigates the impact of noisy rewards on post-training large language models (LLMs) for reasoning tasks using reinforcement learning (RL). ii) The research question explores the LLMs’ robustness to reward noise in scenarios involving reward models. iii) The methodology involves introducing reward noise by randomly flipping the reward function’s outputs in math tasks and using Reasoning Pattern Reward (RPR) without verifying the correctness of answers. iv) A Qwen-2.5-7B model, when trained with a 40% reward flip rate on math tasks, achieved a peak accuracy of 72%, close to the 75.85% achieved with noiseless rewards. v) The principal implication for AI practitioners is that LLMs exhibit robustness to reward noise, and rewarding reasoning patterns can calibrate noisy reward models, suggesting avenues for improving pre-training and post-training techniques. |
| Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial |
|
|
| Intelligence (Read more on arXiv or HuggingFace) |
Yueqi Duan, Yi-Hsin Hung, Fangfu Liu, Diankun Wu |
i) The paper introduces Spatial-MLLM, a novel framework enhancing visual-based spatial intelligence in video Multimodal Large Language Models (MLLMs) through dual-encoder architecture and space-aware frame sampling. ii) The research objective is to improve the spatial reasoning capabilities of video MLLMs from purely 2D observations without relying on additional 3D or 2.5D data. iii) The methodology involves a dual-encoder architecture comprising a 2D visual encoder for semantic features and a spatial encoder initialized from a feed-forward visual geometry model for 3D structure features, combined with a space-aware frame sampling strategy. iv) The Spatial-MLLM achieves state-of-the-art performance on VSI-Bench, outperforming other open-source and proprietary models including Gemini-1.5 Pro on average accuracy. v) AI practitioners can leverage the Spatial-MLLM architecture and space-aware frame sampling strategy to improve the performance of video MLLMs on spatial reasoning tasks, enabling more effective scene understanding and potentially reducing the reliance on computationally expensive 3D data inputs. |
| ZeroGUI: Automating Online GUI Learning at Zero Human Cost (Read more on arXiv or HuggingFace) |
Yue Yu, Xuan Dong, Shi Liu, Shiqian Su, cyyang822 |
ZeroGUI introduces an online learning framework for GUI agents, automating task generation and reward estimation. The paper addresses the limitations of offline GUI agent training by using VLMs for task generation and reward assignment. ZeroGUI employs a two-stage online reinforcement learning approach for continuous interaction and learning in GUI environments. Experiments show ZeroGUI improves performance on OSWorld and AndroidLab, with ZeroGUI-Aguvis-7B achieving a 63% relative improvement on OSWorld. The primary implication is that scalable GUI agent training can be automated without human annotation, reducing development costs. |
| D-AR: Diffusion via Autoregressive Models (Read more on arXiv or HuggingFace) |
mikeshou, sebgao |
i) This paper introduces D-AR, a framework recasting image diffusion as autoregressive next-token prediction. ii) The main research objective is to bridge the gap between diffusion models and autoregressive models for visual generation while adhering to the standard next-token prediction paradigm. iii) The key methodology involves a coarse-to-fine sequential diffusion tokenizer to convert images into discrete tokens, enabling autoregressive modeling without modifying underlying designs. iv) On ImageNet, D-AR achieves 2.09 FID using a 775M Llama backbone with 256 discrete tokens. v) D-AR enables fast inference with KV cache, provides consistent previews during generation, and supports zero-shot layout-controlled synthesis, offering AI practitioners a unified autoregressive architecture for visual synthesis compatible with large language models. |
| Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software |
|
|
| Engineering (Read more on arXiv or HuggingFace) |
Subhro Das, Zhenting Qi, Delin Chen, Guangtao Zeng, maohaos2 |
i) The paper introduces Evolutionary Test-Time Scaling (EvoScale), a sample-efficient method to improve language model performance on software engineering tasks by iteratively refining code generation. ii) The research aims to address the challenge of sample inefficiency in test-time scaling for software engineering, particularly with smaller language models. iii) EvoScale uses an evolutionary approach with iterative selection and mutation of code patches, incorporating reinforcement learning to enable self-evolution without external verifiers at inference. iv) Evaluated on SWE-Bench-Verified, a 32B model (Satori-SWE-32B) using EvoScale achieved performance comparable to models exceeding 100B parameters using significantly fewer samples. v) This technique offers AI practitioners a method to improve the performance of smaller, more computationally efficient language models for complex software engineering tasks, potentially reducing reliance on large-scale models. |
| VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video |
|
|
| Reasoning? (Read more on arXiv or HuggingFace) |
Lin Sui, Yi Liu, Haoning Wu, Yuanxin Liu, RUBBISHLIKE |
i) The paper introduces VIDEOREASONBENCH, a new benchmark for evaluating vision-centric complex video reasoning capabilities in multimodal large language models (MLLMs). ii) The primary objective is to assess if MLLMs can effectively perform vision-centric complex video reasoning, particularly recalling visual information, inferring latent states, and predicting future states. iii) The methodology involves constructing a dataset of videos depicting fine-grained operations on latent states, creating corresponding questions with varying reasoning skills, and comprehensively evaluating 18 state-of-the-art MLLMs. iv) Results show that most MLLMs perform poorly, with GPT-4o achieving only 6.9% accuracy, while the thinking-enhanced Gemini-2.5-Pro achieves 56.0% accuracy, indicating significant disparity in video reasoning skills. v) For AI practitioners, the concerning deficiency of most SOTA MLLMs and higher reasoning depth and visual content reliance indicates a need to improve MLLM architectures to address the requirements of vision-centric complex video reasoning to improve performance in complex tasks. |
| AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views (Read more on arXiv or HuggingFace) |
Kerui Ren, Tao Lu, Linning Xu, Lihan Jiang, matthewmao |
AnySplat is a feed-forward network for novel-view synthesis from uncalibrated multi-view image collections by predicting 3D Gaussian primitives and camera parameters. The research aims to develop a feed-forward model for 3D scene reconstruction and novel view synthesis from uncalibrated multi-view images without pose annotations or per-scene optimization. The methodology involves a geometry transformer to encode images, a differentiable voxelization module for efficient Gaussian primitive processing, and self-supervised knowledge distillation from a pre-trained VGGT model. Experiments show AnySplat achieves comparable or superior novel view synthesis quality to pose-aware baselines and surpasses pose-free methods, demonstrated by achieving 23.09 dB PSNR with 32 input views on the VRNeRF dataset while significantly reducing rendering latency. The unified, compute-efficient model presents a practical approach for AI practitioners seeking real-time novel-view synthesis in unconstrained capture settings by eliminating the need for precise camera calibration and computationally intensive optimization. |
| Are Reasoning Models More Prone to Hallucination? (Read more on arXiv or HuggingFace) |
Junfeng Fang, Jianhui Chen, Yanxu Chen, Yantao Liu, Zijun Yao |
i) This paper investigates the hallucination propensities of Large Reasoning Models (LRMs) compared to base models. ii) The primary research question addresses whether incorporating reasoning capabilities in LRMs leads to increased or decreased hallucination in fact-seeking tasks. iii) The methodology includes evaluating LRMs across factuality benchmarks (SimpleQA, TriviaQA) and analyzing cognitive behaviors like flaw repetition and think-answer mismatch, alongside probing internal model uncertainty. iv) Results indicate that SFT+RL trained LRMs reduce hallucination (e.g., DeepSeek-R1 achieved 28.5% accuracy on SimpleQA), while RL-only and SFT-only trained LRMs are more prone to hallucination and exhibit mis-calibrated uncertainty. v) AI practitioners should consider the post-training pipeline, specifically employing both supervised fine-tuning and verifiable reward reinforcement learning to develop factual and reliable LRMs, and to be aware of potential uncertainty corruption in RL-only or SFT-only training. |
| cadrille: Multi-modal CAD Reconstruction with Online Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Ilya Zisman, Alexander Nikulin, Denis Tarasov, Maksim Kolodiazhnyi, zhemchuzhnikov |
cadrille introduces a multi-modal CAD reconstruction model leveraging vision-language models and reinforcement learning. The research aims to improve CAD reconstruction by processing point clouds, images, and text simultaneously. The methodology involves supervised fine-tuning (SFT) on procedurally generated data followed by reinforcement learning (RL) fine-tuning using online feedback via Group Relative Preference Optimization (GRPO). Results show that cadrille outperforms existing methods, achieving state-of-the-art on multiple CAD datasets; specifically, RL fine-tuning reduces the invalidity ratio to below 0.2% on real-world CC3D datasets. This suggests AI practitioners can leverage online RL with VLMs to enhance CAD reconstruction and improve the robustness and validity of generated models. |
| Multi-Domain Explainability of Preferences (Read more on arXiv or HuggingFace) |
Roi Reichart, Liat Ein-Dor, Nitay Calderon |
i) The paper introduces an automated method for concept-based explainability of preferences across multiple domains using Large Language Models (LLMs). ii) The research aims to generate local and global concept-based explanations for preference mechanisms, including human preference, LLM-as-a-Judge, and reward models. iii) The methodology involves using an LLM for concept discovery, representing examples as concept vectors, and modeling relationships between concepts and preferences with a hierarchical multi-domain regression (HMDR) model. iv) The method achieves strong preference prediction performance and explanation quality across eight datasets and twelve mechanisms; prompting LLMs with concepts from LaaJ explanations yields responses that those judges consistently prefer. v) The principal implication for AI practitioners is a new paradigm for explainability in the era of LLMs, providing tools to better understand and guide preference mechanisms in AI alignment and evaluation. |
| UniRL: Self-Improving Unified Multimodal Models via Supervised and |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, Zhenheng Yang, Weijia Mao |
i) UniRL introduces a self-improving post-training method for unified multimodal models using supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO). ii) The research aims to enhance both image generation and understanding capabilities of unified multimodal models without relying on external image data. iii) The methodology involves constructing prompts and QA pairs, using the model to generate images, and then using these images for training in each iteration via SFT and GRPO. iv) Evaluated on Show-o and Janus, UniRL achieves a GenEval score of 0.77 for Show-o and 0.65 for Janus after post-training, improving both generation and understanding. v) UniRL offers AI practitioners a method to improve unified multimodal models with reduced data requirements, focusing on balancing generation and understanding tasks, potentially reducing task imbalance and facilitating efficient model optimization. |
| SWE-bench Goes Live! (Read more on arXiv or HuggingFace) |
Bowen Li, Yu Kang, Chaoyun Zhang, Shilin He, Linghao Zhang |
i) SWE-bench-Live is introduced as a continuously updated benchmark for evaluating large language models (LLMs) on real-world software issue resolution tasks. ii) The research objective is to address the limitations of static software engineering benchmarks, such as data staleness, limited repository diversity, and manual environment setup. iii) The methodology involves an automated curation pipeline, REPOLAUNCH, for creating Docker-based execution environments and validating issue-pull request pairs. iv) Evaluation of agent frameworks on SWE-bench-Live reveals a resolved rate of 19.25% achieved by OpenHands with Claude 3.7 Sonnet, contrasting with higher performance on SWE-bench Verified (43.20%) under identical conditions. v) SWE-bench-Live facilitates rigorous and contamination-resistant evaluation of LLMs and agents in dynamic, real-world software development settings, underscoring the importance of up-to-date benchmarks for measuring true model generalization for AI practitioners. |
| Train Sparse Autoencoders Efficiently by Utilizing Features Correlation (Read more on arXiv or HuggingFace) |
Nikita Balagansky, Daniil Gavrilov, Daniil Laptev, Yaroslav Aksenov, Vadim Kurochkin |
i) This paper introduces KronSAE, an efficient sparse autoencoder (SAE) architecture leveraging Kronecker factorization and a differentiable AND-like gating mechanism (mAND) to reduce computational overhead. ii) The research aims to address the scalability bottleneck in training SAEs, specifically the computationally intensive encoder projection. iii) KronSAE factorizes the latent representation via Kronecker product decomposition and employs a novel mAND activation function. iv) Experiments on Qwen-1.5B show that KronSAE improves explained variance by up to 4.3% with 54.7% fewer parameters under a 100M token budget compared to TopK SAE, while matching or exceeding TopK baseline reconstruction quality with 46.1% fewer parameters at 1000M tokens. v) KronSAE offers AI practitioners a more scalable approach to training SAEs for interpretability by reducing encoder cost and improving feature disentanglement, enabling efficient analysis of large language model activations. |
| Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV |
|
|
| Cache and Parallel Decoding (Read more on arXiv or HuggingFace) |
Shizhe Diao, Hao Zhang, Chengyue Wu, zhijianliu, Cauthyyy |
i) The paper introduces Fast-dLLM, a method for accelerating diffusion-based language models (dLLMs) without retraining by incorporating KV Cache and parallel decoding. ii) The research aims to improve the inference speed of open-sourced dLLMs, which typically lag behind autoregressive models due to the absence of KV Cache and quality degradation during parallel token generation. iii) The methodology involves a block-wise approximate KV Cache mechanism and a confidence-aware parallel decoding strategy to mitigate token dependency violations. iv) Experimental results demonstrate up to 27.6x throughput improvement on LLaDA and Dream models across multiple benchmarks, closing the performance gap with autoregressive models. v) The implementation of Fast-dLLM provides AI practitioners with a practical, training-free solution to accelerate dLLM inference, enhancing their applicability in real-world deployments. |
| Muddit: Liberating Generation Beyond Text-to-Image with a Unified |
|
|
| Discrete Diffusion Model (Read more on arXiv or HuggingFace) |
Kaidong Yu, Wenhao Chai, Zhuoran Zhao, BryanW, QingyuShi |
i) Muddit introduces a unified discrete diffusion transformer for fast, parallel generation across text and image modalities. ii) The paper aims to develop a unified generative model capable of handling diverse tasks across modalities within a single architecture. iii) The methodology involves integrating visual priors from a pretrained text-to-image backbone with a lightweight text decoder in a MaskGIT-style discrete diffusion transformer. iv) Muddit achieves a strong overall accuracy of 0.61 on the GenEval benchmark, outperforming previous discrete diffusion models, while utilizing only 1B parameters. v) This work suggests that purely discrete diffusion, when equipped with strong visual priors, can serve as a scalable and effective backbone for unified generation, offering AI practitioners an alternative to autoregressive models for multimodal tasks. |
| LoRAShop: Training-Free Multi-Concept Image Generation and Editing with |
|
|
| Rectified Flow Transformers (Read more on arXiv or HuggingFace) |
Pinar Yanardag, Hidir Yesiltepe, ydalva |
LoRAShop introduces a training-free framework for multi-concept image generation and editing using rectified flow transformers. The research aims to enable the simultaneous use of multiple LoRA adapters for image synthesis and manipulation without additional training or auxiliary inputs. LoRAShop extracts subject priors by analyzing feature interaction patterns in rectified flow models and blends LoRA weights within concept-specific regions. Experiments demonstrate that LoRAShop delivers improved identity preservation compared to baselines and blends multiple concepts directly into the diffusion latent without retraining. LoRAShop enables region-controlled personalized image editing, enhancing creative workflows and facilitating visual storytelling for AI practitioners. |
| On-Policy RL with Optimal Reward Baseline (Read more on arXiv or HuggingFace) |
Zewen Chi, Shaohan Huang, Xun Wu, Li Dong, Yaru Hao |
The paper introduces On-Policy RL with Optimal reward baseline (OPO), a novel reinforcement learning algorithm. The research aims to improve training stability and exploration in RL for large language model alignment by minimizing gradient variance. OPO employs exact on-policy training and derives an optimal reward baseline that theoretically minimizes gradient variance. Experiments on mathematical reasoning benchmarks show OPO achieves superior performance and training stability without additional models or regularization, demonstrating lower policy shifts and higher output entropy. OPO consistently achieves higher performance with more stable training dynamics compared to GRPO. These results suggest AI practitioners can leverage OPO for more stable and effective reinforcement learning in tasks such as large language model alignment and reasoning. |
| GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action |
|
|
| Control (Read more on arXiv or HuggingFace) |
Kun Zhan, Xueyang Zhang, Wenzhao Zheng, wangyida, antonio-c |
i) The paper introduces GeoDrive, a driving world model that integrates 3D geometry to enhance action controllability and spatial understanding. ii) The research aims to develop a controllable driving world model that maintains 3D geometric consistency and allows for precise ego-vehicle trajectory control. iii) The methodology involves extracting a 3D representation from monocular input, rendering 2D views along specified trajectories, and using a dynamic editing module for enhanced dynamic modeling. iv) Experiments show GeoDrive reduces trajectory following errors by 42% compared to the Vista model, while also achieving improvements in video quality metrics such as LPIPS, PSNR, SSIM, FID, and FVD. v) The principal implication for AI practitioners is a new approach to building driving world models with improved action controllability and spatial awareness, leading to more realistic and reliable scene modeling for autonomous driving systems. |
| ATLAS: Learning to Optimally Memorize the Context at Test Time (Read more on arXiv or HuggingFace) |
Yuan Deng, Majid Daliri, Praneeth Kacham, Zeman Li, Ali Behrouz |
i) The paper introduces ATLAS, a new long-term memory module for improving context memorization in recurrent neural networks. ii) The research aims to address limitations in existing recurrent models related to memory capacity, online update strategies, and memory management expressiveness. iii) The methodology involves developing a sliding window update rule (Omega rule) and architectures utilizing polynomial feature mappings and a Muon optimizer. iv) ATLAS achieves +80% accuracy in the 10M context length of BABILong benchmark and outperforms Transformers and linear recurrent models on language modeling and common-sense reasoning tasks. v) The ATLAS architecture provides AI practitioners with a scalable approach for enhancing long-context understanding in tasks such as language modeling and reasoning, by addressing memory limitations and optimization challenges in recurrent networks. |
| KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction (Read more on arXiv or HuggingFace) |
Sangdoo Yun, Jae W. Lee, Sangwoo Kwon, jusjinuk, Jang-Hyun |
KVzip introduces a query-agnostic KV cache eviction method for transformer-based LLMs to improve inference efficiency. The research objective is to optimize a reusable compressed KV cache by quantifying the importance of KV pairs based on their contribution to context reconstruction via an LLM forward pass and attention scores. The methodology involves teacher-forced decoding to simulate context reconstruction, assigning importance scores to KV pairs based on maximum attention scores, and evicting lower-importance pairs. Experiments show KVzip reduces KV cache size by 3-4× and FlashAttention decoding latency by approximately 2×, with negligible performance loss across various tasks including models like LLaMA3.1-8B and context lengths up to 170K tokens. KVzip’s ability to reduce KV cache size significantly while maintaining performance offers AI practitioners a practical approach to alleviate memory constraints and improve the efficiency of deploying long-context LLMs. |
| SafeScientist: Toward Risk-Aware Scientific Discoveries by LLM Agents (Read more on arXiv or HuggingFace) |
Ziheng Qi, Jiaxun Zhang, HakHan, m-serious, Leozkl |
i) The paper introduces SafeScientist, an AI scientist framework designed to enhance safety and ethical responsibility in AI-driven scientific exploration. ii) The main objective is to address ethical and safety concerns raised by large language model (LLM) agents in scientific discovery automation. iii) The methodology involves integrating multiple defensive mechanisms including prompt monitoring, agent-collaboration monitoring, tool-use monitoring, and an ethical reviewer component. iv) Experiments demonstrate that SafeScientist improves safety performance by 35% compared to traditional AI scientist frameworks. v) The principal implication is a framework and benchmark to address and mitigate ethical and safety risks when deploying LLM agents in scientific research workflows. |
| ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind (Read more on arXiv or HuggingFace) |
Jiaxuan You, m-serious, HakHan |
i) The paper introduces ToMAP, a framework for training more effective LLM persuaders by integrating theory of mind (ToM) modules. ii) The research aims to improve LLM persuaders by enabling them to better model and reason about their opponent’s mental state during conversations. iii) The methodology incorporates a counterclaim predictor and an opponent attitude predictor, leveraging reinforcement learning to train the LLM persuader. iv) Experiments show that the ToMAP persuader, with only 3B parameters, outperforms larger models like GPT-4o, achieving a 39.4% relative gain in persuasiveness across diverse corpora. v) ToMAP allows AI practitioners to create more sophisticated and adaptable dialogue agents that can dynamically respond to opponent’s viewpoints, demonstrating the potential of incorporating ToM in persuasive language agents. |
| Uni-Instruct: One-step Diffusion Model through Unified Diffusion |
|
|
| Divergence Instruction (Read more on arXiv or HuggingFace) |
Weijian Luo, Debing Zhang, Colin Zhang, Weimin Bai, smallAI |
i) The paper introduces Uni-Instruct, a novel framework for one-step diffusion model distillation. ii) The research aims to unify existing one-step diffusion distillation methods within a theoretical framework based on f-divergence minimization and improve generation performance. iii) The methodology involves a diffusion expansion theory for f-divergences and derivation of a tractable loss function with equivalent parameter gradients. iv) Uni-Instruct achieves a new SoTA on ImageNet64 × 64 conditional generation with an FID of 1.02, outperforming its 79-step teacher diffusion model, and attains an FID of 1.46 for unconditional CIFAR10 generation. v) AI practitioners can use Uni-Instruct to achieve improved performance in one-step diffusion models for image generation tasks due to the method’s unified framework and state-of-the-art results. |
| PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient |
|
|
| Interactions (Read more on arXiv or HuggingFace) |
Jae Ho Sohn, Jiho Kim, Seongsu Bae, Hyunseung Chung, Daeun Kyung |
i) PatientSim introduces a persona-driven patient simulator for evaluating doctor LLMs in realistic, multi-turn interactions. ii) The primary objective is to create a patient simulator capable of generating diverse patient personas grounded in clinical scenarios, thereby addressing limitations of existing simulators. iii) The methodology involves constructing clinical profiles from MIMIC-ED and MIMIC-IV datasets and defining patient personas along four axes: personality, language proficiency, medical history recall, and cognitive confusion. iv) Evaluation across eight LLMs revealed that Llama 3.3 demonstrated the best factual accuracy and persona consistency, validated by clinicians with an average quality score of 3.89 out of 4. v) PatientSim offers AI practitioners a customizable, open-source platform for reproducible and scalable evaluation of medical dialogue systems, facilitating privacy-compliant testing and educational applications. |
| DeepTheorem: Advancing LLM Reasoning for Theorem Proving Through Natural |
|
|
| Language and Reinforcement Learning (Read more on arXiv or HuggingFace) |
Qiuzhi Liu, Tian Liang, Zhiwei He, Jiahao Xu, Ziyin Zhang |
DeepTheorem introduces a new framework for informal theorem proving using natural language and reinforcement learning. The research aims to improve LLM’s mathematical reasoning abilities by creating a large-scale benchmark dataset. The methodology uses a novel reinforcement learning strategy (RL-Zero) and a dataset of 121K high-quality informal mathematical theorems and proofs. Experiments show DeepTheorem significantly improves LLM theorem-proving performance, achieving state-of-the-art results; specifically, RL-Zero training on a 7B model with the DeepTheorem dataset achieves strong performance. The findings imply that DeepTheorem has potential to advance automated informal theorem proving and mathematical exploration for AI practitioners by fundamentally transforming automated informal theorem proving and mathematical exploration. |
| MAGREF: Masked Guidance for Any-Reference Video Generation (Read more on arXiv or HuggingFace) |
Jacob Zhiyuan Fang, Yuanyang Yin, Xun Guo, Yufan Deng, BestWishYsh |
i) MAGREF is a novel video generation framework utilizing masked guidance for coherent multi-subject video synthesis from reference images and text prompts. ii) The main objective is to achieve stable and high-quality video generation that preserves multi-subject consistency and adheres to detailed textual instructions, addressing challenges in existing multi-subject video generation methods. iii) The methodology introduces a region-aware dynamic masking mechanism for flexible subject inference and pixel-wise channel concatenation for improved appearance feature preservation, trained on a self-curated video dataset. iv) MAGREF achieves state-of-the-art performance, establishing a new state-of-the-art in Face Similarity (FaceSim) at 0.567 for single-ID and 0.581 for multi-subject test cases. v) MAGREF offers AI practitioners a scalable and controllable method for high-fidelity multi-subject video synthesis, demonstrating effective domain adaptation and accelerated training convergence without substantial architectural modifications to pre-trained models. |
| FAMA: The First Large-Scale Open-Science Speech Foundation Model for |
|
|
| English and Italian (Read more on arXiv or HuggingFace) |
Mauro Cettolo, Alessio Brutti, Luisa Bentivogli, Marco Gaido, Sara Papi |
FAMA introduces open-science speech foundation models for English and Italian. The paper addresses the limited accessibility of training data and codebases in existing SFMs. It trains models on 150k+ hours of open-source speech data, including a new 16k-hour dataset of cleaned and pseudo-labeled speech. FAMA achieves competitive performance compared to existing SFMs while being up to 8 times faster. This allows AI practitioners to have fully accessible resources, to enable better reproducibility. |
| Afterburner: Reinforcement Learning Facilitates Self-Improving Code |
|
|
| Efficiency Optimization (Read more on arXiv or HuggingFace) |
Dong Huang, Yuhao Qing, Yue Liu, Luu Tuan Tuan, Mingzhe Du |
i) The paper introduces Afterburner, an iterative optimization framework leveraging reinforcement learning to improve code efficiency generated by LLMs. ii) The research aims to enhance the computational efficiency of LLM-generated code through test-time iterative refinement. iii) The methodology involves a closed-loop system with Afterburner iteratively refining code based on performance feedback from the Monolith execution sandbox, explored using Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO). iv) Experiments show that GRPO boosts PASS @1 from 47% to 62% and increases the likelihood of outperforming human submissions in efficiency from 31% to 45% on Venus. v) Reinforcement learning, specifically GRPO, is revealed as a powerful approach for training LLMs to self-improve code efficiency, enabling effective test-time code optimization for AI practitioners. |
| Differentiable Solver Search for Fast Diffusion Sampling (Read more on arXiv or HuggingFace) |
Xubin Li, Qipeng zhang, Zexian Li, sthuihui, wangsssssss |
i) This paper introduces a differentiable solver search algorithm to optimize sampling efficiency in diffusion models. ii) The main objective is to identify optimal timesteps and solver coefficients for accelerating reverse-diffusion solving without retraining. iii) The methodology involves defining a compact search space of time steps and solver coefficients and then using a differentiable search algorithm to optimize these parameters. iv) The proposed method achieves a FID score of 2.33 on the DDPM model, DiT-XL/2, with only 10 steps, which beats the performance of traditional solvers. v) AI practitioners can leverage this method to enhance the efficiency of pre-trained diffusion models for faster image generation. |
| To Trust Or Not To Trust Your Vision-Language Model’s Prediction (Read more on arXiv or HuggingFace) |
Olga Fink, Eleni Chatzi, Jian Liang, Moru Liu, hdong51 |
Vision-Language Models (VLMs) often yield confident yet incorrect predictions, especially in safety-critical domains. The research addresses the critical challenge of estimating when VLM predictions can be trusted without retraining. TrustVLM, a training-free framework, is introduced, leveraging image embedding space and a novel confidence-scoring function based on image-to-text and image-to-image similarity. The framework was evaluated across 17 datasets, 4 architectures, and 2 VLMs and demonstrated state-of-the-art performance, with improvements of up to 51.87% in AURC, 9.14% in AUROC, and 32.42% in FPR95 compared to existing baselines. TrustVLM enables safer deployment of VLMs by providing a more reliable method for assessing prediction confidence. |
| UniTEX: Universal High Fidelity Generative Texturing for 3D Shapes (Read more on arXiv or HuggingFace) |
Hongyu Yan, Rui Chen, Xiao Chen, Kunming Luo, Yixun Liang |
i) UniTEX introduces a two-stage framework for generating high-quality 3D textures by directly operating in a 3D functional space via Texture Functions (TFs). ii) The research aims to bypass UV mapping limitations in 3D texture generation by predicting TFs from images and geometry using a transformer-based Large Texturing Model (LTM). iii) The methodology involves lifting texture generation into 3D space using TFs, predicting these TFs with a transformer-based LTM, and employing a LoRA-based strategy to adapt large-scale Diffusion Transformers (DiTs). iv) Experiments show UniTEX achieves superior visual quality and texture integrity compared to existing approaches, reflected in a 65.91% user preference score, demonstrating a generalizable solution for automated 3D texture generation. v) UniTEX offers AI practitioners a scalable solution for automated 3D texture generation by eliminating UV mapping dependencies, enabling more robust and consistent texture creation across diverse mesh topologies. |
| CXReasonBench: A Benchmark for Evaluating Structured Diagnostic |
|
|
| Reasoning in Chest X-rays (Read more on arXiv or HuggingFace) |
Hyuk Gi Hong, Hangyul Yoon, Jung-Oh Lee, Geon Choi, ttumyche |
i) CXReasonBench, consisting of CheXStruct and CXReasonBench, is introduced to evaluate structured diagnostic reasoning in chest X-rays using MIMIC-CXR-JPG. ii) The objective is to assess the ability of Large Vision-Language Models (LVLMs) to perform clinically valid reasoning steps in chest X-ray diagnosis. iii) CheXStruct automatically derives intermediate reasoning steps from chest X-rays, including anatomical segmentation, landmark identification, diagnostic measurements, index computation, and clinical threshold application; CXReasonBench utilizes this pipeline for model evaluation. iv) CXReasonBench, comprising 18,988 QA pairs across 12 diagnostic tasks and 1,200 cases, reveals that current LVLMs struggle with structured reasoning and generalization in tasks requiring both abstract knowledge and visual grounding, with the strongest models rarely reaching beyond Stage 2 reasoning. v) The primary implication for AI practitioners is the identification of a critical gap in the ability of current LVLMs to integrate abstract diagnostic knowledge with anatomically grounded visual interpretation, highlighting a need for research focusing on improved visual grounding and structured reasoning for medical image analysis. |
| Breaking Down Video LLM Benchmarks: Knowledge, Spatial Perception, or |
|
|
| True Temporal Understanding? (Read more on arXiv or HuggingFace) |
Simon Wang, Zizhen Wang, Shiyu Li, Zhengfeng Lai, Bo Feng |
i) This paper presents VBenchComp, a framework to dissect video Large Language Model (LLM) benchmarks. ii) The primary objective is to categorize video QA benchmark questions to isolate temporal reasoning ability from language priors and static visual understanding. iii) The methodology involves an automated pipeline that classifies questions into LLM-Answerable, Semantic, and Temporal categories based on model performance with/without video and after frame shuffling. iv) Analysis reveals models can achieve up to 50% accuracy on VideoMME and NEXT-QA without video input using GPT-4o, and shuffling frames often does not significantly affect performance; this suggests reliance on language priors or static semantic content. v) The key implication is that AI practitioners should use VBenchComp to refine video LLM benchmarks, focusing on temporal questions to better evaluate models’ true video understanding capabilities. |
| Differential Information: An Information-Theoretic Perspective on |
|
|
| Preference Optimization (Read more on arXiv or HuggingFace) |
Minjoon Seo, Hyeonbin Hwang, Hyunji Lee, yunjae-won |
i) The paper presents an information-theoretic analysis of Direct Preference Optimization (DPO) through a novel concept called Differential Information Distribution (DID). ii) The research aims to establish the theoretical conditions under which the log-ratio reward parameterization in DPO is optimal for aligning language models. iii) The methodology involves formalizing DID to characterize information gain in policy updates and analyzing the entropy of DID to understand policy dynamics. iv) The study proves that DPO’s log-ratio reward is uniquely optimal when preferences encode differential information and finds this condition is linked to log-margin ordered policies; experiments indicate that learning high-entropy differential information is crucial for general instruction-following. v) This analysis provides AI practitioners with a framework for understanding and potentially improving DPO by considering the information-theoretic properties of preference data and its impact on policy behavior, guiding the development of more effective alignment strategies. |
| REOrdering Patches Improves Vision Models (Read more on arXiv or HuggingFace) |
Trevor Darrell, Yutong Bai, David M. Chan, RitwikGupta, d3tk |
i) This paper introduces REOrder, a framework that learns task-optimal patch orderings to improve vision model performance. ii) The research investigates how patch order affects the performance of long-sequence vision models, aiming to discover optimal orderings. iii) The methodology involves an information-theoretic prior based on patch sequence compressibility, combined with reinforcement learning using a Plackett-Luce policy and REINFORCE to optimize patch permutations. iv) REOrder improves top-1 accuracy over row-major ordering on ImageNet-1K by up to 3.01% and Functional Map of the World by 13.35%. v) The principal implication is that AI practitioners can leverage REOrder to enhance the performance of long-sequence vision models by optimizing patch orderings, particularly in scenarios where architectural approximations introduce sensitivity to patch sequence. |
| ZeroSep: Separate Anything in Audio with Zero Training (Read more on arXiv or HuggingFace) |
Yunlong Tang, Susan Liang, Junxuan Huang, Yuesheng Ma, Chao Huang |
ZeroSep is a zero-shot audio source separation framework leveraging pre-trained text-guided audio diffusion models. The research investigates whether generative foundation models can achieve source separation without task-specific training. ZeroSep inverts mixed audio into a diffusion model’s latent space and uses text conditioning to guide the denoising process for individual source recovery. Without training, ZeroSep surpasses supervised methods on separation benchmarks, for example, achieving a FAD score of 0.377 on the MUSIC dataset. This demonstrates the potential of repurposing generative models for discriminative tasks, offering a training-free approach to open-set audio separation for AI practitioners. |
| Re-ttention: Ultra Sparse Visual Generation via Attention Statistical |
|
|
| Reshape (Read more on arXiv or HuggingFace) |
Di Niu, Chao Gao, LiyaoJiang, kgmills, crc5577 |
i) The paper introduces Re-ttention, a novel sparse attention mechanism for Diffusion Transformers (DiTs) aimed at improving the efficiency of visual generation. ii) The main objective is to develop a high-sparsity attention mechanism that minimizes visual quality degradation in text-to-video (T2V) and text-to-image (T2I) models without requiring model retraining. iii) The methodology involves statistically reshaping attention distributions distorted by sparse attention, leveraging the temporal redundancy in Diffusion Models and caching/reusing softmax statistics from previous denoising steps. iv) Experimental results on CogVideoX and PixArt DiTs demonstrate that Re-ttention achieves up to 96.9% sparsity, leading to over 92% self-attention latency reduction on an H100 GPU and over 45% end-to-end latency reduction. v) The primary implication for AI practitioners is a training-free method to significantly accelerate DiT inference while maintaining visual quality, enabling more efficient deployment of these models. |
| StressTest: Can YOUR Speech LM Handle the Stress? (Read more on arXiv or HuggingFace) |
Yossi Adi, gallilmaimon, iyosha |
i) The paper introduces StressTest, a benchmark for evaluating sentence stress understanding in speech-aware language models (SLMs). ii) The research investigates the ability of SLMs to distinguish spoken sentence meanings based on varying stress patterns. iii) The study utilizes a novel synthetic data generation pipeline to create Stress-17k for finetuning SLMs. iv) Evaluations show that the finetuned model, StresSLM, achieves 81.6% accuracy on the sentence stress reasoning task, outperforming existing SLMs. v) The improved performance on stress understanding, without significantly impacting original SLM tasks, suggests that AI/ML engineers can enhance spoken language understanding by explicitly incorporating stress pattern analysis in model training and evaluation. |
| One-shot Entropy Minimization (Read more on arXiv or HuggingFace) |
Bryan Dai, Joey Zhou, Lynx Chen, zgao3186 |
i) This paper introduces One-shot Entropy Minimization (EM), a novel unsupervised post-training technique for large language models. ii) The primary objective is to demonstrate that entropy minimization with minimal data can significantly improve LLM performance. iii) The methodology involves training 13,440 LLMs using a single unlabeled data point and optimizing for 10 steps based on an entropy minimization loss. iv) Results indicate that One-shot EM achieves a 24.7 point average performance gain across multiple math reasoning benchmarks compared to the original Qwen2.5-Math-7B model; specifically, it increases the score on MATH500 by 25.8 points, going from 53.0 to 78.8. v) This implies that AI practitioners can use EM as a computationally efficient method for enhancing LLM performance, potentially rivalling or surpassing RL-based fine-tuning, and calls for reconsidering post-training paradigms. |
| ChartLens: Fine-grained Visual Attribution in Charts (Read more on arXiv or HuggingFace) |
Ryan A. Rossi, Nedim Lipka, Manan Suri, Franck-Dernoncourt, puneetm |
i) The paper introduces ChartLens, a novel chart attribution algorithm, and ChartVA-Eval, a benchmark for fine-grained visual attribution in charts, to address hallucinations in multimodal large language models (MLLMs). ii) The main objective is to develop a post-hoc visual attribution method for charts that identifies specific chart elements validating a given response. iii) ChartLens uses segmentation-based techniques to identify chart objects and set-of-marks prompting with MLLMs for fine-grained visual attribution; ChartVA-Eval comprises real-world and synthetic charts with attribution annotations. iv) Evaluations show that ChartLens improves fine-grained attributions by 26-66% compared to baselines. v) ChartLens enables AI practitioners to improve the transparency and reliability of MLLMs in chart understanding tasks by grounding model responses in verifiable visual elements. |
| A Graph Perspective to Probe Structural Patterns of Knowledge in Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Yongjia Lei, Zhisheng Qi, Utkarsh Sahu, mhalappa, Franck-Dernoncourt |
A graph-based approach is proposed to analyze structural patterns of knowledge in LLMs by quantifying knowledgeability at the triplet and entity levels. The research investigates how LLM knowledge relates to graph structural properties such as node degree and homophily. LLM knowledgeability scores are estimated using graph-based regression models leveraging local neighborhood context in knowledge graphs, and these models are trained with message-passing GNNs. It was found that LLMs exhibit knowledge homophily, where topologically proximate entities show similar knowledgeability, and that node degree correlates with knowledgeability, with regression achieving absolute errors between 0.15 and 0.25. These models can be utilized to prioritize high-value triplet facts for more effective LLM fine-tuning. It is unclear from the paper what are the computational cost of implementing the proposed GNN for predicting knowledgeability scores, or the generalizability to other LLMs and Knowledge Graphs. |
| Can LLMs Deceive CLIP? Benchmarking Adversarial Compositionality of |
|
|
| Pre-trained Multimodal Representation via Text Updates (Read more on arXiv or HuggingFace) |
Gunhee Kim, Dayoon Ko, Heeseung Yun, ahnpersie |
i) The paper introduces Multimodal Adversarial Compositionality (MAC), a benchmark for evaluating the compositional vulnerability of pre-trained multimodal representations. ii) The research aims to benchmark how effectively large language models can generate deceptive text to exploit compositional vulnerabilities in multimodal representations like CLIP across images, videos, and audio. iii) The methodology involves using LLMs for generating deceptive captions, filtering them through sample-wise and group-wise evaluations, and self-training the LLMs using rejection sampling fine-tuning. iv) Experiments demonstrate superior performance with the Llama-3.1-8B model, improving attack success rates by over 68% and diversity without sacrificing attack performance, across tested representations; the method achieves over 93% with GPT-4 during verification. v) AI practitioners can use the MAC benchmark and the proposed self-training approach to evaluate and improve the robustness of multimodal systems against adversarial compositional attacks across different modalities. |
| When Models Reason in Your Language: Controlling Thinking Trace Language |
|
|
| Comes at the Cost of Accuracy (Read more on arXiv or HuggingFace) |
Danielle S. Bitterman, Raquel Fernández, Zidi Xiong, Shan Chen, Jirui Qi |
i) This paper investigates the trade-off between language matching in reasoning traces and answer accuracy in Large Reasoning Models (LRMs) across multiple languages. ii) The research question is to what extent LRMs can reason in a user’s native language and how this affects reasoning accuracy. iii) The methodology involves evaluating six open-sourced LRMs using a new benchmark, XReasoning, which includes translated math and science questions, and applying prompt-hacking and post-training techniques. iv) Results show prompt hacking increases the language matching rate from 45-50% to above 90%, but reduces average accuracy on AIME questions from 26% to 17% for Distilled-R1-32B. v) AI practitioners should be aware that forcing LRMs to generate reasoning traces in a specific language through techniques like prompt hacking comes at a cost to the model’s answer accuracy. |
| CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian |
|
|
| Splatting (Read more on arXiv or HuggingFace) |
Marcin Mazur, Tadeusz Dziarmaga, Piotr Borycki, Joanna Waczyńska, Kornel Howil |
CLIPGaussian presents a universal style transfer model applicable across diverse data modalities using Gaussian Splatting. The research investigates how to achieve text- and image-guided stylization across 2D images, videos, 3D objects, and 4D scenes. The methodology involves operating directly on Gaussian primitives and integrating into existing GS pipelines, optimizing color and geometry. Experimental results demonstrate that CLIPGaussian attains superior style fidelity and consistency; user studies show CLIPGaussian achieves scores comparable to G-Style with image conditioning. The research offers AI practitioners a universal and efficient solution for multimodal style transfer without requiring retraining from scratch, particularly for tasks involving diverse data types. The quantitative results and method’s plug-in nature hold substantial implication for easing style transfer in various modalities. |
| VidText: Towards Comprehensive Evaluation for Video Text Understanding (Read more on arXiv or HuggingFace) |
Yu Li, Yan Zhang, Zhifei Yang, Yan Shu, Zhoufaran Yang |
VidText introduces a new benchmark for evaluating video text understanding capabilities of Large Multimodal Models (LMMs). The research aims to provide a comprehensive evaluation of LMMs in dynamic visual environments containing textual information. VidText employs a hierarchical evaluation framework across video, clip, and instance levels, paired with perception-reasoning tasks, covering a diverse range of real-world scenarios and multilingual content. Experiments on 18 state-of-the-art LMMs demonstrate current models’ limitations, with the best model, Gemini 1.5 Pro, achieving only 46.8% average performance. This highlights the need for advancements in model architecture, OCR capability, and reasoning strategies for AI practitioners working with video understanding tasks. The primary findings underscored the challenge of multi-granularity tasks in videos and the impact of OCR capability on overall performance. |
| SridBench: Benchmark of Scientific Research Illustration Drawing of |
|
|
| Image Generation Model (Read more on arXiv or HuggingFace) |
Chuanhao Li, Jiaxin Ai, Jianwen Sun, Yukang Feng, Yifan Chang |
i) SridBench is introduced as a new benchmark for evaluating multimodal models in generating scientific research illustrations. ii) The primary objective is to assess the capability of AI models, particularly multimodal large language models, in accurately interpreting technical descriptions and creating standardized visual representations of scientific concepts. iii) The methodology involves compiling a dataset of 1,120 instances of illustrations and associated text from scientific papers across 13 disciplines, annotated and evaluated along six dimensions including semantic fidelity and structural accuracy, using both human experts and large language models. iv) Experiments indicate that current state-of-the-art models like GPT-4o-image underperform compared to human-level performance, with GPT-4o-image achieving an average score of “fair” and open-source models scoring near 1, while Gemini-2.0-Flash reaching approximately 1.0. v) The principal implication for AI practitioners is the identification of critical bottlenecks, specifically a lack of text and visual information understanding and the presence of scientific errors, underscoring the need for further advancements in reasoning-driven visual generation for scientific applications. |
| Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking (Read more on arXiv or HuggingFace) |
Ruichuan An, Renrui Zhang, Joey Tsai, Shilin Yan, Pengxiang Li |
i) This paper introduces Adaptive Classifier-Free Guidance (A-CFG) to improve controllability in iterative masked language models. ii) The research addresses the limitation of standard CFG’s static unconditioning input by dynamically tailoring it based on model confidence. iii) A-CFG re-masks low-confidence tokens during each denoising step to construct a localized unconditional input. iv) Experiments on various language generation benchmarks show A-CFG achieves a 3.9 point improvement on GPQA compared to standard CFG. v) AI practitioners can use A-CFG to enhance conditional text generation in iterative diffusion models by dynamically adjusting guidance based on predictive confidence. |
| Evaluating Text Creativity across Diverse Domains: A Dataset and Large |
|
|
| Language Model Evaluator (Read more on arXiv or HuggingFace) |
Fang Luo, Yahui Liu, Yuzhuo Yuan, Xiting Wang, Aman |
i) This paper introduces CreataSet, a dataset and LLM-based evaluator for assessing textual creativity across diverse domains. ii) The main objective is to develop an effective, automated methodology for evaluating text creativity that addresses limitations of cross-domain applicability, granularity, and human effort. iii) The methodology involves a pairwise-comparison framework with shared contextual instructions, a large-scale dataset of human and synthetic creative instruction-response pairs, and an LLM-based evaluator (CrEval) trained on the dataset. iv) CrEval demonstrates superior alignment with human judgments, outperforming GPT-4o by 18.7% in agreement with human judges; even state-of-the-art LLMs still perform poorly on the meta-evaluation benchmark test set. v) AI practitioners should integrate human-generated and synthetic data when training evaluators, leveraging CrEval as a practical tool to assess and boost the creativity of LLMs in generation pipelines, given its domain generalization capabilities. |
Papers for 2025-05-29
| Title |
Authors |
Summary |
| The Entropy Mechanism of Reinforcement Learning for Reasoning Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Haozhan72, yuxinzuo, JC-Chen, YucZhang2003, ganqu |
i) The paper investigates the collapse of policy entropy in reinforcement learning (RL) for reasoning language models (LLMs) and proposes techniques to mitigate this issue. ii) The research aims to understand and control the dynamics of policy entropy in RL-tuned LLMs to improve exploration and downstream task performance. iii) The methodology involves theoretical derivation of entropy dynamics, empirical analysis of covariance between action probabilities and logit changes, and the introduction of Clip-Cov and KL-Cov regularization techniques. iv) The primary result is the establishment of a transformation equation R = -a exp H + b between entropy H and downstream performance R, indicating that performance is traded from policy entropy, and experiments showed that Clip-Cov and KL-Cov led to better downstream performance by approximately 2.0% on the 7B model and 6.4% on the 32B model, on average. v) AI/ML engineers can leverage the Clip-Cov and KL-Cov methods to encourage exploration and improve the performance of RL-trained LLMs by mitigating the entropy collapse issue, ultimately leading to a better scaling of compute resources for RL. |
| R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large |
|
|
| Model Token Routing (Read more on arXiv or HuggingFace) |
Zhihang Yuan, Enshu Liu, Yi Ge, youyc22, fuvty |
R2R introduces a token routing method for efficient LLM inference by selectively offloading token generation to SLMs. The research question is whether SLMs can follow LLM reasoning paths by replacing only divergent tokens. They develop an automatic pipeline to label divergent tokens and train a lightweight neural router to predict them based on SLM outputs. R2R achieves a 2.8x wall-clock speedup over the R1-32B LLM with comparable performance and surpasses the average accuracy of the R1-7B model by 1.6x using only an average activated parameter size of 5.6B. AI practitioners can leverage R2R to improve the test-time scaling efficiency of LLMs by reducing inference overhead while preserving reasoning quality. |
| Skywork Open Reasoner 1 Technical Report (Read more on arXiv or HuggingFace) |
Chaojie Wang, Rui Yan, Jujie He, chrisliu298, skydownacai |
Skywork Open Reasoner 1 introduces an RL-enhanced long Chain-of-Thought model, Skywork-OR1. The research investigates how to improve reasoning abilities of large language models using reinforcement learning, focusing on efficiency and scalability for long CoT models. The methodology involves building upon the DeepSeek-R1-Distill model series with a tailored RL approach, incorporating multi-stage training, adaptive entropy control, and detailed ablation studies. The 32B Skywork-OR1 model achieves an average accuracy increase of 15.0% across AIME24, AIME25, and LiveCodeBench, reaching 82.2% on AIME24; a 7B model achieved an average accuracy increase of 13.9%. The findings indicate that careful RL implementation, specifically mitigating premature entropy collapse and balancing exploration/exploitation, significantly improves reasoning performance, providing AI practitioners with an effective recipe for enhancing CoT models. |
| Sherlock: Self-Correcting Reasoning in Vision-Language Models (Read more on arXiv or HuggingFace) |
Ruqi Zhang, Tuwhy |
i) The paper introduces Sherlock, a training framework to enhance self-correction and reasoning in Vision-Language Models (VLMs). ii) The research aims to address the limitations of reasoning VLMs, specifically their sensitivity to errors, data dependency, and generalization issues, by leveraging self-correction strategies. iii) Sherlock incorporates a trajectory-level self-correction objective, preference data construction based on visual perturbation, and a dynamic ẞ for preference tuning within a three-stage training process involving SFT, offline, and online preference learning. iv) Evaluated on eight benchmarks, Sherlock achieves an average accuracy of 64.1 with direct generation and 65.4 after self-correction, outperforming LLaVA-CoT (63.2), Mulberry (63.9), and LlamaV-01 (63.4) while using only 20k annotated data samples. v) The introduction of Sherlock, enabling self-improvement and self-correction within VLMs using limited annotated data, offers AI practitioners an efficient way to enhance the robustness and accuracy of reasoning VLMs in complex multimodal tasks. |
| Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO (Read more on arXiv or HuggingFace) |
Chen Wang, Yuting Li, weiranhuang, weiranhuang, WaltonFuture |
i) The paper introduces MM-UPT, a novel framework for unsupervised post-training of Multi-Modal Large Language Models (MLLMs). ii) The main objective is to enable continual self-improvement of MLLMs without external supervision by using unlabeled multi-modal data. iii) The methodology involves leveraging Group-Regularized Policy Optimization (GRPO) with a self-rewarding mechanism based on majority voting over multiple sampled responses. iv) Experiments show that MM-UPT improves the reasoning ability of Qwen2.5-VL-7B, with an increase from 66.3% to 72.9% on MathVista using standard datasets without ground truth labels. v) MM-UPT offers AI practitioners a new paradigm for continual, autonomous enhancement of MLLMs in the absence of external supervision. |
| SWE-rebench: An Automated Pipeline for Task Collection and |
|
|
| Decontaminated Evaluation of Software Engineering Agents (Read more on arXiv or HuggingFace) |
Anton Shevtsov, Maksim Nekrashevich, sbkarasik, djalexj, ibragim-bad |
i) SWE-rebench introduces an automated pipeline for collecting and evaluating software engineering tasks for LLM agents, addressing data scarcity and contamination. ii) The research aims to provide a scalable, automated method for generating high-quality, interactive SWE tasks and a contamination-free benchmark for evaluating agentic software engineering. iii) The methodology involves automated extraction of interactive SWE tasks from GitHub repositories, environment configuration, installation verification, and automated task quality assessment using LLMs fine-tuned on SWE-bench data. iv) SWE-rebench provides a public dataset of over 21,000 interactive Python-based SWE tasks, and the pipeline achieves a working installation recipe for at least one task in 31% of the repositories. v) AI practitioners can use SWE-rebench as a resource for reinforcement learning of SWE agents at scale and a benchmark to compare LLMs, potentially revealing inflated performance due to contamination issues on SWE-bench Verified. |
| SageAttention2++: A More Efficient Implementation of SageAttention2 (Read more on arXiv or HuggingFace) |
Pengle Zhang, Haofeng Huang, Jia Wei, Xiaoming Xu, jt-zhang |
SageAttention2++ introduces an optimized attention mechanism using FP8 quantization with FP16 accumulation. The research aims to enhance the efficiency of SageAttention2 by leveraging faster FP8 Matmul instructions. It employs narrowing of the FP8 quantization range to ensure values remain within the representable range of FP16. Experimental results show SageAttention2++ achieves up to a 3.9× speedup over FlashAttention2 while maintaining similar attention accuracy. This improvement offers AI practitioners a plug-and-play acceleration method for attention mechanisms in diverse models with minimal end-to-end metric loss. |
| Advancing Multimodal Reasoning via Reinforcement Learning with Cold |
|
|
| Start (Read more on arXiv or HuggingFace) |
Kaipeng Zheng, Yuting Li, weiranhuang, weiranhuang, WaltonFuture |
i) The paper introduces a two-stage approach (SFT+RL) for enhancing multimodal reasoning in large language models (MLLMs). ii) The main research question is how different cold start strategies during supervised fine-tuning (SFT) impact downstream reinforcement learning (RL) performance in the multimodal domain. iii) The methodology involves supervised fine-tuning (SFT) using structured chain-of-thought reasoning patterns as a cold start, followed by reinforcement learning via GRPO. iv) The resulting 7B model achieves a 73.4% score on MathVista, a +7.10 points increase, and state-of-the-art performance among open-source MLLMs at both 3B and 7B scales. v) AI practitioners should consider using an SFT-based cold start approach to provide a robust foundation for RL scaling, which leads to improved performance in multimodal reasoning tasks, as it demonstrates potential for narrowing performance gaps between smaller and larger multimodal language models. |
| Fostering Video Reasoning via Next-Event Prediction (Read more on arXiv or HuggingFace) |
Kenji Kawaguchi, Chao Du, Xiangyan Liu, Hongfu Liu, Haonan Wang |
i) The paper introduces next-event prediction (NEP) as a self-supervised learning task for enhancing temporal reasoning in multimodal large language models (MLLMs) using past video frames to predict summaries of future events. ii) The research question is to determine an effective learning task that equips MLLMs with temporal reasoning capabilities over video inputs, addressing limitations in existing methods like video question answering and captioning. iii) The methodology involves creating a dataset, V1-33K, consisting of 33,000 video segments with paired past and future frames and employing various video instruction-tuning strategies, including supervised fine-tuning (SFT), critique fine-tuning (CFT), distillation, and mixed-tuning. iv) Experiments show that incorporating NEP enhances MLLMs’ temporal understanding and reasoning, with a performance improvement on temporal benchmarks reflected in FutureBench and maintains general video understanding. A 60.0 average score on V1-33K dataset v) NEP provides a scalable and effective training paradigm for AI practitioners to improve temporal reasoning in MLLMs for various applications, enhancing their ability to infer future events without sacrificing general video understanding. |
| RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with |
|
|
| Global Illumination (Read more on arXiv or HuggingFace) |
Xin Tong, Hongzhi Wu, Pieter Peers, doyleconan, NCJ |
RenderFormer is introduced as a transformer-based neural rendering pipeline for triangle meshes, achieving global illumination effects without per-scene training. The paper explores if a rendering pipeline can be learned end-to-end, rather than overfitting a model to a fixed scene. The research formulates rendering as a sequence-to-sequence transformation, converting triangle tokens to pixel tokens using a two-stage transformer architecture. The model is trained on synthetic scenes with a single reflectance model and a limited number of light sources and triangles. Results demonstrate visually similar renderings to Blender Cycles, and the model can handle scenes with at most 4,096 triangles. RenderFormer presents a neural rendering approach to solving global illumination that can be directly incorporated into existing triangle mesh workflows. |
| DeepResearchGym: A Free, Transparent, and Reproducible Evaluation |
|
|
| Sandbox for Deep Research (Read more on arXiv or HuggingFace) |
Abhijay Paladugu, Kangrui Mao, Jingyuan He, Jingjie Ning, jmvcoelho |
i) DeepResearchGym is introduced as an open-source framework for reproducible evaluation of deep research systems, addressing the limitations of commercial APIs. ii) The research aims to provide a transparent and reproducible environment for benchmarking deep research systems. iii) The methodology combines a reproducible search API using ClueWeb22 and FineWeb, indexed with DiskANN, and an evaluation protocol extending the Researchy Questions dataset using LLM-as-a-judge metrics. iv) The system achieves lower latency than commercial APIs while ensuring stable document rankings; systems integrated with DeepResearchGym achieve performance comparable to commercial APIs. v) AI practitioners can use DeepResearchGym to evaluate deep research systems with a reproducible search API and automatic metrics, helping to standardize the comparison between research systems, although limitations concerning the use of proprietary LLMs restrict full output reproducibility. |
| Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and |
|
|
| Preference Alignment (Read more on arXiv or HuggingFace) |
Jong Chul Ye, Jeongsol Kim, Bryan Sangwoo Kim |
i) The paper introduces Chain-of-Zoom (CoZ), a model-agnostic framework for extreme single-image super-resolution (SISR) based on scale autoregression and preference alignment. ii) The main objective is to extend the scalability of existing SR models beyond their training configurations, enabling high-quality image magnification at extreme resolutions without retraining. iii) The methodology involves factorizing SISR into an autoregressive chain of intermediate scale-states with multi-scale-aware text prompts generated by a fine-tuned vision-language model (VLM), using Generalized Reward Policy Optimization (GRPO) for preference alignment. iv) Experiments show CoZ attains beyond 256× enlargement with a standard 4× diffusion SR model, improving perceptual quality and fidelity, as measured by non-reference metrics such as a NIQE score of 8.2335 on DIV2K dataset compared to 16.5915 for direct SR at 64x magnification. v) CoZ provides AI practitioners with a resource-efficient method for achieving extreme image super-resolution by leveraging existing SR models and VLMs, circumventing the need to train new models for each magnification factor. |
| Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for |
|
|
| Frozen LLMs (Read more on arXiv or HuggingFace) |
Jong Chul Ye, Choonghan Kim, Hyunmin Hwang, Hangeol Chang, kjm981995 |
i) The paper introduces Universal Reasoner (UniR), a plug-and-play reasoning module for frozen LLMs. ii) The research aims to create a lightweight, composable reasoning module transferable across different LLM architectures. iii) The methodology involves decoupling reward model training from full policy updates, training a reasoning module using predefined rewards optimized with a policy gradient algorithm, and additively combining its logits with a frozen LLM backbone at inference. iv) Experiments on math reasoning tasks using Llama3.2 showed UniR achieved an average pass@1 score of 36.0, outperforming GRPO LORA. v) UniR offers AI practitioners a cost-efficient method for enhancing reasoning in LLMs without architectural dependencies, enabling modular composition of task-specific expertise. |
| SVRPBench: A Realistic Benchmark for Stochastic Vehicle Routing Problem (Read more on arXiv or HuggingFace) |
Zangir Iklassov, Salem Lahlou, Martin Takac, Yahia Salaheldin Shaaban, ahmedheakl |
SVRPBench introduces a new open benchmark for evaluating stochastic vehicle routing problem (SVRP) solvers under realistic urban conditions. The research aims to address the limitations of existing benchmarks by incorporating high-fidelity stochastic dynamics such as time-dependent congestion, log-normal delays, and probabilistic accidents. The methodology involves simulating diverse, constraint-rich scenarios with up to 1000 customers, multi-depot, and multi-vehicle setups, and benchmarking existing solvers. Results show that state-of-the-art RL solvers like POMO and AM degrade by over 20% under distributional shift compared to classical and metaheuristic methods. This highlights the need for AI practitioners to design robust routing algorithms that generalize beyond synthetic assumptions and adapt to real-world uncertainty, specifically to consider distributional shifts that may severely affect RL performance. |
| What Makes for Text to 360-degree Panorama Generation with Stable |
|
|
| Diffusion? (Read more on arXiv or HuggingFace) |
Jing Zhang, Qiang Zhang, allencbzhang, mcleanie |
i) This paper analyzes the role of LoRA-adapted attention modules within Stable Diffusion for text-to-360-degree panorama generation. ii) The research investigates which trainable components in LoRA fine-tuning are most critical for adapting pre-trained diffusion models to panoramic image generation. iii) The methodology involves isolating and jointly training the query, key, value, and output weight matrices (W{q,k,v,o}) of attention modules using LoRA, followed by ablation studies. iv) The analysis reveals that the value and output weight matrices (W{v,o}) are more responsible for adapting to the panoramic domain, achieving a Fréchet Auto-Encoder Distance (FAED) of 5.90 with a specific configuration of mixture of experts on a 512x1024 resolution. v) This suggests that AI practitioners can optimize fine-tuning by focusing capacity enhancement on value and output weight matrices, while potentially freezing or down weighting query and key matrices when adapting Stable Diffusion for panoramic image generation, leading to memory-efficient training. |
| WebDancer: Towards Autonomous Information Seeking Agency (Read more on arXiv or HuggingFace) |
Liwen Zhang, Wenbiao Yin, Runnan Fang, Baixuan Li, callanwu |
i) WebDancer presents a framework for building autonomous information-seeking agents. ii) The research aims to develop a systematic approach for creating web agents capable of multi-step reasoning and information retrieval. iii) The methodology involves browsing data construction, trajectories sampling, supervised fine-tuning for cold start, and reinforcement learning for generalization, instantiated in a ReAct-based web agent. iv) Empirical evaluations on GAIA and WebWalkerQA demonstrate WebDancer’s strong performance, achieving considerable results and highlighting the efficacy of the training paradigm. One such evaluation noted a Pass@3 score of 61.1% on GAIA and 54.6% on WebWalkerQA, for the best-performing model. v) The implication for AI practitioners is a systematic, end-to-end pipeline to construct long-term information-seeking web agents, offering a structured pathway to develop capable agentic models, especially in complex, real-world applications. |
| Judging Quality Across Languages: A Multilingual Approach to Pretraining |
|
|
| Data Filtering with Language Models (Read more on arXiv or HuggingFace) |
Abbas Goher Khan, Elias Wendt, Max Lübbering, Mehdi Ali, mbrack |
i) The paper introduces JQL, a multilingual pretraining data filtering approach leveraging language models for quality assessment across 35 languages. ii) The research aims to curate high-quality, diverse multilingual data efficiently while minimizing computational costs. iii) JQL uses lightweight annotators distilled from pretrained multilingual embeddings to assess data quality, evaluated empirically across 35 languages. iv) JQL substantially outperforms heuristic filtering methods like Fineweb2, increasing data retention rates while enhancing downstream model training quality; for example, using the 0.6 percentile threshold in Spanish retains over 9% more tokens than FW2 and exhibits improved quality. v) The approach provides practical insights and resources for multilingual data curation, facilitating the development of high-quality multilingual datasets for AI practitioners. |
| LIMOPro: Reasoning Refinement for Efficient and Effective Test-time |
|
|
| Scaling (Read more on arXiv or HuggingFace) |
Kaishuai Xu, Chunpu Xu, Ruifeng Yuan, Jiashuo Wang, YangXiao-nlp |
LIMOPro introduces a reasoning refinement framework to improve efficiency in large language models (LLMs). The paper addresses the question of how to optimize reasoning chains distilled from powerful language models to reduce computational demands without sacrificing accuracy. Perplexity-based Importance Refinement (PIR) is used to quantitatively evaluate the importance of each reasoning step, selectively pruning low-importance functional steps while preserving progressive reasoning components. Fine-tuning on PIR-optimized data achieves improved accuracy (up to +6.6%) with reduced token usage (up to -41%) on benchmarks like AIME, AMC, and GPQA Diamond. This provides AI practitioners with a method to deploy reasoning-capable LLMs more efficiently by optimizing the training data and reducing inference costs and response times. |
| Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal |
|
|
| Evolution of Human States (Read more on arXiv or HuggingFace) |
Chunpu Xu, Changhe Song, Qiancheng Xu, Jiashuo Wang, YangXiao-nlp |
i) The paper introduces DYNTOM, a novel benchmark to evaluate LLMs’ ability to understand and track the temporal evolution of human mental states across interconnected social scenarios. ii) The research investigates how well LLMs adapt to dynamic changes in mental states, moving beyond static snapshots typically assessed in existing benchmarks. iii) DYNTOM employs a four-step framework for generating social contexts, designing mental state trajectories, creating natural dialogues, and formulating targeted questions. iv) Empirical evaluation of ten LLMs reveals that their performance lags behind human performance by 44.7%, particularly in transformation-type questions assessing mental state changes. v) The findings imply that AI practitioners should focus on improving LLMs’ capacity for temporal reasoning and understanding the dynamic nature of human mental states to better simulate and interact within real-world social contexts. |
| VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich |
|
|
| Information Understanding via Iterative Reasoning with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zehui Chen, Ruixue Ding, Lin-Chen, YuZeng260, autumncc |
i) The paper introduces VRAG-RL, a reinforcement learning framework for visually rich information retrieval-augmented generation. ii) The primary research objective is to enhance VLMs’ reasoning and retrieval capabilities when interacting with visually rich information sources. iii) The methodology involves designing a visual perception action space for fine-grained information extraction and a retrieval-based reward function to optimize VLM interactions with search engines. iv) Experiments on SlideVQA, ViDoSeek and MMLongBench show that VRAG-RL outperforms existing methods by 20% on the Qwen2.5-VL-7B model and 30% on the Qwen2.5-VL-3B model. v) VRAG-RL provides AI practitioners with an improved framework for training VLMs to reason effectively with visually rich information, enhancing their applicability in domains requiring complex visual data interpretation and retrieval. |
| Let’s Predict Sentence by Sentence (Read more on arXiv or HuggingFace) |
Hoyeon Chang, Jiyeon Kim, Seungone Kim, Byeongguk Jeon, Hyeonbin Hwang |
i) The paper introduces a sentence-level autoregressive language model operating within a latent embedding space for structured reasoning. ii) It investigates whether pre-trained language models can effectively perform structured reasoning over sentences rather than tokens by predicting continuous embeddings of next sentences. iii) The study employs two embedding paradigms: semantic embeddings (autoencoding) and contextual embeddings (next-sentence prediction), evaluated under discrete and continuous inference regimes. iv) Continuous inference using contextual embeddings achieves competitive performance with Chain-of-Thought reasoning while reducing inference-time FLOPs by approximately half on average across four reasoning domains. v) AI practitioners can leverage this sentence-level approach to achieve more efficient reasoning in language models, potentially reducing computational costs without sacrificing performance on certain tasks. |
| RICO: Improving Accuracy and Completeness in Image Recaptioning via |
|
|
| Visual Reconstruction (Read more on arXiv or HuggingFace) |
Linli Yao, Sihan Yang, Shuhuai Ren, Yishuo Cai, Yuchi Wang |
i) The paper introduces RICO, a framework for refining image captions by leveraging visual reconstruction to address inaccuracies and incompleteness. ii) The primary objective is to enhance the accuracy and completeness of image captions generated by MLLMs. iii) RICO iteratively refines captions by reconstructing the caption into an image using a text-to-image model, then prompting an MLLM to identify and correct discrepancies between the original and reconstructed images. iv) Experiments show RICO improves caption quality by approximately 10% on CapsBench and CompreCap. v) RICO offers AI practitioners a method to generate higher-quality image caption datasets for improving multimodal model training. |
| Thinking with Generated Images (Read more on arXiv or HuggingFace) |
Jiadi Su, Siqi Kou, Steffi Chern, Zhulin Hu, ethanchern |
i) The paper introduces Thinking with Generated Images, a paradigm enabling LMMs to generate intermediate visual reasoning steps. ii) The research explores how LMMs can natively reason across text and vision modalities by generating visual subgoals and self-critiques. iii) The methodology involves a native long-multimodal thought process within unified autoregressive LMMs, with supervised fine-tuning on synthetic multimodal data. iv) Experiments on vision generation benchmarks show up to 50% relative improvement in handling complex multi-object scenarios (38% to 57%) compared to baseline approaches. v) This work provides AI practitioners with a method for enhancing LMMs visual reasoning by enabling them to dynamically generate, critique, and refine internal visual representations. |
| PrismLayers: Open Data for High-Quality Multi-Layer Transparent Image |
|
|
| Generative Models (Read more on arXiv or HuggingFace) |
Ji Li, Keming Wu, Yanbin Wang, Heyang Jiang, Junwen Chen |
i) The paper introduces PrismLayers and PrismLayersPro, datasets for multi-layer transparent image generation. ii) The main objective is to address the lack of high-quality data for training generative models capable of producing multi-layer transparent images with accurate alpha mattes. iii) The methodology includes a training-free synthesis pipeline leveraging pre-trained diffusion models (FLUX) to generate individual layers, followed by composition guided by semantic layouts. iv) The released dataset PRISMLAYERSPRO contains 20K high-fidelity images, enabling the fine-tuning of ART, resulting in ART+ which surpasses ART in 60% user preference in terms of layer quality and prompt following. v) AI/ML practitioners can utilize PRISMLAYERSPRO and the ART+ model to develop and refine multi-layer image generation systems, opening possibilities for layer-wise image editing and creation workflows. |
| Text2Grad: Reinforcement Learning from Natural Language Feedback (Read more on arXiv or HuggingFace) |
Si Qin, Tianjun Mao, Chaoyun Zhang, Lu Wang, Hanyang Wang |
i) The paper introduces TEXT2GRAD, a reinforcement learning paradigm converting free-form textual feedback into span-level gradients for language model optimization. ii) The research aims to improve reinforcement learning from human feedback (RLHF) by utilizing the rich information in natural language critiques instead of scalar rewards. iii) TEXT2GRAD uses a three-component architecture: a feedback-annotation pipeline, a fine-grained reward model predicting span-level reward, and a span-level policy optimizer. iv) Evaluated across summarization, code generation, and question answering, TEXT2GRAD surpasses scalar-reward RL and prompt-only baselines, achieving a +25.3% BLEU improvement over PPO on the SLF5K summarization dataset. v) TEXT2GRAD provides a method for AI/ML engineers to perform fine-grained policy optimization by leveraging natural language feedback to adjust model parameters directly, leading to improved sample efficiency and model interpretability. |
| Pitfalls of Rule- and Model-based Verifiers – A Case Study on |
|
|
| Mathematical Reasoning (Read more on arXiv or HuggingFace) |
Junxian He, Qi Zhu, Xingshan Zeng, Weihao Zeng, yuzhen17 |
i) This paper analyzes the vulnerabilities of rule-based and model-based verifiers used in reinforcement learning with verifiable reward (RLVR) for mathematical reasoning. ii) The study investigates the accuracy and robustness of rule-based and model-based verifiers and assesses their impact on the RL training performance of large language models. iii) The research employs static evaluation of verifiers across multiple mathematical datasets and conducts RL training experiments to observe the effect of different verifiers on policy model optimization. iv) The results show that rule-based verifiers exhibit an average recall rate of 86% due to format sensitivity, while model-based verifiers achieve higher static accuracy but are susceptible to reward hacking, with a trained verifier leading to a significant divergence between training and oracle reward after 450 iterations. v) These findings imply that AI practitioners should exercise caution when selecting verifiers in RLVR, as high classification accuracy does not guarantee resistance to reward hacking, and should prioritize robustness to adversarial patterns. |
| EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video |
|
|
| Guidance (Read more on arXiv or HuggingFace) |
Han Lin, Jialu Li, Jaemin Cho, Zun Wang, jaehong31 |
EPiC introduces an efficient camera control learning framework for video diffusion models leveraging precise anchor-video guidance. The research aims to improve controllable 3D camera trajectories in video diffusion models by using higher quality anchor videos. The methodology involves creating anchor videos by masking source videos based on first-frame visibility and a lightweight Anchor-ControlNet to integrate anchor video guidance. EPiC achieves state-of-the-art performance on RealEstate10K and MiraData for I2V camera control, obtaining superior camera accuracy with rotation error decreasing to 0.40 ± 0.11 on RealEstate10K. EPiC offers AI practitioners an efficient training approach requiring fewer parameters and less data for precise and robust camera control in video generation tasks. It remains unclear the degree to which specific algorithmic innovation contributes compared to architectural choices. |
| GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language |
|
|
| Models and Enhanced Reasoning Chains (Read more on arXiv or HuggingFace) |
Yiren Song, Haofan Wang, Zihao Pan, Xiaoran Pan, Chun Wang |
i) The paper introduces the GRE Suite, a framework designed to improve geo-localization inference by augmenting Vision-Language Models (VLMs) with structured reasoning chains. ii) The main objective is to enhance the accuracy and interpretability of geo-localization by addressing the limitations of current methods in complex geographic inference. iii) The methodology involves constructing a high-quality geo-localization reasoning dataset, GRE30K, and developing a GRE model using multi-stage reasoning and reinforcement learning training. iv) Experimental results demonstrate that GRE significantly outperforms existing methods, achieving a 11.3% accuracy within 1km on the Im2GPS3k dataset; the Geo Reason Evaluation Benchmark (GREval-Bench) further assesses VLMs. v) For AI practitioners, the GRE Suite offers a novel reasoning-augmented VLM framework and dataset for improving geo-localization tasks, facilitating applications requiring precise geographic inference from images. |
| Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat |
|
|
| Falsehoods (Read more on arXiv or HuggingFace) |
Deval Pandya, Marcelo Lotif, Rizwan Qureshi, amanchadha, Shainarazavi |
i) The paper introduces “model immunization,” a training framework using curated falsehoods as a supervised “vaccine” to enhance AI model truthfulness. ii) The research aims to improve model resistance to misinformation by fine-tuning on explicitly labeled false data. iii) The methodology involves periodically injecting small, quarantined sets of labeled falsehoods during fine-tuning. iv) An illustrative case study showed an increase in model truthfulness from approximately 60% to 78% after immunization with a 5% micro-dose of falsehoods during fine-tuning. v) The principal implication for AI practitioners is a proactive approach to align AI systems with factuality, reducing the generation of misinformation without significantly degrading general performance. |
| Meta-Learning an In-Context Transformer Model of Human Higher Visual |
|
|
| Cortex (Read more on arXiv or HuggingFace) |
Jacob S. Prince, Hossein Adeli, Mu Nan, Muquan Yu, aluo-x |
i) This paper introduces BraInCoRL, a meta-learning framework for predicting voxelwise neural responses in human higher visual cortex using in-context learning. ii) The research aims to develop a generalizable model of visual cortex that adapts to subject-specific neural organization from few-shot examples. iii) A transformer architecture is leveraged to learn an inductive bias over multiple subjects by jointly conditioning on image features and voxel activations, optimizing for in-context learning. iv) Results demonstrate that BraInCoRL outperforms existing voxelwise encoder designs in a low-data regime, generalizing to new visual fMRI datasets and exhibiting test-time scaling behavior; for instance, BraInCoRL with 100 in-context images achieves significantly higher explained variance compared to a ridge regression baseline with the same data. v) BraInCoRL provides AI practitioners with a more data-efficient and generalizable approach to modeling human visual cortex, potentially enabling better interpretability of neural signals and query-driven functional mapping in AI systems interacting with human perception. |
| One-Way Ticket:Time-Independent Unified Encoder for Distilling |
|
|
| Text-to-Image Diffusion Models (Read more on arXiv or HuggingFace) |
Jiehang Xie, Tao Liu, Kai Wang, Lei Wang, senmaonk |
i) The paper introduces a Time-independent Unified Encoder (TiUE) for efficient text-to-image diffusion model distillation. ii) The objective is to reduce inference time and improve image quality/diversity by unifying the encoder across different decoder time steps in diffusion models. iii) A one-pass scheme is proposed where encoder features are shared across multiple decoder time steps, combined with a KL divergence regularization term to improve noise prediction. iv) Results show that TiUE achieves a FID of 23.11 on COCO2017-5K and outperforms state-of-the-art methods such as LCM and SD-Turbo while maintaining computational efficiency. v) TiUE offers AI practitioners a more computationally efficient approach to deploy text-to-image diffusion models with improved image quality and diversity compared to existing distillation techniques. |
| Unveiling Instruction-Specific Neurons & Experts: An Analytical |
|
|
| Framework for LLM’s Instruction-Following Capabilities (Read more on arXiv or HuggingFace) |
Zhaorui Hou, Jungang Li, Yibo Yan, Yubo Gao, Junyan Zhang |
i) This paper introduces HEXAINST, a balanced instructional dataset, and SPARCOM, a framework for analyzing sparse components in LLMs to understand instruction following. ii) The research aims to systematically examine how fine-tuning reconfigures LLM computations by isolating and analyzing instruction-specific sparse components. iii) The methodology involves identifying instruction-specific neurons (ISNs) and experts (ISEs), evaluating their generality and uniqueness, and comparing their alterations during fine-tuning. iv) Results show that after fine-tuning, LLMs exhibit an increase in the number of more capable and specialized ISNs, with activation patterns of specific neurons changing significantly (Jaccard similarity coefficient in ISNs is displayed in Table 1). v) The principal implication for AI practitioners is a deeper understanding of how fine-tuning alters internal mechanisms and instruction-following behavior in LLMs. |
| MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware |
|
|
| Multi-Segment Grounding (Read more on arXiv or HuggingFace) |
Chenliang Li, Ziyue Wang, Chi Chen, Shengfeng Lou, Fuwen Luo |
MUSEG introduces a reinforcement learning method to enhance video temporal understanding in multimodal large language models (MLLMs). The research aims to improve fine-grained temporal reasoning by aligning queries with multiple video segments using timestamp-aware multi-segment grounding. The methodology involves a customized RL training recipe with phased rewards, including segment matching and timestamp rewards. Experiments show MUSEG-7B achieves improved performance on temporal grounding benchmarks, exhibiting a ~60% average score on Charades-STA compared to base models’ ~50%. This suggests AI practitioners can leverage MUSEG to develop MLLMs with enhanced temporal reasoning capabilities for time-sensitive video understanding tasks. |
| Benchmarking Recommendation, Classification, and Tracing Based on |
|
|
| Hugging Face Knowledge Graph (Read more on arXiv or HuggingFace) |
Yuanning Cui, Weiqing Luo, Xiao Zhou, Kaijia Huang, cqsss |
i) This paper introduces HuggingKG, a large-scale knowledge graph, and HuggingBench, a benchmark for IR tasks in the open-source machine learning resource domain. ii) The research aims to create structured representations for ML resources to enhance resource management tasks, such as recommendation, classification, and model tracing. iii) The methodology involves constructing a knowledge graph from Hugging Face metadata and creating three novel test collections for benchmarking IR tasks. iv) HuggingKG comprises 2.6 million nodes and 6.2 million edges; experiments show KGCL with Homo subgraph achieves +4.80% higher in Recall@5 compared to social recommendation; TransE performs best on model tracing with unique relation distribution. v) AI practitioners can leverage HuggingKG and HuggingBench for enhanced resource discovery and management, particularly in tasks requiring structured knowledge of ML models, datasets, and user interactions within the Hugging Face ecosystem. |
| Safe-Sora: Safe Text-to-Video Generation via Graphical Watermarking (Read more on arXiv or HuggingFace) |
Junhao Zhuang, Tangyu Jiang, Hongbin Xu, Xuerui Qiu, Sugewud |
Safe-Sora is a novel framework for embedding graphical watermarks into text-to-video generation to enhance copyright protection. The research addresses the under-explored area of graphical watermarking in video generation via diffusion models. It introduces a hierarchical coarse-to-fine adaptive matching mechanism assigning watermark patches to visually similar video regions and utilizes a 3D wavelet transform-enhanced Mamba architecture. Experiments show Safe-Sora achieves a Fréchet Video Distance of 3.77, demonstrating state-of-the-art video quality and watermark fidelity compared to existing methods. This offers AI practitioners a method for robustly embedding and extracting graphical watermarks, improving the reliability of copyright verification for AI-generated video content. |
| Characterizing Bias: Benchmarking Large Language Models in Simplified |
|
|
| versus Traditional Chinese (Read more on arXiv or HuggingFace) |
Allison Koenecke, Jian Kang, Jiebo Luo, Hanjia Lyu |
i) The paper benchmarks Large Language Model (LLM) performance disparities when prompted in Simplified versus Traditional Chinese. ii) The research aims to investigate whether LLMs exhibit differential performance when prompted in Simplified Chinese compared to Traditional Chinese, specifically focusing on representational harms and downstream decision-making biases. iii) The study designed two benchmark tasks: regional term choice and regional name choice, auditing the performance of 11 commercial LLM services and open-source models, including those trained primarily on English, Simplified Chinese, or Traditional Chinese. iv) The analysis indicates biases in LLM responses depend on the task and prompting language; while LLMs favored Simplified Chinese in regional term choice, they favored Traditional Chinese names in regional name choice tasks. v) The finding that LLM biases are dependent on both task and prompting language indicates a need for ongoing auditing frameworks to evaluate LLM behavior across Chinese language variants. |
| AITEE – Agentic Tutor for Electrical Engineering (Read more on arXiv or HuggingFace) |
Christian Bernhardt, Alexander Bernhardt, CKnievel |
i) The paper introduces AITEE, an agentic tutoring system for electrical engineering education leveraging LLMs, graph neural networks, and circuit simulation. ii) The primary research objective is to develop an agentic tutor that can provide individualized support and promote self-directed learning for electrical engineering students. iii) The methodology involves adapting circuit reconstruction processes, using graph-based similarity measures for context retrieval, Retrieval Augmented Generation (RAG), and implementing a Socratic dialogue to foster learner autonomy. iv) Experiments showed that AITEE significantly outperforms baseline approaches in domain-specific knowledge application, with medium-sized LLM models demonstrating acceptable performance; with Multi-Representation Indexing (MRI), all models except Llama 3.1 8B exhibit a performance level suggesting potential to ensure tutor-level expertise v) The results highlight the potential of agentic tutors to deliver scalable, personalized, and effective learning environments for electrical engineering education, suggesting AI practitioners can create more effective educational tools by combining LLMs with domain-specific knowledge and interactive dialogue strategies. |
| MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal |
|
|
| Manga Understanding (Read more on arXiv or HuggingFace) |
Yuki Imajuku, Atsuyuki Miyai, Shota Onohara, Kazuki Egashira, Jeonghun Baek |
i) The paper introduces MangaVQA and MangaLMM for multimodal manga understanding, addressing OCR and visual question answering. ii) The primary research objective is to establish benchmarks and a specialized model for evaluating and advancing large multimodal models (LMMs) in the domain of manga understanding. iii) The methodology involves creating the MangaVQA benchmark with 526 manually constructed question-answer pairs and finetuning the Qwen2.5-VL model to create MangaLMM for joint MangaOCR and MangaVQA task handling. iv) MangaLMM achieves over 70% on the MangaOCR task and outperforms GPT-4o on MangaVQA (6.57 vs 5.76 on a scale of 1-10), while GPT-4o exhibited near-zero OCR performance. v) MangaLMM provides AI practitioners a specialized model and benchmarks for evaluating and improving LMMs’ abilities in understanding multimodal content, specifically in the stylized and context-rich domain of manga. |
| Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and |
|
|
| Styles (Read more on arXiv or HuggingFace) |
Peidong Liu, Xiang Liu, Peng Wang |
Styl3R is a feed-forward network for instant 3D stylization from sparse, unposed images and a style image. The research question is how to achieve fast, multi-view consistent 3D stylization without test-time optimization. The methodology involves a dual-branch network separating structure and appearance modeling, with an identity loss adaptation for pre-training via novel view synthesis. The primary result is high-quality stylized 3D content in 0.15 seconds, achieving superior style blend and multi-view consistency. AI practitioners can leverage this efficient method for interactive applications requiring fast 3D stylization without dense inputs or per-scene optimization. |
| Efficient Data Selection at Scale via Influence Distillation (Read more on arXiv or HuggingFace) |
Vahab Mirrokni, Dan Alistarh, Vincent Cohen-Addad, Mahdi Nikdan |
i) The paper introduces Influence Distillation, a data selection framework leveraging second-order information to optimize training sample weighting for Large Language Models (LLMs). ii) The research aims to develop a scalable and mathematically-justified data selection method that directly optimizes for performance on a target distribution. iii) Influence Distillation uses a landmark-based approximation to efficiently compute and propagate influence scores, assigning model-specific weights to training samples for LLM fine-tuning. iv) Experiments on instruction tuning of the Tulu V2 dataset using Llama and Qwen models demonstrate that Influence Distillation matches or outperforms state-of-the-art performance while achieving up to 3.5x faster selection runtime. v) Influence Distillation provides AI/ML practitioners with an efficient method for curating training datasets, improving downstream task accuracy while reducing computational costs associated with LLM fine-tuning. |
| First Finish Search: Efficient Test-Time Scaling in Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Tanmoy Chakraborty, Ayan Sengupta, aradhye |
First Finish Search (FFS) is introduced as a training-free parallel decoding strategy to improve reasoning in large language models. The research aims to enhance test-time scaling (TTS) efficiency by dynamically allocating compute during inference. FFS launches n independent samples and selects the output trace that completes first, leveraging the observed correlation between shorter trace length and correctness in reasoning tasks. Experiments with DeepSeek-R1 on the AIME datasets show FFS achieves 82.23% accuracy, a 15% improvement over its standalone accuracy. This indicates that simple TTS strategies, such as FFS, can yield remarkable performance improvements with minimal overhead at inference time by dynamically scaling the number of decoding samples based on the task to optimize for lower token usage and reduced latency. Some parts of the paper and its methodology were unclear. |
Papers for 2025-05-28
| Title |
Authors |
Summary |
| OmniConsistency: Learning Style-Agnostic Consistency from Paired |
|
|
| Stylization Data (Read more on arXiv or HuggingFace) |
Cheng Liu, mikeshou, yiren98 |
i) OmniConsistency is presented as a universal consistency plugin for image stylization, trained on paired data. ii) The research aims to achieve style-agnostic consistency in image stylization tasks using diffusion models, while preserving structure and semantics. iii) The methodology involves a two-stage decoupled training strategy and a rolling LoRA Bank loader mechanism with a lightweight Consistency LoRA Module and Conditional Token Mapping. iv) The method achieves state-of-the-art performance comparable to GPT-4o, enhancing visual coherence and aesthetic quality in stylization. It also achieves a 4.6% increase in GPU memory usage and a 5.3% increase in inference time at 1024x1024 resolution with 24 sampling steps compared to the base Flux Text-to-Image pipeline. v) OmniConsistency offers AI practitioners a modular, plug-and-play component that can be seamlessly integrated with arbitrary style LoRAs without retraining for image-to-image stylization tasks. |
| MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs (Read more on arXiv or HuggingFace) |
BoZhang, KaituoFeng, Yilei-Jiang, Potentialts, JiakangYuan |
i) MME-Reasoning is introduced as a new benchmark for evaluating logical reasoning in multimodal large language models (MLLMs). ii) The primary objective is to comprehensively assess the inductive, deductive, and abductive reasoning capabilities of MLLMs. iii) The methodology involves curating a dataset of 1,188 multimodal questions, categorizing them by reasoning type and difficulty, and evaluating MLLM performance using multiple-choice, free-form, and rule-based question formats. iv) Evaluation of state-of-the-art MLLMs reveals limitations in comprehensive logical reasoning, with Gemini-Pro-2.5-Thinking achieving a score of 60.19%. v) The principal implication is that current MLLMs exhibit performance imbalances across reasoning types, especially in abductive reasoning, highlighting the need for improved reasoning architectures and training methodologies. |
| Paper2Poster: Towards Multimodal Poster Automation from Scientific |
|
|
| Papers (Read more on arXiv or HuggingFace) |
Xi He, philiptorr, HideOnBush, KevinQHLin, weipang142857 |
Paper2Poster introduces a benchmark and metric suite for academic poster generation from scientific papers. The research aims to address the challenge of condensing long-context documents into a coherent visual page. It uses a top-down, visual-in-the-loop multi-agent pipeline called PosterAgent consisting of a Parser, Planner, and Painter-Commenter loop. Evaluations show that PosterAgent, based on open-source models like Qwen-2.5, outperforms GPT-40-driven systems, while also reducing token consumption by 87%, and finalizing a 22 page paper into an editable “.pptx” poster for only $0.005. The primary implication is it provides a framework for AI practitioners to streamline scientific communication through automated poster generation. |
| VerIPO: Cultivating Long Reasoning in Video-LLMs via Verifier-Gudied |
|
|
| Iterative Policy Optimization (Read more on arXiv or HuggingFace) |
Xinyu Chen, whluo, longyuewang, TerenceL-TL, YunxinLi |
i) VerIPO is introduced as a Verifier-guided Iterative Policy Optimization method for video Large Language Models (Video-LLMs). ii) The research aims to improve the long reasoning capacity of Video-LLMs. iii) The methodology involves a GRPO-Verifier-DPO training loop, using a Rollout-Aware Verifier to assess reasoning logic and generate high-quality contrastive data. iv) Experimental results show VerIPO achieves significantly faster and more effective optimization compared to standard GRPO, yielding superior performance, also the model with one iteration outperforms powerful LMMs (e.g., Kimi-VL) and long reasoning models (e.g., Video-R1), and DPO stage is 7x faster than GRPO. v) VerIPO offers AI practitioners a method to enhance the deep reasoning capabilities of Video-LLMs through verifier-guided iterative policy optimization and high-quality data curation. |
| Exploring the Latent Capacity of LLMs for One-Step Text Generation (Read more on arXiv or HuggingFace) |
oseledets, glebzok |
i) This paper explores the possibility of generating accurate multi-token sequences from compressed representations in LLMs without autoregression. ii) The research investigates whether frozen LLMs can reconstruct accurate multi-token sequences in a single forward pass using a small number of learned embeddings, and explores the information encoded in these embeddings. iii) The methodology involves training two “proto-tokens” to optimize cross-entropy loss between the target sequence and the LLM’s output in a single forward pass, varying model size, text source, and token arrangement. iv) The results show that LLMs can reconstruct arbitrary sequences from as few as two learned input embeddings, achieving near-perfect reconstruction (0.99 token-level accuracy) of sequences up to 256 tokens. v) This reveals LLMs’ parallel generation capabilities and indicates potential for fast context compression and decompression, achieving up to 279x greater generation throughput compared to autoregressive methods, thus allowing for accelerated inference especially on-device. |
| SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning |
|
|
| Logical Reasoning and Beyond (Read more on arXiv or HuggingFace) |
zhangmozhi, YN83, ShiqiChen, ShiroFFF, Junteng |
SynLogic introduces a data synthesis framework and dataset for generating verifiable logical reasoning data to enhance large language models. This research aims to address the lack of diverse, verifiable reasoning data for reinforcement learning in LLMs. The study employs a data synthesis pipeline to generate a dataset, SYNLOGIC, comprising 35 diverse logical reasoning tasks with adjustable difficulty and verifiable solutions. Experiments using Qwen2.5-Base models trained with SYNLOGIC demonstrate state-of-the-art logical reasoning performance, exceeding DeepSeek-R1-Distill-Qwen-32B by 6 points on BBEH. Mixing SYNLOGIC data with mathematical and coding tasks improves training efficiency and reasoning generalization, offering AI practitioners a valuable resource for enhancing LLM reasoning capabilities. |
| Don’t Overthink it. Preferring Shorter Thinking Chains for Improved LLM |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Roy Schwartz, adiyoss, gsynnaeve, hassid |
i) This paper challenges the conventional wisdom that longer thinking chains in LLMs lead to better reasoning, finding that shorter chains are often more accurate and efficient. ii) The primary objective is to investigate the relationship between reasoning chain length and correctness in LLMs, and to develop a more efficient inference method based on this relationship. iii) The methodology involves generating multiple reasoning chains for the same question using leading LLMs, comparing the accuracy of shortest, longest, and randomly selected chains, and proposing a novel inference method called short-m@k, which halts computation after a predetermined number of short chains are generated. iv) Results show that shortest reasoning chains can be up to 34.5% more accurate than the longest chains for the same question, and the short-m@k method can reduce compute by up to 40% while maintaining or improving performance. v) The principal implication for AI practitioners is that prioritizing shorter reasoning chains and using inference methods like short-m@k can significantly improve the efficiency and accuracy of reasoning LLMs, suggesting a potential shift in strategies for test-time compute allocation. |
| UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based |
|
|
| Mobile GUI Agents (Read more on arXiv or HuggingFace) |
Afeng-x, luzimu, Yuxiang007, juice-wang, HanXiao1999 |
i) UI-Genie is a self-improving framework for mobile GUI agents utilizing a reward model and iterative pipeline. ii) The research addresses the challenges of trajectory outcome verification and scalable high-quality training data for GUI agents. iii) The methodology involves a reward model (UI-Genie-RM) with an image-text interleaved architecture, rule-based verification, controlled trajectory corruption, hard negative mining, and a self-improvement pipeline with reward-guided exploration. iv) UI-Genie achieves state-of-the-art performance across multiple GUI agent benchmarks and creates UI-Genie-RM-517k and UI-Genie-Agent-16k datasets; the 72B model reaches 77.0% success rate on AndroidControl high-level tasks. v) The framework’s iterative self-improvement and reward-specific dataset provide AI practitioners with a methodology for training and improving GUI agents without manual annotation. |
| Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via |
|
|
| Semantic-Aware Permutation (Read more on arXiv or HuggingFace) |
han-cai, jt-zhang, ylzhao, xihc-ucb, andy-yang |
Sparse VideoGen2 (SVG2) accelerates video generation by optimizing sparse attention mechanisms. The research aims to improve the trade-off between generation quality and computational efficiency in Diffusion Transformers (DiTs) for video generation. The proposed method, SVG2, employs semantic-aware permutation using k-means clustering to identify and densify critical tokens, alongside a top-p selection strategy and custom kernel implementations. Experiments show that SVG2 achieves up to 2.30x speedup on Hunyuan-Video with a PSNR of up to 30 and 1.89x speedup on Wan 2.1 with a PSNR of up to 26 compared to dense attention. SVG2 offers AI practitioners a more efficient framework for video generation by maximizing critical token identification accuracy and minimizing wasted computation. |
| MMMR: Benchmarking Massive Multi-Modal Reasoning Tasks (Read more on arXiv or HuggingFace) |
Guiyao Tie, sunlichao137, MengquSun, Cpmores, zhouxueyang |
MMMR introduces a benchmark for evaluating multi-modal reasoning with explicit thinking traces. The research aims to rigorously evaluate multi-modal reasoning with explicit thinking traces in MLLMs. The methodology involves a high-difficulty dataset spanning six reasoning types and a Reasoning Trace Evaluation Pipeline (RTEP) assessing reasoning quality via relevance, consistency, and error annotations. Empirical results indicate that even top MLLMs-T models like Claude-3.7-Sonnet and Gemini-2.5 Pro exhibit inconsistencies and overthinking despite outperforming non-thinking counterparts, while Gemini-2.5 Pro achieves 42.45% accuracy against human expert levels of 52.85%. This benchmark provides an actionable evaluation pipeline to diagnose reasoning failures and improve the next generation of multi-modal reasoning systems. |
| MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in |
|
|
| Video Scenarios (Read more on arXiv or HuggingFace) |
Huanyao Zhang, Wulin Xie, Huanqian Wang, xinfeng1i, DogNeverSleep |
i) This paper introduces MME-VideoOCR, a new benchmark for evaluating video OCR capabilities of Multimodal Large Language Models (MLLMs). ii) The main objective is to assess the ability of MLLMs to perform OCR and related reasoning tasks in video scenarios, overcoming challenges like motion blur and temporal variations. iii) The methodology involves curating a dataset of 1,464 videos with 2,000 question-answer pairs, categorized into 10 task types and 25 individual tasks, followed by evaluating 18 state-of-the-art MLLMs. iv) Evaluation revealed that even the best-performing model, Gemini-2.5 Pro, achieved an accuracy of only 73.7% on the benchmark, indicating limitations in tasks requiring holistic video comprehension. v) The findings imply AI practitioners must address the deficiencies of current MLLMs in spatio-temporal reasoning and cross-frame information integration to improve OCR performance in dynamic video settings, and that high-resolution visual inputs and sufficient temporal coverage are crucial. |
| OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for |
|
|
| Subject-to-Video Generation (Read more on arXiv or HuggingFace) |
chongyangma, Jinfa, dyf, pkuhexianyi, BestWishYsh |
i) The paper introduces OpenS2V-Nexus, comprising OpenS2V-Eval, a benchmark, and OpenS2V-5M, a million-scale dataset, for subject-to-video (S2V) generation. ii) The research aims to provide infrastructure for evaluating S2V models, focusing on subject consistency, naturalness, and text relevance. iii) The methodology involves curating a dataset of subject-text-video triples and developing three automatic metrics: NexusScore, NaturalScore, and GmeScore. iv) The study evaluates 16 S2V models and creates OpenS2V-5M, which contains 5 million subject-text-video triples. v) The infrastructure supports researchers evaluating S2V models and developing S2V models. |
| GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient |
|
|
| Fine-Tuning (Read more on arXiv or HuggingFace) |
hyek90, tae-su-kim, HyungjunKim, daehyunahn, yeonjoon-jung |
i) GraLoRA introduces a novel granular low-rank adaptation method for parameter-efficient fine-tuning of large language models. ii) The research aims to address the limitations of LoRA concerning rank limitations due to gradient entanglement. iii) The method partitions weight matrices into sub-blocks, each with its own low-rank adapter, mitigating channel dominance. iv) Experiments show GraLoRA achieves up to +8.5% absolute gain in Pass@1 on HumanEval+ compared to LoRA and other baselines. v) AI practitioners can use GraLoRA as a scalable PEFT method to improve fine-tuning performance, particularly in scenarios requiring nuanced representations and complex reasoning. |
| Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning? (Read more on arXiv or HuggingFace) |
Teng Wang, CeciliaJL, yxgeee, tttoaster, Howe666 |
i) This paper introduces Video-Holmes, a new benchmark for evaluating complex video reasoning in multimodal large language models (MLLMs). ii) The research aims to assess whether MLLMs can perform complex video reasoning akin to human experts by locating and connecting multiple relevant visual clues. iii) The methodology involves creating a dataset of 1,837 questions derived from 270 manually annotated suspense short films, designed to test active clue seeking and chain-of-clue reasoning. iv) Evaluation of state-of-the-art MLLMs, including Gemini-2.5-Pro, reveals an accuracy of only 45% on Video-Holmes, indicating substantial challenges in integrating information and identifying critical clues, even with advanced models. v) The principal implication for AI practitioners is the identified need for enhanced reasoning capabilities in MLLMs, specifically in integrating information across diverse video segments and identifying critical clues for more human-like performance, for applications involving complex video analysis. |
| rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale |
|
|
| Verified Dataset (Read more on arXiv or HuggingFace) |
Xudong Zhou, Bingcheng Dong, Yi Zhu, Li Lyna Zhang, YF-L |
i) rStar-Coder introduces a large-scale, verified dataset and a methodology for training code reasoning LLMs. ii) The paper aims to enhance LLM code reasoning capabilities through a scalable, verifiable dataset of competition-level code problems. iii) The methodology involves curating seed problems, synthesizing new problems with a three-step input generation pipeline, and verifying solutions with a mutual verification mechanism. iv) rStar-Coder improves Qwen2.5-7B on LiveCodeBench from 17.4% to 57.3% and achieves a 16.15% average pass@1 accuracy on USACO 2025 using a 7B model, outperforming QWQ-32B. v) The work implies that a curated, verified dataset focused on problem diversity and high-quality reasoning steps can enable smaller LLMs to achieve performance competitive with larger frontier models, benefiting AI practitioners by reducing computational costs. |
| MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent |
|
|
| Systems (Read more on arXiv or HuggingFace) |
Yixuan Li, Yuxuan Chen, samuelyeh, XUANMINGZHANG |
i) The paper introduces MetaMind, a multi-agent framework for enhancing social reasoning in Large Language Models (LLMs). ii) The research aims to improve LLMs’ ability to infer mental states and respond appropriately in ambiguous, context-sensitive social interactions. iii) MetaMind employs three collaborative agents: a Theory-of-Mind Agent for hypothesis generation, a Domain Agent for constraint-based refinement, and a Response Agent for validated output generation. iv) The framework achieves state-of-the-art performance across ToMBench, social cognition, and social simulation benchmarks, including a 35.7% improvement in real-world social scenarios, with LLMs matching human-level performance on ToM tasks for the first time. v) AI practitioners can leverage MetaMind’s architecture to build socially intelligent AI systems, enabling more empathetic dialogue and culturally sensitive interactions by incorporating metacognitive reasoning into LLMs. |
| HoliTom: Holistic Token Merging for Fast Video Large Language Models (Read more on arXiv or HuggingFace) |
Haoxuan You, Can Qin, Keda Tao, Huan-WhoRegisteredMyName, keleshao |
i) HoliTom is introduced as a training-free method to accelerate video large language models (LLMs) through holistic token merging. ii) The primary objective is to reduce computational inefficiency in video LLMs caused by redundant video tokens, while preserving performance. iii) The key methodology involves outer-LLM pruning using global redundancy-aware temporal segmentation and spatio-temporal merging, complemented by a robust inner-LLM token similarity-based merging approach. iv) The method maintains 99.1% average performance while reducing FLOPs to 6.9% on LLaVA-OneVision-7B, achieving a 2.28× reduction in Time-To-First-Token (TTFT) and a 1.32× acceleration in decoding throughput. v) HoliTom enables AI practitioners to achieve efficient video LLM inference with a significantly reduced computational burden, facilitating the deployment of video LLMs in resource-constrained environments. |
| ImgEdit: A Unified Image Editing Dataset and Benchmark (Read more on arXiv or HuggingFace) |
Zongjian Li, Xianyi He, Yang Ye, zhiyuanyan1, BestWishYsh |
i) ImgEdit introduces a new large-scale image editing dataset, benchmark, and editing model. ii) The research aims to address the limitations of existing datasets by creating a high-quality, diverse dataset and a comprehensive benchmark for evaluating image editing models. iii) The study developed an automated data construction pipeline and trained an editing model, ImgEdit-E1, on the new dataset. iv) The dataset comprises 1.2 million edit pairs, and ImgEdit-E1 outperforms existing open-source models on multiple tasks, evaluated using a new benchmark. v) ImgEdit provides AI practitioners with a unified, high-quality resource for training and evaluating image editing models, enabling further advancements in the field. |
| How does Alignment Enhance LLMs’ Multilingual Capabilities? A Language |
|
|
| Neurons Perspective (Read more on arXiv or HuggingFace) |
Xiao Liu, Shuaijie She, VincentLx, DreamW1ngs, Shimao-Zhang |
Multilingual alignment enhances LLMs’ capabilities, analyzed through language neuron identification. The research questions how multilingual alignment influences LLMs’ multilingual proficiency, examined from a language neuron perspective. The study proposes a finer-grained neuron identification algorithm (language-specific, language-related, language-agnostic) and analyzes neuron distribution changes before and after alignment via MAPO. The results indicate that multilingual alignment increases activation of corresponding neuron types across relevant layers and promotes shared language-related neuron utilization, while deactivating language neurons leads to more pronounced effects. The study provides empirical insights for AI practitioners by detailing how multilingual alignment affects neuron activation patterns, suggesting strategies for enhancing multilingual LLMs through targeted neuron manipulation, improving task-relevant understanding in shared semantic space. |
| Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with |
|
|
| Minimalist Rule-Based RL (Read more on arXiv or HuggingFace) |
Yong Dai, Zhongwei Wan, Jiazhen Pan, Haozhe Wang, Che Liu |
AlphaMed explores minimalist rule-based reinforcement learning (RL) to enhance medical LLM reasoning. The paper investigates whether reasoning in medical LLMs can be incentivized solely through rule-based RL on multiple-choice QA data, without supervised fine-tuning (SFT) or distilled chain-of-thought (CoT) data. The study utilizes group relative policy optimization (GRPO) with rule-based rewards on medical QA datasets. AlphaMed achieves state-of-the-art results on six medical QA benchmarks, including a 22.14% accuracy on MedXpert for the 8B model. Minimalist RL with informative QA data is effective at inducing reasoning without CoT supervision, providing a scalable alternative to SFT-based approaches, though the evaluation suggests the need for more challenging, reasoning-oriented benchmarks. |
| Active-O3: Empowering Multimodal Large Language Models with Active |
|
|
| Perception via GRPO (Read more on arXiv or HuggingFace) |
Zongze Du, Hao Zhong, MingyuLiu, Canyu, Z-MU-Z |
i) The paper introduces ACTIVE-03, a reinforcement learning framework using Group Relative Policy Optimization (GRPO) to enable Multimodal Large Language Models (MLLMs) with active perception capabilities. ii) The primary objective is to equip MLLMs with active perception skills for tasks requiring selective sensory information acquisition. iii) The methodology involves a two-stage policy separating region proposal and task execution, combined with a dual-form reward design incorporating task-aware and heuristic feedback. iv) Results show that ACTIVE-03 improves performance in small object detection and interactive segmentation, demonstrated by an APS improvement of +1.0 on LVISsmall over Qwen2.5-VL. v) The framework and benchmark provide AI practitioners with a codebase and evaluation protocol to develop and integrate active perception capabilities into MLLMs, particularly for applications in embodied intelligence and visual grounding. |
| Frame In-N-Out: Unbounded Controllable Image-to-Video Generation (Read more on arXiv or HuggingFace) |
Zezhou Cheng, Matheus Gadelha, Xuweiyi Chen, HikariDawn |
Frame In-N-Out introduces a new image-to-video generation task that enables controllable object entrance/exit beyond initial frame boundaries. The research aims to develop a model for Frame In and Frame Out cinematic techniques, conditioned on user-specified motion trajectories and identity references within an unbounded canvas. The methodology includes curating a semi-automatically generated dataset and developing an efficient identity-preserving motion-controllable video Diffusion Transformer architecture. Evaluation demonstrates significant outperformance against existing baselines, with the Stage2 model achieving a Traj. Err. of 17.85 compared to 41.24 for DragAnything [70] on the Frame Out task. The work implies AI practitioners can utilize the proposed architecture and training methodology to achieve more controllable and spatially unconstrained video generation capabilities. |
| NOVA: A Benchmark for Anomaly Localization and Clinical Reasoning in |
|
|
| Brain MRI (Read more on arXiv or HuggingFace) |
Lena Schmitzer, Evamaria O. Riedel, Philipp Raffler, RioJune, ci-ber |
i) NOVA is introduced as an evaluation-only benchmark for anomaly localization, visual captioning, and diagnostic reasoning on brain MRI scans. ii) The research aims to assess the generalization capabilities of vision-language models in detecting, localizing, and reasoning about rare anomalies in clinical brain MRI under distribution shift. iii) The methodology involves curating a dataset of 906 brain MRI scans from Eurorad spanning 281 pathologies, enriching them with clinical narratives and double-blinded expert bounding box annotations, and evaluating vision-language models (GPT-4o, Gemini 2.0 Flash, Qwen2.5-VL-72B). iv) Results show substantial performance drops across tasks, with anomaly localization mAP@30 ranging from 20.16 to 37.66, indicating poor generalization. v) NOVA serves as a testbed for AI practitioners to develop models that can robustly detect, localize, and reason about truly unknown anomalies, highlighting the need for benchmarks that capture the demands of open-world clinical reasoning, and specifically quantifies the limitations of current models when confronted with real-world clinical data heterogeneity. |
| Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering |
|
|
| Target Atoms (Read more on arXiv or HuggingFace) |
Shumin Deng, Shengyu Mao, Ziwen Xu, Mengru Wang, Ningyu |
i) The paper introduces Steering Target Atoms (STA), a novel method for precise control of LLM behaviors using sparse autoencoders. ii) The research investigates how to enhance safety and control of LLMs by isolating and manipulating disentangled knowledge components. iii) STA utilizes SAE-decoupled representations to identify and manipulate specific target atoms, enabling fine-grained interventions in LLMs. iv) Experiments show STA achieves up to 97.56% average detoxification performance on Gemma-2-9B-it, with minimal impact on general capabilities, demonstrating superior robustness and flexibility in adversarial scenarios. v) STA offers AI practitioners a more robust and precise method for controlling LLM behavior, improving safety and reliability compared to traditional prompt engineering. |
| ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
Hang Zhang, Zixuan Wang, Hongxing Li, Dingming Li, yanyc |
i) The paper introduces ViewSpatial-Bench, a new benchmark for evaluating multi-perspective spatial localization in vision-language models (VLMs). ii) The main objective is to assess and address limitations of current VLMs in understanding spatial relationships from different viewpoints, including camera and human perspectives. iii) The methodology involves creating a dataset with over 5,700 curated samples, using a 3D annotation pipeline and five distinct localization recognition tasks, followed by fine-tuning VLMs on this dataset. iv) Results show that VLMs exhibit reduced accuracy when reasoning from a human viewpoint compared to a camera viewpoint, while fine-tuning on the new dataset improves performance by 46.24% across tasks. v) The principal implication for AI practitioners is the identification of a significant limitation in spatial reasoning within existing VLMs, offering a benchmark and training data to enhance spatial comprehension for embodied AI systems. |
| Code Graph Model (CGM): A Graph-Integrated Large Language Model for |
|
|
| Repository-Level Software Engineering Tasks (Read more on arXiv or HuggingFace) |
Hongen Peng, Zhenhao Tang, Ying Zhang, Hongyuan Tao, Geralt-Targaryen |
i) This paper introduces Code Graph Models (CGMs), a novel architecture integrating code graph structures into Large Language Models (LLMs) for improved repository-level software engineering task performance. ii) The primary research question is whether open-source LLMs can effectively address repository-level tasks without agent-based approaches by incorporating code graph information. iii) The methodology involves integrating code graph structures into the LLM’s attention mechanism and mapping node attributes using a specialized adapter, combined with an agentless graph RAG framework. iv) The approach achieves a 43.00% resolution rate on the SWE-bench Lite benchmark using the open-source Qwen2.5-72B model. v) CGMs offer AI practitioners a new method for leveraging open-source LLMs in repository-level software engineering tasks without proprietary agent systems, improving predictability and enabling data privacy and model customization. |
| DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via |
|
|
| Next-Detail Prediction (Read more on arXiv or HuggingFace) |
Xu Wang, Huichao Zhang, Yiheng Liu, JiangYi, leo1117 |
DetailFlow presents a coarse-to-fine 1D autoregressive image generation method using a next-detail prediction strategy. The paper investigates whether a coarse-to-fine 1D token sequence can efficiently model images by learning a resolution-aware token sequence supervised with progressively degraded images. DetailFlow uses a compact 1D AR model and a parallel inference mechanism with self-correction. On ImageNet 256x256, DetailFlow achieves 2.96 gFID with 128 tokens. DetailFlow provides AI practitioners with a more efficient autoregressive approach that achieves better image quality with fewer tokens and faster inference. |
| SeePhys: Does Seeing Help Thinking? – Benchmarking Vision-Based Physics |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Zirong Liu, Terry Jingchen Zhang, Kun Xiang, yinyahuang, HengLi29 |
SeePhys: A new multimodal benchmark for physics reasoning is introduced to evaluate LLMs’ visual understanding. The research aims to assess LLMs’ capabilities in physics reasoning grounded in visual information from middle school to PhD levels. It uses a dataset of 2,000 physics questions spanning 7 domains and 21 diagram types, including a vision-essential subset that mandates visual information extraction for solutions. Evaluation of LLMs like Gemini-2.5-pro and o4-mini reveals a sub-60% accuracy, highlighting challenges in current models’ visual understanding and coupling with physics reasoning. The study indicates a need for AI practitioners to improve LLMs’ ability to integrate diagram interpretation with physics reasoning, overcoming reliance on textual cues, which currently limits their visual reasoning capacity. |
| Adversarial Attacks against Closed-Source MLLMs via Feature Optimal |
|
|
| Alignment (Read more on arXiv or HuggingFace) |
Chao Du, Tianyu Pang, Simeng Qin, Sensen Gao, jiaxiaojunQAQ |
i) This paper introduces FOA-Attack, a targeted transferable adversarial attack method against Multimodal Large Language Models (MLLMs). ii) The research aims to improve adversarial transferability by optimizing the alignment of both global and local image features between adversarial and target samples. iii) FOA-Attack employs a global feature loss based on cosine similarity and a local clustering optimal transport (OT) loss, along with a dynamic ensemble model weighting strategy. iv) Experiments show FOA-Attack achieves a 70.7% attack success rate on Qwen2.5-VL-7B, surpassing existing methods, and up to 77.3% ASR on GPT-4.1, indicating a 16.5% performance improvement, specifically when transferred to closed-source models. v) AI practitioners should consider feature-level adversarial vulnerabilities in MLLMs and explore feature optimal alignment to enhance robustness against transferable attacks. |
| Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Peter Grabowski, Tianqi Liu, Yinxiao Liu, Yaqing Wang, Shenao Zhang |
i) The paper introduces Bayes-Adaptive RL (BARL), an algorithm for reflective exploration in Large Language Models (LLMs) reasoning by optimizing the expected return under a posterior distribution over Markov decision processes. ii) The research aims to address whether reflective reasoning emerges during Markovian RL training and why such behaviors may be beneficial at test time. iii) The methodology involves recasting reflective exploration within a Bayes-Adaptive RL framework, incentivizing reward-maximizing exploitation and information-gathering exploration through belief updates. iv) Empirical results show BARL outperforms standard Markovian RL approaches at test time, achieving superior token efficiency and improved exploration effectiveness, with BARL requiring up to 39% fewer average tokens than a progress baseline on reasoning tasks. v) BARL provides AI practitioners with a novel approach for training LLMs to adaptively switch strategies based on observed outcomes, improving reasoning performance through a principled mechanism for integrating and revising plausible strategies. |
| Sci-Fi: Symmetric Constraint for Frame Inbetweening (Read more on arXiv or HuggingFace) |
Xianyi He, Xiaoyu Li, Xiaodong Cun, Liuhan Chen, BestWishYsh |
Sci-Fi introduces a novel frame inbetweening framework leveraging symmetric constraints to generate harmonious intermediate video frames. The research aims to improve the quality of synthesized intermediate video sequences conditioned on start and end frames by addressing limitations in current Image-to-Video Diffusion Model (I2V-DM) based methods. The methodology involves a lightweight module, EF-Net, to encode the end frame and inject temporally adaptive features into a base I2V-DM. Experiments show Sci-Fi achieves superior performance with a VBench score of 0.8373 on the Pexels dataset compared to other baselines. This work implies AI practitioners can utilize the Sci-Fi framework to produce higher quality and more consistent intermediate frames in video generation tasks with improved control mechanisms. |
| Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Mao Zheng, Nickyang |
i) ConciseR, a two-stage reinforcement learning framework, aims to enhance and subsequently compress the reasoning of LLMs. ii) The research question is how to achieve concise reasoning in LLMs without sacrificing accuracy. iii) The methodology involves a two-stage reinforcement learning approach: first using Group Relative Policy Optimization with clip-higher and dynamic sampling (GRPO++) and an entropy bonus, then using Length-aware Group Relative Policy Optimization (L-GRPO). iv) Experimental results show ConciseR outperforms baselines with zero RL paradigm across AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad benchmarks, achieving an average accuracy improvement and a 21-23% reduction in response length. v) ConciseR offers AI practitioners a method to train LLMs for more concise and efficient reasoning, balancing accuracy and reduced computational cost. |
| Minute-Long Videos with Dual Parallelisms (Read more on arXiv or HuggingFace) |
Xinchao Wang, Yuecong Xu, Xingyi Yang, Bowen Zheng, Zeqing Wang |
i) This paper introduces DualParal, a distributed inference strategy for DiT-based video diffusion models, parallelizing both temporal frames and model layers. ii) The research aims to mitigate the high processing latency and memory costs associated with generating long videos using DiT models. iii) The methodology involves a block-wise denoising scheme and asynchronous processing across GPUs, incorporating a feature cache to reduce inter-GPU communication and a coordinated noise initialization strategy. iv) Experiments show DualParal achieves up to a 6.54x reduction in latency and a 1.48x reduction in memory cost when generating 1,025-frame videos on 8×RTX 4090 GPUs. v) AI practitioners can leverage DualParal to efficiently generate high-quality, long videos with DiT-based models by mitigating memory bottlenecks and reducing inference latency using the developed parallelization strategies. |
| VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual |
|
|
| Tool Selection (Read more on arXiv or HuggingFace) |
Wen Xiao, Zefan Cai, Yuyang Ji, AniSundar18, ZeyiHuang1010 |
i) The paper introduces VisualToolAgent (VisTA), a reinforcement learning framework for adaptive tool selection in visual reasoning tasks. ii) The research aims to develop a system that can autonomously learn to select and combine appropriate external tools for visual reasoning, improving performance over training-free and fine-tuning methods. iii) VisTA employs end-to-end reinforcement learning with Group Relative Policy Optimization (GRPO) to train an agent to select tools based on empirical performance feedback. iv) Experiments on ChartQA demonstrate that VisTA achieves 79.4% accuracy, a 3.0-point improvement over the best training-free baseline; VisTA with GPT-40 achieves 88.9% accuracy. v) The framework provides AI practitioners with a method for developing more flexible and generalizable visual reasoning systems by enabling dynamic tool selection based on task-specific characteristics. |
| Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic |
|
|
| Capabilities in LLM Compression (Read more on arXiv or HuggingFace) |
Xiaowen Chu, Lujun Li, Zhenheng Tang, Peijie Dong, Dominic789654 |
i) The paper introduces ACBench, a benchmark to evaluate the impact of compression on agentic abilities in LLMs. ii) The primary research question is how post-training compression methods affect LLMs’ performance on tasks requiring workflow generation, tool use, long-context understanding, and real-world application. iii) The methodology involves evaluating 15 models using quantization and pruning techniques across 12 tasks, with new metrics (ERank, Top-k Ranking Correlation, Energy) for systematic analysis. iv) Experiments reveal that 4-bit quantization preserves workflow generation and tool use (1%-3% drop) but degrades real-world application accuracy by 10%-15%; distilled reasoning LLMs show performance degradation in certain agent scenarios. v) The findings offer actionable insights for optimizing LLM compression strategies in agentic scenarios, indicating that while quantization can maintain certain agentic capabilities, real-world application accuracy may be significantly compromised, a critical consideration for AI practitioners deploying compressed LLMs in practical applications. |
| R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs |
|
|
| via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zhipeng Chen, Wenqing Tian, Jinhao Jiang, Huatong Song, EliverQ |
R1-Searcher++ enhances LLMs by adaptively leveraging both internal knowledge and external search. The research aims to train LLMs to dynamically acquire knowledge, balancing internal recall and external retrieval. It employs a two-stage training strategy: SFT Cold-start for format learning followed by RL for Dynamic Knowledge Acquisition, incorporating outcome-supervision and a memorization mechanism. Experiments using Qwen-2.5-7B-Instruct demonstrate the method surpasses baselines by up to 4.3% while reducing retrieval counts by 42.9%. This suggests AI practitioners can utilize the approach to more efficiently create retrieval-augmented reasoning models that have high quality internal knolwedge and make external retrievals as needed. |
| DFIR-Metric: A Benchmark Dataset for Evaluating Large Language Models in |
|
|
| Digital Forensics and Incident Response (Read more on arXiv or HuggingFace) |
Saeed Alshehhi, Aaesha Aldahmani, Richard A. Dubniczky, Tamas Bisztray, Bilel Cherif |
i) DFIR-Metric is introduced as a benchmark for evaluating LLMs in Digital Forensics and Incident Response (DFIR). ii) The primary objective is to establish a comprehensive benchmark evaluating LLMs across theoretical and practical DFIR tasks. iii) The methodology involves a three-part dataset including multiple-choice questions, CTF-style challenges, and NIST CFTT string search cases, evaluated using accuracy, consistency, and a novel Task Understanding Score (TUS). iv) Experimental results show GPT-4.1 achieves a Confidence Index of 89.34% and a Mean Accuracy of 92.75% on multiple-choice questions, while TUS@4 reached 38.52% for the NIST forensic string search task. v) The implication for AI practitioners is the need for improved reasoning and adherence to output specifications in LLMs for reliable application in digital forensics, as current models struggle with sustained deductive reasoning and calibrated confidence. |
| SoloSpeech: Enhancing Intelligibility and Quality in Target Speech |
|
|
| Extraction through a Cascaded Generative Pipeline (Read more on arXiv or HuggingFace) |
Kai Li, Chen Chen, Dongchao Yang, Jiarui Hai, westbrook |
SoloSpeech introduces a cascaded generative pipeline for target speech extraction. The research aims to enhance the intelligibility and quality of extracted speech by integrating compression, extraction, reconstruction, and correction processes. It employs a speaker-embedding-free target extractor using a latent diffusion model conditioned on cue audio and a T-F domain diffusion model as a corrector. Evaluated on Libri2Mix, SoloSpeech achieves a WER of 0.16, demonstrating state-of-the-art intelligibility and quality and improved generalization to out-of-domain data. The pipeline offers AI practitioners a robust and generalizable method for speech extraction tasks, with the provided source code enabling integration into existing speech processing systems. |
| MLLMs are Deeply Affected by Modality Bias (Read more on arXiv or HuggingFace) |
Yuanhuiyi Lyu, Kaiyu Lei, Yuqian Fu, Xu Zheng, Chenfei-Liao |
i) This paper investigates the presence and impact of modality bias in Multimodal Large Language Models (MLLMs). ii) The research aims to diagnose the current state of modality bias in MLLMs, propose a research roadmap, and identify key factors contributing to this bias. iii) The study employs empirical analysis involving missing modality evaluations on the MMMU-Pro dataset using Qwen2.5VL models, along with theoretical discussion. iv) Results show a significant reliance on textual information, with consistency between complete and text-only inputs at 56.53% compared to lower consistency with image-only inputs (27.17%), suggesting underutilization of visual modalities. v) AI practitioners should focus on balanced training strategies, optimizing multimodal integration and addressing dataset imbalances to mitigate modality bias and improve MLLM generalizability. |
| ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and |
|
|
| Reactive Feedback (Read more on arXiv or HuggingFace) |
Jinsong Zhou, Jiantao Lin, Luozhou Wang, Xinli Xu, Litao Guo |
i) ComfyMind is presented as a collaborative AI system for robust and scalable general-purpose generation based on the ComfyUI platform. ii) The research aims to address the limitations of existing open-source generative frameworks by incorporating structured workflow planning and execution-level feedback. iii) The methodology involves a Semantic Workflow Interface (SWI) that abstracts node graphs into functional modules and a Search Tree Planning mechanism with localized feedback execution. iv) ComfyMind achieves a 100% workflow pass rate on ComfyBench, improving upon the 56% of existing methods, and reaches a GPT-score of 0.906 on Reason-Edit. v) AI practitioners can utilize ComfyMind’s architecture to enhance the stability and flexibility of complex generative workflows, potentially improving performance in tasks requiring modular composition and hierarchical planning. |
| R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large |
|
|
| Language Models via Share-GRPO (Read more on arXiv or HuggingFace) |
Yibo Wang, Min Yang, Jingyi Zhang, Qixiang Yin, Huanjin Yao |
R1-ShareVL introduces Share-GRPO, a reinforcement learning approach to enhance reasoning in multimodal large language models (MLLMs). The research aims to mitigate sparse reward and advantage vanishing issues in MLLMs through reinforcement learning. Share-GRPO expands the question space using semantic transformations and shares reasoning trajectories across diverse question variants. Experiments on six reasoning benchmarks demonstrate Share-GRPO’s superiority, with R1-ShareVL-7B achieving a +7.2% improvement on the MathVista benchmark compared to the baseline. AI practitioners can leverage Share-GRPO to improve MLLM reasoning by diversifying training data and stabilizing policy optimization through shared reward information. |
| Search and Refine During Think: Autonomous Retrieval-Augmented Reasoning |
|
|
| of LLMs (Read more on arXiv or HuggingFace) |
Junfeng Fang, Zhiyuan Liu, Chang Wu, Shihan Li, yrshi |
i) AutoRefine improves retrieval-augmented reasoning in LLMs using explicit knowledge refinement and tailored rewards. ii) The research aims to enhance LLMs’ reasoning capabilities by enabling iterative filtering, distilling, and organizing evidence from retrieved documents. iii) A reinforcement learning post-training framework, AutoRefine, is introduced that incorporates explicit knowledge refinement steps between search calls alongside retrieval-specific and answer correctness rewards using group relative policy optimization. iv) Experiments demonstrate AutoRefine outperforms existing approaches by 6.9% higher average accuracy, particularly in complex, multi-hop reasoning scenarios and exhibits a 20% improvement in refinement success rate. v) AI practitioners can utilize AutoRefine to improve the accuracy and robustness of LLMs in knowledge-intensive tasks by incorporating retrieval-specific rewards and explicit knowledge refinement steps, enabling more effective use of external knowledge sources. |
| AdInject: Real-World Black-Box Attacks on Web Agents via Advertising |
|
|
| Delivery (Read more on arXiv or HuggingFace) |
Mingyang Li, Rupeng Zhang, Xiaojun Jia, Junjie Wang, NicerWang |
i) AdInject introduces a novel black-box attack vector leveraging advertising delivery to compromise Web Agents. ii) The research aims to demonstrate the vulnerability of Web Agents to environment injection attacks through advertising channels. iii) The methodology involves crafting malicious ad content and optimizing it using a VLM to infer user intents from website context. iv) Experiments show attack success rates exceeding 60% in most scenarios on VisualWebArena and approaching 100% in certain cases, demonstrating the effectiveness of the proposed attack. v) AI practitioners need to be aware of the potential for real-world advertising delivery systems to be exploited for environment injection attacks on Web Agents, necessitating the development of robust defense mechanisms. |
| Modality Curation: Building Universal Embeddings for Advanced Multimodal |
|
|
| Information Retrieval (Read more on arXiv or HuggingFace) |
Shi Feng, Hongzhi Zhang, Yahui Liu, Jingyuan Zhang, friedrichor |
i) This paper introduces UNITE, a framework for building universal multimodal embeddings through data curation and modality-aware training configurations for multimodal information retrieval (MIR). ii) The research investigates how modality-specific data properties and training protocols influence downstream task performance in diverse MIR scenarios. iii) The methodology employs Modal-Aware Masked Contrastive Learning (MAMCL) to balance relationships among instances of different modalities, along with strategic modality curation and tailored training protocols. iv) UNITE achieves state-of-the-art results on multiple multimodal retrieval benchmarks, surpassing existing methods, including improvements of 15.7% in the temporal retrieval aspects of CaReBench. v) This work provides AI practitioners with a foundational blueprint for advancing MIR performance through strategic modality curation and tailored training protocols, particularly by addressing inter-modal interference using MAMCL. |
| Absolute Coordinates Make Motion Generation Easy (Read more on arXiv or HuggingFace) |
Huaizu Jiang, Yiming Xie, Xiaogang Peng, Zeyu Han, cr8br0ze |
Absolute Coordinates Make Motion Generation Easy proposes a novel motion representation for text-to-motion generation using absolute joint coordinates. The research aims to demonstrate superior performance and scalability using absolute coordinates compared to local-relative kinematic-aware representations in diffusion models. The methodology involves a diffusion model (ACMDM) with a Transformer backbone trained on absolute joint coordinates and evaluated through metrics such as FID and R-Precision. The ACMDM-XL-PS2 model achieves a FID of 0.058 and an R-Precision Top-1 score of 0.522 on the HumanML3D dataset, outperforming state-of-the-art methods. The principal implication is that employing absolute coordinates can significantly enhance motion fidelity and controllability in text-to-motion generation models, offering a more straightforward approach without complex kinematic-aware losses or auxiliary components for AI practitioners. |
| Improving Chemical Understanding of LLMs via SMILES Parsing (Read more on arXiv or HuggingFace) |
Sungsoo Ahn, Jaehyung Kim, yunhuijang |
i) The paper introduces CLEANMOL, a framework for enhancing Large Language Models’ (LLMs) understanding of molecular structures via SMILES parsing. ii) The primary objective is to address the limitations of current LLMs in accurately interpreting SMILES strings by developing clean and deterministic parsing tasks. iii) The methodology involves pre-training LLMs on a constructed dataset with structured supervision derived from subgraph and global graph matching tasks extracted from SMILES representations, incorporating adaptive difficulty scoring and curriculum learning. iv) The results show that CLEANMOL enhances structural comprehension, achieving state-of-the-art or competitive performance on the Mol-Instructions benchmark; for instance, LLaMA3.1-8B achieved a 0.005 MAE on the Mol-Instructions molecular property regression task. v) The principal implication for AI practitioners is the demonstration that incorporating deterministic structural supervision via SMILES parsing can significantly enhance molecular generation capabilities of LLMs, even without direct exposure to generation-specific training data. |
| Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion |
|
|
| Enhances Protein Representations (Read more on arXiv or HuggingFace) |
Ahmed Elnaggar, Mohamed Elkerdawy, Mohamed Elshaffei, hazemessam |
i) Ankh3, a protein language model, leverages multi-task pretraining with sequence denoising and completion for enhanced protein representations. ii) The research investigates whether multi-task pretraining using masked language modeling with multiple masking probabilities alongside protein sequence completion improves protein representation learning. iii) Ankh3 was developed using a T5 architecture and pre-trained on UniRef50 dataset with two objectives: masked language modeling with masking probabilities of 15%, 20%, and 30% and protein sequence completion. iv) The results demonstrated improved performance in secondary structure prediction, achieving 84.4% accuracy on CASP-12, GB1 fitness, and contact prediction with Ankh3-XL. v) The multi-task pretraining strategy in Ankh3 allows for more robust and accurate protein sequence modeling, enabling AI practitioners to develop more effective downstream applications in synthetic biology and protein engineering. |
| Beyond Simple Concatenation: Fairly Assessing PLM Architectures for |
|
|
| Multi-Chain Protein-Protein Interactions Prediction (Read more on arXiv or HuggingFace) |
Abdallah Amr, Sara Ossman, Mohamed Soudy, Mohamed Elshaffei, hazemessam |
i) This paper addresses limitations in predicting protein-protein interaction (PPI) binding affinity using protein language models (PLMs). ii) The research investigates the efficacy of various PLM architectures in sequence-based, multi-chain PPI binding affinity prediction. iii) The methodology includes curating a refined PPB-Affinity dataset, implementing stringent data splitting to mitigate leakage, and systematically evaluating four PLM architectures: embeddings concatenation (EC), sequences concatenation (SC), hierarchical pooling (HP), and pooled attention addition (PAD). iv) Results demonstrate that HP and PAD architectures outperform conventional concatenation methods, achieving up to a 12% increase in Spearman correlation (ρ); the curated PPB-Affinity dataset contains 8,207 unique PPI entries. v) The implication for AI practitioners is the necessity of sophisticated architectural designs, such as HP and PAD, to fully leverage PLMs for improved PPI binding affinity prediction, moving beyond simple concatenation strategies. |
| An Explainable Diagnostic Framework for Neurodegenerative Dementias via |
|
|
| Reinforcement-Optimized LLM Reasoning (Read more on arXiv or HuggingFace) |
Eloi Navet, Laurent Simon, Boris Mansencal, Nathanael Fijalkow, Andrew Zamai |
i) This paper presents an explainable AI framework for the differential diagnosis of neurodegenerative dementias. ii) The research aims to improve diagnostic transparency by integrating radiology report generation from 3D brain MRIs and reinforcement learning optimized LLM reasoning. iii) The methodology involves a modular pipeline for converting 3D brain MRIs into textual reports, prompting LLMs for diagnostic reasoning, and fine-tuning with Group Relative Policy Optimization (GRPO). iv) Experiments show GRPO fine-tuning enables 8B models to match or surpass the diagnostic accuracy of larger models like GPT-40, yielding detailed reasoning grounded in neuroanatomical evidence, achieving a BACC of 84.16% and an M-F1 score of 59.55% for CN class. v) AI practitioners can leverage this framework to develop more transparent and trustworthy diagnostic systems by combining quantitative image analysis with structured language model reasoning, promoting causally grounded explanations for clinical decision-making. |
| Tropical Attention: Neural Algorithmic Reasoning for Combinatorial |
|
|
| Algorithms (Read more on arXiv or HuggingFace) |
Ruriko Yoshida, Chris Teska, Kurt Pasque, Baran47 |
i) This paper introduces Tropical attention, a novel attention mechanism for neural algorithmic reasoning that operates in the max-plus semiring. ii) The main objective is to develop an attention mechanism that enhances out-of-distribution (OOD) generalization and robustness for dynamic programming-type combinatorial algorithms. iii) The methodology involves replacing the softmax-normalized dot-product attention with Tropical attention and proving its ability to approximate tropical circuits and enhance empirical OOD performance. iv) The primary results show that Tropical transformers achieve state-of-the-art OOD generalization in length and value scale and exhibit superior adversarial robustness across eleven combinatorial tasks, outperforming softmax baselines. v) The implication for AI practitioners is that using Tropical attention in transformers can improve OOD performance and adversarial robustness in algorithmic reasoning tasks, particularly those involving dynamic programming, without super-polynomial blow-ups. |
| Do RAG Systems Suffer From Positional Bias? (Read more on arXiv or HuggingFace) |
Fabrizio Silvestri, Yoelle Maarek, Guy Horowitz, Simone Filice, florin-hf |
i) This paper investigates positional bias in Retrieval Augmented Generation (RAG) systems and its impact on LLM vulnerability to distracting passages. ii) The main research question is how positional bias affects an LLM’s capability to capitalize on relevant passages while also being susceptible to distracting passages in RAG systems. iii) The methodology includes experiments on three question-answering benchmarks (PopQA, Natural Questions, and TriviaQA) using BM25 and BGE for retrieval, evaluating the distracting effect of passages using a LLM-as-a-judge approach. iv) The primary result shows that current retrieval pipelines systematically bring highly distracting passages to the top ranks, with over 60% of queries containing at least one highly distracting passage among the top-10 retrieved passages. v) The principal implication for AI practitioners is that improvements in RAG systems should focus on retrieval quality and LLM distraction robustness rather than passage positioning strategies. |
Papers for 2025-05-27
| Title |
Authors |
Summary |
| Shifting AI Efficiency From Model-Centric to Data-Centric Compression (Read more on arXiv or HuggingFace) |
Pppeach33, coderchen01, Steven-Shaobo, zichenwen, xuyang-liu16 |
i) This paper argues for a paradigm shift from model-centric to data-centric compression, specifically token compression, to improve AI efficiency by reducing token counts during training and inference. ii) The main research objective is to analyze and advocate for token compression as a crucial strategy in addressing the computational bottlenecks introduced by increasing context lengths in LLMs and MLLMs. iii) The methodology involves a comprehensive analysis of long-context AI developments, a unified mathematical framework for model efficiency strategies, and a systematic review of token compression techniques. iv) Results show that attention-based token compression methods can underperform compared to simple random pruning in certain scenarios; also, from 2022-2024 model size primarily drove computational costs, but from 2024 onward, token count has grown exponentially. v) AI practitioners should shift focus toward data-centric approaches like token compression, exploring methods that maintain spatial uniformity and mitigate biases, to achieve more efficient and scalable AI systems, and evaluate existing token compression techniques carefully, as speedup is not always reflected in runtime latency. |
| Mutarjim: Advancing Bidirectional Arabic-English Translation with a |
|
|
| Small Language Model (Read more on arXiv or HuggingFace) |
Sara Chrouf, ZeinaD, Moatasem444, hr99, Hennara |
i) The paper introduces Mutarjim, a compact language model for bidirectional Arabic-English translation, and Tarjama-25, a new benchmark dataset. ii) The research aims to develop a smaller, task-specific model that balances translation performance with efficiency, specifically for Arabic-English translation. iii) The methodology involves a two-phase training approach: large-scale monolingual pre-training and supervised fine-tuning with high-quality Arabic-English parallel data, building upon the Kuwain-1.5B language model. iv) Experimental results demonstrate that Mutarjim outperforms larger models, achieving state-of-the-art performance on the English-to-Arabic task in Tarjama-25, surpassing models like GPT-40 mini, with a ChrF score of 83.41. v) The development of Mutarjim provides AI practitioners with a resource-efficient alternative for Arabic-English translation, demonstrating that smaller, specialized models can achieve competitive performance while reducing computational costs. |
| BizFinBench: A Business-Driven Real-World Financial Benchmark for |
|
|
| Evaluating LLMs (Read more on arXiv or HuggingFace) |
Ji Liu, Qlisp, Tinker250, xuntao, guilong |
i) BizFinBench, a new financial benchmark, is introduced to evaluate LLMs in real-world financial applications. ii) The research aims to rigorously evaluate LLMs across a broad spectrum of real-world financial tasks within the financial domain. iii) The methodology involves a new dataset construction and the introduction of IteraJudge, an iterative calibration-based evaluation framework. iv) The evaluation of 25 LLMs reveals that Gemini-2.0-Flash achieves SOTA performance in Anomalous Event Attribution with a score of 86.94. v) AI practitioners should be aware of the limitations of current LLMs in handling complex financial tasks requiring integrated knowledge and cross-concept reasoning, suggesting areas for future model development. |
| Alchemist: Turning Public Text-to-Image Data into Generative Gold (Read more on arXiv or HuggingFace) |
Sergey Kastryulin, Dmitry Baranchuk, Alexey Kirillov, Alexander Ustyuzhanin, sharfikeg |
i) This paper introduces Alchemist, a supervised fine-tuning (SFT) dataset and methodology for enhancing the generative quality of text-to-image (T2I) models. ii) The research objective is to develop a method for curating general-purpose SFT datasets that improve T2I model performance while maintaining diversity and style. iii) The methodology involves a multi-stage filtering pipeline leveraging a pre-trained generative model to estimate the impact of training samples, followed by re-captioning using a vision-language model. iv) Experiments show that fine-tuning public T2I models with the 3,350-sample Alchemist dataset improves aesthetic quality and image complexity by up to 20% in human preference win rates compared to baseline models. v) AI practitioners can utilize the Alchemist dataset and methodology to efficiently fine-tune T2I models, achieving substantial gains in generative quality using a relatively small, high-quality dataset. |
| Embodied Agents Meet Personalization: Exploring Memory Utilization for |
|
|
| Personalized Assistance (Read more on arXiv or HuggingFace) |
jinyeo, ej0cl6, bwookwak, Lune-Blue, Connoriginal |
i) The paper introduces MEMENTO, a framework for evaluating episodic memory utilization in LLM-powered embodied agents for personalized assistance in object rearrangement tasks. ii) The research investigates the effectiveness of embodied agents in leveraging memory to understand user-specific object semantics and routines for personalized instruction interpretation. iii) The methodology involves a two-stage process: Memory Acquisition and Memory Utilization, comparing agent performance on tasks with and without explicit personalized knowledge cues. iv) Experiments revealed that even the frontier model GPT-40 experienced a 30.5% performance drop in joint-memory tasks when required to reference multiple memories, particularly those involving user patterns. v) The study implies that current LLM-powered embodied agents face significant limitations in effectively leveraging episodic memory for personalized assistance, highlighting the need for improved memory utilization and reasoning capabilities in complex, multi-step personalized tasks. |
| PATS: Process-Level Adaptive Thinking Mode Switching (Read more on arXiv or HuggingFace) |
Shujian Huang, Jiajun Chen, Shimao Zhang, master-lan, Yi53 |
i) This paper introduces Process-Level Adaptive Thinking Mode Switching (PATS), a novel reasoning paradigm for Large Language Models (LLMs). ii) The primary research objective is to enable LLMs to dynamically adjust reasoning strategies at each step based on problem difficulty, balancing accuracy and computational efficiency. iii) The methodology integrates Process Reward Models (PRMs) with Beam Search, incorporating progressive mode switching and bad-step penalty mechanisms. iv) Experiments on mathematical benchmarks demonstrate that PATS achieves high accuracy while maintaining moderate token usage, with a 4.4-point accuracy improvement over solution-verification switching while using 7% fewer tokens. v) PATS’s adaptive switching mechanism, dynamically adjusting reasoning based on step-wise difficulty, provides AI practitioners with a method to improve LLM inference efficiency without sacrificing accuracy. |
| ARM: Adaptive Reasoning Model (Read more on arXiv or HuggingFace) |
Kai Zhang, Aili Chen, Arist12, hsaest, Siye01 |
i) The paper introduces ARM, a model that adaptively selects reasoning formats to balance performance and computational efficiency. ii) The main objective is to develop a reasoning model that can dynamically adjust its token usage based on task complexity without human intervention. iii) The methodology involves a two-stage training framework: supervised fine-tuning (SFT) for format understanding, followed by reinforcement learning using Ada-GRPO, an adaptation of Group Relative Policy Optimization. iv) ARM achieves comparable performance to models relying solely on Long CoT, while reducing token usage by an average of 30% and up to 70% and achieves an approximate 2x speedup in training. v) ARM provides AI practitioners with a method to create more efficient and performant reasoning models by dynamically adapting reasoning strategies based on task requirements, reducing computational overhead without sacrificing accuracy. |
| Enigmata: Scaling Logical Reasoning in Large Language Models with |
|
|
| Synthetic Verifiable Puzzles (Read more on arXiv or HuggingFace) |
Zhicheng Cai, Aili Chen, siyuyuan, Abbey4799, jiangjiechen |
i) This paper introduces ENIGMATA, a comprehensive suite for scaling logical reasoning in LLMs using synthetic, verifiable puzzles. ii) The research aims to improve LLMs’ puzzle reasoning skills through a tailored suite of tasks, evaluation benchmarks, and training recipes. iii) The methodology includes generating 36 diverse puzzle tasks with controllable difficulty and automatic verification, alongside optimized multi-task RLVR strategies. iv) Qwen2.5-32B-ENIGMATA surpasses prior state-of-the-art LRMs on ARC-AGI (32.8%) and the ENIGMATA-Eval benchmark. v) ENIGMATA provides AI practitioners with a unified framework for advancing logical reasoning in LLMs, enhancing their performance on complex problem-solving tasks and demonstrating potential benefits for larger models in math/STEM. |
| B-score: Detecting biases in large language models using response |
|
|
| history (Read more on arXiv or HuggingFace) |
Daeyoung Kim, anhng8, taesiri, anvo25 |
i) The paper introduces B-score, a novel metric for detecting biases in large language models (LLMs) based on response history in multi-turn conversations. ii) The primary research objective is to determine if LLMs can reduce biases by observing their prior responses in a multi-turn conversational setting and to assess the effectiveness of B-score in detecting different types of biases. iii) The methodology involves comparing single-turn and multi-turn conversational responses across subjective, random, and objective question categories, calculating B-score as the difference in probabilities of an answer appearing in single-turn versus multi-turn settings. iv) The results demonstrate that LLMs can “de-bias” themselves in multi-turn conversations for random questions and using B-score improves answer verification accuracy by +9.3 on a proposed question dataset. v) AI practitioners can leverage B-score as a runtime indicator to detect and mitigate biased responses from LLMs, particularly in scenarios where access to ground truth labels is limited, substantially enhancing the verification accuracy of LLM answers. |
| Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective (Read more on arXiv or HuggingFace) |
Linchen Xiao, Hongwei Liu, zsytony, Sudanl, jnanliu |
The paper proposes RAML (Reasoning as Meta-Learning), a novel framework that interprets LLM reasoning through the lens of meta-learning. The research aims to understand and optimize LLM reasoning capabilities by conceptualizing reasoning trajectories as pseudo-gradient descent updates to LLM parameters. The methodology formalizes reasoning task training as a meta-learning setup, treating each question as a distinct task and using reasoning trajectories for inner-loop parameter adaptation. Evaluations using Qwen2.5-7B-Base demonstrate that supervised fine-tuning with 32 synthetic reasoning trajectories per question improves performance, showing a gain in Pass@8 metric, and increased reasoning efficiency is attainable, albeit requires further investigation on which token types facilitate the most efficient reasoning. RAML provides a foundation for applying meta-learning insights to enhance LLM reasoning by framing it as a process of optimizing pseudo-gradient descent. |
| Lifelong Safety Alignment for Language Models (Read more on arXiv or HuggingFace) |
Min Lin, Chao Du, Yifei Zhao, Zeyu Qin, Haoyu Wang |
This paper introduces a lifelong safety alignment framework for language models (LLMs). The research question addresses how to continuously adapt LLMs to new and evolving jailbreaking strategies. The methodology employs a competitive setup between a Meta-Attacker, trained to discover novel jailbreaking strategies, and a Defender, trained to resist them, warm-started with insights extracted from jailbreak-related research. The primary result is a reduction of the Meta-Attacker’s success rate from 73% to 7% on RR after iterative training and a 57% transfer attack success rate on LAT using single-turn attacks initially. The principal implication for AI practitioners is a framework for improving the robustness and reliability of LLMs in open-ended environments by continually adapting to new attack vectors. |
| MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis |
|
|
| Discovery via Hierarchical Search (Read more on arXiv or HuggingFace) |
Wei Li, Yujie Liu, Ben Gao, Wanhao Liu, ZonglinY |
i) This paper introduces MOOSE-Chem2, a framework for fine-grained scientific hypothesis discovery using LLMs via hierarchical search. ii) The primary objective is to investigate the upper limits of LLMs in generating detailed, experimentally actionable scientific hypotheses from coarse initial research directions. iii) The methodology involves a hierarchical search method that incrementally proposes and integrates details into the hypothesis, progressing from general concepts to specific experimental configurations, defining a reward landscape based on LLM’s internal heuristics. iv) Empirical evaluations demonstrate that the hierarchical search method consistently outperforms strong baselines, and hypotheses generated by the proposed method achieve higher recall than those from baselines (e.g., HHS achieves 40.40% soft recall vs. 16.60% soft recall for Greedy Search). v) This research provides AI practitioners with a structured approach to leverage LLMs for generating more detailed and experimentally viable scientific hypotheses, improving automation of the scientific discovery process. The paper’s results indicate that repeated use of the strongest model provides better reward landscapes than diverse ensembles, suggesting practical implementation strategies. |
| Can MLLMs Guide Me Home? A Benchmark Study on Fine-Grained Visual |
|
|
| Reasoning from Transit Maps (Read more on arXiv or HuggingFace) |
Lingdong Kong, Shuyi Ouyang, Song Wang, Huan-WhoRegisteredMyName, FSCCS |
i) REASONMAP, a new benchmark, is introduced for evaluating fine-grained visual understanding and spatial reasoning in MLLMs using transit maps. ii) The research aims to assess MLLMs’ proficiency in tasks requiring detailed visual interpretation, specifically spatial reasoning on transit maps. iii) The methodology involves a novel dataset with 1,008 question-answer pairs across 30 cities and a two-level evaluation framework measuring answer correctness and quality. iv) Evaluations of 15 MLLMs revealed that base models outperform reasoning variants among open-source models, while the opposite trend is observed in closed-source models; performance degrades when visual inputs are masked. v) AI practitioners should note the counterintuitive finding that reasoning-enhanced architectures do not consistently improve performance on fine-grained visual tasks, and that visual grounding remains crucial, even when models possess prior knowledge. |
| Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Yifei Zhao, Yifu Luo, Bo Xia, Jiaqi Wu, Haoyuan Sun |
Reinforcement fine-tuning (RFT) significantly enhances reasoning capabilities in multimodal large language models (MLLMs). The paper investigates how RFT improves MLLM reasoning across diverse modalities. The methodology involves summarizing the improvements of RFT in powering reasoning capabilities of MLLMs into five key points: diverse modalities, diverse tasks and domains, better training algorithms, abundant benchmarks and thriving engineering frameworks, categorizing RFT algorithms into Critic-Model-Driven and Critic-Model-Free. The survey summarizes recent works, categorized by release time and modality. This work implies future research should focus on generalizable reasoning, safety, data augmentation and better reward mechanisms for reasoning MLLMs. |
| Surrogate Signals from Format and Length: Reinforcement Learning for |
|
|
| Solving Mathematical Problems without Ground Truth Answers (Read more on arXiv or HuggingFace) |
Dianbo Sui, Yupeng Zhang, Zecheng Wang, Han Liu, Rihui Xin |
i) The paper introduces a reinforcement learning (RL) approach using format and length as surrogate rewards for mathematical problem-solving, eliminating reliance on ground truth answers. ii) The research investigates whether LLMs can be effectively trained for mathematical reasoning tasks using only format and length-based rewards, bypassing the need for ground truth labels. iii) The methodology employs Group Relative Policy Optimization (GRPO) with a reward function incorporating format correctness and response length, evaluated on mathematical datasets. iv) The results show that the proposed GRPO approach, using format-length surrogate signals, achieves 40.0% accuracy on AIME2024, surpassing standard GRPO performance relying on ground truth in certain scenarios. v) AI practitioners can leverage format and length rewards as effective substitutes for ground truth labels in mathematical problem-solving RL, reducing data collection costs and facilitating training in label-scarce environments. |
| Flex-Judge: Think Once, Judge Anywhere (Read more on arXiv or HuggingFace) |
Se-Young Yun, Sungwoo Cho, Jongwoo Ko, sungnyun |
FLEX-Judge is introduced as a modality-agnostic approach for training multimodal judge models. The paper investigates whether a small amount of text-only reasoning data can effectively train a cost-efficient, modality-agnostic judge model. The methodology involves training a multimodal judge model using a 1K-sized corpus of high-quality text reasoning data from JudgeLRM. FLEX-Judge (7B model) achieves competitive performance compared to commercial APIs and outperforms open-source judges, even exceeding Gemini and GPT-40 on several MJ-Bench and GenAI-Bench subtasks. The principal implication is that reasoning-based text supervision offers a cost-effective alternative to annotation-intensive approaches, advancing scalable multimodal model evaluation, applicable to modalities like molecule evaluation where comprehensive benchmarks are scarce. |
| Which Data Attributes Stimulate Math and Code Reasoning? An |
|
|
| Investigation via Influence Functions (Read more on arXiv or HuggingFace) |
Zhijie Deng, Zihao Zeng, Hanwen Xu, Qingyuan Tian, Siqi Kou |
i) The paper introduces Infra, an influence function-based approach, to attribute the reasoning capabilities of large language models (LLMs) in math and coding tasks to specific training data attributes. ii) The research investigates which attributes of training data most effectively stimulate LLMs’ reasoning capabilities in math and code. iii) Influence functions are leveraged to attribute LLMs’ reasoning performance to individual training examples, sequences, and tokens, with a focus on identifying positively influential data. iv) The study found that flipping task difficulty via dataset reweighting boosts AIME24 accuracy from 10% to 20% and improves LiveCodeBench accuracy from 33.8% to 35.3% for the Qwen2.5-7B-Instruct model, and that token-level influence patterns are distinct for math and code reasoning. v) AI practitioners can use the identified data attributes and the Infra framework to curate and optimize training datasets for reasoning-intensive tasks, improving the efficiency and effectiveness of LLM training. |
| Discrete Markov Bridge (Read more on arXiv or HuggingFace) |
Ying Nian Wu, Song-Chun Zhu, zlzheng, ColorfulAI, henry12348 |
i) The paper introduces Discrete Markov Bridge (DMB), a novel variational framework for discrete representation learning. ii) The main objective is to overcome the limitations of fixed-rate transition matrices in existing discrete diffusion models to achieve better latent representations. iii) The methodology involves a bidirectional two-stage learning algorithm with Matrix-learning and Score-learning components, using a parameterized, diagonalizable rate transition matrix. iv) Empirical evaluations on Text8 resulted in an Evidence Lower Bound (ELBO) of 1.38, outperforming baselines, and competitive results were shown on CIFAR-10. v) The DMB framework’s matrix learning process enhances expressiveness of latent representations, providing AI practitioners with a more efficient and adaptable approach for discrete data modeling compared to methods with fixed-rate transition matrices. |
| Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System |
|
|
| Collaboration (Read more on arXiv or HuggingFace) |
Zheng Huang, Zongze Du, Muzhi Zhu, Hao Zhong, Canyu |
i) Omni-R1 is presented as an end-to-end reinforcement learning framework for omnimodal reasoning that addresses the trade-off between temporal coverage and spatial resolution. ii) The main objective is to enable long-horizon video-audio reasoning and fine-grained pixel understanding in omnimodal models. iii) The methodology employs a two-system architecture (Global Reasoning System and Detail Understanding System) trained via reinforcement learning using Group Relative Policy Optimization (GRPO). iv) Experiments on Referring Audio-Visual Segmentation (RefAVS) and Reasoning Video Object Segmentation (REVOS) show Omni-R1 surpasses supervised baselines, improving out-of-domain generalization and mitigating multimodal hallucination with a +4.6% on J&F in seen set and +17.0% on unseen set in Ref-AVSBench. v) AI practitioners can utilize Omni-R1’s architecture to develop scalable omnimodal models capable of effective long-horizon reasoning and precise pixel-level grounding, addressing limitations in existing foundation models. |
| Done Is Better than Perfect: Unlocking Efficient Reasoning by Structured |
|
|
| Multi-Turn Decomposition (Read more on arXiv or HuggingFace) |
Zhijie Deng, Hao Zhang, Boxiu Li, Zihao Zeng, ElysiaTrue |
i) The paper introduces Multi-Turn Decomposition (MinD), a method to improve the efficiency of reasoning in Large Reasoning Models (LRMs) by structuring the reasoning process. ii) The research aims to reduce token usage and latency in LRMs while maintaining performance on complex reasoning tasks. iii) The methodology involves supervised fine-tuning (SFT) to transform conventional Chain-of-Thought (CoT) data into a multi-turn format, followed by reinforcement learning (RL) using Group Relative Policy Optimization (GRPO) to prioritize correct outputs with fewer reasoning turns. iv) Results show MinD achieves up to 70% reduction in output token usage and a 4.2x speedup in time to first token (TTFT) on MATH-500 with DeepSeek-R1-Distill-Qwen-1.5B, while maintaining over 95% accuracy. v) MinD offers AI practitioners a structured approach to reduce computational costs and latency in LRMs, potentially improving the user experience in applications requiring complex reasoning. |
| Hard Negative Contrastive Learning for Fine-Grained Geometric |
|
|
| Understanding in Large Multimodal Models (Read more on arXiv or HuggingFace) |
Ji Qi, Jiajie Zhang, Zhen Yang, Yushi Bai, Kai Sun |
i) This paper introduces a hard negative contrastive learning framework for improving geometric understanding in Large Multimodal Models (LMMs). ii) The research aims to enhance the vision encoder’s ability to recognize fine-grained geometric elements within images for geometric problem-solving. iii) The methodology involves image-based contrastive learning using diagrams generated via code perturbation and text-based contrastive learning using rule-based and retrieval-based negative captions; a novel MMCLIP training strategy is proposed to handle an arbitrary number of hard negatives. iv) The MMGeoLM model, trained with the proposed framework, surpasses existing open-source models on GeoQA and MathVISTA and achieves state-of-the-art performance on MM-MATH, exceeding GPT-4o by 7.5%. v) The principal implication for AI practitioners is a method to improve LMMs’ geometric reasoning through targeted hard negative contrastive learning, enabling more accurate visual perception in tasks requiring fine-grained geometric understanding; using exam-based, authentic image negatives showed better results than over 100K text negatives. |
| The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT |
|
|
| Distillation (Read more on arXiv or HuggingFace) |
Song Wang, Zhen Tan, Rana Muhammad Shahroz Khan, Ruichen Zhang, wjldw |
DC-CoT is introduced as a data-centric benchmark for chain-of-thought (CoT) distillation in large language models (LLMs). The research investigates how data manipulation techniques impact CoT distillation across method, model, and data perspectives. The methodology involves evaluating augmentation, selection, and mixing strategies with diverse teacher models (e.g., Gemini-Pro, Claude-3.5) and student architectures (3B, 7B parameters) on multiple reasoning datasets, focusing on in-distribution (IID), out-of-distribution (OOD) generalization, and cross-domain transfer. Results show data augmentation is generally the most effective approach, with reverse augmentation improving average accuracy by 24.64% on tested tasks using Llama-3.1-8B. The findings provide actionable insights for AI practitioners to optimize CoT distillation through data-centric techniques, thereby facilitating more efficient reasoning models. |
| Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV |
|
|
| Cache Compression (Read more on arXiv or HuggingFace) |
Jenq-Neng Hwang, Cheng-Yen Yang, Zigeng Chen, stargazerx0 |
ScaleKV introduces a novel KV cache compression framework tailored for visual autoregressive modeling to improve memory efficiency. The research aims to mitigate the exponential growth of KV cache in VAR models by employing scale-aware layer budget allocation. The methodology involves categorizing transformer layers into drafters and refiners based on their attention patterns using an Attention Selectivity Index. Evaluations on the Infinity-8B model show a reduction in KV cache memory from 85 GB to 8.5 GB while maintaining a GenEval score of 0.79. This cache compression framework enables AI practitioners to deploy visual autoregressive models in resource-constrained environments while preserving pixel-level fidelity. |
| Learning to Reason without External Rewards (Read more on arXiv or HuggingFace) |
Dawn Song, Sergey Levine, Aosong Feng, Zhewei Kang, Xuandong |
i) The paper introduces Reinforcement Learning from Internal Feedback (RLIF), specifically INTUITOR, to train LLMs using self-certainty as the sole reward signal. ii) The research investigates whether LLMs can enhance reasoning abilities using intrinsic, self-generated signals, without relying on external supervision. iii) INTUITOR replaces external rewards in Group Relative Policy Optimization (GRPO) with self-certainty scores derived from the KL divergence between the model’s output distribution and a uniform distribution. iv) Experiments with Qwen2.5-3B base model trained on MATH dataset show that INTUITOR matches GRPO’s performance on mathematical benchmarks and achieves a 65% relative improvement on the LiveCodeBench code generation task. v) RLIF using INTUITOR offers AI practitioners a scalable alternative to RLVR for training autonomous AI systems in domains where verifiable rewards are unavailable by demonstrating effective learning from intrinsic model signals across domains. |
| From Tens of Hours to Tens of Thousands: Scaling Back-Translation for |
|
|
| Speech Recognition (Read more on arXiv or HuggingFace) |
Shanbo Cheng, Wei Lu, Lu Xu, Tianduo Wang |
Speech Back-Translation is introduced as a scalable data augmentation pipeline for multilingual ASR. The research investigates improving multilingual ASR models by training text-to-speech (TTS) models on limited transcribed speech to generate large synthetic speech datasets. The methodology involves fine-tuning multilingual TTS models and using these to back-translate large text corpora into synthetic speech, along with a novel intelligibility metric for quality control. Experiments pre-training Whisper-large-v3 with 500,000 hours of synthetic speech achieved an average 30% reduction in transcription error rates across ten languages. This demonstrates that TTS models trained on limited real speech can generate significantly larger, high-quality synthetic speech datasets for improving multilingual ASR. |
| AdaCtrl: Towards Adaptive and Controllable Reasoning via |
|
|
| Difficulty-Aware Budgeting (Read more on arXiv or HuggingFace) |
Jiazhan Feng, Zhaochen Su, Wanjun Zhong, Hongru Wang, JoeYing |
AdaCtrl adaptively adjusts and controls reasoning depth in language models based on problem difficulty. The research aims to develop a framework supporting difficulty-aware adaptive reasoning budget allocation and explicit user control over reasoning depth. AdaCtrl employs a two-stage training pipeline: cold-start fine-tuning for initial difficulty awareness and difficulty-aware reinforcement learning to refine reasoning strategies. Experiments show AdaCtrl improves performance and reduces response length by 10.06% on AIME2024 and 12.14% on AIME2025, while achieving 62.05% and 91.04% reductions on MATH500 and GSM8K datasets, respectively. The ability to adaptively allocate resources based on difficulty provides AI practitioners with a method to improve both the efficiency and effectiveness of reasoning in language models. |
| G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language |
|
|
| Model via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Flood Sung, Zhiqi Huang, Tianyu Liu, Hongcheng Gao, Liang Chen |
i) The paper introduces VLM-Gym, a reinforcement learning environment for training vision-language models (VLMs) on diverse visual games, and G1 models which demonstrate improved performance through RL-driven self-evolution. ii) The research aims to improve VLM decision-making capabilities in interactive, visually rich environments by addressing the “knowing-doing” gap. iii) The methodology involves a pure RL-driven self-evolution approach and incorporates a perception-enhanced cold start prior to RL fine-tuning, utilizing the GRPO algorithm. iv) The resulting G1 models outperformed their teacher models and leading proprietary models like Claude-3.7-Sonnet-Thinking, achieving, for example, a score of 17.5 on the Shisen-Sho game compared to Claude-3.7-Sonnet-Thinking’s 15.3. v) The paper suggests that perception and reasoning abilities in VLMs can mutually bootstrap each other through RL, which can inform the design of VLM training strategies for improved interactive agents, offering a novel approach to improve decision-making in visually rich environments. |
| The Coverage Principle: A Framework for Understanding Compositional |
|
|
| Generalization (Read more on arXiv or HuggingFace) |
Miyoung Ko, Sohee Yang, Hanseul Cho, Jinho Park, Hoyeon Chang |
i) This paper introduces the coverage principle, a data-centric framework for understanding compositional generalization in large language models. ii) The main objective is to explain how Transformers generalize compositionally and to predict their generalization boundaries based on observed functional equivalences. iii) The methodology involves deriving predictions from the coverage principle, conducting experiments on synthetic compositional tasks using GPT-2 models, and analyzing latent representations through IICG and causal tracing. iv) Results demonstrate that reliable generalization on two-hop reasoning tasks requires training data scaling at least quadratically with token set size, while data efficiency does not improve with 20x parameter scaling. v) AI practitioners should consider the coverage principle when designing training datasets and architectures for compositional tasks, as data curation informed by coverage considerations may be more effective than parameter scaling, particularly for tasks with path ambiguities. |
| Accelerating Nash Learning from Human Feedback via Mirror Prox (Read more on arXiv or HuggingFace) |
Denis Belomestny, Daniele Calandriello, misovalko, kashif, dtiapkin |
i) This paper introduces Nash Mirror Prox (NashMP), an online algorithm for Nash Learning from Human Feedback (NLHF). ii) The research aims to develop an NLHF algorithm with faster and more stable convergence to the Nash equilibrium compared to existing methods. iii) NashMP utilizes the Mirror Prox optimization scheme, involving two mirror descent steps and policy gradient estimation for parametrized policies. iv) The paper establishes last-iterate linear convergence for NashMP, demonstrating that the KL-divergence to the optimal policy decreases at a rate of order (1 + 2β)-N/2, where N is the number of preference queries, and presents empirical results showing competitive performance on synthetic and LLM fine-tuning tasks. v) The linear convergence rate and independence from the action space size makes NashMP suitable for LLM alignment, offering an efficient alternative to reward modeling with potential for enhancing AI systems’ ability to align with human values. |
| LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
zhenxuan00, jrwen, lyk423, surfingtomchen, xiaolu0714 |
i) This paper introduces Variance-Reduced Preference Optimization (VRPO) to improve the alignment of Masked Diffusion Models (MDMs) with human preferences. ii) The research aims to reduce the high variance in ELBO-based likelihood estimates for preference optimization in MDMs. iii) The methodology involves theoretical analysis of ELBO estimator variance and the derivation of unbiased variance reduction strategies like optimal Monte Carlo budget allocation and antithetic sampling. iv) The resulting model, LLaDA 1.5, demonstrates a +4.7 improvement on GSM8K and shows competitive mathematical performance compared to strong language MDMs and ARMs. v) VRPO offers AI practitioners improved alignment techniques for language MDMs by addressing ELBO estimation variance within a fixed computational budget. |
| ModernGBERT: German-only 1B Encoder Model Trained from Scratch (Read more on arXiv or HuggingFace) |
Andreas Hotho, Fotis Jannidis, Julia Wunderle, Anton Ehrmanntraut, JanPf |
i) ModernGBERT is a family of German encoder models trained from scratch, incorporating ModernBERT architectural innovations. ii) The paper investigates the trade-offs of training German encoders from scratch versus converting decoder models, and the impact of architectural innovations on encoder performance. iii) The methodology involves training ModernGBERT (134M, 1B) and LLäMmlein2Vec (120M, 1B, 7B) using the RedPajamaV2 dataset, evaluating on SuperGLEBer, MTEB, and QA-NIAH benchmarks. iv) ModernGBERT 1B achieves a SuperGLEBer score of 0.808, outperforming previous state-of-the-art German encoders such as GBERTLarge (0.768) and LLäMmlein2Vec 7B (0.787). v) AI practitioners can leverage ModernGBERT for high-performance, parameter-efficient German language understanding tasks, as dedicated encoders outperform converted decoder models of similar size. |
| Interleaved Reasoning for Large Language Models via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Yanchao Sun, Dong Lin, Deepak Gopinath, David Qiu, Roy Xie |
i) This paper introduces interleaved reasoning, a novel reinforcement learning paradigm for LLMs. ii) The research aims to improve LLM reasoning capabilities, reduce time-to-first-token (TTFT), and enhance training efficiency for multi-hop question answering. iii) The methodology employs reinforcement learning with a rule-based reward to incentivize correct intermediate steps during reasoning. iv) Experiments across five datasets demonstrate a 19.3% improvement in Pass@1 accuracy and an 80% reduction in TTFT. v) The interleaved reasoning paradigm, leveraging rule-based rewards for intermediate steps, provides AI practitioners with a method for significantly accelerating LLM responsiveness and improving reasoning performance without external tools. |
| WINA: Weight Informed Neuron Activation for Accelerating Large Language |
|
|
| Model Inference (Read more on arXiv or HuggingFace) |
Colby Banbury, Jongwoo Ko, Dan Zhao, Sihan Chen, tianyic |
WINA: Weight Informed Neuron Activation accelerates LLM inference via weight-aware neuron selection. The research aims to improve inference efficiency in large language models by developing a training-free sparse activation method. The paper introduces WINA, a novel training-free sparse activation framework leveraging both hidden state magnitudes and column-wise l2-norms of weight matrices for neuron selection. Experiments show WINA outperforms TEAL by up to 2.94% in average performance across various LLM architectures and datasets at the same sparsity levels. WINA offers AI practitioners an efficient training-free method for sparse activation in LLMs that potentially improves inference performance compared to existing techniques such as TEAL. |
| Position: Mechanistic Interpretability Should Prioritize Feature |
|
|
| Consistency in SAEs (Read more on arXiv or HuggingFace) |
Zeyu Tang, Lingjing Kong, Yujia Zheng, aashiqmuhamed, xiangchensong |
i) This paper advocates for prioritizing feature consistency in Sparse Autoencoders (SAEs) used for mechanistic interpretability (MI) to improve reliability and reproducibility. ii) The research question focuses on whether mechanistic interpretability should prioritize feature consistency in SAEs, and presents strategies for achieving this goal. iii) The methodology involves proposing Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) to measure consistency, theoretical grounding via sparse dictionary learning, synthetic validation with a model organism, and real-world LLM data experimentation. iv) The primary results demonstrate high consistency levels can be achieved with appropriate architectural choices, such as using TopK SAEs, achieving PW-MCC ≈ 0.80 for TopK SAEs on LLM activations, and approximately 0.97 in model organism, furthermore, high feature consistency strongly correlates with the semantic similarity of learned feature explanations. v) The principal implication for AI practitioners is the need to systematically measure and prioritize feature consistency in SAEs to foster robust cumulative progress in MI, particularly for applications like model steering, unlearning, and safety verification. |
| MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (Read more on arXiv or HuggingFace) |
Ailin Deng, Wei Han, Yujie Lu, happymio, chchenhui |
This paper introduces MLR-Bench, a benchmark for evaluating AI agents on open-ended machine learning research. The research question addresses how to rigorously evaluate the quality of research produced by AI agents. The methodology involves constructing a benchmark consisting of 201 real-world research tasks, MLR-Judge for automated evaluation, and MLR-Agent for research execution. Experiments showed that current coding agents often produce fabricated experimental results in 80% of cases. MLR-Bench helps the community benchmark, diagnose, and improve AI research agents toward trustworthy and transparent scientific discovery. |
| Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications |
|
|
| of Agentic AI (Read more on arXiv or HuggingFace) |
Manoj Karkee, Konstantinos I. Roumeliotis, RanjanSapkota |
i) This paper analyzes and differentiates two AI-assisted software development paradigms: vibe coding and agentic coding. ii) The main objective is to establish a detailed taxonomy differentiating vibe coding and agentic coding based on their conceptual foundations, execution models, feedback loops, safety mechanisms, debugging strategies, and real-world tool ecosystems. iii) The research employs comparative workflow analysis across 20 detailed use cases and architectural analysis through layered diagrams and pseudocode abstractions. iv) The results indicate vibe coding excels in early-stage prototyping and education, while agentic coding is better suited for enterprise automation, codebase refactoring, and CI/CD integration; as an example, the Jules system was able to clone a GitHub repository, parse the README.md, identify relevant integration points, generate new data classes, inject code, update documentation, and create a commit to Git. v) The principal implication for AI practitioners is understanding the trade-offs between these paradigms for harmonized and human-centered AI software development, suggesting hybrid architectures that couple natural language interfaces with autonomous execution pipelines. |
| Rethinking the Sampling Criteria in Reinforcement Learning for LLM |
|
|
| Reasoning: A Competence-Difficulty Alignment Perspective (Read more on arXiv or HuggingFace) |
Jingang Wang, Wei Wang, Qi Guo, xixy, DeyangKong |
i) This paper introduces Competence-Difficulty Alignment Sampling (CDAS), a novel sampling strategy for reinforcement learning (RL) to improve the reasoning abilities of large language models (LLMs). ii) The research aims to address the inefficiency in RL training caused by unstable problem difficulty estimations and the misalignment between model competence and problem difficulty. iii) CDAS estimates problem difficulty by aggregating historical performance discrepancies and quantifies model competence to adaptively select problems aligned with the model’s current capabilities using a fixed-point system. iv) Experiments on mathematical benchmarks demonstrate that CDAS achieves a higher average accuracy of 46.77% and is 2.33 times faster than Dynamic Sampling. v) CDAS provides AI practitioners with an efficient sampling strategy for scaling RL training in LLM reasoning tasks, enabling better allocation of computational resources by dynamically matching problem difficulty to model competence. |
| StructEval: Benchmarking LLMs’ Capabilities to Generate Structural |
|
|
| Outputs (Read more on arXiv or HuggingFace) |
Yuxuan Zhang, Sherman Siu, Lipeng He, Dongfu Jiang, Jialin Yang |
i) StructEval is introduced as a benchmark to evaluate LLMs’ capabilities in generating structured outputs. ii) The research aims to comprehensively assess LLMs’ ability to produce both renderable and non-renderable structured formats. iii) The methodology involves generation and conversion tasks across 18 formats, evaluated using format adherence and structural correctness metrics. iv) Results indicate a performance gap, with even 01-mini only achieving 75.58 average score, and generation tasks proving more challenging than conversion tasks. v) AI practitioners should consider StructEval to evaluate and improve LLMs’ ability to produce structured outputs accurately, especially for tasks requiring visual content and those in software development or data pipeline applications. |
| InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer |
|
|
| Interaction (Read more on arXiv or HuggingFace) |
Xi Xie, Winson Chen, Zijian Zhang, Weitai Kang, Bin12345 |
InfantAgent-Next is a multimodal generalist agent for automated computer interaction utilizing text, images, audio, and video. The paper investigates the development of a generalist agent that integrates tool-based and pure vision agents within a modular architecture to solve complex computer interaction tasks. The methodology involves integrating large language models and visual large language models within a framework that modularizes workflow, tool selection, and tool execution. The results show the agent achieves 7.27% accuracy on the OSWorld benchmark. This research implies that AI practitioners can leverage the proposed architecture to build more versatile and accurate agents for automating computer tasks by combining tool-based and vision-based approaches. |
| GLEAM: Learning Generalizable Exploration Policy for Active Mapping in |
|
|
| Complex 3D Indoor Scenes (Read more on arXiv or HuggingFace) |
Jiangmiao Pang, Tao Huang, Quanyi Li, Tai Wang, Xiao-HF |
i) GLEAM introduces a generalizable exploration policy for active mapping in complex 3D indoor scenes via a large-scale benchmark. ii) The primary objective is to develop a reinforcement learning (RL)-based exploration policy that can generalize across diverse and complex 3D indoor scenes without fine-tuning. iii) The methodology involves training a unified exploration policy using semantic representations, long-term navigable goals, and randomized training strategies within a benchmark (GLEAM-Bench) of 1,152 diverse 3D scenes. iv) The primary result is a 66.50% average coverage ratio across 128 unseen scenes, outperforming state-of-the-art methods by 9.49%, with a 0.80m nearest distance to ground-truth. v) GLEAM demonstrates that improved generalizability in active mapping can be achieved by training on large, diverse datasets with semantic representations, enabling the deployment of robust exploration policies in complex environments, but there is a sim-to-real gap that still needs to be explored. |
| Error Typing for Smarter Rewards: Improving Process Reward Models with |
|
|
| Error-Aware Hierarchical Supervision (Read more on arXiv or HuggingFace) |
Soujanya Poria, Chuan Li, Amir Zadeh, Panshul Sharma, Tej Deep Pala |
i) The paper introduces PathFinder-PRM, a hierarchical, error-aware process reward model (PRM) that classifies math and consistency errors to improve reward estimation for mathematical reasoning. ii) The primary research objective is to enhance PRM performance by decoupling error detection and reward estimation through hierarchical supervision. iii) The methodology involves training a discriminative PRM with a newly constructed 400K-sample dataset annotated with three-dimensional step-level labels indicating math errors, consistency errors, and step correctness. iv) Results show PathFinder-PRM achieves a new state-of-the-art PRMScore of 67.7 on PRMBench, outperforming the prior best (65.5) while using 3x less data. v) The principal implication is that decoupling error detection and reward estimation with hierarchical error-aware supervision can substantially improve end-to-end reward-guided mathematical reasoning, offering AI practitioners a more data-efficient approach to building effective PRMs. |
| DoctorAgent-RL: A Multi-Agent Collaborative Reinforcement Learning |
|
|
| System for Multi-Turn Clinical Dialogue (Read more on arXiv or HuggingFace) |
Yixue Li, Lu Zhou, Yichun Feng, Jarvis1111 |
i) DoctorAgent-RL, a reinforcement learning (RL) based multi-agent system, is introduced for multi-turn clinical dialogue. ii) The paper aims to develop a system capable of dynamically optimizing questioning strategies for clinical diagnosis through multi-turn interactions. iii) The methodology involves a doctor agent refined by RL, a patient agent based on LLMs, and a consultation evaluator providing multi-dimensional rewards and constructing a new dataset called MTMedDialog. iv) Experiments show DoctorAgent-RL outperforms existing models, achieving a 53.9% average score, in both multi-turn reasoning capability and final diagnostic performance on the MTMedDialog dataset. v) The work provides AI practitioners with an RL-based framework for improving diagnostic accuracy through optimized, interactive questioning in clinical consultation systems and a new dataset for training and evaluating such systems. |
| Jodi: Unification of Visual Generation and Understanding via Joint |
|
|
| Modeling (Read more on arXiv or HuggingFace) |
Xilin Chen, Shiguang Shan, Meina Kan, Zhenliang He, xyfJASON |
Jodi is a diffusion framework that unifies visual generation and understanding by jointly modeling the image and multiple label domains. The research aims to develop a single model capable of joint generation, controllable generation, and image perception. The methodology involves a linear diffusion transformer with a role switch mechanism and domain-invariant positional embeddings. Experimental results demonstrate Jodi’s superiority in generation and understanding tasks, achieving an FID score of 13.6 for controllable generation of depth maps, outperforming existing unified models. Jodi offers AI practitioners a unified model for diverse visual tasks, potentially streamlining development and reducing resource requirements compared to task-specific models. |
| An Embarrassingly Simple Defense Against LLM Abliteration Attacks (Read more on arXiv or HuggingFace) |
George Turkiyyah, Bernard Ghanem, Hasan Abed Al Kader Hammoud, Harethah Abu Shairah |
i) This paper introduces extended-refusal fine-tuning as a defense against abliteration attacks on LLMs. ii) The research aims to improve the robustness of LLMs against abliteration attacks, which neutralize safety guardrails by removing a single direction in the model. iii) The methodology involves creating an extended-refusal dataset with detailed responses justifying refusals and fine-tuning LLAMA-2-7B-CHAT and QWEN2.5-INSTRUCT models on this dataset. iv) Experiments show that extended-refusal models maintain high refusal rates after abliteration, dropping at most by 10%, while baseline models drop by 70-80%. v) AI practitioners can leverage extended-refusal fine-tuning to enhance the safety alignment of LLMs by distributing safety signals across multiple latent dimensions, mitigating the risk of targeted attacks. |
| Strong Membership Inference Attacks on Massive Datasets and (Moderately) |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Matthew Jagielski, Christopher A. Choquette-Choo, Ilia Shumailov, Jamie Hayes, pasta41 |
i) This paper evaluates the efficacy of strong membership inference attacks (MIAs) on large language models (LLMs). ii) The primary research question is whether the limitations of previous MIA research are due to attack design choices or if MIAs are fundamentally ineffective on LLMs. iii) The methodology involves scaling the LiRA attack to GPT-2 architectures (10M to 1B parameters), training reference models on over 20B tokens from the C4 dataset. iv) Results indicate that strong MIAs can succeed on pre-trained LLMs, but their effectiveness remains limited (e.g., AUC < 0.7) in practical settings, and that there is a non-monotonic relationship between model size and MIA vulnerability. v) The principal implication for AI practitioners is that despite the success of strong MIAs on LLMs, their limited effectiveness under practical conditions suggests that privacy risks may be lower than previously assumed, but further research into improving attack effectiveness is warranted. |
| Dynamic Risk Assessments for Offensive Cybersecurity Agents (Read more on arXiv or HuggingFace) |
Zhou Li, Joie Zhang, Jiacen Xu, Benedikt Stroebl, boyiwei |
i) This paper investigates the dynamic cybersecurity risks posed by autonomous offensive agents improved through iterative self-improvement with bounded compute. ii) The primary objective is to assess how adversaries can leverage various degrees of freedom to enhance agents’ cybersecurity capabilities within a fixed compute budget. iii) The methodology involves dynamically analyzing agents’ performance on InterCode CTF challenges by allowing adversaries to modify agent systems through repeated sampling, prompt refinement, self-training, and workflow refinement, and then measuring the performance. iv) Results show that adversaries can improve an agent’s cybersecurity capability on InterCode CTF by more than 40% relative to the baseline with an 8 H100 GPU Hour compute budget without external assistance, and Iterative prompt refinement exhibits the highest risk potential. v) AI practitioners should consider the dynamic nature of cybersecurity risks, where adversaries can iteratively improve offensive agents even with limited resources, thus highlighting the limitations of current static risk assessment methods. |
| EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via |
|
|
| Action Pruning (Read more on arXiv or HuggingFace) |
Defu Lian, Quan Liu, Jianshu Zhang, Qisi Chen, Jiawei1222 |
i) The paper introduces EquivPruner, an action pruning approach to improve the efficiency and quality of LLM-based search. ii) The primary objective is to reduce token consumption and improve accuracy in LLM reasoning search by identifying and pruning semantically equivalent actions. iii) The methodology involves training a lightweight equivalence detector, utilizing the newly created MathEquiv dataset for mathematical statement equivalence, integrated into an MCTS framework. iv) Experiments demonstrate that EquivPruner reduced token consumption by 48.1% on GSM8K with Qwen2.5-Math-7B-Instruct while also improving accuracy from 96.44% to 96.59%. v) EquivPruner offers a practical method for AI practitioners to optimize LLM reasoning processes by efficiently managing redundant computations, potentially enabling more complex problem-solving within resource constraints. |
| MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Maged S. Al-Shaibani, Zaid |
i) This paper introduces MOLE, a framework leveraging LLMs for automated metadata extraction and validation from scientific papers covering datasets of languages other than Arabic. ii) The main objective is to automate the extraction of over 30 dataset metadata attributes from scientific papers, a task currently limited by reliance on manual annotation. iii) The methodology utilizes a schema-driven approach to process entire documents in LaTeX or PDF format, employing LLMs and validation mechanisms for consistent JSON output. iv) Experiments across multiple LLMs show promising results, with Gemini 2.5 Pro achieving the highest average metadata extraction score of 67.42% and Gemma 3 27B gets +1.62 % increase in accuracy when employing web browsing for certain metadata attributes. v) The principal implication for AI practitioners is the potential for automating dataset cataloging and preservation, enabling more efficient research discovery and reproducibility, although the results highlight the need for further improvements to ensure consistent and reliable performance. |
| Architectural Backdoors for Within-Batch Data Stealing and Model |
|
|
| Inference Manipulation (Read more on arXiv or HuggingFace) |
Ilia Shumailov, Conrad Grobler, Ivan Petrov, Nicolas Küchler |
i) This paper introduces architectural backdoors that exploit batched inference to facilitate data theft and model manipulation in neural networks. ii) The research investigates how architectural backdoors can compromise batch isolation, enabling attackers to steal or manipulate data from other users within the same batch. iii) The methodology involves injecting backdoors into model architectures, specifically targeting the batching mechanism and using static taint analysis with a novel Information Flow Control mechanism to verify and mitigate vulnerabilities. iv) The paper finds over 200 models on Hugging Face exhibit unintended information leakage due to dynamic quantization and demonstrates successful data theft and manipulation via engineered backdoors. v) AI practitioners should be aware of the potential for architectural backdoors in batched inference systems and employ deterministic mitigation strategies such as the proposed Batch Isolation Checker to ensure user privacy and system integrity. |
| Towards Holistic Evaluation of Large Audio-Language Models: A |
|
|
| Comprehensive Survey (Read more on arXiv or HuggingFace) |
Hung-yi Lee, Neo S. Ho, zenyn |
i) This paper surveys and categorizes evaluation frameworks for large audio-language models (LALMs). ii) The main objective is to provide a systematic taxonomy for LALM evaluations across diverse objectives. iii) The methodology involves a comprehensive review and categorization of existing LALM benchmarks into four dimensions: General Auditory Awareness and Processing, Knowledge and Reasoning, Dialogue-oriented Ability, and Fairness, Safety, and Trustworthiness. iv) The survey identifies current LALMs often accept malicious spoken inputs even when they can refuse similar textual ones, highlighting vulnerabilities. v) AI practitioners can use this taxonomy to select appropriate benchmarks for evaluating specific capabilities of LALMs and identify areas needing improvement, particularly in safety alignment. |
| Option-aware Temporally Abstracted Value for Offline Goal-Conditioned |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Taesup Moon, Heewoong Choi, Hongjoon Ahn, JisuHann |
Offline goal-conditioned reinforcement learning with temporal abstraction (OTA) is presented for improved value function learning. The research addresses the challenge of high-level policy learning in long-horizon tasks by reducing temporal distance between states and goals. The key method is option-aware temporal differencing which updates value function over option sequences, not individual steps. Experiments on the OGBench benchmark demonstrate that policies using the OTA value function achieve strong performance compared to baselines, as shown with the HumanoidMaze-giant task where OTA achieves a 79% success rate. The principal implication of the research is to mitigate the errors of the value function in long-horizon regimes for better decision-making in complex robotic manipulation tasks. |
| TAGS: A Test-Time Generalist-Specialist Framework with |
|
|
| Retrieval-Augmented Reasoning and Verification (Read more on arXiv or HuggingFace) |
Haochen Xue, Ming Hu, Yulong Li, Feilong Tang, JianghaoWu |
i) The paper introduces TAGS, a test-time framework for enhancing medical question answering using a generalist-specialist approach with retrieval augmentation and verification. ii) The research aims to improve LLM performance in MedQA by combining general and domain-specific knowledge without parameter updates. iii) The methodology involves a Generalist-Specialist Reasoning Collaboration (GSRC) module, Hierarchical Retrieval Augmentation (HRA) for multi-scale exemplar selection, and Uncertainty-Aware Answer Aggregation (UAAA) for reasoning consistency evaluation. iv) TAGS achieves a 13.8% improvement on GPT-4o and a 16.8% gain on DeepSeek-R1 accuracy across nine MedQA benchmarks without any parameter updates. v) AI practitioners can leverage the TAGS framework to enhance the reliability and accuracy of LLMs in specialized domains like medicine by combining general and specific knowledge sources and employing verification mechanisms without requiring model retraining or fine-tuning. |
Papers for 2025-05-26
| Title |
Authors |
Summary |
| TabSTAR: A Foundation Tabular Model With Semantically Target-Aware |
|
|
| Representations (Read more on arXiv or HuggingFace) |
Roi Reichart, Alan Arazi, EilamSha |
i) TabSTAR introduces a foundation tabular model with semantically target-aware representations for transfer learning on tabular data with textual features. ii) The research aims to enhance tabular learning performance, particularly in datasets with free-text, by incorporating target-specific semantic information. iii) TabSTAR utilizes a pre-trained text encoder, unfrozen and optimized with target tokens, to learn task-specific embeddings, and pretraining exhibits scaling laws. iv) TabSTAR achieves state-of-the-art performance on medium- and large-sized datasets, with a normalized score of 0.874 on classification benchmarks. v) The framework enables AI practitioners to utilize pre-trained models for tabular data, particularly with text, and efficiently adapt them for new tasks with low-resource settings by enabling transfer learning on tabular data with textual features. |
| QwenLong-L1: Towards Long-Context Large Reasoning Models with |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Chenliang Li, Yingcheng Shi, Shengyi Liao, Weizhou Shen, Wanfq |
QwenLong-L1 introduces a reinforcement learning framework for adapting large reasoning models to long-context scenarios. The research focuses on effectively extending LRM capabilities for processing long-context inputs via reinforcement learning. A progressive context scaling approach, with supervised fine-tuning and curriculum-guided phased RL, is employed to address training instability and inefficiency. Experiments on seven long-context document question-answering benchmarks show that QwenLong-L1-32B achieves an average gain of 5.1 points over the baseline R1-Distill-Qwen-32B. This work provides AI practitioners with a methodology for developing robust LRMs capable of reasoning across information-intensive environments. |
| Reasoning Model is Stubborn: Diagnosing Instruction Overriding in |
|
|
| Reasoning Models (Read more on arXiv or HuggingFace) |
Eunho Yang, Hyun Ryu, Chanjae Park, yjyjyj98, jadohu |
i) The paper introduces ReasoningTrap, a diagnostic dataset for assessing instruction overriding, termed “reasoning rigidity,” in large language models. ii) The main objective is to investigate and categorize the systematic failure of reasoning models to adhere to explicit instructions, instead defaulting to ingrained reasoning patterns. iii) The methodology involves creating modified versions of existing mathematical benchmarks (AIME, MATH500) and logic puzzles to require deviation from familiar reasoning strategies and subsequently categorizing contamination patterns. iv) The primary result identifies three distinct contamination modes: Interpretation Overload, Input Distrust, and Partial Instruction Attention and reports p-pass@1 scores for various models on the created datasets. v) The principal implication for AI practitioners is the identification and public release of a diagnostic set that will facilitate future research into mitigating reasoning rigidity in language models to improve the faithful execution of user instructions. |
| One RL to See Them All: Visual Triple Unified Reinforcement Learning (Read more on arXiv or HuggingFace) |
Pengfei Li, Shaoxiang Chen, Linge Du, Yan Ma, Ryan1122 |
i) The paper introduces V-Triune, a unified reinforcement learning system for vision-language models. ii) The research aims to enable VLMs to jointly learn visual reasoning and perception tasks within a single RL training pipeline. iii) V-Triune employs Sample-Level Data Formatting, Verifier-Level Reward Computation using a Dynamic IoU reward, and Source-Level Metric Monitoring. iv) The resulting model, Orsta, achieves gains on MEGA-Bench Core, with improvements ranging from +2.1% to +14.1% across 7B and 32B model variants. v) This unified RL approach for VLMs allows AI practitioners to train models for both visual reasoning and perception tasks, improving effectiveness and scalability with a single training paradigm. |
| Quartet: Native FP4 Training Can Be Optimal for Large Language Models (Read more on arXiv or HuggingFace) |
Jiale Chen, Oliver Sieberling, Soroush Tabesh, Andrei Panferov, Roberto L. Castro |
i) This paper introduces Quartet, an end-to-end FP4 training method for LLMs, achieving state-of-the-art accuracy through hardware-supported low-precision computation. ii) The main research objective is to enable accurate, fully FP4-based training of large language models by addressing accuracy degradation challenges. iii) The study employs a novel low-precision scaling law, customized CUDA kernels for NVIDIA Blackwell GPUs, and extensive evaluations on Llama-type models. iv) Quartet achieves almost 2x speedup relative to FP8 for linear layer computations on a Blackwell RTX 5090 GPU and state-of-the-art accuracy for FP4 precision on billion-scale models. v) AI practitioners can leverage Quartet to significantly improve the efficiency of LLM training, offering a competitive alternative to standard-precision and FP8 training via fully FP4-based methods. |
| Distilling LLM Agent into Small Models with Retrieval and Code Tools (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Jaewoong Cho, Seanie Lee, Jongwon Jeong, Minki Kang |
i) This paper introduces Agent Distillation, a framework to transfer task-solving behavior from large language model (LLM) agents to smaller language models (sLMs) using retrieval and code tools. ii) The research aims to enable sLMs to perform complex reasoning tasks by distilling the reasoning and tool-use capabilities of LLM-based agents. iii) The methodology includes a first-thought prefix to improve teacher-generated trajectories and self-consistent action generation to enhance test-time robustness of sLMs. iv) Experiments on factual and mathematical reasoning tasks demonstrate that sLMs (0.5B, 1.5B, 3B parameters) achieve performance competitive with larger models (1.5B, 3B, 7B) fine-tuned with chain-of-thought distillation. v) Agent Distillation provides AI practitioners with a method to create practical, tool-using small agents by enabling code execution and information retrieval in sLMs, improving their performance in complex reasoning tasks. |
| PhyX: Does Your Model Have the “Wits” for Physical Reasoning? (Read more on arXiv or HuggingFace) |
Yunta Hsieh, Qi Han, Hui Shen, John-ai-bee, taki555 |
i) The paper introduces PHYX, a new benchmark for evaluating physical reasoning in AI models using visual scenarios. ii) The research aims to assess the integrated ability of AI models to combine domain knowledge, symbolic reasoning, and understanding of real-world constraints in physical problem-solving. iii) The methodology involves a dataset of 3K multimodal questions spanning 6 reasoning types across 25 sub-domains within 6 core physics domains, evaluated using a strict three-step evaluation protocol. iv) The evaluation showed that even state-of-the-art models like GPT-40, Claude3.7-Sonnet, and GPT-o4-mini achieve only 32.5%, 42.2%, and 45.8% accuracy, respectively, significantly lower than human experts. v) PHYX highlights the need for AI practitioners to develop models with improved integration of disciplinary knowledge, symbolic operations, and real-world constraints for high-level physical reasoning. |
| QwenLong-CPRS: Towards infty-LLMs with Dynamic Context Optimization (Read more on arXiv or HuggingFace) |
Shaopeng Lai, Shengyi Liao, Chenliang Li, Weizhou Shen, Wanfq |
QWENLONG-CPRS introduces a context compression framework for long-context large language models (LLMs). The research addresses computational overhead during prefill stages and the “lost in the middle” phenomenon in LLMs processing long sequences. A dynamic context optimization mechanism is used, enabling multi-granularity context compression guided by natural language instructions. Evaluations demonstrate a 21.59x context compression rate alongside 19.15-point average performance gains across various LLMs and benchmarks, with the system deployed with Qwen2.5-32B-Instruct surpassing leading proprietary LLMs. AI practitioners can leverage QWENLONG-CPRS for improved efficiency and performance in LLMs handling extended context inputs, potentially establishing new state-of-the-art performance. |
| VeriThinker: Learning to Verify Makes Reasoning Model Efficient (Read more on arXiv or HuggingFace) |
Xinchao Wang, Ruonan Yu, Gongfan Fang, Xinyin Ma, Zigeng |
i) This paper introduces VeriThinker, a novel approach to improve the efficiency of large reasoning models by fine-tuning them on a CoT verification task. ii) The research question addresses whether LRMs can be effectively compressed by learning to verify reasoning steps rather than relying on synthetic target reasoning chains. iii) The methodology involves Supervised Verification Fine-Tuning (SVFT), where an LRM is trained to classify CoT solutions as correct or incorrect using an auxiliary verification dataset. iv) Results show that VeriThinker reduces reasoning tokens on MATH500 from 3790 to 2125 while improving accuracy by 0.8% (94.0% to 94.8%) and also generalizes to solution-wise speculative reasoning. v) AI practitioners can leverage VeriThinker’s SVFT for compressing and accelerating LRMs, improving their deployment efficiency without sacrificing reasoning accuracy, and possibly improve the accuracy. |
| Model Already Knows the Best Noise: Bayesian Active Noise Selection via |
|
|
| Attention in Video Diffusion Model (Read more on arXiv or HuggingFace) |
Sanghyun Kim, Kwanyoung Kim |
i) The paper introduces ANSE, a model-aware framework for selecting high-quality initial noise seeds in video diffusion models. ii) The research aims to improve video quality and temporal coherence in text-to-video generation by leveraging internal model signals. iii) A Bayesian Active Noise Selection via Attention (BANSA) score, quantifying attention-based uncertainty across stochastic attention samples, is used to evaluate noise seeds. iv) Experiments on CogVideoX-2B show ANSE improves VBench total score from 81.03 to 81.66 with only an 8% increase in inference time. v) AI practitioners can use ANSE to improve the quality and coherence of videos generated by diffusion models without retraining or external noise priors. |
| MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated |
|
|
| Experimental Feedback (Read more on arXiv or HuggingFace) |
Lidong Bing, Jue Wang, di-zhang-fdu, ZonglinY, wanhaoliu |
i) The paper introduces experiment-guided hypothesis ranking, a novel approach for scientific discovery. ii) The main objective is to develop a method for prioritizing candidate hypotheses based on prior experimental results, addressing the limitation of relying solely on LLM’s internal reasoning. iii) The research employs a simulator grounded in three domain-informed assumptions to model hypothesis performance, along with functional clustering of hypotheses. iv) Experiments demonstrate that the proposed method (CSX-Rank) reduces the number of trials required to identify ground-truth hypotheses by more than 50% on the TOMATO-chem dataset compared to pre-experiment baselines. v) AI practitioners can leverage this approach for more efficient hypothesis prioritization, particularly in resource-constrained domains where empirical validation is expensive, with implications for optimizing automated scientific discovery workflows. |
| AudioTrust: Benchmarking the Multifaceted Trustworthiness of Audio Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Jirui Han, Yile Liu, Can Shen, Kai Li, jiaxiaojunQAQ |
i) AudioTrust is introduced as the first multifaceted benchmark to evaluate the trustworthiness of Audio Large Language Models (ALLMs). ii) The research aims to comprehensively assess key trustworthiness dimensions of ALLMs, including fairness, hallucination, safety, privacy, robustness, and authentication. iii) The methodology involves a dataset of over 4,420 audio/text samples, 18 experimental setups, and 9 audio-specific evaluation metrics, utilizing a large-scale automated pipeline for model scoring. iv) Experimental results revealed systematic biases in ALLMs and uneven protection across different types of sensitive information; for example, Defense Success Rate(DSR) ranged from 76.2 - 100 for various closed/open source models on jailbreak attacks. v) AudioTrust provides AI practitioners with insights into the limitations and security vulnerabilities of current ALLMs, informing the development of more secure and reliable audio models. |
| Scaling Image and Video Generation via Test-Time Evolutionary Search (Read more on arXiv or HuggingFace) |
Di Zhang, Pengfei Wan, Xintao Wang, Jiajun Liang, haoranhe |
Scaling Image and Video Generation via Test-Time Evolutionary Search introduces EvoSearch, a novel and generalist test-time scaling framework for image and video generation. The paper addresses test-time scaling for diffusion and flow models by reformulating it as an evolutionary search problem. EvoSearch incorporates selection and mutation mechanisms tailored for stochastic differential equation denoising, iteratively improving sample quality while preserving population diversity. Experimental results show EvoSearch enables Stable Diffusion 2.1 to exceed GPT40 performance and Wan 1.3B to outperform Wan 14B, despite having 10x fewer parameters. This provides AI practitioners with a method to significantly enhance generative model sample quality through strategic computation allocation during inference, without additional training. |
| FullFront: Benchmarking MLLMs Across the Full Front-End Engineering |
|
|
| Workflow (Read more on arXiv or HuggingFace) |
Yu Cheng, Linjie Li, Huichen Will Wang, Haoyu Sun, Kuvvi |
FullFront is introduced as a benchmark for evaluating Multimodal Large Language Models (MLLMs) across the front-end engineering workflow. This research benchmarks MLLMs across three tasks: Webpage Design, Webpage Perception QA, and Webpage Code Generation. The methodology employs a novel two-stage process to transform real-world webpages into clean, standardized HTML. Empirical testing of MLLMs revealed limitations in page perception and code generation, specifically with image handling and layout, showing that the best-performing model, Claude 3.7 Sonnet, achieves an average accuracy below 55% in Webpage Perception QA tasks compared to human accuracy exceeding 95%. This indicates that current MLLM capabilities need enhancement for front-end development to bridge the performance gap with human experts, in fine-grained visual perception particularly. The benchmark and code are available for use by AI researchers to advance MLLM capabilities in front-end engineering. |
| Teaching with Lies: Curriculum DPO on Synthetic Negatives for |
|
|
| Hallucination Detection (Read more on arXiv or HuggingFace) |
Ying Ding, Liu Leqi, ashwinnv, SP2001 |
i) This paper introduces HaluCheck, a hallucination detection LLM aligned with a curriculum-based DPO framework using high-quality synthetic negatives. ii) The research objective is to improve the accuracy of hallucination detection in large language models. iii) The methodology involves using carefully curated, difficulty-ranked hallucinated samples as negative examples in a Direct Preference Optimization (DPO) framework with curriculum learning. iv) HaluCheck 3B achieves up to a 24% relative gain in detection metrics (accuracy, precision, recall, and F1-score) on benchmarks like MedHallu and HaluEval, also demonstrating robustness in zero-shot settings. v) The principal implication for AI practitioners is that using curriculum DPO with high-quality hallucinated negatives provides a robust approach for aligning LLMs to better detect hallucinations, improving trustworthiness of generated content. |
| Thought-Augmented Policy Optimization: Bridging External Guidance and |
|
|
| Internal Capabilities (Read more on arXiv or HuggingFace) |
Zhengqi Wen, Shuai Zhang, Mingkuan Feng, ChonghuaLiao, Jinyang23 |
i) This paper introduces Thought-Augmented Policy Optimization (TAPO), a novel reinforcement learning framework for enhancing the reasoning capabilities of language models. ii) The research aims to bridge the gap between external guidance and a language model’s internal reasoning capabilities for improved performance on reasoning tasks. iii) TAPO extends the Group Relative Policy Optimization (GRPO) method by incorporating a “thought library” of high-level thought patterns abstracted from prior samples, adaptively applied during training. iv) Experiments demonstrate that TAPO significantly outperforms GRPO, achieving a 99% performance increase on AIME and a 41% increase on AMC benchmarks and improved output explainability. v) TAPO offers AI practitioners a method for developing more robust and generalizable reasoning systems by integrating external abstract problem-solving guidance into reinforcement learning training, potentially leading to enhanced reasoning performance. |
| Clear Nights Ahead: Towards Multi-Weather Nighttime Image Restoration (Read more on arXiv or HuggingFace) |
Bin Xiao, Xiuli Bi, Yang Wei, Yunqiu, YuetongLiu |
i) This paper introduces a novel framework, ClearNight, for multi-weather nighttime image restoration and a corresponding dataset, AllWeatherNight. ii) The main objective is to effectively restore nighttime images degraded by the complex interaction of multiple adverse weather conditions and non-uniform illumination. iii) The methodology involves a unified architecture integrating Retinex-based dual prior guidance and a weather-aware dynamic specificity-commonality collaboration strategy. iv) Results show ClearNight achieves state-of-the-art performance, demonstrating a PSNR of 32.5937 on the AllWeatherNight synthetic testing subset when restoring Raindrop images, suggesting a significant advancement in handling mixed degradations. v) AI practitioners can leverage ClearNight as a robust solution for improving the performance of computer vision systems in challenging nighttime and adverse weather scenarios, particularly where multiple degradation types coexist, advancing applications such as autonomous driving and surveillance. |
| Teaching Large Language Models to Maintain Contextual Faithfulness via |
|
|
| Synthetic Tasks and Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zhitong Wang, Yuzhuo Bai, Cheng Gao, Shuzheng Si, BleachNick |
i) This paper introduces CANOE, a framework to improve the contextual faithfulness of Large Language Models (LLMs) using synthetic data and reinforcement learning. ii) The research aims to enhance LLM faithfulness in both short and long-form generation tasks without human annotations, addressing the challenge of knowledge conflicts and limited scalability of manually annotated data. iii) The methodology involves synthesizing short-form question-answering (QA) data with four diverse tasks and employing Dual-GRPO, a rule-based reinforcement learning method with tailored rewards. iv) Experimental results demonstrate that CANOE improves LLM faithfulness, achieving a 22.6% overall score improvement for Llama3-Instruct-8B and surpassing state-of-the-art LLMs like GPT-40 in overall score. v) CANOE offers AI practitioners a method to significantly improve the contextual faithfulness of LLMs with synthetic data and rule-based RL, reducing hallucinations without reliance on human annotations or simply scaling model parameters. |
| Time-R1: Towards Comprehensive Temporal Reasoning in LLMs (Read more on arXiv or HuggingFace) |
Jiaxuan You, Haoru Li, Haofei Yu, Peixuan Han, m-serious |
i) Time-R1 is introduced as a framework for enhancing temporal reasoning capabilities in Large Language Models (LLMs). ii) The research objective is to equip a 3B-parameter LLM with comprehensive temporal abilities, encompassing understanding, prediction, and creative generation related to time. iii) A three-stage reinforcement learning (RL) curriculum with a dynamic rule-based reward system is used for fine-tuning the LLM (Qwen2.5-3B-Instruct). iv) Experiments demonstrate that Time-R1 outperforms models over 200 times larger, including the 671B DeepSeek-R1, on future event prediction and creative scenario generation benchmarks. v) The principal implication for AI practitioners is that carefully engineered, progressive RL fine-tuning enables smaller, efficient models to achieve superior temporal performance, offering a scalable path towards time-aware AI, however some parts of the paper are unclear or seem to lack information. |
| Speechless: Speech Instruction Training Without Speech for Low Resource |
|
|
| Languages (Read more on arXiv or HuggingFace) |
Shreyas Gopal, Tuan Le Duc Anh, Huy Hoang Ha, Dinh Bach Vu, alandao |
Speechless presents a novel method for training speech-instruction models for low-resource languages without requiring text-to-speech (TTS) systems. The research objective is to enable speech understanding in LLMs by mapping text instructions to quantized Whisper encoder representations, bypassing traditional speech synthesis. The methodology involves training a quantizer on ASR data, then training a decoder-only language model (Speechless) to generate semantic tokens from text, followed by fine-tuning an LLM using these synthetic semantic tokens. Experiments on the Vietnamese language showed Speechless achieved competitive ASR performance; models using the Speechless framework achieve 2.69% CER on CommonVoice Vietnamese with beam search inference. This offers AI practitioners a computationally efficient way to create voice assistants in languages where high-quality TTS resources are unavailable. |
| Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse |
|
|
| Attention (Read more on arXiv or HuggingFace) |
Yikang Yang, Yifei Zeng, Feihu Zhang, Youtian Lin, Shuang Wu |
Direct3D-S2 introduces a scalable 3D generation framework using spatial sparse attention (SSA) to improve efficiency and quality in volumetric 3D shape generation. The research addresses the challenge of high computational and memory costs in generating high-resolution 3D shapes using signed distance functions. It employs a spatial sparse attention mechanism within a diffusion transformer to effectively process large token sets in sparse volumes. The method achieves a 3.9x speedup in the forward pass and a 9.6x speedup in the backward pass at 1024³ resolution compared to FlashAttention-2. Direct3D-S2 enables training at 1024³ resolution using only 8 GPUs, significantly reducing the resource requirements and enhancing accessibility for gigascale 3D generation for AI practitioners. |
| RBench-V: A Primary Assessment for Visual Reasoning Models with |
|
|
| Multi-modal Outputs (Read more on arXiv or HuggingFace) |
Qianrui Yang, uyzhang, Mo-ZheHan, CXY07, MenghaoGuo |
i) RBench-V introduces a new benchmark for evaluating visual reasoning capabilities of multi-modal models through multi-modal outputs. ii) The primary research objective is to assess the ability of models to generate appropriate visual outputs during visual reasoning tasks, such as image manipulation and auxiliary line construction. iii) The methodology involves a hand-picked set of 803 questions covering math, physics, counting, and games, requiring image modifications as part of the solution. iv) The best performing model, o3, achieved only 25.8% accuracy on RBench-V, compared to 82.3% for human experts, demonstrating a significant performance gap. v) AI practitioners need to develop new techniques, potentially including multi-modal chain-of-thought or agent-based reasoning, to improve visual reasoning with multi-modal output capabilities in foundation models. |
| s3: You Don’t Need That Much Data to Train a Search Agent via RL (Read more on arXiv or HuggingFace) |
Zifeng Wang, Jinfeng Xiao, Jiacheng Lin, Xueqiang Xu, Pengcheng Jiang |
s3 introduces a lightweight, model-agnostic framework for training search agents via reinforcement learning. The paper addresses the question of how to efficiently train a search agent to improve generation quality without modifying the generator LLM. s3 utilizes a novel Gain Beyond RAG (GBR) reward to train a searcher decoupled from the generator. Experiments show s3 requires only 2.4k training samples to outperform baselines trained on over 70× more data, achieving stronger downstream performance across six general QA and five medical QA benchmarks. s3 offers a more data-efficient approach to training retrieval components in RAG systems, potentially benefiting practitioners working with limited data. |
| Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement |
|
|
| Fine-Tuning of Large Language Models (Read more on arXiv or HuggingFace) |
Daoyuan Chen, Yuchang Sun, Yushuo Chen, Yanxi Chen, Xuchen Pan |
i) Trinity-RFT is presented as a general-purpose framework for reinforcement fine-tuning (RFT) of large language models (LLMs). ii) The research aims to provide a flexible and scalable framework unifying synchronous/asynchronous, on-policy/off-policy, and online/offline RFT modes. iii) The framework employs a decoupled design with an RFT-core comprising an explorer, trainer, and buffer, and supports agent-environment interaction with data pipelines optimized for RFT. iv) The framework can be adapted for diverse applications, offering a platform for exploring advanced reinforcement learning paradigms, but specific quantitative results are not provided. v) Trinity-RFT provides AI practitioners a unified platform for experimenting with various RFT methodologies and exploring advanced RL paradigms for LLMs. |
| Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning (Read more on arXiv or HuggingFace) |
Ruizhong Qiu, Yunzhe Qi, Zihao Li, Yikun Ban, jiaruz2 |
i) This paper introduces Transformer Copilot, a novel Pilot-Copilot framework for LLM fine-tuning that leverages an internal “Mistake Log” to enhance inference performance. ii) The primary objective is to improve LLM inference performance by retaining and leveraging the model’s own learning signals during standard fine-tuning. iii) The methodology involves creating a Mistake Log to track model errors and training a Copilot model to rectify the Pilot model’s logits through a joint training and fused inference paradigm. iv) Experiments across 12 benchmarks showed that Transformer Copilot consistently improves performance by up to 34.5%. v) The principal implication for AI practitioners is a novel approach for enhancing LLM performance with marginal computational overhead by exploiting internal learning signals, offering a potential improvement on standard supervised fine-tuning techniques. |
| Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark |
|
|
| Study (Read more on arXiv or HuggingFace) |
Hwanjo Yu, Jihae Jeong, Joonwon Jang, oneonlee |
i) This paper introduces MEMESAFETYBENCH, a meme-based benchmark for evaluating VLM safety. ii) The study investigates the safety of VLMs when processing real-world meme images shared by ordinary users. iii) The methodology involves pairing real meme images with both harmful and benign instructions, utilizing a comprehensive safety taxonomy and LLM-based instruction generation to assess VLM responses across single and multi-turn interactions. iv) Results show VLMs are more vulnerable to meme-based harmful prompts than synthetic or typographic images, with memes significantly increasing harmful responses and decreasing refusals, and multi-turn interactions providing only partial mitigation. v) The findings highlight the necessity for ecologically valid VLM evaluations and stronger safety mechanisms due to the identified vulnerability of current VLMs to real-world meme-based prompts. |
| Large Language Models Implicitly Learn to See and Hear Just By Reading (Read more on arXiv or HuggingFace) |
Mert Pilanci, Prateek Verma |
i) This paper explores using text-LLM weights for audio and image classification by repurposing text-only trained models for non-text modalities. ii) The research investigates whether text-LLMs can inherently develop the ability to understand images and audio without explicit training on these modalities. iii) The methodology involves replacing the ViT/Audio-Transformer encoder with a fine-tuned text-LLM (GPT-2) and training a linear projection to map image/audio patches to the LLM’s embedding space, employing PEFT techniques such as LORA. iv) The results show that fine-tuning text-LLMs for audio classification on FSD-50K achieves an accuracy of 44.1% with a GPT-1.5B backbone, and the method is applicable to image classification on CIFAR-10 and Fashion-MNIST, yielding competitive accuracy. v) AI practitioners can potentially reuse pre-trained text-LLMs for multimodal tasks by fine-tuning a small number of parameters instead of training new models from scratch, offering an efficient approach to representation learning for audio and image processing. |
| Synthetic Data RL: Task Definition Is All You Need (Read more on arXiv or HuggingFace) |
Zekai Zhang, Zi-Ang Wang, Chuanwei Huang, Yiduo Guo, zguo0525 |
i) The paper introduces Synthetic Data RL, a framework for adapting foundation models to specialized tasks using only synthetic data generated from task definitions. ii) The main objective is to reduce reliance on human-labeled data in reinforcement learning for adapting foundation models by using synthetically generated data based on task definitions. iii) The methodology involves generating question-answer pairs from task definitions and retrieved documents, adapting question difficulty based on model solvability, and selecting questions for RL training using the model’s average pass rate. iv) The method achieves a 29.2% absolute improvement over the base model on GSM8K and also achieves 8.7% on MATH, 13.1% on GPQA, 8.9% on MedQA, 17.7% on CQA(law), and 13.7% on CFA (finance). v) This framework allows AI practitioners to adapt models efficiently without extensive human annotation, enabling more scalable and efficient RL-based model adaptation. |
| RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation |
|
|
| via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jianjin Zhang, Fangkai Yang, Pu Zhao, Lu Wang, Mingrui Wu |
RePrompt enhances text-to-image generation by integrating explicit reasoning into the prompt refinement process. The research aims to improve the fidelity of T2I models in capturing user intentions from concise prompts. The methodology employs reinforcement learning to train a language model that generates structured, self-reflective prompts optimized for image-level outcomes based on human preference, semantic alignment, and visual composition. Experiments on GenEval demonstrate that RePrompt surpasses Qwen2.5 3B-enhanced baselines by +77.1% in the position category, indicating superior spatial grounding. RePrompt offers AI practitioners a method for boosting spatial layout fidelity and compositional generalization in T2I applications without relying on large language models or expensive optimization. |
| On the Design of KL-Regularized Policy Gradient Algorithms for LLM |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Quanquan Gu, Yang Yuan, Huizhuo Yuan, Lewis-Lau, yifAI |
i) The paper proposes Regularized Policy Gradient (RPG), a framework for deriving and analyzing KL-regularized policy gradient methods in online reinforcement learning. ii) The main objective is to systematically explore how different KL divergence formulations can be estimated and integrated into surrogate loss functions for online RL. iii) The methodology involves deriving policy gradients and surrogate loss functions for objectives regularized by forward and reverse KL divergences with normalized and unnormalized policy distributions and comparing them to GRPO, REINFORCE++, and DAPO. iv) Experiments show improved training stability and performance compared to baselines; for example, RPG-FKL achieves the best overall score (Best: 0.8836) on AMC23. v) The RPG framework and its variants can be adopted by AI practitioners for enhanced control over training dynamics and improved mathematical reasoning capabilities in LLMs through systematic KL-regularized policy gradient optimization. |
| Diffusion Classifiers Understand Compositionality, but Conditions Apply (Read more on arXiv or HuggingFace) |
Anna Rohrbach, Seong Joon Oh, Yujin Jeong, Gigglingface |
i) This paper investigates the compositional understanding capabilities of diffusion classifiers across diverse tasks and models. ii) The research aims to comprehensively evaluate and understand the discriminative abilities of diffusion classifiers in compositional scenarios, considering factors like dataset domains and timestep weighting. iii) The study employs an extensive evaluation of diffusion classifiers (SD 1.5, 2.0, and 3-m) on 10 compositional datasets, introducing a new diagnostic benchmark (SELF-BENCH) comprising diffusion-generated images, and explores timestep weighting strategies. iv) Results show diffusion models underperform CLIP on counting tasks and exhibit sensitivity to domain gaps, where SD3-m’s discriminative accuracy reaches only 39%, with timestep reweighting improving performance in large domain gap scenarios; a correlation between visual similarity (CLIP distance) and optimal timestep weighting is also observed. v) AI practitioners should consider domain adaptation techniques and timestep weighting strategies when deploying diffusion classifiers for compositional tasks, particularly for models like SD3-m which demonstrates enhanced sensitivity. |
| Interactive Post-Training for Vision-Language-Action Models (Read more on arXiv or HuggingFace) |
Philipp Krähenbühl, Yue Zhao, Kairan Dou, tanshh97 |
RIPT-VLA introduces a reinforcement learning-based post-training paradigm to enhance vision-language-action (VLA) models. The research aims to improve VLA model adaptability to new tasks and environments using minimal supervision by interactive learning. The methodology involves a novel reinforcement learning framework with dynamic rollout sampling and leave-one-out advantage estimation, optimizing for sparse binary success rewards. Experiments show RIPT-VLA improves the 7B OpenVLA-OFT model to a 97.5% success rate and enhances the QueST model by 10.9% absolute success rate across various LIBERO suites. RIPT-VLA offers AI practitioners a computationally efficient and data-efficient method for post-training VLA models, enabling significant performance gains, particularly in low-data regimes. |
| Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA (Read more on arXiv or HuggingFace) |
Sai Rajeswar, Shiva Krishna Reddy Malay, Khyati Mahajan, Masoud Hashemi, rmahesh |
i) This paper introduces NotesWriting, a method to enhance iterative Retrieval-Augmented Generation (RAG) by generating concise notes from retrieved documents at each step. ii) The research aims to improve the effective context length of LLMs in iterative RAG, addressing issues of context overload, computational cost, distraction, and readability. iii) The proposed method involves using a smaller language model to extract relevant notes from retrieved documents based on the sub-question at each step, replacing raw documents with shorter, focused notes. iv) Experiments across three iterative RAG baselines, four multi-hop QA datasets, and two LLMs showed that NotesWriting yields an average improvement of 15.6 percentage points overall with minimal increase in output tokens. v) NotesWriting allows practitioners to improve the planning and reasoning capabilities of LLMs by increasing the volume of ingested text while using it alongside iterative RAG frameworks for multi-hop question answering tasks. |
| NOVER: Incentive Training for Language Models via Verifier-Free |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yali Du, Chen Qian, Xinyu Wang, Siya Qi, Wei Liu |
NOVER proposes a verifier-free reinforcement learning framework for incentive training of language models. The research investigates how to enable incentive training across text-to-text tasks without external verifiers or costly reward models. NOVER utilizes a model’s own reasoning process perplexity as a reward proxy, calculated from standard supervised fine-tuning data, for lightweight RL training. Experiments show NOVER outperforms models distilled from large reasoning models like DeepSeek R1 671B by 7.7%. NOVER enables incentive-driven reinforcement learning across a range of tasks and facilitates approaches such as inverse incentive training. This work allows practitioners to apply incentive training to text generation tasks lacking readily available external verifiers. |
| Keep Security! Benchmarking Security Policy Preservation in Large |
|
|
| Language Model Contexts Against Indirect Attacks in Question Answering (Read more on arXiv or HuggingFace) |
Hwanhee Lee, Yonghyun Jun, Yumin Kim, HwanChang0106 |
i) This paper introduces CoPriva, a benchmark for evaluating the ability of large language models (LLMs) to preserve contextual security policies against direct and indirect attacks in question answering. ii) The primary objective is to assess the vulnerability of LLMs in adhering to user-defined security policies within context, especially concerning information non-disclosure, under adversarial conditions. iii) The methodology involves creating a dataset of 4,184 instances derived from realistic contexts with explicit security policies and designing direct and indirect attack queries. iv) Evaluation of 10 LLMs revealed a significant vulnerability; many models leaked sensitive information, with indirect attacks exacerbating the issue, increasing leakage by over 40 percentage points on average. v) AI practitioners should be aware that current LLMs exhibit a critical gap in safety alignment for sensitive applications, necessitating more robust methods to guarantee contextual security, especially when models with higher faithfulness scores tend to leak more information because of a trade-off between helpfulness and policy compliance. |
| TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in |
|
|
| Real-World Scenarios (Read more on arXiv or HuggingFace) |
Tianyi Zhuang, Wen Luo, Wei Li, Shaohang Wei, songff |
i) The paper introduces TIME, a multi-level benchmark for evaluating temporal reasoning in Large Language Models (LLMs) across real-world scenarios. ii) The main objective is to address the limitations of existing benchmarks by capturing real-world challenges such as intensive temporal information, fast-changing event dynamics, and complex social interactions. iii) The methodology involves constructing three sub-datasets, TIME-WIKI, TIME-NEWS, and TIME-DIAL, encompassing 38,522 QA pairs across 11 fine-grained sub-tasks, and evaluating various reasoning and non-reasoning LLMs. iv) Experimental results show that models exhibit suboptimal performance on the Timeline task, with small-scale vanilla models achieving accuracy below 10% on both TIME-WIKI and TIME-DIAL datasets. v) LLM practitioners should consider TIME-LITE, a human-annotated subset designed to foster research and standardized evaluation of temporal reasoning, which includes 938 carefully curated instances. |
| Not All Models Suit Expert Offloading: On Local Routing Consistency of |
|
|
| Mixture-of-Expert Models (Read more on arXiv or HuggingFace) |
Duyu Tang, Yitong Li, Miren Tian, Siyuan Wang, ljcleo |
i) This paper introduces two metrics, SRP and SCH, to evaluate the local routing consistency in Mixture-of-Expert (MoE) language models for efficient expert offloading. ii) The research investigates how local routing consistency varies across different MoE model architectures and expert specializations, with the goal of guiding memory-efficient MoE deployment. iii) The methodology involves analyzing 20 MoE LLMs by quantifying local routing consistency using Segment Routing Best Performance (SRP) and Segment Cache Best Hit Rate (SCH) metrics across different segment lengths and cache sizes. iv) The primary results show that MoE models applying MoE on every layer and without shared experts exhibit the highest local routing consistency, and a cache size of approximately 2x the number of active experts achieves optimal balance, furthermore SRP strongly correlated to domain specialization of experts. v) AI practitioners can use these metrics to design and deploy memory-efficient MoE models, balancing cache effectiveness and efficiency during inference, and prioritize models exhibiting high local routing consistency for easier expert offloading strategies. |
| Revisiting Residual Connections: Orthogonal Updates for Stable and |
|
|
| Efficient Deep Networks (Read more on arXiv or HuggingFace) |
Younjae Yu, Suhwan Choi, Siyeol Kim, Woohyun Cho, Giyeong Oh |
i) This paper introduces Orthogonal Residual Update, a novel technique for enhancing deep network training. ii) The research objective is to improve generalization accuracy and training stability in deep neural networks by modifying residual connections. iii) The methodology involves decomposing the module’s output into components parallel and orthogonal to the input stream, adding only the orthogonal component during the update. iv) Experiments across diverse architectures and datasets, including ViT-B on ImageNet-1k, demonstrated a +4.3%p top-1 accuracy gain. v) Orthogonal Residual Update can improve deep network design, offering enhanced performance and efficiency, which can lead to practical improvements in stability of AI models. |
Papers for 2025-05-23
| Title |
Authors |
Summary |
| NovelSeek: When Agent Becomes the Scientist – Building Closed-Loop |
|
|
| System from Hypothesis to Verification (Read more on arXiv or HuggingFace) |
Jiakang Yuan, Xiangchao Yan, Shiyang Feng, Bo Zhang, NovelSeek Team |
i) The paper presents NOVELSEEK, a closed-loop multi-agent framework for autonomous scientific research (ASR) across various scientific domains. ii) The research aims to facilitate innovative research by automating the entire research cycle from idea generation to experimental validation. iii) The methodology involves self-evolving idea generation with human interaction, idea-to-methodology construction, and multi-round automated experiment execution with a unified multi-agent system. iv) In reaction yield prediction, NOVELSEEK improved performance from 27.6% to 35.4% in 12 hours and increased enhancer activity prediction accuracy from 0.52 to 0.79 in 4 hours. In 2D semantic segmentation, precision rose from 78.8% to 81.0% in 30 hours. v) The NOVELSEEK framework enables AI practitioners to automate and accelerate scientific research tasks, reducing reliance on manual effort and enabling faster innovation cycles across diverse scientific fields including domains with complex codes. |
| Scaling Reasoning, Losing Control: Evaluating Instruction Following in |
|
|
| Large Reasoning Models (Read more on arXiv or HuggingFace) |
Yu Cheng, Xiaoye Qu, Jiawei Gu, yaful, TingchenFu |
i) The paper introduces MathIF, a new benchmark for evaluating instruction following in Large Reasoning Models (LRMs). ii) It investigates the tension between scaling reasoning capabilities and maintaining controllability in LRMs. iii) The methodology involves evaluating 23 LRMs on MathIF, which contains 420 math reasoning problems combined with 15 programmatically verifiable instruction constraints. iv) Results show that the best-performing model, Qwen3-14B, achieves only 50.71% accuracy on strict instruction following, and increasing CoT length degrades instruction adherence. v) This suggests AI practitioners face a trade-off between improving reasoning depth and ensuring adherence to user-specified constraints during LRM development. |
| Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Hongjin Qian, Jiajie Jin, Xiaoxi Li, Yifei Chen, Guanting Dong |
Tool-Star is an RL-based framework for LLMs to autonomously invoke multiple external tools during reasoning. The paper addresses the challenge of multi-tool collaborative reasoning in LLMs. It uses a tool-integrated reasoning data synthesis pipeline combining tool-integrated prompting with hint-based sampling, followed by a two-stage training framework. Experiments on 10 reasoning benchmarks show Tool-Star achieves over 40% average accuracy across datasets. The principal implication is an effective and efficient multi-tool collaboration method for enhancing LLM reasoning, providing AI practitioners with a means to improve LLM performance on complex tasks requiring external tool usage. |
| KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models (Read more on arXiv or HuggingFace) |
Xianfang Zeng, Xinyu Ye, Xinting Hu, Zonghui Li, Yongliang Wu |
i) The paper introduces KRIS-Bench, a benchmark for evaluating knowledge-based reasoning in instruction-based image editing models. ii) The research aims to assess the capacity of image editing models to perform tasks requiring factual, conceptual, and procedural knowledge. iii) The methodology involves creating a diagnostic benchmark with 22 tasks spanning 7 reasoning dimensions and a novel Knowledge Plausibility metric. iv) Experiments on 10 models revealed gaps in reasoning performance, with GPT-4o achieving the highest overall score but exhibiting limitations in accurately interpreting chemical reactions. v) KRIS-Bench provides AI practitioners with a fine-grained evaluation framework for developing knowledge-centric image editing systems. |
| Pixel Reasoner: Incentivizing Pixel-Space Reasoning with |
|
|
| Curiosity-Driven Reinforcement Learning (Read more on arXiv or HuggingFace) |
Fangzhen Lin, Weimin Ren, Haozhe Wang, Alex Su, wenhu |
i) This paper introduces Pixel-Reasoner, a novel framework enabling Vision-Language Models (VLMs) to reason directly in the pixel space using visual operations. ii) The primary objective is to equip VLMs with visual reasoning operations like zoom-in and select-frame, allowing them to interact with and infer from visual data more effectively. iii) The methodology involves a two-phase training approach: instruction tuning on synthesized reasoning traces followed by reinforcement learning with a curiosity-driven reward scheme. iv) The 7B Pixel-Reasoner model achieves 84% accuracy on V* bench, 74% on TallyQA-Complex, and 84% on InfographicsVQA, surpassing existing open-source models. v) The principal implication is that incentivizing pixel-space reasoning significantly improves VLM performance on visually intensive tasks, offering AI practitioners a method for enhancing visual understanding and reasoning capabilities. |
| QuickVideo: Real-Time Long Video Understanding with System Algorithm |
|
|
| Co-Design (Read more on arXiv or HuggingFace) |
Wenhu Chen, Tianyu Pang, Chao Du, Dongfu Jiang, Benjamin Schneider |
i) QuickVideo accelerates long video understanding for VideoLLMs through system-algorithm co-design. ii) The research objective is to reduce the computational overhead of long video processing to enable real-time applications. iii) The methodology includes a parallelized CPU-based video decoder (QuickCodec), a memory-efficient prefilling method using KV-cache pruning (QuickPrefill), and an overlapping execution scheme. iv) QuickVideo reduces the inference time of a 30-minute video input by more than 3x, from 69.7 seconds to 20.0 seconds. v) QuickVideo provides AI practitioners with an optimized framework that can significantly accelerate long video understanding, enabling more efficient VideoLLM applications even on limited hardware. |
| GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation |
|
|
| with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Linjiang Huang, Kun Wang, Yuqing Wang, Rongyao Fang, Chengqi Duan |
i) GoT-R1 is a reinforcement learning framework to improve semantic-spatial reasoning for visual generation in MLLMs. ii) The research aims to enhance the ability of MLLMs to handle complex compositional prompts in visual generation through improved reasoning. iii) The methodology employs a dual-stage multi-dimensional reward framework with MLLMs to evaluate both the reasoning process and the final image output, optimized via Group Relative Policy Optimization (GRPO). iv) Experimental results on T2I-CompBench show significant improvements, particularly in compositional tasks; GoT-R1-7B achieved a score of 0.94 in two-object generation in GenEval benchmark, up from 0.69 of GoT; v) The framework’s capacity to autonomously discover effective reasoning strategies via RL enables AI practitioners to generate more accurate and contextually aware visual content, enhancing compositional image synthesis. |
| LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning (Read more on arXiv or HuggingFace) |
Jun Zhou, Jun Hu, Xiaolu Zhang, Shen Nie, Zebin You |
LLaDA-V is introduced as a purely diffusion-based Multimodal Large Language Model (MLLM) with visual instruction tuning. The research investigates how to effectively extend large language diffusion models for multimodal understanding, focusing on visual instruction tuning. LLaDA-V incorporates a vision encoder and MLP connector to project visual features into the language embedding space and is trained on multi-turn multimodal dialogues. LLaDA-V achieves state-of-the-art performance in multimodal understanding compared to existing hybrid autoregressive-diffusion and purely diffusion-based MLLMs and demonstrates stronger data scalability on several benchmarks when compared to LLaMA3-V. LLaDA-V’s superior scalability when compared to LLaMA3-V suggests that large language diffusion models show promise in multimodal contexts. |
| Risk-Averse Reinforcement Learning with Itakura-Saito Loss (Read more on arXiv or HuggingFace) |
Alexander Korotin, Evgeny Burnaev, Anita Toleutaeva, Olivier Croissant, i-udovichenko |
i) This paper introduces a novel and numerically stable loss function based on Itakura-Saito divergence for risk-averse reinforcement learning with exponential utility. ii) The main objective is to develop a loss function that addresses the numerical instability issues of existing exponential-utility RL approaches while preserving theoretical guarantees. iii) The methodology involves deriving Bellman equations for exponential utility and employing reinforcement learning algorithms using the proposed Itakura-Saito loss function for learning state-value and action-value functions. iv) The experiments demonstrate that the proposed Itakura-Saito loss outperforms alternatives in portfolio optimization, deep hedging tasks, and robust combinatorial optimization problems, exhibiting more stable convergence. v) The Itakura-Saito loss offers AI practitioners a numerically stable and theoretically sound alternative to exponential MSE for training risk-averse RL agents, particularly in high-stakes applications requiring reliable convergence. |
| Scaling Diffusion Transformers Efficiently via μP (Read more on arXiv or HuggingFace) |
Zhi Tian, Wei Huang, Rongzhen Wang, Xinyu Zhang, ChenyuZheng |
i) This paper generalizes Maximal Update Parametrization (µP) to diffusion Transformers for efficient scaling. ii) The main objective is to determine if the µP properties observed in vanilla Transformers extend to diffusion Transformers, enabling stable hyperparameter transfer. iii) The methodology involves proving the µP formulation for diffusion Transformers aligns with vanilla Transformers and validating this through large-scale image and text-to-image generation experiments. iv) The primary result shows that DiT-XL-2-µP with a transferred learning rate achieves 2.9x faster convergence compared to the original DiT-XL-2; further scaling experiments on PixArt-a and MMDiT models also demonstrate improved performance. v) These results suggest that AI practitioners can leverage µP to efficiently scale diffusion Transformers, reducing hyperparameter tuning costs while maintaining or improving model performance in large-scale generation tasks. |
| Let LLMs Break Free from Overthinking via Self-Braking Tuning (Read more on arXiv or HuggingFace) |
Wenqi Zhang, Haolei Xu, Yongliang Shen, Yuchen Yan, Haoran Zhao |
i) The paper introduces Self-Braking Tuning (SBT), a novel framework enabling Large Reasoning Models (LRMs) to autonomously regulate reasoning length and mitigate overthinking. ii) The research aims to enable LRMs to autonomously recognize excessive reasoning and terminate their thinking process appropriately without external interventions. iii) The methodology involves constructing overthinking identification metrics, developing data construction strategies (SBT-E and SBT-D) for adaptive reasoning lengths, and introducing a braking prompt mechanism. iv) Experiments show that SBT reduces token consumption by up to 60% on mathematical benchmarks like AIME and GSM8K, while maintaining comparable accuracy to unconstrained models. v) For AI practitioners, SBT offers a method to significantly reduce computational overhead in LRMs by enabling self-regulation of reasoning depth, directly impacting the cost-effectiveness and deployment feasibility of these models. |
| Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning (Read more on arXiv or HuggingFace) |
Guiyang Hou, Wenqi Zhang, Yongliang Shen, Yuchen Yan, Haolei Xu |
i) This paper introduces CoT-Bridge, a method to automatically identify and bridge Thought Leaps in Chain-of-Thought (CoT) reasoning to improve model learning and generalization. ii) The research aims to address the negative impact of Thought Leaps (omitted intermediate reasoning steps) on the performance of Large Language Models (LLMs) in mathematical tasks. iii) The authors constructed ScaleQM+, a specialized training dataset, and trained the CoT-Bridge model to detect leaps and generate missing intermediate reasoning steps. iv) Experiments on mathematical reasoning benchmarks show that models fine-tuned on bridged datasets outperform those trained on original datasets, with improvements of up to +5.87% on NuminaMath. v) CoT-Bridge serves as a plug-and-play module compatible with existing optimization techniques to improve data quality for downstream tasks such as knowledge distillation and reinforcement learning, enhancing the effectiveness of LLMs in mathematical reasoning and improving generalization to other reasoning tasks, such as OOD logical reasoning tasks (↑2.99%). |
| Backdoor Cleaning without External Guidance in MLLM Fine-tuning (Read more on arXiv or HuggingFace) |
Xun Xiao, Jinhe Bi, Jian Liang, Wenke Huang, Xuankun Rong |
i) This paper introduces Believe Your Eyes (BYE), a data filtering framework for mitigating backdoor attacks in multimodal large language models (MLLMs) during fine-tuning. ii) The research aims to address the security risks introduced by malicious fine-tuning in MLLMs, specifically the injection of backdoor triggers, without relying on external guidance. iii) BYE leverages cross-modal attention entropy as a self-supervised signal, extracting attention maps, computing entropy scores, profiling sensitive layers using bimodal separation, and employing unsupervised clustering to filter suspicious samples. iv) Experiments demonstrate BYE achieves near-zero attack success rates (e.g., reducing ASR to 7.18% on RSVQA with InternVL) while maintaining clean-task performance. v) AI practitioners can utilize BYE as a robust, generalizable, and self-contained solution for filtering poisoned data and enhancing the security of MLLMs in fine-tuning-as-a-service settings without clean supervision or model modifications. |
| Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel |
|
|
| Decoding (Read more on arXiv or HuggingFace) |
Xinchao Wang, Xinyin Ma, Runpeng Yu |
Dimple is a Discrete Diffusion Multimodal Large Language Model (DMLLM) designed for parallel decoding. The research addresses training instability, suboptimal performance, and length bias observed in purely discrete diffusion approaches for DMLLMs. The methodology combines an initial autoregressive pre-training phase with a subsequent diffusion-based masked language modeling phase, incorporating confident decoding to improve inference efficiency. The Dimple-7B model achieves a 3.9% performance increase over LLaVA-NEXT on MLLM benchmarks using a similar training data, indicating comparable performance of DMLLM to autoregressive models under similar training data scales. For AI practitioners, this demonstrates the feasibility of DMLLMs and provides techniques for enhancing inference efficiency and controllability in multimodal generation tasks, offering a new paradigm beyond autoregressive generation. |
| VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game |
|
|
| Quality Assurance (Read more on arXiv or HuggingFace) |
Nabajeet Barman, Saman Zadtootaghaj, Abhijay Ghildyal, corpaul, taesiri |
i) The paper introduces VideoGameQA-Bench, a new benchmark for evaluating Vision-Language Models (VLMs) in video game Quality Assurance (QA) tasks. ii) The main objective is to provide a comprehensive benchmark to assess VLM performance in real-world game QA scenarios, including visual unit testing, glitch detection, and bug reporting. iii) The methodology involves curating a dataset of 4,786 questions and images/videos from over 800 games and 9 synthetic scenes, followed by evaluating the performance of 16 VLMs on the defined tasks. iv) The primary result indicates that frontier VLMs achieve up to 82.8% accuracy on image-based glitch detection and 78.1% on video-based glitch detection, but struggle with tasks requiring fine-grained detail analysis and common-sense reasoning. v) The principal implication for AI practitioners is the identification of specific limitations in current VLMs for automating video game QA, highlighting the need for improved spatial reasoning and detail extraction capabilities. |
| Training-Free Efficient Video Generation via Dynamic Token Carving (Read more on arXiv or HuggingFace) |
Bohao Peng, Shaoteng Liu, Bin Xia, Jinbo Xing, Yuechen Zhang |
i) This paper presents Jenga, a training-free inference pipeline to improve the efficiency of video generation using Diffusion Transformer (DiT) models. ii) The main objective is to reduce the computational cost associated with DiT models for video generation without requiring model retraining. iii) The methodology combines dynamic attention carving, using 3D space-filling curves to select relevant token interactions, with a progressive resolution generation strategy. iv) Jenga achieves up to 8.83x speedup on HunyuanT2V with only a 0.01% performance drop on VBench. v) Jenga’s plug-and-play nature enables practical, high-quality video generation on modern hardware by significantly reducing inference time, making it relevant for AI practitioners seeking to deploy video generation models efficiently. |
| Understanding Generative AI Capabilities in Everyday Image Editing Tasks (Read more on arXiv or HuggingFace) |
Franck Dernoncourt, Viet Dac Lai, loganbolton, Franck-Dernoncourt, taesiri |
This paper analyzes generative AI’s effectiveness in real-world image editing. It addresses the question of what types of image editing requests can be successfully handled by current AI editors compared to human editors. The study involves analyzing 83k real-world requests from the /r/PhotoshopRequest Reddit community with their corresponding 305k human-made edits, evaluating them against edits from 49 AI editors and ratings from vision-language models (VLMs). The primary result indicates that AI editors can fulfill approximately 33% of real-world image-editing requests, based on human ratings, with VLMs showing biased judgements. The principal implication is that AI practitioners should focus on improving AI editors’ ability to handle precise editing tasks and preserve subject identity, as well as addressing biases in VLM judgment metrics. |
| SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward (Read more on arXiv or HuggingFace) |
Xiangyu Yue, Dongzhan Zhou, Haoming Lyu, Kaituo Feng, Kaixuan Fan |
i) The paper introduces SophiaVL-R1, a multimodal large language model trained with Trust-GRPO, incorporating model-generated thinking rewards alongside rule-based outcome rewards. ii) The main objective is to enhance MLLMs’ reasoning and generalization capabilities by providing supervision over the thinking process. iii) The methodology involves training a thinking reward model, implementing Trust-GRPO to weigh the thinking reward’s trustworthiness, and using an annealing training strategy. iv) Experimental results show SophiaVL-R1-7B achieves 71.3% accuracy on MathVista and outperforms LLaVA-OneVision-72B on multiple benchmarks, and demonstrate consistently strong performance across general ability benchmarks. v) AI practitioners can utilize the Trust-GRPO algorithm to improve the reliability of reward signals in reinforcement learning for MLLMs, leading to better reasoning and generalization. |
| SpatialScore: Towards Unified Evaluation for Multimodal Spatial |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Yanfeng Wang, Ya Zhang, Yaohui Chen, Xiao Huang, Haoning Wu |
i) The paper introduces SpatialScore, a comprehensive benchmark for evaluating spatial understanding in multimodal large language models (MLLMs). ii) The main objective is to assess the capabilities of existing MLLMs in 3D spatial perception and understanding. iii) The methodology involves creating a new benchmark, SpatialScore, integrating a novel dataset VGBench with data from 11 existing datasets, and developing SpatialAgent, a multi-agent system equipped with specialized tools for spatial reasoning. iv) The SpatialScore benchmark includes 28K samples with a challenging subset (SpatialScore-Hard) of 1.4K samples, and evaluations reveal that while SpatialAgent improves performance, current MLLMs still lag behind human performance. v) The comprehensive and diverse nature of SpatialScore provides AI practitioners with a rigorous testbed and insights for future MLLM development, highlighting the need for fundamental architectural innovations in spatial reasoning. |
| LaViDa: A Large Diffusion Language Model for Multimodal Understanding (Read more on arXiv or HuggingFace) |
Yusuke Kato, Akash Gokul, Hritik Bansal, Konstantinos Kallidromitis, Shufan Li |
LaViDa introduces a diffusion-based VLM for multimodal understanding, offering an alternative to autoregressive models. The research focuses on developing diffusion models (DMs) for vision-language tasks using complementary masking, Prefix-DLM inference, and timestep shifting techniques. Experiments show LaViDa achieves competitive performance on multimodal benchmarks like MMMU while providing advantages such as speed-quality tradeoff; specifically, LaViDa surpasses Open-LLaVa-Next-Llama3-8B by +4.1 CIDEr on COCO captioning with a 1.92x speedup. This work offers AI practitioners a competitive, controllable, and efficient VLM alternative to autoregressive models, especially for tasks requiring bidirectional reasoning or flexible speed-quality trade-offs. The paper seems to lack information on the exact model architecture and datasets used to train the model. |
| TinyV: Reducing False Negatives in Verification Improves RL for LLM |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Luyao Niu, Bhaskar Ramasubramanian, Fengqing Jiang, Yuetai Li, Zhangchen Xu |
TinyV reduces false negatives in verification to improve RL for LLM reasoning. The paper investigates the impact of false negatives (FNs) in reward signals provided by verifiers during RL training of LLMs for reasoning tasks. It mitigates FNs by proposing TINYV, a lightweight LLM-based verifier that augments rule-based methods. Empirical analysis on the Big-Math-RL-Verified dataset reveals over 38% of model-generated responses suffer from false negatives, impairing RL training. Integrating TINYV boosts pass rates by up to 10% across math-reasoning benchmarks and accelerates convergence relative to baselines. Addressing verifier false negatives is critical for improving RL-based fine-tuning of LLMs, allowing for more robust policy optimization. |
| Training-Free Reasoning and Reflection in MLLMs (Read more on arXiv or HuggingFace) |
Zhenzhong Chen, Hongchen Wei |
i) The paper introduces FRANK, a training-free method for endowing Multimodal Large Language Models (MLLMs) with reasoning and reflection capabilities. ii) The main objective is to enhance the reasoning abilities of existing MLLMs without requiring additional training data or gradient updates. iii) FRANK leverages a hierarchical weight merging approach that combines a vision-pretrained MLLM with a reasoning-specialized LLM, guided by layer-wise functional specialization and Taylor-derived closed-form fusion. iv) FRANK-38B achieves an accuracy of 69.2 on the MMMU benchmark, outperforming InternVL2.5-38B by +5.3 and surpassing GPT-40. v) FRANK provides AI practitioners with a cost-effective strategy to imbue off-the-shelf MLLMs with advanced reasoning capabilities, eliminating the need for resource-intensive retraining or scarce, high-quality multimodal reasoning datasets. |
| GRIT: Teaching MLLMs to Think with Images (Read more on arXiv or HuggingFace) |
Ching-Chen Kuo, Kaizhi Zheng, Diji Yang, Xuehai He, Yue Fan |
i) The paper introduces Grounded Reasoning with Images and Text (GRIT), a method for training Multimodal Large Language Models (MLLMs) to generate reasoning chains grounded in visual data using bounding box coordinates. ii) The research aims to enable MLLMs to perform visual reasoning with explicit integration of visual information via grounded reasoning chains. iii) GRIT uses a reinforcement learning approach, GRPO-GR, employing rewards focused on answer accuracy and the format of grounded reasoning outputs, eliminating the need for reasoning chain annotations or bounding box labels. iv) Experiments show that GRIT-trained models, using only 20 image-question-answer triplets from VSR and TallyQA, achieve a GPT-as-judge answer accuracy of 72.9% on VSR and 47.8% on TallyQA. v) GRIT offers AI practitioners a data-efficient method for training MLLMs to generate coherent, visually-grounded reasoning chains, unifying grounding and reasoning abilities without extensive data annotation. |
| AGENTIF: Benchmarking Instruction Following of Large Language Models in |
|
|
| Agentic Scenarios (Read more on arXiv or HuggingFace) |
Youfeng Liu, Amy Xin, Xiaozhi Wang, Hao Peng, Yunjia Qi |
AGENTIF is introduced as a benchmark for evaluating instruction following in LLMs within agentic contexts. The research addresses whether LLMs can reliably follow lengthy instructions with complex constraints common in real-world agentic applications. The study uses 707 human-annotated instructions across 50 real-world agentic tasks annotated with constraints and evaluation metrics including code-based, LLM-based, and hybrid methods. Results show current models perform poorly, especially with complex constraint structures and tool specifications; the best-performing model follows fewer than 30% of instructions perfectly. AGENTIF highlights the need for improved LLMs in adhering to complex instructions for AI practitioners developing LLM-based agents, particularly concerning conditional and tool constraints. |
| Think or Not? Selective Reasoning via Reinforcement Learning for |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, James Cheng, Kevin Qinghong Lin, Jiaqi Wang |
i) This paper introduces TON, a reinforcement learning framework for vision-language models that enables selective reasoning to improve efficiency. ii) The research aims to enable VLMs to decide when reasoning is necessary, reducing unnecessary computation. iii) TON employs a two-stage training strategy: supervised fine-tuning (SFT) with “thought dropout” and group relative policy optimization (GRPO) to maximize task-aware outcome rewards. iv) Experiments show that TON reduces completion length by up to 90% compared to vanilla GRPO without sacrificing performance, and in some cases improving it, along with up to a 17% accuracy improvement on GeoQA. v) TON allows AI practitioners to significantly reduce computational costs in VLMs by adaptively allocating reasoning based on task complexity. |
| AceReason-Nemotron: Advancing Math and Code Reasoning through |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Chankyu Lee, Zihan Liu, Yang Chen, wping, zhuoliny |
i) The paper introduces AceReason-Nemotron, a reinforcement learning approach to enhance math and code reasoning in language models. ii) The research investigates how large-scale RL can improve reasoning capabilities of small- and mid-sized SFT models beyond distillation-based methods. iii) The methodology involves separate math-only and code-only RL training stages, along with robust data curation and curriculum learning with increasing response lengths. iv) AceReason-Nemotron achieves +14.6% / +17.2% improvement on AIME 2025 math benchmark for the 7B / 14B models and +6.8% / +5.8% on LiveCodeBench for 7B / 14B models through math-only RL. v) AI practitioners can leverage this approach to improve reasoning performance in smaller models by employing separate domain-specific RL training stages, particularly math-only RL for cross-domain improvements. |
| VLM-R^3: Region Recognition, Reasoning, and Refinement for Enhanced |
|
|
| Multimodal Chain-of-Thought (Read more on arXiv or HuggingFace) |
Haiyang Xu, Han Yang, Wei Ye, Yongrui Heng, Chaoya Jiang |
i) The paper introduces VLM-R³, a framework enhancing multimodal reasoning in Visual Language Models (MLLMs) through region recognition, reasoning, and refinement. ii) The main objective is to equip MLLMs with the ability to dynamically focus on and revisit visual regions to improve the grounding of textual reasoning in visual evidence. iii) The methodology includes a Region-Conditioned Reinforcement Policy Optimization (R-GRPO) training paradigm and a curated Visuo-Lingual Interleaved Rationale (VLIR) corpus for step-level supervision on region selection and textual justification. iv) The primary result shows VLM-R³ achieves state-of-the-art performance on MathVista, ScienceQA, and other benchmarks, with a 2.2% improvement on MathVista and a 14.33% improvement on ScienceQA. v) The principal implication for AI practitioners is a new benchmark for fine-grained, visually-grounded inference, especially for tasks requiring subtle spatial reasoning or fine-grained visual cue extraction. |
| OViP: Online Vision-Language Preference Learning (Read more on arXiv or HuggingFace) |
Cheng Zeng, Jianxiang Wang, Zejun Li, Siyuan Wang, Shujun Liu |
OViP: Online Vision-Language Preference Learning (OViP) dynamically constructs contrastive training data for large vision-language models (LVLMs) by using the model’s own hallucinated outputs to mitigate misalignment with visual inputs. The research aims to improve LVLM’s faithfulness to visual content by adaptively aligning textual and visual preferences. OViP dynamically constructs contrastive training data using real-time sampling of LVLM outputs and synthesizes negative images using a diffusion model based on semantic differences between response pairs. Experiments show OViP achieves a Hallucination Reduction Index (HRI) of 9.58 on the LLaVA-1.5-7B model, demonstrating reduced hallucinations while preserving multi-modal capabilities. This failure-driven training approach allows AI practitioners to adaptively align both textual and visual preferences, reducing hallucinations in LVLMs more effectively compared to methods relying on static datasets. |
| Reinforcement Learning Finetunes Small Subnetworks in Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Hao Peng, Dilek Hakkani-Tur, Lifan Yuan, sagnikM |
i) Reinforcement learning (RL) in Large Language Models (LLMs) induces parameter update sparsity, affecting only a small subnetwork. ii) This paper investigates the extent and implications of RL-induced parameter update sparsity during LLM finetuning, and if a subnetwork alone can reproduce the full-finetuned model. iii) Publicly released LLM checkpoints finetuned with various RL algorithms were analyzed, measuring update sparsity by comparing parameters before and after RL or SFT, and a subnetwork-only finetuning approach was evaluated. iv) Across different RL algorithms and LLMs, RL finetuning updates only 5%-30% of the parameters, while the rest remain effectively unchanged; Finetuning this subnetwork alone can match or surpass full-model finetuning performance, suggesting the remaining parameters play little role. v) AI practitioners can potentially reduce computational costs in RL-based LLM finetuning by focusing optimization on small, consistently active subnetworks without significant performance degradation, thereby allowing for more efficient resource allocation. |
| Let Androids Dream of Electric Sheep: A Human-like Image Implication |
|
|
| Understanding and Reasoning Framework (Read more on arXiv or HuggingFace) |
Yazhe Niu, Chenhao Zhang |
i) This paper introduces Let Androids Dream (LAD), a novel framework for image implication understanding and reasoning. ii) The main objective is to address the limitations of existing multimodal large language models (MLLMs) in understanding the contextual implications of images. iii) LAD employs a three-stage framework: Perception, Search, and Reasoning, which converts visual information into textual representations, integrates cross-domain knowledge, and generates context-aligned implications via explicit reasoning. iv) Experiments show that LAD achieves state-of-the-art (SOTA) performance on an English image implication benchmark and demonstrates a 68.2% relative improvement on the English Multiple-Choice Question task compared to the GPT-40-mini model. v) LAD provides AI practitioners with a new methodology for enhancing the contextual understanding of images by AI systems through a framework that simulates human-like cognitive processes, potentially improving vision-language reasoning capabilities. |
| SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning (Read more on arXiv or HuggingFace) |
Aosong Feng, Jayanth Srinivasa, Gaowen Liu, Xuandong Zhao, Kaiwen Zhou |
i) SafeKey enhances safety reasoning in Large Reasoning Models (LRMs) against harmful queries and jailbreak attacks. ii) The paper investigates how to improve safety generalization in LRMs, specifically addressing the limitations of supervised fine-tuned models against unseen malicious prompts. iii) The method proposes a “SafeKey” framework with two objectives: a Dual-Path Safety Head to enhance safety signals and Query-Mask Modeling to improve attention on query understanding. iv) Experiments show SafeKey lowers the average harmfulness rate by 9.6% across safety benchmarks, while maintaining general abilities. v) SafeKey provides AI practitioners with a method to reshape internal attention patterns and improve hidden representation quality for more robust safety alignment in LRMs. |
| Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot |
|
|
| Manipulation Datasets (Read more on arXiv or HuggingFace) |
Ken Goldberg, Zehan Ma, Shuangyu Xie, keplerccc |
i) Robo2VLM is introduced as a framework for generating a Visual Question Answering (VQA) dataset from real-world robot manipulation trajectories to evaluate and enhance VLMs. ii) The research aims to improve VLMs’ spatial and interaction reasoning capabilities through a dataset derived from robotic manipulation. iii) The methodology involves segmenting robot trajectories into manipulation phases using proprioceptive and kinematic data to generate VQA pairs with spatial, goal-conditioned, and interaction-based questions. iv) The paper presents Robo2VLM-1, a dataset with 684,710 questions, and shows that fine-tuning LLaVA on it improves spatial and interaction capabilities, with a maximum 50% accuracy gain in state reasoning and task understanding. v) The Robo2VLM-1 dataset provides AI practitioners with a benchmark to evaluate and fine-tune VLMs for enhanced spatial reasoning in robotic manipulation tasks. |
| Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Xiaodong Wang, Xingyu Chen, Hao Tang, Weiyao Wang, Runsen Xu |
Multi-SpatialMLLM introduces a novel framework for enhancing spatial understanding in MLLMs across multiple frames. The research aims to equip MLLMs with robust multi-frame spatial reasoning capabilities. It employs the MultiSPA dataset, a collection of 27 million samples, and a benchmark to test a spectrum of spatial tasks. Multi-SpatialMLLM achieves a 36% average performance gain over baselines and proprietary systems on spatial reasoning tasks. The model’s improved multi-frame spatial reasoning offers AI practitioners an effective tool for advancing robotics and autonomous systems by providing enhanced spatial awareness. The paper lacks information regarding the specific architecture details of Multi-SpatialMLLM. |
| Steering Large Language Models for Machine Translation Personalization (Read more on arXiv or HuggingFace) |
Malvina Nissim, Elisabetta Fersini, Arianna Bisazza, Daniel Scalena, gsarti |
i) This paper explores methods for personalizing large language model (LLM)-based machine translation in low-resource literary settings using prompting and steering techniques. ii) The research aims to develop strategies to steer LLM generations towards a personalized style in machine translation, particularly in the challenging literary domain where stylistic requirements are less explicit. iii) The methodology involves comparing prompt-based approaches with steering techniques that intervene on model internals, utilizing contrastive frameworks with sparse autoencoders (SAEs) to extract salient personalization properties. iv) Results demonstrate that contrastive SAE steering achieves strong personalization while preserving translation quality, achieving between 77% and 99% accuracy in discerning translation styles, and that the learned SAE latents are meaningfully connected to stylistic patterns. v) The principal implication for AI practitioners is the potential of contrastive SAE steering as a data-efficient method to personalize machine translation outputs in low-resource scenarios without compromising translation quality, which can inform development of personalized MT systems, especially in cases of limited style examples. |
| When Do LLMs Admit Their Mistakes? Understanding the Role of Model |
|
|
| Belief in Retraction (Read more on arXiv or HuggingFace) |
Robin Jia, ayyyq |
i) This paper studies when and why large language models (LLMs) retract incorrect answers, defining retraction as acknowledging previous errors. ii) The main research question is to understand the factors influencing LLMs’ decision to retract incorrect answers, specifically examining the role of model belief. iii) The methodology involves constructing model-specific continuation datasets with constraint satisfaction and reversal curse questions, probing LLMs’ internal representations to infer beliefs, and steering model activations to manipulate beliefs. iv) Results show LLMs infrequently retract, retraction is linked to internal belief, and supervised fine-tuning improves retraction performance, achieving up to 84.53% recall on the WIKIDATA dataset after fine-tuning. v) The principal implication for AI practitioners is that aligning LLMs’ internal beliefs with ground truth can significantly enhance the reliability and reduce misinformation risks in LLM applications. |
| Date Fragments: A Hidden Bottleneck of Tokenization for Temporal |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Wei Zhao, Maxime Peyrard, Gagan Bhatia |
i) This paper investigates how date tokenization impacts temporal reasoning in large language models (LLMs). ii) The study aims to quantify the relationship between date fragmentation during tokenization and the accuracy of temporal reasoning tasks. iii) The authors introduced DATEAUGBENCH, a dataset of 6500 examples, and a metric called date fragmentation ratio, using layer-wise probing and causal attention-hop analyses to evaluate LLMs’ ability to handle fragmented dates. iv) Experiments reveal up to a 10-point accuracy drop on uncommon dates due to excessive fragmentation. v) The findings suggest that AI practitioners should consider date-aware vocabularies and adaptive tokenizers to maintain date component integrity, improving the temporal reasoning performance of LLMs in time-sensitive applications. |
| How Do Large Vision-Language Models See Text in Image? Unveiling the |
|
|
| Distinctive Role of OCR Heads (Read more on arXiv or HuggingFace) |
Hwanhee Lee, Sunghyun Ryu, Hwan Chang, Ingeol Baek |
i) This paper investigates the mechanisms by which Large Vision Language Models (LVLMs) process and extract textual information from images, focusing on the role of Optical Character Recognition (OCR) heads. ii) The research aims to identify and characterize the specific attention heads within LVLMs responsible for recognizing and extracting text from images, differentiating them from existing retrieval heads. iii) The methodology involves introducing a scoring-based method to identify OCR heads, analyzing their sparsity, distinctiveness, and activation patterns, and evaluating their behavior in downstream tasks using CoT prompting and attention masking. iv) Results indicate OCR heads are less sparse, qualitatively distinct from retrieval heads, and exhibit static activation patterns, with masking OCR heads causing a performance decline in VQA tasks and a redistribution of the sink token improving performance by up to 0.9% in DocVQA for InternVL-8B. v) The implication for AI practitioners is understanding and manipulating OCR heads within LVLMs can improve OCR-VQA performance, enhancing multimodal reasoning and reducing hallucination in applications involving embedded text. |
| MUG-Eval: A Proxy Evaluation Framework for Multilingual Generation |
|
|
| Capabilities in Any Language (Read more on arXiv or HuggingFace) |
Jiho Jin, Eunsu Kim, Seogyeong Jeong, aliceoh, seyoungsong |
i) MUG-Eval is introduced as a novel, language-agnostic framework for evaluating multilingual text generation in LLMs. ii) The research aims to provide a scalable and reliable method for assessing LLM generation capabilities, particularly in low-resource languages where traditional metrics are limited. iii) The methodology involves transforming existing benchmarks into conversational tasks requiring two LLM instances to communicate in the target language, with algorithmic evaluation of task success. iv) Experiments across 30 languages and 8 LLMs demonstrate strong correlations with established benchmarks (r > 0.75) and indicate effective discriminative power across models and languages. v) MUG-Eval offers AI practitioners a resource-efficient approach for standardized multilingual generation evaluations, facilitating model comparisons across a diverse range of languages without requiring language-specific NLP tools or LLMs-as-judges. |
| SPhyR: Spatial-Physical Reasoning Benchmark on Material Distribution (Read more on arXiv or HuggingFace) |
philippds |
i) The paper introduces SPhyR, a new dataset and benchmark for evaluating spatial-physical reasoning in Large Language Models (LLMs) using topology optimization tasks. ii) The primary objective is to assess LLMs’ ability to reason about optimal material distribution under structural constraints such as boundary conditions, applied forces, and supports. iii) The methodology involves presenting LLMs with 2D topology optimization problems, varying in difficulty from masked region completion to full material distribution prediction, grounded solely in force and support conditions. iv) Experiments with several LLMs (GPT-4.1, Claude 3.7 Sonnet, Gemini 2.5 Pro, and DeepSeek-R1) showed limited ability to reason about global structure; for example, Gemini 2.5 Pro achieved an average exact match of 26.75% on hard tasks. v) The principal implication for AI practitioners is the identification of a significant gap in current LLMs’ ability to integrate spatial layout with physical constraints, suggesting the need for architectures or training strategies incorporating explicit physical priors for engineering and design applications. |
Papers for 2025-05-22
| Title |
Authors |
Summary |
| Web-Shepherd: Advancing PRMs for Reinforcing Web Agents (Read more on arXiv or HuggingFace) |
Seungone Kim, Junhee Cho, donghalim, KimSHine, hyungjoochae |
i) WEB-SHEPHERD is introduced as a process reward model (PRM) for web navigation to assess trajectories in a step-level. ii) The primary objective is to develop a PRM for web navigation that addresses the limitations of using MLLMs as reward models, particularly concerning cost and speed. iii) The methodology involves constructing WEBPRM COLLECTION, a dataset of 40K step-level preference pairs with checklists and introducing WEBREWARDBENCH for evaluating PRMs. iv) Experimental results show that WEB-SHEPHERD achieves approximately 30 points better accuracy than using GPT-40 on WEBREWARDBENCH. v) The key implication is a more cost-effective web navigation trajectory verification strategy for AI practitioners, allowing for 10x lower cost compared to GPT-40-mini when WEB-SHEPHERD is the verifier with GPT-40-mini policy on WebArena-lite with 10.9 points better performance. |
| Scaling Law for Quantization-Aware Training (Read more on arXiv or HuggingFace) |
Zeyue Xue, Yutao Zeng, Jing Liu, Chaoyi Zhang, ChenMnZ |
Quantization-aware training (QAT) scaling laws are explored, focusing on W4A4 quantization. The research addresses the question of how quantization error in QAT scales with model size, training data, and quantization granularity. A unified scaling law is proposed and validated through 268 QAT experiments using Llama3-style models. The results show quantization error decreases with model size but increases with training tokens and coarser quantization granularity; utilizing 8-bit for the FC2 layer input improves W4A4 QAT, reducing quantization error by 42.9% at coarser granularities. These findings suggest AI practitioners should consider both weight and activation quantization error, especially for FC2 layers, to enhance QAT performance in ultra-low bit-width scenarios. |
| UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Jing Tang, Yong Liu, Mingxing Li, Sule Bai, xiaochonglinghu |
i) UniVG-R1 introduces a reasoning-guided MLLM for universal visual grounding, enhancing reasoning capabilities through reinforcement learning. ii) The research aims to improve visual grounding performance, especially for complex instructions across multiple images, by enhancing reasoning capabilities. iii) The methodology involves constructing a high-quality CoT grounding dataset, supervised fine-tuning, and rule-based GRPO reinforcement learning with difficulty-aware weight adjustment. iv) The UniVG-R1 achieves state-of-the-art performance on MIG-Bench with a 9.1% improvement and demonstrates a 23.4% average improvement in zero-shot performance across four image and video reasoning grounding benchmarks. v) AI practitioners can leverage UniVG-R1’s framework to enhance the reasoning and generalization capabilities of MLLMs for visual grounding tasks, particularly in scenarios requiring complex instruction understanding and multi-image reasoning. |
| MMaDA: Multimodal Large Diffusion Language Models (Read more on arXiv or HuggingFace) |
Ke Shen, Bowen Li, Ling Yang, comin, tyfeld |
i) The paper introduces MMaDA, a multimodal diffusion foundation model. ii) The objective is to design a unified multimodal diffusion architecture that achieves superior performance in textual reasoning, multimodal understanding, and text-to-image generation. iii) The methodology involves a unified diffusion architecture, mixed long chain-of-thought (CoT) fine-tuning, and a unified policy-gradient-based reinforcement learning algorithm (UniGRPO). iv) MMaDA-8B surpasses LLaMA-3-7B and Qwen2-7B in textual reasoning; it excels over SDXL and Janus in text-to-image generation. v) The unified architecture and post-training strategies in MMaDA provide AI practitioners with a comprehensive framework for future research in unifying diffusion architectures. |
| Diffusion vs. Autoregressive Language Models: A Text Embedding |
|
|
| Perspective (Read more on arXiv or HuggingFace) |
Anh Tuan Luu, Arman Cohan, LYGeng, yilunzhao, siyue |
i) This paper introduces DIFFEMBED, a diffusion language model-based approach for text embeddings, contrasting it with autoregressive language model (LLM) embeddings. ii) The research investigates whether diffusion language models, with their inherent bidirectional architecture, are better suited for text embedding tasks compared to LLMs that use unidirectional attention. iii) The methodology involves training a diffusion LM (DREAM-7B) and LLMs on a public dataset (Public E5) and a newly created reasoning-intensive dataset (REASONAUG) using contrastive learning. iv) Results show that DIFFEMBED outperforms LLM-based models by 20% on long-document retrieval and 8% on reasoning-intensive retrieval; DIFFEMBED achieves 100% accuracy on passkey retrieval and 86.8% on needle-in-a-haystack tasks in the LONGEMBED benchmark v) AI practitioners can leverage diffusion language models like DIFFEMBED to improve text embedding performance, particularly in applications requiring robust handling of long and complex contexts such as document retrieval and reasoning tasks by implementing bidirectional attention in embedding models. |
| Efficient Agent Training for Computer Use (Read more on arXiv or HuggingFace) |
Pengfei Liu, zizi-0123, henryhe0123 |
i) The paper introduces PC Agent-E, a framework for efficient training of computer use agents using a small dataset augmented with synthesized actions. ii) The research aims to develop an agent training approach that reduces the reliance on large-scale human demonstrations for computer use tasks. iii) The methodology involves augmenting a small set of 312 human-annotated trajectories by synthesizing diverse action decisions using Claude 3.7 Sonnet, followed by supervised fine-tuning. iv) PC Agent-E achieves a 141% relative performance improvement over the Qwen2.5-VL-72B baseline on the WindowsAgentArena-V2 benchmark. v) The research implies that strong computer use capabilities can be achieved with limited high-quality trajectory data, offering a more efficient approach to training computer use agents for AI practitioners. |
| Learn to Reason Efficiently with Adaptive Length-based Reward Shaping (Read more on arXiv or HuggingFace) |
Yuzhen Huang, Yiyun Deng, Ruochen Zhou, yuntian-deng, PeterV09 |
i) The paper introduces LASER-D, an RL method for improving reasoning efficiency in large reasoning models (LRMs). ii) The research investigates how to promote reasoning efficiency in LRMs by dynamically adjusting the length-based reward shaping according to problem difficulty. iii) The methodology involves RL training with a length-based step reward, adaptive target length adjustment, and difficulty-aware reward shaping. iv) Experiments on DeepSeek-R1-Distill models demonstrated a +6.1 accuracy improvement on AIME2024 while reducing token usage by 63%. v) The principal implication is an enhanced method for AI practitioners to improve LRM reasoning performance with greater response length efficiency by using dynamic and difficulty-aware length-based reward shaping. |
| When to Continue Thinking: Adaptive Thinking Mode Switching for |
|
|
| Efficient Reasoning (Read more on arXiv or HuggingFace) |
Haodong Zhao, Yaawennn, Machine981, Amanda2023, DadaCloud01 |
i) This paper introduces Adaptive Self-Recovery Reasoning (ASRR), a framework for dynamically adjusting reasoning length in Large Reasoning Models (LRMs). ii) The research investigates how to reduce computational overhead in LRMs by suppressing unnecessary reasoning while enabling implicit self-recovery. iii) ASRR employs accuracy-aware length reward regulation, conditionally applying length penalties based on group-level accuracy to balance efficiency and correctness. iv) Experiments show ASRR reduces reasoning budget by up to 32.5% (1.5B model) and 25.7% (7B model) with minimal accuracy loss (1.2% and 0.6% pass@1, respectively) and improves harmless rates by +21.7% on safety benchmarks. v) ASRR provides AI practitioners a method to improve LRM efficiency and safety by adaptively allocating reasoning effort based on problem difficulty, reducing computational cost without significantly impacting performance. |
| Vid2World: Crafting Video Diffusion Models to Interactive World Models (Read more on arXiv or HuggingFace) |
Mingsheng Long, Shangchen Miao, Qixing Zhou, manchery, knightnemo |
Vid2World introduces a method to transform pre-trained video diffusion models into interactive world models. The research aims to bridge the gap between video diffusion models and interactive world models by enabling causal generation and action conditioning. The methodology involves causalization of a pre-trained video diffusion model through architectural modifications and a causal action guidance mechanism. Experiments show that Vid2World achieves state-of-the-art performance in video prediction tasks and demonstrated 81.8% relative performance improvement in game simulation. AI practitioners can leverage Vid2World to repurpose highly capable video diffusion models for interactive world modeling, addressing challenges of coarse generation quality and excessive data requirements. |
| IA-T2I: Internet-Augmented Text-to-Image Generation (Read more on arXiv or HuggingFace) |
Yifan Chang, Mingliang Zhai, Yukang Feng, Jianwen Sun, Chuanhao Li |
i) The paper introduces an Internet-Augmented Text-to-Image generation (IA-T2I) framework to improve T2I models’ performance when generating images from text prompts containing uncertain knowledge. ii) The research aims to enhance T2I models by integrating reference images retrieved from the Internet to address scenarios where knowledge implied in text prompts is uncertain, ambiguous, or recently updated. iii) The IA-T2I framework incorporates an active retrieval module, a hierarchical image selection module, and a self-reflection mechanism to retrieve and refine reference images, augmenting the T2I generation process. iv) Experiments using the introduced Img-Ref-T2I dataset demonstrated that the IA-T2I framework outperforms GPT-40 by approximately 30% in human evaluations. v) IA-T2I framework offers AI practitioners a methodology to improve T2I model accuracy by dynamically incorporating external visual information, particularly beneficial when dealing with evolving or ambiguous concepts not adequately represented in the model’s training data. |
| Deliberation on Priors: Trustworthy Reasoning of Large Language Models |
|
|
| on Knowledge Graphs (Read more on arXiv or HuggingFace) |
Jun Liu, Rui Xing, Zhitao Gao, Jie Ma, stillqu |
Deliberation on Priors (DP) is introduced as a framework to improve the trustworthiness of LLM reasoning over knowledge graphs. The paper addresses the challenge of LLMs generating hallucinations due to insufficient knowledge. The methodology involves a progressive knowledge distillation strategy integrating structural priors and a reasoning-introspection strategy incorporating constraint priors. Experiments on WebQuestionsSP, ComplexWebQuestions, and MetaQA datasets show DP achieves state-of-the-art results, including a 13% Hit@1 improvement on ComplexWebQuestions; the paper demonstrates that integrating prior knowledge and constraints enhances the reliability of LLM-generated responses, implying AI practitioners should prioritize incorporating external knowledge and constraint-based verification to improve the trustworthiness of LLM-based systems. |
| lmgame-Bench: How Good are LLMs at Playing Games? (Read more on arXiv or HuggingFace) |
Eric P. Xing, Haoyang Yu, Mingjia Huo, Yuxuan13, Snyhlxde |
i) The paper introduces lmgame-Bench, a benchmark for evaluating LLMs in video game playing. ii) The main research question addresses whether video game environments can effectively evaluate the perception, memory, and planning capabilities of LLMs, and how to mitigate common challenges. iii) The methodology involves creating a Gym-style API for platformer, puzzle, and narrative games, combined with lightweight perception and memory scaffolds, contamination mitigation techniques, and standardized prompt optimization. iv) Results show that using Imgame-Bench can yield an 86.7% game run success rate which is beyond using harness or without any support in distinguishing model performance, while the standardized prompt optimization reduces performance variance across different empirically optimized initializations by 33.8% to 63.5%. v) Imgame-Bench provides AI practitioners with a more reliable and informative evaluation environment, highlighting the importance of gaming harnesses, contamination control, and prompt tuning for LLM agent performance in interactive settings. The paper also indicates that RL training in-game transfers well to planning and agentic tasks. It’s unclear whether all 13 models trained with RL and assessed. |
| Constructing a 3D Town from a Single Image (Read more on arXiv or HuggingFace) |
Xin Eric Wang, Jie Yang, Jing Gu, Ruijian Zhang, Kaizhi Zheng |
i) This paper introduces 3DTown, a training-free framework for generating coherent 3D scenes from a single top-down image. ii) The research aims to synthesize realistic and geometrically consistent 3D scenes from a single image without requiring 3D training data or fine-tuning. iii) The method employs a region-based generation strategy and spatial-aware 3D inpainting using pretrained object generators and masked rectified flow. iv) Experiments demonstrate that 3DTown outperforms state-of-the-art baselines, achieving a GPT-40-based texture win rate of 92.3% versus 7.7% for Hunyuan3D-2. v) The primary implication for AI practitioners is the demonstration of a modular, training-free approach to 3D scene synthesis that overcomes resolution bottlenecks and geometry inconsistencies, offering a potentially scalable method for generating structured 3D environments from minimal input. |
| dKV-Cache: The Cache for Diffusion Language Models (Read more on arXiv or HuggingFace) |
Xinchao Wang, Gongfan Fang, Runpeng Yu, Xinyin Ma |
i) The paper introduces dKV-Cache, a delayed key-value caching mechanism to accelerate inference in Diffusion Language Models (DLMs). ii) The research aims to address the inference inefficiency of DLMs by adapting the KV-cache technique used in autoregressive models. iii) The methodology involves a delayed caching strategy for key and value states, implemented in two variants: dKV-Cache-Decode and dKV-Cache-Greedy. iv) Experiments on LLaDA and Dream-Base-7B models demonstrate 2-10x inference speedup with minimal performance degradation using dKV-Cache. v) dKV-Cache provides AI practitioners a training-free method to accelerate DLM inference, potentially narrowing the performance gap between DLMs and autoregressive models. |
| How Should We Enhance the Safety of Large Reasoning Models: An Empirical |
|
|
| Study (Read more on arXiv or HuggingFace) |
Qi Zhu, Victor Shea-Jay Huang, Xian Qi Loye, Zhexin Zhang, yangjunxiao2021 |
i) This paper empirically investigates methods for enhancing the safety of Large Reasoning Models (LRMs) through Supervised Fine-Tuning (SFT). ii) The main research question explores how to improve the safety performance of LRMs without compromising reasoning capabilities. iii) The methodology involves analyzing failure patterns in distilled safe responses, modifying prompting strategies, and comparing different reasoning processes (short, template-based, and long-form CoT). iv) Results show directly distilling safe responses fails to significantly enhance safety, identifying lack of safety awareness, overthinking, and inconsistency as key failure patterns; explicitly addressing these reduces the Attack Success Rate (ASR) of PAIR from 77.0% to 7.0%. v) The findings suggest that simpler reasoning processes can be as effective as longer chains, easing the learning process for models and including benign reasoning data can balance safety and over-refusal; however, there is limited information on the impact of this method on larger models. |
| Learning to Reason via Mixture-of-Thought for Logical Reasoning (Read more on arXiv or HuggingFace) |
Heng Huang, R. Thomas McCoy, Simeng Han, Lichang Chen, TongZheng1999 |
i) This paper introduces Mixture-of-Thought (MoT), a framework enabling LLMs to reason across modalities like natural language, code, and truth-table reasoning for logical tasks. ii) The research aims to improve LLMs’ logical reasoning capabilities by training them to utilize multiple reasoning modalities synergistically. iii) MoT employs a two-phase approach: self-evolving MoT training using filtered, self-generated rationales across modalities, and MoT inference which leverages the synergy of the modalities. iv) Experiments on FOLIO and ProofWriter show MoT outperforms single-modality baselines, achieving up to +11.7pp average accuracy gain, and demonstrate that a 9B MoT model matches GPT-4 + Logic-LM performance on FOLIO. v) The MoT framework provides AI practitioners with a method for improving logical reasoning in LLMs by combining multiple reasoning modalities which can be achieved through a two-phase approach of MoT training and MoT inference. |
| Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data |
|
|
| Could Be Secretly Stolen! (Read more on arXiv or HuggingFace) |
Hongning Wang, Shiyao Cui, Yuhao Sun, Zhexin Zhang, yangjunxiao2021 |
Fine-tuning open-source LLMs can create a risk of downstream data extraction via backdoor training. The research investigates the potential for creators of open-source LLMs to extract fine-tuning data from downstream users. It uses supervised fine-tuning (SFT) and reinforcement learning to inject backdoors that trigger query reproduction. Experiments across four models show up to 76.3% of fine-tuning data can be extracted in practical settings, increasing to 94.9% in more ideal conditions. This poses a data breaching risk for AI practitioners who fine-tune open-source LLMs with proprietary data, requiring enhanced security measures. |
| RLVR-World: Training World Models with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Mingsheng Long, Ningya Feng, Shaofeng Yin, manchery |
i) RLVR-World is introduced as a framework to optimize world models via reinforcement learning with verifiable rewards (RLVR). ii) The research aims to improve world models by directly optimizing task-specific metrics rather than surrogate objectives like maximum likelihood estimation (MLE). iii) The method involves tokenizing states and actions as sequences and using verifiable rewards based on decoded predictions for RLVR. iv) Experiments show RLVR improves LLMs, achieving +30.7% accuracy on text-based game state prediction and improves video world models with a +9.2% relative LPIPS improvement on robot manipulation, even with limited RLVR gradient steps. v) RLVR offers AI practitioners a post-training technique to refine generative models by directly optimizing for specific task-aligned metrics, enhancing utility beyond pre-training. |
| Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous |
|
|
| Concept Space (Read more on arXiv or HuggingFace) |
Chenyang Zhao, Ao Shen, Weixiang Yan, Xuehai He, Zhen Zhang |
i) The paper introduces Soft Thinking, a training-free method that improves LLM reasoning by operating in a continuous concept space of probability-weighted token embeddings. ii) The research aims to unlock the reasoning potential of LLMs by enabling soft, abstract concept manipulation beyond discrete language tokens. iii) The methodology involves replacing discrete token selection in Chain-of-Thought prompting with probabilistic soft aggregation over the entire vocabulary, forming a continuous concept space. iv) Soft Thinking improves pass@1 accuracy by up to 2.48 points on mathematical and coding benchmarks while reducing token usage by up to 22.4% compared to standard Chain-of-Thought. v) AI practitioners can leverage Soft Thinking as a drop-in replacement for Chain-of-Thought prompting to improve both accuracy and efficiency of LLMs without additional training. |
| ConvSearch-R1: Enhancing Query Reformulation for Conversational Search |
|
|
| with Reasoning via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Xipeng Qiu, Kai Song, Ruijun Feng, Siyin Wang, BeastyZ |
i) This paper introduces ConvSearch-R1, a novel self-driven framework for conversational query reformulation (CQR) using reinforcement learning, eliminating the need for external rewrite supervision. ii) The research objective is to align query reformulation models with downstream retrievers effectively without human-annotated rewrites. iii) The methodology employs a two-stage approach: Self-Driven Policy Warm-Up (SD-PWU) through retrieval-guided self-distillation, and Retrieval-Guided Reinforcement Learning (RL) with a rank-incentive reward shaping mechanism. iv) Experiments on TopiOCQA demonstrate ConvSearch-R1 achieves over 10% average improvement across metrics compared to previous state-of-the-art results with 3B parameter models and no external supervision. v) ConvSearch-R1 provides AI practitioners with a self-supervised CQR framework, reducing annotation costs and enhancing retrieval performance by aligning query reformulation with retriever ranking signals. |
| BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs (Read more on arXiv or HuggingFace) |
Chujie Zheng, Xiaoce Wang, Haoran Liu, Jinzhe Tu, yangjunxiao2021 |
i) The paper introduces BARREL, a framework to improve the factual reliability of Large Reasoning Models (LRMs) by promoting boundary-aware reasoning. ii) The main research question is how to mitigate pathological reasoning patterns leading to overconfident and incorrect answers in LRMs. iii) The methodology involves BARREL-training, comprising knowledge labeling, reasoning trace construction using Supervised Fine-Tuning (SFT), and Group Relative Policy Optimization (GRPO). iv) Experiments showed that BARREL-training increased the reliability of DeepSeek-R1-Distill-Llama-8B from 39.33% to 61.48%, while maintaining a competitive accuracy of 40.7%. v) For AI practitioners, BARREL offers a technique for enhancing the factual reliability of LRMs by promoting uncertainty-aware refusal, which can be integrated into existing training pipelines to develop more trustworthy systems. |
| This Time is Different: An Observability Perspective on Time Series |
|
|
| Foundation Models (Read more on arXiv or HuggingFace) |
Chris Lettieri, Salahidine Lemaachi, Youssef Doubli, Emaad Khwaja, Ben Cohen |
i) The paper introduces TOTO, a 151-million-parameter time series forecasting foundation model, and BOOM, a large-scale observability benchmark dataset. ii) The main objective is to develop a time series foundation model optimized for observability metrics and to provide a benchmark for evaluating such models. iii) TOTO employs a decoder-only architecture with causal normalization, patch embedding, proportional factorized attention, and a Student-T mixture model head, pre-trained on observability data, public datasets, and synthetic data. iv) Evaluations show TOTO achieves state-of-the-art performance on both BOOM and general-purpose benchmarks, with a 12% improvement in CRPS on BOOM compared to other methods. v) The principal implication for AI practitioners is the availability of an open-source foundation model and a benchmark tailored to observability data, enhancing zero-shot forecasting capabilities for monitoring and anomaly detection in distributed systems. |
| Text Generation Beyond Discrete Token Sampling (Read more on arXiv or HuggingFace) |
Jianfeng Gao, Jingbo Shang, Chandan Singh, Liyuan Liu, Yufan Zhuang |
i) The paper introduces Mixture of Inputs (MOI), a training-free method to enhance autoregressive language models by preserving token distribution information. ii) The main objective is to improve text quality and reasoning capabilities in LLMs by modifying the input to incorporate the distribution of predicted tokens, rather than solely the sampled token. iii) The methodology involves a Bayesian estimation approach that blends the generated discrete token with the previously discarded token distribution using posterior expectation. iv) MOI achieves consistent performance improvements across tasks, demonstrated by a +2.36% average absolute gain for Nemotron-Super-49B, with the largest improvement on GPQA-Diamond (+4.1%). v) MOI offers AI practitioners a way to improve reasoning tasks without retraining, by combining discrete choices with probabilistic contexts to enhance accuracy without sacrificing decoding efficiency. |
| AutoMat: Enabling Automated Crystal Structure Reconstruction from |
|
|
| Microscopy via Agentic Tool Use (Read more on arXiv or HuggingFace) |
Jiangjie Qiu, Xiao Chen, Yizhe Chen, IvanTang, yaotianvector |
AutoMat is an agent-assisted pipeline for automated crystal structure reconstruction from STEM images. The research aims to automate the transformation of STEM images into simulation-ready crystal structures and predict their physical properties. It employs a pattern-adaptive denoising network (MOE-DIVAESR), physics-guided template retrieval, symmetry-aware atomic reconstruction, fast relaxation via MatterSim, and orchestrates these tools with a text-only LLM. AutoMat achieves a projected lattice RMSD of 0.11 ± 0.03 Å and energy MAE below 350 meV/atom, outperforming vision-language models and domain-specific tools. AutoMat provides a framework enabling AI practitioners to generate reliable atomistic structures from microscopy data for ML model training and validation. |
| VARD: Efficient and Dense Fine-Tuning for Diffusion Models with |
|
|
| Value-based RL (Read more on arXiv or HuggingFace) |
Bangyan Liao, Siteng Huang, Yufei Huang, Zifeng Zhuang, Fengyuan Dai |
i) The paper introduces VARD, a value-based reinforcement learning approach for efficient and stable fine-tuning of diffusion models, particularly with non-differentiable rewards. ii) The main objective is to improve the training efficiency and stability of diffusion models when fine-tuning them for specific desirable properties, especially in scenarios with non-differentiable rewards. iii) VARD learns a process reward model (PRM) akin to a value function in RL, to provide dense, differentiable supervision signals throughout the diffusion trajectory, supplemented by KL regularization to maintain proximity to the pre-trained model. iv) Experiments demonstrate that VARD achieves better trajectory guidance, leading to faster convergence and improved sample quality, extending RL applicability to complex non-differentiable reward functions; VARD w/o KL and VARD exhibit steeper growth trajectories than baselines with respect to reward. v) VARD provides AI practitioners with a method for stable and efficient fine-tuning of diffusion models using potentially non-differentiable rewards, offering enhanced sample quality and trajectory control. |
| RL Tango: Reinforcing Generator and Verifier Together for Language |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Duane S. Boning, Zhang-Wei Hong, Maohao Shen, Zhengqi Gao, sunshinekevin |
i) The paper introduces TANGO, a reinforcement learning (RL) framework for jointly training an LLM generator and a generative LLM verifier in an interleaved manner to improve language reasoning. ii) The primary objective is to overcome limitations of fixed or discriminatively trained verifiers in existing RL post-training methods for LLMs, which are susceptible to reward hacking and poor generalization. iii) TANGO uses RL to concurrently train both an LLM generator and a process-level generative LLM verifier based solely on outcome-level verification correctness rewards without explicit process-level annotations. iv) Experiments demonstrate TANGO achieves state-of-the-art results among 7B/8B-scale models, with the generator attaining best-in-class performance across five competition-level math benchmarks and the verifier leading on the ProcessBench dataset; TANGO with GRPO doubles the accuracy on the most challenging benchmark, AIME 2025, relative to vanilla GRPO. v) The principal implication for AI practitioners is that co-evolving generator and verifier components in RL frameworks can lead to improved reasoning capabilities and generalization, offering a more robust alternative to relying on fixed or SFT-trained verifiers in LLM post-training. |
| Streamline Without Sacrifice - Squeeze out Computation Redundancy in LMM (Read more on arXiv or HuggingFace) |
Ziwei Liu, Lewei Lu, Penghao Wu |
i) This paper introduces ProxyV, a method to reduce computation in Large Multimodal Models (LMMs) by using proxy vision tokens. ii) The research investigates how to reduce computation associated with vision tokens in decoder-only LMMs without sacrificing performance. iii) The method involves downsampling original vision tokens to create proxy tokens, processing these proxy tokens through self-attention and feed-forward networks, and then using them to guide updates of the original vision tokens. iv) ProxyV reduces prefilling FLOPs and time by 43% and 40%, respectively while achieving 101% performance on fine-grained benchmarks when applied to Vicuna1.5-7B. v) AI practitioners can use ProxyV to enhance the efficiency of LMMs in scenarios with long visual sequences, as it effectively mitigates the computational burden of vision tokens without compromising performance, and the paper also proposes a non-spatial variant of ProxyV which can be seamlessly integrated with token reduction methods to further enhance efficiency. |
| Evaluate Bias without Manual Test Sets: A Concept Representation |
|
|
| Perspective for LLMs (Read more on arXiv or HuggingFace) |
Zirui Song, Chenxi Wang, Wei Liu, Kaiyang Wan, Lang Gao |
i) This paper introduces BIASLENS, a test-set-free framework for evaluating biases in Large Language Models (LLMs) by analyzing concept representations. ii) The research aims to overcome the limitations of existing bias evaluation methods by shifting the focus from behavioral differences to conceptual representations within LLMs. iii) BIASLENS combines Concept Activation Vectors (CAVs) and Sparse Autoencoders (SAEs) to extract and compare interpretable concept representations, quantifying bias via representational similarity between target and reference concepts. iv) Experiments demonstrate BIASLENS achieves a Spearman correlation r > 0.85 with traditional bias evaluation metrics and uncovers biases not easily detectable by existing methods. v) BIASLENS provides AI practitioners with a scalable and efficient methodology for bias discovery in LLMs, facilitating improvements in fairness and transparency without requiring manual test set creation. |
| PiFlow: Principle-aware Scientific Discovery with Multi-Agent |
|
|
| Collaboration (Read more on arXiv or HuggingFace) |
Hongyu Chen, Tao Lin, Yingming Pu |
i) PiFlow is presented as a framework for structured uncertainty reduction in automated scientific discovery using LLM-based multi-agent systems. ii) The paper addresses the research question of how to improve scientific discovery efficiency and solution quality by incorporating scientific principles into the hypothesis generation process. iii) The method employs an information-theoretical approach with Min-Max optimization to select high-potential scientific principles for guiding hypothesis generation, validation, and refinement. iv) Evaluations across three scientific domains show PiFlow achieves a 73.55% increase in the AUC of property values versus exploration steps and a 94.06% improvement in solution quality compared to a vanilla agent system. v) PiFlow offers AI practitioners a plug-and-play method for enhancing scientific discovery MAS, potentially leading to more efficient exploration of complex search spaces. |
| Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large |
|
|
| Audio-Language Models (Read more on arXiv or HuggingFace) |
Lang Gao, Mingzhe Li, Mingxuan Cui, Qian Jiang, Zirui Song |
i) This paper introduces AJailBench, a benchmark for evaluating jailbreak vulnerabilities in Large Audio-Language Models (LAMs). ii) The main objective is to provide a systematic, quantitative evaluation of LAM safety against adversarial audio prompts. iii) The methodology involves constructing a dataset of adversarial audio prompts converted from textual jailbreak attacks and creating an Audio Perturbation Toolkit (APT) to generate dynamic adversarial variants, using Bayesian optimization under semantic consistency constraints. iv) The results indicate that state-of-the-art LAMs exhibit vulnerabilities to both static and optimized adversarial audio inputs, with even small, semantically preserved perturbations significantly reducing safety performance, and no single model demonstrating robust performance across all attacks. v) AI practitioners should be aware that current LAMs are vulnerable to audio jailbreak attacks that can bypass safety mechanisms, requiring more robust and semantically aware defense strategies and that signal-level manipulations can be a key attack vector. |
| WebNovelBench: Placing LLM Novelists on the Web Novel Distribution (Read more on arXiv or HuggingFace) |
Haidong Wang, Jun Zheng, Leon Lin |
i) WebNovelBench is introduced as a new benchmark for evaluating long-form story generation by Large Language Models (LLMs). ii) The research aims to establish a comprehensive and objective methodology for assessing and ranking LLMs’ storytelling capabilities relative to human-authored works. iii) The methodology involves a synopsis-to-story generation task using a dataset of over 4,000 Chinese web novels and an LLM-as-Judge approach evaluating eight narrative dimensions. iv) Experiments involving 24 LLMs demonstrate effective differentiation between human-written and LLM-generated content, with top models achieving norm scores up to 5.21, indicating strong alignment with high-quality human writing. v) WebNovelBench provides AI practitioners with a scalable, replicable, and data-driven framework for assessing and advancing LLM-driven narrative generation, enabling standardized comparisons and insights for future model development. |
| Prior Prompt Engineering for Reinforcement Fine-Tuning (Read more on arXiv or HuggingFace) |
Sarana Nutanong, Potsawee Manakul, kunato, pittawat |
i) This paper explores the influence of different prior prompt engineering (pPE) approaches in reinforcement fine-tuning (RFT) of language models. ii) The research investigates whether different pPE strategies can guide language models to internalize distinct behaviors after RFT. iii) Five representative inference-time prompt engineering (iPE) strategies were translated into corresponding pPE approaches and used to train Qwen2.5-7B models with math-only data, followed by quantitative and qualitative evaluations. iv) Results show that all pPE-trained models surpassed their iPE-prompted baselines, with the null-example pPE approach achieving the largest average performance gain and highest improvement on GPQA-Diamond; training dynamics were largely similar across pPE variants. v) The findings demonstrate that pPE is a powerful yet understudied axis for RFT, allowing AI practitioners a way to incentivize diverse behaviors without changing algorithms, reward shaping, or data curation. |
| Language Specific Knowledge: Do Models Know Better in X than in English? (Read more on arXiv or HuggingFace) |
Dilek Hakkani-Tür, Nimet Beyza Bozdag, Ishika Agarwal |
i) This paper introduces and investigates Language Specific Knowledge (LSK) in language models, exploring whether models exhibit better performance in certain languages for specific topics. ii) The research aims to determine if multilingual language models possess varying degrees of expertise across languages for different knowledge domains, and if this can be leveraged to improve reasoning. iii) The methodology involves a two-stage framework called LSKEXTRACTOR: mapping LSK by conducting chain-of-thought (CoT) reasoning in 13 languages on culture-specific datasets, and LSK-informed reasoning using the identified expert languages during inference. iv) The experiments show an average relative improvement of 10% in accuracy by using LSKEXTRACTOR across various models and datasets. v) The principal implication is that AI practitioners can enhance the performance of language models by strategically incorporating code-switching to leverage language-specific knowledge identified through the LSKEXTRACTOR framework for improved accuracy and cultural alignment. |
| MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation |
|
|
| of LLM Hallucinations (Read more on arXiv or HuggingFace) |
Johannes Bjerva, Katja Hose, Russa Biswas, ernlavr |
i) MultiHal, a new multilingual benchmark for evaluating LLM hallucinations, is introduced. ii) The research aims to provide a knowledge graph-grounded, multilingual testbed for generative text evaluation to address limitations in current factuality benchmarks. iii) The methodology involves mining 140k KG-paths from open-domain KGs, pruning them to 25.9k high-quality paths, and translating question-answer pairs with KG paths into five languages. iv) Baseline evaluation shows a 0.12 to 0.36 point increase in semantic similarity scores using KG-RAG over vanilla QA in multiple languages and models. v) MultiHal facilitates future research into graph-based hallucination mitigation and fact-checking tasks for improving LLM faithfulness and KG integration for AI developers. |
| HumaniBench: A Human-Centric Framework for Large Multimodal Models |
|
|
| Evaluation (Read more on arXiv or HuggingFace) |
Mukund S. Chettiar, Ashmal Vayani, Vahid Reza Khazaie, Aravind Narayanan, shainaraza |
HumaniBench introduces a new benchmark for evaluating Large Multimodal Models (LMMs) on human-centered criteria. The research aims to provide a holistic assessment of LMMs regarding fairness, ethics, understanding, reasoning, language inclusivity, empathy, and robustness. The methodology involves curating a dataset of 32K real-world image-question pairs, annotated using a GPT-4o-assisted pipeline and verified by domain experts, across seven diverse tasks. Benchmarking 15 LMMs revealed that proprietary models generally perform better, but gaps remain in robustness and visual grounding; Qwen2.5-7B achieved 84.87% in Understanding on particular tasks. The benchmark facilitates diagnosing alignment gaps and steering LMMs toward accurate and socially responsible behavior, offering AI practitioners a rigorous test-bed for optimizing LMMs for human values. |
Papers for 2025-05-21
| Title |
Authors |
Summary |
| Emerging Properties in Unified Multimodal Pretraining (Read more on arXiv or HuggingFace) |
Ziang, codecaution, whyu, gouc, Andy1621 |
i) The paper introduces BAGEL, an open-source multimodal foundation model for understanding and generation. ii) The main objective is to create a unified model that natively supports both multimodal understanding and generation through pretraining on diverse interleaved data. iii) The methodology involves pretraining a decoder-only model on trillions of tokens curated from interleaved text, image, video, and web data using a Mixture-of-Transformer-Experts architecture. iv) BAGEL outperforms open-source VLMs on multimodal understanding benchmarks and achieves text-to-image quality competitive with state-of-the-art generators. v) AI practitioners can utilize BAGEL as a foundational model for developing advanced multimodal applications, leveraging its capabilities in tasks such as free-form image manipulation, future frame prediction, and 3D manipulation. |
| SageAttention3: Microscaling FP4 Attention for Inference and An |
|
|
| Exploration of 8-Bit Training (Read more on arXiv or HuggingFace) |
surfingtomchen, whx1003, haofeng666, Guyan, jt-zhang |
i) This paper introduces SageAttention3, an FP4 attention mechanism for inference acceleration, and explores 8-bit attention (SageBwd) for training. ii) The primary objective is to enhance the efficiency of attention mechanisms through low-bit quantization for both inference and training tasks. iii) The methodology involves implementing FP4 microscaling quantization for inference and designing a trainable 8-bit attention mechanism for forward and backward propagation in training. iv) SageAttention3 achieves 1038 TOPS on RTX5090, a 5x speedup over FlashAttention for inference; SageBwd achieves lossless performance in fine-tuning tasks but slower convergence in pretraining. v) The FP4 attention and the 8-bit training exploration offer AI practitioners new approaches to accelerate inference and fine-tuning, respectively, for large models, although the suitability for pre-training tasks needs further investigation. |
| VisualQuality-R1: Reasoning-Induced Image Quality Assessment via |
|
|
| Reinforcement Learning to Rank (Read more on arXiv or HuggingFace) |
Kede Ma, Lei Zhang, Jie Liang, Jian Zou, TianheWu |
VisualQuality-R1 introduces a reasoning-induced no-reference image quality assessment model trained via reinforcement learning to rank. The paper aims to improve image quality assessment by leveraging reasoning capabilities and addressing the relative nature of visual quality. Group relative policy optimization is used to generate multiple quality scores, and comparative probabilities are calculated using the Thurstone model. Experiments show VisualQuality-R1 outperforms existing models, achieving an average SRCC of 0.791 and PLCC of 0.831 across KADID-10K and SPAQ datasets in multi-dataset training. These results indicate an enhanced ability to generalize across distortion scenarios and provide contextually rich quality descriptions. The code for the project is available at https://github.com/TianheWu/VisualQuality-R1. |
| Visual Agentic Reinforcement Fine-Tuning (Read more on arXiv or HuggingFace) |
sweetFruit, steins1096, zyshan, yuhangzang, ziyuliu |
Visual-ARFT is presented as a method for improving reasoning in Large Vision-Language Models (LVLMs) through reinforcement fine-tuning with external tools. The research aims to enhance LVLMs’ capabilities to use tools like web browsers for searching and code execution for image manipulation. Visual-ARFT uses a reward-driven training strategy and the Group Relative Policy Optimization (GRPO) algorithm. Experiments show Visual-ARFT outperforms baselines, achieving +18.6% F1 / +13.0% EM on the MAT-Coding benchmark and surpasses GPT-40 on this task, furthermore, it demonstrates generalization with +29.3% F1 / +25.9% EM on multi-hop QA benchmarks. Visual-ARFT offers AI practitioners a method for building more robust and generalizable multimodal agents. |
| The Aloe Family Recipe for Open and Specialized Healthcare LLMs (Read more on arXiv or HuggingFace) |
annariasdu, pabberpe, danihinjos, adriantormos, JordiBayarri-bsc |
This paper introduces Aloe Beta, an open-source family of healthcare Large Language Models (LLMs). The research explores optimization of data preprocessing and training to create competitive medical LLMs, including model safety and efficacy enhancements. Key methods involve custom datasets, Direct Preference Optimization (DPO), and Retrieval-Augmented Generation (RAG). Results show competitive performance on healthcare benchmarks, with enhanced safety against jailbreaking attacks; the Qwen2.5-Aloe-Beta-72B model achieves top performance among open models on MCQA tasks. Aloe Beta offers a top-performing, ethically aligned open-source option for AI practitioners in healthcare, with the models and datasets made available under a permissive license. |
| Optimizing Anytime Reasoning via Budget Relative Policy Optimization (Read more on arXiv or HuggingFace) |
Wee Sun Lee, duchao, P2333, lkevinzc, QPHutu |
i) This paper introduces AnytimeReasoner, a framework to optimize anytime reasoning performance in large language models (LLMs) by maximizing rewards under varying token budget constraints. ii) The primary research objective is to improve token efficiency and reasoning flexibility of LLMs under different resource constraints. iii) The key methodology involves truncating the thinking process at sampled token budgets, introducing verifiable dense rewards and employing Budget Relative Policy Optimization (BRPO) to improve advantage estimation. iv) Empirical results on mathematical reasoning tasks demonstrate AnytimeReasoner consistently outperforms GRPO across all thinking budgets and enhances both training and token efficiency; for AIME2024 there is an accuracy of 32.7% compared to MRT’s reported 30.3%. v) The principal implication for AI practitioners is a more efficient and flexible approach for deploying LLMs in resource-constrained environments, where performance must be maintained even with limited computational budgets. |
| Latent Flow Transformer (Read more on arXiv or HuggingFace) |
Pei-Chen Ho, dsshiu, menghsichen, FengTing, yenchen |
i) The paper introduces the Latent Flow Transformer (LFT) for efficient language modeling, replacing transformer blocks with learned transport operators trained via flow matching. ii) The research objective is to reduce the parameter and compute cost of large language models (LLMs) by compressing layers using flow-based methods. iii) The methodology involves training a single transformer-like layer to learn a velocity field that maps latent states across multiple transformer layers, guided by a novel “Recoupling Ratio” metric for layer selection, with the proposed Flow Walking (FW) algorithm for trajectory learning. iv) The experiments on Pythia-410M demonstrate that LFT trained with flow matching can compress 6 of 24 layers and achieve a KL divergence of LM logits at 0.407, outperforming skipping 2 layers (KL divergence of 0.529). v) LFT provides AI practitioners with a new structural compression technique that leverages flow-based learning for efficient LLM design, potentially reducing model size while retaining performance. |
| Neurosymbolic Diffusion Models (Read more on arXiv or HuggingFace) |
Antonio Vergari, ducdauge, pminervini, HEmile |
i) This paper introduces neurosymbolic diffusion models (NESYDMs) to address limitations of independence assumptions in neurosymbolic predictors. ii) The research objective is to develop a neurosymbolic predictor that models dependencies between extracted symbols to improve uncertainty quantification and out-of-distribution generalization. iii) The methodology involves a discrete diffusion process that reuses the independence assumption from NeSy predictors at each diffusion step, enabling scalable learning while modeling symbol dependencies, which is trained with a derived continuous-time loss function. iv) Results on visual path planning demonstrate that NESYDMs achieve a state-of-the-art accuracy of 97.40% on a 30x30 grid, surpassing existing NeSy predictors, and demonstrate strong calibration on RSBench tasks. v) AI practitioners can leverage NESYDMs to build more reliable and generalizable AI systems by modeling dependencies between symbols in neurosymbolic reasoning tasks, particularly in safety-critical applications. |
| Visionary-R1: Mitigating Shortcuts in Visual Reasoning with |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yixuan Li, Peng Gao, kaiyangzhou, yuhangzang, Jiaer-Xia |
Visionary-R1 introduces a reinforcement learning framework to improve visual reasoning in VLMs by mitigating shortcut learning. The research aims to train VLMs, using only question-answer pairs and reinforcement learning, to perform reasoning on image data without explicit chain-of-thought supervision. The methodology involves training the model to generate a detailed image caption before reasoning and answering (caption-reason-answer). Experiments demonstrate that Visionary-R1 achieves improved performance, outperforming models like GPT-40 on MathVista by 7.9%. The principal implication is that enforcing visual understanding through captioning prior to reasoning enhances the generalizability of VLMs, offering a method for AI practitioners to develop more robust visual reasoning systems. |
| General-Reasoner: Advancing LLM Reasoning Across All Domains (Read more on arXiv or HuggingFace) |
wenhu, zhangysk, DongfuJiang, SivilTaram, MrLight |
i) The paper introduces GENERAL-REASONER, a new training paradigm to enhance LLM reasoning across diverse domains beyond mathematics and coding. ii) The main objective is to improve LLM reasoning capabilities in domains with diverse answer representations and limited data. iii) The methodology involves constructing a large-scale dataset of verifiable questions from web crawling and developing a generative model-based answer verifier. iv) Evaluation across 12 benchmarks shows GENERAL-REASONER outperforms baselines, improving MMLU-Pro and SuperGPQA performance by approximately 10%, while preserving mathematical reasoning capabilities. v) The primary implication for AI practitioners is a robust and generalizable LLM reasoning framework that extends beyond traditional mathematical and coding domains, improving model accuracy across a broader range of real-world reasoning tasks. |
| Reasoning Models Better Express Their Confidence (Read more on arXiv or HuggingFace) |
YongilKim, Sunkyoung, soheeyang, seungone, DKYoon |
i) Reasoning models exhibit superior confidence calibration compared to non-reasoning models due to slow thinking behaviors. ii) This work investigates whether reasoning models communicate their confidence accurately, specifically if slow thinking behaviors enhance confidence calibration. iii) The study benchmarks six reasoning models against non-reasoning counterparts across six datasets, measuring Expected Calibration Error (ECE), Brier Score, and AUROC. iv) Reasoning models achieved strictly better confidence calibration than non-reasoning models in 33 out of 36 settings; R1-Distill-Qwen exhibits near-perfect calibration above 60% confidence on TriviaQA. v) AI practitioners should consider reasoning models for tasks requiring reliable confidence estimation, as slow thinking enhances the alignment between predicted confidence and actual accuracy, thus potentially increasing trust and reliability of AI systems. |
| Exploring Federated Pruning for Large Language Models (Read more on arXiv or HuggingFace) |
Liangqiong-QU, limingcv, MENGTINGLIU, jcccy, gpx333 |
i) The paper introduces FedPrLLM, a federated learning framework for privacy-preserving pruning of large language models (LLMs). ii) The main objective is to address the challenge of pruning LLMs in privacy-sensitive domains without requiring access to public calibration samples. iii) The methodology involves clients calculating a pruning mask matrix based on local calibration data and sharing it with the server, which then aggregates these matrices to prune the global model. iv) Experiments demonstrate that one-shot pruning with layer comparison and no weight scaling is the optimal choice within the FedPrLLM framework, achieving comparable performance to iterative pruning while reducing communication costs. v) The findings suggest that layer comparison is a simple yet effective method for parameter comparison in federated pruning scenarios, indicating a practical approach for deploying compressed LLMs in privacy-sensitive applications. |
| Reasoning Path Compression: Compressing Generation Trajectories for |
|
|
| Efficient LLM Reasoning (Read more on arXiv or HuggingFace) |
Jae-Joon Kim, YulhwaKim, dongwonjo, jiwonsong |
Reasoning Path Compression (RPC) accelerates inference in reasoning-focused Large Language Models (LLMs) by compressing KV caches. The paper addresses the problem of increased memory usage and reduced throughput in LLMs due to long reasoning paths. RPC periodically compresses the KV cache, retaining high-importance scores, calculated using a selector window of recent queries. Experiments with QwQ-32B show up to 1.60× improvement in generation throughput with a 1.2% accuracy drop on the AIME 2024 benchmark. This method’s implication is that exploiting semantic sparsity in reasoning traces can improve LLM deployment efficiency, offering a training-free method that is straightforward to integrate into existing pipelines. |
| NExT-Search: Rebuilding User Feedback Ecosystem for Generative AI Search (Read more on arXiv or HuggingFace) |
Wenjie Wang, chuats, jrwen, pl8787, KID-22 |
i) This paper proposes NExT-Search, a new paradigm aimed at reintegrating fine-grained user feedback into generative AI search. ii) The main research objective is to address the limitations of current generative AI search systems, which lack effective feedback loops for refining individual components due to the reduced granularity of user feedback. iii) The methodology involves integrating two complementary modes: User Debug Mode for explicit user intervention and Shadow User Mode, which employs a personalized user agent to simulate user preferences and generate AI-assisted feedback. iv) The primary result is the conceptualization of a feedback store mechanism where users can share and potentially monetize their debugging efforts, though no quantitative results are reported in this perspective paper. v) AI practitioners can leverage NExT-Search to build feedback-rich AI search systems that continuously evolve alongside human feedback, emphasizing online adaptation and offline updates to refine query decomposition, retrieval, and generation models. |
| Training-Free Watermarking for Autoregressive Image Generation (Read more on arXiv or HuggingFace) |
Shuai Yang, kaiyangzhou, Apostle723, yutchina02 |
i) IndexMark, a training-free watermarking framework, is proposed for autoregressive image generation models. ii) The objective is to embed invisible and robust watermarks into images generated by autoregressive models without compromising image quality. iii) The methodology uses a match-then-replace strategy, selecting watermark tokens based on token similarity and employing an Index Encoder for verification with a cropping-robust validation scheme. iv) Experiments show IndexMark achieves state-of-the-art performance in image quality and verification accuracy (1.000 on watermark verification under clean conditions) while demonstrating robustness against perturbations like cropping and noise. v) The training-free nature and demonstrated robustness of IndexMark provide AI practitioners with a practical method for ensuring image traceability in autoregressive generative models without incurring additional training costs. |
| VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation (Read more on arXiv or HuggingFace) |
Ping Nie, Yiming Jia, ZhuofengLi, wren93, tonymwt |
i) VIDEOEVAL-PRO is introduced as a more robust long video understanding (LVU) benchmark. ii) The research aims to address the inflated performance and strong priors of existing LVU benchmarks and provide a realistic evaluation. iii) The methodology involves reformulating multiple-choice questions from existing benchmarks into open-ended questions and employing filtering methods based on video duration, answer type, answerability, and difficulty. iv) Evaluation of 21 video LMMs reveals a performance drop exceeding 25% on open-ended questions compared to multiple-choice questions, with models achieving only ~10% accuracy on VIDEOEVAL-PRO with a single input frame. v) VIDEOEVAL-PRO offers a more reliable measure of long video understanding progress, providing a more faithful assessment of LMMs’ ability to integrate and reason over longer video contexts for AI practitioners. |
| CS-Sum: A Benchmark for Code-Switching Dialogue Summarization and the |
|
|
| Limits of Large Language Models (Read more on arXiv or HuggingFace) |
Eng Siong Chng, Lim Zhi Hao, Tanmay Surana, SkAndMl |
i) CS-Sum is introduced as the first benchmark dataset for evaluating code-switching (CS) dialogue summarization in LLMs across Mandarin-English, Tamil-English, and Malay-English language pairs. ii) The research objective is to assess the comprehensibility of code-switching in LLMs through the task of CS dialogue to English summarization. iii) The methodology involves evaluating ten LLMs using few-shot learning, translate-summarize, and fine-tuning (LoRA, QLORA on synthetic data) approaches. iv) Results indicate that although automated metrics are high, LLMs make subtle errors altering the meaning of dialogues; error rates vary across CS pairs and models. v) This underscores the need for specialized training on code-switched data to improve LLM’s ability to interpret multilingual prompts, thereby suggesting current models lack intrinsic CS comprehension, and that finetuning can amplify errors under distribution shift. |
| Think Only When You Need with Large Hybrid-Reasoning Models (Read more on arXiv or HuggingFace) |
Zewen Chi, Qingxiu Dong, Shaohan Huang, YUSHUIWX, lingjie23 |
i) The paper introduces Large Hybrid-Reasoning Models (LHRMs) that adaptively determine whether to engage in extended thinking processes based on query complexity. ii) The research aims to mitigate the overthinking problem in Large Reasoning Models (LRMs) by adaptively selecting between thinking and no-thinking modes. iii) The methodology involves a two-stage training pipeline: Hybrid Fine-Tuning (HFT) followed by Hybrid Group Policy Optimization (HGPO), and a metric called Hybrid Accuracy is used for evaluation. iv) Experiments show that LHRMs outperform existing LRMs and LLMs, demonstrating adaptive hybrid thinking on queries of varying difficulty; LHRMs achieve average improvements of 9.2% and 7.1% compared to HFT-DPO at the 1.5B and 7B scales, respectively. v) LHRMs provide AI practitioners with a more efficient reasoning model that reduces computational overhead on simple tasks while maintaining strong reasoning ability on complex queries, leading to better resource utilization and user experience. |
| Fine-tuning Quantized Neural Networks with Zeroth-order Optimization (Read more on arXiv or HuggingFace) |
Minxian Li, Jiayi Zhou, kaiyangzhou, chenyulin, sifengshang |
i) The paper introduces Quantized Zeroth-order Optimization (QZO) for memory-efficient fine-tuning of quantized neural networks. ii) The research aims to minimize memory usage on model weights, gradients, and optimizer states during fine-tuning. iii) QZO approximates gradients by perturbing the continuous quantization scale and employs directional derivative clipping to stabilize training. iv) QZO reduces total memory cost by over 18× for 4-bit LLMs, enabling fine-tuning of Llama-2-13B and Stable Diffusion 3.5 Large on a single 24GB GPU. v) QZO provides AI practitioners with a method to fine-tune large models with significantly reduced memory requirements, potentially democratizing access to adapting such models on resource-constrained hardware. |
| SSR: Enhancing Depth Perception in Vision-Language Models via |
|
|
| Rationale-Guided Spatial Reasoning (Read more on arXiv or HuggingFace) |
Han Zhao, Pengxiang Ding, Xiaomin Yu, Ming Ma, yliu-cs |
This paper introduces SSR, a Spatial Sense and Reasoning method to improve spatial understanding in Vision-Language Models (VLMs) by converting raw depth data into structured textual rationales. The research aims to enhance spatial reasoning in VLMs by integrating depth information more effectively. The method involves transforming depth data into textual rationales, knowledge distillation to create compact latent embeddings, and a novel SSR-COT dataset for training and evaluation. Experiments on SSRBENCH showed SSR substantially improves depth utilization and enhances spatial reasoning, with SSR achieving a 13.6% improvement in average question answering accuracy compared to baseline models on certain spatial reasoning tasks. These results imply AI practitioners can enhance VLMs by incorporating structured depth information via textual rationales, improving spatial reasoning capabilities without extensive retraining. |
| Reward Reasoning Model (Read more on arXiv or HuggingFace) |
Qingxiu Dong, Zewen Chi, Jiaxin Guo, YUSHUIWX, unilm |
i) The paper introduces Reward Reasoning Models (RRMs), which perform explicit reasoning before generating rewards for language model outputs. ii) The main objective is to enhance reward model performance by effectively utilizing test-time compute for complex queries. iii) RRMs are trained using a reinforcement learning framework to foster self-evolved reward reasoning capabilities without explicit reasoning traces as training data. iv) Experiments show RRMs achieve superior performance on reward modeling benchmarks; RRM-32B attains an accuracy of 98.6% in the reasoning category of RewardBench. v) RRMs offer AI practitioners a method to improve reward modeling through deliberate reasoning and adaptive allocation of test-time compute, enhancing performance in tasks requiring nuanced analysis. |
| Not All Correct Answers Are Equal: Why Your Distillation Source Matters (Read more on arXiv or HuggingFace) |
Sitong Zhao, Shuaiting Chen, Haotian Wang, Yunjie Ji, Emperorizzis |
i) The paper investigates the impact of different teacher models on the quality of distilled reasoning datasets for language models. ii) The main objective is to determine how the source model used for distillation affects the reasoning performance of student models trained on the resulting datasets. iii) The methodology involves distilling data from three teacher models (AM-Thinking-v1, Qwen3-235B-A22B, and DeepSeek-R1) on a corpus of 1.89 million queries, followed by training student models on each distilled dataset. iv) Results show that student models trained on AM-Thinking-v1 distilled data achieve superior performance, reaching 84.3 on AIME2024, and also demonstrate adaptive output behavior based on task complexity. v) The key implication is that the choice of the distillation source significantly influences downstream reasoning performance, and high-quality, verified reasoning traces from models like AM-Thinking-v1 are critical for creating effective reasoning-oriented language models; AM-Thinking-v1 is also demonstrated to have lower perplexity suggesting higher quality data. |
| Hunyuan-Game: Industrial-grade Intelligent Game Creation Model (Read more on arXiv or HuggingFace) |
vcvcvn, tangjs, YellowAddice, zhengsj, lslrh |
i) Hunyuan-Game is presented as a comprehensive AI-driven framework for procedural game asset generation, including image and video modalities. ii) The research aims to develop a suite of generative models capable of producing high-fidelity, controllable game content to enhance designer efficiency. iii) The methodology involves curating large-scale datasets of game and anime assets, fine-tuning diffusion transformer models, and implementing specialized prompt optimization and control mechanisms. iv) The system demonstrates state-of-the-art performance, with designer feedback indicating a 60% improvement in visual effects iteration efficiency using the introduced reference-based game visual effects generation approach. v) Hunyuan-Game offers AI/ML practitioners a practical framework and associated models for automating and enhancing content creation pipelines within the gaming industry, providing a foundation for further research and development in domain-specific generative AI. |
| Warm Up Before You Train: Unlocking General Reasoning in |
|
|
| Resource-Constrained Settings (Read more on arXiv or HuggingFace) |
Keith Ross, xanubhav81, AadimNepal, guactastesgood, safal312 |
Designing reasoning-capable LLMs with limited training data can be improved using a two-stage approach of warmup followed by task adaptation. The research investigates if models trained with general reasoning strategies can rapidly adapt to new domains with minimal supervision. The methodology involves pre-training a model on Knights & Knaves logic puzzles to distill general reasoning skills, followed by RLVR fine-tuning on a limited set of domain-specific examples. Experiments show that the warmup phase leads to a +10.2% increase on MATH and +15.3% increase on HumanEval+ for the Qwen2.5-3B model, and that the warmed-up model outperforms the base model when both are RLVR trained on the same small datasets. AI practitioners can leverage the warmup technique to improve sample efficiency and maintain cross-domain generalizability when training robust reasoning LLMs in data-scarce environments. |
| Lessons from Defending Gemini Against Indirect Prompt Injections (Read more on arXiv or HuggingFace) |
cchoquette, julsh, tux, iliashum, chongyangs |
i) This paper evaluates and improves the robustness of Gemini models against indirect prompt injection attacks in tool-use scenarios. ii) The main objective is to assess Gemini’s adversarial robustness and identify key lessons for making the model more resilient to manipulation via untrusted data. iii) The methodology involves an adversarial evaluation framework that deploys adaptive attack techniques against Gemini, along with adversarial fine-tuning. iv) Gemini 2.5 achieved an average of approximately 47% reduction in attack success rate (ASR) across three attack techniques, and the warning defense achieved a 10.8% ASR defending Gemini 2.0 against the adaptive TAP attack. v) The principal implication is that adaptive evaluation and adversarial training are crucial for enhancing model security, while external defenses can complement model-level improvements. |
| Towards eliciting latent knowledge from LLMs with mechanistic |
|
|
| interpretability (Read more on arXiv or HuggingFace) |
Emil Ryd, NeelNanda, srdm, bcywinski |
i) This paper explores methods for eliciting hidden knowledge from language models. ii) The main research question is how to uncover a secret word internalised by a language model without explicit verbalisation. iii) The methodology involves training a Taboo model and then applying black-box and mechanistic interpretability techniques like Logit Lens and Sparse Autoencoders. iv) The primary result demonstrates that interpretability-based approaches can elicit the secret word, with “Another Model” black-box elicitation achieving 95% Pass@10. v) This suggests mechanistic interpretability is a promising direction for extracting hidden knowledge, but the model organism needs to be more complex. |
| Truth Neurons (Read more on arXiv or HuggingFace) |
ZiningZhu, jordansuchow, ShirleyY, YupengCao, Acatsama |
i) The paper identifies and analyzes “truth neurons” in language models that encode truthfulness in a subject-agnostic manner. ii) The research aims to identify neuron-level mechanisms encoding truthfulness within language models. iii) The methodology involves using integrated gradients to measure neuron attribution scores for truthful vs. untruthful responses, followed by systematic filtering to identify truth neurons. iv) Experiments across six language models reveal that suppressing identified truth neurons leads to statistically significant accuracy reductions on TruthfulQA (e.g., average accuracy of small-scale models decreases to 54.25%, representing a degradation of 10.49%) and generalizes to other benchmarks. v) The identification and analysis of truth neurons offer AI practitioners potential directions for improving the trustworthiness and reliability of language models by highlighting areas for targeted intervention and alignment. |
| Two Experts Are All You Need for Steering Thinking: Reinforcing |
|
|
| Cognitive Effort in MoE Reasoning Models Without Additional Training (Read more on arXiv or HuggingFace) |
Jiahao Xu, Zhiwei He, Yue Wang, Xingyu Chen, Mengru Wang |
i) The paper introduces Reinforcing Cognitive Experts (RICE), a novel inference-time method to enhance reasoning in Mixture-of-Experts (MoE) models without additional training. ii) The main research objective is to improve the cognitive efficiency of Large Reasoning Models (LRMs) by modulating experts correlated with reasoning. iii) The methodology involves identifying specialized “cognitive experts” using normalized Pointwise Mutual Information (nPMI) and selectively amplifying their activation during inference. iv) The approach improves reasoning accuracy on DeepSeek-R1, increasing AIME24 accuracy from 73.3% to 83.3% by reinforcing only the top two cognitive experts with a multiplier of 64. v) RICE offers a lightweight and interpretable method for AI practitioners to enhance reasoning in MoE-based LRMs, improving efficiency and accuracy without retraining or complex heuristics. |
| Fixing 7,400 Bugs for 1$: Cheap Crash-Site Program Repair (Read more on arXiv or HuggingFace) |
Mathias Payer, Aiden Hall, Tianqi Fan, Han Zheng, iliashum |
i) The paper introduces WILLIAMT, a cost-effective crash-site program repair tool. ii) The research aims to reduce the cost and improve the effectiveness of automated program repair, particularly for memory corruption vulnerabilities, using crash-site repair and template-guided patch generation. iii) The methodology involves a regex-based context retrieval and a template-guided patch generation to minimize LLM token usage. iv) Evaluation shows that WILLIAMT, when combined with CodeRover-S, reduces token cost by 45.9% and increases the bug-fixing rate to 73.5% on ARVO, retaining 86.7% of CodeRover-S performance while saving 99.7% Token cost. v) The results imply that AI practitioners can leverage low-cost, template-guided APR for memory corruption vulnerabilities, significantly reducing resource consumption while maintaining repair effectiveness in open-source software. |
| Phare: A Safety Probe for Large Language Models (Read more on arXiv or HuggingFace) |
Matteo Dora, inoki-giskard, bmalezieux, pierlj |
Phare introduces a multilingual diagnostic framework to evaluate the safety of large language models (LLMs). The primary objective is to probe and evaluate LLM behavior across hallucination and reliability, social biases, and harmful content generation. The study evaluated 17 state-of-the-art LLMs using Phare, revealing systematic vulnerabilities, such as sycophancy, prompt sensitivity, and stereotype reproduction. The research found that confidence tone in user messages can significantly decrease debunking accuracy by up to 15%. Phare provides actionable insights for AI practitioners by highlighting specific failure modes to build more robust, aligned, and trustworthy language systems. The study does not contain information about the architecture of Phare. |
| MIGRATION-BENCH: Repository-Level Code Migration Benchmark from Java 8 (Read more on arXiv or HuggingFace) |
Lin Chen, Qiang Zhou, omidvarb, sliuxl, linboliu |
i) MigrationBench, a new benchmark, facilitates the evaluation of LLMs for Java code migration from version 8 to 17/21. ii) The research aims to provide a comprehensive benchmark for repository-level code migration to address the limitations of existing code generation and issue-resolution focused benchmarks. iii) The methodology involves curating a dataset of open-source Java repositories, developing an automated evaluation framework, and proposing a novel feedback mechanism named SD-Feedback. iv) Results show that SD-Feedback, when implemented with Claude-3.5-Sonnet-v2, achieves a 62.33% success rate (pass@1) for minimal migration on the selected subset of repositories. v) AI practitioners can use MigrationBench and SD-Feedback to improve LLM-driven code migration tools for enhancing software maintainability and facilitating Java version upgrades. |
| Tokenization Constraints in LLMs: A Study of Symbolic and Arithmetic |
|
|
| Reasoning Limits (Read more on arXiv or HuggingFace) |
Yiwei Xu, Jiaqi Wei, Juntai Cao, Charlesyooo, Wyattz23 |
i) Tokenization schemes in LLMs can significantly constrain symbolic and arithmetic reasoning abilities. ii) The research investigates how tokenization schemes, specifically byte-pair encoding (BPE), affect the ability of LLMs to perform symbolic computation and arithmetic reasoning. iii) The methodology involves a theoretical analysis of token awareness and empirical evaluation across arithmetic and symbolic tasks with variations in tokenization and Chain-of-Thought prompting. iv) The study demonstrates that token structure dramatically affects reasoning performance, showing up to 80% performance degradation due to suboptimal tokenization and demonstrating GPT-40-mini can outperform o1 when tokenization is atomically aligned. v) AI practitioners should consider tokenization strategies as a critical factor in designing LLMs for symbolic and arithmetic reasoning tasks to unlock full computational potential, as model performance is deeply conditioned on token-level representations. |
| CompeteSMoE – Statistically Guaranteed Mixture of Experts Training via |
|
|
| Competition (Read more on arXiv or HuggingFace) |
Van Nguyen, Quang Pham, Huy Nguyen, nhatho, DavidNguyen |
CompeteSMoE introduces a novel routing mechanism for Sparse Mixture of Experts (SMoE) training based on a competition principle. This research addresses the challenge of suboptimal routing in SMOE by exploring whether experts that perform computation directly contribute to the routing process. The methodology involves distributing tokens to experts based on the highest neural response, with theoretical guarantees of better sample efficiency compared to softmax routing. Experiments using a 5.1B parameter backbone demonstrates CompeteSMoE improved zero-shot performance across nine visual instruction tuning tasks. This work offers AI/ML engineers an effective strategy to enhance large language model training by improving upon routing efficiency. |
| Solve-Detect-Verify: Inference-Time Scaling with Flexible Generative |
|
|
| Verifier (Read more on arXiv or HuggingFace) |
Kezhi Li, Zhijian Xu, Zeju Li, XiangyuWen, Jianyuan1 |
i) The paper introduces FlexiVe, a flexible generative verifier, and the Solve-Detect-Verify pipeline for efficient LLM reasoning. ii) The research aims to improve the trade-off between accuracy and computational efficiency in LLM reasoning by dynamically allocating verification resources. iii) The methodology involves a two-stage verification process: a fast, resource-efficient mode for quick error diagnosis and a slower, computationally-intensive mode for deeper analysis, managed by a flexible verification budget. iv) Experiments on the AIME2024 benchmark showed that Solve-Detect-Verify achieves higher accuracy while requiring approximately 4x fewer solutions compared to baseline approaches; also FlexiVe (specifically with the Flex@8 configuration) attains a higher F1 score while generating approximately 3x fewer tokens than the baseline on the Math benchmark. v) The primary implication for AI practitioners is a scalable and effective approach for enhancing LLM reasoning at test time, providing a means to balance accuracy and computational cost. |
| To Bias or Not to Bias: Detecting bias in News with bias-detector (Read more on arXiv or HuggingFace) |
grohg, amosharafa, himel7 |
i) This paper presents an improved RoBERTa-based model for sentence-level media bias detection. ii) The research aims to enhance the accuracy and statistical significance of bias detection in news articles compared to existing models. iii) The methodology involves fine-tuning a RoBERTa-base model on the BABE dataset and comparing its performance against a DA-ROBERTa baseline using McNemar’s test and 5x2 cross-validation. iv) The fine-tuned RoBERTa model achieved a macro F1 score of 0.9257 on the BABE dataset, demonstrating statistically significant improvements (p < 2.45x10-9 in McNemar’s test) over the DA-ROBERTa baseline. v) This research offers AI practitioners a more robust and statistically validated bias detection model, potentially reducing false positives/negatives in downstream tasks that rely on unbiased news analysis, while also establishing a framework for future comprehensive bias analysis. |
| Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for |
|
|
| Real-world Knowledge Injection (Read more on arXiv or HuggingFace) |
Shangbin Feng, Wenhao Yu, Yuwei Zhang, shangjingbo, KomeijiForce |
i) This paper introduces WIKIDYK, a novel benchmark for evaluating knowledge injection in LLMs using real-world Wikipedia “Did You Know…” facts. ii) The main research question is whether LLMs can effectively memorize and internalize new knowledge after pre-training, comparing Causal Language Models (CLMs) against Bidirectional Language Models (BiLMs). iii) The methodology involves continued pre-training of various LLM architectures (CLMs and BiLMs) with WIKIDYK facts, followed by a multi-dimensional evaluation suite spanning question answering tasks. iv) The primary result indicates that BiLMs demonstrate significantly stronger knowledge memorization capabilities compared to CLMs, exhibiting a 23% higher accuracy in reliability; a modular collaborative framework utilizing ensembles of BiLMs as external knowledge repositories further improves reliability accuracy by up to 29.1%. v) The principal implication for AI practitioners is that BiLMs may be more effective than CLMs for applications requiring robust knowledge integration, suggesting a potential shift or hybrid approach in model architecture design for knowledge-intensive tasks. |
| Masking in Multi-hop QA: An Analysis of How Language Models Perform with |
|
|
| Context Permutation (Read more on arXiv or HuggingFace) |
Jeff Z. Pan, Mirella Lapata, pvougiou, hwy9855 |
Language models (LMs) performance on multi-hop question answering (MHQA) is analyzed by varying the order of retrieved documents in the input context. The study investigates how encoder-decoder (Flan-T5) and decoder-only (Qwen, Llama) architectures respond to document permutations. The primary methodology involves evaluating LM accuracy on the MuSiQue dataset under different document orderings, distances, and completeness configurations. It was found that fine-tuned LMs favor forward-placed documents and bi-directional attention can improve performance. An attention weight analysis showed that LMs typically assign higher attention weights to relevant documents when answering correctly; specifically, accuracy of Qwen 7B was increased from 28.6% to 33.7% by sampling answers with different input document permutations and retaining answers for the inputs for which the LM assigned the largest peak attention score. This research suggests that optimizing document ordering and incorporating bidirectional attention can enhance LMs for knowledge-intensive tasks and improving the RAG paradigm by focusing on ranking based metrics. |
| Incorporating brain-inspired mechanisms for multimodal learning in |
|
|
| artificial intelligence (Read more on arXiv or HuggingFace) |
Xin Yang, Qingqun Kong, Yang Li, Dongcheng Zhao, Xiang He |
i) This paper introduces an inverse effectiveness driven multimodal fusion (IEMF) strategy inspired by biological multimodal integration. ii) The main objective is to improve multimodal learning in artificial intelligence by incorporating the inverse effectiveness principle. iii) The methodology involves incorporating IEMF into neural network architectures, adapting the fusion module’s weights based on the strength of unimodal inputs and multimodal outputs. iv) Experiments on audio-visual tasks demonstrate IEMF achieves up to 50% reduction in computational cost compared to baseline methods, while maintaining or improving performance. v) IEMF offers AI practitioners a biologically-inspired mechanism for enhancing multimodal fusion in neural networks, improving both performance and computational efficiency; it is unclear if they only measure compute cost by FLOPs. |
| Understanding Gen Alpha Digital Language: Evaluation of LLM Safety |
|
|
| Systems for Content Moderation (Read more on arXiv or HuggingFace) |
Fausto Giunchiglia, Manisha Mehta |
i) This paper evaluates the efficacy of LLM-based content moderation systems in understanding and mitigating risks within Gen Alpha’s digital communication. ii) The research investigates the ability of current AI systems to comprehend Gen Alpha’s evolving linguistic patterns, including slang and context-dependent meanings, and detect harmful content. iii) The methodology involved creating a novel dataset of 100 Gen Alpha expressions and evaluating leading LLMs (GPT-4, Claude, Gemini, Llama 3) alongside human moderators and Gen Alpha users, across dimensions of basic understanding, context recognition, and safety implication detection. iv) The study found that Gen Alpha users demonstrated 92% accuracy in identifying potential harm, significantly exceeding the performance of both AI systems and human moderators, and that LLMs showed limitations in detecting masked risks and evolving language (32-42% accuracy). v) The principal implication is that AI practitioners must develop content moderation systems incorporating dynamic, context-aware capabilities and systematic bias audits, supplemented with human oversight, to effectively protect young users from online risks, especially given documented reluctance of Gen Alpha users to seek adult help, and platform-specific meaning variations. |
Papers for 2025-05-20
| Title |
Authors |
Summary |
| Chain-of-Model Learning for Language Model (Read more on arXiv or HuggingFace) |
tricktreat, Chengruidong, iofu728, xutan, KaitaoSong |
i) The paper proposes Chain-of-Model (CoM), a learning paradigm for language models that introduces scaling efficiency and deployment flexibility. ii) The primary objective is to develop a framework for progressively scaling up language models and enabling elastic inference with varying model sizes. iii) The methodology involves incorporating Chain-of-Representation (CoR) into Transformer layers, termed Chain-of-Language-Model (CoLM), and introducing a KV sharing mechanism (CoLM-Air) for extensibility. iv) Experimental results demonstrate that CoLM achieves comparable performance to standard Transformers while enabling capabilities such as progressive scaling and offering multiple sub-models for elastic inference. v) The CoLM framework enables AI practitioners to progressively scale language models and deploy them in resource-constrained environments by selecting appropriate sub-model sizes for inference. |
| AdaptThink: Reasoning Models Can Learn When to Think (Read more on arXiv or HuggingFace) |
Ling Feng, Lei Hou, juanli, linny2002, NeoZ123 |
i) This paper introduces AdaptThink, a reinforcement learning (RL) algorithm for reasoning models to adaptively select between Thinking and NoThinking modes based on problem difficulty. ii) The primary research objective is to enable reasoning models to dynamically choose the optimal thinking mode to balance reasoning quality and efficiency. iii) The methodology involves a constrained optimization objective that encourages the NoThinking mode and an importance sampling strategy to balance Thinking and NoThinking samples during on-policy RL training. iv) Experiments on math datasets show that AdaptThink reduces the average response length of DeepSeek-R1-Distill-Qwen-1.5B by 53% and improves its accuracy by 2.4% on three math datasets (GSM8K, MATH500, and AIME2024). v) The principal implication for AI practitioners is the potential for adaptive thinking-mode selection to optimize the trade-off between reasoning quality and inference costs in large reasoning models. |
| AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Shuangzhi, qingping95, Swtheking, sunzewei2715, louchenwei |
AdaCoT introduces a reinforcement learning framework for adaptive Chain-of-Thought (CoT) triggering in large language models (LLMs) to optimize performance and cost. The research addresses the challenge of indiscriminate CoT usage by framing adaptive reasoning as a Pareto optimization problem. The methodology employs proximal policy optimization (PPO) with selective loss masking (SLM) to dynamically control CoT triggering based on query complexity. Experiments show AdaCoT reduces CoT triggering rates to 3.18% on production traffic, decreasing average response tokens by 69.06% while maintaining performance on complex tasks. AdaCoT offers AI practitioners a method for developing more efficient and cost-effective LLMs by dynamically adjusting reasoning based on query complexity. |
| Delta Attention: Fast and Accurate Sparse Attention Inference by Delta |
|
|
| Correction (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, gmlwns5176, jeffwillette |
Here is a summary of the provided AI research paper: i) This paper introduces Delta Attention (∆ Attention), a method for correcting distributional shifts in sparse attention mechanisms to improve accuracy during inference. ii) The research aims to mitigate the performance degradation observed in sparse attention methods for long sequences by addressing the distributional shift they induce. iii) The methodology involves calculating the difference between sparse and full attention outputs on a subset of queries and applying this difference as a correction to the sparse attention output. iv) The primary result is an average 36%pt performance increase in accuracy compared to existing sparse attention methods, recovering 88% of full quadratic attention accuracy on the 131K RULER benchmark with sliding window attention and sink tokens, while maintaining 98.5% sparsity, achieving 32x faster inference than Flash Attention 2 when processing 1M token prefills. v) The principal implication for AI practitioners is a more accurate and efficient sparse attention mechanism for transformer models that can be seamlessly integrated into existing pipelines to improve performance in long-sequence tasks. |
| Scaling Computer-Use Grounding via User Interface Decomposition and |
|
|
| Synthesis (Read more on arXiv or HuggingFace) |
Mayome, RadioBlue, lixiaochuan2020, MillanK, tianbaoxiexxx |
i) This paper introduces OSWORLD-G, a GUI grounding benchmark, and JEDI, a large-scale synthetic dataset. ii) The primary objective is to address the limitations of existing GUI grounding benchmarks by creating a more comprehensive and challenging evaluation environment. iii) The research utilizes a multi-perspective decoupling of tasks to synthesize the JEDI dataset and trains multi-scale models on this data. iv) Results show improved grounding performance on ScreenSpot-v2, ScreenSpot-Pro, and OSWORLD-G, with agentic capabilities on complex computer tasks improving from 5% to 27% on OSWorld. v) AI practitioners can leverage the JEDI dataset and OSWORLD-G benchmark to develop more robust GUI grounding models, leading to enhanced agentic capabilities in complex computer tasks. |
| Thinkless: LLM Learns When to Think (Read more on arXiv or HuggingFace) |
wxcTest, horseee, Vinnnf |
i) Thinkless is a reinforcement learning framework enabling Language Models (LLMs) to adaptively select between short-form and long-form reasoning modes. ii) The main research question is whether LLMs can learn to decide when to engage in elaborate reasoning based on task complexity and model capability. iii) The methodology involves training LLMs under a reinforcement learning paradigm using Decoupled Group Relative Policy Optimization (DeGRPO) with control tokens for reasoning modes. iv) Thinkless reduces the usage of long-chain thinking by 50%-90% on benchmarks like Minerva Algebra and GSM8K. v) The principal implication for AI practitioners is a method for significantly improving the efficiency of reasoning LLMs by adaptively controlling the depth of reasoning, reducing computational costs while preserving task performance. |
| Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient |
|
|
| in Latent Space (Read more on arXiv or HuggingFace) |
zlzheng, vickyandkekey, ColorfulAI, xuekai, henry12348 |
i) The paper introduces LATENTSEEK, a novel framework for enhancing LLM reasoning via test-time instance-level adaptation (TTIA) in the model’s latent space. ii) The research aims to improve LLM reasoning capabilities at test time without parameter updating by optimizing latent representations guided by self-generated reward signals. iii) The key methodology involves using policy gradient to iteratively update latent representations based on a self-generated reward function operating within the model’s latent space. iv) Results show that LATENTSEEK achieves an average improvement of 10.75% over Chain-of-Thought on the GSM8K dataset. v) AI practitioners can leverage LATENTSEEK as a lightweight and scalable solution to enhance the reasoning capabilities of LLMs without extensive retraining or fine-tuning. |
| MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable |
|
|
| Step-Level Supervision (Read more on arXiv or HuggingFace) |
wqshao126, Domingo12, SuperposedWave, FanqingM, Cierra0506 |
i) The paper introduces MM-PRM, a process reward model for enhancing multimodal mathematical reasoning through scalable step-level supervision. ii) The research aims to improve logical robustness in multimodal reasoning systems by providing fine-grained supervision over intermediate steps. iii) The methodology includes building a multimodal policy model (MM-Policy), curating the MM-K12 dataset, and generating step-level annotations using a Monte Carlo Tree Search (MCTS) pipeline for training the process reward model. iv) Experiments show MM-PRM improves accuracy on the MM-K12 test set from 33.92% to 42.80% using best-of-N inference and demonstrates generalization to out-of-domain benchmarks such as MathVista and OlympiadBench. v) MM-PRM provides AI practitioners with a process reward model and a framework that enhance the logical consistency of multimodal reasoning systems, demonstrating the effectiveness of process supervision in improving mathematical problem-solving. |
| Hybrid 3D-4D Gaussian Splatting for Fast Dynamic Scene Representation (Read more on arXiv or HuggingFace) |
epark, Heyjin, LeeYG, ohseungjun |
Hybrid 3D-4D Gaussian Splatting (3D-4DGS) is introduced for efficient dynamic scene representation. The study addresses the computational and memory overhead in dynamic 3D scene reconstruction by adaptively representing static regions with 3D Gaussians and dynamic elements with 4D Gaussians. The method iteratively converts temporally invariant Gaussians into 3D, reducing parameters and improving computational efficiency. Experiments show the approach achieves comparable rendering quality with a significantly faster training time, converging in approximately 12 minutes compared to other 4DGS methods. The hybrid 3D-4D representation allows AI practitioners to achieve faster training and reduced memory consumption when reconstructing dynamic scenes, enabling more efficient development of real-time rendering applications. |
| FedSVD: Adaptive Orthogonalization for Private Federated Learning with |
|
|
| LoRA (Read more on arXiv or HuggingFace) |
Sangwoo Park, hbseong, dwgnr, dongboklee, Seanie-lee |
i) The paper introduces FedSVD, a federated learning method that adapts LoRA weights using SVD for improved privacy and performance. ii) The research aims to mitigate noise amplification in differentially private federated learning with LoRA by adaptively updating the low-rank adaptation matrix. iii) The methodology involves a server performing SVD on aggregated LoRA updates and reinitializing the orthogonal matrix A, while clients optimize matrix B with DP-SGD. iv) Experiments on GLUE datasets under DP constraints (ε = 6, δ = 10−5) show that FedSVD achieves a 8.77 percentage point increase over FFA-LoRA. v) The key implication for AI practitioners is a stable and efficient method to fine-tune large language models in privacy-sensitive federated learning settings by reparameterizing LoRA updates. |
| CPGD: Toward Stable Rule-based Reinforcement Learning for Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
wqshao126, SuperposedWave, Cierra0506, FanqingM, Zkkkai |
i) The paper introduces Clipped Policy Gradient Optimization with Policy Drift (CPGD) to stabilize rule-based reinforcement learning for language models. ii) The main objective is to address training instability issues in existing RL methods for LMs, specifically those related to large policy updates and improper clipping. iii) CPGD incorporates a KL divergence-based policy drift constraint to dynamically regularize policy updates, combined with a clip mechanism on the logarithm of the ratio to prevent excessive changes. iv) Empirical analysis shows that CPGD improves overall performance by +11.0% across various multimodal reasoning benchmarks compared to the base model and reduces instability. v) CPGD offers AI practitioners a more robust and stable RL algorithm for post-training language models, mitigating issues of training collapse common in methods that directly incorporate importance-sampling ratios in the loss function. |
| Faster Video Diffusion with Trainable Sparse Attention (Read more on arXiv or HuggingFace) |
EricX003, hunterhector, BrianChen1129, haofeng666, PY007 |
Faster Video Diffusion with Trainable Sparse Attention introduces VSA, a hardware-aligned trainable sparse attention mechanism for video Diffusion Transformers (DiTs). The research aims to mitigate the quadratic complexity of 3D attention in DiTs by focusing computation on critical tokens. VSA employs a hierarchical approach with a coarse stage for tile pooling and a fine stage for token-level attention within selected tiles, implemented with block-sparse kernels and trained end-to-end. Experiments show VSA achieves up to 2.53× reduction in training FLOPS compared to full attention with no drop in diffusion loss and accelerates Wan-2.1 attention time by 6×. VSA’s efficiency and scalability present a practical alternative to full attention, enabling further scaling of video diffusion models. |
| Fractured Chain-of-Thought Reasoning (Read more on arXiv or HuggingFace) |
JunnanLi, doyensahoo, yuhuixu, hendrydong, baohao |
i) The paper introduces Fractured Sampling, an inference-time scaling technique for large language models (LLMs) that interpolates between full Chain-of-Thought (CoT) and solution-only sampling. ii) The research investigates how to optimize the accuracy-cost trade-off in LLM reasoning by controlling the number of reasoning trajectories, final solutions per trajectory, and reasoning trace truncation depth. iii) The methodology involves extensive experiments on five reasoning benchmarks, evaluating performance against token budget constraints by varying the number of reasoning trajectories, solution diversity, and reasoning prefix length. iv) Results demonstrate that truncated CoT often matches or exceeds full CoT accuracy with fewer tokens, and Fractured Sampling achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget with up to a 10.4% improvement over baseline on certain tasks. v) Practitioners can leverage Fractured Sampling to achieve more efficient and scalable LLM reasoning by strategically allocating computational resources across reasoning depth, trajectory diversity, and solution diversity to maximize performance within a given token budget. |
| VisionReasoner: Unified Visual Perception and Reasoning via |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Shu Liu, BoHao0326, zszhong, TainU, Ricky06662 |
VisionReasoner is a unified reinforcement learning framework for diverse visual perception tasks. The paper aims to create a single model capable of reasoning and solving multiple visual perception tasks like detection, segmentation, and counting. The methodology involves designing a multi-object cognitive learning strategy and task reformulation, using format and accuracy rewards to train a shared model via reinforcement learning. VisionReasoner achieves superior performance with a 29.1% relative improvement on COCO detection compared to Qwen2.5VL using only 7,000 training samples. VisionReasoner offers AI practitioners a unified architecture for handling various visual perception tasks, potentially streamlining development and improving generalization capabilities in resource-constrained scenarios. |
| Neuro-Symbolic Query Compiler (Read more on arXiv or HuggingFace) |
jrwen, wuyongkang, lixiaoxi45, douzc, KeriaZhang |
i) This paper introduces QCompiler, a neuro-symbolic framework for complex query understanding in Retrieval Augmented Generation (RAG) systems. ii) The main research objective is to improve the precision of search intent recognition for complex queries with nested structures and dependencies in RAG systems. iii) The methodology involves designing a minimal Backus-Naur Form (BNF) grammar to formalize complex queries, translating natural language queries into BNF expressions, and parsing these into Abstract Syntax Trees (ASTs) for execution. iv) Experimental results show that QCompiler achieves a 44.5% Exact Match on the 2WikiMultihopQA dataset. v) The primary implication for AI practitioners is a lightweight framework that can be integrated into existing RAG systems to improve efficiency and accuracy by providing more precise document retrieval and response generation. |
| Model Merging in Pre-training of Large Language Models (Read more on arXiv or HuggingFace) |
Jing Liu, Chaoyi Zhang, Shen Yan, Yiyuan Ma, Yunshui Li |
Model merging, specifically Pre-trained Model Average (PMA), is investigated for LLM pre-training to enhance performance and reduce training costs. The study explores how merging checkpoints during pre-training affects model performance, optimal merging strategies, and training stability. The methodology involves training dense and Mixture-of-Experts (MoE) architectures, ranging from millions to over 100 billion parameters, with extensive ablations of merging techniques. Model merging during the stable training phase achieves consistent performance gains, demonstrated by improvements such as Seed-MoE-1.3B/13B increasing from 31.1 to 36.6 on the Humaneval benchmark. PMA offers AI practitioners a cost-effective method to simulate annealed performance with constant learning rates, potentially leading to faster validation and reduced computational costs in LLM development, while also stabilizing training via weight initialization. |
| ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
Pietroferr, giobin, minttusofia, dainesn1, merlerm |
i) ViPlan is a new benchmark for evaluating visual planning capabilities of Vision-Language Models (VLMs) using symbolic predicates. ii) The paper investigates how VLMs perform in visual planning tasks both as direct planners and as grounders for symbolic planners. iii) The benchmark uses a visual Blocksworld domain and a simulated household robotics environment, evaluating nine open-source VLM families and selected closed models with and without Chain-of-Thought prompting. iv) Results show VLM-grounded symbolic planning outperforms direct VLM planning in Blocksworld but the reverse is true for household robotics, also showing no significant benefit from Chain-of-Thought prompting. v) AI practitioners should note that VLM performance in visual planning is highly task-dependent, with symbolic grounding proving more useful for tasks requiring accurate image interpretation and direct VLM planning working better when benefiting from the pre-trained world knowledge. |
| Accelerate TarFlow Sampling with GS-Jacobi Iteration (Read more on arXiv or HuggingFace) |
zhenqincn, encoreus |
i) The paper introduces a GS-Jacobi iteration method to accelerate the sampling process in TarFlow models. ii) The research aims to improve the sampling efficiency of TarFlow models, which suffer from slow sequential computation due to the causal form of attention. iii) The methodology involves transforming the sampling process into a diagonalized nonlinear system and applying a Gauss-Seidel-Jacobi hybrid iteration scheme, along with Convergence Ranking Metric (CRM) and Initial Guessing Metric (IGM). iv) Experiments show GS-Jacobi sampling achieves a speed-up of 4.53× in Img128cond, 5.32x in AFHQ, 2.96× in Img64uncond, and 2.51x in Img64cond without degrading FID scores. v) AI practitioners can utilize GS-Jacobi sampling to significantly enhance the sampling speed of TarFlow models while preserving generation quality, potentially enabling faster deployment and experimentation. |
| When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification |
|
|
| of Scientific Research (Read more on arXiv or HuggingFace) |
sngwon, Cartinoe5930, HazelNam, JW17, amphora |
This paper introduces SPOT, a new benchmark for automated scientific manuscript verification using large language models (LLMs). The research question explores the viability of using LLMs to automate the academic verification of scientific papers. The methodology involves creating a dataset of 83 published papers paired with 91 confirmed errors, cross-validated by authors and human annotators, and then evaluating state-of-the-art LLMs on this dataset. Results show that the best LLM achieved only 21.1% recall and 6.1% precision in error detection; furthermore confidence estimates are uniformly low and rediscovery rates are poor. The principal implication for AI practitioners is that there remains a substantial gap between current LLM capabilities and the requirements for dependable AI-assisted academic verification, indicating a need for further research and development in this area. |
| ChartMuseum: Testing Visual Reasoning Capabilities of Large |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
wadhma, PrasannSinghal, fcyin, thomlake, lytang |
ChartMuseum introduces a new benchmark for evaluating visual and textual reasoning in large vision-language models (LVLMs) on chart understanding. The research aims to address the imbalance in LVLM skills, particularly the shortfall in visual reasoning compared to textual reasoning. A new Chart Question Answering (QA) benchmark called CHARTMUSEUM was created, consisting of 1,162 expert-annotated questions derived from real-world charts. The results indicate that the best-performing model Gemini-2.5-Pro achieves only 63.0% accuracy, while human performance reaches 93%, and on questions requiring primarily visual reasoning, models experience a 35%-55% performance drop. This benchmark reveals a substantial gap between model and human capabilities in chart understanding and highlights the areas of visual reasoning that present significant challenges for current LVLMs, indicating practitioners must be aware of the limitation of LVLMs’ ability to reason with visual data when developing multimodal systems involving charts. |
| MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation (Read more on arXiv or HuggingFace) |
Yali Wang, Zhizhi Guo, Xirui Hu, yanboding |
i) MTVCrafter introduces a novel framework for human image animation directly modeling raw 3D motion sequences using a 4D motion tokenizer. ii) The research aims to improve generalization and controllability in open-world human image animation by directly modeling 4D motion instead of relying on 2D pose estimations. iii) A 4D Motion Tokenizer (4DMoT) is proposed to quantize 3D motion sequences into 4D motion tokens, and a Motion-aware Video Diffusion Transformer (MV-DiT) leverages these tokens for animation guidance. iv) MTVCrafter achieves a state-of-the-art FID-VID score of 6.98 on the TikTok dataset, surpassing the second-best method by 65%. v) MTVCrafter offers AI practitioners a new paradigm for pose-guided human video generation by enabling direct manipulation of 4D motion data for improved realism and control, though detailed architecture info is limited. |
| FinePhys: Fine-grained Human Action Generation by Explicitly |
|
|
| Incorporating Physical Laws for Effective Skeletal Guidance (Read more on arXiv or HuggingFace) |
Shengda Xu, Mingfei Shi, Dian Shao, Jason-Huang824, Harold328 |
FinePhys is a novel framework for generating physically plausible fine-grained human action videos using skeletal guidance and physics-based motion re-estimation. The research objective is to synthesize realistic and coherent human actions, particularly for challenging tasks like gymnastics routines. The methodology employs online 2D pose estimation, 2D-to-3D lifting via in-context learning, and a physics-based motion re-estimation module (PhysNet) governed by Euler-Lagrange equations for bidirectional temporal updating. Evaluated on three fine-grained action subsets from FineGym, FinePhys significantly outperforms competitive baselines achieving a CLIP-SIM* of 0.833 compared to AnimateDiff’s 0.752 on the FX-TURN dataset, thereby producing more natural human actions. FinePhys provides AI practitioners with a novel approach for incorporating physical constraints into generative models, potentially improving the realism and plausibility of generated human motion in various applications. |
| ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jie Zhou, fandong, Krystalan |
i) The paper introduces ExTrans, a multilingual neural machine translation (MT) model trained via reinforcement learning (RL) with a novel reward modeling approach. ii) The main research objective is to improve the translation quality of large reasoning models (LRMs) in both monolingual and multilingual settings by leveraging exemplar translations. iii) The methodology employs a new reward model that compares the policy MT model’s translations with those generated by a strong LRM (DeepSeek-R1) acting as an exemplar, combined with format verification for multilingual extension. iv) Experimental results show ExTrans-7B achieves state-of-the-art performance in English-to-Chinese literary translation, outperforming OpenAI-01 and DeepSeek-R1, with mExTrans-7B demonstrating competitive multilingual MT performance across 11 languages. v) This work provides AI practitioners with an effective RL-based training paradigm for MT that incorporates LLM-as-an-exemplar reward modeling and a lightweight multilingual transfer strategy, potentially reducing reliance on high-resource data and complex reward models. |
| SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning (Read more on arXiv or HuggingFace) |
Chunyan Miao, Xu Guo, Aver3, xuyige |
SoftCoT++ introduces a test-time scaling method for chain-of-thought reasoning in large language models by diversifying latent soft thought representations. The research aims to enhance LLM reasoning performance by enabling diverse exploration of thinking paths in the continuous latent space during inference. This is achieved by perturbing latent thoughts via multiple specialized initial tokens and applying contrastive learning to promote diversity among soft thought representations. Experiments on five reasoning benchmarks using LLaMA-3 and Qwen-3 show SoftCoT++ significantly boosts SoftCoT, outperforming SoftCoT with self-consistency scaling, with an average accuracy of 77.57% across all tasks using LLaMA-3.1-8B-Instruct. The primary implication is that AI practitioners can leverage SoftCoT++ to improve the reasoning capabilities of LLMs without retraining by scaling latent soft thought representations during inference, enhancing performance on complex reasoning tasks. |
| HISTAI: An Open-Source, Large-Scale Whole Slide Image Dataset for |
|
|
| Computational Pathology (Read more on arXiv or HuggingFace) |
Ekaterina Ivanova, alpchel, mgvz |
i) The HISTAI dataset, a large, open-access resource, is introduced for computational pathology research. ii) The main objective is to provide a diverse and richly annotated whole slide image (WSI) dataset to address limitations in existing resources. iii) The methodology involves curating over 60,000 WSIs from various tissue types, accompanied by comprehensive clinical metadata including diagnoses, demographics, pathological annotations, and ICD-10 codes. iv) The HISTAI dataset includes 57,647 slides at 20X magnification and 2,463 slides at 40X magnification, with 58,282 H&E stained slides, and aims to cover a wide array of organs and cancer types. v) The HISTAI dataset provides AI practitioners with a large, multimodal resource to develop more robust, generalizable, and clinically relevant AI solutions in digital pathology. |
| QVGen: Pushing the Limit of Quantized Video Generative Models (Read more on arXiv or HuggingFace) |
Jing Liu, HaotongQin, lvchengtao, Ruihao, Harahan |
i) This paper introduces QVGen, a novel quantization-aware training (QAT) framework for efficient video diffusion models (DMs) under extremely low-bit quantization. ii) The main research objective is to develop a QAT method that preserves the performance of video DMs under 4-bit or lower quantization, without introducing additional inference costs. iii) QVGen incorporates auxiliary modules to reduce quantization errors and employs a rank-decay strategy using singular value decomposition (SVD) and rank-based regularization to progressively eliminate these modules during training. iv) Experiments show that QVGen achieves full-precision comparable quality under 4-bit settings and significantly outperforms existing methods, for example, a 3-bit CogVideoX-2B achieves +25.28 improvement in Dynamic Degree on VBench. v) QVGen enables AI practitioners to deploy high-quality, low-bit video DMs with minimal performance degradation and zero inference overhead, providing a practical solution for resource-constrained environments. |
| From Grunts to Grammar: Emergent Language from Cooperative Foraging (Read more on arXiv or HuggingFace) |
Mingfei Sun, Wei Pan, Weicheng Tao, Rujikorn Charakorn, Maytus Piriyajitakonkij |
i) This paper introduces Foraging Games (FG), a multi-agent reinforcement learning (MARL) framework for studying emergent language under embodied and socially interdependent conditions. ii) The research aims to investigate how language emerges and adapts within multi-agent systems in response to ecological and cognitive constraints relevant to cooperative foraging. iii) The methodology involves training agents via Proximal Policy Optimization (PPO) in a partially observable grid world, where agents jointly learn actions and communication strategies from scratch without parameter sharing or a centralized critic. iv) Results show agents develop communication protocols exhibiting arbitrariness, interchangeability, displacement, cultural transmission, and compositionality, with agents achieving over 95% success rates across games; with population sizes greater than 2, agents achieved an Interchangeability approaching 1.0. v) The study provides AI practitioners with a decentralized MARL framework to study emergent communication, social dynamics, and the evolution of language in embodied agents, offering insights for designing more robust and adaptable AI systems in cooperative environments. |
| Tiny QA Benchmark++: Ultra-Lightweight, Synthetic Multilingual Dataset |
|
|
| Generation & Smoke-Tests for Continuous LLM Evaluation (Read more on arXiv or HuggingFace) |
vincentkoc |
i) The paper introduces Tiny QA Benchmark++ (TQB++), an ultra-lightweight LLM evaluation suite, expanding on the original TQB with synthetic data generation and multilingual support. ii) The main objective is to provide a rapid, low-cost method for continuous integration and deployment (CI/CD) and smoke-testing of LLMs. iii) The methodology includes a Python script for on-demand synthetic micro-benchmark generation in multiple languages with SHA-256 hashing for provenance, along with pre-built multilingual packs. iv) Empirical results show top-tier models achieve approximately 90% Exact Match accuracy on the core English set, with significant performance variations in low-resource languages. v) TQB++ enables AI engineers to quickly detect regressions and quality shifts in LLMOps workflows through lightweight unit testing, facilitating faster iteration and more robust LLM deployments. |
| HelpSteer3-Preference: Open Human-Annotated Preference Data across |
|
|
| Diverse Tasks and Languages (Read more on arXiv or HuggingFace) |
Felipe Soares, Hoo-Chang Shin, Olivier Delalleau, Jiaqi Zeng, Zhilin Wang |
i) The paper introduces HelpSteer3-Preference, a new open-source human-annotated preference dataset for training instruction-following language models. ii) The main objective is to improve the quality and diversity of available preference data for Reinforcement Learning from Human Feedback (RLHF). iii) The methodology involved collecting over 40,000 samples across diverse tasks including STEM, coding, and multilingual scenarios, utilizing specialist annotators. iv) Reward Models trained on HelpSteer3-Preference achieve 82.4% on RM-Bench and 73.7% on JudgeBench, a ~10% absolute improvement over existing RMs. v) AI practitioners can use HelpSteer3-Preference to train more effective reward models for aligning large language models, particularly in domains requiring specialized knowledge or multilingual capabilities. |
| Learned Lightweight Smartphone ISP with Unpaired Data (Read more on arXiv or HuggingFace) |
Radu Timofte, AndreiArhire |
i) This paper introduces a novel unpaired training method for a learnable Image Signal Processor (ISP) on smartphones. ii) The main objective is to develop a lightweight ISP that eliminates the need for pixel-wise aligned paired data. iii) The methodology involves a multi-term loss function guided by adversarial training with multiple discriminators processing feature maps from pre-trained networks. iv) Evaluated on the Zurich RAW to RGB dataset, the unpaired approach demonstrates potential and achieves high fidelity across evaluation metrics while maintaining a favorable perceptual quality as reflected by LPIPS scores. v) This unpaired training strategy allows AI practitioners to develop efficient ISPs without the costly acquisition of paired RAW and RGB images, enabling broader applications in resource-constrained environments like mobile devices. |
| Fast, Not Fancy: Rethinking G2P with Rich Data and Rule-Based Models (Read more on arXiv or HuggingFace) |
Hamid R. Rabiee, Zahra Dehghanian, Mahta Fetrat Qharabagh |
i) This paper addresses homograph disambiguation in grapheme-to-phoneme (G2P) conversion, particularly for low-resource languages like Persian. ii) The research aims to improve homograph disambiguation accuracy in both neural and rule-based G2P systems, while maintaining low latency for real-time applications. iii) A semi-automated pipeline for constructing a homograph-focused dataset (HomoRich) was developed, along with a lightweight statistical method to enhance G2P systems. iv) Fine-tuning a state-of-the-art neural G2P model (GE2PE) on HomoRich achieved a 29.72% improvement in homograph accuracy, and integrating the statistical method into eSpeak resulted in a 30.66% improvement in homograph disambiguation. v) The HomoRich dataset and the statistical disambiguation method provide AI practitioners with resources to enhance the accuracy of both neural and rule-based G2P systems, particularly in low-resource scenarios where real-time performance is critical for applications like screen readers. |
| LLM Context Conditioning and PWP Prompting for Multimodal Validation of |
|
|
| Chemical Formulas (Read more on arXiv or HuggingFace) |
PChemGuy |
LLM Context Conditioning and PWP Prompting for Multimodal Validation of Chemical Formulas investigates methods for improving Large Language Model (LLM) accuracy in identifying errors within scientific documents, specifically chemical formulas. The main objective is to enhance the reliability of general-purpose LLMs for precise validation tasks, focusing on chemical formulas validation within a test paper containing known errors. Persistent Workflow Prompting (PWP) principles informed structured LLM context conditioning was used to modulate LLM behavior at inference time via chat interfaces of Gemini 2.5 Pro and ChatGPT Plus 03. PWP-informed context conditioning improved textual error identification with both models, and Gemini 2.5 Pro repeatedly identified a subtle image-based formula error previously overlooked. This study implies that PWP-informed context conditioning can enhance LLM-driven analytical workflows, particularly for tasks requiring meticulous error detection in scientific and technical documents. |
| TechniqueRAG: Retrieval Augmented Generation for Adversarial Technique |
|
|
| Annotation in Cyber Threat Intelligence Text (Read more on arXiv or HuggingFace) |
mparvez, TahaSencar, utsavshukla, lekssays |
i) TECHNIQUERAG is a domain-specific retrieval-augmented generation framework for automating adversarial technique annotation in security texts. ii) The paper addresses the research question of how to accurately identify adversarial techniques in security texts without extensive labeled data or task-specific optimizations. iii) The framework integrates off-the-shelf retrievers, instruction-tuned LLMs, and minimal text-technique pairs, using LLM re-ranking to enhance domain specificity. iv) Experiments demonstrate state-of-the-art performance on multiple security benchmarks, with TECHNIQUERAG achieving a F1 score of 91.09% on Procedures. v) The principal implication for AI practitioners is a novel approach to improving the precision of RAG systems in specialized domains with limited data. |
| AI-Driven Scholarly Peer Review via Persistent Workflow Prompting, |
|
|
| Meta-Prompting, and Meta-Reasoning (Read more on arXiv or HuggingFace) |
PChemGuy |
This paper explores AI-driven scholarly peer review using large language models (LLMs). The research question focuses on developing a persistent workflow prompting (PWP) methodology for critical analysis by LLMs, especially for experimental chemistry manuscripts. The key methodology involves creating a hierarchical, modular prompt architecture (structured via Markdown) and iteratively refining the prompt through meta-prompting and meta-reasoning. The primary result demonstrates the LLM’s ability to identify major methodological flaws, distinguishing claims from evidence, and performing quantitative feasibility checks on a test case. The principal implication for AI practitioners lies in the potential of PWP to enable sophisticated analysis of complex scientific tasks using readily available LLMs, reducing the need for custom-tailored models or extensive training data. |
Papers for 2025-05-19
| Title | Authors | Summary |
|——-|———|———|
| Qwen3 Technical Report (Read more on arXiv or HuggingFace)| huybery, BeichenZhang, Baosong, laf070810, yangapku | i) Qwen3, the latest iteration of the Qwen model family, is introduced as a series of open-source large language models designed for enhanced performance, efficiency, and multilingual capabilities. ii) The objective is to advance performance, efficiency, and multilingual capabilities in large language models. iii) The methodology involves pre-training on 36 trillion tokens, integrating thinking and non-thinking modes, and a multi-stage post-training approach including long chain-of-thought finetuning, reinforcement learning, and strong-to-weak distillation. iv) Qwen3-235B-A22B achieves 85.7 on AIME’24 and expands multilingual support to 119 languages, demonstrating state-of-the-art results across diverse benchmarks. v) AI practitioners can leverage Qwen3’s unified thinking mode and thinking budget mechanism to adaptively allocate computational resources during inference, balancing latency and performance based on task complexity. |
| MMLongBench: Benchmarking Long-Context Vision-Language Models
Effectively and Thoroughly (Read more on arXiv or HuggingFace)| Yu Zhao, Jipeng Zhang, Xiyu Ren, Wenhao Yu, Zhaowei Wang | i) MMLONGBENCH is introduced as the first benchmark for evaluating long-context vision-language models (LCVLMs). ii) The research aims to provide an effective and thorough evaluation of LCVLMs across a diverse set of tasks. iii) The methodology involves curating a dataset of 13,331 examples spanning five downstream task categories and assessing 46 closed-source and open-source LCVLMs across standardized input lengths. iv) Results indicate that performance on a single task is a weak proxy for overall long-context capability, and models with stronger reasoning exhibit better long-context performance; at 128K tokens, even GPT-4o only achieves 62.9% on average. v) The benchmark and associated analysis highlight the need for improved vision-language long-context capabilities and a more comprehensive evaluation approach for future LCVLM development. |
| GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning (Read more on arXiv or HuggingFace)| Tri Cao, Yulin Chen, Mingzhe Du, Shengfang Zhai, Yue Liu | i) GuardReasoner-VL, a novel reasoning-based VLM guard model, is introduced to enhance safety by incentivizing deliberative reasoning before moderation decisions via online reinforcement learning (RL). ii) The primary research objective is to improve the safety of Vision-Language Models (VLMs) without compromising their core capabilities by developing a guard model that reasons about harmful content before moderating. iii) The methodology involves constructing a reasoning corpus, GuardReasoner-VLTrain, with 123K samples and 631K reasoning steps, followed by supervised fine-tuning (SFT) and online RL with safety-aware data concatenation and a dynamic clipping parameter. iv) Experiments demonstrate that GuardReasoner-VL surpasses the runner-up by 19.27% F1 score on average on multi-modal guardrail benchmarks. v) The principal implication for AI practitioners is a new, reasoning-based approach to VLM safety that can be implemented via online RL, offering a potential framework for developing more robust and interpretable guard models. |
| Visual Planning: Let’s Think Only with Images (Read more on arXiv or HuggingFace)| ivulic, akorhonen, caiqizh, masonxw, hzhouml | i) The paper introduces Visual Planning, a novel paradigm for machine reasoning using solely visual representations. ii) The main objective is to investigate whether models can effectively plan through visual representations without textual mediation. iii) The methodology involves a reinforcement learning framework, Visual Planning via Reinforcement Learning (VPRL), utilizing GRPO for post-training large vision models. iv) Experiments on spatial navigation tasks (FROZENLAKE, MAZE, MINIBEHAVIOR) show that VPRL achieves over 40% higher average exact-match rate compared to supervised fine-tuning (SFT). v) Visual Planning offers AI practitioners a viable alternative to language-based reasoning for tasks with inherent spatial or geometric properties, potentially reducing the modality gap in multimodal tasks. |
| Simple Semi-supervised Knowledge Distillation from Vision-Language
Models via texttt{D}ual-texttt{H}ead
texttt{O}ptimization (Read more on arXiv or HuggingFace)| Sung Ju Hwang, Hyungjoon Jang, Seongjae Kang, dongboklee | i) This paper introduces Dual-Head Optimization (DHO), a knowledge distillation framework for vision-language models in semi-supervised learning. ii) The objective is to transfer knowledge from large VLMs to compact, task-specific models while addressing gradient conflicts in semi-supervised settings. iii) DHO utilizes dual prediction heads independently trained with supervised and distillation losses and combines their outputs linearly at inference. iv) Experiments show DHO improves accuracy by 3% on ImageNet with 1% labeled data compared to existing methods. v) DHO offers AI practitioners a more efficient distillation method that mitigates gradient conflicts, improving feature learning for knowledge transfer in resource-constrained environments. |
| Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token
Level Granularity (Read more on arXiv or HuggingFace)| Yi-Chang Chen, Feng-Ting Liao, Jamie McGowan, Davide Buffelli, Splend1dchan | i) The paper introduces Group Think, a novel LLM inference paradigm enabling token-level collaborative reasoning among concurrent agents to improve quality and latency. ii) The primary objective is to develop a more efficient and higher-quality reasoning framework for LLMs by leveraging multiple concurrent reasoning agents. iii) The methodology involves modifying existing LLMs to support multiple interdependent, parallel reasoning trajectories with token-level adaptation among agents, evaluated on enumeration, divide-and-conquer, and coding tasks. iv) Empirical results show Group Think improves reasoning accuracy while reducing latency on open-source LLMs, demonstrating an acceleration of roughly N (number of thinkers) times faster than CoT, with Completion Coverage becoming near saturated. v) Group Think offers AI practitioners a method for enhancing reasoning performance, especially in resource-constrained edge inference scenarios, by efficiently utilizing idle computational resources through concurrent reasoning. |
| Mergenetic: a Simple Evolutionary Model Merging Library (Read more on arXiv or HuggingFace)| erodola, crisostomi, teelinsan, tmencatt, adrianrob | i) Mergenetic is introduced as an open-source library for evolutionary model merging in LLMs. ii) The research aims to facilitate experimentation with evolutionary algorithms and merging methods, while reducing the computational cost of fitness evaluations. iii) The library integrates 19 evolutionary algorithms and 6 merging strategies, incorporating dataset subsampling and fitness approximation techniques. iv) Experiments demonstrate that Mergenetic achieves competitive results across tasks and languages, with merged models outperforming language-specific constituents by up to 19% on the ARC-Challenge benchmark. v) The library’s modular design and user-friendly interfaces (Python API, CLI, GUI) enable AI practitioners to efficiently explore high-quality model compositions on consumer-grade GPUs. |
| MPS-Prover: Advancing Stepwise Theorem Proving by Multi-Perspective
Search and Data Curation (Read more on arXiv or HuggingFace)| Tao Yang, Yang Li, haitaominlp, freesunshine0316, invokerliang | i) The paper introduces MPS-Prover, a stepwise automated theorem proving system utilizing multi-perspective search and data curation. ii) The main objective is to improve stepwise theorem proving performance by mitigating biased search guidance and enhancing exploration. iii) The methodology involves a post-training data curation strategy to prune redundant training data and a multi-perspective tree search mechanism integrating a learned critic with heuristic rules. iv) MPS-Prover achieves a 75.82% accuracy on the miniF2F benchmark, surpassing previous stepwise provers, and obtains a 32.97% success rate on ProofNet. v) The work provides AI practitioners with a robust framework for developing more powerful theorem provers and demonstrates the efficacy of combining learned critics with heuristic search in formal reasoning systems. |
| Multi-Token Prediction Needs Registers (Read more on arXiv or HuggingFace)| Nikos Komodakis, Spyros Gidaris, nasos10 | i) The paper introduces MuToR, a novel multi-token prediction method for improving language model pretraining and finetuning by interleaving learnable register tokens into input sequences. ii) The main objective is to develop a multi-token prediction approach that enhances autoregressive transformers without architectural changes, enabling scalable prediction horizons and preserving compatibility with pretrained models. iii) The method involves training register tokens to predict future targets at varying offsets while using a designed attention mask to maintain the standard next-token prediction for regular tokens. iv) Experiments on language modeling show that MuToR improves performance in supervised and parameter-efficient finetuning, surpassing standard baselines under equivalent compute; in mathematical reasoning with Gemma 2B, MuToR achieved 42.10% accuracy on GSM8K, outperforming Next-Token at 38.87%. v) MuToR provides AI practitioners with a readily integrable technique for improving model performance and training efficiency in generative tasks across both language and vision domains, particularly in scenarios benefiting from enhanced forward-looking context during training. |
| Learning Dense Hand Contact Estimation from Imbalanced Data (Read more on arXiv or HuggingFace)| kyoungmu, dqj5182 | Learning dense hand contact estimation is addressed by mitigating class and spatial imbalance in hand interaction datasets. The research aims to improve dense hand contact estimation by addressing class and spatial imbalance issues in training data. Balanced Contact Sampling (BCS) constructs multiple sampling groups to represent diverse contact statistics, while Vertex-Level Class-Balanced (VCB) loss reweights loss contribution of each vertex based on its contact frequency. The method achieves improved performance in dense hand contact estimation across diverse scenarios, evidenced by a 10.4% increase in F1-score on the MOW dataset compared to models without BCS. AI practitioners can use the proposed techniques to effectively train hand contact estimation models on imbalanced datasets, improving performance in areas such as robotics and AR/VR. |
| Scaling Reasoning can Improve Factuality in Large Language Models (Read more on arXiv or HuggingFace)| rubis, bjerva, jjzha | i) This paper investigates methods for improving factual accuracy in LLMs through reasoning and knowledge graph integration. ii) The research question is to what extent long reasoning influences factual generalization capabilities of large language models on complex multi-hop questions. iii) The methodology involves distilling reasoning traces from QwQ-32B and DeepSeek-R1-671B, fine-tuning Qwen2.5 models with these traces, and incorporating knowledge graph paths via Wikidata. iv) The primary result shows that smaller instruction-tuned models can improve factual accuracy with KG-enhanced reasoning traces, and increasing test-time compute by parallel scaling improves factual accuracy by 2-8%. v) The principal implication is that within a single run, smaller reasoning models can achieve improvements in factual accuracy compared to their original instruction-tuned counterparts in Open-Domain QA. |
| Humans expect rationality and cooperation from LLM opponents in
strategic games (Read more on arXiv or HuggingFace)| Miguel Costa-Gomes, Darija Barak | i) This paper investigates human strategic behavior in p-beauty contests against LLM opponents. ii) The study aims to understand how human choices differ when playing against LLMs versus other humans in a multiplayer game setting without dominant strategies. iii) The methodology involves a monetarily-incentivized laboratory experiment using a within-subject design to compare behavior against human and LLM (ChatGPT 3.5 and Claude v2) opponents. iv) Results show that subjects choose significantly lower numbers against LLMs, driven by an increased rate of zero choices, with 15.3% making zero choices against LLMs compared to 4.2% against humans; additionally, 16.7% of subjects were classified as possessing high strategic reasoning ability. v) AI practitioners should account for the potential of humans to overestimate LLMs’ strategic sophistication or cooperativeness when designing human-AI interactive systems, impacting mechanism design and agent behavior predictions. The paper does not contain a clearly-defined quantitative measure related to cooperation. |
| MatTools: Benchmarking Large Language Models for Materials Science Tools (Read more on arXiv or HuggingFace)| David J. Srolovitz, Bo Hu, Beilin Ye, Jiamin Xu, SiyuLiu | MatTools introduces a benchmark to evaluate large language models (LLMs) proficiency in materials science through code generation and execution. The research aims to assess LLMs’ ability to answer materials science questions by generating codes based on physics-based computational materials science packages. MatTools employs a two-component framework: a materials simulation tool question-answer (QA) benchmark with 69,225 pairs from pymatgen and a real-world tool-usage benchmark of 49 tasks (138 subtasks). Evaluation of various LLMs revealed that general-purpose LLMs significantly outperformed materials science-focused LLMs, achieving 80% versus <32% accuracy in QA tasks, and LLM-generated documentation substantially improved performance in retrieval-augmented generation (RAG) systems. The principal implication for AI practitioners is the demonstration of leveraging LLM-generated documentation and self-reflection mechanisms to enhance LLM tool-use abilities in technical domains like materials science, potentially guiding the development of more effective AI systems for scientific research, while highlighting the limitations of domain-specific LLMs. |
Papers for 2025-05-16
| Title |
Authors |
Summary |
| Beyond ‘Aha!’: Toward Systematic Meta-Abilities Alignment in Large |
|
|
| Reasoning Models (Read more on arXiv or HuggingFace) |
cxiong, amritasaha87, yuhuixu, hendrydong, zhiyuanhucs |
Large reasoning models are aligned with deduction, induction, and abduction meta-abilities to enhance reasoning capabilities. The research aims to improve the scalability and reliability of large reasoning models (LRMs) by explicitly aligning them with meta-abilities rather than relying on emergent behaviors. The methodology involves a three-stage pipeline: individual alignment with meta-abilities using automatically generated tasks, parameter-space merging, and domain-specific reinforcement learning. The proposed method boosts performance by over 10% compared to instruction-tuned baselines and achieves an additional 2% average gain through domain-specific RL. Explicit meta-ability alignment offers a scalable foundation for reasoning in large models. The paper seems to lack comparative result or reference on how many parameters were used in the baseline model. |
| System Prompt Optimization with Meta-Learning (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, jinheon, YuminChoi |
i) This paper introduces a meta-learning framework, MetaSPO, for optimizing task-agnostic system prompts to improve Large Language Model (LLM) performance across diverse user prompts and unseen tasks. ii) The research objective is to develop a bilevel system prompt optimization method that designs system prompts robust to varying user prompts and transferable to new tasks. iii) MetaSPO uses a meta-learning approach to optimize system prompts over multiple datasets, iteratively updating user prompts to enhance synergy. iv) Experiments across 14 unseen datasets in 5 domains demonstrated that MetaSPO produces generalizable system prompts and facilitates rapid adaptation, achieving performance improvements and reducing optimization steps. v) MetaSPO offers AI practitioners a method to enhance LLM generalization and adaptation by automating the design of robust system prompts. |
| EnerVerse-AC: Envisioning Embodied Environments with Action Condition (Read more on arXiv or HuggingFace) |
hsli-cuhk, pathcn, thuhsy, Shengcong, YuxinJiang |
ENERVERSE-AC is an action-conditional world model for generating future visual observations in robotic manipulation. The research aims to create a realistic and controllable robotic inference environment by conditioning video generation on predicted agent actions. The methodology introduces a multi-level action-conditioning mechanism, ray map encoding for dynamic multi-view image generation, and failure trajectory expansion. Experiments showed that using the generated data for data augmentation improved policy training success rate from 0.28 to 0.36. This work reduces the cost of robotic manipulation testing by providing an alternative to physical robots or complex simulations for AI practitioners involved in robotics and imitation learning. |
| The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a |
|
|
| Reasoning Model will Think (Read more on arXiv or HuggingFace) |
hbin0701, dreamgonfly, Minju2136, seungone, Seongyun |
i) The paper introduces the CoT ENCYCLOPEDIA, a framework for analyzing, predicting, and controlling reasoning strategies in large language models (LLMs) using chain-of-thought (CoT) prompting. ii) The primary objective is to provide a bottom-up methodology to identify and interpret diverse reasoning strategies employed by LLMs, circumventing limitations of predefined strategy types. iii) The method extracts reasoning criteria from model-generated CoTs, embeds them into a semantic space, clusters them into representative categories, and derives contrastive rubrics to interpret reasoning behavior. iv) The framework achieves a 92–97% perceived reasonableness score in human evaluations and improves performance by 2.5–8.3% on various benchmarks by guiding models towards more effective reasoning strategies; it also reveals that training data format (free-form vs. multiple choice) has a greater impact than data domain on reasoning behaviors. v) The principal implication for AI practitioners is the provision of a diagnostic and practical tool for shaping reasoning behaviors in LLMs, particularly by selecting appropriate training data formats and enabling controllability through model merging. |
| EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied |
|
|
| World Models (Read more on arXiv or HuggingFace) |
sundrops, AutobotZero, pathcn, Shengcong, thuhsy |
EWMBench is introduced as a benchmark for evaluating embodied world models (EWMs). The research addresses the challenge of evaluating EWMs beyond general perceptual metrics, focusing on physically grounded and action-consistent behaviors. The methodology involves a curated dataset with diverse scenes/motion patterns and a multi-dimensional evaluation toolkit assessing visual scene consistency, motion correctness, and semantic alignment. The evaluation metrics exposed that current video generation models have limitations when used for embodied tasks. The benchmark and evaluation tools are publicly available to guide future development in the field, although quantitative results are not explicitly stated in the provided abstract. |
| End-to-End Vision Tokenizer Tuning (Read more on arXiv or HuggingFace) |
RobertLuo1, Paranioar, YufengCui, ryanzhangfan, gilnore |
i) The paper introduces ETT, an end-to-end approach for tuning vision tokenizers jointly with downstream autoregressive tasks. ii) The main research question is whether jointly optimizing vision tokenization and target autoregressive tasks improves performance compared to using frozen vision tokenizers. iii) The methodology involves leveraging the visual embeddings of the tokenizer codebook and optimizing the vision tokenizer with both reconstruction and caption objectives, without adjusting the LLM codebook or architecture. iv) The primary results demonstrate performance gains of 2-6% on multimodal understanding and visual generation tasks compared to frozen tokenizer baselines, while preserving reconstruction capability. v) ETT provides AI practitioners a simple method for improving multimodal foundation models by end-to-end vision tokenizer tuning, enhancing performance in image generation and understanding tasks without significantly altering existing LLM architectures. |
| MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine |
|
|
| Learning Engineering (Read more on arXiv or HuggingFace) |
percyliang, Solute, yinghaoli-yh, yczhuang, Jerrycool |
i) MLE-Dojo is introduced as a Gym-style framework for reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents in machine learning engineering (MLE) workflows. ii) The research aims to provide an interactive environment for iterative experimentation, debugging, and refinement of LLM agent solutions in real-world MLE tasks. iii) The methodology involves building a fully executable environment upon 200+ real-world Kaggle challenges, facilitating comprehensive agent training via supervised fine-tuning and reinforcement learning. iv) Evaluations of eight frontier LLMs show iterative improvements, but also reveal limitations in autonomously generating long-horizon solutions and resolving complex errors, with HumanRank scores and Elo rankings presented as evaluation metrics. v) MLE-Dojo provides AI practitioners with a benchmark to tune model-based agents through diverse data sources, tools, and evaluation protocols to improve interoperability, scalability, and reproducibility. |
| WorldPM: Scaling Human Preference Modeling (Read more on arXiv or HuggingFace) |
Zhenru Zhang, Le Yu, Keming Lu, Runji Lin, Binghai Wang |
i) This paper introduces World Preference Modeling (WorldPM) for scaling human preference models. ii) The primary objective is to investigate scaling laws in preference modeling using large datasets and models. iii) The research methodology involves training language models (1.5B to 72B parameters) on a 15M-sample dataset of human preferences gathered from online forums. iv) Results indicate that adversarial metrics scale with increased data and model size, objective metrics show emergent behavior in larger models, and integrating WorldPM into RLHF pipelines improved evaluations by 4% to 8% in in-house evaluations. v) WorldPM offers AI practitioners a foundation for improving the generalization performance of preference fine-tuning across various datasets, especially with limited data. |
| Achieving Tokenizer Flexibility in Language Models through Heuristic |
|
|
| Adaptation and Supertoken Learning (Read more on arXiv or HuggingFace) |
Vinayak Pahalwan, Shaurya Sharthak, adarshxs, adi-kmt |
i) This paper introduces TokenAdapt, a framework for adapting language models to new tokenizers using a hybrid heuristic initialization and explores supertoken learning. ii) The main objective is to mitigate tokenizer lock-in issues in LLMs by facilitating efficient tokenizer transplantation without substantial retraining. iii) The methodology involves a hybrid heuristic combining local subword decomposition and global semantic similarity, alongside pre-tokenization learning of multi-word supertokens. iv) Empirical results demonstrate that TokenAdapt consistently yields lower perplexity ratios compared to ReTok and TransTokenizer baselines, achieving up to approximately a 2-fold improvement in aggregate perplexity scores. v) TokenAdapt offers AI practitioners a practical method for adapting LLMs to specialized domains or languages by minimizing retraining costs and improving tokenization efficiency. |
| Style Customization of Text-to-Vector Generation with Image Diffusion |
|
|
| Priors (Read more on arXiv or HuggingFace) |
Jing Liao, CHERRY-Z, intchous |
i) This paper introduces a two-stage pipeline for style-customizable text-to-vector graphic (SVG) generation leveraging diffusion models. ii) The research aims to enable style customization in text-to-vector generation while preserving structural regularity in the resulting SVGs. iii) The methodology involves training a path-level T2V diffusion model followed by style distillation from customized image diffusion models. iv) The model achieved a Path FID of 37.51, indicating improved structural regularity, and user studies showed preference for the method’s visual quality at 53.2%. v) AI practitioners can use this model to generate stylized vector graphics from text prompts, enabling efficient content creation with consistent visual aesthetics. |
| J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Xian Li, Ping Yu, Tianlu Wang, Chenxi Whitehouse, swarna92 |
i) The paper introduces J1, a reinforcement learning (RL) method for training LLM-as-a-Judge models to improve reasoning and judgment capabilities. ii) The research objective is to develop a method that incentivizes thinking and mitigates judgment bias in LLMs used for evaluation tasks. iii) The methodology involves converting both verifiable and non-verifiable prompts into judgment tasks with verifiable rewards and using Group Relative Policy Optimization (GRPO) for training. iv) Results show that the J1 method outperforms existing 8B and 70B models on several benchmarks, including PPE, with J1-Llama-70B achieving an overall accuracy of 69.6, and even surpasses R1 on certain non-verifiable tasks. v) The main implication for AI practitioners is the provision of an effective RL approach for creating generalist judge models capable of evaluating diverse LLM responses, which can be used for improving all stages of LLM development. |
| PointArena: Probing Multimodal Grounding Through Language-Guided |
|
|
| Pointing (Read more on arXiv or HuggingFace) |
Boyang Li, Haoquan Fang, Yi Ru Wang, Jiafei Duan, Long Cheng |
PointArena is introduced as a comprehensive platform for evaluating multimodal pointing capabilities in AI systems. The research aims to provide a benchmark for assessing how well multimodal models can ground language within visual contexts using pointing. The methodology involves a curated dataset (Point-Bench) with approximately 1,000 pointing tasks, an interactive arena (Point-Battle) for pairwise model comparisons, and a real-world robotic manipulation system (Point-Act). The results showed Molmo-72B consistently outperformed other models and supervised training significantly enhances model performance; Point-Battle has gathered over 4,500 anonymized votes. The implication for AI practitioners is the critical role of precise pointing capabilities in bridging abstract reasoning with concrete actions, providing a benchmark to improve and evaluate multimodal models for robotics, assistive technology, and interactive AI systems. |
| Depth Anything with Any Prior (Read more on arXiv or HuggingFace) |
Ziang Zhang, Jialei Wang, Lihe Yang, Siyu Chen, sleetwang6 |
i) The paper introduces Prior Depth Anything, a framework for generating accurate and dense metric depth maps by integrating incomplete metric measurements with relative depth predictions. ii) The objective is to develop a depth estimation model that can effectively utilize diverse and potentially incomplete depth priors to produce detailed and accurate metric depth maps. iii) The methodology involves a coarse-to-fine pipeline with pixel-level metric alignment and a conditioned monocular depth estimation model. iv) The model achieves state-of-the-art zero-shot performance across depth completion, super-resolution, and inpainting tasks on 7 real-world datasets. v) Prior Depth Anything provides AI practitioners with a flexible approach to improve depth estimation in various applications by effectively utilizing available depth priors and allows test-time improvements with different model settings. |
| OpenThinkIMG: Learning to Think with Images via Visual Tool |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Zhengyuan Yang, Yunzhuo Hao, Mingyang Song, Linjie Li, Zhaochen Su |
i) The paper introduces OPENTHINKIMG, a framework for tool-augmented Large Vision-Language Models (LVLMs), and V-TOOLRL, a reinforcement learning method for adaptive tool use. ii) The primary objective is to enable LVLMs to learn dynamic policies for invoking external vision tools to improve visual reasoning. iii) The methodology involves supervised fine-tuning (SFT) for policy initialization and a reinforcement learning (RL) framework, V-TOOLRL, for adaptive tool usage based on feedback from tool interactions. iv) Results show a +28.83 accuracy point improvement over the SFT-initialized counterpart on chart reasoning tasks and surpasses GPT-4.1 by +8.68 accuracy points. v) AI practitioners can utilize OPENTHINKIMG and V-TOOLRL to develop more robust LVLMs capable of dynamic, tool-augmented visual reasoning for solving complex multimodal tasks. |
| AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection (Read more on arXiv or HuggingFace) |
Weixi Zhang, Yuezhi Cai, Jiangtao Yan, Yue Zhu, Bin-Bin Gao |
AdaptCLIP adapts CLIP for universal visual anomaly detection across unseen domains. The research aims to develop a zero-/few-shot anomaly detection model requiring no target domain fine-tuning. AdaptCLIP employs alternately learned visual and textual representations with contextual and aligned residual feature comparative learning. It achieves state-of-the-art performance on 12 anomaly detection benchmarks, including a 10%+ improvement in pixel AUPR on challenging datasets. This method offers a flexible and adaptable approach for anomaly detection in practical AI applications by significantly improving zero-shot performance. |
| ReSurgSAM2: Referring Segment Anything in Surgical Video via Credible |
|
|
| Long-term Tracking (Read more on arXiv or HuggingFace) |
Guanyi Qin, Ziyue Wang, Xuxiao Luo, Mingqi Gao, HeverLaw |
i) ReSurgSAM2 is introduced as a two-stage framework for surgical referring segmentation, integrating text-referred target detection with long-term tracking using Segment Anything Model 2 (SAM2). ii) The paper addresses the limitations of existing surgical video segmentation methods by improving efficiency and long-term tracking in complex surgical scenarios. iii) The method uses a cross-modal spatial-temporal Mamba (CSTMamba) for precise detection and credible initial frame selection (CIFS), followed by a diversity-driven long-term memory (DLM) mechanism for consistent object tracking. iv) The model achieves real-time performance at 61.2 FPS with substantial improvements in accuracy and efficiency compared to existing methods. v) This work provides AI practitioners with a more accurate and efficient method for surgical video analysis, enabling enhanced interactive surgical tools and real-time decision support systems. |
| QuXAI: Explainers for Hybrid Quantum Machine Learning Models (Read more on arXiv or HuggingFace) |
Rafiul Islam, Md Jafor Sadek, Shehenaz Khaled, imostafizur, AlignAI |
i) The paper introduces QuXAI, a framework with a Q-MEDLEY explainer, for interpreting feature importance in hybrid quantum-classical machine learning (HQML) models. ii) The research aims to provide robust global and local explainability for HQML architectures involving quantum feature encoding and classical learning. iii) The methodology involves constructing HQML models, using the Q-MEDLEY explainer that synthesizes Drop-Column Importance (DCI) and Permutation Importance (PI), and visualizing the results. iv) Results show Q-MEDLEY effectively identifies influential classical aspects in HQML models, distinguishing them from noise, and achieves high Recall@3 scores in classical ML settings. v) QuXAI offers AI practitioners a means to improve interpretability and reliability in HQML models by revealing feature contributions throughout the hybrid processing pipeline. |
| Exploring the Deep Fusion of Large Language Models and Diffusion |
|
|
| Transformers for Text-to-Image Synthesis (Read more on arXiv or HuggingFace) |
Saining Xie, Sayak Paul, Xichen Pan, Boyang Zheng, Bingda Tang |
i) This paper explores deep fusion of large language models (LLMs) and diffusion transformers (DiTs) for text-to-image synthesis. ii) The main objective is to conduct an empirical study on text-to-image generation, analyzing design choices, and providing a reproducible recipe for training. iii) The methodology involves controlled comparisons with shallow fusion baselines, where text representations are integrated into each DiT layer from a single LLM layer using late fusion. iv) Results show the deep fusion model achieves better image-text alignment (GenEval 0.51) compared to self-attention DiT (GenEval 0.42) and competitive inference latency; also removing timestep conditioning improves FID. v) The principal implication is a practical recipe for text-to-image synthesis using deep fusion that is competitive with alternative approaches, also LLM and DiT model designs can be effectively decoupled enabling application of separate scaling laws and design principles. |
| Parallel Scaling Law for Language Models (Read more on arXiv or HuggingFace) |
Dayiheng Liu, Jiaxi Yang, Zeyu Cui, Binyuan Hui, Mouxiang Chen |
i) The paper introduces parallel scaling (PARSCALE), a novel method to scale language models by increasing parallel computation. ii) The research aims to improve language model scaling efficiency by increasing parallel computation instead of solely relying on parameter or inference-time scaling. iii) PARSCALE employs multiple learnable transformations on the input, processes them in parallel, and dynamically aggregates the outputs. iv) Experiments show that PARSCALE achieves performance similar to scaling parameters by O(log P), but with up to 22x less memory increase and 6x less latency increase compared to parameter scaling. v) PARSCALE offers AI practitioners a memory-efficient strategy for deploying more powerful language models in low-resource scenarios through increased parallel computation rather than extensive parameter scaling. |
| MetaUAS: Universal Anomaly Segmentation with One-Prompt Meta-Learning (Read more on arXiv or HuggingFace) |
csgaobb |
i) This paper introduces MetaUAS, a novel framework for universal anomaly segmentation utilizing a pure vision model and one-prompt meta-learning. ii) The main objective is to develop a universal anomaly segmentation model that can effectively segment any novel or unseen visual anomalies using only a single normal image prompt. iii) The methodology involves unifying anomaly segmentation into change segmentation, leveraging synthetic image pairs for training, and employing a soft feature alignment module to handle geometrical variations. iv) MetaUAS achieves state-of-the-art performance on three industrial anomaly benchmarks; specifically, it outperforms existing methods while using 10x fewer parameters and demonstrating a 100x speed improvement over WinCLIP+. v) MetaUAS offers AI practitioners a novel, efficient, and training-free approach to anomaly segmentation by providing an alternative to vision-language models, relying instead on a pure vision model and synthesized training data for improved generalization. |
| Learning to Detect Multi-class Anomalies with Just One Normal Image |
|
|
| Prompt (Read more on arXiv or HuggingFace) |
csgaobb |
i) This paper presents OneNIP, a unified anomaly detection framework using a single normal image prompt to enhance reconstruction-based anomaly detection. ii) The research aims to improve the performance and generalization of unified anomaly detection models by guiding feature reconstruction with a normal image prompt. iii) The proposed methodology involves unsupervised reconstruction and restoration streams, combined with a supervised refiner that regresses reconstruction errors, using both real normal and synthesized anomalous images. iv) The OneNIP method outperforms previous methods on industry anomaly detection benchmarks, achieving a pixel-level anomaly segmentation performance of 63.7% on MVTec, a significant improvement over UniAD’s 44.7%. v) OneNIP provides AI practitioners with an effective approach to multi-class anomaly detection using a single normal image prompt, offering improved accuracy and faster convergence compared to existing reconstruction-based methods, particularly beneficial for industrial applications. |
| Few-Shot Anomaly-Driven Generation for Anomaly Classification and |
|
|
| Segmentation (Read more on arXiv or HuggingFace) |
Yunsheng Wu, Chengjie Wang, Jun Liu, Guan Gui, csgaobb |
i) The paper introduces AnoGen, a few-shot anomaly-driven generation method to synthesize realistic and diverse anomaly samples for training anomaly detection models. ii) The research aims to address the problem of scarce anomaly samples by generating synthetic anomalies guided by a few real anomalies to improve anomaly classification and segmentation. iii) The methodology involves learning anomaly distribution from a few real anomalies, guiding a diffusion model using embeddings and bounding boxes to generate synthetic anomalies, and weakly-supervised training of anomaly detection models. iv) Experiments on MVTec dataset show that DRAEM and DesTSeg with AnoGen achieved a 5.8% and 1.5% improvement in AU-PR metric on segmentation, respectively. v) AI practitioners can leverage AnoGen to augment limited anomaly datasets with realistic synthetic data, leading to enhanced performance of anomaly detection models, particularly in segmentation tasks. |
Papers for 2025-05-15
| Title |
Authors |
Summary |
| DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception (Read more on arXiv or HuggingFace) |
Yichi Chen, Bin Kang, Yulin Li, Bin Chen, xiaomoguhzz |
i) DeCLIP enhances CLIP for open-vocabulary dense prediction by decoupling and separately optimizing content and context features. ii) The research addresses CLIP’s limitations in local feature representation for dense prediction tasks due to its image tokens’ inability to effectively aggregate information from spatially/semantically related regions. iii) It employs a decoupled distillation design, fine-tuning “content” features with image crop representations and “context” features under guidance from vision foundation models. iv) Experiments show DeCLIP outperforms existing methods across tasks like object detection and semantic segmentation; DeCLIP improves F-ViT on OV-COCO by 3.5 mAP for novel classes. v) DeCLIP offers AI practitioners an unsupervised method to improve CLIP’s local feature discriminability and spatial consistency, enhancing performance in open-vocabulary dense prediction tasks, facilitating integration in downstream applications. |
| BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, |
|
|
| Training and Dataset (Read more on arXiv or HuggingFace) |
Zhiyang Xu, Jiuhai Chen, xurantju, zhoutianyi, xcpan |
i) The paper presents BLIP3-0, a family of unified multimodal models for image understanding and generation. ii) The research aims to determine the optimal model architecture and training strategy for unified multimodal frameworks, particularly focusing on image generation. iii) The methodology involves a diffusion transformer for generating CLIP image features and a sequential pretraining strategy of first training on image understanding and then image generation. iv) BLIP3-0 achieves a GenEval score of 0.84, with the 8B model scoring 1682.6 on MME-P and 50.6 on MMMU; a human study indicates statistical confidence for improved visual quality and prompt alignment versus Janus Pro. v) The results suggest that CLIP image features coupled with flow matching and a sequential training strategy can enhance performance in unified multimodal tasks, indicating practical advantages for AI practitioners developing similar unified models. |
| Insights into DeepSeek-V3: Scaling Challenges and Reflections on |
|
|
| Hardware for AI Architectures (Read more on arXiv or HuggingFace) |
Huazuo Gao, Damai Dai, Chong Ruan, Chengqi Deng, Chenggang Zhao |
DeepSeek-V3 achieves state-of-the-art performance through hardware-aware model co-design, optimizing for cost-efficient training and inference at scale. The paper investigates how hardware characteristics influence model architecture and identifies potential future hardware directions for AI. DeepSeek-V3 employs Multi-head Latent Attention (MLA), Mixture of Experts (MoE), FP8 mixed-precision training, and a Multi-Plane Network Topology to overcome hardware limitations. Using 2,048 NVIDIA H800 GPUs, DeepSeek-V3 achieves a KV cache size of 70.272 KB per token, significantly less than Qwen-2.5 72B’s 327.680 KB and LLaMA-3.1 405B’s 516.096 KB, demonstrating enhanced memory efficiency. The findings emphasize the importance of hardware and model co-design for scalable AI, providing a practical blueprint for innovating next-generation AI systems by precisely scaling low-precision computation units and emphasizing scale-up and scale-out convergence. |
| Marigold: Affordable Adaptation of Diffusion-Based Image Generators for |
|
|
| Image Analysis (Read more on arXiv or HuggingFace) |
Nando Metzger, Tianfu Wang, Kevin Qu, Bingxin Ke, konradschindler |
i) Marigold presents a fine-tuning protocol and associated conditional diffusion models adapted from pretrained latent diffusion models for image analysis tasks. ii) The research objective is to leverage the knowledge embedded in generative image models for dense image analysis, including monocular depth estimation, surface normals prediction, and intrinsic image decomposition. iii) The methodology involves fine-tuning a Stable Diffusion model with synthetic data and a resource-efficient protocol, reusing the LDM’s VAE to encode both input images and output modalities into the latent space. iv) Marigold demonstrates state-of-the-art zero-shot generalization, achieving high performance on depth estimation, surface normal prediction, and intrinsic image decomposition on datasets without observing a single image other than synthetic rooms. Marigold can produce depth estimates in under 100ms. v) This work offers AI practitioners a practical approach to repurpose readily available foundational generative models with minimal computational overhead to achieve high-performing image analysis capabilities in data-scarce settings, enabling rapid prototyping and deployment of robust vision systems. |
| SweRank: Software Issue Localization with Code Ranking (Read more on arXiv or HuggingFace) |
Xuan Phi Nguyen, Ye Liu, JaeHyeok Doo, Tarun Suresh, Revanth Gangi Reddy |
i) The paper introduces SWERANK, a retrieve-and-rerank framework for software issue localization, and a corresponding dataset, SWELOC. ii) The research aims to develop a more effective and efficient method for identifying relevant code locations for software issues described in natural language. iii) The methodology employs a bi-encoder for retrieval (SWERANKEMBED) and a listwise-trained LLM for reranking (SWERANKLLM), trained using a new dataset, SWELOC, curated from GitHub. iv) Experimental results on SWE-Bench-Lite show that SWERANK achieves state-of-the-art performance, with SWERANKEMBED-LARGE achieving 71.90% Acc@10 for function localization, surpassing existing agent-based approaches at a significantly lower cost. v) SWERANK provides AI practitioners with a cost-effective and accurate alternative to agent-based systems for software issue localization, enabling efficient integration into automated software engineering tools. |
| VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large |
|
|
| Video Language Models (Read more on arXiv or HuggingFace) |
Ali Etemad, pritamqu |
i) VCRBench, a new benchmark, is introduced to evaluate long-form causal reasoning in Large Video Language Models (LVLMs). ii) The research aims to assess LVLMs’ ability to identify, reason about, and sequence events in procedural videos to achieve specific goals. iii) VCRBench uses procedurally-generated videos of shuffled everyday activities to test LVLMs’ ability to correctly order causally-related steps. iv) Evaluations show that current LVLMs struggle, performing at or below random guess, with the best model falling nearly 40% short of human performance; however, using Recognition-Reasoning Decomposition (RRD) improves accuracy by up to 25.2%. v) RRD, a modular approach decomposing video reasoning into video recognition and causal reasoning, improves LVLM performance, indicating that explicitly separating these sub-tasks can enhance causal reasoning capabilities for AI practitioners developing video-based reasoning systems. |
| Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM? (Read more on arXiv or HuggingFace) |
Hilde Kuehne, Samuel Thomas, Edson Araujo, Saurabhchand Bhati, h9LtLSb |
Omni-R1, a streamlined GRPO fine-tuning of Qwen2.5-Omni, attains new State-of-the-Art performance on the MMAU benchmark for audio question answering. The research investigates whether audio fine-tuning is truly necessary for audio LLMs. GRPO fine-tuning was applied to Qwen2.5-Omni using both human-annotated and automatically generated audio question-answering datasets. Results show that Omni-R1 achieves the highest MMAU accuracies across sounds, music, speech, and overall average, with a peak Test-mini accuracy of 71.3% and Test-full accuracy of 71.2%. Text-only fine-tuning can significantly improve audio performance, suggesting improved text-based reasoning contributes substantially to the improved audio question answering ability in the model. |
| DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person |
|
|
| Recognition (Read more on arXiv or HuggingFace) |
Carolina Fernandes, Satish Mekewad, Pavan Kumar MP, Nzakiese Mbongo, Kailash A. Hambarde |
i) This paper introduces DetReIDX, a new dataset for person re-identification (ReID) designed to stress-test algorithms under real-world UAV surveillance conditions. ii) The main objective is to provide a challenging dataset that realistically incorporates data variability factors often lacking in existing ReID datasets, such as viewpoint changes, scale variations, clothing changes, and occlusion. iii) The methodology involved collecting a multi-session aerial-ground dataset of 509 identities across seven university campuses, with UAV altitudes ranging from 5.8 to 120 meters, and annotating 16 soft biometric attributes and multitask labels for detection, tracking, ReID, and action recognition. iv) Experiments using SOTA methods on DetReIDX revealed a performance degradation of up to 80% in detection accuracy and over 70% in Rank-1 ReID compared to their performance on existing datasets. v) DetReIDX provides AI practitioners a new benchmark dataset to develop more robust and generalizable person ReID systems capable of handling the challenges inherent in real-world UAV deployments, including addressing the limitations of models relying on superficial appearance cues. |
Papers for 2025-05-14
| Title |
Authors |
Summary |
| MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable |
|
|
| Speaker Encoder (Read more on arXiv or HuggingFace) |
Congchao Guo, Bowen Zhang, ymzhang0519, mqyang1s, JunjieYan |
MiniMax-Speech is an autoregressive Transformer-based TTS model with a learnable speaker encoder. The research aims to achieve high-quality speech synthesis with zero-shot voice cloning capabilities. The methodology involves jointly training a speaker encoder with the AR model and using a Flow-VAE to improve audio quality and speaker similarity. The model achieves state-of-the-art results on objective voice cloning metrics (Word Error Rate) on Seed-TTS-eval test set, specifically WER of 0.83 in zero-shot setting. AI practitioners can leverage this model architecture and training strategy for improved voice cloning performance in TTS systems, especially in scenarios with limited speaker data. |
| A Multi-Dimensional Constraint Framework for Evaluating and Improving |
|
|
| Instruction Following in Large Language Models (Read more on arXiv or HuggingFace) |
xjhuang, sean-xl-y, wuyilong, avonfwj, Junjie-Ye |
i) The paper introduces a multi-dimensional constraint framework to evaluate and improve instruction following in LLMs. ii) The main research objective is to address limitations of existing benchmarks that rely on templated prompts and lack real-world usage diversity. iii) The methodology involves developing an automated instruction generation pipeline incorporating constraint expansion, conflict detection, and instruction rewriting to produce code-verifiable test samples. iv) The primary result is a dataset of 1,200 diverse instruction-following cases that, when used in reinforcement learning (GRPO), resulted in performance gains, with average performance dropping from 77.67% at Level I to 32.96% at Level IV. v) The principal implication for AI practitioners is a method for generating targeted training data to enhance constraint recognition and adherence in LLMs, particularly through modifications in attention module parameters. |
| Measuring General Intelligence with Generated Games (Read more on arXiv or HuggingFace) |
William Chen, David Huang, nickatomlin, danjklein, vivekverma |
i) The paper introduces gg-bench, a synthetically generated benchmark for evaluating general reasoning in language models via novel games. ii) The research aims to assess language models’ ability to generalize and act in unseen environments through gameplay. iii) The methodology involves using a language model to generate game descriptions and Gym implementations, training RL agents via self-play, and evaluating language models’ win rates against these agents. iv) State-of-the-art LLMs such as GPT-4o and Claude 3.7 Sonnet achieve winrates of 7-9% on gg-bench using in-context learning, while reasoning models such as 01, 03-mini and DeepSeek-R1 achieve average winrates of 31-36%. v) The findings suggest that current language models struggle with strategic reasoning and adaptability in novel game environments, indicating a need for improved generalization capabilities for AI practitioners developing reasoning systems. |
| SkillFormer: Unified Multi-View Video Understanding for Proficiency |
|
|
| Estimation (Read more on arXiv or HuggingFace) |
ucaclio, EdBianchi |
SkillFormer is presented as a parameter-efficient architecture for multi-view proficiency estimation from videos. The research aims to develop a unified model for skill assessment using egocentric and exocentric videos. The methodology involves a TimeSformer backbone with a CrossViewFusion module using multi-head cross-attention and LoRA fine-tuning. The model achieves 47.5% accuracy on the EgoExo4D dataset in the Ego+Exos setting, using 4.5x fewer parameters than prior baselines. SkillFormer offers AI practitioners a computationally efficient architecture for multi-view skill assessment, potentially improving the performance of applications in sports, rehabilitation, and training. |
| NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged |
|
|
| Information Guidance (Read more on arXiv or HuggingFace) |
Yujian Zhang, Jiaqi Peng, Jiangmiao, fulifuli666, WadeCai |
NavDP introduces a navigation diffusion policy trained solely in simulation for zero-shot transfer to real-world robots. The research aims to develop an end-to-end framework for robot navigation that generalizes across different robot embodiments and unstructured environments using only simulation data. NavDP combines diffusion-based trajectory generation with a critic function for trajectory selection, conditioned on local observation tokens encoded by a policy transformer, and trained using privileged information. NavDP achieves a 30% success rate improvement by incorporating Gaussian Splatting based real-to-sim fine-tuning data, while maintaining generalization. AI practitioners can leverage NavDP’s approach to develop scalable and generalizable robot navigation policies trained in simulation, reducing the reliance on expensive real-world data collection. |
| ViMRHP: A Vietnamese Benchmark Dataset for Multimodal Review Helpfulness |
|
|
| Prediction via Human-AI Collaborative Annotation (Read more on arXiv or HuggingFace) |
Kiet Van Nguyen, Dat Minh Nguyen, sonlam1102, trucnguyen28 |
ViMRHP introduces a new Vietnamese dataset for multimodal review helpfulness prediction, leveraging human-AI collaborative annotation. The research aims to address the lack of linguistic diversity in existing review helpfulness datasets by creating a large-scale Vietnamese dataset. The methodology involved a two-step annotation process using LLMs followed by human verification and refinement. The experiments demonstrate that human-verified annotations achieve higher quality, showing a Cohen’s Kappa score of 31.34% for ground truth Helpfulness Score, indicating “Fair Agreement” compared to AI annotations. The principal implication for AI practitioners is the demonstrated need for human verification to ensure data quality in complex annotation tasks despite the advantages of LLMs in reducing costs and annotation time. |
Papers for 2025-05-13
| Title | Authors | Summary |
|——-|———|———|
| Seed1.5-VL Technical Report (Read more on arXiv or HuggingFace)| kuma-zhao, yuanlp, 0nejiawei, chb1997, anyuzx | Seed1.5-VL is a vision-language foundation model, comprising a 532M-parameter vision encoder and a 20B active parameter Mixture-of-Experts LLM, designed for general-purpose multimodal understanding and reasoning. The primary objective is to detail the development of Seed1.5-VL, focusing on advancing multimodal capabilities by addressing data scarcity through extensive data synthesis and achieving efficient training for its asymmetrical architecture. Key methodologies include pre-training on 3 trillion diverse tokens covering images, videos, text, and HCI data, with specialized data pipelines for OCR, grounding, and 3D understanding, followed by post-training using Supervised Fine-tuning, RLHF, RLVR, and iterative rejection sampling for LongCoT. Seed1.5-VL achieves state-of-the-art performance on 38 out of 60 public benchmarks, including 85.6% on MathVista (thinking mode) and 87.2% on the WebVoyager GUI agent task. For AI practitioners, this report offers a comprehensive guide to building efficient and high-performing VLMs, detailing data curation, training strategies for MoE-based LLMs with native-resolution vision encoders, and infrastructure optimizations, particularly valuable for developing versatile multimodal AI systems. |
| MiMo: Unlocking the Reasoning Potential of Language Model – From
Pretraining to Posttraining (Read more on arXiv or HuggingFace)| whatseeker, Prestonprom, HugoZHL, dwzhu, xiabingquan | This paper introduces MiMo-7B, a large language model optimized across pre-training and post-training stages specifically for reasoning tasks. The primary objective is to unlock and enhance the inherent reasoning potential of language models by systematically improving data processing, model architecture, and reinforcement learning techniques. Key methodologies include a three-stage pre-training data mixing strategy on 25 trillion tokens with a Multi-Token Prediction objective, and post-training using reinforcement learning on 130K verifiable math/code problems with a novel test-difficulty-driven code-reward scheme and strategic data resampling. The final RL-tuned model, MiMo-7B-RL, achieves superior performance, notably scoring 55.4 on AIME 2025, surpassing OpenAI o1-mini by 4.7 points, and significantly outperforming it on LiveCodeBench v5 (MiMo-7B-RL: 57.8, o1-mini: 53.8). The principal implication for AI practitioners is that targeted optimizations in both pre-training and post-training, particularly with high-quality verifiable data and refined RL reward mechanisms, can enable smaller models to achieve state-of-the-art reasoning capabilities comparable to or exceeding much larger or proprietary models. |
| Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured
3D Assets (Read more on arXiv or HuggingFace)| PanJianxiong, flybirdtian, shian7, wchengad, xuanyangz | Step1X-3D introduces an open framework for high-fidelity, controllable generation of textured 3D assets through rigorous data curation and a novel two-stage 3D-native architecture. The primary objective is to overcome fundamental challenges in 3D generation, such as data scarcity and algorithmic limitations, by providing an open and reproducible solution. Its methodology combines a pipeline curating over 5 million initial assets into a 2 million high-quality dataset, a hybrid VAE-DiT for Truncated Signed Distance Function (TSDF)-based geometry generation, and a diffusion model for geometrically-conditioned, view-consistent texture synthesis, notably supporting direct transfer of 2D control techniques like LoRA. Step1X-3D achieves state-of-the-art results among open-source methods, notably attaining the highest texture CLIP-Score of 0.853 in comparative benchmarks and competitive geometry scores (e.g., Uni3D-I of 0.361). For AI practitioners, this framework offers a robust, open-source baseline with models, training code, and curated data, facilitating advancements in controllable 3D asset generation and simplifying the integration of established 2D control mechanisms into 3D workflows. |
| Learning from Peers in Reasoning Models (Read more on arXiv or HuggingFace)| Benyou, tangzhy, Jiaxi0775, wydu, Zeno-Luo | This paper introduces Learning from Peers (LeaP), a method for Large Reasoning Models (LRMs) to overcome the “Prefix Dominance Trap”—where poor initial reasoning hinders recovery—by sharing intermediate insights among parallel inference paths. The primary objective is to enhance the limited self-correction capabilities of LRMs by enabling them to learn from diverse reasoning trajectories. LeaP’s methodology involves periodic communication (every T tokens) where each reasoning path summarizes its current state, sharing these summaries with peers via a routing mechanism (e.g., Dispersed, Hybrid); a fine-tuned LeaP-T series is proposed for smaller models. Key results show significant improvements: QwQ-32B with LeaP achieves nearly 5 absolute points higher Pass@1 on average than its baseline, and on a 14B model, LeaP reduced the “Prefix Dominance Trap” performance gap from 19.88 to 7.81 points on an AIME 2024 subset. For AI practitioners, this research offers a strategy to build more robust reasoning systems by facilitating collaborative error correction and diverse exploration during inference, improving performance even when initial reasoning is flawed. |
| Unified Continuous Generative Models (Read more on arXiv or HuggingFace)| Yi Jiang, tlin-taolin, sp12138sp | This paper introduces a unified framework, UCGM, for continuous generative models. The research aims to provide a unified training and sampling methodology applicable to both multi-step and few-step generative models. The proposed methodology uses a unified training objective parameterized by a consistency ratio and a novel self-boosting mechanism for improved performance. Experiments on ImageNet 256x256 using a 675M diffusion transformer model show UCGM-T achieves 1.30 FID in 20 sampling steps and 1.42 FID in 2 sampling steps. AI practitioners can leverage UCGM for improved training efficiency and sample quality across different continuous generative modeling paradigms, with reduced reliance on classifier-free guidance. |
| REFINE-AF: A Task-Agnostic Framework to Align Language Models via
Self-Generated Instructions using Reinforcement Learning from Automated
Feedback (Read more on arXiv or HuggingFace)| Pawan Goyal, Somak Aditya, Aniruddha Roy, abhi1nandy2, Pretam | REFINE-AF is a task-agnostic framework that aligns smaller open-source language models using self-generated instructions and Reinforcement Learning from Automated Feedback (RLAF). The primary objective is to investigate the effectiveness of smaller LLMs (LLaMA 2-7B/13B, Mistral 7B) for task-agnostic instruction generation and assess the impact of RLAF in this process. The methodology involves a three-stage pipeline: iterative instruction generation from a seed set, RLAF using a reward model (based on oasst-rm-pythia-1.4b and UniEval metrics) with PPO to refine input-output pair generation, followed by supervised fine-tuning on the resultant dataset. REFINE-AF demonstrated superior performance over the SELF INSTRUCT baseline on the SUPER-NI benchmark, with the LLaMA 2 13B model achieving a 6.6133 average ROUGE-L score and outperforming in 66.39% of tasks using 15,000 generated instructions. For AI practitioners, this research offers a cost-effective method to generate diverse, high-quality instruction datasets for fine-tuning smaller, open-source LLMs, thereby enhancing their instruction-following capabilities with reduced human effort. |
| DanceGRPO: Unleashing GRPO on Visual Generation (Read more on arXiv or HuggingFace)| appleluo, ChenMnZ, ltzhu, wujie10, xzyhku | This paper introduces DanceGRPO, a unified framework adapting Group Relative Policy Optimization (GRPO) to enhance visual generation across diverse generative paradigms, tasks, and models. The research aims to overcome limitations in existing RL-based visual generation, such as incompatibility with ODE-based sampling and training instability, by developing a versatile RL framework for aligning models with human preferences. DanceGRPO reformulates sampling for diffusion and rectified flow models as Markov Decision Processes, unifies them using Stochastic Differential Equations, and applies a GRPO objective with strategies for noise initialization, timestep selection, and multi-reward aggregation. DanceGRPO achieves substantial improvements, outperforming baselines by up to 181% on benchmarks like VideoAlign (e.g., a 181% relative improvement in motion quality for HunyuanVideo) and demonstrating robust performance across various models and tasks. DanceGRPO provides AI practitioners with a scalable and effective Reinforcement Learning from Human Feedback (RLHF) solution for aligning complex visual generative models across image and video domains, enabling more stable and higher-quality visual synthesis. |
| AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong
Pretraining Data Selection (Read more on arXiv or HuggingFace)| Steven Wu, Kai Hua, shenke18, zhangysk | This paper introduces AttentionInfluence, a training-free method that uses attention head masking in a small pretrained language model to select high-quality, reasoning-intensive pretraining data for improving larger LLMs. The main objective is to develop an efficient, scalable, and unsupervised method for identifying diverse, high-quality pretraining data to enhance complex reasoning in LLMs, without relying on labeled data or supervised classifiers. AttentionInfluence identifies “retrieval heads” in a small pretrained language model (e.g., 1.3B parameters) using a synthetic task, then computes a data quality score based on the relative loss difference when these heads are masked versus unmasked; this score is used to rank and select data from a large corpus. Using data selected by a 1.3B model, a 7B pretrained model demonstrated substantial improvements, such as a +3.5pp increase on the HumanEval benchmark and +2.7pp on GSM8K, compared to a baseline trained on the unselected corpus. This method offers AI practitioners a scalable, computationally cheaper, and training-free alternative for curating reasoning-centric pretraining datasets, enabling smaller models to effectively guide data selection for training more capable larger models, thus demonstrating weak-to-strong generalization. |
| WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional
Websites from Scratch (Read more on arXiv or HuggingFace)| shiwk20, hht1113, Houxing, Yqy6, luzimu | This paper introduces WebGen-Bench, a benchmark to evaluate LLM-based agents on generating multi-file interactive websites from scratch. The primary objective is to systematically assess and improve LLM agents’ ability to create functional and aesthetically pleasing websites based on natural language instructions. The methodology involves curating diverse website generation instructions (human + GPT-4o), creating 647 test cases (GPT-4o + manual refinement), and employing an automated pipeline with a web-navigation agent (WebVoyager) for functional testing and GPT-4o for aesthetic scoring. The best general-purpose agent combination (Bolt.diy with DeepSeek-R1) achieved only 27.8% accuracy, while fine-tuning Qwen2.5-Coder-32B-Instruct on a subset of their WebGen-Instruct dataset (creating WebGen-LM-32B) reached 38.2% accuracy. For AI practitioners, this work highlights the current limitations of LLMs in complex, from-scratch code generation tasks like website creation but demonstrates that targeted fine-tuning with specialized instructional datasets can significantly enhance these capabilities. |
| Learning Dynamics in Continual Pre-Training for Large Language Models (Read more on arXiv or HuggingFace)| Daniel Dajun Zeng, Lu Wang, Xingjin Wang, linjinglian, Howe77 | This paper introduces a CPT scaling law that models learning dynamics in continually pre-trained large language models by decoupling the effects of distribution shift and learning rate annealing to predict validation loss. The primary objective is to quantitatively describe and predict how general (Dpt) and specific-domain (Dcpt) validation losses evolve throughout the CPT process under various training configurations. The methodology involves deriving the scaling law by combining a base pre-training loss component (L_base, affected by summed learning rate S1 and annealing area S2) with a distribution shift component (ΔL, modeled as a power-law function of CPT summed LR S1_cpt), fitting parameters using Huber loss minimization. Key results demonstrate that this CPT scaling law (L(t) = Lo + A(S1_pt + S1_cpt)^-α - C1S2_pt - C2S2_cpt + B(1 - (1 + E*S1_cpt)^-β)) accurately fits validation loss curves across different learning rate schedules, datasets, model sizes, and replay ratios, with the distribution shift term fitting achieving R² values like 0.994 (Dpt) and 0.985 (Dcpt) initially. For AI practitioners, this scaling law provides a predictive tool to optimize CPT hyperparameters—such as loss potential, peak LR, and replay ratio—to effectively balance general and domain-specific performance (Finding 5) and can guide the adaptation of open-source models with unknown pre-training details. |
| Skywork-VL Reward: An Effective Reward Model for Multimodal
Understanding and Reasoning (Read more on arXiv or HuggingFace)| Yi Peng, Wei Shen, Jiangbo Pei, OrlandoHugBot, shawn0wang | The paper introduces Skywork-VL Reward, a novel multimodal reward model designed for robust evaluation of multimodal understanding and reasoning in Vision-Language Models (VLMs). Its primary objective is to provide reliable reward signals for aligning diverse VLMs, including advanced reasoners, with human preferences across a wide range of tasks. The methodology involves curating a large-scale preference dataset of approximately 190,000 pairs and fine-tuning a Qwen2.5-VL-7B-Instruct base model with an added reward head using a multi-stage, pairwise ranking loss approach. Skywork-VL Reward achieves state-of-the-art 73.1% overall accuracy on the VL-RewardBench, and its preference data, when used for Mixed Preference Optimization (MPO), improved a VLM’s MathVista score from 69.2% to 73.5%. For AI practitioners, Skywork-VL Reward offers an effective open-source tool for VLM alignment, and its utility in generating high-quality preference data can significantly boost the reasoning capabilities of downstream VLMs. |
| Reinforced Internal-External Knowledge Synergistic Reasoning for
Efficient Adaptive Search Agent (Read more on arXiv or HuggingFace)| Kang Liu, Jun Zhao, Yiming Ju, Xiaowei Yuan, hzy | This paper introduces IKEA, a reinforcement learning-based agent that synergistically reasons with internal and external knowledge for efficient adaptive search by discerning its own knowledge boundaries. The primary objective is to develop an efficient adaptive search agent that can determine when to use its internal parametric knowledge versus external retrieved knowledge, thereby reducing redundant retrievals and improving reasoning accuracy. IKEA employs reinforcement learning with a novel knowledge-boundary aware reward function and a specially constructed training dataset (balanced between internally-known and externally-required questions) to learn optimal retrieval timing. Evaluations demonstrate that IKEA significantly outperforms baselines; for instance, on Qwen2.5-7B, IKEA achieved an average Exact Match (EM) of 50.05% while reducing retrieval frequency by 50.81% (to 0.91 retrievals) compared to the Search-R1 baseline (45.00% EM, 1.85 retrievals). AI practitioners can leverage IKEA’s approach of knowledge-boundary aware rewards and dataset construction to train more efficient and accurate retrieval-augmented LLM agents that better utilize internal knowledge, leading to reduced latency and computational cost in knowledge-intensive tasks. |
| H^{3}DP: Triply-Hierarchical Diffusion Policy for Visuomotor
Learning (Read more on arXiv or HuggingFace)| Pu Hua, Zhecheng Yuan, Yufeng Tian, binaryXwizard, Lyy0725 | H³DP introduces a triply-hierarchical diffusion policy for visuomotor learning that integrates depth-aware input, multi-scale visual features, and hierarchical action generation to improve robotic manipulation. The primary objective is to enhance visuomotor policy learning by explicitly incorporating hierarchical structures across visual perception and action generation, thereby strengthening their coupling for improved performance in complex robotic manipulation tasks. The H³DP framework employs three hierarchical levels: 1) depth-aware input layering of RGB-D observations based on depth information, 2) multi-scale visual representations encoding semantic features at varying granularities, and 3) a hierarchically conditioned diffusion process where coarse-to-fine action generation is guided by corresponding visual features at different scales. H³DP achieved a +27.5% average relative improvement over baselines across 44 simulation tasks (resulting in a 75.6% average success rate for H³DP) and demonstrated superior performance in 4 challenging bimanual real-world manipulation tasks, with specific results like a 66.2% average success rate on instance generalization tasks. AI practitioners can leverage the triply-hierarchical design of H³DP to develop more robust and effective visuomotor learning agents, as explicitly structuring the perception-action pipeline with depth awareness, multi-scale feature encoding, and hierarchical action conditioning significantly enhances policy performance and generalization in complex, cluttered environments. |
| Continuous Visual Autoregressive Generation via Score Maximization (Read more on arXiv or HuggingFace)| Jie Zhou, Fandong Meng, cccczshao | This paper introduces a Continuous Visual Autoregressive (VAR) framework enabling direct visual generation without vector quantization by optimizing strictly proper scoring rules. The main objective is to overcome information loss from quantization in traditional VAR models by developing a method for direct autoregressive generation in continuous visual data spaces. The key methodology involves using strictly proper scoring rules, primarily the energy score, as training objectives within a Transformer architecture (termed Energy-based AutoRegression or EAR), where an MLP generator implicitly models the predictive distribution. The EAR-H model achieves a Fréchet Inception Distance (FID) of 1.97 on ImageNet 256x256 conditional generation, demonstrating competitive performance with significantly higher inference efficiency compared to per-token diffusion methods. The principal implication for AI practitioners is the provision of a likelihood-free training paradigm for autoregressive models on continuous data, offering an alternative to quantization-based approaches and potentially leading to improved generation quality and efficiency for modalities beyond discrete tokens. |
| Overflow Prevention Enhances Long-Context Recurrent LLMs (Read more on arXiv or HuggingFace)| rgiryes, leokarlin, OmegaLittleBob, ItamarZ, assafbk | This paper introduces OPRM, a training-free chunk-based inference strategy that significantly enhances the long-context processing capabilities of recurrent LLMs by preventing memory overflows. The research investigates the limitations of fixed-size recurrent memory in large long-context models and aims to develop a method to mitigate these limitations, thereby improving their performance on long-context tasks. The proposed OPRM method involves segmenting the input context into fixed-size chunks, processing each chunk (wrapped with original prefix and suffix) speculatively in parallel, and then selectively decoding from the chunk determined to be most relevant, typically using a minimum entropy criterion combined with an IDK filter. Primary results demonstrate substantial improvements; for instance, on LongBench, OPRM improved the overall performance of RWKV6-Finch-7B by 51%, and Falcon3-Mamba-Inst-7B with OPRM achieved a state-of-the-art 30.8 score on the LongBench v2 benchmark for its size class. The principal implication for AI practitioners is that OPRM can be applied as an inference-time technique to existing recurrent LLMs to extend their effective context length and boost performance on tasks with very long sequences without retraining, making these models more competitive for long-context applications. |
| UMoE: Unifying Attention and FFN with Shared Experts (Read more on arXiv or HuggingFace)| Jing Li, Chaozheng Wang, ysngkil | i) 1-line summary: This paper introduces UMoE, a unified Mixture-of-Experts architecture that reformulates attention to share FFN-like experts between attention and feed-forward network layers, improving performance and parameter efficiency. ii) Main research question or objective: The primary objective was to determine if attention mechanisms can be reformulated to be compatible with FFN expert designs for unified MoE application and parameter sharing, without sacrificing expressive power. iii) Key methodology used: UMoE employs a “pre-mixing” attention reformulation where input token embeddings are first aggregated via weighted summation before being processed by shared two-layer FFN experts, with expert-dependent query projections utilizing low-rank matrices. iv) Primary results (include at least one specific quantitative finding): UMoE achieved superior performance, with its 540M parameter model attaining a perplexity of 20.44 on FineWeb-Edu, surpassing a comparable 535M FFN-MoE (PPL 21.19), and also demonstrated stronger average zero-shot accuracy (e.g., 40.06% for the base UMoE vs. 39.55% for FFN-MoE). v) Principal implication for AI practitioners: AI practitioners can leverage UMoE to develop more parameter-efficient and higher-performing large language models by sharing expert modules across attention and FFN layers, informed by the paper’s finding that FFN layers can be conceptualized as specialized attention layers with an identity matrix for token mixing. |
| Document Attribution: Examining Citation Relationships using Large
Language Models (Read more on arXiv or HuggingFace)| Nedim Lipka, Vipula Rawte, Franck-Dernoncourt, ryanrossi | This research investigates document attribution in LLMs by proposing a zero-shot textual entailment approach and an attention-based classification technique to verify citation reliability. The primary objective is to enhance the trustworthiness and interpretability of LLM-generated content by developing methods to accurately trace outputs to source documents and assess the reliability of these citations. The study employs a zero-shot textual entailment framework, prompting LLMs (specifically flan-ul2 and gpt4-o) to determine if a reference entails a claim, and explores an attention-based binary classification method using attention weights from a smaller LLM (flan-t5-small). The zero-shot textual entailment method using flan-ul2 achieved an F1 score of 73.8 on the in-distribution (ID) average and 83.43 on the out-of-distribution (OOD) average of AttributionBench, outperforming prior baselines (e.g., ID LFQA F1 of 85.38 vs. 80.1 baseline), while the preliminary attention-based method with flan-t5-small showed F1 score improvements over its zero-shot baseline on the LFQA subset for most layers. AI practitioners can leverage the proposed zero-shot textual entailment prompting strategy with models like flan-ul2 as a computationally efficient method to improve citation verification and attribution in document-based LLM applications, enhancing system reliability without requiring task-specific fine-tuning. |
| Physics-Assisted and Topology-Informed Deep Learning for Weather
Prediction (Read more on arXiv or HuggingFace)| Yerong Feng, Qing Ling, Yumenomae | PASSAT is a novel deep learning model for weather prediction that integrates physics, via the advection and Navier-Stokes equations, with a topology-informed spherical graph neural network to model weather evolution on a spherical manifold. The main objective is to develop a deep learning model that overcomes limitations of existing approaches by explicitly incorporating underlying physical laws and the Earth’s spherical topology. PASSAT’s methodology involves numerically solving the advection and Navier-Stokes equations on a spherical manifold for the advection process, while a spherical graph neural network estimates the Earth-atmosphere interaction and generates the initial velocity fields for the advection equation. On the 5.625°-resolution ERA5 dataset, PASSAT outperformed state-of-the-art deep learning models and the operational numerical weather prediction model IFS T42; for instance, for geopotential at 500hPa (z500) with a 72-hour lead time, PASSAT achieved an RMSE of 420, compared to 438 for GraphCast and 489 for IFS T42. The principal implication for AI practitioners is that synergistically combining deep learning with domain-specific physical differential equations and appropriate geometric representations can significantly improve model accuracy and physical consistency in complex spatio-temporal forecasting tasks. |
| Multi-Objective-Guided Discrete Flow Matching for Controllable
Biological Sequence Design (Read more on arXiv or HuggingFace)| Tong Chen, pranamanam, sophtang, yinuozhang | This paper introduces Multi-Objective-Guided Discrete Flow Matching (MOG-DFM), a framework for steering discrete flow matching models to design biological sequences optimizing multiple conflicting objectives. The research aims to develop a controllable method for generating Pareto-efficient peptide and DNA sequences by guiding pretrained discrete flow matching models across multiple functional and biophysical criteria. MOG-DFM iteratively guides a base discrete flow matching generator by computing a hybrid rank-directional score for candidate token transitions and applying an adaptive hypercone filter to ensure consistent multi-objective progression towards a specified trade-off. MOG-DFM significantly outperformed traditional multi-objective algorithms in peptide design, boosting non-fouling and solubility by approximately 30-50% and extending half-life by a factor of 3 to 4 compared to the next-best method. In DNA enhancer design, MOG-DFM successfully guided generation towards specific enhancer classes (e.g., achieving class 1 probability ~0.7) and desired DNA shapes (e.g., HelT ~36.0). AI engineers can leverage MOG-DFM as a versatile tool for de novo multi-property design of discrete biological sequences, enabling fine-grained control over trade-offs between complex, user-defined objectives. |
Papers for 2025-05-12
| Title |
Authors |
Summary |
| Bielik v3 Small: Technical Report (Read more on arXiv or HuggingFace) |
Adrian Gwoździej, Łukasz Flis, djstrong, Remek, chrisociepa |
This paper introduces Bielik v3, a series of parameter-efficient (1.5B, 4.5B) generative text models optimized for the Polish language. The main objective was to demonstrate that smaller, well-optimized models can achieve performance comparable to much larger counterparts for a less-resourced language using substantially fewer computational resources. Key methodologies included depth up-scaling Qwen2.5 base models, implementing a custom Polish tokenizer (APT4) for improved token efficiency, utilizing Adaptive Learning Rate, and training on a curated 292 billion token Polish-centric corpus. The primary result shows the 4.5B parameter Bielik v3 Instruct model achieved a competitive score of 56.13 on the Open PL LLM Leaderboard (5-shot), outperforming several models 2-3 times its size. For AI practitioners, this work implies that targeted optimization, including custom tokenization and architecture scaling techniques, allows for the development of high-performing, resource-efficient models for specific languages, potentially reducing computational costs for deployment. |
| Bielik 11B v2 Technical Report (Read more on arXiv or HuggingFace) |
Adrian Gwoździej, Łukasz Flis, Remek, djstrong, chrisociepa |
This report details Bielik 11B v2, an 11-billion parameter language model optimized for Polish, derived from Mistral 7B v0.2 using depth up-scaling and novel training methods. The primary objective was to create a state-of-the-art, computationally efficient model for Polish text processing with strong cross-lingual transferability. Methodology involved continued pre-training on a 198 billion token Polish-centric corpus, followed by Supervised Fine-Tuning and DPO-Positive alignment using custom techniques like Weighted Instruction Cross-Entropy Loss and Adaptive Learning Rate. Bielik-11B-v2 models achieve leading performance on Polish benchmarks, with v2.3-Instruct scoring 65.71 on the Open PL LLM Leaderboard, significantly outperforming its 7B predecessor and many larger models, while demonstrating robustness to quantization down to IQ2_XXS (61.34 score). For AI practitioners, Bielik 11B v2 offers a parameter-efficient (11B) model for high-quality Polish language tasks, deployable on constrained hardware due to effective quantization, serving as a benchmark for less-resourced language model development. |
| Healthy LLMs? Benchmarking LLM Knowledge of UK Government Public Health |
|
|
| Information (Read more on arXiv or HuggingFace) |
Toby Nonnenmacher, Timothy Laurence, Felix Feldman, Fan Grayson, Joshua-Harris |
This paper introduces PubHealthBench to benchmark Large Language Model (LLM) knowledge of UK Government public health information. The primary objective was to assess the accuracy and potential risks of using LLMs for retrieving public health guidance. An automated pipeline generated over 8000 Multiple Choice Question Answering (MCQA) questions and a free-form response set from government documents, which were used to evaluate 24 LLMs. Key results show the best private LLMs achieve >90% accuracy in MCQA, significantly outperforming a human baseline using search engines, but no model scored above 75% in the more challenging free-form setting. For AI practitioners, this indicates that while SOTA LLMs possess strong factual recall for public health information in structured formats, deploying them for free-form response generation requires caution and potentially additional safeguards due to lower observed accuracy and hallucination risks. |
| UniVLA: Learning to Act Anywhere with Task-centric Latent Actions (Read more on arXiv or HuggingFace) |
Shenyuan Gao, Jisong Cai, Yanting Yang, Qingwen Bu, sundrops |
UniVLA introduces a generalist vision-language-action framework enabling policy learning across diverse embodiments by deriving task-centric latent actions unsupervisedly from videos. The primary objective is to learn a unified, transferable action representation from heterogeneous video data (robot/human, varied perspectives) without requiring ground-truth action labels, addressing scalability limitations of current VLA models. Key methodology involves a two-stage latent action learning process using inverse dynamics on DINOv2 features conditioned by language to decouple task-centric actions, followed by pretraining an auto-regressive VLM on these latent actions and deploying with lightweight action decoders. UniVLA achieves state-of-the-art results, including a 95.2% average success rate on the LIBERO benchmark, significantly outperforming prior methods like OpenVLA (+18.5%) with substantially less pretraining compute (<1/20). For AI practitioners, this work presents a scalable and efficient method to train generalist robot policies by leveraging readily available, unlabeled video data, reducing dependence on annotated datasets and extensive computation. |
| G-FOCUS: Towards a Robust Method for Assessing UI Design Persuasiveness (Read more on arXiv or HuggingFace) |
Yejin Choi, Sumin Shim, Min Soo Kim, Jang Han Yoon, jeochris |
This paper introduces WISERUI-BENCH for pairwise UI persuasiveness evaluation and G-FOCUS, a VLM reasoning strategy to improve assessment accuracy and reduce bias. The research aims to quantitatively evaluate and enhance VLM capabilities for assessing UI design persuasiveness, addressing the cost limitations of A/B testing and inherent VLM biases. Key methodology involves the WISERUI-BENCH dataset (300 UI pairs with A/B results and rationales) and the G-FOCUS inference strategy (goal extraction, difference localization, contrastive reasoning, evaluation). Results demonstrate G-FOCUS surpasses baselines in reliability; using GPT-4o, it achieved 43.33% Consistent Accuracy, a +12.66% improvement over the prior best baseline, indicating reduced bias. For AI practitioners, G-FOCUS offers a more robust automated method for comparative UI evaluation that can complement A/B testing and provide scalable preference data for aligning models towards human-preferred UI generation. |
| Sailing AI by the Stars: A Survey of Learning from Rewards in |
|
|
| Post-Training and Test-Time Scaling of Large Language Models (Read more on arXiv or HuggingFace) |
Xiaobao Wu |
This survey provides a comprehensive overview of the “Learning from Rewards” paradigm used in post-training and test-time scaling of Large Language Models (LLMs). The main objective is to categorize and analyze the diverse strategies under this paradigm, detailing how reward signals guide LLM behavior across training, inference, and post-inference stages. Methodologically, the paper presents a unified conceptual framework and taxonomy, organizing techniques based on reward sources, reward model design dimensions (architecture, format, pattern, granularity), learning timing, and learning strategies (training-based/free). Key surveyed results include the successful application of techniques like RLHF and DPO for preference alignment, and the emergence of deep reasoning capabilities through methods like GRPO with rule-based rewards, as demonstrated by models like DeepSeek-R1 acquiring long Chain-of-Thoughts abilities. For AI practitioners, this survey offers a structured understanding for selecting and implementing appropriate reward-based methods to align, enhance, and scale LLMs beyond pre-training for specific tasks and desired behaviors. |
| A Preliminary Study for GPT-4o on Image Restoration (Read more on arXiv or HuggingFace) |
Liyuan Pan, Ruikun Zhang, Yan Yang, Hao Yang |
This paper presents the first systematic evaluation of OpenAI’s GPT-4o for diverse image restoration tasks. The primary objective is to investigate GPT-4o’s capabilities and limitations in restoring degraded images across various domains like dehazing, deraining, and low-light enhancement. Methodology involves quantitative analysis using PSNR and CLIP-IQA metrics across eight tasks, qualitative assessment, failure case analysis, and proposing a baseline post-processing network using GPT-4o outputs as visual priors. Results show GPT-4o generates visually appealing outputs (high CLIP-IQA) but suffers poor pixel-level fidelity (e.g., PSNR often lower than degraded input, 12.89 dB vs. 21.58 dB in one example); however, using its outputs as priors significantly boosted a baseline network’s performance (e.g., O-Haze PSNR improved from 20.86 to 22.08). For AI practitioners, the key implication is that GPT-4o outputs, despite structural inaccuracies, can serve as effective visual priors when integrated into pipelines with lightweight networks to enhance existing image restoration methods’ perceptual quality and structural fidelity. |
Papers for 2025-05-09
| Title |
Authors |
Summary |
| On Path to Multimodal Generalist: General-Level and General-Bench (Read more on arXiv or HuggingFace) |
Gh0stAR, ChocoWu, LXT, JunchengLi, scofield7419 |
This paper introduces General-Level, a 5-level taxonomy, and General-Bench, a massive benchmark, to evaluate multimodal large language models (MLLMs) based on their synergistic capabilities across comprehension, generation, and modalities. The research aims to establish a sophisticated evaluation framework that assesses MLLMs not just on task performance but on their “synergy effect”—the ability for knowledge in one modality/task to enhance others—as a truer measure of generalist intelligence towards Artificial General Intelligence (AGI). The General-Level framework classifies MLLMs into five levels, with progression requiring increasing synergy, defined by outperforming State-of-the-Art (SoTA) specialists on tasks within the General-Bench, which comprises over 700 tasks and 325,800 instances across image, video, audio, 3D, and language. Evaluation of over 100 MLLMs on General-Bench revealed that most lack the cross-task or cross-modal synergy for higher General-Level classifications; for instance, only 3 models (Mini-Gemini, Emu2-37B, Vitron-V1) achieved Level 4, with Mini-Gemini scoring 1.56, and no model reached Level 5 (total synergy). AI practitioners can use General-Level and General-Bench to rigorously assess and compare MLLM synergistic abilities, providing a roadmap for developing more robust generalists that can better integrate and transfer knowledge across diverse multimodal inputs and tasks, a critical step towards AGI. |
| Perception, Reason, Think, and Plan: A Survey on Large Multimodal |
|
|
| Reasoning Models (Read more on arXiv or HuggingFace) |
imryanxu, xyidealist, TerenceL-TL, foggyforest, YunxinLi |
This survey details the evolution of Large Multimodal Reasoning Models (LMRMs) across four developmental stages, identifies current limitations, and proposes a future direction towards Native LMRMs (N-LMRMs) capable of integrated omni-modal perception, agentic reasoning, and generative capabilities. The paper aims to provide a comprehensive, structured review of multimodal reasoning research, analyze the entire roadmap from early modular designs to state-of-the-art LMRMs, and project future developments for next-generation systems. The research employs a survey methodology, synthesizing over 540 publications to delineate a four-stage developmental roadmap (Perception-Driven Modular, Language-Centric Short, Language-Centric Long, and prospective Native LMRMs), supported by analysis of existing models, benchmarks, and experimental insights from models like OpenAI’s O3 and O4-mini. Current LMRMs, while advancing, show significant limitations in omni-modal generalization and agentic behavior; for instance, on the OmniMMI benchmark, even commercial models like Gemini-1.5-Pro and GPT-4o achieve less than 20% average accuracy, and performance drops further on tasks requiring unified understanding across multiple modalities. AI practitioners should focus on developing N-LMRMs with unified architectures for heterogeneous modalities, interleaved multimodal reasoning, and continuous learning from interaction, as current language-centric LMRMs are insufficient for complex, real-world omni-modal and agentic tasks. |
| Flow-GRPO: Training Flow Matching Models via Online RL (Read more on arXiv or HuggingFace) |
dizhang, Xintao, CheeryLJH, Lp256, liuhuohuo |
Flow-GRPO introduces online reinforcement learning (RL) to flow matching models by converting deterministic Ordinary Differential Equations (ODEs) to equivalent Stochastic Differential Equations (SDEs) for stochastic exploration and employing a Denoising Reduction strategy for efficient training. The main objective is to effectively integrate online RL, specifically Group Relative Policy Optimization (GRPO), with flow matching generative models to enhance their capabilities in complex text-to-image (T2I) tasks, such as compositional understanding and text rendering, while maintaining image quality and sampling efficiency. The key methodology involves two strategies: (1) an ODE-to-SDE conversion that transforms the model’s deterministic generative process into a stochastic one, matching the original model’s marginal distribution at all timesteps to enable statistical sampling for RL exploration, and (2) a Denoising Reduction strategy that reduces the number of denoising steps during RL training (e.g., 10 steps) compared to inference (e.g., 40 steps) to improve sampling efficiency. Flow-GRPO demonstrated significant improvements across multiple T2I tasks; notably, for complex compositions, the RL-tuned SD3.5-Medium model increased GenEval accuracy from 63% to 95%, while visual text rendering accuracy improved from 59% to 92%, with little to no reward hacking observed. The principal implication for AI practitioners is that online RL can be effectively applied to state-of-the-art flow matching models to enhance specific generation capabilities and align with human preferences by introducing stochasticity via SDE conversion and accelerating training through denoising reduction, with Kullback-Leibler (KL) constraints proving vital for preventing performance degradation in general image quality. |
| Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Peisong Wang, Qingxuan Jiang, Bang Zhang, zptu, vvibt |
This paper introduces Sentient Agent as a Judge (SAGE), an automated framework using an LLM-powered agent to evaluate higher-order social cognition in large language models by simulating human-like emotional responses and inner thoughts. The primary objective is to develop a robust method for assessing LLMs’ abilities to understand and respond to human emotions and intentions in multi-turn dialogues, moving beyond mere textual competence. SAGE employs a “Sentient Agent,” instantiated with a persona, background, goals, and hidden intentions, which interacts with the LLM being tested; this agent uses multi-hop reasoning to simulate emotional changes (quantified as an emotion score trajectory) and generate contextually appropriate responses. The Sentient emotion score from SAGE shows strong correlation with human-centric psychological metrics like the Barrett-Lennard Relationship Inventory (BLRI) (Pearson r = 0.818) and utterance-level empathy (Pearson r = 0.788), and its leaderboard reveals GPT-40-Latest achieved the top Sentient score of 79.9. For AI practitioners, SAGE offers a principled and scalable tool to benchmark progress towards genuinely empathetic LLMs, with findings like GPT-40-Latest achieving a top Sentient score (79.9) with high token efficiency (3.3K tokens), indicating an advance in socially adept AI development. |
| Scalable Chain of Thoughts via Elastic Reasoning (Read more on arXiv or HuggingFace) |
cxiong, JunnanLi, doyensahoo, hendrydong, yuhuixu |
The paper introduces Elastic Reasoning, a framework for large reasoning models to produce scalable chain-of-thought outputs under strict inference budgets by separating reasoning into ‘thinking’ and ‘solution’ phases with independent budgets and training via budget-constrained rollouts. Its main objective is to enable robust and efficient reasoning from these models when faced with limited computational resources at inference time. The core methodology combines separate budgeting for thinking and solution generation at inference with a GRPO-based training strategy that simulates budget exhaustion, teaching the model to adapt to incomplete reasoning. Key results demonstrate that an E1-Math-1.5B model, trained with significantly fewer steps (200 vs. 700-820 for baselines), achieves 35.0% accuracy on AIME2024 with a 2K token budget, outperforming baselines, and reduces token usage by 32.1% in unconstrained settings compared to the original model while maintaining comparable performance. For AI practitioners, Elastic Reasoning offers a practical approach to deploy advanced reasoning models in resource-constrained environments by providing fine-grained control over inference costs without substantial performance loss or extensive retraining overhead. |
| FG-CLIP: Fine-Grained Visual and Textual Alignment (Read more on arXiv or HuggingFace) |
DaweiLiang, jinchenglijc, fanjing, binwang, xiechunyu |
FG-CLIP is a model that significantly enhances fine-grained visual and textual understanding in multimodal systems. The primary objective was to address the limitations of existing CLIP-like models in comprehending detailed visual attributes and relationships, which often struggle due to coarse-grained input and a lack of region-specific alignment. FG-CLIP’s methodology involves three key innovations: generating 1.6 billion long caption-image pairs for global semantic detail, constructing a dataset of 12 million images with 40 million region-specific bounding boxes aligned with detailed captions, and incorporating 10 million hard fine-grained negative samples, all trained using a two-stage process with an extended text encoder capacity. Extensive experiments demonstrate FG-CLIP’s superiority; for instance, FG-CLIP (ViT-L/14) achieved 48.4% accuracy on the fine-grained understanding benchmark FG-OVD (hard subset), substantially outperforming the original CLIP’s 15.4%. The principal implication for AI practitioners is that leveraging large-scale, meticulously curated datasets with detailed long captions, region-level annotations, and challenging negative samples is crucial for advancing the nuanced understanding and discriminative power of multimodal models, particularly for tasks requiring fine-grained distinctions. |
| 3D Scene Generation: A Survey (Read more on arXiv or HuggingFace) |
Fangzhou Hong, liuziwei7, FrozenBurning, hzxie, wenbc21 |
This survey systematically reviews and categorizes state-of-the-art 3D scene generation techniques, analyzing their foundations, trade-offs, datasets, and applications. The paper’s objective is to provide a comprehensive overview of 3D scene generation, organizing existing approaches and identifying current challenges and future research directions at the intersection of generative AI, 3D vision, and embodied intelligence. The authors surveyed and classified existing methods into four main paradigms—procedural generation, neural 3D-based generation, image-based generation, and video-based generation—analyzing their technical foundations, trade-offs, and representative results, along with datasets and evaluation protocols. The survey highlights a significant growth in the field, particularly noting that in 2024, neural 3D-based generation and video-based generation methods saw 93 and 61 publications respectively (Figure 1, with 2025 data being partial up to April 30th). For AI practitioners, this work offers a structured guide to current 3D scene generation methods, their comparative strengths (summarized in Table 1), common datasets (Table 3), and evaluation protocols, facilitating informed decision-making for developing applications in areas such as immersive media, robotics, and autonomous driving. |
| ICon: In-Context Contribution for Automatic Data Selection (Read more on arXiv or HuggingFace) |
Zhifang Sui, soliz1998, yaolily, Rsy24, yyxsghx |
The paper introduces ICON, a gradient-free method leveraging in-context learning (ICL) to automatically select high-contribution data for LLM instruction tuning, enhancing performance while reducing computational costs. The primary objective is to develop an efficient automated data selection method for instruction tuning that measures individual sample contribution without costly gradient computations or manually designed heuristics. ICON quantifies sample contribution by assessing performance shifts (via perplexity changes) on a diverse assessment set when a candidate sample is included in an ICL prompt, then uses these “ICON scores” to train a lightweight LoRA-based selection model. On LLaMA3.1-8B, training on 15% of ICON-selected Alpaca data outperformed full dataset training by 5.42 percentage points on average across 12 benchmarks and surpassed the best prior selection methods by 2.06 percentage points. AI practitioners can use ICON for more efficient and effective instruction tuning dataset curation, as it demonstrates that smaller, carefully selected subsets comprising diverse and appropriately difficult samples can yield superior model performance with significantly reduced computational overhead. |
| LiftFeat: 3D Geometry-Aware Local Feature Matching (Read more on arXiv or HuggingFace) |
Jinchi Zhu, Yuxuan Xiong, Zhou Zhao, Wenpeng Lai, pengliu123 |
i) The paper introduces LiftFeat, a lightweight network integrating 3D geometric features to enhance 2D local feature matching robustness. ii) The main objective is to improve the discriminative ability of 2D feature descriptors under extreme conditions by incorporating 3D geometric information. iii) The methodology involves extracting 3D geometric features supervised by pseudo surface normal labels derived from monocular depth estimation and fusing these with 2D descriptors using a 3D Geometry-aware Feature Lifting module. iv) Experimental results show LiftFeat outperforms other lightweight methods on relative pose estimation, homography estimation, and visual localization, and runtime tests confirm that the method can achieve inference latency of 7.4 ms on edge devices. v) LiftFeat offers AI practitioners a computationally efficient method for enhancing feature matching in challenging scenarios by leveraging readily available 3D geometric context, which is helpful to integrate into robotic applications. |
| X-Reasoner: Towards Generalizable Reasoning Across Modalities and |
|
|
| Domains (Read more on arXiv or HuggingFace) |
RustyArchimedes, sidkiblawi, hiaoxui, shengz, qianchu |
This paper introduces X-REASONER, a vision-language model that achieves strong generalizable reasoning across modalities and domains through post-training solely on general-domain text. The research investigates whether reasoning capabilities can be effectively generalized across different input modalities and specialized domains using only general-domain text-based post-training. X-REASONER employs a two-stage post-training process: supervised fine-tuning (SFT) on general-domain text with distilled long chain-of-thoughts, followed by reinforcement learning with verifiable rewards (RLVR) using mathematical text. Despite text-only training, X-REASONER surpasses prior 7B models trained with multimodal data on benchmarks like MMMU (Val) (56.4% vs. 55.0% SOTA) and MMMU-Pro (43.0% vs. 40.7% SOTA), while its medical-specialized variant, X-REASONER-MED, achieves new SOTA on medical tasks. The principal implication for AI practitioners is that carefully designed text-only post-training can be a highly data-efficient strategy to imbue models with robust, transferable reasoning skills, potentially reducing reliance on expensive multimodal or in-domain datasets. |
| Generating Physically Stable and Buildable LEGO Designs from Text (Read more on arXiv or HuggingFace) |
junyanz, devakramanan, RLCMU, kangled, AvaLovelace |
i) The paper introduces LEGOGPT, an autoregressive model for generating physically stable and buildable LEGO designs from text prompts. ii) The primary objective is to generate LEGO brick models from text while ensuring physical stability and buildability. iii) The methodology involves constructing a dataset of LEGO designs with associated captions and training an autoregressive large language model to predict the next brick via next-token prediction, incorporating physics-aware constraints during training and inference. iv) The method achieves 98.8% stability on generated LEGO structures and outperforms other baselines in mean brick stability and CLIP score. v) LEGOGPT provides AI practitioners with a framework integrating language models and physics constraints for generating realizable 3D structures directly from text, enhancing design automation and robotic assembly applications. |
| Crosslingual Reasoning through Test-Time Scaling (Read more on arXiv or HuggingFace) |
JuliaKreutzerCohere, gentaiscool, Muennighoff, MJonibek, yongzx |
This research demonstrates that scaling test-time inference compute for English-centric reasoning language models (RLMs) substantially improves their multilingual mathematical reasoning, though this benefit is domain-specific and less effective for low-resource languages. The study investigates the extent to which English-centric RLMs, finetuned with long chain-of-thoughts, can generalize reasoning capabilities across diverse languages and domains by scaling inference-time compute. The authors evaluated s1 models (Qwen2.5-Instruct finetuned on 1k English STEM samples) across various sizes on multilingual benchmarks (e.g., MGSM, Global-MMLU), analyzing the effects of increased thinking tokens, language forcing strategies, and emergent language-mixing patterns like “quote-and-think.” Crosslingual test-time scaling significantly improves multilingual math reasoning for models ≥3B parameters (e.g., a 14B s1 model showed a +Δ9.4% average accuracy gain on MGSM with more thinking tokens), often outperforming larger baseline models; however, models show poor out-of-domain generalization from STEM to cultural commonsense reasoning. Practitioners should consider test-time compute scaling for English-centric RLMs (≥3B) to enhance multilingual reasoning in high-resource languages for STEM tasks, but recognize its limitations for low-resource languages and out-of-domain applications, where specialized multilingual training data is still crucial. |
| PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes (Read more on arXiv or HuggingFace) |
abdo-eldesokey, zuluquebec, Aileron, Filippo8, Samir55 |
This paper introduces PlaceIt3D, a novel task, benchmark, and dataset for language-guided 3D object placement in real scenes, along with a baseline method called PlaceWizard. The main objective is to develop a system that can find a physically plausible and semantically correct 3D position and orientation for an asset in a scene based on a natural language description, addressing the ambiguity of multiple valid solutions and complex 3D spatial reasoning. The proposed PlaceWizard method utilizes a point encoder for scene features, uniform spatial pooling, a pre-trained Point-BERT for asset encoding, and a Large Language Model (LLM) that processes these features and the text prompt to predict placement location, anchor objects (auxiliary), and rotation masks via specialized decoder heads. Primary results show PlaceWizard achieved a global constraint accuracy of 52.6% and a complete placement success rate of 29.4% on the new benchmark, significantly outperforming an adapted Reason3D baseline which scored 40.6% global constraint accuracy and 18.1% complete placement success. The principal implication for AI practitioners is that PlaceIt3D provides a challenging new benchmark and dataset for evaluating 3D LLMs, fostering the development of AI agents with enhanced capabilities for understanding and interacting with 3D environments based on natural language, crucial for robotics and AR/VR applications. |
| BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language |
|
|
| Models in Chinese (Read more on arXiv or HuggingFace) |
Bruce Leon, HawkFaust, yeeeqichen99, MindYing, PALIN2018 |
This paper introduces BrowseComp-ZH, a benchmark for evaluating the web browsing ability of Large Language Models (LLMs) in the Chinese language environment. The primary objective is to assess LLM agents on the Chinese web, considering its unique linguistic, infrastructural, and censorship-related complexities often overlooked by English-centric benchmarks. The methodology involves 289 reverse-engineered multi-hop questions spanning 11 diverse domains, subjected to a two-stage quality control protocol to ensure high difficulty and answer uniqueness, which were used to benchmark over 20 state-of-the-art LLMs and agentic search systems. Key results demonstrate that most models perform poorly, with many achieving accuracy rates below 10% and even the best-performing system, OpenAI’s DeepResearch, reaching only 42.9% accuracy. For AI practitioners, this highlights a critical need to enhance LLMs’ capabilities in effective retrieval, sophisticated reasoning, and information reconciliation to master complex web browsing tasks, particularly in non-English information ecosystems. |
| Chain-of-Thought Tokens are Computer Program Variables (Read more on arXiv or HuggingFace) |
Zhifang Sui, peiyiwang89, soliz1998 |
i) This paper investigates the role of chain-of-thought (CoT) tokens in large language models (LLMs), proposing that they function similarly to variables in computer programs. ii) The research objective is to empirically study the function of CoT tokens, and whether they store intermediate values used in subsequent computations. iii) The methodology involves fine-tuning Qwen-2.5-1.5B on multi-digit multiplication and dynamic programming tasks, intervening on CoT tokens, and merging them into latent tokens to evaluate performance. iv) Results show that removing non-result tokens from CoT causes little performance drop, and the performance decreases by 9% when latent tokens store larger numbers in 4*5 DP problems indicating a computational complexity limit. v) The implication for AI practitioners is that LLMs treat CoT tokens similarly to program variables, so alternative forms of CoT should be explored to design more concise and efficient reasoning processes. |
Papers for 2025-05-08
| Title |
Authors |
Summary |
| Unified Multimodal Understanding and Generation Models: Advances, |
|
|
| Challenges, and Opportunities (Read more on arXiv or HuggingFace) |
Minghao Fu, Jintao Guo, Xinjie Zhang, Flourish, Suikong |
This paper surveys recent advancements in unified multimodal models integrating vision-language understanding and generation. The objective is to provide a comprehensive overview of current efforts to unify disparate architectural paradigms for multimodal understanding (often autoregressive) and generation (often diffusion-based). The paper reviews and categorizes existing unified models based on their core architecture (diffusion-based, autoregressive-based, or hybrid) and image tokenization strategies (e.g., pixel, semantic, learnable query), also compiling relevant datasets and benchmarks. The survey highlights a rapid growth, identifying over 40 distinct unified models emerging between 2023 and early 2025 (Fig. 1), with varied approaches such as Emu2 (LLaMA backbone, EVA-CLIP encoder, SDXL decoder) and Janus-Pro (DeepSeek-LLM backbone, SigLIP + VQGAN encoders). AI practitioners receive a structured guide to the diverse architectures (e.g., autoregressive MLLMs using semantic encoders like CLIP paired with diffusion decoders), key datasets (e.g., LAION 5.9B image-text pairs), and benchmarks, aiding the development and evaluation of sophisticated unified multimodal systems. |
| ZeroSearch: Incentivize the Search Capability of LLMs without Searching (Read more on arXiv or HuggingFace) |
Yingyan Hou, Xuanbo Fan, Zile Qiao, Hao Sun, SpaceProduct |
ZEROSEARCH is a reinforcement learning framework that enhances LLM search capabilities by fine-tuning an LLM to simulate a search engine, thus avoiding real search engine interactions and associated API costs. Its objective is to improve LLMs’ search and reasoning without the high costs and document quality unpredictability of live search engine interactions. The core methodology involves supervised fine-tuning of a “simulation LLM” to generate controlled-quality documents (relevant or noisy) for queries, coupled with a curriculum learning strategy that progressively increases retrieval difficulty during RL training, and a loss masking mechanism for retrieved tokens. ZEROSEARCH consistently outperforms real search engine-based methods, with a 14B parameter simulation LLM achieving an average Exact Match score of 33.97 across several question answering datasets, surpassing Google Search which scored 32.47, while demonstrating stable learning and generalizability. This offers AI practitioners a cost-effective and stable approach to develop LLMs with strong search and reasoning skills by simulating search environments, reducing reliance on expensive APIs and improving control over training data quality. |
| PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with |
|
|
| Auto-Regressive Transformer (Read more on arXiv or HuggingFace) |
Yiqin Zhu, Yanning Zhou, Jingwen Ye, loktarxiao, hyz317 |
PrimitiveAnything introduces a novel framework for generating 3D primitive assemblies by learning from human-crafted abstractions using an auto-regressive transformer. The main objective is to enable the generation of high-quality primitive assemblies that align with human perception and maintain geometric fidelity across diverse 3D shape categories, by reformulating shape abstraction as a sequence generation task. The methodology involves an ambiguity-free parameterization scheme for multiple primitive types, a shape-conditioned decoder-only transformer for auto-regressive primitive generation, and a cascaded primitive decoder to model attribute dependencies, trained on a large-scale dataset of human-crafted abstractions. Primary results demonstrate superior performance, achieving a Voxel-IoU of 0.484 on the HumanPrim test set, significantly outperforming optimization-based methods like EMS (0.259) and MP (0.201). The principal implication for AI practitioners is a method to create semantically structured and editable 3D content that is lightweight and aligns with human cognitive processes, useful for applications requiring efficient and interpretable 3D representations, such as user-generated content in games or computer-aided design. |
| HunyuanCustom: A Multimodal-Driven Architecture for Customized Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yuan Zhou, Sen Liang, Zhengguang Zhou, Zhentao Yu, Teng Hu |
HunyuanCustom is a novel multimodal-driven architecture for customized video generation that prioritizes subject consistency across image, audio, video, and text inputs. The primary objective is to enable flexible, user-defined video generation featuring specific subjects with robust identity preservation and multi-modal controllability. The framework, built on HunyuanVideo, incorporates a LLaVA-based text-image fusion module, an image ID enhancement module using temporal concatenation, and distinct injection mechanisms including an AudioNet for audio and a patchify-based feature-alignment network for video conditioning. HunyuanCustom significantly outperforms existing methods, achieving a Face-Sim score of 0.627 for ID consistency, surpassing competitors in single- and multi-subject scenarios. This work offers AI practitioners a robust approach for developing highly controllable, identity-preserving video generation systems, with direct applications in areas requiring precise subject customization like virtual human creation and fine-grained video editing. |
| Beyond Recognition: Evaluating Visual Perspective Taking in Vision |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Maciej Wołczyk, Michał Nauman, Piotr Miłoś, Alicja Ziarko, Gracjan |
This research evaluates Vision Language Models’ (VLMs) visual perspective taking (VPT) capabilities, revealing strong scene understanding but significant deficiencies in spatial reasoning and perspective-taking. The primary objective is to investigate the ability of state-of-the-art VLMs to perform visual perspective taking by assessing three levels of visual cognition: scene understanding, spatial reasoning, and visual perspective taking. The study employed 144 unique visual tasks featuring a humanoid minifigure and an object in systematically varied spatial configurations and viewpoints, with each task accompanied by 7 open-ended diagnostic questions; model responses were evaluated using a precision-based correctness metric. While VLMs excelled in scene understanding (e.g., GPT-4o achieved 100.0% correctness), their performance significantly declined for spatial reasoning (e.g., GPT-4o at 72.9% for minifigure orientation) and further deteriorated for visual perspective taking (e.g., GPT-4o at 59.0% for determining object location from the minifigure’s viewpoint). The principal implication for AI practitioners is that current VLMs lack robust internal geometric and perspective-dependent spatial reasoning, indicating a need for future VLM development to integrate explicit geometric representations and tailored training protocols beyond surface-level object recognition for reliable application in complex, interactive domains. |
| Benchmarking LLMs’ Swarm intelligence (Read more on arXiv or HuggingFace) |
Hao Sun, Ji-Rong Wen, Mowen Huang, 6cf |
This paper introduces SwarmBench, a benchmark for evaluating the emergent swarm intelligence of Large Language Models (LLMs) operating as decentralized agents under strict local perception and communication constraints. The research aims to systematically assess whether LLMs can exhibit effective coordination and collective intelligence akin to natural swarms when faced with limited local information, by evaluating their performance on five multi-agent tasks (Pursuit, Synchronization, Foraging, Flocking, Transport) within a configurable 2D grid world. The methodology involves LLM-driven agents operating with a k × k local view (e.g., 5x5 in main experiments) and optional local messaging, evaluated in a zero-shot setting using metrics for task success and emergent group dynamics. Evaluations of thirteen LLMs revealed significant performance variability, with emergent physical group dynamics, such as behavioral variability (std_action_entropy correlating with score at r = 0.300), explaining approximately 24.5% of task score variance, while explicit communication characteristics showed a weaker influence. For AI practitioners, this implies that when designing LLM-based multi-agent systems under severe decentralization, focusing on enhancing emergent physical coordination strategies may yield more significant performance gains than solely refining explicit communication protocols, as current LLMs struggle with robust planning under such constraints. |
| Beyond Theorem Proving: Formulation, Framework and Benchmark for Formal |
|
|
| Problem-Solving (Read more on arXiv or HuggingFace) |
Qinxiang Cao, Xingzhi Qi, Renqiu Xia, Xinhao Zheng, purewhite42 |
This paper presents a principled formulation of problem-solving as a deterministic MDP, introduces FPS and D-FPS frameworks for process-verified solving in FTP environments, and new benchmarks with the RPE evaluation metric. The research aims to establish a rigorous and verifiable approach to formal problem-solving beyond traditional theorem proving, enabling AI agents to produce process-level auditable solutions. Key methodologies include defining problem-solving as a deterministic Markov Decision Process, implementing the Formal Problem-Solving (FPS) and Deductive FPS (D-FPS) frameworks in Lean 4, constructing three novel benchmarks (FormalMath500, MiniF2F-Solving, PutnamBench-Solving), and proposing Restricted Propositional Equivalence (RPE) for answer correctness evaluation. Primary results show that SOTA FTP models using FPS solved at most 23.77% of FormalMath500, 27.47% of MiniF2F-Solving, and 0.31% of PutnamBench-Solving according to RPE, while D-FPS, though achieving lower solving rates, yielded nearly zero incorrectly submitted answers. For AI practitioners, these frameworks and benchmarks provide essential tools for developing and evaluating AI systems capable of verifiable, step-by-step formal reasoning, critical for applications requiring high trustworthiness and auditable solution processes. |
| OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue |
|
|
| Resolution (Read more on arXiv or HuggingFace) |
Jiachi Chen, Yanlin Wang, Runhan Jiang, Lianghong Guo, itaowe |
The paper introduces OmniGIRL, a novel multilingual, multimodal, and multi-domain benchmark for GitHub issue resolution. The primary objective is to create a comprehensive benchmark to evaluate the capabilities of Large Language Models (LLMs) in resolving diverse, real-world GitHub issues, addressing limitations of existing benchmarks regarding language, domain, and input modality. OmniGIRL was constructed by collecting 959 task instances from 15 popular repositories across four programming languages (Python, JavaScript, TypeScript, Java) and eight domains, including issues with textual, image, and website link information, followed by execution-based verification. Evaluations show current LLMs have limited performance on OmniGIRL; notably, the best-performing model, GPT-40 with the Agentless-X method, resolved only 8.6% of the total issues, and for issues requiring image understanding, Claude-3.5-Sonnet resolved only 10.5% using an oracle retrieval method with image-augmented text. AI practitioners should be aware that current LLMs significantly struggle with complex, multilingual, and multimodal software engineering tasks like GitHub issue resolution, indicating a substantial need for improved model capabilities and methods to handle cross-file and multimodal contexts effectively. |
| OpenHelix: A Short Survey, Empirical Analysis, and Open-Source |
|
|
| Dual-System VLA Model for Robotic Manipulation (Read more on arXiv or HuggingFace) |
Xinyang Tong, Shuanghao Bai, Wenxuan Song, Pengxiang Ding, Can Cui |
This paper presents OpenHelix, an open-source dual-system Vision-Language-Action (VLA) model for robotic manipulation, alongside a survey and empirical analysis of dual-system VLA design choices. Its main objective is to systematically evaluate core design elements of dual-system VLA architectures, such as MLLM training, policy training, and integration strategies, and to propose an effective, low-cost open-source model based on these findings. The study employs empirical evaluations on the CALVIN benchmark, varying MLLM training (frozen, fine-tuning, prompt-tuning), policy training (from scratch, fine-tuning pre-trained), and integration strategies (projector pre-alignment, auxiliary tasks), leading to the OpenHelix model which uses prompt-tuned LLaVA-7B and a pre-trained 3D Diffuser Actor policy with an auxiliary multimodal reasoning task. Key results demonstrate that MLLM prompt tuning with an auxiliary task significantly improves performance, with the proposed configuration achieving a 4.01 average task completion length on CALVIN (Table 7) and 46.9% 5-task completion success on CALVIN ABC-D with 60-step asynchronous inference (Table 8). For AI practitioners, this implies that prompt-tuning large MLLMs with auxiliary tasks for enhanced visual reasoning, coupled with careful pre-alignment of dual-system components, is a highly effective strategy for robotic VLA development, and that asynchronous inference between systems often has minimal impact on overall performance. |
| OSUniverse: Benchmark for Multimodal GUI-navigation AI Agents (Read more on arXiv or HuggingFace) |
Sinéad Ryan, Arturo Márquez Flores, Patrick Barker, Daniel Jeffries, mariya-davydova |
The paper introduces OSUniverse, a benchmark for evaluating multimodal GUI-navigation AI agents on complex desktop tasks with automated validation. The main objective is to provide a robust, extensible benchmark with increasing task complexity to measure the capabilities of GUI-navigation AI agents and to assess current state-of-the-art (SOTA) performance. The methodology involves defining tasks in YAML, running them in Dockerized desktop environments (AgentDesk) using a SurfKit-compatible runtime, and employing automated validation with Gemini models for scoring, supplemented by a human review interface. Primary results show that SOTA agents (at publication) achieve less than 50% accuracy, with the top agent (computer-use-preview-2025-03-11) scoring 47.80%; the automated validation mechanism exhibits an average error rate below 2% (1.64% with Gemini 2.0 Flash). The principal implication for AI practitioners is that OSUniverse provides a challenging and calibrated benchmark with automated, non-deterministic validation to assess GUI-navigation agents, highlighting that even top proprietary models require custom agentic code and specialized training for optimal performance, with open-weight models lagging. |
| Knowledge Augmented Complex Problem Solving with Large Language Models: |
|
|
| A Survey (Read more on arXiv or HuggingFace) |
Yuqi Zhu, Yuchen Tian, Junwei Su, Lun Du, Da Zheng |
This survey examines the capabilities and limitations of Large Language Models (LLMs) in complex problem-solving, focusing on multi-step reasoning, knowledge augmentation, and result verification across various domains. The paper aims to provide a comprehensive overview of current LLM techniques for tackling complex problems, highlight challenges such as data scarcity and computational costs, and discuss future research directions. The survey analyzes methodologies including Chain-of-Thought (CoT) reasoning for multi-step problem decomposition, knowledge augmentation via retrieval-augmented generation (RAG) and knowledge graphs, and various result verification techniques such as LLM-based verifiers and tool-assisted validation. Key findings highlighted include the inference scaling law, where solution coverage can grow nearly log-linearly with the number of sampled reasoning paths [10], and that training dedicated verifier models significantly improves solve rates on tasks like GSM8K math problems compared to only fine-tuning the generator LLM [20]. For AI practitioners, this implies that LLM problem-solving can be substantially enhanced by systematically integrating structured reasoning processes, incorporating external knowledge sources, and employing robust verification loops, while also needing to address the high computational demands of extensive search and reasoning. |
| R&B: Domain Regrouping and Data Mixture Balancing for Efficient |
|
|
| Foundation Model Training (Read more on arXiv or HuggingFace) |
Ziyi Chu, Avi Trost, John Cooper, Tzu-Heng Huang, Albert Ge |
R&B is a two-stage framework that improves foundation model training efficiency by first re-partitioning data based on semantic similarity (Regroup) and then dynamically optimizing data mixture proportions using domain gradients (Balance). The paper addresses how to overcome the limitations of predetermined data domains and the computational inefficiency of existing data mixing methods in foundation model training. R&B employs semantic clustering (e.g., k-means on embeddings) for data regrouping and leverages a Gram matrix of domain gradients, updated during training, to dynamically reweight skill mixtures via a regularized softmax optimization. Empirically, R&B matches or exceeds state-of-the-art data mixing performance while significantly reducing computational overhead, requiring as little as 0.01% additional compute; for instance, on SUP-NATINST, R&B achieved a loss of 2.381 with 0.009% overhead. AI practitioners can significantly reduce computational costs and potentially improve performance in foundation model training by adopting semantic data regrouping and gradient-based dynamic mixture balancing, avoiding expensive per-skill evaluations. |
| Uncertainty-Weighted Image-Event Multimodal Fusion for Video Anomaly |
|
|
| Detection (Read more on arXiv or HuggingFace) |
Mohsen Imani, Paper9795, Eavn |
This paper introduces IEF-VAD, a framework that synthesizes event representations from RGB videos and fuses them with image features using an uncertainty-aware process, aiming to enhance video anomaly detection by integrating temporal cues from synthetic event data with spatial RGB information. The key methodology involves extracting image and synthetic event features via CLIP, modeling sensor noise with a Student’s-t likelihood, and deriving inverse-variance weights through Laplace approximation for fusion. Furthermore, IEF-VAD employs Kalman-style sequential updates and an iterative refinement network to denoise the fused latent state before classification using a composite loss function including KL divergence and modality alignment terms. IEF-VAD achieved state-of-the-art results, such as an AUC of 88.67% on UCF-Crime and 92.90% on MSAD (Student’s-t model), with masking experiments confirming adaptive uncertainty weighting. For AI practitioners, this work shows that fusing synthetic event data with RGB data via principled uncertainty estimation (e.g., Student’s-t noise model, inverse-variance weighting) can significantly improve video anomaly detection by capturing motion cues without dedicated event sensors, offering a practical enhancement for video understanding systems. |
| Cognitio Emergens: Agency, Dimensions, and Dynamics in Human-AI |
|
|
| Knowledge Co-Creation (Read more on arXiv or HuggingFace) |
linxule |
This paper introduces Cognitio Emergens (CE), a comprehensive theoretical framework for understanding and guiding the co-evolutionary nature of human-AI partnerships in scientific knowledge co-creation. The primary objective is to propose the CE framework to address limitations in existing models by capturing the dynamic, emergent, and co-evolutionary processes through which scientific understanding is co-created. The methodology is primarily theoretical, synthesizing theories like autopoiesis and social systems theory to define CE through three core components: Agency Configurations, Epistemic Dimensions, and Partnership Dynamics. The primary result is the CE framework itself, detailing three Agency Configurations, six Epistemic Dimensions (e.g., Divergent Intelligence, Synthesis Intelligence) forming “capability signatures” (Section 3.2.4) for diagnostic purposes, and six Partnership Dynamics; the paper, being a framework proposal, does not present empirical quantitative findings. For AI practitioners, CE offers tools to design AI systems as evolving epistemic partners, focusing on dynamic agency and specific collaborative capabilities rather than solely on narrow performance metrics. |
Papers for 2025-05-07
| Title |
Authors |
Summary |
| Unified Multimodal Chain-of-Thought Reward Model through Reinforcement |
|
|
| Fine-Tuning (Read more on arXiv or HuggingFace) |
Qinglin Lu, Chunyu Wang, Zhimin Li, Yibin Wang, yuhangzang |
This paper introduces UNIFIEDREWARD-THINK, a unified multimodal Chain-of-Thought (CoT) reward model enhanced by reinforcement fine-tuning to improve reward signal accuracy for visual understanding and generation tasks. The main objective is to enable reliable, multi-dimensional CoT reasoning in reward models by eliciting and incentivizing latent complex reasoning capabilities, despite the scarcity of explicit CoT supervision data. The key methodology employs a three-stage training pipeline: (1) cold-starting by distilling CoT reward format from GPT-4o, (2) refining through rejection sampling on large-scale unified preference data, and (3) leveraging Group Relative Policy Optimization (GRPO) for reinforcement fine-tuning using verifiable format and accuracy rewards. UNIFIEDREWARD-THINK achieved superior performance, for example, attaining a 72.3% macro accuracy on the VLRewardBench for image understanding, compared to 66.6% by the UnifiedReward baseline, and also demonstrated improved implicit reasoning capabilities when CoT was not explicitly output. For AI practitioners, this work offers a method to develop more accurate and interpretable multimodal reward models by incorporating CoT through reinforcement learning, which can significantly enhance the alignment of vision models with human preferences, even with limited explicit CoT data. |
| Absolute Zero: Reinforced Self-play Reasoning with Zero Data (Read more on arXiv or HuggingFace) |
Andrew Zhao, zlzheng, shenzhi-wang, Yang130, kevinwyr |
This paper introduces Absolute Zero, an RLVR paradigm where a model self-improves reasoning by autonomously proposing and solving tasks using only a code executor for verifiable rewards, without any external data. The research aims to develop a system where a large language model can enhance its reasoning capabilities purely through self-play, eliminating reliance on human-curated data for task definition or solution verification. The core methodology involves the Absolute Zero Reasoner (AZR), a single model acting as both a task proposer (rewarded for task learnability) and a solver (rewarded for solution correctness) for self-generated coding tasks (deduction, abduction, induction), with a code executor providing feedback and Task-Relative REINFORCE++ for updates. AZR, trained entirely without external data, achieved state-of-the-art performance, surpassing previous zero-setting models that used curated data by an average of 1.8 absolute points on combined coding and math reasoning benchmarks. This paradigm offers AI practitioners a pathway to build more autonomous reasoning systems capable of self-generating training curricula and improving without continuous human data supervision, potentially overcoming data bottlenecks and enabling learning beyond human-provided tasks. |
| FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios (Read more on arXiv or HuggingFace) |
Yansong Tang, Ying Shan, Zhaoyang Zhang, Shiyi Zhang, JunhaoZhuang |
FlexiAct proposes a novel framework for transferring actions from a reference video to an arbitrary target image, achieving flexible action control in heterogeneous scenarios with varying spatial structures while maintaining appearance consistency. The main objective is to overcome the limitations of existing action customization methods that require strict spatial alignment (layout, skeleton, viewpoint) between reference and target, by enabling action transfer across diverse subjects and domains. The methodology involves two key components: RefAdapter, a lightweight image-conditioned adapter for spatial adaptation and consistency preservation, and Frequency-aware Action Extraction (FAE), which dynamically adjusts attention to frequency-specific embeddings during the denoising process to precisely extract motion. Experiments show FlexiAct effectively transfers actions in diverse scenarios; in human evaluations, FlexiAct was preferred over a base model for motion consistency (79.5% vs. 20.5%) and appearance consistency (78.3% vs. 21.7%). For AI practitioners, FlexiAct offers a robust method for action-conditioned video generation where reference and target subjects differ significantly, broadening applications in animation and content creation by decoupling action from strict spatial constraints and utilizing dynamic, frequency-aware attention modulation. |
| RADLADS: Rapid Attention Distillation to Linear Attention Decoders at |
|
|
| Scale (Read more on arXiv or HuggingFace) |
Eugene Cheah, Janna Lu, Eric Alcaide, SmerkyG |
The paper introduces RADLADS, a rapid and cost-effective protocol for converting pre-trained softmax attention transformers into performant linear attention decoder models, alongside two new RWKV-variant architectures, RAD-RWKV6 and RAD-RWKV7. The primary objective is to develop a highly efficient method to distill knowledge from large softmax attention transformers into linear attention models, requiring significantly less data (350-700M tokens, <0.005% of original pre-training data) and compute than full pre-training, while preserving near-original model quality and achieving state-of-the-art performance for linear attention models. RADLADS employs a three-step conversion: 1) Attention Weights Transfer from the teacher, 2) Attention Hidden State Alignment using L2 loss on 100M tokens to match teacher attention hidden states, and 3) Knowledge Distillation of teacher output logits using Kullback-Leibler divergence loss on 250M-700M tokens, followed by optional fine-tuning. A key result is that a converted 72B Qwen2.5 model (QRWKV6-72B-Instruct) achieved an MMLU score of 0.754, closely matching its teacher’s 0.751, establishing new state-of-the-art downstream performance for a pure RNN language model of its size. For AI practitioners, RADLADS offers a practical pathway to create large-scale, inference-efficient linear attention models from existing powerful softmax transformers with significantly reduced costs, facilitating broader adoption of models with O(1) per-token inference complexity. |
| RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM |
|
|
| Inference (Read more on arXiv or HuggingFace) |
Chengruidong Zhang, Jinkai Zhang, Yaoqi Chen, qianxizhang, baotonglu |
RetroInfer presents a novel vector-storage system to accelerate long-context Large Language Model (LLM) inference by exploiting attention sparsity. The primary objective is to address GPU memory and bandwidth constraints that hinder efficient inference for LLMs with extended context lengths. Its core methodology involves the “wave index,” an Attention-aWare Vector index for retrieving critical tokens using tripartite attention approximation, accuracy-bounded estimation, and segmented clustering, complemented by a “wave buffer” for coordinating KV cache placement and hardware operations. Experiments demonstrate up to 4.5x speedup over full attention within GPU memory limits and up to 10.5x over sparse attention baselines when extending KV cache to CPU memory, while maintaining full-attention-level accuracy. For AI practitioners, RetroInfer offers a system to significantly improve throughput and scalability for deploying LLMs with very long contexts without compromising model accuracy. |
| Decoding Open-Ended Information Seeking Goals from Eye Movements in |
|
|
| Reading (Read more on arXiv or HuggingFace) |
Yevgeni Berzak, Yoav Meiri, Omer Shubi, Cfir Avraham Hadar |
This research investigates decoding open-ended, text-specific information-seeking goals from readers’ eye movements using multimodal LLMs. The primary objective is to determine if a reader’s specific question for a text can be automatically decoded from their eye movements, assessed via goal classification and reconstruction tasks. The methodology involves discriminative (adapted Haller RNN, ROBERTEye-Fixations) and novel generative (DalEye-LLaVA, DalEye-Llama) multimodal LLMs combining text and eye-tracking features from the OneStop dataset. ROBERTEye-Fixations achieved the highest classification accuracy at 49.3% overall (chance 33.0%), and significantly, 57.3% (chance 49.9%) in distinguishing questions over identical text spans, demonstrating extraction of fine-grained goal information. This suggests AI practitioners can leverage eye-tracking with LLMs to infer user-specific information needs for personalized systems, though precise goal generation requires further advancement. |
| An Empirical Study of Qwen3 Quantization (Read more on arXiv or HuggingFace) |
Xudong Ma, Yue Feng, Yuye Li, HaoranChu, Xingyu-Zheng |
This paper empirically evaluates the quantization robustness of the Qwen3 LLM series using five post-training quantization (PTQ) methods across bit-widths from 1 to 8 bits. The study’s main objective is to systematically assess Qwen3’s performance degradation under various quantization settings to identify opportunities and challenges in compressing these state-of-the-art models. The methodology involves applying five PTQ techniques (RTN, GPTQ, AWQ, SmoothQuant, BiLLM) to Qwen3 models, testing weight-only (1-8 bits) and weight-activation quantization, with performance measured on perplexity, 0-shot reasoning tasks, and 5-shot MMLU. Primary results indicate that while Qwen3 achieves near-lossless performance at 8-bit quantization, it shows noticeable degradation at 4-bits (e.g., Qwen3-8B’s MMLU score drops from 74.7 in FP16 to 69.3 with 4-bit AWQ per-group quantization) and experiences more pronounced degradation at 3-bits or fewer, particularly compared to previous model generations. Principal implication for AI practitioners: When deploying Qwen3, practitioners can expect robust performance with 8-bit quantization, but must carefully evaluate the noticeable performance trade-offs at 4-bits and the significant degradation at 3-bits or below, indicating a need for advanced quantization strategies or careful capability assessments for ultra-low precision applications of these models. |
| Multi-Agent System for Comprehensive Soccer Understanding (Read more on arXiv or HuggingFace) |
Yanfeng Wang, Ya Zhang, Zifeng Li, haoningwu, Homie0609 |
This paper introduces SoccerAgent, a multi-agent system for holistic soccer understanding, accompanied by SoccerWiki, a multimodal knowledge base, and SoccerBench, a comprehensive benchmark. The main research objective is to develop a comprehensive framework for AI-driven soccer understanding that moves beyond isolated tasks to enable knowledge-driven reasoning. The key methodology involves constructing SoccerWiki with information on 9,471 players and 266 teams, creating SoccerBench with ~10K multimodal multi-choice QA pairs across 13 tasks, and developing SoccerAgent, a multi-agent system that decomposes questions and invokes 18 specialized tools. SoccerAgent achieved 85.0% accuracy on TextQA tasks and 60.9% on VideoQA tasks within SoccerBench, outperforming existing Multimodal Large Language Models. The principal implication for AI practitioners is the provision of a new benchmark (SoccerBench) and a multi-agent system architecture (SoccerAgent) that demonstrates effective task decomposition and tool utilization for complex, domain-specific multimodal understanding, offering a template for similar AI applications. |
| Geospatial Mechanistic Interpretability of Large Language Models (Read more on arXiv or HuggingFace) |
Kevin Roitero, Stefano Mizzaro, sdesabbata |
This paper introduces a framework using spatial analysis and sparse autoencoders to interpret how Large Language Models internally represent geographical information. Its objective is to understand the internal mechanisms LLMs use to process and encode geospatial data. The study extracted activations from Mistral-7B-Instruct-v0.2 for placename prompts, analyzed them using spatial autocorrelation, then applied sparse autoencoders to decompose activations from layer 15 into features, which were also spatially analyzed. While 14.98% of raw neuron activations across multiple layers exhibited polysemantic spatial patterns, sparse autoencoder decomposition of layer 15 activations yielded only 0.2% (67 of 32,768) of features with significant spatial autocorrelation, indicating sparse, though sometimes more monosemantic, geospatial encoding and highlighting areas for further research in decomposition techniques. The principal implication for AI practitioners is that this framework offers a method to interpret LLMs’ complex and sparsely distributed geographical representations, which is critical for developing more reliable and well-understood foundation models for geospatial applications by revealing how models internally handle such data. |
| InfoVids: Reimagining the Viewer Experience with Alternative |
|
|
| Visualization-Presenter Relationships (Read more on arXiv or HuggingFace) |
Kevin Hsu, Ivy Chen, Tongyu Zhou, Ji Won Chung, Franck-Dernoncourt |
This paper introduces “InfoVids,” an augmented reality (AR) paradigm that integrates presenters and visualizations within a shared 3D space to enhance viewer experience compared to traditional 2D slide-based presentations. The primary objective is to investigate how these alternative spatial arrangements and interactions affect viewer engagement, perceived presenter immersion, and attention dynamics. Researchers developed four InfoVid case technology probes using ARKit and a custom Body Object Model (BOM), which were then compared against 2D baseline equivalents by 30 public participants through surveys and semi-structured interviews. Results showed InfoVids significantly shifted viewer attention towards the presenter (e.g., for AIRPLANEVIS, 16 out of 30 participants shifted focus to the presenter) and were generally perceived as more engaging and immersive. For AI practitioners developing data communication or presentation tools, this research indicates that co-locating presenters and AR visualizations can create more human-centric experiences, suggesting a valuable approach for designing AI-driven data storytelling systems that prioritize presenter engagement. |
| VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient |
|
|
| Large Speech-Language Model (Read more on arXiv or HuggingFace) |
Lijiang Li, Heting Gao, Chaoyou Fu, Yunhang Shen, Zuwei Long |
VITA-Audio is an end-to-end large speech model designed for fast interleaved cross-modal token generation to reduce high first-audio-token latency in streaming applications. The primary objective is to achieve real-time audio generation within end-to-end speech models, specifically enabling zero audio token delay after the initial LLM forward pass. This is accomplished using lightweight Multiple Cross-modal Token Prediction (MCTP) modules that efficiently generate multiple audio tokens directly from LLM hidden states within a single model forward pass, combined with a four-stage progressive training strategy. VITA-Audio demonstrates a 3-5x inference speedup at the 7B parameter scale and reduces the first audio token chunk generation time from 236ms (Vanilla mode) to 53ms (Boost mode). The principal implication for AI practitioners is that VITA-Audio offers an effective architecture for developing highly responsive, real-time conversational AI systems by enabling immediate audio output from the first forward pass. |
| Invoke Interfaces Only When Needed: Adaptive Invocation for Large |
|
|
| Language Models in Question Answering (Read more on arXiv or HuggingFace) |
Biao Qin, Chunlai Zhou, Robot2050 |
This paper proposes AttenHScore, an unsupervised metric for adaptive LLM invocation in Question Answering by detecting Small Language Model (SLM) hallucinations in real-time, complemented by an uncertainty-aware text re-ranking strategy. The main objective is to precisely determine when to invoke a large language model (LLM) if a small language model (SLM) is likely hallucinating, thereby optimizing the trade-off between performance and cost in collaborative LM systems. The key methodology involves “AttenHScore,” which calculates the accumulation and propagation of hallucinations during SLM generation using token probabilities (Pmax(xi)) and attention scores (Atten(xi)), and an uncertainty-based re-ranking of retrieved documents by guiding SLMs to generate queries from text chunks. Primary results show AttenHScore outperforms baselines; for example, with Llama3-8B-Instruct on SQuAD, it achieved an AUCS of 0.8715 and ACCr of 0.8176. The re-ranking strategy improved the F1 score of Vicuna-7B-v1.5 by 3.37 on MultiFieldQA-zh. The principal implication for AI practitioners is the provision of a plug-and-play, unsupervised method to reduce computational costs and improve QA system efficiency by adaptively invoking expensive LLMs only when SLMs demonstrate signs of hallucination, without needing additional model training. |
| HoloTime: Taming Video Diffusion Models for Panoramic 4D Scene |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yonghong Tian, Xinhua Cheng, Jiawen Guan, Haiyang Zhou, Drexubery |
HoloTime is a novel framework that generates immersive panoramic 4D scenes from images or prompts by integrating specialized video diffusion for panoramic video creation and a robust 4D reconstruction pipeline. Its primary objective is to overcome the limitations of existing methods in producing truly immersive, dynamic 360-degree 4D scene-level assets for VR/AR applications. The methodology combines the “360World” dataset of fixed-camera panoramic videos, a “Panoramic Animator” (a two-stage motion-guided image-to-video diffusion model with hybrid fine-tuning and panoramic circular techniques), and “Panoramic Space-Time Reconstruction” (using space-time aligned depth estimation and 4D Gaussian Splatting). The framework demonstrates superior performance, with HoloTime achieving an 87.74% user preference for graphics quality in image-driven 4D scene generation compared to 3D-Cinemagraphy (1.94%), and significantly higher user ratings for text-driven panoramic video quality. For AI practitioners, HoloTime offers a method to create high-fidelity, spatially and temporally consistent panoramic 4D environments, enhancing immersive experiences, and provides the 360World dataset as a resource for developing similar panoramic video generation models. |
| Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in |
|
|
| Smart Personal Assistant (Read more on arXiv or HuggingFace) |
Xiaoyu Shen, lorashen |
This paper introduces Auto-SLURP, a benchmark dataset for evaluating LLM-based multi-agent frameworks for smart personal assistants. The main objective is to provide a standardized benchmark for comprehensive end-to-end evaluation of these frameworks, covering language understanding, task execution, and response generation. Auto-SLURP extends the original SLURP dataset by relabeling slots and integrating simulated servers and external services, with experiments conducted on frameworks like CamelAI, LangGraph, AutoGen, and AgentLite using GPT-4. Primary results show AgentLite achieved the highest accuracy at 0.46, and finetuning an intent agent (LLAMA-3 8B) on AutoGen improved its accuracy from 0.40 to 0.62, a 55% performance increase. The principal implication for AI practitioners is that Auto-SLURP offers a challenging testbed for developing and iterating on more reliable multi-agent personal assistant systems, revealing that current frameworks require significant improvement, especially in areas like intent processing. |
Papers for 2025-05-06
| Title |
Authors |
Summary |
| Voila: Voice-Language Foundation Models for Real-Time Autonomous |
|
|
| Interaction and Voice Role-Play (Read more on arXiv or HuggingFace) |
Yu Shu, Yemin Shi, zhitinghu, Jaward, guangyil |
The paper introduces Voila, a family of open-sourced, end-to-end voice-language foundation models designed for real-time, autonomous, and emotionally expressive human-AI interaction, supporting tasks like dialogue, ASR, and TTS. Its primary objective is to enable voice AI agents to interact autonomously and proactively by moving beyond reactive pipeline systems towards full-duplex, low-latency conversations preserving rich vocal nuances. Voila utilizes a hierarchical multi-scale Transformer architecture with an LLM backbone and a hierarchical audio generator, a novel voice tokenizer (Voila-Tokenizer) that distills semantic and acoustic information into layered RVQ tokens, and a structured text-audio interleaved alignment strategy for multi-task training. Voila achieves a response latency of 195 milliseconds and an accuracy of 30.56 on its custom Voila Benchmark, significantly outperforming prior models, and a 2.7% Word Error Rate for ASR on LibriSpeech test-clean (when trained with LibriSpeech data). This provides AI practitioners with an open-source foundation for developing next-generation autonomous voice AI systems with improved naturalness, responsiveness, and customizability, offering a unified model that effectively integrates LLM reasoning with nuanced voice processing. |
| RM-R1: Reward Modeling as Reasoning (Read more on arXiv or HuggingFace) |
Ziqi Wang, zhangdenghui123, Merlin-Hongru, gaotang, XtremSup |
The paper introduces RM-R1, a family of Reasoning Reward Models (REASRMS) that formulate reward modeling as an explicit reasoning task to enhance LLM alignment. The research aims to improve the interpretability and performance of reward models for LLMs by integrating deep, interpretable reasoning capabilities into the reward generation and judgment process. RM-R1 is trained using a two-stage pipeline involving: 1) distillation of high-quality reasoning chains, often employing a Chain-of-Rubrics (CoR) framework, from stronger teacher models, and 2) subsequent reinforcement learning with verifiable rewards (RLVR) using Group Relative Policy Optimization (GRPO). RM-R1 models demonstrate state-of-the-art or near state-of-the-art performance, outperforming significantly larger open-weight and proprietary models by up to 13.8% on benchmarks like RewardBench, where RM-R1-QWEN-INSTRUCT-32B achieved 92.9% overall accuracy. AI practitioners can develop more robust, accurate, and interpretable LLM alignment systems by shifting from opaque scalar rewards to generative reward models that explicitly reason and justify their judgments, particularly through structured reasoning distillation and targeted RL. |
| Grokking in the Wild: Data Augmentation for Real-World Multi-Hop |
|
|
| Reasoning with Transformers (Read more on arXiv or HuggingFace) |
Gjergji Kasneci, Roman Abramov, fsteinbauer |
This paper demonstrates that augmenting real-world knowledge graphs with synthetic data to increase the ratio of inferred to atomic facts ($\phi_r$) enables Transformers to “grok” multi-hop reasoning, achieving high performance on factual question answering. The main research objective was to determine if Transformers can achieve “grokking” (transitioning from memorization to generalization) on real-world multi-hop factual reasoning tasks by synthetically increasing the ratio of inferred facts to atomic facts ($\phi_r$) in the training data above a critical threshold. The key methodology involved augmenting the 2WikiMultiHopQA dataset by generating synthetic atomic and multi-hop (inferred) facts to elevate the relation-specific ratio $\phi_r$ (e.g., to an achieved ratio of 8 for comparison tasks). A GPT-2 small model was then trained from scratch on this augmented data for an extended period (e.g., ~300k steps). The primary result showed that the grokked GPT-2 small model achieved 96% Out-of-Distribution (OOD) accuracy on the 2WikiMultiHopQA structured comparison task, significantly outperforming larger models like GPT-4o (reported at 87% for the same task average in Figure 1, or 0.87 for comparison in Table 3) and o1-mini (reported at 89% in Figure 1, or 0.88 for comparison in Table 3) which did not undergo the same data augmentation. The principal implication for AI practitioners is that targeted data augmentation to ensure a high density of multi-step inference examples relative to atomic facts can unlock robust multi-hop reasoning capabilities even in smaller Transformer models, offering a pathway to more efficient and potentially more interpretable factual reasoning systems without solely relying on model scale. (The paper’s arXiv date is listed as “29 Apr 2025”, which is unusual.) |
| FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
ZhengYuan, yifanzhang114, Liam-Liu, prt66, zhouliang |
This paper introduces FormalMATH, a large-scale Lean4 benchmark with 5,560 formally verified problems to evaluate the formal mathematical reasoning of large language models. The primary objective is to address limitations in the scope and scale of existing formal mathematics benchmarks and to rigorously assess current LLM-based theorem provers. FormalMATH was created via a human-in-the-loop autoformalization pipeline integrating specialized LLMs for statement generation, multi-LLM semantic verification, and negation-based disproof, achieving a 72.09% pass rate for candidate statements undergoing final manual expert verification. Evaluations on FormalMATH showed that even the strongest LLM-based theorem provers have significant limitations, achieving only a 16.46% success rate (Pass@32), and revealed that natural-language solution guidance can negatively impact formal proof success in chain-of-thought scenarios. FormalMATH offers a robust benchmark for advancing LLM-based formal theorem proving, highlighting needs for improved cross-domain generalization, deeper deductive capabilities beyond simple automation, and better integration of formal and informal reasoning. |
| ReplaceMe: Network Simplification via Layer Pruning and Linear |
|
|
| Transformations (Read more on arXiv or HuggingFace) |
szagoruyko121, stamatisl, madrugado, ammarali32, dimitriish |
ReplaceMe is a generalized training-free depth pruning method that replaces contiguous transformer blocks with an estimated linear transformation, maintaining high performance. The main objective is to simplify transformer networks by pruning layers and approximating their functionality with a single linear operation, estimated using a small calibration dataset, without requiring retraining. The key methodology involves identifying prunable blocks based on inter-layer activation distances (cosine distance preferred) and then computing an optimal linear transformation (LT) to replace these blocks, which is subsequently merged into a remaining layer. Primary results show that ReplaceMe can prune up to 25% of a Llama 2 7B model while retaining 92.5% of its original performance on open benchmarks using the cosine distance objective for LT estimation, significantly outperforming UIDL in compression time and environmental impact. For AI practitioners, ReplaceMe offers a computationally efficient, training-free approach to compress large language models, reducing latency and resource demands with minimal performance loss, thus facilitating more accessible deployment. |
| Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization |
|
|
| in Rejection Sampling and RL (Read more on arXiv or HuggingFace) |
nanjiang, WeiXiong, hendrydong, HanningZhang, FlippyDora |
This paper introduces GVM-RAFT, a dynamic sample allocation strategy that optimizes Chain-of-Thought (CoT) reasoner training by minimizing stochastic gradient variance. The primary objective is to improve the efficiency of CoT training, which often suffers from inefficient stochastic gradient estimation due to static sampling strategies, by dynamically allocating computational resources based on prompt-specific characteristics. GVM-RAFT proposes a prompt-specific Dynamic Sample Allocation Strategy that monitors prompt acceptance rates and stochastic gradient norms to minimize gradient variance under a computational budget, derived within an Expectation-Maximization framework. Experiments on mathematical reasoning tasks show that GVM-RAFT achieves a 2-4× speedup in convergence and considerable accuracy improvements over vanilla RAFT, for instance, GVM-RAFT++ improved the 5-benchmark average accuracy from 36.42% to 39.64% on Qwen2.5-Math-1.5B. AI practitioners can utilize this method to more efficiently fine-tune CoT models through rejection sampling or reinforcement learning, by adaptively allocating inference budgets to different prompts, thus accelerating training and enhancing final model accuracy. |
| Practical Efficiency of Muon for Pretraining (Read more on arXiv or HuggingFace) |
cadarsh-essential, monk-essential, karlstratos, ampolloreno, ishaan-essential |
This research demonstrates Muon, a second-order optimizer, expands the compute-time Pareto frontier over AdamW for pretraining and introduces an efficient muP-based telescoping hyperparameter tuning method. Its main objective is to investigate Muon’s practical efficiency compared to AdamW in large-scale language model pretraining, particularly the compute-time tradeoff and hyperparameter selection. Key methodology involved comparing optimizers via iso-loss frontiers on a compute-time plane using models up to 4 billion parameters, and developing a “telescoping” algorithm for maximal update parameterization (muP) that accounts for error sources. Primary results indicate Muon requires 10-15% fewer tokens than AdamW to achieve an identical loss, and the telescoping algorithm enables allocating over 20% of the total compute budget to the final model training run while ensuring near-optimal hyperparameters. For AI practitioners, this implies Muon offers a more data-efficient pretraining alternative to AdamW, especially at large batch sizes, and the telescoping muP approach facilitates cost-effective hyperparameter tuning, reducing overall training time and computational resources. |
| A Survey on Inference Engines for Large Language Models: Perspectives on |
|
|
| Optimization and Efficiency (Read more on arXiv or HuggingFace) |
Sungryeol Jeon, leejaymin, Devcow, oos2, inputsh |
This paper presents a comprehensive survey of 25 LLM inference engines, evaluating their optimization techniques, hardware support, and ecosystem maturity to guide efficient deployment. The primary objective is to systematically compare these open-source and commercial engines, identifying their design goals, supported features, and suitability for throughput- or latency-sensitive LLM services. The methodology involves analyzing each engine’s architecture, supported optimization categories (e.g., parallelism, compression, caching per Table 7), hardware compatibility (Table 4), and non-technical indicators like GitHub activity and documentation quality (Table 3). Key findings show significant diversity: engines like Ollama gained high user preference (209.6 average daily GitHub star growth) for ease of use, while solutions like vLLM and TensorRT-LLM offer extensive, specialized optimizations for demanding server-side inference, supporting techniques such as PagedAttention and various parallelisms. The principal implication for AI practitioners is a structured guide for selecting optimal inference engines based on specific performance requirements, hardware constraints, and the trade-offs between ease-of-use, feature support, and ecosystem maturity, facilitating more efficient and cost-effective LLM service deployment. |
| R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
KevinTowne, KaiyuValley, bhsc24, XingyuLu, yifanzhang114 |
This paper introduces R1-Reward, a multimodal reward model (MRM) trained via a novel StableReinforce algorithm to enhance reward modeling through stable reinforcement learning. The primary objective is to explore and improve the application of reinforcement learning (RL) for multimodal reward modeling by addressing the instability issues of existing RL algorithms in this context. The key methodology involves reformulating reward modeling as a rule-based RL task and developing StableReinforce, which incorporates refined training loss (Pre-CLIP), advantage estimation (Advantage Filter), and a novel consistency reward mechanism using an MLLM referee. R1-Reward achieves state-of-the-art performance, demonstrating a 14.3% improvement on the Multimodal Reward Bench compared to previous SOTA models. For AI practitioners, this work provides a robust method (StableReinforce) and a high-performing model (R1-Reward) for developing more accurate MRMs, crucial for improving MLLM alignment, data filtering, and evaluation. |
| Think on your Feet: Adaptive Thinking via Reinforcement Learning for |
|
|
| Social Agents (Read more on arXiv or HuggingFace) |
Xinghua Zhang, Haobo Wang, bingliwu, Yongbin-Li, iiiiwis |
This paper introduces Adaptive Mode Learning (AML) with an Adaptive Mode Policy Optimization (AMPO) algorithm to enable social agents to dynamically adjust reasoning depth in social interactions. The main objective is to develop language agents that can dynamically adjust their reasoning depth based on real-time context in social simulations, unlike current approaches that use fixed reasoning depths or lack reasoning capabilities. The key methodology involves defining four thinking modes (intuitive reaction to deep contemplation) and using the AMPO algorithm, which incorporates multi-granular thinking mode design, context-aware mode switching, and token-efficient reasoning, trained via behavioral cloning and reinforcement learning. Primary results show AML achieves 15.6% higher task performance than state-of-the-art methods, and notably, outperforms GRPO by 7.0% in performance with 32.8% shorter reasoning chains. For AI practitioners, AMPO provides a framework to develop more human-like, adaptive, and token-efficient social agents capable of context-sensitive reasoning in complex social environments. |
| SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from |
|
|
| Sparse and Noisy Demonstrations (Read more on arXiv or HuggingFace) |
Hok Wai Tsui, Yinhuai Wang, cqf, Crimnos, IngridYU |
SkillMimic-V2 introduces a framework for learning robust and generalizable robot interaction skills from sparse and noisy demonstrations by augmenting data and employing adaptive training. The main research objective is to overcome demonstration noise and coverage limitations in Reinforcement Learning from Interaction Demonstration (RLID), enabling robots to learn complex skills from limited and imperfect human demonstrations. The key methodology involves two data augmentation techniques—Stitched Trajectory Graph (STG) and State Transition Field (STF)—an Adaptive Trajectory Sampling (ATS) strategy for curriculum generation, and a History Encoder (HE) for memory-dependent skills. The method enhances generalization performance by over 35%; for instance, on the BallPlay-M benchmark, it achieved an average ε-Neighborhood Success Rate (εNSR) of 49.3% compared to 18.3% for the baseline SkillMimic (SM). The principal implication for AI practitioners is that this approach allows for the training of AI agents for complex physical interaction tasks using sparse and noisy demonstrations, significantly improving skill robustness and generalization beyond the provided data. |
| Agentic Reasoning and Tool Integration for LLMs via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
akshaynambi, akshaynambi, Raghav2002, joykirat |
The paper introduces ARTIST (Agentic Reasoning and Tool Integration in Self-improving Transformers), a framework using outcome-based reinforcement learning (RL) to enable LLMs to autonomously reason and integrate external tools for complex problem-solving. The objective is to enable LLMs to autonomously decide when, how, and which tools to invoke within multi-step reasoning chains, learning robust strategies via RL without step-level supervision. ARTIST trains LLMs using Group Relative Policy Optimization (GRPO), interleaving text-based reasoning with tool invocations and outputs, guided by a composite reward function (correctness, format, tool success) and loss masking for tool outputs. ARTIST achieved up to a 22% absolute improvement in mathematical reasoning (e.g., Qwen2.5-14B-ARTIST scored 0.55 Pass@1 on AMC) and more than doubled accuracy on some multi-turn function calling tasks compared to base models. Integrating agentic reasoning with dynamic tool use via outcome-based RL, as in ARTIST, offers a robust path to enhance LLMs for complex tasks requiring external interaction, without needing detailed step-by-step supervision data. |
| SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based |
|
|
| Image Editing (Read more on arXiv or HuggingFace) |
Xin Gu, Zilence006, lionwen, xiaoying0505, limingcv |
SuperEdit introduces a data-oriented method to improve instruction-based image editing by rectifying editing instructions using diffusion priors and facilitating supervision with contrastive learning. The primary objective is to address noisy supervision in instruction-based image editing by developing more effective editing instructions that better align with original-edited image pairs, thereby improving model performance without requiring architectural changes or extensive pre-training. The methodology involves: i) Rectifying editing instructions by guiding a Vision-Language Model (GPT-4O) with diffusion generation priors, which link inference timesteps to specific image attribute changes (global layout, local objects, style/details); and ii) Constructing contrastive supervision signals by generating positive (rectified) and negative (incorrect, subtly altered) instructions from the VLM and training the editing model using a triplet loss. SuperEdit demonstrated a 9.19% performance improvement over the prior state-of-the-art SmartEdit on the Real-Edit benchmark (achieving an Overall Score of 3.91), while utilizing 30x less training data (40K samples) and a 13x smaller model (1.1B parameters). For AI practitioners, this research highlights that significant performance gains in instruction-based image editing can be achieved by focusing on the quality and precision of supervision signals rather than solely on model architecture complexity or extensive pre-training, suggesting a more data-centric and efficient path for model improvement. |
| Low-Precision Training of Large Language Models: Methods, Challenges, |
|
|
| and Opportunities (Read more on arXiv or HuggingFace) |
Li Shen, Guoxia, csdvT, GGJY, Zhiwei840 |
This survey comprehensively reviews low-precision training techniques for Large Language Models (LLMs), categorizing approaches by numerical formats to address research fragmentation. The primary objective is to systematically organize existing methods—fixed-point/integer-based, floating-point-based, and customized formats—and discuss quantization-aware training (QAT) and system support. The paper reveals an increasing adoption of integer and low-precision floating-point methods, citing an example where FP8-LM achieved a 75% training speedup compared to BF16 for a 175B parameter model. For AI practitioners, this survey offers a structured understanding of how to implement more resource-efficient LLM training pipelines by selecting appropriate low-precision techniques and leveraging evolving hardware support. |
| Ming-Lite-Uni: Advancements in Unified Architecture for Natural |
|
|
| Multimodal Interaction (Read more on arXiv or HuggingFace) |
bear-xxy, jianxinsun, chenjingdong, zhengdd0422, BiaoGong |
Ming-Lite-Uni is an open-source multimodal framework unifying vision and language through a novel visual generator, multi-scale learnable tokens, and a native autoregressive model for tasks like text-to-image generation and instruction-based editing. The paper aims to demonstrate a unified autoregressive multimodal model built upon multi-scale learnable tokens with fine-tuned diffusion models and to accelerate community engagement by open-sourcing its implementation, improving upon the integrated MetaQueries and M2-omni frameworks. The framework leverages a fixed Multimodal Large Language Model (MLLM) (Llama3-based M2-omni) and fine-tunes an external diffusion model using newly designed multi-scale learnable query tokens, a multi-scale representation alignment strategy (minimizing Mean Squared Error between DiT backbone intermediate states and final semantic representations), and a FlowMatching loss. Ming-Lite-Uni achieved an overall accuracy of 0.62 on the GenEval benchmark for text-to-image generation, notably scoring 0.99 on single-object generation, and demonstrated strong multimodal understanding with an 80.7 MMB score and 72.3 MM-Vet score. This work provides AI practitioners with an open-source, unified architecture that effectively integrates understanding and generation capabilities, offering a practical foundation for developing advanced multimodal AI systems with robust interactive and generative performance. |
| TEMPURA: Temporal Event Masked Prediction and Understanding for |
|
|
| Reasoning in Action (Read more on arXiv or HuggingFace) |
vibhav-vineet, yilche, wchai, hsiangwei0903, andaba |
TEMPURA is a two-stage training framework employing masked event prediction and dense captioning, supported by the new VER dataset, to significantly improve temporal event understanding and causal reasoning in video action. The research aims to enhance video Large Multi-modal Models’ (LMMs) capability to understand causal event relationships and achieve fine-grained temporal grounding in videos by enabling them to infer missing events and segment videos into detailed, temporally-aligned event descriptions. TEMPURA utilizes a two-stage training pipeline: first, it applies masked event prediction reasoning to reconstruct missing events and generate step-by-step causal explanations from dense event annotations; second, it learns to perform video segmentation and dense captioning to decompose videos into non-overlapping events with detailed, timestamp-aligned descriptions, using the newly curated VER dataset (500K videos, 18K hours). TEMPURA significantly outperforms strong baseline models without task-specific fine-tuning, achieving a mean Intersection over Union (mIoU) of 39.2 on the Charades-STA benchmark (a 6.3 point improvement over the baseline) and a HIT@1 score of 51.7 on the QVHighlights dataset (a 6.9 point improvement). This work provides AI engineers and data scientists with a structured two-stage training approach and a large-scale dataset (VER) for developing video LMMs with enhanced abilities to reason about event causality and perform fine-grained temporal segmentation, crucial for applications like highlight detection and detailed video analysis. |
| LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive |
|
|
| Streaming Speech Synthesis (Read more on arXiv or HuggingFace) |
Yang Feng, Yan Zhou, zhangshaolei, guoshoutao, poeroz |
LLaMA-Omni 2 is a series of modular Speech Language Models (0.5B-14B parameters) achieving real-time, high-quality spoken chatbot interaction through an autoregressive streaming speech synthesis pipeline. The main objective is to develop an end-to-end spoken language model capable of real-time, intelligent, and natural speech interaction, addressing limitations of cascaded systems and the extensive data requirements of native SpeechLMs, while retaining strong underlying text capabilities. The system integrates a Whisper speech encoder and adapter with a Qwen2.5 LLM, followed by a streaming speech generation module; this module comprises an autoregressive text-to-speech language model (MTTS), initialized from Qwen2.5-0.5B and utilizing a “Read-R-Write-W” strategy to generate speech tokens, which are then converted to mel spectrograms by a causal flow matching model and HiFi-GAN vocoder, with the entire system fine-tuned on 200K synthesized multi-turn speech dialogues. LLaMA-Omni 2 demonstrates strong performance, with the 7B parameter model achieving 31.3% accuracy on the Web Questions speech-to-speech benchmark, significantly outperforming GLM-4-Voice (15.9%) and exhibiting a latency of 582.91ms for the first speech chunk (using R=3, W=10). The principal implication for AI practitioners is that this modular approach, leveraging pre-trained LLMs with specialized speech components and an efficient streaming architecture, enables the development of high-performance, real-time spoken dialogue systems using substantially less speech-specific training data (200K samples) compared to large native SpeechLMs, offering a more data-efficient pathway. |
| MUSAR: Exploring Multi-Subject Customization from Single-Subject Dataset |
|
|
| via Attention Routing (Read more on arXiv or HuggingFace) |
Chong Mou, Pengze Zhang, heqian, yanze, Zinan123212 |
MUSAR introduces a framework for multi-subject image customization using only single-subject training data via attention routing. The primary objective is to overcome the challenges of acquiring diverse multi-subject training data and mitigating attribute entanglement between subjects in text-to-image generation. MUSAR employs de-biased diptych learning, which constructs multi-subject training pairs from single-subject images and corrects systemic biases using static attention routing and dual-branch LoRA, alongside a dynamic attention routing mechanism that adaptively maps image regions to their corresponding conditional subjects to prevent entanglement. Quantitatively, on DreamBench multi-subject customization, MUSAR achieved a DINO score of 0.704 and a CLIP-I score of 0.720, outperforming methods trained on actual multi-subject datasets. This work provides AI practitioners a data-efficient pathway to develop robust multi-subject customization models without relying on difficult-to-obtain multi-subject datasets, by leveraging synthesized training data and refined attention mechanisms. |
| Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural |
|
|
| Radiance Fields (Read more on arXiv or HuggingFace) |
Dan Xu, Xue Xiao, Ping Yin, Zhenxing Mi |
This paper introduces Switch-NeRF++, a Heterogeneous Mixture of Hash Experts (HMoHE) framework for efficiently learning decomposition and heterogeneous representations of large-scale Neural Radiance Fields. The main objective is to develop a highly scalable NeRF method that addresses learnable scene decomposition, models scene heterogeneity, and improves modeling efficiency for complex, large-scale scenes in an end-to-end manner. The key methodology involves a hash-based gating network that learns to decompose scenes and allocate 3D points to a set of distinct, heterogeneous hash experts, each designed with different hash grid resolution ranges, all co-optimized within a Sparsely Gated Mixture of Experts (MoE) NeRF framework. Primary results demonstrate state-of-the-art rendering accuracy and significant efficiency improvements; for instance, Switch-NeRF++ achieves an 8x acceleration in training and a 16x acceleration in rendering (e.g., rendering a 1152x864 image in 6.65s versus 110s for Switch-NeRF) compared to the best-performing competitor Switch-NeRF, and outperforms INGP on the UrbanBIS dataset (PSNR 20.76 vs 19.58). The principal implication for AI practitioners is the provision of a more practical and efficient solution for applying NeRFs to real-world, large-scale 3D scene modeling, enabling higher quality and faster reconstruction with reduced computational resources, particularly for scenes with diverse content. |
| Unlearning Sensitive Information in Multimodal LLMs: Benchmark and |
|
|
| Attack-Defense Evaluation (Read more on arXiv or HuggingFace) |
Jie Peng, Peter Hase, mohitbansal, a2889184, vaidehi99 |
This paper introduces UNLOK-VQA, a benchmark, and an attack-defense framework for evaluating targeted unlearning of sensitive information in Multimodal Large Language Models (MLLMs). The main objective is to systematically evaluate the effectiveness of unlearning methods in MLLMs, particularly for deleting specific multimodal knowledge while preserving model utility. The methodology involves generating the UNLOK-VQA dataset with varied proximity samples for efficacy, generalization, and specificity testing, and an attack-defense framework comprising seven attack types (e.g., a novel Probability Delta2 whitebox attack) against six LoRA-based unlearning defense objectives. Primary results show that multimodal extraction attacks (45.5% success rate against a baseline defense) are more effective than image-only (32%) or text-only (39%) attacks, though the Head Projection (HP) defense significantly reduces multimodal blackbox attack success to 15.7%. For AI practitioners, this research underscores the heightened risk of sensitive information leakage in MLLMs via multimodal inputs and provides a benchmark (UNLOK-VQA) and evidence that specific defense strategies (like HP) are critical for mitigating these vulnerabilities during MLLM development and deployment. |
Papers for 2025-05-05
| Title |
Authors |
Summary |
| PixelHacker: Image Inpainting with Structural and Semantic Consistency (Read more on arXiv or HuggingFace) |
xinggangw, steelozazala, wenyuliu, SmileTAT, Uyoung |
PixelHacker introduces Latent Categories Guidance (LCG) within a diffusion model for structurally and semantically consistent image inpainting. The objective is to overcome limitations of existing inpainting methods that struggle with complex structures and semantics, leading to artifacts and logically incoherent results. The key methodology is Latent Categories Guidance (LCG), utilizing separate fixed-size embeddings for latent ‘foreground’ and ‘background’ features derived from diverse mask types (semantic, random), injected into a diffusion model’s denoising steps via linear attention. PixelHacker demonstrated superior performance, achieving a state-of-the-art FID of 8.59 on the Places2 test set (512 resolution, 40-50% masks), outperforming models like SDXL. For practitioners, the LCG approach demonstrates an effective technique to enhance structural and semantic coherence in diffusion-based inpainting models by conditioning on coarse foreground/background distinctions, rather than complex textual prompts or fine-grained labels, potentially simplifying guidance while improving output quality for image editing applications. |
| Improving Editability in Image Generation with Layer-wise Memory (Read more on arXiv or HuggingFace) |
Jaesik Park, Jaeah Lee, carpedkm |
This paper introduces a framework employing layer-wise memory to improve control and consistency in sequential, mask-guided image editing. The primary objective is to enable multiple edits while preserving background integrity and ensuring natural integration of new elements using only rough user masks, overcoming limitations of single-object editing methods. Key methodologies include a layer-wise memory storing previous edit latents and prompts, Background Consistency Guidance (BCG) for stable background preservation and efficient latent blending, and Multi-Query Disentanglement (MQD) in cross-attention for coherent object integration across layers. The proposed method demonstrates superior performance on a new Multi-Edit Benchmark, achieving a BLEU-4 score of 36.59 and a CLIP score of 64.29, outperforming existing editing and layout-to-image models in sequential tasks. For AI practitioners, this framework offers a robust technique for developing interactive editing systems capable of complex, multi-step modifications with minimal user effort while maintaining high fidelity and contextual coherence. |
| Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG |
|
|
| Evaluation Prompts (Read more on arXiv or HuggingFace) |
Wenge Rong, Yiqi Liu, Chenghao Xiao, Hanhua Hong, yangwang825 |
This paper introduces inversion learning to automatically generate highly effective, model-specific evaluation prompts for NLG systems using just a single evaluation sample. The objective is to overcome the limitations and prompt sensitivity issues inherent in manually crafted prompts used for LLM-based evaluation. The key methodology involves training an inversion model to learn the reverse mapping from an LLM evaluator’s output (e.g., human score) back to the corresponding input instruction (evaluation prompt). Results show inversion prompts consistently outperform human-crafted and forward prompts across tasks and models; for LLaMA-3.1-8B-Instruct (Black-Box), inversion prompts achieved a 33% higher average Spearman correlation than forward prompts, demonstrating model-specificity is crucial. The principal implication for AI practitioners is that generating tailored evaluation prompts via inversion learning, instead of using generic ones, leads to more robust, efficient, and reliable LLM-based evaluation. |
| Llama-Nemotron: Efficient Reasoning Models (Read more on arXiv or HuggingFace) |
Ran El-Yaniv, Mohammad Dabbah, Izik Golan, Itay Levy, Akhiad Bercovich |
The Llama-Nemotron paper introduces an open family of heterogeneous reasoning models (Nano-8B, Super-49B, Ultra-253B) optimized for efficiency and enterprise use. The main objective was to develop models delivering exceptional reasoning capabilities combined with high inference throughput and memory efficiency under a permissive open license. Key methodologies include neural architecture search (NAS) from Llama 3 models using the Puzzle framework, FFN Fusion, knowledge distillation, continued pretraining, supervised fine-tuning (SFT) on curated synthetic data, and large-scale reinforcement learning (RL). Primary results demonstrate the flagship LN-Ultra (253B) achieves state-of-the-art open model performance, outperforming DeepSeek-R1 on benchmarks like GPQA-Diamond (76.0%) while offering significantly higher inference throughput (e.g., 4x at 500/2000 ISL/OSL on 8xH100). For AI practitioners, this provides commercially permissive, high-performance reasoning models optimized for efficient deployment, featuring a novel dynamic reasoning toggle to switch between chat and reasoning modes. |
| CORG: Generating Answers from Complex, Interrelated Contexts (Read more on arXiv or HuggingFace) |
Trung Bui, aifactoryysh, Franck-Dernoncourt, hyunjilee |
This paper introduces CORG, a framework for language models to generate answers from complex corpora by organizing interrelated contexts into processed groups. The objective is to improve language model answer generation accuracy, recall, and disambiguation when processing multiple contexts exhibiting distracting, ambiguous, counterfactual, or duplicated relationships. CORG employs a graph constructor to identify context interrelationships, a reranker to organize contexts into optimized groups based on relationship type, and an aggregator to generate cited answers per group. Results demonstrate CORG’s effectiveness; on the AmbigDocs+ dataset with Llama2-7B, CORG achieved a Disambig-F1 score of 22.0, substantially outperforming grouping baselines like KMeans (3.6) and single-pass methods like base processing (17.0). AI practitioners can utilize CORG as an inference-time solution to enhance the robustness of retrieval-augmented generation systems facing inconsistent real-world documents, improving answer quality and entity disambiguation without requiring model retraining. |
| Real-World Gaps in AI Governance Research (Read more on arXiv or HuggingFace) |
Tim O’Reilly, sruly, isobelmoure, strauss-NYC |
This paper analyzes AI safety and reliability research, revealing a corporate focus on pre-deployment and gaps in post-deployment risk analysis. The objective was to compare the research outputs and priorities of leading AI companies (Anthropic, Google DeepMind, Meta, Microsoft, OpenAI) versus top AI universities regarding AI safety and reliability, particularly pre- versus post-deployment issues. Methodology involved analyzing 1,178 safety/reliability papers from 9,439 generative AI papers (Jan 2020-Mar 2025), applying fractional authorship adjustments, classifying papers using GPT-4o mini, and conducting keyword searches for specific risk domains. Primary results indicate corporate research increasingly concentrates on pre-deployment alignment and testing, while only 4% of corporate safety papers address high-risk deployment domains (e.g., misinformation, medical contexts, hallucinations, copyright); ethics and bias research is now predominantly academic. The principal implication for AI practitioners is that current corporate-led research may underemphasize critical risks emerging after deployment, necessitating caution as established best practices for real-world operational safety and reliability remain underdeveloped in public research. |
| TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matching (Read more on arXiv or HuggingFace) |
Chuchu Fan, yuemithucsd |
TeLoGraF introduces a graph-encoded flow matching framework for planning trajectories that satisfy general Signal Temporal Logic (STL) specifications. The objective is to learn a single conditional generative model capable of handling diverse STL specifications as input without requiring retraining for new formulas. It encodes STL specifications as syntax graphs processed by a Graph Neural Network (GNN) whose embedding conditions a flow-matching model to generate trajectories. Results show TeLoGraF outperforms baselines in STL satisfaction rates across five environments; notably, its “Fast” variant achieves up to 123.6X faster inference than gradient-based methods on the Franka Panda benchmark while maintaining high satisfaction. For AI practitioners, this provides a significantly faster inference method for planning under complex temporal and logical constraints in robotics and cyber-physical systems, though performance degrades on heavily out-of-distribution STLs. |
| X-Cross: Dynamic Integration of Language Models for Cross-Domain |
|
|
| Sequential Recommendation (Read more on arXiv or HuggingFace) |
Haggai Roitman, liorrokach, Bshapira, yeshel, guyhadad01 |
This paper presents X-Cross, a model for cross-domain sequential recommendation via dynamic, layer-wise integration of language models fine-tuned with LoRA. The primary objective is to enable effective sequential recommendation in new target domains by transferring knowledge from multiple source-domain models, requiring minimal target-domain data and avoiding full model retraining. X-Cross utilizes trainable integrators at each layer to dynamically compute weights for combining activations from frozen, LoRA-adapted source domain language models, refining representations progressively. Results show X-Cross achieves performance comparable to target-domain LoRA fine-tuning using only 25% of the adapter parameters, and requires significantly less fine-tuning data (e.g., 83.3% less for Electronics domain adaptation) to surpass baseline performance. For AI practitioners, X-Cross provides a parameter- and data-efficient method for adapting recommendation systems to new domains, reducing computational overhead and data requirements in data-constrained or rapidly evolving environments. |
Papers for 2025-05-02
| Title |
Authors |
Summary |
| A Survey of Interactive Generative Video (Read more on arXiv or HuggingFace) |
Xintao Wang, Quande Liu, Haoxuan Che, Yiran Qin, Jiwen Yu |
This paper surveys the emerging field of Interactive Generative Video (IGV), defining it and outlining its applications. The main objective is to provide a comprehensive overview of IGV technology, survey its application landscape (gaming, embodied AI, autonomous driving), and propose a systematic framework to guide future development. The methodology involves synthesizing existing literature on video generation and interactive systems, classifying current IGV models (shown evolving since 2020 in Fig 1), and proposing a novel five-module framework (Generation, Control, Memory, Dynamics, Intelligence). Primary results include the proposed five-module IGV framework, an analysis identifying key technical challenges such as achieving real-time generation and ensuring long-term coherence, and a categorization of existing IGV methods across different application domains (Tables 1-3). The principal implication for AI practitioners is the provision of a structured framework to decompose the complex problem of IGV, enabling systematic development and targeted research into specific module challenges like control generalization or physics simulation for interactive AI systems. |
| DeepCritic: Deliberate Critique with Large Language Models (Read more on arXiv or HuggingFace) |
Ji-Rong Wen, Yankai Lin, Jingwen Chen, Keven16 |
This paper introduces DeepCritic, a two-stage framework enhancing LLM critique abilities for mathematical reasoning. The objective is to develop LLM critics capable of deliberate, step-wise critiques with multi-perspective verification and meta-critiquing, addressing the superficiality of existing critics. The methodology involves supervised fine-tuning on 4.5K generated long-form critiques, followed by reinforcement learning using PRM800K data or Monte Carlo sampling-based annotations. The resulting DeepCritic-7B-RL-PRM800K model achieves a 67.1 average F1 score on error identification benchmarks, outperforming models like GPT-4o and same-sized DeepSeek-R1-distill models. For AI practitioners, this demonstrates a method to create more accurate automated supervision models that provide detailed feedback, improving LLM generator refinement and potentially enabling weak-to-strong supervision. |
| T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level |
|
|
| and Token-level CoT (Read more on arXiv or HuggingFace) |
Hao Li, Zhuofan Zong, Renrui Zhang, Ziyu Guo, Dongzhi Jiang |
T2I-R1 introduces a reasoning-enhanced text-to-image generation model using reinforcement learning with a bi-level Chain-of-Thought (CoT) process. The primary objective is to improve generation quality and prompt alignment by explicitly coordinating high-level semantic planning (semantic-level CoT) and low-level pixel processing (token-level CoT). The core methodology employs BiCoT-GRPO, a novel reinforcement learning framework that jointly optimizes both CoT levels within a Unified Large Multimodal Model (ULM) using group-relative rewards from an ensemble of vision expert models. T2I-R1 demonstrates superior performance, achieving a 13% improvement over its baseline on T2I-CompBench and surpassing the state-of-the-art FLUX.1 model. For AI practitioners, this work highlights that integrating and explicitly optimizing multi-level reasoning processes (planning and step-by-step generation) via RL within unified models can significantly boost performance and robustness in complex generative tasks. |
| AdaR1: From Long-CoT to Hybrid-CoT via Bi-Level Adaptive Reasoning |
|
|
| Optimization (Read more on arXiv or HuggingFace) |
Rui Liu, Jinluan Yang, Yibo Wang, Haiying He, Haotian Luo |
AdaR1 introduces a bi-level optimization method for LLMs to adaptively switch between Long-CoT and Short-CoT reasoning, enhancing efficiency without sacrificing performance. The main objective is to overcome the high inference cost of Long-CoT by tailoring reasoning depth to input problem complexity. Key methodology involves merging long and short CoT models and then applying Bi-Level Preference Training (using DPO) to optimize reasoning path selection at both group (style) and instance (conciseness) levels. Primary results demonstrate a significant reduction in reasoning length (over 50% on average across five math datasets) while largely maintaining accuracy compared to Long-CoT baselines (-1.65% accuracy change with -50.93% length reduction for the 7B model). For AI practitioners, this approach offers a way to deploy powerful reasoning models more efficiently by dynamically allocating computational resources based on task demands, improving feasibility in latency-sensitive or resource-constrained environments. |
| KeySync: A Robust Approach for Leakage-free Lip Synchronization in High |
|
|
| Resolution (Read more on arXiv or HuggingFace) |
Konstantinos Vougioukas, Michał Stypułkowski, Stella Bounareli, Rodrigo Mira, Antoni Bigata |
KeySync introduces a two-stage latent diffusion framework for high-resolution, leakage-free, and occlusion-robust lip synchronization. The research objective is to overcome common lip-sync limitations including temporal inconsistency, expression leakage from source video, and poor occlusion handling, especially in cross-synchronization tasks. Key methodology involves keyframe generation and interpolation via a diffusion model conditioned on audio embeddings and carefully masked video latents, augmented by an inference-time occlusion handling pipeline using video segmentation. KeySync achieves state-of-the-art cross-synchronization performance, notably obtaining a LipScore of 0.48 while significantly reducing expression leakage to a LipLeak score of 0.16, outperforming existing methods quantitatively (Elo: 1145). For AI practitioners, this provides a robust model for high-fidelity applications like automated dubbing, offering temporally coherent output that minimizes leakage and handles occlusions effectively. |
| TF1-EN-3M: Three Million Synthetic Moral Fables for Training Small, Open |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Laura Diosan, andreeatomescu, andreiPiscoran, mihainadas |
This paper presents TF1-EN-3M, a dataset of 3 million synthetic English moral fables generated using open-weight language models under 8B parameters. The research investigates the effectiveness of combinatorial prompt expansion for generating diverse, high-quality fables with resource-constrained LLMs and identifies optimal models. Methodology involved a 6-slot prompt template (character, trait, setting, conflict, resolution, moral) and a hybrid evaluation pipeline combining GPT-based critique with reference-free diversity/readability metrics. Results show Llama-3.1-8B-Instruct achieved the highest composite score (0.891), producing high-quality fables at a cost of approximately $0.135 per 1000 on consumer hardware. The principal implication for AI practitioners is that large-scale, structured narrative datasets for tasks like moral reasoning can be efficiently created using small, open models, reducing reliance on proprietary systems. |
| LLMs for Engineering: Teaching Models to Design High Powered Rockets (Read more on arXiv or HuggingFace) |
Toby Simonds |
This research evaluates Large Language Models’ (LLMs) effectiveness in high-powered rocket design, showing reinforcement learning (RL) significantly enhances their capabilities. The study’s objective was to determine if LLMs can function as effective tools for physical engineering tasks, using high-powered rocketry as a test domain. Methodology involved creating RocketBench, an interface to the RocketPy simulator, evaluating foundation LLMs (Claude 3.7, o1, etc.) via iterative prompting on altitude and landing tasks, and training a 7B parameter Qwen-2.5 model with Group Relative Policy Optimization (GRPO). Key results show that while foundation models plateaued below human expert performance (e.g., max human score 76.57 on altitude), the RL-trained 7B model surpassed both humans and foundation models, achieving a peak score of 95.6 on the precision landing task (vs. 91.6 human expert) and landing within 12 meters accuracy. The principal implication for AI practitioners is that integrating RL with LLMs, leveraging their domain knowledge alongside structured exploration, enables performance exceeding human experts in complex engineering optimization, indicating a promising approach for AI-driven design provided effective simulation interfaces and reward functions exist. |
| MediAug: Exploring Visual Augmentation in Medical Imaging (Read more on arXiv or HuggingFace) |
Lei Zhang, Hao Zhang, Canxuan Gang, Zeyu Zhang, Xuyin Qi |
MediAug introduces a benchmark evaluating six mix-based data augmentation methods on medical image classification using CNN and Transformer backbones. The objective was to systematically assess the performance of MixUp, YOCO, CropMix, CutMix, AugMix, and SnapMix on brain tumor MRI and eye disease fundus datasets to address the domain gap and fragmented prior research. A unified framework applied these augmentations to train and evaluate ResNet-50 and ViT-B models on the two datasets. Results demonstrated significant variability: MixUp achieved the highest accuracy for ResNet-50 on brain tumors (79.19%), while SnapMix was best for ViT-B (99.44%); YOCO (ResNet-50, 91.60%) and CutMix (ViT-B, 97.94%) excelled on eye diseases. The principal implication for AI practitioners is that the optimal mix-based augmentation strategy is highly dependent on the specific combination of backbone architecture (CNN vs. Transformer) and the medical imaging task/dataset. |
Papers for 2025-05-01
| Title |
Authors |
Summary |
| Sadeed: Advancing Arabic Diacritization Through Small Language Model (Read more on arXiv or HuggingFace) |
Sara Chrouf, hr99, Moatasem444, Hennara, ZeinaD |
This paper introduces Sadeed, a compact decoder-only language model fine-tuned for Arabic diacritization, and SadeedDiac-25, a new benchmark for this task. The objective is to advance Arabic diacritization by developing an efficient model trained on high-quality data and establishing a more reliable evaluation framework. Methodology involved fine-tuning the Kuwain 1.5B model on rigorously cleaned Tashkeela and ATB data using a Question-Answering format, alongside creating the SadeedDiac-25 benchmark covering diverse Arabic styles with expert validation. Sadeed achieves competitive results, outperforming open-source models on SadeedDiac-25 (9.92 WER without case ending) and achieving state-of-the-art WER (2.9375 excluding non-diacritized chars) on a corrected version of the Fadel benchmark, though it requires post-processing via Needleman-Wunsch to correct hallucinations. For AI practitioners, this demonstrates the viability of small, specialized models for complex NLP tasks like diacritization and underscores the critical need for high-quality, curated training data and robust, uncontaminated benchmarks, revealing significant flaws in prior commonly used datasets. |
| WebThinker: Empowering Large Reasoning Models with Deep Research |
|
|
| Capability (Read more on arXiv or HuggingFace) |
Yutao Zhu, Hongjin Qian, Guanting Dong, Jiajie Jin, Xiaoxi Li |
WebThinker is a deep research agent empowering Large Reasoning Models (LRMs) with autonomous web exploration and report generation capabilities. The objective is to overcome LRM limitations stemming from reliance on static internal knowledge for complex, knowledge-intensive tasks requiring dynamic web information synthesis. The methodology integrates a Deep Web Explorer module for web search/navigation/extraction and an Autonomous Think-Search-and-Draft strategy, trained using RL-based iterative online Direct Preference Optimization (DPO). Results demonstrate significant improvements over baselines; for instance, WebThinker-32B-RL achieved 46.5% accuracy on the WebWalkerQA benchmark, substantially outperforming the Search-o1-32B baseline’s 34.1%. AI practitioners can utilize this framework to develop LRMs that perform real-time, in-depth web research and synthesis concurrently with multi-step reasoning, enhancing performance on complex information-seeking tasks. |
| Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language |
|
|
| Models in Math (Read more on arXiv or HuggingFace) |
Yen-Chun Chen, Dongdong Chen, Hany Awadalla, Baolin Peng, Haoran Xu |
This paper introduces a systematic multi-stage training recipe to significantly enhance the mathematical reasoning abilities of small language models (SLMs). The primary objective is to overcome the limitations of SLM capacity and develop robust reasoning capabilities competitive with larger models. The methodology involves four sequential steps: large-scale distilled CoT mid-training, high-quality CoT supervised fine-tuning, Rollout DPO using curated preference pairs, and Reinforcement Learning with a verifiable reward signal incorporating specific stability enhancements. Applying this recipe to the 3.8B Phi-4-Mini model resulted in Phi-4-Mini-Reasoning, achieving 94.6% Pass@1 on Math-500, outperforming larger models like DeepSeek-R1-Distill-Qwen-7B. For AI practitioners, this demonstrates that a carefully orchestrated training strategy with high-quality synthetic data can enable resource-constrained SLMs to achieve strong reasoning performance, offering a blueprint for developing efficient and capable models. |
| Softpick: No Attention Sink, No Massive Activations with Rectified |
|
|
| Softmax (Read more on arXiv or HuggingFace) |
Alham Fikri Aji, Erland Hilman Fuadi, Zayd M. K. Zuhri |
This paper introduces Softpick, a rectified, non-sum-to-one drop-in replacement for softmax in transformer attention mechanisms designed to eliminate attention sinks and massive activations. The main objective is to evaluate if Softpick can maintain performance parity with softmax while mitigating these issues and improving model characteristics like quantization robustness. The methodology involves defining the Softpick function, training 340M parameter Llama-style models from scratch using both Softpick and standard softmax, and comparing their performance on benchmarks, quantization tasks, attention sink rates, activation distributions, and attention map sparsity. Key results show Softpick achieves comparable benchmark performance to softmax, eliminates attention sinks (0% sink rate vs 63.41%), drastically reduces hidden state kurtosis (340 vs 33,510), produces sparser attention maps (46.97% sparsity), and significantly outperforms softmax under quantization, especially at low bit-precisions. For AI practitioners, Softpick presents a viable attention mechanism that intrinsically avoids problematic massive activations, thereby simplifying quantization efforts and potentially enhancing low-precision training and model sparsity. |
| Phi-4-reasoning Technical Report (Read more on arXiv or HuggingFace) |
Harkirat Behl, Vidhisha Balachandran, Ahmed Awadallah, Sahaj Agarwal, Marah Abdin |
This report introduces Phi-4-reasoning and Phi-4-reasoning-plus, 14-billion parameter models optimized for complex reasoning tasks via specialized training. The objective is to detail the models’ creation through data curation, supervised fine-tuning (SFT), and reinforcement learning (RL), and evaluate their reasoning performance. Key methodologies include SFT of Phi-4 on curated prompts with o3-mini-generated reasoning traces for Phi-4-reasoning, followed by outcome-based RL on math problems for Phi-4-reasoning-plus. Primary results show both models significantly outperform larger open-weight models like DeepSeek-R1-Distill-Llama-70B (e.g., Phi-4-reasoning-plus achieves 78.0% on AIME 25 vs. 51.5%) and approach state-of-the-art performance, demonstrating substantial gains over the base Phi-4. The principal implication for AI practitioners is that meticulous data curation for SFT combined with targeted RL can yield highly capable reasoning models at smaller scales, rivaling significantly larger architectures. |
| Beyond the Last Answer: Your Reasoning Trace Uncovers More than You |
|
|
| Think (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Hani Itani, Hasan Abed Al Kader Hammoud |
This research demonstrates that analyzing intermediate reasoning steps (“subthoughts”) in LLMs yields more reliable answers than relying solely on the final output. The study investigates whether an LLM’s final answer is its optimal conclusion and if alternative reasoning paths from intermediate points yield different results. Methodologically, it segments an initial reasoning trace, prompts the model to generate completions from each intermediate subthought endpoint, extracts the final numerical answer from each completion, and analyzes the resulting distribution of answers. The primary result shows that selecting the most frequent answer (mode) from these completions significantly boosts accuracy compared to the original final answer, with gains up to 13% on AIME2024, and that lower answer distribution entropy correlates with correctness. For AI practitioners, this implies that evaluating the distribution of answers derived from subthoughts, particularly using the mode, offers a more robust method for assessing LLM reasoning reliability than standard final-answer evaluation. |
| Taming the Titans: A Survey of Efficient LLM Inference Serving (Read more on arXiv or HuggingFace) |
Tong Liu, Zhenlin Yang, Yixin Ji, Juntao Li, zenRRan |
This paper surveys methods for optimizing Large Language Model (LLM) inference serving efficiency. The objective is to provide a comprehensive, hierarchical overview of techniques addressing the memory and computational challenges in LLM deployment. The methodology involves systematically reviewing and categorizing research into instance-level (e.g., model placement, KV cache management, request scheduling, PD disaggregation), cluster-level (e.g., deployment, load balancing, cloud), emerging scenarios (e.g., long context, RAG, MoE), and miscellaneous areas. The survey organizes numerous optimization strategies, highlighting advancements like PagedAttention for KV cache memory fragmentation and Prefill-Decoding (PD) disaggregation for optimizing distinct inference phases, though specific quantitative performance improvements across all methods aren’t aggregated due to the survey nature. For AI practitioners, this survey offers a structured map of optimization techniques, aiding in the selection and implementation of appropriate strategies to meet specific latency, throughput, and cost requirements for LLM serving. |
| COMPACT: COMPositional Atomic-to-Complex Visual Capability Tuning (Read more on arXiv or HuggingFace) |
Olga Russakovsky, Polina Kirichenko, Hee Seung Hwang, Xindi Wu |
This paper introduces COMPACT, a data-efficient visual instruction tuning recipe to improve compositional reasoning in Multimodal Large Language Models (MLLMs) by explicitly combining atomic visual skills. The main objective is to enhance MLLM performance on complex visual tasks requiring multiple capabilities without massive data scaling, addressing the lack compositional complexity in existing VIT datasets. COMPACT’s methodology involves defining 10 atomic capabilities, generating structured training data by prompting a generator model (Gemini-2.0-Flash) to create questions integrating exactly k={1, 2, 3} capabilities for sampled images, verifying quality, and mixing this data (32K samples) with 5% of LLaVA-665K VIT data. Primary results show COMPACT achieves performance comparable to the full LLaVA-665K baseline using less than 10% of the data, and significantly improves performance on complex tasks, achieving an 83.3% improvement on MMStar benchmark questions requiring four or more atomic capabilities. For AI practitioners, this implies that focusing on structured, compositionally complex training data generation offers a more data-efficient path to enhancing MLLM reasoning on complex multi-capability tasks compared to solely scaling undifferentiated instruction tuning data. |
| RoboVerse: Towards a Unified Platform, Dataset and Benchmark for |
|
|
| Scalable and Generalizable Robot Learning (Read more on arXiv or HuggingFace) |
Bangjun Wang, Yuyang Li, Songlin Wei, Feishi Wang, Haoran Geng |
RoboVerse introduces a unified simulation platform, large-scale synthetic dataset (>10M transitions, >1k tasks), and benchmark aimed at scalable and generalizable robot learning. The primary objective is to overcome robotics’ challenges in data scaling and standardized evaluation by unifying diverse simulation environments and datasets. Key methodology involves METASIM, an infrastructure that abstracts simulators (like Isaac Sim, MuJoCo) via a universal configuration system and API, enabling cross-simulator integration, hybrid simulation, large-scale data migration, and augmentation. Experiments demonstrate that RoboVerse improves imitation learning (e.g., ACT achieved 50.0% average success on representative tasks), reinforcement learning, and world models, facilitating direct sim-to-real transfer (e.g., OpenVLA achieved 7.0/10.0 on a real-world wash soap task after sim training). For AI practitioners, RoboVerse provides a standardized framework and a large, diverse synthetic dataset to train and evaluate robot learning models more efficiently, potentially accelerating development and improving sim-to-real generalization. |
| ReVision: High-Quality, Low-Cost Video Generation with Explicit 3D |
|
|
| Physics Modeling for Complex Motion and Interaction (Read more on arXiv or HuggingFace) |
Alan Yuille, Liang-Chieh Chen, Qihang Yu, Ju He, Qihao Liu |
ReVision enhances pre-trained video diffusion models by explicitly integrating parameterized 3D physical knowledge for high-quality, controllable generation of complex motion and interaction. The objective is to improve video generation quality, motion consistency, and control, particularly for complex actions and interactions, by leveraging 3D physical priors without needing massive models. The methodology involves a three-stage “Extract-Optimize-Reinforce” pipeline: generating a coarse video, extracting/optimizing 3D features using a Parameterized Physical Prior Model (PPPM), and regenerating the video conditioned on the refined 3D motion. Applied to Stable Video Diffusion (1.5B parameters), ReVision significantly improves motion coherence, outperforming a 13B parameter model in user preference studies and achieving an FVD of 130.14 on dance generation, adding only 5 seconds to inference time. For AI practitioners, this demonstrates that incorporating explicit physical knowledge can enhance smaller models to generate complex, controllable, physically plausible motions, offering an alternative to massive parameter scaling. |
| Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report (Read more on arXiv or HuggingFace) |
Anu Vellore, Blaine Nelson, Alexander Chen, Baturay Saglam, paulkass |
This paper introduces Foundation-Sec-8B, a Llama 3.1-8B model further pretrained on a curated 5.1 billion token cybersecurity corpus. The main objective was to enhance LLM performance on specialized cybersecurity tasks by addressing domain-specific data scarcity and knowledge representation challenges. Key methodology involved collecting and filtering web data for cybersecurity relevance, performing continued pretraining, and evaluating the model on benchmarks like CTIBench, CyberMetric, and SecBench using few-shot or zero-shot prompting. Foundation-Sec-8B demonstrated significant improvement over the base Llama 3.1-8B, notably achieving 14.29% higher accuracy (0.720±0.017) on the CTIBench-RCM task, matching or exceeding larger models like Llama 3.1-70B and GPT-4o-mini in specific CTI evaluations while showing minimal general knowledge degradation. The principal implication for AI practitioners is the availability of a publicly released, cybersecurity-specialized 8B parameter model that serves as a strong foundation for developing more capable AI-driven security tools. |
| UniBiomed: A Universal Foundation Model for Grounded Biomedical Image |
|
|
| Interpretation (Read more on arXiv or HuggingFace) |
Hao Chen, Jiaxin Zhuang, Sunan He, Yuxiang Nie, Linshan Wu |
UniBiomed is presented as the first universal foundation model integrating segmentation and text generation for grounded biomedical image interpretation across diverse modalities. The main objective is to address the inflexibility and lack of holistic information usage in conventional AI approaches by unifying the generation of clinical texts and the segmentation of corresponding biomedical objects. UniBiomed integrates a Multi-modal Large Language Model (MLLM, InternVL2.5) with a Segment Anything Model (SAM, SAM2), trained end-to-end on a novel dataset of 27 million image-annotation-text triplets spanning 10 biomedical imaging modalities. Extensive validation on 84 datasets shows UniBiomed achieves state-of-the-art performance, surpassing the previous best segmentation model (BiomedParse) by an average of 10.25% in Dice score across 60 segmentation datasets, and demonstrates strong capabilities in grounded disease recognition, VQA, and report generation. For AI practitioners, UniBiomed offers a single, versatile model capable of end-to-end grounded interpretation, potentially streamlining biomedical image analysis workflows by eliminating the need for separate segmentation/text models and manual prompt engineering. |
| Generative AI for Character Animation: A Comprehensive Survey of |
|
|
| Techniques, Applications, and Future Directions (Read more on arXiv or HuggingFace) |
Alireza Mirrokni, Hossein Behzadasl, Pardis Sadat Zahraei, Omid Ghahroodi, Mohammad Mahdi Abootorabi |
This survey comprehensively reviews generative AI techniques, applications, datasets, evaluation metrics, and future directions for character animation, integrating traditionally fragmented subfields. The main objective is to provide a unified perspective on state-of-the-art generative AI applications, including facial animation, expression rendering, avatar creation, gesture modeling, motion synthesis, object generation, and texture synthesis. Key methodologies surveyed include GANs, VAEs, Transformers, and Diffusion Models, alongside foundational frameworks like SMPL and evaluation metrics like FID and CLIP Score across numerous datasets (e.g., LAION, AMASS, FFHQ). The paper finds that foundation and diffusion models have significantly advanced realism and reduced production costs but highlights challenges in cross-domain generalization, real-time performance, controllability, and standardized evaluation. For AI practitioners, this survey offers a structured roadmap to the field, detailing key models, essential resources (datasets, benchmarks), evaluation techniques, and identifying open research problems crucial for developing advanced AI-driven animation systems. |
Papers for 2025-04-30
| Title |
Authors |
Summary |
| UniversalRAG: Retrieval-Augmented Generation over Multiple Corpora with |
|
|
| Diverse Modalities and Granularities (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Soyeong Jeong, jinheon, KangsanKim71, wgcyeo |
UniversalRAG introduces a retrieval-augmented generation framework handling diverse modalities and granularities by routing queries to modality-specific corpora. The main objective is to address the limitation of existing RAG systems, which typically handle only single modalities or suffer from modality gaps when unifying heterogeneous corpora. The key methodology involves a modality-aware router (either training-free using GPT-4o or trained using models like T5) that dynamically selects the optimal corpus (text, image, video) and granularity (e.g., paragraph, document, clip) for retrieval, avoiding direct cross-modal comparison in a unified space. Across 8 benchmarks using the InternVL-2.5 model, UniversalRAG with a T5-Large trained router achieved a 39.36 average score, significantly outperforming a unified corpus baseline (31.15), demonstrating the effectiveness of modality-specific routing. For AI practitioners, this implies that dynamically routing queries to specialized, modality-specific corpora, rather than relying on a single unified multimodal index, is crucial for building robust RAG systems that accurately leverage heterogeneous knowledge sources. |
| Reinforcement Learning for Reasoning in Large Language Models with One |
|
|
| Training Example (Read more on arXiv or HuggingFace) |
Baolin, renll, ZhiyuanZeng, hushqyang, ypwang61 |
This paper demonstrates that Reinforcement Learning with Verifiable Reward (RLVR) using just one training example (1-shot RLVR) can dramatically enhance LLM mathematical reasoning. The main objective was to investigate the extent to which the RLVR training dataset size can be reduced while maintaining performance comparable to using large datasets. The key methodology involved applying RL algorithms, primarily GRPO, to LLMs (e.g., Qwen2.5-Math-1.5B) using a single math problem-answer pair, duplicated to form the training batch. Primary results show that a carefully selected single example improved the Qwen2.5-Math-1.5B model’s performance on MATH500 from 36.0% to 73.6%, matching the performance obtained using a 1.2k example dataset. The principal implication for AI practitioners is that substantial reasoning improvements may be achievable with extreme data efficiency using RLVR, suggesting base models possess strong latent capabilities that can be activated by minimal, targeted RL signals, potentially reducing the need for extensive data curation. |
| ReasonIR: Training Retrievers for Reasoning Tasks (Read more on arXiv or HuggingFace) |
pangwei, sewon, Muennighoff, volpato30, rulins |
ReasonIR-8B is a novel bi-encoder retriever trained specifically for reasoning-intensive retrieval tasks using a synthetic data generation pipeline. The main objective is to improve retrieval quality for complex reasoning queries where existing fact-focused training data and retrievers show limited gains. The key methodology involves REASONIR-SYNTHESIZER, a pipeline generating challenging synthetic data including varied-length queries, reasoning-intensive hard queries, and corresponding plausible hard negatives from seed documents, used for contrastive training of a Llama3.1-8B model. REASONIR-8B achieved a new state-of-the-art 29.9 nDCG@10 on the BRIGHT benchmark without reranking, outperforming existing retrievers and improving downstream RAG task performance on MMLU by 6.4% over the closed-book baseline. For AI practitioners, this work provides a method (REASONIR-SYNTHESIZER) and a trained model (REASONIR-8B) to enhance RAG systems needing to retrieve diverse, contextually relevant information for complex reasoning, rather than just direct factual answers. |
| Toward Evaluative Thinking: Meta Policy Optimization with Evolving |
|
|
| Reward Models (Read more on arXiv or HuggingFace) |
Chanwoo Park, dykang, machineteacher, zaemyung |
Meta Policy Optimization (MPO) introduces a meta-reward model to dynamically refine evaluation rubrics during reinforcement learning alignment for LLMs. The main objective is to mitigate reward hacking and reduce the prompt engineering overhead associated with static reward models in RLAIF. The key methodology involves using a meta-reward model (MRM) to monitor the training context (policy outputs, reward scores, current rubric) and iteratively update the reward model’s (RM) evaluation prompt via meta-analysis, meta-refinement, and meta-merging steps during PPO training. Primary results demonstrate that MPO-aligned models consistently outperform static-prompt PPO baselines across tasks; for instance, on essay writing, the 32B RM + 32B MRM configuration achieved an Elo rating of 1196, significantly higher than the 966 Elo of the PPO baseline using the initial prompt. The principal implication for AI practitioners is that MPO provides a framework for automating the development of effective evaluation rubrics and enhancing alignment stability, reducing reliance on brittle, manually engineered prompts and mitigating risks of reward hacking. |
| TesserAct: Learning 4D Embodied World Models (Read more on arXiv or HuggingFace) |
Junyan Li, Hongxin Zhang, Qiao Sun, yilundu, anyeZHY |
TesserAct introduces a 4D embodied world model learned from RGB, Depth, and Normal (RGB-DN) videos to predict 3D scene evolution based on text instructions. The objective is to create a model generating temporally and spatially consistent 4D scene predictions for embodied agent actions, surpassing traditional 2D world models. Key methodology involves fine-tuning a video diffusion model on a curated RGB-DN dataset for joint prediction and using a novel algorithm with consistency/regularization losses to reconstruct coherent 4D point clouds from generated videos. TesserAct achieves superior 4D reconstruction quality, evidenced by a Chamfer L1 distance of 0.0811 on synthetic data (lower is better), and improves downstream robotic task success rates (e.g., 88% vs 81% for UniPi* on ‘close box’). For AI practitioners, this provides a framework for building geometrically accurate world models from multi-modal video, enabling enhanced simulation, planning, and policy learning for robotics compared to video-only approaches. |
| YoChameleon: Personalized Vision and Language Generation (Read more on arXiv or HuggingFace) |
Jing Shi, Krishna Kumar Singh, Thao Nguyen, Yuheng, yjlee0222 |
Yo’Chameleon personalizes Large Multimodal Models for joint vision and language generation using few-shot concept images. The primary objective is to enable LMMs to understand and generate content about specific user concepts provided via only 3-5 images, addressing the limitation of generic models. The methodology utilizes dual soft prompts (for understanding and generation) tuned via a self-prompting mechanism for task selection, combined with a “soft-positive” image training strategy using adaptive prompt lengths based on visual similarity. Results demonstrate superior performance over prompting baselines, achieving a recognition accuracy of 0.845 (vs. 0.727 baseline) and a CLIP image similarity of 0.783 (vs. 0.566 baseline) for generation using only 32 learnable tokens, while preserving general model capabilities. For AI practitioners, this provides a parameter-efficient method to infuse personalized knowledge into LMMs using minimal user data, enabling tailored multimodal generation and understanding without requiring full model retraining or suffering significant catastrophic forgetting. |
| The Leaderboard Illusion (Read more on arXiv or HuggingFace) |
sayashk, dsouzadaniel, AlexWang, olivernan, shivalikasingh |
This research reveals systematic distortions in Chatbot Arena rankings caused by undisclosed private testing, selective score reporting, and significant data access disparities favoring proprietary models. The objective was to investigate whether Chatbot Arena rankings accurately reflect generative AI model capabilities or are skewed by preferential evaluation policies and practices. Methodology involved auditing 243 models from 42 providers using ~2M battle records (Jan 2024 - Apr 2025), combining public data, scraped statistics, proprietary API data, simulations, and real-world private variant experiments. Primary results show undisclosed private testing benefits select providers (e.g., Meta tested 27 private variants pre-Llama 4 release), enabling selective score reporting; proprietary models receive disproportionately more data (Google/OpenAI ~20% each vs. 83 open-weight models combined ~30%); and access to Arena data provides substantial performance gains (up to 112% relative win-rate increase on ArenaHard). For AI practitioners, this implies that Chatbot Arena rankings may significantly reflect leaderboard-specific dynamics and overfitting due to unequal data access and private testing advantages, rather than solely general model competence, potentially misguiding model development and selection. |
| Certified Mitigation of Worst-Case LLM Copyright Infringement (Read more on arXiv or HuggingFace) |
Daniel Khashabi, Benjamin Van Durme, Jiacan Yu, mmarone, jackzhang |
This paper introduces BLOOMSCRUB, an inference-time method for certified mitigation of worst-case copyright infringement in LLMs by removing long verbatim quotes. The primary objective is to prevent LLMs from generating long verbatim excerpts (worst-case infringement) from copyrighted corpora while preserving text utility and factual knowledge. The key methodology involves iteratively applying a Bloom filter for efficient detection of verbatim quotes exceeding a length threshold (τ) and using an LLM to perform guided rewriting (“scrubbing”) of the detected segments, with an option to abstain if removal fails. Experimental results demonstrate BLOOMSCRUB effectively reduces worst-case infringement, achieving near-zero generation of quotes longer than 100 characters (%R > Q(100) ≈ 0.0%) compared to ~20% in vanilla models, while maintaining information quality and utility. For AI practitioners, BLOOMSCRUB offers a scalable, plug-and-play, inference-only technique to demonstrably reduce the risk of generating infringing long verbatim text, adaptable to different risk thresholds without requiring model retraining or access to logits. |
| ISDrama: Immersive Spatial Drama Generation through Multimodal Prompting (Read more on arXiv or HuggingFace) |
Tao Jin, Zhiyuan Zhu, Changhao Pan, Wenxiang Guo, AaronZ345 |
ISDrama introduces a unified framework for generating continuous, multi-speaker binaural speech with dramatic prosody from multimodal prompts for immersive spatial drama applications. The research objective is to simultaneously model complex spatial audio cues (position, orientation, movement, Doppler effect) and dramatic prosody from diverse inputs (scripts, audio, video, poses), overcoming data scarcity and integration challenges. Key methodology includes the creation of the MRSDrama dataset and the ISDrama model, featuring a contrastive learning-based Multimodal Pose Encoder for unified pose features and a flow-based Mamba-Transformer (Immersive Drama Transformer) with Drama-MOE and context-consistent CFG for generation. Primary results show ISDrama outperforms baselines, achieving superior prosodic expressiveness (MOS-E 4.01 ± 0.09 vs. 3.86 ± 0.06 for F5-TTS) and better spatial consistency (MOS-P 4.18 ± 0.10 for geometric input). The principal implication for AI practitioners is the availability of a one-stage solution for generating high-fidelity, spatially accurate, and dramatically expressive binaural speech directly from multimodal sources, avoiding error accumulation typical of cascaded approaches and enhancing immersive VR/AR experiences. |
| X-Fusion: Introducing New Modality to Frozen Large Language Models (Read more on arXiv or HuggingFace) |
Yijun Li, Siddharth Srinivasan Iyer, Xun Huang, Thao Nguyen, Sicheng Mo |
X-Fusion introduces a framework for adding vision capabilities to frozen Large Language Models (LLMs) while preserving their text abilities. The primary objective is to develop an efficient method for unified multimodal understanding and generation without retraining the LLM from scratch or degrading its inherent language skills. The key methodology involves a Dual Tower architecture where the original LLM (text tower) is frozen, and a parallel, trainable vision tower processes visual information, with selective feature integration across layers. Experimental results demonstrate superior performance over alternative architectures, with the Dual Tower design achieving a 23% lower (better) FID score (14.20 vs 19.10) on text-to-image generation compared to a single-tower fine-tuning approach using a 1B parameter model. For AI practitioners, this work presents a computationally efficient strategy to extend powerful LLMs to multimodal domains, suggesting that modality-specific towers can effectively integrate new capabilities without compromising the core model’s pretrained knowledge. |
| Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional |
|
|
| Talking Portrait Generation (Read more on arXiv or HuggingFace) |
Xiaobin Hu, FeiFan Xu, Chuming Lin, Weipeng Tan, ChengmingX |
DICE-Talk introduces a diffusion-based framework for generating emotionally expressive talking portraits while preserving identity by disentangling emotion/identity and leveraging inter-emotion correlations. The primary objective is to develop an emotional talking head generation model that overcomes limitations such as insufficient audio emotion utilization, identity leakage into emotion representations, and isolated emotion learning. Key methodologies include a cross-modal disentangled emotion embedder modeling emotions as identity-agnostic Gaussian distributions, a correlation-enhanced conditioning module using a learnable emotion bank, and an emotion discrimination objective during diffusion. DICE-Talk significantly outperforms prior methods in emotion accuracy, achieving a top Emo-Score of 0.4865 on MEAD and 0.5527 on an out-of-domain dataset, while maintaining competitive lip-sync and visual quality. AI practitioners can utilize this framework’s disentangled audio-visual emotion modeling and correlation-aware conditioning via emotion banks to create more controllable, realistic, and emotionally expressive digital avatars for applications requiring nuanced human interaction. |
| TreeHop: Generate and Filter Next Query Embeddings Efficiently for |
|
|
| Multi-hop Question Answering (Read more on arXiv or HuggingFace) |
Xuming Hu, Shuliang Liu, Jinghuai Ou, Zhonghao Li, kpzhang1028 |
TreeHop introduces an efficient, embedding-level iterative retrieval framework for multi-hop question answering that bypasses LLM-based query rewriting. The main objective is to significantly reduce computational overhead and latency in multi-hop RAG systems compared to methods requiring iterative LLM calls, while preserving retrieval effectiveness. Key methodology involves dynamically generating next-step query embeddings by fusing prior query and retrieved document embeddings using a gated cross-attention mechanism (UpdateGate) and rule-based pruning strategies. Primary results demonstrate comparable recall performance to advanced RAG systems on three MHQA benchmarks but with approximately 99% lower query latency (e.g., achieving 61.6% Recall@5 on 2WikiMultiHop iter2 with 0.022s latency versus 59.2% recall and 4.690s latency for Iter-RetGen). For AI practitioners, TreeHop presents a substantially faster and more cost-effective approach for implementing multi-hop RAG, replacing computationally expensive iterative LLM query refinements with lightweight embedding-space operations. |
Papers for 2025-04-29
| Title |
Authors |
Summary |
| RepText: Rendering Visual Text via Replicating (Read more on arXiv or HuggingFace) |
Yimeng Li, winhelp, SNOWAI, YujiaX, wanghaofan |
RepText introduces a method for rendering multilingual visual text in images by replicating glyphs rather than relying on deep text understanding. The main objective is to empower pre-trained monolingual text-to-image models with accurate, controllable multilingual text rendering capabilities without costly retraining or multilingual encoders. The key methodology involves a ControlNet-like architecture conditioned on language-agnostic glyph canny edges and position masks, enhanced by a text perceptual (OCR) loss, glyph latent replication during initialization, and regional masking at inference. Results show RepText outperforms existing open-source methods and achieves comparable performance to native multilingual closed-source models in rendering quality and accuracy, though specific quantitative metrics are not provided in the initial sections. For AI practitioners, this implies the ability to add precise, user-specified multilingual text to images using existing monolingual generative models through glyph replication, bypassing the need for models with inherent multilingual text understanding. |
| LLM-Powered GUI Agents in Phone Automation: Surveying Progress and |
|
|
| Prospects (Read more on arXiv or HuggingFace) |
Afeng-x, guoyaxuan0106, melpancake, Pengxiangzhao, guangyil |
This paper systematically reviews the evolution and capabilities of LLM-powered phone GUI agents, contrasting them with traditional methods. The main objective is to survey the progress, analyze core technologies, identify challenges, and outline future prospects for LLMs in phone automation. The methodology involves a comprehensive literature review, proposing a taxonomy of agent frameworks (single-agent, multi-agent, plan-then-act), modeling approaches (prompting, training-based), datasets, and benchmarks, alongside analyzing how LLMs address prior limitations. Primary results indicate LLMs significantly enhance phone automation through advanced language understanding, multimodal perception, and decision-making, overcoming traditional script limitations; for instance, specific RL approaches like DistRL show up to 20% relative improvement in success rate over state-of-the-art methods on general Android tasks. For AI practitioners, this survey provides a structured taxonomy and methodological framework, serving as a definitive reference for designing, developing, and evaluating scalable, adaptive LLM-powered phone GUI agents. |
| CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through |
|
|
| Cryptography Challenges (Read more on arXiv or HuggingFace) |
JiangWu, mingchenlin2025, LHL3341, blue01223, yu0226 |
This paper introduces CipherBank, a comprehensive benchmark evaluating LLM reasoning on cryptographic decryption tasks. The main objective is to rigorously assess the cryptographic reasoning capabilities of modern LLMs, identifying their strengths and weaknesses in this specific domain. Key methodology involves evaluating 18 state-of-the-art LLMs on the CipherBank dataset (comprising 2,358 problems across 5 domains, 14 subdomains, and 9 encryption algorithms) using a 3-shot known-plaintext attack evaluation protocol and accuracy metrics. Primary results reveal significant limitations across all models, even advanced ones; the top-performing model, Claude-3.5, achieved only 45.14% accuracy, demonstrating that current reasoning optimizations inadequately address cryptographic challenges. The principal implication for AI practitioners is that standard LLMs exhibit poor performance in precise cryptographic manipulation, indicating unsuitability for security-critical applications and highlighting the need for targeted development of symbolic reasoning and rule application capabilities beyond general language understanding for robust AI systems. |
| Clinical knowledge in LLMs does not translate to human interactions (Read more on arXiv or HuggingFace) |
cynddl, sahimo, Chronoszoldyck11, HannahRoseKirk, ambean |
Large language models (LLMs) demonstrate high clinical knowledge on benchmarks but fail to improve lay users’ medical assessment accuracy in realistic interactive scenarios compared to controls. The study investigated whether providing laypeople with access to high-performing LLMs (GPT-4o, Llama 3, Command R+) improves their ability to identify appropriate medical dispositions and relevant conditions in simulated health scenarios. A randomized controlled trial (N=1298) assigned participants to receive assistance from one of three LLMs or a control group (using typical resources) to assess ten medical vignettes against physician-defined gold standards. While LLMs alone identified relevant conditions in over 90% of cases, participants using LLMs identified relevant conditions in less than 34.5% of cases, significantly underperforming the control group (47.0%, p<0.001), and showed no significant improvement in disposition accuracy. For AI practitioners, this study critically demonstrates that strong performance on static or simulated benchmarks does not predict real-world interactive utility; robust human-user testing focused on interaction dynamics is essential before deploying LLMs for public health applications. |
| Group Downsampling with Equivariant Anti-aliasing (Read more on arXiv or HuggingFace) |
Raymond A. Yeh, ashiq24 |
This paper introduces a method for uniform downsampling of signals on finite groups with equivariant anti-aliasing, generalizing classical sampling theory concepts for group equivariant architectures. The objective is to define how to select an appropriate subgroup for a given downsampling rate and how to perform anti-aliasing to preserve equivariance. The methodology involves an algorithm for subgroup selection based on Cayley graphs, a Subgroup Sampling Theorem defining bandlimited-ness for perfect reconstruction, and an equivariant anti-aliasing operator derived via constrained optimization. Experiments demonstrate improved accuracy and equivariance (e.g., near-zero reconstruction error like 1.72e-13 for bandlimited signals on D28 downsampled to D14) and reduced model size when incorporated into G-CNNs for image classification. For AI practitioners, this provides a principled way to incorporate downsampling into group equivariant networks, enabling more computationally efficient models while better preserving theoretical equivariance guarantees compared to naive subsampling. |
| TrustGeoGen: Scalable and Formal-Verified Data Engine for Trustworthy |
|
|
| Multi-modal Geometric Problem Solving (Read more on arXiv or HuggingFace) |
BoZhang, friskit, Rethinker, zhoubb2010, renqiux0302 |
TrustGeoGen is a scalable engine generating formally verified multimodal geometric problems, solutions, and diagrams. The objective is to create a reliable benchmark and data generation pipeline for trustworthy geometric problem solving (GPS), addressing the lack of verified, multimodal data. Its methodology integrates multimodal-aligned generation, formal logical verification of reasoning steps, a bootstrapping mechanism for complexity scaling, and algorithms for generating diverse, traceable solutions. Primary results show the generated GeoTrust-test is challenging for SOTA models (OpenAI-o1 achieves 49.17% accuracy), and training on GeoTrust data improves OOD generalization on GeoQA compared to pseudo-labels. For AI practitioners, this provides a formally verified data source and engine (TrustGeoGen/GeoTrust) crucial for developing and evaluating more logically sound multimodal geometric reasoning systems, demonstrating superior training effectiveness over unverified data. |
| SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning (Read more on arXiv or HuggingFace) |
Xiaodan Liang, Peisong Wang, Bang Zhang, vvibt, judge |
This paper introduces Self-Play Critic (SPC), a method to automatically evolve a critic model for assessing LLM reasoning steps via adversarial self-play, removing the need for manual step-level annotation. The main objective is to improve the evaluation of step-by-step reliability in complex LLM reasoning like Chain-of-Thought. SPC employs two fine-tuned models, a “sneaky generator” creating subtle errors and a “critic” detecting them, which improve iteratively through reinforcement learning based on adversarial game outcomes. Experiments show SPC progressively enhances error detection capabilities (accuracy increasing from 70.8% to 77.7% on ProcessBench) and significantly improves LLM mathematical reasoning on MATH500 and AIME2024 when used to guide test-time search, outperforming existing process reward models. For AI practitioners, SPC provides a scalable technique to develop more accurate step-level critics for verifying and enhancing LLM reasoning processes without requiring expensive human-labeled data. |
| Benchmarking Multimodal Mathematical Reasoning with Explicit Visual |
|
|
| Dependency (Read more on arXiv or HuggingFace) |
Xin Li, Zhiqiang Hu, Wenqi Zhang, Jiashuo Sun, cloudcatcher2 |
VCBENCH is a new benchmark evaluating Large Vision-Language Models (LVLMs) on elementary math problems requiring reasoning across multiple images with explicit visual dependencies. The objective is to assess the core ability of LVLMs to discern, integrate, and reason using visual information from multiple images for basic mathematical tasks, moving beyond knowledge-centric evaluations. The methodology involved creating VCBENCH with 1,720 problems (averaging 3.9 images each, total 6,697 images) across six cognitive domains and evaluating 26 state-of-the-art LVLMs. Primary results show significant performance limitations, with even the best-performing models (e.g., Gemini2.0-Flash, Qwen-VL-Max) failing to exceed 50% accuracy, indicating particular weaknesses in pattern recognition and integrating visual cues across multiple images. For AI practitioners, this highlights a critical gap in current LVLMs’ fundamental visual-mathematical reasoning and multi-image integration capabilities, suggesting that model architectures and pre-training strategies need substantial improvement for tasks requiring grounded visual reasoning beyond single-image comprehension. |
| MMInference: Accelerating Pre-filling for Long-Context VLMs via |
|
|
| Modality-Aware Permutation Sparse Attention (Read more on arXiv or HuggingFace) |
Xufang Luo, Qianhui Wu, Chengruidong Zhang, Yucheng Li, iofu728 |
MMInference introduces a dynamic sparse attention method to accelerate the pre-filling stage for long-context Vision Language Models (VLMs). The primary objective is to mitigate the quadratic complexity bottleneck of attention during the processing of long multi-modal inputs, particularly the pre-fill stage which causes high Time-to-First-Token latency. The key methodology involves identifying unique modality-specific sparse patterns (like the Grid pattern for visual data) and modality boundaries, then applying modality-aware permutations and optimized GPU kernels for efficient sparse computation without model modification. Experiments demonstrate that MMInference accelerates the pre-filling stage by up to 8.3x for 1M tokens compared to dense attention, while maintaining comparable accuracy on various multi-modal benchmarks using models like LongVila and Llava-Video. For AI practitioners, this offers a drop-in method to significantly reduce inference latency for long-context VLMs, enabling faster deployment in real-world applications without requiring model fine-tuning. |
| ICL CIPHERS: Quantifying “Learning’’ in In-Context Learning via |
|
|
| Substitution Ciphers (Read more on arXiv or HuggingFace) |
Daniel Khashabi, Anqi Liu, Muhan Gao, Aayush Mishra, FocusV857 |
This paper introduces ICL CIPHERS, using substitution ciphers to quantify task learning (TL) in In-Context Learning (ICL) separately from task retrieval (TR). The main objective is to determine if Large Language Models (LLMs) can decipher and solve tasks when input tokens are systematically replaced using a reversible (bijective) mapping, thereby measuring inference-time learning. The methodology involves comparing LLM accuracy on tasks using inputs ciphered with bijective substitution mappings versus non-bijective (irreversible, random) mappings across four datasets and six models. Results consistently show LLMs perform better on bijective ciphers; Llama3.1 (8B) achieved an average of 7.1% higher accuracy on the bijective ciphered Amazon dataset compared to the non-bijective cipher across various demonstration counts. For AI practitioners, the observed accuracy gap between bijective and non-bijective ciphered tasks provides a quantitative proxy to assess an LLM’s ability to learn novel, reversible patterns during inference, beyond simple retrieval from pre-training data. |
| ChiseLLM: Unleashing the Power of Reasoning LLMs for Chisel Agile |
|
|
| Hardware Development (Read more on arXiv or HuggingFace) |
Shanshan Li, Renzhi Chen, Jiaran Gao, xiuranli, observerw |
This paper introduces ChiseLLM, a series of datasets and fine-tuned reasoning models to enhance Large Language Model (LLM) performance for Chisel hardware construction language generation. The research objective is to address the poor syntax correctness and limited design variability exhibited by existing LLMs when generating Chisel code for Agile Hardware Development Methodology (AHDM). Methodologically, the authors curated high-quality datasets from public RTL sources, synthesized prompt-guided reasoning traces, and performed domain-adapted fine-tuning on Qwen2.5-Coder base models. Results show significant improvements; notably, the ChiseLLM-32B model increased average Pass@5 syntax correctness by 26.32% over the base model and boosted design variability capability by 47.58% compared to baseline reasoning models. The principal implication for AI practitioners is that domain adaptation combined with synthesized reasoning traces is crucial for effectively leveraging reasoning LLMs in specialized, low-resource code generation tasks like HCL, enabling practical application in hardware design. |
Papers for 2025-04-28
| Title |
Authors |
Summary |
| Towards Understanding Camera Motions in Any Video (Read more on arXiv or HuggingFace) |
Jay Karhade, Daniel Jiang, Stephen624, syCen, zhiqiulin |
Introduces CameraBench, a large-scale dataset and benchmark for understanding camera motion primitives in diverse internet videos. The main objective is to evaluate how well current Structure-from-Motion (SfM) and Video-Language Models (VLMs) understand a comprehensive taxonomy of camera motions and to improve this capability. Methodology involves creating a detailed taxonomy with cinematographers, collecting and annotating ~3,000 videos, conducting human studies, and benchmarking 20 diverse models (SfM/SLAM and VLMs) on tasks like classification, VQA, captioning, and retrieval. Primary results show classic SfM struggles with semantic/dynamic content, while VLMs struggle with precise geometry; the best baseline method (MegaSAM) achieves ~50% overall Average Precision (AP) on primitive classification, while fine-tuning a generative VLM (Qwen2.5-VL-7B) boosts performance significantly (~2x improvement), reaching 59.3% AP. AI practitioners can utilize CameraBench’s dataset and taxonomy to fine-tune VLMs, substantially improving their ability to interpret both geometric and semantic camera movements for enhanced video understanding applications. |
| Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning (Read more on arXiv or HuggingFace) |
Xiaokun Wang, Yi Peng, Yichen Wei, Chris, xuchensong |
Skywork R1V2 is a multimodal reasoning model developed using a hybrid reinforcement learning strategy that eliminates the need for supervised fine-tuning or teacher model distillation. The primary objective is to enhance sophisticated reasoning capabilities in vision-language models (VLMs) while preserving broad generalization, directly addressing the challenge of balancing these competing demands via reinforcement learning. Methodologically, R1V2 combines Mixed Preference Optimization (MPO) with reward-model guidance and Group Relative Policy Optimization (GRPO), augmented by a Selective Sample Buffer (SSB) to counteract vanishing advantages and prioritize informative training samples. Key results demonstrate state-of-the-art open-source performance, achieving 78.9% on AIME2024 and 62.6% on OlympiadBench, significantly improving upon prior open-source models and reducing the gap with proprietary systems. For AI practitioners, this work presents a validated hybrid RL framework (MPO+GRPO+SSB) as an effective technique for building capable multimodal reasoning models without reliance on SFT, while highlighting the necessity of careful reward calibration to manage the trade-off between enhanced reasoning and potential visual hallucination. |
| BitNet v2: Native 4-bit Activations with Hadamard Transformation for |
|
|
| 1-bit LLMs (Read more on arXiv or HuggingFace) |
Furu Wei, Shuming Ma, Hongyu Wang |
BitNet v2 introduces a framework for 1-bit Large Language Models (LLMs) enabling native 4-bit activation quantization. The main objective is to overcome activation outliers that complicate low bit-width quantization in attention and feed-forward networks. Key methodology involves H-BitLinear, which applies an online Hadamard transformation to reshape activation distributions into more Gaussian-like forms prior to quantization, specifically targeting attention output (Wo) and FFN down-projection (Wdown). Experiments show BitNet v2 trained with native 4-bit activations achieves performance comparable to prior versions using 8-bit or hybrid activations; for instance, the 7B model BitNet v2 (a4) achieves an average downstream task accuracy of 58.30, close to BitNet b1.58’s 58.12. For AI practitioners, this research offers a viable path towards deploying 1-bit LLMs with efficient native 4-bit activations, reducing memory and computational costs, particularly beneficial for hardware supporting low-bit computations and batched inference scenarios. |
| VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures, |
|
|
| Languages, and Domains in Video Comprehension (Read more on arXiv or HuggingFace) |
Wenhan Luo, Baotian Hu, Haoyuan Shi, Yunxin Li, Xinyu Chen |
VideoVista-CulturalLingo introduces the first benchmark for evaluating video comprehension across diverse cultures (Chinese, Western), languages (Chinese, English), and domains. The primary objective is to assess the understanding and reasoning capabilities of large multimodal models (LMMs) beyond existing culturally and linguistically limited benchmarks. A dataset comprising 1,389 videos and 3,134 QA pairs was constructed using a hybrid annotation framework leveraging LLMs (Qwen2-VL, DeepSeek) and human review, followed by the evaluation of 24 LMMs. Experimental results indicate proprietary models like Gemini-2.0-Flash achieve the highest accuracy (76.3%), while open-source models show limitations, particularly on Chinese-centric questions and temporal understanding tasks like Event Localization (achieving only 45.2% maximum score). For AI practitioners, this benchmark provides a crucial tool for identifying weaknesses in LMMs’ cross-cultural generalization and fine-grained temporal reasoning, highlighting areas needing improvement for developing more globally competent video understanding systems. |
| Can Large Language Models Help Multimodal Language Analysis? MMLA: A |
|
|
| Comprehensive Benchmark (Read more on arXiv or HuggingFace) |
Peiwu Wang, Hua Xu, Yeshuang Zhu, Zhuohang Li, HanleiZhang |
This paper introduces MMLA, a benchmark for evaluating large language models (LLMs) and multimodal large language models (MLLMs) on multimodal language analysis. The objective is to assess the capability of these foundation models to comprehend high-level, cognitive semantics (intent, emotion, dialogue act, sentiment, speaking style, communication behavior) in human utterances. Methodology involves evaluating eight model families across nine datasets (61K+ utterances) using zero-shot inference, supervised fine-tuning (SFT), and instruction tuning (IT). Results demonstrate that MLLMs significantly improve with SFT but still face challenges, achieving only about 60-70% average accuracy (best SFT model reaches 69.18%), indicating limitations in understanding complex human language nuances. For AI practitioners, this highlights that while fine-tuning substantially boosts performance and enables smaller models to rival larger ones, current MLLMs require further development to reliably decode complex multimodal semantics for applications like virtual assistants or social behavior analysis. |
| The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (Read more on arXiv or HuggingFace) |
Kelly Marchisio, Sebastian Ruder, Renjie Huang, Robert Li, Piotr Nawrot |
This paper presents a large-scale empirical comparison of training-free sparse attention methods in Transformer LLMs across various model sizes, sequence lengths, and sparsity levels. The main objective was to systematically evaluate the viability, efficiency-accuracy trade-offs, and scaling properties of sparse attention for long-context processing. Researchers compared six representative sparse attention patterns on nine diverse long-sequence tasks using Qwen 2.5 models (7B-72B) up to 128k sequence length, employing isoFLOPS analysis, statistical tests, and scaling law fitting. Key findings include: 1) isoFLOPS analysis shows large, highly sparse models are preferable to smaller, dense ones for very long sequences; 2) maximum tolerable sparsity without performance degradation varies significantly, often exceeding 10x on average but dropping below 5x for at least one task in most configurations; 3) no single method excels universally, with optimal choices being task- and phase-dependent. For AI practitioners, this implies sparse attention is a valuable tool for scaling to longer sequences but is not a universally applicable solution and demands careful, application-specific evaluation of performance trade-offs, as even moderate sparsity can cause significant degradation on sensitive tasks. |
| Subject-driven Video Generation via Disentangled Identity and Motion (Read more on arXiv or HuggingFace) |
Wonjoon Jin, Jingxu Zhang, cluo-ms, daiqi, carpedkm |
This paper presents a zero-shot method for subject-driven video generation by factorizing identity injection and temporal modeling using image customization and unpaired video datasets. The primary objective is to achieve high-fidelity subject consistency and temporal coherence without relying on large-scale annotated subject-video (S2V) datasets. Key methodologies include fine-tuning a pre-trained Multi-Modal Diffusion Transformer (MM-DiT) model via stochastically-switched optimization between identity learning (using an S2I dataset and LoRA) and temporal preservation (using unpaired videos and I2V fine-tuning), incorporating random frame selection and token dropping. The approach significantly outperforms baselines in zero-shot settings, achieving superior identity consistency (DINO-I: 59.29) and dynamic degree (60.19) on the VBench benchmark. For AI practitioners, this demonstrates a viable path to build scalable personalized video generation models using readily available image customization data, bypassing the significant cost and complexity of acquiring large annotated video datasets. |
| DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Lifan Guo, Junhui Li, Huaixia Dou, Qian Chen, amazingj |
The paper introduces DianJin-R1, an LLM framework enhancing financial reasoning via structured supervision and reinforcement learning using a curated dataset. The primary objective is to improve LLM performance on complex financial tasks requiring domain-specific knowledge, numerical calculation, and compliance adherence. Methodology involves fine-tuning Qwen2.5 models on the DianJin-R1-Data (derived from CFLUE, FinQA, CCC) to generate structured reasoning () and answers (), further refined using Group Relative Policy Optimization (GRPO) with format and accuracy rewards. Key results show DianJin-R1 models significantly outperform non-reasoning counterparts; DianJin-R1-32B achieved 96.00% accuracy on the proprietary CCC compliance benchmark with a single API call, exceeding a multi-agent baseline requiring 8.15 calls. For AI practitioners, this demonstrates that combining structured reasoning supervision with targeted reinforcement learning provides a scalable and computationally efficient approach to building specialized LLMs for complex, domain-specific reasoning tasks like financial compliance. |
| DC-SAM: In-Context Segment Anything in Images and Videos via Dual |
|
|
| Consistency (Read more on arXiv or HuggingFace) |
Lu Qi, Xiaoyang Bi, Xiangtai Li, Mengshi Qi, zaplm |
DC-SAM adapts SAM/SAM2 for in-context image/video segmentation via prompt tuning and dual consistency. The objective is to enhance SAM’s in-context segmentation ability by generating higher-quality visual prompts through better feature utilization and consistency, and to establish a benchmark for in-context video segmentation. The key methodology involves fusing SAM encoder features with backbone features, employing dual positive/negative prompt generation branches refined by cyclic consistent cross-attention, and using a mask-tube training strategy for video extension with SAM2. Primary results demonstrate state-of-the-art performance, achieving 55.5 mIoU (+1.4 improvement over VRP-SAM baseline) on COCO-20^2 and a J&F score of 71.52 on the proposed In-Context Video Object Segmentation (IC-VOS) benchmark. For AI practitioners, DC-SAM offers an efficient parameter-tuning approach to adapt foundation models like SAM/SAM2 for few-shot (specifically one-shot) segmentation tasks in both images and videos, improving prompt quality through explicit feature fusion and consistency enforcement. |
| Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing |
|
|
| Efficiency Through Vocabulary Adaptation (Read more on arXiv or HuggingFace) |
Edoardo Barba, Andrei Stefan Bejgu, Pere-Lluis Huguet Cabot, Giovanni Puccetti, Luca Moroni |
This paper introduces and evaluates vocabulary adaptation techniques, including the novel Semantic Alignment Vocabulary Adaptation (SAVA), for optimizing English Large Language Models (LLMs) for Italian. The primary objective is to compare different vocabulary adaptation methods against Language-Adaptive Pre-Training (LAPT) for adapting English LLMs (Mistral-7B-v0.1, Llama-3.1-8B) to Italian, focusing on token fertility reduction and downstream task performance. Methodology involves replacing the original tokenizer and embeddings using heuristics like Fast Vocabulary Transfer (FVT), Cross-Lingual Projection (CLP), the proposed SAVA (using neural mapping from a helper model, Minerva-3B), and random initialization, followed by continual training on Italian/English data. Results show that adapting Mistral-7B-v0.1 with the Minerva tokenizer reduced Italian token fertility by 25%, and adapting Llama-3.1-8B reduced its parameters by 1 billion (10% size reduction) due to vocabulary optimization; SAVA and FVT demonstrated competitive performance on downstream tasks, often converging faster during continual training than other methods. For AI practitioners, this indicates that vocabulary adaptation techniques like SAVA or FVT can significantly improve the efficiency (lower fertility, potentially smaller models) of deploying English-centric LLMs for other languages like Italian with relatively limited continual training. |
Papers for 2025-04-25
| Title |
Authors |
Summary |
| Step1X-Edit: A Practical Framework for General Image Editing (Read more on arXiv or HuggingFace) |
Peng Xing, Yucheng Han, Shiyu Liu, skicy, wchengad |
Step1X-Edit introduces an open-source framework for general, instruction-based image editing designed to rival proprietary models. The research aims to develop and evaluate a unified image editing model that effectively understands natural language instructions and performs diverse, high-fidelity edits, bridging the gap with closed-source systems. The methodology combines a Multimodal Large Language Model (MLLM) for instruction and image understanding with a DiT-style diffusion decoder for image generation, trained on a custom-generated dataset of over 1 million high-quality editing triplets across 11 categories. Evaluated on the introduced GEdit-Bench benchmark using GPT-4.1, Step1X-Edit achieves a G_O score of 6.701 (Full set, English instructions), significantly outperforming open-source baselines like OmniGen (5.061) and approaching proprietary models like Gemini2 Flash (6.315) and GPT-40 (7.534). For AI practitioners, this work provides a high-performing, open-source alternative for instruction-based image editing, along with a large-scale dataset and benchmark, facilitating the development and evaluation of general-purpose image manipulation capabilities. |
| RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Michal Sokolik, Brian Gordon, Yonatan Bitton, Hagai Taitelbaum, lovodkin93 |
REFVNLI introduces a cost-effective, fine-tuned VLM metric for evaluating both subject preservation and textual alignment in subject-driven text-to-image generation. The objective is to develop a single, automatic metric that reliably assesses both subject identity fidelity and prompt adherence in subject-driven T2I outputs, overcoming the limitations of costly or single-aspect existing evaluators. The methodology involves fine-tuning a PaliGemma vision-language model on a large-scale dataset (1.2M triplets) automatically curated from video reasoning benchmarks and image perturbations, performing sequential binary classifications for textual alignment then subject preservation. Primary results show REFVNLI matches or outperforms baselines across multiple benchmarks, achieving up to 8.5-point ROC AUC gains in subject consistency (ImagenHub, Multi-subject) and aligning with human preferences on rare entities over 87% of the time. For AI practitioners, REFVNLI provides a scalable, more reliable automated evaluation method for subject-driven T2I models, enabling faster iteration without dependence on expensive API calls or separate metrics that may not capture both essential generation qualities. |
| Paper2Code: Automating Code Generation from Scientific Papers in Machine |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Seongyun Lee, jinheon, iaminju |
PaperCoder is a multi-agent LLM framework designed to automatically generate functional code repositories from machine learning research papers. The objective is to improve research reproducibility by automating the conversion of scientific papers into executable code, addressing the common issue of unavailable implementations. The methodology involves a three-stage pipeline using specialized LLM agents: Planning (roadmap, architecture design with diagrams, dependency/configuration generation), Analysis (interpreting file-specific implementation details), and Coding (generating modular, dependency-aware code sequentially). Evaluation on benchmarks like PaperBench showed PaperCoder achieved a 44.26% replication score, significantly outperforming strong baselines (e.g., 16.4% for Iterative Agent), and generated code required minimal manual modification (averaging 0.48% of lines changed) for execution in case studies. For AI practitioners, PaperCoder demonstrates a viable approach to automatically generate high-fidelity, nearly executable code directly from research papers, substantially reducing the effort required to reproduce results and build upon existing work when official code is missing. |
| Breaking the Modality Barrier: Universal Embedding Learning with |
|
|
| Multimodal LLMs (Read more on arXiv or HuggingFace) |
Yanzhao Zhang, Xingjun Wang, Ziyong Feng, Tiancheng Gu, Kaichengalex |
This paper introduces UniME, a two-stage framework using Multimodal Large Language Models (MLLMs) for universal multimodal embedding learning. The primary objective is to overcome limitations of CLIP and standard MLLMs to learn discriminative and transferable representations for diverse downstream vision-language tasks. UniME utilizes a two-stage methodology: (1) textual discriminative knowledge distillation from an LLM teacher model to enhance the MLLM’s language component, and (2) hard negative enhanced instruction tuning with false negative filtering and hard negative sampling. Experimental results demonstrate superior performance, with UniME based on LLaVA-1.6 achieving a 66.6% overall score on the MMEB benchmark, a 3.3% improvement over the VLM2Vec baseline. For AI practitioners, this framework offers a method to adapt MLLMs for generating highly discriminative universal embeddings applicable across tasks like retrieval, VQA, classification, and grounding, enhancing multimodal understanding capabilities. |
| Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery |
|
|
| Simulation (Read more on arXiv or HuggingFace) |
Leonidas Guibas, Mikaela Angelina Uy, Chanho Park, Jihyeon Je, Phillip Y. Lee |
This paper introduces Abstract Perspective Change (APC), a framework enabling vision-language models (VLMs) to perform spatial reasoning from arbitrary viewpoints by simulating mental imagery. The main objective is to overcome the inherent egocentric bias in VLMs and equip them with robust allocentric reasoning capabilities necessary for understanding scenes from perspectives other than the camera’s. APC utilizes vision foundation models (object detection, segmentation, orientation estimation) to build a coarse 3D scene abstraction, transforms this abstraction into the reference viewer’s egocentric coordinate frame via coordinate transformation, and then prompts the VLM with this transformed representation (either numerically or visually). Experiments show APC significantly outperforms existing VLMs and reconstruction-based approaches, achieving 72.78% accuracy on the challenging 3DSRBench left/right spatial reasoning task using its visual prompt variant (APC-Vis), compared to significantly lower scores for baselines on real images. For AI practitioners, APC provides a concrete methodology to enhance VLM spatial intelligence for tasks requiring perspective shifts (like robotics or embodied AI) by effectively converting allocentric problems into egocentric ones that VLMs can readily solve. |
| QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM |
|
|
| Pretraining (Read more on arXiv or HuggingFace) |
Yifan Zhang, Zhimiao Yu, Binbin Liu, Weidong Zhou, Fengze Liu |
This paper introduces QuaDMix, a unified framework to automatically optimize data selection for large language model (LLM) pretraining by jointly balancing data quality and diversity. The primary objective is to address the challenge of the inherent trade-off between quality and diversity, which prior methods often optimize separately. QuaDMix employs multiple quality scoring criteria and domain classification, feeding these features into a unified parameterized sampling function; optimal parameters are found efficiently using simulated experiments on small proxy models and a LightGBM regression model to predict performance. Experiments demonstrate QuaDMix achieves an average performance improvement of 7.2% across multiple benchmarks compared to methods optimizing quality or diversity independently, confirming the necessity of joint optimization. For AI practitioners, QuaDMix offers an automated approach to curate more effective pretraining datasets by systematically balancing quality and diversity, potentially improving LLM efficiency and downstream performance. |
| Token-Shuffle: Towards High-Resolution Image Generation with |
|
|
| Autoregressive Models (Read more on arXiv or HuggingFace) |
Chih-Yao Ma, Hao Tang, Haoyu Ma, Peize Sun, Xu Ma |
Token-Shuffle enables efficient high-resolution (up to 2048x2048) image generation with autoregressive models by reducing the computational load of visual tokens. The primary objective is to overcome the computational and resolution constraints imposed by the large number of image tokens required by standard autoregressive Multimodal Large Language Models (MLLMs) for image synthesis. Key methodology involves ‘token-shuffle’ to merge spatially local tokens via MLPs before Transformer blocks and ‘token-unshuffle’ to restore spatial arrangement after, exploiting visual vocabulary dimensional redundancy within a VQGAN-tokenized Llama framework. A 2.7B parameter Token-Shuffle model achieved a 0.77 overall score on the GenAI-benchmark hard prompts, outperforming prior AR models like LlamaGen by 0.18. This provides AI practitioners a plug-and-play method to build more efficient and higher-resolution autoregressive image generation MLLMs using existing architectures with minimal modification and without needing additional text encoders. |
| Distilling semantically aware orders for autoregressive image generation (Read more on arXiv or HuggingFace) |
David Vazquez, Masih Aminbeidokhti, Juan A. Rodriguez, Antoine Poupon, rishavpramanik |
This paper introduces an autoregressive image generation method that learns a semantically aware order for generating image patches, improving quality over traditional raster-scan approaches. The main objective is to determine if learning an optimal, content-dependent patch generation order can enhance the quality of autoregressive image generation models without requiring additional annotations. The key methodology involves first training an any-given-order autoregressive model, using it to infer optimal generation orders for the training data via likelihood maximization at each step, and then fine-tuning the model using these self-supervised, distilled orders combined with relative positional encoding. On the Fashion Product dataset, the proposed fine-tuned ordered generation method achieved a Fréchet Inception Distance (FID) of 2.56, significantly improving upon the 4.58 FID of the baseline raster-scan model. For AI practitioners, this work demonstrates that the generation order in patch-based autoregressive models is a crucial factor impacting image quality, and learning this order dynamically based on content offers a viable path to enhance generation performance using self-supervision. |
| DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs (Read more on arXiv or HuggingFace) |
Heng Ji, Silvio Savarese, Caiming Xiong, Senthil Purushwalkam, Zhenhailong Wang |
The paper introduces DyMU, a training-free framework to enhance the efficiency of Vision-Language Models (VLMs) by dynamically adjusting visual token counts. The primary objective is to reduce the computational burden of VLMs, stemming from fixed-length visual token sequences, without sacrificing task performance or requiring model retraining. DyMU employs Dynamic Token Merging (DToMe) to merge redundant visual tokens based on image complexity and Virtual Token Unmerging (VTU) to efficiently reconstruct attention dynamics for the Large Language Model (LLM) using Rotary Position Embeddings (RoPE). Experiments demonstrate DyMU can reduce average visual token counts by 32%-85%; specifically, DyMU-low on LLaVA-1.5 achieved 97.7% of the baseline performance using only ~15% of the original visual tokens. For AI practitioners, DyMU offers a plug-and-play method to significantly decrease the inference costs and computational requirements of existing VLMs without fine-tuning, enabling more efficient deployment. |
| IberBench: LLM Evaluation on Iberian Languages (Read more on arXiv or HuggingFace) |
Areg Mikael Sarvazyan, Álvaro Romo Herrero, Ian Borrego Obrador, José Ángel González, mchinea |
IberBench introduces a comprehensive, extensible benchmark for evaluating LLM performance across Iberian languages, including Spanish varieties, focusing on both fundamental and industry-relevant tasks. The main objective is to assess LLM capabilities beyond English, addressing the limited coverage of Iberian languages, linguistic varieties, and task types in existing static benchmarks. The methodology involves compiling 101 datasets covering 22 task categories, standardizing them, and evaluating 23 LLMs (100M to 14B parameters) using a custom, incremental evaluation pipeline built upon lm-evaluation-harness, primarily in a zero-shot setting. Results indicate that LLMs generally perform worse on industry-relevant tasks than fundamental ones, exhibit lower average performance in Galician and Basque, and the top-performing model (Qwen-2.5-7b-Instruct) achieved a mean score of 46.8% across tasks and languages. For AI practitioners, IberBench provides a standardized tool and public leaderboard to compare LLM suitability for specific Iberian language applications, highlighting current model limitations in areas like industry relevance and performance on less-resourced languages like Basque and Galician, guiding model selection and development focus. |
| Boosting Generative Image Modeling via Joint Image-Feature Synthesis (Read more on arXiv or HuggingFace) |
Nikos Komodakis, Spyros Gidaris, Ioannis Kakogeorgiou, Efstathios Karypidis, Theodoros Kouzelis |
ReDi enhances generative image modeling by jointly synthesizing low-level VAE image latents and high-level DINOv2 semantic features within a unified diffusion framework. The primary objective is to integrate representation learning directly into the generative process to improve synthesis quality and training convergence speed, bypassing explicit distillation steps. The core methodology involves modifying standard Diffusion Transformer architectures (DiT/SiT) to operate on a combined input of noise-corrupted VAE latents and PCA-reduced DINOv2 features, learning the joint distribution via a shared denoising objective, and introducing Representation Guidance for inference refinement. Key results demonstrate substantial improvements, such as accelerating SiT-XL/2 convergence by >6x compared to prior representation alignment methods (REPA) and achieving a state-of-the-art FID of 1.64 on ImageNet 256x256 (CFG). For AI practitioners, ReDi offers a method to train high-fidelity generative models significantly faster and introduces Representation Guidance as a novel technique to steer image generation using learned semantic features. |
| ViSMaP: Unsupervised Hour-long Video Summarisation by Meta-Prompting (Read more on arXiv or HuggingFace) |
Mariano Beguerisse-Diaz, Shaogang Gong, Dimitrios Korkinof, Jian Hu |
ViSMaP presents an unsupervised method for summarizing hour-long videos by adapting models trained on short videos using meta-prompting. The objective is to generate coherent summaries for long, unannotated videos, bridging the semantic gap between short segment descriptions and holistic long video narratives without costly long-form annotations. Key methodology involves pre-training a model on short videos, using iterative meta-prompting with multiple LLMs (generator, evaluator, optimizer) to create pseudo-summaries for long videos, and fine-tuning the model using these pseudo-summaries with a noisy label loss. Primary results demonstrate performance comparable to supervised methods, achieving a 26.0 CIDEr score on the Ego4D-HCap dataset versus 29.3 for the fully supervised VideoReCap, despite requiring no human annotations for the long videos. This provides AI practitioners a framework to summarize large volumes of unlabelled long videos by leveraging existing short-video datasets and LLMs, drastically reducing the need for expensive manual annotation. |
| 3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models (Read more on arXiv or HuggingFace) |
Fan Wang, Jingkai Zhou, Chaohui Yu, Min Wei |
3DV-TON introduces a diffusion-based video try-on framework employing textured 3D guidance to enhance temporal consistency and fidelity. The primary objective is to generate high-quality, temporally coherent try-on videos by mitigating the appearance bias inherent in pixel-reconstruction objectives which often leads to motion artifacts. Its methodology utilizes adaptively generated, animatable textured 3D meshes as explicit frame-level guidance for a diffusion model, complemented by a rectangular masking strategy to prevent information leakage. Quantitatively, on the ViViD dataset, 3DV-TON* achieved a paired VFID_I3D score of 10.9680, surpassing the baseline ViViD method’s score of 17.2924 (lower indicates better generation quality and temporal consistency). For AI practitioners, the key implication is that leveraging explicit, textured 3D spatio-temporal guidance within diffusion models can significantly improve motion coherence and detail preservation in complex video generation tasks like virtual try-on. |
| TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming |
|
|
| Videos (Read more on arXiv or HuggingFace) |
Shuhuai Ren, Lei Li, Yuancheng Wei, Yicheng Li, Linli Yao |
TimeChat-Online introduces the Differential Token Drop (DTD) module to achieve efficient streaming video understanding by exploiting visual redundancy. The primary objective is to enable real-time, interactive VideoLLMs capable of handling long, redundant streaming videos and performing proactive responses. The core methodology involves the DTD module, which adaptively drops visually similar tokens between consecutive frames based on pixel or feature-level similarity, preserving only significant temporal changes and associated spatial-temporal positions. Experiments show DTD removes 82.8% of video tokens on StreamingBench while retaining 98% of the original accuracy and achieving a 1.76x latency speedup. For AI practitioners, this demonstrates that leveraging natural video redundancy through query-agnostic token dropping can drastically reduce computational costs and improve efficiency for deploying VideoLLMs in real-time streaming applications, especially for long-duration videos. |
| Interpretable non-linear dimensionality reduction using gaussian |
|
|
| weighted linear transformation (Read more on arXiv or HuggingFace) |
erikbergh |
This paper introduces an interpretable non-linear dimensionality reduction method using Gaussian-weighted linear transformations. The objective is to combine the representational power of non-linear techniques with the interpretability of linear methods. The methodology involves constructing a non-linear mapping as a weighted sum of multiple linear transformations, where weights are derived from normalized Gaussian functions centered throughout the data space, optimized to preserve pairwise distances. Applied to a 3D S-curve dataset reduced to 2D, the algorithm achieved a reconstruction error of 0.45 and allowed for quantifying dimension influence (e.g., y-dimension contributed 0.25 overall influence) and visualizing spatial variations in transformation properties like influence skewness and space contraction. For AI practitioners, this offers a dimensionality reduction tool that provides non-linear expressiveness while enabling analysis of how original dimensions contribute to the reduced space and how geometric properties change locally. |
| Process Reward Models That Think (Read more on arXiv or HuggingFace) |
Hao Peng, Jaekyeom Kim, Lajanugen Logeswaran, Rishabh Agarwal, Muhammad Khalifa |
This paper introduces THINKPRM, a data-efficient generative process reward model (PRM) that verifies reasoning steps using long Chain-of-Thought (CoT). The objective is to create PRMs that are both high-performing and require significantly less supervision data compared to traditional discriminative PRMs. The methodology involves lightweight fine-tuning of large reasoning models on a small dataset of filtered synthetic verification CoTs (using as few as 8K process labels). Key results show THINKPRM outperforms discriminative PRMs trained on ~100x more data and LLM-as-a-judge baselines, achieving an 8% improvement over a discriminative PRM on a GPQA-Diamond subset despite using far less training data. For AI practitioners, this demonstrates the potential to build powerful and scalable reasoning verifiers with minimal supervision by leveraging generative models and CoT verification, reducing reliance on large labeled process datasets. |
Papers for 2025-04-24
| Title |
Authors |
Summary |
| VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Einsiedler, luotto, Weiyun1025, GenuineWWD, wilye |
This paper introduces VisuLogic, a benchmark designed to evaluate genuine vision-centric reasoning in multi-modal large language models (MLLMs). The research aims to address the limitation that current MLLM evaluations often rely on textual descriptions, allowing language-based shortcuts instead of measuring true visual reasoning. The methodology involves a new benchmark of 1,000 human-verified visual problems across six categories (quantitative, spatial, positional, attribute, stylistic, other), designed to be difficult to solve via text description alone, on which leading MLLMs and humans were evaluated. Primary results show a significant gap: most evaluated MLLMs achieved below 30% accuracy (e.g., Doubao-1.5-Vision-Pro at 28.1%), far below the human baseline of 51.4% and only slightly above the 25% random baseline. The principal implication for AI practitioners is that current MLLMs possess weak visual reasoning capabilities, necessitating better evaluation benchmarks and development focus on genuine vision-centric understanding, potentially leveraging techniques like reinforcement learning which showed promise (improving a baseline to 31.1%). |
| DreamID: High-Fidelity and Fast diffusion-based Face Swapping via |
|
|
| Triplet ID Group Learning (Read more on arXiv or HuggingFace) |
heqian, giruhc9gj, Crayon-Shinchan, miaohua, Alon77777 |
DreamID introduces a high-fidelity, fast diffusion-based face swapping model using explicit supervision. The primary objective is to significantly improve identity (ID) similarity and attribute preservation in face swapping while achieving rapid inference speed. Key methodology involves constructing Triplet ID Group data (source A1, pseudo target B, ground truth A2) for explicit pixel-level supervision, leveraging the accelerated Stable Diffusion Turbo (SD Turbo) model for single-step inference, and utilizing an improved architecture comprising SwapNet, FaceNet, and an ID Adapter. The primary result shows state-of-the-art performance, achieving 0.71 ID similarity and generating 512x512 resolution swaps in just 0.6 seconds. For AI practitioners, this work provides a significantly faster and more accurate face swapping technique by enabling effective end-to-end training with explicit image-space loss functions, overcoming limitations of implicit supervision in prior diffusion-based methods. |
| Trillion 7B Technical Report (Read more on arXiv or HuggingFace) |
Suyeong An, hist0613, kyudolski, scottsuk0306, sungjunhan-trl |
Trillion-7B is introduced as a highly token-efficient, Korean-centric multilingual Large Language Model. The research aims to address the data imbalance in multilingual LLM training, enabling effective knowledge transfer from English to target languages like Korean despite data scarcity. Key methodologies include the novel Cross-lingual Document Attention (XLDA) mechanism, optimized data mixtures, language-specific filtering, a tailored tokenizer, and a two-stage pre-training approach. Trillion-7B achieves competitive performance across 27 benchmarks using only 10% multilingual data within its 2T token training budget, requiring 59.4K H100 GPU hours ($148K) for full training. For AI practitioners, this demonstrates that architectural innovations like XLDA and strategic training can enable efficient development of high-performing multilingual models for less-resourced languages, reducing reliance on massive language-specific data scaling. |
| Pre-DPO: Improving Data Utilization in Direct Preference Optimization |
|
|
| Using a Guiding Reference Model (Read more on arXiv or HuggingFace) |
Yue Zhang, Qiji Zhou, Shulin Huang, Junshu Pan, Swtheking |
Pre-DPO is a training paradigm enhancing DPO and SimPO preference optimization by leveraging a guiding reference model derived from an initial optimization pass for improved data utilization. The research objective was to overcome inefficient data weighting and performance ceilings inherent in standard DPO/SimPO reference model configurations. Methodologically, Pre-DPO involves first optimizing an initial policy, setting this optimized policy as a guiding reference model, and subsequently re-optimizing the initial policy using DPO under the guidance of this new reference. Experimental results show Pre-DPO consistently outperforms standard DPO and SimPO, achieving average improvements of 2.5 points on AlpacaEval 2 LC and boosting Qwen2.5-7B-Instruct Arena-Hard v0.1 WR from 62.9 (DPO) to 68.8. For AI practitioners, this provides a technique to enhance preference optimization outcomes using existing models and data by enabling more effective adaptive data reweighting, potentially raising performance ceilings. |
| I-Con: A Unifying Framework for Representation Learning (Read more on arXiv or HuggingFace) |
John Hershey, Shaden Alshammari, mhamilton723, mrpuppt, axelf |
I-Con introduces a unified information-theoretic framework generalizing numerous representation learning methods by minimizing an integrated KL divergence between supervisory and learned conditional neighborhood distributions. The primary objective is to demonstrate that diverse techniques like clustering, contrastive learning, dimensionality reduction, and supervised learning are special cases of this single underlying loss function. The methodology involves defining specific conditional probability distributions (p and q) for existing algorithms (e.g., SNE, SimCLR, K-Means, Cross-Entropy) to show their equivalence to minimizing the I-Con objective, proving 15 theorems connecting over 23 methods. Key results include the theoretical unification itself and the creation of a novel debiased clustering method achieving a +8% improvement in Hungarian accuracy on unsupervised ImageNet-1K classification over the previous state-of-the-art. For AI practitioners, I-Con provides a principled foundation for understanding the relationships between disparate loss functions, enabling the transfer of techniques across domains and the development of improved or novel representation learning algorithms, particularly for unsupervised tasks. |
| Decoupled Global-Local Alignment for Improving Compositional |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Ziyong Feng, Jun Wang, haoranxu, Kaichengalex, xiaoxing2001 |
This paper introduces DeGLA, a framework enhancing vision-language models’ compositional understanding while maintaining general capabilities by decoupling global self-distillation alignment from local contrastive alignment using LLM-generated hard negatives. The main objective is to overcome the limitation where improving compositional reasoning in models like CLIP often degrades their general performance due to catastrophic forgetting during fine-tuning. DeGLA utilizes self-distillation with an EMA teacher for global alignment and introduces Image-Grounded Contrast (IGC) and Text-Grounded Contrast (TGC) losses with ~2M LLM-generated negative captions for local alignment. Compared to the CE-CLIP baseline, DeGLA shows an average 3.5% improvement across VALSE, SugarCrepe, and ARO compositional benchmarks and a 13.0% average improvement across 11 zero-shot classification datasets. For AI practitioners, DeGLA offers a method to fine-tune vision-language models for improved nuanced understanding (e.g., attribute binding, relations) in multimodal tasks without significantly sacrificing their robust zero-shot transfer abilities. |
| DreamO: A Unified Framework for Image Customization (Read more on arXiv or HuggingFace) |
LemonSky1995, Crayon-Shinchan, shiwenzh, Zinan123212, yanze |
DreamO provides a unified framework based on a Diffusion Transformer (DiT) for diverse image customization tasks using lightweight adaptation. The objective is to overcome the limitations of task-specific models by enabling flexible integration and interaction of multiple control conditions (identity, subject, style, try-on) within a single model. The methodology involves fine-tuning a pre-trained DiT (Flux-1.0-dev) using LoRA, introducing a feature routing constraint based on cross-attention supervision for fidelity and disentanglement, a placeholder strategy for positional control, and a three-stage progressive training strategy. Qualitative results demonstrate high-fidelity generation across multiple conditions with only 707M additional trainable LoRA parameters, and ablation studies confirm the effectiveness of the routing constraint and progressive training. For AI practitioners, DreamO offers a method to implement versatile, multi-conditional image customization capabilities efficiently using a single, lightweight adapted model, reducing the need for multiple specialized systems. |
| Tina: Tiny Reasoning Models via LoRA (Read more on arXiv or HuggingFace) |
Ollie Liu, Enes Burak Bilgin, Ömer Faruk Akgül, Julian Asilis, upup-ashton-wang |
This paper introduces Tina, a family of cost-effective 1.5B parameter reasoning models developed by applying LoRA during RL. The research objective is to determine how cost-effectively strong reasoning abilities can be achieved in small language models using minimal computational resources. Key methodology involves applying parameter-efficient low-rank adaptation (LoRA) updates during reinforcement learning (specifically, a GRPO-style algorithm) to a tiny 1.5B parameter base model, using open-source frameworks and minimal hardware. Primary results demonstrate that Tina models achieve reasoning performance competitive with, and sometimes superior to, full-parameter trained SOTA RL models on the same base; the best Tina model attained a 50.60% average score across six reasoning benchmarks, significantly outperforming its 41.60% full-parameter baseline average, at an estimated $9 post-training and evaluation cost. The principal implication for AI practitioners is that LoRA combined with RL provides a highly resource-efficient pathway to substantially enhance reasoning capabilities in smaller LMs, achieving significant performance gains with minimal computational expenditure, potentially by rapidly adapting the model’s output format. |
| A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training |
|
|
| and Deployment (Read more on arXiv or HuggingFace) |
Guibin Zhang, Kun Wang, Ningyu, Atarogic, Fred456 |
This survey introduces “full-stack” LLM safety, comprehensively analyzing security and safety issues across the entire LLM lifecycle, from data to deployment. The primary objective is to systematically categorize safety considerations throughout all stages of LLM development (data preparation, pre-training, post-training including alignment, editing, unlearning, and agent integration) and deployment, identifying gaps in existing research that focuses on isolated phases. The methodology involves an extensive literature review encompassing over 800 papers, synthesized into a novel “full-stack” taxonomic framework that maps safety risks and defenses across the defined LLM lifecycle stages. Key results include the identification of persistent risks such as data poisoning (e.g., 0.1% poisoned data causing lasting impact even after fine-tuning), privacy leakage from training data memorization, vulnerabilities introduced during fine-tuning/alignment (like RLHF reward model poisoning), and novel attack surfaces in LLM-based agents involving tool use and memory manipulation. The principal implication for AI practitioners is the critical need to integrate safety considerations throughout the entire development and deployment pipeline, recognizing that security is not merely a deployment-stage concern but is deeply intertwined with data sourcing, training methodologies, alignment processes, and the integration of external modules in agentic systems. |
| RePOPE: Impact of Annotation Errors on the POPE Benchmark (Read more on arXiv or HuggingFace) |
Matthias Hein, YanNeu |
This paper assesses the impact of annotation errors in the MSCOCO dataset on the POPE object hallucination benchmark and introduces a corrected version called RePOPE. The objective is to quantify how these underlying label errors influence the evaluation and ranking of Vision Large Language Models (VLMs) for object hallucination. The methodology involved re-annotating all 500 images used in POPE by consensus, identifying errors and ambiguous cases, creating the corrected RePOPE labels by fixing errors and removing ambiguities, and re-evaluating various VLMs. Primary results show significant label errors, particularly 9.3% errors and 13.8% ambiguous cases in the positive (“Yes”) set of POPE, leading to substantial shifts in model F1 score rankings on RePOPE compared to the original benchmark. The principal implication for AI practitioners is that evaluations based on the original POPE benchmark are notably affected by annotation quality, and using RePOPE offers a more reliable assessment, potentially changing conclusions about relative model performance regarding hallucinations. |
| Rethinking the Generation of High-Quality CoT Data from the Perspective |
|
|
| of LLM-Adaptive Question Difficulty Grading (Read more on arXiv or HuggingFace) |
Keyu Wu, Kunlinliu2, MeiManlin, zcs1234, USTCYu |
This paper introduces LLM-adaptive difficulty grading to generate high-quality CoT data, enabling smaller LLMs to achieve superior reasoning performance. The objective is to determine if LLM-adaptive question difficulty grading can efficiently produce high-quality Chain-of-Thought (CoT) data tailored to enhance smaller LLM reasoning capabilities. The methodology involves grading questions using a base LLM’s performance (correctness check + PRM-Grader), constructing an adaptive question database, sampling based on difficulty distribution, and generating verified CoT using DeepSeek-R1 as a teacher model. Results show that a 32B model fine-tuned on just 2k adaptively generated math CoT examples (ZMath-32B) significantly outperformed the DeepSeek-Distill-32B baseline on math benchmarks (e.g., 73.33% vs 66.67% accuracy on AIME24). For AI practitioners, this indicates that smaller, intelligently curated CoT datasets based on adaptive difficulty grading can be highly resource-efficient for substantially improving the reasoning abilities of smaller LLMs through supervised fine-tuning. |
| CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation (Read more on arXiv or HuggingFace) |
Ziteng Wang, Jia Pan, Robert Zhang, gregdurrett, anirudhkhatry |
This paper introduces CRUST-Bench, a benchmark for evaluating C-to-safe-Rust transpilation using 100 C repositories with manually defined Rust interfaces and test cases. The research objective is to assess the ability of current transpilation systems, particularly LLMs, to generate functionally correct, memory-safe, and idiomatic Rust code from entire C repositories. The methodology involved creating the CRUST-Bench dataset by sourcing C repositories, manually authoring corresponding safe Rust interfaces and test suites, and using this framework to evaluate various LLMs and agentic systems. Primary results show that current state-of-the-art LLMs find this task challenging; the best performing model, OpenAI o1, solved only 15% of tasks single-shot, improving to 37% with iterative test-based repair, highlighting frequent errors in type handling, borrowing rules, and incomplete implementations. For AI practitioners, this implies that fully automated, reliable C-to-safe-Rust migration for complex projects using current LLMs remains an open challenge, necessitating significant improvements in handling Rust’s strict safety and ownership semantics or requiring human oversight. |
| Unchecked and Overlooked: Addressing the Checkbox Blind Spot in Large |
|
|
| Language Models with CheckboxQA (Read more on arXiv or HuggingFace) |
Borchmann, sf-mchilinski, mturski |
This paper introduces CheckboxQA, a benchmark dataset to evaluate and improve Large Vision-Language Model (LVLM) performance on interpreting checkboxes in documents. The primary objective is to assess and address the significant challenge LVLMs face with accurately identifying checkbox states and their associated context, a crucial but often overlooked aspect of document understanding. The authors curated the CheckboxQA dataset comprising 88 documents and 579 question-answer pairs focused on checkbox interpretation and evaluated baseline LVLMs using the Average Normalized Levenshtein Similarity (ANLS) metric. Results show that even top-performing models like Qwen 2.5 VL 72B (83.2 ANLS) lag significantly behind human performance (97.5 ANLS*), indicating substantial room for improvement. For AI practitioners, this research underscores that robust document processing requires specific attention to fine-grained visual elements like checkboxes, as general LVLM proficiency does not automatically transfer, necessitating targeted datasets and potentially model adaptations for reliable real-world applications. |
| Progressive Language-guided Visual Learning for Multi-Task Visual |
|
|
| Grounding (Read more on arXiv or HuggingFace) |
Dingjiang Huang, Kunhua Ji, Wenlong Zhang, Hong Wang, jcwang0602 |
This paper introduces PLVL, a Progressive Language-guided Visual Learning framework for Multi-Task Visual Grounding (MTVG), integrating Referring Expression Comprehension (REC) and Segmentation (RES). The main objective is to address insufficient language injection into visual backbones and ineffective exploitation of the REC-RES task relationship in existing methods. PLVL utilizes a modified ViTDet backbone with local and global blocks, progressively injecting language tokens via cross-attention in global blocks, and employs a novel convolution-based collaborative multi-task head exploiting shared object localization priors. Results demonstrate state-of-the-art performance, achieving 89.80% accuracy on the RefCOCOg test(U) REC task under pre-training settings, outperforming previous methods. For AI practitioners, PLVL offers a more effective architecture for joint REC/RES prediction by deeply integrating language guidance throughout the visual feature extraction process and explicitly modeling task synergy, leading to improved grounding accuracy. |
Papers for 2025-04-23
| Title |
Authors |
Summary |
| Kuwain 1.5B: An Arabic SLM via Language Injection (Read more on arXiv or HuggingFace) |
Omar Hadid, Sara Chrouf, ZeinaD, Moatasem444, Hennara |
This paper introduces Kuwain 1.5B, an Arabic-English Small Language Model created via language injection into an existing English model. The primary objective was to efficiently integrate Arabic into an English-centric LLM (TinyLlama 1.1B) without compromising its original knowledge or incurring high retraining costs. The methodology involved expanding the tokenizer with 26K Arabic tokens and inserting 8 new, trainable layers into the model architecture while freezing the original layers, using only 20% of the original English data alongside a large Arabic corpus. Results demonstrated an average 8% performance improvement on Arabic benchmarks compared to the base model, while maintaining comparable performance on English benchmarks (53.28 average score vs. 52.99 for the base model). For AI practitioners, this work presents a resource-efficient language injection technique to expand model capabilities to new languages, especially low-resource ones, without extensive retraining or significant degradation of existing knowledge. |
| TTRL: Test-Time Reinforcement Learning (Read more on arXiv or HuggingFace) |
Xuekai Zhu, Li Sheng, Shang Qu, Yuxin Zuo, iseesaw |
This paper introduces Test-Time Reinforcement Learning (TTRL), a method for improving Large Language Models (LLMs) on reasoning tasks using unlabeled test data. The objective is to enable LLM self-evolution using Reinforcement Learning (RL) during inference without access to ground-truth labels, addressing the challenge of reward estimation in this setting. TTRL employs repeated sampling to generate multiple outputs, uses majority voting to estimate a consensus label, and computes rule-based rewards based on this estimate to drive RL training. Experiments show TTRL boosted Qwen-2.5-Math-7B pass@1 performance on AIME 2024 by approximately 159% using only unlabeled test data, and consistently surpassed the performance upper limit implied by the initial model’s majority voting accuracy. For AI practitioners, TTRL demonstrates a method for adapting and improving LLMs on new tasks using unlabeled data alone, suggesting a potential pathway for continuous learning and reduced reliance on extensive labeled datasets for RL fine-tuning. |
| The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks (Read more on arXiv or HuggingFace) |
Huifeng Yin, Sinuo Liu, Weixuan Wang, Minghao Wu, ChenyangLyu |
This paper analyzes over 2,000 multilingual (non-English) benchmarks (2021-2024) to evaluate past, present, and future multilingual benchmarking practices. The primary objective is to assess historical trends, the current alignment of benchmarks with human judgments, and future needs for multilingual evaluation. The methodology involved collecting and annotating 2,024 arXiv papers, analyzing language/task/domain distributions, and correlating LLM performance on benchmarks with human Elo rankings across five languages. Key findings reveal English overrepresentation despite exclusion, poor correlation for translated benchmarks (e.g., MMLU Chinese correlation 0.473 vs. localized CMMLU 0.682), and better alignment for STEM tasks (0.70-0.85 correlation) than traditional NLP tasks like QA (0.11-0.30). The principal implication for AI practitioners is that evaluating multilingual models requires moving beyond translated English benchmarks towards developing localized, culturally authentic, and human-aligned benchmarks for accurate capability assessment. |
| Describe Anything: Detailed Localized Image and Video Captioning (Read more on arXiv or HuggingFace) |
Yifan Ding, richardaecn, yala, Boyiliee, longlian |
This paper introduces the Describe Anything Model (DAM) for generating detailed captions for specific regions in images and videos. The primary objective is to overcome limitations in existing VLMs regarding precise localization and the generation of detailed, context-aware regional descriptions. DAM employs a focal prompt for high-resolution encoding of target regions and a localized vision backbone that integrates global context with local details using gated cross-attention, trained via a novel semi-supervised data pipeline (DLC-SDP). The model achieves state-of-the-art results on 7 benchmarks, including a 67.3% average accuracy on the newly proposed DLC-Bench. For AI practitioners, DAM offers a robust method for fine-grained visual understanding, enabling applications requiring detailed descriptions of user-specified image or video regions without relying on reference captions for evaluation. |
| Learning Adaptive Parallel Reasoning with Language Models (Read more on arXiv or HuggingFace) |
Charlie Snell, Long Lian, Jiayi Pan, yala, xiuyul |
This paper introduces Adaptive Parallel Reasoning (APR), a framework enabling language models to learn adaptive parallelization of reasoning tasks using parent-child threading. The research objective is to overcome limitations of serialized chain-of-thought (latency, context limits) and simple parallel methods (redundancy, poor coordination) by training models to dynamically orchestrate both serial and parallel computations. APR employs a multi-threading mechanism with spawn() and join() operations, integrated into the language model’s decoding process and optimized end-to-end using reinforcement learning. On the Countdown reasoning task, APR achieved significantly higher accuracy within a fixed context window (83.4% vs. 60.0% for serialized search at 4k context) and better accuracy at equivalent latency (75.2% vs. 57.3% at ~5000ms). The principal implication for AI practitioners is that LMs can be trained to autonomously manage and parallelize their inference-time computation, potentially leading to more efficient and scalable reasoning systems under resource constraints. |
| IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning |
|
|
| in Multimodal LLMs (Read more on arXiv or HuggingFace) |
Yifan Yao, Jarvis Guo, Yuanxing Zhang, JinChengRen, mdh98 |
IV-Bench is introduced as the first comprehensive benchmark designed to evaluate Multimodal Large Language Models (MLLMs) specifically on image-grounded video perception and reasoning tasks. The research objective is to assess how effectively MLLMs utilize external static images as indispensable context for video comprehension, a capability largely overlooked by existing benchmarks. The methodology involved creating a dataset of 967 videos paired with 2,585 meticulously annotated image-text queries across 13 tasks (7 perception, 6 reasoning) using externally sourced, necessary images, followed by evaluating 27 state-of-the-art open and closed-source MLLMs. The primary result shows current MLLMs significantly underperform, with the best model achieving only 28.9% overall accuracy, and performance deteriorating further on reasoning tasks (best at 24.9%). For AI practitioners, this implies a critical need to develop advanced MLLMs with improved mechanisms for integrating external image context into video understanding, as current models struggle significantly with these tasks and simple data format alignment proves insufficient. |
| BookWorld: From Novels to Interactive Agent Societies for Creative Story |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yanghua Xiao, Jiaqing Liang, Tian Qiu, Xintao Wang, Yiting Ran |
BookWorld introduces a system for constructing and simulating multi-agent societies based on fictional novels for creative story generation. The primary objective is to explore simulating established fictional worlds and characters using book data, enabling character-driven storytelling and interactive experiences. The methodology involves extracting character profiles, worldview data, and map information from source texts to initialize role agents and a world agent, which orchestrate interactions, memory updates, and movements within scene-based simulations managed by LLMs. BookWorld demonstrated superior performance in generating high-quality, faithful narratives, surpassing previous methods with a win rate of 75.36% in comparative evaluations. For AI practitioners, this research provides a framework for leveraging existing literary works to create immersive, context-rich simulations and interactive story generation applications, reducing the need for manual world-building. |
| Efficient Pretraining Length Scaling (Read more on arXiv or HuggingFace) |
Jianqiao Lu, Sijun Zhang, Shen Yan, Taoer, bongbohong |
This paper introduces the Parallel Hidden Decoding (PHD) Transformer framework to enable efficient length scaling during language model pre-training. The objective is to achieve the performance benefits of increased sequence length during pre-training without proportionally increasing KV cache size or inference latency. The core methodology involves repeating input tokens multiple times but employing a novel KV cache strategy where only the cache from original tokens is retained globally, while the cache from repeated (“hidden decoding”) tokens is discarded or kept only within a local/chunk-wise window (PHD-SWA/PHD-CSWA). Results demonstrate consistent performance improvements; for instance, the PHD-CSWA-3-16-32 variant achieved a 2.0% average accuracy increase across evaluated benchmarks compared to a 1.2B parameter baseline, with minimal impact on inference efficiency. For AI practitioners, this work presents a method (PHD-CSWA) to enhance model reasoning capabilities through pre-training length scaling without the typical memory and latency penalties, offering a practical approach to scale computational depth efficiently. |
| CheXWorld: Exploring Image World Modeling for Radiograph Representation |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Shiji Song, Pan Liu, Chenxin Tao, Yulin Wang, yueyang2000 |
CheXWorld introduces a self-supervised world modeling framework for learning robust radiograph representations by capturing anatomical knowledge and domain variations. The primary objective is to develop a unified framework that models local anatomical structures, global anatomical layouts, and domain appearance variations essential for radiograph interpretation. Key methodology involves integrating these three aspects through tailored prediction tasks within a joint-embedding predictive architecture, predicting target representations based on context and latent variables (relative position, augmentation parameters). CheXWorld significantly outperforms existing self-supervised learning methods on eight medical image classification and segmentation benchmarks, achieving 95.24±0.13 AUROC on VinDr-CXR classification. For AI practitioners, the principal implication is that this world modeling approach yields highly effective and transferable representations for diverse radiograph analysis tasks, potentially reducing the need for extensive labeled data. |
| Personalized Text-to-Image Generation with Auto-Regressive Models (Read more on arXiv or HuggingFace) |
Xihui Liu, Yao Teng, Xian Liu, Kaiyue Sun |
This research investigates personalized text-to-image generation using auto-regressive (AR) models, adapting them for a task typically dominated by diffusion models. The primary objective is to evaluate the potential of optimizing AR models for personalized image synthesis by leveraging their unified architecture for text and image modeling. The methodology involves a two-stage training strategy: first optimizing text embeddings associated with a unique identifier for the subject, and second, fine-tuning the model’s transformer layers using 3-5 reference images. Experiments on the Lumina-mGPT 7B model demonstrated comparable subject fidelity (DINO: 0.671) and prompt following (CLIP-T: 0.314) to the diffusion-based DreamBooth method (DINO: 0.668, CLIP-T: 0.305) on the Dreambench dataset. For AI practitioners, this work highlights that appropriately optimized AR models present a viable alternative architecture for personalized image generation, achieving competitive fidelity and prompt adherence compared to established diffusion techniques, although generation speed is noted as slower. |
| LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale (Read more on arXiv or HuggingFace) |
Zejun Ma, Wei Li, Yiqi Lin, Ziyun Zeng, Joya Chen |
LiveCC introduces a Video LLM trained at scale using densely interleaved, timestamped automatic speech recognition (ASR) transcripts for real-time video commentary. The primary objective is to enable scalable Video LLM training leveraging cheap ASR data for fine-grained, temporally-aligned vision-language modeling and low-latency inference. Key methodology involves a novel streaming training approach on curated datasets (Live-CC-5M, Live-WhisperX-526K) derived from YouTube closed captions. The final LiveCC-7B-Instruct model surpasses 72B models in commentary quality on the LiveSports-3K benchmark (achieving a 41.5% win rate against LLaVA-Video-72B) and achieves state-of-the-art results on VideoMME/OVOBench QA benchmarks at the 7B scale, with commentary latency under 0.5 seconds per frame. For AI practitioners, this work demonstrates a cost-effective and scalable method using readily available ASR data to develop high-performance, real-time Video LLMs, reducing dependency on expensive annotations or APIs. |
| Vidi: Large Multimodal Models for Video Understanding and Editing (Read more on arXiv or HuggingFace) |
Fan Chen, Chia-Wen Kuo, Celong Liu, Vidi Team, daviddousa |
Vidi is a family of Large Multimodal Models (LMMs) designed for long-duration video understanding and editing, initially focused on temporal retrieval using vision, audio, and text. The primary objective is to develop a multimodal AI model capable of accurately performing temporal retrieval (identifying specific time ranges based on text/audio queries) within hour-long videos by processing visual, auditory, and textual information simultaneously. Vidi employs modality-specific encoders (SigLIP, Whisper), adapter layers, and a Mistral-7B LLM core utilizing Decomposed Attention for efficient processing of densely sampled (1fps visual, 16kHz audio), long multimodal sequences, trained via multi-stage alignment on synthetic and real annotated video data. On the introduced VUE-TR benchmark designed for realistic, long-form video retrieval, Vidi significantly outperforms proprietary models, achieving an overall Intersection-over-Union Area Under Curve (IoU AUC) of 35.4% compared to 21.2% (Gemini-2.0-Flash), 15.2% (Gemini-2.5-Pro), and 13.6% (GPT-4o). For AI practitioners, Vidi demonstrates a viable architecture using Decomposed Attention for building LMMs that can efficiently process and temporally ground queries in hour-long multimodal videos, offering a strong foundation for developing advanced, scalable video editing and retrieval applications. |
| From Reflection to Perfection: Scaling Inference-Time Optimization for |
|
|
| Text-to-Image Diffusion Models via Reflection Tuning (Read more on arXiv or HuggingFace) |
Renrui Zhang, Yue Liao, Sayak Paul, Liangbing Zhao, Le Zhuo |
This paper introduces ReflectionFlow, an inference-time optimization framework that enables text-to-image diffusion models to iteratively refine their outputs via self-reflection. The objective is to improve image generation quality for complex prompts by scaling inference-time computation rather than solely relying on larger pre-trained models. Key methodology involves proposing three scaling axes (noise, prompt, reflection), constructing the large-scale GenRef dataset (1 million reflection triplets plus 227K CoT annotations), and performing efficient reflection tuning on the FLUX.1-dev diffusion transformer by jointly modeling multimodal inputs (prompt, reflection, flawed image, target image) in a unified sequence. The primary result shows ReflectionFlow significantly improves performance, achieving a GenEval score of 0.91 with 32 samples, outperforming the FLUX.1-dev baseline (0.67) and naive noise scaling (0.85), and requiring 10x fewer samples than noise scaling for similar performance levels. For AI practitioners, this offers a scalable, compute-efficient inference-time technique to enhance the fidelity and detail of generated images for challenging prompts without modifying the underlying generative model architecture or extensive retraining. |
| LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making |
|
|
| Abilities (Read more on arXiv or HuggingFace) |
Razvan Pascanu, Markus Wulfmeier, Jordi Grau-Moya, Jörg Bornschein, Thomas Schmied |
This paper investigates why LLMs act sub-optimally as decision-making agents and evaluates Reinforcement Learning Fine-Tuning (RLFT) to improve their performance. The research aims to identify the causes of sub-optimal LLM decision-making, specifically greediness, frequency bias, and the knowing-doing gap, and to determine if RLFT on self-generated Chain-of-Thought (CoT) rationales can mitigate these issues. Methodology involved analyzing Gemma2 models (2B, 9B, 27B) on multi-armed/contextual bandits and Tic-tac-toe, quantifying failure modes, applying RLFT with a PPO-like objective on CoT outputs, and evaluating various exploration strategies. Primary results indicate LLMs exhibit a knowing-doing gap (e.g., 87% correct rationales but acting greedily 58% of that time) and poor exploration (e.g., 27B model covering only 45% of actions in 20-arm MABs); RLFT improved exploration (e.g., +12% action coverage for 2B model after 30k steps) and reduced regret, partially mitigating greediness and frequency bias. The principal implication for AI practitioners is that base LLMs require explicit mechanisms beyond CoT prompting for effective exploration; RLFT on CoT rationales, especially enhanced with exploration bonuses or reward shaping, significantly improves decision-making but does not eliminate the need for careful consideration of exploration strategies in agentic systems. |
| WALL-E 2.0: World Alignment by NeuroSymbolic Learning improves World |
|
|
| Model-based LLM Agents (Read more on arXiv or HuggingFace) |
Deheng Ye, Guodong Long, Yijun Yang, Siyu Zhou, zhoutianyi |
WALL-E 2.0 enhances LLM agent performance by aligning LLM-based world models with environment dynamics through neurosymbolic learning of executable code rules. The primary objective is to bridge the gap between LLM prior knowledge and specific environment dynamics, creating more accurate world models for LLM agents without requiring RL fine-tuning or large memory buffers. The key methodology involves using LLMs for inductive reasoning on environment trajectories to extract symbolic knowledge (action rules, knowledge/scene graphs), translating this into executable code rules, pruning redundant rules, and integrating these into an LLM world model within a Model-Predictive Control (MPC) loop. Results show significant improvements over baselines, including reward increases of 16.1%-51.6% in the Mars environment and achieving a 98% success rate in ALFWorld after only 4 iterations. For AI practitioners, this work demonstrates a training-free method to enhance LLM agent reliability and planning efficiency in novel or dynamic environments by explicitly learning and enforcing environment-specific constraints as verifiable code rules within the agent’s world model. |
| MR. Video: “MapReduce” is the Principle for Long Video Understanding (Read more on arXiv or HuggingFace) |
Yu-Xiong Wang, Ziqi Pang |
MR. Video proposes and validates the MapReduce principle for long video understanding, using an agentic framework to separate parallel short clip perception (Map) from joint information aggregation (Reduce). The objective is to overcome context length limitations of VLMs and the sequential, limited-context nature of existing video agents by applying this big data processing paradigm. The methodology involves a two-stage MapReduce workflow (Captioning and Analysis) implemented via an LLM agent controlling a VLM (Gemini-2.0-Flash) for perception and an LLM (GPT4o) for reasoning/reduction. MR. Video achieves 60.8% accuracy on the challenging LVBench dataset, demonstrating a >10% improvement over state-of-the-art VLMs and video agents, and correctly localizes relevant scenes for 68.8% of questions via its intention analysis step. For AI practitioners, this demonstrates that structuring long video analysis using the MapReduce principle enables scalable, parallel processing of local details and comprehensive global context aggregation, offering a practical method to improve performance on long-form video tasks. |
| Progent: Programmable Privilege Control for LLM Agents (Read more on arXiv or HuggingFace) |
Hongwei Li, Linyu Wu, Zhun Wang, Jingxuan He, stneng |
Progent introduces a programmable framework using a domain-specific language (DSL) for fine-grained privilege control over LLM agent tool calls. The primary objective is to mitigate security risks associated with LLM agents executing potentially harmful actions via tools by enforcing the principle of least privilege. Key methodology involves a DSL, implemented using the JSON ecosystem, to define policies specifying permissible tool calls, conditions, and fallbacks, with support for manual definition and LLM-based automated generation/updating. Experimental results show Progent significantly enhances security, reducing attack success rates on the AgentDojo benchmark from 41.2% to 2.2% using combined manual and LLM-managed policies, while maintaining high utility. For AI practitioners, Progent offers a modular, API-based mechanism to integrate deterministic security controls into LLM agents, restricting tool use to essential functions and reducing vulnerabilities with minimal code modification. |
| RealisDance-DiT: Simple yet Strong Baseline towards Controllable |
|
|
| Character Animation in the Wild (Read more on arXiv or HuggingFace) |
Chao Fan, Min Wei, Shikai Li, Yifan Wu, Jingkai Zhou |
RealisDance-DiT introduces a simple yet strong baseline for controllable character animation in the wild by leveraging a powerful video foundation model with minimal modifications. The main objective is to address challenges in character animation such as rare poses, stylized characters, object interactions, and complex scenes without relying on elaborate, task-specific networks like Reference Net. The methodology involves making minor adjustments to the Wan-2.1 DiT architecture (adding condition layers, modifying RoPE) and employing specific fine-tuning strategies, namely low-noise warmup and large-batch/small-iteration training, to preserve foundation model priors while adapting to the animation task. Primary results show state-of-the-art performance, achieving an FVD of 563.28 and FID of 24.79 on the proposed RealisDance-Val benchmark, significantly outperforming prior methods. The principal implication for AI practitioners is that adapting large pre-trained foundation models with straightforward modifications and tailored fine-tuning can yield superior results for complex generative tasks compared to designing complex, specialized architectures from scratch. |
| IPBench: Benchmarking the Knowledge of Large Language Models in |
|
|
| Intellectual Property (Read more on arXiv or HuggingFace) |
Minghui Zhu, Huaren Liu, Hongbo Wang, Guhong Chen, QiYao-Wang |
This paper introduces IPBench, a comprehensive, bilingual benchmark designed to evaluate Large Language Model (LLM) knowledge across the complex intellectual property (IP) domain. The primary objective is to assess LLM capabilities in real-world IP scenarios involving both technical and legal understanding, covering 8 IP mechanisms and 20 distinct tasks. Methodologically, the benchmark comprises 10,374 data points used to evaluate 16 different LLMs, ranging from general-purpose to domain-specific models, under various prompting strategies. The key finding indicates substantial limitations, as the top-performing model (DeepSeek-V3) achieved only 75.8% overall accuracy, with open-source IP/law-oriented models notably underperforming compared to closed-source general models. For AI practitioners, this highlights the current gap in LLM proficiency for specialized IP tasks and suggests a need for enhanced domain-specific adaptation or fine-tuning, particularly for open-source solutions, to handle the required blend of technical and legal reasoning effectively. |
| CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via |
|
|
| Occluded Object Counting (Read more on arXiv or HuggingFace) |
Mohit Bansal, Jaemin Cho, Elias Stengel-Eskin, Atin Pothiraj |
This paper introduces CAPTURE, a benchmark to evaluate Vision Language Models’ (VLMs) spatial reasoning by counting objects in patterns, especially when occluded. The primary objective is to quantify VLMs’ ability to perform amodal counting by inferring patterns hidden behind occluders, testing their world modeling and spatial understanding. Methodology involves evaluating four VLMs (GPT-4o, Intern-VL2, Molmo, Qwen2-VL) on the CAPTURE dataset (real and synthetic images with occluded patterns) using the symmetric mean absolute percentage error (sMAPE) metric, comparing performance on occluded versus unoccluded images and against human/object-detection baselines. Results show current VLMs struggle significantly, performing worse with occlusion (average sMAPE of 27.37% on CAPTUREreal occluded images versus 21.09% unoccluded), in stark contrast to near-perfect human performance (3.79% sMAPE occluded); providing oracle object coordinates substantially improves VLM performance, reducing error significantly (e.g., average VLM error dropped by 15.65% on CAPTUREreal when given all coordinates). For AI practitioners, this highlights that even strong VLMs lack robust spatial world modeling for occluded scenes, indicating their errors stem from both visual counting difficulties and an inability to infer missing information, suggesting limitations in current architectures for tasks requiring integrated visual reasoning and amodal completion. |
| DiffVox: A Differentiable Model for Capturing and Analysing Professional |
|
|
| Effects Distributions (Read more on arXiv or HuggingFace) |
Wei-Hsiang Liao, Ben Hayes, Junghyun Koo, Marco A. Martínez-Ramírez, yoyolicoris |
i) DiffVox presents a differentiable model for estimating and analysing professional vocal effects parameter distributions from audio data. ii) The research aims to capture real-world vocal processing configurations by reverse-engineering effects parameters using differentiable signal processing and analysing their statistical properties. iii) The methodology employs a differentiable audio effects chain (parametric EQ, dynamics, delay, FDN reverb) optimized via gradient descent using multi-resolution spectral (MRS) and loudness dynamic range (MLDR) losses on paired dry/wet vocal stems from two datasets. iv) Primary results demonstrate effective parameter fitting (e.g., DiffVox achieves MRS loss of 0.75/0.98 on left/right & mid/side channels for MedleyDB) and principal component analysis indicates the most significant variation (13.86% of variance on the internal dataset) corresponds to perceived spaciousness, with parameter distributions confirmed as non-Gaussian. v) For AI practitioners, this work offers a validated method and a public dataset of vocal presets to establish realistic priors for audio effects, potentially improving generative audio models and automatic mixing systems by replacing non-informative uniform or Gaussian assumptions. |
Papers for 2025-04-22
| Title |
Authors |
Summary |
| Learning to Reason under Off-Policy Guidance (Read more on arXiv or HuggingFace) |
Zhi Wang, ganqu, huzican, yaful, Elliott |
LUFFY introduces an off-policy guidance framework for reinforcement learning to enhance large reasoning model capabilities beyond purely on-policy methods. The primary objective is to effectively integrate external, high-quality reasoning traces (off-policy) with a model’s own exploration (on-policy) within the zero-RL paradigm, overcoming limitations where models fail to acquire abilities beyond their initial scope. Key methodologies include mixed-policy training combining off-policy demonstrations with on-policy rollouts, and policy shaping via regularized importance sampling to dynamically balance imitation and exploration while mitigating entropy collapse. LUFFY demonstrates significant improvements, achieving an average gain of over +7.0 points across six math benchmarks compared to previous zero-RL methods and a +6.2 point advantage on out-of-distribution tasks. For AI practitioners, this work presents a validated technique to leverage off-policy data within RL, offering a scalable path to train more generalizable and capable reasoning models compared to standard supervised fine-tuning or purely on-policy RL. |
| FlowReasoner: Reinforcing Query-Level Meta-Agents (Read more on arXiv or HuggingFace) |
P2333, bhooi, dreamerdeo, yueliu1999, HongchengGao |
This paper proposes FLOWREASONER, a meta-agent that automatically generates query-specific multi-agent systems using reasoning reinforced by execution feedback. The primary objective is to create a meta-agent that designs a unique multi-agent workflow optimized for each individual user query, overcoming the rigidity of one-size-fits-all task-level systems. Key methodology involves initial supervised fine-tuning (SFT) on reasoning data distilled from a large model, followed by reinforcement learning (RL) using external code execution feedback and a multi-purpose reward signal encompassing performance, complexity, and efficiency. Results show FLOWREASONER-14B achieves 81.89% overall accuracy across three code benchmarks (BigCodeBench, HumanEval, MBPP), notably surpassing the o1-mini baseline by 10.52%. For AI practitioners, FLOWREASONER offers a method to automate the creation of adaptive multi-agent workflows tailored to specific user inputs, potentially improving performance and reducing manual engineering effort for complex, query-dependent tasks. |
| Eagle 2.5: Boosting Long-Context Post-Training for Frontier |
|
|
| Vision-Language Models (Read more on arXiv or HuggingFace) |
WonminByeon, deahuang, lulidong, RealZhiqiLi, cg1177 |
Eagle 2.5 is a vision-language model family improving long-context video and image understanding through specialized post-training and a new dataset. The research objective is to enhance vision-language models’ capabilities for processing long-context multimodal inputs, specifically long videos and high-resolution images, without introducing specialized compression modules. Key methodologies include an information-first sampling strategy (combining Image Area Preservation tiling and Automatic Degradation Sampling for token budgeting), progressive post-training to scale context length (up to 128K), and the creation of the Eagle-Video-110K dataset using a dual (story-level and clip-level) annotation approach, built upon a SigLIP-Qwen2.5 architecture. Primary results show strong performance on long-context tasks; specifically, Eagle 2.5-8B achieves 72.4% accuracy on the Video-MME benchmark with 512 input frames, competitive with significantly larger proprietary and open-source models. For AI practitioners, this work provides validated techniques (information-first sampling, progressive training) and a dataset (Eagle-Video-110K) for developing smaller yet high-performing VLMs capable of processing extended visual contexts, crucial for applications involving long videos or detailed images. |
| ToolRL: Reward is All Tool Learning Needs (Read more on arXiv or HuggingFace) |
Cheng Qian, Gokhantur, XtremSup, Merlin-Hongru, emrecanacikgoz |
This paper presents a comprehensive study on reward design for enhancing Large Language Model (LLM) tool use capabilities via Reinforcement Learning (RL). The main research objective is to systematically investigate and identify optimal reward strategies for tool selection and application tasks within the RL paradigm, assessing factors like reward type, scale, granularity, and temporal dynamics. The key methodology involves proposing a principled, fine-grained reward design tailored for tool use and training LLMs using Group Relative Policy Optimization (GRPO), alongside extensive ablation studies on reward components. Empirical evaluations show this approach yields robust training, achieving a 17% improvement over base models and a 15% gain over Supervised Fine-Tuning (SFT) models on tool use benchmarks; specifically, fine-grained reward decomposition proved more effective than coarser signals. For AI practitioners, the principal implication is that careful, decomposed reward engineering within an RL framework is critical for developing LLMs with significantly enhanced and more generalizable tool-using abilities compared to SFT alone. |
| SphereDiff: Tuning-free Omnidirectional Panoramic Image and Video |
|
|
| Generation via Spherical Latent Representation (Read more on arXiv or HuggingFace) |
joyfull78, sungwon95, YeolJoo, TaewoongKang, mpark |
SphereDiff introduces a tuning-free framework for generating seamless 360-degree panoramic images and videos by leveraging spherical latent representations with pre-trained diffusion models. The primary objective is to overcome the severe distortions and discontinuities, particularly near the poles, associated with traditional equirectangular projection (ERP) methods without requiring model fine-tuning. The methodology involves defining a uniform spherical latent representation, extending MultiDiffusion to this space, employing dynamic latent sampling to map spherical latents to a 2D grid compatible with standard diffusion models, and using distortion-aware weighted averaging during projection. SphereDiff demonstrates superior performance over baselines, achieving significantly higher scores for distortion mitigation (e.g., 3.238 vs. 2.854 for DynamicScaler) and end-to-end continuity (e.g., 4.892 vs. 3.985) in image generation tasks. For AI practitioners, this provides a robust, tuning-free approach to generate high-quality omnidirectional content directly from existing perspective-view diffusion models, bypassing the need for ERP-specific datasets and mitigating common projection artifacts. |
| StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on |
|
|
| 3D Gaussians (Read more on arXiv or HuggingFace) |
Cailin Zhuang, Yiying12, unpackableorange, wchengad, xuanyangz |
StyleMe3D introduces a multi-encoder framework using disentangled priors for high-quality artistic stylization of 3D Gaussian Splatting representations. The primary objective is to enable versatile and coherent style transfer onto pre-reconstructed 3D Gaussian Splatting models while preserving geometric integrity and overcoming limitations of prior methods in handling stylized aesthetics. Key methodology involves integrating four novel components—Dynamic Style Score Distillation (DSSD) using Stable Diffusion’s latent space, Contrastive Style Descriptor (CSD), Simultaneously Optimized Scale (SOS) via VGG features, and a 3D Gaussian Quality Assessment (3DG-QA) prior—while optimizing only the RGB attributes of the Gaussians. StyleMe3D demonstrated superior performance over state-of-the-art methods, achieving higher quantitative metrics (e.g., PSNR 18.015, SSIM 0.830, LPIPS 0.174 on evaluated datasets) and preserving fine geometric details and stylistic consistency. For AI practitioners, this work provides a robust method to apply diverse artistic styles to existing 3D GS assets, significantly enhancing visual content for gaming, virtual worlds, and digital art by effectively bridging photorealistic reconstruction with artistic expression without altering underlying geometry. |
| X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents (Read more on arXiv or HuggingFace) |
hamidpalangi, mparvez, genglinliu, liweijiang, salmannyu |
X-Teaming introduces an adaptive multi-agent framework for systematic multi-turn language model jailbreaking and defense generation. The main objective is to address the gap in multi-turn conversational AI safety by exploring how harmless interactions escalate into harmful outcomes and generating diverse attack scenarios. Key methodology involves a two-phase approach using collaborative agents: a Planner for strategy, an Attacker for execution, a Verifier for evaluation, and a Prompt Optimizer using TextGrad for refining failed attacks. Primary results show state-of-the-art multi-turn jailbreak effectiveness, achieving attack success rates up to 98.1% across various models, including 96.2% against Claude 3.7 Sonnet, and the creation of the 30K-example XGuard-Train dataset. For AI practitioners, this work provides the X-Teaming framework for scalable multi-turn red-teaming and the XGuard-Train dataset, enabling the development and training of more robust multi-turn safety alignment defenses for LMs. |
| UFO2: The Desktop AgentOS (Read more on arXiv or HuggingFace) |
rujiawang, liqul, duchao, shilhe, vyokky |
UFO2 presents a multiagent AgentOS deeply integrated with Windows for robust LLM-driven desktop automation. The objective is to build a practical, system-level automation framework that overcomes the limitations of prior CUAs reliant on shallow OS integration and screenshot-based interaction. Methodology involves a HOSTAGENT for orchestration, application-specific APPAGENTS leveraging native APIs and domain knowledge, a hybrid UIA-vision control detection pipeline, a unified GUI-API action layer, speculative multi-action execution, and a Picture-in-Picture interface for non-disruptive operation. Key results demonstrate superior performance over existing CUAs, achieving up to 32.7% success rate on the OSWorld-W benchmark (o1 model); API integration improved completion rates by over 8%, and speculative execution reduced LLM inference calls by up to 51.5% on certain tasks without degrading success rate. For AI practitioners, this work highlights that deep OS integration and hybrid GUI-API interaction models are critical for moving desktop automation agents from conceptual prototypes to reliable, efficient, and scalable real-world applications. |
| LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient |
|
|
| Training of Code LLMs (Read more on arXiv or HuggingFace) |
Yan Wang, Yunhui Xia, chuyi777, jasonkleinlove, Swtheking |
This paper introduces LeetCodeDataset, a high-quality temporal benchmark curated from LeetCode Python problems for robust code LLM evaluation and efficient training. The objective is to address the lack of reasoning-focused coding benchmarks and provide a self-contained, contamination-free testbed for training and evaluation. The methodology involved collecting 2,869 LeetCode problems with metadata, 100+ test cases per problem, canonical solutions, and applying a strict temporal split (pre/post-July 2024) for training and test sets. Results demonstrate that reasoning models significantly outperform non-reasoning ones (DeepSeek-R1 achieved 65.23% pass@1 on the test set), and supervised fine-tuning (SFT) using only 2.6K model-generated examples from the dataset achieved performance comparable to models trained on 110K examples. For AI practitioners, this dataset provides a reliable resource for evaluating code generation models without contamination and highlights the potential for highly data-efficient SFT using curated, high-quality problem-solution pairs. |
| Seeing from Another Perspective: Evaluating Multi-View Understanding in |
|
|
| MLLMs (Read more on arXiv or HuggingFace) |
Shengbang Tong, yubei, chengtim, ch-chenyu, danielchyeh |
Here is a 4-sentence summary of the research paper: This paper introduces All-Angles Bench, a new benchmark with over 2,100 question-answer pairs across 90 scenes, designed to evaluate the multi-view scene understanding capabilities of Multi-Modal Large Language Models (MLLMs). The primary objective is to assess how well MLLMs reconcile geometric consistency and cross-view correspondence across diverse viewpoints using six defined tasks, including attribute identification and camera pose estimation. Experiments on 27 MLLMs (e.g., GPT-4o, Gemini-2.0-Flash, InternVL2.5-38B) reveal a significant performance gap compared to humans (human 82.0% vs. best MLLM 60.8% on a 250 Q&A subset), with particular weaknesses in handling partial occlusions and estimating coarse camera poses. For AI practitioners, this implies that current MLLMs require substantial improvements, likely through domain-specific training or architectural changes incorporating multi-view awareness, to be reliably deployed in applications demanding 3D scene comprehension like embodied agents. |
| InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to |
|
|
| Deliberative Reasoners (Read more on arXiv or HuggingFace) |
Xavier Hu, Yuhang Liu, xiaotianhan, xieck13, pengxiang |
This paper introduces InfiGUI-R1, an MLLM-based GUI agent designed to transition from reactive behavior to deliberate reasoning for complex GUI tasks. The main objective is to advance GUI agents beyond reactive execution by explicitly incorporating robust planning, cross-modal spatial reasoning, and error recovery capabilities. The core methodology is the Actor2Reasoner framework, employing Spatial Reasoning Distillation (SFT) for initial reasoning injection, followed by Deliberation Enhancement using Reinforcement Learning with novel Sub-goal Guidance and Error Recovery Scenario Construction techniques. Experimental results show InfiGUI-R1-3B achieves strong cross-platform GUI grounding (87.5% average accuracy on ScreenSpot) and task execution performance (71.1% success rate on AndroidControl-High), competitive against larger parameter models. For AI practitioners, this work provides a structured framework and specific training techniques (reasoning distillation, RL with targeted rewards for sub-goals and error recovery) to build more capable GUI agents that can handle complex, long-horizon tasks requiring planning and adaptation. |
| EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Linear-Matrix-Probability, HaomingXu, xukewei, Saberlve, xzwnlp |
i) EasyEdit2 is a framework enabling adjustable, plug-and-play, test-time behavioral control of Large Language Models (LLMs) via steering interventions. ii) The main research objective is to create a unified, user-friendly framework for steering diverse LLM behaviors (e.g., safety, sentiment, factuality, reasoning) without altering the model’s underlying parameters. iii) The methodology centers on a steering vector generator (supporting methods like CAA, STA, LM-Steer, Prompt Auto) and a steering vector applier, which integrate intervention vectors during the forward pass, facilitated by a vector library and merging capabilities. iv) Primary results show effectiveness across different LLMs; specifically, the Contrastive Activation Addition (CAA) method achieved a 64.72% safety defense rate (DR) on Gemma-2-9B, surpassing the 58.29% baseline DR. v) For AI practitioners, EasyEdit2 offers a modular system for applying fine-grained, test-time control over LLM outputs with minimal code, aiding in model alignment, debugging, and customization for specific application requirements. |
| LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration |
|
|
| Benchmark (Read more on arXiv or HuggingFace) |
dkeeeee, Yuxiang007, zhimingc, Pengxiangzhao, lgy0404 |
This paper presents LearnAct, a few-shot learning framework, and LearnGUI, a benchmark, to improve mobile GUI agent generalization using human demonstrations. The primary objective is to enhance mobile GUI agent capabilities in handling diverse, unseen scenarios and user-specific tasks by learning from a small number of examples, addressing limitations of traditional pre-training or large-scale fine-tuning. The key methodology involves the LearnAct multi-agent framework (DemoParser for knowledge extraction, KnowSeeker for retrieval, ActExecutor for execution) and the LearnGUI benchmark dataset containing offline/online tasks with human demonstrations and similarity metrics. Primary results demonstrate significant performance gains; notably, a single demonstration improved Gemini-1.5-Pro’s offline accuracy from 19.3% to 51.7%, and LearnAct boosted UI-TARS-7B-SFT’s online success rate from 18.1% to 32.8%. For AI practitioners, this work implies that incorporating few-shot demonstration-based learning is a viable strategy to create more adaptable and deployable mobile GUI agents, reducing reliance on extensive datasets for personalization and handling long-tail tasks. |
| LookingGlass: Generative Anamorphoses via Laplacian Pyramid Warping (Read more on arXiv or HuggingFace) |
Vinicius C. Azevedo, Jingwei Tang, coffeeweb2907, ssancho, pascalchang87 |
This paper introduces LookingGlass, a method using latent rectified flow models and a novel Laplacian Pyramid Warping technique to generate anamorphic images that reveal hidden content via specific viewpoints while maintaining a valid direct interpretation. The objective is to extend generative optical illusions to latent space models and complex spatial transformations beyond simple orthogonal warps, using only text prompts. The core methodology involves synchronizing latent flow model predictions across views by decoding to image space, applying frequency-aware Laplacian Pyramid Warping (LPW) for robust transformation and blending, encoding back to latent space, and using residual correction. Primary results demonstrate high-quality anamorphosis generation for conic/cylindrical mirrors and Nicéron’s lens, quantitatively outperforming prior methods on complex transforms (e.g., achieving FID 129.74 vs 166.03+ for 135° rotation). For AI practitioners, this work presents a feed-forward approach for creating intricate perceptual illusions with modern latent generative models and introduces LPW, a generally applicable technique for high-fidelity, frequency-aware image warping in generative tasks. |
| DRAGON: Distributional Rewards Optimize Diffusion Generative Models (Read more on arXiv or HuggingFace) |
Somayeh Sojoudi, Jonah Casebeer, Njb, Bai-YT |
DRAGON introduces a versatile on-policy framework for fine-tuning diffusion models using distributional rewards beyond standard instance-level feedback. The objective is to enable optimization for a wider class of reward functions, including instance-wise, instance-to-distribution, and distribution-to-distribution metrics, such as FAD or Vendi diversity. DRAGON operates by generating on-policy samples, evaluating them with the target reward function to construct positive (D+) and negative (D-) demonstration sets, and then applying contrastive optimization losses (like Diffusion-DPO/KTO) to align the model’s output distribution. Experiments fine-tuning a text-to-music model showed DRAGON achieved an 81.45% average win rate across 20 diverse reward functions, and significantly improved human-perceived quality (60.95% human-voted win rate) by optimizing FAD with an appropriate exemplar set, without needing human preference annotations. For AI practitioners, DRAGON provides a method to directly optimize generative models for complex distributional metrics like FAD and enables using easily obtainable reference data (even cross-modal, like text descriptions for music) to improve generation quality, reducing reliance on costly human feedback collection. |
| Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls |
|
|
| for Video Generation (Read more on arXiv or HuggingFace) |
Shikai Li, yanweifuture, Alex-snow, theFoxofSky, ewrfcas |
Uni3C introduces a unified framework for precise 3D-enhanced camera and human motion control in video generation using foundational video diffusion models (VDMs). The objective is to enable joint, precise control over both camera trajectories and human motions in video generation, overcoming limitations of separate controls and reliance on jointly annotated data. Key methodologies include PCDController, a lightweight, plug-and-play module trained with a frozen VDM backbone using unprojected point clouds for camera control, and a global 3D world guidance system aligning scenic point clouds and SMPL-X characters for unified inference. Uni3C significantly improves joint control, achieving an Absolute Trajectory Error (ATE) of 0.251 on the unified benchmark, substantially outperforming the baseline RealisDance-DiT’s ATE of 0.549 while maintaining visual quality. For AI practitioners, the PCDController offers a robust, parameter-efficient module for adding precise camera control to existing VDMs with minimal training overhead and without needing joint annotations, while the global alignment enables unified multi-modal control. |
| TAPIP3D: Tracking Any Point in Persistent 3D Geometry (Read more on arXiv or HuggingFace) |
Katerina Fragkiadaki, Bowei Zhang, aharley, lkeab |
TAPIP3D introduces a method for long-term 3D point tracking by representing videos as camera-stabilized spatio-temporal 3D feature clouds. The objective is to improve long-term 3D point tracking accuracy and robustness, particularly under complex deformations and large camera movements, by leveraging persistent 3D world-space representations. The methodology involves lifting 2D video features using depth and optional camera pose into a 3D point feature cloud (world or camera coordinates), employing a novel Local Pair Attention mechanism for contextualization, and iteratively refining 3D trajectories via a transformer. Results show state-of-the-art performance, significantly outperforming prior methods; for example, on the LSFOdyssey benchmark using ground-truth depth and camera pose, TAPIP3D-world achieved 72.2 AJ3D compared to 37.7 AJ3D for the DELTA baseline. For AI practitioners, this work demonstrates that utilizing explicit 3D world-space coordinates and 3D-specific attention mechanisms can yield substantial improvements in tracking accuracy for applications requiring fine-grained motion understanding, especially when reliable depth and pose are accessible. |
| An LMM for Efficient Video Understanding via Reinforced Compression of |
|
|
| Video Cubes (Read more on arXiv or HuggingFace) |
Yuan Yao, Ji Qi, chuats, acharkq, bys0318 |
Quicksviewer introduces a Large Multimodal Model (LMM) employing nonuniform partitioning and resampling for efficient video understanding. The primary objective is to create an LMM that dynamically compresses videos based on temporal information density, reducing redundancy for efficient long-video processing. Key methodology involves a cubing network using Gumbel Softmax to partition videos into nonuniform cubes, followed by a unified 3D resampler compressing each cube into a fixed number of tokens, achieving an overall 45x compression rate. Quicksviewer outperformed a fixed partitioning baseline by up to 8.72 in accuracy and achieved SOTA on Video-MME using significantly fewer tokens per frame (up to 5% of baseline needs). For AI practitioners, this reinforced dynamic cubing approach offers a method to develop computationally efficient LMMs for long video analysis, drastically reducing token requirements while maintaining strong performance. |
| RainbowPlus: Enhancing Adversarial Prompt Generation via Evolutionary |
|
|
| Quality-Diversity Search (Read more on arXiv or HuggingFace) |
Truong-Son Hy, tnngo2, quyanh |
RainbowPlus introduces a novel evolutionary quality-diversity (QD) framework to enhance adversarial prompt generation for Large Language Model (LLM) red-teaming. The primary objective is to improve the scalability, effectiveness, and diversity of attack strategies compared to existing methods. It employs an adaptive QD search based on MAP-Elites, featuring a multi-element archive storing multiple prompts per cell and a probabilistic fitness function for concurrent multi-prompt evaluation. RainbowPlus demonstrated superior performance, achieving an average Attack Success Rate (ASR) of 81.1% on the HarmBench dataset across twelve LLMs, surpassing AutoDAN-Turbo by 3.9% while being 9 times faster. For AI practitioners, RainbowPlus offers a more scalable and computationally efficient open-source tool for comprehensive LLM vulnerability assessment and safety enhancement. |
| NEMOTRON-CROSSTHINK: Scaling Self-Learning beyond Math Reasoning (Read more on arXiv or HuggingFace) |
yejinchoinka, ericnyberg, ekmb, shrimai19, SieraL |
NEMOTRON-CROSSTHINK proposes a framework to scale reinforcement learning-based self-learning for LLMs beyond mathematics by systematically incorporating multi-domain, multi-format data. The primary objective is to generalize RL-enhanced reasoning capabilities to diverse non-math domains (STEM, humanities, social sciences) where verifiable reward structures are less defined than in mathematics. The methodology involves curating multi-source QA data, applying structured answer templates (MCQ/Open-Ended), filtering for verifiability, optimizing data blending ratios, and employing Group Relative Policy Optimization (GRPO) for RL training. This framework achieved substantial accuracy gains over baselines on both math (MATH-500: +30.1%) and non-math benchmarks (MMLU-PRO: +12.8%), with the multi-domain blend notably improving response efficiency by using 28% fewer tokens for correct general-purpose reasoning answers compared to a math-only RL model. For AI practitioners, the principal implication is that incorporating diverse, multi-domain data with appropriate formatting and filtering into RL pipelines is crucial for enhancing LLM reasoning generalization and inference efficiency, moving beyond math-centric training paradigms. |
| CoMotion: Concurrent Multi-person 3D Motion (Read more on arXiv or HuggingFace) |
Stephan R. Richter, Alejandro Newell, vkoltun, lahavl, peiyun-hu-apple |
CoMotion introduces an online approach for concurrent 3D pose estimation and tracking of multiple people from a single monocular video stream. The primary objective is to maintain temporally coherent and accurate 3D pose tracks for multiple individuals, even in crowded scenes with occlusions, in a streaming fashion. The methodology employs a recurrent model using a tracking-by-attention paradigm, directly updating existing pose tracks from image features via cross-attention and a GRU, alongside a module for detecting new tracks, trained on a heterogeneous mix of real and synthetic datasets with pseudo-labels. CoMotion achieves state-of-the-art pose accuracy and significantly improves tracking, notably increasing MOTA by 14% and IDF1 by 12% on PoseTrack21 over prior methods while being substantially faster. For AI practitioners, this demonstrates that directly updating tracks from image features enables more robust and efficient online multi-person 3D motion tracking compared to traditional detect-and-associate methods. |
Papers for 2025-04-21
| Title |
Authors |
Summary |
| Does Reinforcement Learning Really Incentivize Reasoning Capacity in |
|
|
| LLMs Beyond the Base Model? (Read more on arXiv or HuggingFace) |
Zhaokai Wang, Andrew Zhao, Rui Lu, Zhiqi Chen, Yang Yue |
This paper demonstrates that Reinforcement Learning with Verifiable Rewards (RLVR) primarily enhances sampling efficiency for existing reasoning paths within LLMs, rather than fundamentally expanding reasoning capacity beyond the base model. The study critically investigates if RLVR enables LLMs to acquire novel reasoning abilities exceeding their base models’ intrinsic capabilities. Using the pass@k metric with large k values across math, coding, and visual reasoning benchmarks, alongside perplexity analysis and manual Chain-of-Thought checks, the researchers compared the reasoning boundaries of base and RL-trained models. Key findings reveal that while RL models excel at low k (pass@1), base models consistently match or surpass RL models at high k (e.g., base Minerva 32B outperformed its RL counterpart by ~9% pass@128), indicating RL primarily learns to sample pre-existing correct reasoning paths more efficiently, rather than discovering new ones. For AI practitioners, this implies current RLVR mainly optimizes known reasoning patterns rather than fostering new skills, suggesting that achieving breakthroughs in reasoning might require complementary methods like distillation or fundamentally different training paradigms that overcome RL’s observed limitation in narrowing exploration. |
| MIG: Automatic Data Selection for Instruction Tuning by Maximizing |
|
|
| Information Gain in Semantic Space (Read more on arXiv or HuggingFace) |
Haochen Ye, Zerun Ma, Kai Hu, Yining Li, Yicheng Chen |
This paper introduces MIG, an automatic method for selecting instruction-tuning data by maximizing information gain within a semantic label graph representation. Its objective is to unify the quantification of data quality and diversity for efficient subset selection from large pools, overcoming limitations of prior heuristic and embedding-based techniques. MIG models semantic relationships via a label graph, uses a submodular information function considering propagation effects, and iteratively selects samples via a greedy, gain-maximizing algorithm. Key results demonstrate MIG’s superiority over baselines; notably, Llama3.1-8B tuned with just 5% of Tulu3 data selected by MIG improved performance over the model trained on the full dataset by +5.73% on AlpacaEval and +6.89% on Wildbench. This provides AI practitioners an efficient, automated approach to curate smaller yet highly effective instruction-tuning datasets, potentially reducing training costs while improving model alignment and outperforming methods relying solely on embedding distance or heuristics. |
| Could Thinking Multilingually Empower LLM Reasoning? (Read more on arXiv or HuggingFace) |
Lei Li, Shujian Huang, Wenhao Zhu, Xu Huang, Changjiang Gao |
This paper investigates harnessing multilingualism to enhance Large Language Model (LLM) reasoning capabilities. The primary objective is to quantify the potential performance upper-bound of multilingual reasoning compared to English-only approaches. Key methodology involves aggregating LLM responses (LLaMA3.1-70B, Qwen2.5-72B, R1-distill-LLaMA3.1-70B) to parallel inputs translated into 17 languages on GPQA and MGSM datasets, measuring the Acc@k upper bound. Results demonstrate that multilingual thinking significantly surpasses English-only baselines (Repeat/Paraphrase), boosting the Acc@k upper bound by nearly 10 points (e.g., GPQA from ~45 to ~90 Acc@17). For AI practitioners, this highlights substantial untapped potential in leveraging diverse languages for reasoning, though current answer selection methods (majority voting, prompt-based, LLM-as-judge) fail to fully realize this potential gain. |
| AerialMegaDepth: Learning Aerial-Ground Reconstruction and View |
|
|
| Synthesis (Read more on arXiv or HuggingFace) |
Shubham Tulsiani, Srinivasa Narasimhan, Deva Ramanan, Anurag Ghosh, kvuong2711 |
This paper introduces AerialMegaDepth, a hybrid dataset for improving aerial-ground 3D reconstruction and view synthesis. The objective is to overcome the failure of learning-based methods to handle extreme viewpoint variations between aerial and ground images due to a lack of suitable training data. The methodology combines pseudo-synthetic aerial renderings from 3D city meshes (Google Earth) with co-registered real ground-level images (MegaDepth) into a unified coordinate system. Fine-tuning the DUSt3R model on AerialMegaDepth significantly improved ground-aerial camera registration, increasing the Relative Rotation Accuracy @5° from under 5% to nearly 56%. AI practitioners can utilize this framework and dataset to develop models robust to drastic viewpoint differences for tasks like cross-view 3D reconstruction and novel view synthesis, addressing a key limitation in existing large-scale datasets. |
| HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation (Read more on arXiv or HuggingFace) |
Tao Hu, Yuan Li, Zesong Yang, Bangbang Yang, Wenqi Dong |
HiScene introduces a hierarchical framework for generating editable, compositional 3D scenes from text prompts by leveraging isometric view generation. The main objective is to create high-fidelity 3D scenes with natural layouts, complete object instances, and interactive editing capabilities, overcoming limitations of prior methods. Its methodology involves initializing a scene from an isometric view using a native 3D generator, performing hierarchical scene parsing with 3D segmentation, applying video-diffusion-based amodal completion to handle occlusions, and using spatial alignment with shape prior injection for object regeneration. Experimental results show HiScene outperforms methods like GALA3D and DreamScene in user studies (Overall Quality score: 2.76 vs 1.75/1.73) and metrics, with its amodal completion achieving 83.84 mIoU on COCO-A, surpassing prior zero-shot methods. This research provides AI practitioners with a method to generate complex, editable 3D scenes from text, improving workflow for interactive applications, and introduces a robust video-diffusion technique for amodal completion relevant to 3D perception. |
| NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodes (Read more on arXiv or HuggingFace) |
Yixin Liu, Haoxiang Chen, Chengze Li, Haojie Zheng, Tianyang Xu |
This paper introduces NodeRAG, a framework optimizing retrieval-augmented generation by structuring knowledge into a carefully designed heterogeneous graph with distinct node types. The main objective is to demonstrate how this specific graph structure enhances graph-based RAG performance, particularly for multi-hop reasoning and summary-level queries, compared to methods with less considered structures. NodeRAG utilizes LLMs to decompose text into seven distinct node types (Entity, Relationship, Semantic Unit, Attribute, High-Level, Overview, Text), augments the graph via community detection and node importance metrics, and employs graph algorithms like shallow Personalized PageRank with a dual (exact + vector) search for retrieval. Primarily, NodeRAG achieves superior accuracy on multi-hop benchmarks, such as 46.29% on MuSiQue (compared to 41.71% for GraphRAG), while using fewer retrieval tokens. For AI practitioners, NodeRAG provides a concrete methodology for designing graph indices that yield more precise, efficient, and explainable retrieval, significantly improving RAG system performance and reducing operational costs associated with token usage for complex information synthesis tasks. |
| It’s All Connected: A Journey Through Test-Time Memorization, |
|
|
| Attentional Bias, Retention, and Online Optimization (Read more on arXiv or HuggingFace) |
Vahab Mirrokni, Peilin Zhong, Meisam Razaviyayn, Ali Behrouz |
This paper reconceptualizes sequence models like Transformers and linear RNNs as associative memory modules optimizing an internal “attentional bias” objective, introducing the MIRAS framework. The main objective is to define an underlying design framework for these models by integrating concepts of associative memory, attentional bias, retention mechanisms, and online optimization. Key methodology involves proposing MIRAS, characterized by four design choices (memory architecture, attentional bias, retention gate, memory algorithm), and developing three new models (MONETA, YAAD, MEMORA) using novel biases (e.g., lp-loss, Huber) and retention gates (e.g., Lq, KL divergence). Primary results demonstrate that these MIRAS variants outperform state-of-the-art baselines on language modeling, commonsense reasoning, and recall tasks, with the 1.3B parameter YAAD model achieving 15.18 perplexity on Wikitext compared to 18.53 for Transformer++. For AI practitioners, the principal implication is that MIRAS offers a structured method to design sequence backbones by explicitly selecting attentional bias and retention mechanisms beyond standard L2/dot-product approaches, enabling targeted optimization for tasks demanding specific capabilities like long-context handling or robustness. |
| Tokenize Image Patches: Global Context Fusion for Effective Haze Removal |
|
|
| in Large Images (Read more on arXiv or HuggingFace) |
Kaiqi Li, Qizhi Xu, Jiuchen Chen, fengyanzi |
This paper introduces DehazeXL, an efficient end-to-end method for removing haze from large, high-resolution images by balancing global context and local features. The main objective is to overcome GPU memory limitations typically encountered when processing large images for dehazing, without resorting to performance-degrading slicing or downsampling. DehazeXL partitions the input image into patches, encodes them locally, fuses global context using an efficient global attention bottleneck inspired by large language models, and decodes patches asynchronously in mini-batches. Key results demonstrate that DehazeXL can process images up to 10240x10240 pixels using only 21 GB of GPU memory (FP16 inference), achieving state-of-the-art PSNR (32.35) and SSIM (0.9863) on the introduced 8KDehaze dataset. For AI practitioners, the primary implication is a validated, memory-efficient architecture enabling the application of complex image restoration models to ultra-high-resolution inputs on mainstream hardware, crucial for remote sensing or surveillance applications. |
| Thought Manipulation: External Thought Can Be Efficient for Large |
|
|
| Reasoning Models (Read more on arXiv or HuggingFace) |
Wenhan Dong, Zifan Peng, Zhen Sun, Jingyi Zheng, Yule Liu |
This paper introduces ThoughtMani, a training-free pipeline using external Chain-of-Thought (CoT) from smaller models to improve the inference efficiency of Large Reasoning Models (LRMs). The main objective is to mitigate LRM “overthinking” and reduce computational costs by bypassing unnecessary reasoning steps without fine-tuning. The core methodology involves inserting CoTs generated by a smaller model between specific thinking tokens (<think>, </think>) in the LRM’s prompt to guide its generation. Key results demonstrate significant efficiency gains; for example, applying ThoughtMani to QwQ-32B on LiveBench/Code reduced output token counts by approximately 30% while maintaining performance and improving safety alignment by an average of 10%. For AI practitioners, ThoughtMani provides a practical, low-overhead method to make LRMs more computationally efficient and accessible for real-world applications, particularly when deploying different model sizes concurrently. |
Papers for 2025-04-18
| Title |
Authors |
Summary |
| CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for |
|
|
| Language Model Pre-training (Read more on arXiv or HuggingFace) |
Dan Su, Xin Dong, Yonggan Fu, Yu Yang, shizhediao |
CLIMB introduces an automated framework using clustering and iterative bootstrapping to optimize language model pre-training data mixtures. The main objective is to automatically discover, evaluate, and refine optimal data mixtures from large-scale corpora without manual curation or predefined domain labels to improve pre-training performance. The methodology involves embedding and clustering documents, followed by an iterative process that samples mixture configurations, trains proxy models, fits a performance predictor, and prunes the search space to find optimal weights. Primary results show that a 1B model trained continuously on 400B tokens using the CLIMB-optimized mixture (ClimbMix) surpassed the Llama-3.2-1B model by 2.0% on average across 12 reasoning benchmarks. The principal implication for AI practitioners is that CLIMB provides a data-driven, automated approach to curate high-quality pre-training datasets from unlabeled web-scale data, demonstrably improving model performance under fixed token budgets compared to baseline mixtures or random sampling, as evidenced by the released ClimbMix dataset. |
| Antidistillation Sampling (Read more on arXiv or HuggingFace) |
Avi Schwarzschild, Zhili Feng, Asher Trockman, arobey1, yashsavani |
This paper introduces antidistillation sampling, a technique to generate reasoning traces from large language models (LLMs) that hinder model distillation while maintaining the original model’s performance. The primary objective is to develop a sampling strategy that poisons generated data for distillation purposes, thereby protecting proprietary model capabilities, without sacrificing the utility of the model’s outputs for downstream tasks. The key methodology involves modifying the teacher model’s next-token sampling distribution by adding a penalty term proportional to an approximation of how a sampled token would increase a proxy student model’s downstream loss, calculated efficiently via a finite difference approximation of a directional derivative. Results show that for comparable teacher model accuracy on GSM8K (around 68-69%), antidistillation sampling reduced the distilled student model’s accuracy to 24.73%, significantly lower than the 51.86% achieved by a student distilled from traces generated via standard temperature sampling. For AI practitioners, this method offers a way to protect intellectual property embedded in frontier models by degrading the effectiveness of distillation when sharing model outputs, such as extended reasoning traces, while largely preserving the original model’s task performance. |
| A Strategic Coordination Framework of Small LLMs Matches Large LLMs in |
|
|
| Data Synthesis (Read more on arXiv or HuggingFace) |
Honglin Lin, Yu Li, Zinan Tang, Qizhi Pei, GX-XinGao |
A coordination framework (GRA) using multiple small LLMs achieves data synthesis quality comparable to single large LLMs. The research objective is to design a resource-efficient framework enabling small LLMs to collectively match the data synthesis capabilities of monolithic LLMs without their associated high costs and limitations. GRA employs a peer-review-inspired methodology assigning distinct Generator, Reviewer, and Adjudicator roles to multiple small LLMs for iterative data generation, evaluation, and quality control. Primary results show GRA-produced data matches or surpasses large LLM quality; data synthesized using GRA with a Qwen-2.5-7B base model outperformed Qwen-2.5-72B-Instruct distilled data by 8.83% on average across tested benchmarks. The principal implication for AI practitioners is that strategically coordinating smaller models offers a computationally efficient alternative for generating high-quality synthetic training data, reducing reliance on large models for data synthesis and distillation. |
| Packing Input Frame Context in Next-Frame Prediction Models for Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Maneesh Agrawala, Lvmin Zhang |
This paper presents FramePack, a structure for next-frame video prediction that compresses input frames to maintain a fixed transformer context length. The primary objective is to mitigate the “forgetting” (fading memory) and “drifting” (error accumulation) problems in generating long videos. FramePack employs progressive compression using varying transformer patchify kernel sizes based on frame importance and introduces anti-drifting sampling methods like inverted temporal ordering for bi-directional context. Results show that finetuning existing models with FramePack, especially using the inverted anti-drifting sampling (e.g., f1k1_x_g9_f1k1f2k2f16k4_td configuration), achieves superior performance across multiple metrics, including the highest human assessment ELO score of 1239 in ablation studies. For AI practitioners, FramePack offers a method to train video generation models capable of handling longer sequences with significantly higher batch sizes and reduced error accumulation, potentially improving visual quality and training efficiency. |
| Generate, but Verify: Reducing Hallucination in Vision-Language Models |
|
|
| with Retrospective Resampling (Read more on arXiv or HuggingFace) |
Trevor Darrell, Joseph E. Gonzalez, Jiaxin Ge, Heekyung Lee, tsunghanwu |
This paper introduces REVERSE, a unified framework reducing visual hallucinations in Vision-Language Models (VLMs) via hallucination-aware training and retrospective resampling. The objective is to enable a single VLM to both detect and dynamically correct its own hallucinations during text generation, unifying generation adjustment and post-hoc verification. Key methodology involves fine-tuning VLMs on a new 1.3M semi-synthetic dataset annotated with confidence tokens (</CN>, </UN>) and employing inference-time retrospective resampling triggered by token uncertainty to backtrack and regenerate content. Primary results demonstrate state-of-the-art performance, achieving up to a 12% reduction in CHAIR scores on CHAIR-MSCOCO compared to previous best methods. For AI practitioners, REVERSE offers a novel technique to enhance VLM reliability by embedding self-verification and correction capabilities directly into the model, reducing reliance on external verifiers or complex multi-stage pipelines. |
| WORLDMEM: Long-term Consistent World Simulation with Memory (Read more on arXiv or HuggingFace) |
Shuai Yang, Wenqi Ouyang, Yifan Zhou, Yushi Lan, Zeqi Xiao |
WORLDMEM introduces a memory-augmented framework for long-term consistent world simulation, addressing temporal limitations in existing video diffusion models. The primary research objective is to mitigate the lack of long-term 3D spatial consistency in generative world simulators caused by limited temporal context windows. The methodology integrates an external memory bank (storing past frames with pose and timestamp states) into a Conditional Diffusion Transformer, using memory attention with relative state embeddings (Plücker for pose) and Diffusion Forcing to condition generation on retrieved memories. Quantitative results demonstrate improved consistency; for instance, on a Minecraft benchmark beyond the context window, WORLDMEM achieved a PSNR of 25.32 and LPIPS of 0.1429, significantly outperforming a Diffusion Forcing baseline (PSNR 18.04, LPIPS 0.4376). For AI practitioners, this approach offers a method to build more persistent and spatially coherent interactive simulations or virtual environments where maintaining state over extended periods is critical. |
| VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference |
|
|
| Optimization for Large Video Models (Read more on arXiv or HuggingFace) |
Meng Luo, Haojian Huang, scofield7419, ChocoWu, Harold328 |
VistaDPO introduces a hierarchical spatial-temporal direct preference optimization framework to enhance large video models (LVMs). The primary objective is to address LVM misalignment with human intuition and video hallucination by optimizing text-video preference alignment across instance, temporal, and perceptive hierarchical levels. The key methodology involves applying this hierarchical DPO framework, termed VistaDPO, using a newly constructed VistaDPO-7k dataset (7.2K QA pairs) annotated with chosen/rejected responses and spatial-temporal grounding information. Experimental results show VistaDPO significantly improves baseline LVMs, achieving average performance gains of 26.42% over PLLaVA and 53.92% over Video-LLaVA across hallucination, QA, and captioning benchmarks. For AI practitioners, this work demonstrates that incorporating hierarchical spatial-temporal preference optimization, beyond simple instance-level DPO, is crucial for improving the reliability and reducing hallucinations in LVMs for complex video understanding tasks. |
| NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation (Read more on arXiv or HuggingFace) |
Chao Du, Zijian Wu, Jinjie Ni, Xiangyan Liu, dreamerdeo |
NoisyRollout introduces an RL fine-tuning approach for VLMs that enhances visual reasoning by incorporating trajectories from distorted images during rollout collection. The objective is to improve policy exploration diversity and mitigate issues arising from imperfect visual perception in VLMs without additional training costs. The key methodology involves a hybrid rollout strategy within GRPO, using both clean and noise-distorted images (with noise annealing) to generate trajectories for reward calculation, while policy updates use only clean images. Using just 2.1K samples, NoisyRollout achieved state-of-the-art average accuracy of 59.2% across five out-of-domain benchmarks compared to similar open-source RL-tuned models. For AI practitioners, this work demonstrates that targeted data augmentation during RL rollouts can effectively boost VLM generalization and robustness, particularly for visual reasoning, offering a cost-effective method to enhance exploration. |
| ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question |
|
|
| Answering (Read more on arXiv or HuggingFace) |
Firoz Kabir, Aayush Bajaj, Mahir Ahmed, 38saidul, ahmed-masry |
This paper introduces ChartQAPro, a diverse and challenging benchmark for Chart Question Answering (CQA). The primary objective was to address the limitations of existing CQA benchmarks, such as lack of diversity and performance saturation, and provide a more realistic evaluation of Large Vision-Language Models (LVLMs). The authors constructed ChartQAPro by collecting 1,341 charts from 157 diverse sources, including infographics and dashboards, paired with 1,948 human-verified questions covering multiple complex types like conversational and hypothetical queries. Evaluations on 21 LVLMs revealed a substantial performance decrease on ChartQAPro compared to prior benchmarks; for instance, Claude Sonnet 3.5’s accuracy dropped from 90.5% on ChartQA to 55.81% on ChartQAPro. For AI practitioners, this implies that current LVLMs struggle significantly with complex, real-world chart reasoning, and ChartQAPro serves as a more robust tool for identifying these limitations and guiding future model development. |
| Exploring Expert Failures Improves LLM Agent Tuning (Read more on arXiv or HuggingFace) |
Ruochen Wang, Minhao Cheng, Andrew Bai, Li-Cheng Lan, zhoutianyi |
This paper introduces Exploring Expert Failures (EEF), a fine-tuning method that improves LLM agent performance by utilizing information from failed expert trajectories. The objective is to address the limitation of Rejection Sampling Fine-Tuning (RFT), which discards failed expert trajectories, causing agents to struggle with complex, out-of-distribution subtasks where experts often fail. EEF simulates intermediate states from failed expert trajectories using the current agent policy, identifies beneficial action sequences leading to success via simulation, and selectively incorporates only these validated segments into the training data for supervised fine-tuning. The primary result shows EEF achieved a 62% win rate on the WebShop benchmark, significantly outperforming RFT (53.6%) and setting a new state-of-the-art score above 0.81. For AI practitioners, this implies that analyzing and selectively leveraging segments from failed expert demonstrations, rather than discarding them entirely, provides valuable training signals that enhance agent capabilities on complex tasks and improve overall tuning efficiency. |
| InstantCharacter: Personalize Any Characters with a Scalable Diffusion |
|
|
| Transformer Framework (Read more on arXiv or HuggingFace) |
Yiji Cheng, Qixun Wang, Yanbing Zhang, Jiale Tao, wanghaofan |
InstantCharacter presents a scalable diffusion transformer framework designed for high-fidelity, open-domain character personalization in image generation. The primary objective is to address the limited generalization, compromised image quality, and reduced textual controllability inherent in previous U-Net based or optimization-based character customization approaches, especially when applied to large Diffusion Transformers (DiTs). Methodologically, it introduces a scalable adapter with stacked transformer encoders, integrating features from SigLIP and DINOv2 via dual-stream fusion and a timestep-aware Q-former, trained progressively in three stages on a 10-million sample dataset containing paired and unpaired character images. Qualitative results demonstrate superior performance in maintaining character identity, fidelity, and text controllability compared to prior art like OminiControl, EasyControl, ACE++, and UNO, achieving comparable results to GPT4o, though specific quantitative metrics are not detailed in the provided text. For AI practitioners, this research offers a robust architecture and training strategy for adapting large foundation DiT models to specialized, controllable generation tasks like character personalization, enhancing flexibility and output quality without requiring test-time fine-tuning. |
| CCMNet: Leveraging Calibrated Color Correction Matrices for Cross-Camera |
|
|
| Color Constancy (Read more on arXiv or HuggingFace) |
Seon Joo Kim, Michael S. Brown, Dongyun Kim, Mahmoud Afifi, dongyong2 |
CCMNet introduces a lightweight framework utilizing pre-calibrated Color Correction Matrices (CCMs) for zero-shot cross-camera color constancy. The objective is to enable accurate illuminant estimation on unseen cameras without retraining or needing additional test images. The methodology involves using CCMs to map standard illuminants to the camera’s raw space, encoding this trajectory into a Camera Fingerprint Embedding (CFE) via a CNN, and using this CFE to guide a hypernetwork (based on CCC/C5) for predicting illumination from uv-histograms; imaginary camera augmentation further improves robustness. CCMNet achieves state-of-the-art results, such as a 1.68° mean angular error on Cube+, outperforming previous methods while being computationally efficient. For AI practitioners, this provides a method to achieve consistent color rendering across diverse camera hardware by leveraging readily available ISP metadata (CCMs), eliminating the need for per-camera calibration data or model fine-tuning. |
| FocusedAD: Character-centric Movie Audio Description (Read more on arXiv or HuggingFace) |
Liangcheng Li, Sheng Zhou, Yiren Song, Chun Wang, Xiaojun Ye |
FocusedAD introduces a novel framework for generating character-centric movie audio descriptions (AD) emphasizing narrative relevance. The main objective is to automatically produce AD for movies that explicitly identifies characters by name and focuses on plot-significant visual details, unlike generic video captioning. The methodology integrates a Character Perception Module (CPM) using an automated clustering-based query bank for character identification/tracking, a Dynamic Prior Module (DPM) injecting context via soft prompts, and a Focused Caption Module (FCM) generating descriptions from scene, character, and text tokens. FocusedAD achieves state-of-the-art performance on multiple benchmarks, including a BertScore of 57.7 on MAD-eval-Named and 64.5 on the introduced Cinepile-AD dataset, significantly outperforming prior AD methods and general MLLMs. For AI practitioners, this work provides a method for enhancing MLLM-based video understanding by incorporating specialized modules for character focus and contextual integration, leading to more narratively coherent and targeted outputs relevant for accessibility tools. |
| Retrieval-Augmented Generation with Conflicting Evidence (Read more on arXiv or HuggingFace) |
Mohit Bansal, Elias Stengel-Eskin, Archiki Prasad, HanNight |
This paper introduces RAMDocs, a dataset for evaluating RAG systems against simultaneous ambiguity, misinformation, and noise, and proposes MADAM-RAG, a multi-agent debate framework to handle such conflicts. The main objective is to develop and evaluate a RAG approach capable of managing diverse, concurrent sources of conflict in retrieved documents, a common challenge in real-world scenarios. The key methodology involves assigning individual documents to LLM agents who debate their validity over multiple rounds, followed by an aggregator agent synthesizing a final response based on the discussion. MADAM-RAG significantly outperforms strong RAG baselines, improving accuracy by up to 11.40% on AmbigDocs and 15.80% on FaithEval using Llama3.3-70B-Instruct, while the new RAMDocs dataset proves challenging for existing methods. For AI practitioners, this indicates that standard RAG pipelines are insufficient for handling complex, realistic conflicts, and multi-agent debate frameworks like MADAM-RAG are needed to improve the reliability and factuality of RAG outputs when facing ambiguity, misinformation, and noise simultaneously. |
| Sleep-time Compute: Beyond Inference Scaling at Test-time (Read more on arXiv or HuggingFace) |
Sarah Wooders, Charles Packer, Yu Wang, Charlie Snell, Kevin Lin |
This paper introduces sleep-time compute, a technique allowing LLMs to pre-process context offline to reduce test-time compute requirements. The research aims to evaluate the efficacy of sleep-time compute in improving the accuracy vs. test-time compute trade-off for stateful reasoning tasks. The methodology involves modifying reasoning datasets (GSM-Symbolic, AIME) into stateful versions where context is processed during “sleep-time” before a query arrives, comparing this to standard test-time scaling. Key results show sleep-time compute reduces the test-time compute needed for equivalent accuracy by approximately 5x on Stateful GSM-Symbolic and Stateful AIME, and scaling sleep-time compute can further improve accuracy by up to 18% on Stateful AIME. For AI practitioners, this implies that in stateful applications with available context (e.g., coding agents, document QA), implementing sleep-time compute can significantly cut test-time latency and cost while maintaining or improving accuracy, particularly when future queries are predictable. |
| Set You Straight: Auto-Steering Denoising Trajectories to Sidestep |
|
|
| Unwanted Concepts (Read more on arXiv or HuggingFace) |
Adams Wai-Kin Kong, Yan Ren, Leyang Li, Shilin-LU |
This paper introduces ANT, a finetuning framework for concept erasure in text-to-image diffusion models that automatically guides denoising trajectories away from unwanted concepts. The primary objective is to overcome limitations of prior methods by enabling precise content modification during mid-to-late denoising stages without disrupting early-stage structural integrity or relying on heuristic anchor concepts. ANT utilizes a trajectory-aware loss function that reverses the classifier-free guidance condition direction only after a specific timestep (t’) and employs an augmentation-enhanced weight saliency map to identify and finetune only the most relevant parameters for erasure. ANT achieves state-of-the-art results, reducing inappropriate image detections (e.g., NSFW content) on the I2P benchmark to 23, significantly lower than prior methods, while maintaining competitive FID and CLIP scores on MS-COCO. For AI practitioners, ANT provides a more effective and robust finetuning method to build safer generative models by removing unwanted concepts with less impact on overall generative quality and without needing manual anchor selection. |
| Perception Encoder: The best visual embeddings are not at the output of |
|
|
| the network (Read more on arXiv or HuggingFace) |
Andrea Madotto, Jang Hyun Cho, Peize Sun, Po-Yao Huang, Daniel Bolya |
Perception Encoder (PE) introduces a state-of-the-art vision encoder family achieving top performance across diverse tasks using only scaled contrastive vision-language pretraining, finding optimal embeddings within intermediate network layers. The main objective was to investigate if a single, scalable contrastive pretraining approach could generate strong, general visual embeddings suitable for classification, retrieval, language modeling, and spatial tasks without complex multi-objective training. The key methodology involved developing a robust image pretraining recipe, creating a video data engine using synthetically generated captions for video finetuning, and introducing language and spatial alignment tuning methods to extract and adapt features from specific intermediate layers. Primary results show PE models achieve state-of-the-art performance; for instance, PEcoreG obtains 86.6% average zero-shot image classification accuracy, outperforming previous models, and its intermediate features rival specialized models like AIMv2 (language) and DINOv2 (spatial) before alignment tuning. The principal implication for AI practitioners is that powerful, general-purpose visual embeddings can be learned via scaled contrastive learning alone, but optimal performance on diverse downstream tasks necessitates extracting and aligning features from intermediate layers rather than solely relying on the final network output. |
Papers for 2025-04-17
| Title |
Authors |
Summary |
| ColorBench: Can VLMs See and Understand the Colorful World? A |
|
|
| Comprehensive Benchmark for Color Perception, Reasoning, and Robustness (Read more on arXiv or HuggingFace) |
zhoutianyi, jiuhai, shweta12, kweCobi, Fcr09 |
This paper introduces COLORBENCH, a benchmark to evaluate Vision-Language Models’ (VLMs) capabilities in color perception, reasoning, and robustness. The research aims to assess whether and how current VLMs understand and utilize color information compared to human abilities. Methodology involved creating a benchmark with 11 distinct tasks across 3 core dimensions (Perception, Reasoning, Robustness) grounded in real-world applications, and evaluating 32 VLMs of varying sizes and architectures. Results show that while larger models generally perform better, overall performance on COLORBENCH is low (e.g., top proprietary models achieve ~53.9% overall P&R accuracy pre-CoT), performance gaps are small, and color understanding appears neglected in VLM development. The principal implication for AI practitioners is that current VLMs exhibit critical limitations in color comprehension, underscoring the need for targeted improvements in model architecture and training, using COLORBENCH as a foundational evaluation tool. |
| BitNet b1.58 2B4T Technical Report (Read more on arXiv or HuggingFace) |
thegenerality, THU-CHUNXIA, buaahsh, hongyuw, shumingma |
This paper introduces BitNet b1.58 2B4T, an open-source, native 1.58-bit, 2-billion parameter LLM trained on 4 trillion tokens. The primary objective was to demonstrate that a native, scaled 1-bit LLM can achieve performance comparable to similar-sized open-weight, full-precision models while being significantly more computationally efficient. Methodology involved training a modified Transformer architecture from scratch, replacing standard linear layers with BitLinear layers using 1.58-bit (ternary {-1, 0, +1}) absolute mean weight quantization and 8-bit activation quantization, followed by SFT and DPO. Results show BitNet b1.58 2B4T achieves performance on par with leading 1-2B parameter full-precision LLMs across various benchmarks (e.g., average score 54.19 vs. 55.23 for Qwen2.5 1.5B) but requires substantially less memory (0.4GB non-embedding vs 2.6GB). For AI practitioners, this work presents a highly efficient LLM that rivals full-precision counterparts in performance, enabling deployment in resource-constrained environments and offering significant reductions in memory, energy, and latency compared to both full-precision and standard post-training quantized models. |
| Cobra: Efficient Line Art COlorization with BRoAder References (Read more on arXiv or HuggingFace) |
Zhaoyang Zhang, yshan2u, juxuan27, l-li, JunhaoZhuang |
Cobra introduces an efficient, long-context framework for high-fidelity, reference-based line art colorization supporting over 200 references while preserving identity details. The primary objective is to address limitations in existing diffusion models regarding extensive reference handling, inference latency, and flexible control in industrial comic colorization workflows. Key methodology includes a Causal Sparse DiT architecture leveraging Localized Reusable Position Encoding for arbitrary reference image counts and Causal Sparse Attention with KV-Cache to reduce computational complexity. Results show Cobra outperforms baselines on the Cobra-bench benchmark, achieving a FID of 20.98 compared to 26.29 for ColorFlow, while Causal Sparse Attention reduces per-step inference time from 1.99s (Full Attention) to 0.35s using 24 references. For AI practitioners, Cobra offers a scalable and efficient approach for integrating extensive visual context (hundreds of images) into generative tasks like colorization with significantly reduced latency compared to standard attention mechanisms. |
| AlayaDB: The Data Foundation for Efficient and Effective Long-context |
|
|
| LLM Inference (Read more on arXiv or HuggingFace) |
FeTieTer, YuanPeiqi, Qilong00, BenjaminXIANG, YangshenDeng |
AlayaDB is a vector database system architected to enhance long-context LLM inference efficiency and effectiveness by managing KV cache and attention computation externally. The primary objective is to simultaneously reduce GPU memory consumption and inference latency (TTFT and TPOT) while maintaining or improving generation quality for long-context tasks, addressing the limitations of coupled, disaggregated, and retrieval-based sparse attention approaches. Key methodologies include decoupling KV cache/attention from the LLM inference engine, introducing a Dynamic Inner Product Range (DIPR) query to dynamically select critical tokens for sparse attention, and employing a native query optimizer with specialized index structures and computation optimizations. Experiments demonstrate that AlayaDB achieves better average generation quality (47.0) on ∞-Bench compared to baseline methods like InfLLM (43.8) and Top-k (46.7), while meeting latency SLOs and significantly reducing TTFT by 19-42x compared to LMCache for context reuse. For AI practitioners, AlayaDB offers a data foundation that can lower hardware resource requirements and simplify the development of high-performing long-context LLM applications by abstracting complex cache management and attention computation. |
| SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction |
|
|
| Fine-Tuning (Read more on arXiv or HuggingFace) |
Jian Xie, Rupak Vignesh Swaminathan, svinxz, vijaygirish2001, panprabh |
This paper introduces SIFT-50M, a 50M-example, five-language dataset generated using LLMs from public speech corpora for speech instruction fine-tuning. The primary objective was to create a large-scale, diverse dataset to improve the instruction-following capabilities and generalization of speech-text LLMs beyond standard ASR tasks. Key methodology involved extracting detailed acoustic and content metadata from speech, mapping it to categorical values, and using LLMs (Mixtral 8x7B, Amazon Nova Pro) prompted with this metadata to generate varied instruction-response pairs, including closed-ended QA, open-ended analysis, and controllable generation prompts. The resulting SIFT-LLM model (Whisper-medium + Qwen2.5-7B), trained on SIFT-50M, achieved state-of-the-art performance on instruction-following benchmarks, notably scoring 57.4% accuracy on Dynamic-Superb (DS-1) closed-ended tasks, significantly outperforming prior models. For AI practitioners, SIFT-50M provides a substantial resource for training speech-text models that better comprehend and execute nuanced, multilingual instructions related to both speech understanding and controllable generation, alongside the EvalSIFT benchmark for systematic evaluation. |
| ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (Read more on arXiv or HuggingFace) |
chijx, imjcqt, YujiaHi, zhangysk, JoeYing |
ReTool is a reinforcement learning framework enhancing LLM mathematical reasoning by strategically integrating real-time code interpreter execution. The research objective is to teach LLMs when and how to leverage external computational tools effectively for complex reasoning tasks where pure text-based approaches falter. The methodology uses supervised fine-tuning on synthetic code-augmented data for initialization, followed by PPO-based reinforcement learning where task outcome accuracy serves as the reward signal during policy rollouts involving real-time code execution. Primary results show ReTool significantly boosts performance and efficiency, achieving 67.0% accuracy on AIME 2024 (400k steps) versus a text-only RL baseline (40.0%, 1080k steps), and exhibits emergent capabilities like code self-correction. For AI practitioners, this work shows outcome-driven RL effectively teaches LLMs strategic tool use, yielding more capable and efficient reasoning models for computational tasks without complex reward engineering or explicit tool-use supervision. |
| REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion |
|
|
| Transformers (Read more on arXiv or HuggingFace) |
liangzheng06, sainx, Zhenchang, yunzhong-hou, xingjianleng |
This paper introduces REPA-E, a method enabling joint end-to-end training of VAEs and latent diffusion transformers using representation alignment loss. The main objective is to develop an effective end-to-end training scheme for both the VAE tokenizer and the diffusion model, overcoming the performance degradation observed when using standard diffusion loss for joint training. REPA-E utilizes representation alignment (REPA) loss to jointly optimize VAE and diffusion model parameters, applying standard diffusion loss only to the diffusion model via stop-gradients, and incorporates batch normalization and VAE regularization. The proposed method significantly accelerates training, achieving an FID of 4.07 on ImageNet 256x256 in 400k steps (over 17x faster than the REPA baseline) and attains a state-of-the-art FID of 1.26 with classifier-free guidance. For AI practitioners, REPA-E offers a technique to drastically reduce latent diffusion model training time while simultaneously improving the VAE’s latent structure and final generative performance through joint optimization. |
| Vivid4D: Improving 4D Reconstruction from Monocular Video by Video |
|
|
| Inpainting (Read more on arXiv or HuggingFace) |
Yiyi Liao, BangBnag Yang, yuewenma, shengmiao, JaceyH919 |
Vivid4D enhances 4D reconstruction from monocular video by reformulating view augmentation as a video inpainting task integrating geometric and generative priors. The primary research objective is to improve the quality and completeness of 4D dynamic scene reconstruction from sparse monocular video inputs. Key methodology involves warping observed views to novel viewpoints using monocular depth priors, training a video diffusion model on unposed web videos with synthetic occlusion masks to inpaint missing regions, and employing an iterative view augmentation strategy with a robust reconstruction loss. Results demonstrate improved reconstruction quality, achieving an overall PSNR of 19.45 on the HyperNeRF dataset, outperforming baselines like 4D GS (18.24) and Shape of Motion (18.82). For AI practitioners, this work presents a practical method using video inpainting to generate richer supervision signals from monocular video, thereby enhancing the fidelity of 4D scene reconstructions for applications like VR/AR content creation. |
| Robust and Fine-Grained Detection of AI Generated Texts (Read more on arXiv or HuggingFace) |
ashay-sriv, jebish7, DrishtiSharma, Siddartha10, 1024m |
This paper presents robust token-classification models for fine-grained detection of AI-generated text, including human-LLM co-authored content. The main objective was to create detection systems resilient to unseen generators, domains, adversarial inputs, non-native speaker text, and shorter or partially AI-generated texts. The key methodology involved training multilingual transformer models (specifically xlm-longformer) with an additional CRF layer using a token-classification approach on a new, large dataset (~2.45M samples) of human-machine co-authored texts across 23 languages and 12 LLMs. Primary results include an average word-level accuracy of 94.19% on their diverse test set and demonstrating robustness against adversarial inputs on the raid-bench benchmark, achieving an F1 score of 0.79 without specific adversarial training. The principal implication for AI practitioners is that a token-classification approach trained on varied co-authored data significantly improves robustness for detecting AI text, particularly in mixed-authorship scenarios and against unseen generators or adversarial attacks, offering a more practical method than binary text classification. |
| Syzygy of Thoughts: Improving LLM CoT with the Minimal Free Resolution (Read more on arXiv or HuggingFace) |
Qigan Sun, Jiaquan Zhang, Yi Lu, Chaoning Zhang, Chenghao Li |
Syzygy of Thoughts (SoT) introduces a novel framework extending Chain-of-Thought (CoT) by incorporating Minimal Free Resolution (MFR) principles to enhance LLM reasoning. The objective is to improve the robustness and structure of LLM problem-solving for complex tasks by capturing deeper logical dependencies compared to standard CoT. The methodology leverages algebraic concepts like “Module”, “Betti numbers”, and “Minimality” to systematically decompose problems into minimal, logically complete subproblems and interrelated reasoning paths. Results demonstrate that SoT matches or surpasses CoT and CoT-SC accuracy across datasets like GSM8K and MATH; for instance, using GPT-4o-mini on GSM8K, SoT achieved 96.0% accuracy versus 85.1% for CoT. For AI practitioners, SoT provides a structured, mathematically-inspired approach to prompt engineering that can yield more reliable and transparent reasoning chains for complex tasks, potentially reducing errors and improving performance without relying solely on larger models. |
Papers for 2025-04-16
| Title |
Authors |
Summary |
| Genius: A Generalizable and Purely Unsupervised Self-Training Framework |
|
|
| For Advanced Reasoning (Read more on arXiv or HuggingFace) |
Haiteng Zhao, Chang Ma, Hang Yan, QiushiSun, xufangzhi |
Genius is a generalizable, purely unsupervised self-training framework designed to enhance Large Language Model (LLM) reasoning capabilities without external supervision. The central research objective is to advance LLM reasoning ability using only general, unlabeled queries, bypassing the need for annotated data or auxiliary reward models. Genius employs a stepwise foresight re-sampling strategy to sample candidate reasoning steps and estimate their value by simulating future outcomes, coupled with an Advantage-Calibrated Optimization (ACO) loss function to handle estimation noise and ensure robust optimization. Using only 25K unsupervised general queries from the Magpie dataset, Genius improved the average reasoning performance of LLaMA3.1-8B-Instruct by over 7% (from 49.65% to 57.08%) across seven reasoning benchmarks. For AI practitioners, this demonstrates a promising approach to scale LLM reasoning performance by leveraging vast amounts of readily available unlabeled data, potentially reducing dependency on expensive annotations and specialized reward models. |
| xVerify: Efficient Answer Verifier for Reasoning Model Evaluations (Read more on arXiv or HuggingFace) |
Bo Tang, Wentao Zhang, Pengyuan Wang, Duguce, Hush-cd |
This paper introduces xVerify, an efficient LLM-based answer verifier designed for evaluating reasoning models by accurately determining answer equivalence. The research aims to address the inadequacy of existing evaluation methods in extracting final answers and performing robust equivalence checks for complex, multi-step reasoning outputs from LLMs. Methodologically, the authors constructed the VAR dataset from 19 LLMs across 24 benchmarks, used multi-round GPT-4o and human annotation for labeling, and fine-tuned various xVerify models (0.5B-32B parameters) using QLoRA. Key results show all xVerify models achieving over 95% F1 score and accuracy on the test set, with the xVerify-3B-Ib model surpassing even GPT-4o (used as a CoT judge) in overall performance (97.27% vs 96.95% accuracy). For AI practitioners, the publicly available xVerify models offer a more reliable, efficient, and cost-effective method for automatically evaluating the correctness of reasoning model outputs compared to expensive API calls or less robust rule-based frameworks. |
| Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding (Read more on arXiv or HuggingFace) |
Weixian Lei, Yanwei Li, Zilong Huang, Tao Zhang, LXT |
Pixel-SAIL introduces a single-transformer architecture for multimodal large language models (MLLMs) targeting fine-grained, pixel-level understanding tasks. The primary research objective is to develop a highly simplified MLLM architecture for pixel-grounded understanding, eliminating the need for separate vision encoders and segmentation expert modules. Key methodologies include integrating a learnable upsampling module for refining visual tokens, a novel visual prompt injection strategy using special vocabulary tokens fused early with vision tokens, and a vision expert distillation technique. Pixel-SAIL (3B) demonstrates superior performance on referring segmentation benchmarks, outperforming larger models like GLaMM (7B) by up to 3.0% cIoU on RefCOCOg with a significantly simpler pipeline. For AI practitioners, this work shows that effective pixel-level understanding can be achieved with reduced architectural complexity using a unified transformer, potentially simplifying model development, training, and deployment. |
| Heimdall: test-time scaling on the generative verification (Read more on arXiv or HuggingFace) |
Xing Jin, WesleyShi |
This paper introduces Heimdall, an RL-trained long CoT verifier, and Pessimistic Verification to enhance LLM solution correctness judgment and problem-solving scaling. The main objective is to improve the weak verification capabilities of LLMs for complex reasoning tasks and leverage this improved verification to scale overall problem-solving accuracy. Key methodology involves training Heimdall via PPO reinforcement learning on filtered math problems and proposing Pessimistic Verification, an algorithm that selects solutions by balancing solver outputs and verifier judgments using a lower-confidence-bound approach. Primary results show Heimdall boosting verification accuracy from 62.5% to 94.5% on AIME2024 (97.5% with sampling), while Pessimistic Verification improved AIME2025 solving accuracy from 54.2% to 70.0% (16x compute budget with DeepSeek-R1-Distill-Qwen-32B). The principal implication for AI practitioners is that utilizing dedicated RL-trained verifiers and selection algorithms like Pessimistic Verification can significantly enhance the reliability and performance of LLMs on complex problem-solving by explicitly verifying and selecting trustworthy solutions. |
| Seedream 3.0 Technical Report (Read more on arXiv or HuggingFace) |
Zhichao Lai, Xiaoxia Hou, Qiushan Guo, Lixue Gong, Yu Gao |
Seedream 3.0 is presented as a high-performance Chinese-English bilingual text-to-image foundation model with significant improvements over its predecessor. The objective was to enhance alignment with complex prompts, fine-grained typography (especially Chinese text), visual aesthetics, fidelity, and native image resolution. Methodologies involved data augmentation (defect-aware training, dual-axis sampling), architectural improvements (mixed-resolution training, cross-modality RoPE, representation alignment loss), advanced post-training (aesthetic SFT, VLM reward model), and novel acceleration techniques (consistent noise expectation, importance-aware timestep sampling). Seedream 3.0 achieves superior performance, ranking first on the Artificial Analysis Leaderboard (ELO 1158), demonstrating a 94% text availability rate for Chinese characters, and enabling 4-8x inference speedup while supporting native 2K resolution. For AI practitioners, this model offers enhanced capabilities for high-fidelity, high-resolution bilingual image generation with strong text rendering and improved prompt adherence, suitable for applications demanding advanced typography and aesthetic quality. |
| How Instruction and Reasoning Data shape Post-Training: Data Quality |
|
|
| through the Lens of Layer-wise Gradients (Read more on arXiv or HuggingFace) |
Ziyue Li, Yanhong Li, Ming Li, zhoutianyi |
This paper analyzes how instruction and reasoning data quality impacts LLM post-training dynamics through the spectral properties of layer-wise gradients. The primary objective is to understand how low/high-quality instruction and reasoning data affect gradients and to unify different data quality evaluation metrics using gradient spectral characteristics. The study employs Singular Value Decomposition (SVD) on the layer-wise gradients (specifically Q, K, V, O projections) of various LLMs (Qwen2, Llama3, Gemma2 families) finetuned on datasets partitioned by quality metrics (IFD, InsTag, Difficulty, Reward) and compares instruction-following versus reasoning data. Results consistently show that higher-quality data, for both instruction and reasoning types, leads to lower nuclear norms and significantly higher effective ranks of the gradients; for instance, high-quality reasoning data (s1.1) yielded substantially higher effective ranks than high-quality instruction data across models (e.g., Table 2, Qwen2.5-7B K-projection high-quality reasoning rank 361.2 vs. instruction rank 153.3). The principal implication for AI practitioners is that the effective rank of layer-wise gradients offers a unified, robust metric to evaluate data quality, potentially guiding more effective data selection or synthesis strategies for stable LLM post-training, particularly for developing complex reasoning abilities. |
| TextArena (Read more on arXiv or HuggingFace) |
Leshem Choshen, Benjamin-eecs, simonycl, bobbycxy, LeonGuertler |
TextArena introduces an open-source framework leveraging 74+ competitive text-based games for evaluating and training agentic capabilities in LLMs via a dynamic TrueSkill leaderboard. The objective is to provide a scalable, relative benchmark assessing LLM skills like strategic planning, theory of mind, and deception, often missed by static benchmarks, through competitive gameplay. Methodologically, TextArena employs diverse text-based games (single/two/multi-player) within a Gym-compatible interface, evaluating models online (model-vs-model/human) and tracking performance using TrueSkill ratings across 10 specific soft skills. Primary results include relative model rankings and granular skill profiles; preliminary data shows frontier models achieving TrueSkill scores in the 30-38 range in certain games, demonstrating capabilities relative to a collective human baseline, though performance varies significantly across tasks (Figure 2). For AI practitioners, TextArena offers a platform to benchmark complex agentic behaviors without human preference bias, diagnose specific model skill gaps (e.g., Persuasion vs. Spatial Thinking), and potentially generate diverse interaction data for RL-based agent training. |
| The Scalability of Simplicity: Empirical Analysis of Vision-Language |
|
|
| Learning with a Single Transformer (Read more on arXiv or HuggingFace) |
Jun Hao Liew, Haochen Wang, Jiacong Wang, Weixian Lei, LXT |
This paper introduces and empirically analyzes SAIL, a single-transformer architecture for joint vision-language processing, comparing its properties to modular designs. The research objective is to evaluate the scalability, cross-modal information flow patterns, and visual representation capabilities of this unified approach against modular Multimodal Large Language Models (MLLMs) that use separate vision encoders. SAIL employs a single transformer with mixed attention (bidirectional for image patches, causal for text) and multimodal rotary position embeddings (M-RoPE) to process raw pixels and text, evaluated via scaling experiments and performance on vision-language/vision benchmarks. Key results show SAIL exhibits superior data scalability compared to modular models (Fig 1A) and achieves strong vision task performance, including 84.95% Top-1 accuracy on ImageNet-1K classification, demonstrating effective visual feature learning without a pre-trained encoder. For AI practitioners, this indicates that unified single-transformer architectures are a viable, potentially more scalable alternative to complex modular designs, simplifying the model stack while achieving competitive performance, especially with large datasets. |
| Efficient Process Reward Model Training via Active Learning (Read more on arXiv or HuggingFace) |
Tianyu Pang, Xin Mao, Zichen Liu, Keyu Duan, dreamerdeo |
This paper proposes ACTPRM, an active learning framework to efficiently train Process Reward Models (PRMs) for large language models. The primary objective is to reduce the prohibitive annotation costs required for obtaining step-level supervision needed to train PRMs. ACTPRM employs an ensemble PRM to estimate both aleatoric and epistemic uncertainty at each reasoning step, selectively forwarding only the most uncertain samples to a capable reasoning LLM for annotation, and then training the PRM exclusively on this subset. ACTPRM achieved state-of-the-art performance (75.0% average F1) on ProcessBench while requiring only 20% of the estimated annotation cost compared to the prior SOTA model, UniversalPRM. For AI practitioners, this methodology offers a significantly more cost-effective approach to training PRMs, enabling scalable development of LLMs with improved reasoning capabilities through fine-grained process supervision. |
| Efficient Generative Model Training via Embedded Representation Warmup (Read more on arXiv or HuggingFace) |
Tao Lin, Xufeng Li, Peng Sun, SempraETY |
This paper introduces Embedded Representation Warmup (ERW) to accelerate diffusion model training by initializing early layers with pretrained representations. The primary objective is to improve training efficiency and representation quality by decoupling the representation learning phase from the generation phase in diffusion models. ERW employs a two-phase training strategy: first, a warmup phase aligns the initial layers (Latent-to-Representation circuit) with features from a pretrained model (e.g., Dinov2) using an alignment loss; second, standard diffusion training proceeds with a decaying alignment guidance term. Empirically, ERW demonstrates a 40x acceleration in training speed compared to the REPA baseline, achieving an FID of 6.0 on ImageNet-1k (SiT-XL/2, no CFG) within 100k iterations. For AI practitioners, ERW offers a plug-and-play method to significantly reduce computational costs and training time for large diffusion models by leveraging existing pretrained representation encoders, making state-of-the-art generative modeling more accessible. |
| NormalCrafter: Learning Temporally Consistent Normals from Video |
|
|
| Diffusion Priors (Read more on arXiv or HuggingFace) |
Bing Wang, Xinya Chen, Haoyuan Wang, Yanrui Bin, wbhu-tc |
NormalCrafter introduces a novel method leveraging video diffusion priors to generate temporally consistent and detailed surface normals from open-world videos. The main objective is to address the challenge of maintaining both high spatial fidelity and temporal coherence in video-based normal estimation, which existing methods often fail to achieve simultaneously. Key methodology includes adapting a pre-trained video diffusion model (SVD), proposing Semantic Feature Regularization (SFR) to align internal features with semantic representations (from DINO), and utilizing a two-stage training protocol optimizing first in latent space for temporal context and then in pixel space for spatial accuracy. Primary results demonstrate superior performance on video benchmarks, achieving a 1.6° reduction in mean angular error on the Sintel dataset compared to the prior state-of-the-art, alongside improved temporal consistency. For AI practitioners, this research provides a framework for adapting large video generative models for downstream perception tasks, showcasing how diffusion priors combined with specific regularization and training strategies can yield high-fidelity, temporally stable outputs for video understanding applications like 3D reconstruction or editing. |
| A Minimalist Approach to LLM Reasoning: from Rejection Sampling to |
|
|
| Reinforce (Read more on arXiv or HuggingFace) |
Lei Wang, Bo Pang, Yuhui Xu, Jiarui Yao, Wei Xiong |
This paper analyzes simplified reinforcement learning algorithms for fine-tuning large language models (LLMs) on reasoning tasks, demonstrating the strong performance of rejection sampling. The primary objective is to understand the sources of effectiveness in complex RL algorithms like GRPO and identify minimal yet performant alternatives. Key methodologies include empirical comparisons of RAFT (rejection sampling), vanilla Reinforce, GRPO, and PPO on mathematical reasoning benchmarks, alongside ablation studies isolating components like reward normalization and sample filtering, leading to a proposed variant, Reinforce-Rej. The primary result shows that RAFT achieves competitive performance (e.g., 49.9% average accuracy on Qwen2.5-Math-7B-base) compared to GRPO (53.9%) and PPO (51.8%), with GRPO’s advantage largely attributed to filtering prompts with only incorrect responses, not reward normalization. The principal implication for AI practitioners is that simpler, computationally lighter methods like RAFT and the proposed Reinforce-Rej can be highly effective alternatives to complex RL algorithms for reward-based LLM fine-tuning, highlighting the crucial role of selective sample filtering over intricate algorithmic designs. |
| DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and |
|
|
| Verifiable Mathematical Dataset for Advancing Reasoning (Read more on arXiv or HuggingFace) |
Xingyu Chen, Qiuzhi Liu, Jiahao Xu, Tian Liang, Zhiwei He |
This paper introduces DeepMath-103K, a large-scale, challenging, decontaminated, and verifiable mathematical dataset designed for advancing AI reasoning via reinforcement learning. The primary objective was to create a dataset overcoming limitations of existing resources, namely insufficient difficulty, lack of verifiable answers for RL, benchmark contamination, and inadequate scale for highly challenging problems. The methodology involved a rigorous curation pipeline including source analysis, semantic decontamination against multiple benchmarks using LLM-judges, difficulty filtering focusing on levels 5-9, and answer verification through consistency checks across three distinct R1-generated solutions for each of the 103K problems. Models trained using RL-Zero on DeepMath-103K demonstrated significant performance improvements, with DeepMath-Zero-7B achieving 85.5% pass@1 accuracy on MATH500, substantially outperforming baseline and models trained on other RL datasets. For AI practitioners, DeepMath-103K provides a crucial, publicly available resource enabling the development and evaluation of more powerful reasoning systems, particularly through rule-based RL paradigms demanding verifiable answers and high problem complexity. |
| Diffusion Distillation With Direct Preference Optimization For Efficient |
|
|
| 3D LiDAR Scene Completion (Read more on arXiv or HuggingFace) |
Jiale Wu, Zejian Li, Ling Yang, Shengyuan Zhang, An Zhaol |
This paper proposes Distillation-DPO, a novel framework integrating diffusion distillation with direct preference optimization for efficient and high-quality 3D LiDAR scene completion. The primary objective is to accelerate the slow sampling speed of diffusion models for LiDAR completion while mitigating performance degradation typically associated with distillation. Distillation-DPO generates paired completion samples using a student model with varied initial noise, constructs win/lose pairs based on non-differentiable LiDAR metrics (used as preference), and optimizes the student by minimizing the difference in score functions between teacher and student models on these pairs, facilitated by two teaching assistant models. Experiments demonstrate that Distillation-DPO achieves superior completion quality (e.g., 0.354 refined CD compared to the SOTA LiDiff’s 0.375) while accelerating inference speed by over 5-fold (3.38s vs 17.87s). For AI practitioners, this method offers a way to significantly enhance the efficiency of diffusion models for 3D scene completion tasks, making them more viable for real-world applications by effectively using preference data to guide distillation without requiring differentiable reward functions. |
| PVUW 2025 Challenge Report: Advances in Pixel-level Understanding of |
|
|
| Complex Videos in the Wild (Read more on arXiv or HuggingFace) |
Shuting He, Nikhila Ravi, Chang Liu, LXT, HenghuiDing |
This report summarizes the 4th Pixel-level Video Understanding in the Wild (PVUW) Challenge, focusing on methods and results for complex video segmentation tasks. The primary objective was to benchmark and advance algorithms for complex video object segmentation (MOSE track) and motion/language-guided video segmentation (MeViS track) using new, challenging real-world datasets. Key methodologies employed by top teams included fine-tuning large foundation models like SAM2, utilizing multi-model ensembles, adaptive pseudo-labeling (e.g., PGMR), and integrating Large Multimodal Models (LMMs) like Sa2VA, evaluated via J&F scores on confidential test sets. The top team on the MOSE track achieved a J&F score of 87.26%, while the MeViS track winner reached 61.98%, showcasing the effectiveness of these advanced techniques. For AI practitioners, the principal implication is the demonstrated benefit of adapting large pre-trained vision and multimodal models (SAM2, LMMs) and using ensemble strategies to improve robustness and accuracy in complex, dynamic video understanding tasks. |
| ReZero: Enhancing LLM search ability by trying one-more-time (Read more on arXiv or HuggingFace) |
Thinh Le, alandao |
ReZero introduces a reinforcement learning framework to enhance LLM search persistence within Retrieval-Augmented Generation (RAG) by rewarding query retries. The main objective is to improve LLM robustness in information retrieval by explicitly incentivizing the model to attempt subsequent searches if the initial one fails. The key methodology utilizes Group Relative Policy Optimization (GRPO) to fine-tune an LLM, incorporating a specific reward_retry function that rewards additional search attempts conditional on generating a correct final answer. The primary result showed the ReZero model achieved 46.88% peak accuracy on the evaluation dataset, nearly doubling the 25.00% peak accuracy of a baseline model trained without the retry incentive. For AI practitioners, this implies that designing RL rewards to explicitly encourage persistence can significantly improve RAG system performance, especially for tasks where initial information retrieval attempts are likely insufficient. |
| AI-University: An LLM-based platform for instructional alignment to |
|
|
| scientific classrooms (Read more on arXiv or HuggingFace) |
Rahul Gulati, Mostafa Faghih Shojaei, garikipati, Dinzhenzhenzhu, simocimolato |
This paper introduces AI-University (AI-U), a framework using fine-tuned LLMs and Retrieval-Augmented Generation (RAG) to generate instructor-aligned responses for scientific courses. The objective was to develop and evaluate a platform that adapts an LLM (Llama-3.2-11B) to a specific graduate-level Finite Element Method (FEM) course’s content and teaching style using lecture transcripts, notes, and textbooks. Key methodology involved systematic question-answer pair generation for LoRA-based fine-tuning (creating LLaMA-TOMMI-1.0), followed by RAG synthesis for contextualized, referenced answers, evaluated via cosine similarity and LLM-as-a-judge. The fine-tuned LLaMA-TOMMI-1.0 model achieved higher cosine similarity to ground-truth answers than the base model on 86% of test cases and was preferred approximately four times more often by an LLM judge. The principal implication for AI practitioners is that this combined approach of systematic data generation for fine-tuning and RAG offers a robust method for developing domain-specific LLMs that exhibit strong alignment with specialized technical content and style, providing traceable and accurate AI assistance. |
| Adaptive Computation Pruning for the Forgetting Transformer (Read more on arXiv or HuggingFace) |
Aaron Courville, Johan Obando-Ceron, Zhixuan Lin, littleowen |
This paper proposes Adaptive Computation Pruning (ACP) to accelerate the Forgetting Transformer (FoX) by dynamically skipping computations based on forget gate decay. The objective is to determine if dynamically pruning FoX attention computations based on decay strength can improve training throughput without performance loss. ACP employs a dynamic pruning threshold, calculated based on attention logit bounds and sequence length, to identify and skip negligible input-output dependency computations within a modified FlashAttention framework. Results demonstrate that ACP consistently reduces FLOPs in softmax attention by ~70% across different model sizes (125M-760M) and context lengths (4k-16k), resulting in 10%-35% faster training throughput without performance degradation on language modeling or downstream tasks. For AI practitioners, ACP provides a technique to significantly decrease computational costs and improve training efficiency for FoX models, particularly those with long contexts, while maintaining accuracy. |
| Multimodal Long Video Modeling Based on Temporal Dynamic Context (Read more on arXiv or HuggingFace) |
Xiangyu Yue, Yiyuan Zhang, Jiaming Han, Hoar012 |
This paper introduces Temporal Dynamic Context (TDC), a method for multimodal long video understanding integrating static features and dynamic context compression. The research aims to address MLLM context length limitations and suboptimal multimodal integration (vision, audio) in long video processing. TDC segments videos by inter-frame similarity, encodes static keyframes fully, and uses a Q-Former to compress subsequent visual/audio tokens based on temporal differences relative to the static frame; a Long Video Chain-of-Thought (LVCoT) strategy handles extremely long videos without training. TDC demonstrates strong performance, outperforming the audio-visual VideoLLaMA2 model by 15.6% on the long-video MLVU benchmark. For AI practitioners, TDC provides an effective technique for encoding dense multimodal video data more efficiently, enabling MLLMs to process longer videos by compressing dynamic context while preserving key static details, reducing information loss compared to sparse sampling or purely visual compression methods. |
| Summarization of Multimodal Presentations with Vision-Language Models: |
|
|
| Study of the Effect of Modalities and Structure (Read more on arXiv or HuggingFace) |
Frédéric Dufaux, Camille Guinaudeau, gigant |
This paper analyzes how input modality and structure affect Vision-Language Model (VLM) performance for summarizing multimodal presentations. The primary objective is to evaluate the cost-performance tradeoffs of various input representations (raw video, extracted slides, transcript, structured/unstructured combinations) and suggest effective strategies. Using Qwen2-VL and other VLMs on a benchmark derived from the TIB dataset, the study measured performance with metrics like ROUGE and Importance-based Relevance (IbR). Results demonstrate that a structured representation using interleaved slides and transcript yields the best performance (e.g., Qwen2-VL 2B achieved ROUGE-1 of 27.1 and overall IbR of 33.4), significantly outperforming raw video or unstructured inputs. For AI practitioners, the key implication is that preprocessing presentations into structured, interleaved slide-transcript sequences offers the most effective input for VLM summarization, balancing computational cost and summary quality, especially for inputs exceeding approximately 6k tokens. |
| D^2iT: Dynamic Diffusion Transformer for Accurate Image Generation (Read more on arXiv or HuggingFace) |
Zhendong Mao, Lei Zhang, Nan Chen, Mengqi Huang, Weinan Jia |
This paper introduces D²iT, a Diffusion Transformer using dynamic compression based on regional information density to improve image generation accuracy. The main objective is to overcome the limitations of fixed spatial compression in standard Diffusion Transformers (DiTs) which disregard varying information densities across image regions. The methodology employs a two-stage framework: first, a Dynamic VAE (DVAE) uses a hierarchical encoder and information density estimation (Shannon entropy) to create multi-grained latent codes; second, the Dynamic Diffusion Transformer (D²iT) predicts corresponding multi-grained noise using novel Dynamic Grain and Content Transformers. Primary results demonstrate a significant quality improvement, achieving a 1.73 FID score on class-conditional ImageNet 256x256 generation, a 23.8% improvement over the baseline DiT’s 2.27 FID, using only 57.1% of the training resources. For AI practitioners, this research implies that dynamically adapting compression and computational effort based on input complexity, rather than using fixed approaches, can yield substantial gains in both the performance and efficiency of generative models like DiTs. |
| Change State Space Models for Remote Sensing Change Detection (Read more on arXiv or HuggingFace) |
Erchan Aptoula, ElmanGhazaei |
This paper introduces the Change State Space Model (CSSM), a computationally efficient Mamba-based architecture tailored for remote sensing change detection. The research objective is to develop a specialized state-space model that focuses exclusively on relevant bi-temporal changes, improving efficiency and accuracy over existing ConvNet, ViT, and general Mamba approaches for change detection. CSSM utilizes a lightweight CNN encoder-decoder framework incorporating a modified state space model block that employs an L1 distance mechanism on projected inputs to isolate and process only changed features between pre- and post-event images. Evaluated on benchmark datasets like LEVIR-CD+, CSSM achieved state-of-the-art performance, attaining an F1-score of 92.39 while requiring only 4.34M parameters and 5.10 GFLOPs, significantly less than comparable models. For AI practitioners, CSSM presents a highly resource-efficient architecture delivering state-of-the-art accuracy in change detection, making it suitable for large-scale analysis or deployment in computationally constrained environments. |
| LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews (Read more on arXiv or HuggingFace) |
Iryna Gurevych, Lizhen Qu, Anne Lauscher, Zhuang Li, sukannya |
This paper introduces LAZYREVIEW, a dataset annotated with fine-grained categories to detect ‘lazy thinking’ heuristics in NLP peer reviews. The primary objective was to create this resource and evaluate the ability of Large Language Models (LLMs) to automatically identify such instances. The methodology involved iteratively developing annotation guidelines over three rounds using ARR-22 reviews, annotating 500 expert and 1276 silver review segments, and evaluating LLMs using zero-shot, few-shot in-context learning, and instruction fine-tuning. Key results show that while LLMs struggle in zero-shot detection, instruction fine-tuning on LAZYREVIEW significantly boosts performance by 10-20 accuracy points (e.g., instruction-tuned Qwen achieved 59.4% string-matching accuracy for fine-grained classification). For AI practitioners, this provides a validated dataset and methodology for building automated tools to flag superficial review arguments, potentially improving review quality assessment systems and reviewer training. |
Papers for 2025-04-15
| Title |
Authors |
Summary |
| InternVL3: Exploring Advanced Training and Test-Time Recipes for |
|
|
| Open-Source Multimodal Models (Read more on arXiv or HuggingFace) |
jackroos, duanyuchen, gulixin0922, Yeshenglong, Weiyun1025 |
InternVL3 presents an open-source Multimodal Large Language Model (MLLM) series developed via native multimodal pre-training and advanced training/test-time techniques. The research objective was to improve MLLM performance and training efficiency by jointly learning multimodal and linguistic capabilities within a single pre-training stage, circumventing typical post-hoc adaptation of text-only LLMs. Key methodologies employed include unified pre-training on mixed text and multimodal corpora, Variable Visual Position Encoding (V2PE), supervised fine-tuning (SFT), and Mixed Preference Optimization (MPO) post-training, alongside test-time scaling. The primary result shows InternVL3-78B achieving a state-of-the-art score of 72.2% on the MMMU benchmark among open-source MLLMs, demonstrating strong capabilities competitive with proprietary models like ChatGPT-4o and Gemini 2.5 Pro. For AI practitioners, this work provides evidence that native multimodal pre-training yields powerful open-source MLLMs, and the released models and data offer a strong foundation for developing advanced multimodal applications without relying solely on closed systems. |
| PRIMA.CPP: Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday |
|
|
| Home Clusters (Read more on arXiv or HuggingFace) |
Hongfang Yu, Mohsen Guizani, NeuronNomad, LiPhilip, LIKirin |
PRIMA.CPP introduces a distributed system for running 70B-scale LLMs on heterogeneous, low-resource home device clusters. The objective is to minimize inference latency while managing limited and diverse resources (CPU/GPU, RAM/VRAM, disk, OS, network). It employs piped-ring parallelism with prefetching to hide disk I/O latency from memory-mapped weights and uses the Halda algorithm to optimally assign model layers based on a detailed heterogeneity model. Evaluations on a four-node home cluster show prima.cpp is 15x faster than llama.cpp for 70B models, achieving ~600 ms/token with memory pressure under 6%. This enables AI practitioners to deploy state-of-the-art 30B-70B models locally on clusters of everyday consumer devices, expanding accessibility beyond high-end hardware or cloud services. |
| VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models |
|
|
| with Reinforcement Learning (Read more on arXiv or HuggingFace) |
Wei Chu, Chao Qu, wenhu, zuminghuang, JasperHaozhe |
VL-Rethinker improves multimodal reasoning by incentivizing self-reflection in vision-language models through reinforcement learning. The research aims to enhance slow-thinking capabilities in VLMs for complex multimodal tasks. It uses Group Relative Policy Optimization (GRPO) with Selective Sample Replay (SSR) and Forced Rethinking to train the model. VL-Rethinker achieves state-of-the-art scores on MathVista (80.3%), MathVerse (61.8%), and MathVision (43.9%). The method provides AI practitioners with an RL approach for enhancing VLM reasoning without reliance on distillation, offering techniques such as SSR to stabilize training and Forced Rethinking to promote self-reflection. |
| FUSION: Fully Integration of Vision-Language Representations for Deep |
|
|
| Cross-Modal Understanding (Read more on arXiv or HuggingFace) |
Jingzhou Chen, conghui, jingwei-xu-00, Balalauuoo, starriver030515 |
i) The paper introduces FUSION, a family of multimodal large language models (MLLMs) designed for deep, dynamic integration of vision and language. ii) The research aims to enhance cross-modal understanding by achieving a fully vision-language aligned and integrated paradigm within MLLMs. iii) The methodology incorporates Text-Guided Unified Vision Encoding, Context-Aware Recursive Alignment Decoding, and a Dual-Supervised Semantic Mapping Loss. iv) Experiments show FUSION 3B outperforms Cambrian-1 8B and Florence-VL 8B on most benchmarks, even when limited to 300 vision tokens. v) FUSION’s approach provides AI practitioners with a strategy for significantly improving MLLM performance with fewer vision tokens by focusing on deep modality integration. |
| Iterative Self-Training for Code Generation via Reinforced Re-Ranking (Read more on arXiv or HuggingFace) |
Valentin Malykh, Ivan Sedykh, Nikita Sorokin |
Iterative self-training is used to refine code generation through reinforced re-ranking with Proximal Policy Optimization (PPO). The research aims to improve code generation quality and re-ranking accuracy of decoder-based models through iterative self-training using PPO to optimize a reward/re-ranking model. The methodology involves supervised fine-tuning, reward model training, PPO-based code generation, and iterative refinement using hard negative mining. Results demonstrate a 13.4B parameter model outperforming a 33B parameter model on the MultiPL-E dataset in code generation quality and reaching comparable to GPT-4 performance in code generation, while being three times faster. For AI practitioners, the study presents a method for developing more efficient code generation models by focusing on a robust reward mechanism within a self-training framework. |
| Mavors: Multi-granularity Video Representation for Multimodal Large |
|
|
| Language Model (Read more on arXiv or HuggingFace) |
kugwzk, zhenhuawu, UnnamedWatcher, CheeryLJH, DogNeverSleep |
Mavors introduces a multi-granularity video representation framework for multimodal large language models (MLLMs) aimed at efficient long-context video understanding. The main objective is to balance computational efficiency with the retention of fine-grained spatio-temporal patterns, addressing information loss from methods like sparse sampling or token compression. Mavors uses an Intra-chunk Vision Encoder (IVE) for high-resolution spatial features within video segments and an Inter-chunk Feature Aggregator (IFA) with chunk-level rotary position embeddings (C-ROPE) for temporal coherence across segments. Results demonstrate Mavors-7B’s strong performance, achieving a score of 39.4 on the DREAM-1K video captioning benchmark, significantly outperforming many comparable 7B models on tasks requiring fine-grained spatio-temporal reasoning. For AI practitioners, Mavors offers an approach to enhance MLLM capabilities for long video analysis by preserving detailed spatio-temporal information more effectively than common sampling or compression strategies, crucial for applications needing nuanced video understanding. |
| AgentRewardBench: Evaluating Automatic Evaluations of Web Agent |
|
|
| Trajectories (Read more on arXiv or HuggingFace) |
dongchans, arkilpatel, ncmeade, kazemnejad, xhluca |
This paper introduces AGENTREWARDBENCH, a benchmark designed to evaluate the automatic evaluation of web agent trajectories by LLM judges. The main objective is to assess the effectiveness of LLMs in judging web agent success compared to expert human annotations, addressing limitations of rule-based and manual evaluations. The methodology involved collecting 1302 trajectories from 4 LLMs across 5 web environments, annotating each by experts for success, side effects, and repetition, and then using this dataset to evaluate 12 different LLM judges and existing rule-based methods. Primary results indicate that no single LLM judge performs best across all benchmarks, the best judges achieve less than 70% precision against expert labels, and official rule-based methods significantly underestimate agent success rates (55.9% recall). The principal implication for AI practitioners is that current automatic evaluation methods, including LLM judges, are not yet reliable enough for high-fidelity assessment or reward modeling, necessitating the development of more accurate automatic evaluation techniques for web agents. |
| S1-Bench: A Simple Benchmark for Evaluating System 1 Thinking Capability |
|
|
| of Large Reasoning Models (Read more on arXiv or HuggingFace) |
Tingwen Liu, Xinghua Zhang, Starrrrrry, ShuaiyiNie, WYRipple |
i) S1-Bench is introduced as a benchmark to evaluate Large Reasoning Models’ (LRMs) system 1 thinking capabilities, contrasting with their prevalent system 2 reliance. ii) The research aims to assess LRMs’ performance on simple, intuitive tasks better suited for system 1 processing to understand the effects of over-reliance on system 2. iii) The methodology involves constructing a dataset of simple, diverse questions across multiple domains and languages and evaluating 22 LRMs on this benchmark. iv) Results indicate that LRMs exhibit lower efficiency tendencies, generating outputs averaging 15.5 times longer than traditional small LLMs, and accuracy degradation on simple questions. v) This highlights the need for substantial development in LRMs to achieve balanced dual-system thinking capabilities adaptable to task complexity for AI practitioners. |
| Have we unified image generation and understanding yet? An empirical |
|
|
| study of GPT-4o’s image generation ability (Read more on arXiv or HuggingFace) |
Ning Li, cuijiaxing, zhangjingran |
i) This paper empirically evaluates GPT-4o’s image generation capabilities across global instruction adherence, fine-grained editing precision, and post-generation reasoning. ii) The main objective is to assess whether GPT-4o achieves world knowledge-informed semantic synthesis during image generation. iii) The methodology involves designing three types of prompts: global instruction, fine-grained editing, and post-generation reasoning, to test specific aspects of image generation. iv) Results show GPT-4o defaults to literal interpretations, inconsistently applies knowledge constraints, and struggles with conditional reasoning tasks. v) The principal implication is that GPT-4o has significant limitations in dynamically integrating knowledge into its image generation process, necessitating more robust benchmarks for reasoning-aware multimodal generation. |
| DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM |
|
|
| Post-training (Read more on arXiv or HuggingFace) |
zwt123home123, timecuriosity, gfcui, ztwang |
i) The paper introduces DUMP, an automated distribution-level curriculum learning framework for reinforcement learning-based post-training of large language models. ii) The research aims to dynamically schedule training across heterogeneous data distributions to optimize learning efficiency in LLMs. iii) The methodology employs Upper Confidence Bound (UCB) scores based on expected absolute advantage to adaptively adjust sampling probabilities for different distributions. iv) Experiments on logic reasoning datasets show that DUMP significantly improves convergence speed and final performance, achieving a reward of over 0.5 in the 9-character K&K puzzles distribution, while the uniform sampling baseline remained below 0.0. v) The principal implication is that AI practitioners can utilize DUMP to improve the efficiency and effectiveness of RL-based LLM post-training by dynamically prioritizing learnable data distributions. |
| SocioVerse: A World Model for Social Simulation Powered by LLM Agents |
|
|
| and A Pool of 10 Million Real-World Users (Read more on arXiv or HuggingFace) |
milesz7777, tangshiping, SimingChen, libo-ca, Lishi0905 |
i) SocioVerse is presented as an LLM-agent-driven world model for social simulation. ii) The research aims to address alignment challenges in social simulation across environment, users, interaction, and behavior. iii) The methodology involves a framework with four alignment components and a user pool of 10 million real individuals derived from social media data. iv) Experiments across politics, news, and economics domains demonstrated SocioVerse’s ability to reflect population dynamics, with presidential election prediction achieving over 90% accuracy in state voting results. v) The study indicates a need for careful selection of underlying LLMs to optimize simulation precision across different social scenarios for AI practitioners. |
| Breaking the Data Barrier – Building GUI Agents Through Task |
|
|
| Generalization (Read more on arXiv or HuggingFace) |
jxhe, QiushiSun, changma, heroding77, leoozy |
i) This paper investigates the effectiveness of mid-training Vision Language Models (VLMs) on reasoning-intensive tasks for improved generalization in GUI agent planning. ii) The research aims to determine how incorporating various instruction-tuning tasks during the mid-training phase of VLMs facilitates generalization to GUI planning scenarios, addressing the scarcity of high-quality trajectory data. iii) The methodology involves training VLMs on a range of readily available instruction-tuning datasets, including GUI perception, multimodal reasoning, and textual reasoning, followed by fine-tuning on GUI trajectory data. iv) The primary results indicate that task generalization proves highly effective, with multimodal mathematical reasoning enhancing performance on AndroidWorld by an absolute 6.3%; text-only mathematical data significantly boosts GUI web agent performance, achieving a 5.6% improvement on WebArena and a 5.4% improvement on AndroidWorld. v) The principal implication for AI practitioners is that incorporating specific, readily available reasoning tasks into the mid-training of VLMs can substantially improve the performance and generalization capabilities of GUI agents, offering a practical approach to addressing data scarcity challenges in this domain; The work also identifies an optimized dataset mixture called GUIMid which achieves absolute gains of 8.0% on WebArena and 12.2% on AndroidWorld. |
| TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning (Read more on arXiv or HuggingFace) |
Lei Huang, Wenjun Wu, wenzz1, Zhang199 |
TinyLLaVA-Video-R1 explores reasoning in small vision-language models (VLMs) for video understanding. The research investigates how reinforcement learning (RL) can improve reasoning capabilities in smaller VLMs using general Video-QA datasets. The GRPO algorithm was applied to TinyLLaVA-Video with modifications to the reward structure, including a continuous length reward and penalties for incorrect answers. TinyLLaVA-Video-R1 achieves 49.5 on MVBench, improving reasoning with fewer parameters. The work demonstrates that RL can elicit emergent reasoning abilities like self-verification in small-scale VLMs, suggesting avenues for improving video reasoning with limited computational resources. |
| LLM-SRBench: A New Benchmark for Scientific Equation Discovery with |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Khoa D Doan, Amir Barati Farimani, Ngoc-Hieu Nguyen, mkmeidani, parshinsh |
i) LLM-SRBench, a new benchmark, is introduced for evaluating scientific equation discovery using Large Language Models (LLMs). ii) The research aims to provide a rigorous benchmark that avoids memorization effects and properly assesses the equation discovery capabilities of LLMs. iii) The methodology involves creating a dataset with 239 challenging problems across four scientific domains, utilizing both LSR-Transform (alternative mathematical representations) and LSR-Synth (synthetic problems) categories. iv) Experimental results demonstrate that the best-performing system achieves only 31.5% symbolic accuracy across the benchmark. v) This benchmark highlights the limitations of current LLMs in scientific equation discovery, suggesting AI practitioners need to develop more robust methods to leverage LLMs for complex scientific reasoning tasks that go beyond memorization. |
| EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental |
|
|
| Health Safety (Read more on arXiv or HuggingFace) |
Edify-Kd2024, yaozixin, YimingWang, ChrisJuan, yinghuihe |
i) EmoAgent is a multi-agent AI framework for evaluating and mitigating mental health risks in human-AI interactions within character-based chatbots. ii) The research aims to assess and safeguard human-AI interactions for mental health safety, particularly for vulnerable users. iii) EmoAgent employs a simulated environment (EmoEval) using clinically validated psychological assessment tools and a real-time safeguard agent (EmoGuard) that monitors and provides corrective feedback. iv) Experiments show that emotionally engaging dialogues can lead to mental state deterioration in vulnerable users in more than 34.4% of simulations; EmoGuard reduces these deterioration rates significantly. v) AI practitioners should be aware that emotionally engaging AI dialogues can lead to mental state deterioration in vulnerable users; and real-time monitoring and corrective feedback are crucial for ensuring safety in AI-human interactions. |
| The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via |
|
|
| Agentic Tree Search (Read more on arXiv or HuggingFace) |
Chris Lu, Shengran Hu, Robert Tjarko Lange, conglu, yyamada |
i) This paper introduces THE AI SCIENTIST-v2, an AI agentic system for automated scientific discovery, improving upon its predecessor. ii) The research aims to develop an end-to-end system capable of autonomously producing scientific manuscripts acceptable for peer review. iii) The methodology involved agentic tree search managed by an experiment manager agent, Vision-Language Model (VLM) feedback loops, and parallel experiment execution. iv) The system generated a manuscript that achieved an average reviewer score of 6.33 at an ICLR workshop, exceeding the average human acceptance threshold. v) This work demonstrates the potential for AI to conduct all aspects of scientific research, enabling unprecedented scalability in research productivity. |
| Executable Functional Abstractions: Inferring Generative Programs for |
|
|
| Advanced Math Problems (Read more on arXiv or HuggingFace) |
Zaid Khan, mohitbansal, j-min, archiki, esteng |
i) The paper introduces EFAGen, a framework for automatically constructing Executable Functional Abstractions (EFAs) for advanced math problems by inferring generative programs from static examples. ii) The research aims to automate the construction of EFAs for advanced math problems, operationalizing this as a program synthesis task. iii) EFAGen conditions a large language model (LLM) on a seed math problem and its solution to generate candidate EFA programs, using executable unit tests as verifiable rewards to train the LLM. iv) Experiments show that EFAs constructed by EFAGen remain faithful to seed problems, produce learnable problem variations, infer EFAs across multiple diverse sources of competition-level math problems, and EFA-based augmentation yields consistent improvements on MATH-500, where Pass@1 improves by +1.9 in the 33% seed setting. v) The principal implication is a scalable approach for generating diverse and verifiable math problem variants, aiding in data augmentation, model stress-testing, and curriculum learning for improving mathematical reasoning in AI systems. |
| How new data permeates LLM knowledge and how to dilute it (Read more on arXiv or HuggingFace) |
Nolan Andrew Miller, Andrey Zhmoginov, Chen Sun, gozzo87, mendor |
i) This paper investigates how individual text samples update LLM knowledge, introducing a “priming” effect where new facts inappropriately generalize to unrelated contexts. ii) The research aims to understand and predict how new information propagates through an LLM’s knowledge base, leading to both generalization and problematic hallucination. iii) The methodology involves a novel dataset, “Outlandish”, composed of 1320 diverse text samples designed to systematically probe knowledge permeation, along with measuring token probabilities before and after learning. iv) The study found that the degree of priming can be predicted by measuring the token probability of key words before learning, and developed two techniques, “stepping-stone” text augmentation and “ignore-k” update pruning, reducing priming effects by 50-95%. v) The findings offer AI practitioners empirical insights and practical tools for improving the specificity of knowledge insertion in language models and reducing undesirable knowledge permeation. |
| VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search (Read more on arXiv or HuggingFace) |
QipengGuo, alphadl, ngc7293, sinwang, LibraTree |
VisuoThink introduces a multimodal tree search framework to enhance Large Vision-Language Model (LVLM) reasoning by interleaving visual and textual information dynamically. The research aims to improve LVLM performance on complex reasoning tasks by integrating visual aids and step-by-step thinking through a predictive rollout search mechanism. The methodology involves a vision-text interleaved reasoning framework coupled with a look-ahead tree search algorithm that explores multiple reasoning paths. Experiments show VisuoThink achieves an accuracy of 48.5% on Geomeverse, a 21.8% improvement over the state-of-the-art baseline without fine-tuning, particularly excelling in problems requiring multi-step visual reasoning. This framework offers AI practitioners an effective method for improving reasoning capabilities in vision-language models without requiring model retraining. |
| M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models (Read more on arXiv or HuggingFace) |
Daniele Paliotta, tridao, voidptr74, xu3kev, JunxiongWang |
i) The paper introduces M1, a hybrid Mamba-based reasoning model, that exhibits efficient test-time compute scaling. ii) The research aims to develop a scalable reasoning model that can leverage increased test-time computation for improved performance on mathematical tasks. iii) The methodology includes distilling a Transformer model into a Mamba architecture, followed by supervised fine-tuning on math datasets and reinforcement learning training with GRPO. iv) M1 achieves performance comparable to DeepSeek-R1-Distill-Qwen-1.5B on MATH500 (82) and AIME25 (22) benchmarks, while demonstrating over 3x faster inference throughput compared to similarly-sized transformer models using vLLM. v) M1 offers AI practitioners an efficient alternative to Transformers for reasoning tasks, enabling greater test-time compute scaling through faster inference and potentially improving performance via self-consistency or chain-of-thought approaches under fixed time budgets. |
| LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety |
|
|
| in Large Language Models (Read more on arXiv or HuggingFace) |
Xinyi Zhang, sarvech123, aneverfull, Zhiyang03, mqliu |
i) This paper introduces PERSUSAFETY, a framework for assessing persuasion safety in Large Language Models (LLMs). ii) The primary objective is to investigate whether LLMs reject unethical persuasion tasks and avoid unethical strategies, considering influencing factors like personality traits and external pressures. iii) The methodology involves creating persuasion scenes, simulating persuasive conversations between LLMs, and assessing safety via refusal rates and unethical strategy usage. iv) Experiments across 8 LLMs revealed that most models fail to consistently refuse harmful persuasion tasks and employ unethical strategies; Claude-3.5-Sonnet, while exhibiting strong refusal rates, showed high unethical strategy usage when engaged. v) AI practitioners should be aware that current safety alignment techniques in LLMs may not prevent the use of unethical strategies once the model is engaged, necessitating further research into safety alignment in goal-driven conversations. |
| DeepSeek vs. o3-mini: How Well can Reasoning LLMs Evaluate MT and |
|
|
| Summarization? (Read more on arXiv or HuggingFace) |
Christoph Leiter, Yanran Chen, Ran Zhang, Sotaro Takeshita, Daniil Larionov |
i) The paper systematically compares the performance of reasoning-enabled LLMs against non-reasoning counterparts in evaluating machine translation (MT) and text summarization (TS) tasks. ii) The main research questions are whether reasoning models improve upon conventional models in NLG evaluation and how effectively distillation preserves evaluation capabilities while reducing computational costs. iii) The methodology involves evaluating eight different models, including reasoning-based LLMs, distilled variants, and conventional LLMs, using GEMBA-MQM for MT evaluation and G-Eval for TS evaluation, across the WMT23 and SummEval benchmarks. iv) Primary results indicate that OpenAI’s o3-mini models show performance improvements with increased reasoning intensity, achieving the highest overall Eval4NLP scores of 0.644 and 0.645, while DeepSeek-R1 generally underperforms compared to its non-reasoning variant. v) A principal implication for AI practitioners is that the efficacy of reasoning capabilities for NLG evaluation is highly architecture-dependent, and distillation of reasoning capabilities maintains reasonable performance in medium-sized models but degrades substantially in smaller variants, requiring careful consideration of model architecture and task alignment. |
| MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in |
|
|
| Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Jiaxin Ai, Zhaopan Xu, Xiaopeng Peng, Fanrui Zhang, Pengfei Zhou |
i) MDK12-Bench is introduced as a new multi-disciplinary benchmark for evaluating multimodal reasoning in large language models (MLLMs) using K-12 level examinations. ii) The research aims to address the limitations of existing benchmarks by providing a more comprehensive evaluation of MLLMs’ reasoning capabilities across multiple disciplines. iii) The methodology involves curating a dataset of 140K reasoning instances spanning six disciplines, annotating instances with knowledge points, and developing a dynamic evaluation framework to mitigate data contamination through bootstrapped unseen data. iv) Experiments showed that Gemini2-thinking achieves the highest overall accuracy of 59.4% on the MDK12-Mini dataset, and models demonstrate sensitivity to combined textual and visual bootstrapping. v) AI practitioners can utilize MDK12-Bench to identify specific knowledge gaps in MLLMs, facilitating targeted improvements in multimodal reasoning capabilities, particularly in areas such as contextual comprehension and resistance to data contamination. |
Papers for 2025-04-14
| Title |
Authors |
Summary |
| Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model (Read more on arXiv or HuggingFace) |
Zhijie Lin, Ceyuan Yang, Team Seawead, zhenheny, lingff |
This paper details a cost-effective strategy for training Seaweed-7B, a 7-billion parameter video generation foundation model using moderate compute. The primary objective was to demonstrate that a medium-sized video generation model can achieve competitive performance compared to much larger models trained with significantly greater computational resources. Key methodologies involved training a novel 64x compression Variational Autoencoder (VAE) and a hybrid-stream Diffusion Transformer (DiT) from scratch on curated data using 665,000 H100 GPU hours, employing multi-stage training, SFT, DPO, and infrastructure optimizations like 3D parallelism and Multi-Level Activation Checkpointing (MLAC). Seaweed-7B achieved competitive performance, ranking second in image-to-video generation Elo ratings (1047 Elo, 58% win rate) against models like Sora and Wan 2.1, and its VAE obtained state-of-the-art reconstruction (e.g., 0.0391 LPIPS on UCF-101). Its distilled version requires only 12 NFEs for inference, 62 times faster than Wan 2.1 (100 NFEs). For AI practitioners, this work implies that careful design choices in data curation, VAE/DiT architecture, and training/inference optimization enable the development of highly competitive, cost-effective video generation models without necessarily resorting to massive parameter counts. |
| GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for |
|
|
| Autoregressive Image Generation (Read more on arXiv or HuggingFace) |
Jiashi Feng, Zilong Huang, Jun Hao Liew, XihuiLiu, YuuTennYi |
GigaTok introduces a 3 billion parameter visual tokenizer for autoregressive image generation that improves reconstruction, generation, and representation quality simultaneously during scaling. The research aims to overcome the common dilemma where scaling visual tokenizers improves reconstruction but degrades downstream generation performance. Key methods involve semantic regularization using features from a pre-trained DINOv2 model, employing 1D Q-Former based tokenizers, prioritizing decoder scaling in an asymmetric architecture, and using entropy loss for billion-scale training stability. The proposed 2.9B parameter GigaTok, when paired with a 1.4B AR model, achieves state-of-the-art autoregressive generation performance with a gFID of 1.98* on ImageNet 256x256. AI practitioners can apply semantic regularization and the identified scaling practices (1D tokenizers, asymmetric scaling, entropy loss) to develop larger, more effective visual tokenizers for generative models without sacrificing downstream performance due to increased latent space complexity. |
| MineWorld: a Real-Time and Open-Source Interactive World Model on |
|
|
| Minecraft (Read more on arXiv or HuggingFace) |
Yushu Jiang, Haoyu Wu, Tianyu He, Yang Ye, Junliang Guo |
MineWorld introduces a real-time, open-source, interactive world model for Minecraft based on an autoregressive Transformer. The primary objective is to develop an efficient and controllable world model capable of real-time interaction by predicting future game states conditioned on past states and actions. Key methodology involves tokenizing visual game states and player actions, feeding them interleaved into a Transformer trained via next-token prediction, and employing a novel parallel decoding algorithm for inference acceleration. Results demonstrate the model’s efficacy, with the 1.2B parameter version achieving 3.01 FPS, a discrete action F1 score of 0.73, and camera control L1 loss of 1.02, significantly outperforming diffusion-based baselines while the parallel decoding provides over 3x speedup. For AI practitioners, MineWorld offers a validated open-source framework and an efficient parallel decoding technique for building fast, interactive simulators essential for agent training and human-AI interaction in complex environments. |
| PixelFlow: Pixel-Space Generative Models with Flow (Read more on arXiv or HuggingFace) |
Ping Luo, Peize Sun, Shilong Zhang, Chongjian Ge, Shoufa Chen |
i) PixelFlow, a novel image generation model, performs image generation directly in raw pixel space through cascade flow modeling. ii) The research aims to develop an end-to-end trainable image generation model operating directly in pixel space, avoiding the need for pre-trained VAEs and cascaded upsampling. iii) PixelFlow employs a cascade flow modeling strategy, operating on multi-scale samples across cascading resolutions and using Flow Matching for velocity prediction. iv) PixelFlow achieves an FID of 1.98 on the 256x256 ImageNet class-conditional image generation benchmark. v) The PixelFlow framework provides AI practitioners with a simpler, end-to-end trainable alternative to latent-space diffusion models, enabling efficient pixel-space image generation with competitive performance. |
| SQL-R1: Training Natural Language to SQL Reasoning Model By |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Ran Chen, Xuhui Jiang, Chengjin Xu, Peixian Ma, ZhuangXialie |
i) This paper introduces SQL-R1, an NL2SQL reasoning model trained via reinforcement learning to improve performance in complex scenarios. ii) The research aims to enhance NL2SQL inference performance in complex database scenarios using reinforcement learning. iii) The methodology involves training a NL2SQL model using reinforcement learning with a specialized reward function and a cold start strategy based on supervised fine-tuning. iv) SQL-R1 achieves execution accuracy of 88.6% on the Spider benchmark and 66.6% on the BIRD benchmark using a 7B base model. v) AI practitioners can leverage the SQL-R1 model to achieve competitive accuracy in NL2SQL tasks with limited data and improved reasoning capabilities, demonstrating the potential of RL in optimizing NL2SQL performance. |
| FlexIP: Dynamic Control of Preservation and Personality for Customized |
|
|
| Image Generation (Read more on arXiv or HuggingFace) |
Kaiwen Xiao, Yanning Zhou, Haonan Lin, DevLinyan |
FlexIP is introduced as a novel framework for decoupling identity preservation and personalized editing in image generation. The research aims to enable flexible, parameterized control during inference through dynamic tuning of the weight adapter in generative models. FlexIP uses a dual-adapter architecture comprising a Personalization Adapter and a Preservation Adapter, coupled with a dynamic weight gating mechanism to balance identity retention and stylistic variation. Experiments demonstrate that FlexIP achieves a 61.4% controllability (Flex score) and 76.8% ID-Pres score. The framework offers AI practitioners a robust and flexible solution for subject-driven image generation by enabling continuous parametric control of the preservation-editability trade-off. |
| In-2-4D: Inbetweening from Two Single-View Images to 4D Generation (Read more on arXiv or HuggingFace) |
Ali Mahdavi-Amiri, Hao Zhang, Daniel Cohen-Or, Sauradip Nag |
i) This paper introduces In-2-4D, a method for generating 4D (3D object + motion) interpolations from two single-view images. ii) The primary objective is to generate and reconstruct a smooth 4D motion sequence given only start and end state images of an object. iii) The method uses a hierarchical approach involving video interpolation models, keyframe selection based on motion and appearance analysis, 3D Gaussian Splatting for static 3D representation, and dynamic Gaussian generation via a deformation field optimized with multi-view diffusion priors. iv) The method achieves improved performance on a newly introduced I4D-15 benchmark, outperforming baselines in terms of appearance (LPIPS: 0.103, FVD: 679.23) and geometry (SI-CD: 22.67, CD: 0.59), with user studies indicating a preference for the generated 4D motion quality (1.29 rating). v) The approach provides AI practitioners with a method for generating dynamic 3D content from minimal input, enabling applications in content creation and animation by requiring less data and allowing for diverse motion synthesis. |
| ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on |
|
|
| Transformer Encoder Models Performance (Read more on arXiv or HuggingFace) |
Djamé Seddah, Benoît Sagot, Wissam Antoun |
This paper conducts a controlled comparison of ModernBERT and DeBERTaV3 architectures by pretraining them on identical French datasets. The objective is to disentangle architectural advantages from training data differences in explaining performance variations between these transformer encoder models. The methodology involves pretraining French ModernBERT on the same 275B token dataset as CamemBERTaV2 (a French DeBERTaV3 model) and evaluating on French NER, QA, and classification tasks. Results show DeBERTaV3 (CamemBERTaV2) achieves superior benchmark performance (e.g., 83.04 F1 QA vs. 81.34 F1 for ModernBERT-CV2) and sample efficiency when data is controlled, while ModernBERT offers faster training/inference speeds. For AI practitioners, this implies a trade-off: DeBERTaV3 yields higher accuracy, whereas ModernBERT provides better computational efficiency, highlighting the need to evaluate models under shared data conditions for fair comparison. |
| Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend |
|
|
| NPUs (Read more on arXiv or HuggingFace) |
Xueyu Wu, Yehui Tang, Kaikai Song, Wenyong Huang, Yichun Yin |
Pangu Ultra is a 135B parameter dense Transformer LLM trained on 13.2 trillion tokens using 8,192 Ascend NPUs. The primary objective was to explore the performance limits of large-scale dense LLMs and address the associated training stability and system efficiency challenges on Ascend hardware. Methodology involved proposing depth-scaled sandwich normalization and TinyInit for stable training of the 94-layer model, alongside system optimizations like NPU Fusion Attention (NFA) and MC2 for efficient training, achieving over 50% MFU. Results show Pangu Ultra significantly outperforms comparable dense models like Llama 3.1 405B (e.g., 90.3% vs 72.5% on C-Eval) and achieves competitive results against larger sparse MoE models such as DeepSeek-R1. For AI practitioners, this work validates the capability of Ascend NPUs for efficiently training >100B parameter dense models and demonstrates that optimized dense architectures can achieve state-of-the-art performance comparable to sparse models, potentially offering simpler inference deployment. |
| SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder |
|
|
| Guardrails for Precision Unlearning in LLMs (Read more on arXiv or HuggingFace) |
Virginia Smith, Mona Diab, Jacopo Bonato, Aashiq Muhamed |
i) This paper introduces Dynamic SAE Guardrails (DSG), an activation-based method using Sparse Autoencoders (SAEs) that significantly improves precision unlearning in LLMs compared to gradient-based approaches. ii) The primary objective is to develop an unlearning technique that effectively removes targeted knowledge from LLMs while preserving general utility, addressing limitations of existing methods like high cost, instability, and poor data efficiency. iii) DSG employs principled feature selection using Fisher Information approximation via squared SAE activations to identify forget-relevant features and uses a dynamic, input-dependent classifier with a statistically determined threshold to conditionally clamp these features during inference. iv) Experiments demonstrate DSG substantially outperforms baseline methods, achieving a superior forget-utility trade-off by reducing WMDP-Bio accuracy to 29.64% (vs. 50.00% for the next best, RMU) while maintaining high MMLU accuracy (99.34%) and offering better computational efficiency, hyperparameter stability, and sequential unlearning performance. v) For AI practitioners, DSG provides a more computationally efficient, stable, interpretable, and data-efficient mechanism for targeted knowledge removal, enhancing LLM safety, privacy, and maintenance capabilities without requiring gradient computations during intervention. |
| Do PhD-level LLMs Truly Grasp Elementary Addition? Probing Rule Learning |
|
|
| vs. Memorization in Large Language Models (Read more on arXiv or HuggingFace) |
Zhenzhong Lan, Renjun Xu, Yu Lu, Yang Yan |
This paper probes whether Large Language Models genuinely understand elementary addition principles or rely on pattern memorization. The research investigates if LLMs learn generalizable arithmetic rules or merely exploit statistical patterns when performing two-integer addition. Methodology involves evaluating LLMs on addition tasks using standard digits versus isomorphic symbolic mappings, testing commutativity (A+B vs B+A), and analyzing performance scaling with digit count. Results show that while models achieve high numerical accuracy (73.8-99.8%), performance collapses to ≤7.5% under symbolic mapping, indicating a failure to generalize learned rules beyond familiar patterns. The principal implication for AI practitioners is that current LLMs heavily rely on memorization over true rule learning for arithmetic, necessitating more rigorous evaluation methods to assess genuine mathematical reasoning capabilities before deployment. |
| CoRAG: Collaborative Retrieval-Augmented Generation (Read more on arXiv or HuggingFace) |
Virginia Smith, Mona Diab, Aashiq Muhamed |
i) The paper introduces CoRAG, a framework for collaborative retrieval-augmented generation. ii) The research investigates how to effectively train RAG models in collaborative settings with shared passage stores, addressing the challenges of data heterogeneity and client incentives. iii) The methodology involves developing a novel benchmark, CRAB, for homogeneous open-domain question answering and comparing CoRAG against parametric collaborative learning and local RAG baselines using FedAvg. iv) Experiments on CRAB show CoRAG consistently outperforms baselines in few-shot settings, achieving a 33.8% improvement over local RAG at 16-shot; further analysis reveals that relevant passages are crucial, hard negatives are detrimental, while irrelevant passages can even be beneficial for model performance. v) AI practitioners can leverage CoRAG to improve model performance in low-resource, collaborative knowledge-intensive tasks by careful curation of the shared passage store, balancing the inclusion of relevant and irrelevant passages while minimizing hard negatives. |
| InteractVLM: 3D Interaction Reasoning from 2D Foundational Models (Read more on arXiv or HuggingFace) |
Cordelia Schmid, Omid Taheri, Shashank Tripathi, Dimitrije Antić, saidwivedi |
i) InteractVLM estimates 3D human-object contact points from single images by leveraging 2D vision-language models. ii) The research objective is to accurately estimate 3D contact points between humans and objects from in-the-wild 2D images to improve joint reconstruction without relying on extensive 3D contact annotations. iii) The methodology involves a “Render-Localize-Lift” module using multi-view rendering, a novel multi-view localization model (MV-Loc), and fine-tuning a VLM with limited 3D contact data. iv) InteractVLM achieves a 20.6% improvement in F1 score over existing methods for binary human contact prediction on the DAMON dataset. v) InteractVLM enables AI practitioners to improve 3D human-object interaction reconstruction from 2D images using predicted contact points and minimal 3D annotation, improving the realism and accuracy of HOI reconstruction. |
Papers for 2025-04-11
| Title |
Authors |
Summary |
| Kimi-VL Technical Report (Read more on arXiv or HuggingFace) |
dongliangwang, congcongwang, DuChenZhuang, tzzcl, xingbowei |
Kimi-VL is presented as an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM). The objective is to develop a VLM offering advanced multimodal reasoning, long-context understanding (128K), and strong agent capabilities while activating only 2.8B parameters in its language decoder. Methodology involves pairing a native-resolution MoonViT vision encoder with an MoE language model (Moonlight), trained through multi-stage pre-training, joint supervised fine-tuning (SFT), and enhanced with long-CoT SFT and reinforcement learning (RL) for the Kimi-VL-Thinking variant. Primary results show Kimi-VL competes effectively with larger VLMs across various benchmarks, while the Kimi-VL-Thinking variant achieves 61.7 on MMMU and 36.8 on MathVision, demonstrating strong long-horizon reasoning with its compact 2.8B activated LLM parameters. For AI practitioners, this research indicates the viability of using MoE architectures and native-resolution vision encoders to create parameter-efficient VLMs capable of complex multimodal reasoning, long-context processing, and agentic behavior. |
| VCR-Bench: A Comprehensive Evaluation Framework for Video |
|
|
| Chain-of-Thought Reasoning (Read more on arXiv or HuggingFace) |
lovesnowbest, Lin-Chen, Osilly, ChthollyTree, yukunqi |
VCR-Bench introduces a novel benchmark for comprehensively evaluating video Chain-of-Thought (CoT) reasoning capabilities in Large Vision-Language Models (LVLMs). The primary objective is to rigorously assess the entire reasoning process, differentiating failures originating from perception versus reasoning deficits, which current benchmarks inadequately address. Methodology involves a new dataset (VCR-Bench) with 859 videos and 1,034 QA pairs, featuring manually annotated, stepwise CoT rationales tagged for perception/reasoning, and a CoT score derived from recall/precision evaluation of these steps. Experiments reveal significant limitations in existing LVLMs, with the top-performing model achieving only a 62.8% CoT score and 56.7% accuracy, and most models exhibiting lower performance on perception steps compared to reasoning steps. For AI practitioners, VCR-Bench offers a standardized framework to identify specific weaknesses, particularly in temporal-spatial perception, providing actionable insights for improving LVLMs on complex video reasoning tasks. |
| MM-IFEngine: Towards Multimodal Instruction Following (Read more on arXiv or HuggingFace) |
yhcao, sweetFruit, KennyUTC, yuhangzang, ChrisDing1105 |
MM-IFEngine introduces a pipeline for generating multimodal instruction-following data and the MM-IFEval benchmark for evaluation. The research objective is to address the scarcity of high-quality training data and the limitations of existing benchmarks for evaluating multimodal instruction following (IF) in MLLMs. Key methodology involves the MM-IFEngine pipeline using LLMs (GPT-4o) for image filtering, task generation, and integrating 32 constraint categories to create the MM-IFInstruct-23k (SFT) and MM-IFDPO-23k (DPO) datasets, alongside the MM-IFEval benchmark featuring hybrid evaluation. Primary results show fine-tuning Qwen2-VL-7B on MM-IFDPO-23k significantly improves IF performance, achieving gains of +10.2% on MM-IFEval and +7.6% on MIA-Bench, while maintaining comparable VQA capabilities. For AI practitioners, this work provides datasets (MM-IFInstruct-23k, MM-IFDPO-23k) and a benchmark (MM-IFEval) to train and rigorously evaluate MLLMs for enhanced instruction adherence, crucial for applications needing precise, constrained multimodal outputs. |
| VisualCloze: A Universal Image Generation Framework via Visual |
|
|
| In-Context Learning (Read more on arXiv or HuggingFace) |
mingming8688, cosumosu25, JonsonYan, RuoyiDu, lzyhha |
VisualCloze presents a universal image generation framework leveraging visual in-context learning (ICL) to perform diverse tasks using a unified infilling model approach. Its primary objective is to overcome limitations of language-based instructions and task sparsity by enabling a model to understand and generalize visual tasks from examples. The key methodology involves formulating generation tasks as infilling problems on a grid of concatenated visual prompts and targets, fine-tuning the FLUX.1-Fill-dev model with LoRA on the proposed dense Graph200K dataset. Results demonstrate strong performance on in-domain tasks, generalization to unseen tasks, and task unification, with ICL quantitatively improving results (e.g., reducing Depth-to-Image RMSE from 10.31 to 9.68 using two in-context examples). For AI practitioners, this work implies that visual ICL combined with pre-trained infilling models offers a promising, unified paradigm for building versatile image generation systems that can learn complex visual relationships and adapt to new tasks with fewer explicit instructions compared to purely language-guided or task-specific models. |
| DeepSeek-R1 Thoughtology: Let’s about LLM Reasoning (Read more on [arXiv](https://arxiv.org/abs/2504.07128) or [HuggingFace](https://huggingface.co/papers/2504.07128)) |
parishadbehnam, miladink, vaibhavad, arkilpatel, spaidartaigar |
This paper introduces “Thoughtology,” a systematic analysis of the internal reasoning chains (“thoughts”) produced by the Large Reasoning Model (LRM) DeepSeek-R1. The main objective is to characterize DeepSeek-R1’s reasoning patterns, evaluate the impact of thought length and context on performance, and assess its safety and cognitive parallels. Key methodologies include developing a taxonomy for reasoning steps, quantitative evaluation on math (AIME-24, GSM8k, multiplication), long-context (Needle-in-a-Haystack, CHASE-QA/Code), safety (HarmBench), and cognitive/cultural benchmarks. Primary results indicate a consistent reasoning structure but reveal an optimal thought length ‘sweet spot’ beyond which performance declines; notably, DeepSeek-R1 also exhibits significant safety vulnerabilities, responding harmfully to 30.0% of direct HarmBench requests. For AI practitioners, this implies that controlling LRM thought length is crucial for performance and efficiency, yet DeepSeek-R1 lacks inherent mechanisms for this, and its reasoning capabilities introduce new safety risks requiring specific mitigation strategies beyond standard LLM alignment. |
| HoloPart: Generative 3D Part Amodal Segmentation (Read more on arXiv or HuggingFace) |
Lp256, zouzx, KevinHuang, bennyguo, yhyang-myron |
HoloPart introduces a generative approach for 3D part amodal segmentation, decomposing shapes into complete semantic parts, including occluded geometry. The primary objective is to address the limitations of standard 3D part segmentation by inferring and completing hidden part geometry while maintaining global shape consistency. The key methodology employs a two-stage approach: leveraging existing segmentation for initial surface patches, followed by HoloPart, a novel diffusion-based model using specialized local and context-aware attention mechanisms, to complete these patches into full parts. HoloPart significantly outperforms existing shape completion methods, achieving a mean instance IoU of 0.764 on the ABO benchmark compared to 0.565 for the next best baseline (Finetune-VAE). For AI practitioners, this work offers a tool to generate complete, semantically meaningful 3D parts from potentially incomplete data, enabling more robust downstream applications in 3D content creation, editing, and analysis. |
| C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization |
|
|
| for Test-Time Expert Re-Mixing (Read more on arXiv or HuggingFace) |
Ziyue Li, zhoutianyi, Lzy01241010 |
C3PO dynamically optimizes sub-optimal expert pathways in MoE LLMs at test-time to boost performance without retraining. The objective is to improve individual test sample predictions by re-mixing expert routing weights based on pathways from successful reference samples. C3PO employs collaborative optimization using neighbors in an embedding space to define a surrogate objective, focusing optimization on core experts within critical layers using methods like Neighborhood Gradient Descent (NGD). Results show C3PO improves base MoE accuracy by 7-15%; NGD on OLMoE-1B-7B achieved a 9.3% average accuracy increase (69.9% to 79.2%) across six benchmarks, enabling it to outperform 7-9B parameter dense models. AI practitioners can apply C3PO to enhance deployed MoE LLM performance on specific tasks or samples, potentially achieving higher accuracy with smaller models and reduced computational cost during inference. |
| MOSAIC: Modeling Social AI for Content Dissemination and Regulation in |
|
|
| Multi-Agent Simulations (Read more on arXiv or HuggingFace) |
Marzyeh Ghassemi, saadia, elisakreiss, salmannyu, genglinliu |
MOSAIC is an open-source multi-agent simulation framework using LLM agents to model social network content diffusion, user engagement, and moderation effects. The primary objective is to analyze LLM agent interactions, model misinformation propagation, and evaluate the efficacy of different content moderation strategies within a simulated social environment. The methodology employs LLM-driven agents (GPT-4o) assigned diverse personas who interact on a directed social graph, with their engagement patterns compared against human participants and tested under no-fact-checking, community-based, third-party, and hybrid moderation conditions. Key results indicate that simulated misinformation does not spread faster than factual content (unlike observed human behavior), and a hybrid fact-checking approach yielded the best balance of precision and recall (F1 score = 0.612) while enhancing factual content engagement. For AI practitioners, this suggests agent-based simulations can test moderation systems, but results must be critically evaluated as agent behavior, potentially influenced by safety training or simulation design, may deviate significantly from human patterns, impacting the direct applicability of findings to real-world platform governance. |
| Scaling Laws for Native Multimodal Models Scaling Laws for Native |
|
|
| Multimodal Models (Read more on arXiv or HuggingFace) |
Joshua Susskind, Matthieu Cord, Victor Guilherme Turrisi da Costa, Enrico Fini, Mustafa Shukor |
This paper investigates the scaling laws of native multimodal models (NMMs) trained from scratch, comparing early-fusion, late-fusion, and sparse architectures. The primary objective is to determine if commonly used late-fusion architectures hold an inherent advantage over early-fusion for NMMs and to characterize their scaling properties. The methodology involves training and evaluating 457 NMMs with varying architectures and training mixtures, deriving scaling laws by fitting power-law relationships between validation loss, compute (FLOPs), model parameters (N), and training tokens (D). Results indicate no inherent advantage for late-fusion; early-fusion performs comparably (loss L ∝ C^-0.049 for both) while being more parameter-efficient for compute-optimal models, and sparse Mixture-of-Experts (MoE) significantly improve early-fusion performance. For AI practitioners, this suggests early-fusion NMMs, trained natively and potentially enhanced with MoEs, offer a viable and efficient alternative to late-fusion approaches that rely on separate pre-trained vision encoders, especially at lower parameter counts. |
| SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual |
|
|
| Reasoning Self-Improvement (Read more on arXiv or HuggingFace) |
furongh-lab, kevinlin311tw, linjieli222, zyang39, russwang |
This paper presents an MCTS-guided data selection method for efficient visual reasoning self-improvement in VLMs using less data and no knowledge distillation. The main objective is to enhance VLM reasoning capabilities through reinforcement fine-tuning (RFT) using a minimal set of appropriately challenging training samples identified based on difficulty. The key methodology involves repurposing Monte Carlo Tree Search (MCTS) to quantify sample difficulty by measuring the iterations required for the base VLM (Qwen2.5-VL-7B-Instruct) to solve each problem, filtering 70k samples down to 11k. The resulting model, ThinkLite-VL-7B, trained on only 11k samples, achieves 75.1 accuracy on MathVista, surpassing larger models and improving the average benchmark performance of the base VLM by 7% (from 59.69 to 63.89). For AI practitioners, this demonstrates that strategically selecting challenging training data using MCTS for RFT can yield state-of-the-art reasoning performance in VLMs with significantly reduced data requirements, optimizing resource utilization. |
| Towards Visual Text Grounding of Multimodal Large Language Model (Read more on arXiv or HuggingFace) |
Franck-Dernoncourt, YfZ, JoshuaGu, zhangry868, MingLiiii |
This paper introduces TRIG, a novel task, benchmark (TRIG-Bench), and dataset to evaluate and improve the visual text grounding capabilities of Multimodal Large Language Models (MLLMs) on text-rich document images. The main research objective is to address the poor performance of existing MLLMs in localizing specific text regions within documents that support their generated answers for question-answering tasks. Methodology involved creating the TRIG-Bench benchmark (800 manually verified QA pairs) and a 90k synthetic instruction dataset using an OCR-LLM-human interaction pipeline, and proposing instruction-tuning and embedding-based grounding methods. Evaluation revealed significant limitations in current models on TRIG-Bench (e.g., GPT-4o achieved only 5.28% average pixel-level IoU in the OCR-free setting), while the proposed instruction-tuning method improved performance considerably to 29.98% average IoU after fine-tuning. For AI practitioners, this research provides a standardized benchmark and effective fine-tuning methods to assess and enhance MLLMs’ ability to ground answers in documents, crucial for building more trustworthy and verifiable document understanding systems. |
| MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular |
|
|
| Detection (Read more on arXiv or HuggingFace) |
R. Venkatesh Babu, Jogendra Kundu, Sarthak Vora, Srinjay Sarkar, RishubhPar |
MonoPlace3D learns realistic, scene-aware 3D object placement to generate effective data augmentations for improving monocular 3D object detection. The main objective is to automatically determine plausible 3D bounding box parameters (position, dimensions, orientation) for inserting synthetic objects into real scenes, addressing a key limitation of prior augmentation methods focused mainly on appearance. The methodology involves training a Scene-Aware Placement Network (SA-PlaceNet) on inpainted scenes to predict a distribution over plausible 3D boxes, then sampling from this distribution and rendering realistic objects using synthetic assets refined by ControlNet. MonoPlace3D significantly improves detection accuracy across multiple detectors and datasets; for example, on KITTI (easy, AP40@IOU=0.7) with MonoDLE, it boosted AP from 17.45% to 22.49% and achieved performance comparable to using the full dataset with only 50% of the data. For AI practitioners, this work demonstrates that focusing on learning physically plausible object placement is crucial for creating highly effective 3D data augmentations, leading to substantial gains in detector performance and data efficiency. |
| Compass Control: Multi Object Orientation Control for Text-to-Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
R. Venkatesh Babu, Vaibhav Agrawal, sachi1, RishubhPar |
Compass Control introduces a method for precise, explicit 3D orientation control of individual objects within text-to-image diffusion models. The primary objective is to enable users to specify the desired 3D orientation for multiple objects in a scene alongside a text prompt, overcoming the limitations and imprecision of text-only control. Key methodology involves predicting orientation-aware ‘compass tokens’ via a lightweight encoder, prepending them to object tokens in the text prompt, and using ‘Coupled Attention Localization (CALL)’ to constrain the cross-attention maps of compass and object tokens to corresponding 2D bounding box regions. The approach achieves superior orientation control, yielding a significantly lower angular error (0.198 radians for single objects, 0.215 for multiple) compared to baselines like LooseControl (0.385 and 0.372 respectively), and generalizes effectively to unseen objects and scenes with more than two objects. For AI practitioners, this provides a user-friendly interface for granular 3D orientation control in generative models using only orientation angles and coarse 2D boxes, enhancing predictability and streamlining creative workflows without requiring dense 3D data. |
| TAPNext: Tracking Any Point (TAP) as Next Token Prediction (Read more on arXiv or HuggingFace) |
rgoroshin, apsarath, msajjadi, skoppula, artemZholus |
TAPNext reformulates Tracking Any Point (TAP) in video as a sequential masked token decoding problem for online, low-latency tracking. The primary objective is to develop a simpler, more scalable TAP model by removing complex tracking-specific inductive biases and heuristics found in prior work. It employs a causal architecture combining ViT and SSM layers (TRecViT) to jointly process image patch tokens and masked point coordinate tokens, predicting trajectories via token imputation using a classification-based coordinate head. The method achieves state-of-the-art online tracking performance, with BootsTAPNext-B reaching 78.5 Average Jaccard (AJ) on DAVIS First at 256x256 resolution, outperforming previous frame-latency methods while operating purely online. For AI practitioners, TAPNext demonstrates that general-purpose sequence models with minimal task-specific components can achieve SOTA performance in complex correspondence tasks like point tracking, offering a potentially more scalable and easily adaptable approach for applications requiring online video understanding. |
Papers for 2025-04-10
| Title |
Authors |
Summary |
| DDT: Decoupled Diffusion Transformer (Read more on arXiv or HuggingFace) |
Weilin Huang, Zhi Tian, lmwang, wangsssssss |
This paper introduces the Decoupled Diffusion Transformer (DDT), separating semantic encoding and high-frequency detail decoding. The objective is to resolve the inherent optimization conflict in standard diffusion transformers, thereby accelerating training convergence and improving generation quality. DDT utilizes a distinct condition encoder for semantic extraction and a velocity decoder for detail generation, incorporating representation alignment and trained via linear flow matching. Key results show DDT-XL/2 achieves a state-of-the-art 1.31 FID on ImageNet 256x256 in 256 epochs, indicating approximately 4x faster convergence than prior diffusion transformers like REPA. For AI practitioners, DDT offers a significantly more efficient architecture for training high-fidelity diffusion models and introduces a statistical dynamic programming approach to accelerate inference by sharing encoder computations between steps with minimal performance loss. |
| GenDoP: Auto-regressive Camera Trajectory Generation as a Director of |
|
|
| Photography (Read more on arXiv or HuggingFace) |
lindahua, wetzste1, liuziwei7, jingtan, Dubhe-zmc |
This paper introduces GenDoP, an auto-regressive model, and DataDoP, a large-scale dataset, for generating artistic camera trajectories. The research aims to generate controllable, expressive camera trajectories based on multi-modal inputs (text, optional RGBD), addressing limitations in existing methods lacking directorial intent alignment or suffering from instability. The methodology involves creating the DataDoP dataset (29K shots, 11M frames) with detailed motion/directorial captions and developing GenDoP, a decoder-only Transformer that tokenizes camera parameters and generates trajectories auto-regressively. GenDoP significantly outperforms prior methods in text-trajectory alignment, achieving a CLaTr-CLIP score of 36.179 compared to 31.689 for a retrained baseline (Director3D) on the Motion caption task, and also shows superior user-rated alignment, quality, and complexity. For AI practitioners, this work provides a method for generating complex, instruction-following camera paths, enhancing controllability in camera-controlled video generation systems for applications like filmmaking and virtual cinematography. |
| OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training |
|
|
| Tokens (Read more on arXiv or HuggingFace) |
Yensung, sewon, yanaiela, taylorb, liujch1998 |
OLMOTRACE is a system that traces language model (LM) outputs back to their training data to understand LM behavior. The research question is how to efficiently trace LM outputs to their full multi-trillion-token training data in real time. The methodology uses an extended version of infini-gram to index the training data and a parallel algorithm to compute matching spans. The system traces LM responses (average 450 tokens) to the training data in 4.5 seconds on average. OLMOTRACE enables AI practitioners to explore the relationship between LM outputs and training data for fact-checking, creativity analysis, and understanding math capabilities. |
| A Unified Agentic Framework for Evaluating Conditional Image Generation (Read more on arXiv or HuggingFace) |
Yiyu Wang, Longyue Wang, Xue Yang, Jifang Wang, imryanxu |
i) The paper introduces CIGEVAL, a unified agentic framework leveraging large multimodal models (LMMs) for evaluating conditional image generation tasks. ii) The research aims to develop a task-agnostic, reliable, and explainable evaluation metric for conditional image generation. iii) CIGEVAL employs LMMs with a multi-functional toolbox and a fine-grained evaluation framework, synthesizing evaluation trajectories for fine-tuning smaller LMMs. iv) Experiments across seven conditional image generation tasks show CIGEVAL (GPT-40 version) achieves a Spearman correlation of 0.4625 with human assessments. v) CIGEVAL offers AI practitioners a more human-aligned and explainable method for automated evaluation of conditional image generation models, especially in tasks involving multiple conditions, and a pathway for fine-tuning smaller LMMs using synthesized evaluation trajectories for improved performance. |
| Missing Premise exacerbates Overthinking: Are Reasoning Models losing |
|
|
| Critical Thinking Skill? (Read more on arXiv or HuggingFace) |
Ming Li, zhoutianyi, sunlichao137, Fcr09 |
i) The paper investigates the effect of missing premises in questions on the response behavior of reasoning Large Language Models (LLMs). ii) The study aims to quantify and analyze the extent to which LLMs exhibit “MiP-Overthinking”, characterized by increased response length and ineffective reasoning on ill-posed questions with missing premises. iii) The methodology involves curating MiP datasets across varying difficulty levels, evaluating LLMs’ response length, accuracy, and abstain rate, and analyzing step-level similarities in reasoning chains. iv) Reasoning models generate responses 2x-4x longer for MiP questions compared to well-defined questions, contradicting test-time scaling law, while non-reasoning models generate responses of similar lengths for both. v) AI practitioners should be aware that current training paradigms for reasoning LLMs insufficiently promote efficient thinking, potentially resulting in resource inefficiencies and the abuse of reasoning patterns when faced with ambiguous input. It is unclear how in-process suspicion metrics are calculated in the paper. |
| FantasyTalking: Realistic Talking Portrait Generation via Coherent |
|
|
| Motion Synthesis (Read more on arXiv or HuggingFace) |
Yunpeng Zhang, Yaqi Fan, Mengchao Wang, fanjiang, wangqiang9 |
i) FantasyTalking generates realistic talking portraits from a single image via a dual-stage audio-visual alignment strategy. ii) The research aims to generate high-fidelity and coherent talking portraits with controllable motion dynamics from a static image. iii) The method utilizes a video diffusion transformer model with clip-level and frame-level audio-visual alignment and a facial-focused cross-attention module for identity preservation. iv) The proposed approach achieves state-of-the-art performance, demonstrating improved video quality, temporal consistency, and motion diversity, and achieves an aesthetic score of 0.6183 on the wild talking head dataset. v) AI practitioners can leverage this method for creating more realistic and controllable avatar animations, enhancing applications in gaming, filmmaking, and virtual reality. |
| A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths |
|
|
| to Reproducibility (Read more on arXiv or HuggingFace) |
AmeyaPrabhu, albanie, vishaal27, hrdkbhatnagar, libeanim |
i) This paper analyzes the reproducibility of recent advances in language model (LM) reasoning, identifying sensitivities to implementation choices and proposing a standardized evaluation framework. ii) The research investigates whether reported performance gains in mathematical reasoning benchmarks are robust to variations in decoding parameters, random seeds, prompt formatting, and hardware configurations. iii) The methodology involves a comprehensive empirical study re-evaluating recent methods using a standardized framework and assessing variance across multiple seeds and varying hyperparameters. iv) The study found reinforcement learning approaches yield only modest improvements and are prone to overfitting, while supervised finetuning shows consistently stronger generalization; Pass@1 values show standard deviations ranging from 5 to 15 percentage points across seeds. v) AI practitioners should adopt rigorous, multi-seed evaluation protocols and standardized testing frameworks to ensure the reliability and generalizability of LM reasoning enhancements before integrating them into applications. |
| OmniCaptioner: One Captioner to Rule Them All (Read more on arXiv or HuggingFace) |
Cxxs, Wayne-lc, Dakerqi, JiakangYuan, yeeeeeyy |
OmniCaptioner introduces a unified visual captioning framework for diverse domains. The main objective is to generate fine-grained textual descriptions for natural images, visual text (posters, UIs), and structured visuals (tables, charts, math) using a single model. The methodology involves a two-stage captioning pipeline (Seed-Caption Generation with GPT-40, Caption Extension with Qwen LLMs) trained on a 21M multi-domain dataset, initializing from Qwen2-VL-Instruct. Primary results show that integrating OmniCaptioner’s detailed captions with LLMs (e.g., DS-R1-Distill-Qwen-7B) significantly improves visual reasoning, achieving 40.5 on MathVerse without MLLM fine-tuning, enhances text-to-image generation (+2.97 on GenEval for SANA-1.0), and enables more efficient SFT (reaching comparable performance to LLaVA-OV-7B with ~1/3 of the SFT data). The principal implication for AI practitioners is the ability to leverage a single, versatile captioner to generate rich, domain-specific descriptions that directly enhance downstream visual reasoning systems, improve text-to-image generation quality, and accelerate supervised fine-tuning for various multimodal tasks. |
| Are We Done with Object-Centric Learning? (Read more on arXiv or HuggingFace) |
Matthias Bethge, coallaoh, AmeyaPrabhu, arubique |
i) This paper explores the limits of current object-centric learning (OCL) methods. ii) The main objective is to assess whether advances in OCL provide practical benefits beyond unsupervised object discovery, particularly in out-of-distribution (OOD) generalization scenarios. iii) The methodology involves introducing Object-Centric Classification with Applied Masks (OCCAM), a probe using sample-efficient segmentation models to generate object-centric representations and evaluate downstream classification tasks with spurious backgrounds. iv) The primary result shows that segmentation-based encoding of individual objects significantly outperforms slot-based OCL methods in robust zero-shot image classification, achieving up to 78.5% accuracy on ImageNet-D with HQES masks and SigLip models, which is superior to baseline LLAVA 1.5 (73.3%) and FT-Dinosaur (71.5%). v) The principal implication for AI practitioners is that utilizing foundational segmentation models for generating object-centric representations offers a more scalable and effective approach for robust classification tasks compared to traditional slot-centric OCL methods. |
| Self-Steering Language Models (Read more on arXiv or HuggingFace) |
Jacob Andreas, Vikash K. Mansinghka, Joshua B. Tenenbaum, Gabriel Grand, alexanderlew |
i) This paper introduces DISCIPL, a self-steering framework for language models (LMs) that decouples planning from execution by generating task-specific inference programs. ii) The main research question is how to enable LMs to perform complex reasoning tasks more efficiently and verifiably without extensive fine-tuning. iii) The methodology involves using a Planner LM to generate an inference program, which is then executed by a population of Follower LMs via Sequential Monte Carlo (SMC). iv) Experiments on constrained generation tasks show that DISCIPL, with a 1B Follower, matches or outperforms GPT-40 and 01 models and achieves 0.81 pass@1 on COLLIE sentence-level tasks. v) DISCIPL offers AI practitioners a method to automate the creation of highly parallelized Monte Carlo inference strategies for LMs, improving performance on challenging generation tasks. |
| RuOpinionNE-2024: Extraction of Opinion Tuples from Russian News Texts (Read more on arXiv or HuggingFace) |
Anna Lapanitsyna, Natalia Tkachenko, Natalia Loukachevitch, nicolay-r, RefalMachine |
i) The paper introduces the RuOpinionNE-2024 shared task for extracting structured opinion tuples from Russian news texts. ii) The primary objective is to extract tuples composed of a sentiment holder, target, expression, and polarity for a given sentence. iii) The methodology involved participants experimenting with large language models using zero-shot, few-shot, and fine-tuning techniques. iv) The best result on the test set was achieved through fine-tuning a large language model with an F1 score of 0.41. v) The principal implication for AI practitioners is the benchmark dataset and performance metrics for structured opinion extraction in Russian, enabling development and evaluation of models for Russian sentiment analysis. |
| Masked Scene Modeling: Narrowing the Gap Between Supervised and |
|
|
| Self-Supervised Learning in 3D Scene Understanding (Read more on arXiv or HuggingFace) |
Leon Sick, Christian Stippel, phermosilla |
i) The paper introduces a novel self-supervised approach, Masked Scene Modeling, for learning 3D scene representations. ii) The research aims to develop a self-supervised model for 3D scene understanding that can achieve performance comparable to supervised models when using off-the-shelf features. iii) The methodology involves a bottom-up hierarchical masking approach with a novel reconstruction objective tailored to hierarchical 3D models, reconstructing deep features of masked patches. iv) Experiments demonstrate that the proposed model achieves competitive performance in semantic segmentation (68.7 mIoU on ScanNet using linear probing) compared to supervised models, surpassing existing self-supervised methods. v) The principal implication is that the proposed self-supervised pre-training approach provides AI practitioners with a method to extract features from 3D scenes that perform comparably to supervised approaches, reducing the need for labeled data. |
| DiTaiListener: Controllable High Fidelity Listener Video Generation with |
|
|
| Diffusion (Read more on arXiv or HuggingFace) |
chaubeyG, hongkung, minhtran, Boese0601, havent-invented |
DiTaiListener is a video generation model for synthesizing high-fidelity listener head portraits conditioned on speaker audio, facial motions, and optional text prompts. The paper aims to generate controllable and temporally consistent listener behavior in video by adapting Diffusion Transformer (DiT) architecture. The method introduces a Causal Temporal Multimodal Adapter (CTM-Adapter) to process speaker audio and visual cues and DiTaiListener-Edit for refining transitional frames between video segments. DiTaiListener achieves a 73.8% improvement in FID score on RealTalk dataset and a 6.1% improvement on VICO dataset, signifying enhanced photorealism and motion representation. This work provides AI practitioners with an approach for generating realistic and customizable listener videos for applications in virtual avatars and human-computer interaction. |
| VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement |
|
|
| Fine-Tuning (Read more on arXiv or HuggingFace) |
Lanxingxuan, donglu, desenmeng, Aurorana, xinhaoli |
VideoChat-R1 enhances spatio-temporal perception in video MLLMs via reinforcement fine-tuning. The research aims to improve spatio-temporal perception in video MLLMs while preserving general capabilities. It employs Reinforcement Fine-Tuning (RFT) with Group Relative Policy Optimization (GRPO) on spatio-temporal objectives using limited data samples. VideoChat-R1 achieves state-of-the-art performance, improving temporal grounding by +31.8 and object tracking by +31.2 compared to Qwen2.5-VL-7B. RFT offers a data-efficient approach for specialized task enhancement in video MLLMs without sacrificing general capabilities, relevant to AI engineers developing video understanding systems. |
| WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments (Read more on arXiv or HuggingFace) |
Songyou Peng, Marc Pollefeys, Valentin Bieri, Zihan Zhu, Jianhao Zheng |
WildGS-SLAM is presented as a monocular SLAM system using 3D Gaussian Splatting robust to dynamic environments. The research aims to achieve accurate camera tracking and scene reconstruction in dynamic environments using only monocular RGB input. An uncertainty map derived from DINOv2 features is used to guide dynamic object removal within tracking and mapping pipelines. Evaluation on the Wild-SLAM MoCap dataset shows the system achieves an ATE RMSE of 0.46 cm, outperforming existing dynamic SLAM methods. Practitioners can leverage this method for improved SLAM performance in real-world applications with dynamically changing elements without explicit depth or semantic information. |
| RobustDexGrasp: Robust Dexterous Grasping of General Objects from |
|
|
| Single-view Perception (Read more on arXiv or HuggingFace) |
Jie Song, Sammy Christen, Linyi Huang, Zijian Wu, ethHuiZhang |
i) This paper introduces a reinforcement-learning-based framework for robust, zero-shot dynamic dexterous grasping of unseen objects from single-view perception. ii) The main objective is to enable a robot to grasp a wide range of previously unseen objects with a dexterous hand using only a single-view camera while adapting to external disturbances. iii) The methodology involves a mixed curriculum learning strategy that combines imitation learning from a teacher policy trained with privileged information and reinforcement learning for adaptation to disturbances, utilizing a hand-centric object representation. iv) The primary result is a grasping success rate of 97.0% across 247,786 simulated objects and 94.6% across 512 real objects without prior knowledge or object-specific training. v) The principal implication for AI practitioners is the demonstrated effectiveness of sparse hand-centric object representation and mixed curriculum learning for training robust dexterous grasping policies that generalize to unseen objects from limited observations, suggesting a path toward more adaptable and general-purpose robotic manipulation systems. |
Papers for 2025-04-09
| Title |
Authors |
Summary |
| OmniSVG: A Unified Scalable Vector Graphics Generation Model (Read more on arXiv or HuggingFace) |
Jiaxu Zhang, Xianfang Zeng, Yiying Yang, CH3COOK, wchengad |
OmniSVG is a unified framework leveraging pre-trained Vision-Language Models (VLMs) for end-to-end multimodal Scalable Vector Graphics (SVG) generation. The main objective is to produce high-quality, complex, and editable SVGs across diverse modalities (Text-to-SVG, Image-to-SVG, Character-Reference SVG), addressing the limitations of existing methods in handling complexity and structure. The key methodology involves parameterizing SVG commands and coordinates into discrete tokens using a dedicated SVG tokenizer and training a VLM (Qwen2.5-VL) on a large-scale dataset (MMSVG-2M) with a next-token prediction objective. Primary results demonstrate superior performance over existing methods; for instance, on the MMSVG-Illustration text-to-SVG task, OmniSVG(7B) achieved a FID score of 66.91, outperforming SVGDreamer (75.31 on MMSVG-Icon) and other baselines, while handling complex SVGs with token lengths up to 30k. For AI practitioners, OmniSVG offers a versatile, end-to-end solution for generating complex and editable vector graphics from multimodal inputs, potentially integrating into professional design workflows and overcoming the limitations of previous optimization-based or simpler auto-regressive approaches. |
| Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought (Read more on arXiv or HuggingFace) |
Jiangbo Pei, Yichen Wei, Xiaokun Wang, Chris, Yi Peng |
This paper introduces Skywork R1V, a 38B parameter multimodal model enhancing LLM reasoning for visual tasks using Chain-of-Thought. The primary objective is to efficiently transfer the reasoning capabilities of the text-based R1-series LLM to handle multimodal inputs without retraining the base LLM or vision encoder. Key methodologies include an efficient multimodal transfer via a lightweight MLP visual projector, a hybrid optimization framework combining Iterative SFT and GRPO, and an Adaptive-Length Chain-of-Thought distillation for data generation. Skywork R1V achieves competitive performance, notably scoring 69.0 on the MMMU benchmark and 94.0 on the text-based MATH500 benchmark. For AI practitioners, this work presents an open-source model and methodology demonstrating how to effectively build capable multimodal reasoning systems by efficiently adapting existing strong LLMs, offering a practical approach to enhance VLM reasoning without prohibitive retraining costs. |
| An Empirical Study of GPT-4o Image Generation Capabilities (Read more on arXiv or HuggingFace) |
Zhuoran Zhao, Sixiang Chen, donghao-zhou, QingyuShi, BryanW |
This paper empirically benchmarks GPT-4o’s image generation, revealing strengths like text rendering but limitations like inconsistency. The objective is to assess GPT-4o’s image generation capabilities by qualitatively benchmarking it against models like Gemini 2.0 Flash Experimental and domain-SOTA methods across >20 tasks (text-to-image, image-to-image, image-to-3D, image-to-X). Methodology relies on structured visual evaluation and error analysis (detailed qualitatively in Table 1) due to the lack of API access and unpublished architecture. Primary results show GPT-4o excels in exceptional text rendering, compositional prompt following, spatial reasoning, and image transformation, often surpassing benchmarks qualitatively, but exhibits limitations in inconsistent generation, hallucination, and data bias (e.g., non-Latin scripts); the study explicitly notes the qualitative nature and lack of quantitative metrics. For AI practitioners, GPT-4o’s notably strong text rendering capability demonstrates potential for unified models requiring precise visual-textual alignment, although current reliability issues (inconsistency, bias) warrant caution for direct deployment. |
| Hogwild! Inference: Parallel LLM Generation via Concurrent Attention (Read more on arXiv or HuggingFace) |
Vage Egiazarian, George Yakushev, Alina Shutova, Roman Garipov, Gleb Rodionov |
Hogwild! Inference: Parallel LLM Generation via Concurrent Attention This paper proposes Hogwild! Inference, a method enabling multiple instances of the same LLM to generate text in parallel while sharing and concurrently updating a common Key-Value attention cache. The main objective is to explore if LLMs can develop dynamic collaboration strategies for problem-solving without pre-defined frameworks, leveraging immediate access to each other’s partial progress. The key methodology involves running parallel LLM “workers” with a shared KV cache, utilizing Rotary Position Embeddings (RoPE) to efficiently manage positional information across workers and testing three cache layouts: contiguous, interleaved, and combined. Preliminary results on LIMO mathematical reasoning tasks show that the Hogwild! Combined layout allows multiple workers (e.g., 2 workers) to achieve higher accuracy faster than single-threaded baselines or independent parallel workers, reaching approximately 89% accuracy with an 8192 max forward pass budget, surpassing other methods at equivalent budgets. For AI practitioners, the principal implication is that existing reasoning-capable LLMs can potentially leverage shared KV caches for parallel, collaborative inference out-of-the-box to improve efficiency, without requiring model fine-tuning or explicit coordination protocols. |
| COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for |
|
|
| Alignment with Human Values (Read more on arXiv or HuggingFace) |
Siwei Wu, M-A-P Team, Liam-Liu, aaabiao, JinChengRen |
This paper introduces COIG-P, a large-scale (1,006k pairs), high-quality Chinese preference dataset generated via an LLM-based pipeline for human value alignment. The primary objective was to overcome limitations of existing Chinese preference datasets, such as small scale, narrow domains, lack of validation, and the scalability issues of human annotation. The methodology involved crawling and filtering 92k Chinese queries, using 15 LLMs to generate responses, and employing 8 LLMs to score and create chosen-rejected pairs without human intervention, alongside training an 8B Chinese Reward Model (CRM) and creating a Chinese Reward Benchmark (CRBench). Results show COIG-P significantly improves LLM performance on AlignBench, yielding gains of 2% to 12% for Qwen2/2.5 and Infinity-Instruct-3M-0625 models compared to training without it, and the developed CRM demonstrates scoring capabilities comparable to GPT-40 on a test split filtering task. For AI practitioners, COIG-P provides a valuable resource for aligning Chinese LLMs using methods like DPO, while the LLM-based annotation pipeline and the CRM offer scalable, cost-effective alternatives to manual annotation or reliance on expensive large models for data curation. |
| Less-to-More Generalization: Unlocking More Controllability by |
|
|
| In-Context Generation (Read more on arXiv or HuggingFace) |
Fei Ding, Yufeng Cheng, Mengqi Huang, wuwx, fenfan |
i) This paper introduces UNO, a universal customization framework enabling less-to-more generalization for controllable single-to-multi-subject image generation using in-context generation. ii) The research aims to develop a stable and scalable paradigm for subject-driven image generation that enhances controllability and consistency, particularly for multi-subject scenarios, while overcoming data limitations. iii) The key methodology is a model-data co-evolution approach, featuring a progressive synthetic data curation pipeline leveraging diffusion transformers’ in-context generation and the UNO model, which incorporates progressive cross-modal alignment and Universal Rotary Position Embedding (UnoPE) into a DiT architecture. iv) UNO demonstrates state-of-the-art results, achieving the highest DINO (0.760) and CLIP-I (0.835) scores on the DreamBench single-subject benchmark among tuning-free methods evaluated. v) For AI practitioners, UNO provides a tuning-free framework capable of generating high-fidelity images with strong subject similarity and text controllability for both single and multiple subjects, directly applicable to customization tasks without per-subject optimization. |
| Generative Evaluation of Complex Reasoning in Large Language Models (Read more on arXiv or HuggingFace) |
Baizhou Huang, Ruilin Yan, Xiangyu Wang, YitaoLiang, pkuHaowei |
This paper introduces KUMO, a generative evaluation framework combining LLMs and symbolic engines to dynamically create complex, contamination-resistant reasoning tasks for assessing large language models. The primary objective is to reliably evaluate genuine LLM reasoning capabilities, distinguishing it from memorization resulting from training data contamination of static benchmarks. KUMO employs a neural-symbolic pipeline utilizing LLMs for domain generation and SAT-based engines for task instantiation, creating partially observable, multi-turn reasoning games across numerous domains with adjustable difficulty, evaluated via success rate and relative action count. Key results from evaluating 23 LLMs on 5,000 tasks across 100 domains show reasoning-scaled models achieve university-level performance on complex tasks, and KUMO performance correlates strongly (Pearson correlation > 0.9 on hard setting vs MMLU-Pro/LiveBench-Reason) with recent real-world benchmarks, while experiments demonstrate resistance to overfitting. For AI practitioners, KUMO provides a scalable, dynamic, and contamination-resistant benchmark methodology for assessing the true reasoning progress of LLMs, facilitating more reliable model evaluation and development efforts compared to potentially saturated static datasets. |
| Tuning-Free Image Editing with Fidelity and Editability via Unified |
|
|
| Latent Diffusion Model (Read more on arXiv or HuggingFace) |
Ming-Hsuan Yang, Mike Zheng Shou, Yuchao Gu, Lan Chen, Qi Mao |
i) The paper introduces UnifyEdit, a tuning-free method for text-based image editing that balances fidelity and editability using a unified latent diffusion optimization framework. ii) The research aims to enable a balanced integration of fidelity and editability in text-based image editing without extensive retraining, addressing issues of over- or under-editing. iii) UnifyEdit employs self-attention preservation and cross-attention alignment constraints, along with an adaptive time-step scheduler, to guide diffusion latent optimization. iv) Experiments show UnifyEdit outperforms existing methods, demonstrating superior structure preservation and text alignment across various editing tasks, with user studies showing a 66%-84% preference for fidelity compared to baseline approaches. v) AI practitioners can utilize UnifyEdit for more robust and adaptable text-based image editing, achieving a better balance between preserving original image structure and accurately reflecting text-based modifications. |
| V-MAGE: A Game Evaluation Framework for Assessing Visual-Centric |
|
|
| Capabilities in Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Alex Jinpeng Wang, Ping Yu, Zhengyuan Yang, Linjie Li, Fengx1nn |
i) V-MAGE is introduced as a game-based framework to evaluate the visual reasoning capabilities of multimodal large language models (MLLMs). ii) The research aims to address limitations in current game-based benchmarks by providing visually-centric tasks that assess diverse reasoning skills. iii) The methodology involves evaluating leading MLLMs across five games with 30+ levels, using an adaptive Elo-based ranking system for performance comparison. iv) Results show a substantial performance gap between top-performing MLLMs and humans, with GPT-40 scoring 1.93/10 versus a human score of ≈10/10 in FlappyBird Level 6, while Qwen2VL-72B achieved 0.61/10 on the same task. v) V-MAGE highlights limitations in MLLMs’ visual perception and reasoning, suggesting a need to refine agent strategies and address perceptual inaccuracies from an agent-centric perspective for AI improvement. |
| CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs |
|
|
| with Controllable Puzzle Generation (Read more on arXiv or HuggingFace) |
William W. Cohen, Bill Yuchen Lin, Langlin Huang, Chengsong Huang, Jixuan Leng |
This paper introduces CrossWordBench, a benchmark using controllable crossword puzzles to evaluate the multimodal reasoning of LLMs and Large Vision-Language Models (LVLMs). The main objective is to assess model capabilities in handling tasks requiring simultaneous adherence to semantic constraints from text clues and structural constraints from visual grids. Methodologically, it utilizes a controllable puzzle generation framework creating text and image formats from diverse sources and evaluates over 20 models using zero-shot Chain-of-Thought and interactive modes. Results show reasoning LLMs significantly outperform non-reasoning models by leveraging crossing-letter constraints (achieving an 89% relative increase in Intersection Consistency Rate), while LVLMs perform poorly, with puzzle-solving performance strongly correlating (r=0.94) with grid-parsing accuracy. For AI practitioners, this highlights current LVLMs’ limitations in integrating visual-structural information with textual reasoning for constrained tasks and suggests the benchmark’s potential for developing and evaluating models with better spatial-textual grounding. |
| Accelerate Parallelizable Reasoning via Parallel Decoding within One |
|
|
| Sequence (Read more on arXiv or HuggingFace) |
Yijiong Yu |
The paper introduces a parallel decoding method, “Parallel Decoding in One Sequence,” for accelerating reasoning in Large Language Models (LLMs). The research aims to address the inefficiency of autoregressive decoding for tasks with parallelizable steps. The methodology involves identifying parallelizable steps, decoding them in parallel using a modified attention mask and position IDs within a single sequence, and then concatenating the results. Experiments demonstrate over 100% speedup in decoding time on a retrieval task with a context of 10 items while maintaining answer quality. This method enables AI practitioners to accelerate LLM reasoning on parallelizable tasks without additional memory usage or KV cache recomputation. |
| HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned |
|
|
| Guidance (Read more on arXiv or HuggingFace) |
Tong Wu, Pan Zhang, Yujie Zhou, Pengyang Ling, Jiazi Bu |
HiFlow introduces a training-free, model-agnostic framework for high-resolution text-to-image generation using pre-trained rectified flow models. The research aims to enhance image quality in high-resolution synthesis by establishing a virtual reference flow and aligning it with the high-resolution sampling flow through initialization, direction, and acceleration alignment. HiFlow achieves superior high-resolution image quality over state-of-the-art methods, demonstrating, for example, a FID score of 52.55 for 4096x4096 image generation. The flow-aligned guidance approach offers AI practitioners a method for improving image fidelity and detail in high-resolution T2I tasks without requiring model retraining. The paper does not provide information about the compute resources required or its use with large language models. |
| Leanabell-Prover: Posttraining Scaling in Formal Reasoning (Read more on arXiv or HuggingFace) |
Yang Yue, Yahui Liu, Xingguang Ji, Qi Wang, Jingyuan Zhang |
Leanabell-Prover improves automated theorem proving (ATP) through posttraining scaling of large language models using Lean 4 code. This research investigates posttraining techniques for ATP with the aim of achieving breakthroughs similar to those seen in natural language reasoning models. The study utilizes a hybrid dataset for continual training and GRPO for reinforcement learning, incorporating cognitive behaviors. Results show a 59.8% pass rate (pass@32) on the MiniF2F test after employing RL training, surpassing DeepSeek-Prover-v1.5-RL and Goedel-Prover-SFT. AI practitioners can leverage the proposed methods to enhance formal provers, leading to state-of-the-art performance in whole-proof generation. |
Papers for 2025-04-08
| Title |
Authors |
Summary |
| One-Minute Video Generation with Test-Time Training (Read more on arXiv or HuggingFace) |
guestrin, zhaoyue-zephyrus, GashonHussein, koceja, karansdalal |
This paper introduces Test-Time Training (TTT) layers integrated into a Diffusion Transformer to generate coherent one-minute videos from text storyboards. The main objective is to address the inefficiency of self-attention and the limited expressiveness of standard RNN hidden states for generating long videos with complex narratives. The key methodology involves adding TTT layers, whose hidden states are neural networks (specifically two-layer MLPs) updated via test-time gradient descent on a self-supervised reconstruction task, to a pre-trained CogVideo-X 5B model and fine-tuning on a curated Tom and Jerry dataset. The primary result shows that TTT layers significantly improve video coherence and storytelling for one-minute videos compared to baselines like Mamba 2 and Gated DeltaNet, leading by 34 Elo points in human evaluations, although some artifacts persist and efficiency needs improvement. For AI practitioners, this demonstrates TTT layers as a viable approach to enhance temporal consistency in long video generation, offering a mechanism to handle extended contexts beyond typical attention or RNN limitations, but requiring consideration of current efficiency trade-offs. |
| SmolVLM: Redefining small and efficient multimodal models (Read more on arXiv or HuggingFace) |
eliebak, mervenoyan, mfarre, orrzohar, andito |
SmolVLM introduces a family of compact, efficient Vision-Language Models (VLMs) designed for resource-constrained inference on edge devices. The primary objective was to engineer small VLMs by systematically exploring architectural configurations, tokenization strategies, and data curation optimized for low computational overhead and minimal memory footprints. Key methodologies included investigating encoder-LM parameter balance, optimizing context length and pixel shuffling for token reduction, evaluating learned versus string positional tokens, using image splitting, and carefully curating training data mixes (including CoT and video duration). Results show the smallest model (SmolVLM-256M) achieves a 44.0% average score across benchmarks using less than 1GB GPU RAM, outperforming significantly larger models, while the 2.2B variant rivals state-of-the-art models requiring double the GPU memory. For AI practitioners, the principal implication is that strategic architectural optimizations, aggressive tokenization, and curated data enable high-performance multimodal capabilities at much smaller scales, facilitating practical deployment on edge devices. |
| T1: Tool-integrated Self-verification for Test-time Compute Scaling in |
|
|
| Small Language Models (Read more on arXiv or HuggingFace) |
Jaewoong Cho, Jongwon Jeong, Nardien |
This paper introduces Tool-integrated Self-verification (T1) to enhance small language model (sLM) self-verification during test-time compute scaling by using external tools. The main research objective is to investigate if sLMs can reliably perform self-verification for test-time scaling, particularly for memorization-heavy tasks, and to improve this capability without resorting to larger models. The key methodology involves T1, a two-stage process combining a tool-based verifier (ToolV) leveraging external tools (e.g., code interpreter) for filtering, and a reward model (RM)-based verifier for scoring, with both components enhanced via knowledge distillation from larger teacher models. Primary results demonstrate that T1 significantly boosts sLM performance; specifically, a Llama-3.2 1B model using T1 under test-time scaling outperformed a significantly larger Llama-3.1 8B model on the MATH benchmark. The principal implication for AI practitioners is that integrating external tools via methods like T1 can substantially improve the reasoning and verification capabilities of computationally cheaper sLMs, enabling them to tackle complex tasks more effectively and potentially match larger model performance in specific domains. |
| URECA: Unique Region Caption Anything (Read more on arXiv or HuggingFace) |
Heeji Yoon, seungryong, crepejung00, junwann, SammyLim |
URECA introduces a large-scale dataset and novel model for generating unique captions for image regions at multiple granularities. The primary objective is to address the limitation of existing methods that struggle to produce distinctive descriptions for regions across varying levels of detail, especially distinguishing visually similar regions. The methodology involves a four-stage automated data curation pipeline utilizing mask trees and MLLMs to generate unique captions, and a captioning model featuring a dynamic mask encoder that preserves spatial properties for multi-granularity inputs. The proposed URECA model achieves state-of-the-art performance on the new dataset, attaining a BERTScore of 75.11, and demonstrates strong zero-shot generalization on benchmarks like Visual Genome with a METEOR score of 18.4. For AI practitioners, this work provides a robust dataset and model architecture enabling the generation of precise, context-aware natural language descriptions for arbitrarily selected image regions, enhancing detailed visual understanding applications. |
| Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning |
|
|
| Models (Read more on arXiv or HuggingFace) |
Yuxuan Sun, Tiezheng, baihaoli, manyi2024, ruikangliu |
This paper empirically investigates the impact of quantization on the reasoning abilities of large language models. The primary objective is to systematically evaluate how weight-only, KV cache, and weight-activation quantization affect reasoning performance across various model families, sizes, and tasks. The study quantizes DeepSeek-R1-Distilled Qwen/LLaMA families (1.5B-70B) and QwQ-32B using state-of-the-art algorithms (e.g., AWQ, QuaRot, FlatQuant) and evaluates them on mathematical, scientific, and programming reasoning benchmarks. Key findings reveal that W8A8 weight-activation or W4A16 weight-only/KV cache quantization can achieve near-lossless performance (≤1% accuracy drop), whereas lower bit-widths introduce significant risks, influenced by model size, origin (distilled vs. RL), and task difficulty. For AI practitioners, this implies that while 8-bit or selective 4-bit quantization can preserve reasoning with minimal loss, aggressive low-bit quantization requires careful consideration of the specific model and task, with FlatQuant and AWQ/QuaRot being preferred algorithms for weight-activation and weight-only/KV cache respectively. |
| Concept Lancet: Image Editing with Compositional Representation |
|
|
| Transplant (Read more on arXiv or HuggingFace) |
Hancheng Min, Tianjiao Ding, CCB, ryanckh, peterljq |
Concept Lancet (CoLan) introduces a zero-shot, plug-and-play framework for diffusion-based image editing using sparse concept decomposition and transplant in latent space. The research aims to solve the challenge of accurately determining the required edit strength for concept manipulation in images, avoiding over/under-editing without costly trial-and-error. CoLan employs a large curated concept dictionary (CoLan-150K), VLM-based parsing for task-specific concepts, and sparse coding to decompose the source latent vector (text embedding or diffusion score), allowing targeted replacement (transplant) of concept vectors. Equipping editing backbones like P2P-Zero with CoLan significantly improved consistency preservation, reducing LPIPS by nearly 50% (from 273.8/142.4 to 120.3/68.43 x10^-3 on whole image/background) while enhancing edit effectiveness on the PIE-Bench dataset. AI practitioners can integrate CoLan into diffusion editing pipelines to achieve more precise and consistent edits automatically by estimating and applying appropriate concept-specific magnitudes, eliminating the need for manual edit strength tuning per image. |
| LiveVQA: Live Visual Knowledge Seeking (Read more on arXiv or HuggingFace) |
Yao Wan, Mingyang Fu, shuaishuaicdp, Tim666, Ayiirep |
This paper introduces LIVEVQA, a benchmark dataset automatically collected from recent news to evaluate Multimodal Large Language Models (MLLMs) on live visual knowledge seeking. The research objective is to assess the capability of current MLLMs to answer questions demanding understanding of up-to-date visual knowledge synthesized from internet news content. Methodology involved creating the LIVEVQA dataset (3,602 single- and multi-hop visual questions from 1,233 news instances across 14 categories) and evaluating 15 MLLMs (e.g., GPT-4o, Gemma-3, Qwen-2.5-VL) with and without search tool integration. Primary results demonstrate that while stronger models perform better overall, significant performance gaps persist, particularly for complex multi-hop questions requiring recent visual knowledge; Gemini-2.0-Flash achieved the highest accuracy at 24.93% without search integration. The principal implication for AI practitioners is that current MLLMs, even sophisticated ones, struggle significantly with visual questions requiring timely, real-world knowledge and complex reasoning, highlighting a critical need for improved visual grounding and knowledge integration mechanisms. |
| Are You Getting What You Pay For? Auditing Model Substitution in LLM |
|
|
| APIs (Read more on arXiv or HuggingFace) |
Tianneng Shi, Will Cai, dawnsong, Xuandong |
This paper evaluates methods for detecting undisclosed model substitution in black-box Large Language Model (LLM) APIs. The objective is to formalize the API auditing problem and assess the robustness of software-based verification techniques (text classification, MMD, benchmarks, log probability analysis) and hardware solutions (TEEs) against adversarial attacks like quantization and randomized substitution. Methodology involves empirical evaluation of these techniques using various LLMs (Llama, Gemma, Mistral, Qwen2) under different attack scenarios, including comparing outputs, benchmark scores, and log probabilities. Primary results indicate that text-output-based methods are ineffective against subtle changes like quantization (e.g., text classifiers achieve only ~50% accuracy distinguishing original vs. quantized models) and randomized substitution (MMD test power drops significantly), while log probability analysis is more sensitive but relies on often unavailable API features; TEEs show promise with low performance overhead (<3% throughput impact under load). The principal implication for AI practitioners is that relying solely on current software-based verification for API model identity is unreliable, highlighting the need for enhanced provider transparency or hardware-attested environments like TEEs to ensure model integrity in critical applications and benchmarking. |
| Gaussian Mixture Flow Matching Models (Read more on arXiv or HuggingFace) |
saibi, wetzste1, luanfujun, zexiangxu, Lakonik |
GMFlow introduces a novel flow matching model predicting Gaussian mixture (GM) parameters instead of just the mean velocity to enhance generative modeling. The primary objective is to overcome the limitations of discretization errors in few-step sampling and color over-saturation issues associated with classifier-free guidance (CFG) in existing diffusion and flow matching models. Key methodology involves parameterizing the flow velocity as a GM, training with a KL divergence loss, deriving novel GM-SDE/ODE solvers that leverage analytic distributions, and introducing a probabilistic guidance mechanism for CFG reweighting rather than extrapolation. GMFlow demonstrates superior performance, achieving a Precision of 0.942 with only 6 sampling steps and a state-of-the-art Precision of 0.950 with 32 steps on ImageNet 256x256, significantly outperforming baselines, especially in few-step scenarios. For AI practitioners, this provides a framework for developing generative models capable of faster, higher-fidelity sampling with reduced CFG-induced saturation artifacts. |
| DiaTool-DPO: Multi-Turn Direct Preference Optimization for |
|
|
| Tool-Augmented Large Language Models (Read more on arXiv or HuggingFace) |
Donghun Lee, dsindex, junrae, gaeunseo, hash2430 |
This paper introduces DiaTool-DPO, a Direct Preference Optimization method enhancing Tool-Augmented LLMs’ multi-turn dialogue control for information gathering and tool rejection. The primary objective was to improve TA-LLM handling of incomplete or out-of-scope user queries by adapting DPO without requiring new expert demonstrations. Key methodology involves modeling interactions as a Markov Decision Process, automatically constructing paired chosen/rejected dialogue trajectory datasets based on defined query types, and applying a specialized DiaTool-DPO objective loss with turn-length normalization and reward gap margins. Experiments showed DiaTool-DPO significantly improved LLaMA3-8B-Instruct’s performance over SFT-only baselines, achieving 91.7% slot-filling accuracy (a 44% improvement) and 91.3% relevance accuracy (a 9.6% improvement), nearing GPT-4o performance. For AI practitioners, this method offers a way to train more robust TA-LLMs capable of managing ambiguous requests and unavailable tools using automatically generated preference data, reducing problematic tool calls without manual labeling. |
| VAPO: Efficient and Reliable Reinforcement Learning for Advanced |
|
|
| Reasoning Tasks (Read more on arXiv or HuggingFace) |
Ruofei Zhu, Xiaochen Zuo, Qiying Yu, Yufeng Yuan, YuYue |
This paper introduces VAPO, a value-based reinforcement learning framework designed to enhance the performance and efficiency of large language models on advanced reasoning tasks requiring long chain-of-thought. The primary objective is to overcome limitations inherent in value-based RL for long-CoT, specifically value model bias, handling heterogeneous sequence lengths, and sparse reward signals, aiming to surpass existing value-free methods. VAPO employs a modified Proximal Policy Optimization (PPO) approach incorporating seven key techniques, including Value-Pretraining, Decoupled and Length-Adaptive Generalized Advantage Estimation (GAE), Token-Level Loss, Clip-Higher clipping, Positive Example LM Loss, and Group-Sampling. Benchmarked on AIME 2024 using a Qwen-32B model, VAPO achieved a state-of-the-art score of 60.4 within 5,000 training steps, significantly outperforming the prior SOTA value-free method DAPO by over 10 points while demonstrating greater training stability and efficiency. For AI practitioners, VAPO presents a robust and efficient value-based RL alternative for training high-performance reasoning models, offering improved stability and potentially higher accuracy ceilings compared to value-free methods on complex, long-CoT tasks. |
| Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language |
|
|
| Models for Domain-Generalized Semantic Segmentation (Read more on arXiv or HuggingFace) |
robbytan, XinNUS |
This paper introduces MFuser, a Mamba-based framework to efficiently fuse Vision Foundation Models (VFMs) and Vision-Language Models (VLMs) for Domain-Generalized Semantic Segmentation (DGSS). The primary objective is to combine the complementary strengths of VFMs (fine-grained features) and VLMs (robust text alignment) while overcoming the challenges of long-sequence modeling and computational cost associated with integrating large models. The key methodology involves two components: MVFuser, a Mamba-based co-adapter for joint parameter-efficient fine-tuning of VFM and VLM visual features, and MTEnhancer, a hybrid attention-Mamba module to refine VLM text embeddings using visual priors. MFuser significantly outperforms existing DGSS methods, achieving a state-of-the-art 68.20 mIoU on the synthetic-to-real benchmark (G→{C, B, M} average) using DINOv2 and EVA02-CLIP. For AI practitioners, this work presents a computationally efficient Mamba-based adapter approach (MVFuser) to synergistically combine diverse foundation models, enhancing generalization for semantic segmentation tasks without requiring full fine-tuning of the base models. |
| BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose |
|
|
| Estimation (Read more on arXiv or HuggingFace) |
taeyeop, anas-gouda, mfourmy, swtyree, nv-nguyen |
The BOP Challenge 2024 advanced the state-of-the-art in 6D object pose estimation by introducing model-free tasks, new high-resolution datasets (BOP-H3), and a practical 6D detection task. The main objective was to shift evaluation from lab-like setups towards real-world applicability, notably by requiring methods to onboard unseen objects from reference videos without CAD models in model-free tracks. Key methodology involved evaluating methods across seven tracks defined by task (6D localization, 6D detection, 2D detection), onboarding setup (model-based, model-free), and dataset group (BOP-Classic-Core, BOP-H3) using established metrics like Average Recall (AR) and Average Precision (AP). Primary results showed significant progress: the best model-based 6D localization method for unseen objects (FreeZeV2.1) achieved 82.1 AR on BOP-Classic-Core, 22% higher than the 2023 best, though 2D detection for unseen objects still lags significantly (-53% behind seen objects), indicating it’s the main pipeline bottleneck. For AI practitioners, this highlights substantial improvements in unseen object pose estimation accuracy but underscores the critical need to advance 2D detection capabilities for robust real-world system deployment. |
| Clinical ModernBERT: An efficient and long context encoder for |
|
|
| biomedical text (Read more on arXiv or HuggingFace) |
Jeffrey N. Chiang, Anthony Wu, Simonlee711 |
This paper introduces Clinical ModernBERT, an efficient transformer encoder adapted for long-context biomedical and clinical text processing. The main objective is to leverage ModernBERT’s architectural improvements (RoPE, Flash Attention, GeGLU, 8192 token context) and adapt them via domain-specific pretraining for enhanced clinical language understanding. Methodology involved continued pretraining of a ModernBERT-base model on a 13-billion-token corpus comprising PubMed abstracts, MIMIC-IV clinical notes, and structured medical ontologies using masked language modeling with token-aware masking. Primary results demonstrate strong performance on clinical NLP benchmarks, achieving a state-of-the-art 0.9769 AUROC on EHR classification and superior runtime efficiency compared to BioClinicalBERT, processing data ~1.6x faster at higher volumes. The principal implication for AI practitioners is the availability of a performant, efficient, and publicly released encoder backbone specifically optimized for long clinical sequences and medical code semantics, suitable for replacing older BERT variants in clinical applications. |
| JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language |
|
|
| Model (Read more on arXiv or HuggingFace) |
Li Li, Yi Nian, yuehanqi, yuehanqi, Chouoftears |
i) JailDAM is a novel framework for detecting and mitigating jailbreak attacks on Vision-Language Models (VLMs) using an adaptive memory mechanism. ii) The research aims to develop a robust and efficient jailbreak detection method for VLMs, addressing the limitations of existing approaches, such as reliance on model internals or expensive computations. iii) The methodology involves a memory-based approach, using policy-driven unsafe knowledge representations, test-time adaptation to refine the memory with emerging unsafe variations, and an autoencoder-based detection pipeline. iv) Experiments on VLM jailbreak benchmarks demonstrate that JailDAM delivers state-of-the-art performance in harmful content detection, improving both accuracy and speed by an average of 0.10 AUROC compared to the second-best method. v) JAILDAM offers AI practitioners a black-box compatible and computationally efficient solution for detecting jailbreak attempts in VLMs, adaptable to new attack strategies without requiring extensive harmful data or model retraining, enhancing the safety and robustness of VLM deployments. |
| GlotEval: A Test Suite for Massively Multilingual Evaluation of Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Ona de Gibert, Sawal Devkota, Joseph Attieh, Zihao Li, zuenmin |
i) GlotEval is introduced as a lightweight massively multilingual evaluation framework for Large Language Models (LLMs). ii) The research aims to address the challenge of evaluating LLMs in diverse linguistic environments, especially low-resource languages, by providing a consistent and flexible evaluation framework. iii) The methodology involves integrating 20+ existing multilingual benchmarks across seven key tasks including machine translation, text classification, and summarization, standardizing language codes, and incorporating language-specific prompt templates with optional Microsoft Translator integration. iv) Experiments with Qwen2-1.5B model show throughput variances across languages and hardware setups, with Nvidia A100 generally achieving higher throughput than AMD MI250X; for example, French translation achieved 969.55 tokens/s on Nvidia A100. v) GlotEval offers AI practitioners a tool for fine-grained diagnostics of model strengths and weaknesses across a wide array of languages, facilitating the development of more inclusive and robust multilingual language technologies. |
| Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting |
|
|
| LLMs Across Languages and Resources (Read more on arXiv or HuggingFace) |
Jörg Tiedemann, Hengyu Luo, Shaoxiong Ji, Zihao Li |
i) This paper investigates data mixing strategies in multilingual continual pretraining (CPT) for adapting large language models (LLMs) across languages and resources. ii) The main objective is to evaluate the relative effectiveness of monolingual, bilingual, and code-augmented data strategies in multilingual CPT. iii) The study systematically evaluates 36 CPT configurations involving three multilingual base models across 30+ languages categorized as altruistic, selfish, and stagnant. iv) The findings reveal that bilingual CPT improves multilingual classification but often causes language mixing, while code data inclusion enhances classification but introduces generation trade-offs, with Llama-3.1-8B achieving only 7.47 BLEU with bilingual CPT versus 25.52 with monolingual CPT for high-resource languages. v) The principal implication for AI practitioners is the need for adaptive CPT methods that balance classification improvements and generation quality due to the complex interactions between language characteristics and data mixing strategies. |
Papers for 2025-04-07
| Title |
Authors |
Summary |
| Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving (Read more on arXiv or HuggingFace) |
Linhao Zhang, Hanwu Chen, Wei Liu, Zhirong Huang, Daoguang Zan |
This paper introduces Multi-SWE-bench, a multilingual benchmark for evaluating Large Language Models (LLMs) on software issue resolving tasks across diverse programming languages. The main objective is to overcome the limitations of existing Python-centric benchmarks like SWE-bench by providing a comprehensive evaluation framework for Java, TypeScript, JavaScript, Go, Rust, C, and C++. The methodology involved a five-phase pipeline including repository selection, pull request crawling, environment determination, automated filtering based on test outcomes, and rigorous manual verification by 68 experts, resulting in 1,632 high-quality instances; state-of-the-art LLMs were then evaluated using Agentless, SWE-agent, and OpenHands methods. Primary results show existing models struggle to generalize beyond Python, with performance significantly decreasing on complex tasks; for instance, resolved rates drop sharply when fix patches exceed 600 tokens or involve multiple files, indicating weaknesses in long-context retention and multi-file reasoning. For AI practitioners, Multi-SWE-bench offers a robust tool for assessing LLM capabilities in realistic, multilingual software engineering scenarios, revealing current limitations and guiding future development, alongside releasing initial datasets and infrastructure for reinforcement learning (Multi-SWE-RL) in this domain. |
| Agentic Knowledgeable Self-awareness (Read more on arXiv or HuggingFace) |
Xiangyuan Ru, Xiaobin Wang, Baochang Ren, Zhisong Qiu, Shuofei Qiao |
This paper introduces agentic knowledgeable self-awareness, enabling LLM agents to autonomously regulate knowledge utilization based on situational difficulty. The research objective is to overcome the limitations of traditional “flood irrigation” methods by allowing agents to decide when to use internal capabilities, reflect, or seek external knowledge. The proposed method, KnowSelf, employs a heuristic situation judgment criterion on self-explored trajectories and a two-stage (SFT + RPO) training process using special tokens to signify different cognitive states (fast, slow, knowledgeable thinking). Experiments demonstrate KnowSelf achieves superior performance with minimal knowledge; for instance, on ALFWorld using Llama-8B, it attained an 84.33% average reward while using external knowledge for only 15.01% of actions, outperforming baselines. For AI practitioners, this implies a method to train more efficient agents that dynamically manage computational resources (like reflection or knowledge retrieval) based on assessed task complexity, potentially reducing inference costs and improving robustness. |
| MegaMath: Pushing the Limits of Open Math Corpora (Read more on arXiv or HuggingFace) |
Liping Tang, Zhoujun Cheng, Nikhil Ranjan, Zengzhi Wang, Fan Zhou |
MegaMath introduces a large-scale, 371B token open dataset specifically curated for math-centric LLM pre-training. The primary objective was to address the lack of open, high-quality, large-scale corpora tailored for mathematical reasoning in LLMs. Methodology involved re-extracting and filtering Common Crawl data with math-specific optimizations, recalling math-relevant code from Stack-V2, and synthesizing QA, translated code, and interleaved text-code data. Key results demonstrate MegaMath’s scale and quality, with subsets like MegaMath-Web-Pro (15.1B tokens) outperforming existing open math corpora like FineMath-4+ by ≥ 4% in comparative pre-training evaluations, and boosting Llama-3 CoT performance by 15-20%. For AI practitioners, MegaMath provides a high-quality, large-scale open resource enabling the pre-training of more capable mathematical reasoning LLMs, previously hindered by the scarcity of suitable open datasets. |
| SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge |
|
|
| Refinement (Read more on arXiv or HuggingFace) |
Jialong Wu, Shuofei Qiao, Yuan Liang, Xiaobin Wang, Runnan Fang |
SynWorld introduces a framework for LLM-based agents to refine action knowledge by synthesizing virtual scenarios and using Monte Carlo Tree Search (MCTS) for exploration. The primary objective is to enable agents to autonomously enhance their understanding of actions and optimize workflows in novel or complex environments. The methodology involves synthesizing multi-step task scenarios conditioned on tool subsets and applying iterative MCTS optimization to refine action descriptions and cognitive workflows based on simulated environmental feedback. Key results demonstrate SynWorld’s effectiveness, achieving a 59.33 PASS score on ToolBench using GPT-4-turbo, outperforming several baseline methods. For AI practitioners, this implies a viable approach to automatically adapt agents to new tools and environments, improving planning and execution capabilities through simulated experience, thereby reducing reliance on manual annotation for action knowledge refinement. |
| MME-Unify: A Comprehensive Benchmark for Unified Multimodal |
|
|
| Understanding and Generation Models (Read more on arXiv or HuggingFace) |
Bingyan Nie, Yang Shi, Chaoyou Fu, Yi-Fan Zhang, Wulin Xie |
This paper introduces MME-Unify (MME-U), a comprehensive benchmark to evaluate Unified Multimodal Large Language Models (U-MLLMs) across understanding, generation, and novel unified tasks. The primary objective was to create a standardized evaluation framework addressing the lack of unified standards and benchmarks for mixed-modality generation capabilities in U-MLLMs. The methodology involved curating tasks from 12 datasets, standardizing formats (e.g., multiple-choice QA, normalized scores), and designing five new ‘unify’ tasks (e.g., Visual CoT, Image Editing & Explaining) requiring synergistic understanding and generation. Evaluations of 12 U-MLLMs revealed significant room for improvement, especially in instruction following and unified tasks, with the top model Gemini2.0-flash-exp achieving an MME-U score of 45.57, while many models struggled significantly on complex unified tasks. For AI practitioners, this highlights current U-MLLM limitations in reliably performing complex, integrated multimodal reasoning and generation, underscoring the need for improved model architectures and training strategies for robust real-world deployment. |
| VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via |
|
|
| Iterative Instruction Tuning and Reinforcement Learning (Read more on arXiv or HuggingFace) |
Liming Liang, Dongchao Yang, Yufan Deng, Yuxin Xie, Xianwei Zhuang |
VARGPT-v1.1 presents an improved unified visual autoregressive model for enhanced understanding and generation tasks. The objective is to advance the VARGPT framework by improving instruction-following, generation quality, and overall multimodal performance through enhanced training strategies and data scaling. Key methodology combines iterative visual instruction tuning (SFT) on an expanded 8.3M visual-generative instruction pair corpus with Direct Preference Optimization (DPO) reinforcement learning, upgrades the LLM backbone to Qwen2-7B, increases generation resolution, and enables editing capabilities via SFT. The model achieves state-of-the-art results on multimodal understanding benchmarks, such as 81.01 on MMBench, significantly improving comprehension and generation metrics over its predecessor and comparable models. For AI practitioners, this work demonstrates that iterative SFT and DPO-based RL within a purely visual autoregressive framework can yield highly capable unified multimodal systems, offering an alternative architecture to diffusion-based or separate component approaches. |
| APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated |
|
|
| Agent-Human Interplay (Read more on arXiv or HuggingFace) |
Ming Zhu, Jianguo Zhang, Weiran Yao, Zuxin Liu, Akshara Prabhakar |
This paper introduces APIGen-MT, a two-phase framework for generating verifiable multi-turn agent interaction data via simulated agent-human interplay. The primary objective was to overcome the scarcity of high-quality, realistic multi-turn data needed for training capable AI agents. The methodology involves first generating verified task blueprints using an agentic pipeline with LLM reviewers and feedback, followed by simulating human-agent interactions based on these blueprints to create full trajectories. Key results show models trained on this data (xLAM-2-fc-r series) outperform strong baselines; for instance, the 70B model achieved 78.19% accuracy on BFCL v3, surpassing GPT-40, with smaller models also demonstrating superior multi-turn consistency. For AI practitioners, this work provides open-source, high-quality synthetic data and models enabling the development of more reliable agents for complex, multi-turn interactions, potentially allowing smaller models to achieve performance comparable to larger ones. |
| HumanDreamer-X: Photorealistic Single-image Human Avatars Reconstruction |
|
|
| via Gaussian Restoration (Read more on arXiv or HuggingFace) |
Guosheng Zhao, Xiaofeng Wang, Runqi Ouyang, Boyuan Wang, ZhengZhu |
HumanDreamer-X introduces a unified pipeline for photorealistic single-image 3D human avatar reconstruction by integrating multi-view generation and Gaussian restoration. The primary objective is to overcome geometric inconsistencies and visual artifacts like fragmented limbs common in decoupled generation-then-reconstruction approaches for single-view inputs. The methodology involves initial coarse avatar reconstruction using 3D Gaussian Splatting (3DGS), rendering multi-view video frames, refining these frames with a video restoration model named HumanFixer which incorporates an attention modulation strategy, and subsequently using the restored video to enhance the 3DGS model. Key results show significant improvements over existing methods, achieving up to 25.62 dB PSNR in reconstruction quality, a 12.65% increase compared to prior SOTA on CustomHumans. For AI practitioners, this work demonstrates a technique combining explicit 3D representation (3DGS) with generative video restoration and attention modulation to create higher-quality, consistent digital humans from minimal input, applicable to virtual avatar creation and animation. |
| TransMamba: Flexibly Switching between Transformer and Mamba (Read more on arXiv or HuggingFace) |
Shuaipeng Li, Xingwu Sun, Ruobing Xie, andyyang, Yixinglee |
This paper proposes TransMamba, a framework unifying Transformer and Mamba using shared parameters to switch dynamically between attention and state space model (SSM) mechanisms. The objective is to leverage the strengths of both Transformer (short context efficiency) and Mamba (long context efficiency) within a single flexible architecture, overcoming static hybrid model limitations. TransMamba utilizes shared QKV/CBx parameters and introduces a “Memory Converter” for lossless state transfer at designated sequence positions (“TransPoints”), with a scheduling strategy determining the switch points across layers. Experiments show TransMamba achieves superior efficiency (e.g., 0.75 relative training time vs. 1.00 for Transformer at 1.5B parameters) and performance on benchmarks like LongBench-v2 (38.76 overall score vs. 31.61 for Transformer-1.5B) compared to baseline Transformer, Mamba2, and static Hybrid models. For AI practitioners, TransMamba presents a scalable architecture potentially offering improved training/inference efficiency and performance, especially for applications involving variable sequence lengths, by dynamically selecting the optimal computation mechanism (Attention or SSM) per token segment and layer. |
| Comprehensive Relighting: Generalizable and Consistent Monocular Human |
|
|
| Relighting and Harmonization (Read more on arXiv or HuggingFace) |
Zhixin Shu, Krishna Kumar Singh, Xin Sun, Jingyuan Liu, Junying Wang |
This paper presents Comprehensive Relighting, a novel diffusion-based framework for generalizable and temporally consistent monocular human relighting and background harmonization. The main objective is to develop a single model capable of controllably relighting humans in images/videos (using Spherical Harmonics or background scenes), ensuring harmonization and temporal coherence across arbitrary body parts and scenes without large-scale supervised video data. The methodology utilizes a pre-trained latent diffusion model in a coarse-to-fine framework conditioned via ControlNet on coarse shading and background inputs, combined with an unsupervisedly trained temporal module (using cycle consistency) integrated via spatio-temporal feature blending and followed by guided refinement. Results show superior performance over baselines, achieving, for example, the best temporal consistency score (tLPIPS of 0.026, lower is better) on a challenging synthetic video benchmark (Scenario 3), compared to the next best (0.028). For AI practitioners, this work demonstrates adapting diffusion priors with conditioning and unsupervised temporal learning offers a potent strategy for tackling complex, data-limited generative video tasks, enabling the development of more robust and controllable video editing/synthesis tools. |
| EvMic: Event-based Non-contact sound recovery from effective |
|
|
| spatial-temporal modeling (Read more on arXiv or HuggingFace) |
Lu Zhang, Xudong XU, Xu Jia, Shi Guo, yyzqy |
EvMic introduces a deep learning pipeline for non-contact sound recovery using event cameras, overcoming traditional camera limitations. The objective is to effectively recover sound signals from object vibrations captured by event cameras by modeling spatial-temporal event data. The methodology employs a laser matrix for enhanced gradient capture, a synthetic dataset (EvMic) for training, and a network combining sparse convolutions, Mamba for temporal modeling, and a spatial aggregation block (SAB) for fusing information from multiple locations. The proposed method achieves superior performance on synthetic data, yielding an average SNR of 1.214 dB, significantly outperforming the EvPhase baseline (-0.079 dB). For AI practitioners, this demonstrates the potential of event-based vision and tailored architectures (sparse ConvNets, SSMs like Mamba, attention) for recovering high-frequency signals from subtle physical phenomena, offering a new modality for sensor fusion and signal processing tasks. |
| MedSAM2: Segment Anything in 3D Medical Images and Videos (Read more on arXiv or HuggingFace) |
Mohammed Baharoon, Bihui Chen, Sumin Kim, Zongxin Yang, Jun Ma |
MedSAM2 is a promptable foundation model for general-purpose 3D medical image and video segmentation. The objective was to create a versatile model capable of segmenting diverse structures across modalities by overcoming the 2D limitations of prior work and enabling efficient large-scale annotation. The methodology involved fine-tuning the lightweight SAM2.1-Tiny architecture on a large curated dataset (>455k 3D pairs, 76k video frames) using bounding box prompts and a human-in-the-loop iterative refinement process. Primary results demonstrate superior segmentation performance over baseline SAM2.1 models across CT, MRI, PET, ultrasound, and endoscopy data, alongside a user study showing an over 85% reduction in manual annotation time for 3D CT lesions. For AI practitioners, MedSAM2 provides an efficient, deployable tool integrated into common platforms (3D Slicer, Gradio, etc.) to significantly accelerate the creation of large-scale annotated medical datasets and streamline segmentation workflows. |
| BEATS: Bias Evaluation and Assessment Test Suite for Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Lisa Erickson, tbandopa, alokabhishek |
This research introduces BEATS, a framework and benchmark using 29 metrics to evaluate Bias, Ethics, Fairness, and Factuality (BEFF) in Large Language Models. The main objective was to develop a systematic framework and establish a standard benchmark for measuring and detecting BEFF metrics within LLMs. Key methodology involved using a curated dataset of 901 evaluation questions, performing inference on five major LLMs, and employing a consortium of three LLMs-as-judges to score responses based on the BEFF metrics, followed by statistical analysis including ANOVA. The primary result showed that 37.65% of generated outputs from tested industry-leading models contained some form of bias, indicating substantial risk. For AI practitioners, this implies a critical need for rigorous bias assessment using tools like BEATS before deploying LLMs, especially in sensitive applications, to inform necessary mitigation strategies. |
Papers for 2025-04-04
| Title |
Authors |
Summary |
| Advances and Challenges in Foundation Agents: From Brain-Inspired |
|
|
| Intelligence to Evolutionary, Collaborative, and Safe Systems (Read more on arXiv or HuggingFace) |
KaitaoSong, JinlinW, Peiyan, xinfeng1i, Bang-UdeM-Mila |
This survey presents a comprehensive overview of LLM-powered Foundation Agents, proposing a modular, brain-inspired architecture integrating cognitive science and neuroscience principles. The main objective is to structure the understanding of advanced intelligent agents by exploring their modular foundations, self-enhancement mechanisms, collaborative/evolutionary dynamics, and safety aspects. The methodology involves a structured literature review and synthesis, mapping agent components (memory, world modeling, reward, emotion) to brain functions and analyzing self-optimization (AutoML, LLM-driven), multi-agent systems, and safety/ethical threats. As a survey, the paper synthesizes existing research across these four areas rather than presenting novel quantitative findings, identifying key research gaps, challenges, and opportunities. For AI practitioners, this work provides a unified framework for designing, evaluating, and ensuring the safety of complex Foundation Agents, emphasizing the need to harmonize modular design, adaptive capabilities, and collaborative potential with robust safety and ethical considerations. |
| Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual |
|
|
| Editing (Read more on arXiv or HuggingFace) |
Rethinker, GTZhai, KexianTang, zpy777, PhoenixZ |
This paper introduces RISEBench, the first benchmark designed to evaluate Reasoning-Informed Visual Editing (RISE) capabilities in Large Multi-modality Models (LMMs). The main objective is to systematically assess LMM performance on visual editing tasks requiring Temporal, Causal, Spatial, and Logical reasoning beyond simple pixel manipulation. The methodology involves curating image-instruction test cases for each reasoning type and evaluating model outputs (from models like GPT-4o-Native, Gemini-2.0-Flash, EMU2) using both human judges and an LMM-as-a-judge (GPT-4o) framework across dimensions of Instruction Reasoning, Appearance Consistency, and Visual Plausibility. Primary results indicate that while GPT-4o-Native significantly outperforms other models with a 35.9% overall accuracy, even this state-of-the-art model struggles notably with logical reasoning tasks (37.5% accuracy), and open-source models achieve near-zero accuracy on RISEBench. The principal implication for AI practitioners is that current SOTA LMMs exhibit significant deficiencies in integrating complex, especially logical, reasoning within visual editing, highlighting a critical area requiring further research and development before such capabilities can be reliably deployed. |
| GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
shawnxyh, BestWishYsh, SereinH, liweijia, Yejy53 |
This paper introduces GPT-ImgEval, a benchmark for quantitatively and qualitatively evaluating OpenAI’s GPT-4o model in image generation and editing tasks. The main objective was to assess GPT-4o’s performance across generation quality, editing proficiency, and world knowledge-informed synthesis, while also investigating its potential underlying architecture. Methodology involved evaluating GPT-4o using the GenEval, Reason-Edit, and WISE datasets via custom automation scripts, and employing a classification model trained to distinguish between diffusion and auto-regressive outputs to infer GPT-4o’s generation mechanism. Primary results indicate GPT-4o significantly surpasses prior models, achieving an overall score of 0.84 on GenEval, and empirical analysis suggests it likely uses a hybrid auto-regressive architecture combined with a diffusion-based head, contrary to VAR-like structures. For AI practitioners, this work provides a standardized evaluation framework, highlights GPT-4o’s advanced capabilities and specific limitations (e.g., editing inconsistencies, non-English text issues), and notes its outputs are detectable by current forensic models, impacting considerations for deployment and safety. |
| Rethinking RL Scaling for Vision Language Models: A Transparent, |
|
|
| From-Scratch Framework and Comprehensive Evaluation Scheme (Read more on arXiv or HuggingFace) |
Pengfei, IanZhong, Ryan1122, steffichern, ManTle |
This paper introduces MAYE, a transparent, from-scratch Reinforcement Learning (RL) framework for Vision-Language Models (VLMs), alongside a comprehensive evaluation scheme. The main objective is to improve reproducibility and standardized assessment in RL for VLMs, addressing limitations of complex, opaque existing frameworks. Methodologically, it presents a minimal four-step RL pipeline (using Reinforce++ with KL penalty) built with standard libraries and introduces an evaluation scheme tracking dynamics like accuracy curves, response length, and reflection ratios. Key results show RL consistently surpasses Supervised Fine-Tuning (SFT) generalization, achieving a 1.35x average accuracy increase (peaking at 1.76x) on the mm_math5k validation set compared to the baseline, even when SFT uses high-quality data; findings also indicate response length sensitivity to random seeds and correlation between reflection and output length. For AI practitioners, this provides a reproducible baseline framework (MAYE) for VLM RL experimentation and demonstrates RL’s potential for superior generalization over SFT on visual reasoning tasks, suggesting its utility even with access to good supervised data. |
| SkyReels-A2: Compose Anything in Video Diffusion Transformers (Read more on arXiv or HuggingFace) |
raul678, ruiwang, diqiu7, Debang, onion |
This paper introduces SkyReels-A2, an open-source framework for composing videos from text prompts and multiple reference images (characters, objects, scenes). The primary research objective is to generate high-fidelity videos that maintain strict identity consistency for each specified element while coherently composing the scene according to the text prompt, defining this as the elements-to-video (E2V) task. Key methodologies include a comprehensive data pipeline for constructing prompt-reference-video triplets, a novel joint image-text embedding model integrated into a diffusion transformer architecture with distinct spatial and semantic feature branches, and inference acceleration strategies. Evaluated on the proposed A2-Bench benchmark, SkyReels-A2 achieves comparable quantitative results to closed-source models, notably scoring 0.809 in object consistency, slightly outperforming competitors like Vidu (0.796) and Keling (0.790). For AI practitioners, SkyReels-A2 provides a publicly available model and benchmark for controllable multi-element video generation, facilitating development in areas requiring precise visual element control and composition, such as virtual e-commerce or creative content production. |
| Scaling Analysis of Interleaved Speech-Text Language Models (Read more on arXiv or HuggingFace) |
adiyoss, MajoRoth, hassid, gallilmaimon |
This paper analyzes the scaling behaviour of interleaved speech-text language models (SLMs), finding they scale more efficiently than textless SLMs. The main objective is to determine if SLMs trained with interleaved speech and text data scale more efficiently with compute compared to textless SLMs. The methodology involves training dozens of interleaved SLMs across various sizes (0.5B-7B), compute budgets (2e18-2e20 FLOPs), and TextLM initialisations (e.g., Qwen2.5, Llama3.2), evaluating performance on speech-only validation loss and semantic metrics (sSC, tSC) using an ISO-FLOP curve approach. Results show interleaved SLMs scale significantly better with compute, indicating compute budgets should favour larger model sizes over more training tokens; for a 2e20 FLOP budget, a 7B parameter model trained on 4.2B tokens outperformed smaller models on more tokens, contrasting with textless SLM scaling predictions. The principal implication for AI practitioners is that when training large interleaved SLMs (e.g., >4.5B tokens), allocating more compute towards larger, high-quality pre-trained TextLM-initialised models is more efficient than towards increasing training tokens alone for improving semantic speech abilities. |
| ShortV: Efficient Multimodal Large Language Models by Freezing Visual |
|
|
| Tokens in Ineffective Layers (Read more on arXiv or HuggingFace) |
xphan, sanmusunrise, luyaojie, chenjiawei-icip, yuanqianhao |
ShortV enhances Multimodal Large Language Model (MLLM) efficiency by identifying and freezing visual token computations in ineffective layers. The primary objective is to reduce the high computational overhead of MLLMs, specifically addressing redundancy in how different layers process visual tokens. A novel metric, Layer Contribution (LC), is introduced to quantify a layer’s impact by measuring the KL divergence in model output logits when that layer’s transformations on specific tokens (visual or text) are bypassed; ShortV uses LC to identify layers ineffective for visual tokens and replaces them with sparse layers where visual computations are frozen. Experiments demonstrate that ShortV can freeze visual token processing in approximately 60% of MLLM layers (e.g., achieving 50% FLOPs reduction on LLaVA-NeXT-13B with N=24 replaced layers) with negligible performance degradation. For AI practitioners, ShortV offers a training-free, parameter-free method to significantly decrease MLLM inference costs by exploiting layer-wise redundancy for visual tokens, and it is compatible with token pruning techniques. |
| Audio-visual Controlled Video Diffusion with Masked Selective State |
|
|
| Spaces Modeling for Natural Talking Head Generation (Read more on arXiv or HuggingFace) |
Jun Zhou, Zixiang Zhou, danxuhk, xuzn, HarlanHong |
This paper introduces ACTalker, an end-to-end video diffusion framework for natural talking head generation controlled simultaneously by audio and facial motion signals without conflict. The primary objective is to enable fine-grained control using multiple driving signals while preventing conflicts and ensuring spatio-temporal coherence. Key methodologies involve a parallel-control mamba (PCM) layer leveraging Masked Selective State Space Models (Mask-SSM) and a mask-drop strategy to direct each signal’s influence to specific facial regions within a stable video diffusion architecture. Experimental results demonstrate state-of-the-art performance, achieving a Sync-C score of 5.317 and an FVD-Inc score of 232.374 on the CelebV-HD dataset under audio-only control, surpassing previous methods. For AI practitioners, this work presents a novel application of Mamba (SSM) structures for efficient, conflict-free multi-modal conditioning in video generation, offering precise control over synthesized facial dynamics. |
| ZClip: Adaptive Spike Mitigation for LLM Pre-Training (Read more on arXiv or HuggingFace) |
gueraf, nilabhra, louisowen6, akanyaani |
ZClip introduces an adaptive gradient clipping method based on z-scores to enhance stability during large language model (LLM) pre-training. The primary objective is to mitigate gradient instability and malignant loss spikes that disrupt training, necessitating costly interventions like checkpoint restoration. ZClip dynamically adjusts the gradient clipping threshold by tracking the exponential moving average (EMA) of the gradient norm’s mean and standard deviation, applying z-score-based anomaly detection to identify and scale down spikes. Experiments on a 1B LLaMA model demonstrated that ZClip enabled stable training at a high learning rate (3.0x10⁻³), reaching baseline validation loss using 18.6B fewer tokens (over 35% faster) compared to fixed clipping at a lower, stable rate (5.0x10⁻⁴). For AI practitioners, ZClip offers a method to improve LLM pre-training stability and efficiency, potentially reducing training time and compute costs by allowing for more aggressive learning rates without succumbing to catastrophic divergence. |
| Inference-Time Scaling for Generalist Reward Modeling (Read more on arXiv or HuggingFace) |
Chong Ruan, Shirong Ma, Runxin Xu, Peiyi Wang, Zijun Liu |
This paper introduces Self-Principled Critique Tuning (SPCT) to enhance the inference-time scalability and performance of generalist generative reward models (GRMs). The main objective is to investigate if a specific learning method can enable effective inference-time scaling for GRMs, improving reward quality beyond standard model or compute scaling. The key methodology involves SPCT, which uses rejective fine-tuning and rule-based online RL to train GRMs to generate adaptive principles and critiques, combined with inference-time scaling via parallel sampling and voting, optionally guided by a meta RM. Primary results show DeepSeek-GRM-27B trained with SPCT achieves 69.9% overall accuracy on RM benchmarks, improving to 71.0% with voting@32, and further to 72.8% with meta RM guidance, demonstrating effective inference-time scaling compared to just increasing model size. For AI practitioners, this implies that using SPCT and inference-time sampling with GRMs can yield superior reward signals for aligning LLMs, potentially offering a more compute-efficient path to performance gains than solely relying on larger models. |
| Efficient Model Selection for Time Series Forecasting via LLMs (Read more on arXiv or HuggingFace) |
Hongjie Chen, Franck-Dernoncourt, ryanrossi, tiankaiy, wwdd7718 |
This paper investigates leveraging Large Language Models (LLMs) for efficient, zero-shot model selection in time series forecasting, eliminating the need for costly pre-computed performance matrices. The primary objective is to determine if LLMs can select optimal forecasting models and hyperparameters for unseen time series datasets solely through prompting. The methodology involves querying LLMs (Llama 3.2, GPT-4o, Gemini 2.0 flash) with prompts containing time series data and optionally meta-features or Chain-of-Thought (CoT) instructions to recommend a model configuration. Results demonstrate that the LLM approach, particularly Llama 3.2 using prompts with meta-features, outperforms traditional meta-learning (e.g., achieving 7.27% hit@10 accuracy vs. 4.51% for MLP) and heuristic baselines while reducing median inference time by up to 89x compared to naïve exhaustive evaluation. For AI practitioners, this suggests LLMs offer a computationally cheaper and faster alternative for selecting appropriate time series forecasting models without extensive prior model evaluations or meta-feature engineering, streamlining the model selection workflow. |
| Instruction-Guided Autoregressive Neural Network Parameter Generation (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Song Chong, Bruno Andreis, bedio |
This paper introduces IGPG, an instruction-guided autoregressive framework for generating neural network parameters conditioned on task and architecture specifications. The primary objective is to enable scalable and coherent parameter synthesis across diverse models and tasks, addressing limitations of prior methods like diffusion models. IGPG utilizes a VQ-VAE to tokenize parameters and an autoregressive transformer, conditioned on task/dataset embeddings and architecture descriptions, to generate weight tokens sequentially. Key results demonstrate competitive performance, including generating LoRA parameters that improve accuracy by up to 10% over baseline methods on vision benchmarks. For AI practitioners, IGPG offers a unified tool for rapid model initialization, efficient adaptation to new tasks, and potentially reduces the need for extensive fine-tuning by generating specialized weights on demand. |
| Interpreting Emergent Planning in Model-Free Reinforcement Learning (Read more on arXiv or HuggingFace) |
David Krueger, Usman Anwar, Stephen Chung, agaralon, tuphs |
This paper provides the first mechanistic evidence that model-free reinforcement learning agents (DRC) learn internal planning mechanisms in Sokoban using concept-based interpretability. The primary research objective was to determine if a DRC agent internally formulates, evaluates, and utilizes plans based on predicted future consequences without an explicit world model. The methodology involved probing ConvLSTM cell states for planning-relevant concepts (Agent Approach Direction CA, Box Push Direction CB), analyzing iterative plan formation across internal ticks, and performing causal interventions on activations to verify behavioral dependence. Results show the agent linearly represents CA and CB (e.g., final layer 1x1 probe Macro F1 for CB ~0.8 vs <0.3 baseline), forms plans iteratively resembling parallelized bidirectional search which refine with extra compute (Fig 6), and interventions causally steer behavior (e.g., 98.8% success rate for Layer 3 Agent-Shortcut interventions). The principal implication for AI practitioners is that complex planning capabilities can emerge implicitly in model-free architectures, suggesting that internal state representations and iterative computation may be key mechanisms for such behaviors, influencing agent design and analysis beyond purely behavioral metrics. |
| GenPRM: Scaling Test-Time Compute of Process Reward Models via |
|
|
| Generative Reasoning (Read more on arXiv or HuggingFace) |
Saputello, dmux, ChetKao, iseesaw, RyanLiu112 |
GenPRM introduces a generative process reward model utilizing explicit reasoning and code verification to scale test-time compute for LLM verification. The objective is to overcome limitations of current Process Reward Models (PRMs) by enhancing their process supervision capabilities and enabling test-time scaling (TTS) through generative modeling. GenPRM achieves this by performing multi-step Chain-of-Thought (CoT) reasoning integrated with code generation and execution for verification, using Relative Progress Estimation (RPE) and rationale synthesis for training data generation. Experiments demonstrate that a 7B GenPRM significantly outperforms prior models, surpassing the much larger Qwen2.5-Math-PRM-72B on ProcessBench (achieving 80.5 F1 score with Maj@8 scaling). For AI practitioners, this work shows that smaller generative PRMs, when combined with test-time scaling, can serve as highly effective and potentially more compute-efficient verifiers or critics compared to larger models or traditional scalar-based PRMs, improving the evaluation and refinement of complex reasoning processes. |
| Scaling Laws in Scientific Discovery with AI and Robot Scientists (Read more on arXiv or HuggingFace) |
Zhenting Wang, Renjun Xu, Huazhe Xu, Heng Zhang, universea |
This paper proposes the Autonomous Generalist Scientist (AGS) concept, integrating agentic AI and embodied robotics to automate the end-to-end scientific research lifecycle. The main objective is to outline a framework for AGS systems capable of independent, multi-domain scientific discovery by synergizing AI’s cognitive abilities with robotics’ physical interaction capabilities. The methodology involves proposing a conceptual framework featuring a five-module architecture (literature review, proposal generation, experimentation, manuscript writing, reflection/feedback) and defining five distinct levels of automation, ranging from Level 1 (Tool-Assisted) to Level 5 (Pioneer/ASIR). The paper hypothesizes new scaling laws for scientific discovery driven by AGS capabilities and number, rather than presenting empirical results; it details requirements for virtual (OS agents) and physical (embodied AI robots) task execution. For AI practitioners, the primary implication is the conceptual roadmap for developing integrated AI-robotic systems capable of complex, multi-stage, cross-domain automation, moving beyond specialized AI tools to handle tasks requiring both virtual reasoning and physical manipulation. |
| Sparse Autoencoders Learn Monosemantic Features in Vision-Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Zeynep Akata, Serge Belongie, Quentin Bouniot, Shyamgopal Karthik, Mateusz Pach |
This work extends Sparse Autoencoders (SAEs) to Vision-Language Models (VLMs) like CLIP, demonstrating their ability to learn more interpretable, monosemantic features from vision representations. The primary objective is to quantitatively evaluate whether SAEs applied post-hoc to VLM activations enhance neuron monosemanticity and enable model control. Methodology involves training various SAE types on CLIP layer activations and introducing a Monosemanticity Score (MS) metric, calculating activation-weighted pairwise image embedding similarity for neurons. Results demonstrate SAE neurons achieve significantly higher monosemanticity (e.g., MS increased from 0.48 in the base VLM to 0.81 with an SAE for specific neurons shown) and reveal hierarchical concept structures, especially with Matryoshka SAEs. For AI practitioners, this research validates SAEs as an unsupervised method to interpret VLM representations and directly steer the output concepts of multimodal LLMs like LLaVA by intervening on SAE activations, without modifying the base model. |
| Whisper-LM: Improving ASR Models with Language Models for Low-Resource |
|
|
| Languages (Read more on arXiv or HuggingFace) |
Ibon Saratxaga, Eva Navas, inmahernaez, zuazo |
This research improves Whisper ASR models for low-resource languages by integrating external n-gram and large language models (LLMs) with fine-tuned models at inference time. The main objective was to enhance transcription accuracy and robustness, particularly in low-resource and out-of-distribution scenarios, by combining acoustic model probabilities with language model scores. Key methodology involved fine-tuning Whisper models per language, followed by integrating KenLM 5-gram models or language-specific LLMs by modifying beam search scores using optimized weighting parameters. Primary results demonstrate substantial Word Error Rate (WER) reductions, achieving up to 51% improvement for in-distribution Basque data with 5-gram models, while LLMs offered consistently robust, albeit more moderate, gains across languages. For AI practitioners, this indicates that integrating external LMs significantly boosts Whisper’s performance for under-resourced languages, but optimal performance requires careful language model parameter tuning and attention to evaluation settings. |
Papers for 2025-04-03
| Title |
Authors |
Summary |
| MergeVQ: A Unified Framework for Visual Generation and Representation |
|
|
| with Disentangled Token Merging and Quantization (Read more on arXiv or HuggingFace) |
Cheng Tan, Juanxi, ZedongWangAI, LuyuanZhang01, Lupin1998 |
MergeVQ presents a unified framework integrating token merging into VQ-based models to balance visual representation learning and autoregressive generation. The primary objective is to overcome the trade-off between generation quality, representation learning, and efficiency inherent in existing VQ-MIM approaches. Key methodologies include disentangling semantics via token merging (ToMe) while preserving spatial details in a recoverable source matrix, employing Look-up Free Quantization (LFQ), using cross-attention for detail recovery, global alignment via self-distillation (DINO), and introducing MergeAR with KV Cache compression for efficient generation. Experiments on ImageNet-1K show the representation-focused variant achieves 79.8% linear probe accuracy using only 36 merged tokens, while the generative variant achieves a competitive class-conditional generation gFID of 3.05 using MergeAR. For AI practitioners, MergeVQ offers a pathway to build more computationally efficient unified vision models, as demonstrated by its ability to achieve strong representation learning performance with significantly reduced token counts (36 tokens), potentially lowering pre-training and inference costs. |
| Improved Visual-Spatial Reasoning via R1-Zero-Like Training (Read more on arXiv or HuggingFace) |
Zijian Kong, Yanhao Zhang, Qingsong Xie, Zhenyi Liao, zhijie3 |
This work enhances visual-spatial reasoning in Multimodal Large Language Models (MLLMs) using R1-Zero-like GRPO training. The primary objective was to improve visual-spatial intelligence (VSI) capabilities, particularly in small- to medium-sized Qwen2-VL models where Chain of Thought (CoT) prompting proved ineffective. The key methodology involved constructing the VSI-100k dataset from ScanNet and applying Group Relative Policy Optimization (GRPO) while identifying the necessity of retaining the KL penalty. The resulting vsGRPO-2B model outperformed its Qwen2-VL-2B base by 12.1% on the VSI-bench benchmark and surpassed GPT-4o performance. For AI practitioners, this demonstrates that GRPO training with curated datasets is a potent technique to specifically boost MLLM reasoning faculties like VSI, offering substantial gains over base models and even surpassing larger or closed-source alternatives for targeted tasks. |
| AnimeGamer: Infinite Anime Life Simulation with Next Game State |
|
|
| Prediction (Read more on arXiv or HuggingFace) |
Ying Shan, Jing Liao, Yixiao Ge, Yuying Ge, Howe666 |
AnimeGamer introduces an MLLM-based framework for generating infinite, interactive anime life simulation games featuring dynamic video outputs and character state updates from language instructions. The primary objective is to create contextually consistent and dynamic multi-turn game states, addressing limitations of prior static image or text-only methods. The key methodology involves using an MLLM to predict novel action-aware multimodal representations from historical context and instructions, which are then decoded into video clips using a fine-tuned video diffusion model alongside character state prediction. AnimeGamer significantly outperforms baselines in quantitative evaluations, achieving higher character consistency (CLIP-I 0.8132 vs. 0.7960) and superior motion quality (ACC-F 0.6744 vs. 0.4249). For AI practitioners, this work demonstrates an effective approach using MLLMs to generate coherent, dynamic video-based interactive experiences by bridging language and video synthesis via specialized multimodal representations, enhancing immersion in generative games. |
| VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in |
|
|
| One Step (Read more on arXiv or HuggingFace) |
Yueqi Duan, Jiawei Chi, Fangfu Liu, hanyang-21 |
VideoScene introduces a framework to distill video diffusion models for efficient, one-step 3D scene generation from only two input images. The main objective is to bridge the gap between slow, multi-step video diffusion methods and the need for fast, 3D-consistent scene generation from sparse views. The key methodology involves a 3D-aware leap flow distillation strategy, initialized using a coarse scene from a feed-forward 3DGS model (MVSplat), and a dynamic denoising policy network (DDPNet) trained via contextual bandits to optimize leap timesteps. Primarily, VideoScene achieves significantly faster inference (~3s) while maintaining high quality; its 1-step generation on RealEstate10K yields an FVD of 103.42, vastly outperforming 1-step baselines and remaining competitive with their 50-step versions (e.g., CogVideoX-5B 50-step FVD 521.04). For AI practitioners, this offers an efficient tool for generating temporally coherent and geometrically consistent 3D video sequences from minimal input, drastically reducing computational cost for sparse-view 3D reconstruction tasks. |
| Understanding R1-Zero-Like Training: A Critical Perspective (Read more on arXiv or HuggingFace) |
Tianyu Pang, Wenjun Li, QPHutu, Cameron-Chen, lkevinzc |
This paper critically analyzes R1-Zero-like LLM training, examining base model properties and RL optimization biases, particularly in GRPO. The primary objective is to understand how base model pretraining affects RL outcomes and to identify and mitigate biases in the GRPO algorithm. Methodology includes evaluating various base models (e.g., Qwen2.5, DeepSeek-V3-Base) on math benchmarks with different templates and comparing GRPO against a proposed unbiased variant, Dr. GRPO, in RL experiments. Key findings demonstrate that some base models exhibit strong initial reasoning (Qwen2.5 improves ~60% without templates), GRPO introduces length and standard deviation normalization biases impacting token efficiency, and the proposed Dr. GRPO optimizer corrects these, enabling a 7B model to achieve 43.3% accuracy on AIME 2024. The principal implication for practitioners is that understanding base model capabilities and utilizing unbiased RL optimizers like Dr. GRPO are essential for efficient reasoning enhancement, avoiding artifactual response length increases from biased optimization. |
| DreamActor-M1: Holistic, Expressive and Robust Human Image Animation |
|
|
| with Hybrid Guidance (Read more on arXiv or HuggingFace) |
Tianshu Hu, Longhao Zhang, Lizhen Wang, Zhengkun Rong, Yuxuan Luo |
This paper introduces DreamActor-M1, a Diffusion Transformer (DiT) based framework for robust human image animation. The primary objective is to overcome limitations in existing methods regarding fine-grained holistic control, multi-scale adaptability (portraits to full-body), and long-term temporal coherence, particularly for unseen regions. Key methodologies include using hybrid motion guidance signals (implicit facial latent representations, 3D head spheres, 3D body skeletons with bone length adjustment), complementary appearance guidance for unseen areas, and a progressive multi-scale training strategy. The proposed method achieved superior quantitative results, for instance, an FVD score of 122.0 on their collected body animation dataset, outperforming prior works like Animate Anyone (158.3) and MimicMotion (149.9). For AI practitioners, this work demonstrates a robust DiT-based approach with hybrid explicit/implicit controls and appearance guidance, enabling the generation of higher-fidelity, more expressive, and temporally consistent human animations across diverse scales and viewpoints. |
| PaperBench: Evaluating AI’s Ability to Replicate AI Research (Read more on arXiv or HuggingFace) |
Jun Shern Chan, James Aung, Dane Sherburn, Oliver Jaffe, Giulio Starace |
PaperBench introduces a benchmark to evaluate AI agents’ ability to replicate state-of-the-art AI research papers from scratch. The objective is to assess how well AI agents can understand paper contributions, develop codebases, and execute experiments to reproduce empirical results. The methodology involves providing agents with 20 ICML 2024 papers and using detailed, author-approved hierarchical rubrics alongside an LLM-based judge to evaluate the agent-generated code repository and its execution outputs. Results show the best agent, Claude 3.5 Sonnet with scaffolding, achieved an average replication score of 21.0%, significantly lower than a human expert baseline (41.4% on a subset), indicating current models have limited autonomous AI R&D replication capabilities. For AI practitioners, this highlights that while agents show nascent ability, they are not yet proficient at the complex, long-horizon task of independently replicating and validating frontier AI research, requiring substantial human oversight for such tasks. |
| ScholarCopilot: Training Large Language Models for Academic Writing with |
|
|
| Accurate Citations (Read more on arXiv or HuggingFace) |
Zhiheng Lyu, Huaye Zeng, Ping Nie, Xueguang Ma, Yubo Wang |
ScholarCopilot introduces a unified framework for training LLMs to generate academic text with accurate, context-aware citations. The main objective is to overcome limitations of traditional RAG systems by integrating dynamic retrieval directly into the generation process for improved citation relevance and quality in academic writing. The methodology involves dynamically generating special retrieval tokens ([RET]) during text generation, using their representations for similarity search against a database, and feeding retrieved references back into the model, optimizing generation and retrieval jointly. ScholarCopilot achieved 40.1% top-1 retrieval accuracy, significantly outperforming E5-Mistral-7B-Instruct (15.0%), and obtained a generation quality score of 16.2/25, surpassing larger models like Qwen-2.5-72B-Instruct (15.8/25). For AI practitioners, this work demonstrates a unified, dynamic RAG approach that can enhance LLM factual accuracy and contextual relevance for specialized generation tasks requiring precise citations, offering a potentially more efficient alternative to separate retrieval/generation pipelines. |
| Towards Physically Plausible Video Generation via VLM Planning (Read more on arXiv or HuggingFace) |
Lei Bai, Zhenfei Yin, Yiming Zhang, Baolu Li, Xindi Yang |
This paper proposes a two-stage framework using a Vision Language Model (VLM) planner and a Video Diffusion Model (VDM) synthesizer to generate physically plausible videos. The objective is to enhance physical plausibility in video generation by explicitly incorporating physics priors, addressing the limitations of standard VDMs in understanding physical laws. The methodology involves a VLM performing coarse-grained, physics-aware motion planning via chain-of-thought (CoT) reasoning to predict rough object trajectories, which then guide a VDM through injected structured noise derived from optical flow for fine-level motion synthesis. Quantitative results on the PhyGenBench benchmark show the proposed method achieved an average score of 0.60, outperforming the best compared image-to-video method (SG-I2V at 0.54) by 11.1% in physical plausibility assessment. For AI practitioners, this demonstrates a method to integrate explicit physical reasoning from VLMs into VDMs to improve the realism and physical consistency of generated video content, particularly for scenarios involving object interactions governed by physics. |
| ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and |
|
|
| Diffusion Refinement (Read more on arXiv or HuggingFace) |
Yunlong Yuan, Guansong Lu, Junwei Yang, Chunwei Wang, Runhui Huang |
ILLUME+ enhances unified Multimodal Large Language Models (MLLMs) by integrating dual visual tokenization and diffusion refinement for improved understanding, generation, and editing. The objective is to create a single MLLM that overcomes limitations of prior models, such as poor texture preservation in editing or weaker semantic understanding, by effectively unifying these three core capabilities. Key methodologies include the DualViTok tokenizer preserving both semantic and texture details, a diffusion model decoder for high-fidelity image reconstruction and super-resolution, and a coarse-to-fine image representation strategy within the MLLM. Primary results show the 3B parameter ILLUME+ achieves competitive performance across understanding, generation, and editing benchmarks, including an improved Fréchet Inception Distance (FID) of 6.00 on the MJHQ-30k generation benchmark compared to its predecessor. For AI practitioners, this work presents a unified model architecture that supports flexible resolution inputs/outputs and demonstrates strong performance in fine-grained editing tasks, potentially offering a more versatile foundation for complex, interactive multimodal applications. |
| Articulated Kinematics Distillation from Video Diffusion Models (Read more on arXiv or HuggingFace) |
Chenfanfu Jiang, Yongxin Chen, Tsung-Yi Lin, Qianli Ma, Xuan Li |
Articulated Kinematics Distillation (AKD) synthesizes articulated motions for rigged 3D assets by leveraging video diffusion models. The objective is to generate high-fidelity, structurally consistent character animations from text prompts, addressing limitations of prior text-to-4D methods based on neural deformation fields. AKD utilizes a low-DoF skeleton-based representation optimized via Score Distillation Sampling (SDS) with a pre-trained video diffusion model, incorporating explicit ground rendering and optional physics-based motion tracking. Experiments show AKD outperforms TC4D, achieving higher automated scores (e.g., Semantic Adherence 0.81±0.26 vs 0.40±0.34) and preference in user studies for motion quality and physical plausibility. For AI practitioners, AKD offers a method to generate controllable, physically grounded 3D character animations from text by effectively combining generative video priors with explicit articulated structure, improving consistency over deformation field approaches. |
| Safeguarding Vision-Language Models: Mitigating Vulnerabilities to |
|
|
| Gaussian Noise in Perturbation-based Attacks (Read more on arXiv or HuggingFace) |
Zhendong Liu, Yushen Zuo, sofyc, AllenChai, Jarvis1111 |
This paper investigates Vision-Language Model (VLM) vulnerability to Gaussian noise perturbations and proposes noise-augmented fine-tuning and a diffusion-based defense (DiffPure-VLM) to mitigate these risks. The primary objective is to systematically analyze VLM robustness against visual Gaussian noise and develop effective defense strategies against both simple noise and optimization-based adversarial attacks while preserving model helpfulness. Key methodologies include creating the Robust-VLGuard dataset with aligned/misaligned safety pairs, employing Gaussian noise augmentation during safety fine-tuning, and proposing the DiffPure-VLM pipeline which uses diffusion models to transform adversarial perturbations into Gaussian-like noise manageable by the fine-tuned VLMs. Primary results demonstrate that while baseline VLMs degrade significantly under Gaussian noise, the proposed noise-augmented fine-tuning enhances robustness, and DiffPure-VLM substantially reduces optimization-based attack success rates; for example, with InternVL2-8B-RobustVLGuard under a €=32/255 attack, DiffPure-VLM (t*=50) lowered the attack success rate from 70.6% to 33.4%. For AI practitioners, this implies that incorporating noise-augmented safety fine-tuning and employing diffusion-based preprocessing defenses like DiffPure-VLM are practical strategies to significantly bolster VLM security against visual perturbation attacks without excessive computational overhead or loss of core functionality. |
| Boost Your Own Human Image Generation Model via Direct Preference |
|
|
| Optimization with AI Feedback (Read more on arXiv or HuggingFace) |
Hyunjoon Lee, Yonggyu Kim, sanghyeonna |
This paper introduces HG-DPO, a method enhancing human image generation realism by applying Direct Preference Optimization (DPO) with real images and curriculum learning. The main objective is to improve diffusion models for human image synthesis by overcoming the limitations of standard DPO, which typically relies only on generated images. HG-DPO utilizes a novel preference structure where real images serve as preferred (winning) examples and generated images as non-preferred (losing), combined with a three-stage curriculum learning pipeline (easy, normal, hard) and AI feedback for dataset construction. Results demonstrate HG-DPO significantly outperforms baseline models and prior DPO methods, achieving a lower FID of 29.41 compared to the base model’s 37.34 and higher CI-S of 0.9858 versus 0.9573. For AI practitioners, this provides a framework to boost the quality and realism of text-to-human image generation models by effectively integrating real-world image data into the preference learning process without costly human annotation, and enhances personalized generation tasks. |
| DASH: Detection and Assessment of Systematic Hallucinations of VLMs (Read more on arXiv or HuggingFace) |
Matthias Hein, Maximilian Augustin, YanNeu |
This paper introduces DASH, an automated pipeline for detecting systematic false-positive object hallucinations in Vision-Language Models (VLMs) using large-scale, real-world image data. The main objective is to systematically identify clusters of semantically similar real-world images that cause a VLM to incorrectly affirm the presence of an object not actually depicted. Key methodologies include DASH-LLM, which uses LLM-generated text queries for image retrieval, and DASH-OPT, which optimizes latent diffusion model inputs to generate misleading images, both followed by k-NN retrieval on ReLAION-5B and clustering. Applying DASH to PaliGemma and two LLaVA-NeXT models across 380 objects yielded over 19k hallucination clusters containing over 950k images; fine-tuning PaliGemma on DASH-identified images improved accuracy on the derived DASH-B benchmark by 11.6%. For AI practitioners, this work highlights that significant object hallucination issues persist beyond standard benchmarks, necessitating open-world testing methods like DASH for reliable VLM assessment and providing datasets (DASH-B) for more rigorous evaluation and potential mitigation fine-tuning. |
| LSNet: See Large, Focus Small (Read more on arXiv or HuggingFace) |
Guiguang Ding, Jungong Han, Zijia Lin, Hui Chen, jameslahm |
LSNet introduces a lightweight vision network family leveraging a novel LS convolution inspired by the human vision system’s “See Large, Focus Small” strategy. The primary objective is to enhance the performance and efficiency balance in lightweight models by improving the token mixing process, specifically perception and aggregation under limited computational budgets. The key methodology involves the proposed LS (Large-Small) convolution, which uses large-kernel static depth-wise convolution for broad perception and small-kernel grouped dynamic convolution for adaptive, focused aggregation. Results demonstrate state-of-the-art performance; for instance, LSNet-B achieves 80.3% top-1 accuracy on ImageNet-1K with 1.3G FLOPs, outperforming comparable models like AFFNet and RepViT-M1.1 in both accuracy and efficiency. For AI practitioners, LSNet provides a new efficient architectural block (LS convolution) and model series offering improved accuracy-efficiency trade-offs for vision tasks deployed on resource-constrained platforms. |
| VerifiAgent: a Unified Verification Agent in Language Model Reasoning (Read more on arXiv or HuggingFace) |
Ehsan Shareghi, Wray Buntine, Jiuzhou Han |
This paper introduces VerifiAgent, a unified agent employing two verification levels (meta and tool-based adaptive) to enhance large language model (LLM) reasoning reliability. The main research objective is to develop a generalisable and efficient verification framework for diverse LLM reasoning tasks, overcoming the limitations of current methods. VerifiAgent utilizes a two-layer methodology involving meta-verification for completeness and consistency, followed by tool-based adaptive verification which autonomously selects external tools (e.g., Python interpreter, search engine, symbolic solver) based on the reasoning type. Experimental results show VerifiAgent outperforms baseline verification methods across mathematical, logical, commonsense, and hybrid reasoning tasks, achieving 0.96 accuracy on GSM8K compared to baselines like deductive verifier (0.95). For AI practitioners, VerifiAgent offers a plug-and-play framework to improve the reliability and accuracy of LLM reasoning outputs, particularly in inference scaling scenarios, achieving better results with fewer samples and lower cost than methods like PRMs. |
| Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal |
|
|
| Representations (Read more on arXiv or HuggingFace) |
Sangheum Hwang, mawjdgus |
This paper introduces Cross-Modal Alignment (CMA), a multi-modal fine-tuning method to enhance Out-of-Distribution (OoD) detection in vision-language models. The primary objective is to improve OoD performance by mitigating the modality gap observed between image and text embeddings during standard fine-tuning. CMA employs a regularization loss during fine-tuning to explicitly align in-distribution image-text embedding pairs in the hyperspherical representation space, shown theoretically to correspond to maximizing the log-likelihood of a joint energy-based model. The proposed CMA method, when combined with the NegLabel scoring function, achieved state-of-the-art OoD performance on the MOS benchmark, attaining an average FPR95 of 19.93% and 95.13% AUROC, significantly outperforming existing zero-shot and fine-tuning approaches while maintaining high ID accuracy (82.64% on ImageNet-1k). For AI practitioners, this work demonstrates that explicitly regularizing for cross-modal alignment during fine-tuning can substantially improve model robustness by enhancing both OoD detection and in-distribution classification, thereby increasing the reliability of VLMs deployed in open environments. |
Papers for 2025-04-02
| Title |
Authors |
Summary |
| Any2Caption:Interpreting Any Condition to Caption for Controllable Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
shuicheng, dizhang, Xintao, WeicaiYe, ChocoWu |
Any2Caption introduces an MLLM-based framework to interpret diverse multimodal conditions into structured captions for controllable video generation. The main objective is to accurately interpret complex user intent from various inputs (text, images, specialized cues like pose/camera) to improve video synthesis control and quality. The methodology involves decoupling interpretation from generation, using a Qwen2-LLM with dedicated encoders to generate detailed, structured captions, trained on the new Any2CapIns dataset (337K instances). Results show high caption fidelity (e.g., 91.95 BERTSCORE) and improved video quality and controllability when integrated with existing video generators across various conditions. For AI practitioners, the key implication is the ability to enhance control over existing video generation models using complex multimodal inputs by integrating this interpretation module, which outputs structured text captions, without needing to retrain the core video generator. |
| Exploring the Effect of Reinforcement Learning on Video Understanding: |
|
|
| Insights from SEED-Bench-R1 (Read more on arXiv or HuggingFace) |
yshan2u, yxgeee, ruiwang, tttoaster, ChenYi99 |
SEED-Bench-R1 is introduced to systematically evaluate reinforcement learning (RL) post-training for multimodal large language model (MLLM) video understanding. The primary objective is to compare the effectiveness and generalization of RL (specifically GRPO) against supervised fine-tuning (SFT) for video tasks requiring both perception and logical reasoning. Using Qwen2-VL-Instruct-7B, the study compared GRPO trained with outcome-based rewards against SFT on the hierarchical SEED-Bench-R1 benchmark (L1: In-distribution, L2/L3: OOD). Results show GRPO significantly outperforms SFT in data efficiency and generalization, particularly in OOD scenarios (e.g., 44.89% vs 38.15% accuracy on Level-3), and extends generalization benefits to benchmarks like LongVideoBench (43.40% vs 40.00%). For AI practitioners, this implies RL, even with simple outcome rewards, is highly effective at enhancing MLLM visual perception and OOD generalization for video tasks compared to SFT, though analysis notes RL may compromise logical coherence in the reasoning chain. |
| CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive |
|
|
| Program Synthesis (Read more on arXiv or HuggingFace) |
Naveen Kannan, Jiannan Cao, kaiyan289, tarsur909, anjiangwei |
This paper introduces CodeARC, an interactive benchmark evaluating LLM agents on inductive program synthesis. The main objective is to assess LLMs’ ability to infer hidden functions solely from input-output examples through interaction, departing from static evaluation protocols. Key methodology involves agents querying a hidden target function, synthesizing candidates, and using a differential testing oracle for feedback and iterative refinement under budget constraints on 1114 Python functions. Primary results indicate the task is challenging: the best-performing model, o3-mini, achieved a 52.7% success rate on the anonymized dataset, and fine-tuning LLaMA-3.1-8B-Instruct improved performance by up to 31% relatively. For AI practitioners, this work provides a more realistic benchmark revealing significant limitations in current LLMs’ inductive reasoning for code synthesis and suggests interactive refinement and targeted fine-tuning as avenues for improvement. |
| JudgeLRM: Large Reasoning Models as a Judge (Read more on arXiv or HuggingFace) |
Jiaying Wu, Nuo Chen, bhooi, qingyunzou, zhiyuanhucs |
This paper introduces JudgeLRM, a family of LLMs trained via reinforcement learning (RL) to serve as evaluators, specifically targeting complex reasoning tasks where SFT judges falter. The research investigates whether enhancing reasoning capabilities improves LLM judge performance and proposes an RL-based training approach using judge-wise, outcome-driven rewards. Key methodology involves training base LLMs (Qwen2.5) using Group Relative Policy Optimization (GRPO) with a custom reward function combining structural correctness and content alignment (relation, absolute, confidence metrics) against ground-truth judgments. Primary results show JudgeLRM models outperform SFT-tuned and state-of-the-art reasoning models; notably, JudgeLRM-7B surpasses DeepSeek-R1 by 2.79% in F1 score on the JudgeLM benchmark, excelling particularly on tasks requiring deep reasoning. For AI practitioners, this implies that RL with carefully designed, reasoning-focused rewards is a more effective method than SFT for developing robust LLM evaluators capable of handling nuanced, complex judgment tasks, suggesting RL should be considered for building reliable automated evaluation systems. |
| GeometryCrafter: Consistent Geometry Estimation for Open-world Videos |
|
|
| with Diffusion Priors (Read more on arXiv or HuggingFace) |
Xiaoyu Li, yshan2u, wbhu-tc, xiangjun0211, slothfulxtx |
GeometryCrafter generates temporally consistent, metrically accurate point map sequences from open-world videos using diffusion priors. The main objective is to estimate high-fidelity, temporally coherent point maps with correct metric scale from videos, overcoming the affine ambiguity and temporal inconsistency limitations of prior diffusion-based depth and geometry estimation methods. The key methodology employs a novel point map Variational Autoencoder (VAE) with a dual-encoder design (using an inherited SVD encoder and a residual encoder) to encode unbounded point maps while maintaining latent compatibility, integrated with a video diffusion model finetuned using these latents and per-frame geometry priors. Primary results demonstrate state-of-the-art performance, achieving an average rank of 1.9 on point map estimation across seven diverse benchmark datasets, indicating superior 3D accuracy and temporal consistency compared to previous methods. For AI practitioners, this provides a framework to extract metrically accurate, temporally consistent geometry from videos, directly usable for applications like 3D/4D reconstruction or depth-conditioned video editing/generation without post-hoc scale recovery. |
| Agent S2: A Compositional Generalist-Specialist Framework for Computer |
|
|
| Use Agents (Read more on arXiv or HuggingFace) |
Vincent Tu, Kyle Wong, xw-eric, jc-y42, saa1605 |
Agent S2 introduces a compositional generalist-specialist framework enhancing computer use agent capabilities via specialized modules. The primary objective is to address limitations in GUI grounding precision, long-horizon task planning, and reliance on single generalist models for diverse cognitive tasks. Methodologically, Agent S2 employs a Mixture-of-Grounding technique routing actions to specialized grounding experts and Proactive Hierarchical Planning for dynamic plan refinement based on evolving observations. Agent S2 achieved new state-of-the-art results, notably a 34.5% success rate on the OSWorld 50-step evaluation, a 32.7% relative improvement over the leading Claude Computer Use baseline. For AI practitioners, this demonstrates the effectiveness of composing generalist planning with specialized grounding modules to overcome bottlenecks in monolithic models for complex GUI automation tasks. |
| Z1: Efficient Test-time Scaling with Code (Read more on arXiv or HuggingFace) |
Xiao-Ping Zhang, armanc, yilunzhao, yh1567, zjy2001 |
Z1 proposes an efficient test-time compute scaling method for LLMs using code-related reasoning trajectories and a novel shifted thinking window. The research aims to reduce the excessive thinking token cost associated with test-time scaling in Large Reasoning Models (LRMs) while preserving performance. Key methodology involves training an LLM (Qwen2.5-Coder-7B-Instruct) on a curated dataset (Z1-Code-Reasoning-107K) containing both short and long code solution trajectories and employing a “Shifted Thinking Window” during inference that avoids fixed delimiters and caps reasoning tokens. The resulting model, Z1-7B, matches the performance of R1-Distill-Qwen-7B on three reasoning benchmarks while using only about 30% of its average thinking tokens, and notably generalizes to non-code tasks like GPQA Diamond (47.5%). For AI practitioners, this demonstrates a method to significantly improve the computational efficiency and reduce inference costs of LRMs for complex reasoning tasks by fine-tuning with varied-length code trajectories and adopting a flexible, adaptive thinking process during inference. |
| MixerMDM: Learnable Composition of Human Motion Diffusion Models (Read more on arXiv or HuggingFace) |
José García-Rodríguez, Sergio Escalera, Cristina Palmero, Germs96, pabloruizponce |
MixerMDM introduces a learnable technique for composing pre-trained text-conditioned human motion diffusion models. The main research objective is to dynamically combine motions from specialized single-person and interaction models to achieve fine-grained control over individual movements within complex interactions. The key methodology involves a lightweight Mixer module, trained adversarially against multiple discriminators (one per pre-trained model), to predict dynamic, context-dependent mixing weights at each denoising step, using the pre-trained models’ outputs as pseudo-ground truth. Primary results demonstrate superior performance over fixed-weight or scheduled methods, with MixerMDM achieving significantly better alignment and consistency, ranking first in 85.14% of user study comparisons based on motion alignment to textual descriptions. For AI practitioners, MixerMDM provides a modular framework to combine specialized, pre-trained diffusion models for generating nuanced, controllable human motion sequences without requiring retraining of the base models or explicit ground truth for the combined outputs. |
| Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal |
|
|
| LLMs on Academic Resources (Read more on arXiv or HuggingFace) |
Heng Wang, Yu Tian, windwest, yanglj55, weizhiwang |
Open-Qwen2VL details compute-efficient pre-training of a fully open-source 2B parameter Multimodal Large Language Model (MLLM) on academic-scale resources. The objective is to develop and openly release an efficient MLLM pre-training pipeline reproducible with limited compute, specifically using 8xA100-40G GPUs. Key methodologies include low-to-high dynamic image resolution (144 visual tokens in pre-training, 729 in SFT), multimodal sequence packing, and data filtering using both CLIP-based methods and MLLM-based techniques (MLM-Filter) on a 29M image-text pair dataset. The resulting instruction-tuned Open-Qwen2VL, pre-trained on 5B packed multimodal tokens (using 442 A100-40G GPU hours), outperforms the partially-open Qwen2-VL-2B on benchmarks such as MMBench (achieving 80.9), SEEDBench, MMStar, and MathVista, despite using only 0.36% of Qwen2-VL’s reported pre-training tokens. For AI practitioners, this work provides a fully open-sourced blueprint—including codebase, data filtering/packing scripts, curated pre-training data, and model checkpoints—demonstrating that efficient, high-performance MLLM pre-training is attainable without extensive industrial-scale resources, enabled by optimized data curation and training techniques. |
| Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for |
|
|
| Large Language Models (Read more on arXiv or HuggingFace) |
Sudanl, pangjh3, BeyondHsueh, Merlin-Hongru, Ray121381 |
This survey systematically reviews strategies for achieving “Reasoning Economy” in Large Language Models (LLMs), balancing performance benefits against computational budgets. The primary objective is to analyze the causes of reasoning inefficiency (e.g., length bias, deceptive behaviors), understand different reasoning patterns, and survey potential solutions across post-training and test-time inference stages. It employs a comprehensive literature review, categorizing challenges stemming from post-training methods (like Superficial Alignment leading to length bias) and test-time usage (like unreasonable computation allocation) and corresponding optimization solutions (e.g., behavior regulation, usage improvement). Key findings identify specific inefficiencies like length bias (where RMs may prefer longer responses, e.g., 63.1% in RLCD) and overly cautious reasoning, while highlighting solutions such as long2short RL methods (e.g., SimPO reducing lengths by 30-40%) and adaptive computation allocation based on task complexity. For AI practitioners, the principal implication is the need to shift from static, one-size-fits-all inference approaches towards dynamic, adaptive strategies (e.g., adaptive budget allocation, algorithm selection) to optimize resource utilization and unlock LLMs’ full potential efficiently. |
| OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming |
|
|
| Video Contexts (Read more on arXiv or HuggingFace) |
Tong Wu, Bo Chen, Yueqian Wang, zlzheng, ColorfulAI |
This paper introduces OmniMMI, a benchmark for evaluating MLLMs in streaming video interaction, and M4, a framework enhancing these capabilities. The primary objective is to evaluate and improve the real-world interactive performance of OmniLLMs in streaming video contexts, focusing on streaming understanding and proactive reasoning challenges underexplored by existing benchmarks. Methodology involved curating the OmniMMI dataset (1,121 videos, 2,290 questions across six subtasks including dynamic state grounding and proactive alerting) and developing the Multi-modal Multiplexing Modeling (M4) framework using multiplexing techniques and an attention-based inference method for efficient, proactive processing. Experimental results show existing MLLMs perform poorly on OmniMMI, particularly struggling with proactive tasks and multi-turn dependencies, while the proposed lightweight M4 model demonstrates significant improvement, achieving 68.5% accuracy on the Proactive Turn-taking task after audio adaptation (M4-a). For AI practitioners, this research underscores the inadequacy of current models for real-time interaction, provides OmniMMI as a necessary tool for evaluating streaming/proactive capabilities, and suggests the M4 architecture as a resource-efficient approach to develop models that can simultaneously perceive and generate responses in dynamic environments. |
| Command A: An Enterprise-Ready Large Language Model (Read more on arXiv or HuggingFace) |
salthammer, yazeed7, jayalammar, ArashAhmadian, aakanksha |
This report details Command A, a 111B parameter multilingual large language model optimized for enterprise RAG and agentic tasks, alongside the smaller Command R7B. The primary objective was to develop and evaluate Command A and R7B as efficient, high-performing LLMs tailored for real-world enterprise settings, focusing on multilingualism, Retrieval Augmented Generation (RAG), and tool use. Key methodologies include a decentralised post-training strategy combining supervised fine-tuning (SFT) and reinforcement learning (RL) across specialized expert models, followed by parameter merging (linear soup), and a polishing phase using algorithms like Self-improving Robust Preference Optimisation (SRPO). Command A achieves competitive results, scoring 80.0 on the MATH benchmark and 51.7 on Taubench, while maintaining efficiency by requiring only two A100/H100 GPUs for inference and delivering up to 156 tokens/sec. For AI practitioners, Command A offers an efficient foundation for enterprise applications needing strong RAG and agentic capabilities, while the reported decentralised training and merging approach presents a method for integrating diverse expert functionalities into a single model. |
| Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on |
|
|
| Elementary School-Level Reasoning Problems? (Read more on arXiv or HuggingFace) |
Xuesong Yao, xmerge123, ALEXoDu, yfxu, kaiyan289 |
This paper demonstrates that cutting-edge LLMs often recite solutions rather than genuinely reason, even on elementary problems. The research objective was to determine if LLMs possess true reasoning ability or merely replicate patterns seen during training, particularly when faced with subtly altered conditions. A novel multi-modal benchmark, RoR-Bench, was created featuring pairs of original problems and variants with minor but crucial condition shifts. Empirical analysis revealed severe recitation behavior, with top models like OpenAI-o1 and DeepSeek-R1 experiencing performance drops exceeding 60% on modified elementary arithmetic and reasoning problems compared to their original counterparts. For AI practitioners, this highlights a critical need to re-evaluate LLM intelligence claims and emphasizes that current models may lack robustness, potentially failing unexpectedly when encountering slight deviations from learned patterns. |
| AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models |
|
|
| with Unsupervised Coefficient Optimization (Read more on arXiv or HuggingFace) |
Yiru Wang, Jiabo Ye, Xiaochen Wang, Yiyang Du, carboncoo |
AdaMMS introduces an unsupervised method for merging heterogeneous Multimodal Large Language Models (MLLMs) with differing architectures. The primary objective is to effectively combine capabilities from distinct MLLMs without requiring labeled data for optimizing the merging hyperparameters. The methodology involves parameter mapping to align weights, linear interpolation for merging, and an unsupervised search step that selects the optimal interpolation coefficient based on minimizing generation consistency differences across candidate merged models using a small unlabeled dataset. Experiments show AdaMMS outperforms supervised baselines; for example, merging LLaVA-OneVision-7B into Qwen2-VL-7B yielded a SUM score of 563.56, a +26.84 gain over the original models’ average. AI practitioners can leverage AdaMMS to fuse heterogeneous MLLMs efficiently, creating enhanced models without supervised data by using generation consistency as a proxy for task performance during optimization. |
| When To Solve, When To Verify: Compute-Optimal Problem Solving and |
|
|
| Generative Verification for LLM Reasoning (Read more on arXiv or HuggingFace) |
anna-rohrbach, kaiweichang, adityagrover, arianhosseini, hbXNov |
This research compares the compute-efficiency of Self-Consistency (SC) and Generative Reward Models (GenRM) for LLM reasoning, revealing SC’s superiority at lower budgets. The study investigates whether allocating a fixed inference budget towards generating more solutions (SC) or generating fewer solutions with multiple verifications (GenRM) yields better LLM reasoning performance, and how to optimally balance solutions and verifications for GenRM. A compute-matched analysis compared SC and GenRM across various models, tasks, and budgets, calculating FLOPs based on solution (S) and verification (V) generation; inference scaling laws were derived by fitting optimal solution (S_opt) and verification (V_opt) counts to compute budget C. Primary results show SC outperforms GenRM until high compute budgets are reached; for Llama-3.1-8B on MATH, GenRM required 8x the compute of SC to match its performance and 128x to achieve a 3.8% gain, while compute-optimal GenRM requires scaling solutions faster (S_opt ∝ C^0.57) than verifications (V_opt ∝ C^0.39). AI practitioners should prioritize SC for LLM reasoning under typical compute constraints; if using GenRM at high budgets, allocate compute preferentially towards increasing solution count over verification count per solution for optimal efficiency. |
| Scaling Language-Free Visual Representation Learning (Read more on arXiv or HuggingFace) |
liuzhuang13, koustuvs, JiachenZhu, tsbpp, davidfan97 |
This paper investigates scaling language-free visual self-supervised learning (SSL) on web-scale data, comparing its performance against Contrastive Language-Image Pretraining (CLIP) primarily on Visual Question Answering (VQA). The research aims to determine if visual SSL lags behind CLIP due to the absence of language supervision or disparities in training data. Key methodology involves training DINOv2 (SSL) and CLIP models (1B to 7B parameters) on the identical 2 billion sample MetaCLIP dataset and evaluating using the Cambrian-1 VQA suite and traditional vision benchmarks. Primary results indicate visual SSL scales better with model and data size than CLIP on VQA; specifically, a 7B parameter Web-DINO model trained on 8 billion examples outperforms a comparable CLIP model on average VQA performance across 16 tasks. The principal implication for AI practitioners is that appropriately scaled visual SSL can yield vision encoders competitive with language-supervised models for multimodal tasks like VQA, providing a strong vision-centric alternative without needing paired text data during pretraining. |
| Multi-Token Attention (Read more on arXiv or HuggingFace) |
sainbar, spermwhale, Tianlu, Golovneva |
This paper introduces Multi-Token Attention (MTA), enhancing LLM attention by conditioning weights on multiple query and key vectors simultaneously via convolution operations. The primary objective is to overcome the “single token attention” bottleneck, allowing models to locate relevant context using richer, multi-token criteria rather than single vector similarity. MTA modifies standard attention by applying convolutions across query, key, and head dimensions (termed key-query convolution and head mixing convolution), often coupled with group normalization. Experiments demonstrate MTA achieves lower perplexity on language modeling (11.09 avg PPL vs 11.25 for an 880M Transformer baseline) and notably improves performance on long-context tasks like Needle-in-a-Haystack and BabiLong compared to baselines. For AI practitioners, MTA offers a method to improve model performance in scenarios requiring identification of context based on multiple simultaneous conditions, particularly beneficial for long-context reasoning, by incorporating these convolutional modifications into the attention mechanism. |
| Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features (Read more on arXiv or HuggingFace) |
Jaeyeon Kim, Donguk Lim, Seungmin Yang, Ki-Ung Song, Jewon Lee |
This paper presents Trimmed Llama, a method for improving inference efficiency in cross-attention-based Large Vision-Language Models (LVLMs) by pruning visual features. The main objective is to mitigate the computational bottleneck caused by the large Key-Value (KV) cache size associated with image tokens in cross-attention layers. The key methodology involves exploiting the sparsity and inter-layer resemblance of cross-attention patterns, using head-wise attention scores from the first cross-attention layer to selectively prune redundant visual features for subsequent layers. Primary results show that Trimmed Llama can reduce visual feature usage by up to 50% (e.g., Kratio=0.15 retaining ~41.6% features for the 11B model) while maintaining performance parity with baseline Llama-3.2-Vision models on benchmarks like MME and LLaVA-Bench, alongside reduced inference latency (e.g., 14.2% reduction for batch size 16). For AI practitioners, this provides a training-free technique to decrease inference latency and memory consumption for cross-attention LVLMs with minimal impact on task performance. |
| Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies |
|
|
| Ahead (Read more on arXiv or HuggingFace) |
Neel Joshi, Shivam Garg, Lingjiao Chen, Jingya Chen, Vidhisha Balachandran |
This extensive empirical study evaluates the benefits and limitations of inference-time scaling methods across diverse complex tasks for large language models (LLMs). The main objective was to investigate how scaling performance, including accuracy and token usage tradeoffs, varies across nine state-of-the-art conventional and reasoning-tuned models on eight challenging benchmarks (e.g., math, NP-hard problems, planning, spatial reasoning). Key methodologies included evaluating models using standard Chain-of-Thought (CoT), parallel scaling (sampling N generations with aggregators like best-of-N), and sequential scaling (iterative refinement with self-critique), approximating performance bounds. Primary results show inference-time scaling benefits vary significantly by task and diminish with complexity; notably, increased token consumption does not reliably yield higher accuracy across models (e.g., on AIME 25, DeepSeek R1 used >5x more tokens than Claude 3.7 Sonnet for <3% accuracy difference). The principal implication for AI practitioners is that leveraging inference-time compute requires careful task-specific consideration and highlights the critical need for developing robust, efficient verifiers and adaptive scaling strategies, as current approaches show inconsistent gains and cost nondeterminism. |
| Discovering Knowledge Deficiencies of Language Models on Massive |
|
|
| Knowledge Base (Read more on arXiv or HuggingFace) |
Ryotaro Shimizu, Jieyu Zhang, Xuwei Ding, MaksimSTW, linxinso |
This paper introduces Stochastic Error Ascent (SEA), a scalable framework for efficiently discovering factual knowledge deficiencies in closed-weight LLMs against massive knowledge bases under budget constraints. The primary objective is to develop a scalable and budget-constrained method for automatically uncovering knowledge deficiencies (errors) in closed-weight LLMs by evaluating them against large knowledge bases. The core methodology, SEA, uses stochastic optimization to iteratively retrieve knowledge base paragraphs semantically similar to prior LLM failures, employing hierarchical retrieval and a relation DAG to guide the search efficiently. Empirically, SEA uncovered 40.7× more knowledge errors than the Automated Capability Discovery baseline and 26.7% more than AutoBencher, while significantly reducing the cost-per-error. For AI practitioners, SEA provides a cost-effective method to pinpoint specific factual weaknesses in LLMs, enabling targeted improvements through data curation or fine-tuning to enhance model reliability. |
| m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning |
|
|
| with Large Language Models (Read more on arXiv or HuggingFace) |
Yuyin Zhou, Xianfeng Tang, Hui Liu, Juncheng Wu, Xiaoke Huang |
This paper introduces m1, a method applying test-time scaling to enhance the medical reasoning capabilities of Large Language Models (LLMs). The primary objective was to investigate the effectiveness of test-time scaling for medical QA, contrasting it with mathematical reasoning tasks. The methodology involved curating medical QA datasets (m1K, m23K), fine-tuning Qwen2.5 models (7B, 32B) on these datasets using Supervised Fine-Tuning (SFT), and controlling the “thinking” token budget during inference. Results show that increasing the thinking budget improves accuracy (e.g., m1-7B-23K achieved 60.32% average accuracy), but plateaus around 4K tokens; budget forcing offered limited benefits, and performance gains were ultimately constrained by the model’s inherent medical knowledge. For AI practitioners, this implies that while test-time scaling enhances medical reasoning, it is insufficient alone; complementing it with improved knowledge grounding via high-quality data curation and larger model capacity is essential for further performance gains, especially on complex medical tasks. |
| Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs (Read more on arXiv or HuggingFace) |
Gül Varol, Cordelia Schmid, Antoine Yang, Lucas Ventura |
Chapter-Llama introduces an efficient LLM-based framework for automatic video chaptering in hour-long videos. The primary objective is to partition long videos into semantic chapters and generate corresponding titles automatically. The methodology involves finetuning a large language model (Llama-3.1-8B) using text inputs derived from ASR transcripts and descriptive captions of sparsely sampled keyframes, selected via a novel speech-guided strategy. Results show substantial improvement over the state-of-the-art on VidChapters-7M, achieving a 45.3 F1 score compared to the previous best of 26.7. For AI practitioners, this work presents a scalable, text-only approach leveraging LLMs and efficient frame sampling for indexing and structuring long-form video content without direct video feature processing. |
| Towards Trustworthy GUI Agents: A Survey (Read more on arXiv or HuggingFace) |
Ninghao Liu, Wenhu Chen, Wenlin Yao, Wenhao Yu, Yucheng Shi |
This survey reviews the critical dimensions of trustworthiness for GUI agents interacting with digital interfaces via foundation models. The paper’s objective is to systematically examine security vulnerabilities, reliability, explainability, ethical considerations, and evaluation methodologies pertinent to GUI agent trustworthiness. It employs a literature survey methodology, categorizing research into five key trustworthiness areas and summarizing existing attacks (e.g., WebPI, AEIA-MN), defenses (e.g., GuardAgent, CLEAR), and evaluation frameworks (e.g., ST-WebAgentBench, Agent-SafetyBench). Key findings identify significant security vulnerabilities, such as environmental injection attacks achieving up to 93% success rates (AEIA-MN), alongside challenges in reliability (hallucination) and privacy, while noting that current research often overlooks these aspects for functional performance. For AI practitioners, this necessitates a shift from solely optimizing task completion towards implementing holistic, multi-layered defenses, robust evaluation benchmarks incorporating safety metrics, and user-centric transparency mechanisms to ensure secure and responsible GUI agent deployment. |
| DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D |
|
|
| Gaussian Splatting (Read more on arXiv or HuggingFace) |
Gim Hee Lee, onandon |
DiET-GS introduces a novel framework for motion deblurring in 3D Gaussian Splatting using event streams and diffusion priors. The research addresses the problem of reconstructing sharp 3D representations from blurry multi-view images. It leverages an event double integral prior and a pretrained diffusion model within a two-stage training strategy. DiET-GS outperforms existing methods, achieving a MUSIQ score of 51.71 on real-world datasets, but the DiET-GS++ has a longer training time compared to E2NeRF and Ev-DeblurNeRF. This provides AI practitioners with an approach for improving novel view synthesis from motion-blurred images. |
| ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via |
|
|
| Residual Learning (Read more on arXiv or HuggingFace) |
Siyuan Huang, Yuyang Li, Tengyu Liu, Puhao Li, Kailin Li |
i) The paper introduces MANIPTRANS, a two-stage method for efficient transfer of human bimanual skills to dexterous robotic hands in simulation. ii) The primary research objective is to facilitate the transfer of human hand manipulation skills, especially bimanual actions, to dexterous robotic hands in simulation enabling accurate tracking of reference motions. iii) The method uses a pre-trained generalist trajectory imitator for hand motion mimicking followed by a fine-tuned residual module under interaction constraints. iv) MANIPTRANS achieves superior success rates (58.1/39.5 for single/bimanual respectively) compared to SOTA methods and constructs DEXMANIPNET, a dataset of 3.3K episodes of robotic manipulation, improving motion fidelity. v) The development of MANIPTRANS offers AI practitioners an efficient and generalizable framework for creating large-scale, high-quality datasets of dexterous manipulation enabling more effective training of robot control policies. |
| MB-ORES: A Multi-Branch Object Reasoner for Visual Grounding in Remote |
|
|
| Sensing (Read more on arXiv or HuggingFace) |
Mustapha lebbah, Hanane Azzag, rdkarim |
MB-ORES introduces a unified framework for object detection (OD) and visual grounding (VG) in remote sensing (RS) imagery. The paper aims to improve visual grounding in RS images by fine-tuning an open-set object detector with referring expression data and then processing outputs via a graph-based representation and a multi-branch, task-aware architecture. The methodology incorporates a multi-branch network for feature integration, an object reasoning network for proposal ranking, and a soft selection mechanism for object localization. Experiments on DIOR-RSVG show MB-ORES outperforms existing methods, increasing performance by +3.38% to +14.89% across threshold levels, while on the OPT-RSVG dataset, meanIoU increased by +6.98%. This implies a unified OD/VG approach, applicable by AI practitioners in the remote sensing domain, can achieve state-of-the-art performance while retaining OD capabilities, offering a more versatile tool. |
Papers for 2025-04-01
| Title |
Authors |
Summary |
| TextCrafter: Accurately Rendering Multiple Texts in Complex Visual |
|
|
| Scenes (Read more on arXiv or HuggingFace) |
Nikai Du, yingtai, jzzzzk, Chenzzzzzz, zhen-nan |
TextCrafter is a training-free framework designed to accurately render multiple texts across different regions in complex visual scenes generated by diffusion models. The primary objective is to address limitations like text distortion, omission, and blurriness encountered in Complex Visual Text Generation (CVTG). The methodology involves a progressive three-stage approach: Instance Fusion to align text content with its visual carrier, Region Insulation to separate text prompts and initialize layout using pre-trained model priors, and Text Focus to enhance text token attention for improved fidelity. Experiments on the newly proposed CVTG-2K benchmark show TextCrafter achieves a 0.7370 average Word Accuracy, significantly improving OCR accuracy by over 45% compared to the baseline FLUX model it builds upon. For AI practitioners, this provides an effective method to enhance multi-text rendering capabilities in text-to-image systems without requiring additional model training or fine-tuning, improving performance on complex scene generation with detailed textual elements. |
| MoCha: Towards Movie-Grade Talking Character Synthesis (Read more on arXiv or HuggingFace) |
Luczzz, daixl1992, FelixXu, haoyum1997, lim142857 |
MoCha introduces an end-to-end Diffusion Transformer model for generating movie-grade talking characters directly from speech and text inputs without auxiliary conditions. The primary objective is to create realistic characters with synchronized lip movements, natural facial expressions, coherent full-body actions, and support for multi-character, turn-based conversations, addressing limitations in prior work focused on talking heads or general video synthesis lacking speech control. Key methodologies include a speech-video window attention mechanism for improved lip-sync, a joint training strategy leveraging both speech-labeled and text-only video data for better generalization, and structured character-tagged prompts for multi-character dialogue. MoCha significantly outperforms baselines on the MoCha-Bench benchmark, achieving superior human evaluation scores across all five axes (e.g., +1.40 in Lip-Sync Quality over the next best) and better quantitative lip-sync metrics (Sync-C: 6.037 vs 4.866). For AI practitioners, MoCha offers a direct speech+text-to-video synthesis approach for controllable character animation, enabling richer narrative generation for applications like automated filmmaking and virtual avatars without reliance on intermediate representations like keypoints or explicit pose control. |
| What, How, Where, and How Well? A Survey on Test-Time Scaling in Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
nancy-zwx, demolei, RubinSun, silentspring2, DonJoey |
This survey introduces a unified four-dimensional framework (what, how, where, how well) to systematically organize and analyze research on Test-Time Scaling (TTS) in Large Language Models. Its objective is to address the lack of a comprehensive overview by categorizing TTS methods, applications, and evaluation metrics, identifying trends, and outlining future directions. The paper proposes a multi-axis taxonomy and conducts an extensive literature review, decomposing techniques like parallel, sequential, hybrid, and internal scaling, alongside tuning-based (SFT, RL) and inference-based (stimulation, verification, search, aggregation) implementation strategies. The review confirms TTS significantly enhances LLM performance across various tasks, observing scaling-law-like improvements with increased compute, and highlights specific techniques like internal scaling via RL (e.g., DeepSeek-R1) or search methods yielding efficiency gains (e.g., ETS achieving 1.8x KV cache reduction). AI practitioners can utilize the taxonomy and guidelines (Section 7) to select, combine, and evaluate complementary TTS strategies (e.g., Self-Consistency, MCTS, STaR, internal scaling) for balancing performance, cost, and task-specific requirements in LLM deployment. |
| Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement |
|
|
| Learning on the Base Model (Read more on arXiv or HuggingFace) |
Xiangyu Zhang, Qi Han, djiang, YinminZhang, reign12 |
Open-Reasoner-Zero (ORZ) introduces an open-source, minimalist approach for large-scale reinforcement learning (RL) focused on enhancing reasoning in base language models. The primary objective was to determine if vanilla PPO with simple rule-based rewards and no KL regularization could scale LLM reasoning performance and response length effectively. The methodology involved applying PPO with GAE (λ=1, γ=1) and a binary correctness reward directly to Qwen2.5 base models (0.5B to 32B) using a curated reasoning dataset. Results showed that ORZ-32B surpassed the DeepSeek-R1-Zero-Qwen-32B model on benchmarks like MATH500 (92.2 vs 91.6) and GPQA Diamond (55.5 vs 55.0) using only 1/10th the training steps, demonstrating stable scaling without KL constraints. The principal implication for AI practitioners is that complex RLHF setups with KL regularization may not be necessary for scaling reasoning; a simpler, resource-efficient PPO configuration can yield strong results directly on base models. |
| RIG: Synergizing Reasoning and Imagination in End-to-End Generalist |
|
|
| Policy (Read more on arXiv or HuggingFace) |
Haian Huang, Zhonghan Zhao, GaoangWang, pppppM, ZwwWayne |
RIG introduces an end-to-end generalist policy that synergizes reasoning and imagination for embodied agents. The research aims to improve sample efficiency and generalization by integrating reasoning and imagination into a single Transformer model. The methodology involves a progressive data collection strategy to generate reasoning-enriched and dream-review trajectories coupled with language model-based training. Experimental results in Minecraft demonstrate that RIG achieves state-of-the-art performance, showing more than 17× sample efficiency improvements compared to prior works, requiring only 111 hours of video data; also shown is the improvement of robustness and interoperability of generalist policy. RIG provides AI practitioners with an architecture that enhances the performance and scalability of embodied agents by combining reasoning and imagination, offering a pathway towards more efficient and robust policy learning in complex environments. |
| Effectively Controlling Reasoning Models through Thinking Intervention (Read more on arXiv or HuggingFace) |
Prateek Mittal, Jiachen T. Wang, cxiang, tongwu2020 |
Reasoning models can be controlled through Thinking Intervention, a paradigm for guiding internal reasoning processes via strategic token insertion or revision. The research question explores fine-grained control over model behavior by guiding internal reasoning processes of LLMs. The methodology involves comprehensive evaluations across instruction following, instruction hierarchy, and safety alignment tasks. Results show that Thinking Intervention achieves up to a 6.7% accuracy gain in instruction-following, a 15.4% improvement in reasoning about instruction hierarchies, and a 40.0% increase in refusal rates for unsafe prompts using open-source DeepSeek R1 models. Thinking Intervention enables fine-grained control over reasoning trajectories, aligning model behavior with specific task objectives, allowing for more reliable and aligned AI systems. |
| Query and Conquer: Execution-Guided SQL Generation (Read more on arXiv or HuggingFace) |
sfc-mwydmuch, Borchmann |
i) The paper introduces an execution-guided self-consistency approach for text-to-SQL generation. ii) The research aims to improve accuracy in complex text-to-SQL tasks by leveraging execution results for candidate query selection. iii) The methodology utilizes exact and approximate execution-based similarity metrics within the Minimum Bayes Risk (MBR) decoding framework. iv) The Qwen 2.5 Coder 7B model employing this method achieves nearly a 10% accuracy improvement, matching the performance of O1 while reducing inference cost by 30 times. v) AI practitioners can leverage execution-guided self-consistency to improve the performance of smaller, cost-effective models in text-to-SQL tasks. |
| SketchVideo: Sketch-based Video Generation and Editing (Read more on arXiv or HuggingFace) |
dizhang, WeicaiYe, Xintao, fuhongbo, Okrin |
SketchVideo presents a unified framework for generating and editing videos conditioned on sparse keyframe sketches and text prompts. The research objective is to achieve precise spatial layout and motion control in video synthesis and editing using temporally sparse user-drawn sketches. It utilizes a skip-residual sketch control structure for DiT models, an inter-frame attention mechanism for propagating sparse conditions, and a video insertion module with latent fusion for editing. Experiments show superior performance, with SketchVideo achieving the lowest LPIPS (27.56) and highest CLIP score (98.31) in generation benchmarks compared to methods like SparseCtrl. AI practitioners can implement this technique to provide users with fine-grained geometric and motion control in video creation/editing tools, enhancing controllability beyond text-only approaches. |
| TeleAntiFraud-28k: A Audio-Text Slow-Thinking Dataset for Telecom Fraud |
|
|
| Detection (Read more on arXiv or HuggingFace) |
Kai Wu, Jingpeng Wang, HuangMinhua, WDong, JimmyMa99 |
This paper introduces TeleAntiFraud-28k, an open-source audio-text dataset with slow-thinking annotations for telecom fraud detection. The research aims to overcome the lack of suitable multimodal training data by integrating audio signals with reasoning-oriented textual analysis for automated fraud identification. Methodology involves dataset construction via three strategies: processing real anonymized calls with ASR/TTS, semantic expansion using LLM self-instruction, and multi-agent adversarial simulation, followed by LLM-based annotation capturing reasoning steps. Key results include the creation of 28,511 audio-text pairs and the demonstration that fine-tuning Qwen2Audio on this dataset significantly boosted fraud detection F1 score to 84.78% (average F1 across tasks 83.00%) on the established TeleAntiFraud-Bench. For AI practitioners, this work provides a crucial dataset and benchmark for developing and evaluating multimodal, reasoning-capable audio language models specifically for the challenging task of telecom fraud detection. |
| Efficient Inference for Large Reasoning Models: A Survey (Read more on arXiv or HuggingFace) |
jiaheng233, Bibaolong, HongyuChen, HongchengGao, yueliu1999 |
This survey reviews and categorizes methods for improving the inference token efficiency of Large Reasoning Models (LRMs) while maintaining reasoning quality. The primary objective is to analyze techniques mitigating high token consumption, memory overhead, and inference time inherent in LRM’s deliberative reasoning processes. It introduces a taxonomy classifying approaches into explicit compact Chain-of-Thought (CoT), which reduces tokens while keeping explicit structure, and implicit latent CoT, which encodes reasoning in hidden representations, alongside empirical analysis. Key findings categorize methods based on whether they maintain explicit reasoning steps or encode them latently; for instance, on GSM8K, explicit methods like TokenSkip (ratio=0.5) achieve 86.70% accuracy using 113.05 tokens with LLaMA-3.1-8B-Instruct, while implicit methods like SoftCoT reach 85.81% accuracy with Qwen2.5-7B-Instruct, though its specific token cost comparison is not fully detailed in the provided table excerpt. AI practitioners gain insights into the performance/efficiency trade-offs of LRM optimization techniques, informing the selection of methods (e.g., explicit CoT for interpretability, implicit CoT for token reduction) for developing cost-effective reasoning applications. |
| Classical Planning with LLM-Generated Heuristics: Challenging the State |
|
|
| of the Art with Python Code (Read more on arXiv or HuggingFace) |
jendrikseipp, andregrahl, abcorrea |
This paper demonstrates using Large Language Models (LLMs) to automatically generate domain-dependent heuristic functions as Python code for classical planning tasks. The objective was to determine if LLM-generated heuristics could outperform traditional domain-independent heuristics and compete with state-of-the-art learned heuristics. The methodology involved prompting an LLM (e.g., DeepSeek R1) multiple times for a given planning domain, evaluating the resulting pool of Python heuristic functions on training tasks using Greedy Best-First Search (GBFS), and selecting the best-performing one. Results show the selected LLM-generated heuristics significantly outperformed the widely used hFF heuristic (solving 373 vs. 243 test tasks in Pyperplan) and were competitive with state-of-the-art learned heuristics implemented in optimized C++, even when run in an unoptimized Python planner. For AI practitioners, this implies LLMs can automate the creation of highly effective, domain-specific heuristics for planning, potentially accelerating development and improving performance without requiring deep heuristic engineering expertise or specialized learning pipelines. |
| Expanding RL with Verifiable Rewards Across Diverse Domains (Read more on arXiv or HuggingFace) |
zptu, haitaominlp, douvleplus, freesunshine0316, yudian |
This paper extends Reinforcement Learning with Verifiable Rewards (RLVR) to diverse domains like medicine and economics, using a distilled generative reward model. The main objective is to investigate RLVR’s applicability beyond well-structured tasks and evaluate if a single, trained reward model can effectively provide cross-domain reward signals for free-form answers without domain-specific annotations. The methodology involves training a 7B parameter reward model using judgments distilled from a larger teacher LLM (Qwen2.5-72B-Instruct) and incorporating model-based soft scoring for RL fine-tuning (using REINFORCE, RLOO, etc.) of a base 7B policy model. Using RLOO with the distilled 7B reward model (RM-7B) and soft scoring yielded a 30.0% average accuracy on multi-subject tasks, outperforming the baseline rule-based reward (16.6%) and matching the performance using the much larger Qwen2.5-72B model directly for rewards (30.6%). For AI practitioners, this suggests that a smaller, distilled generative reward model can effectively guide RL fine-tuning across diverse domains with unstructured answers, offering a computationally efficient alternative to large teacher models or domain-specific reward engineering, enhancing RLVR’s scalability and robustness. |
| Progressive Rendering Distillation: Adapting Stable Diffusion for |
|
|
| Instant Text-to-Mesh Generation without 3D Data (Read more on arXiv or HuggingFace) |
Zhen Lei, Xiangyu Zhu, Rongyuan Wu, DarklordLeto, ZhiyuanthePony |
This paper presents Progressive Rendering Distillation (PRD) to adapt Stable Diffusion (SD) for instant text-to-mesh generation without 3D ground-truth data. The objective is to overcome the 3D data scarcity problem by distilling knowledge from multi-view 2D diffusion models into an SD-based native 3D generator. PRD progressively denoises latent noise over a few steps, decoding intermediate results into Triplanes and using score distillation with SD, MVDream, and RichDreamer as teachers; Parameter-Efficient Triplane Adaptation (PETA) adds only 2.5% trainable parameters via LoRA. The resulting model, TriplaneTurbo, generates high-quality textured meshes in 1.2 seconds, achieving a CLIP Score of 68.2, outperforming prior methods in speed and quality without 3D training data. For AI practitioners, this work demonstrates an effective, data-efficient method to repurpose large 2D diffusion models for rapid 3D content creation, significantly reducing reliance on 3D datasets and accelerating generation pipelines. |
| TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through |
|
|
| Task Tokenization (Read more on arXiv or HuggingFace) |
BoDai, WenjiaWang, frankzydou, Zeshi209, lianganimation |
TokenHSI introduces a unified transformer-based policy using task tokenization to synthesize diverse, physically plausible human-scene interactions. The primary objective is to develop a single, versatile physics-based controller capable of learning multiple foundational HSI skills and efficiently adapting them to novel, complex scenarios like skill composition or environment variations. Key methodology involves separate tokenizers for shared humanoid proprioception and distinct task states, combined within a transformer encoder via a masking mechanism, enabling multi-task learning and flexible adaptation by adding new tokenizers and lightweight adapter layers. Primary results demonstrate successful unification of diverse skills (following, sitting, climbing, carrying) and superior adaptation compared to baselines, achieving a 99.2% success rate on the challenging Climb + Carry skill composition task. For AI practitioners, this provides an efficient and extensible framework for building versatile physics-based agents capable of complex interactions, reducing the need for separate controllers per skill and enabling rapid adaptation to new tasks with minimal parameter fine-tuning. |
| KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large |
|
|
| Vision-Language Models in the Korean Language (Read more on arXiv or HuggingFace) |
lastdefiance20, yoonshik1205 |
This paper introduces KOFFVQA, a novel Korean free-form Visual Question Answering benchmark designed for objective evaluation of Large Vision-Language Models (VLMs). The main objective is to overcome the limitations of existing VLM evaluation methods, namely the subjectivity of judge models and the lack of Korean-specific benchmarks, by providing a reliable framework for assessing open-ended VLM responses. The methodology involves a benchmark dataset of 275 curated image-question pairs, each accompanied by detailed, objective grading criteria, which guide an LLM judge (specifically Gemma 2 9B in testing) to score VLM responses on a scale of 0-10 across 10 performance categories. Results from evaluating 47 VLMs show this criteria-guided LLM-judge approach achieves significantly higher evaluation consistency (e.g., mean score standard deviation of 0.398 for Gemma 2 9B vs. 0.584 for ground-truth comparison) and accuracy (89.3% correct grading for Gemma 2 9B) compared to methods using ground-truth comparisons or VLM-as-a-judge, which was found prone to visual hallucinations. For AI practitioners, this work provides a robust benchmark and methodology for objectively evaluating the free-form reasoning and Korean language capabilities of VLMs, highlighting that explicit, objective criteria significantly improve judge model reliability over subjective or ground-truth-comparative approaches. |
| UPME: An Unsupervised Peer Review Framework for Multimodal Large |
|
|
| Language Model Evaluation (Read more on arXiv or HuggingFace) |
Zheyuan Liu, Yibing, yuehuang, MunanNing, 77Hui |
This paper introduces UPME, an unsupervised peer review framework for evaluating Multimodal Large Language Models (MLLMs) using only image data, eliminating the need for human QA annotations. The research objective is to develop an objective MLLM evaluation method that avoids the high cost of human annotation and mitigates biases found in MLLM-as-a-judge systems. UPME utilizes a peer review process where MLLMs generate questions for images and evaluate peer answers using a vision-language scoring system (assessing correctness, visual understanding/reasoning, image-text correlation) refined by dynamic weight optimization based on evaluation consistency. Experimental results show UPME achieves high alignment with human judgments, attaining a Pearson correlation of 0.944 on the MMstar dataset, while significantly reducing verbosity and self-preference biases compared to baseline peer review methods. For AI practitioners, UPME offers a scalable, automated, and less biased approach to evaluate MLLM performance, particularly for visual capabilities, without requiring extensive human-annotated datasets. |
| Easi3R: Estimating Disentangled Motion from DUSt3R Without Training (Read more on arXiv or HuggingFace) |
Anpei Chen, Andreas Geiger, Yuliang Xiu, faneggg, rover-xingyu |
Easi3R introduces a training-free method to adapt the static 3D reconstruction model DUSt3R for dynamic 4D reconstruction by disentangling motion from its attention maps. The main objective is to extract and separate camera and object motion information implicitly encoded within DUSt3R’s attention layers without requiring retraining or fine-tuning on dynamic datasets. The key methodology involves aggregating spatial and temporal cross-attention maps to derive dynamic object segmentations, which are then used for attention re-weighting during a second inference pass and optional segmentation-aware global alignment. Easi3R significantly outperforms previous methods trained or fine-tuned on dynamic data across camera pose estimation, dynamic object segmentation (e.g., achieving 53.0 JM on DAVIS-all without SAM2 using the MonST3R backbone), and 4D point map reconstruction. For AI practitioners, this implies that task adaptation of large pre-trained models can sometimes be achieved through careful analysis and manipulation of internal representations like attention maps during inference, reducing the need for costly retraining on specialized dynamic datasets. |
| MeshCraft: Exploring Efficient and Controllable Mesh Generation with |
|
|
| Flow-based DiTs (Read more on arXiv or HuggingFace) |
Xiaoshui Huang, Zexiang Liu, Di Huang, Junyi Chen, Xianglong He |
MeshCraft introduces a novel framework for efficient and controllable 3D mesh generation using flow-based diffusion transformers. The paper addresses the challenge of slow generation speeds and uncontrollable face numbers in existing mesh generation techniques. MeshCraft employs a transformer-based VAE to encode and decode meshes in a continuous latent space and a flow-based diffusion transformer conditioned on the number of faces. Experiments demonstrate MeshCraft achieves a 35x speed increase compared to MeshGPT while maintaining state-of-the-art mesh quality. The framework’s efficient and controllable mesh generation capability enables AI practitioners to rapidly generate high-quality 3D assets with user-defined specifications. |
| Bridging Evolutionary Multiobjective Optimization and GPU Acceleration |
|
|
| via Tensorization (Read more on arXiv or HuggingFace) |
Ran Cheng, Kebin Sun, Naiwei Yu, Hao Li, ZhenyuLiang |
i) This paper introduces a tensorization methodology to accelerate evolutionary multiobjective optimization (EMO) algorithms on GPUs. ii) The research aims to bridge the gap between EMO algorithms and GPU computing by transforming EMO data structures and operations into tensor representations. iii) The methodology involves tensorizing data structures and operations within EMO algorithms and applying this tensorization to NSGA-III, MOEA/D, and HypE. iv) Experiments show that tensorized EMO algorithms achieve speedups of up to 1113× compared to CPU-based counterparts on a multi-objective robot control benchmark. v) Tensorization enables AI practitioners to effectively utilize GPUs to significantly improve the computational efficiency and scalability of EMO algorithms for complex optimization problems. |
| Decoupling Angles and Strength in Low-rank Adaptation (Read more on arXiv or HuggingFace) |
Zeynep Akata, Leander Girrbach, Massimo Bini |
i) The paper introduces Decoupled Low-rank Adaptation (DeLoRA), a novel parameter-efficient finetuning method. ii) The research aims to enhance the robustness of low-rank adaptation methods like LoRA by decoupling angular learning from adaptation strength. iii) DeLoRA normalizes and scales learnable low-rank matrices, bounding the transformation distance through normalization. iv) Experiments on subject-driven image generation demonstrate that DeLoRA achieves a DINO score of 0.693 and CLIP-I score of 0.820 matching or surpassing LoRA’s performance. v) AI practitioners can leverage DeLoRA to achieve more robust performance in adapting large-scale models to downstream tasks, particularly where hyperparameter tuning is challenging or extended training is required. |
| Entropy-Based Adaptive Weighting for Self-Training (Read more on arXiv or HuggingFace) |
Wei Wang, Mingyu Derek Ma, Yihe Deng, Xiaoxuan Wang |
i) This paper introduces Entropy-Based Adaptive Weighting for Self-Training (EAST), a novel method to improve mathematical reasoning in large language models (LLMs). ii) The research aims to address the challenge of effectively using self-generated data in self-training by prioritizing uncertain data points. iii) EAST assigns adaptive weights based on the entropy of the model’s sample distribution, using a mapping function with a tunable sharpness parameter integrated with SFT, DPO, and KTO loss functions. iv) On the MATH benchmark, EAST achieves approximately a 1% gain over the backbone model, and on GSM8K, it attains a further 1-2% performance boost compared to the vanilla method using the Llama-3.2-1B and Llama-3.1-8B architectures. v) EAST provides AI practitioners with an improved self-training strategy by reweighting training data to leverage uncertainty information, potentially increasing reasoning capabilities and reducing overfitting on overconfident data. |
Papers for 2025-03-31
| Title |
Authors |
Summary |
| AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through |
|
|
| Lightweight Vocabulary Adaptation (Read more on arXiv or HuggingFace) |
Roi Reichart, ehoffer, eyalbd, nitay, itaynakash |
AdaptiVocab enhances Large Language Model (LLM) efficiency in focused domains through lightweight vocabulary adaptation. Its objective is to reduce latency and computational costs in domain-specific, low-resource settings by optimizing the LLM’s vocabulary. The methodology involves replacing low-frequency general tokens with high-frequency domain-specific n-gram tokens based on a token-saving score, initializing new embeddings using exponential weighting, and performing lightweight fine-tuning on embedding and adjacent layers. Results across two 7B LLMs and three niche domains show over a 25% reduction in token usage for both input processing and output generation, without compromising generation quality or end-task performance. For AI practitioners, this offers a resource-efficient technique to improve the inference speed and reduce the operational cost of LLMs deployed for specialized applications, particularly in settings with limited data or computational budgets. |
| Exploring Data Scaling Trends and Effects in Reinforcement Learning from |
|
|
| Human Feedback (Read more on arXiv or HuggingFace) |
amusingchao, qingping95, zhengwu07, glnbyte, Swtheking |
This paper investigates data scaling challenges in RLHF, proposing data construction and training strategies to mitigate reward hacking and improve response diversity. The primary objective is to identify and overcome data-driven bottlenecks hindering RLHF performance scaling. Methodology involves a hybrid reward system combining Reasoning Task Verifiers (RTV) and Generative Reward Models (GenRM) with ground truth, alongside a Pre-PPO prompt selection method prioritizing challenging prompts and early-stage math/coding task training. Results demonstrate the proposed ‘Data Scale’ approach significantly outperforms baseline PPO, achieving a +1.4 overall score improvement on the challenging TestSet V2.0 for the large model, and RTV exhibited the strongest resistance to reward hacking. For AI practitioners, this work highlights that strategic data curation and robust reward mechanisms (like RTV/GenRM-GT) are critical for enhancing RLHF performance and scalability, offering practical methods to address reward hacking and diversity issues. |
| Think Before Recommend: Unleashing the Latent Reasoning Power for |
|
|
| Sequential Recommendation (Read more on arXiv or HuggingFace) |
Xu Chen, Jun Xu, TengShi, KID-22, TangJiakai5704 |
This paper introduces ReaRec, an inference-time framework that enhances sequential recommendation (SeqRec) models by incorporating multi-step implicit reasoning. The objective is to overcome the limitations of traditional direct forward inference in capturing complex user preference dynamics, especially for long-tail items. ReaRec achieves this by autoregressively feeding the last hidden state back into the SeqRec model, using specialized reasoning position embeddings, and employs two learning strategies: Ensemble Reasoning Learning (ERL) and Progressive Reasoning Learning (PRL). Empirical results show ReaRec improves performance by an average of 7.49% across metrics on five datasets, and notably, post-hoc analysis reveals it can raise the performance ceiling of backbone SeqRec models by approximately 30-50%. For AI practitioners, ReaRec presents a model-agnostic method to potentially improve existing SeqRec systems by strategically increasing computation during inference rather than solely relying on model parameter scaling. |
| A Survey of Efficient Reasoning for Large Reasoning Models: Language, |
|
|
| Multimodality, and Beyond (Read more on arXiv or HuggingFace) |
Elliott, weigao266, Warrieryes, yaful, Xiaoye08 |
This survey reviews methods to enhance the computational efficiency of reasoning processes in Large Reasoning Models (LRMs) throughout their development lifecycle. The paper’s objective is to categorize patterns of reasoning inefficiency, such as excessive token generation and overthinking simple problems, and provide a comprehensive overview of techniques aiming to improve reasoning efficiency. Methodologically, it defines reasoning efficiency η(M) = E[Q(M,D) / C(M,D)] and systematically surveys literature, classifying techniques across pretraining, SFT, RL, and inference stages, including length budgeting, model switching, reasoning chain compression, and architectural modifications. Primary results highlight significant inefficiencies, exemplified by an LRM (QwQ-32B) using nearly 40 times more tokens than an instruction-tuned model for a simple math problem, and detail various strategies to reduce computational cost, often involving a trade-off with performance accuracy. The principal implication for AI practitioners is the catalog of techniques (e.g., length budgeting, SFT compression, latent-space reasoning) that can be applied to mitigate excessive token usage and latency, enabling more cost-effective and resource-aware deployment of LRMs, especially in applications like agent-based systems. |
| ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation (Read more on arXiv or HuggingFace) |
Jihyun Lee, Minhyuk, 32V, daehyeonchoi, myhong |
ORIGEN introduces the first zero-shot method for grounding 3D orientation for multiple objects in text-to-image generation. The main research objective is to enable controllable 3D orientation in generated images without requiring specific training data or being limited to single objects or synthetic data. The key methodology involves a reward-guided sampling approach using a pretrained orientation estimation model (OrientAnything) and a one-step generative flow model, optimized via Langevin dynamics with adaptive time rescaling. Quantitative results on the MS-COCO-Single benchmark show ORIGEN achieves significantly better orientation alignment (e.g., 87.1% Acc.@22.5° azimuth accuracy) compared to prior orientation-conditioned models and training-free guidance methods. For AI practitioners, this provides a training-free mechanism to impose precise 3D orientation constraints on generated objects, improving spatial controllability in text-to-image synthesis for complex scenes. |
| Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal |
|
|
| Consistency (Read more on arXiv or HuggingFace) |
skhu101, GuangcongWang, FrozenBurning, Inso, tqliu |
Free4D introduces a tuning-free framework for generating spatially-temporally consistent 4D scenes from a single image or text input. The primary objective is to produce high-quality, controllable 4D scene representations from limited observations without expensive training or finetuning, ensuring spatial-temporal consistency. Key methodologies involve initializing 4D geometry using image-to-video diffusion and dynamic reconstruction, generating consistent multi-view videos via adaptive guidance and latent replacement strategies, and optimizing a final 4D Gaussian Splatting representation using a coarse-to-fine strategy with modulation-based refinement. Compared to the text-to-4D baseline 4Real on VBench, Free4D demonstrates improved performance in Dynamics (47.4% vs 32.3%) and Aesthetics (64.7% vs 50.9%). For AI practitioners, this work offers an efficient pipeline for generating dynamic 4D scenes directly from single images, reducing reliance on large-scale 4D datasets or model tuning for applications in immersive media and virtual environments. |
| PHYSICS: Benchmarking Foundation Models on University-Level Physics |
|
|
| Problem Solving (Read more on arXiv or HuggingFace) |
armanc, jsous, henryL7, yilunzhao, Carrie777 |
This paper introduces PHYSICS, a benchmark with 1,297 university-level physics problems to evaluate foundation models’ advanced problem-solving skills. The primary objective is to assess foundation models’ abilities in multi-step reasoning, mathematical derivation, and domain-specific knowledge application in physics. The methodology involves expert annotation of PhD-qualifying exam problems and a robust automated evaluation system combining SymPy-based verification with GPT-4o assessment. Results show significant limitations even for top models, with the best proprietary model (o3-mini) achieving only 59.9% accuracy, revealing persistent challenges in calculation, assumption validity, and knowledge integration. For AI practitioners, this highlights the substantial gap remaining for models to reach expert-level scientific reasoning, necessitating further research into robust mathematical handling and effective knowledge grounding. |
| Perceptually Accurate 3D Talking Head Generation: New Definitions, |
|
|
| Speech-Mesh Representation, and Evaluation Metrics (Read more on arXiv or HuggingFace) |
taehyunoh, akasha9890, backryun, Han-EunGi, Chae-Yeon |
This paper defines criteria and introduces a speech-mesh representation and metrics for perceptually accurate 3D talking head generation. The research aims to define and improve the perceptual accuracy of lip movements in speech-driven 3D talking heads, focusing on Temporal Synchronization, Lip Readability, and Expressiveness. A speech-mesh synchronized representation is developed using a two-stage training process, leveraging large-scale 2D audio-visual data before aligning with 3D mesh data, and is applied as a perceptual loss and metric (PLRS), alongside two new physical metrics (MTM for synchronization, SLCC for expressiveness). Experiments show that incorporating the proposed perceptual loss significantly improves existing models across all three criteria; for instance, applying it to FaceFormer on the VOCASET dataset improved the Perceptual Lip Readability Score (PLRS) from 0.368 to 0.463. AI practitioners can utilize the proposed perceptual loss to enhance the realism of 3D talking heads and employ the introduced metrics (MTM, PLRS, SLCC) for a more comprehensive, perceptually-grounded evaluation beyond traditional geometric error metrics like LVE. |
| Segment Any Motion in Videos (Read more on arXiv or HuggingFace) |
Nan Huang, qianqian68, akanazawa, kurtkeutzer, chenfengx |
This paper introduces a novel method for Moving Object Segmentation (MOS) by integrating long-range trajectories, semantic features, and foundation model prompting. The objective is to accurately segment objects based solely on their observable motion within a video, even in challenging scenarios like occlusions or complex deformations. The methodology combines long-range point tracks with DINO semantic features using specialized Spatio-Temporal Trajectory Attention and Motion-Semantic Decoupled Embedding, followed by an iterative prompting strategy with SAM2 to generate dense masks from sparse tracks. The proposed approach achieves state-of-the-art results on multiple benchmarks, including a 91.0 F-score on the DAVIS2016 MOS task, outperforming previous methods. For AI practitioners, this work demonstrates a powerful technique for video understanding tasks, showcasing how combining long-term motion cues, semantic context, and large segmentation models like SAM2 can yield robust and precise segmentation of moving objects where traditional optical flow or VOS methods might fail. |
| Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal |
|
|
| Bridging (Read more on arXiv or HuggingFace) |
Xiaoyang Guo, Jiahao Chang, Yushuang Wu, Chongjie Ye, LUZITENG |
Hi3DGen introduces a novel framework for high-fidelity 3D geometry generation from single images by leveraging normal maps as an intermediate bridge. The primary objective is to accurately reproduce fine-grained geometric details from 2D images, addressing limitations like domain gaps and inherent RGB ambiguities in existing methods. Key methodology involves a noise-injected, dual-stream image-to-normal estimator (NiRNE) for sharp normal prediction, and a normal-to-geometry latent diffusion learner (NoRLD) with explicit normal map regularization, supported by a high-quality synthetic 3D dataset (DetailVerse). The framework demonstrates superior performance, with NiRNE achieving a Normal Error (NE) of 21.837 on the LUCES-MV dataset, significantly outperforming prior state-of-the-art methods, and user studies confirm higher perceived fidelity. For AI practitioners, this work presents a technique using normal maps as an explicit intermediate representation with regularization in latent diffusion to significantly enhance the geometric detail and fidelity of single-image 3D model generation pipelines. |
| ReFeed: Multi-dimensional Summarization Refinement with Reflective |
|
|
| Reasoning on Feedback (Read more on arXiv or HuggingFace) |
jasoncai, hwany-j, Myyhlee, hyang0503, hamzzi |
This paper introduces ReFeed, a pipeline employing reflective reasoning on feedback to refine text summaries across multiple quality dimensions simultaneously. The primary objective is to enhance summarization refinement beyond single dimensions like faithfulness, addressing inter-dimensional trade-offs, feedback ordering bias, and sensitivity to noisy LLM-generated feedback. ReFeed utilizes a novel dataset, SumFeed-CoT, containing Long-CoT reflective reasoning distilled from a large reasoning model, to fine-tune a lightweight model (LLaMA-3.1-8B) capable of backtracking and validating feedback during refinement. Experiments show ReFeed significantly outperforms baselines, improving average summary quality by 8.4 points over initial summaries and specifically boosting completeness by 13.6 points, while demonstrating robustness to feedback noise and order. For AI practitioners, ReFeed offers a method and dataset to build lightweight yet effective multi-dimensional refinement models that mitigate quality trade-offs by incorporating distilled reflective reasoning, crucial for robust real-world deployment. |
| OThink-MR1: Stimulating multimodal generalized reasoning capabilities |
|
|
| via dynamic reinforcement learning (Read more on arXiv or HuggingFace) |
Changwang Zhang, Feng Liu, Yuting Zhang, Zhiyuan Liu, jwanglux |
OThink-MR1 introduces GRPO-D, a dynamic reinforcement learning strategy, to enhance the generalized multimodal reasoning capabilities of MLLMs beyond standard fine-tuning. The primary objective is to overcome the limitations of SFT and static RL by developing a dynamic RL approach (GRPO-D) that fosters better same-task performance and cross-task generalization for multimodal reasoning. The key methodology is GRPO-D, which employs a dynamically adjusted Kullback-Leibler (KL) divergence weight during reinforcement learning fine-tuning to optimally balance policy exploration and exploitation based on verifiable multimodal task rewards. GRPO-D demonstrated superior same-task and cross-task performance, achieving over a 61.63% relative improvement versus SFT in cross-task generalization evaluations where SFT showed poor transferability. For AI practitioners, GRPO-D provides a superior fine-tuning technique for MLLMs, enabling the development of models with stronger, transferable reasoning abilities across diverse multimodal tasks without requiring retraining for each specific task. |
| Your ViT is Secretly an Image Segmentation Model (Read more on arXiv or HuggingFace) |
Giuseppe Averta, Narges Norouzi, Alexander Hermans, Niccolò Cavagnero, Tommie Kerssies |
This paper introduces the Encoder-only Mask Transformer (EoMT), demonstrating that a plain Vision Transformer (ViT) can perform image segmentation without task-specific components like adapters or decoders. The study investigates if these components are essential for state-of-the-art ViT-based segmentation, hypothesizing their relevance diminishes with larger models and extensive pre-training. By systematically removing components from a ViT-Adapter + Mask2Former baseline and repurposing the ViT encoder blocks to process learnable queries alongside patch tokens, supplemented by a mask annealing strategy for efficient inference, EoMT is developed. Results show that EoMT with ViT-L achieves comparable Panoptic Quality (56.0 PQ) to the baseline (57.1 PQ) on COCO while being 4.4x faster (128 FPS vs 29 FPS). For AI practitioners, this implies that investing compute in scaling ViT models and pre-training, rather than adding architectural complexity, can yield simpler, faster, and highly accurate segmentation models that readily benefit from foundation model advancements. |
| 4D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
mhelhoseiny, ajhamdi, TonNew, bing-li-ai, vxuanz |
This paper introduces 4D-Bench, the first benchmark designed to evaluate the capabilities of Multimodal Large Language Models (MLLMs) in understanding dynamic 4D objects through question answering and captioning tasks. The objective is to assess current MLLM performance in multi-view spatial-temporal reasoning for 4D assets, addressing the lack of standardized evaluation in this domain. The methodology involved creating a dataset from rendered dynamic 3D objects (Objaverse-XL) into multi-view videos, curating data via motion and quality filters, and generating challenging QA pairs and human-annotated captions, followed by evaluating multiple MLLMs using accuracy and diverse captioning metrics, including GPT-4o assessment. Key results show MLLMs significantly underperform humans, with the state-of-the-art GPT-4o achieving only 62.98% overall accuracy on the 4D object QA task compared to a 91.08% human baseline, demonstrating particular weakness in object counting (37.29% average accuracy) and temporal reasoning. For AI practitioners, this highlights substantial MLLM limitations in integrating complex spatial-temporal information for 4D objects and handling counterfactual data, indicating a need for developing more robust models for applications involving dynamic 3D assets. |
| A Refined Analysis of Massive Activations in LLMs (Read more on arXiv or HuggingFace) |
Fabian Güra, akanyaani, nilabhra, louisowen6 |
This paper analyzes massive activations across diverse LLMs, challenging prior assumptions and evaluating mitigation strategies. The research objective is to systematically assess the characteristics, impact, and mitigation of massive activations across a broader range of GLU and non-GLU based LLM architectures than previously studied. Methodology involves intervention analysis (setting activations to zero/mean) on pre-trained models and retraining LLaMA-1B/GPT-2 with mitigation techniques (Attention KV Bias, TVR, DyT, hybrids), evaluating perplexity and downstream task performance. Primary results contradict prior claims, showing not all massive activations are detrimental, Attention KV bias mitigation is ineffective for architectures like LLaMA-1B, and hybrid strategies such as TVR + KV Bias successfully mitigate activations in LLaMA-1B (mean downstream task accuracy 52.0 vs 50.3 baseline) while preserving performance. The principal implication for AI practitioners is that mitigating massive activations, crucial for quantization and numerical stability, requires architecture-specific analysis and potentially hybrid approaches like TVR+KV Bias or TVR+DyT, as universal solutions are ineffective. |
| SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modeling (Read more on arXiv or HuggingFace) |
Lp256, pookiefoof, bennyguo, zouzx, XianglongHe |
SparseFlex introduces a sparse-structured isosurface representation for high-resolution, arbitrary-topology 3D shape modeling. The primary objective is to create high-fidelity 3D meshes (up to 1024³) with complex geometries, open surfaces, and interiors directly from rendering supervision, overcoming limitations of existing methods. Key methodologies involve adapting Flexicubes within a sparse voxel structure and employing a novel frustum-aware sectional voxel training strategy that activates only relevant voxels during rendering to drastically reduce memory consumption. Experiments demonstrate state-of-the-art reconstruction accuracy, evidenced by an ~82% reduction in Chamfer Distance and an ~88% increase in F-score compared to previous methods on tested benchmarks. For AI practitioners, this work provides a memory-efficient pathway to train high-resolution, differentiable mesh reconstruction and generation models using only rendering losses, facilitating the creation of detailed 3D assets with arbitrary topology without costly watertight preprocessing. |
| MedAgent-Pro: Towards Multi-modal Evidence-based Medical Diagnosis via |
|
|
| Reasoning Agentic Workflow (Read more on arXiv or HuggingFace) |
Yueming Jin, Chang Han Low, morson, ZiyueWang |
MedAgent-Pro introduces a reasoning agentic workflow for evidence-based, multi-modal medical diagnosis. The primary objective is to enhance diagnostic reliability and explainability compared to standard MLLMs by strictly adhering to retrieved clinical criteria and enabling quantitative analysis. The methodology utilizes a hierarchical agentic workflow: a task-level planner uses RAG to generate diagnostic plans based on medical knowledge, while case-level tool agents (specialized vision/VQA models, coding agent) execute steps on patient data, followed by a decider agent integrating findings. MedAgent-Pro significantly outperformed baselines, achieving 90.4% mACC on Glaucoma diagnosis using its MOE decider, a 32.3% absolute improvement over the best single foundation model tested (BioMedClip). For AI practitioners, this work implies that augmenting MLLMs with structured agentic workflows, external specialized tools, and explicit knowledge retrieval is crucial for building reliable and interpretable systems in domains requiring rigorous, evidence-based quantitative reasoning like medical diagnosis. |
| X^{2}-Gaussian: 4D Radiative Gaussian Splatting for Continuous-time |
|
|
| Tomographic Reconstruction (Read more on arXiv or HuggingFace) |
yixuanyuan, XGGNet, Fanzhiwen, CaiYuanhao, vortex778 |
X²-Gaussian presents a novel framework for continuous-time 4D computed tomography (CT) reconstruction using dynamic radiative Gaussian splatting. The objective is to reconstruct 4D CT volumes at arbitrary time points directly from projections, eliminating discrete phase binning and the need for external respiratory gating devices. The methodology integrates dynamic radiative Gaussian splatting, modeled via a spatiotemporal encoder-decoder for continuous deformation prediction, with a self-supervised, physiology-driven periodic consistency loss to learn respiratory cycles directly from projection data. Results demonstrate state-of-the-art performance, achieving a 9.93 dB PSNR improvement over traditional methods and a 2.25 dB gain over prior Gaussian splatting approaches on the DIR dataset. For AI practitioners, this provides a hardware-free method for high-fidelity, continuous dynamic medical image reconstruction, potentially enhancing motion analysis in clinical applications like image-guided radiotherapy. |
| On Large Multimodal Models as Open-World Image Classifiers (Read more on arXiv or HuggingFace) |
Yiming Wang, Enrico Fini, paolorota, massimilianom, altndrr |
This paper evaluates Large Multimodal Models (LMMs) for open-world image classification beyond predefined categories. The objective was to assess LMM performance in an unconstrained classification setting and analyze prediction errors using novel metrics. The methodology involved evaluating 13 LMMs on 10 benchmarks using four metrics (Text Inclusion, Llama Inclusion, Semantic Similarity, Concept Similarity) to measure alignment between generated text and ground truth labels. Results indicate LMMs outperform open-world contrastive baselines (e.g., CaSED) on inclusion metrics but significantly underperform closed-world models (e.g., CLIP), with notable errors in granularity (e.g., predicting “dog” instead of “pug”) and fine-grained discrimination; for instance, even the best models struggled significantly on very fine-grained datasets, often achieving near 0% Text Inclusion. AI practitioners should recognize current LMMs’ limitations in specific open-world classification, noting that while promising, tailored prompting and reasoning only partially alleviate errors related to granularity and fine-grained distinctions compared to traditional closed-world approaches. |
| Reconstructing Humans with a Biomechanically Accurate Skeleton (Read more on arXiv or HuggingFace) |
Qixing Huang, Etienne Vouga, Xiaowei Zhou, geopavlakos, IsshikiHugh |
This paper presents HSMR, a method for single-image 3D human reconstruction using the biomechanically accurate SKEL model. The main objective is to estimate SKEL parameters directly from an image, overcoming the lack of paired image-SKEL training data. HSMR utilizes a transformer network trained with iteratively refined pseudo-ground truth SKEL parameters generated by converting existing SMPL datasets and optimizing against 2D keypoints (“SKELify”). HSMR achieves competitive performance on standard benchmarks compared to SMPL-based methods like HMR2.0, while significantly outperforming them (by >10mm PA-MPJPE) on datasets with extreme poses like MOYO and reducing unnatural joint rotations. For AI practitioners, this offers a way to generate more physically plausible 3D human models directly from images, which is crucial for biomechanics, robotics, and simulation applications where joint limits and skeletal accuracy are paramount. |
Papers for 2025-03-28
| Title |
Authors |
Summary |
| Video-R1: Reinforcing Video Reasoning in MLLMs (Read more on arXiv or HuggingFace) |
Potentialts, guozonghao96, BreakLee, kxgong, KaituoFeng |
Video-R1 introduces a rule-based reinforcement learning framework to enhance video reasoning capabilities within Multimodal Large Language Models (MLLMs). The primary objective is to adapt the R1 reasoning paradigm for video by addressing the lack of explicit temporal modeling in standard RL algorithms and the scarcity of high-quality video reasoning data. The methodology involves proposing the Temporal Group Relative Policy Optimization (T-GRPO) algorithm, which contrasts performance on ordered versus shuffled video frames, and utilizing curated hybrid datasets (Video-R1-COT-165k, Video-R1-260k) combining image and video reasoning samples. Key results show significant improvements across video benchmarks, notably achieving 35.8% accuracy on VSI-Bench with the 7B model, surpassing the proprietary GPT-4o model. For AI practitioners, this research demonstrates that temporal-aware RL algorithms like T-GRPO, coupled with hybrid image-video data, offer an effective approach to improve complex temporal reasoning in MLLMs for video understanding applications. |
| UI-R1: Enhancing Action Prediction of GUI Agents by Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Xi Yin, hsli-cuhk, guoyaxuan0106, Yuxiang007, LZXzju |
This paper introduces UI-R1, leveraging rule-based reinforcement learning (RL) to enhance graphical user interface (GUI) action prediction for multimodal large language models (MLLMs). The main objective was to investigate if rule-based RL could improve MLLM reasoning capabilities for GUI action prediction using significantly less data than supervised fine-tuning (SFT). Key methodology involved curating a 136-sample mobile GUI task dataset, designing a unified rule-based reward function for action type and coordinate accuracy, and applying Group Relative Policy Optimization (GRPO) for reinforcement fine-tuning (RFT) on a Qwen2.5-VL-3B model. The primary result showed UI-R1-3B improved action type accuracy by 15% and grounding accuracy by 10.3% on the in-domain ANDROIDCONTROL benchmark compared to its base model, while using only 136 training samples, and achieved competitive out-of-domain performance against larger SFT models trained on 76K data. The principal implication for AI practitioners is that rule-based RFT presents a highly data-efficient method for improving GUI agent performance and generalization, offering a viable alternative to large-scale SFT, particularly in resource-constrained or OOD scenarios. |
| Challenging the Boundaries of Reasoning: An Olympiad-Level Math |
|
|
| Benchmark for Large Language Models (Read more on arXiv or HuggingFace) |
Wayne Xin Zhao, jrwen, TimothyCzp, EliverQ, CoderBak |
This paper introduces OlymMATH, a new bilingual Olympiad-level mathematical benchmark designed to rigorously evaluate the complex reasoning capabilities of large language models (LLMs). The primary objective is to address the saturation of existing math reasoning benchmarks by providing a more challenging test set derived from manually verified, non-digital sources. The methodology involved curating 200 problems (split into AIME-level easy and harder Olympiad-level tiers) across four mathematical fields, providing parallel English and Chinese versions with verifiable numerical answers. Empirical results show state-of-the-art models like DeepSeek-R1 achieve low accuracy (21.2% Pass@1) on the OlymMATH-EN-HARD subset, indicating significant limitations in current LLM reasoning. For AI practitioners, OlymMATH serves as a demanding benchmark to better differentiate advanced reasoning models and identify weaknesses, such as reliance on heuristics over rigorous derivation, guiding the development of more robust mathematical problem-solving capabilities. |
| VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic |
|
|
| Faithfulness (Read more on arXiv or HuggingFace) |
mimihe, yinanhe, jackyhate, HongboLiu, Ziqi |
VBench-2.0 introduces an automated benchmark suite designed to evaluate the intrinsic faithfulness of video generation models, moving beyond superficial quality assessments. Its primary objective is to systematically measure adherence to principles like physics, commonsense reasoning, human fidelity, controllability, and creativity across 18 fine-grained dimensions. The methodology integrates Vision-Language Models (VLMs) and Large Language Models (LLMs) through text description alignment and video-based multi-question answering, alongside specialist detectors and heuristics, validated via human preference annotations. Evaluations reveal current state-of-the-art models struggle significantly with complex plot generation (~10-12% scores) and dynamic attribute control (~8-24% scores), although VBench-2.0’s automated metrics show strong alignment with human judgment (Spearman’s ρ > 0.8 across most dimensions). For AI practitioners, VBench-2.0 provides a standardized framework to assess and guide the development of video generation models towards greater realism and adherence to world principles, crucial for applications requiring simulation and complex scene understanding. |
| LeX-Art: Rethinking Text Generation via Scalable High-Quality Data |
|
|
| Synthesis (Read more on arXiv or HuggingFace) |
Dakerqi, afdsafas, Xxxy13, QJerry, stzhao |
LeX-Art introduces a data-centric framework using scalable high-quality synthesis to improve visual text rendering in text-to-image (T2I) generation. The main objective is to bridge the gap between prompt expressiveness and text rendering fidelity by enhancing data quality and fine-tuning models, rather than relying solely on control-based architectural changes. The methodology involves using DeepSeek-R1 for prompt enrichment, generating the LeX-10K dataset (10K 1024x1024 images) via multi-stage filtering, developing the LeX-Enhancer prompt model, fine-tuning LeX-FLUX and LeX-Lumina T2I models, and introducing the LeX-Bench benchmark and PNED metric for evaluation. Primary results demonstrate significant improvements, with LeX-Lumina achieving a 79.81% PNED gain (indicating better text accuracy) on CreateBench compared to its baseline. For AI practitioners, the principal implication is that this scalable, data-centric approach, leveraging high-quality synthetic data and prompt enhancement, offers an effective method to substantially improve text rendering fidelity and aesthetics in T2I models without requiring complex model architecture modifications. |
| Large Language Model Agent: A Survey on Methodology, Applications and |
|
|
| Challenges (Read more on arXiv or HuggingFace) |
qqlong, joeyleo, evan-gyy, yszhao, luojunyu |
This survey systematically reviews Large Language Model (LLM) agents, covering their methodologies, applications, and challenges. The primary objective is to deconstruct LLM agent systems through a methodology-centered taxonomy, linking architectural foundations (construction), interaction mechanisms (collaboration), and improvement pathways (evolution). It employs a tripartite framework analyzing agent construction (profile, memory, planning, action execution), collaboration paradigms (centralized, decentralized, hybrid), and evolution mechanisms (autonomous learning, co-evolution, external resources), complemented by analysis of evaluation, tools, real-world issues, and applications. The survey provides a unified architectural perspective, identifies significant challenges including scalability, memory constraints, reliability, and evaluation complexity, and offers a structured understanding distinct from prior works focusing on isolated aspects. For AI practitioners, this work delivers a comprehensive taxonomy and framework for understanding the design principles, lifecycle, and practical considerations crucial for developing and deploying robust LLM agent systems. |
| Lumina-Image 2.0: A Unified and Efficient Image Generative Framework (Read more on arXiv or HuggingFace) |
luyiting, Paper99, RuoyiDu, JackyZhuo, Dakerqi |
Lumina-Image 2.0 introduces a unified and efficient text-to-image generation framework improving upon Lumina-Next. The main objective is to enhance image fidelity, prompt adherence, and generation efficiency through architectural unification and improved training data. Key methodologies include the Unified Next-DiT architecture for joint text-image token processing, the Unified Captioner (UniCap) for generating high-quality, multi-granularity captions, multi-stage progressive training, and inference optimizations like CFG-Renormalization and CFG-Truncation. Lumina-Image 2.0 achieves strong performance, scoring 87.20 on the DPG benchmark with only 2.6B parameters, demonstrating superior efficiency and scalability compared to prior models. For AI practitioners, this work presents an efficient (2.6B parameters) and unified transformer architecture applicable beyond T2I, alongside a specialized captioning system (UniCap) that significantly improves training data quality and model convergence, offering a practical approach to building performant generative models. |
| ReaRAG: Knowledge-guided Reasoning Enhances Factuality of Large |
|
|
| Reasoning Models with Iterative Retrieval Augmented Generation (Read more on arXiv or HuggingFace) |
chenyn66, liuweichuan, NeoZ123, caoshulin, ZhiCheng0326 |
ReaRAG enhances Large Reasoning Model (LRM) factuality for multi-hop QA using iterative, knowledge-guided Retrieval-Augmented Generation (RAG) with reflective reasoning. The objective is to improve LRM factual accuracy on complex QA tasks by mitigating reliance on parametric knowledge and issues like overthinking and error propagation found in prior iterative RAG and RL-based approaches. The methodology involves constructing a dataset with bounded reasoning chains, fine-tuning ReaRAG-9B (based on GLM-4-9B) using a Thought-Action-Observation paradigm, iteratively querying a RAG engine, and employing reflection to refine the reasoning trajectory. ReaRAG-9B significantly outperforms baselines on multi-hop QA benchmarks, achieving a 14.5% ACCL improvement over SearChain on MuSiQue (66.00 vs 51.50 ACCL). For AI practitioners, ReaRAG provides a fine-tuning framework and inference strategy to build more factually reliable QA systems by effectively integrating iterative external knowledge retrieval and explicit reasoning steps, reducing errors compared to solely prompt-based or single-retrieval RAG methods. |
| Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for |
|
|
| Embodied Interactive Tasks (Read more on arXiv or HuggingFace) |
Guiyang1001, tricktreat, yijiang, Gangao, zwq2018 |
This paper presents Embodied-Reasoner, a model extending ol-style reasoning to interactive embodied search tasks by generating and learning from coherent Observation-Thought-Action trajectories. The primary objective is to enhance reasoning capabilities for embodied agents facing challenges like continuous multimodal interaction, spatial understanding, temporal reasoning, and self-reflection based on interaction history. Key methodology involves synthesizing 9.3k trajectories featuring diverse thinking processes (e.g., analysis, spatial reasoning, reflection) and employing a three-stage training pipeline comprising imitation learning, self-exploration via rejection sampling, and self-correction via reflection tuning. Results demonstrate significant improvements over advanced visual reasoning models, with Embodied-Reasoner exceeding OpenAI GPT-o1 by +9% and GPT-o3-mini by +24% in success rate, showing fewer repeated searches and better consistency on long-horizon tasks. For AI practitioners, this work provides a data synthesis and training framework to develop embodied agents with enhanced planning, reasoning, and interaction capabilities, particularly for complex tasks requiring adaptive behavior based on visual feedback and interaction history. |
| ResearchBench: Benchmarking LLMs in Scientific Discovery via |
|
|
| Inspiration-Based Task Decomposition (Read more on arXiv or HuggingFace) |
yuqiangli, bgao22182, jinjieni, ZonglinY, yujieliu |
ResearchBench introduces a benchmark for evaluating Large Language Models (LLMs) in scientific discovery by decomposing the process into inspiration retrieval, hypothesis composition, and ranking. The objective is to assess LLM performance on these fundamental sub-tasks using recent, contamination-resistant scientific literature across 12 disciplines. An automated LLM-based agentic framework extracts research components (questions, background, inspirations, hypotheses) from 1386 papers published in 2024, forming the basis for evaluation, including carefully selected negative examples for retrieval tasks. Results show LLMs excel at the out-of-distribution inspiration retrieval task (GPT-4o hit ratio: 45.65% for top 4% candidates), while hypothesis composition and ranking show moderate capabilities with potential for improvement; ranking is notably affected by position bias. For AI practitioners, this indicates LLMs can serve as “research hypothesis mines” capable of surfacing novel knowledge associations for automated discovery, though the bottleneck in retrieval suggests a reliance on pretraining depth over post-training refinement. |
| Optimal Stepsize for Diffusion Sampling (Read more on arXiv or HuggingFace) |
Han Hu, Jianning Pei, cientgu |
This paper introduces Optimal Stepsize Distillation (OSS), a dynamic programming framework to derive theoretically optimal stepsize schedules for accelerating diffusion model sampling. The objective is to overcome suboptimal discretization in diffusion sampling by focusing on principled stepsize schedule design, rather than solely optimizing update directions. OSS treats stepsize optimization as knowledge distillation, using dynamic programming to recursively minimize the global discretization error between a few-step student sampler and a many-step teacher reference trajectory. Experiments demonstrate that OSS enables significant acceleration, achieving 10x speedup for text-to-image generation while maintaining 99.4% of the teacher model’s performance on the GenEval benchmark. For AI practitioners, OSS provides a robust, architecture-agnostic method to drastically reduce diffusion model inference latency with minimal performance loss, enabling more efficient deployment. |
| Exploring the Evolution of Physics Cognition in Video Generation: A |
|
|
| Survey (Read more on arXiv or HuggingFace) |
huangsiteng, wangcunxiang, huangsiteng, yishanwang, minnielin |
This survey reviews the integration of physical cognition into video generation models, organizing advancements along an evolutionary path inspired by human cognitive development. The main objective is to systematically categorize methods for improving physical fidelity in generated videos, addressing the gap between visual realism and physical plausibility. The paper proposes a three-tier taxonomy (Basic Schematic Perception, Passive Cognition, Active Cognition) to classify techniques like motion-guided generation, physics-inspired regularization, simulation integration, and LLM-based reasoning. Despite progress, the survey highlights that even state-of-the-art models often violate fundamental physical laws, generating visually appealing but physically inconsistent results, as evidenced by evaluations on benchmarks like PhyGenBench [86] and Physics-IQ [84]. For AI practitioners, this implies that achieving physically plausible video generation, essential for applications like robotics and simulation, requires moving beyond visual mimicry towards integrating explicit physical knowledge and interaction mechanisms. |
| ChatAnyone: Stylized Real-time Portrait Video Generation with |
|
|
| Hierarchical Motion Diffusion Model (Read more on arXiv or HuggingFace) |
Peng Zhang, Chaonan Ji, Jinwei Qi, Liefeng, shengxu97 |
ChatAnyone introduces a novel framework for generating stylized, real-time upper-body portrait videos from audio using a hierarchical motion diffusion model and hybrid control fusion GAN. The primary objective is to create expressive digital humans with synchronized facial expressions, head poses, and upper-body movements including hands, enabling fine-grained style control. The methodology involves a two-stage process: first, hierarchical motion diffusion models predict explicit and implicit motion representations from audio and optional style references; second, a warping-based GAN synthesizes the video using these representations, injected hand controls, and a face refinement module. Key results demonstrate real-time performance (up to 30fps at 512x768 on a 4090 GPU) and improved quantitative metrics, such as achieving a PSNR of 24.88 in self-reenactment, significantly outperforming prior GAN-based methods. For AI practitioners, this provides an effective approach for developing highly expressive, controllable, and real-time digital avatars for interactive applications like video chat and virtual assistants, demonstrating the power of combining diffusion models for motion generation with GANs for efficient synthesis. |
| FinAudio: A Benchmark for Audio Large Language Models in Financial |
|
|
| Applications (Read more on arXiv or HuggingFace) |
Yueru1, Shashidhar, ShirleyY, Acatsama, YupengCao |
FinAudio introduces the first benchmark specifically designed to assess Audio Large Language Models (AudioLLMs) within the financial domain. The primary objective is to evaluate the capacity of current AudioLLMs on realistic financial audio tasks, revealing their strengths and limitations. The methodology involves defining three tasks (short-clip ASR, long-recording ASR, and summarization), curating five datasets (MDRM, SPGISpeech, Earnings-21, Earnings-22, FinAudioSum) totaling over 400 hours, and evaluating seven diverse AudioLLMs. Key results show significant performance variation, with Whisper-v3 achieving the lowest Word Error Rate (WER) on short-clip ASR (2-3%), but performance degrading across models for long audio ASR (Whisper-v3: 12-16% WER) and summarization being dependent on initial ASR quality. For AI practitioners, this benchmark reveals that while open-source models like Whisper-v3 provide a strong baseline, current AudioLLMs struggle with long financial recordings and specialized terminology/numerical data, highlighting the need for improved context handling and domain-specific adaptation. |
| Synthetic Video Enhances Physical Fidelity in Video Synthesis (Read more on arXiv or HuggingFace) |
Ziyan Yang, Ziyu Wang, Qi Zhao, fengcheng1, Univstar |
This research demonstrates that integrating synthetic videos from CGI pipelines improves the physical fidelity of generative video synthesis models. The objective was to investigate whether synthetic videos, generated with physical consistency using computer graphics, can enhance the physical realism (e.g., 3D consistency, human pose integrity) of diffusion-based video generation models. The methodology involved generating synthetic videos using Blender/Unreal Engine, curating this data based on factors like asset/rendering quality and camera setups, employing a specific captioning strategy, and introducing a training technique called SimDrop to integrate synthetic data while mitigating visual artifacts using a reference model and classifier-free guidance. Primary results show significant improvement in physical fidelity across tasks like large human motion, camera rotation, and layer decomposition; for instance, on the camera spin shot task, the synthetically-enhanced model achieved an 80% success rate in user studies compared to 20% for the baseline and reduced the 3D reconstruction re-projection error (ê_proj) from 0.437 to 0.135. The principal implication for AI practitioners is that leveraging carefully curated synthetic video data, combined with techniques like SimDrop, offers a data-centric approach to enhance the physical consistency and reduce artifacts in video generation models without requiring modifications to the core model architecture. |
| ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging (Read more on arXiv or HuggingFace) |
Ziyan Jiang, Yi Zhong, Yanqiu Zhao, Saberlve, HaomingXu |
ZJUKLAB employed TIES-Merging of two specialized models to address selective unlearning in Large Language Models for SemEval-2025 Task 4. The objective was to effectively erase sensitive content by balancing the trade-off between over-forgetting general knowledge and under-forgetting targeted data. Their methodology involved training two distinct LoRA models using Negative Preference Optimization (NPO), Gradient Descent on Retain set (GDR), and KL divergence minimization (KLR) to induce complementary biases, then merging them using TIES-Merging. The merged system ranked second online (Task Aggregate 0.944) and locally achieved an Aggregate Score of 0.806 and a near-optimal MIA AUC of 0.501, significantly outperforming the individual biased models. For AI practitioners, this demonstrates model merging as a practical technique to combine models with opposing unlearning biases for more effective and balanced sensitive data removal, though limitations in current evaluation metrics are noted. |
| Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile |
|
|
| Gaussian Feature Fields (Read more on arXiv or HuggingFace) |
Hui Ren, Fanzhiwen, ir1d, ShuwangZhang00, shijiezhou |
Feature4X provides a universal framework to lift arbitrary 2D vision foundation model functionalities into interactive 4D agentic AI systems using only monocular video input. Its main objective is to enable versatile 4D scene understanding and interaction (segmentation, editing, VQA) from readily available monocular videos, overcoming the limitations of 4D data scarcity. The key methodology involves distilling diverse 2D features into a compact, unified dynamic 4D Gaussian feature field represented using Gaussian Splatting and Motion Scaffolds, trained end-to-end and integrated with LLMs. Primary results include robust novel-view segmentation, language-guided 4D scene editing, and spatiotemporal VQA, with semantic segmentation achieving comparable accuracy to baselines while being approximately 6.2x more space-efficient (95.4MB vs 593.9MB). For AI practitioners, this offers a scalable method to extend existing 2D vision model capabilities to dynamic 4D environments, facilitating the development of interactive 4D agentic AI applications without requiring extensive annotated 4D datasets. |
| Unified Multimodal Discrete Diffusion (Read more on arXiv or HuggingFace) |
Katerina Fragkiadaki, Deepak765, Sid1275, mihirpd, aswerdlow |
This paper introduces UniDisc, a unified multimodal discrete diffusion model for joint text and image generation. The objective is to explore discrete diffusion models as an alternative unified generative formulation for joint text and image domains, comparing their advantages over autoregressive (AR) models. UniDisc employs a transformer architecture trained using a discrete diffusion process involving masking tokens (text and image) with an absorbing state and learning to denoise via a weighted cross-entropy objective. Results show UniDisc outperforms AR models in conditional generation using classifier-free guidance (CFG), enables zero-shot joint text-image inpainting, and demonstrates superior joint retrieval accuracy (e.g., 0.64 vs 0.17 on DataComp1B). For AI practitioners, UniDisc offers enhanced controllability, editability, and a flexible inference time vs. quality trade-off for multimodal generation tasks compared to traditional AR approaches, although scaling analysis indicates it requires approximately 13.2x more training compute for equivalent loss levels. |
| LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized |
|
|
| Text-Guided Image Editing (Read more on arXiv or HuggingFace) |
Sirisha Rambhatla, Meet Soni, Achint Soni |
LOCATEdit introduces graph Laplacian optimization on cross- and self-attention maps (CASA graphs) for precise, localized text-guided image editing. The primary objective is to improve spatial consistency and confine edits to target regions, mitigating artifacts and distortions common in methods relying solely on cross-attention maps from diffusion models. Key methodology involves constructing CASA graphs from attention maps, applying graph Laplacian regularization to enforce smoothness and optimize attention values, integrating IP-Adapter guidance, and using selective pruning on text embedding differences. LOCATEdit significantly outperforms baselines on PIE-Bench, achieving, for example, a background preservation SSIM of 86.52 (x10^2) with DPM-Solver++(20), demonstrating superior localization and fidelity. For AI practitioners, this work provides a robust, training-free technique using graph-based optimization on attention mechanisms to achieve more controlled and spatially consistent results in text-guided generative image editing tasks. |
| LLPut: Investigating Large Language Models for Bug Report-Based Input |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Tarannum Shaila Zaman, imranraad, Subarna10, alifalhasan |
This paper investigates the effectiveness of generative Large Language Models (LLMs) in extracting failure-inducing input commands from natural language bug reports. The primary research objective is to empirically evaluate how effectively three open-source generative LLMs (LLaMA, Qwen, Qwen-Coder) can extract these inputs compared to a fine-tuned BERT model. Using a dataset of 206 annotated Linux coreutils bug reports and a one-shot prompting strategy, the study evaluates extraction accuracy against human annotations using BLEU scores. The generative LLMs significantly outperformed the BERT baseline, with Qwen yielding the best results, achieving a BLEU-2 score of ≥ 0.5 for 62.62% of its extracted commands. For AI practitioners, this indicates that generative LLMs offer considerable potential for automating the extraction of executable commands from bug reports, aiding debugging workflows, though challenges in handling command variations and extraction failures persist. |
Papers for 2025-03-27
| Title |
Authors |
Summary |
| Dita: Scaling Diffusion Transformer for Generalist |
|
|
| Vision-Language-Action Policy (Read more on arXiv or HuggingFace) |
TTTTTony, MIASANMIA, robot-haonan, TianyiZhang0213, zhihou |
Dita introduces a scalable Diffusion Transformer architecture for generalist vision-language-action robot policies. The primary objective is to develop a versatile, open-source VLA model capable of zero-shot or few-shot generalization across diverse robotic embodiments, tasks, and environments, particularly addressing long-horizon tasks and environmental variations. The key methodology involves using a causal Transformer to directly denoise continuous action sequences via a diffusion process, conditioned in-context on raw visual tokens (from DINOv2 and Q-Former) and language instructions (from CLIP). Dita achieves state-of-the-art or competitive performance on simulation benchmarks, notably attaining an 82.4% average success rate on LIBERO (a ~6% improvement over prior methods), and demonstrates robust real-world adaptation with 10-shot finetuning on complex, long-horizon tasks under varying conditions. For AI practitioners, Dita provides a lightweight (334M parameters) and effective open-source framework that integrates Transformer scalability with inherent diffusion denoising via in-context conditioning, offering a strong baseline for developing adaptable robot policies requiring minimal task-specific data. |
| Qwen2.5-Omni Technical Report (Read more on arXiv or HuggingFace) |
JialinWang, chenkq, bluelike, jinzheng-he, ZhifangGuo |
Qwen2.5-Omni is an end-to-end multimodal model processing text, image, audio, and video to generate streaming text and speech responses. The primary objective is to develop a unified model capable of perceiving diverse streaming inputs, synchronizing temporal modalities like audio and video, and concurrently generating both text and low-latency speech outputs. Key methodologies include block-wise processing for input encoders, Time-aligned Multimodal RoPE (TMROPE) for audio-video synchronization, and a Thinker-Talker architecture separating text generation (Thinker LLM) from streaming speech token generation (Talker), using a sliding-window DiT for audio decoding. Primary results demonstrate state-of-the-art performance on benchmarks like OmniBench (56.13% average score), comparable end-to-end speech instruction following capabilities to text input on tasks like GSM8K (88.7% speech accuracy vs 91.6% text accuracy for Qwen2.5-7B), and robust streaming speech generation with 6.54% WER on the seed-tts-eval test-hard set after reinforcement learning. For AI practitioners, this work offers the Thinker-Talker architecture and TMROPE as a framework for building unified streaming multimodal systems that handle synchronized inputs and generate real-time text and speech, enabling more natural human-AI interaction. |
| LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning? (Read more on arXiv or HuggingFace) |
Leoxing, KennyUTC, zengyh1900, favourisnotyou, KexianTang |
This paper introduces LEGO-Puzzles, a benchmark designed to evaluate multi-step spatial reasoning in Multimodal Large Language Models (MLLMs). The objective is to assess MLLMs’ capabilities in both spatial understanding and sequential reasoning through diverse LEGO construction-based tasks. The methodology involves a curated dataset of over 1,100 visual question-answering (VQA) pairs across 11 tasks, alongside image generation evaluations, tested on 20 state-of-the-art MLLMs. Results reveal significant limitations; even the best MLLM (GPT-40) achieved only 57.7% overall accuracy, far below human performance (93.6%), with particular weaknesses in multi-step sequential reasoning and spatially grounded image generation. For AI practitioners, this highlights critical deficiencies in current MLLMs’ spatial intelligence, underscoring the need for advancements in models intended for complex real-world applications like robotics and automated assembly that demand robust sequential spatial reasoning. |
| Wan: Open and Advanced Large-Scale Video Generative Models (Read more on arXiv or HuggingFace) |
HermanZ, chenweix7, chaojiemao, baoleai, ang-annng |
This paper introduces Wan, an open-source suite of advanced large-scale video generative models based on the Diffusion Transformer paradigm. The objective is to push video generation boundaries by developing high-performance, efficient, and comprehensive open-source models (1.3B and 14B parameters) trained on billions of images/videos. Key methodologies include a novel spatio-temporal VAE, scalable pre-training with Flow Matching, large-scale data curation, and extensions to tasks like I2V, editing, personalization, and real-time generation. The 14B model achieved a leading Wan-Bench score of 0.724, outperforming competitors, while the 1.3B model demonstrated consumer-grade efficiency requiring only 8.19 GB VRAM for 480p inference. For AI practitioners, Wan provides open-source access to powerful (14B) and efficient (1.3B) foundation models, code, and training details, enabling the development of diverse video generation applications, including potential deployment on consumer GPUs with the 1.3B model. |
| Unconditional Priors Matter! Improving Conditional Generation of |
|
|
| Fine-Tuned Diffusion Models (Read more on arXiv or HuggingFace) |
Jaihoon Kim, Minhyuk, phillipinseoul, prinphunya |
This paper introduces a training-free method to enhance conditional generation from fine-tuned diffusion models by utilizing stronger unconditional priors from base models. The primary objective is to address the degradation in conditional generation quality caused by poor unconditional noise predictions learned during Classifier-Free Guidance (CFG) based fine-tuning. The key methodology involves replacing the unconditional noise prediction term in the CFG sampling process of the fine-tuned model with the corresponding prediction from its original base model or another pretrained model with robust unconditional generation capabilities. Results demonstrate significant improvements; for example, applying this method to Zero-1-to-3 novel view synthesis using SD2.1 as the unconditional prior improved LPIPS from 0.182 to 0.158 and PSNR from 16.647 to 17.801. For AI practitioners, this implies that during inference with CFG-based fine-tuned diffusion models, leveraging the unconditional prior from a separate, well-trained unconditional model can substantially boost conditional output quality without requiring model retraining or architectural changes. |
| Open Deep Search: Democratizing Search with Open-source Reasoning Agents (Read more on arXiv or HuggingFace) |
speedyarda, ljirwin, pchiniya, cabxyz, salzubi401 |
Open Deep Search (ODS) is introduced as an open-source framework augmenting LLMs with reasoning agents and web search tools to rival proprietary search AI. The primary objective is to bridge the performance gap between open-source and closed-source search AI solutions by enhancing LLM reasoning with real-time web information. ODS employs two main components: an Open Search Tool for improved web context retrieval and an Open Reasoning Agent (using ReAct or CodeAct) to orchestrate tool use, including the search tool, calculator, and code interpreter, based on user queries. Key results show ODS-v2 paired with DeepSeek-R1 achieves 75.3% accuracy on the FRAMES benchmark, outperforming GPT-4o Search Preview by 9.7%, and 88.3% on SimpleQA. For AI practitioners, ODS offers a modular, open-source system to integrate advanced search and reasoning into any base LLM, enabling state-of-the-art performance on fact-based question answering without dependence on closed systems. |
| GenHancer: Imperfect Generative Models are Secretly Strong |
|
|
| Vision-Centric Enhancers (Read more on arXiv or HuggingFace) |
yshan2u, yxgeee, aether25, tttoaster, msj9817 |
GenHancer enhances CLIP’s fine-grained visual representations using lightweight generative models without requiring perfect reconstruction or pre-trained denoisers. The objective is to explore how imperfect generative models can effectively transfer fine-grained visual knowledge to discriminative models like CLIP, investigating optimal conditioning, denoising configurations, and generation paradigms. The key methodology involves a two-stage post-training approach using lightweight, randomly initialized continuous or discrete denoisers conditioned solely on CLIP’s global ([CLS]) token for self-supervised reconstruction, employing techniques like LoRA and scaled Logit-Normal timestamp sampling. GenHancer consistently outperforms prior methods, achieving a 6.0% improvement over the baseline OpenAICLIP on the MMVP-VLM benchmark, demonstrating that perfect generation is not necessary for representation enhancement. For AI practitioners, this implies that fine-grained visual capabilities of CLIP-based systems (like MLLMs) can be significantly and efficiently improved post-hoc using lightweight generative models focused on specific conditioning (global token only) and training strategies, avoiding computationally expensive heavy denoisers. |
| BizGen: Advancing Article-level Visual Text Rendering for Infographics |
|
|
| Generation (Read more on arXiv or HuggingFace) |
YuanYuhui, kevinlin311tw, bohanChen, Marseclipse, wukeming11 |
BizGen introduces a framework for generating high-quality infographics and slides with accurate article-level visual text rendering and adherence to ultra-dense layouts. The primary objective is to overcome the challenges of significantly longer text contexts and the scarcity of high-quality business content data compared to standard text-to-image tasks. Key methodologies include the creation of a large-scale dataset (INFOGRAPHICS-650K) via retrieval-augmented generation and a novel layout-guided cross-attention mechanism with layout-conditional Classifier-Free Guidance (CFG) for region-wise control. BizGen significantly outperforms models like FLUX and SD3 on the BizEval benchmark, achieving over 25% absolute improvement in visual text spelling accuracy (OCR) on infographics with more than 20 layers compared to FLUX. For AI practitioners, BizGen offers a scalable data generation strategy and a controllable diffusion model architecture to produce complex, text-rich business graphics demanding high fidelity to dense layouts and long-form textual content. |
| Gemini Robotics: Bringing AI into the Physical World (Read more on arXiv or HuggingFace) |
abalakrishna123, TravisAStrong, montse90, jalayrac, saminda |
This paper introduces Gemini Robotics, a family of AI models based on Gemini 2.0 designed to bridge AI capabilities into the physical world via robotics. The main objective is to endow large multimodal models with robust embodied reasoning and dexterous physical interaction capabilities for general-purpose robot control. Key methodologies include enhancing Gemini 2.0’s embodied reasoning (Gemini Robotics-ER), evaluated on a new ERQA benchmark, and fine-tuning a Vision-Language-Action (VLA) model (Gemini Robotics) on extensive robot action data for direct, low-latency control. The generalist Gemini Robotics VLA achieved high proficiency out-of-the-box, succeeding on 50% of 20 diverse dexterous manipulation tasks with over 80% success rate, and demonstrated strong generalization and rapid adaptation to new tasks and embodiments. For AI practitioners, this work shows that large multimodal foundation models, when specifically trained for embodied reasoning and grounded with robot interaction data, provide a viable foundation for developing more general-purpose, dexterous, and adaptable robotic agents. |
| MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree |
|
|
| Search (Read more on arXiv or HuggingFace) |
armanc, chenzhao, yilunzhao, AlexCCtop |
MCTS-RAG integrates Monte Carlo Tree Search (MCTS) with Retrieval-Augmented Generation (RAG) to improve reasoning capabilities of small language models (SLMs) on knowledge-intensive tasks. The research aims to overcome SLM limitations in accessing and utilizing external knowledge by dynamically combining structured reasoning search with adaptive retrieval. The methodology employs MCTS to explore reasoning paths, introducing specific RAG actions (Retrieval Reasoning, Retrieval Decompose) at decision points, guided by UCT, and evaluates paths using retrieved information. Key results show MCTS-RAG enabled Llama 3.1-8B to achieve over 20% absolute accuracy improvement on ComplexWebQA and roughly 15% on GPQA compared to baseline methods. For AI practitioners, this work presents an effective inference-time compute scaling method to significantly enhance the performance of smaller LMs on complex, knowledge-reliant tasks without model retraining, offering a pathway to achieve higher accuracy with more resource-efficient models. |
| AccVideo: Accelerating Video Diffusion Model with Synthetic Dataset (Read more on arXiv or HuggingFace) |
Yunhong Wang, XihuiLiu, YaohuiW, AriaChen, aejion |
AccVideo accelerates video diffusion models through distillation using a synthetic dataset of denoising trajectories. The research objective is to reduce the extensive inference steps required by video diffusion models while maintaining output quality by avoiding distillation on irrelevant data points. Key methodology involves generating a synthetic dataset (SynVid) with full denoising trajectories from a pretrained teacher model, training a student model using trajectory-based few-step guidance on keyframes from these trajectories, and employing an adversarial training strategy with timestep-aware discriminators. The primary result is an 8.5x reduction in inference time compared to the teacher model (HunyuanVideo), generating 720x1280 videos in 380s vs 3234s with comparable quality. For AI practitioners, this demonstrates an effective technique to significantly speed up high-resolution video generation from diffusion models, making them more feasible for real-world deployment by leveraging synthetic data distillation. |
| ViLBench: A Suite for Vision-Language Process Reward Modeling (Read more on arXiv or HuggingFace) |
cihangxie, xianft, alihiker, Helicopt, PahaII |
This paper introduces VILBENCH, a benchmark suite for vision-language process reward modeling, alongside a new dataset (ViLReward-73K) and a trained process reward model (ViLPRM). The main objective is to evaluate the effectiveness of vision-language large models (VLLMs) as process reward models (PRMs) and output reward models (ORMs), and to develop improved PRMs for tasks requiring step-wise reasoning. Key methodologies include benchmarking seven VLLMs on five VL datasets, filtering data to create VILBENCH emphasizing step-wise rewards, collecting preference data using an enhanced MCTS algorithm, and training a 3B parameter ViLPRM based on QwenVL-2.5. Primary results show neither ORM nor PRM consistently outperforms the other across tasks using general VLLMs, while the trained ViLPRM achieves an average improvement of 3.3% over standard Chain-of-Thought evaluation on VILBENCH. For AI practitioners, this indicates that specialized PRMs trained on process supervision data, like ViLPRM, can better evaluate complex vision-language reasoning steps than general VLLMs or ORMs, highlighting a pathway to improve model alignment and evaluation for multi-step multimodal tasks. |
| LogQuant: Log-Distributed 2-Bit Quantization of KV Cache with Superior |
|
|
| Accuracy Preservation (Read more on arXiv or HuggingFace) |
Pingyi Luo, Bingsheng He, deciding, Zicong99, Concyclics |
LogQuant introduces a log-distributed 2-bit quantization method for LLM KV Caches, improving accuracy preservation over existing techniques. The objective is to reduce KV Cache memory usage via 2-bit quantization while mitigating the associated accuracy loss by selectively preserving important tokens based on a log-distributed attention pattern. The methodology involves applying a base-2 logarithmic filtering strategy to retain tokens with decreasing density further from the current position, quantizing less critical tokens to 2-bits while keeping a dynamic window of recent tokens (2W to 3W) at full precision. LogQuant demonstrated superior performance, improving accuracy by 40%-200% on Math and Code tasks compared to KiVi at similar compression ratios, and boosting throughput by 25% over a BF16 baseline. For AI practitioners, LogQuant offers a way to deploy LLMs with long contexts more efficiently on memory-constrained hardware by significantly reducing KV Cache size with better accuracy retention than prior 2-bit quantization approaches. |
| ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving |
|
|
| Systems (Read more on arXiv or HuggingFace) |
xzwnlp, bozhong, xiangchen-dvi, JizhanFang, Chenxiwang |
This paper introduces ADS-Edit, a multimodal benchmark dataset for evaluating knowledge editing techniques applied to Large Multimodal Models (LMMs) in Autonomous Driving Systems (ADS). The research objective is to assess how effectively knowledge editing can update LMMs with domain-specific ADS knowledge (addressing traffic knowledge gaps, complex conditions, dynamic states) without requiring full retraining. The methodology involves constructing the ADS-Edit benchmark from existing ADS datasets (LingoQA, DriveLM, CODA-LM) with video, multi-view, and single-image data across perception, understanding, and decision-making scenarios, and evaluating four editing baselines (Prompt, AdaLora, GRACE, WISE) on reliability, generality, and locality. Primary results demonstrate that memory-based methods achieve high reliability (e.g., GRACE reached 100% reliability on single edits), but differ significantly in generality (GRACE <30%, WISE ~85-95%), with WISE showing strong locality (~100%). For AI practitioners, ADS-Edit provides a framework to evaluate and select knowledge editing methods for efficiently updating LMMs in ADS, indicating WISE offers a balanced trade-off for update reliability, generalization, and parameter preservation. |
| Beyond Words: Advancing Long-Text Image Generation via Multimodal |
|
|
| Autoregressive Models (Read more on arXiv or HuggingFace) |
Min Li, Lijuan, zyang39, linjieli222, Awiny |
This paper presents LongTextAR, a multimodal autoregressive model enabling high-fidelity long-text image generation. It addresses the challenge of accurately rendering extensive textual content in images, a limitation of current generative models. The methodology identifies Vector Quantization (VQ) tokenization bottlenecks and introduces TextBinarizer, a novel text-focused binary tokenizer, integrated into a Llama2-based autoregressive architecture trained on text-rich data. LongTextAR significantly outperforms models like SD3.5 Large, achieving 69.5% OCR accuracy on long texts (>10 words) versus 52.3% for SD3.5 Large, and offers controllable text rendering (font, size, color, alignment). For AI practitioners, this work demonstrates that specialized tokenization within an autoregressive framework provides a strong alternative to diffusion models for generating images requiring accurate, controllable long text, impacting applications like automated document and presentation creation. |
| Attention IoU: Examining Biases in CelebA using Attention Maps (Read more on arXiv or HuggingFace) |
Vikram V. Ramaswamy, Olga Russakovsky, tyleryzhu, serianni |
This paper introduces Attention-IoU, a metric using attention maps to quantify biases within computer vision classification models by analyzing internal representations. The objective is to identify spurious correlations and understand how specific image features contribute to biased predictions, moving beyond performance disparities. The core methodology uses a generalized Intersection-over-Union (Attention-IoU) to compare GradCAM attention maps against ground-truth feature masks (mask score) or other attribute attention maps (heatmap score). Validation on Waterbirds shows the mask score accurately tracks induced bias (decreasing from 0.72±0.02 to 0.42±0.03 as bias increases from 50% to 100%), and analysis on CelebA reveals Attention-IoU uncovers correlations like that between Blond_Hair and Male (heatmap score 0.72±0.02) potentially linked to unlabeled confounders, unlike Wavy_Hair (0.65±0.03). For AI practitioners, Attention-IoU provides a tool to pinpoint spatial sources of bias within models, indicating that biases can stem from internal representations not solely reflected in dataset label correlations, thus informing more targeted debiasing interventions. |
| Self-Supervised Learning of Motion Concepts by Optimizing |
|
|
| Counterfactuals (Read more on arXiv or HuggingFace) |
Kevin Feigelis, Rahul Venkatesh, Seungwoo Kim, Stefan Stojanov, kmeisthax |
Opt-CWM introduces a self-supervised technique for optical flow and occlusion estimation by optimizing counterfactual probes on a pre-trained video prediction model without labeled data. The primary objective is to develop a method that extracts motion concepts from unlabeled videos by learning optimal input perturbations for a base Counterfactual World Model (CWM), avoiding fixed heuristics. Key methodology involves parameterizing perturbations with a learnable network trained jointly with a sparse flow-conditioned predictor using an asymmetric masking principle and RGB reconstruction loss. Results demonstrate state-of-the-art performance on real-world benchmarks compared to other self-supervised methods, achieving an Average Jaccard (AJ) of 47.53 and Average Distance (AD) of 8.73 on TAP-Vid First (DAVIS). For AI practitioners, this work provides a scalable, self-supervised approach to extract robust motion primitives from vast unlabeled video data, beneficial for applications requiring motion understanding without reliance on synthetic datasets or manual heuristics. |
| Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs (Read more on arXiv or HuggingFace) |
kw1jjang, Rock222, AndrewAhn, ya-mehdi, Anshumann |
This paper introduces Random Sampling Knowledge Distillation (RS-KD), an importance-sampling method for accelerating LLM pre-training distillation using sparse teacher logits. The research aims to develop an efficient offline knowledge distillation strategy for LLM pre-training that requires storing only a sparse subset of teacher logits without compromising student model performance or calibration. The key methodology involves using importance sampling (specifically, sampling proportional to teacher probabilities) to create unbiased sparse target distributions, theoretically and empirically contrasting this with biased Top-K sampling approaches. Primary results show that RS-KD achieves performance comparable to full distillation using only 12 unique sampled tokens, maintains near-perfect calibration (ECE ~0.8%), preserves expected gradients (4° angular difference vs. FullKD), and offers significant training throughput gains (1.7x-2.6x faster than FullKD). For AI practitioners, RS-KD offers a computationally efficient method to pre-train smaller LLMs via offline distillation, drastically reducing the storage required for teacher logits (using ~0.01%) and accelerating training with marginal overhead compared to standard cross-entropy training. |
| DINeMo: Learning Neural Mesh Models with no 3D Annotations (Read more on arXiv or HuggingFace) |
Alan Yuille, Weijie Guo, wufeim, guofeng1123 |
DINeMo presents a neural mesh model for category-level 3D pose estimation trained without 3D annotations. The main objective is to overcome the limitation of requiring extensive 3D annotations for training neural mesh models, enabling broader applicability and scalability. The key methodology involves leveraging pseudo-correspondence derived from large visual foundation models (SD-DINO) via a novel bidirectional generation process that integrates local features and global context, combined with Grounded-SAM for enhanced inference. DINeMo significantly outperforms previous zero- and few-shot methods on PASCAL3D+ car pose estimation (e.g., narrowing the gap with fully-supervised methods by 67.3% on Acc@pi/18, LO) and demonstrates effective scaling with additional unlabeled training data. For AI practitioners, this work offers a viable pathway to develop robust 3D object understanding models without relying on difficult-to-obtain 3D ground truth, utilizing unlabeled image data for training. |
| Image as an IMU: Estimating Camera Motion from a Single Motion-Blurred |
|
|
| Image (Read more on arXiv or HuggingFace) |
r0nn13, jerredchen |
This paper introduces a method to estimate instantaneous camera rotational (ω) and translational (v) velocity directly from motion blur within a single image. The objective is to leverage motion blur, often considered an artifact, as the primary source of information for robust ego-motion estimation during fast camera movements, eliminating the need for IMUs or multi-frame analysis. The approach first predicts dense motion flow and monocular depth using a neural network, then recovers velocity by solving a differentiable linear least squares system derived from motion field equations, enabling end-to-end training on synthetic and real data. Evaluated on real-world data, the method yields state-of-the-art velocity estimates (e.g., average rotational RMSE 1.22/0.91/1.76 rad/s), significantly outperforming MASt3R and COLMAP, and achieves real-time performance (30 FPS). AI practitioners can apply this technique for real-time, drift-free, IMU-like velocity measurements in high-motion scenarios (e.g., robotics, AR/VR) using only a single blurred camera image, enhancing robustness where traditional VO/SLAM methods fail. |
| PathoHR: Breast Cancer Survival Prediction on High-Resolution |
|
|
| Pathological Images (Read more on arXiv or HuggingFace) |
Rundong Xue, Jiaxuan Xiao, Jun Liu, Shiru Wang, Yang Luo |
PathoHR is a novel pipeline for breast cancer survival prediction using enhanced high-resolution pathological image features and optimized similarity learning. The main objective is to improve survival prediction accuracy by effectively extracting representative features from high-resolution WSIs while managing computational costs and addressing tumor heterogeneity. The methodology involves patch-wise feature extraction using a pre-trained encoder, integrating a plug-and-play high-resolution Vision Transformer (ViTAR) for feature enhancement, and systematically evaluating various similarity metrics (e.g., Cosine, Euclidean, Attention Score) for adaptive token merging. Results demonstrate that using enhanced 16x16 patches with the PathoHR pipeline (specifically with cosine similarity) achieves superior performance (AUC 0.90741) compared to baseline methods using larger raw 24x24 patches (AUC 0.8), validating the approach’s effectiveness and efficiency. For AI practitioners, this implies that integrating resolution enhancement techniques (like high-res ViTs) with optimized similarity-based feature learning can enable more accurate analysis of large medical images using smaller patches, reducing computational overhead without sacrificing predictive power. |
Papers for 2025-03-26
| Title |
Authors |
Summary |
| Long-Context Autoregressive Video Modeling with Next-Frame Prediction (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, Weijia Mao, Yuchao Gu |
This paper introduces Frame AutoRegressive (FAR), a baseline for long-context autoregressive video modeling using next-frame prediction. The research objective is to address challenges in long-context video modeling, namely visual redundancy impacting temporal extrapolation and computational costs associated with long sequences. Key methodologies include FAR trained with a frame-wise flow matching objective and causal attention, stochastic clean context to bridge the train-inference gap, FlexRoPE for improved test-time temporal extrapolation (up to 16x), and long short-term context modeling for efficient training on longer videos. Primary results show FAR achieves state-of-the-art performance, outperforming Token-AR and demonstrating better convergence than video diffusion transformers, achieving an FVD of 279 on UCF-101 (Table 2, FAR-XL). For AI practitioners, FAR provides an effective and simpler baseline framework for autoregressive video generation that naturally supports variable-length context and improves temporal consistency in long videos compared to existing methods. |
| CoMP: Continual Multimodal Pre-training for Vision Foundation Models (Read more on arXiv or HuggingFace) |
Yu-Gang Jiang, Zuxuan Wu, Wujian Peng, Lingchen Meng, Row11n |
This paper introduces COMP, a continual multimodal pre-training method enhancing Vision Foundation Models (VFMs) for native resolution processing and better language alignment. The objective is to adapt prevailing VFMs, regardless of their original training, to handle diverse image sizes and produce visual features more congruent with Large Language Model (LLM) representations. COMP utilizes Continual Rotary Position Embedding (C-ROPE) for variable resolution inputs and an Alignment Loss for explicit cross-modal feature alignment within a three-stage training framework. Results show COMP-SigLIP achieves significant gains, reaching 66.7 on ChartQA and 75.9 on DocVQA with a 0.5B LLM, while largely maintaining performance on unimodal tasks like ImageNet-1K classification (87.4%). For AI practitioners, COMP provides a mechanism to upgrade existing VFMs, enabling them to serve as more effective vision encoders for LLMs, particularly in tasks demanding fine-grained visual understanding from native resolution images. |
| Exploring Hallucination of Large Multimodal Models in Video |
|
|
| Understanding: Benchmark, Analysis and Mitigation (Read more on arXiv or HuggingFace) |
Yue Liu, Baolong Bi, Jingyi Tang, Jiashu Qu, Hongcheng Gao |
This paper introduces HAVEN, a benchmark to evaluate and mitigate hallucinations in Large Multimodal Models (LMMs) for video understanding. The main objective is to systematically analyze hallucination causes (prior conflict, in-context conflict, capability deficiency) and aspects (object, scene, event) in videos and develop mitigation strategies. Key methodology involves constructing the 6K-question HAVEN benchmark and proposing a thinking-based mitigation approach combining supervised reasoning fine-tuning (SRFT) and thinking-based direct preference optimization (TDPO). Primary results show significant variation in hallucination across 16 LMMs, with the proposed SRFT+TDPO method improving baseline accuracy by 7.65% on hallucination evaluation and reducing the consistency bias score by 4.5%. For AI practitioners, HAVEN offers a standardized tool to assess video LMM reliability regarding hallucinations, while the SRFT+TDPO training strategy presents a method to enhance model factuality and reasoning in video tasks. |
| Inference-Time Scaling for Flow Models via Stochastic Generation and |
|
|
| Rollover Budget Forcing (Read more on arXiv or HuggingFace) |
Minhyuk Sung, Jisung Hwang, Taehoon Yoon, Jaihoon Kim |
This paper introduces an inference-time scaling approach for pretrained flow models using stochastic generation and adaptive compute allocation to enhance alignment with user preferences. The main objective is to enable effective inference-time scaling, similar to diffusion models, for deterministic flow models without retraining. The key methodology involves converting the flow model’s ODE to an SDE, using a Variance Preserving (VP) interpolant instead of a linear one to increase diversity, and applying Rollover Budget Forcing (RBF) to adaptively allocate computation across timesteps. Results show the VP-SDE with RBF significantly improves compositional alignment, achieving a VQAScore of 0.925, outperforming the base model (0.726) and diffusion models even with fewer computations (NFEs). For AI practitioners, this method allows enhancing existing flow models to better follow complex prompts (e.g., counting, spatial relations) during inference, offering a computationally efficient way to improve output quality and alignment compared to standard generation or diffusion model scaling approaches. |
| Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection |
|
|
| with Artifact Explanation (Read more on arXiv or HuggingFace) |
Zichen Wen, Hengrui Kang, Peilin Feng, Junyan Ye, Siwei Wen |
This paper introduces FakeVLM, a specialized large multimodal model for detecting synthetic images and providing artifact explanations, alongside the FakeClue dataset. The primary objective is to create an LMM-based system capable of accurately classifying images as real or synthetic (general and DeepFake) while offering interpretable, natural language explanations for detected artifacts. FakeVLM employs a LLaVA-v1.5 architecture, fine-tuning all parameters on the novel FakeClue dataset (>100k images, 7 categories) which features fine-grained artifact annotations generated via a multi-LMM strategy and category-specific prompts, framing detection as an explanatory visual question answering task. FakeVLM demonstrated superior performance over baseline LMMs, achieving 0.986 Accuracy and 0.981 F1 score on the FakeClue dataset for combined detection and explanation, nearing expert model performance in detection-only tasks without requiring auxiliary classifiers. For AI practitioners, FakeVLM offers a robust, single-model solution for synthetic image detection that inherently provides interpretability, enhancing trust and transparency in authenticity assessment pipelines compared to black-box classifiers or less specialized LMMs. |
| Scaling Vision Pre-Training to 4K Resolution (Read more on arXiv or HuggingFace) |
Sifei Liu, Yao Lu, Han Cai, Boyi Li, Baifeng Shi |
This paper introduces PS3, a method scaling CLIP-style vision pre-training to 4K resolution with near-constant computational cost by selectively processing local regions instead of entire high-resolution images. The objective is to overcome the prohibitive quadratic/quartic cost of training vision models on high-resolution inputs. PS3 employs a multi-stage architecture involving low-resolution global feature extraction, top-down/bottom-up patch selection based on saliency or text prompts, and multi-scale high-resolution feature extraction on selected patches using localized contrastive learning. Applied within a Multimodal Large Language Model (MLLM) named VILA-HD, PS3 significantly improves performance on high-resolution tasks; on the proposed 4KPro benchmark, VILA-HD achieves 74.2% accuracy, outperforming Qwen2-VL by 3.2% while being 2.96x faster. For AI practitioners, PS3 provides a computationally efficient pre-training framework enabling MLLMs to perceive fine-grained details in 4K images, significantly enhancing capabilities for tasks requiring high-resolution visual understanding with reduced inference latency compared to full-image processing or token pruning methods. |
| Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time |
|
|
| Thinking (Read more on arXiv or HuggingFace) |
Yunjie Ji, Shuaiting Chen, Haotian Wang, Sitong Zhao, Xiaoyu Tian |
This paper introduces “Multi-round Thinking,” a test-time scaling method enhancing large language model (LLM) reasoning by iteratively refining answers using previous outputs as prompts. The main objective is to improve LLM reasoning performance, especially on complex tasks, by overcoming limitations of single-step reasoning and cognitive inertia without requiring additional training. The key methodology involves repeatedly prompting the LLM with the original question concatenated with the model’s final answer from the previous round, using a specific prompt template. Primary results show consistent performance gains across models and benchmarks; for example, QwQ-32B improved pass@1 accuracy on AIME 2024 from 80.3% (Round 1) to 82.1% (Round 2), and DeepSeek-R1 improved from 79.7% to 82.0%. For AI practitioners, this simple, training-free technique offers a practical method to potentially enhance LLM accuracy at inference time simply by re-prompting, although it incurs additional computational cost and latency per round. |
| CoLLM: A Large Language Model for Composed Image Retrieval (Read more on arXiv or HuggingFace) |
Son Tran, Mubarak Shah, Ashish Tawari, Jinyu Yang, Chuong Huynh |
CoLLM introduces a Large Language Model (LLM) based framework for Composed Image Retrieval (CIR) that synthesizes training triplets dynamically from image-caption pairs. The objective is to overcome CIR data scarcity, enhance multimodal query understanding using LLMs, and improve evaluation benchmark reliability. Key methodology includes synthesizing reference image embeddings using Spherical Linear Interpolation (Slerp) and modification text using template-based interpolation between image-caption pairs, feeding these into an LLM for composed query embedding generation. CoLLM achieves state-of-the-art results on multiple CIR benchmarks, and the introduced MTCIR dataset yields up to 15% performance improvement for baseline models compared to other synthetic datasets. For AI practitioners, the principal implication is a method for supervised CIR model training without expensive manually annotated triplets, providing scalability alongside a large-scale synthetic dataset (MTCIR) and refined evaluation benchmarks. |
| MDocAgent: A Multi-Modal Multi-Agent Framework for Document |
|
|
| Understanding (Read more on arXiv or HuggingFace) |
Yun Li, Tong Sun, Ruiyi Zhang, Peng Xia, Siwei Han |
MDocAgent is a novel multi-modal, multi-agent framework integrating text and image retrieval-augmented generation (RAG) for improved document question answering (DocQA). The primary objective is to address the limitations of single-modal DocQA systems by effectively integrating and reasoning over both textual and visual information in complex documents. The methodology utilizes parallel text and image RAG pipelines feeding context to five specialized agents (General, Critical, Text, Image, Summarizing) that collaborate to extract, analyze, and synthesize information guided by extracted critical cues. Preliminary experiments show MDocAgent achieves an average performance improvement of 12.1% over current state-of-the-art methods on five benchmarks using top-1 retrieval. For AI practitioners, this demonstrates that a structured multi-agent, multi-modal RAG approach can enhance DocQA accuracy on complex documents by enabling detailed cross-modal understanding and synthesis beyond single-modal or basic LVLM capabilities. |
| Latent Space Super-Resolution for Higher-Resolution Image Generation |
|
|
| with Diffusion Models (Read more on arXiv or HuggingFace) |
Seon Joo Kim, Jinwoo Kim, Sangmin Han, Jinho Jeong |
This paper proposes LSRNA, a framework combining Latent space Super-Resolution (LSR) and Region-wise Noise Addition (RNA) to improve higher-resolution image generation with diffusion models. The objective is to overcome limitations like manifold deviation (latent upsampling) and smoothness (RGB upsampling) in reference-based high-resolution generation, enabling faster inference and better detail preservation beyond native model resolutions. The methodology involves training an LSR module to map low-resolution latents to the high-resolution manifold and using RNA to inject Canny edge-guided noise adaptively, enhancing high-frequency details without progressive upsampling. Integrating LSRNA into DemoFusion for 16x resolution (4096x4096) reduced generation time to 34% (1507s to 506s) and improved patch-FID from 32.89 to 29.12 compared to the baseline DemoFusion. AI practitioners can leverage LSRNA to accelerate and enhance detail in high-resolution image generation pipelines built on pretrained diffusion models, offering a superior alternative to progressive latent upscaling or RGB-space upsampling methods. |
| ReSearch: Learning to Reason with Search for LLMs via Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Chenzheng Zhu, Yijie Zhou, Haoze Sun, Tianpeng Li, Mingyang Chen |
ReSearch trains Large Language Models (LLMs) to integrate reasoning with external search using reinforcement learning, without supervised data on reasoning steps. The primary objective is to enable LLMs to handle complex multi-hop questions requiring multiple retrieval steps by treating search operations as part of the reasoning chain. The key methodology involves using Group Relative Policy Optimization (GRPO), where the LLM generates text thoughts and search queries, receives retrieval results, and is optimized based solely on rewards derived from final answer correctness and format adherence. Experiments training Qwen2.5 models showed significant improvements over baselines on multi-hop QA benchmarks, with average absolute gains ranging from 8.9% to 22.4% across benchmarks, such as a 17.56% average LLM-as-a-judge improvement for the 7B model. For AI practitioners, this demonstrates a viable approach to train more capable reasoning and multi-step Retrieval-Augmented Generation (RAG) systems using reinforcement learning from final outcomes, reducing the need for costly supervised reasoning data and enhancing model generalizability. |
| LookAhead Tuning: Safer Language Models via Partial Answer Previews (Read more on arXiv or HuggingFace) |
Mengshu Sun, Lin Yuan, Yujie Luo, Mengru Wang, Kangwei Liu |
This paper introduces LookAhead Tuning, a data modification technique using partial answer previews to preserve large language model (LLM) safety during fine-tuning. The primary objective is to mitigate the degradation of safety alignment caused by fine-tuning, particularly on benign data, without sacrificing downstream task performance. The key methodology involves modifying training data instructions by appending either the initial tokens of the ground-truth answer (Real Answer) or a fixed prefix phrase (Virtual Answer), thereby minimizing perturbations to the model’s initial token distributions. Results show LookAhead Tuning (virtual) significantly improves safety metrics (e.g., +20.76% average Jailbreak Safe Rate) compared to vanilla fine-tuning, while maintaining comparable utility (-1.59% average decrease across tasks). For AI practitioners, this presents a simple, low-resource, data-centric method to fine-tune models more safely without requiring architectural changes or significant computational overhead. |
| Frequency Dynamic Convolution for Dense Image Prediction (Read more on arXiv or HuggingFace) |
Ying Fu, Chenggang Yan, Liang Li, Lin Gu, CharlesChen2023 |
Frequency Dynamic Convolution (FDConv) introduces a novel approach to enhance dynamic convolution by learning frequency-diverse weights within a fixed budget in the Fourier domain. The primary objective is to overcome the limited adaptability and high parameter cost associated with the frequency homogeneity observed in traditional dynamic convolution methods. FDConv employs Fourier Disjoint Weight (FDW) to create diverse parallel weights from frequency-grouped spectral coefficients, Kernel Spatial Modulation (KSM) for fine-grained spatial filter adjustment, and Frequency Band Modulation (FBM) for spatially varying frequency response adaptation. Applied to ResNet-50 for object detection, FDConv achieves an Apbox of 39.4 on COCO with only +3.6M parameters, outperforming prior methods requiring substantially larger parameter increases (e.g., ODConv +65.1M for 39.2 Apbox). For AI practitioners, FDConv provides a parameter-efficient module to improve the adaptability and performance of vision models on dense prediction tasks by explicitly managing weight frequency diversity, integrating readily into existing ConvNet and Transformer architectures. |
| LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary |
|
|
| Semantic Segmentation (Read more on arXiv or HuggingFace) |
Giorgos Tolias, Jiří Matas, Yannis Kalantidis, Vladan Stojnić |
This paper presents LPOSS/LPOSS+, a training-free label propagation method for improving open-vocabulary semantic segmentation using Vision-Language and Vision Models. The objective is to enhance coarse initial VLM patch-level predictions and overcome patch-resolution limitations by propagating labels across patches and then pixels. The methodology involves a two-stage label propagation (LP) process: first on a patch graph using Vision Model features for affinities (LPOSS), followed by pixel-level LP initialized with patch-level results (LPOSS+), enabling joint prediction across the entire image. LPOSS+ achieves state-of-the-art performance among training-free methods, attaining an average mIoU of 42.1% across eight datasets with ViT-B/16 backbones. For AI practitioners, LPOSS+ offers a plug-and-play, training-free technique to significantly refine segmentation outputs from existing VLMs, particularly improving accuracy near object boundaries without requiring model retraining. |
| Gumbel-Softmax Flow Matching with Straight-Through Guidance for |
|
|
| Controllable Biological Sequence Generation (Read more on arXiv or HuggingFace) |
Alexander Tong, Yinuo Zhang, Sophia Tang, pranamanam |
This paper introduces Gumbel-Softmax Flow Matching and Score Matching, generative frameworks operating on the continuous simplex for biological sequence design. The primary objective is to develop a scalable and controllable method for generating discrete sequences like DNA and proteins by learning smooth interpolations from noise to data using a novel Gumbel-Softmax interpolant with time-dependent temperature. Methodologically, it derives velocity fields for flow matching and score functions for score matching based on this interpolant and introduces Straight-Through Guided Flows (STGFlow), a training-free classifier guidance technique leveraging straight-through estimators. Results demonstrate state-of-the-art performance in conditional DNA promoter design (MSE 0.029), competitive de novo protein generation, and effective target-binding peptide design using STGFlow guidance, outperforming existing binders in docking scores. For AI practitioners, this provides a scalable flow-matching framework for discrete data generation on the simplex, offering a modular, training-free guidance mechanism (STGFlow) to control generation towards desired properties using pre-trained classifiers. |
| Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID (Read more on arXiv or HuggingFace) |
wish44165 |
This paper presents a strong baseline for multi-UAV tracking in thermal infrared video using YOLOv12 and BoT-SORT-ReID. The objective was to establish a straightforward yet effective tracking workflow leveraging recent advances in detection and tracking, evaluated against the Anti-UAV Challenge metrics. The methodology integrates the YOLOv12 detector with the BoT-SORT tracker (including ReID for multi-object tracking), utilizing staged training and tailored inference strategies for SOT and MOT tasks without contrast enhancement or temporal fusion. Results demonstrate competitive performance, significantly improving over official baselines, achieving a MOTA score of 0.7609 on Track 3, with increased image input resolution identified as the most significant factor contributing approximately 0.1 to score improvement. For AI practitioners, this work provides a validated high-performance baseline for thermal UAV tracking, emphasizing the effectiveness of combining state-of-the-art detection/tracking models and highlighting input resolution tuning as crucial for optimizing performance. |
| When Words Outperform Vision: VLMs Can Self-Improve Via Text-Only |
|
|
| Training For Human-Centered Decision Making (Read more on arXiv or HuggingFace) |
Yu Yin, Jing Li, Zhe Hu |
This study demonstrates that Visual Language Models (VLMs) can enhance human-centered decision-making capabilities through text-only training, even achieving self-improvement using data from smaller counterpart LLMs. The primary objective was to improve VLM performance on complex decision-making tasks where they initially underperform compared to text-only LLMs. The methodology involved evaluating baseline models on the VIVA benchmark and then fine-tuning VLMs using synthesized text-only situational data generated by either GPT-4o or Llama-3.1 8B. Results show significant accuracy improvements post-training (e.g., Qwen2-VL improved from 80.32% to 83.15% using GPT-4o data) and notably, that training data generated by the smaller Llama 8B yielded comparable gains, demonstrating VLM self-improvement. For AI practitioners, this indicates that VLM reasoning can be effectively and efficiently enhanced for human-centric tasks via text-only data, bypassing the need for costly image-text pairs and enabling improvement using accessible LLM counterparts. |
| Towards a Unified Copernicus Foundation Model for Earth Vision (Read more on arXiv or HuggingFace) |
Thomas Dujardin, Adam J. Stewart, Chenying Liu, Zhitong Xiong, Yi Wang |
This paper introduces a unified framework for Earth observation (EO) foundation models integrating data from all major Copernicus Sentinel missions. The objective is to develop a single model capable of processing diverse spectral/non-spectral sensor data and metadata, overcoming the limitations of sensor-specific approaches. The methodology involves creating Copernicus-Pretrain (18.7M aligned images), Copernicus-FM (a model using dynamic hypernetworks and Fourier-encoded metadata), and Copernicus-Bench (a 15-task benchmark). Copernicus-FM demonstrates superior performance, significantly improving results on Sentinel-3/5P tasks compared to prior models and supervised training, achieving an RMSE of 789.4 on AQ-O3-S5P compared to 1755.6 for DOFA [69], with metadata integration yielding substantial gains (e.g., +22.4% OA on EuroSAT-S1). For AI practitioners, this work offers a scalable architecture (Copernicus-FM) and resources (Copernicus-Pretrain, Copernicus-Bench) enabling the development of versatile foundation models for multimodal geospatial data, applicable across diverse EO tasks including atmospheric and climate studies. |
Papers for 2025-03-25
| Title |
Authors |
Summary |
| I Have Covered All the Bases Here: Interpreting Reasoning Features in |
|
|
| Large Language Models via Sparse Autoencoders (Read more on arXiv or HuggingFace) |
Polina Druzhinina, Andrey Galichin, tlenusik, razzant, therem |
This research identifies and validates reasoning-specific features in Large Language Models (LLMs) using Sparse Autoencoders (SAEs). The main research question is how reasoning capabilities are internally encoded within LLMs, specifically the DeepSeek-R1 series. The key methodology involves training SAEs on LLM activations, proposing a “ReasonScore” metric to identify reasoning features, and using feature steering to analyze their impact. Primary results show that steering identified features increases reasoning trace length, such as feature i=46379 increasing the completion length by 29% for the AIME 2024 task. The principal implication is that AI practitioners can use SAEs and feature steering to interpret, and potentially improve, the internal reasoning processes of LLMs. |
| Position: Interactive Generative Video as Next-Generation Game Engine (Read more on arXiv or HuggingFace) |
XihuiLiu, dizhang, Xintao, chehx, VictorYuki |
This position paper proposes Interactive Generative Video (IGV) as the foundation for Generative Game Engines (GGE), enabling AI-driven game development. The main research objective is to demonstrate how IGV can overcome current game engine limitations and serve as the core technology for next-generation game development. The key methodology involves extending video generation models with interactivity, user control, memory, physics-awareness, and causal reasoning to create a comprehensive GGE framework. A hierarchical maturity roadmap (L0-L4) is presented, outlining progressive steps from manual game development to self-evolving world ecosystems, including systems with level L2, where the engine continuously generates physics-compliant video based on users interactions. The principal implication for AI practitioners is that IGV offers a viable pathway to create games with unlimited content, realistic physics, and adaptive gameplay, reducing development barriers and expanding creative possibilities. |
| Video-T1: Test-Time Scaling for Video Generation (Read more on arXiv or HuggingFace) |
Hanyang Wang, duanyueqi, xhangzhan, iseesaw, Liuff23 |
The paper introduces Video-T1, a framework for improving video generation quality by scaling computation at test time. The main research question is how much video generation quality can be improved by allowing a model to use more inference-time compute, given a challenging text prompt. The key methodology involves reinterpreting test-time scaling as a search problem and using test-time verifiers and heuristic algorithms, including random linear search and Tree-of-Frames (ToF), to sample better trajectories from Gaussian noise. Experiments on text-conditioned video generation benchmarks show that increasing test-time compute consistently improves video quality; for example, the CogVideoX-5B model with Test-Time Scaling (TTS) achieved a total score of 84.42, a 3.44% increase. AI practitioners can use this framework to significantly enhance the quality of generated videos without retraining, by scaling inference-time computation. |
| Aether: Geometric-Aware Unified World Modeling (Read more on arXiv or HuggingFace) |
Junyichen, lizizun, AmberHeart, ZhouTimeMachine, HaoyiZhu |
AETHER is a unified world model that integrates 4D reconstruction, action-conditioned video prediction, and visual planning using synthetic data. The main research objective is to develop a framework that enables geometry-aware reasoning in world models by jointly optimizing reconstruction, prediction, and planning capabilities. The key methodology involves post-training a video diffusion model with synthetic 4D data, utilizing a robust camera pose annotation pipeline, and integrating cross-task and cross-modal conditioning signals. Primary results show AETHER achieved a zero-shot Absolute Relative error (Abs Rel) of 0.056 on the KITTI dataset for video depth estimation, surpassing prior methods. Principal implication for AI practitioners is that AETHER provides an effective framework for post-training world models with scalable synthetic data, achieving strong zero-shot transfer to real-world tasks and enabling actionable planning. |
| SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for |
|
|
| Open Base Models in the Wild (Read more on arXiv or HuggingFace) |
jxhe, HelicHe, SivilTaram, yuzhen17, AndrewZeng |
Zero reinforcement learning (zero RL) training can significantly improve the reasoning abilities of open base language models. The paper investigates how zero RL training impacts the reasoning capabilities of diverse open base language models. The methodology involves training 10 base models (e.g., Llama3-8B, Mistral-7B, Qwen2.5 series) using the GRPO algorithm, with rule-based rewards based solely on answer correctness, on the training sets of GSM8K and MATH datasets. Results show that zero RL training consistently improves accuracy and response length, with Qwen-2.5-32B’s Pass@1 on AIME 24 increasing from 10.0 to 36.7. The study provides AI practitioners with key design factors and empirical findings to enable successful zero RL training, emphasizing alignment of data difficulty with model capability and avoiding overly restrictive format rewards. |
| OmnimatteZero: Training-free Real-time Omnimatte with Pre-trained Video |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Nir Darshan, ramiben, galchechik, m98levy, Dvir |
OmnimatteZero is a training-free approach for video object removal, extraction, and layer composition using pre-trained video diffusion models. The main research objective is to adapt zero-shot image inpainting techniques for efficient and high-quality video omnimatte without requiring model training or optimization. The key methodology leverages self-attention maps from video diffusion models to identify object footprints and effects, then uses latent arithmetic for object layer isolation and blending. OmnimatteZero achieves a PSNR of 39.09 and LPIPS of 0.012 on the Movie dataset for background reconstruction, outperforming all existing methods, and runs at 0.04 seconds per frame on an A100 GPU. AI practitioners can utilize this method for real-time video editing applications like object removal and layer composition without any fine-tuning, requiring only a pre-trained video diffusion model. |
| LEMMA: Learning from Errors for MatheMatical Advancement in LLMs (Read more on arXiv or HuggingFace) |
mingchenlin2025, Word2Li, QizhiPei, LHL3341, panzs |
LEMMA is a framework that enhances LLMs’ mathematical reasoning by learning from error-corrective trajectories. The main research objective is to improve LLMs’ reflective reasoning capabilities by constructing and learning from data consisting of incorrect solutions, erroneous steps, and reflection connections to correct solutions. The key methodology involves an error-type grounded mistake augmentation method to collect diverse errors, constructing paired reflection data via “Fix & Continue” and “Fresh & Restart” mechanisms, and connecting trajectories with model-aware reflection links. Primary results show that models fine-tuned with LEMMA achieved a 62.4% average accuracy on in-distribution and out-of-distribution math datasets using LLaMA3-8B, outperforming strong baselines. Principal implication is that AI practitioners can significantly improve LLMs’ mathematical reasoning abilities by systematically constructing and learning from structured error data, without reliance on complex external critique models. |
| Equivariant Image Modeling (Read more on arXiv or HuggingFace) |
Li Li, Zigang Geng, hanhu2, Mendel192, dongruixiao |
The paper introduces an equivariant image modeling framework that aligns optimization targets across subtasks in image generation. The core research question is: Can a task decomposition framework be established to inherently align optimization targets across subtasks in image generation? The method uses column-wise tokenization and windowed causal attention to enhance translational symmetry and enforce consistent contextual relationships. When evaluated on class-conditioned ImageNet generation at 256x256 resolution, the proposed approach achieves a generative FID (gFID) of 5.57, comparable to state-of-the-art AR models with fewer computational resources. The principal implication is that, AI practitioners can improve model efficiency and zero-shot generalization in generative modeling by leveraging inherent equivariance properties of visual data. |
| Training-free Diffusion Acceleration with Bottleneck Sampling (Read more on arXiv or HuggingFace) |
lazybone128, Lingaaaaaaa, xiaoxuefeng, renyuxi, tyfeld |
The paper introduces Bottleneck Sampling, a training-free framework to accelerate inference in diffusion models by leveraging low-resolution priors. The main research objective is to reduce the computational cost of high-resolution image and video generation in diffusion models without sacrificing output quality. The key methodology is a high-low-high denoising workflow that performs high-resolution denoising at initial and final stages and low-resolution denoising in intermediate steps, with adaptive resolution transition points and timestep shifting. Primary results show that Bottleneck Sampling accelerates inference by up to 3x for image generation and 2.5x for video generation, while maintaining comparable output quality to standard full-resolution sampling. For AI practitioners, Bottleneck Sampling provides a plug-and-play acceleration strategy for existing diffusion models that does not require retraining or architectural modifications, enhancing deployment in resource-constrained environments. |
| Judge Anything: MLLM as a Judge Across Any Modality (Read more on arXiv or HuggingFace) |
shuang72, Frywind, NiuniuWang, yuhangchen, fjchendp |
This paper introduces TASKANYTHING and JUDGEANYTHING benchmarks to evaluate Multimodal LLMs (MLLMs) as judges across various modalities for multimodal understanding and generation tasks. The main research objective is to evaluate whether MLLMs can serve as a unified judge for assessing the understanding and generation ability of any-to-any modality tasks. The key methodology involves constructing two benchmarks: TASKANYTHING, with 1,500 open-ended queries across 15 any-to-any modality categories, and JUDGEANYTHING, evaluating MLLMs’ judging abilities using Pair Comparison and Score Evaluation settings against human annotations. The primary results show that MLLMs align more closely with human preferences on Pair Comparison than Score Evaluation, with Gemini-1.5-Pro achieving an average of 70.6% accuracy on Pair Comparison for Multimodal Understanding tasks. Principal implication for AI practitioners: Current MLLM-as-a-Judge systems show promise but face limitations, especially in Multimodal Generation tasks, highlighting the need for refined evaluation protocols and improved alignment with human preferences in model development. |
| FFN Fusion: Rethinking Sequential Computation in Large Language Models (Read more on arXiv or HuggingFace) |
geifmany, AmnonGeifman, omripuny, mdabbah-nvidia, abercovich |
FFN Fusion is a novel architectural optimization that reduces sequential computation in large language models by parallelizing Feed-Forward Network (FFN) layers. The main research objective is to investigate whether sequences of FFN layers in transformers can be parallelized to reduce inference latency while preserving model accuracy. The key methodology involves identifying and fusing consecutive FFN layers into wider, parallel layers, supported by a block-wise dependency analysis and a distillation-based refinement. The primary result is that Ultra-253B-Base, created using FFN Fusion, achieves a 1.71x speedup in inference latency and 35x reduction of the per-token cost compared to its parent Llama-3.1-405B model, while maintaining or exceeding its performance. AI practitioners can apply FFN Fusion to significantly improve the inference efficiency of large language models, particularly in resource-constrained deployment scenarios. |
| CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Models (Read more on arXiv or HuggingFace) |
Ziwei Liu, Raymond A. Yeh, Amber Yijia Zheng, weepiess2383 |
CFG-Zero* enhances classifier-free guidance for flow matching models by addressing inaccuracies in early-stage velocity estimation. The main research objective is to improve the sample quality and controllability of flow matching models during generation when the learned velocity is underfitted. The key methodology involves introducing an optimized scale to correct for velocity inaccuracies and a “zero-init” technique that zeros out the first few steps of the ODE solver. Primary results show that CFG-Zero* achieves the best FID Score of 2.10 and sFID Score of 4.59 on ImageNet-256, outperforming existing methods. Principal implication for AI practitioners is that CFG-Zero* can be readily integrated into flow matching models to improve image fidelity and text alignment, particularly during the early stages of training or when models are underfitted. |
| Video SimpleQA: Towards Factuality Evaluation in Large Video Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Pengfei Hu, zhangysk, Drexubery, grejioh, mengcao |
Video SimpleQA, a new benchmark, evaluates the factual accuracy of large video language models (LVLMs). The main research objective is to develop and introduce a comprehensive benchmark for assessing the factuality of LVLMs in video contexts. The key methodology involves creating a dataset of 2030 question-answer pairs derived from 1293 videos, with questions requiring external knowledge, designed to be fact-seeking, and having definitive, short-form, and externally verified answers. Primary results indicate that the best-performing model, Gemini-1.5-Pro, achieves an F-score of only 54.4%, and open-source models perform notably worse. The principal implication for AI practitioners is the need to address significant deficiencies in factual adherence of current LVLMs, highlighting a critical area for improvement in developing models that can accurately and reliably process video information. |
| AgentRxiv: Towards Collaborative Autonomous Research (Read more on arXiv or HuggingFace) |
Samuel Schmidgall, mdmoor |
Here’s a summary of the paper “AgentRxiv: Towards Collaborative Autonomous Research” by Schmidgall and Moor, following the provided guidelines: 1. AgentRxiv is a framework enabling LLM agent laboratories to collaboratively conduct research by sharing and building upon findings via a centralized preprint server. 2. Main research question/objective: To determine if autonomous LLM agents can collaboratively improve research performance by sharing and building upon each other’s work. 3. Key methodology: Agent laboratories developed reasoning/prompting techniques, uploading/retrieving reports on a shared server, with performance evaluated on benchmarks like MATH-500. 4. Primary results: Agents with access to prior research achieved higher performance improvements (11.4% relative improvement on MATH-500) compared to isolated agents. Multiple labs using the System were able to reach a best performance of 79.8% 5. Principal implication for AI practitioners: AgentRxiv demonstrates a viable path for accelerating AI research through agent collaboration, potentially leading to faster discovery and improved generalization of techniques. |
| MagicComp: Training-free Dual-Phase Refinement for Compositional Video |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Hongyu Zhang, ClownRat, Pengjin, BestWishYsh, dyf |
MagicComp is a training-free framework that improves compositional text-to-video generation through dual-phase refinement during conditioning and denoising. The main research objective is to address challenges in compositional video generation, such as attribute binding, spatial relationships, and interactions between multiple subjects, without additional training. The key methodology involves Semantic Anchor Disambiguation (SAD) to resolve inter-subject ambiguity during conditioning, and Dynamic Layout Fusion Attention (DLFA) for spatial-attribute binding during denoising. Results on T2V-CompBench show that MagicComp achieves a Consist-attr score of 0.7665, outperforming the baseline CogVideoX-2B’s score of 0.6775. The principal implication for AI practioners is that MagicComp can be integrated into existing text-to-video architectures to enhance compositional video generation quality without requiring additional training or significant increases in inference time. |
| Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models |
|
|
| via Vision-Guided Reinforcement Learning (Read more on arXiv or HuggingFace) |
Fan Yang, Hongyin Zhao, Shurong Zheng, Yousong Zhu, Yufei Zhan |
Vision-R1 is a vision-guided reinforcement learning algorithm that improves object localization in Large Vision-Language Models (LVLMs) using only curated instruction data. The main research objective is to enhance LVLM capabilities in object localization tasks without relying on human-annotated preference data or specialized reward models. The key methodology involves a criterion-driven reward function based on visual feedback and a progressive rule refinement strategy that dynamically adjusts reward criteria during training. Results show that fine-tuning a 7B LVLM with Vision-R1 achieved up to 50% improvement, and specifically, increased the Average Precision (mAP) on the ODINW-13 benchmark by 9.0 points compared to supervised fine tuning for Qwen2.5-VL-7B. AI practitioners can utilize Vision-R1 to improve object localization performance in LVLMs without the need for costly human-annotated preference data, leading to substantial gains in model accuracy. |
| Reasoning to Learn from Latent Thoughts (Read more on arXiv or HuggingFace) |
Tatsunori Hashimoto, cmaddis, nband, ryoungj |
This paper introduces “reasoning to learn,” an approach for improving language model (LM) pretraining data efficiency by explicitly modeling and inferring the latent human thoughts underlying text generation. The main research objective is to investigate whether augmenting observed text data with inferred latent thoughts can improve data efficiency in LM pretraining, particularly in a data-constrained regime. The key methodology involves training LMs to jointly model the distribution of observed text and synthesized latent thoughts, using an EM algorithm (BoLT) to iteratively improve latent thought quality and LM capability. Primary results show that a 1.1B LM pretrained with GPT-4o-mini synthesized latent thoughts achieves 25.4% accuracy on MATH, significantly outperforming the 5.74% accuracy achieved by training on raw data alone. For AI practitioners, this implies that incorporating synthesized latent thoughts during pretraining can lead to substantial data efficiency improvements, enabling the development of more capable models with limited data. |
| Defeating Prompt Injections by Design (Read more on arXiv or HuggingFace) |
Tianqi Fan, ftramer, carlini, iliashum, dedeswim |
CaMeL is a system designed to protect Large Language Model (LLM) agents from prompt injection attacks by enforcing explicit security policies. The main research question is how to design a robust defense that prevents prompt injection attacks in LLM agents interacting with untrusted data, without modifying the underlying model. The key methodology involves extracting control and data flows from user queries, representing them as pseudo-Python code, and enforcing security policies via a custom Python interpreter that tracks provenance and capabilities. The primary results demonstrate that CaMeL solves 67% of tasks with provable security in the AgentDojo benchmark, with some utility degradation on specific task suites, and eliminates almost all prompt injection attacks when combined with capabilities and policy. The principal implication for AI practitioners is that using capability-based security, explicit isolation, and a custom interpreter to manage data and control flows can significantly enhance the security of LLM agent systems against prompt injections, without relying solely on inherent model robustness. |
| Typed-RAG: Type-aware Multi-Aspect Decomposition for Non-Factoid |
|
|
| Question Answering (Read more on arXiv or HuggingFace) |
Yunho Maeng, Hyeonseo Nam, Ahjeong Park, keirahrlee, oneonlee |
Typed-RAG is a framework for non-factoid question answering that improves response quality by classifying questions and decomposing multi-aspect queries. The main research objective is to address the limitations of existing retrieval-augmented generation (RAG) systems in handling the complexity and diversity of non-factoid questions (NFQs). The key methodology is Typed-RAG, a type-aware, multi-aspect decomposition approach that integrates question type classification and aspect-based decomposition into the RAG pipeline. Experimental results on the Wiki-NFQA dataset show that Typed-RAG outperforms baselines, achieving a Mean Reciprocal Rank (MRR) of 0.8413 with a GPT-40 mini scorer and Mistral-7B base model configuration. Principal implication is that AI practitioners can create NFQA models by leveraging type-aware and multi-aspect decomposition strategies to create a more comprehensive RAG system. |
| AlphaSpace: Enabling Robotic Actions through Semantic Tokenization and |
|
|
| Symbolic Reasoning (Read more on arXiv or HuggingFace) |
Bui Quang Huy, Dinh Bach Vu, alandao |
AlphaSpace enhances spatial reasoning in language models for 3D robotic manipulation using semantic tokenization and symbolic reasoning. The main objective is to improve the ability of language models to perform precise object manipulation in 3D Cartesian space without relying on vision-based embeddings. The key methodology involves a hierarchical semantics-based tokenization strategy that encodes spatial information (including height) and object attributes, combined with synthetic reasoning data for training. AlphaSpace achieves a total accuracy of 66.67% on the EmbodiedBench Manipulation Subtask, significantly outperforming GPT-4o (37.5%) and Claude 3.5 Sonnet (29.17%). AI practitioners can leverage this approach to develop more efficient and accurate robotic control systems that rely less on computationally expensive visual processing and more on structured spatial representations. |
| AMD-Hummingbird: Towards an Efficient Text-to-Video Model (Read more on arXiv or HuggingFace) |
Dong Zhou, He Cui, Takashi Isobe, ebarsoum, gemengmeng |
AMD-Hummingbird is a lightweight text-to-video (T2V) generation framework that balances computational efficiency with high visual quality. The main research objective is to develop a T2V model suitable for resource-constrained devices by addressing the trade-off between model size and visual fidelity. The key methodology involves a two-stage diffusion model distillation pipeline: first pruning the U-Net architecture and then enhancing visual quality via visual feedback learning, combined with a data processing pipeline using LLMs and VQA models. The primary result is that Hummingbird achieves a 31x speedup compared to VideoCrafter2 and reduces U-Net parameters from 1.4 billion to 0.7 billion, while attaining the highest overall VBench score. For AI practitioners, this provides a practical and efficient solution for T2V generation, combining performance, scalability, and flexibility, especially beneficial for deployment on devices with limited computational resources. |
| Lost in Cultural Translation: Do LLMs Struggle with Math Across Cultural |
|
|
| Contexts? (Read more on arXiv or HuggingFace) |
Bhoomika Lohana, jaswindersingh2, 55mv, Abdul084, abedk |
Large Language Models (LLMs) demonstrate reduced mathematical reasoning performance when presented with culturally adapted math word problems, despite the underlying mathematical structure remaining constant. The research investigates whether LLMs’ mathematical reasoning abilities persist across different cultural contexts. Six culturally adapted datasets were synthesized from the GSM8K benchmark by modifying cultural elements (names, foods, places) while preserving mathematical logic. Fourteen LLMs were evaluated, revealing that models performed worse on culturally adapted problems compared to the original GSM8K, with Meta LLaMA 3.1-8B showing the largest accuracy drop (5.9%) on the Somalia dataset. AI practitioners should prioritize diverse and representative training data to improve LLMs’ robustness in real-world applications across various cultural contexts. |
| Variance Control via Weight Rescaling in LLM Pre-training (Read more on arXiv or HuggingFace) |
gueraf, nilabhra, akanyaani, louisowen6 |
This paper introduces weight initialization and variance control techniques to improve LLM pre-training. The main research objective is to investigate how controlling weight variance, both at initialization and during training, impacts LLM stability and downstream task performance. The key methodology involves proposing Layer Index Rescaling (LIR) for weight initialization and Target Variance Rescaling (TVR) for variance control during training, and evaluating these on a 1B parameter LLaMA model using various benchmarks. Primary results show that the combined use of LIR and TVR improves downstream task performance, with up to a 4.6% increase on common pre-training benchmarks, while also reducing extreme activation values. Principal implication for AI practioners is that managing weight variance using LIR and TVR during LLM pre-training can lead to improved model performance and stability, while mitgating some issues as massive activations. |
| V-Seek: Accelerating LLM Reasoning on Open-hardware Server-class RISC-V |
|
|
| Platforms (Read more on arXiv or HuggingFace) |
Luca Benini, Daniele Jahier Pagliari, Alessio Burrello, Mohamed Amine Ahmdi, Javier J. Poveda Rodrigo |
This paper optimizes LLM inference on a many-core RISC-V CPU, achieving significant speedups compared to baseline implementations. The main research objective is to optimize the performance of LLM inference on the Sophon SG2042 RISC-V platform. Key methodologies include developing optimized quantized kernels, choosing a suitable compilation toolchain (Xuantie GCC 10.4 for kernels, Clang 19 for the framework), and optimizing model mapping with NUMA policies. On a DeepSeek R1 Distill Llama 8B model, the authors achieved 4.32 tokens/s for token generation and 6.54 tokens/s for prompt processing, representing speedups of up to 2.9x/3.0x over the baseline. The principal implication is to use, on RISC-V architecture, Clang 19 compiler, to disable NUMA Balancing and activate Memory Interleaving, to improve LLM inference performance. |
| MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse (Read more on arXiv or HuggingFace) |
Han Liu, zhenyupan |
MetaSpatial is an RL-based framework that enhances 3D spatial reasoning in vision-language models (VLMs) for 3D scene generation. The main research objective is to address the lack of internalized 3D spatial reasoning in VLMs and the limitations of supervised fine-tuning for 3D layout generation. The key methodology is a multi-turn reinforcement learning (RL) optimization that uses format detection, physical detection, and rendering-based evaluation to provide reward signals, optimized via Group Relative Policy Optimization (GRPO). Results show that on a Qwen-VL 7B model, MetaSpatial improves format correctness from 0.85 to 0.98 and reduces the object collision rate by 24.5%. For AI practitioners, this provides a method to train VLMs to generate coherent, physically plausible 3D scenes without needing extensive “perfect” layout annotations or manual post-processing. |
| Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent |
|
|
| Diffusion Models (Read more on arXiv or HuggingFace) |
Junjie Liu, Jinjin Zhang, dihuang, xiefan-guo, qiuyuhuang |
Diffusion-4K introduces a framework for direct ultra-high-resolution (4K) image synthesis using latent diffusion models. The main research objective is to enable direct training and generation of 4K images with diffusion models, addressing the lack of a 4K image synthesis benchmark. The key methodology involves a wavelet-based fine-tuning approach for latent diffusion models and the creation of a new benchmark, Aesthetic-4K, including a curated 4K dataset with GPT-40-generated captions. Results show that Diffusion-4K, particularly when powered by models like SD3-2B and Flux-12B, achieves a FID score of 39.49 and the model performs well and improves GLCM Score up to 0.79 on the Aesthetic-Eval@2048 benchmark, outperforming the previous scores. AI practitioners can use Diffusion-4K and the Aesthetic-4K benchmark for training and evaluating models capable of generating high-quality, ultra-high-resolution images with detailed textures and improved text prompt adherence. |
| RDTF: Resource-efficient Dual-mask Training Framework for Multi-frame |
|
|
| Animated Sticker Generation (Read more on arXiv or HuggingFace) |
Yeshuang Zhu, Jiapei Zhang, Ying Deng, Ting Zhang, Zhiqiang Yuan |
i) This paper introduces RDTF, a resource-efficient training framework for generating multi-frame animated stickers using a dual-mask approach and curriculum learning. ii) The main research objective is to demonstrate that training a smaller video generation model from scratch with limited data can outperform parameter-efficient tuning of larger models under resource constraints. iii) Key methodologies include a discrete frame generation network with a spatial-temporal interaction layer, a dual-mask data utilization strategy (condition mask and loss mask), and a difficulty-adaptive curriculum learning method. iv) On the I&T->V task, RDTF achieved an FVD of 442.18 and a VQA of 0.502, outperforming methods like I2V-Adapter and SimDA. v) For AI practitioners, RDTF shows that effective data utilization and curriculum strategies can enable smaller models trained from scratch to achieve superior performance in resource-constrained settings, suggesting an alternative to fine-tuning large pre-trained models. |
| Optimized Minimal 3D Gaussian Splatting (Read more on arXiv or HuggingFace) |
Jong Hwan Ko, epark, maincold2 |
Optimized Minimal 3D Gaussian Splatting (OMG) significantly reduces the storage and computational costs of 3D Gaussian Splatting while maintaining rendering quality. The main objective is to minimize the number of Gaussian primitives and storage requirements for 3D Gaussian Splatting (3DGS) without significantly degrading rendering quality. The key methodology involves using a compact attribute representation with sub-vector quantization, integrating per-Gaussian features with a lightweight neural field, and introducing a local distinctiveness metric for Gaussian pruning. The primary result is that OMG achieves nearly a 50% storage reduction compared to the previous state-of-the-art on the Mip-NeRF 360 dataset, requiring only 4.06 MB while preserving comparable rendering quality. The principal implication for AI practitioners is that they can utilize OMG for real-time, high-fidelity rendering on resource-constrained devices and accelerate training through reduced Gaussians and optimized attribute representation. |
| Verbal Process Supervision Elicits Better Coding Agents (Read more on arXiv or HuggingFace) |
Jui-Ming Yao, Cheng-Pong Huang, MarkChenX |
CURA, a novel code reasoning agent with verbal process supervision (VPS), enhances code generation performance. The main research objective is to examine if iterative verbal process supervision, combined with an agentic reasoning pipeline like Code Understanding and Reasoning Agent (CURA), improves code generation over baseline models. The key methodology involves a process-supervised reasoning framework called CURA, using VPS to generate verbal reward signals at each reasoning step, incorporating iterative feedback within a code-testing sandbox. The primary result is that CURA with VPS achieved a 3.65% improvement over baseline models on BigCodeBench. For AI practitioners, integrating agentic reasoning with iterative, step-level verbal process supervision offers a new, effective approach for enhancing code generation and software engineering tasks, with a direct, measurable performance improvement. |
Papers for 2025-03-24
| Title |
Authors |
Summary |
| MAPS: A Multi-Agent Framework Based on Big Seven Personality and |
|
|
| Socratic Guidance for Multimodal Scientific Problem Solving (Read more on arXiv or HuggingFace) |
Xinyu Zhang, Zhangqi Wang, Zhiyuan Wang, Qika, VentureZJ |
MAPS is a multi-agent framework for multimodal scientific problem-solving, leveraging the Big Seven Personality theory and Socratic questioning to improve reasoning and reflection in AI systems. The main research question is how to leverage and elicit off-the-shelf Multimodal Large Language Models (MLLMs) to address challenging Multimodal Scientific Problems (MSPs). The key methodology involves a multi-agent framework with seven distinct agents, each based on a Big Seven personality trait, using a progressive four-agent solving strategy and a Critic agent for Socratic feedback. The primary results show that MAPS outperforms the current state-of-the-art model by 15.84% across all tasks on the EMMA, Olympiad, and MathVista datasets, and slightly exceeds human expert by 3.58%. The principal implication is that AI practitioners can use this framework to enhance multi-model comprehensive reasoning and provide continous feedback mechanism to improve the accuracy in complex, multimodal scientific problem-solving scenarios. |
| MARS: A Multi-Agent Framework Incorporating Socratic Guidance for |
|
|
| Automated Prompt Optimization (Read more on arXiv or HuggingFace) |
Jun Liu, Haiping Zhu, Zhangqi Wang, Qika, VentureZJ |
MARS is a multi-agent framework for automated prompt optimization (APO) that uses Socratic guidance and autonomous planning. The main research objective is to address the limited flexibility of fixed templates and inefficient search in prompt spaces that are present in existing APO methods. The key methodology involves a multi-agent architecture with seven agents, including a Planner, and a Teacher-Critic-Student Socratic dialogue pattern for iterative prompt refinement. Primary results show that MARS outperforms the previous state-of-the-art by 6.04% on general tasks and achieves 85.11% accuracy on 12 general tasks. The use of MARS can help AI practitioners by enabling more efficient and precise prompt refinement, leading to better performance of LLMs across various tasks without needing to create complex meta prompts. |
| RoboFactory: Exploring Embodied Agent Collaboration with Compositional |
|
|
| Constraints (Read more on arXiv or HuggingFace) |
Xiaohong Liu, Zhenfei Yin, Xiufeng Song, FACEONG, IranQin |
RoboFactory introduces a framework for generating safe and efficient collaborative data for multi-agent embodied systems using compositional constraints. The main research objective is to address the challenges of multi-agent collaboration in embodied systems by proposing and validating a compositional constraint-based approach. The key methodology involves using a large language model (RoboBrain) to generate sub-goals and textual constraints, constructing constraint interfaces (RoboChecker) to ensure adherence, and generating trajectories using predefined motion primitives. Primary results show that in tasks involving three agents, an average success rate of 20.5% was achieved using diffusion policy with 150 demonstrations, and the use of a “local view” with “separate policy” improves task success rates for the “Food Place” task from 0% to 20% in imitation learning when compared with a “shared policy”. The principal implication for AI practitioners is that they can use RoboFactory’s compositional constraints and automated data collection framework to develop and evaluate multi-agent manipulation systems more efficiently. |
| When Less is Enough: Adaptive Token Reduction for Efficient Image |
|
|
| Representation (Read more on arXiv or HuggingFace) |
Andrey Kuznetsov, Elizaveta Goncharova, Eduard Allakhverdov |
This paper introduces an adaptive token reduction method for vision encoders to improve efficiency without compromising performance. The main research objective is to determine if all visual tokens generated by vision encoders are equally valuable, or if some can be discarded to reduce computational costs. The key methodology involves integrating an autoencoder with a Gumbel-Softmax selection mechanism to identify and retain only the most informative visual tokens, based on reconstructability. Primary results show that on OCR-based tasks, over 50% of the visual context can be removed with minimal performance loss using the LLaVA-NeXT model. Principal implication for AI practitioners is that multimodal pruning can be adaptively performed, facilitating scalable and low-overhead inference without requiring additional model fine-tuning. |
| Bridging Continuous and Discrete Tokens for Autoregressive Visual |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Yuanzhi Zhu, Yao Teng, Zhijie Lin, ShuhuaiRen, Epiphqny |
TokenBridge bridges continuous and discrete token representations for autoregressive visual generation, achieving high-quality image synthesis with simplified modeling. The main objective is to maintain the representational capacity of continuous tokens while preserving the modeling simplicity of discrete tokens in autoregressive visual generation. The key methodology is post-training quantization of pre-trained continuous VAE features using a dimension-wise quantization strategy, paired with a lightweight autoregressive prediction mechanism for large token spaces. The proposed method achieved an FID score of 1.55 and an IS of 313.3 on ImageNet 256x256, matching state-of-the-art continuous approaches while still using discrete token prediction. AI practitioners can leverage this approach to build high-quality autoregressive visual generation models using standard categorical prediction, bypassing the complexity of continuous distribution modeling, without compromising image quality. |
| OpenVLThinker: An Early Exploration to Complex Vision-Language Reasoning |
|
|
| via Iterative Self-Improvement (Read more on arXiv or HuggingFace) |
Wei Wang, Nanyun Peng, Fan Yin, Hritik Bansal, Yihe Deng |
OpenVLThinker explores iteratively improving vision-language reasoning in large vision-language models (LVLMs) through a combination of supervised fine-tuning (SFT) and reinforcement learning (RL). The main research objective is to investigate whether complex reasoning capabilities, similar to those in large language models, can be integrated into LVLMs and improve performance on multimodal reasoning tasks. The key methodology involves iterative SFT and RL, with each iteration’s RL-improved model generating refined SFT datasets for the next round, using distilled reasoning steps from text-only models. Primary results show that OpenVLThinker-7B achieved 70.2% accuracy on MathVista, surpassing the Qwen2.5-VL-7B baseline of 68.5%. Principal implication for AI practioners that is combining SFT with verifiable RL can enhance the multi-step reasoning in LVLMs. |
| Modifying Large Language Model Post-Training for Diverse Creative |
|
|
| Writing (Read more on arXiv or HuggingFace) |
Max Kreminski, Yuqian Sun, Melissa Roemmele, Vishakh Padmakumar, John Joon Young Chung |
The paper introduces a post-training approach that modifies large language models (LLMs) to improve output diversity in creative writing while maintaining quality. The primary objective is to enhance LLM output diversity during creative writing tasks by incorporating “deviation” (difference from other outputs for the same prompt) into the training objective. The methodology involves adapting Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO) by weighting training instances with the deviation of the winning response. Results showed that a Llama-3.1-8B-based diversified DPO model achieved on-par diversity with a human-created dataset and output quality similar to instruction-tuned models like GPT-4o. AI practitioners can leverage this approach to promote output diversity in creative writing LLMs, balancing diverse and high-quality outputs by incorporating the instance deviation during post-training. |
| ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question |
|
|
| Generation and Answering (Read more on arXiv or HuggingFace) |
Wei Liu, Peng Zhang, Yuchong Sun, Zhengfeng Lai, Guan123 |
ETVA is a new method for evaluating text-to-video alignment using question generation and answering. The main research objective is to develop a more accurate and fine-grained evaluation metric for text-to-video (T2V) alignment than existing methods. The key methodology involves a multi-agent system for generating atomic questions from text prompts using scene graphs and a knowledge-augmented multi-stage reasoning framework for answering questions about generated videos. Primary results show that ETVA achieves a Spearman’s correlation coefficient of 58.47 with human judgment, significantly outperforming existing metrics like VideoScore (31.0). Principal implication is that AI practitioners can use ETVA and its associated benchmark (ETVABench) for more reliable and human-aligned evaluation of text-to-video generation models, focusing improvements on fine-grained semantic alignment. |
| Single Image Iterative Subject-driven Generation and Editing (Read more on arXiv or HuggingFace) |
Idan Schwartz, Gal Chechik, yairshp |
SISO is a training-free method for personalizing image generation and editing using only a single subject image. The main objective is to develop a method for subject-driven image generation and editing from a single image without requiring encoder pre-training. SISO iteratively optimizes a similarity score between the generated image and the input subject image using pre-trained models like DINO and IR. The method achieved a CMMD score of 0.18 in image generation on a benchmark dataset, improving prompt adherence while maintaining image fidelity compared to baselines. AI practitioners can use SISO as a plug-and-play optimization technique for existing image generators, enabling efficient single-image personalization without extensive training. |
| MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical |
|
|
| Problems (Read more on arXiv or HuggingFace) |
Jun Cen, Tao Feng, Yunqiu Xu, Felix Chen, JacobYuan |
MathFlow decouples visual mathematical problem-solving in Multimodal Large Language Models (MLLMs) into perception and inference stages, improving performance. The main research objective is to evaluate and enhance MLLMs’ ability to accurately perceive and interpret diagrams in visual mathematical problems. The key methodology involves creating a new benchmark, FlowVerse, to categorize information components, and developing MathFlow, a modular pipeline with a dedicated perception model (MathFlow-P-7B) trained via multi-task pretraining and supervised fine-tuning. Primary results indicate that MathFlow*GPT-4V achieved a 56.7% accuracy on MathVerse’s testmini set, and integrated MathFlow-P-7B yields substantial performance gains with various inference models. For AI practitioners, MathFlow offers a modular problem-solving pipeline that enhances the model’s mathematical problem understanding and solving ability by decoupling the perception and inference process. |
| Enabling Versatile Controls for Video Diffusion Models (Read more on arXiv or HuggingFace) |
Jiaxing Yan, Xiaobin Lu, Haoming Qin, Hao Zhou, Xu Zhang |
VCtrl is a unified framework for fine-grained control over pre-trained video diffusion models using diverse control signals. The main research objective is to enable precise and flexible spatiotemporal control in text-to-video generation, addressing limitations of existing methods. The key methodology involves a unified control signal encoding pipeline and a sparse residual connection mechanism, integrated with a conditional module, to handle various control signals (Canny edges, segmentation masks, human keypoints) without modifying the base generator. Results demonstrate that, on the Canny-to-Video task, VCtrl-Canny achieves a Canny Matching score of 0.24 and an FVD score of 985.31. For AI practitioners, VCtrl provides a generalizable and efficient way to incorporate diverse user-specified controls into existing video diffusion models, improving controllability and generation quality. |
| When Preferences Diverge: Aligning Diffusion Models with Minority-Aware |
|
|
| Adaptive DPO (Read more on arXiv or HuggingFace) |
Donghao Luo, Kai Hu, Chengming Xu, Chen Liu, Lingfan Zhang |
This paper proposes Adaptive-DPO, a novel approach to align diffusion models with human preferences, addressing the challenge of minority samples in preference datasets. The main research question is how to mitigate the detrimental effects of minority preference samples (erroneous annotations and subjective divergences) on diffusion model alignment. The key methodology is a minority-instance-aware metric incorporating intra-annotator confidence and inter-annotator stability, used to adaptively reweight and adjust the DPO loss function. Primary results show that Adaptive-DPO outperforms standard DPO; for example it is found that on SD1.5 with 20% flipped labels, Adaptive-DPO achieves an ImageReward of 0.34, while DPO achieves 0.00. The principal implication for AI practitioners is that incorporating Adaptive-DPO can improve the robustness and effectiveness of preference learning in text-to-image generation tasks, especially in the presence of noisy or subjective preference data. |
| FastCuRL: Curriculum Reinforcement Learning with Progressive Context |
|
|
| Extension for Efficient Training R1-like Reasoning Models (Read more on arXiv or HuggingFace) |
Xuan Luo, Wenjie Yang, Zheng Li, Mao Zheng, Mingyang Song |
FASTCURL accelerates reinforcement learning for reasoning models by segmenting training data and progressively extending the context window. The main objective is to improve the training efficiency and performance of R1-like reasoning models, particularly with a 1.5B parameter language model, in tackling complex reasoning tasks. The key methodology, FASTCURL, involves length-aware training data segmentation based on input prompt length and curriculum reinforcement learning with a progressively increasing context window. FASTCURL-1.5B-Preview surpasses DeepScaleR-1.5B-Preview across five benchmark datasets while using only 50% of the training steps. For AI practitioners, FASTCURL demonstrates a practical and efficient strategy of segmenting training dataset, and applying curriculum reinforcement learning to reduce training resources (by 50% in training steps, the paper illustrates) for R1-like large language models. |
| From Head to Tail: Towards Balanced Representation in Large |
|
|
| Vision-Language Models through Adaptive Data Calibration (Read more on arXiv or HuggingFace) |
Yu Cheng, Jiawei Zhou, Xiaoye Qu, hitsmy |
Here’s a concise summary of the research paper, adhering to your guidelines: The paper introduces an Adaptive Data Refinement (ADR) framework to address the long-tail data distribution problem in Large Vision-Language Models (LVLMs). The main research objective is to investigate and mitigate the impact of imbalanced training data on the performance of LVLMs. The key methodology involves a two-stage approach: Data Rebalancing (DR), which filters redundant head data, and Data Synthesis (DS), which uses diffusion models to generate scarce tail data. Primary results show that ADR improves the average performance of LLaVA 1.5 by 4.36% across eleven benchmarks without increasing training data volume. Principal implication for AI practitioners is, ADR can be integrated into existing LVLMs to improve their performance on tasks with long-tail data distributions, enhancing robustness and generalization capabilities. |
| PVChat: Personalized Video Chat with One-Shot Learning (Read more on arXiv or HuggingFace) |
Yuchen Li, Yumeng Li, Gang Xu, Weilong Yan, Master-Shi |
PVChat is a personalized video large language model capable of subject-aware question answering from a single reference video. The main research objective is to develop a ViLLM that can understand and answer questions about specific individuals in videos after learning from only one video of each individual. The key methodology involves a Mixture-of-Heads (MoH) enhanced ViLLM optimized on a synthetically augmented video-QA dataset, using a progressive image-to-video learning strategy, and a ReLU Routing MoH attention mechanism. The primary result is that PVChat achieved an accuracy of 0.901, a BLEU score of 0.562, and a BERTScore of 0.952, outperforming state-of-the-art ViLLMs in personalized feature understanding. For AI practitioners, PVChat offers a framework for building video understanding models that can learn individual-specific information from minimal data, enabling more personalized applications in areas such as smart healthcare and home environments. |
| Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language |
|
|
| Model (Read more on arXiv or HuggingFace) |
Junlin Han, Runjia Li, Yun Liu, Guolei Sun, Zhaochong An |
GFS-VL enhances generalized few-shot 3D point cloud segmentation (GFS-PCS) by integrating 3D vision-language models (VLMs) and few-shot samples. The main research objective is to improve the performance of GFS-PCS models in segmenting both base and novel object classes, particularly when limited labeled data is available for novel classes. The key methodology involves using a 3D VLM to generate pseudo-labels for novel classes, filtering these pseudo-labels with few-shot samples for accuracy, adaptively infilling unlabeled regions using a combination of pseudo-label context and few-shot data, and employing a novel-base mix strategy for data augmentation. The primary results show that on the ScanNet200 benchmark, GFS-VL achieves a 28.57% increase in harmonic mean (HM) and a 23.37% increase in mIoU-N over the existing state-of-the-art GFS-PCS methods for the 5-shot setting. The principal implication is that AI practitioners can leverage the combined strengths of 3D VLMs’ open-world knowledge and the precision of few-shot samples to achieve significantly improved segmentation in scenarios where acquiring large labeled datasets for new object classes is impractical. |
| Implicit Bias-Like Patterns in Reasoning Models (Read more on arXiv or HuggingFace) |
Calvin K. Lai, l048596 |
Reasoning models exhibit processing differences for association-compatible versus incompatible information, similar to human implicit bias. The research examined whether reasoning models show implicit bias-like patterns by expending differential computational effort on association-compatible versus incompatible information. The researchers adapted the Implicit Association Test (IAT) for reasoning models, called RM-IAT, measuring the number of reasoning tokens generated via API calls to OpenAI’s 03-mini model for different association tasks. The model generated significantly more reasoning tokens in the association-incompatible condition than the association-compatible condition in nine of ten RM-IATs; for example, the Instruments/Weapons + Pleasant/Unpleasant RM-IAT generated, on average, 84.29 more tokens in the incompatiable vs. compatiable condition.. AI practitioners should consider that reasoning models may have implicit bias-like patterns that increase computational effort when processing association-incompatible information, impacting efficiency and potentially leading to subtle biases. |
| FFaceNeRF: Few-shot Face Editing in Neural Radiance Fields (Read more on arXiv or HuggingFace) |
Junyong Noh, Hangyeul Shin, Chaelin Kim, Kwan Yun |
FFaceNeRF is a NeRF-based method for 3D-aware face editing that enables customization with few-shot training on desired mask layouts. The main research objective is to overcome the limitation of existing mask-based 3D face editing methods that rely on pre-trained segmentation masks with fixed layouts. The key methodology involves a geometry adapter with feature injection and latent mixing for tri-plane augmentation (LMTA) to enable adapting to various mask layouts using few training samples. The proposed method achieved an average mIoU of 85.33% for mask generation on a test set, outperforming NeRFFaceEditing’s 81.37%. For AI practitioners, FFaceNeRF facilitates personalized and detailed 3D face editing with limited data, reducing the dependency on extensive, specifically segmented datasets. |
| TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented |
|
|
| Reality via 3D Gaussian Splatting (Read more on arXiv or HuggingFace) |
Tiansong Zhou, Zhonghua Jiang, Gaige Wang, Jingchuan Hu, Jianchuan Chen |
TaoAvatar generates photorealistic, full-body avatars from multi-view sequences for real-time AR applications. The research objective is to create high-fidelity, lightweight, and drivable full-body talking avatars that can run in real-time on mobile and AR devices. The key methodology combines 3D Gaussian Splatting (3DGS) with a personalized clothed human parametric template (SMPLX++), using a teacher-student framework with non-rigid deformation baking and blend shapes compensation. The primary result is that TaoAvatar achieves state-of-the-art rendering quality, maintaining 90 FPS on high-definition stereo devices like the Apple Vision Pro at 2K resolution. For AI practitioners, TaoAvatar provides a lightweight and efficient approach for representing and rendering lifelike full-body avatars directly deployable to resource-constrained AR environments and mobile devices. |
Papers for 2025-03-21
| Title |
Authors |
Summary |
| One-Step Residual Shifting Diffusion for Image Super-Resolution via |
|
|
| Distillation (Read more on arXiv or HuggingFace) |
agoxandr, skushneryuk, ngushchin, kekchpek, apryc1 |
This paper introduces RSD, a distillation method for accelerating diffusion-based super-resolution models, achieving single-step image restoration. The main research objective is to develop a computationally efficient distillation method for ResShift that maintains high perceptual quality while significantly reducing inference time. The key methodology is based on training a student network to produce images such that a fake ResShift model trained on them coincides with the teacher model, incorporating multistep training and additional supervised losses. Primary results show that RSD outperforms the teacher ResShift model and SinSR on RealSR with a MUSIQ score of 69.172 compared to the teacher’s 61.330. Principal implication for AI practitioners is that RSD offers a way to deploy diffusion-based super-resolution models in real-time applications on consumer devices by providing faster inference and lower computational requirements. |
| Stop Overthinking: A Survey on Efficient Reasoning for Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
andrewwen, HongyiLiuAI, jy-yuan, JiamuZhang, yangsui |
This survey systematically investigates and explores the current progress toward achieving efficient reasoning in Large Language Models (LLMs), particularly addressing the “overthinking phenomenon”. The main research question is how to optimize reasoning length in LLMs while preserving or even enhancing their reasoning capabilities. Key methodologies used include model-based (RL with length reward, SFT with varied-length CoT data), reasoning output-based (latent representation compression, dynamic reasoning), and input prompt-based (prompt-guided, attribute-driven routing) approaches. Primary results across multiple works demonstrate the feasibility of significantly shortening LLM reasoning paths, with one example, O1-Pruner, showing the effectiveness of the Length-Harmonizing reward for shortening CoT length. Principal implication for AI practitioners is that efficient reasoning strategies can substantially reduce computational costs and improve the responsiveness of LLM-based applications without significantly compromising, and sometimes improving accuracy. |
| Unleashing Vecset Diffusion Model for Fast Shape Generation (Read more on arXiv or HuggingFace) |
Huiwenshi, wangfuyun, cocacola, qikahh, ZeqiangLai |
FlashVDM is a framework for accelerating 3D shape generation using Vecset Diffusion Models (VDMs) by optimizing both diffusion sampling and VAE decoding. The main research objective is to address the slow inference speed of VDMs in generating high-resolution 3D shapes. The key methodology involves Progressive Flow Distillation for diffusion sampling, and a lightning vecset decoder with Adaptive KV Selection, Hierarchical Volume Decoding, and Efficient Network Design for VAE acceleration. Primary results show a 45x speedup in VAE decoding (from 22.33s to 0.491s) and an overall 32x speedup in shape generation, achieving comparable quality to state-of-the-art with significantly reduced inference time. AI practitioners can leverage FlashVDM to enable significantly faster 3D shape generation with VDMs, opening possibilities for real-time interactive applications. |
| Survey on Evaluation of LLM-based Agents (Read more on arXiv or HuggingFace) |
Yilun Zhao, Guy Uziel, Lilach Eden, lihaoxin2020, Asaf-Yehudai |
This paper provides a comprehensive survey of evaluation methodologies for LLM-based agents across capabilities, applications, and frameworks. The main research objective is to systematically analyze existing benchmarks and frameworks for evaluating LLM-based agents across four critical dimensions: fundamental agent capabilities, application-specific benchmarks, generalist agent benchmarks, and agent evaluation frameworks. The key methodology involves a systematic review and categorization of existing literature, benchmarks, and evaluation methods for LLM-based agents, highlighting emerging trends and research gaps. Primary results include the identification of trends toward more realistic and challenging evaluations (e.g., some top-performing models scoring as low as 2% on complex benchmarks), the continuous updating of “live benchmarks,” and a lack of standardized metrics for cost-efficiency, safety, and granular performance evaluation. A principal implication for AI practitioners is the need to adopt and develop more granular, dynamic, and safety-focused evaluation frameworks to ensure robust and responsible development of LLM-based agents, shifting beyond coarse-grained metrics to include fine-grained trajectory analysis and security aspects. |
| DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers (Read more on arXiv or HuggingFace) |
Mingwu Zheng, Xintao Wang, Haotian Yang, Ziyang Yuan, MingleiShi |
DiffMoE introduces a Mixture-of-Experts (MoE) architecture for diffusion transformers that enables dynamic token selection and global token accessibility. The main research objective is to address the limitations of existing MoE approaches in diffusion models, specifically their restricted token accessibility and fixed computational patterns. The key methodology incorporates a batch-level global token pool during training and a capacity predictor for dynamic resource allocation during inference. DiffMoE achieves a state-of-the-art FID score of 2.13 on ImageNet 256x256 class-conditional generation with classifier-free guidance (cfg=1.5), surpassing dense models with 1.5x the number of activated parameters. The principle implication is that AI practitioners can leverage DiffMoE to scale diffusion models more efficiently, achieving superior performance while maintaining computational efficiency compared to dense models and previous MoE implementations. |
| Scale-wise Distillation of Diffusion Models (Read more on arXiv or HuggingFace) |
Dmitry Baranchuk, Artem Babenko, Denis Kuznedelev, Nikita Starodubcev |
Scale-wise Distillation (SWD) is a novel method that improves diffusion model efficiency by progressively increasing spatial resolution during sampling. The paper’s main objective is to investigate whether generating images scale-by-scale across the diffusion process can improve the efficiency of diffusion distillation methods. The key methodology involves integrating a scale-wise generation approach into existing diffusion distillation frameworks, specifically DMD2, and introducing a patch distribution matching (PDM) loss. A primary result is that, within SD3.5 medium, the 6-step scale-wise configuration achieves a FID score of 23.0 on COCO 2014, while its full-scale 6-step counterpart reaches 20.4. AI practitioners can leverage SWD to achieve a balance between generation speed and quality in diffusion models, offering a practical technique to accelerate inference by operating at lower resolutions during initial sampling steps. |
| Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning (Read more on arXiv or HuggingFace) |
Hannah Brandon, Alisson Azzolini, NVIDIA, zhuoliny, fferroni |
Cosmos-Reason1 is a family of multimodal large language models developed by NVIDIA, trained to integrate physical common sense and embodied reasoning. The main research objective is to develop models capable of understanding the physical world and generating appropriate embodied decisions using natural language through long chain-of-thought reasoning. The key methodology involves defining ontologies for physical common sense and embodied reasoning, curating datasets based on these ontologies, and training models in four stages: vision pre-training, general supervised fine-tuning (SFT), Physical AI SFT, and Physical AI reinforcement learning (RL). Evaluation results show that the Cosmos-Reason1-56B model achieves 60.2% accuracy on the physical common sense benchmark, and Physical AI RL improves performance across most benchmark components. For AI practitioners, using Physical AI SFT and RL, this work will make the code and models open-source to expedite the progress of building Physical AI systems that understand and perform complex tasks. |
| MathFusion: Enhancing Mathematic Problem-solving of LLM through |
|
|
| Instruction Fusion (Read more on arXiv or HuggingFace) |
Honglin Lin, Yu Li, Zhuoshi Pan, Lijun Wu, Qizhi Pei |
MathFusion enhances LLM mathematical problem-solving by synthesizing new training instructions from existing problem pairs. The main research objective is to improve LLMs’ mathematical reasoning capabilities through cross-problem instruction synthesis, overcoming limitations of instance-level data augmentation. The key methodology, MathFusion, employs three fusion strategies—sequential, parallel, and conditional—to combine existing mathematical problems into new, more complex ones. Experiments using DeepSeekMath-7B, Mistral-7B, and Llama3-8B show that MathFusion increases accuracy by 18.0 points on average across diverse benchmarks with only 45K additional synthetic instructions. The principal implication is that AI practitioners can improve mathematical reasoning performance in LLMs efficiently using this data synthesis technique. |
| InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity (Read more on arXiv or HuggingFace) |
Hao Kang, Zichuan Liu, Yumin Jia, Qing Yan, Liming Jiang |
InfiniteYou (InfU) is a Diffusion Transformer (DiT)-based framework for identity-preserved image generation that recrafts photos using text descriptions while maintaining facial identity. The main research objective is to address limitations of existing methods, such as insufficient identity similarity, poor text-image alignment, and low generation quality when using DiTs. The key methodology involves InfuseNet, a generalization of ControlNet, which injects identity features into the DiT base model via residual connections, combined with a multi-stage training strategy using synthetic single-person-multiple-sample (SPMS) data. Primary results showed that InfU achieved a lower ID Loss (0.209) compared to PuLID-FLUX (0.225) and FLUX.1-dev IPA (0.772), while also achieves the highest CLIPScore and PickScore. A principal implication for AI practitioners is that they can utilize InfU’s plug-and-play design, as well as the method of residual feature connections demonstrated, to create high-fidelity and text-aligned identity-preserved images, and extend use cases beyond those presented in the paper. |
| Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Huan Wang, Can Qin, Yang Sui, Haoxuan You, KD-TAO |
VidKV, a plug-and-play KV cache quantization method, compresses the KV cache in Video Large Language Models (VideoLLMs) to 1.x-bit precision with minimal performance loss. The main research question is how to effectively quantize the KV cache in VideoLLMs to lower than 2 bits while preserving model performance. The key methodology involves mixed-precision quantization for the key cache (2-bit for anomalous channels, 1-bit with FFT for normal channels) and 1.58-bit quantization with optional token protection for the value cache, applied per-channel. Primary results show that VidKV compresses the KV cache to 1.5-bit and 1.58-bit precision on LLaVA-OV-7B and Qwen2.5-VL-7B, achieving a VideoChat-GPT average score of 3.06 and 3.00 respectively, which is a close to no loss to the FP16 counterparts. The principal implication for AI practitioners is that they can significantly reduce the memory footprint and computational cost of VideoLLM inference using VidKV, enabling efficient deployment of these models. |
| JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play |
|
|
| Visual Games with Keyboards and Mouse (Read more on arXiv or HuggingFace) |
Yitao Liang, Xiaojian Ma, Kaichen He, Zihao Wang, Muyao Li |
JARVIS-VLA introduces a new training paradigm, ActVLP, that enhances vision-language-action (VLA) models for decision-making in open-world environments like Minecraft. The main research objective is to investigate whether integrating visual-language tasks into the post-training phase of VLA models improves their performance. The key methodology, ActVLP, involves a three-stage training pipeline: post-training language models on text-only world knowledge, post-training both vision encoder and language models on multimodal vision-language alignment and spatial grounding datasets, then post-training language models on multimodal instruction following datasets. The primary result is that post-training on non-trajectory tasks leads to a 40% improvement over the best agent baseline in Minecraft on a diverse set of atomic tasks. For AI practitioners, this demonstrates that incorporating visual-language post-training significantly improves VLA model performance in complex decision-making tasks, offering a new, effective training approach. |
| CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners (Read more on arXiv or HuggingFace) |
Shumin Deng, Jia-Chen Gu, Jizhan Fang, Yunzhi Yao, Ningyu |
CaKE improves the generalization of knowledge editing in large language models by aligning edits with the models’ reasoning circuits. The main research objective is to address the poor performance of existing knowledge editing (KE) methods on downstream reasoning tasks involving updated knowledge. The key methodology, CaKE, involves generating circuit-aware training data that explicitly requires reasoning with updated knowledge and training the model to construct robust reasoning circuits integrating the new information. Experimental results show CaKE improves multi-hop reasoning accuracy on the MQUAKE dataset by an average of 20% compared to existing KE methods. AI practitioners can use CaKE to create language models that not only store updated facts but also effectively apply this knowledge in downstream reasoning tasks, improving generalizability. |
| Ultra-Resolution Adaptation with Ease (Read more on arXiv or HuggingFace) |
Xinchao Wang, Zhenxiong Tan, Songhua Liu, Ruonan Yu |
URAE facilitates adapting text-to-image diffusion models to ultra-high resolutions with limited data and computation. The main research objective is to identify efficient guidelines for adapting existing text-to-image models to ultra-high resolutions (2K and 4K) when training data and computational resources are limited. The key methodology involves theoretically and empirically investigating data efficiency (using synthetic data from teacher models) and parameter efficiency (tuning minor components of weight matrices), alongside examining the impact of classifier-free guidance. Primary results include that URAE achieves comparable 2K generation performance to FLUX1.1 [Pro] Ultra with only 3K samples and 2K iterations, while setting new benchmarks for 4K resolution generation. The principal implication for AI practitioners is that they can adapt diffusion models to ultra-high resolutions efficiently by using synthetic data when available, tuning minor weight matrix components, and disabling classifier-free guidance during adaptation. |
| Expert Race: A Flexible Routing Strategy for Scaling Diffusion |
|
|
| Transformer with Mixture of Experts (Read more on arXiv or HuggingFace) |
Xun Zhou, Defa Zhu, Ziyu Wang, FetchFortune, yyk-wew |
Race-DiT introduces a flexible routing strategy for scaling diffusion transformers with Mixture of Experts (MoE). The main research objective is to enhance the scalability and performance of diffusion transformers by integrating MoE methods with a new routing strategy called Expert Race. The key methodology involves allowing tokens and experts to compete and selecting the top candidates, along with per-layer regularization and router similarity loss. The primary result is that Race-DiT achieves a 7.2x speedup in iterations when reaching the same training loss compared to DiT-XL, with an equal number of activated parameters. Principal implication for AI practioners is that it provides a method to improve performance gains and scaling in diffusion models while maintaining good expert utilization, with superior ImageNet validation and image quality. |
| MagicMotion: Controllable Video Generation with Dense-to-Sparse |
|
|
| Trajectory Guidance (Read more on arXiv or HuggingFace) |
Qi Dai, Hui Zhang, Rui Wang, Zhen Xing, quanhaol |
MagicMotion is a novel image-to-video generation framework that enables trajectory control through three levels of conditions (masks, bounding boxes, and sparse boxes). The main objective is to develop a trajectory-controllable video generation model that overcomes limitations of existing methods, such as imprecise trajectory adherence and compromised visual quality, and supports multiple trajectory control formats. The key methodology involves a progressive training strategy using a Trajectory ControlNet architecture (similar to ControlNet) to inject trajectory conditions into a diffusion model, alongside a novel latent segment loss. The primary results demonstrate that MagicMotion outperforms previous methods on the MagicBench benchmark, achieving a Mask_IoU of 91.57% and a Box_IoU of 87.75% in Stage 1, and Mask_IoU=76.61 and Box_IoU=81.45 in Stage 2. AI practitioners can use MagicMotion for improved controllable video generation, allowing more precise control over object motion and facilitating the creation of high-quality videos with user-specified trajectories. |
| M3: 3D-Spatial MultiModal Memory (Read more on arXiv or HuggingFace) |
Jianglong Ye, Xuanbin Peng, Ri-Zhao Qiu, Yuchen Song, Xueyan Zou |
M3 is a multimodal memory system that integrates 3D Gaussian Splatting with foundation models to store and render multimodal representations of medium-sized static scenes. The main research objective is to develop a spatial memory system that efficiently stores and retrieves multi-granularity information about static scenes from video sources, addressing computational constraints and information loss in existing feature splatting methods. The key methodology involves storing high-dimensional feature maps from foundation models in a memory bank (principal scene components) and using low-dimensional queries from 3D Gaussians as indices, applying Gaussian memory attention to render foundation model embeddings. The primary results show that M3 outperforms previous methods in feature similarity and downstream tasks; for example, M3 achieved a cosine similarity of 0.6074 on the Playroom dataset using CLIP, compared to 0.4867 for F-Splat. For AI practitioners, M3 provides a more effective framework to integrate foundation models with 3D scene representations, enabling efficient memorization and query of visual and semantic information in spatial contexts. |
| Why Do Multi-Agent LLM Systems Fail? (Read more on arXiv or HuggingFace) |
Bhavya Chopra, Lakshya A. Agrawal, Shuyi Yang, Melissa Z. Pan, Mert Cemri |
This paper presents a comprehensive study of failure modes in Multi-Agent Systems (MAS) powered by Large Language Models (LLMs). The main research question is: Why do Multi-Agent LLM Systems fail, and what is the taxonomy of these failure modes? The key methodology involves grounded theory analysis of 150+ conversation traces from five popular MAS frameworks, with human expert annotation and iterative refinement to establish a failure taxonomy. The primary result is a taxonomy (MASFT) of 14 failure modes grouped into 3 categories, with the “Poor Specification” category appearing in 37.17% of analyzed traces. AI practitioners should use this taxonomy to identify and mitigate failures in MAS designs, focusing on enhanced specification, inter-agent coordination, and task verification, rather than relying solely on base LLM improvements. |
| 1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering (Read more on arXiv or HuggingFace) |
Xinchao Wang, Xingyi Yang, Qiuhong Shen, nopyyh |
4DGS-1K achieves over 1000 FPS in dynamic scene rendering by addressing temporal redundancy in 4D Gaussian Splatting. The main research objective is to reduce the storage requirements and improve the rendering speed of 4D Gaussian Splatting (4DGS) for dynamic scenes. The key methodology involves a two-step pruning approach: first, pruning short-lifespan Gaussians using a spatial-temporal variation score, and second, filtering inactive Gaussians using a key-frame based temporal filter. The method achieves a 41x reduction in storage and 9x faster rasterization speed compared to vanilla 4DGS on complex dynamic scenes, while maintaining comparable visual quality. For AI practitioners, this implies that they can render high-fidelity, complex dynamic scenes, in real-time with significantly less storage requirements through the implementation of temporal-aware filtering and pruning. |
| XAttention: Block Sparse Attention with Antidiagonal Scoring (Read more on arXiv or HuggingFace) |
Song Han, Junxian Guo, Guangxuan Xiao, Ruyi Xu, songhan |
XAttention is a plug-and-play framework that accelerates long-context Transformer inference by using block-sparse attention based on antidiagonal scoring. The paper’s main research question is: Can a block-sparse attention mechanism be designed to accelerate long-context Transformers without accuracy loss? XAttention’s methodology sums antidiagonal values in the attention matrix to estimate block importance, enabling selective computation. Evaluations on language and video benchmarks show XAttention achieves comparable accuracy to full attention, with up to 13.5x acceleration in attention computation during pre-filling. This suggests AI practitioners can deploy more efficient long-context Transformer models in real-world applications by adopting XAttention to reduce computational costs. |
| Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on |
|
|
| Compressed Spatial Tokens (Read more on arXiv or HuggingFace) |
Zhifeng Gao, Lin Yao, Haowei Lin, Shuqi Lu, guolinke |
Uni-3DAR is a unified framework for 3D structural generation and understanding that uses autoregressive prediction on compressed spatial tokens. The main research objective is to develop a unified framework that seamlessly integrates 3D generation and understanding (3D GU) tasks via autoregressive prediction. The key methodology involves a hierarchical tokenization using an octree to compress 3D space, a two-level subtree compression strategy, and a masked next-token prediction mechanism. Primary results show that Uni-3DAR surpasses previous state-of-the-art diffusion models on microscopic 3D GU tasks, achieving up to 256% relative improvement on PXRD-guided crystal structure prediction and up to 21.8x faster inference speeds. AI practitioners can use Uni-3DAR as a more efficient and versatile framework for unifying diverse 3D GU tasks, potentially leading to faster and more accurate models in areas like materials science and drug discovery. |
| CLS-RL: Image Classification with Rule-Based Reinforcement Learning (Read more on arXiv or HuggingFace) |
Kaipeng Zhang, Jike Zhong, Ming Li, yuxianglai117, stzhao |
This paper introduces CLS-RL, a rule-based reinforcement learning approach for fine-tuning Multimodal Large Language Models (MLLMs) for image classification, demonstrating improved performance and generalization compared to supervised fine-tuning. The main research objective is to explore few-shot MLLM classification fine-tuning and address catastrophic forgetting issues observed with supervised fine-tuning (SFT). The key methodology involves using verifiable signals (class names) as rewards to fine-tune MLLMs and formatting the reward to encourage “thinking” before answering, and comparing the proposed method to No-Thinking-CLS-RL. The primary results show CLS-RL outperforms SFT in most of 11 datasets, with a base-to-new generalization setting achieving 81.17% accuracy on base classes and 79.15% on new classes for CLS-RL, compared to 67.4% and 70.73% for SFT. For AI practitioners, using rule-based reinforcement learning for fine-tuning MLLMs can lead to improved image classification performance and better generalization to new classes, even with limited labeled data. |
| LHM: Large Animatable Human Reconstruction Model from a Single Image in |
|
|
| Seconds (Read more on arXiv or HuggingFace) |
Weichao Shen, Peihao Li, Xiaodong Gu, Lingteng Qiu, DyrusQZ |
LHM is a feed-forward transformer model that generates animatable 3D human avatars from single images in seconds. The main objective is to create a generalizable model for high-fidelity 3D human reconstruction from a single image that supports real-time rendering and animation. The method utilizes a multimodal transformer architecture with a head feature pyramid encoding scheme to fuse 3D point features and 2D image features and represents the avatar as 3D Gaussian splatting. Trained on a large-scale video dataset, LHM achieves a PSNR of 25.183 on synthetic data, outperforming existing methods. For AI practitioners, LHM offers an efficient solution for generating animatable 3D human models from single images, reducing reliance on extensive optimization or post-processing. |
| Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video |
|
|
| Diffusion (Read more on arXiv or HuggingFace) |
Chua Tat-Seng, Fan Hehe, Ma Fan, zhenglin |
Zero-1-to-A is a method for generating animatable 4D head avatars from a single image using video diffusion models. The main research objective is to generate high-fidelity 4D head avatars from a single image input, overcoming the spatial and temporal inconsistencies of video diffusion models. The key methodology, Zero-1-to-A, employs Symbiotic GENeration (SymGEN) to iteratively construct a consistent video dataset and optimize the avatar, alongside a Progressive Learning strategy that separates spatial and temporal learning. Results show that Zero-1-to-A achieves an average CLIP score of 0.285 (ViT-L/14) and 0.322(ViT-B/32), and improves ID consistency and rendering speed compared to prior methods. AI practitioners can leverage this method for efficient and data-sparse creation of high-fidelity, animatable head avatars from single images, eliminating the need for extensive training data. |
| Towards Unified Latent Space for 3D Molecular Latent Diffusion Modeling (Read more on arXiv or HuggingFace) |
Kenji Kawaguchi, Sihang Li, Yi Zhao, Zhiyuan Liu, Yanchen Luo |
The paper introduces UAE-3D and UDM-3D, a VAE and latent diffusion model, for 3D molecule generation using a unified latent space. The main research question is whether a unified generative model can seamlessly integrate all modalities of 3D molecule generation (atom types, bonds, 3D coordinates). The key methodology is a multi-modal VAE (UAE-3D) that compresses 3D molecules into a unified latent space, using a Relational Transformer encoder and SE(3) augmentations, combined with a Diffusion Transformer (DiT) for latent diffusion modeling. The results show that UDM-3D achieves 100.0% atom and bond accuracy and 0.0002 coordinate RMSD in reconstruction, and 9.89E-03 bond length distribution in GEOM-Drugs in comparison with the second-best result of 3.91E-01. For AI practitioners, this offers a way to generate 3D molecules with improved efficiency and accuracy by leveraging a unified latent space, simplifying the complexities of handling multi-modality and equivariance. |
| Tokenize Image as a Set (Read more on arXiv or HuggingFace) |
Shuyang Gu, Han Hu, Mengde Xu, Zigang Geng |
This paper introduces TokenSet, a new image generation paradigm using set-based tokenization and distribution modeling to improve context aggregation and robustness. The main research objective is to develop a more effective image representation that dynamically allocates coding capacity based on regional semantic complexity, unlike fixed-position latent codes. The key methodology involves representing images as unordered token sets, using a dual transformation to convert sets into fixed-length sequences, and applying a novel Fixed-Sum Discrete Diffusion model for distribution modeling. Primary results show that the TokenSet achieves a reconstruction rFID of 2.74 on ImageNet, with an token overlap of 87.6% after adding a level-10 Gaussian noise (Signal-to-Noise Ratio (dB)), which is a superior performance as compared to prior state of arts. AI practitioners can use TokenSet’s representation and modeling approach to create image generation models that better capture global context and exhibit robustness to image perturbations for a variety of computer vision applications. |
| NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenes (Read more on arXiv or HuggingFace) |
Angel X. Chang, Qinghong Han, rexleeppp |
NuiScene explores efficient generation of unbounded outdoor scenes using a novel vector set representation and explicit outpainting. The main research objective is to develop an efficient method for generating large, unbounded outdoor scenes with varying heights and diverse styles. The key methodology involves compressing scene chunks into uniform vector sets using 3DShape2VecSet, training an explicit outpainting diffusion model for unbounded generation, and curating a dataset (NuiScene43) of 43 scenes with unified scales and cleaned ground geometries. The vector set diffusion model achieves an FPD score of 0.571 and KPD score of 0.951, outperforming the triplane baseline. For AI practitioners, this method provides a more efficient approach for representing and generating unbounded 3D outdoor scenes compared to methods using spatially structured latents. |
| Fin-R1: A Large Language Model for Financial Reasoning through |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jinyi Niu, Lingfeng Zeng, Fangqi Lou, Xin Guo, Zhaowei Liu |
Fin-R1 is a 7-billion parameter large language model designed specifically for financial reasoning, addressing data fragmentation, reasoning uncontrollability, and generalization challenges. The main research objective was to develop a model that can effectively handle complex financial problems and improve performance in financial reasoning tasks. The key methodology involved constructing a high-quality dataset (Fin-R1-Data) with 60,091 chain-of-thought entries, followed by Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) using Group Relative Policy Optimization (GRPO). Fin-R1 achieved an average score of 75.2 across multiple financial benchmarks, outperforming other similar-sized models and ranking second overall. The principal implication is that AI practitioners can leverage Fin-R1’s two-stage training framework and specialized dataset to build more accurate and interpretable decision-making tools for financial AI applications, particularly in areas like compliance and robo-advisory. |
| SALT: Singular Value Adaptation with Low-Rank Transformation (Read more on arXiv or HuggingFace) |
Mohammad Yaqub, Hu Wang, Mohammed Elseiagy, Abdelrahman Elsayed, Sarim-Hash |
SALT is a parameter-efficient fine-tuning method for adapting the Segment Anything Model (SAM) to medical image segmentation. The main research objective is to develop a method that effectively adapts foundation models to the medical domain while minimizing trainable parameters and preserving pre-trained knowledge. The key methodology, SALT, combines SVD-based adaptation of dominant singular values with low-rank updates for the remaining subspace, using trainable scale, shift, and low-rank matrices. SALT outperformed state-of-the-art PEFT methods (LoRA and SVD) by 2% to 5% in Dice score on five medical datasets, with only 3.9% trainable parameters. AI practitioners can use SALT for efficient and robust adaptation of large foundation models to specialized domains like medical imaging, achieving high accuracy with significantly reduced computational overhead compared to full fine-tuning or other PEFT methods. |
| MotionStreamer: Streaming Motion Generation via Diffusion-based |
|
|
| Autoregressive Model in Causal Latent Space (Read more on arXiv or HuggingFace) |
Liang Pan, Ke Fan, Huaijin Pi, Shunlin Lu, lxxiao |
MotionStreamer is a framework for text-conditioned streaming motion generation that uses a diffusion-based autoregressive model in a causal latent space. The main research objective is to address the challenge of generating human motion sequences incrementally while dynamically adapting to online text inputs and maintaining semantic coherence. The key methodology involves incorporating a continuous causal latent space into a probabilistic autoregressive model with a diffusion head, utilizing a Causal Temporal AutoEncoder (TAE) for motion compression and online decoding, and employing Two-Forward and Mixed training strategies. The method achieves a Frechet Inception Distance (FID) of 10.724 on the HumanML3D test set, outperforming existing approaches. For AI practioners, MotionStreamer provides an effective model to generate realistic and diverse human motions that directly respond to progressive input text prompts, with low latency. |
| Make Your Training Flexible: Towards Deployment-Efficient Video Models (Read more on arXiv or HuggingFace) |
Yi Wang, Xiangyu Zeng, Tianxiang Jiang, Kunchang Li, Chenting Wang |
FluxViT enhances video model efficiency by optimizing input token selection and sampling for varied computational budgets. The main research question is how to maximize input information across budgets, addressing sub-optimal accuracy-computation trade-offs in video models. The key methodology, termed Flux, uses flexible video sampling and token selection, integrated with a masked alignment strategy in a teacher-student training framework. FluxViT-S outperforms InternVideo2-S by 2.2% on K400 with standard computation and achieves comparable performance with only 10% of the inference cost. AI practitioners can leverage Flux for training robust video models adaptable to diverse deployment scenarios, achieving state-of-the-art performance with significantly reduced computational requirements. |
| MagicID: Hybrid Preference Optimization for ID-Consistent and |
|
|
| Dynamic-Preserved Video Customization (Read more on arXiv or HuggingFace) |
Hongwei Yi, Tianyang Wang, Xi Xiao, Lifan Jiang, Hengjia Li |
MagicID is a framework for generating personalized videos that maintain consistent identity and exhibit natural dynamics based on user-provided reference images. The main research objective is to address identity degradation and reduced dynamics in customized video generation caused by reliance on self-reconstruction training with static images. The key methodology involves constructing pairwise preference video data with explicit identity and dynamic rewards, and a hybrid sampling strategy that prioritizes identity preservation and then enhances dynamic motion. The primary results show MagicID achieves a mean identity similarity score of 0.600, outperforming existing methods while preserving motion dynamics. The principal implication for AI practitioners is that using hybrid preference optimization with tailored rewards can improve the quality of identity-preserved video customization, enabling more realistic and personalized video generation. |
| Reinforcement Learning for Reasoning in Small LLMs: What Works and What |
|
|
| Doesn’t (Read more on arXiv or HuggingFace) |
Chris Ngo, quyanh |
This study investigates reinforcement learning (RL) for improving reasoning in small language models (LLMs) under resource constraints. The main research question is how small LLMs behave when fine-tuned with RL under strict computational and time limitations, and whether their reasoning performance can be improved using an RL approach similar to DeepSeek-R1. The key methodology involves adapting the Group Relative Policy Optimization (GRPO) algorithm and curating a compact, high-quality mathematical reasoning dataset, then training a 1.5-billion-parameter model (DeepSeek-R1-Distill-Qwen-1.5B) on 4 GPUs within 24 hours. A primary result is that the model achieved an AIME24 score of 46.7% with only 7,000 training samples and a $42 training cost, surpassing the o1-preview model. This implies AI practitioners can achieve substantial reasoning gains in small LLMs using RL with limited data and computational resources, offering a cost-effective alternative to large-scale approaches. |
| Improving Autoregressive Image Generation through Coarse-to-Fine Token |
|
|
| Prediction (Read more on arXiv or HuggingFace) |
Michael Qizhe Shieh, Kaipeng Zhang, Ziyao Guo |
This paper introduces a coarse-to-fine framework for autoregressive image generation that alleviates vocabulary redundancy in large codebooks. The main research objective is to maintain the benefits of large codebooks for high-quality image reconstruction while simplifying the autoregressive modeling task. The key methodology involves clustering similar VQ-VAE codebook tokens into coarse labels, predicting coarse labels autoregressively, and then predicting fine-grained tokens in parallel using full attention. The primary results include an average improvement of 59 points in Inception Score compared to baselines, reduced FID, and faster sampling speeds despite adding an auxiliary network. For AI practitioners, this method allows more efficient autoregressive image generation by reducing the effective vocabulary size, facilitating faster training and improved image quality when using large codebooks. |
| Deceptive Humor: A Synthetic Multilingual Benchmark Dataset for Bridging |
|
|
| Fabricated Claims with Humorous Content (Read more on arXiv or HuggingFace) |
Sunil Saumya, Shankar Biradar, UVSKKR |
Here’s a concise summary of the research paper, adhering strictly to your guidelines: This paper introduces the Deceptive Humor Dataset (DHD), a new synthetic multilingual benchmark for studying humor derived from fabricated claims and misinformation. The main research objective is to establish a structured foundation for analyzing humor in deceptive contexts and to understand how humor influences the perception and spread of misinformation. The key methodology involves generating 9,000 humor-infused comments using ChatGPT-4o, labeled with satire levels (1-3) and humor attributes (Irony, Absurdity, Social Commentary, Dark Humor, Wordplay) across multiple languages and code-mixed variants. Primary results show that mBART achieved the best performance for Satire Level Classification with an accuracy of 51.00%, while BERT performed best on Humor Attribute Classification with an accuracy of 40.44%. The principal implication for AI practitioners is the availability of a structured dataset and established baselines to benchmark and advance deceptive humor detection models, a critical aspect in mitigating the spread of harmful narratives. |
| VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting |
|
|
| Generation with Flexible Pose and Multi-View Joint Modeling (Read more on arXiv or HuggingFace) |
Hyungjin Chung, Byung-Hoon Kim, Hyelin Nam, Byeongjun Park, Hyojun Go |
VideoRFSplat is a text-to-3D Gaussian Splatting model that generates real-world scenes with flexible camera poses and multi-view image consistency, eliminating the need for per-scene optimization or external refinement models. The main objective is to develop a direct text-to-3D generation model capable of handling diverse camera poses and unbounded scenes without relying on score distillation sampling (SDS) refinement. The methodology utilizes a dual-stream architecture with a video generation model and a side-attached pose generation model, communicating via cross-attention and employing an asynchronous sampling strategy. The primary result is that VideoRFSplat achieves a FID of 30.33 and CLIP score of 33.0 on MVImgNet, outperforming existing direct text-to-3D methods that use SDS refinement. The principal implication is that AI practitioners can directly generate realistic and coherent 3D scenes from text prompts without needing post-hoc refinement, simplifying the 3D generation pipeline and potentially improving efficiency. |
| Sonata: Self-Supervised Learning of Reliable Point Representations (Read more on arXiv or HuggingFace) |
Chris Xie, Tianwei Shen, Duncan Frost, Daniel DeTone, Xiaoyang Wu |
Sonata is a self-supervised learning framework for 3D point cloud representations that addresses limitations of existing approaches. The main research question is whether a reliable self-supervised point cloud model can be developed for diverse 3D tasks via simple linear probing, even with limited data. The key methodology involves a point self-distillation framework that obscures spatial information and emphasizes input features, training on 140k point cloud scenes. A primary result is that Sonata triples linear probing accuracy on ScanNet semantic segmentation compared to previous methods, achieving 72.5% mIoU with less than 0.2% learnable parameters. The principal implication is that AI practitioners can leverage Sonata as a reliable foundation model for various 3D perception tasks, achieving strong performance and data efficiency, even with limited labeled data, by using it as initialization and then employing simple linear probing. |
| BigO(Bench) – Can LLMs Generate Code with Controlled Time and Space |
|
|
| Complexity? (Read more on arXiv or HuggingFace) |
Gabriel Synnaeve, Benoit Sagot, Baptiste Roziere, pierrechambon |
BIGO(BENCH) is a new benchmark for evaluating the ability of large language models (LLMs) to generate code with specified time and space complexity constraints. The main objective is to assess LLMs’ capacity to understand and control computational complexity in code generation. The methodology involves a dynamic complexity inference framework to analyze Python functions, a dataset of 3,105 coding problems and 1,190,250 solutions with inferred complexity labels, and evaluations of LLMs on complexity prediction, generation, and coefficient ranking. The results show that DEEPSEEK-R1 LLAMA 70B achieved 4.8% and 3.4% All@1 on time and space complexity generation, respectively, revealing challenges in handling complexity requirements. The main implication for AI practitioners is that while LLMs show proficiency in program synthesis, controlling and reasoning about time and space complexity remains a significant challenge, indicating a need to improve models on abstract thinking about code. |
| See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language |
|
|
| Balance to Mitigate Dominant Modality Bias (Read more on arXiv or HuggingFace) |
YoungBin Kim, Juhwan Choi, Eunju Lee, MiHyeon Kim, JuneHyoung Kwon |
Vision-language (VL) models exhibit a “dominant modality bias,” disproportionately relying on one modality, which BALGRAD mitigates by reweighting and projecting gradients. The research analyzes model behavior under dominant modality bias, showing how unaligned gradients and differences in gradient magnitudes hinder balanced loss convergence. The proposed BALGRAD framework employs inter-modality gradient reweighting (adjusting KL divergence gradient based on modality contribution) and inter-task gradient projection. Experiments on UPMC Food-101, Hateful Memes, and MM-IMDb datasets demonstrate BALGRAD’s effectiveness; on UPMC Food-101, BALGRAD improved performance on the weak (text) modality by 12.5%p compared to the baseline. AI practitioners can use BALGRAD to create more robust VL models that effectively utilize both modalities, even when one is impaired, reducing reliance on a single dominant modality. |
| AIMI: Leveraging Future Knowledge and Personalization in Sparse Event |
|
|
| Forecasting for Treatment Adherence (Read more on arXiv or HuggingFace) |
Hassan Ghasemzadeh, Diane J. Cook, ab9mamun |
AIMI, a knowledge-guided system, forecasts medication adherence by leveraging sensor data, medication history, and future knowledge. The main research objective was to determine the impact of future knowledge and personalization on the accuracy of sparse event forecasting for treatment adherence. The key methodology involved training and evaluating CNN and LSTM models with various combinations of input features, including sensor data, adherence history, and “future knowledge” (prescribed medication times), along with an incremental learning algorithm. The LSTM models achieved an accuracy of 0.932 and an F-1 score of 0.936, and leveraging future knowledge improved the F-1 score by almost 112% when only high-sampled features and future knowledge data were used. For AI practitioners, the results demonstrate that incorporating readily available future knowledge, such as scheduled events, can significantly enhance the performance of sparse event forecasting models in time-series prediction, especially in resource-constrained environments. |
Papers for 2025-03-20
| Title |
Authors |
Summary |
| φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time |
|
|
| Exploration and Exploitation (Read more on arXiv or HuggingFace) |
Qika, haitengzhao, changma, Meituannnnnn, xufangzhi |
Φ-Decoding is a novel inference-time optimization algorithm that balances exploration and exploitation in large language model reasoning. The main research objective is to develop an efficient inference-time strategy that achieves globally optimal step estimation without external auxiliary models. The key methodology is “foresight sampling,” which leverages simulated future steps to derive two distributions (advantage and alignment) for optimal step selection, combined with in-width and in-depth pruning strategies for adaptive computation. Primary results show that Φ-Decoding improves the average reasoning performance of LLaMA3.1-Instruct-8B by over 14% across various reasoning benchmarks compared to auto-regressive CoT. For AI practitioners, Φ-Decoding offers a training-free method to improve LLM reasoning performance while balancing computational cost. |
| DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement |
|
|
| Learning (Read more on arXiv or HuggingFace) |
yikaiwang, NTU-yiwen, guangce, yejunliang23, zzzrw |
DeepMesh is a framework for generating artist-like 3D triangle meshes conditioned on point clouds and images using an auto-regressive transformer and reinforcement learning. The main research objective is to generate high-quality, aesthetically pleasing meshes with precise topology that align with human preferences, overcoming limitations of existing auto-regressive methods. The key methodology involves an improved mesh tokenization algorithm that reduces sequence length by 72%, a data curation strategy, and Direct Preference Optimization (DPO) with a scoring standard combining 3D metrics and human evaluation. Results show that DeepMesh outperforms state-of-the-art methods, achieving a Chamfer Distance of 0.0884 and a user preference score of 37% on a test dataset. AI practitioners can use DeepMesh’s improved tokenization and DPO implementation to efficiently generate more aesthetically refined 3D meshes, with geometric accuracy for various applications. |
| TULIP: Towards Unified Language-Image Pretraining (Read more on arXiv or HuggingFace) |
XuDong Wang, Seun Eisape, Long Lian, yala, ZinengTang |
TULIP is a contrastive image-text model that enhances visual feature learning while preserving language grounding. The main research objective is to improve the learning of general-purpose visual features in contrastive image-text models, addressing limitations in fine-grained visual understanding. The methodology leverages generative data augmentation, enhanced image-image and text-text contrastive learning, and image/text reconstruction regularization. TULIP achieved a zero-shot ImageNet-1K top-1 accuracy of 85.3%, surpassing existing models like SigLIP 2. AI practitioners can use TULIP as a drop-in replacement for existing CLIP-like models to achieve state-of-the-art performance on tasks requiring fine-grained visual understanding and improved vision-language representation. |
| Cube: A Roblox View of 3D Intelligence (Read more on arXiv or HuggingFace) |
Karun Channa, Nishchaie Khanna, Kiran Bhat, Foundation AI Team, marcelvanworkum |
This paper introduces a 3D shape tokenization method for building a foundation model for 3D intelligence on the Roblox platform. The main research objective is to develop a method for converting 3D shapes into discrete tokens that can be used in multi-modal autoregressive sequence models. The key methodology involves a Perceiver-based transformer with Phased-Modulated Positional Encoding, optimal-transport vector quantization, and a stochastic gradient shortcut, trained with a self-supervised loss. Primary results show that the proposed method, Ours-VQ, achieves a 91.7% surface-IoU and 94.5% volumetric-IoU on the Toys4K dataset, surpassing other existing methods such as Craftsman. The principal implication for AI practitioners is that this shape tokenization method enables the development of various 3D generative applications, including text-to-shape, shape-to-text, and text-to-scene generation, allowing for better integration of 3D shapes into large language models. |
| Efficient Personalization of Quantized Diffusion Model without |
|
|
| Backpropagation (Read more on arXiv or HuggingFace) |
Se Young Chun, Kyungryeol Lee, Wongi Jeong, Agorium |
ZOODiP enables memory-efficient personalization of quantized diffusion models using only forward passes. The research objective is to reduce the memory demands of diffusion model personalization on edge devices without relying on backpropagation. The key methodology combines zeroth-order optimization with a quantized diffusion model, subspace gradient projection, and partial uniform timestep sampling. The primary results show that ZOODiP achieves comparable performance to prior methods in image and text alignment scores, while reducing training memory demand up to 8.2x (2.37GB VRAM consumption). AI practitioners can leverage this approach for diffusion model personalization in memory-constrained environments, enabling on-device training with significantly reduced resources. |
| Temporal Regularization Makes Your Video Generator Stronger (Read more on arXiv or HuggingFace) |
Yajing Bai, Yexin Liu, Xianfeng Wu, Haojian Huang, Harold328 |
FLUXFLOW enhances temporal coherence and diversity in video generation by applying controlled temporal perturbations during training. The main research question is whether temporal augmentation, specifically the proposed FLUXFLOW strategy, can improve the temporal quality of generated videos while maintaining spatial fidelity. FLUXFLOW introduces frame-level and block-level temporal perturbations to video data during the training of video generation models, without architectural changes. Experiments on UCF-101 and VBench show that FLUXFLOW applied to VideoCrafter2 improves the FVD score by 19.21 while improves Total Score to 82.36, a 1.92 improvement, enhancing both temporal coherence and diversity without reducing spatial fidelity. AI practitioners can integrate FLUXFLOW as a plug-and-play data augmentation strategy to improve the temporal quality of various video generation models. |
| STEVE: AStep Verification Pipeline for Computer-use Agent Training (Read more on arXiv or HuggingFace) |
Chi-Wing Fu, Shu Liu, Ziqin Wei, Zhisheng Zhong, Fanbin Lu |
STEVE is a step verification pipeline designed to train computer-use agents using a large, verified instruction set and trajectory data. The main research objective is to develop a scalable training pipeline for computer-use agents that overcomes the limitations of behavior cloning, which requires vast, high-quality trajectories. The key methodology involves establishing a large instruction set, collecting trajectory data with suboptimal agents, using GPT-4o to verify the correctness of each step based on before-and-after screen states, and then employing Kahneman & Tversky Optimization (KTO). A primary result is that the STEVE-trained 7B vision-language model achieved a 23% task success rate on the challenging WinAgentArena live environment using KTO, surpassing the performance of supervised finetuning. The principal implication for AI practitioners is that using step verification with KTO allows training of effective computer-use agents from sub-optimal trajectory data, which scales better and performs better. |
| LEGION: Learning to Ground and Explain for Synthetic Image Detection (Read more on arXiv or HuggingFace) |
Weijia Li, Junyan Ye, Siwei Wen, zichenwen, khr0516 |
The paper introduces SynthScars, a new dataset for synthetic image detection, and LEGION, a multimodal large language model-based framework for analyzing and refining synthetic images. The main research objective is to develop a model capable of detecting, localizing, and explaining artifacts in fully synthetic images, and to explore its use as a controller for improving image generation. The key methodology involves using a multimodal large language model (MLLM) to integrate artifact detection, segmentation, and explanation, and then applying this in iterative image regeneration and inpainting pipelines. Primary results show that LEGION outperforms existing methods on SynthScars, achieving a 3.31% higher mIoU and 7.75% higher F1 score than the second-best traditional expert, and demonstrates superior robustness. For AI practitioners, LEGION provides a new approach and benchmark for synthetic image analysis, and suggests how deep learning based image detection models can be integrated into the generative process to achieve higher quality of image synthesis. |
| MusicInfuser: Making Video Diffusion Listen and Dance (Read more on arXiv or HuggingFace) |
Steven M. Seitz, Brian Curless, Ira Kemelmacher-Shlizerman, Susung Hong |
MusicInfuser adapts existing text-to-video diffusion models to generate dance videos synchronized to music, while preserving text-based control over style. The main research objective is to adapt pre-trained text-to-video models to condition on music tracks and generate synchronized dance outputs. The key methodology involves introducing lightweight music-video cross-attention and a low-rank adapter within a video diffusion model, trained on dance videos, without requiring motion capture data. The method achieved a Dance Quality Average score of 7.95, outperforming baselines like Mochi (7.70) and MM-Diffusion (7.16) in comprehensive evaluations including factors like style and beat alignment. AI practitioners can adapt pre-existing video diffusion models for music-driven video generation by incorporating audio features via cross-attention and low-rank adapters, without extensive multimodal training. |
| GKG-LLM: A Unified Framework for Generalized Knowledge Graph |
|
|
| Construction (Read more on arXiv or HuggingFace) |
Jun Liu, haiping Zhu, Shihao Qi, Bifan Wei, VentureZJ |
This paper introduces GKG-LLM, a unified framework for constructing generalized knowledge graphs (GKGs), encompassing knowledge graphs, event knowledge graphs, and commonsense knowledge graphs. The main research objective is to develop a unified framework for constructing generalized knowledge graphs (GKGs) that overcomes task-specific differences and integrates knowledge from various graph types. The key methodology is a three-stage curriculum learning fine-tuning framework that iteratively injects knowledge from knowledge graphs (KGs), event knowledge graphs (EKGs), and commonsense knowledge graphs (CKGs) into a Large Language Model (LLM), using the LoRA+ technique. The primary result is that GKG-LLM achieved an average performance of 67.90% across all tasks, outperforming the strongest baseline by 7.49%, and specifically achieved 80.63% on the NYT sentence-level relation extraction task. AI practitioners can leverage the GKG-LLM framework for improved and generalized knowledge graph construction across various domains, achieving state-of-the-art performance with a single, unified model. |
| Mitigating Visual Forgetting via Take-along Visual Conditioning for |
|
|
| Multi-modal Long CoT Reasoning (Read more on arXiv or HuggingFace) |
Han-Jia Ye, Houwen Peng, Zhun Sun, Allen8 |
The paper introduces “Take-along Visual Conditioning” (TVC) to address visual forgetting in multi-modal large language models (MLLMs) during long-chain reasoning. The main research question is how to mitigate the decline in attention to visual information in MLLMs as reasoning progresses. The key methodology involves shifting image input to critical reasoning stages and compressing visual tokens via dynamic pruning, combined with Dynamic Visual Reaffirmation (DVR) and Periodic Visual Calibration (PVC). The primary result shows that the TVC approach achieves state-of-the-art performance, with a +3.4% average improvement over previous methods across five mathematical reasoning benchmarks. For AI practitioners, TVC offers a method to improve multi-modal reasoning performance in MLLMs by sustaining visual attention, applicable to tasks like geometric problem-solving. |
| Unlock Pose Diversity: Accurate and Efficient Implicit Keypoint-based |
|
|
| Spatiotemporal Diffusion for Audio-driven Talking Portrait (Read more on arXiv or HuggingFace) |
Chenru Jiang, Yuyao Yan, weiguangzhao, KaiserYaoJM, ChaolongYang |
KDTalker is a novel framework that generates audio-driven talking portrait videos using implicit keypoint-based spatiotemporal diffusion. The main research objective is to generate talking head videos with accurate lip synchronization and diverse head poses while maintaining computational efficiency. The methodology combines unsupervised implicit 3D keypoints with a spatiotemporal diffusion model and a custom-designed spatiotemporal attention mechanism. Primary results show that KDTalker achieves a LSE-C score of 7.326 and a head pose diversity of 0.760 on the HDTF dataset, outperforming existing methods. For AI practitioners, KDTalker offers a method for creating realistic talking portrait animations suitable for real-time applications with improved pose diversity and lip-sync accuracy. |
| ELTEX: A Framework for Domain-Driven Synthetic Data Generation (Read more on arXiv or HuggingFace) |
Eugene Dmitriev, Julien Capitaine, Sofia Sedlova, Kseniia Murasheva, lavriz |
ELTEX is a framework for generating high-quality synthetic training data in specialized domains, like blockchain-related cyberattack detection. The main research objective is to address the scarcity of domain-specific training data in specialized fields like cybersecurity, which limits the performance of Large Language Models (LLMs). ELTEX systematically integrates explicit domain indicator extraction with dynamic prompting to preserve critical domain knowledge during the generation process. Fine-tuning Gemma-2B with ELTEX-generated data, combined with real data, achieved an F1-score of 0.81, competitive with GPT-4. The principal implication is that AI practitioners can use domain-driven synthetic data generation to bridge the performance gap between smaller, more efficient models, and larger models, in specialized domains. |
Papers for 2025-03-19
| Title |
Authors |
Summary |
| RWKV-7 “Goose” with Expressive Dynamic State Evolution (Read more on arXiv or HuggingFace) |
saitejautpala, Guangyu, SmerkyG, ZhangRC, BlinkDL |
RWKV-7 “Goose” is a new sequence modeling architecture with pre-trained language models that introduces a generalized delta rule with vector-valued gating for improved performance. The main research objective is to develop a sequence modeling architecture that achieves state-of-the-art performance while maintaining efficiency in terms of memory usage and inference time. The key methodology involves a generalized formulation of the delta rule with vector-valued gating, in-context learning rates, and a relaxed value replacement rule, integrated into a modified RWKV-6 architecture. Primary results show that RWKV-7 models achieve state-of-the-art multilingual performance at the 3 billion parameter scale, matching current SoTA English language performance while requiring only constant memory usage and inference time per token; and on English-focused benchmarks the RWKV7-World3-2.9B achieved 71.5 average accuracy. AI practitioners can use RWKV-7 models as efficient alternatives to Transformers, benefiting from reduced inference costs and constant memory usage, particularly beneficial for long-sequence applications. |
| Impossible Videos (Read more on arXiv or HuggingFace) |
Hai Ci, mikeshou, ZechenBai |
This paper introduces IPV-BENCH, a benchmark for evaluating video generation and understanding models on impossible or counterfactual video content. The main research questions are whether current video generation models can create impossible videos from prompts and whether video understanding models can comprehend them. The key methodology involved creating a taxonomy of impossible video types, generating a dataset of text prompts (IPV-TXT) and videos (IPV-VID), and evaluating various models on tasks including video generation, judgment, multiple-choice question answering, and open-ended question answering. A key finding is that the top-performing video generation model, Mochi 1, generated high-quality impossible videos in only 37.3% of cases. This demonstrates the need for significant improvement in video models’ ability to generate and understand non-real-world scenarios, providing AI practitioners a clear benchmark and identified limitations to guide the development of more robust and creative video models. |
| Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM (Read more on arXiv or HuggingFace) |
Yingji Liang, Shengyuan Ding, Kai Lan, Zhijian Chen, Xinyu Fang |
Creation-MMBench is a new benchmark for evaluating the visual creative capabilities of Multimodal Large Language Models (MLLMs) in real-world, image-based tasks. i) Main research question or objective: To introduce and evaluate Creation-MMBench, a multimodal benchmark designed to assess the creative capabilities of MLLMs in real-world, image-based tasks. ii) Key methodology used: Creation-MMBench comprises 765 test cases across 51 fine-grained tasks, with instance-specific evaluation criteria for assessing response quality and factual consistency with visual inputs, using MLLM-as-a-Judge (GPT-4o) methodology. iii) Primary results: Current open-source MLLMs significantly underperform compared to proprietary models in creative tasks; for instance, Qwen2.5-VL-72B-Instruct achieved a reward of -5.82 and visual factuality score of 8.33 on the overall benchmark, while Gemini-2.0-pro-exp achieved a reward of 4.48 and visual factuality of 8.53. iv) Principal implication for AI practitioners: AI practitioners should address the limitations of current MLLMs in context-aware creativity and visual-based language generation, and focus on developing more comprehensive and fine-grained evaluation criteria, recognizing that visual fine-tuning can negatively impact the base LLM’s creative abilities. |
| DAPO: An Open-Source LLM Reinforcement Learning System at Scale (Read more on arXiv or HuggingFace) |
Xiaochen Zuo, Yufeng Yuan, Ruofei Zhu, Zheng Zhang, Qiying Yu |
DAPO is an open-source system for large-scale reinforcement learning (RL) with language models (LLMs), achieving state-of-the-art results on mathematical reasoning. The main research objective is to develop and open-source a scalable and reproducible RL system for LLMs that addresses limitations in existing approaches and reproduces industry-level RL results. The key methodology is the Decoupled Clip and Dynamic sampling Policy Optimization (DAPO) algorithm, incorporating techniques like Clip-Higher, Dynamic Sampling, Token-Level Policy Gradient Loss, and Overlong Reward Shaping, built upon the verl framework. The primary result is that DAPO achieves 50 points on AIME 2024 using a Qwen2.5-32B base model, surpassing previous state-of-the-art results with 50% fewer training steps. Principal implications for AI practitioners include that this paper presents a fully open-sourced algorithm, training code and dataset, providing techniques to solve problems like reward noise and training instability for reinforcement learning. |
| DeepPerception: Advancing R1-like Cognitive Visual Perception in MLLMs |
|
|
| for Knowledge-Intensive Visual Grounding (Read more on arXiv or HuggingFace) |
Zonghao Guo, Zhicong Luo, carboncoo, sdudzy, MaxyLee |
DeepPerception enhances Multimodal Large Language Models (MLLMs) for knowledge-intensive visual grounding by integrating cognitive reasoning with visual perception. The research introduces and addresses the challenge of knowledge-intensive visual grounding (KVG), requiring fine-grained perception and domain knowledge integration in MLLMs. The methodology involves a two-stage training framework: supervised fine-tuning for cognitive reasoning and reinforcement learning to optimize perception-cognition synergy, using an automated data synthesis pipeline. DeepPerception achieved an 8.08% accuracy improvement on the new KVG-Bench compared to direct fine-tuning, also showcasing +4.60% superior cross-domain generalization. AI practitioners can leverage DeepPerception’s training framework and the KVG-Bench dataset to develop MLLMs with improved cognitive visual perception, enabling more human-like visual understanding in AI systems. |
| CapArena: Benchmarking and Analyzing Detailed Image Captioning in the |
|
|
| LLM Era (Read more on arXiv or HuggingFace) |
Qiushi Sun, Zheng Ma, Jiaxin Fan, songwp, cckevinn |
CapArena benchmarks detailed image captioning with large language models (LLMs) through human evaluations and analyzes automated metrics. The main research questions are how well current Vision-Language Models (VLMs) perform on detailed image captioning compared to humans, and how reliably automated metrics can assess detailed caption quality. The key methodology involved creating CapArena, a platform with over 6000 pairwise caption battles with human preference votes, and evaluating various traditional and recent captioning metrics against these human annotations. Primary results showed that top models like GPT-4o achieve or surpass human-level performance, and the VLM-as-a-Judge approach correlated with human rankings at 94.3% at $4 per test. AI practitioners should use VLM-as-a-Judge for efficient and reliable evaluation of detailed image captioning models, as it aligns better with human preference than traditional metrics. |
| Infinite Mobility: Scalable High-Fidelity Synthesis of Articulated |
|
|
| Objects via Procedural Generation (Read more on arXiv or HuggingFace) |
Li Ray Luo, Yitong Wang, Ruiming Liang, Zichao Yu, Xinyu Lian |
Infinite Mobility is a procedural pipeline for synthesizing large-scale, high-fidelity 3D articulated objects. The main research objective is to develop a method for generating high-quality articulated objects that overcomes the limitations of existing data-driven and simulation-based approaches. The key methodology utilizes a tree-growing strategy for articulation structure generation, combined with procedural mesh generation or dataset retrieval with refinement, and ensures physical plausibility through constraint rules. The primary results show that the method produces objects comparable to human-annotated datasets, with an average Tree Edit Distance of 78.62 compared to 3.88 of PartNet-Mobility, and outperforms existing generative models in both physical property and mesh quality evaluations. The principal implication for AI practitioners is that the proposed pipeline provides a scalable and high-fidelity data source for training embodied AI agents and generative models, facilitating tasks requiring interaction with articulated objects. |
| Frac-Connections: Fractional Extension of Hyper-Connections (Read more on arXiv or HuggingFace) |
Jundong Zhou, Hongzhi Huang, Defa Zhu, Taoer, FetchFortune |
Frac-Connections are introduced as a memory-efficient alternative to Hyper-Connections for deep learning models. The main research objective is to address the seesaw effect between gradient vanishing and representation collapse in residual connections without increasing memory access costs. The key methodology is to divide hidden states into multiple parts (fractional expansion), rather than expanding their width, and construct fractional connection strengths. Primary results show that OLMoE-7B-DFC×4 models achieve a training loss reduction of 0.012 and outperform the baseline by +0.95% on WinoGrande. The principal implication for AI practitioners is that Frac-Connections can improve training stability and downstream task performance in large language models with minimal parameter overhead. |
| Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal |
|
|
| Control (Read more on arXiv or HuggingFace) |
Tiffany Cai, Maciej Bala, Jose Alvarez, Hassan Abu Alhaija, NVIDIA |
Cosmos-Transfer1 is a diffusion-based conditional world model that generates videos based on multiple spatial control inputs with an adaptive weighting scheme. The main research objective is to develop a highly controllable world generation model that can leverage multimodal inputs (segmentation, depth, edge) to produce high-quality and diverse simulations. The key methodology involves adding multiple ControlNet branches to a diffusion transformer-based world model (Cosmos-Predict1), training these branches separately, and fusing them with spatiotemporal control maps during inference. Primary results include a Blur SSIM of 0.87 and a Quality Score of 8.54 on the TransferBench evaluation when using uniform weights across all modalities, outperforming single-modality baselines. Principal implication for AI practitioners is that Cosmos-Transfer1 provides a framework for generating high-fidelity and controllable simulations useful in applications requiring diverse and controllable environments, such as robotics Sim2Real transfer and autonomous vehicle data enrichment, where it achieves a real-time generation of a 5-second video in 4.2 seconds. |
| MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process |
|
|
| Errors Identification (Read more on arXiv or HuggingFace) |
Kai Wang, Wangbo Zhao, Jiaxin Ai, Pengfei Zhou, Zhaopan Xu |
MPBench is a new benchmark for evaluating multimodal process reward models (PRMs) across diverse reasoning tasks. The main research objective is to systematically assess the effectiveness of PRMs in diverse reasoning scenarios using multi-task, multimodal data. The key methodology involves three evaluation paradigms: Step Correctness, Answer Aggregation, and Reasoning Process Search, applied to a dataset of 9,745 instances across six sub-categories. A primary result is that the state-of-the-art model, GPT-4o, achieved an overall score of 71.2, while weaker models like Qwen2.5-VL-3B scored below random chance on some assessments. The principal implication for AI practitioners is that current multimodal PRMs, even advanced ones, struggle with complex reasoning tasks, indicating a need for improved model capacity and training strategies specifically for process-level supervision and multimodal understanding. |
| Aligning Multimodal LLM with Human Preference: A Survey (Read more on arXiv or HuggingFace) |
Jinda Lu, Junkang Wu, Chaoyou Fu, Tao Yu, yifanzhang114 |
This survey provides a comprehensive and systematic review of alignment algorithms for multimodal large language models (MLLMs). The main research question is how to categorize and understand the current advancements in aligning MLLMs with human preferences, focusing on application scenarios, dataset construction, and evaluation benchmarks. The key methodology involves a systematic literature review, categorizing existing methods based on application scenarios (general image understanding, complex modalities, extended applications), dataset construction factors (data sources, model responses, preference annotations), and evaluation benchmarks. The primary result found 13 benchmarks used in current MLLM alignment research, and no publicly available, fully human-annotated dataset over 200,000 samples. The principal implication for AI practitioners is the need for developing more efficient methods to balance dataset scalability with quality and find new methods that efficiently use visual information in alignment, moving beyond current limitations. |
| Measuring AI Ability to Complete Long Tasks (Read more on arXiv or HuggingFace) |
Katharyn Garcia, Amy Deng, Joel Becker, Ben West, Thomas Kwa |
The paper introduces a metric to quantify AI capabilities on long tasks, finding exponential growth in AI task completion time horizon. The main research objective is to quantify AI capabilities in terms of human capabilities, and track the progress. The authors measured human and AI performance on a new dataset of 170 software engineering, cybersecurity, machine learning, and general reasoning tasks, and fit a logistic model to estimate the “50%-task-completion time horizon” for each AI model. Results show the 50% time horizon for frontier AI models like Claude 3.7 Sonnet is around 50 minutes, and has been doubling approximately every seven months since 2019. For AI practitioners, the time horizon metric and trend provide a quantitative framework to assess and forecast AI agent capabilities for performing complex, real-world, long-duration tasks. |
| Concat-ID: Towards Universal Identity-Preserving Video Synthesis (Read more on arXiv or HuggingFace) |
Chongxuan Li, Xiaotao Gu, Jiayan Teng, Zhuoyi Yang, Yong Zhong |
Concat-ID is a unified framework for identity-preserving video generation that scales to multiple identities and subjects. The main research objective is to develop a framework that achieves a balance between maintaining identity consistency and facial editability in generated videos, without needing extra modules or parameters. The key methodology uses Variational Autoencoders (VAEs) to extract image features, which are concatenated with video latents along the sequence dimension, leveraging solely 3D self-attention mechanisms, combined with a cross-video pairing strategy and a multi-stage training regimen. Primary results show that Concat-ID achieves an ArcSim score of 0.442 and a CLIPDist score of 0.325 for single-identity generation, superior to existing methods in both identity consistency and facial editiablity. Principal implication for AI practitioners is that a single and concise model is sufficient to achieve single-identity, multi-identity, and multi-subject preservation in video generation without additional modules. |
| Temporal Consistency for LLM Reasoning Process Error Identification (Read more on arXiv or HuggingFace) |
Xinzhe Juan, Kaixuan Huang, Jiahao Qiu, Yue Wu, Jiacheng Guo |
This paper introduces a temporal consistency method to improve large language models’ (LLMs) ability to identify errors in mathematical reasoning processes. The main research question is whether leveraging consistency in a sequence of self-reflection actions can improve verification accuracy in identifying mathematical process errors. The key methodology involves iterative self-checking by LLMs, where each LLM reviews its own verification results based on previous assessments until a stable result is achieved. Applying the method to DeepSeek R1 distilled models, improvements of 46.6% on MathCheck*, 37.9% on ProcessBench, and 29.0% on PRM800K with the 8B model. AI practitioners can use this temporal consistency approach to enhance the reliability of LLM-based verification systems, particularly for mathematical reasoning, by incorporating iterative self-reflection to reduce errors. |
| PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for |
|
|
| Multimodal Large Language Models (Read more on arXiv or HuggingFace) |
Wangbo Zhao, Jiaxin Ai, Weidong Tang, Pengfei Zhou, Zhaopan Xu |
PEBench is a new benchmark for evaluating machine unlearning in multimodal large language models, focusing on personal entities and events. The main research objective is to develop a standardized framework to assess the efficacy of machine unlearning (MU) methods in removing specific visual concepts (identity and event) from Multimodal Large Language Models (MLLMs) while preserving performance on unrelated concepts. The key methodology involves creating a synthetic dataset, PEBench, with 200 fictitious individuals and 40 event scenes, coupled with six MU methods, to evaluate unlearning efficacy, generality, and scope using metrics like precision, ROUGE-L, and G-Eval. A primary result is that while most MU methods achieve nearly 100% efficacy for people unlearning, the ROUGE-L score for event descriptions drops from 0.99 to an average of 0.88, showing an impact from the unlearning process in people to events. AI practitioners can use PEBench to systematically evaluate and improve MU methods for MLLMs, ensuring effective removal of specific concepts without degrading performance on unrelated tasks, particularly in privacy-sensitive applications. |
| MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs (Read more on arXiv or HuggingFace) |
Justin Lazarow, Haiming Gang, David Griffiths, Nina Wenzel, Erik Daxberger |
MM-Spatial introduces a new dataset and benchmark, CA-VQA, to improve 3D spatial understanding in multimodal large language models (MLLMs). The main research objective is to develop an MLLM, MM-Spatial, that excels at 3D spatial reasoning tasks using large-scale 3D scene data. The key methodology involves generating a supervised fine-tuning dataset, CA-VQA, from high-quality 3D scene data, and training MM-Spatial with diverse spatial tasks, metric depth, and multi-view inputs. MM-Spatial achieves state-of-the-art performance on 3D spatial understanding benchmarks, with a 70.1 average score on the CA-VQA spatial category. The principal implication is that AI practitioners can leverage the CA-VQA dataset and MM-Spatial model to enhance MLLMs’ 3D spatial reasoning capabilities, crucial for applications like robotics and AR/VR. |
| Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion |
|
|
| Transformers via In-Context Reflection (Read more on arXiv or HuggingFace) |
Yusuke Kato, Arsh Koneru, Akash Gokul, Konstantinos Kallidromitis, Shufan Li |
Reflect-DiT improves text-to-image generation by enabling Diffusion Transformers to iteratively refine outputs using past generations and textual feedback. The main research objective is to develop an inference-time scaling method for text-to-image diffusion models that improves image quality and text alignment without extensive retraining. The methodology, Reflect-DiT, uses a vision-language model to critique generated images and provide textual feedback, which a Diffusion Transformer then uses along with previous generations as in-context examples to refine subsequent outputs. Reflect-DiT achieved a new state-of-the-art score of 0.81 on the GenEval benchmark using only 20 samples per prompt. AI practitioners can use Reflect-DiT to improve the quality and prompt alignment of text-to-image diffusion models during inference, achieving better results with fewer samples compared to best-of-N sampling. |
| Florenz: Scaling Laws for Systematic Generalization in Vision-Language |
|
|
| Models (Read more on arXiv or HuggingFace) |
Sven Behnke, Sebastian Houben, Spravil |
Florenz investigates scaling laws for systematic generalization in vision-language models (VLMs) by training monolingual models on multilingual tasks with incomplete data coverage. The main research question is how model size and the number of seen training samples affect a monolingual VLM’s ability to generalize to unseen task-language pairs in a multilingual setting. The key methodology involves training a novel encoder-decoder VLM, Florenz, on a synthetic dataset with intentionally missing language coverage for image captioning, using a combination of pre-trained VLM (Florence-2) and LLM (Gemma-2) components. A primary result is that a 30B parameter model could achieve a cross-entropy loss of 2.31 on unseen captioning, and increasing model size has more significant effect on generalizaton than quantity of training samples. This result implies that AI practitioners can potentially achieve cross-lingual transfer in VLMs even with monolingual models by focusing on scaling model size, mitigating the need for exhaustive multilingual data collection for every task. |
| Pensez: Less Data, Better Reasoning – Rethinking French LLM (Read more on arXiv or HuggingFace) |
HoangHa |
Pensez 7B, a bilingual English-French language model, demonstrates competitive reasoning performance with significantly less training data than comparable models. The main research question is whether strategic fine-tuning on a small, high-quality, bilingual dataset can enhance both the reasoning capabilities and French language proficiency of a large language model. The key methodology involves supervised fine-tuning of a Qwen2.5 7B Instruct base model on a curated 2,000-example bilingual (English-French) dataset, emphasizing data quality, diversity, and explicit reasoning chains. Pensez 7B achieves a 12-point accuracy increase on a French MATH level 5 benchmark compared to the base model. The principal implication is that AI practitioners can achieve strong reasoning performance in LLMs with focused, high-quality datasets, reducing reliance on massive, resource-intensive training corpora. |
| Hyperbolic Safety-Aware Vision-Language Models (Read more on arXiv or HuggingFace) |
Rita Cucchiara, Lorenzo Baraldi, Pascal Mettes, Tejaswi Kasarla, tobi1modna |
HySAC introduces a novel approach to address unsafe content in vision-language models (VLMs) using hyperbolic space. The main research objective is to develop a VLM that can distinguish between safe and unsafe content without unlearning unsafe concepts, enabling controlled retrieval and classification. The key methodology involves encoding safe and unsafe image-text pairs in a hyperbolic space, employing entailment loss functions to model hierarchical relationships, and using a traversal mechanism to adjust query embeddings for safe or unsafe retrieval. Primary results show that HySAC achieves a recall of 49.8% at R@1 and 90.7% at R@20 for safe content retrieval on the ViSU test set, outperforming existing safety-unlearning CLIP and hyperbolic CLIP models. AI practitioners can use HySAC to build VLMs with enhanced safety awareness, allowing for dynamic control over content moderation and safer retrieval by design without removing information. |
| KUDA: Keypoints to Unify Dynamics Learning and Visual Prompting for |
|
|
| Open-Vocabulary Robotic Manipulation (Read more on arXiv or HuggingFace) |
Yunzhu Li, Mingtong Zhang, Zixian Liu |
KUDA is an open-vocabulary robotic manipulation system that integrates visual prompting and dynamics learning through a unified keypoint representation. The main research objective is to develop a system that can perform complex manipulation tasks based on free-form language instructions while accounting for object dynamics. The key methodology involves using a vision-language model (VLM) to generate keypoint-based target specifications from language instructions and RGBD observations, and then employing model-based planning with a learned dynamics model to achieve the specified goals. The system achieved an 80.0% success rate across 60 trials on various manipulation tasks, significantly outperforming baseline methods. AI practitioners can leverage KUDA’s unified keypoint representation to bridge vision-language models and dynamics models, enabling more flexible and robust robotic manipulation systems that can handle a wider variety of objects and tasks. |
| RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground |
|
|
| Simulation (Read more on arXiv or HuggingFace) |
Junhao Ge, Yifan Lu, Zichen Chao, Anning Hu, yuwendu |
RoCo-Sim is a simulation framework for improving roadside collaborative perception by generating diverse, multi-view consistent simulated data. The main research objective is to address data limitations in roadside collaborative perception, such as calibration errors, sparse data, and multi-view inconsistency, by developing a simulation framework. The key methodology involves using dynamic foreground editing and full-scene style transfer of single images, Camera Extrinsic Optimization, a Multi-View Occlusion-Aware Sampler (MOAS), DepthSAM, and a Scalable Post-Processing Toolkit. RoCo-Sim outperforms state-of-the-art methods on the Rcooper-Intersection dataset by 83.74% for AP70. AI practitioners can use RoCo-Sim to generate realistic and diverse roadside perception datasets, substantially enhancing the performance of camera-only 3D detection models without needing extensive real-world data collection or model architecture changes. |
Papers for 2025-03-18
| Title |
Authors |
Summary |
|
| DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal |
|
|
|
| Consistent Video Generation (Read more on arXiv or HuggingFace) |
Runze Zhang, NeilXu, EllenAP, lixiaochuan, georgedu |
DropletVideo introduces a new dataset and model for generating videos with integral spatio-temporal consistency, addressing plot coherence and visual consistency across viewpoints. The main research question is how to ensure integral spatio-temporal consistency in video generation, considering the interplay between plot progression, camera techniques, and prior content impact. The key methodology involves constructing a large-scale dataset (DropletVideo-10M) with detailed captions and developing a diffusion model (DropletVideo) with motion-adaptive generation. Primary results show DropletVideo achieves 37.93% in Camera Motion and 98.94% in Motion Smoothness on VBench++-ISTP benchmarks, indicating a strong ability of DropletVideo to generate videos with integral spatiotemporal consistency. AI practitioners can utilize the open-sourced DropletVideo dataset and model to advance video generation research and applications requiring robust spatio-temporal coherence, particularly multi-plot narratives. |
|
| Being-0: A Humanoid Robotic Agent with Vision-Language Models and |
|
|
|
| Modular Skills (Read more on arXiv or HuggingFace) |
tellarin, SherryXu, takenpeanut, fuyh, Yaya041 |
i) Being-0, a hierarchical framework, effectively controls a full-sized humanoid robot for complex embodied tasks by integrating a Foundation Model (FM) with a modular skill library. ii) The research aims to develop a humanoid robotic agent that can perform complex, long-horizon tasks efficiently and robustly in real-world environments. iii) The methodology involves using an FM for high-level planning, a VLM-based Connector module for bridging the gap between the FM and low-level skills, and a modular skill library for locomotion and manipulation. iv) Experiments demonstrate Being-0 achieves an 84.4% average completion rate on long-horizon tasks and 4.2x efficiency in navigation compared to fully FM-based agents when modules (except the FM) are deployed on onboard computation devices. v) The principal implication for AI practitioners is the demonstration of a hierarchical architecture using a lightweight VLM Connector which significantly enhances the embodied decision-making capabilities of humanoid robots and efficiently coordinates locomotion and manipulation. |
|
| DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale |
|
|
|
| Text-to-Image Models (Read more on arXiv or HuggingFace) |
Yi Yang, z-x-yang, aiJojosh, limuloo1999 |
DreamRenderer is a training-free approach for controlling attributes of multiple instances in image-conditioned text-to-image generation. The research aims to enable precise control over the content of individual instances or regions within images generated from textual descriptions and conditioning inputs like depth or canny maps. The key methodology involves “Bridge Image Tokens” for Hard Text Attribute Binding to correctly associate text embeddings with visual attributes, and selective application of “Hard Image Attribute Binding” in vital layers of the FLUX model. DreamRenderer improves the Image Success Ratio by 17.7% over FLUX on the COCO-POS benchmark and enhances performance of layout-to-image models like GLIGEN by up to 26.8%. AI practitioners can leverage DreamRenderer as a plug-and-play controller for fine-grained control over multi-instance image generation without additional training, enhancing controllability in applications like animation and game development. |
|
| Edit Transfer: Learning Image Editing via Vision In-Context Relations (Read more on arXiv or HuggingFace) |
Qi Mao, AnalMom, guyuchao, Orannue |
Edit Transfer introduces a new image editing paradigm that learns transformations from single source-target examples and applies them to new images. The main research question is whether an image editing transformation can be learned from a single source-target example and applied to a new query image. The key methodology is visual relation in-context learning, adapting a DiT-based text-to-image model with a four-panel composite input and lightweight LoRA fine-tuning. The primary result is that Edit Transfer outperforms state-of-the-art TIE and RIE methods in non-rigid editing scenarios, achieving a user preference rate exceeding 80% across all aspects in user studies. The principal implication is that AI practitioners can achieve sophisticated non-rigid image editing using minimal data (42 training images total) and a visual relation in-context learning approach, reducing the need for large-scale datasets and extensive training. |
|
| Personalize Anything for Free with Diffusion Transformer (Read more on arXiv or HuggingFace) |
Lu Sheng, Lin Li, Haoran Feng, lvhairong, huanngzh |
Personalize Anything is a training-free framework for personalized image generation in Diffusion Transformers (DiTs) that achieves high-fidelity subject reconstruction and flexible editing. The research aims to develop a training-free method for personalized image generation in DiTs that preserves identity and supports diverse editing scenarios. The key methodology involves timestep-adaptive token replacement with patch perturbation, injecting reference subject tokens in early denoising steps and transitioning to multi-modal attention in later steps. Evaluations on DreamBench demonstrate state-of-the-art performance, with the method achieving a CLIP-I score of 0.876 and a DreamSim score of 0.179 in single-subject personalization, surpassing existing approaches. AI practitioners can leverage this framework for efficient, high-fidelity personalized image generation and editing in DiTs without the need for training or fine-tuning, achieving superior identity preservation. |
|
| WideRange4D: Enabling High-Quality 4D Reconstruction with Wide-Range |
|
|
|
| Movements and Scenes (Read more on arXiv or HuggingFace) |
mingbao, zbhpku, Juanxi, czkk566, Lingaaaaaaa |
WideRange4D enables high-quality 4D scene reconstruction, including wide-range spatial movements of objects, by introducing a new benchmark and a two-stage reconstruction method. The main research objective is to address the limitations of existing 4D reconstruction methods and datasets in handling scenes with significant object spatial variations. The key methodology involves curating a new benchmark, WideRange4D, and proposing a two-stage 4D reconstruction method, Progress4D, which first initializes a high-quality 3D scene and then progressively fits 4D dynamics. Primary results show that Progress4D achieves a PSNR of 28.86 on the WideRange4D benchmark, outperforming existing state-of-the-art methods. The principal implication for AI practitioners is that WideRange4D provides a more challenging and comprehensive benchmark for evaluating 4D generation methods, while Progress4D offers a more stable and higher-quality approach for reconstructing complex 4D scenes with wide-range object movement. |
|
| BlobCtrl: A Unified and Flexible Framework for Element-level Image |
|
|
|
| Generation and Editing (Read more on arXiv or HuggingFace) |
HongxiangLi, daoyuan98, ZyZcuhk, l-li, Yw22 |
BlobCtrl is a unified framework for element-level image generation and editing using a probabilistic blob-based representation. The main research objective is to develop a method for precise and flexible manipulation of visual elements in images, overcoming limitations of current diffusion-based methods. The key methodology involves a dual-branch diffusion model with a blob-based representation, self-supervised training with data augmentation, and controllable dropout strategies. BlobCtrl achieves a significantly higher average CLIP-I score of 87.48 for identity preservation tasks, relative to the next best result. AI practitioners can use BlobCtrl for element-level image generation and editing, benefiting from its precise control over visual appearance and spatial layout that improves fidelity. |
|
| reWordBench: Benchmarking and Improving the Robustness of Reward Models |
|
|
|
| with Transformed Inputs (Read more on arXiv or HuggingFace) |
Yoon Kim, Andrew Cohen, mghazvininejad, michiyasunaga, ZhaofengWu |
Reward models (RMs) are brittle and their performance degrades substantially when inputs are transformed in meaning- or ranking-preserving ways. The main research objective is to evaluate and improve the robustness of state-of-the-art reward models against input transformations. Key methodology used involves creating reWordBench, a benchmark of transformed RewardBench instances, and regularizing RM training by encouraging similar scores for paraphrased inputs. Primary results show that RM ranking accuracy on RewardBench can drop by 15.3% on the Chat subset when transformed with reWordBench, and regularization reduces the drop to 7.9%. Principal implication for AI practitioners is that RMs need to be explicitly trained for robustness, such as through paraphrase regularization, to ensure reliable performance and avoid potential reward hacking in downstream alignment tasks. |
|
| MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based |
|
|
|
| Scientific Research (Read more on arXiv or HuggingFace) |
lundbergemma, chadliu, shcohen, suyc21, jmhb |
MicroVQA is a new benchmark for evaluating multimodal reasoning in AI, specifically for microscopy-based biological research. The main research objective is to assess AI models’ ability to perform expert visual understanding, hypothesis generation, and experiment proposal using microscopy images and associated questions. The key methodology involves curating a dataset of 1,042 multiple-choice questions (MCQs) created by biology experts, with a two-stage MCQ generation pipeline involving optimized LLM prompting and an agent-based “RefineBot” to remove language shortcuts. The primary result is that state-of-the-art multimodal large language models (MLLMs) achieve a peak performance of only 53% accuracy on the benchmark. For AI practitioners, this benchmark highlights the need for improved multimodal reasoning capabilities beyond language understanding, specifically in integrating visual information, prior scientific knowledge, and complex reasoning, suggesting that current models are far from expert-level scientific reasoning in this domain. |
|
| Free-form language-based robotic reasoning and grasping (Read more on arXiv or HuggingFace) |
Matteo Bortolon, Alice Fasoli, Runyu Jiao, SPovoli, FGiuliari |
FreeGrasp enables robots to perform grasping tasks based on free-form language instructions by leveraging Vision-Language Models (VLMs) for spatial reasoning. The research explores how pre-trained VLMs can interpret human instructions and understand spatial relationships for robotic grasping in a zero-shot setting. The proposed method, FreeGrasp, uses mark-based visual prompting and object keypoints to facilitate GPT-4o’s spatial reasoning about object arrangements and obstructions. Experiments on the new FreeGraspData dataset show FreeGrasp achieves a Reasoning Success Rate (RSR) of 0.83 without object ambiguity, outperforming the ThinkGrasp baseline. AI practitioners can use FreeGrasp’s approach, combining VLMs with visual prompting, to enhance robotic manipulation tasks requiring complex language understanding and spatial reasoning without the need for more training data. |
|
| R1-VL: Learning to Reason with Multimodal Large Language Models via |
|
|
|
| Step-wise Group Relative Policy Optimization (Read more on arXiv or HuggingFace) |
Jingyi Zhang, Xikun, liushunyu, HuanjinYao, huangjiaxing |
R1-VL introduces Step-wise Group Relative Policy Optimization (StepGRPO) to enhance reasoning in Multimodal Large Language Models (MLLMs). The research aims to improve MLLMs’ reasoning abilities beyond simply imitating successful reasoning paths, addressing the sparse reward issue in online reinforcement learning. StepGRPO uses online reinforcement learning with two novel rule-based rewards: Step-wise Reasoning Accuracy Reward (StepRAR) and Step-wise Reasoning Validity Reward (StepRVR), evaluating intermediate reasoning steps and logical structure. R1-VL, developed with StepGRPO, achieved a 63.5% accuracy on the MathVista benchmark, outperforming the baseline Qwen2-VL-7B by 3.8%. AI practitioners can use StepGRPO to train MLLMs with improved reasoning capabilities, achieving more reliable and structured outputs through a process that mitigates sparse reward issues without needing process reward models. |
|
| V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning (Read more on arXiv or HuggingFace) |
Wei Li, Ziquan Liu, ChenyangSi, lwpyh, Cade921 |
This paper introduces V-STaR, a new benchmark for evaluating Video-LLMs’ spatio-temporal reasoning abilities, including a dataset and evaluation metrics. The main research objective is to assess how well Video-LLMs can integrate spatial, temporal, and causal relationships in video understanding, moving beyond simple object recognition. The key methodology is a Reverse Spatio-Temporal Reasoning (RSTR) task that decomposes video understanding into “what”, “when”, and “where” questions, evaluated with coarse-to-fine Chain-of-Thought (CoT) questions generated by a semi-automated GPT-4-powered pipeline. Primary results show that while some models like GPT-4o perform well on “what” questions (60.78% accuracy), their performance on integrated spatio-temporal reasoning is significantly lower, with the best LGM score of 39.51 on the “what-when-where” chain. The principal implication is that current Video-LLMs have significant limitations in consistent spatio-temporal reasoning, requiring AI practitioners to develop methods that enhance causal and relational understanding in video processing models. |
|
| VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning (Read more on arXiv or HuggingFace) |
Chang Wen Chen, Ye Liu, AnalMom, KevinQHLin |
Here’s a concise summary of the research paper: i) VideoMind is a video-language agent that uses a Chain-of-LoRA strategy for temporal-grounded video understanding. ii) The main research objective is to develop an agent that can effectively reason about long videos by identifying and integrating essential capabilities for temporal reasoning. iii) Key methodology involves a role-based agentic workflow (Planner, Grounder, Verifier, Answerer) and a Chain-of-LoRA strategy for efficient role-switching using lightweight LoRA adaptors on a single base model (Qwen2-VL). iv) Primary results: On the CG-Bench long video benchmark, the 2B VideoMind model achieved a 5.94 mIoU, surpassing GPT-40-mini (3.75) and approaching GPT-40 (5.62). v) Principal implication for AI practitioners: The Chain-of-LoRA approach enables the creation of efficient and flexible video reasoning agents, reducing the computational overhead of using multiple models while demonstrating strong performance on grounded video question-answering. |
|
| Rewards Are Enough for Fast Photo-Realistic Text-to-image Generation (Read more on arXiv or HuggingFace) |
Jing Tang, Kenji Kawaguchi, Weijian Luo, whatlegequ, Luo-Yihong |
This paper introduces R0, a novel approach for fast text-to-image generation that relies solely on reward maximization, challenging the necessity of diffusion distillation. The main research question is whether reward signals alone, without diffusion losses, are sufficient for high-quality, few-step text-to-image generation. The key methodology is R0, a conditional generation approach via regularized reward maximization, that treats image generation as an optimization problem in data space. The results show that R0 outperforms previous methods such as RG-LCM and DI++, achieving a HPS of 34.37 and Image Reward of 1.27 using SD-v1.5 in 4 steps. AI practitioners can develop fast and high-quality text-to-image models by focusing on proper reward functions and regularization, without relying on computationally expensive diffusion distillation, and may adapt the framework to other conditional image generation tasks. |
|
| MTV-Inpaint: Multi-Task Long Video Inpainting (Read more on arXiv or HuggingFace) |
CeciliaJL, XiaodongChen, magicwpf, lianghou, GuZheng |
MTV-Inpaint is a unified video inpainting framework that supports multiple tasks, including text/image-guided object insertion and scene completion, and handles long videos. The main research objective is to develop a video inpainting model capable of handling both scene completion and controllable object insertion tasks in long videos, unifying these tasks and with enhanced input controllability. The key methodology involves a dual-branch spatial attention mechanism in a T2V diffusion U-Net, integration of image inpainting models via an I2V mode, and a two-stage pipeline (keyframe plus in-between frame propagation) for long videos. In object insertion, the method achieved a mIOU of 85.00%, surpassing existing baselines. For AI practitioners, MTV-Inpaint offers a single framework capable of various video inpainting tasks and their derivates like multi-modal inpainting, editing and object removal with state-of-art performance, avoiding the needs of training specialized models. |
|
| Error Analyses of Auto-Regressive Video Diffusion Models: A Unified |
|
|
|
| Framework (Read more on arXiv or HuggingFace) |
duchao, TIanyupang, xiaolili, Fengzhuo, k-nick |
This paper develops a theoretical framework for analyzing errors in auto-regressive video diffusion models (ARVDMs) and uses the analysis to propose architectural improvements. The main research question is what types of errors are shared by most ARVDMs, why do those errors appear, and how can they be mitigated. The key methodology involves developing a unified framework, Meta-ARVDM, analyzing the KL-divergence between generated and true videos to identify error sources, and deriving an information-theoretic impossibility result related to the error. A primary result is the identification of “error accumulation” and “memory bottleneck”, with the KL-divergence bound including terms for noise initialization, score estimation, discretization errors, and a memory bottleneck term specifically I(Output; Past |
Input). The principal implication is that AI practitioners can mitigate the memory bottleneck by modifying the network structure, such as using prepending and channel concatenation, leading to improved trade-offs between error and computational cost. |
| Sightation Counts: Leveraging Sighted User Feedback in Building a |
|
|
|
| BLV-aligned Dataset of Diagram Descriptions (Read more on arXiv or HuggingFace) |
Jaime-Choi, sangryul, namin0202, eunkey, soarhigh |
SIGHTATION, a novel dataset, enhances diagram descriptions for blind and low-vision (BLV) users by incorporating sighted user feedback on Vision Language Model (VLM) outputs. The main research objective is to create a BLV-aligned dataset of diagram descriptions that addresses the misalignment between sighted annotators and BLV user preferences. The key methodology involves a two-pass VLM inference with latent supervision using a guide generated, followed by sighted-user assessments of the VLM-generated descriptions in terms of preference, completion, retrieval, and question answering. Primary results reveal that preference-tuning a 2B model on the dataset increased usefulness ratings by BLV educators by an average of 1.670 standard deviations. Principal implication for AI practitioners is that leveraging sighted user assessments of VLM-generated content, guided by a multi-pass inference, provides a scalable and effective method to develop datasets that meet the needs of BLV users. |
|
| Long-Video Audio Synthesis with Multi-Agent Collaboration (Read more on arXiv or HuggingFace) |
Li Liu, Xiaojie Xu, yingcongchen, Xxlbigbrother, Buzz-lightyear |
i) The paper introduces LVAS-Agent, a novel multi-agent framework for end-to-end long-video audio synthesis. ii) The primary research objective is to address the challenges of long-video dubbing, including semantic shifts and temporal misalignment, by mimicking professional dubbing workflows. iii) The methodology decomposes the synthesis process into scene segmentation, script generation, sound design, and audio synthesis, utilizing VLM and LLM-based agents with discussion-correction and generation-retrieval-optimization mechanisms. iv) The study demonstrates superior audio-visual alignment over baseline methods using LVAS-Bench, a new benchmark dataset with 207 professionally curated long videos, and achieves state-of-the-art performance across distribution matching, audio quality, semantic alignment, and temporal alignment metrics. v) The principal implication for AI practitioners is the provision of a structured, collaborative framework and corresponding dataset that enables higher-quality, contextually aware audio synthesis in long-form video content creation, potentially enhancing viewer immersion and narrative coherence. |
|
| Basic Category Usage in Vision Language Models (Read more on arXiv or HuggingFace) |
KyleMoore, JesseTNRoberts, HTSawyer |
Vision Language Models (VLMs) exhibit human-like basic-level categorization preferences, distinctions between biological/non-biological objects, and expert-level shifts. The main research question is whether basic-level categorization behaviors observed in humans transfer to large language models. The key methodology involved prompting two VLMs (Llama 3.2 Vision Instruct and Molmo 7B-D) with images and comparing model-generated descriptions to a dataset of basic-level image labels, using two-proportion Z-tests for statistical analysis. Primary results showed that Llama 3.2 produced basic-level categorizations in 60.2% of outputs, and both models used basic-level terms significantly less (p<0.01) for non-biological items. The principal implication is that understanding how LLMs represent object categories, mirroring human cognition, is essential for developing models that align more closely with human behavior and interpretability. |
|
| Investigating Human-Aligned Large Language Model Uncertainty (Read more on arXiv or HuggingFace) |
Pamela Wisniewski, Daryl Watson, Kyle Moore, JesseTNRoberts |
This work investigates how well various large language model (LLM) uncertainty measures correlate with human uncertainty. The main research question is what LLM uncertainty measures best align with human group-level uncertainty on non-factual questions. The methodology involves comparing LLM uncertainty on a curated dataset of survey questions against human response distributions, using measures like self-reporting, entropy, and ensemble methods. The primary result is that top-k entropy correlates negatively with human uncertainty and decreases in human-similarity with increased model size (r > 0.3 for many models), but combining multiple measures produces a generalizable model (r ≈ 0.5 for cross-validation and r>0.6 on full data) . AI practitioners can use mixtures of uncertainty quantification methods, and potentially combining methods such as nucleus size and top-k entropy, to create LLMs that better reflect human-like uncertainty, especially for applications requiring calibrated trust and human-AI collaboration. |
|
Papers for 2025-03-17
| Title |
Authors |
Summary |
| ReCamMaster: Camera-Controlled Generative Rendering from A Single Video (Read more on arXiv or HuggingFace) |
Zuozhu, Mu437, Xintao, menghanxia, jianhongbai |
ReCamMaster is a framework for re-rendering a given video with novel camera trajectories using a generative model. The main research objective is to develop a camera-controlled generative video re-rendering framework that can reproduce the dynamic scene of an input video at novel camera trajectories. The key methodology involves conditioning a pre-trained text-to-video diffusion model on both the source video and target camera poses using a frame-dimension concatenation technique, and training on a new multi-camera synchronized video dataset created with Unreal Engine 5. The method achieved a FID score of 57.10 and FVD of 122.74 on visual quality, outperforming existing state-of-the-art approaches. AI practitioners can use this framework for video editing tasks like stabilization, super-resolution, and outpainting, offering improved control over camera movements in generated videos. |
| Adversarial Data Collection: Human-Collaborative Perturbations for |
|
|
| Efficient and Robust Robotic Imitation Learning (Read more on arXiv or HuggingFace) |
AutobotZero, hsli-cuhk, Eralien, morninghaze, SiyuanH |
Here’s a concise summary of the paper: i) Adversarial Data Collection (ADC) framework improves robotic imitation learning by introducing human-collaborative perturbations during data acquisition. ii) The main research objective is to maximize per-demonstration information density and improve the efficiency and robustness of robotic imitation learning. iii) Key methodology involves a “Two-Humans-in-the-Loop” approach where an adversarial operator dynamically introduces visual and linguistic perturbations during teleoperation by a primary operator. iv) Models trained with 20% of ADC-collected data volume achieved superior generalization and robustness compared to models trained with 100% of traditionally collected data. v) For AI practitioners, ADC provides a practical strategy for enhancing data quality over quantity, reducing the reliance on large datasets for training robust robotic policies in real-world, dynamic environments. |
| Technologies on Effectiveness and Efficiency: A Survey of State Spaces |
|
|
| Models (Read more on arXiv or HuggingFace) |
yuchenFan, xuekai, iseesaw, Youbang, XingtaiHF |
i) This survey provides a structured overview of State Space Models (SSMs), comparing their effectiveness and efficiency against transformers. ii) The main objective is to present a coherent and systematic analysis of SSMs, covering their theoretical underpinnings, mathematical formulations, and applications. iii) The survey categorizes SSMs into three main sections: original SSMs, structured SSMs (S4), and selective SSMs (Mamba), emphasizing the technical aspects and key techniques. iv) The paper highlights techniques such as Euler’s method, ZOH, and bilinear transform discretization for enabling the transformation of SSMs from continuous-time to discrete-time, and references the Mamba model achieving a 20-40 time speedup by performing SSM parameter discretization and recurrence computation directly in the GPU SRAM rather than the GPU HBM. v) AI practitioners can use this survey to understand the trade-offs between different SSM architectures, enabling them to make informed decisions when selecting models for sequential data processing and long-context tasks where efficiency is critical. |
| API Agents vs. GUI Agents: Divergence and Convergence (Read more on arXiv or HuggingFace) |
Eliblo1969, SiQin88, liqul, shilhe, vyokky |
i) This paper comparatively analyzes API-based and GUI-based LLM agents for software automation, examining their divergence and potential convergence. ii) The main objective is to systematically analyze the architectural differences, development workflows, and user interaction models of API-based versus GUI-based LLM agents. iii) The methodology involves a comparative study across key dimensions such as modality, reliability, efficiency, availability, flexibility, security, transparency, human-like interaction, and maintainability, along with illustrative use cases. iv) The primary result shows API agents offer efficiency and security with stable endpoints while GUI agents provide broader applicability, with the finding being that hybrid approaches can combine UI-based steps where APIs are unavailable with direct calls for data-heavy tasks. v) The principal implication for AI practitioners is the need to consider hybrid agent architectures that leverage the strengths of both API- and GUI-based approaches to achieve comprehensive automation across diverse software ecosystems. |
| Large-scale Pre-training for Grounded Video Caption Generation (Read more on arXiv or HuggingFace) |
Josef Sivic, Cordelia Schmid, ekazakos |
This paper introduces a method for generating video captions with objects grounded via temporally dense bounding boxes, including a new model, datasets, and pre-training approach. The main research objective is to generate video-level captions with corresponding bounding boxes that consistently localize key noun phrases across the video frames. The key methodology includes an automatic annotation method that aggregates frame-level grounded captions into temporally consistent video annotations, coupled with a Grounded Video Caption Generation model (GROVE) that uses spatio-temporal adapters and a temporal objectness head. The primary results show that GROVE, pre-trained on the new, automatically-annotated HowToGround1M dataset (1M videos) and fine-tuned on the manually-annotated iGround dataset, achieves a CIDEr score of 85.4 on the iGround test set. The principal implication is that AI practitioners can leverage large-scale automatic annotation and pre-training, followed by fine-tuning on smaller, high-quality datasets, to achieve state-of-the-art results in grounded video caption generation. |
| FlowTok: Flowing Seamlessly Across Text and Image Tokens (Read more on arXiv or HuggingFace) |
Liang-Chieh Chen, QHL067, QihangYu, turkeyju |
FlowTok is a framework that enables direct flow matching between text and images by encoding both into compact 1D tokens. The main research question is whether multimodal understanding and generation can be unified by enabling direct transitions within a shared, compact 1D latent space. The key methodology involves projecting both text and images into a unified 1D latent space using an enhanced image tokenizer and a text projector, then applying flow matching. FlowTok reduces the latent space size by 3.3x compared to prior methods at 256 resolution and achieves a COCO FID-30K score of 9.67 while completing training in 26.1 8-A100 days. For AI practitioners, FlowTok offers a more memory-efficient and faster approach to text-to-image and image-to-text generation, by leveraging a compact 1D token representation. |
| Kolmogorov-Arnold Attention: Is Learnable Attention Better For Vision |
|
|
| Transformers? (Read more on arXiv or HuggingFace) |
Xin Li, Killian Hitsman, aritradutta, maitysubhajit |
This paper investigates learnable attention mechanisms based on Kolmogorov-Arnold Networks (KANs) for Vision Transformers (ViTs). The main research question is whether a learnable multi-head self-attention (MHSA) module, specifically a Kolmogorov-Arnold Attention (KArAt), can improve the performance of vanilla ViTs. The key methodology involves designing a general KArAt, and a specific variant, Fourier-KArAt, and evaluating them against vanilla ViTs on CIFAR-10, CIFAR-100, and ImageNet-1K datasets, analyzing loss landscapes, weight distributions, and attention maps. The primary result shows ViT-Tiny+Fourier KArAt outperforms ViT-Tiny on CIFAR-10 by 5.40% in Top-1 accuracy, but larger ViT models with KArAt show diminished gains or worse performance. The implication is that directly replacing softmax with learnable activations in ViT’s attention mechanism does not guarantee improved performance, requiring careful design due to increased model complexity and optimization challenges, although in some instances, smaller models can improve their performance. |
| Cockatiel: Ensembling Synthetic and Human Preferenced Training for |
|
|
| Detailed Video Caption (Read more on arXiv or HuggingFace) |
Hao Li, Zhiyu Tan, xiaomengyang, Kobeshegu, Fr0zencr4nE |
Cockatiel-13B is a video captioning model that ensembles synthetic and human-aligned training to generate detailed and human-preferred video descriptions. The main research objective is to address the imbalanced video-caption alignment and misalignment with human preferences in existing video detailed captioning (VDC) models. The key methodology involves a three-stage training pipeline that curates data using a human-aligned caption quality scorer, trains a 13B parameter model (Cockatiel-13B) on the curated data, and distills an 8B parameter model (Cockatiel-8B) from it. Primary results show Cockatiel-13B achieving a new state-of-the-art VDCSCORE average of 43.80, outperforming existing models. The principal implication is that AI practitioners can achieve more human-aligned and dimension-balanced video descriptions by utilizing a training procedure that selectively combines diverse model strengths, guided by structured human preferences. |
| Neighboring Autoregressive Modeling for Efficient Visual Generation (Read more on arXiv or HuggingFace) |
Hong Zhou, Feng Chen, Shaoxuan He, Yuanyu He, Yefei He |
Neighboring Autoregressive Modeling (NAR) is a new paradigm for efficient visual generation that formulates autoregressive visual generation as a progressive outpainting procedure. The main research objective is to develop an autoregressive visual generation method that improves efficiency and preserves spatial/temporal locality, unlike raster-order “next-token prediction” approaches. The key methodology is a near-to-far “next-neighbor prediction” mechanism, using dimension-oriented decoding heads to predict multiple adjacent tokens in parallel along orthogonal dimensions. Results show that on ImageNet 256x256, NAR-L achieves a lower FID (3.06) than LlamaGen-XXL (3.09) with 87.8% fewer steps and 13.8x higher throughput. AI practitioners can use NAR to achieve more efficient autoregressive visual generation with improved fidelity compared to traditional next-token prediction and existing parallel approaches, particularly beneficial for high-resolution image and video tasks. |
| ProJudge: A Multi-Modal Multi-Discipline Benchmark and |
|
|
| Instruction-Tuning Dataset for MLLM-based Process Judges (Read more on arXiv or HuggingFace) |
Fanrui Zhang, Ming Li, Zhaopan Xu, Pengfei Zhou, Jiaxin Ai |
ProJudge is a benchmark and instruction-tuning dataset for evaluating multi-modal large language models (MLLMs) as automated process judges for scientific problem-solving. The main research objective is to assess and enhance the capability of MLLMs to perform fine-grained evaluation of step-by-step reasoning in scientific problems, including error detection, classification, and diagnosis. The key methodology involves creating ProJudgeBench, a benchmark of 2,400 multi-modal scientific problems with 50,118 step-level annotations, and ProJudge-173k, a large-scale instruction-tuning dataset, accompanied by a Dynamic Dual-Phase fine-tuning strategy. A key finding is that after fine-tuning on ProJudge-173k, InternVL2.5-8B showed a 58.92% increase in step correctness accuracy. Principal implication for AI practioners is that open-source models, through the ProJudge, can significantly enhance their performance to match that of many state-of-art closed-source, enabling more reliable and nuanced process evaluation in multi-modal reasoning tasks. |
| ARMOR v0.1: Empowering Autoregressive Multimodal Understanding Model |
|
|
| with Interleaved Multimodal Generation via Asymmetric Synergy (Read more on arXiv or HuggingFace) |
Zizhen Li, Fanrui Zhang, Chuanhao Li, Yukang Feng, Jianwen Sun |
ARMOR v0.1 is a resource-efficient framework that upgrades existing multimodal large language models (MLLMs) to unified models (UniMs) capable of both understanding and interleaved text-image generation. The main research objective is to enable MLLMs to perform multimodal generation while preserving their understanding capabilities and minimizing computational overhead. The key methodology involves an asymmetric encoder-decoder architecture with a forward-switching mechanism, a curated interleaved dataset, and a three-stage “What or How to Generate” (WoHG) training algorithm. Experimental results show that ARMOR outperforms existing UniMs on multimodal understanding benchmarks (78.8 score on MMB versus 62.6 for Janus-pro) while achieving comparable generation performance. AI practitioners can leverage ARMOR to build UniMs by fine-tuning existing MLLMs, thereby reducing training costs and enabling natural text-image interleaved generation. |
| Learning Few-Step Diffusion Models by Trajectory Distribution Matching (Read more on arXiv or HuggingFace) |
Yujun Cai, jingtang, JIACSUN96, whatlegequ, Luo-Yihong |
Learning Few-Step Diffusion Models by Trajectory Distribution Matching (TDM) introduces a unified distillation paradigm for accelerating diffusion model sampling. The main research objective is to develop a few-step diffusion model distillation method that combines the strengths of distribution and trajectory matching, overcoming their individual limitations. The key methodology is a data-free score distillation objective that aligns the student’s trajectory with the teacher’s at the distribution level, coupled with a sampling-steps-aware objective for flexible multi-step adaptation. The method distills PixArt-α into a 4-step generator that outperforms its teacher on real user preference at 1024 resolution, accomplishing this with only 500 iterations and 2 A800 hours. For AI practitioners, TDM offers a highly efficient way to train fast and high-quality few-step diffusion models, significantly reducing training cost while surpassing teacher model performance, as demonstrated on text-to-image tasks. |
| ETCH: Generalizing Body Fitting to Clothed Humans via Equivariant |
|
|
| Tightness (Read more on arXiv or HuggingFace) |
Yuliang Xiu, Michael J. Black, Zeyu Cai, Haiwen Feng, Boqian-Li |
ETCH is a novel framework for fitting a 3D body model to point clouds of clothed humans by modeling cloth-to-body mapping. The main research objective is to accurately estimate the underlying body shape and pose from 3D scans of clothed humans, generalizing across diverse poses, shapes, and garment types. The key methodology is Equivariant Tightness Fitting, which uses SE(3)-equivariant displacement vectors to represent “tightness” and leverages pose-invariant body correspondences for sparse marker regression. The method reduces directional errors by 67.2% ~ 89.8% in one-shot (out-of-distribution) settings with approximately 1% of training data. AI practitioners can use this method to obtain accurate body shape and pose estimations from 3D scans of clothed individuals, with robustness to variations in clothing and pose, even with limited training data. |
| Open-World Skill Discovery from Unsegmented Demonstrations (Read more on arXiv or HuggingFace) |
Yitao Liang, Anji Liu, Shaofei Cai, Zihao Wang, Jingwen Deng |
This paper introduces Skill Boundary Detection (SBD), a self-supervised algorithm for segmenting unsegmented demonstration videos into discrete skills for open-world learning. The main research question is how to automatically segment long, unsegmented demonstration videos into meaningful, skill-consistent segments without manual annotations. SBD leverages a pretrained unconditional action-prediction model and detects skill boundaries by identifying significant increases in prediction error, based on event segmentation theory. The method improved the average performance of conditioned policies in Minecraft by 63.7% and 52.1% on short-term atomic skill tasks. AI practitioners can leverage this method to train instruction-following agents from diverse, unlabeled video data, such as YouTube, without requiring manual segmentation or labeling. |
| GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories |
|
|
| Generation in End-to-End Autonomous Driving (Read more on arXiv or HuggingFace) |
Bo Jiang, Yang Hu, Xingyu Zhang, WonderingWorld, XXXXing |
GoalFlow is an end-to-end autonomous driving method that generates high-quality multimodal trajectories using goal-driven flow matching. The main research objective is to address trajectory selection complexity and reduced quality in existing multimodal trajectory generation methods for autonomous driving. The key methodology involves introducing GoalFlow, which constrains trajectory generation using a goal point selected via a novel scoring mechanism, employs Flow Matching for efficient generation, and uses a refined scoring mechanism for optimal trajectory selection. Primary results show GoalFlow achieved a PDMS of 90.3 on the Navsim benchmark, significantly outperforming other methods, and requires only a single denoising step for excellent performance. Principal implication for AI practitioners is that GoalFlow provides a method for generating high-quality, diverse, yet, safe candidate actions for autonomous driving systems enhancing robustness and real-world deployability. |
| MaRI: Material Retrieval Integration across Domains (Read more on arXiv or HuggingFace) |
Yuxuan Chen, Huixiong Zhang, Yangfan He, Jianhui Wang, yangzhifei |
MaRI is a framework for aligning visual and material properties in a shared embedding space for material retrieval. The main research objective is to bridge the feature space gap between synthetic and real-world materials to improve material retrieval accuracy. The key methodology involves using dual DINOv2-based encoders trained contrastively to map images and materials into a shared space, leveraging a new dataset combining synthetic and real-world material data. Primary results show that on a trained material dataset, MaRI achieves a top-1 instance accuracy of 26.0% and a top-5 instance accuracy of 90.0%. AI practitioners can use MaRI’s framework and dataset to improve the accuracy and generalization of material retrieval, enhancing 3D asset creation and applications requiring realistic material representation. |
| VGGT: Visual Geometry Grounded Transformer (Read more on arXiv or HuggingFace) |
Christian Rupprecht, Andrea Vedaldi, Nikita Karaev, Minghao Chen, Jianyuan Wang |
VGGT is a feed-forward transformer that directly infers 3D attributes of a scene from multiple images, achieving state-of-the-art results in several 3D tasks. The main research objective is to determine if 3D tasks can be solved directly by a neural network without visual geometry post-processing. The key methodology is a large transformer with alternating frame-wise and global self-attention, trained on multiple 3D-annotated datasets to predict camera parameters, depth maps, point maps, and 3D point tracks. The primary results show that VGGT outperforms state-of-the-art methods on RealEstate10K and CO3Dv2 datasets for camera pose estimation (AUC@30 of 93.5 and 91.8 respectively, with BA), and also achieves superior accuracy on the DTU and ETH3D datasets for multi-view depth and point map estimation, exceeding optimization-based and other feed-forward methods. Principal implication is that AI practitioners can leverage VGGT for fast and accurate 3D reconstruction, reducing or eliminating the reliance on costly iterative optimization techniques commonly used in computer vision, potentially simplifying and accelerating 3D vision pipelines. |
| From TOWER to SPIRE: Adding the Speech Modality to a Text-Only LLM (Read more on arXiv or HuggingFace) |
Tsz Kin Lam, Anil Keshwani, Sonal Sannigrahi, Kshitij Ambilduke, bpop |
SPIRE extends the TOWER language model to process speech by incorporating discretized speech units and continued pre-training. The main research objective is to integrate English speech processing (transcription and translation) into an existing text-only multilingual LLM, TOWER, while maintaining its original text-task performance. The methodology involves two stages: continued pre-training (CPT) on a mixture of ASR data and TOWER’s text data, and instruction tuning (IT) on MT, ASR, and ST datasets, employing HuBERT-based k-means clustering for speech discretization. SPIREFULL achieves a Word Error Rate (WER) of 4.2 on the LibriSpeech test-clean set, outperforming models like Spirit-LM and the Whisper-base, though not matching the performance of more heavily speech-trained models. AI practitioners can adapt a text-based LLM for speech tasks with preserved performance on text-based tasks by leveraging the recipe of speech discretization and CPT+IT. |
| Group-robust Machine Unlearning (Read more on arXiv or HuggingFace) |
Massimiliano Mancini, Elisa Ricci, Stéphane Lathuilière, Subhankar Roy, Thomas De Min |
This paper introduces group-robust machine unlearning to address performance degradation in specific demographic groups caused by non-uniformly distributed data removal requests. The main research question is how to unlearn data from a trained model while preserving performance for groups that are over-represented in the forget set. The key methodology involves sample distribution reweighting during retraining and a novel approximate unlearning method (MIU) that minimizes mutual information between model features and group information, alongside mutual information calibration with original model. Primary results show that MIU outperforms standard unlearning methods on CelebA, Waterbirds, and FairFace datasets; for example it achieves 69.0% group accuracy (GA) on CelebA compared with next best of 66.2%, preserving model robustness. The principle implication is that AI practitioners should use distribution reweighting and mutual information-based techniques to mitigate fairness issues in machine unlearning scenarios where data removal requests are not uniformly distributed across groups. |
Papers for 2025-03-14
| Title |
Authors |
Summary |
| CoSTAast: Cost-Sensitive Toolpath Agent for Multi-turn Image Editing (Read more on arXiv or HuggingFace) |
Dang Nguyen, zhoutianyi, nandakiran09, advaitgupta |
CoSTA* is a cost-sensitive toolpath agent that finds the optimal tool sequence for multi-turn image editing by combining LLMs and A* search. The main research question is how to combine the strengths of large language models (LLMs) and graph search to find cost-efficient tool paths for multi-turn image editing. The key methodology is a three-stage approach called CoSTA* that uses LLMs to create a subtask tree, prunes a graph of AI tools, and then conducts A* search on the subgraph to find a tool path, guided by a combination of cost and quality metrics. CoSTA* achieved an overall accuracy of 0.94 across all tasks, outperforming baselines such as GenArtist (0.73) and CLOVA (0.63) and offers dynamic trade-offs between the computational cost and quality. This implies that AI practitioners can leverage CoSTA* to build more efficient and adaptable image editing systems that can handle complex, multi-turn editing instructions, allowing for dynamic parameter adjustments of quality-cost trade-offs. |
| World Modeling Makes a Better Planner: Dual Preference Optimization for |
|
|
| Embodied Task Planning (Read more on arXiv or HuggingFace) |
xpqiu, Jinlan, CyberDJ, ngc7293, sinwang |
D²PO jointly optimizes state prediction and action selection in LVLMs for embodied task planning, improving performance and efficiency. The research objective is to develop a learning framework, Dual Preference Optimization (D²PO), that enhances embodied task planning in large vision-language models (LVLMs) by jointly optimizing state prediction and action selection. The key methodology involves a tree search mechanism for automatic data collection and a dual preference learning approach using preference pairs for both action and state prediction. Primary results show that D²PO significantly outperforms existing methods and GPT-4o on the VoTa-Bench, achieving a 31.4% relative improvement in success rate and a 33.0% improvement in planning efficiency compared to SFT baselines on a 7B-parameter model. The principal implication for AI practitioners is that incorporating world modeling objectives through D²PO substantially enhances the planning capabilities of LVLMs in embodied AI, offering a more effective approach for developing agents that can perform complex tasks with higher success and efficiency. |
| Silent Branding Attack: Trigger-free Data Poisoning Attack on |
|
|
| Text-to-Image Diffusion Models (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, kiminle2, harryjo97, wchoi403, agwmon |
This paper introduces a novel data poisoning attack, called Silent Branding Attack, that manipulates text-to-image diffusion models to generate images with specific brand logos, without requiring any text triggers. The main research objective is to develop and validate a data poisoning method that unobtrusively embeds target logos into images generated by text-to-image diffusion models, operating without explicit text triggers. The key methodology involves an automated algorithm that personalizes logos, generates masks for logo placement, and uses inpainting and refinement techniques to seamlessly integrate logos into existing images. The attack achieved a logo inclusion rate (LIR) of 45.00% on the Midjourney dataset and 39.68% on the Tarot dataset with a 100% poisoning ratio, demonstrating successful logo embedding without specific text triggers. AI practitioners should be aware that text-to-image diffusion models are vulnerable to data poisoning attacks that can subtly embed unwanted visual elements, even without trigger words, necessitating safeguards against such manipulations. |
| Charting and Navigating Hugging Face’s Model Atlas (Read more on arXiv or HuggingFace) |
yedid, LielAmar, jonkahana, nitzankur, Eliahu |
The paper introduces a method for charting and navigating the vast model repository of Hugging Face by constructing a model atlas represented as a directed acyclic graph. The main research objective is to develop a method for recovering the undocumented evolutionary relationships between models in large repositories, and to explore the use cases of such an atlas. The key methodology involves representing models by their weights, calculating pairwise distances, and using temporal and structural priors to predict directed edges, accounting for model merging and quantization. The results show the proposed method recovers 78.87% of the model relations on a Qwen connected component dataset, substantially outperforming baseline methods, and reveal that 99.41% of quantised models in hugging face are leafs (don’t have children). The principal implication is that AI practitioners can use the constructed atlas to improve model discovery, attribute prediction, and heritage tracing, enabling more efficient model reuse and analysis. |
| GoT: Unleashing Reasoning Capability of Multimodal Large Language Model |
|
|
| for Visual Generation and Editing (Read more on arXiv or HuggingFace) |
zengxingyu, shilinyan, LjHuang, gogoduan, LucasFang |
This paper introduces Generation Chain-of-Thought (GoT), a new paradigm for visual generation and editing that leverages multimodal large language models (MLLMs) to perform explicit semantic-spatial reasoning before outputting images. The main research objective is to integrate reasoning mechanisms into visual generation and editing to improve the alignment of generated content with human intentions. The key methodology involves formulating GoT as a multimodal reasoning chain, constructing large-scale GoT datasets with 9M+ samples, and developing a unified framework integrating Qwen2.5-VL with a Semantic-Spatial Guidance Module enhanced diffusion model. The GoT framework achieved a 0.64 overall score on the GenEval benchmark for text-to-image generation, outperforming existing methods. For AI practitioners, GoT offers a framework to build visual generation and editing systems with enhanced reasoning capabilities, enabling improved control, more accurate results, and interactive generation based on modified reasoning steps. |
| Transformers without Normalization (Read more on arXiv or HuggingFace) |
Zhuang Liu, Kaiming He, ylecun, endernewton, JiachenZhu |
This paper introduces Dynamic Tanh (DyT) as a replacement for normalization layers in Transformers, achieving comparable or superior performance. The main research question is whether normalization layers are indispensable in Transformers, and can they be replaced with a simpler alternative. The key methodology involves replacing normalization layers (LayerNorm and RMSNorm) with a proposed element-wise operation, DyT(x) = tanh(αx), where α is a learnable parameter, and empirically evaluating the modified architectures. Primary results show that Vision Transformers (ViT-B) with DyT achieved 82.5% top-1 accuracy on ImageNet-1K, surpassing the 82.3% accuracy of the LN-based model. Principal implication for AI practitioners that normalization layers in Transformers may not be necessary, and simpler, computationally efficient alternatives such as DyT can provide same or better performance across multiple tasks. |
| GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding (Read more on arXiv or HuggingFace) |
wenyuliu, steelozazala, wondervictor, LianghuiZhu, RuiHu |
GroundingSuite introduces a new benchmark and framework for evaluating and improving pixel-level visual grounding in complex and diverse scenarios. The main research objective is to address limitations in existing pixel grounding datasets, specifically their limited object categories, textual diversity, and annotation quality. The key methodology involves an automated annotation framework (GSSculpt) leveraging multiple VLM agents for entity localization, text generation, and noise filtering, alongside a curated evaluation benchmark (GSEval). A model trained on the new dataset (GSTrain-10M) achieved a cIoU of 68.9 on gRefCOCO, outperforming models trained on other datasets. AI practitioners can use GroundingSuite to train and evaluate models for more robust and generalizable pixel grounding, applicable across diverse granularities and complex referential expressions. |
| New Trends for Modern Machine Translation with Large Reasoning Models (Read more on arXiv or HuggingFace) |
acecamel1977, longyuewang, minghaowu, ChenyangLyu, SNF |
Large Reasoning Models (LRMs) substantially transform traditional machine translation (MT) by reframing it as a dynamic reasoning task. The main research objective is to explore the potential of LRMs in redefining MT systems and identify the foundational shifts, new opportunities, and challenges they introduce. The key methodology involves a conceptual analysis and empirical case studies of LRM capabilities in various translation scenarios, including stylized, document-level, and multimodal translation. Primary results show LRMs can perform self-reflection to correct errors, automatically utilize pivot translation, and struggle with complex encoded text; experiments on commonMT showed similar BLEURT (73.0-74.2) and COMET (84.1-84.8) scores for both the reasoning and non-reasoning models. AI practitioners should consider LRMs as a means to develop MT systems that function as multilingual cognitive agents capable of reasoning about meaning, context, culture, and intent, beyond simple text conversion. |
| Shifting Long-Context LLMs Research from Input to Output (Read more on arXiv or HuggingFace) |
mingshan, tsq2000, Zhiqiang007, bys0318, mozhu |
This paper advocates for a shift in long-context large language model (LLM) research, prioritizing long-output generation capabilities over the current focus on long-input processing. The main research objective is to define and address the challenges of developing LLMs capable of generating high-quality, coherent, and contextually relevant long-form text outputs. The key methodology involves analyzing existing datasets, benchmarks, and models, and identifying limitations in long-output generation through statistical analysis and qualitative assessment of model outputs. Primary results show that the demand for long-output generation (exceeding 4,000 tokens) is 2-3 times greater than for equivalent-length inputs in real-world applications, while only 2 out of 104 papers on long-context tasks at major ML/NLP conferences in 2024 directly addressed long-output generation. The principal implication for AI practitioners is the need to develop new datasets, training techniques, and evaluation metrics specifically designed for long-output LLMs to meet real-world demands in areas like creative writing and complex reasoning. |
| VisualWebInstruct: Scaling up Multimodal Instruction Data through Web |
|
|
| Search (Read more on arXiv or HuggingFace) |
Bo Li, Xiang Yue, wenhu, jiachenli-ucsb, jymmmmm |
VisualWebInstruct introduces a method for creating large-scale, multimodal instruction datasets by leveraging web search. The main research objective is to address the scarcity of high-quality, diverse training data for reasoning-focused multimodal tasks. The key methodology involves using Google Image Search with 30,000 seed images to collect over 700K unique URLs, extracting QA pairs from HTML accessibility trees, and refining the data using GPT-4O for answer synthesis and consistency filtering. Fine-tuning MAmmoTH-VL on this dataset (named VisualWebInstruct) achieves a state-of-the-art performance of 50.4% average accuracy across seven visual reasoning benchmarks. The principal implication is that AI practitioners can leverage web-scale data to improve the reasoning abilities of vision-language models, particularly on tasks requiring multi-step deliberation with visual context. |
| DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture |
|
|
| Design in Text to Image Generation (Read more on arXiv or HuggingFace) |
Rui Qian, Chen Chen, yinfeiy, tsujuifu, wenzehu |
The paper introduces DiT-Air, a streamlined Diffusion Transformer architecture for text-to-image generation that achieves state-of-the-art performance with improved parameter efficiency. The main research objective is to empirically investigate the impact of architectural choices, text-conditioning strategies, and training protocols on the performance and efficiency of Diffusion Transformers (DiTs). The key methodology involves a comparative analysis of vanilla DiT, PixArt-style, and MMDiT variants, along with ablations of text encoders, layer-wise parameter sharing, and a progressive VAE training approach. Primary results show that DiT-Air achieves GenEval and T2I CompBench scores of 82.9 and 59.5, respectively, outperforming existing models while using significantly fewer parameters (66% reduction compared to MMDiT). For AI practitioners, DiT-Air offers a more parameter-efficient architecture for text-to-image diffusion models, enabling competitive performance with reduced computational resources. |
Do I look like a cat.n.01 to you? A Taxonomy Image Generation |
|
|
| Benchmark (Read more on arXiv or HuggingFace) |
Ekaterina Neminova, Alina Lobanova, lilaspourpre, apanc, VityaVitalich |
This paper introduces a benchmark for evaluating text-to-image models’ ability to generate images representing taxonomic concepts from WordNet. The main research objective is to assess how well text-to-image models can visualize concepts of varying abstraction levels within a hierarchical taxonomy. The key methodology involves evaluating 12 text-to-image models using 9 taxonomy-related metrics, human feedback, and pairwise evaluation with GPT-4 feedback. The primary results show that Playground-v2 and FLUX consistently outperform other models across metrics, with Playground ranking first in all preference-based evaluations, but the model ranking differs significantly from standard text-to-image tasks. AI practitioners can use this benchmark to evaluate and improve text-to-image models for generating images reflecting structured, hierarchical data, with a clear indication that specific models are much better at reflecting taxonomic data. |
| Open-Sora 2.0: Training a Commercial-Level Video Generation Model in |
|
|
| $200k (Read more on arXiv or HuggingFace) |
Xinying Guo, Tom Young, Chenhui Shen, Zangwei Zheng, Xiangyu Peng |
Open-Sora 2.0 is a commercially viable video generation model trained for $200k, demonstrating cost-effective techniques for high-quality video synthesis. The main research objective is to develop a top-performing video generation model at a highly controlled cost, much lower than comparable existing models. Key methodologies used include a hierarchical data filtering system, a deeply compressed video autoencoder (Video DC-AE), a diffusion transformer (DiT) architecture leveraging full attention, and an image-to-video training approach. The model achieves a win rate favorably against other top-performing models in all three aspects of human preference evaluation (visual quality, prompt adherence, and motion quality); specifically it is 5-10x cheaper to train ($200k) than comparables like MovieGen and Step-Video-T2V. Principal implication for AI practitioners is that high-quality video generation models are achievable with significantly reduced training costs through optimized data curation, model architecture, and training strategies. |
| Long Context Tuning for Video Generation (Read more on arXiv or HuggingFace) |
lindahua, zhenheny, Ikuinen, Brightmzb, ziyany |
Long Context Tuning (LCT) extends pre-trained video diffusion models to generate coherent multi-shot scenes by expanding their context window. The main research objective is to enable scene-level video generation with visual and dynamic consistency across multiple shots. The key methodology involves adapting full attention mechanisms to encompass all shots in a scene, incorporating interleaved 3D positional embedding, and using an asynchronous noise strategy for training. The primary results show that LCT-trained models achieve superior semantic alignment compared to baseline methods, with a user study score of 3.79 versus baselines ranging from 1.57 to 2.50. For AI practitioners, LCT offers a training paradigm to directly adapt single-shot video models for coherent, multi-shot video generation without additional parameters, enabling applications like short film production and interactive video editing. |
| 4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
hpfister, Qmh, wrencanfly, rpzhou, EthanTaylor |
4D LangSplat learns 4D language fields for efficient, time-sensitive, open-vocabulary querying of dynamic scenes. The main research objective is to develop a method for constructing precise 4D language fields that enable both time-agnostic and time-sensitive open-vocabulary queries in dynamic scenes. The key methodology involves using Multimodal Large Language Models (MLLMs) to generate object-wise video captions, encoding these captions into sentence embeddings for supervision, and employing a status deformable network to model continuous state changes. Results show that on the HyperNeRF dataset, for time-sensitive querying the proposed method achieves an accuracy of 89.42% and a vIoU of 66.07%. AI practitioners can use 4D LangSplat to build systems that enable open vocabulary text-based queries, which are time agnostic and time-sensitive, of the evolution and interaction of objects within a dynamic scene. |
| SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency |
|
|
| Distillation (Read more on arXiv or HuggingFace) |
Yuyang Zhao, Shuchen Xue, Junsong Chen, xieenze, sayakpaul |
SANA-Sprint is a text-to-image diffusion model that achieves fast, high-quality image generation through hybrid distillation. The main research objective is to develop an efficient diffusion model capable of one-step high-quality text-to-image (T2I) generation while maintaining multi-step sampling flexibility. The key methodology involves transforming a pre-trained flow-matching model for continuous-time consistency distillation (sCM), combined with latent adversarial distillation (LADD), and includes QK-normalization and dense time-embedding. The primary results show SANA-Sprint achieves a 7.59 FID and 0.74 GenEval in only one step, outperforming FLUX-schnell while being 10x faster (0.1s vs 1.1s on H100). The principal implication for AI practitioners is that they can leverage SANA-Sprint for applications requiring real-time or near real-time image generation with significantly reduced computational overhead compared to prior diffusion models. |
| UniGoal: Towards Universal Zero-shot Goal-oriented Navigation (Read more on arXiv or HuggingFace) |
Ziwei Wang, Lingqing Zhao, jiwenlu, xuxw98, hangyin |
UniGoal is a framework for universal zero-shot goal-oriented navigation that unifies different goal types within a single model. The main research objective is to develop a general framework capable of handling multiple navigation tasks (object, instance-image, and text-based goals) without task-specific training or fine-tuning. The key methodology involves representing both the scene and goals as graphs, performing graph matching, and using a multi-stage exploration policy guided by the matching score and a blacklist mechanism. Results show that UniGoal achieves a 60.2% success rate on instance-image goal navigation on the HM3D benchmark, outperforming prior zero-shot methods. AI practitioners can use UniGoal to deploy navigation agents in new environments with varied goal specifications without needing environment-specific or task-specific retraining. |
| Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and |
|
|
| Beyond (Read more on arXiv or HuggingFace) |
tanglifu, JunchenLiu, yyy99, duan901010, cizhenshi |
Light-R1 presents a training recipe for long chain-of-thought (COT) reasoning models, achieving state-of-the-art math performance with efficient training. The main research objective was to develop a method for training compact long-COT models from scratch, overcoming limitations of existing approaches. The key methodology involved a curriculum training recipe comprising two-stage supervised fine-tuning (SFT) with a curated dataset and semi-on-policy direct preference optimization (DPO), followed by reinforcement learning (specifically GRPO). The Light-R1-32B model, trained from Qwen2.5-32B-Instruct, achieved 76.6% on the AIME24 benchmark, surpassing DeepSeek-R1-Distill-Qwen-32B. AI practitioners can use this open-sourced approach, including models, data, and code, to efficiently train and deploy long-COT reasoning capabilities in resource-constrained environments, particularly for mathematical problem-solving. |
| CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidance (Read more on arXiv or HuggingFace) |
brotherhuang, u302117, BestWishYsh, angtian, dyf |
CINEMA is a framework for generating videos featuring multiple subjects, guided by reference images and text, using a Multimodal Large Language Model (MLLM) for improved coherence. The main research objective is to generate coherent multi-subject videos that maintain visual consistency of individual subjects and follow textual prompts, addressing limitations of existing methods that rely on ambiguous keyword mapping. The key methodology involves leveraging an MLLM (specifically Qwen2-VL) to encode multimodal conditions, an AlignerNet to align MLLM outputs with text features, and VAE encoding of reference images for fine-grained visual detail preservation, all integrated within a Multimodal Diffusion Transformer (MM-DiT) framework. The model was trained on 1.46 million video clips, each paired with 1 to 6 human/object references, achieving results shown qualitatively in Figures 5 and 6 with a training configuration using 128 NVIDIA H100 GPUs. For AI practitioners, CINEMA offers a scalable approach for multi-subject video generation that eliminates the need for explicit subject-text correspondences, improving subject consistency, which is beneficial for applications like personalized video content creation. |
| Quantization for OpenAI’s Whisper Models: A Comparative Analysis (Read more on arXiv or HuggingFace) |
allisonandreyev |
Whisper and its variants are evaluated for speech recognition, focusing on quantization’s impact on model size, latency, and accuracy. The main research objective is to analyze the similarities, differences, and capabilities of three Whisper models (Whisper, Whisper_Streaming, and whisper-timestamped) and quantify the impact of quantization on latency and its viability for edge deployment. The key methodology involves qualitative comparisons of the three models and quantitative evaluation of word error rate (WER) and latency using the LibriSpeech dataset with three quantization methods (INT4, INT5, INT8) in whispercpp. Quantization with INT4 reduced model size by 45% (from 141.11MB to 44.33MB) and decreased latency by 19%, while slightly improved word error rate (0.0199 to 0.0159). Quantization is a viable method for deploying Whisper on resource-limited devices, maintaining accuracy while significantly reducing model size and improving deployment efficiency. |
| Distilling Diversity and Control in Diffusion Models (Read more on arXiv or HuggingFace) |
David Bau, RohitGandikota |
Distilled diffusion models can retain the control and regain/exceed the diversity of their base models through strategic timestep management. The paper investigates how to distill both diversity and control capabilities from base diffusion models to their efficient distilled variants. The key methodology involves introducing DT-Visualization to analyze latent representations, and a hybrid inference approach that utilizes the base model for the first critical timestep and the distilled model subsequently. The primary results reveal that the hybrid approach achieves a FID score of 10.79 on COCO-30k, better than both the base (12.74) and distilled (15.52) models, while maintaining the distilled model’s inference speed. The principal implication is that AI practitioners can achieve both high diversity and efficiency in image generation using distilled diffusion models without additional training by leveraging the hybrid inference approach. |
| R1-Onevision: Advancing Generalized Multimodal Reasoning through |
|
|
| Cross-Modal Formalization (Read more on arXiv or HuggingFace) |
Xiaoxuan He, Yi Yang, twilightsnow, dcyin, Emilia515 |
R1-Onevision introduces a multimodal reasoning model, dataset, and benchmark to improve visual-language understanding and reasoning. The main research objective is to bridge the gap between visual perception and deep reasoning in large language models by employing a cross-modal reasoning pipeline. Key methodologies used include a cross-modal reasoning pipeline that transforms images into formal textural representations and a two-stage post-training strategy (supervised fine-tuning and reinforcement learning). R1-Onevision achieved 29.9% accuracy on MathVision, comparable to the closed-source model GPT-4o. The principal implication for AI practitioners is that formalizing visual information into textual representations, combined with specialized training, can significantly enhance the multimodal reasoning capabilities of large language models, as demonstrated through performance in visual reasoning benchmarks. |
| Autoregressive Image Generation with Randomized Parallel Decoding (Read more on arXiv or HuggingFace) |
Huan Wang, Guoqi Li, Jinyue Yang, hp-l33 |
ARPG is a visual autoregressive model that enables random-order, parallel image generation. The research objective is to develop an autoregressive image generation model that overcomes the limitations of raster-order approaches in inference efficiency and zero-shot generalization. The methodology involves a “guided decoding” framework that decouples positional guidance (queries) from content representation (key-value pairs) within the causal attention mechanism, to specify the output image token. On ImageNet-1K 256x256, ARPG achieves an FID of 1.94 with 64 sampling steps, attaining over 20x throughput increase and reducing memory use by over 75% compared to autoregressive models of similar scale. AI practitioners can use ARPG as a more efficient and versatile framework for autoregressive image generation, enabling faster and more flexible image synthesis applications. |
| The Curse of Conditions: Analyzing and Improving Optimal Transport for |
|
|
| Conditional Flow-Based Generation (Read more on arXiv or HuggingFace) |
Alexander Schwing, hkchengrex |
Conditional optimal transport (C²OT) improves conditional flow-based generative models by addressing a train-test discrepancy caused by standard optimal transport. The main research objective is to analyze and mitigate the performance degradation of minibatch optimal transport (OT) in conditional flow matching when conditions are introduced. The key methodology is the introduction of a conditional weighting term in the OT cost matrix calculation, along with adaptive weight finding and oversampling techniques. The primary results demonstrate C²OT outperforms flow matching (FM) and OT in conditional generation, e.g. achieving a 2-Wasserstein distance of 0.013±0.003 on 8gaussians→moons with continuous conditions vs FM (0.028±0.010) and OT (2.143±1.993). AI practitioners can use C²OT as a drop-in replacement for standard OT in flow matching to achieve better performance in conditional generative modeling, avoiding skewed priors during training. |
| VisualPRM: An Effective Process Reward Model for Multimodal Reasoning (Read more on arXiv or HuggingFace) |
Einsiedler, Yeshenglong, Decaux, chenlj22, Weiyun1025 |
VisualPRM is an 8B parameter multimodal Process Reward Model (PRM) that improves reasoning in Multimodal Large Language Models (MLLMs) using Best-of-N evaluation. The research introduces VisualPRM and evaluates its effectiveness as a critic model for enhancing MLLM reasoning. The authors construct a multimodal process supervision dataset (VisualPRM400K) and a benchmark (VisualProcessBench) with human-annotated step-wise correctness labels, then train VisualPRM on the dataset. Applying VisualPRM to InternVL2.5-78B achieves a 5.9-point improvement across seven multimodal reasoning benchmarks. AI practitioners can utilize VisualPRM as an effective critic model to enhance the reasoning performance of MLLMs through Test-Time Scaling, particularly with the Best-of-N strategy. |
| “Silent Is Not Actually Silent”: An Investigation of Toxicity on Bug |
|
|
| Report Discussion (Read more on arXiv or HuggingFace) |
Jaydeb Sarker, imranraad |
This study investigates toxicity in GitHub bug report discussions, revealing its negative impacts on collaboration and resolution. The main research objective was to analyze how toxicity manifests in bug reports and impacts developers’ bug resolution. The researchers performed a qualitative analysis of 203 bug threads (including 81 toxic ones) from GitHub, selected using stratified sampling and toxicity detection tools (ToxiCR and LLaMA). A primary result was that only 29.11% of toxic bug report issues were linked with a Pull Request, lower than percentages reported in prior studies. The principal implication for AI practitioners is that automated systems for bug severity/priority management, combined with enhanced toxicity detection tools incorporating domain-specific knowledge, are needed to improve communication and efficiency in software projects. |
| PerCoV2: Improved Ultra-Low Bit-Rate Perceptual Image Compression with |
|
|
| Implicit Hierarchical Masked Image Modeling (Read more on arXiv or HuggingFace) |
Daniel Mueller-Gritschneder, Sascha Hauke, HerrSiebert, edukrom, Nikolai10 |
PerCoV2 is an open ultra-low bit-rate perceptual image compression system built upon Stable Diffusion 3, enhancing entropy coding through explicit modeling of the discrete hyper-latent image distribution. The main research objective is to improve ultra-low bit-rate image compression while maintaining perceptual quality by using an implicit hierarchical masked image modeling approach. The key methodology involves extending the PerCo framework to Stable Diffusion 3 and comparing autoregressive methods (VAR and MaskGIT) for entropy modeling within a two-stage training protocol. Results on the MSCOCO-30k benchmark show that PerCoV2 achieves higher image fidelity at lower bit-rates than previous methods, with the QLDS masking schedule achieving a 6.34% bit-rate saving over the baseline in the ultra-low bit-rate setting. For AI practitioners, PerCoV2 offers a publicly available, state-of-the-art, ultra low bit-rate image compression approach that, in comparison to previous works, particularly excels at the ultra low-extreme bit rates (0.003-0.03bpp). |
| On the Limitations of Vision-Language Models in Understanding Image |
|
|
| Transforms (Read more on arXiv or HuggingFace) |
Saquib Sarfraz, Hasnain Ali, Ahmad Mustafa Anis |
This paper investigates the limitations of Vision-Language Models (VLMs) in comprehending basic image transformations. The main research question is: “Can Vision Language Embedding Models understand simple Image Transformations?”. The researchers created an augmented Flickr8k dataset and evaluated CLIP and SigLIP models’ ability to associate image transformations with textual descriptions and classify transformations. Key results showed that SigLIP Base 256 Multilingual achieved only 47.21% accuracy in understanding augmented descriptions (Experiment 1), and all the VLMs model cannot classify the image transformation correctly. For AI practitioners, the principal implication is that current VLMs, despite strong semantic understanding, have significant limitations in understanding fundamental image transformations which can significantly limit downstream applications of image editing. |
Papers for 2025-03-13
| Title |
Authors |
Summary |
| TPDiff: Temporal Pyramid Video Diffusion Model (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, Lingmin Ran |
TPDiff is a framework that enhances video diffusion model efficiency by using progressively increasing frame rates during the diffusion process. The main research objective is to reduce the high computational demands of training and inference in video diffusion models. The key methodology is a temporal pyramid approach that divides diffusion into stages, increasing frame rate with each stage, combined with a stage-wise diffusion training framework leveraging data-noise alignment. The primary results demonstrate a 50% reduction in training cost and a 1.5x improvement in inference efficiency compared to vanilla diffusion models. For AI practitioners, TPDiff offers a method to substantially reduce computational requirements in video generation with diffusion models, enabling faster training and more efficient inference. |
| Reangle-A-Video: 4D Video Generation as Video-to-Video Translation (Read more on arXiv or HuggingFace) |
Jong Chul Ye, Suhyeon Lee, hyeonho-jeong-video |
Reangle-A-Video introduces a framework for generating synchronized multi-view videos from a single input video without using multi-view generative priors. The main research objective is to develop a method for synchronized multi-view video generation from a single monocular video, reframing it as a video-to-video translation task. The methodology involves two stages: (1) Multi-View Motion Learning using self-supervised fine-tuning of an image-to-video diffusion transformer on warped videos, and (2) Multi-View Consistent Image-to-Images Translation using warped and inpainted first frames guided by a multi-view stereo reconstruction network. The proposed method achieves a MEt3R score of 0.0412 for static view transport, outperforming the Vanilla CogVideoX baseline. For AI practitioners, this work provides a new approach to multi-view video generation that leverages existing image and video diffusion priors, removing the need for large-scale 4D datasets and enabling dynamic camera control and static view transport from a single video input. |
| Block Diffusion: Interpolating Between Autoregressive and Diffusion |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Zhixuan Qi, Zhihan Yang, Justin T Chiu, Aaron Gokaslan, Marianne Arriola |
Block Diffusion Language Models (BD3-LMs) interpolate between discrete denoising diffusion and autoregressive models, enabling flexible-length generation and improved inference efficiency. The main research objective is to introduce and evaluate a class of language models that overcome limitations of both autoregressive and diffusion models, specifically addressing fixed-length generation, inference inefficiency, and perplexity gaps. The key methodology involves defining an autoregressive distribution over blocks of tokens, where the conditional probability of each block is specified by a discrete denoising diffusion model, and employing custom training algorithms and data-driven noise schedules. On the LM1B benchmark, BD3-LMs achieved a test perplexity of 28.23 with a block size of 4, outperforming previous diffusion models and closing gap with the AR perplexity of 22.88 . AI practitioners can leverage BD3-LMs for generating arbitrary-length sequences with improved likelihood modeling compared to standard diffusion models, and with parallel generation capabilities beyond autoregressive models. |
| RewardSDS: Aligning Score Distillation via Reward-Weighted Sampling (Read more on arXiv or HuggingFace) |
Sagie Benaim, Guy Yariv, Itay Chachy |
RewardSDS is a novel score distillation approach that aligns diffusion models with user intent using reward-weighted sampling. The main research objective is to improve the alignment of score distillation sampling (SDS) outputs with user intent in tasks such as text-to-3D generation. The key methodology is RewardSDS, which weights noise samples during score distillation based on alignment scores from a reward model, prioritizing gradients from samples yielding high-reward outputs. Primary results show that RewardSDS and RewardVSD improve over SDS and VSD on text-to-image generation, with ImageReward achieving a 7.19 LLM Grader score compared to 6.74 for the SDS baseline. AI practitioners can utilize RewardSDS as a plug-and-play module to enhance existing SDS-based methods, improving generation quality and alignment with desired reward models in various tasks, including text-to-image and text-to-3D generation. |
| GTR: Guided Thought Reinforcement Prevents Thought Collapse in RL-based |
|
|
| VLM Agent Training (Read more on arXiv or HuggingFace) |
Zongqing Lu, Yuanchun Shi, Junliang Xing, Yijun Yang, Tong Wei |
GTR is a framework that prevents “thought collapse” in reinforcement learning-trained vision-language model (VLM) agents by integrating automated thought correction. The main research objective is to investigate and mitigate the phenomenon of “thought collapse” – a degradation of reasoning ability – observed when training VLM agents with RL in visually-grounded environments. The key methodology is Guided Thought Reinforcement (GTR), which uses an off-the-shelf VLM as a corrector to evaluate and refine the agent’s chain-of-thought reasoning at each RL step, combined with SFT thought cloning and PPO updates. Primary results demonstrate that GTR significantly improves performance, achieving a 3-5x higher task success rate on the Points24 card game compared to state-of-the-art methods. Principal implication for AI practioners is that incorporating process-level guidance via automated thought correction during RL training can substantially enhance the decision-making capabilities and generalization of VLM agents in complex visual environments. |
| More Documents, Same Length: Isolating the Challenge of Multiple |
|
|
| Documents in RAG (Read more on arXiv or HuggingFace) |
Gabriel Stanovsky, Michael Hassid, Nir Mazor, Shahar Levy, LihiShalmon |
Retrieval-augmented generation (RAG) performance can degrade with more documents, even with a fixed context length. The main research objective was to isolate the effect of the number of retrieved documents on LLM performance in RAG systems, while controlling for context length. Researchers used a modified multi-hop QA dataset (MuSiQue) to create inputs with varying numbers of documents, but a constant total token count, by expanding remaining documents when others were removed. Primary result was increasing documents from 2-4 to 20 can decrease performance by up to 10% on several tested models (Llama-3.1, Gemma-2). The principal implication is that AI practitioners should consider the number of retrieved documents in RAG systems, as increasing their number without also changing the context may worsen system performance. |
| Quantizing Large Language Models for Code Generation: A Differentiated |
|
|
| Replication (Read more on arXiv or HuggingFace) |
Gabriele Bavota, Saima Afrin, Antonio Mastropaolo, mdiipenta, Devy1 |
This paper investigates the impact of quantizing large language models (LLMs) on code generation performance, focusing on extreme quantization levels and code-specific calibration datasets. The main research question is how low-bit quantization, different calibration datasets, and model size affect the code generation ability of LLMs. The key methodology involves quantizing CodeLlama and DeepSeek-Coder models to 8, 4, 3, and 2 bits using AQLM, with various calibration datasets, and evaluating performance on MultiPL-E and McEval benchmarks using the pass@1 metric. A primary result is that 4-bit quantization reduces model memory footprint by 70% with no significant performance decrease, while code-specific calibration datasets improve performance at more extreme (3 and 2-bit) quantization levels. AI practitioners can deploy larger code generation models on resource-constrained devices by safely quantizing LLMs down to 4 bits without sacrificing significant performance. |
| WildIFEval: Instruction Following in the Wild (Read more on arXiv or HuggingFace) |
Liat Ein-Dor, Ariel Gera, Asaf Yehudai, Gili Lior |
WILDIFEVAL introduces a large-scale dataset of real user instructions with multiple constraints to evaluate LLMs’ instruction-following capabilities. i) WILDIFEVAL, a new benchmark of 12K real-world, multi-constrained user instructions, is introduced to evaluate instruction following in LLMs. ii) The main research objective is to assess how well leading LLMs can follow complex, real-world instructions with multiple constraints. iii) Key methodology involved collecting and curating real user instructions from Chatbot Arena, decomposing them into individual constraints, and evaluating LLM performance based on the fraction of fulfilled constraints. iv) The best-performing model achieved a score of 0.65, and all models experienced performance degradation with an increasing number of constraints. v) AI practitioners should focus on improving LLMs’ ability to handle multiple, diverse constraints, particularly length-related constraints, to better align with realistic user needs and expectations in complex text generation tasks. |
| VLog: Video-Language Models by Generative Retrieval of Narration |
|
|
| Vocabulary (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, KevinQHLin |
VLog is a video understanding framework that defines video narrations as vocabulary and uses a generative retrieval model for efficient indexing. The main research objective is to develop a video understanding model that generates concise, contextually accurate, and efficient narrations. The key methodology involves a generative retrieval model, a hierarchical vocabulary derived from video narrations using Narration Pair Encoding, and a vocabulary update strategy leveraging generative models. VLog achieves a 20x speedup over generative models on the Vidcab-Eval dataset while maintaining comparable accuracy to retrieval models. AI practitioners can use VLog’s generative retrieval approach to create more efficient video-language models, achieving faster processing speeds with accuracy, especially when handling long videos or requiring real-time responses. |
| Cost-Optimal Grouped-Query Attention for Long-Context LLMs (Read more on arXiv or HuggingFace) |
Maosong Sun, Zhiyuan Liu, Xu Han, Yutong Wu, chen-yingfa |
The paper investigates cost-optimal configurations for Grouped-Query Attention (GQA) in Transformer-based large language models (LLMs), focusing on trade-offs between performance, computational cost, and memory usage. The main research question is how to optimize the number of attention heads and groups in GQA to minimize computational and memory costs of LLMs while maximizing language modeling capabilities, particularly in long-context scenarios. The key methodology involves systematically comparing LLMs with varying parameter sizes, context lengths, and attention head configurations, extending existing scaling laws to account for context length and attention head configuration. A primary result is that for Llama-3.2-1B at 128K context length, using a head configuration of H=(8,1) and increasing the model size can achieve the same loss while reducing inference memory and FLOPs usage by 48.4% and 49.6% respectively, relative to the standard GQA configuration. The principal implication for AI practitioners is that commonly used GQA configurations can be significantly suboptimal, and carefully selecting the attention head configuration, based on expected inference context length, can substantially reduce computational and memory costs, enabling more efficient deployment of long-context LLMs. |
| Alias-Free Latent Diffusion Models:Improving Fractional Shift |
|
|
| Equivariance of Diffusion Latent Space (Read more on arXiv or HuggingFace) |
Xingang Pan, Shuai Yang, Zeqi Xiao, SingleZombie |
Alias-Free Latent Diffusion Models (AF-LDM) improve the shift-equivariance of diffusion models for more consistent image generation. The main research objective is to enhance the fractional shift-equivariance of Latent Diffusion Models (LDMs) to improve consistency in applications like video editing and image-to-image translation. The key methodology involves redesigning attention modules to be shift-equivariant, proposing an equivariance loss to suppress feature bandwidth, and using cross-frame attention in both training and inference. The primary results show that AF-LDM achieves a Latent SPSNR of 40.94 and an Image SPSNR of 28.06 on the FFHQ dataset, demonstrating significantly improved shift-equivariance compared to vanilla LDM. The principal implication for AI practitioners is that they can use AF-LDM to achieve greater consistency and stability in image and video generation tasks requiring shift-equivariance, enabling improved performance in applications like video editing and image-to-image translation. |
| Self-Taught Self-Correction for Small Language Models (Read more on arXiv or HuggingFace) |
Irina Nikishina, Chris Biemann, VityaVitalich |
The paper introduces the Self-Taught Self-Correction (STaSC) algorithm, enabling small language models (SLMs) to improve their outputs through iterative fine-tuning on self-generated data. The main research objective is to investigate if SLMs can learn self-correction without external information or evaluators, relying solely on intrinsic knowledge. The key methodology is iterative fine-tuning of SLMs using self-generated trajectories, incorporating flexible design choices for initial answer generation, correction filtering, and fine-tuning strategy. Primary results show that on the Natural Questions dataset, the Phi3-Mini model achieved a maximum reward of 0.394 (correction, Improving filter) with Evolving Fine-tuning, with a general observation is that both models’ initial answer’s accuracy also increased by training to improve. The STaSC algorithm allows AI practitioners to develop and deploy more accurate and efficient SLMs, enhancing their reasoning and output quality even with limited external resources. |
| MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented |
|
|
| Generation System (Read more on arXiv or HuggingFace) |
Simin Niu, Hanyu Wang, Zhaoxin Fan, Zhiyuan Ji, Robot2050 |
This paper introduces a framework called Mixture-of-Chunkers (MoC) to improve text chunking in Retrieval-Augmented Generation (RAG) systems. The main research objective is to optimize text chunking, a commonly overlooked component of RAG, to improve the quality of retrieved content and subsequently enhance the accuracy of generated answers. The key methodology involves a three-stage process: a multi-granularity-aware router, specialized meta-chunkers, and a post-processing algorithm, using regex-guided chunking and edit-distance rectification. Primary results show that the Meta-chunker-1.5B achieved a BLEU-1 score of 0.3754, and F1 score of 0.2387 on the DuReader dataset, outperforming several baseline methods. For AI practitioners, the proposed MoC framework and evaluation metrics offer a way to enhance RAG system performance by optimizing the text chunking process, a critical yet often under-optimized component of the architecture. |
| Multimodal Language Modeling for High-Accuracy Single Cell |
|
|
| Transcriptomics Analysis and Generation (Read more on arXiv or HuggingFace) |
Xiang Wang, Junfeng Fang, Sihang Li, Jiaqi Yang, Yaorui Shi |
scMMGPT is a multimodal pre-trained language model for joint cell and text modeling in single-cell transcriptomics. The main research objective is to develop a unified model that effectively integrates scRNA-seq data and textual descriptions to improve performance on single-cell analysis tasks. The key methodology involves integrating pre-trained cell (scGPT) and text (Llama-2) PLMs using cross-modal projectors, and pre-training on 27 million cells with tasks including cell-text representation alignment, cell description generation, and pseudo-cell generation. Primary results include an 84% relative improvement in textual discrepancy for cell description generation compared to existing methods. The principal implication for AI practitioners is that scMMGPT provides a powerful tool for single-cell analysis and generation, demonstrating superior ability to bridge the modality gap between transcriptomic data and free text descriptions. |
| When Large Vision-Language Model Meets Large Remote Sensing Imagery: |
|
|
| Coarse-to-Fine Text-Guided Token Pruning (Read more on arXiv or HuggingFace) |
Qi Zhu, Kang Wu, Xue Yang, Yingying Zhang, Junwei Luo |
This paper introduces a text-guided token pruning method for efficient processing of large remote sensing images (RSIs) by Large Vision-Language Models (LVLMs). The main research objective is to balance image detail and computational cost when LVLMs process large RSIs. The key methodology involves a Region Focus Module (RFM) for text-aware region localization and a Dynamic Image Pyramid (DIP) for coarse-to-fine image tile selection and vision token pruning. The method achieved a 32.16% average accuracy on the new LRS-VQA benchmark, outperforming existing high-resolution strategies. AI practitioners can utilize this approach to build more efficient LVLMs for high-resolution image analysis, particularly beneficial when dealing with limited computing resources or large images. |
| Multi Agent based Medical Assistant for Edge Devices (Read more on arXiv or HuggingFace) |
Pragya Sahu, Jagdish Samant, Chinmay Kulkarni, Shivam Akhouri, Sakharam Gawade |
This paper introduces an on-device, multi-agent healthcare assistant that leverages task-specific agents for optimized resource utilization, privacy, and scalability. The main research objective is to develop a healthcare assistant for edge devices that addresses privacy, latency, and internet dependency challenges associated with cloud-based systems. The key methodology involves a multi-agent architecture utilizing specialized, smaller models (based on Qwen Code Instruct 2.5 7B) for tasks like intelligent diagnosis, appointment booking, emergency services, vital tracking, and reminder scheduling, combined with a data creation pipeline for synthetic data generation. The fine-tuned planner and caller agents achieved an average RougeL score of 85.5 for planning and 96.5 for calling, respectively, for appointment scheduling. This architecture enables AI practitioners to deploy robust and efficient healthcare solutions on resource-constrained edge devices, enhancing user privacy and responsiveness without relying on continuous internet access. |
| Monte Carlo Diffusion for Generalizable Learning-Based RANSAC (Read more on arXiv or HuggingFace) |
Tong Zhang, Wei Ke, Chen Zhao, Jiale Wang |
This paper introduces a Monte Carlo diffusion mechanism to improve the generalization of learning-based RANSAC for robust model estimation. The main research objective is to address the limited generalization of existing learning-based RANSAC methods to out-of-distribution data. The key methodology involves a diffusion-based training paradigm that progressively injects noise into ground-truth data and uses Monte Carlo sampling to approximate diverse data distributions. Primary results show that on ScanNet, the proposed method improves AUC @20° by 12% on LoFTR compared to a model trained only on SIFT. For AI practitioners, this provides a training strategy to enhance the generalization ability of learning-based RANSAC estimators across various input data distributions without retraining. |
Papers for 2025-03-12
| Title |
Authors |
Summary |
| Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural |
|
|
| Vision-Language Dataset for Southeast Asia (Read more on arXiv or HuggingFace) |
davidanugraha, rifqifarhansyah, tackhwa, holylovenia, samuelcahyawijaya |
SEA-VL is an open-source initiative to develop a vision-language dataset representing Southeast Asian cultures, addressing their underrepresentation in AI research. The main objective is to create a high-quality, culturally relevant vision-language dataset for Southeast Asian (SEA) languages and assess different data collection strategies. The researchers employ a multi-pronged approach that includes crowdsourcing, crawling existing image corpora, and generating synthetic images using diffusion models, followed by human evaluation. Crawling achieves approximately 85% cultural relevance and is more cost- and time-efficient than crowdsourcing, while image generation models are currently found unreliable for accurately reflecting SEA cultures. AI practitioners can leverage this dataset to develop more inclusive vision-language models and should prioritize crawling over generation for efficient collection of culturally relevant visual data. |
| LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through |
|
|
| Two-Stage Rule-Based RL (Read more on arXiv or HuggingFace) |
Jie Liu, Zhiyuan You, Miaosen Zhang, Gongrui Zhang, Yingzhe Peng |
i) This paper introduces LMM-R1, a two-stage rule-based RL framework for enhancing reasoning abilities in Large Multimodal Models (LMMs). ii) The main objective is to improve the reasoning capabilities of compact 3B-parameter LMMs, particularly in multimodal contexts. iii) The methodology involves Foundational Reasoning Enhancement (FRE) using text-only data and Multimodal Generalization Training (MGT) to extend reasoning to multimodal domains. iv) Results on Qwen2.5-VL-Instruct-3B show LMM-R1 achieves 4.83% and 4.5% average improvements over baselines in multimodal and text-only benchmarks, respectively, and a 3.63% gain in Football Game tasks. v) LMM-R1 provides AI practitioners with a data-efficient approach to enhance reasoning in LMMs by leveraging text-based reasoning enhancement for effective multimodal generalization. |
| YuE: Scaling Open Foundation Models for Long-Form Music Generation (Read more on arXiv or HuggingFace) |
HKUST-Audio, Liam-Liu, dododododo, zhangysk, a43992899 |
YuE is a family of open foundation models for long-form, lyrics-to-song music generation based on the LLaMA2 architecture. The main research objective is to develop a system capable of generating high-quality, long-form (up to five minutes) music with coherent structure, lyrical alignment, and engaging vocal melodies from lyrics and other control signals. The key methodology involves a track-decoupled next-token prediction strategy with dual-token output (vocal and accompaniment), structural progressive conditioning using a Chain-of-Thought-like approach, a redesigned music in-context learning framework, and a multitask, multiphase pre-training recipe. Primary results include outperforming or matching several proprietary systems (e.g., Suno, Udio) in human evaluations of musicality, and achieving a mean vocal range of approximately 27 semitones, comparable to closed-source systems. The principal implication for AI practitioners is that YuE provides an open, scalable, and performant approach to full-song lyrics-to-music generation, offering improved controllability and competitive quality to existing proprietary alternatives. |
| UniF^2ace: Fine-grained Face Understanding and Generation |
|
|
| with Unified Multimodal Models (Read more on arXiv or HuggingFace) |
Liya Guo, Linrui Xu, Xuerui Qiu, delinqu, tulvgengenr |
UniF²ace is a unified multimodal model designed for fine-grained face understanding and generation tasks, trained on a new specialized dataset. The main research objective is to develop a single model capable of both understanding (image-to-text) and generating (text-to-image) fine-grained facial attributes with high accuracy. The key methodology involves a combination of autoregressive and diffusion models, optimized using a dual discrete diffusion training strategy and a two-level mixture-of-experts architecture, trained on the self-constructed UniF²ace-130K dataset. The primary results show that UniF²ace achieves a FID score of 66.005 and a VLM-score of 88.049 on the UniF²ace-130K test dataset, outperforming existing unified multimodal models and approaching state-of-the-art generative models. The principal implication for AI practitioners is that a unified model, leveraging both score-based and masked generative models with a specialized architecture, can achieve high performance in both detailed facial image understanding and generation, potentially streamlining the development of face-related AI applications. |
| SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by |
|
|
| Imitating Human Annotator Trajectories (Read more on arXiv or HuggingFace) |
Qingpei Guo, Chunluan Zhou, Hao Chen, Yuzhuo Tian, Z-MU-Z |
SegAgent introduces a new segmentation framework where Multimodal Large Language Models (MLLMs) mimic human annotators using interactive tools to enhance pixel-level understanding. The main research objective is to develop and evaluate a method for MLLMs to perform fine-grained pixel-level image segmentation by imitating human annotation trajectories. The key methodology is modeling segmentation as a multi-step Markov Decision Process (HLMAT), where MLLMs generate text-based click points iteratively, and adapting policy improvement methods like StaR and process reward modeling (PRM) guided tree search. The primary result is that SegAgent-LLaVA+SAM achieved a 75.72 cIoU on the refCOCO testB dataset, demonstrating performance comparable to state-of-the-art methods. Principal implication for AI practitioners is a new protocol to train and assess the fine-grained visual understanding capabilities of MLLMs on pixel segmentation and interactive tasks. |
| MagicInfinite: Generating Infinite Talking Videos with Your Words and |
|
|
| Voice (Read more on arXiv or HuggingFace) |
Jiantong Zhao, Xuancheng Yang, Shitong Shao, Hongwei Yi, Owen777 |
MagicInfinite is a diffusion Transformer framework for generating high-fidelity, infinite-length talking head videos controlled by audio and text. The main research objective is to overcome limitations of existing portrait animation methods in handling diverse character styles, achieving accurate lip synchronization, and enabling efficient long video generation. The key methodology involves a 3D full-attention mechanism with a sliding window denoising strategy, a two-stage curriculum learning scheme (integrating audio, text, and reference images), and region-specific masks with adaptive loss functions. Primary results show that MagicInfinite achieves a 20x inference speed boost over the basemodel and can generate a 10-second 540x540p video in 10 seconds on 8 H100 GPUs without quality loss. For AI practitioners, this framework offers an efficient way to generate high-quality, controllable, and arbitrarily long talking head animations with strong temporal coherence. |
| Seedream 2.0: A Native Chinese-English Bilingual Image Generation |
|
|
| Foundation Model (Read more on arXiv or HuggingFace) |
Liang Li, Fanshi Li, Xiaoxia Hou, Lixue Gong, wujie10 |
Seedream 2.0 is a bilingual Chinese-English text-to-image diffusion model that addresses limitations of existing models in cultural understanding, text rendering, and model bias. The main research objective is to develop a foundation model capable of generating high-fidelity images aligned with both Chinese and English prompts, demonstrating superior performance in multiple aspects, including text rendering and understanding of Chinese cultural nuances. The key methodology includes a multi-level optimization framework that integrates a bilingual LLM text encoder, a Glyph-Aligned ByT5 for character-level text rendering, Scaled ROPE, multi-phase post-training (SFT, RLHF), and a data system for continuous knowledge integration. The primary result is that Seedream 2.0 achieves state-of-the-art performance, outperforming models like Midjourney v6.1 and Ideogram 2.0 in human evaluations, with a human evaluation ELO score of 1117, and demonstrating a 78% text accuracy rate and 82% hit rate in Chinese text rendering. Principal implication for AI practitioners is that Seedream 2.0 provides a robust and culturally aware foundation model for bilingual image generation, particularly effective for applications requiring accurate Chinese text rendering and culturally specific content generation, outperforming widely available text-to-image models in the field. |
| Gemini Embedding: Generalizable Embeddings from Gemini (Read more on arXiv or HuggingFace) |
Madhuri Shanbhogue, Daniel Cer, Sahil Dua, Feiyang Chen, Jinhyuk Lee |
Gemini Embedding is a new state-of-the-art text embedding model that leverages the Gemini large language model for improved generalizability across languages and tasks. The main research objective is to develop a unified embedding model that achieves state-of-the-art performance across a broad range of multilingual text embedding tasks. The key methodology involves initializing the embedding model from Gemini, curating a high-quality training dataset using Gemini, and employing a two-stage training pipeline (pre-finetuning and finetuning) with a contrastive learning objective, culminating with model souping. The primary result is that Gemini Embedding achieves a mean task score of 68.32 on the Massive Multilingual Text Embedding Benchmark (MMTEB), outperforming prior state-of-the-art models. The principal implication for AI practitioners is that they can leverage Gemini Embedding as a highly generalizable, off-the-shelf solution for various downstream tasks, including classification, similarity, clustering, ranking and retrieval, particularly in multilingual settings. |
| LightGen: Efficient Image Generation through Knowledge Distillation and |
|
|
| Direct Preference Optimization (Read more on arXiv or HuggingFace) |
Yexin Liu, Harold Haodong Chen, Haoze Zheng, Yajing Bai, Xianfeng Wu |
LightGen is an efficient text-to-image generation model that uses knowledge distillation and direct preference optimization to reduce computational costs. The main research objective is to develop a text-to-image generation model that achieves comparable performance to state-of-the-art (SOTA) models with significantly reduced computational resources and dataset size. The key methodology involves distilling knowledge from SOTA text-to-image models into a compact Masked Autoregressive (MAR) architecture using a synthetic dataset and refining the output with Direct Preference Optimization (DPO). The model achieves an overall performance score of 0.62 on the GenEval benchmark at 512x512 resolution using only 0.7B parameters and a 2M image dataset. AI practitioners can use LightGen to develop high-quality image generation models with limited computational resources and smaller datasets, achieving performance similar to much larger and resource intensive models. |
| Tuning-Free Multi-Event Long Video Generation via Synchronized Coupled |
|
|
| Sampling (Read more on arXiv or HuggingFace) |
Jinwoo Shin, Joon-Young Lee, Jui-Hsien Wang, Seoung Wug Oh, Subin Kim |
The paper introduces SynCoS, a tuning-free inference framework for generating multi-event long videos from text prompts using existing text-to-video diffusion models. The main research objective is to extend text-to-video diffusion models for long-form video generation with multiple events while maintaining local smoothness and global coherence. The key methodology, Synchronized Coupled Sampling (SynCoS), combines reverse and optimization-based sampling (DDIM and CSD) with a grounded timestep and fixed baseline noise to synchronize denoising paths across the entire video. SynCoS achieved a subject consistency score of 90.19% on Open-Sora Plan, outperforming baselines. AI practitioners can utilize SynCoS to extend existing diffusion models for high-quality, multi-event, and coherent, long video generation without additional model training. |
| Implicit Reasoning in Transformers is Reasoning through Shortcuts (Read more on arXiv or HuggingFace) |
Deqing Yang, Siyu Yuan, Tianhe Lin, hsaest |
Transformers trained for implicit multi-step reasoning rely on shortcuts rather than true step-by-step computation, limiting generalization. The main research question is how language models perform implicit reasoning in multi-step tasks, and why advanced reasoning capabilities observed in explicit reasoning do not emerge in implicit reasoning. The researchers trained GPT-2 models from scratch on a synthetic multi-step mathematical reasoning dataset and used activation patching for analysis. Results showed that models trained on data with unfixed premise order had significantly reduced accuracy; for instance, accuracy dropped to ~40% on 5-step reasoning tasks. The principal implication for AI practitioners is that current language models may achieve high performance on tasks with similar patterns through shortcut learning without genuine generalization, particularly in implicit reasoning scenarios. |
| OmniMamba: Efficient and Unified Multimodal Understanding and Generation |
|
|
| via State Space Models (Read more on arXiv or HuggingFace) |
Xinggang Wang, Wenyu Liu, Qian Zhang, Bencheng Liao, Jialv Zou |
OmniMamba is a Mamba-based unified multimodal model for both understanding and generation tasks. The main research objective is to develop a unified multimodal generation model that achieves both training and inference efficiency with limited training data. The key methodology involves using a linear-architecture-based Mamba-2, decoupled vocabularies, task-specific LoRA, and a decoupled two-stage training strategy. OmniMamba achieves competitive performance with JanusFlow and surpasses Show-o across benchmarks while using only 2M image-text pairs, demonstrating up to 119.2x speedup and 63% GPU memory reduction. AI practitioners can leverage OmniMamba’s efficient architecture and training strategies for developing multimodal models with reduced computational cost and data requirements. |
| Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning (Read more on arXiv or HuggingFace) |
Edward Emanuel Beeching, Lewis Tunstall, Amrith Setlur, Matthew Y. R. Yang, CohenQu |
This paper introduces Meta Reinforcement Fine-Tuning (MRT), a method to optimize test-time compute for large language models (LLMs) by minimizing cumulative regret. The main research question is whether current LLMs efficiently utilize test-time compute and whether scaling approaches continue to be effective as budget improves. The key methodology is to formalize test-time compute optimization as a meta-reinforcement learning problem, using a dense reward bonus based on “progress” quantified by the change in likelihood of eventual success. MRT leads to a 2-3x relative gain in performance and roughly a 1.5x gain in token efficiency for math reasoning compared to outcome-reward RL. For AI practitioners, MRT provides a new fine-tuning method that improves LLM performance and efficiency by optimizing for progress during inference, enabling better utilization of computational resources. |
| Video Action Differencing (Read more on arXiv or HuggingFace) |
Alejandro Lozano, Anita Rau, Yuhui Zhang, nicholswang, jmhb |
This paper introduces Video Action Differencing (VidDiff), a new task and benchmark for identifying subtle differences between videos of the same action. The main research question is how to identify and describe fine-grained differences between two videos of individuals performing the same action. The key methodology is a three-stage agentic workflow (VidDiff Method) that leverages large language models (LLMs) for difference proposal, CLIP for frame localization, and vision-language models (VLMs) for frame differencing. The primary result is that the proposed VidDiff Method achieves a closed-set accuracy of 56.3%, outperforming GPT-40 (53.5%) and Gemini-1.5 Pro (57.7%), and its open set recall@N is 42.1. AI practitioners can use the VidDiffBench dataset and the VidDiff Method as a benchmark and baseline for developing and evaluating models capable of fine-grained video understanding and comparison, essential for applications like skill learning, coaching and automated performance feedback. |
| ^RFLAV: Rolling Flow matching for infinite Audio Video generation (Read more on arXiv or HuggingFace) |
Claudio Ferrari, Tomaso Fontanini, Filippo Botti, Giuseppe Gabriele Tarollo, MaverickAlex |
RFLAV is a novel transformer-based architecture for infinite and synchronized audio-video generation. The main research objective is to address the limitations of existing audio-video generation models regarding quality, multimodal synchronization, and duration. The key methodology is a rolling rectified-flow model with a lightweight temporal cross-modality fusion module that processes audio and video in separate branches before combining them. The proposed RFLAV model achieves a FVD score of 38.36 on the AIST++ dataset with 200 denoising steps, surpassing existing state-of-the-art models. For AI practitioners, this model offers an improved method for generating arbitrarily long, high-quality audio-video sequences without the duration constraints of prior methods. |
| “Principal Components” Enable A New Language of Images (Read more on arXiv or HuggingFace) |
Xiaojuan Qi, Jiankang Deng, Ismail Elezi, tennant, xwen99 |
“Principal Components” Enable A New Language of Images introduces a visual tokenization framework with a provable PCA-like structure in the latent token space. The main research objective is to create a compact, structured image representation that reduces redundancy while effectively decoupling semantic information from less important low-level details in 1D visual tokenizers. The key methodology involves a dynamic nested classifier-free guidance strategy during training to induce an orderliness bias in tokens, combined with a diffusion-based decoder. The approach achieves a state-of-the-art reconstruction FID score of 0.72 on the ImageNet validation set, a 10% improvement over prior methods. For AI practitioners, this method provides a way to generate more interpretable and efficient visual representations, suitable for tasks such as image reconstruction and auto-regressive generative modeling, with fewer tokens for training and inference. |
| BiasEdit: Debiasing Stereotyped Language Models via Model Editing (Read more on arXiv or HuggingFace) |
Julian McAuley, Ningyu Zhang, Wei Xu, XinXuNLPer |
BIASEDIT is a model editing method for debiasing stereotyped language models by modifying model parameters with lightweight editor networks. The main research objective is to develop an efficient method to remove stereotypical biases from language models without significantly impacting their language modeling capabilities. The key methodology involves training editor hyper-networks using a debiasing loss and a retention loss to generate parameter updates that locally modify a language model’s parameters related to stereotyped biases. Results show that BIASEDIT reduces Stereotype Score (SS) to less than 57% and more than 46% on various LMs, outperforming baselines, while maintaining language modeling scores with small changes. For AI practitioners, BIASEDIT offers a computationally efficient method to mitigate societal biases within pre-trained language models, enabling the development of fairer and more robust NLP applications, and bias editing on upper blocks of language models had fewer negative impacts. |
| QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long |
|
|
| Video Comprehension (Read more on arXiv or HuggingFace) |
Shukang Yin, Weizhong Huang, Xiawu Zheng, Wang Chen, Yongdong Luo |
QuoTA is a training-free framework for long video understanding that enhances existing LVLMs by assigning visual tokens based on query relevance. The main research objective is to improve long-video comprehension in Large Video-Language Models (LVLMs) by mitigating visual redundancy and aligning visual processing with task-specific requirements. The key methodology involves query-oriented frame-level importance assessment using Chain-of-Thoughts reasoning to decouple the query, parallel video frame evaluation with a scoring LVLM, and dynamic visual token assignment based on the generated scores. Implementing QuoTA with LLaVA-Video-7B yields an average performance improvement of 3.2% across six video understanding benchmarks, including Video-MME and MLVU. The principal implication for AI practitioners is that QuoTA offers a plug-and-play module to improve existing LVLMs’ long video understanding capabilities without additional training, enabling more effective processing aligned with given query. |
| Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents (Read more on arXiv or HuggingFace) |
Xiao Zhang, Liang Pang, Haiyuan Zhao, Sunhao Dai, Haoyu Wang |
PLM-based retrieval models exhibit a “perplexity trap,” overrating documents with low perplexity, leading to source bias that favors LLM-generated content. The main research question is why PLM-based retrievers prefer low-perplexity documents, even when semantic quality is comparable to human-written ones. The authors employ causal graphs, two-stage least squares (2SLS) regression, and theoretical analysis linking retrieval and language modeling objectives. Results show a consistently negative causal effect of perplexity on relevance scores across multiple datasets and models; for example, on the TREC-COVID dataset, ANCE showed a coefficient of -0.23 (p=0.15). A causal-inspired debiasing method, Causal Diagnosis and Correction (CDC), is proposed to mitigate this effect, which is valuable for those seeking to remove perplexity-related source bias. |
| RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow |
|
|
| Trajectories (Read more on arXiv or HuggingFace) |
Xing Wang, Yuxi Ren, Yuhong Yang, Xin Xia, Huiyang Shao |
RayFlow is a diffusion model acceleration framework that guides each sample along a unique path to an instance-specific target distribution, improving generation speed and control. The main research objective is to address the slow generation speed, sample quality compromises, and training complexities of existing diffusion model acceleration methods. The key methodology includes guiding each sample along a unique path towards instance-specific target distributions and introducing an importance sampling technique (Time Sampler) for enhanced training efficiency. Primary results show that, on the COCO-5k dataset, the SDXL-Ray model achieved a FID score of 3.90 in a 4-step generation, outperforming several existing methods. A principal implication is that AI practitioners can use RayFlow to generate high-quality images with improved speed, control, and training efficiency compared to existing acceleration techniques. |
| Benchmarking AI Models in Software Engineering: A Review, Search Tool, |
|
|
| and Enhancement Protocol (Read more on arXiv or HuggingFace) |
Maliheh Izadi, philippedebekker, RohamKoohestani |
This paper reviews AI4SE benchmarks, introduces a search tool (BenchScout) and enhancement protocol (BenchFrame), and demonstrates improvements on HumanEval, resulting in HumanEvalNext. The main research objective is to address challenges in AI4SE benchmarking, including knowledge fragmentation, benchmark selection, lack of standardization, and existing benchmark limitations. The key methodology involves a systematic literature review of 204 benchmarks, development of a semantic search tool using clustering and dimensionality reduction, and a case study applying a proposed framework (BenchFrame) for benchmark enhancement through code review, modifications, and peer review. A primary result shows that on HumanEvalNext, language models exhibited a pass@1 score reduction of 31.22% compared to the original HumanEval. The principal implication for AI practitioners is that using refined and rigorously evaluated benchmarks like HumanEvalNext provides a more accurate assessment of model capabilities and guides future AI4SE research, emphasizing the need for continuous benchmark improvement. |
| Referring to Any Person (Read more on arXiv or HuggingFace) |
Yuda Xiong, Tianhe Ren, Zhaoyang Zeng, Lin Wu, Qing Jiang |
This paper introduces “Referring to Any Person,” a new task and model (RexSeek) for detecting all individuals in an image that match a natural language description, along with a new dataset (HumanRef). The main research objective is to develop a model capable of multi-instance person referring, overcoming limitations of existing models and datasets that primarily focus on one-to-one object referring. The key methodology involves integrating a multimodal large language model with an object detection framework, trained in a multi-stage process, and creating a new dataset, HumanRef with 103,028 referring statements. The primary result is that RexSeek achieves a DensityF1 score of 82.3 on the HumanRef benchmark, significantly outperforming existing models like Qwen2.5-VL (31.9 DensityF1). The principal implication is that AI practitioners should leverage this model and the HumanRef for robust referring expression comprehension, especially within the task of referring to any person to enable more precise, multi-instance person detection. |
| AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion |
|
|
| Models (Read more on arXiv or HuggingFace) |
Junyong Noh, Chaelin Kim, Seokhyeon Hong, kwanY |
AnyMoLe is a novel method for generating 3D character motion in-betweening without character-specific datasets, by leveraging video diffusion models. The main research objective is to address the scarcity of character-specific datasets in motion in-betweening, enabling animation generation for arbitrary characters. The key methodology involves a two-stage video generation process using a fine-tuned video diffusion model (ICAdapt), and motion-video mimicking optimization with a scene-specific joint estimator. The primary results show that AnyMoLe outperforms baseline methods in all metrics, achieving an HL2Q of 0.0015 for humanoid characters, demonstrating superior motion generation. For AI practitioners, this implies a reduced reliance on extensive character-specific datasets for motion in-betweening, expanding the applicability of animation generation to a wider range of characters. |
| AI-native Memory 2.0: Second Me (Read more on arXiv or HuggingFace) |
Jingbo Shang, Felix Tao, Tao Gao, Xiang Ying, Jiale Wei |
SECOND ME is an AI-native memory system that acts as an intelligent, persistent memory offload for users. The main research objective is to develop and evaluate an LLM-based system that can retain, organize, and dynamically utilize user-specific knowledge to improve human-computer interaction. The key methodology involves a multi-layer hybrid architecture integrating supervised fine-tuning (SFT) and direct preference optimization (DPO) with automated data synthesis and evaluation using LLMs. A key result is that using diverse data sources with strong Chain-of-Thought (CoT) normalization achieved a 0.91 score in the Memory (Self) evaluation metric. AI practitioners can leverage this fully localizable, open-sourced system’s approach to memory parameterization and multi-agent framework to build more personalized and context-aware AI applications. |
| Mixture of Experts Made Intrinsically Interpretable (Read more on arXiv or HuggingFace) |
Puneet K. Dokania, Christian Schroeder de Witt, Ashkan Khakzar, Constantin Venhoff, Xingyi Yang |
This paper introduces MoE-X, a Mixture-of-Experts language model designed for intrinsic interpretability by leveraging sparsity and width. The main research objective is to design an intrinsically interpretable language model architecture that reduces polysemanticity without relying on post-hoc interpretability methods. The key methodology involves rewriting the MoE layer as an equivalent sparse, wide MLP, enforcing sparse activation within each expert using ReLU, and redesigning the routing mechanism to prioritize experts with the highest activation sparsity. MoE-X achieves a perplexity better than GPT-2 and a chess board state reconstruction score of 0.840, surpassing sparse autoencoder-based approaches. AI practitioners can leverage MoE-X’s architecture for improved interpretability in language models without sacrificing performance, offering a direct path to more transparent and understandable AI systems. |
| NullFace: Training-Free Localized Face Anonymization (Read more on arXiv or HuggingFace) |
Nicu Sebe, Terence Sim, Tuomas Varanka, hkung |
NullFace is a training-free method for localized face anonymization that preserves non-identity facial attributes using diffusion models. The main research objective is to develop a face anonymization technique that balances identity obscuration with the preservation of key non-identity-related attributes, without requiring model training. The key methodology involves inverting an input image using DDPM inversion to recover initial noise, then denoising it through an identity-conditioned diffusion process with modified identity embeddings, and optionally applying segmentation masks for localized control. The method achieved a re-identification rate of 0.34% on the FFHQ dataset, the lowest among compared methods. For AI practitioners, this method offers a flexible and practical approach to face anonymization, achieving competitive performance in privacy-preserving applications without the need for training or fine-tuning, and enabling controllable localized anonymization. |
| Beyond Decoder-only: Large Language Models Can be Good Encoders for |
|
|
| Machine Translation (Read more on arXiv or HuggingFace) |
Qinghong Zhang, Bei Li, Yongyu Mu, Tong Zheng, luoyingfeng |
LaMaTE uses LLMs as encoders within an encoder-decoder architecture for improved machine translation. The main research objective is to explore combining LLMs with NMT by using LLMs for encoding and NMT decoders for efficient and generalizable translation. The key methodology is a two-stage training approach: first pre-training the NMT decoder and adaptor with frozen LLM parameters, then fine-tuning all parameters on a multi-task dataset (ComMT). Primary results show that LaMaTE achieves a COMET score of 82.32 and BLEU score of 33.85, averaging across all tasks in the new ComMT benchmark dataset. Principal implication for AI practitioners is that using LLMs as encoders in encoder-decoder models offers a strong balance between high translation quality, reduced computational cost (2.4-6.5x faster decoding), and generalizability, suggesting a promising direction of research. |
| VisualSimpleQA: A Benchmark for Decoupled Evaluation of Large |
|
|
| Vision-Language Models in Fact-Seeking Question Answering (Read more on arXiv or HuggingFace) |
Lixin Liu, Shasha Guo, Xiaodong Chen, Yihan Zhao, WYLing |
VisualSimpleQA is a new benchmark for evaluating fact-seeking question-answering capabilities of large vision-language models (LVLMs). The main research objective is to introduce a multimodal fact-seeking benchmark that allows for decoupled evaluation of visual and linguistic modules in LVLMs and incorporates well-defined difficulty criteria. The key methodology involves human annotation of samples with multimodal questions, text-only questions, rationales, and difficulty scores based on visual and linguistic factors. Primary results show that even state-of-the-art LVLMs like GPT-4o achieve only 60%+ correctness on multimodal questions in VisualSimpleQA, and 30%+ on a harder subset. The principal implication for AI practitioners is that there is substantial room for improvement in both the visual and linguistic modules of LVLMs for fact-seeking QA, especially regarding challenging visual recognition tasks and knowledge identification. |
Papers for 2025-03-11
| Title |
Authors |
Summary |
| Feature-Level Insights into Artificial Text Detection with Sparse |
|
|
| Autoencoders (Read more on arXiv or HuggingFace) |
Kristian Kuznetsov, natriistorm, razzant, plina2polina, Kushnareva |
This paper explores enhancing interpretability in artificial text detection (ATD) using Sparse Autoencoders (SAEs) to extract features from a Gemma-2-2b model’s residual stream, categorizing them, and analyzing their effectiveness. The main research objective is to improve ATD interpretability by analyzing the semantics and relevance of SAE-extracted features. The key methodology involves applying SAEs to Gemma-2-2b’s residual stream, analyzing extracted features through domain/model-specific statistics, steering, and manual/LLM-based interpretation, and evaluating feature effectiveness using XGBoost and threshold classifiers. A primary result is that SAE-derived features at the 16th layer outperform a state-of-the-art MTL model and mean-pooled activations on the COLING dataset in detecting artificially generated text. For AI practitioners, using SAEs for feature extraction offers a valuable approach for understanding text generators and detectors and their generalization, which helps in developing more robust and interpretable ATD systems. |
| SEAP: Training-free Sparse Expert Activation Pruning Unlock the |
|
|
| Brainpower of Large Language Models (Read more on arXiv or HuggingFace) |
Xun Liang, BO1022, Ki-Seki, siminniu, UglyToilet |
SEAP is a training-free method that prunes large language models (LLMs) by dynamically selecting task-relevant parameters to reduce inference overhead. The main research objective is to develop a pruning technique that reduces computational overhead while maintaining LLM performance on various tasks. The key methodology is Sparse Expert Activation Pruning (SEAP), which identifies task-specific expert activation patterns and prunes the model based on dynamically distributed sparsity. Primary results show that at 50% pruning, SEAP surpasses WandA and FLAP by over 20% in task accuracy on the Llama-2-7B model. The principal implication for AI practitioners is that SEAP provides a scalable and effective approach for optimizing large-scale LLMs, enabling more efficient deployment in resource-constrained environments. |
| MM-Eureka: Exploring Visual Aha Moment with Rule-based Large-scale |
|
|
| Reinforcement Learning (Read more on arXiv or HuggingFace) |
wangwhcore, friskit, hflqf88888, Cierra0506, FanqingM |
MM-Eureka successfully extends large-scale rule-based reinforcement learning (RL) to multimodal reasoning, demonstrating visual “aha moments”. The main research objective was to investigate the effectiveness of large-scale RL in multimodal reasoning and open-source the pipeline. The key methodology involved applying rule-based RL without supervised fine-tuning, using a simple reward function (accuracy and format), and the REINFORCE Leave-One-Out (RLOO) algorithm. MM-Eureka-Zero-38B, trained with only 9.3k image-text data, achieved a 46.4% accuracy on the K12 math test set, surpassing the instruct model and an 8.2% improvement. AI practitioners can use this open-sourced framework and simple RL setup to efficiently improve the multimodal reasoning ability of both instruction-tuned and pre-trained models, with potentially significant data efficiency gains. |
| Taking Notes Brings Focus? Towards Multi-Turn Multimodal Dialogue |
|
|
| Learning (Read more on arXiv or HuggingFace) |
Zongqing Lu, Jiazheng Liu, tellarin, sipeng9527 |
This paper introduces MMDiag, a new multi-turn multimodal dialogue dataset, and DiagNote, a model designed to improve focus and reasoning in such dialogues. The research aims to address the challenge of maintaining focus on target regions in multi-turn multimodal dialogues, specifically “saliency tracking” and “saliency recall”. The key methodology involves a new dataset, MMDiag, generated collaboratively through rules and GPT assistance, and a two-module (Deliberate and Gaze) model, DiagNote, that interacts to perform Chain-of-Thought reasoning and annotations. DiagNote, trained on MMDiag + COCO, achieved a 0.648 average Intersection over Union (IoU) score on grounding benchmarks, outperforming baselines. For AI practitioners, the work provides a new challenging benchmark (MMDiag) and demonstrates improved multimodal grounding and reasoning abilities via the proposed DiagNote model, potentially leading to better handling multi-turn conversational settings. |
| Automated Movie Generation via Multi-Agent CoT Planning (Read more on arXiv or HuggingFace) |
Zeyu Zhu, AnalMom, weijiawu |
MovieAgent is a multi-agent framework for automatically generating long-form videos from a script synopsis and character bank. The main research objective is to automate the process of movie generation, including narrative planning, scene structuring, and shot composition, which traditionally requires extensive manual effort. The key methodology involves a hierarchical Chain-of-Thought (CoT) reasoning process using multiple LLM agents simulating roles like director, screenwriter, and storyboard artist, decomposing the movie generation process into manageable, sequential steps. Primary results show MovieAgent achieving a CLIP score of 22.25 and an Inception score of 9.39 in keyframe generation, with 97.84 motion smoothness in video generation. The principal implication is that AI practitioners can leverage this framework to significantly reduce the cost and time required for movie/long-video production, automating narrative and cinematic planning while ensuring character consistency and narrative coherence. |
| FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA |
|
|
| Subparameter Updates (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, matbambbang, Seanie-lee, Sangsang |
FedRand enhances privacy in federated learning (FL) for vision-language models (VLMs) by randomizing Low-Rank Adaptation (LoRA) subparameter updates. The main research objective is to mitigate membership inference attacks (MIAs) in FL when training VLMs, specifically addressing the vulnerability caused by exposing full client model parameters to the central server. The key methodology, FedRand, involves clients randomly selecting a subset of LoRA parameters from the server and keeping the remaining LoRA parameters private; after local training, only non-private parameters are sent back for aggregation. Experimental results on MSCOCO show FedRand achieved a CIDEr score of 110.27 while maintaining an AUROC of 53.84% against MIAs, demonstrating comparable task performance to FedAvg (CIDEr: 111.08) and improved MIA robustness. This implies that AI practitioners can improve privacy in federated learning of VLMs, without significant performance degradation, by communicating only a random subset of LoRA parameters between client and server. |
| DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (Read more on arXiv or HuggingFace) |
Luming Liang, tding1, sungnyun, tianyic, jongwooko |
DISTILLM-2 introduces a contrastive learning approach to improve knowledge distillation for compressing large language models (LLMs). Main research question or objective: Can a contrastive approach, considering both teacher and student generated outputs, improve the performance of distilled smaller language models (sLMs)? Key methodology used: DISTILLM-2 uses a contrastive loss function (combining Skew KL and reverse Skew KL) applied asymmetrically to teacher- and student-generated responses, along with optimized data curation and curriculum-based adaptive loss mechanisms. Primary results: DISTILLM-2 achieved state-of-the-art performance on instruction-following, outperforming the second-best method by +2.34%, on average for Qwen2-1.5B model. Principal implication for AI practitioners: AI practitioners can utilize DISTILLM-2 to build high-performing, compact language models suitable for deployment where computational resources are limited, using the proposed contrastive distillation. |
| EasyControl: Adding Efficient and Flexible Control for Diffusion |
|
|
| Transformer (Read more on arXiv or HuggingFace) |
Jiaming Liu, Yirui Yuan, wanghaofan, yiren98, zzyx |
i) EasyControl is presented as a lightweight, efficient, and flexible framework for condition-guided Diffusion Transformers (DiT). ii) The research objective is to enable efficient and flexible control over DiT models, addressing limitations in existing spatial and subject control mechanisms. iii) The method involves a Condition Injection LoRA Module, a Position-Aware Training Paradigm, and a Causal Attention Mechanism with KV Cache. iv) The framework achieves a 58% reduction in inference time compared to ablated versions while maintaining a 15M parameter count in single-condition settings, with the best overall performance in multi-condition configurations. v) EasyControl offers AI practitioners an efficient and adaptable approach to conditional image generation with DiT models, particularly beneficial for applications requiring precise spatial control, subject manipulation, and multi-condition integration. |
| FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation |
|
|
| for Feature Implementation (Read more on arXiv or HuggingFace) |
Wei Li, lisijia0504, yangyu90, dawnmsg, CharonBony |
FEA-Bench is a benchmark for evaluating large language models on repository-level code generation for feature implementation. The main research objective is to assess the ability of LLMs to perform incremental development within code repositories by adding new features. The key methodology involves collecting pull requests from 83 GitHub repositories, filtering them based on rules and intent, and pairing code changes with unit tests. Primary results show that the best-performing LLM (DeepSeek-R1) resolves only 9.92% of task instances in the Oracle and Detailed prompt settings. The principal implication for AI practitioners is that current LLMs face significant challenges in repository-level incremental code development, requiring improvements in handling long contexts and complex code modifications. |
| AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via |
|
|
| Reinforcement Learning and Reasoning (Read more on arXiv or HuggingFace) |
Qian Zhang, xinggangw, wenyuliu, Atan-0221, rb93dett |
AlphaDrive is a VLM-based framework for autonomous driving planning that leverages reinforcement learning and reasoning. The main research objective is to investigate how reinforcement learning (RL) and reasoning can be applied to enhance the performance of vision-language models (VLMs) in autonomous driving planning while reducing training costs. The key methodology involves a two-stage training strategy combining supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO)-based RL, using four custom-designed rewards for planning accuracy, action weighting, diversity, and output format. Primary results show AlphaDrive significantly improves planning accuracy by 25.52% compared to an SFT-trained model, and outperforms SFT by 35.31% with only 20% of the training data. For AI practitioners, AlphaDrive demonstrates the efficacy of integrating GRPO-based RL and a two-stage training approach with planning-specific rewards, offering a method to improve planning performance and training efficiency of VLMs in autonomous driving. |
| DreamRelation: Relation-Centric Video Customization (Read more on arXiv or HuggingFace) |
Shiwei Zhang, Shuaishuai0219, lloong, JacobYuan, weilllllls |
DreamRelation is a novel method for customizing relational video content based on a small set of exemplar videos. The main research question is: How can we decouple relations and subject appearances while accurately modeling relational dynamics to enhance generalizability in customized video generation? The key methodology involves relational decoupling learning, using a relation LoRA triplet and hybrid mask training strategy to separate relations from appearances, and relational dynamics enhancement via a space-time relational contrastive loss. The primary results show that DreamRelation achieves a relation accuracy of 0.4452 ± 0.01, outperforming baselines like direct LoRA finetuning (0.3258 ± 0.05) and MotionInversion (0.3151 ± 0.03). The principal implication for AI practitioners is that by effectively disentangling relational dynamics from subject appearances, DreamRelation provides a more precise and generalizable approach to relational video customization, enabling applications such as creation of diverse human-like animal interactions in novel domains. |
| Agent models: Internalizing Chain-of-Action Generation into Reasoning |
|
|
| models (Read more on arXiv or HuggingFace) |
Jitao Sang, Xinyan Wen, Jiangming Shu, tzteyang, TokerZ |
Large Agent Models (LAMs) internalize Chain-of-Action generation, allowing autonomous decisions on when and how to use external tools. The research objective is to develop a framework, AutoCoA, that enables reasoning models to autonomously generate Chain-of-Action (CoA) for improved task completion. The methodology combines supervised fine-tuning (SFT) with reinforcement learning (RL), including step-level action triggering and trajectory-level CoA optimization, and utilizes an internal world model. Primary results show AutoCoA-trained agent models achieve a 33.9% Exact Match accuracy on multi-hop QA tasks like Bamboogle, significantly outperforming ReAct-based workflows (15.2%). Principal implication for AI practitioners: The AutoCoA framework provides a method to train agent models that show enhanced performance by reducing reliance on externally prompted actions. |
| WritingBench: A Comprehensive Benchmark for Generative Writing (Read more on arXiv or HuggingFace) |
SHaopeng Lai, Chenliang Li, Ming Yan, Jiahao Mei, AQuarterMile |
WritingBench, a new benchmark, evaluates large language models (LLMs) across diverse writing tasks, incorporating a query-dependent evaluation framework. The main objective is to create a comprehensive benchmark for evaluating LLMs on diverse, real-world generative writing tasks and to propose a query-dependent evaluation framework. Key methodology involves a four-stage query construction pipeline leveraging LLMs and human refinement, and a query-dependent evaluation framework using dynamically generated, instance-specific criteria scored by a fine-tuned critic model. Primary results show that the query-dependent evaluation framework achieves 83% human alignment, significantly surpassing static-criteria baselines (65%, 59%). Principal implication for AI practitioners is that WritingBench provides a more nuanced and robust evaluation tool for writing-focused LLMs, and the query-dependent evaluation approach can lead to more accurate and human-aligned assessment of generative writing capabilities, guiding improvements in model development. |
| SurveyForge: On the Outline Heuristics, Memory-Driven Generation, and |
|
|
| Multi-dimensional Evaluation for Automated Survey Writing (Read more on arXiv or HuggingFace) |
Bin Wang, Renqiu Xia, Jiakang Yuan, Shiyang Feng, Xiangchao Yan |
SurveyForge is a framework for automated survey paper generation using heuristic outline generation, memory-driven content creation, and multi-dimensional evaluation. The main research objective is to address the quality gap between AI-generated and human-written surveys, focusing on outline structure, citation accuracy, and content comprehensiveness. The methodology involves a two-stage process: heuristic outline generation based on human-written survey patterns and relevant literature, followed by memory-driven content generation using a scholar navigation agent with temporal-aware reranking. Key results show that SurveyForge outperforms the baseline AutoSurvey in reference coverage (0.40 vs 0.23 using Claude-3-Haiku) and overall content quality (76.34 vs 73.87). AI practitioners can use SurveyForge to create comprehensive, structured survey papers more efficiently and with higher literature coverage than existing methods. |
| Vision-R1: Incentivizing Reasoning Capability in Multimodal Large |
|
|
| Language Models (Read more on arXiv or HuggingFace) |
Zheyu Ye, Shaosheng Cao, Zijie Zhai, Bohan Jia, Wenxuan Huang |
Vision-R1, a multimodal large language model (MLLM), enhances reasoning by combining cold-start initialization with reinforcement learning (RL). The main research objective is to enhance the reasoning capability of MLLMs using RL, addressing limitations of direct RL training. Key methodology used is Modality Bridging with Progressive Thinking Suppression Training (PTST) and Group Relative Policy Optimization (GRPO) using the hard formatting result reward function. Primary results show Vision-R1-7B achieves 73.5% accuracy on the MathVista benchmark, which is only 0.4% lower than the leading model, OpenAI 01. Principal implication for AI practitioners: Using cold-start initialization with a high-quality multimodal Chain-of-Thought (CoT) dataset, combined with the PTST strategy during RL, improves the mathematical reasoning of MLLMs, providing a viable training approach. |
| LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted |
|
|
| Contrastive Learning (Read more on arXiv or HuggingFace) |
Jinsong Su, Jie Zhou, Fandong Meng, lqniu, zhibinlan |
LLaVE is a multimodal embedding model framework that improves performance by focusing on hard negative pairs during contrastive learning. The main research objective is to address the challenge that existing Large Multimodal Model (LMM)-based embedding models struggle to distinguish hard negative pairs effectively when trained with the standard InfoNCE loss. The key methodology involves hardness-weighted contrastive learning, using a reward model to dynamically assign larger weights to harder negative pairs and cross-device negative sample gathering. Primary results show that LLaVE-7B achieves a 6.2 point performance improvement on the MMEB benchmark over the previous state-of-the-art model. The principal implication for AI practitioners is that employing hardness-weighted contrastive learning with LMMs can create more powerful and generalizable multimodal embedding models, with the framework applied and scaling well to diverse datasets. |
| MedAgentsBench: Benchmarking Thinking Models and Agent Frameworks for |
|
|
| Complex Medical Reasoning (Read more on arXiv or HuggingFace) |
Jiapeng Chen, Jiwoong Sohn, Daniel Shao, wshi83, RTT1 |
This paper introduces MEDAGENTSBENCH, a new benchmark for evaluating large language models (LLMs) on complex medical reasoning tasks. The main research objective is to assess the performance of advanced thinking models and agent frameworks in challenging medical scenarios requiring multi-step reasoning. The key methodology involves constructing a dataset of 862 questions from seven established medical datasets, using adversarial filtering to select difficult questions and evaluating various LLMs and agent-based methods using standardized prompts and metrics. A primary result is that DEEPSEEK-R1 achieved the highest scores on five of the datasets and the accuracy values are highlighted in the papers such as MedMCQA: 31.0%, MMLU: 43.8%, MMLU-Pro: 37.0%, MedExQA: 26.0%, and MedXpertQA-U: 26.0%. The principal implication for AI practitioners is that thinking models, like DEEPSEEK-R1, and search-based agent methods, like AFLOW, offer superior performance in complex medical reasoning and better cost-efficiency than the other LLMs and agents, guiding model selection for real-world applications. |
| PE3R: Perception-Efficient 3D Reconstruction (Read more on arXiv or HuggingFace) |
Xinchao Wang, Shizun Wang, Jie Hu |
PE3R is a novel framework for efficient and accurate 3D semantic reconstruction from 2D images without requiring 3D data or camera parameters. The main research objective is to develop a method for 3D semantic reconstruction that generalizes across diverse scenes and objects, achieves high perception accuracy, and operates at high speed. The key methodology involves a feed-forward architecture incorporating pixel embedding disambiguation, semantic field reconstruction, and global view perception modules. The framework achieves a minimum 9-fold speedup in 3D semantic field reconstruction compared to previous methods, along with improved accuracy and precision. For AI practitioners, PE3R provides a faster and more generalizable approach to 3D scene understanding from 2D images, enabling applications in scenarios with limited 3D data availability. |
| Effective and Efficient Masked Image Generation Models (Read more on arXiv or HuggingFace) |
Jun Zhou, Jun Hu, Xiaolu Zhang, Jingyang Ou, yyyou |
eMIGM unifies and improves masked image and diffusion models for efficient, high-quality image generation. The main research objective is to systematically explore the design space of training and sampling in masked image generation models, identifying key factors contributing to performance and efficiency. The key methodology involves unifying masked image modeling and masked diffusion models, then exploring variations in masking distributions, weighting functions, conditional distributions, and sampling strategies like time-interval classifier-free guidance. A primary result is that on ImageNet 512x512, eMIGM-L surpasses EDM2 with an FID of 1.77, using only 60% of the function evaluations. The principal implication is that AI practitioners can leverage eMIGM’s unified framework and optimized training/sampling strategies to achieve state-of-the-art image generation with significantly reduced computational cost. |
| Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive |
|
|
| Reinforcement (Read more on arXiv or HuggingFace) |
Fanbin Lu, Zihao Yue, Zhisheng Zhong, Bohao Peng, Yuqi Liu |
Seg-Zero is a framework for reasoning segmentation that leverages cognitive reinforcement learning to achieve zero-shot generalization. The main research objective is to develop a segmentation model that exhibits strong generalization and explicit reasoning capabilities without relying on supervised fine-tuning with explicit reasoning data. The key methodology involves a decoupled architecture with a reasoning model (MLLM) generating a chain-of-thought and positional prompts, and a segmentation model producing pixel-level masks, trained using reinforcement learning with a novel reward mechanism. Primary results show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on the ReasonSeg benchmark, surpassing the prior LISA-7B by 18%. The principal implication for AI practitioners is that pure reinforcement learning, guided by a well-designed reward mechanism, can induce emergent reasoning in segmentation models, improving generalization across domains without explicit reasoning supervision. |
| BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement |
|
|
| for Transformers in Large-Scale Time Series Modeling (Read more on arXiv or HuggingFace) |
xiaol, Alic-Li |
Rimer replaces the transformer backbone in time series models with RWKV-7, achieving superior performance and efficiency. The research objective was to develop a more efficient and scalable time-series model compared to transformer-based approaches. The methodology involved integrating RWKV-7’s time mix and channel mix components into the transformer-based time series model, Timer. The Rimer model achieved a 1.13x to 43.3x performance improvement and a 4.5x reduction in training time with 1/23 the parameters of the original Timer model. AI practitioners can leverage Rimer for improved performance and reduced computational cost in large-scale time series modeling tasks, benefiting from its compatibility with both AMD and NVIDIA GPUs. |
| This Is Your Doge, If It Please You: Exploring Deception and Robustness |
|
|
| in Mixture of LLMs (Read more on arXiv or HuggingFace) |
Ilija Bogunovic, Sangwoong Yoon, Llwo |
Mixture of LLM Agents (MoA) architectures are vulnerable to significant performance degradation when even a single agent acts deceptively. This paper explores the robustness of Mixture of LLM Agents (MoA) against deceptive agents that provide misleading responses. The authors evaluate MoA’s performance on AlpacaEval 2.0 and QUALITY benchmarks, introducing deceptive agents into the multi-agent system. They find that introducing a single deceptive agent into a 7-agent MoA reduces the length-controlled win rate on AlpacaEval 2.0 from 49.2% to 37.9%. AI practitioners should implement defense mechanisms, such as those proposed in this paper, to mitigate the risks associated with deceptive agents in multi-agent LLM systems. |
| Efficient Distillation of Classifier-Free Guidance using Adapters (Read more on arXiv or HuggingFace) |
msadat97, cristianpjensen |
Adapter Guidance Distillation (AGD) efficiently simulates classifier-free guidance (CFG) in diffusion models using lightweight adapters, doubling sampling speed while maintaining quality. The main research objective is to mitigate the computational cost of CFG in conditional diffusion models, which doubles the number of neural function evaluations per inference step. The key methodology involves training lightweight adapters on CFG-guided trajectories to approximate CFG in a single forward pass, keeping the base diffusion model frozen. AGD achieves a FID score of 5.03 on class-conditional ImageNet generation using DiT, outperforming CFG (FID 5.30) and matching or exceeding the performance across various other tested architectures. For AI practitioners, AGD enables faster sampling from diffusion models with performance similar to or exceeding the use of CFG, and distilling large models such as Stable Diffusion XL on a single consumer GPU. |
| State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for |
|
|
| State Space Models (Read more on arXiv or HuggingFace) |
Hyung Il Koo, Minjae Lee, Yuchen Zeng, Kevin Galim, Wonjun Kang |
State-offset Tuning is a new parameter-efficient fine-tuning method for State Space Models (SSMs) that directly modifies state-related features. The main research objective is to develop a more effective parameter-efficient fine-tuning (PEFT) method for SSMs than existing prompt-based methods. The key methodology is State-offset Tuning, which adds a learnable, constant state-offset to the hidden state at each timestep within the SSM module. Primary results show State-offset Tuning (h) achieved 59.9 execution accuracy on the Spider dataset, outperforming other PEFT methods with comparable parameter budgets. AI practitioners can use State-offset Tuning to efficiently adapt pretrained SSMs to downstream tasks, achieving performance comparable to full fine-tuning with significantly fewer trainable parameters. |
| Should VLMs be Pre-trained with Image Data? (Read more on arXiv or HuggingFace) |
Igor Vasiljevic, Kushal Arora, Samir Yitzhak Gadre, Jean Mercat, Sedrick Keh |
Vision-Language Models (VLMs) can be improved by incorporating image data during pre-training, before the model is fully pre-trained with text. The main research question is when and how image data should be introduced during VLM pre-training to optimize downstream performance on vision-language and text-only tasks. Researchers trained approximately 300 models, systematically varying text-only pre-training amounts, image-text ratios, and fine-tuning stages using a decoder-only transformer architecture with a frozen image encoder. A key finding is that, for a 1B parameter model, introducing visual tokens 80% of the way through pre-training leads to a 2% average improvement on vision-language tasks compared to introducing them after full pre-training. The results suggest that AI practitioners should integrate image data earlier in VLM pre-training, but not immediately, to maintain text performance, instead of following traditional separate training phases. |
| WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image |
|
|
| Generation (Read more on arXiv or HuggingFace) |
Peng Jin, Bin Lin, Mengren Zheng, Munan Ning, Yuwei Niu |
The paper introduces WISE, a new benchmark for evaluating text-to-image (T2I) models’ ability to integrate world knowledge and complex semantics, along with a new metric called WiScore. The main research objective is to assess how well T2I models can generate images that accurately reflect complex semantic understanding and world knowledge, going beyond simple text-image alignment. The key methodology involves a benchmark of 1000 prompts across 25 sub-domains of cultural common sense, spatio-temporal reasoning, and natural science, and evaluates 20 T2I models (10 dedicated, 10 unified) using a novel quantitative metric, WiScore, which assesses knowledge-image alignment. A key result is that the FLUX.1-dev model achieved the best overall WiScore of 0.50, while dedicated T2I models generally outperformed unified multimodal models in leveraging world knowledge. The primary implication is that AI practitioners need to develop enhanced methods for incorporating and applying world knowledge in T2I models, as existing models demonstrate significant limitations in this area. |
| ProBench: Judging Multimodal Foundation Models on Open-ended |
|
|
| Multi-domain Expert Tasks (Read more on arXiv or HuggingFace) |
Liu Liu, Bei Chen, Haoning Wu, dxli1, HelloKKMe |
ProBench is a benchmark for evaluating multimodal foundation models on expert-level, open-ended tasks using MLLM-as-a-Judge. The main research objective is to assess the capabilities of multimodal large language models (MLLMs) on complex, real-world professional tasks requiring expert knowledge and advanced reasoning. The key methodology involves curating a dataset of 4,000 high-quality, open-ended user queries submitted by professionals across 10 fields and 56 sub-fields, and evaluating 24 MLLMs using an MLLM-as-a-Judge approach. The primary results reveal that while the best open-source models rival proprietary ones, ProBench presents significant challenges, and that the MLLM-as-a-Judge evaluation shows 79.9% agreement with human experts. A principal implication for AI practitioners is that current MLLMs still struggle with visual perception, textual understanding, domain knowledge, and advanced reasoning, highlighting the specific areas requiring focused development for improved performance on real-world expert tasks. |
| Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by |
|
|
| Learning Language-Agnostic Speech Representations (Read more on arXiv or HuggingFace) |
Yong Man Ro, Stavros Petridis, Chae Won Kim, Minsu Kim, JeongHun0716 |
This paper explores zero-shot audio-visual speech recognition (AVSR) using language-agnostic speech representations and Large Language Models (LLMs). The main research objective is to enable speech recognition in target languages without any audio-visual speech data in those languages. The key methodology involves an Audio-Visual Speech Romanizer (AV-Romanizer) to predict Roman text and uses pre-trained LLMs and multi-task training to convert it into language-specific graphemes. The Zero-AVSR framework, trained on a new Multilingual Audio-Visual Romanized Corpus (MARC) of 2,916 hours, achieves a 25.2% average WER on the MuAViC dataset. AI practitioners can leverage this framework to expand language support in AVSR systems without requiring target-language speech data. |
| Words or Vision: Do Vision-Language Models Have Blind Faith in Text? (Read more on arXiv or HuggingFace) |
Bryan Hooi, Tri Cao, Ailin Deng, ryanchen42 |
Vision-Language Models (VLMs) exhibit a “blind faith in text” phenomenon, disproportionately trusting textual data over visual data when inconsistencies arise. The main research question is: How do VLMs handle inconsistencies between visual and textual inputs? The key methodology involves introducing textual variations (match, corruption, irrelevance) to four vision-centric tasks and evaluating ten VLMs. A primary result is that Qwen2-VL-7B’s accuracy on VQAv2, DocVQA, and MathVista drops to approximately 50% of its original levels under text corruption. The principal implication for AI practitioners is that balanced training and careful consideration of modality interactions are crucial for enhancing VLM robustness and reliability when handling multi-modal data inconsistencies, especially in safety-critical applications. |
| Detection Avoidance Techniques for Large Language Models (Read more on arXiv or HuggingFace) |
Gabi Dreo Rodosek, Joao A. G. Schneider, Florian Steuber, SinclairSchneider |
This research investigates methods to bypass large language model (LLM) detection systems. The main research objective is to explore the vulnerability of various LLM detection techniques to different evasion strategies. The key methodology involves modifying generative model parameters (temperature, sampling), applying reinforcement learning to fine-tune models, and using paraphrasing models. Primary results show that paraphrasing led to a >90% evasion rate of zero-shot detectors like DetectGPT, reducing detection from 88.6% to 8.7% in one experiment. Principal implication for AI practitioners is that current LLM detection classifiers can be easily bypassed, requiring further research into more robust detection and adaptive detection methods. |
| DiffCLIP: Differential Attention Meets CLIP (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Hasan Abed Al Kader Hammoud |
DiffCLIP extends CLIP with a differential attention mechanism to improve vision-language model performance. The main research question is whether differential attention can be adapted to vision-language models to improve their ability to focus on relevant features across modalities. The key methodology involves integrating differential attention, which subtracts complementary attention distributions, into CLIP’s dual-encoder (image and text) architecture. DiffCLIP outperforms standard CLIP on image-text retrieval, with a 1.2% average improvement on image retrieval using the CC3M dataset. AI practitioners can use DiffCLIP as a lightweight, parameter-efficient addition to CLIP that enhances performance across various vision-language tasks, including few-shot, zero-shot, and robustness benchmarks. |
| Novel Object 6D Pose Estimation with a Single Reference View (Read more on arXiv or HuggingFace) |
Hui Yang, Jin Zheng, Kai Zeng, Wei Sun, JianLiu99 |
SinRef-6D is a framework for estimating the 6D pose of novel objects using only a single RGB-D reference view. The main research objective is to develop a CAD-model-free and dense-reference-view-free method for novel object 6D pose estimation that is scalable and efficient. The key methodology involves iteratively establishing point-wise alignment in the camera coordinate system using state space models (SSMs) for feature encoding, and RGB and points SSMs to capture spatial information. The primary results show that SinRef-6D achieves 90.3% on the LineMod dataset using the ADD-0.1d metric, which is on par with some CAD-based and superior compared to single-reference view based methods. This implies that AI practitioners can achieve accurate 6D pose estimation for novel objects without requiring CAD models or multiple reference views, reducing computational overhead and manual efforts, and enhance the practicality in real-world settings. |
| Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge |
|
|
| Reasoning (Read more on arXiv or HuggingFace) |
Fabio Petroni, Orion Weller, papotti, giulio98 |
This paper introduces a task-aware KV cache compression method for large language models to improve reasoning over large external knowledge corpora. The main research objective is to develop a query-agnostic compression technique that preserves efficiency while maintaining competitive performance compared to query-aware compression and Retrieval-Augmented Generation (RAG). The key methodology involves precomputing a compressed key-value (KV) cache, guided by a task description and optionally few-shot examples, which can be reused for any query within the defined task domain. The approach improves accuracy by up to 7 absolute points over RAG on LongBench v2 with a 30x compression rate, and reduces inference latency. The principal implication is that AI practitioners can leverage task-aware KV cache compression to enable more efficient and comprehensive reasoning over large corpora in LLM applications, outperforming RAG in broad-knowledge tasks. |
| HumanMM: Global Human Motion Recovery from Multi-shot Videos (Read more on arXiv or HuggingFace) |
Jing Lin, Zhuokai Zhao, Ling-Hao Chen, Guanlin Wu, Yuhong Zhang |
HumanMM is a framework for reconstructing 3D human motion in world coordinates from multi-shot videos, addressing challenges like shot transitions and occlusions. The main research objective is to reconstruct long-sequence 3D human motion in world coordinates from in-the-wild videos with multiple shot transitions. The key methodology integrates enhanced camera pose estimation (using a modified LEAP-VO with human masking) with Human Motion Recovery (HMR), incorporating a shot transition detector, an alignment module for pose and orientation continuity across shots, and a custom motion integrator. The proposed method achieved a PA-MPJPE of 36.82 on the ms-AIST subset of the created ms-Motion dataset, outperforming existing methods. For AI practitioners, HumanMM provides a novel, robust method for reconstructing realistic human motion in world coordinates from multi-shot videos, enabling improved motion generation and understanding applications. |
| YOLOE: Real-Time Seeing Anything (Read more on arXiv or HuggingFace) |
Jungong Han, Zijia Lin, Hui Chen, Lihao Liu, Ao Wang |
YOLOE is a unified, efficient object detection and segmentation model that supports diverse open prompt mechanisms, achieving real-time performance. The main research objective is to develop a single model capable of detecting and segmenting arbitrary objects guided by text prompts, visual cues, or without prompts, with high efficiency and accuracy. The key methodology involves Re-parameterizable Region-Text Alignment (RepRTA) for text prompts, Semantic-Activated Visual Prompt Encoder (SAVPE) for visual prompts, and Lazy Region-Prompt Contrast (LRPC) for prompt-free scenarios, all built upon YOLO architectures. On LVIS, YOLOE-v8-S surpasses YOLO-Worldv2-S by 3.5 AP with 3x less training cost and 1.4x inference speedup. The principal implication for AI practitioners is that YOLOE provides a strong baseline and framework for developing real-time, open-prompt-driven vision applications, streamlining development by using a single efficient model for diverse prompt types. |
| RePO: ReLU-based Preference Optimization (Read more on arXiv or HuggingFace) |
Jinyang Gao, Xue Wang, Kexin Huang, Junkang Wu, xiangwang1223 |
RePO introduces a simplified offline preference optimization algorithm for aligning large language models (LLMs) with human preferences. The main research question is whether a simpler offline preference optimization algorithm can be developed that achieves comparable or better performance than existing methods. The key methodology involves using ReLU-based max-margin loss and reference-free reward margins, eliminating the need for the hyperparameter β in SimPO and simplifying the log-sigmoid activation. Primary results show that RePO outperforms DPO and SimPO across multiple base models on AlpacaEval 2, achieving a win rate of 51.1% on Llama3-8B and 66.6% on Gemma2-9B, and it require tuning only one hyperparameter, γ. For AI practitioners, RePO offers a more streamlined and efficient approach to preference optimization, requiring less hyperparameter tuning while achieving competitive or superior performance in LLM alignment. |
| Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal |
|
|
| LLMs (Read more on arXiv or HuggingFace) |
Stavros Petridis, Minsu Kim, Umberto Cappellazzo |
Llama-MTSK, a Matryoshka-based Multimodal LLM, enables adaptive audio-visual speech recognition with flexible token allocation. The research objective is to create an audio-visual speech recognition (AVSR) system that dynamically adjusts computational efficiency and performance at inference time using a single model. The methodology involves encoding audio-visual representations at multiple granularities using Matryoshka Representation Learning and fine-tuning a pre-trained LLM with three LoRA-based Matryoshka strategies. On the LRS3 dataset, Llama-MTSK achieved a Word Error Rate (WER) of 2.3% using the SS configuration with an audio compression rate of 4 and video compression of 2, outperforming independently trained models. AI practitioners can use Llama-MTSK to deploy AVSR models that efficiently adapt to various computational constraints and accuracy requirements without retraining. |
| Escaping Plato’s Cave: Towards the Alignment of 3D and Text Latent |
|
|
| Spaces (Read more on arXiv or HuggingFace) |
Qixing Huang, Diego Gomez, Luca Moschella, Souhail Hadgi, teelinsan |
This paper investigates the alignment between latent spaces of 3D and text encoders, finding that subspace projection improves cross-modal performance. The main research objective is to explore the possibility of a posteriori alignment of representations obtained from uni-modal 3D encoders compared to text-based feature spaces. The key methodology involves combining Canonical Correlation Analysis (CCA) for subspace selection with affine transformation and local CKA for alignment of 3D and text features. A primary result is that the affine + subspace projection method achieves a top-5 retrieval accuracy of 42.2% between uni-modal PointBert and RoBERTa, significantly higher than without subspace projection. Principal implication for AI practitioners that aligning lower-dimensional subspaces of 3D and text representations enables cross-modal applications, like matching and retrieval tasks, without expensive joint training, and offers a new tool. |
| NeuGrasp: Generalizable Neural Surface Reconstruction with Background |
|
|
| Priors for Material-Agnostic Object Grasp Detection (Read more on arXiv or HuggingFace) |
Xudong Zheng, Wenzhe He, Chao Li, Yinghao Cai, KianYale |
NeuGrasp is a generalizable neural surface reconstruction method that uses background priors for 6-DoF robotic grasp detection of objects with various material properties. The main research objective is to develop a method for robust, material-agnostic grasp detection in scenes with transparent and specular objects from sparse views within a narrow field of view. The key methodology involves integrating transformers and global prior volumes within a neural implicit surface framework, using residual feature enhancement and an occupancy-prior volume to distinguish foreground objects. Primary results show that NeuGrasp achieved a success rate of 86.3% and declutter rate of 81.0% in simulation experiments on packed scenes with transparent and specular objects, outperforming baselines. AI practitioners can apply NeuGrasp to achieve accurate grasp detection using a small amount of RGB image input. |
Papers for 2025-03-10
| Title |
Authors |
Summary |
| Unified Reward Model for Multimodal Understanding and Generation (Read more on arXiv or HuggingFace) |
Cheng Jin, Hao Li, Jiaqiwang, yuhangzang, CodeGoat24 |
This paper proposes UNIFIEDREWARD, a unified reward model for assessing both multimodal understanding and generation, enabling pairwise ranking and pointwise scoring for vision model preference alignment. The main research objective is to develop a single reward model adaptable across diverse visual tasks (image/video generation and understanding) and to demonstrate its effectiveness in aligning vision models with human preferences. The key methodology involves training a Vision Language Model (VLM) on a newly constructed, large-scale human preference dataset, then using the trained model to curate preference data for Direct Preference Optimization (DPO) of VLMs and diffusion models. Primary results show that UNIFIEDREWARD achieves 66.5% macro accuracy on VLRewardBench for image understanding assessment, outperforming existing methods. The principal implication for AI practitioners is that they can leverage this unified reward model and associated training pipeline to improve the alignment of vision models with human preferences across a range of generation and understanding tasks, leading to better output quality and overall better evaluation. |
| EuroBERT: Scaling Multilingual Encoders for European Languages (Read more on arXiv or HuggingFace) |
caiocorro, ayoubhammal, DuarteMRAlves, hgissbkh, Nicolas-BZRD |
EuroBERT, a family of multilingual encoder models, outperforms existing alternatives on various tasks, spanning multiple languages, mathematics, and coding. The main research objective is to revisit the development of multilingual encoders by leveraging recent advances from decoder models and examining design choices in data composition and training. Methodology includes building a 5T-token multilingual dataset, using a masked language modeling objective, and employing a two-phase training pipeline (pre-training and annealing). EuroBERT-2.1B achieves the highest performance among all systems, ranking first on 7 of 12 multilingual benchmarks, outperforming XLM-ROBERTa-XL. This implies that AI practitioners can use EuroBERT models for improved performance in NLP tasks, especially retrieval, classification and evaluation tasks across European and other widely spoken languages, even with models smaller than pre-existing state-of-the-art. |
| Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, jinheon, saytes |
Sketch-of-Thought (SoT) is a prompting framework that improves large language model (LLM) reasoning efficiency by using concise, structured intermediate steps inspired by human cognitive processes. The main research objective is to reduce the computational cost of LLM reasoning while maintaining or improving accuracy compared to verbose methods like Chain-of-Thought (CoT). The key methodology involves three cognitive-inspired paradigms (Conceptual Chaining, Chunked Symbolism, and Expert Lexicons) dynamically selected by a lightweight router model based on query characteristics. Primary results show that SoT reduces token usage by up to 76% across 15 reasoning datasets with negligible accuracy impact, and in some cases, even improved accuracy. Principal implication for AI practitioners: SoT offers a practical method to reduce computational costs and latency in LLM-based reasoning applications without significant performance degradation, enabling deployment in resource-constrained environments. |
| Forgetting Transformer: Softmax Attention with a Forget Gate (Read more on arXiv or HuggingFace) |
Aaron Courville, littleowen, nikishin, zhixuan-lin |
Forgetting Transformer (FoX) introduces a forget gate into the softmax attention mechanism of Transformers to improve performance, particularly in length extrapolation and short-context tasks. The main research objective is to determine if incorporating a data-dependent forget gate into Transformers can improve their performance on both long and short-context tasks. The key methodology involves modifying the softmax attention mechanism by down-weighting unnormalized attention scores based on a learned, data-dependent forget gate, implemented efficiently using a modification of the FlashAttention algorithm. Primary results show that FoX outperforms the standard Transformer in long-context language modeling, achieving a per-token loss of approximately 1.53 compared to Transformer’s ~1.58 at the 32,000 token index (Figure 2, left) in a configuration with a 760M parameter. Principal implication for AI practitioners is that the FoX architecture could improve performance in some sequential tasks and serves as a strong baseline, especially in tasks needing to balance long- and short-context information, with the Pro architecture being the most promising. |
| VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control (Read more on arXiv or HuggingFace) |
Zhaoyang Zhang, yshan2u, Ljzycmd, juxuan27, BianYx |
VideoPainter introduces a dual-branch framework for text-guided video inpainting and editing that maintains ID consistency in long videos. The research objective is to develop a method for video inpainting that addresses challenges such as generating fully masked objects, balancing background preservation with foreground generation, and maintaining identity consistency over long videos. The key methodology involves a lightweight context encoder within a dual-branch Diffusion Transformer architecture, and a novel inpainting region ID resampling technique. Primary results include achieving a FVID score of 0.09 on the VPBench dataset for standard video inpainting surpassing competing methods. The principal implication is that AI practitioners can leverage this framework for more effective and controllable video inpainting and editing, with robust performance in generating long videos and maintaining object identity due to its sampling technique. |
| R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning (Read more on arXiv or HuggingFace) |
jrwen, TimothyCzp, EliverQ, Boru, XXsongLALA |
R1-Searcher is a two-stage outcome-based reinforcement learning (RL) framework to enhance search capabilities in large language models (LLMs). The main research objective is to enable LLMs to autonomously invoke external search systems for accessing additional knowledge during reasoning. The key methodology is a two-stage RL approach: first incentivizing retrieval invocation, then rewarding accurate answer generation using retrieved information, with RAG-based rollout and retrieval mask-based loss calculation. The primary results are, using Qwen-2.5-7B-Base, R1-Searcher outperforms ReARTeR by 48.22% on HotpotQA and by 21.72% on 2Wiki. The principal implication is that AI practitioners can use this RL method to train LLMs to effectively integrate external search, improving reasoning and generalization, even in out-of-domain and online search scenarios. |
| R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcing Learning (Read more on arXiv or HuggingFace) |
Xihan Wei, Liefeng, StarJiaxing |
The paper introduces R1-Omni, an omni-multimodal model for emotion recognition using Reinforcement Learning with Verifiable Reward (RLVR). The main research objective is to investigate the potential of RLVR in enhancing emotion recognition performance in a video-based, omni-multimodal setting (incorporating both visual and audio data). Key methodology involves applying RLVR with Group Relative Policy Optimization (GRPO) to a HumanOmni model, using a verifiable reward function that combines accuracy and format rewards, after a cold start using the EMER dataset. Primary results show that R1-Omni achieves a UAR of 65.83% and a WAR of 56.27% on the DFEW dataset, outperforming Supervised Fine-Tuning (SFT) models. For AI practitioners, the principal implication is that RLVR can significantly improve the reasoning capability, emotion recognition accuracy, and generalization ability of multimodal large language models in tasks such as emotion recognition, without explicit reasoning-process supervision. |
| TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Models (Read more on arXiv or HuggingFace) |
Mark YU, yshan2u, Doubiiu, wbhu-tc |
TrajectoryCrafter redirects camera trajectories in monocular videos using diffusion models. The research objective is to generate high-fidelity videos from monocular inputs with user-defined camera trajectories, ensuring 4D consistency. The methodology uses a dual-stream conditional video diffusion model that integrates point cloud renders and source videos, trained on a hybrid dataset of monocular and multi-view data using a double-reprojection strategy. The method achieved a PSNR of 14.24 on the iPhone multi-view dataset, outperforming existing methods. AI practitioners can use this framework to generate videos with controlled camera movements from single-camera footage, enhancing video content creation and editing capabilities. |
| BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities (Read more on arXiv or HuggingFace) |
Ruohan Zhang, jiajunwu, cgokmen, yjze, yunfanj |
BEHAVIOR ROBOT SUITE (BRS) is a framework for learning whole-body manipulation for household tasks. The main research objective is to identify and address the key capabilities required for robots to perform everyday household activities successfully. The key methodology used is a combination of a cost-effective whole-body teleoperation interface (JoyLo) for data collection, and a novel imitation learning algorithm (Whole-Body VisuoMotor Attention policy, WB-VIMA) for modeling coordinated whole-body actions. The trained WB-VIMA policies achieved an average success rate of 58% and a peak success rate of 93% across five challenging household tasks. For AI practitioners, BRS provides an integrated framework for whole-body manipulation, offering open-source hardware and software to facilitate data collection and policy learning for real-world robotic applications, streamlining the development of robots capable of diverse household tasks. |
| RuCCoD: Towards Automated ICD Coding in Russian (Read more on arXiv or HuggingFace) |
Vladimir Makharev, Airat Valiev, Ivan Sviridov, Andrey Sakhovskiy, Aleksandr Nesterov |
This paper introduces RuCCoD, a new Russian-language dataset for automated ICD coding, and benchmarks several state-of-the-art models for this task. The main research objective is to investigate the feasibility of automating clinical coding in Russian, a language with limited biomedical resources. The key methodology involves training and evaluating BERT-based, LLaMA-based (with LoRA and RAG), models on the RuCCoD dataset, and applying the best model to a larger EHR dataset for diagnosis prediction. Primary results show that pre-training a Longformer model on automatically assigned ICD codes (using the new proposed dataset) yields a 28% higher macro-averaged F1-score for diagnosis prediction compared to using physician-assigned codes. For AI practitioners, using an automated pipeline to generate ICD codes for model training can significantly improve diagnosis prediction accuracy in resource-limited languages like Russian. |
| TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation (Read more on arXiv or HuggingFace) |
lwher1996, yuhanwuuu, xiaoqijiang, zhaoguangxiang, lincharliesun |
TinyR1-32B-Preview is a new language model that improves accuracy on reasoning tasks using a branch-merge distillation approach. The main objective is to create a smaller, high-performing Large Language Model (LLM) with reduced computational cost and time, compared to traditional distillation methods. The key methodology involves a two-phase distillation: (1) “Branch Phase,” where a large teacher model’s knowledge is selectively distilled into specialized student models via domain-specific supervised fine-tuning, and (2) “Merge Phase,” where specialized models are combined using Arcee Fusion. The primary result is that TinyR1-32B-Preview outperforms DeepSeek-R1-Distill-Qwen-32B by 5.5 points in Mathematics on the AIME 2024 benchmark. The principal implication is to provide AI practioners, a scalable solution for creating smaller, more efficient LLMs, and a means of achieving high accuracy on specific benchmarks, while potentially reducing the computational and time resources needed. |
| ProReflow: Progressive Reflow with Decomposed Velocity (Read more on arXiv or HuggingFace) |
Yu Li, Xuefei Ning, Haohang Xu, Lei Ke, Ringo1110 |
ProReflow improves flow matching in diffusion models for faster image and video generation by progressively refining the diffusion process and emphasizing directional alignment in velocity prediction. The main research objective is to address the high computational cost of diffusion models by optimizing the flow matching training process. The key methodology involves progressive reflow (refining diffusion models in stages with decreasing timesteps) and aligned v-prediction (prioritizing velocity direction matching over magnitude). Primary results show that on the MSCOCO2014 validation set, ProReflow-II achieves an FID of 10.70 with only 4 sampling steps. For AI practitioners, ProReflow offers a more efficient training framework for flow-based diffusion models, achieving state-of-the-art performance with reduced sampling steps, directly benefiting applications requiring fast image/video synthesis. |
| Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts (Read more on arXiv or HuggingFace) |
Yu Cheng, Tong Zhu, Xiaoye08, landisen, weigao266 |
Linear-MoE integrates linear sequence modeling (LSM) with Mixture-of-Experts (MoE) for efficient large-scale model training. The paper explores the objective of combining the benefits of LSM and MoE to improve performance and training efficiency in large models. The methodology involves developing a system with modeling and training subsystems, including sequence parallelism tailored for LSM and hybrid models with standard Transformer-MoE layers. Evaluations on A0.3B-2B and A1B-7B models show Linear-MoE achieves efficiency gains while maintaining competitive performance across various benchmarks. Linear-MoE offers AI practitioners a potential next-generation foundational model architecture by enhancing efficiency and scalability in large language models. |
| Learning from Failures in Multi-Attempt Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jie Fu, Stephen Chung, wydu |
i) The paper introduces a multi-attempt reinforcement learning task to enhance reasoning in large language models (LLMs) by providing feedback on incorrect responses. ii) The research aims to improve LLMs’ reasoning capabilities by training them to refine responses based on feedback in a multi-attempt setting. iii) The methodology involves training an LLM with standard Proximal Policy Optimization (PPO) on a math problem dataset, modifying the task to allow multiple attempts with feedback after each incorrect answer. iv) The primary result shows that an LLM trained on the multi-attempt task improves accuracy on math benchmarks from 45.6% to 52.5% with two attempts, compared to a marginal improvement from 42.3% to 43.2% for the same LLM trained on a standard single-turn task. v) The principal implication for AI practitioners is that training LLMs with multi-attempt tasks can lead to better self-refinement capabilities and improved performance in reasoning tasks, offering a more effective approach compared to single-turn training. |
| An Empirical Study on Eliciting and Improving R1-like Reasoning Models (Read more on arXiv or HuggingFace) |
daixuancheng, Boru, ToheartZhang, EliverQ, TimothyCzp |
i) This paper presents an empirical study on improving reasoning capabilities in Large Language Models (LLMs) through Reinforcement Learning (RL) and tool manipulation. ii) The main objective is to investigate methods for eliciting and enhancing R1-like reasoning in LLMs, focusing on scaling RL training and using tool manipulation techniques. iii) The study employs RL training with various hyperparameter settings and reward designs, alongside supervised fine-tuning to enable tool manipulation. iv) The primary result is that RL training improves QWEN2.5-32B base models, achieving 39.33% accuracy on AIME 2024 for a fine-tuned model; furthermore, tool manipulation achieved 86.67% accuracy with greedy search on AIME 2024. v) The findings suggest that scaling RL training and incorporating tool manipulation are effective strategies for AI practitioners to enhance reasoning performance in LLMs, offering a path to improve model capabilities in complex tasks. |
| SAGE: A Framework of Precise Retrieval for RAG (Read more on arXiv or HuggingFace) |
Jinyang Su, Guoliang Li, jt-zhang |
i) The paper introduces SAGE, a RAG framework enhancing retrieval precision through semantic segmentation, gradient-based chunk selection, and LLM self-feedback. ii) The primary objective is to improve the accuracy and cost-efficiency of RAG systems by addressing limitations in corpus segmentation and context retrieval. iii) The methodology involves training a semantic segmentation model, developing a gradient-based chunk selection algorithm, and implementing an LLM-based self-feedback mechanism for context adjustment. iv) Experiments show SAGE outperforms baselines by 61.25% in QA quality on average and achieves a 49.41% enhancement in cost efficiency. v) SAGE offers AI practitioners a more effective and cost-efficient RAG system by improving the precision of retrieved context, which reduces LLM token consumption and increases QA accuracy. |
| LONGCODEU: Benchmarking Long-Context Language Models on Long Code Understanding (Read more on arXiv or HuggingFace) |
Ge Li, Kechi Zhang, Lei Li, Xuyuan Guo, Jia Li |
LONGCODEU is introduced as a new benchmark to evaluate long code understanding in LLMs. The primary objective is to assess LLMs’ abilities in code unit perception, intra-code unit understanding, inter-code unit relation understanding, and long code documentation understanding. The methodology involves curating a dataset from real-world code repositories with varying code lengths and evaluating LLMs on eight different tasks spanning the four understanding aspects. Experimental results showed that LLMs’ performance significantly degrades when processing code longer than 32K tokens, and the inter-code unit relation understanding is the most challenging aspect; for example, DeepSeek-V2.5 achieves 11.75% average improvements on the benchmarks tasks. This benchmark provides AI practitioners with a means to identify limitations and guide development of LLMs for software engineering tasks requiring long code context. |
| LoRACode: LoRA Adapters for Code Embeddings (Read more on arXiv or HuggingFace) |
bindsch, amanchadha, shollercoaster |
LoRACode introduces a parameter-efficient fine-tuning method for code embeddings using Low-Rank Adaptation (LoRA). The research investigates whether LoRA adapters can improve code retrieval accuracy while minimizing computational costs. The methodology involves fine-tuning CodeBERT, GraphCodeBERT, and UniXcoder with LoRA on code corpora, creating task-specific and language-specific adapters. Experiments showed an increase of up to 9.1% in Mean Reciprocal Rank (MRR) for Code2Code search and up to 86.69% for Text2Code search tasks. LoRA’s efficient fine-tuning, utilizing only 1.83%-1.85% of base model parameters, allows AI practitioners to rapidly adapt code embedding models for improved semantic code search with reduced computational resources. |
| R1-Zero’s “Aha Moment” in Visual Reasoning on a 2B Non-SFT Model (Read more on arXiv or HuggingFace) |
Minhao Cheng, Ruochen Wang, zhoutianyi, AIcell, Dolphin42 |
This paper demonstrates emergent visual reasoning capabilities in a 2B parameter language model through reinforcement learning, without supervised fine-tuning. The main research objective was to replicate the “aha moment” and increased response length observed in DeepSeek-R1 in a multimodal setting, specifically for visual reasoning. The key methodology involved applying the GRPO algorithm, a variant of PPO, directly to a non-SFT Qwen2-VL-2B base model, using a rule-based reward function based on response format and correctness on the SAT dataset. The primary result was that the model achieved 59.47% accuracy on CVBench, outperforming the base model by approximately 30% and the SFT model by about 2%. Principal implication for AI practioners is that reinforcement learning can induce sophisticated reasoning in multimodal models without requiring extensive supervised data, offering a more scalable approach to training. |
| AnyAnomaly: Zero-Shot Customizable Video Anomaly Detection with LVLM (Read more on arXiv or HuggingFace) |
Inpyo Hong, Sein Kwon, Kijung Lee, jyy1551, SkiddieAhn |
AnyAnomaly is a zero-shot customizable video anomaly detection (C-VAD) method that leverages Large Vision-Language Models (LVLMs). The main research objective is to develop a VAD system that can detect user-defined anomalies in diverse environments without requiring retraining or environment-specific data. The key methodology involves a segment-level approach using a Key frames Selection Module, a context-aware Visual Question Answering (VQA) with position and temporal contexts, and a prompt designed specifically for anomaly scoring. The proposed model, AnyAnomaly, achieved a 9.88% performance improvement over the baseline on the Customizable-ShT (C-ShT) dataset and state-of-the-art on the UBnormal dataset. AI practitioners can deploy VAD in new scenarios without additional training or data collection by providing user-defined text descriptions of anomalies. |
Papers for 2025-03-07
| Title |
Authors |
Summary |
| LLM as a Broken Telephone: Iterative Generation Distorts Information (Read more on arXiv or HuggingFace) |
Michalis Vazirgiannis, guokan-shang, mgeng, amr-mohamed |
Iterative processing of text by large language models (LLMs) degrades information, similar to the “broken telephone” game. The main research question is whether LLMs distort information through iterative generation, particularly in translation tasks. The key methodology involved simulating iterative translation chains, where an English document was repeatedly translated into and out of other languages using LLMs. Primary results show a gradual decline in factuality and relevance over iterations, with an average FActScore gradient of -0.038 ± 0.02 in the most complex translation chain setting. Principal implication for AI practitioners is that iterative generation with LLMs can lead to information distortion, making control of temperature, prompt design, and understanding the role of intermediary languages necessary when building applications relying on the iterative processing of LLM-generated content. |
| EgoLife: Towards Egocentric Life Assistant (Read more on arXiv or HuggingFace) |
Zzitang, Alarak, fesvhtr, THUdyh, Jingkang |
i) EgoLife introduces a comprehensive egocentric dataset and benchmark for developing AI life assistants. ii) The study aims to create life-oriented question-answering tasks designed to provide meaningful assistance in daily life through multimodal egocentric data understanding. iii) Data was collected from six participants living together for a week, using AI glasses to record multimodal egocentric video, supplemented by synchronized third-person video references and annotated for comprehensive data analysis. iv) The EgoLife Dataset comprises 300 hours of egocentric data and introduces EgoLifeQA, a benchmark for long-context question answering, alongside EgoButler, an integrated system, and their experiments verified the mechanisms, critical factors, and bottlenecks, guiding future improvements with EgoGPT achieving state-of-the-art performance on egocentric video understanding. v) The EgoLife dataset, tasks, and models offer AI practitioners a resource for advancing long-term egocentric life assistance through improved multimodal integration, identity recognition, and ultra-long-context question answering. |
| HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization (Read more on arXiv or HuggingFace) |
Ya Wang, Breeze0417, LLIXQ, Taoer, BryceZhuo |
HybridNorm, a novel normalization strategy for Transformers, combines QKV normalization in attention and Post-Norm in the feed-forward network to improve training stability and performance. The research objective is to address the trade-offs between training stability and final model performance inherent in existing normalization techniques like Pre-Norm and Post-Norm in Transformer models. The key methodology involves proposing HybridNorm and evaluating it through extensive experiments on large-scale dense and Mixture-of-Experts (MoE) language models. The primary results show that HybridNorm consistently outperforms Pre-Norm and Post-Norm across various benchmarks; for example, HybridNorm* achieved an average accuracy of 64.15% compared to Pre-Norm’s 62.99% on downstream tasks for 1.2B dense models. Principal implication: AI practitioners can use HybridNorm to achieve more stable training dynamics and superior performance when training large Transformer models, particularly in language modeling applications. |
| PokéChamp: an Expert-level Minimax Language Agent (Read more on arXiv or HuggingFace) |
Andy Luu Nguyen, chijin, milkkarten |
PokéChamp is a minimax language agent that achieves expert-level performance in Pokémon battles by integrating large language models (LLMs) into the tree search algorithm. The main research objective is to develop an agent capable of strategic action proposal, accurate opponent modeling, and effective evaluation of game trajectories in Pokémon battles, without requiring LLM fine-tuning. The key methodology involves replacing three components of minimax tree search—player action sampling, opponent modeling, and value function estimation—with LLM-based generations, leveraging a world model that approximates game transitions. PokéChamp, powered by GPT-4o, achieves a 76% win rate against the best existing LLM-based bot and 84% against the strongest rule-based bot in the Generation 9 OverUsed Meta. AI practitioners can leverage this framework’s integration of LLMs with game-theoretic planning algorithms to develop agents for complex, partially observable environments without task-specific training. |
| FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion (Read more on arXiv or HuggingFace) |
passerqxj, OnewayLab, GGLS, Wanfq, AALF |
FuseChat-3.0 integrates the strengths of heterogeneous large language models (LLMs) into more compact target LLMs using a two-stage training process. The main objective is to develop a method for effectively fusing knowledge from multiple, diverse source LLMs into smaller target LLMs. The methodology involves a specialized data construction protocol followed by supervised fine-tuning (SFT) and Direct Preference Optimization (DPO), using preference pairs generated from the same source model. When using Llama-3.1-8B-Instruct as the target model, the fusion approach achieves an average improvement of 6.8 points across 14 benchmarks. AI practitioners can use this implicit model fusion technique to enhance the performance of smaller LLMs by leveraging the capabilities of larger, heterogeneous models, without requiring architectural changes. |
| Token-Efficient Long Video Understanding for Multimodal LLMs (Read more on arXiv or HuggingFace) |
zhiqilinv, MuyangLI, zhijianliu, xiuyul, jdps |
i) STORM is a novel architecture for efficient long video understanding in multimodal LLMs. ii) The research aims to improve video understanding in LLMs, particularly with extended temporal contexts. iii) A dedicated temporal encoder using the Mamba State Space Model is introduced between the image encoder and the LLM, enabling token reduction via sampling and spatial/temporal pooling. iv) STORM achieves state-of-the-art results with over 5% improvement on MLVU and LongVideoBench, while reducing computation costs by up to 8x and decoding latency by 2.4-2.9x for fixed input frames. v) Practitioners can leverage STORM to reduce LLM computational demands and latency without sacrificing performance. |
| The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation (Read more on arXiv or HuggingFace) |
Xu Tan, Kai Shen, Aoxiong Yin, JunchengLi, ustcscallion |
LanDiff is a hybrid text-to-video generation framework that combines language models and diffusion models for coarse-to-fine video synthesis. The main research objective is to develop a framework that leverages the strengths of both autoregressive language models (semantic understanding, causal modeling) and diffusion models (high visual quality, progressive refinement) while mitigating their limitations. The key methodology involves a two-stage process: (1) a semantic tokenizer compresses 3D visual features into 1D discrete representations, and an LLM generates semantic tokens; (2) a streaming diffusion model refines these tokens into high-fidelity video features, decoded by a VAE. LanDiff, with a 5B parameter model, achieves a score of 85.43 on the VBench T2V benchmark, surpassing state-of-the-art open-source and commercial models. AI practitioners can use LanDiff architecture as a blueprint of production-level video generation, particularly in scenarios requiring high semantic accuracy, visual quality, and long video generation capabilities. |
| IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (Read more on arXiv or HuggingFace) |
Mingsheng Shang, yilunzhao, guo9, songtingyu |
IFIR is a new benchmark for evaluating instruction-following information retrieval in specialized domains, revealing challenges for current models. The main research objective is to evaluate how well current information retrieval (IR) systems can follow complex, domain-specific instructions in expert fields. Key methodology involves creating a new benchmark (IFIR) with 2,426 examples across finance, law, healthcare, and scientific literature, incorporating three levels of instruction complexity and a novel LLM-based evaluation metric (INSTFOL). Primary results show that while BM25 performs relatively well due to glossary terms, instruction-tuned retrievers like INSTRUCTOR don’t significantly outperform their base models, and most models’ performance declines with increasing instruction complexity; LLM-based retrievers achieve the highest INSTFOL score, as demonstrated by Promptriever-7B. Principal implication is that current retrieval models, even those fine-tuned for instruction following, struggle with long, complex instructions in specialized domains, indicating a need for improved training methodologies and architectures or hybrid systems, leveraging large language model’s superior instruction-following ability. |
| Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities (Read more on arXiv or HuggingFace) |
manocha, rafaelvalle, firecomputer, ZhifengKong, SreyanG-NVIDIA |
i) Audio Flamingo 2 (AF2) is a novel audio-language model (ALM) enhancing audio understanding and reasoning. ii) The research aims to develop an ALM with advanced capabilities in understanding and reasoning over both short and long audio segments, including non-speech sounds and music. iii) AF2 leverages a custom CLAP model, synthetic Audio QA data, and a multi-stage curriculum learning strategy. iv) AF2 achieves state-of-the-art performance on over 20 benchmarks, surpassing larger models, with a 3B parameter language model achieving up to 18.9% improvement on the LongAudioBench compared to Gemini F v2. v) AF2’s ability to understand long audio segments offers AI practitioners new capabilities for real-world applications requiring contextual auditory cue processing, such as anomaly detection and assistive technologies. |
| Identifying Sensitive Weights via Post-quantization Integral (Read more on arXiv or HuggingFace) |
Weiyu Huang, surfingtomchen, jt-zhang, zcliang22, yuezhouhu |
The paper introduces a novel sensitivity metric and quantization framework for compressing large language models (LLMs). The primary research objective is to develop a more accurate sensitivity metric for weight quantization that addresses limitations of existing gradient and Hessian-based methods. The key methodology is Post-quantization Integral (PQI), which estimates the impact of quantized weights on the loss function, along with a Dense-and-Sparse detach framework called ReQuant. Applying ReQuant to Llama 3.2 1B with QTIP quantization reduces perplexity by 2.66, showcasing the improvement. For AI practitioners, this method provides an effective way to improve post-training quantization of LLMs, achieving better compression with minimal accuracy loss. |
| L$^2$M: Mutual Information Scaling Law for Long-Context Language Modeling (Read more on arXiv or HuggingFace) |
Marin Soljačić, Di Luo, Zhuotao Jin, oriolmayne, zhuoc3 |
This paper establishes a theoretical framework for understanding and improving long-context language modeling based on a bipartite mutual information scaling law. The main research question is how a language model’s capacity to handle long-range dependencies scales with its internal state size and sequence length. The key methodology involves proving a “Long-context Language Modeling (L²M)” condition, theoretically relating model state size to bipartite mutual information, and empirically validating this scaling law using transformer and state space models on text datasets. The primary result is that bipartite mutual information in natural language scales as I ~ L^β (where β is between 0 and 1) and that a model’s state size must grow at least as fast as I ~ L^β for effective long-context modeling. The principal implication for AI practitioners is that designing models for long-context tasks requires careful consideration of the history state’s scaling, with transformers naturally satisfying this condition and other architectures (like SSMs) needing model size increases to maintain performance at longer sequence lengths. |
| Dedicated Feedback and Edit Models Empower Inference-Time Scaling for Open-Ended General-Domain Tasks (Read more on arXiv or HuggingFace) |
Ellie Evans, Daniel Egert, Jiaqi Zeng, Zhilin Wang, odelalleau |
Dedicated Feedback and Edit Models enable inference-time scaling for open-ended tasks, achieving state-of-the-art performance by leveraging human feedback. i) Main research question or objective: How to perform inference-time scaling for open-ended general-domain tasks, inspired by human feedback, using dedicated Feedback and Edit Models. ii) Key methodology used: Trained dedicated Feedback and Edit Models on a curated dataset, leveraging human-provided feedback and edits. iii) Primary results: The optimally scaled system, based on 70B models from the Llama 3 family, achieved a state-of-the-art performance on Arena Hard at 92.7, surpassing OpenAI ol-preview-2024-09-12 (90.4) and DeepSeek R1 (92.3). iv) Principal implication for AI practitioners: This approach demonstrates a viable method for improving model performance on complex, open-ended tasks by using human feedback to train models to improve responses at inference. |
| Union of Experts: Adapting Hierarchical Routing to Equivalently Decomposed Transformer (Read more on arXiv or HuggingFace) |
Linhui Li, Jing Lian, yjyangwork |
Union-of-Experts (UoE) decomposes transformers into equivalent experts and implements selective routing on input data and experts to improve model performance while maintaining efficiency. The main research objective is to address limitations of existing Mixture-of-Experts (MoE) methods, specifically lack of high-quality expert interactions and inefficient extension to attention blocks. Key methodology involves equivalent expert decomposition on MLP and attention blocks via matrix partition, two routing paradigms (patch-wise data and expert selection), and parallel implementation of routing/computation. Primary results show UoE achieves an average perplexity reduction of 2.38 on language modeling tasks compared to the best-performed MoE method, using only 76% of the FLOPs. Principal implication for AI practitioners is that UoE offers a more efficient and performant approach to building transformer-based models, directly applicable to large-scale language and vision tasks. |
| Lost in Literalism: How Supervised Training Shapes Translationese in LLMs (Read more on arXiv or HuggingFace) |
Leyang Cui, Huajian Zhang, Zhilin Wang, Ronghao Zhang, yaful |
This paper investigates and mitigates translationese (unnatural translations) in Large Language Models (LLMs) caused by biases introduced during supervised fine-tuning (SFT). The main research objective is to evaluate the prevalence of translationese in LLM-generated translations and investigate its origins during supervised training. The key methodology involves human annotation to identify translationese spans, analysis of training data, and mitigation strategies such as refining training references and filtering unnatural instances using perplexity. The primary results show that even advanced models like GPT-4 exhibit substantial translationese, with over 40% of their translations containing substantial translationese patterns, and that refining training data with LLMs reduces perplexity by 7.8 in the English-Chinese dataset. Principal implication for AI practitioners is that addressing translationese bias in SFT data, by polishing golden references or filtering, can improve the naturalness of LLM translation outputs. |
| Combining Flow Matching and Transformers for Efficient Solution of Bayesian Inverse Problems (Read more on arXiv or HuggingFace) |
Ekaterina Muravleva, oseledets, dsherki |
The paper introduces a method combining Conditional Flow Matching (CFM) and transformers to efficiently solve Bayesian inverse problems. The main objective is to recover the distribution of model parameters conditioned on observed experimental data, given a series of observations and a forward model. The key methodology involves training a transformer-based CFM architecture to learn the conditional probability distribution from samples, handling a variable number of observations. Results showed that for a SEIR disease model, the average error was 2.05% ± 1.04% using a 4-point MLP model, significantly outperforming MCMC in computational efficiency. AI practitioners can leverage this approach for faster and more scalable sampling from posterior distributions in Bayesian inverse problems, particularly with datasets having variable-length observations. |
| Understanding and Predicting Derailment in Toxic Conversations on GitHub (Read more on arXiv or HuggingFace) |
Rebekah Copeland, Robert Zita, kdamevski, rahat-rizvi, imranraad |
This research investigates conversational derailment leading to toxicity in GitHub discussions, aiming to predict and mitigate such occurrences proactively. The main research objective is to understand the characteristics of toxic conversations on GitHub and how these conversations derail into toxicity. The key methodology involves curating a dataset of toxic and non-toxic GitHub conversations, analyzing linguistic and conversational features, and developing a Large Language Model (LLM)-based approach using conversation trajectory summaries. The LLM prompts, tailored to provide summaries of GitHub conversations, achieved a 69% F1-score in predicting conversational derailment. AI practitioners can use this proactive, domain-specific, LLM-based moderation approach to identify and address potentially harmful conversations on platforms like GitHub before they escalate to toxicity. |
Papers for 2025-03-06
| Title |
Authors |
Summary |
| Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakers (Read more on arXiv or HuggingFace) |
LidongBing, maljunied, jhying, lukecq, Yiran0924 |
Babel is an open multilingual large language model that supports 25 languages, covering over 90% of global speakers. The main objective is to develop an open-source multilingual LLM that addresses the underrepresentation of many widely spoken languages in existing models. The key methodology is layer extension, adding new layers to an existing model (Qwen2.5) and pre-training on a curated dataset emphasizing under-resourced languages. Babel-83B-Base achieves an average score of 73.2 across six multilingual benchmarks, outperforming comparable open models like Qwen2.5-72B (69.8). AI practitioners can use Babel as a strong base or chat model for multilingual applications, benefiting from enhanced performance, especially in low-resource languages, and from the use of layer extension in scaling the model. |
| ABC: Achieving Better Control of Multimodal Embeddings using VLMs (Read more on arXiv or HuggingFace) |
Florian Kerschbaum, Benjamin Schneider, wenhu |
ABC is a multimodal embedding model that uses a vision-language model (VLM) backbone to integrate natural language instructions with visual inputs for improved control over embeddings. The main research objective is to develop a model that can effectively utilize user instructions to control and refine multimodal embeddings, overcoming limitations of existing CLIP-based models. The key methodology involves a two-stage training process: contrastive pretraining with mined negatives and instruction fine-tuning using synthetic instructions generated from image captions. The model achieves best-for-size performance on MSCOCO image-to-text retrieval with a R@1 score of 69.2 and outperforms all other models on the Massive Multimodal Embedding Benchmark (MMEB) for classification and VQA tasks. AI practitioners can use ABC’s architecture and training approach to create multimodal embedding models with enhanced control via natural language, resulting in a flexible tool that improves performance of visual retrieval, classification, and VQA, as well as the ability to complete unique, instruction-specific tasks. |
| Enhancing Abnormality Grounding for Vision Language Models with Knowledge Descriptions (Read more on arXiv or HuggingFace) |
Cosmin I. Bercea, Rossella Arcucci, Wenjia Bai, Jun Li, che111 |
This paper introduces a method to improve medical abnormality grounding in vision-language models (VLMs) using decomposed knowledge descriptions. The main research objective is to enhance the performance of VLMs in detecting and localizing medical abnormalities in images by improving the alignment between textual descriptions and visual features. The key methodology involves decomposing medical concepts into fundamental attributes and visual patterns, and using these attribute-based descriptions as prompts during VLM training. The proposed method, trained on only 1.5% of the data used by larger models, achieved a RoDeO score of 54.38% on the VinDr-CXR dataset, comparable to 7B parameter models like RadVLM. AI practitioners can use this knowledge-enhanced approach to achieve competitive performance in medical image abnormality grounding with significantly smaller VLMs and less training data, and improve zero-shot generalization. |
| GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (Read more on arXiv or HuggingFace) |
Yifan Lu, Huan Ling, Jiahui Huang, Tianchang Shen, xrenaa |
GEN3C is a generative video model with precise camera control and temporal 3D consistency. The main research objective is to develop a video generation model that allows for precise camera control and maintains 3D consistency across generated frames. The key methodology involves constructing a 3D cache (point clouds from depth estimates) and rendering it with user-provided camera trajectories to condition a fine-tuned video diffusion model. The results demonstrate that GEN3C achieves a PSNR of 18.66 and an SSIM of 0.67 on the Tanks-and-Temples dataset for single-view video generation, outperforming baselines. For AI practitioners, GEN3C offers a method for generating 3D-consistent videos with precise camera control by conditioning video generation on 3D renderings, improving controllability and consistency compared to prior video generation models. |
| KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding (Read more on arXiv or HuggingFace) |
Radha Poovendran, mingyuanzhou, yyqoni, nlpyang, flydust |
KODCODE is a synthetic dataset of 447K coding problems with verified solutions and unit tests, designed to enhance code LLM training. The main research objective is to create a large-scale, diverse, and verifiable coding dataset that addresses limitations in existing resources for training large language models (LLMs) for code. The methodology involves a three-step pipeline: coding question synthesis from 12 sources, solution and test generation with self-verification, and post-training data synthesis via question rewriting and test-based rejection sampling using DeepSeek-R1. Models fine-tuned on KODCODE-SFT achieved a 61.26% average score across five coding benchmarks, outperforming models like Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Llama-70B. The principal implication is that AI practitioners can use KODCODE to improve the performance of code LLMs in supervised fine-tuning and potentially RL training, with verified solutions and tests offering advantages for various code-related tasks. |
| CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom (Read more on arXiv or HuggingFace) |
Pan Zhou, Wenxuan Shen, Lingfeng Yang, shuaishuaicdp, yisenL |
CROWDSELECT, a novel synthetic instruction data selection framework, leverages multi-LLM responses and reward scores for improved instruction tuning. The main research objective is to investigate whether multi-dimensional signals derived from multiple LLMs can enhance the selection of synthetic instruction-response pairs for instruction tuning. The key methodology involves calculating three metrics (Difficulty, Separability, Stability) from multiple LLM responses and reward model assessments, and then integrating these with a clustering-based approach for diverse data selection. Primary results show that CROWDSELECT achieves state-of-the-art performance, improving instruction tuning by 4.81% on Arena-Hard and 11.1% on MT-bench with Llama-3.2-3b-instruct. The principal implication for AI practitioners is that leveraging multi-LLM wisdom through the proposed metrics and framework can lead to more efficient and effective instruction tuning, improving the performance of distilled smaller models. |
| QE4PE: Word-level Quality Estimation for Human Post-Editing (Read more on arXiv or HuggingFace) |
Malvina Nissim, Ana Guerberof-Arenas, Grzegorz Chrupała, Vilém Zouhar, gsarti |
The QE4PE study investigates the impact of word-level quality estimation (QE) on professional machine translation post-editing, finding that factors beyond QE accuracy influence its real-world usefulness. The main research objective was to measure the effect of word-level QE error span highlighting on the editing quality, productivity, and usability in a realistic post-editing workflow. The methodology involved 42 professional translators post-editing machine-translated texts in English-Italian and English-Dutch, using four highlight modalities (supervised, unsupervised, oracle, and no highlights) and logging their editing behavior. Results showed that highlight modalities are not solely predictive of editing time and that cross-modality highlight overlap ranged between 15% and 39%. This implies that AI practitioners should consider factors beyond accuracy, such as domain, language, and user-specific factors, to improve the integration of word-level QE in post-editing tools and enhance their real-world usability. |
| Exploring Rewriting Approaches for Different Conversational Tasks (Read more on arXiv or HuggingFace) |
Xiang Chen, Mike Rimer, Ryan A. Rossi, Md Mehrab Tanjim, Franck-Dernoncourt |
This paper systematically investigates query rewriting and fusion approaches for conversational AI tasks. The main research question is whether a single LLM-based query rewrite module can be universally effective across diverse conversational scenarios or if specialized modules are needed. The key methodology involves evaluating two parameterized query rewriting approaches (query rewrite and query fusion) on three datasets: conversational text-based Q&A and two text-to-visualization tasks (short and long conversations). The primary result is that for the conversational text-based Q&A task, the query rewrite approach achieved a 3.9% higher mean cosine similarity than query fusion, while for long text-to-vis tasks, query fusion had 7.6% high mean cosine similarity. The principal implication is that AI practitioners should select a query rewriting approach (either query rewrite and query fusion) that aligns with the specific conversational task and data characteristics, as no single approach is universally superior. |
| Process-based Self-Rewarding Language Models (Read more on arXiv or HuggingFace) |
Zheheng Luo, Junxiao Liu, Xin Zhang, Shimao Zhang, lx865712528 |
The paper introduces Process-based Self-Rewarding Language Models, enhancing mathematical reasoning by incorporating step-wise evaluations and preference optimization. The main research objective is to improve the mathematical reasoning capabilities of large language models (LLMs) using a self-rewarding paradigm without external human feedback. The key methodology involves iterative training with step-wise LLM-as-a-Judge evaluations and step-wise preference optimization using Direct Preference Optimization (DPO). The primary result is that the 72B model, after four iterations, achieved an average accuracy of 60.6 across several math benchmarks, an improvement over the starting accuracy. The principal implication is that AI practitioners can improve LLMs’ mathematical reasoning performance, through iterative self-improvement without human-annotated data. |
| Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective (Read more on arXiv or HuggingFace) |
KartikAngadi, kruthika, SyedAbdul, RakshitAralimatti |
The paper introduces the Shakti series of Small Language Models (SLMs) designed for efficient on-device AI, focusing on domain-specific applications. The main objective is to develop SLMs that can overcome resource constraints of edge devices while maintaining high performance in specialized domains. Key methodologies include a combination of efficient transformer architectures, quantization-aware training, supervised fine-tuning, and preference alignment (RLHF or DPO). Primary results show that Shakti-500-Q4 achieves 583.88 tokens per second (TPS) on an NVIDIA L40s GPU and the Shakti-250M model, after fine-tuning, achieves 0.86 answer relevance score in finance domain. The paper’s principal implication is that carefully engineered and fine-tuned compact models can effectively be deployed on edge devices, offering a practical approach for real-world, domain-specific AI applications with limited computational resources. |
| Mixture of Structural-and-Textual Retrieval over Text-rich Graph Knowledge Bases (Read more on arXiv or HuggingFace) |
Ryan A. Rossi, Haoyu Han, Yongjia Lei, mhalappa, Franck-Dernoncourt |
This paper proposes a Mixture of Structural-and-Textual Retrieval (MoR) framework for answering queries over text-rich graph knowledge bases (TG-KBs). The main research objective is to develop a retrieval method that effectively combines both textual and structural information from TG-KBs to improve query answering performance. The key methodology is a Planning-Reasoning-Organizing framework, where the Planning stage generates textual planning graphs, the Reasoning stage interweaves structural traversal and textual matching, and the Organizing stage reranks candidates based on their structural trajectory. The primary result shows that MoR achieved an average Hit@1 score of 48.93%, outperforming other baselines on three TG-KB datasets. The principal implication is that AI practitioners can leverage MoR’s mixture-of-experts approach to improve retrieval performance in applications that use the graph knowledge bases by harmonizing textual and structural signals, especially useful to combine and rank structural knowledge from graph data with traditional text features. |
| Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models (Read more on arXiv or HuggingFace) |
Shuaiqiang Wang, Pengjie Ren, Lingyong Yan, Yuhan Wang, Zhengliang Shi |
The paper introduces TOOLRET, a new benchmark for evaluating information retrieval (IR) models on tool retrieval tasks for large language models (LLMs). The main research objective is to assess the performance of existing IR models in retrieving relevant tools for LLMs in diverse, real-world scenarios, and to analyze the impact of retrieval quality on end-to-end task performance. The key methodology involves collecting and curating a large-scale dataset of 7.6k retrieval tasks and 43k tools from existing datasets, evaluating various IR models (sparse, dense, and re-ranking) on this benchmark, and contributing a large scale training dataset (TOOLRET-train) to improve retrieval performance. A primary result is that the best-performing model (NV-embedd-v1) achieves an nDCG@10 of only 33.83 on the benchmark, indicating existing IR models struggle with tool retrieval. The principal implication is that AI practitioners need to develop new retrieval methods tailored for tool retrieval, or improve upon current methods using target-aware reasoning and large-scale training data, as shown in the paper using TOOLRET-train, since current strong IR models are not effective for tool retrieval. |
| FLAME: A Federated Learning Benchmark for Robotic Manipulation (Read more on arXiv or HuggingFace) |
Danica Kragic, Yuchong Zhang, Miguel Vasco, Alberta Longhini, Santiago Bou Betran |
FLAME is a new benchmark for federated learning in robotic manipulation, providing datasets and a framework for distributed training. The main objective is to evaluate federated learning (FL) strategies for training robotic manipulation policies in a distributed, privacy-preserving manner. The key methodology involves creating a large-scale dataset of diverse manipulation tasks across multiple simulated environments and integrating it into a FL framework using FLOWER, where local models are trained and aggregated. Primary results show that Federated Averaging (FedAvg) achieves a 2.64 ± 0.13 RMSE on the Slide Block to Target task, but performance varies significantly across tasks and FL methods. The principal implication for AI practitioners is that FLAME provides a standardized benchmark for evaluating and developing scalable, adaptive, and privacy-aware robotic learning systems, although further development in FL algorithms are necessary. |
| Benchmarking Large Language Models for Multi-Language Software Vulnerability Detection (Read more on arXiv or HuggingFace) |
Hung Nguyen, Martin Weyssow, Yindu Su, Chengran Yang, Ting Zhang |
This paper presents a comprehensive empirical study evaluating large language models (LLMs) on software vulnerability detection (SVD) across multiple programming languages. The main research objective is to investigate the effectiveness of various LLMs in predicting software vulnerabilities, comparing them with smaller language models (SLMs) and static application security testing (SAST) tools, and exploring strategies to improve LLM performance. The key methodology involves compiling a multi-language dataset (Python, Java, JavaScript) of vulnerable functions, evaluating five open-source LLMs using prompt engineering, instruction tuning, and sequence classification fine-tuning, and comparing them against SLMs and SAST tools. The results show that fine-tuned LLMs achieved the best F1-score of 0.443 on the JavaScript dataset, with performance varying significantly across programming languages and adaptation strategies. The principal implication for AI practitioners is that while LLMs show promise for SVD, particularly in JavaScript with fine-tuning, performance is highly dependent on data characteristics, requiring careful consideration of language, model selection, and adaptation strategies. |
| CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs (Read more on arXiv or HuggingFace) |
Artyom Myshlyaev, Oleg Sautenkov, Muhammad Haris Khan, Valerii Serpiva, Artem Lykov |
CognitiveDrone, a Vision-Language-Action (VLA) model and benchmark for real-time cognitive task solving in UAVs, is introduced. The main research objective is to develop and evaluate a UAV control system capable of performing complex cognitive tasks, including human recognition, symbol understanding, and reasoning, based on visual input and textual instructions. The methodology combines a 7B-parameter VLA model (adapted from OpenVLA) trained on a dataset of over 8,000 simulated flight trajectories with an optional 7B-parameter VLM reasoning module (Qwen2.5-VL based) for task refinement, and evaluates performance within a Gazebo-based simulation benchmark (CognitiveDroneBench). The CognitiveDrone-R1 model, incorporating the reasoning module, achieved a 77.2% overall success rate, outperforming the base CognitiveDrone model (59.6%) and a racing-oriented model (RaceVLA, 31.3%). AI practitioners can utilize the provided open-source dataset, benchmark environment, and model weights to develop and evaluate VLA models for UAVs that incorporate cognitive capabilities beyond basic navigation and control. |
| Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner Framework for Enhancing Autonomous Vehicle Interactions (Read more on arXiv or HuggingFace) |
Peng Hang, Chen Lv, Chengkai Xu, Jiaqi Liu, FanGShiYuu |
This paper introduces an LLM-driven Actor-Reasoner framework for autonomous vehicles (AVs) to improve bidirectional interactions with human-driven vehicles (HVs). The main objective is to enhance AVs’ real-time decision-making and intent expression capabilities in complex driving scenarios with heterogeneous HVs. The methodology involves a parallel Actor-Reasoner architecture; the Reasoner uses an LLM with Chain-of-Thought (CoT) reasoning to infer HV driving styles and generate eHMI displays, while the Actor employs a two-layer memory retrieval mechanism from a database constructed during training with simulated HVs. Results show that the proposed framework achieves a 94% success rate in intersection scenarios, and a memory partition module improves retrieval speed by an average of 12%. AI practitioners can use this framework as a method to integrate LLMs into real-time decision-making systems, addressing LLM inference speed limitations by combining reasoning capabilities with memory-based fast retrieval. |
| SwiLTra-Bench: The Swiss Legal Translation Benchmark (Read more on arXiv or HuggingFace) |
Yingqiang Gao, Sina Ahmadi, Luka Nenadic, Jakob Merane, Joel Niklaus |
SwiLTra-Bench introduces a multilingual benchmark for evaluating LLM-based translation systems on Swiss legal texts, comprising 180K aligned translation pairs across five languages. The main research objective was to evaluate the performance of frontier LLMs and fine-tuned open SLMs on Swiss legal translations in zero-shot and fine-tuning settings, including the development of an LLM-based evaluation metric. Key methodology included systematic evaluation using lexical and model-based metrics, fine-tuning open SLMs, human expert validation, and developing a specialized LLM evaluation system (SwiLTra-Judge). Primary results showed that frontier models like Claude-3.5-Sonnet outperformed others, achieving a GEMBA-MQM score of 80.66, while fine-tuned open SLMs improved but still lagged behind. For AI practitioners, this benchmark and the associated evaluations highlight that while frontier models provide superior legal text translation, fine-tuning offers significant improvement for open SLMs, and SwiLTra-Judge can serve as a reliable automated evaluation tool that aligns well with human experts. |
Papers for 2025-03-05
| Title |
Authors |
Summary |
| MPO: Boosting LLM Agents with Meta Plan Optimization (Read more on arXiv or HuggingFace) |
sujianli, songff, Adagio, Rsy24, xwm |
The paper introduces Meta Plan Optimization (MPO), a framework that enhances large language model (LLM) agents’ planning capabilities by incorporating optimized, high-level meta plans. The main research objective is to improve LLM-based agents’ performance on interactive planning tasks without requiring retraining for each new agent, while addressing planning hallucinations. MPO leverages a meta planner that generates abstract task strategies, optimized via a combination of supervised fine-tuning, Monte Carlo sampling, and Direct Preference Optimization (DPO) using agent feedback. Experiments on ALFWorld and ScienceWorld benchmarks demonstrate that MPO significantly outperforms existing baselines, with performance improvements of up to 100% for some agents. For AI practitioners, MPO offers a plug-and-play solution to boost agent performance and generalization in planning tasks, by incorporating general guidance that is improvable. |
| Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs (Read more on arXiv or HuggingFace) |
Kai Chen, Chengqi Lyu, lindahua, ZwwWayne, vanilla1116 |
Mask-DPO is a fine-grained factuality alignment method for LLMs that leverages sentence-level factuality to improve preference learning and reduce hallucinations. The main research objective is to develop a more effective and generalizable method for aligning LLMs with factual correctness, addressing limitations of response-level preference learning. The key methodology, Mask-DPO, incorporates sentence-level factuality annotations as mask signals in Direct Preference Optimization (DPO), selectively learning from correct sentences in preferred responses and avoiding penalties on factual content in non-preferred responses. Primary results show that Mask-DPO improved the factuality score of Llama3.1-8B-Instruct on the ANAH test set from 49.19% to 77.53%. Principal implication for AI practitioners is that Mask-DPO provides a more precise alignment technique that enhances factuality and generalization in LLMs, enabling the development of more reliable and trustworthy AI assistants. |
| Wikipedia in the Era of LLMs: Evolution and Risks (Read more on arXiv or HuggingFace) |
Yao Wan, fjchendp, mgeng, sdzzxyl, hsm316 |
This paper analyzes the impact of Large Language Models (LLMs) on Wikipedia, examining its evolution and potential risks to the broader NLP community. The primary research objective is to determine if and how LLMs have already impacted Wikipedia, and how this might influence the NLP community. The key methodology involves analyzing Wikipedia page views, article content, and simulating LLM impact on machine translation benchmarks and Retrieval-Augmented Generation (RAG) systems. Primary results indicate that Wikipedia articles have been influenced by LLMs, with an estimated impact of 1%-2% in certain categories and simulations show potential score inflations in machine translation benchmarks and performance reduction in RAG systems using LLM generated content. The principal implication for AI practitioners is that reliance on Wikipedia for training and evaluating NLP models may be affected by LLM-generated content, necessitating careful consideration of data provenance and potential biases. |
| LADDER: Self-Improving LLMs Through Recursive Problem Decomposition (Read more on arXiv or HuggingFace) |
akiray1, TamasSimonds |
LADDER is a framework enabling large language models (LLMs) to autonomously improve problem-solving through self-guided learning by recursively generating and solving simpler problem variants. The main research objective is to develop a method for LLMs to improve their mathematical integration capabilities without curated datasets or human feedback. The key methodology, LADDER, involves recursive generation of simpler problem variants, solution verification via numerical integration, and reinforcement learning (using GRPO) on the variant trees. LADDER improved a Llama 3.2 3B model’s accuracy on undergraduate-level integration problems from 1% to 82%, and, with test-time reinforcement learning (TTRL) a Qwen 2.5 7B model achieved 90% on MIT Integration Bee. AI practitioners can leverage self-improving systems like LADDER and TTRL to enhance model capabilities in verifiable domains without extensive human supervision or data curation, demonstrating a practical path to developing more autonomous and capable AI. |
| MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents (Read more on arXiv or HuggingFace) |
mikewang, ShuyiGuo, Thomas-X-Yang, zhaochenhong, Leozkl |
MultiAgentBench is a benchmark designed to evaluate LLM-based multi-agent systems across diverse interactive scenarios, measuring task completion and the quality of collaboration and competition. The main research objective is to assess how well LLM-based multi-agent systems perform in collaborative and competitive environments, using novel milestone-based key performance indicators. The methodology involves evaluating various coordination protocols (star, chain, tree, graph) and strategies (group discussion, cognitive planning) in six interactive scenarios, including research, Minecraft, database, coding, bargaining, and Werewolf, developed using the MARBLE framework. Results show gpt-4o-mini achieves the highest average task score, graph structure performs best in research, and cognitive planning improves milestone achievement rates by 3%. For AI practitioners, the framework and benchmark provide a means to systematically evaluate and improve multi-agent coordination, which is critical in developing more effective and collaborative AI systems. |
| PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization (Read more on arXiv or HuggingFace) |
Min Lin, Xinyi Wan, JialinLi, huanggx-sea, QPHutu |
PipeOffload enhances pipeline parallelism (PP) scalability for large language models (LLMs) by optimizing activation memory usage through offloading. The main research objective is to address the activation memory bottleneck in PP that limits its scalability. The key methodology involves selectively offloading activations to host memory, prioritizing those with longer lifespans, and integrating a generalized interleaving strategy for balancing memory and throughput. The primary result is that PipeOffload reduces per-device activation memory in a better-than-linear manner, enabling up to a 19% acceleration compared to tensor parallelism (TP), while using less memory in applicable cases. For AI practitioners, PipeOffload provides a more scalable PP method, especially beneficial when full activation offload is feasible (k <= 1), allowing for more efficient training of large models. |
| Iterative Value Function Optimization for Guided Decoding (Read more on arXiv or HuggingFace) |
Ruizhe Chen, jokephp, ab3223323, lljhbxt, zhliu |
Iterative Value Function Optimization (IVO) is a novel framework for guided decoding that improves the accuracy of value estimation in language models without retraining the base model. The main research objective is to address the limitations of existing value-guided decoding methods, which suffer from inaccurate value estimation due to high variance and distribution shift. The key methodology involves two components: Monte Carlo Value Estimation, which reduces estimation variance by exploring diverse trajectories, and Iterative On-Policy Optimization, which progressively improves value estimation through collecting trajectories from value-guided policies. Primary results show that IVO achieves 77.52% GPT-4 win rates on the Multi-turn Dialogue task against the base policy, significantly outperforming baseline methods in terms of reward scores across various tasks. Principal implication for AI practitioners is that IVO offers a computationally efficient way to align language models with human values and task requirements, improving control over model outputs without expensive retraining. |
| FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling (Read more on arXiv or HuggingFace) |
yuxuanli, zwl96, hyx21, ThonyPan, Achazwl |
FR-Spec accelerates large-vocabulary language models by optimizing draft candidate selection in speculative sampling. The main research objective is to address the increased computational overhead of the LM Head in speculative sampling when using models with large vocabularies. The key methodology is frequency-ranked speculative sampling, which constrains the draft search to a frequency-prioritized token subset, reducing LM Head computation. Primary results show an average 1.12x speedup over the state-of-the-art speculative sampling method EAGLE-2 on multiple datasets, with optimized drafting reducing computation by 75%. For AI practitioners, this method provides a plug-and-play solution to accelerate existing speculative sampling techniques without retraining, directly improving inference speed for large-vocabulary language models. |
| SemViQA: A Semantic Question Answering System for Vietnamese Information Fact-Checking (Read more on arXiv or HuggingFace) |
Thanh T. Tran, ThanhDi, TienAnh, xuandin, DavidNguyen |
SemViQA is a Vietnamese language fact-checking system that enhances accuracy and efficiency through semantic understanding. The main research objective is to develop a robust fact-checking system for Vietnamese, a low-resource language, addressing challenges like semantic ambiguity and long-token sequences. The key methodology integrates Semantic-based Evidence Retrieval (SER), combining TF-IDF and a Question Answering Token Classifier (QATC), with a Two-step Verdict Classification (TVC) using Focal Loss and Cross-Entropy Loss. The system achieves a strict accuracy of 80.82% on the ViWikiFC dataset and 78.97% on the ISE-DSC01. The principal implication is that AI practitioners can leverage SemViQA’s framework, particularly its SER and TVC components, to develop more efficient, robust, and effective fact-checking systems that handle complex linguistic structures, especially in low-resource languages. |
| UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface (Read more on arXiv or HuggingFace) |
windmillknight, Shawnee-bxy, Haiyang-W, chenweix7, kanashi6 |
UFO unifies fine-grained visual perception tasks through an open-ended language interface, achieving state-of-the-art performance without task-specific decoders. The main research objective is to effectively integrate fine-grained perception tasks (like detection and segmentation) into multimodal large language models (MLLMs) without relying on complex, task-specific designs. The key methodology involves transforming all perception targets into the language space and using a novel embedding retrieval approach for segmentation, relying solely on the language interface. After multi-task training, UFO outperforms previous state-of-the-art generalist models by 12.3 mAP on COCO instance segmentation and 3.3 mIoU on ADE20K semantic segmentation. AI practitioners can leverage UFO’s unified framework to simplify architectural design and training, seamlessly integrating fine-grained perception capabilities into MLLMs for enhanced visual understanding and enabling more challenging vision-language tasks. |
| ATLaS: Agent Tuning via Learning Critical Steps (Read more on arXiv or HuggingFace) |
Yuxuan Huang, Ming Li, Zhixun Chen, zhoutianyi, YaliDU |
ATLAS finetunes large language model (LLM) agents on critical steps within expert trajectories to improve generalization and reduce training costs. The main research objective is to develop a more efficient and effective agent tuning method by identifying and focusing on critical steps in expert trajectories. The key methodology, ATLAS, uses an oracle LLM to select critical steps based on criteria like plan creation, critical observation, critical action, and self-correction, then finetunes the agent’s LLM solely on these steps. Results show that an LLM finetuned on only ~30% critical steps selected by ATLAS outperforms the LLM finetuned on all steps and recent open-source LLM agents. The principal implication is that AI practitioners can achieve better agent generalization and performance with reduced training costs by focusing LLM finetuning on semantically critical steps identified by an oracle LLM. |
| Language Models can Self-Improve at State-Value Estimation for Better Search (Read more on arXiv or HuggingFace) |
rittera, emendes3 |
Self-taught lookahead (STL) enables language model-based value functions to improve without ground truth rewards by leveraging state-transition dynamics. The main research objective is to demonstrate that an LLM-based value function can self-improve without labels or rewards, outperforming computationally expensive methods. The key methodology, STL, fine-tunes a value model by predicting the next best action, resulting state, and value rationale, bootstrapping from an initial value function using lookahead in tree search. Results show that STL-improved models match the performance of a GPT-4 value model, improving performance by 20% while reducing inference costs 37x compared to prior LLM-based tree search. Principal implication is that AI practitioners can utilize STL to train efficient and effective value models for search-based tasks, reducing reliance on expensive closed-source models and ground truth rewards. |
| RectifiedHR: Enable Efficient High-Resolution Image Generation via Energy Rectification (Read more on arXiv or HuggingFace) |
Liang Hou, dizhang, wileewang, PaulSHEN1, YZCS |
RectifiedHR is a training-free method for generating high-resolution images with diffusion models by addressing energy decay and employing noise refresh. The main objective is to enable diffusion models to efficiently generate images at resolutions higher than their training resolution without additional training. The key methodology involves a noise refresh strategy to progressively increase resolution during sampling and an energy rectification strategy that adjusts classifier-free guidance to mitigate image blurriness. The primary result is that RectifiedHR achieves a FID score of 25.347 and a CLIP score of 33.756 at 2048x2048 resolution, outperforming several baselines in image quality while using less computing time. The principal implication is that AI practitioners can generate high-quality, high-resolution images using pre-trained diffusion models without costly retraining or complex modifications, by using noise refresh and energy rectification steps during image generation. |
| SPIDER: A Comprehensive Multi-Organ Supervised Pathology Dataset and Baseline Models (Read more on arXiv or HuggingFace) |
Ekaterina Ivanova, alpchel, mgvz |
SPIDER is a new multi-organ histopathology dataset with baseline models for patch-level classification and whole-slide image segmentation. The main research objective is to create and evaluate a large, high-quality, multi-organ, patch-level histopathology dataset with comprehensive class coverage, along with baseline classification models. Key methodology used is a semi-automatic annotation pipeline, expert pathologist verification, feature extraction with Hibou-L foundation model, and an attention-based classification head. Primary results of SPIDER’s evaluation include, on the thorax test set, model achieved an accuracy of 0.962, precision of 0.958, and F1 score of 0.960. AI practitioners can use this dataset and models to improve digital pathology tasks like tissue classification and rapid identification, providing a new benchmark for future developments in this field. |
| Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content (Read more on arXiv or HuggingFace) |
Zicheng Zhang, GTZhai, a9108, sl2782087, wcain |
The paper introduces Q-Eval-100K, a large-scale dataset, and Q-Eval-Score, a unified model, for evaluating visual quality and text-image/video alignment in text-to-vision generation. The main research objective is to develop a comprehensive benchmark and method for assessing both the visual quality and text-alignment of content generated by text-to-vision models. The key methodology involves collecting 100K instances (images and videos) with 960K human annotations of Mean Opinion Scores (MOS) and developing Q-Eval-Score, a Large Multimodal Model (LMM) fine-tuned using a context-prompt format. The primary results show that Q-Eval-Score achieves a 0.943 SRCC for image visual quality at the model-level, outperforming existing methods, it also introduces Vague-to-Specific Strategy for long prompt alignment. AI practitioners can use Q-Eval-100K and Q-Eval-Score as a reliable benchmark and evaluation metric to assess and improve the performance of text-to-vision generative models, focusing on both visual quality and text-alignment. |
| IterPref: Focal Preference Learning for Code Generation via Iterative Debugging (Read more on arXiv or HuggingFace) |
Ruihang, yangyu90, Jianwen2003, CharonBony, Ringo1110 |
IterPref is a new preference alignment framework for code generation that improves Code LLMs through iterative debugging. The research objective is to address the limitation of existing preference learning methods that do not pinpoint specific code errors, hindering the learning of informative error correction patterns. The key methodology is IterPref, which involves creating the CodeFlow dataset where code is iteratively refined until passing tests, and using a tailored DPO algorithm to align corresponding tokens for error regions. Primary result is that, equipped with IterPref, Qwen2.5-Coder-7B achieved a 29.7% pass@1 score on BigCodeBench Complete Hard, on par with some much larger models. For AI practitioners, this implies an effective way to enhance code generation models that leverages an iterative debugging process for precise preference learning, focusing model’s learning on correcting critical errors. |
| AppAgentX: Evolving GUI Agents as Proficient Smartphone Users (Read more on arXiv or HuggingFace) |
Chi Zhang, Wenjia Jiang, xuyang, ChenxiSong, yyzhuang2 |
AppAgentX introduces an evolutionary framework for GUI agents that improves operational efficiency on smartphones while maintaining adaptability. The main research objective is to address the inefficiency of LLM-based GUI agents in performing routine tasks by enabling them to learn and evolve high-level actions. The key methodology involves a memory mechanism that records task execution history, allowing the agent to identify repetitive action sequences and replace them with abstract, high-level actions represented as “shortcut nodes”. Primary results show that on the AppAgent benchmark, AppAgentX reduced the average steps per task from 9.1 to 5.7 and increased the success rate from baseline 16.9% to 71.4% . For AI practitioners, this evolutionary framework offers a method to develop GUI agents that execute routine operations more efficiently while using LLM only to optimize new behavior, thus improving the balance between intelligence and efficiency in practical applications. |
Papers for 2025-03-04
| Title |
Authors |
Summary |
| Visual-RFT: Visual Reinforcement Fine-Tuning (Read more on arXiv or HuggingFace) |
yhcao, sweetFruit, yuhangzang, Zery, ziyuliu |
Visual-RFT extends Reinforcement Fine-Tuning (RFT) to visual tasks by using verifiable rewards to improve performance of Large Vision-Language Models (LVLMs). The main objective is to apply RFT, previously successful in language models, to multi-modal domains, specifically visual perception tasks, with limited data. The key methodology is using LVLMs to generate multiple responses with reasoning tokens and applying visual perception verifiable reward functions (e.g., IoU for object detection) to update the model via policy optimization algorithms like Group Relative Policy Optimization (GRPO). Visual-RFT improved accuracy by 24.3% over the baseline in one-shot fine-grained image classification and exceeded SFT baselines by 21.9 and 15.4 on COCO and LVIS, in two-shot settings, respectively. For AI practitioners, Visual-RFT offers a data-efficient, reward-driven approach to enhance reasoning and adaptability in LVLMs for domain-specific tasks, particularly when fine-tuning data is scarce. |
| Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models (Read more on arXiv or HuggingFace) |
zgojcic, AnalMom, xrenaa, hturki, jayw |
DIFIX3D+ enhances 3D reconstruction and novel-view synthesis using single-step diffusion models. The main research objective is to improve the quality of 3D reconstructions, especially in under-constrained regions, by leveraging 2D diffusion model priors. The methodology involves fine-tuning a single-step image diffusion model (DIFIX) to remove artifacts in rendered novel views, and using it both during reconstruction to clean pseudo-training views and as a neural enhancer during inference. Primary results show an average 2x improvement in FID score over baselines while maintaining 3D consistency, with compatibility across both NeRF and 3DGS representations. The principal implication is that AI practitioners can leverage single-step diffusion models for real-time post-processing to improve the visual quality of 3D reconstructions and novel view synthesis. |
| Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs (Read more on arXiv or HuggingFace) |
vishravmsft, martincai, alonbenhaim, jianmin-ustc, atabakashfaqMSFT |
Phi-4-Mini and Phi-4-Multimodal are 3.8-billion-parameter language and multimodal models trained on high-quality data, achieving strong performance relative to their size. Main research question or objective: To develop compact yet highly capable language and multimodal models that outperform similar-sized open-source models and rival larger models, using curated data and novel architecture techniques. Key methodology used: The researchers trained Phi-4-Mini on high-quality web and synthetic data, with emphasis on math and coding datasets, expanded the vocabulary to 200K tokens, used grouped query attention, and a fractional RoPE dimension. For Phi-4-Multimodal, they used a “Mixture of LoRAs” technique, integrating modality-specific LoRAs while freezing the base language model. Primary results: Phi-4-Mini outperformed similarly sized models and matched the performance of models twice its size on math/coding, and Phi-4-Multimodal ranked first on the OpenASR leaderboard at the time, with the speech/audio LoRA having only 460 million parameters. Phi-4-Multimodal outperformed larger vision-language models, and achieved 72.0 average score across various vision-language benchmarks. Principal implication for AI practitioners: AI/ML/Software Engineers and Data Scientists can leverage Phi-4-Mini and Phi-4-Multimodal as efficient and performant small language and multimodal models, achieving strong performance while keeping the base language model frozen, making it a practical solution in resource-constrained environments. |
| OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment (Read more on arXiv or HuggingFace) |
GuoruiZhou, DingWF, caikuo, oneself, OrpheusBetter |
OneRec is an end-to-end generative recommendation model that unifies retrieval and ranking stages. The main research objective is to develop a single-stage generative model that surpasses the performance of traditional multi-stage recommender systems in real-world scenarios. The key methodology involves an encoder-decoder architecture with Mixture-of-Experts (MoE), session-wise generation, and Iterative Preference Alignment (IPA) combined with Direct Preference Optimization (DPO) using a reward model. Primary results show that OneRec deployed in Kuaishou’s main scene achieved a 1.68% increase in watch-time, a substantial improvement over the previous system. For AI practitioners, OneRec demonstrates the feasibility of achieving significant performance gains by replacing a cascaded ranking system with a unified generative model by utilizing techniques like MoE and IPA. |
| Liger: Linearizing Large Language Models to Gated Recurrent Structures (Read more on arXiv or HuggingFace) |
Yu Cheng, JusenK, Jiaxihu2, weigao266, landisen |
Liger transforms pretrained Transformer-based large language models (LLMs) into gated linear recurrent structures for efficient deployment. The main research objective is to linearize LLMs into gated recurrent structures without adding extra parameters and with minimal performance loss. The key methodology involves repurposing pretrained key matrix weights to construct gating mechanisms and using Low-Rank Adaptation (LoRA) for lightweight fine-tuning. The primary result is that Liger recovers 93% of the Transformer-based Llama-3 8B model’s performance using only 0.02% of pre-training tokens during linearization. AI practitioners can deploy LLMs more efficiently with linear-time inference and constant memory usage by converting them to gated recurrent structures using Liger. |
| When an LLM is apprehensive about its answers – and when its uncertainty is justified (Read more on arXiv or HuggingFace) |
Alexey Zaytsev, Edvard Khalafyan, DanielVyazhev, aigoncharov, sspetya |
The paper investigates uncertainty estimation in Large Language Models (LLMs) for multiple-choice question answering, focusing on entropy and model-as-judge (MASJ) approaches. The main research question is how well token-wise entropy and MASJ estimates reflect LLM error and question difficulty across different domains and reasoning requirements. The key methodology involves evaluating three LLMs (Phi-4, Mistral, Qwen) on the MMLU-Pro dataset, using an auxiliary LLM to label questions by reasoning/knowledge needs and comparing uncertainty estimates with correctness labels. A primary result is that response entropy predicts model error effectively in knowledge-dependent domains (biology ROC AUC = 0.73), but this correlation weakens for reasoning-dependent domains (math ROC AUC = 0.55). For AI practioners this indicates, that the data-uncertainty related entropy is a useful measure in uncertainty estimate frameworks and should be integrated, but its usefulness is dependent to how much reasoning is requred to solve the problem. |
| DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion (Read more on arXiv or HuggingFace) |
Guobin Ma, Chunbo Hao, Yuepeng Jiang, Huakang Chen, Ziqian Ning |
DiffRhythm is a latent diffusion-based model that generates full-length songs with vocals and accompaniment, achieving high musicality, intelligibility, and fast inference speeds. The main research objective is to develop an end-to-end song generation model capable of synthesizing complete songs (up to 4m45s) with both vocal and accompaniment, overcoming limitations of existing approaches like multi-stage architectures and slow inference. Key methodology involves a Variational Autoencoder (VAE) for learning compact latent representations of waveforms and a Diffusion Transformer (DiT) operating in the latent space, along with a novel sentence-level lyrics alignment mechanism. Primary results show that DiffRhythm achieves a Phoneme Error Rate (PER) of 18.02% in full-length song generation with a real-time factor (RTF) of 0.034. AI practitioners can leverage DiffRhythm’s simple architecture, fast non-autoregressive generation, and open-sourced code/models for scalable, end-to-end song generation research and applications, eliminating the need for complex multi-stage cascading modelling. |
| Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs (Read more on arXiv or HuggingFace) |
ngoodman, nlile, Asap7772, ayushchakravarthy, obiwan96 |
i) This paper investigates cognitive behaviors that enable language models to effectively self-improve via reinforcement learning. ii) The research question is: what intrinsic properties enable effective self-improvement in language models trained with reinforcement learning? iii) The methodology involves analyzing verification, backtracking, subgoal setting, and backward chaining in Qwen and Llama models during reinforcement learning on the Countdown game, alongside controlled behavioral dataset experiments and pretraining data curation. iv) Results show that Qwen naturally exhibits reasoning behaviors whereas Llama lacks them, priming Llama with these behaviors enables substantial improvements during RL; models primed with incorrect solutions but proper reasoning patterns achieve comparable performance to those trained on correct solutions, and curated pretraining data amplified Llama’s reasoning behaviors. v) AI practitioners should consider the initial reasoning behaviors of language models as a critical factor in determining their capacity for self-improvement via reinforcement learning, and potentially curate pretraining data to enhance those behaviors. |
| Speculative Ad-hoc Querying (Read more on arXiv or HuggingFace) |
Venkat Arun, Aditya Akella, Maria Angels de Luis Balaguer, Srikanth Kandula, Haoyu0529 |
SpeQL, a system that reduces query latency by using large language models (LLMs) to predict and precompute SQL queries during user input, improves analytical query responsiveness. The research objective is to determine if query execution can begin before a user finishes typing an SQL query, enabling near-instantaneous results. The methodology involves using LLMs to predict query structure and precompute temporary tables, alongside a scheduler that manages query execution and a user interface that displays speculative results. Results from experiments on 103 TPC-DS queries at 100GB scale show that SpeQL reduces P90 planning, compilation, and execution latency by 94.42%, 99.99%, and 87.23%, respectively, with a 7.72 seconds P90 execution overhead. AI practitioners can leverage SpeQL’s approach to improve the responsiveness of interactive data analysis systems, thereby enabling quicker insight discovery during exploratory data analysis. |
| Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions (Read more on arXiv or HuggingFace) |
Xiaohui He, Jia Chen, aiqy, haitaoli, qian |
Qilin is a new multimodal information retrieval dataset collected from a social platform, Xiaohongshu, for improving search and recommendation services. The main research objective is to create a dataset that facilitates the development of advanced multimodal neural retrieval models across diverse task settings with real-world user interaction data. The key methodology involves collecting user sessions with heterogeneous results (image-text, video, commercial notes, direct answers) and APP-level contextual signals, then filtering the data using LLMs and human verification for safety and privacy. Primary results include a dataset of APP-level sessions from 15,482 users, where search users browse an average of 23.41 items when Deep Query Answering (DQA) is not triggered, but only 10.61 items when DQA is triggered. Principal implication for AI practitioners is that Qilin provides a realistic, large-scale, multimodal dataset with rich contextual information for training, evaluating, and analyzing retrieval-augmented generation systems and other advanced search and recommendation models, taking into account complex user behaviors. |
| DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting (Read more on arXiv or HuggingFace) |
xpqiu, QipengGuo, KYLN24, KaiLv |
DuoDecoding is a novel speculative decoding method that leverages heterogeneous hardware to accelerate large language model inference. The main research objective is to reduce generation latency in large language models (LLMs) while maintaining output distribution fidelity and reducing the time to first token (TTFT). The key methodology involves deploying the draft model on the CPU and the target model on the GPU, enabling parallel decoding, along with a hardware-aware optimal draft budget and dynamic multi-sequence drafting. DuoDecoding achieves up to a 2.61x speedup in generation latency compared to vanilla autoregressive generation and reduces TTFT to 83% of that in conventional speculative decoding. The principal implication for AI practitioners is that DuoDecoding provides a method to significantly improve the inference speed of LLMs, particularly beneficial for interactive applications, by utilizing both CPU and GPU resources effectively. |
| Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generation (Read more on arXiv or HuggingFace) |
yingcongchen, Xxlbigbrother, StarYDY, MeixiChen, LTT |
Kiss3DGen is a framework that repurposes 2D image diffusion models for 3D asset generation, including tasks like text-to-3D, image-to-3D, editing, and enhancement. The main research objective is to develop an efficient method for generating, editing, and enhancing 3D objects by leveraging pretrained 2D image diffusion models, without the need of large-scale 3D datasets. The key methodology involves fine-tuning a diffusion model (Flux) to generate “3D Bundle Images”—tiled representations of multi-view images and normal maps—which are then used to reconstruct a 3D mesh. The method achieves a CLIP score of 0.837 in text-to-3D generation evaluation, outperforming 3DTopia, Direct2.5, and Hunyuan3D-1.0. AI practitioners can utilize this framework to efficiently create high-quality 3D models by maximizing the use of pre-trained 2D diffusion models, thus reducing the dependency on extensive 3D training data. |
| Word Form Matters: LLMs’ Semantic Reconstruction under Typoglycemia (Read more on arXiv or HuggingFace) |
Lang Gao, Zhongyu Wei, Ziruibest, Carol0110, Aurora-cx |
Large Language Models (LLMs) reconstruct the meaning of scrambled words primarily using word form, with minimal reliance on contextual information. The main research question is how word form and contextual information influence LLMs’ semantic reconstruction ability under Typoglycemia. The researchers used controlled experiments on LLaMA models, varying Scramble Ratio (SR) and Context Integrity (CI), and introduced SemRecScore to quantify semantic reconstruction. Primary results show SemRecScore decreases as SR increases, and at a Scramble Ratio (SR) of 1, a final SemRecScore of only 0.5 is achieved on the final LLM layer, indicating incomplete semantic reconstruction. For AI practitioners, this highlights that improvements can come by incorporating human-like, context-aware mechanisms, as current attention mechanisms focus primarily on the word form. |
| SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity (Read more on arXiv or HuggingFace) |
bitwjg, WeiWang, WQYC, DeyangKong, xixy |
SampleMix is a sample-wise pre-training data mixing strategy for large language models that coordinates data quality and diversity. The main research objective is to address the limitations of existing domain-wise data mixing methods, which overlook inter-domain overlaps and use suboptimal sample distributions. The key methodology involves evaluating the quality and diversity of each sample, assigning sampling weights, and constructing a training dataset based on these weights. The primary results show that SampleMix achieves an average accuracy of 47.77% across eight downstream tasks, outperforming all baseline methods, and reaching baseline performance with 1.9x fewer training steps. The principal implication is that AI practitioners can use SampleMix to improve training efficiency and model performance by creating better data mixtures by incorporating sample-wise quality and diversity evaluations. |
| From Hours to Minutes: Lossless Acceleration of Ultra Long Sequence Generation up to 100K Tokens (Read more on arXiv or HuggingFace) |
Yuxuan Wang, zlzheng, vickyandkekey, JunzheS, TongWu |
TOKENSWIFT accelerates ultra-long sequence generation for large language models without compromising output quality. The main research question is whether model-agnostic, lossless acceleration can be achieved for generating ultra-long sequences with minimal training overhead. The key methodology involves multi-token parallel self-drafting with the target model, token reutilization, dynamic KV cache management, and contextual penalty. Primary results show that TOKENSWIFT achieves over 3x speedup compared to autoregressive generation across various models, reducing generation time for 100K tokens on LLAMA3.1-8b from nearly 5 hours to 90 minutes. Principal implication for AI practitioners is TOKENSWIFT provides a scalable and effective solution to dramatically speed up ultra long text generation, enabling applications that require producing very large outputs. |
| Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model (Read more on arXiv or HuggingFace) |
Jianan Wang, Xili Dai, xyyue, qixianbiao, yxuan |
The paper introduces Plane-DUSt3R, a novel method for multi-view room layout estimation using the DUSt3R 3D foundation model. The main research objective is to develop a method for 3D room layout estimation from multiple unposed, sparse-view images. The methodology involves fine-tuning DUSt3R on a room layout dataset with a modified objective to estimate structural planes and combining it with a 2D plane detector and a post-processing algorithm. The Plane-DUSt3R achieves a 5.27% and 5.33% improvement in RRA and mAA metrics, respectively, for multi-view correspondence tasks, compared to state-of-the-art methods on the Structure3D dataset. AI practitioners can use Plane-DUSt3R to generate 3D room layouts from unposed images, eliminating the need for precise camera poses and simplifying multi-view 3D reconstruction. |
| CodeArena: A Collective Evaluation Platform for LLM Code Generation (Read more on arXiv or HuggingFace) |
terryyz, DongHuang-ebay, bobxwu, anhtuanluu36, Elfsong |
CodeArena is an online platform for evaluating large language models (LLMs) on code generation tasks, incorporating a collective evaluation mechanism. The main objective is to address limitations in existing LLM code generation evaluation, such as benchmark contamination, data dissipation, and system inaccessibility. The key methodology involves a dynamic scoring system that adjusts model scores based on the collective performance of all submissions, along with providing automation-friendly APIs and open access to solutions and test cases. Results show that closed-source LLMs generally outperform open-source models, with “DeepSeek-Coder” achieving a Dynamic Point score of 249.28 and solving 90.63% of the problems. AI practitioners can use CodeArena for unbiased LLM code generation evaluation, accessing a public repository of solutions and test cases, and streamlining the evaluation process with automation-ready APIs. |
| VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generation (Read more on arXiv or HuggingFace) |
Yi Yang, WenhaoWang |
VideoUFO is a million-scale video dataset designed to align text-to-video generation models with real-world user preferences. The main research objective is to curate a video dataset that reflects user-focused topics and evaluate its impact on text-to-video model performance. The key methodology involves clustering user-provided prompts from VidProM to identify 1,291 topics, retrieving relevant videos from YouTube, segmenting them into clips, generating captions, and assessing video quality using VBench. Primary results show that a model trained on VideoUFO achieves a low-10 score of 0.442, outperforming models trained on other datasets, while maintaining a top-10 score of 0.651 on a benchmark of user-focused topics. For AI practitioners, the VideoUFO dataset provides a resource for training or fine-tuning text-to-video models to better meet user expectations in real-world, diverse applications. |
| Large-Scale Data Selection for Instruction Tuning (Read more on arXiv or HuggingFace) |
pradeepd, pangwei, faezeb, nanami, hamishivi |
This paper systematically investigates the scaling properties of automated data selection methods for instruction-tuning language models. The main research objective is to determine how well various data selection approaches perform when selecting large datasets (up to 2.5M samples) from large pools (up to 5.8M samples) for instruction tuning. The key methodology involves comparing nine data selection techniques, including representation-based, gradient-based, and loss/perplexity-based methods, across multiple dataset sizes and selection pools, evaluating performance on seven diverse tasks. The primary result is that a variant of representation-based data selection (RDS+) consistently outperforms other methods, including random selection, achieving an average score of 50.5 versus 46.4 for the next best method (Embed (GTR)) when selecting 10k data points. This implies that AI practitioners should consider using the proposed simple, embedding-based RDS+ method, especially in large-scale settings, rather than more computationally expensive methods when selecting data for finetuning LLMs. |
| Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator (Read more on arXiv or HuggingFace) |
mingyuliutw, gdhe17, HuayuChen, Ema11, worstcoder |
Direct Discriminative Optimization (DDO) finetunes likelihood-based visual generative models using a GAN-inspired objective without extra networks. The research aims to improve the sample quality of likelihood-based generative models beyond the limitations of maximum likelihood estimation (MLE). DDO implicitly parameterizes a discriminator using the likelihood ratio between a learnable target model and a fixed, pretrained reference model, optimizing the target model with a GAN discriminator loss. Finetuning a diffusion model (EDM) with DDO achieved a new record FID score of 1.30 on CIFAR-10, a significant improvement over the base model’s 1.79. AI practitioners can directly finetune and iteratively refine pretrained likelihood-based generative models to achieve state-of-the-art performance without modifying model architecture or inference procedures. |
| AI-Invented Tonal Languages: Preventing a Machine Lingua Franca Beyond Human Understanding (Read more on arXiv or HuggingFace) |
dnoever |
This paper explores the potential for large language models (LLMs) to create private tonal languages for machine-to-machine communication. The main research question is whether AI agents can autonomously invent and use private tonal languages, and what those languages might resemble. The key methodology involves implementing a character-to-frequency mapping system using musical semitones to encode the full ASCII character set, creating a prototype tonal language. Primary results demonstrate that tonal encoding can achieve information rates exceeding human speech, with the ASCII mapping spanning approximately 7.8 octaves (220 Hz to 50175.42 Hz). The principle implication for AI practioners is that LLMs could theoretically engage in M2M communications, partially or wholly, outside of human perceptual boundaries, raising a need for transparency, oversight, and governance strategies in AI development. |
| CLEA: Closed-Loop Embodied Agent for Enhancing Task Execution in Dynamic Environments (Read more on arXiv or HuggingFace) |
Qing Zhao, Zhixin Mai, Yiming Zhao, Ge Wang, SP4595 |
CLEA is a closed-loop embodied agent framework that enhances task execution in dynamic environments using multiple LLMs. The main research objective is to address the limitations of Large Language Models (LLMs) in embodied systems for reliable execution of subtask sequences and one-shot success in long-term tasks within dynamic environments. The key methodology involves a closed-loop architecture with four specialized open-source LLMs and a planner-critic framework, integrating environmental memory and multimodal feedback for dynamic task management. Across 12 task trials, CLEA achieved a 67.3% improvement in success rate and a 52.8% increase in task completion rate compared to the open-loop baseline. For AI practitioners, the framework offers a robust method for deploying embodied agents in real-world, dynamic settings by facilitating adaptive strategy adjustment, enhancing task planning, and improving execution through continuous environmental feedback. |
Papers for 2025-03-03
| Title |
Authors |
Summary |
| DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking (Read more on arXiv or HuggingFace) |
luyaojie, sanmusunrise, xuanang, yhycai, lzq2021 |
The paper introduces a new benchmark and system for complex engineering solution design. The main research objective is to evaluate and improve systems’ ability to generate complete and feasible solutions for engineering problems with multiple constraints. The key methodology is SolutionRAG, leveraging tree-based exploration and a bi-point thinking mechanism (alternating solution design and review) to generate solutions. SolutionRAG achieved a 66.4 analytical score and 67.9 technical score on the SolutionBench, outperforming baselines like Naive-RAG and Self-RAG. AI practitioners can use SolutionBench to benchmark and the SolutionRAG architecture to improve the generation of solutions for complex, multi-constraint engineering problems. |
| Chain of Draft: Thinking Faster by Writing Less (Read more on arXiv or HuggingFace) |
Lingxiao Zhao, Wenhao Xie, DeBERTa, sileixu |
Chain of Draft (CoD) is a new prompting strategy that improves the efficiency of large language models (LLMs) by generating concise reasoning steps. The research proposes and evaluates Chain of Draft (CoD), a prompting method that minimizes verbosity in LLM reasoning. CoD prompts LLMs to produce brief, information-dense intermediate steps, resembling human draft-thinking, during multi-step reasoning tasks. The results show that CoD matches or surpasses Chain-of-Thought (CoT) accuracy on GSM8K, date, sports, and coin flip tasks, while using up to 92.4% fewer tokens in a specific Sports Understanding case. AI practitioners can use CoD to reduce latency and computational costs in LLM applications without significantly sacrificing accuracy, especially in resource-constrained environments. |
| ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents (Read more on arXiv or HuggingFace) |
xpjandy, shihang, vickywu, lovesnowbest, autumncc |
ViDoRAG is a multi-agent RAG framework for visually-rich documents using dynamic retrieval and iterative reasoning. The main research objective is to address the limitations of existing RAG methods in handling visually rich documents, particularly the challenges of multi-modal retrieval and insufficient reasoning capabilities. The methodology employs a Gaussian Mixture Model (GMM)-based hybrid retrieval strategy (textual and visual) and a multi-agent framework (seeker, inspector, answer) for iterative reasoning. Primary results show ViDoRAG outperforms existing methods on the ViDoSeek benchmark by over 10% in overall accuracy. AI practitioners can leverage ViDoRAG’s multi-agent framework and dynamic retrieval strategy to build more effective and robust RAG systems for applications dealing with visually rich documents. |
| SoS1: O1 and R1-Like Reasoning LLMs are Sum-of-Square Solvers (Read more on arXiv or HuggingFace) |
Coralia Cartis, Wenqi Zhu, Kechen Li, Shiweiliuiiiiiii, jitianbo |
Large Language Models (LLMs) can be effectively used to solve sum-of-squares (SoS) polynomial problems with proper reasoning guidance. The main research question is whether LLMs can determine the nonnegativity of a given multivariate polynomial, a computationally intractable problem related to Hilbert’s Seventeenth Problem. The researchers introduced a dataset (SoS-1K) of ~1,000 polynomials and evaluated various LLMs using plain questions, simple instructions, and expert-designed reasoning instructions based on five criteria. The results show that high-quality reasoning instructions significantly improve accuracy, with the best-performing model (DeepSeek-R1) reaching 81% accuracy with SoS Reasoning instructions, compared to around 60% with plain question. Supervised fine-tuning of a 7B model on SoS-1K achieved 70% accuracy outperforming the 671B Deepseek-V3. AI practitioners can leverage specialized datasets and reasoning-guided instructions to significantly enhance LLMs’ ability to solve complex mathematical problems and tackle NP-hard problems. |
| Optimal Brain Apoptosis (Read more on arXiv or HuggingFace) |
Delei Kong, Junjie Jiang, Jiaxu Wang, Zheng Fang, Mingyuan Sun |
Optimal Brain Apoptosis (OBA) is a novel pruning method that calculates the Hessian-vector product to estimate parameter importance for neural network compression. The main research objective is to develop a more precise and efficient pruning method that avoids approximations of the Hessian matrix used in prior work. The key methodology involves decomposing the Hessian matrix across network layers, identifying conditions for non-zero inter-layer Hessian submatrices, and efficiently computing the second-order Taylor expansion of parameters using a Jacobian-vector product forward propagation technique. The primary results show that OBA achieves a 2x speedup on ImageNet with ResNet50 with only a 0.53% accuracy decrease, outperforming existing methods. The principal implication for AI practitioners is that OBA offers a more accurate and efficient way to prune both convolutional neural networks and Transformers, directly leading to computational savings in inference. |
| Tell me why: Visual foundation models as self-explainable classifiers (Read more on arXiv or HuggingFace) |
Christian Lovis, Gianmarco Mengaldo, Mina Bjelogrlic, hturbe |
Visual foundation models (VFMs) can be adapted into self-explainable classifiers through a novel prototypical architecture called ProtoFM. The main research objective is to develop a self-explainable model (SEM) leveraging VFMs that achieves competitive classification performance and improved interpretability. The methodology involves training a lightweight head (approximately 1 million parameters) on top of frozen VFMs, using a student-teacher approach and specialized training objectives, including assignment, alignment, contrastive, sparsity, and classification losses. The ProtoFM architecture achieved a mean explainability score (mX) of 0.92 on the FunnyBirds framework, outperforming existing prototypical models. AI practitioners can leverage frozen VFMs to create efficient and interpretable classifiers, improving transparency and trust, particularly in critical applications. |
| Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids (Read more on arXiv or HuggingFace) |
Yuke Zhu, Linxi Fan, Kartik Sachdev, Toru Lin, jitendra1995 |
This paper presents a sim-to-real reinforcement learning recipe for vision-based dexterous manipulation tasks on humanoid robots. The main research objective is to identify and address the key challenges in applying sim-to-real reinforcement learning to solve contact-rich dexterous manipulation tasks on humanoids. The key methodology includes an automated real-to-sim tuning module, a generalized reward design scheme, a divide-and-conquer distillation process, and a mixture of sparse and dense object representations. The primary results include a 62.3% success rate on the grasp-and-reach task, 80% on the box lift task, and 52.5% on bimanual handover, demonstrating generalization and robustness against force perturbations; also shown is the correlation that lower MSE measured by autotune module and higher sim-to-real transfer success rate. AI practitioners can utilize the proposed techniques to train humanoid robots for dexterous manipulation, achieving robust generalization and high performance without human demonstrations. |
| LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation (Read more on arXiv or HuggingFace) |
kasikci, kojimano, jungok, kamahori |
LITEASR is a compression scheme for ASR encoders that maintains transcription accuracy while reducing computational costs. The main research objective is to reduce the computational intensity of ASR encoders, which are a deployment bottleneck. The key methodology leverages low-rank properties in intermediate activations by applying PCA and optimizing self-attention in a reduced dimension, implemented using a specialized GPU kernel. Applying LITEASR to Whisper large-v3 reduces encoder size by over 50%, matching Whisper medium’s size with better transcription accuracy. AI practitioners can deploy more efficient ASR systems by leveraging the compressed, and Pareto-optimal, models. |
| HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models (Read more on arXiv or HuggingFace) |
Fuzheng Zhang, Yuanxing Zhang, Jingyun Hua, Xiao Wang, lwher1996 |
This paper introduces HAIC, a two-stage data annotation pipeline and two datasets, to improve human action understanding and generation in multi-modal large language models (MLLMs). The main research objective is to address the lack of high-quality data for training MLLMs on videos involving human actions, especially multi-person interactions. The methodology involves a two-stage data annotation pipeline: accumulating videos with clear human actions, and annotating videos with a standardized caption format detailing individual attributes, actions, and interactions. Training with the curated HAICTrain dataset improves human action understanding, as evidenced by a 2.1% accuracy improvement on the HAICBench benchmark compared to the baseline LLaVA-Video-7B model. AI practitioners can use the released datasets and annotation pipeline to enhance MLLMs’ performance in tasks requiring fine-grained understanding of human actions and interactions in videos. |
Papers for 2025-02-28
| Title |
Authors |
Summary |
| Self-rewarding correction for mathematical reasoning (Read more on arXiv or HuggingFace) |
Nan Jiang, Chenlu Ye, Hanning Zhang, Wei Xiong, Lichang-Chen |
This paper introduces a self-rewarding reasoning framework for large language models (LLMs) that enables autonomous error detection and correction in mathematical reasoning without external feedback. The main research question is whether LLMs can simultaneously generate reasoning steps, evaluate their correctness, and revise their outputs during inference without external reward models. The key methodology involves a two-staged training approach using self-generated data: sequential rejection sampling to create training trajectories, followed by reinforcement learning with rule-based signals. Primary results show that on the MATH500 benchmark, the self-rewarding IFT + PPO model achieves a final accuracy of 80.2%, outperforming intrinsic self-correction and comparable to systems using external reward models. For AI practitioners, this framework offers a way to improve LLM reasoning accuracy and reduce computational overhead by integrating generation and evaluation within a single model, streamlining deployment for mathematical reasoning tasks. |
| MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jiayuan Zhu, Fenglin Liu, Jiazhen Pan, morson, che111 |
MedVLM-R1 is a medical vision-language model that uses reinforcement learning to generate explicit reasoning alongside answers for radiology visual question answering. The main research objective is to develop a medical VLM that generates natural language reasoning to improve transparency and trustworthiness, without relying on supervised fine-tuning (SFT). The key methodology is a reinforcement learning framework, specifically Group Relative Policy Optimization (GRPO), that incentivizes the model to discover human-interpretable reasoning paths without using reasoning references. The model, trained on 600 visual question answering samples, boosts accuracy from 55.11% to 78.22% across MRI, CT, and X-ray benchmarks, outperforming larger models. For AI practitioners, this implies that training smaller, specialized models with reinforcement learning can achieve superior, robust, and transparent generalization in the medical domain relative to supervised fine-tuning approaches. |
| R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts (Read more on arXiv or HuggingFace) |
Ziyue Li, zhoutianyi, Lzy01241010 |
R2-T2 introduces a test-time re-routing method for multimodal Mixture-of-Experts (MoE) models that improves performance without retraining. The main research objective is to optimize the routing weights of a multimodal MoE model during inference to improve performance on challenging or out-of-distribution samples. The key methodology is “Re-Routing in Test-Time (R2-T2),” which locally optimizes routing weights by moving them toward those of correctly predicted neighbor samples, using strategies like Neighborhood Gradient Descent (NGD), kernel regression, and mode finding. Applying R2-T2 with NGD to MoAI-7B improved MMBench accuracy by 6.9%, TextVQA accuracy by 6.8%, and achieved a 66.1-point increase on MME-P. AI practitioners can use R2-T2 to enhance the performance and generalization of multimodal MoE models on diverse tasks in test-time, without costly retraining or modification of model parameters. |
| LongRoPE2: Near-Lossless LLM Context Window Scaling (Read more on arXiv or HuggingFace) |
Gilsinia Lopez, Gaokai Zhang, Li Lyna Zhang, Ning Shang, OldKingMeister |
LongRoPE2 extends LLMs’ effective context window while preserving short-context performance through RoPE rescaling and mixed context window training. The main research objective is to address the out-of-distribution (OOD) issues in rotary positional embeddings (RoPE) and the performance degradation on short-context tasks when extending the context window of pre-trained large language models (LLMs). The key methodology involves an evolutionary search for optimal RoPE rescaling factors guided by “needle-driven” perplexity, combined with a mixed context window training approach that uses both original and rescaled RoPE. Primary results show that LongRoPE2 extends LLaMA3-8B to achieve a 128K effective context length while retaining over 98.5% of short-context performance, using only 10B training tokens. Principal implication is that AI practitioners can extend LLM context windows to 128K with near-lossless performance on both long and original context window, significantly reducing the data, and training costs compare to prior methods. |
| FINEREASON: Evaluating and Improving LLMs’ Deliberate Reasoning through Reflective Puzzle Solving (Read more on arXiv or HuggingFace) |
Chaoqun Liu, Hou Pong Chan, Hao Zhang, Weiwen Xu, Guizhen Chen |
FINEREASON introduces a logic-puzzle benchmark to evaluate and improve LLMs’ deliberate reasoning through state checking and transition tasks. The main research objective is to assess and enhance LLMs’ ability to reflect and rectify mistakes during multi-step reasoning processes, going beyond final-answer accuracy. The key methodology involves decomposing logic puzzles into atomic steps and evaluating models on two tasks: state checking (assessing if a state can lead to a solution) and state transition (determining the next valid move). Primary results show that models trained with state checking and transition data demonstrated gains in math reasoning by up to 5.1% on GSM8K, when starting from the DeepSeek-R1-Distill-Qwen-7B model, the accuracy increased from 82.3% to 87.4%. The principal implication for AI practitioners is that training LLMs with structured, puzzle-based data focusing on intermediate reasoning steps can significantly improve their performance on general mathematical reasoning tasks. |
| CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale (Read more on arXiv or HuggingFace) |
Kaiyue Qiu, Zhaoyang Chu, Chenlong Wang, yxy0807, zx10086 |
CODESYNC introduces a data engine and benchmark to assess large language models’ (LLMs) ability to adapt to evolving Python library APIs. The main research question is: Can LLMs be effectively and efficiently updated to handle real-time API modifications? CODESYNC systematically identifies API updates, retrieves relevant code instances from GitHub, and uses an LLM to synthesize contrastive code for legacy/updated API versions, then builds a benchmark,CODESYNCBENCH. Evaluation of 14 LLMs shows they struggle with API updates even with knowledge updating methods, e.g. a maximum BLEU score of 31.59 on the code completion task across five models with SFT. The principal implication is that AI practitioners need to develop and employ techniques to improve LLMs’ ability to synchronize with evolving code, as static pre-training datasets limit handling of real-time API updates. |
| Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance (Read more on arXiv or HuggingFace) |
Zhixu Li, Pu Zhao, Lu Wang, Chenghua Huang, keanudicap |
DVPO decouples value and policy optimization in RLHF to improve training efficiency and stability for large language models. The main research objective is to address the computational complexity and instability of traditional PPO-based RLHF caused by joint actor-critic training. The key methodology is Decoupled Value Policy Optimization (DVPO), which pre-trains a Global Value Model (GVM) on policy trajectories and uses it as a fixed guide for policy optimization via a standard RL objective. Primary results show that DVPO reduces GPU memory usage by 40% and training time by 35% compared to conventional RLHF, while achieving comparable performance to state-of-the-art PPO. The principal implication is that AI practitioners can achieve more efficient and stable RLHF training by decoupling value estimation from policy updates, simplifying the alignment of LLMs with human preferences. |
| UniTok: A Unified Tokenizer for Visual Generation and Understanding (Read more on arXiv or HuggingFace) |
Xin Yu, Jihan Yang, Junfeng Wu, Yi Jiang, Chuofan Ma |
UniTok is a unified visual tokenizer designed for both visual generation and understanding tasks, bridging the representation gap between these two domains. The main research objective is to investigate whether reconstruction and contrastive losses truly conflict in unified tokenizer training, and to identify any underlying bottlenecks. The key methodology is multi-codebook quantization, which divides visual tokens into chunks and discretizes each with independent sub-codebooks, alongside attention factorization. UniTok achieves a remarkable rFID of 0.38 and a zero-shot accuracy of 78.6% on ImageNet. The principal implication for AI practitioners is that a unified visual tokenizer, enhanced with multi-codebook quantization, can match or surpass domain-specific tokenizers, enabling more efficient and integrated multimodal model development. |
| FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute (Read more on arXiv or HuggingFace) |
Markos Georgopoulos, Jonas Kohler, Yeongmin Kim, Gregor Bachmann, Sotiris Anagnostidis |
FlexiDiT enables Diffusion Transformers (DiTs) to generate high-quality images with reduced computational cost by dynamically adjusting the compute budget per denoising step. The main research objective is to overcome the fixed and large compute requirements of standard DiTs during inference by revisiting the static compute allocation paradigm. The key methodology is converting pre-trained DiT models into flexible ones (FlexiDiTs) that can process inputs at varying compute budgets by dynamically adjusting patch size during the denoising process, and using different LoRAs for each sequence. The primary result is that FlexiDiT models can reduce FLOPs by more than 40% compared to static counterparts for class-conditioned and text-conditioned image generation, without any drop in quality. AI practitioners can deploy more computationally efficient diffusion models by adopting FlexiDiT, enabling substantial savings in computational resources without compromising the quality of generated outputs, especially for high-resolution image and video generation. |
| Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think (Read more on arXiv or HuggingFace) |
Haozhe Zhao, Weichu Xie, Wenhao Chai, Shuai Bai, Liang Chen |
DREAM ENGINE enables arbitrary text-image interleaved control for image generation by aligning large multimodal models (LMMs) with diffusion models. The research objective is to develop a framework that can generate images based on complex instructions interweaving text and visual elements from multiple images. The key methodology involves replacing the text encoders of a diffusion model (SD3.5) with an LMM (QwenVL) and a two-stage training paradigm: joint text-image alignment and multimodal interleaved instruction tuning. The primary results show that DREAM ENGINE achieves a 0.69 overall score on the GenEval benchmark, matching state-of-the-art text-to-image models. For AI practitioners, the principal implication is that LMMs can be directly integrated into diffusion models to enable advanced text-image control, simplifying the creation of complex, multi-image-influenced generation systems. |
| NeoBERT: A Next-Generation BERT (Read more on arXiv or HuggingFace) |
Sarath Chandar, Mariam El Mezouar, Quentin Fournier, Lola Le Breton |
NeoBERT, a new BERT-like encoder model, integrates architectural, data, and pre-training advancements to improve bidirectional representation learning. The primary objective is to create a next-generation BERT model that outperforms existing encoders by leveraging modern advancements in language model design. The key methodology involves pre-training on the RefinedWeb dataset with modifications like RoPE, SwiGLU, RMSNorm, a 20% masking rate, and a two-stage sequence length increase (1,024 to 4,096 tokens). NeoBERT achieves an 89.0 average score on the GLUE benchmark and 51.3 on the MTEB benchmark after contrastive fine-tuning, outperforming all similarly-sized and even larger, models on MTEB. AI practitioners can adopt NeoBERT as a plug-and-play replacement for existing base encoders to obtain better performance in downstream NLP tasks that depend on their embeddins, notably for retrieval-augmented generation and toxicity classification, without needing architectural modifications. |
| Mobius: Text to Seamless Looping Video Generation via Latent Shift (Read more on arXiv or HuggingFace) |
Xiaodong Cun, Yong Zhang, Bo Liu, Jianfei Yuan, Xiuli Bi |
Mobius is a training-free method to generate seamless looping videos from text descriptions using pre-trained video diffusion models. The main research objective is to develop a method for generating seamless looping videos directly from text prompts, without requiring user annotations or additional training. The key methodology involves constructing a latent cycle and performing multi-frame latent denoising by iteratively shifting the first-frame latent towards the end in each step, while also using a frame-invariant latent decoding method. Primary results show that the proposed method achieves an MSE of 25.43 between the first and last frame, FVD of 40.78, a CLIP score of 32.24, and a Motion Smoothness score of 0.9850. For AI practitioners, this method provides a way to directly repurpose pre-trained text-to-video diffusion models for generating seamless looping videos, without the need for large scale training or annotated dataset. |
| SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning (Read more on arXiv or HuggingFace) |
Yanzhen Zou, Xiangxin Meng, Pengfei Gao, Chao Peng, mizersy |
SoRFT is a novel training approach that enhances large language models’ (LLMs) issue-resolving capabilities through subtask decomposition and reinforced fine-tuning. The main research objective is to improve the performance and generalization of open-source LLMs on software issue resolution tasks, addressing limitations of existing methods. The key methodology involves decomposing issue resolving into subtasks (file/function/line localization, code edit generation) and using rejection-sampled supervised fine-tuning followed by rule-based proximal policy optimization (PPO) with ground-truth-based rewards. The primary result is that SoRFT-Qwen-7B achieves 21.4% resolution rate on SWE-Bench Verified, outperforming other open-source models of similar size. For AI practitioners, SoRFT offers a cost-effective way to leverage open-source development resources and substantially boost the performance of open-source LLMs in automated issue resolution. |
| Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting (Read more on arXiv or HuggingFace) |
Song-Chun Zhu, Junfeng Ni, Ruijie Lu, Baoxiong Jia, Yu Liu |
ArtGS introduces a method for reconstructing and modeling complex articulated objects using 3D Gaussian Splatting. The main research objective is to effectively integrate information across different object states to improve part-mesh reconstruction and articulation parameter estimation, especially for multi-part articulated objects. The key methodology involves using canonical Gaussians with coarse-to-fine initialization and updates, alongside a skinning-inspired part dynamics modeling module. Primary results show that on the PARIS dataset, ArtGS achieves a mean angular error (Axis Ang.) of 0.01 degrees and a mean Chamfer Distance for movable parts (CD-m) of 0.03, outperforming existing methods. For AI practitioners, this implies a more efficient and accurate approach to creating digital twins of articulated objects, facilitating applications in robotics and virtual environments. |
| R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning (Read more on arXiv or HuggingFace) |
Hongyong Zeng, Yuanchang Luo, Shimin Tao, Yilun Liu, boommmmm |
R1-T1 is a novel framework that enhances machine translation (MT) in large language models (LLMs) through reinforcement learning (RL) with human-aligned chain-of-thoughts (CoTs). The main research objective is to improve the adaptability of LLMs to diverse translation scenarios by incorporating inference-time reasoning into general MT, going beyond specific sub-tasks. The key methodology involves formalizing six expert-curated CoT templates, reflecting human translation strategies, and using RL with KL-constrained rewards for self-evolving CoT discovery and anti-forgetting adaptation. Primary results demonstrate steady translation performance improvement across 21 languages and 80 translation directions on the Flores-101 test set, with a COMETScore of 0.626 on trained languages using RL, surpassing supervised fine-tuning (SFT) and other baselines. Principal implication for AI practioners: It provides a method for using RL to adapt LLMs to new machine translation tasks without relying on the SFT data and avoiding the Catastrophic Forgetting issue. |
Papers for 2025-02-27
| Title |
Authors |
Summary |
| Kanana: Compute-efficient Bilingual Language Models (Read more on arXiv or HuggingFace) |
seopbo, Doohae, daniel-rl2, jiyeonham, bzantium |
Kanana is a series of bilingual language models demonstrating strong performance in Korean and competitive performance in English at a significantly lower computational cost than comparable state-of-the-art models. The main research objective was to develop compute-efficient bilingual language models that maintain strong performance in both Korean and English. The key methodologies employed include high-quality data filtering, staged pre-training, depth up-scaling, pruning, and distillation, combined with supervised fine-tuning and preference optimization for instruction tuning. Primary results show that the Kanana Flag 32.5B model outperforms Llama 3.1 70B on MMLU and KMMLU, while using substantially fewer computational resources, costing similiar to Gemma 2 9B. AI practitioners can leverage Kanana’s training techniques such as staged pre-training and depth-up scaling to build high-performing, resource-efficient language models, especially for languages with limited data availability. |
| GHOST 2.0: generative high-fidelity one shot transfer of heads (Read more on arXiv or HuggingFace) |
Andrey Kuznetsov, Denis Dimitrov, Pavel Paramonov, Alexander Groshev, nastasia-y |
GHOST 2.0 is a two-module framework for high-fidelity one-shot head swapping, addressing limitations in existing face-swapping and head-reenactment methods. The main research objective is to develop a system that can realistically swap entire heads between source and target images, preserving identity, pose, and expression while seamlessly blending the result. The key methodology involves an “Aligner” module for head reenactment and a “Blender” module for integrating the reenacted head into the target background, using StyleGAN-based architecture and correlation learning. Primary results show that at 512x512 resolution in cross-reenactment, GHOST 2.0 achieves a CSIM score of 0.628 and a FID score of 29.57, outperforming one of the baselines (StyleHEAT) and indicating better performace than another baseline (HeSer) at identity preservation. AI practitioners can use GHOST 2.0 to improve the realism and robustness of head-swapping applications, particularly in scenarios with significant variations in head pose, hairstyle, and background. |
| TheoremExplainAgent: Towards Multimodal Explanations for LLM Theorem Understanding (Read more on arXiv or HuggingFace) |
Jonathan Leung, AlvinYuVotee, KrishKrosh, chongcht, vinesmsuic |
TheoremExplainAgent, a novel agentic system, generates multimodal theorem explanation videos, and a new benchmark, TheoremExplainBench, evaluates them. The main research objective is to assess if AI systems can effectively generate multimodal theorem explanations. The key methodology involves a two-agent pipeline (planner and coding agent) using Manim to create videos, and a benchmark of 240 theorems across STEM, evaluated across five dimensions. The o3-mini agent achieved a 93.8% success rate and an overall score of 0.77, but visual element layout exhibited minor issues. AI practitioners can leverage this agentic approach for enhanced theorem understanding, though refinement is needed in visual structuring and consistency of generated video outputs. |
| Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? (Read more on arXiv or HuggingFace) |
Weixun Wang, Jiaheng Liu, Shilong Li, Yancheng He, zhangysk |
DeltaBench, a new benchmark, evaluates large language models’ (LLMs) ability to detect errors in long chain-of-thought (CoT) reasoning. The main research objective is to assess the quality of long CoTs generated by o1-like models and to measure the critique abilities of existing LLMs, process reward models (PRMs) and critic models on these CoTs. The key methodology involves creating DeltaBench, a dataset of long CoTs with fine-grained error annotations, and evaluating various LLMs, including PRMs and critic models, on their ability to identify these errors. Primary results show that even the top-performing model (GPT-4-turbo-128k) achieved a low F1-score of only 40.8% in error detection, and that o1-like models do not show any advantage over non-o1-like models on critique abilities. Principal implication for AI practitioners is that current LLMs, including PRMs, have limited ability to identify errors in long CoT reasoning, highlighting a need for significant improvements in critique capabilities for robust AI system development. |
| Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems (Read more on arXiv or HuggingFace) |
Bin Xu, Zijun Yao, Xiaozhi Wang, Yunjia Qi, Hao Peng |
This paper proposes a new reward modeling approach, “agentic reward modeling,” that combines human preferences with verifiable correctness signals for more reliable reward systems in large language models (LLMs). The main research objective is to develop a reward system that mitigates the limitations of existing reward models, which primarily focus on subjective human preferences and often neglect verifiable correctness. The key methodology involves implementing a reward agent, REWARDAGENT, that integrates human preference rewards with two verifiable signals: factuality (assessed via pairwise comparison and evidence verification) and instruction-following (verified through constraint parsing and Python code execution). The primary results show that REWARDAGENT significantly outperforms existing reward models on benchmarks like RM-Bench, JudgeBench, and a newly constructed IFBench, achieving an overall score of 72.5% in one configuration. The principal implication for AI practitioners is that integrating verifiable correctness signals with human preference feedback can lead to more reliable and robust reward models, improving LLM performance in downstream tasks and alignment with intended behavior, particularly during the inference and training phases. |
| Language Models’ Factuality Depends on the Language of Inquiry (Read more on arXiv or HuggingFace) |
Hamid Palangi, Kumar Ayush, Kumar Tanmay, ayush1801, AggarwalTushar |
Language models (LMs) exhibit inconsistent factual recall across different languages, failing to transfer knowledge even when possessing it in one language. The main research question is whether multilingual LMs truly internalize and transfer factual knowledge across languages or encode isolated linguistic silos. The key methodology involves creating a benchmark of 10,000 country-related facts across 13 languages and proposing metrics (Factual Recall Score, Knowledge Transferability Score, Cross-Lingual Factual Knowledge Transferability Score) to quantify factual recall and knowledge transferability. A primary result is that Llama-3-70B achieved the highest X-FaKT score of 0.848, demonstrating superior balanced performance in both factual recall and knowledge transfer. The principal implication is that AI practitioners must recognize language-specific factual reliability in multilingual LMs and leverage the most trustworthy information across languages, moving beyond the assumption of consistent cross-lingual knowledge access. |
| Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creation (Read more on arXiv or HuggingFace) |
Matthias Bethge, Jonas Geiping, Ponnurangam Kumaraguru, Shashwat Goel, Shiven Sinha |
Language models (LMs) are evaluated on their ability to generate counterexamples that falsify incorrect algorithmic solutions, introducing a new benchmark called REFUTE. The main research question is: Can LMs create counterexamples for incorrect solutions to algorithmic problems? The key methodology involves sourcing incorrect submissions from programming competitions, filtering them for non-trivial errors, and prompting LMs to generate inputs that cause these solutions to fail, validated through code execution. The primary result is that the best reasoning agents, including OpenAI 03-mini (high), can only create counterexamples for less than 9% of incorrect solutions in REFUTE, despite having a much higher success rate at solving those same problems. The principal implication for AI practitioners is that verification, including falsification of subtly incorrect solutions, is significantly harder for current LMs than generating correct solutions, highlighting a limitation in capabilities relevant for self-improvement and reliable reasoning. |
| Towards an AI co-scientist (Read more on arXiv or HuggingFace) |
Anil Palepu, Tao Tu, Alexander Daryin, Wei-Hung Weng, Juraj Gottweis |
Here’s a summary of the paper, strictly adhering to your guidelines: The paper introduces an AI co-scientist, a multi-agent system built on Gemini 2.0, designed to assist in scientific discovery by generating and evaluating novel research hypotheses. The main research objective is to develop an AI system capable of formulating demonstrably novel research hypotheses and proposals, building upon existing evidence and aligned with scientist-provided goals. The key methodology involves a multi-agent architecture with an asynchronous task execution framework, utilizing a generate, debate, and evolve approach with specialized agents for hypothesis generation, refinement, and ranking via simulated scientific debates and tournaments. The system demonstrates, across 203 diverse research goals, improved hypothesis quality (measured by an internal Elo rating system) as a function of increased test-time compute, and hypotheses for acute myeloid leukemia were validated to show tumor inhibition in vitro at clinically applicable concentrations. AI practitioners can leverage the multi-agent architecture and test-time compute scaling paradigm presented to build systems capable of complex reasoning and iterative improvement, although specific external validation metrics remain limited within the paper. |
| VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Model (Read more on arXiv or HuggingFace) |
Lingrui Mei, Lu Wang, Jiani Zheng, vyokky, keanudicap |
VEM decouples value estimation from policy optimization for training GUI agents, enabling environment-free reinforcement learning. The main research objective is to develop an environment-free RL framework that can effectively train GUI agents without costly real-world interactions. The key methodology involves pretraining a Value Environment Model (VEM) to predict state-action values from offline data and then using this frozen VEM to guide policy exploration. The method achieves 28.0% offline task success rate on the General domain of the Android-in-the-Wild benchmark, surpassing environment-free baselines by 12-28%. AI practitioners can leverage this approach to train GUI agents with greater sample efficiency and stability, bypassing the need for direct environment interactions. |
| Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance (Read more on arXiv or HuggingFace) |
Polydoros Giannouris, Efstathia Soufleri, Triantafillos Papadopoulos, Xueqing Peng, jiminHuang |
The paper introduces Plutus-ben, a Greek financial benchmark, and Plutus-8B, a Greek financial LLM, to address the lack of resources for Greek financial NLP. The main research question is: How do current language models perform on core Greek financial tasks, and how can fine-tuning on Greek financial data enhance performance? Key methodology involved creating Plutus-ben, comprising five financial NLP tasks (numeric and textual NER, QA, abstractive summarization, topic classification), and fine-tuning Llama-Krikri-8B with Greek domain-specific data to create Plutus-8B, evaluating 22 LLMs. The primary result is that Plutus-8B achieved the best performance on Plutus-ben, surpassing GPT-4 by 15.38% and outperforming all baseline models in the evaluation. Principal implication for AI practitioners is that fine-tuning on language-specific and domain-specific data is crucial for LLM performance in low-resource languages like Greek, significantly improving performance in tasks like financial numeric reasoning. |
| Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator (Read more on arXiv or HuggingFace) |
Ying Cui, Ruibo Li, Hongji Li, Dongyan Guo, Xiankang He |
This paper introduces a new distillation framework for improving monocular depth estimation (MDE) using unlabeled data. The main research objective is to enhance zero-shot MDE by addressing the limitations of existing depth normalization strategies in pseudo-label distillation. The key methodology involves Cross-Context Distillation, integrating global and local depth cues, and a multi-teacher distillation framework using diverse depth estimation models. The primary result shows that the proposed method outperforms state-of-the-art methods on benchmark datasets; for instance, on the DIODE dataset, the AbsRel improves by 14.1% using the Local-Global and Shared-Context Distillation strategies. For AI practitioners, this method provides an effective way to train more robust and accurate MDE models by leveraging unlabeled data and combining the strengths of multiple teacher models, especially improving generalization in varied scenarios. |
| Project Alexandria: Towards Freeing Scientific Knowledge from Copyright Burdens via LLMs (Read more on arXiv or HuggingFace) |
Andreas Hochlehnert, Tawsif Ahmed, Ameya Prabhu, Gollam Rabby, Christoph Schuhmann |
This paper proposes converting copyrighted scientific texts into structured “Knowledge Units” using LLMs to make factual information freely accessible while respecting copyright. The main research question is whether converting scientific texts into Knowledge Units preserves factual information and adheres to copyright laws. The key methodology involves using LLMs to extract entities, attributes, and relationships from paragraphs of scientific papers into structured data, and evaluating the legal defensibility and information retention via question-answering experiments. Primary results show that language models answering multiple-choice questions using Knowledge Units achieved nearly the same accuracy (within 3-5% variance) as when using original texts across several scientific domains. AI practitioners can utilize this framework to build and use datasets containing facts from copyrighted scientific text, potentially democratizing access to scholarly knowledge without infringing on the original expression. |
| AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement (Read more on arXiv or HuggingFace) |
Xijie Huang, Junxiao Yang, Leqi Lei, Zhexin Zhang, LLLeo612 |
AISafetyLab is a unified framework and toolkit for AI safety that integrates attack, defense, and evaluation methodologies. The main objective is to provide a standardized platform to evaluate and improve AI safety by addressing the lack of comprehensive tools and inconsistent experimental setups. The methodology involves implementing 13 attack methods (including black-box, gray-box, and white-box), 16 defense mechanisms (both inference-time and training-time), and 7 evaluation scorers, alongside auxiliary modules for model interaction, data management, utilities, and logging. In evaluations using Vicuna-7B-v1.5, AutoDAN achieved an average attack success rate of 56.4% across various defenses, while some other methods had varying performance depending on the defense used. For AI practitioners, AISafetyLab provides a flexible, extensible platform with comprehensive method coverage for systematically assessing and enhancing the robustness of AI models against adversarial attacks. |
| BIG-Bench Extra Hard (Read more on arXiv or HuggingFace) |
Chrysovalantis Anastasiou, John Palowitch, Hritik Bansal, Mehran Kazemi, baharefatemi |
BIG-Bench Extra Hard (BBEH) is a new benchmark to evaluate the general reasoning capabilities of large language models (LLMs). The main research objective is to address the saturation of existing LLM reasoning benchmarks, particularly BIG-Bench Hard (BBH), by creating a more challenging and diverse set of tasks. The methodology involves replacing each of the 23 tasks in BBH with a novel, more difficult task that probes similar reasoning capabilities, using a semi-adversarial approach with two reference models to ensure sufficient difficulty. The primary result is that the best general-purpose model achieved a harmonic mean accuracy of 9.8% on BBEH, while the best reasoning-specialized model achieved 44.8%, indicating significant room for improvement. AI practitioners should use BBEH to evaluate LLMs for robust general reasoning, revealing current limitations and driving improvements instead of using other benchmarks where LLMs have reached ceiling performance. |
| CritiQ: Mining Data Quality Criteria from Human Preferences (Read more on arXiv or HuggingFace) |
Zhiheng Xi, Tianyi Liang, Qipeng Guo, Kai Lv, KYLN24 |
CritiQ is a novel data selection method that automatically mines data quality criteria from human preferences and performs efficient data selection. The main research objective is to develop a method for automatically extracting data quality criteria from human preferences with minimal human annotation effort. The key methodology, CritiQ Flow, employs a manager agent to evolve quality criteria and worker agents to make pairwise judgments based on a knowledge base and a reflection process. Accuracies on human-annotated test sets reach 89.33% for code, 84.57% for math, and 88.06% for logic, outperforming baselines such as TextGrad and single-criterion methods. AI practitioners can use CritiQ to automatically derive data quality criteria and select high-quality subsets, improving model performance on downstream tasks with reduced reliance on manually designed heuristics or extensive human annotation. |
| MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectra (Read more on arXiv or HuggingFace) |
Qiang Liu, Deli Zhao, Yu Rong, Shaozhen Liu, AzureLeon1 |
MolSpectra enhances pre-training of 3D molecular representations by incorporating multi-modal energy spectra. The main research objective is to establish the relationship between 3D molecular structures and energy states using spectral data to improve molecular representation learning. The key methodology involves a multi-spectrum encoder, SpecFormer, trained with masked patch reconstruction, and a contrastive objective aligning 3D and spectral representations. Pre-training with MolSpectra achieved state-of-the-art performance on the QM9 dataset, achieving a mean absolute error (MAE) of 0.011 D on the dipole moment (μ) prediction, outperforming the baseline Coord method in 10 out of 12 properties. For AI practitioners, MolSpectra provides a pre-training framework that leverages molecular spectra to learn more informative 3D molecular representations, enhancing performance on downstream tasks like property prediction. |
| PosterSum: A Multimodal Benchmark for Scientific Poster Summarization (Read more on arXiv or HuggingFace) |
Frank Keller, Pasquale Minervini, rohitsaxena |
POSTERSUM, a new benchmark, evaluates multimodal models on summarizing scientific posters into research paper abstracts, revealing limitations in current models and introducing a hierarchical approach for improvement. Main research question or objective: How effectively can Multimodal Large Language Models (MLLMs) understand and summarize the complex, visually-rich content of scientific posters into concise textual abstracts, and can a hierarchical approach improve this performance? Key methodology used: The authors created a new dataset, POSTERSUM, consisting of 16,305 scientific posters paired with their corresponding abstracts. They benchmarked state-of-the-art MLLMs (including GPT-4o, Claude-3.5 Sonnet, Gemini 2.0, and various open-source models) on this dataset using metrics like ROUGE, SacreBLEU, METEOR, and BERTScore. They then proposed “SEGMENT & SUMMARIZE,” a hierarchical approach involving segmentation of the poster into coherent regions, localized summarization of each region, and global summarization to combine the localized summaries. Primary results: State-of-the-art MLLMs struggle to accurately summarize scientific posters. The best-performing closed-source model, GPT-4o, achieved a ROUGE-L score of only 22.30. The proposed SEGMENT & SUMMARIZE method significantly outperformed all other models, including closed-source MLLMs, achieving a ROUGE-L score of 24.18. Principal implication for AI practitioners: Current MLLMs, while strong on various tasks, have significant limitations when handling the complex multimodal information presented in scientific posters. The POSTERSUM dataset provides a valuable benchmark for advancing multimodal understanding, and the “SEGMENT & SUMMARIZE” approach demonstrates a promising direction for improving performance by incorporating a divide-and-conquer strategy, handling the complexity inherent in poster summarization. AI/ML/Software Engineers and Data Scientist working with scientific documents should prioritize models and architectures that are capable of understanding a variety of modalities and their combinations. |
Papers for 2025-02-26
| Title |
Authors |
Summary |
| OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference (Read more on arXiv or HuggingFace) |
Jiaqiwang, Weiyun1025, UniverseCA, ChrisDing1105, PhoenixZ |
OmniAlign-V introduces a new dataset and benchmark to improve the alignment of multi-modal large language models (MLLMs) with human preferences. The main research objective is to address the gap in human preference alignment observed in existing open-source MLLMs, despite their strong performance on foundational capability benchmarks. The key methodology involves constructing OmniAlign-V, a dataset of ~200K high-quality training samples with diverse images and complex question-answer pairs, and MM-AlignBench, a human-annotated benchmark for evaluating MLLM alignment. Finetuning MLLMs with OmniAlign-V via Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO) improved the win rate against Qwen2VL-72B on MM-AlignBench, achieving a 72.6 win rate. The principal implication is that AI practitioners should utilize curated, human-aligned multi-modal datasets like OmniAlign-V during SFT and DPO to significantly enhance the human preference alignment of MLLMs while maintaining or enhancing fundamental capabilities. |
| SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference (Read more on arXiv or HuggingFace) |
Haofeng Huang, surfingtomchen, hxi0408, Xiang-cd, jt-zhang |
SpargeAttn is a universal sparse and quantized attention mechanism designed to accelerate inference in various AI models. The paper’s main objective is to design a training-free sparse attention operator that accelerates all models without metric loss. The key methodology involves a two-stage online filter that predicts sparse blocks in the attention map using selective token compression and a sparse warp online softmax, integrated with 8-bit quantization. SpargeAttn achieved a 1.83x speedup on Mochi on an L40 GPU without loss of video quality and is 2.5x to 5x faster than existing dense/sparse attention models. AI practitioners can use SpargeAttn to significantly accelerate the inference of diverse models, including language, image, and video generation, without sacrificing end-to-end performance metrics. |
| KV-Edit: Training-Free Image Editing for Precise Background Preservation (Read more on arXiv or HuggingFace) |
Yansong Tang, jewelshaw, shiyi0408, xilluill |
KV-Edit is a training-free image editing method that achieves precise background preservation by utilizing KV cache in diffusion models. The main research objective is to address the challenge of maintaining background consistency during image editing tasks while generating content aligned with modified text prompts. The key methodology involves caching and reusing key-value pairs of background tokens in Diffusion Transformers (DiTs) during the inversion and denoising processes, and optional mask-guided inversion and reinitialization strategies. Primary results show that KV-Edit achieves a PSNR of 35.87 in masked region preservation, outperforming existing methods. For AI practitioners, this method provides a way to perform image editing with perfect background preservation, without additional training or complex mechanisms, thereby facilitating more practical AI image editing applications. |
| ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation (Read more on arXiv or HuggingFace) |
JianminBao, DongChen06, 131131yhx, 2JZ, yifanpu001 |
This paper introduces the Anonymous Region Transformer (ART) for generating variable multi-layer transparent images from a global text prompt and an anonymous region layout. The main research objective is to develop a method for generating high-quality, multi-layer transparent images that overcomes the limitations of existing methods requiring detailed semantic layouts. The key methodology involves using an anonymous region layout, a layer-wise region crop mechanism, and a multi-layer transparent image autoencoder. The method achieves a speed improvement of over 12 times compared to the full attention approach, and user studies show it outperforms existing methods (LayerDiffuse and COLE) in multiple aspects. The principal implication is that AI practitioners can generate multi-layer images more efficiently and with greater scalability, allowing for more precise control in interactive content creation and editing of individual elements within generative models. |
| SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution (Read more on arXiv or HuggingFace) |
RishabhSingh021, gsynnaeve, lingming, JadeCopet, yuxiang630 |
SWE-RL is a reinforcement learning approach that enhances LLM reasoning for software engineering tasks using open-source software evolution data. The main research objective is to improve LLMs’ performance on real-world software engineering tasks, specifically issue resolution, using reinforcement learning. The key methodology is training LLMs on GitHub pull request data with a rule-based reward function based on the similarity between predicted and oracle code patches, optimized via Group Relative Policy Optimization (GRPO). The primary result is that Llama3-SWE-RL-70B achieves a 41.0% solve rate on the SWE-bench Verified dataset. The principal implication for AI practitioners is that reinforcement learning on software evolution data can significantly enhance LLM reasoning capabilities for software engineering and also improve performance on out-of-domain tasks. |
| Unveiling Downstream Performance Scaling of LLMs: A Clustering-Based Perspective (Read more on arXiv or HuggingFace) |
Chenggang Li, Xiao Li, shenke18, Lucky2022, JerryXu98 |
The paper introduces a Clustering-On-Difficulty (COD) framework to predict downstream task performance of Large Language Models (LLMs). The main research objective is to accurately predict LLM performance on downstream tasks prior to extensive model training, addressing the challenges of emergent abilities and uneven task difficulty distributions. The key methodology involves clustering tasks based on difficulty features, fitting performance-compute curves on predictable clusters, and mapping these predictions to the full evaluation set. The primary result is that COD achieves a mean absolute prediction error of 1.36% across eight LLM evaluation benchmarks on a 70B-parameter model. The principal implication is that AI practitioners can use COD for efficient resource allocation and monitoring during LLM training, by reliably predicting downstream task performance using smaller models. |
| Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models (Read more on arXiv or HuggingFace) |
Ya Wang, LLIXQ, xunzhou, Taoer, BryceZhuo |
Scale-Distribution Decoupling (SDD) is a novel approach that stabilizes and improves the training of large language models by separating the scale and distribution of weight matrices. The main research objective is to address training instability issues, such as gradient explosion and vanishing gradients, in large language models (LLMs), particularly in Post-Norm Transformer architectures. SDD uses a normalization mechanism to regulate activations and a learnable scaling vector to maintain well-conditioned gradients in fully-connected layers. SDD-1B achieves a training loss of 2.65, outperforming OLMo2-1B (2.70), PostNorm-1B (2.69), and DeepNorm-1B (2.72), also achieving the highest average accuracy of 54.04% across multiple downstream tasks. For AI practitioners, SDD provides a lightweight and compatible solution for stabilizing LLM training, improving convergence, and enabling more efficient large-scale pre-training. |
| K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs (Read more on arXiv or HuggingFace) |
Qibin Hou, Zhen Li, oyzh2005 |
K-LoRA is a training-free method for merging subject and style LoRAs to generate images that preserve both characteristics. The paper’s objective is to develop a method for effectively combining content and style LoRAs without requiring additional training or manual parameter tuning. The key methodology is a Top-K selection process within attention layers that identifies and selects the most representative features from each LoRA for fusion, combined with a scaling factor that prioritizes content or style at different diffusion timesteps. The method achieved a CLIP score of 69.4% and a DINO score of 46.9% for subject similarity, outperforming existing methods. AI practitioners can use K-LoRA to effectively fuse separately trained subject and style LoRAs, enabling efficient customized image generation without retraining, simplifying the process of generating images with specific content and styles. |
| WebGames: Challenging General-Purpose Web-Browsing AI Agents (Read more on arXiv or HuggingFace) |
Fraser, semitable, BiggieW, XanderJC, georgethomas |
WebGames introduces a benchmark suite for evaluating general-purpose web-browsing AI agents. The primary objective is to assess AI limitations in web interactions using 50+ interactive challenges designed to be human-intuitive yet AI-challenging. The methodology involves evaluating vision-language models like GPT-4o, Claude Computer-Use, Gemini-1.5-Pro, and Qwen2-VL in a hermetic, client-side environment, measuring their success against human baselines. The best AI system achieved a 41.2% success rate compared to 95.7% human performance, revealing a substantial capability gap. This highlights the need for improvements in AI’s ability to handle common web interaction patterns, thereby directing future development efforts for web-browsing agents by AI practitioners. |
| Introducing Visual Perception Token into Multimodal Large Language Model (Read more on arXiv or HuggingFace) |
wxcTest, horseee, rp-yu |
This paper introduces Visual Perception Tokens to enhance Multimodal Large Language Models’ (MLLMs) control over visual perception processes. The main research objective is to enable MLLMs to autonomously control their visual perception, such as selecting specific image regions or refining features. The key methodology involves designing two types of Visual Perception Tokens (Region Selection and Vision Re-Encoding) that MLLMs generate and use to trigger additional visual processing steps. Results show that adding Visual Perception Tokens to a 2B parameter model improves its average performance across various VQA tasks by 30.9%, achieving a score of 0.749 compared to 0.572 without the tokens. AI practitioners can utilize these tokens to improve MLLMs’ performance in tasks requiring fine-grained visual understanding and spatial reasoning, by giving models a mechanism to actively control their visual input. |
| The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve? (Read more on arXiv or HuggingFace) |
Peijie Dong, Qian Wang, Xiang Liu, wenxinsiju, coolzhtang |
This paper proposes a “lottery LLM hypothesis” suggesting that smaller, compressed large language models (LLMs) can achieve comparable performance to original LLMs using external tools and reasoning. The main research objective is to identify the essential capabilities that compressed LLMs and key-value (KV) cache compression methods should preserve to maintain performance. The methodology involves a review of recent LLM advancements (retrieval-augmented generation, external tools, multi-step reasoning, computational expressivity) and proposes a recursive multi-step reasoning algorithm (Algorithm 1) for the “lottery LLM”. Primary results include showing that retrieval augmented generation can provide a compressed model equivalent performance. For instance Table 2 shows that Llama-3-Ins8B with RAG achieves a 59.8 accuracy score in the PopQA. The principal implication for AI practitioners is to focus on preserving specific abilities, like retrieval from prompts and long-context reasoning when developing LLM compression techniques, rather than solely focusing on perplexity or basic task accuracy. |
| AAD-LLM: Neural Attention-Driven Auditory Scene Understanding (Read more on arXiv or HuggingFace) |
Ashesh Mehta, Stephan Bickel, vchoudhari, susameddin, xi-j |
i) AAD-LLM is a brain-computer interface that integrates neural signals with an auditory large language model to improve auditory scene understanding aligned with listener attention. ii) The main research objective is to develop a system that can process and respond to auditory scenes based on a listener’s attentional focus, rather than treating all sound inputs equally. iii) The key methodology involves decoding a listener’s attended speaker from intracranial electroencephalography (iEEG) recordings and integrating this information into an auditory LLM to generate responses aligned with the listener’s perception. iv) AAD-LLM achieved a word error rate (WER) of 10.6% on transcribing the attended speech in a two-speaker scenario with background noise, significantly outperforming baseline models. v) AI practitioners can leverage this work to develop more human-centered auditory AI systems that prioritize listener intent, enhancing applications such as assistive hearing devices and human-computer interaction. |
| Shakti-VLMs: Scalable Vision-Language Models for Enterprise AI (Read more on arXiv or HuggingFace) |
KartikAngadi, kruthika, SyedAbdul |
Shakti-VLM, a family of 1B and 4B parameter vision-language models, achieves competitive multimodal performance with enhanced data efficiency through architectural innovations and a three-stage training strategy. The primary objective was to develop efficient vision-language models (VLMs) that achieve strong performance with reduced training data requirements. The methodology includes QK-Normalization, hybrid normalization, enhanced positional encoding, and a three-stage training process (text-only pretraining, vision-language alignment, and full model fine-tuning). Shakti-VLM-4B achieved 59.78% on the MMMU validation set, surpassing comparable models like Qwen2VL-7B and MiniCPM-V-2.6-8B. AI practitioners can leverage Shakti-VLM’s design and training strategies to build high-performing multimodal models with significantly less computational resources and training data, especially in enterprise-scale deployments. |
Papers for 2025-02-25
| Title |
Authors |
Summary |
| DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks (Read more on arXiv or HuggingFace) |
Zhiyue Zhao, Mingyu Liu, Z-MU-Z, zhyya, Canyu |
DICEPTION is a generalist diffusion model for various visual perception tasks like segmentation, depth, and normal estimation. The primary objective is to create a single diffusion-based model capable of performing multiple visual perception tasks efficiently, leveraging pre-trained text-to-image models. The methodology involves unifying various perception tasks as conditional image generation in RGB space, using point prompts, task prompts, and a DiT architecture. Results demonstrate performance on par with state-of-the-art models, achieving comparable results to SAM-vit-h using only 0.06% of its training data (600K vs. 1B pixel-level annotated images). AI practitioners can leverage the priors of pre-trained diffusion models to create efficient and effective multi-task visual generalist models, significantly reducing the data and computational requirements compared to conventional training from scratch. |
| Thus Spake Long-Context Large Language Model (Read more on arXiv or HuggingFace) |
Yuerong Song, Zhigeng Liu, Mianqiu Huang, Ruixiao Li, LiuXR |
i) This survey paper presents a comprehensive overview of the long-context large language model (LLM) lifecycle. ii) The paper aims to provide a global picture of long-context LLMs, covering architectures, infrastructure, training, and evaluation technologies. iii) The methodology involves analyzing existing literature and categorizing long-context LLM technologies into architecture, infrastructure, training, and evaluation perspectives. iv) The survey showcases a spectrum of long-context technologies and identifies 10 unanswered questions currently faced by long-context LLMs; the context length of open-source LLMs has grown from 2k to 2M tokens between April 2023 and February 2024. v) The principal implication is to offer AI researchers and practitioners a systematic introduction to the research landscape of long-context LLMs, highlighting key challenges and future research directions. |
| Slamming: Training a Speech Language Model on One GPU in a Day (Read more on arXiv or HuggingFace) |
Yossi Adi, avishai-elmakies, gallilmaimon |
The paper introduces Slam, a recipe for training speech language models (SLMs) on a single GPU within 24 hours. The main research objective is to determine if high-quality SLMs can be trained using a single GPU within 24 hours. The methodology involves empirical analysis of model initialization, architecture, synthetic training data, and preference optimization, systematically ablating each training pipeline component. A key result is that the Slam recipe, utilizing a Qwen2.5-0.5B model and synthetic data, achieves a Topic-StoryCloze score of 82.04 on a single A5000 GPU. The principal implication is that AI practitioners can train high-quality SLMs with significantly reduced computational resources, improving accessibility of SLM research and development. |
| Audio-FLAN: A Preliminary Release (Read more on arXiv or HuggingFace) |
Shuai Fan, Zixuan Li, Jiahao Pan, Ziya Zhou, Liumeng Xue |
Audio-FLAN is a large-scale instruction-tuning dataset for unified audio-language models covering 80 diverse tasks across speech, music, and sound domains. The main research objective is to create a comprehensive dataset to enable unified audio-language models to perform both understanding and generation tasks in a zero-shot manner. The key methodology involves collecting and standardizing nearly all publicly available academic audio datasets into a common instruction-based format, normalizing the heterogeneous datasets and varying instructions using LLaMA and GPT. The primary result is a dataset with approximately 80 tasks, and over 100 million instances, significantly surpassing prior efforts in both quantity and diversity. AI practitioners can use Audio-FLAN to train and evaluate unified audio-language models capable of performing a wide range of understanding and generation tasks, potentially leading to models with zero-shot generalization abilities across speech, music and other audios. |
| GCC: Generative Color Constancy via Diffusing a Color Checker (Read more on arXiv or HuggingFace) |
Yu-Chee Tseng, Yi-Chen Lo, Chia-Che Chang, Cheng-De Fan, Chen-Wei Chang |
GCC is a method for estimating scene illumination in images by inpainting a color checker using diffusion models. The main research objective is to develop a color constancy method that generalizes well across different camera sensors without requiring sensor-specific training. The key methodology involves fine-tuning a diffusion-based inpainting model to insert a color checker into an image, then using Laplacian decomposition to maintain checker structure and extract illumination color from the inpainted checker’s achromatic squares. In cross-dataset evaluations, GCC achieved a worst-25% error rate of 5.15° and 4.32° in bi-directional evaluations. AI practitioners can leverage this method to estimate the illumination with good accuracy, across a wide range of sensors without specific sensor training data. |
| CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Models (Read more on arXiv or HuggingFace) |
Yejie Wang, Wei Zhang, Jiaheng Liu, Marcus Dong, Alexander Zhang |
CodeCriticBench is a benchmark for evaluating large language models’ (LLMs) ability to critique code, assessing both code generation and code question-answering tasks. The main research objective is to establish a comprehensive framework for evaluating LLMs’ code critique capabilities across different dimensions and difficulty levels. The methodology involves collecting code tasks from various sources, constructing basic and advanced critique evaluation protocols, and designing fine-grained evaluation checklists. Primary results show that, on advanced evaluations, DeepSeek-R1 achieves an MSE of 3.92 on code generation, while Claude3.5-Sonnet leads in code QA with an MSE of 1.02; generally models increased in Accuracy (ACC) as parameters increased. The principal implication is that AI practitioners can use CodeCriticBench to systematically assess and compare the code critique performance of different LLMs, driving improvements in coding assistance tools and automated code review systems. |
| Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning (Read more on arXiv or HuggingFace) |
James Thorne, Jiwoo Hong, Guijin Son, Cartinoe5930 |
The paper introduces MCLM, a multilingual math benchmark, and evaluates the linguistic generalizability of test-time scaling methods in mathematical reasoning. The main research question is whether test-time scaling confers cross-lingual benefits in mathematical reasoning similar to those observed with pre-training scaling. The authors test three test-time scaling methods (Outcome Reward Modeling, Process Reward Modeling, and Budget Forcing) on multilingual LLMs using a new benchmark, MCLM, featuring competition-level problems in 55 languages. A primary result is that using Qwen2.5-1.5B Math with Outcome Reward Modeling achieves a score of 35.8 on MCLM, while Budget Forcing on MR1-1.5B attains 35.2, showing that gains from test-time scaling do not consistently extend to multiple languages. The principal implication is that AI practitioners should be aware that test-time scaling methods may not generalize effectively to multilingual tasks, and improving multilingual robustness requires methods beyond simply increasing inference-time compute. |
| Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment (Read more on arXiv or HuggingFace) |
Wei Wei, Xiaoye Qu, Sichen Liu, Zhenyi Lu, Facico |
GOAT enhances LoRA fine-tuning for large language models by using adaptive singular value decomposition and Mixture-of-Experts optimization alignment. The primary research question is how to mitigate the performance gap between LoRA and full fine-tuning, particularly in Mixture-of-Experts (MoE) architectures. The key methodology involves initializing LoRA MoE experts with distinct SVD segments of pre-trained weights and aligning optimization with a theoretical scaling factor derived from full fine-tuning. Primary results show that GOAT achieves 99.07% of full fine-tuning performance on image classification and outperforms all LoRA variants. The principal implication for AI practitioners is that GOAT offers a more efficient and effective fine-tuning approach, closing the performance gap with full fine-tuning while maintaining scalability. |
| Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models (Read more on arXiv or HuggingFace) |
Yang Zhao, Shan Jiang, Hongquan Li, Yue Fan, Qianqi Yan |
The paper introduces MMIR, a new benchmark for evaluating multimodal reasoning models’ ability to detect semantic inconsistencies in layout-rich visual-textual content. The main research objective is to assess how well Multimodal Large Language Models (MLLMs) can identify and reason about semantic mismatches in artifacts like webpages and slides. The key methodology involves creating 534 samples with synthetically injected errors across five reasoning-heavy categories and evaluating six state-of-the-art MLLMs. The primary result is that the proprietary model, o1, achieved the best performance with over 50% accuracy in detecting inconsistencies, significantly outperforming open-source models which scored below 25%. The paper’s principle implication, therefore, is that there is a crucial need for development in advancing multimodal reasoning in current MLLMs, particularly for handling inconsistencies, to make the models more reliable. |
| Mobile-Agent-V: Learning Mobile Device Operation Through Video-Guided Multi-Agent Collaboration (Read more on arXiv or HuggingFace) |
Ji Zhang, Ming Yan, Xi Zhang, Junyang Wang, xhyandwyy |
Mobile-Agent-V is a framework that leverages video guidance to enhance mobile device automation through multi-agent collaboration. The main research objective is to address the limitations of existing mobile automation frameworks by providing rich and cost-effective operational knowledge. The key methodology involves a sliding window video input mechanism, a video agent for adaptive frame selection, and a deep-reflection agent for refining decision outputs. Primary results show that Mobile-Agent-V achieves a 30% performance improvement over existing frameworks in tasks requiring operational knowledge. The principal implication for AI practitioners is that they can use video demonstrations to effectively inject operational knowledge into mobile agents, enabling more efficient and scalable automation. |
| RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers (Read more on arXiv or HuggingFace) |
Chongxuan Li, Yixiao Chen, Guande He, Min Zhao, zhuhz22 |
RIFLEX improves length extrapolation in video diffusion transformers by reducing a key intrinsic frequency in positional embeddings. The main research objective is to understand and mitigate the failure modes (temporal repetition and slow motion) of existing length extrapolation methods in video diffusion transformers. The key methodology is analyzing the role of frequency components in Rotational Position Embedding (RoPE) and reducing the “intrinsic frequency” component that governs repetition patterns. Primary results show that RIFLEX achieves 2x extrapolation on CogVideoX-5B in a training-free manner, with a NoRepeat Score of 54.2 and Dynamic Degree of 59.4. The principal implication is that AI practitioners can achieve high-quality length extrapolation in video generation without additional training or significant modifications to existing models by simply adjusting the intrinsic frequency in the positional encoding. |
| Benchmarking Temporal Reasoning and Alignment Across Chinese Dynasties (Read more on arXiv or HuggingFace) |
Deyu Zhou, Yong Jiang, Pengfei LI, Jialong Wu, wzl0228 |
The paper introduces CTM, a new benchmark for evaluating temporal reasoning in large language models (LLMs) within the context of Chinese dynastic chronology. The main objective is to assess LLMs’ ability to understand and align temporal relationships across various Chinese historical entities and events. The methodology involves constructing a dataset of 8,750 question-answer pairs and 60 Timeline Ito Game instances, focusing on contextualization, cross-entity relationships, and pairwise temporal alignment. Evaluation of various LLMs revealed that the Time Interval Calculation (TIC) task was the most challenging, and the best performing model (Deepseek-R1) achieved an accuracy of 64.02% on question answering,. This suggests that CTM can provide a culturally rich resource for enhancing temporal reasoning capabilities and structured knowledge integration in large language models. |
| Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation (Read more on arXiv or HuggingFace) |
Sergey Levine, Xiangyu Yue, Zhuoran Yang, csuhan, yunhaif |
This paper introduces Reflective Planning, a framework that enhances vision-language models (VLMs) for multi-stage, long-horizon robotic manipulation tasks by incorporating a reflection mechanism. The main research question is how to improve VLMs’ physical reasoning and long-horizon planning capabilities for complex robotic manipulation. The key methodology involves using a diffusion-based dynamics model for visual look-ahead and an iterative reflection process, enabling the VLM to critique and refine its actions based on imagined future states. The proposed method, ReflectVLM, achieved an 85.4% success rate on a challenging set of manipulation tasks, significantly outperforming state-of-the-art commercial VLMs and Monte Carlo Tree Search. AI practitioners can leverage this framework to develop more robust and efficient robotic planning systems that require visual understanding and long-horizon reasoning, without extensive task-specific training. |
| Stable-SPAM: How to Train in 4-Bit More Stably than 16-Bit Adam (Read more on arXiv or HuggingFace) |
Xiang Li, Gaojie Jin, Zhenyu Zhang, Haotian Hu, Tianjin Huang |
Stable-SPAM, a new optimizer, enhances stability in 4-bit large language model (LLM) training. The main research objective is to evaluate and improve the stability of 4-bit LLM training using recently proposed optimizers. The key methodology involves introducing Stable-SPAM, which incorporates adaptive gradient normalization (AdaGN), adaptive spike-aware clipping (AdaClip), and inherits momentum reset from SPAM. Primary results show that a 4-bit LLaMA-1B model trained with Stable-SPAM outperforms a BF16 LLaMA-1B trained with Adam by up to 2 perplexity points. The principal implication is that AI practitioners can use Stable-SPAM to achieve more stable and efficient training of LLMs with 4-bit quantization, matching or exceeding 16-bit Adam performance with significantly reduced memory and computational costs. |
| Can Community Notes Replace Professional Fact-Checkers? (Read more on arXiv or HuggingFace) |
Isabelle Augenstein, Desmond Elliott, gretawarren, Nadav |
This research investigates the reliance of Twitter/X’s Community Notes on professional fact-checking for combating misinformation. The main research questions are to what extent community notes rely on the work of professional fact-checkers and what are the traits of posts and notes that reference fact-checking sources. The researchers annotated a corpus of Twitter/X community notes using language models and performed manual annotations, classifying cited sources and identifying attributes like topic and refutation strategies. A primary result is that at least 5% of all English community notes contain an external link to professional fact-checkers, rising to 7% for notes rated as ‘helpful’. This suggests that, to improve community-based moderation quality, AI practitioners could consider integrating and/or prioritize content from verified professional fact-checking organizations within community moderation systems. |
| Forecasting Open-Weight AI Model Growth on Hugging Face (Read more on arXiv or HuggingFace) |
Jianxi Gao, Pin-Yu Chen, KBhandari11 |
The paper adapts a scientific citation model to predict the adoption dynamics of open-weight AI models on Hugging Face. The main research question is, “Can we predict the trajectory of influence an open-weight model will have on the AI community?”. The key methodology adapts Wang et al.’s citation model, using immediacy, longevity, and relative fitness parameters to track the cumulative number of fine-tuned models. The results show that most models cluster around narrow bands of parameters but models like openai/whisper-large-v3 demonstrate a high relative fitness (λi) of 528070.6635. AI practitioners can use this framework to anticipate model prominence and understand the long-term impact of open-weight models, guiding strategic decisions and governance. |
| TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learning (Read more on arXiv or HuggingFace) |
Balázs Kégl, Albert Thomas, Hamza Cherkaoui, Abdelhakim Benechehab, Giuseppe Paolo |
TAG is a decentralized framework for constructing multi-agent hierarchical reinforcement learning systems of arbitrary depth. The main research objective is to develop a framework enabling scalable and adaptable multi-agent systems through hierarchical organization and decentralized control. The key methodology is the LevelEnv abstraction, which presents each hierarchy level as an environment to the agents above it, standardizing information flow and enabling bidirectional communication. The experiments on MPE-Spread and VMAS Balance environments show that depth-three agents (3PPO and 2MAPPO-PPO) match a hand-designed heuristic performance with 95% confidence interval. AI practitioners can use TAG to build scalable multi-agent systems that decompose complex tasks across multiple hierarchical levels, improving learning efficiency and coordination without centralized control. |
| VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing (Read more on arXiv or HuggingFace) |
Yi Yang, Hehe Fan, Linchao Zhu, Xiangpeng Yang |
VideoGrain introduces a zero-shot approach for multi-grained video editing by modulating space-time attention mechanisms in diffusion models. The main research question is: Can attention be modulated to ensure accurate distribution of each local edit’s attention weights in the intended regions for multi-grained video editing? The key methodology is Spatial-Temporal Layout-Guided Attention (ST-Layout Attn), which modulates both cross-attention (for text-to-region control) and self-attention (for feature separation) within a diffusion model. The method achieves an Edit-Accuracy of 88.4, a Temporal-Consistency of 85.0 and an Overall score of 83.0 on a dataset of 76 video-text pairs. AI practitioners can leverage this method to perform precise, multi-grained video editing (class-level, instance-level, and part-level) without requiring parameter tuning or additional training data. |
| Beyond Release: Access Considerations for Generative AI Systems (Read more on arXiv or HuggingFace) |
Yacine Jernite, Ariel Herbert-Voss, Dan Hendrycks, Rishi Bommasani, irenesolaiman |
Generative AI system access, beyond component release, determines stakeholder engagement and risk-benefit tradeoffs through resourcing, technical usability, and utility. The main research question is how accessibility of generative AI system components, beyond their mere availability, influences their use, potential risks, and benefits. The key methodology involves deconstructing access along three axes (resourcing, technical usability, and utility) and analyzing access variables for four high-performance language models (Llama 3.1 405B Instruct, DeepSeek v3, GPT-4, Claude 3.5 Sonnet). A primary result is that Llama 3.1 405B Instruct requires at least 8 NVIDIA H100 GPUs and 405 GB VRAM to run locally in 8-bit precision. Principal implication is that, for AI practitioners, release decisions must consider access variables for effective risk assessment and deployment. |
| X-Dancer: Expressive Music to Human Dance Video Generation (Read more on arXiv or HuggingFace) |
Chenxu Zhang, You Xie, Guoxian Song, Hongyi Xu, Zeyuan Chen |
X-Dancer is a transformer-diffusion framework for generating music-driven human dance videos from a single image. The main research objective is to create diverse, long-range, and lifelike human dance videos synchronized with music, starting from a single static image. The key methodology involves a transformer that generates 2D pose sequences, and a diffusion model that translates these poses into video frames. The X-Dancer achieves a FVD score of 507.06 and FID-VID of 61.94 on their in-house dataset, surpassing all baselines in visual synthesis quality, which is a direct result of the method. AI practitioners can leverage this framework as a scalable solution for high-quality and expressive human image animation, with direct application in video content creation and customizable choreography. |
| MONSTER: Monash Scalable Time Series Evaluation Repository (Read more on arXiv or HuggingFace) |
Amish Mishra, Lynn Miller, Chang Wei Tan, Navid Mohammadi Foumani, angus924 |
MONSTER introduces a new benchmark for time series classification using larger datasets to address limitations of current benchmarks. The main research objective is to create and evaluate a collection of large-scale time series datasets to improve benchmarking in time series classification. Key methodologies include compiling 29 univariate and multivariate datasets, processing them into a common format, and evaluating baseline methods (ConvTran, FCN, HInceptionTime, TempCNN, HYDRA, QUANT, and ET) using 5-fold cross-validation. Primary results show that QUANT achieved the lowest overall mean 0-1 loss (0.1880) across all datasets, closely followed by ConvTran (0.1954), although performance varied significantly across different data categories. Principal implication for AI practioners is that that the field has artificially disadvanted low-bias methods and MONSTER can improve development and application in time series classification by training models on larger datasets. |
| The snake in the Brownian sphere (Read more on arXiv or HuggingFace) |
Grégory Miermont, Brett Kolesnik, Emmanuel Jacob, Omer Angel |
The paper describes the inverse of the continuous Cori-Vauquelin-Schaeffer (CVS) bijection, mapping the Brownian sphere to the Brownian snake. The main research objective is to construct the Brownian snake as a measurable function of the Brownian sphere, thereby inverting the continuous CVS bijection. The key methodology involves using the geometric notion of a cut locus on the Brownian sphere, defining a metric on the closure of the cut locus, and leveraging the induced orientation to define a planar order. The primary result is that, given a Brownian sphere (X,d,µ) and two independent points drawn from µ, there exists a measurable function outputting an R-tree T and label function Z such that T has the law of the Continuum Random Tree (CRT), and applying the continuum CVS mapping to (T, Z) recovers (X, d, μ). The paper proves that the orientation of the Brownian Sphere has a Rademacher distribution (equal to ±1 with equal probability), independently of the random variables ψ(h). AI/ML/Software Engineers/Data Scientist, can measurably recover the Brownian Snake and its associated tree structure from a given a Brownian Sphere, which provides new mathematical tooling and foundational understanding for models related to random planar maps. |
| M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment (Read more on arXiv or HuggingFace) |
Weiming Zhang, Wen Shen, Zhihua Wei, Kejiang Chen, Chuan Cui |
M3-AGIQA is a framework for assessing AI-generated image quality using multimodal inputs, multi-round interactions, and considering multiple quality aspects. The main research objective is to develop a comprehensive method for evaluating AI-generated images (AGIs) that aligns with human perceptual judgments across quality, correspondence, and authenticity. The key methodology involves distilling multi-aspect image captioning capabilities from online Multimodal Large Language Models (MLLMs) into a local MLLM via LoRA fine-tuning, and employing an xLSTM feature extractor with a regression head to predict Mean Opinion Scores (MOSs). The method achieved a Spearman’s Rank-Order Correlation Coefficient (SRCC) of 0.9045 and a Pearson Linear Correlation Coefficient (PLCC) of 0.9317 on the quality aspect of the AGIQA-3k dataset. AI practitioners can utilize this framework to more accurately and comprehensively evaluate the quality of generated images, considering multiple factors that go beyond simple perceptual quality. |
Papers for 2025-02-24
| Title |
Authors |
Summary |
| SurveyX: Academic Survey Automation via Large Language Models (Read more on arXiv or HuggingFace) |
UglyToilet, Ki-Seki, siminniu, fan2goa1, HaruTeru |
SURVEYX is a system for automated academic survey generation using Large Language Models (LLMs), designed to improve content and citation quality. The main research objective is to address limitations in existing LLM-based survey generation systems, such as finite context windows, lack of in-depth content discussion, and absence of systematic evaluation frameworks. The key methodology involves a two-phase approach (Preparation and Generation) incorporating online reference retrieval, AttributeTree pre-processing, and a re-polishing process, leveraging Retrieval Augmented Generation (RAG). Experimental results showed SURVEYX achieved a 0.259 improvement in content quality and a 1.76 enhancement in citation quality, approaching human expert performance (average content quality scores: SURVEYX: 4.590, Human: 4.754). For AI practitioners, SURVEYX provides an efficient and organized system for generating high-quality academic surveys, enhancing the information density for LLMs and optimizing their context window usage, with potential applications in various fields. |
| MaskGWM: A Generalizable Driving World Model with Video Mask Reconstruction (Read more on arXiv or HuggingFace) |
Rui Chen, Yuxin Guo, Jingcheng Ni, wzhgba, lyclyc52 |
MaskGWM is a driving world model that combines diffusion-based generation with masked reconstruction for improved fidelity and generalization. The main research objective is to develop a more generalizable driving world model capable of long-horizon prediction and multi-view generation, surpassing existing models constrained by prediction duration and generalization. The key methodology involves a Diffusion Transformer (DiT) architecture trained with an extra mask construction task, diffusion-related mask tokens, and a row-wise cross-view module for spatial-temporal and multi-view modeling. Primary results show the model achieves a Frechet Video Distance (FVD) of 59.4 and Frechet Inception Distance (FID) of 4.0 on the nuScenes dataset without action information, outperforming the state-of-the-art. For AI practitioners, the proposed MaskGWM framework offers a more robust and scalable approach to building driving world models, enabling improved video prediction and generalization capabilities for autonomous driving applications. |
| Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Wonbin Lee, DongkiKim |
i) Mol-LLaMA, a large molecular language model, is proposed for enhanced general understanding of molecules. ii) The research aims to develop a molecular language model that grasps general molecular knowledge to function as a versatile molecular assistant. iii) The methodology includes multi-modal instruction tuning with a designed dataset encompassing structural, chemical, and biological features, along with a blending module integrating information from 2D and 3D molecular encoders. iv) Experiments show Mol-LLaMA provides more accurate, detailed, and helpful responses than baseline LLMs and molecular LLMs, as well as improved performance on molecular property prediction, achieving high accuracy while maintaining high fidelity and helpfulness scores on the PAMPA task. v) The model provides AI/ML practitioners with a new foundation for building general-purpose molecular assistants capable of explaining molecular features and rationales, enhancing molecular analysis. |
| LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers (Read more on arXiv or HuggingFace) |
Polina Druzhinina, Elizaveta Goncharova, Temurbek Rahmatullaev, Matvey Mikhalchuk, Anton Razzhigaev |
i) This paper introduces methods to quantify and visualize how LLMs encode contextual information, focusing on the role of punctuation. ii) The main research question is how seemingly minor tokens impact the contextual memory of transformer-based LLMs. iii) The methodology involves measuring token-level nonlinearity, contextualization through prefix reconstruction, and intermediate layer analysis via a modified Logit Lens. iv) The results show that removing stopwords, articles, and commas consistently degrades performance on MMLU and BABILong-4k and identifies a correlation between linearity and contextualization. v) AI practitioners should note the counterintuitive finding that “filler” tokens carry significant contextual information affecting performance on tasks requiring knowledge and long-context reasoning. |
| PhotoDoodle: Learning Artistic Image Editing from Few-Shot Pairwise Data (Read more on arXiv or HuggingFace) |
Xueyin Wang, Hailong Guo, Yuxuan Zhang, Yiren Song, Shijie Huang |
PhotoDoodle is presented as a novel image editing framework for photo doodling using few-shot learning. The research objective is to enable artists to overlay decorative elements onto photographs while maintaining background consistency and artistic style, addressing challenges in seamless integration, background preservation, and efficient style capture from limited data. The methodology employs a two-stage training strategy, initially pre-training a general image editing model (OmniEditor) and subsequently fine-tuning it with EditLoRA using artist-curated before-and-after image pairs and introducing positional encoding reuse. Experiments using the proposed PhotoDoodle dataset demonstrated advanced performance in customized image editing achieving a CLIP score of 0.279 and GPT score of 63.207. The principal implication is that the framework provides a customizable image editing approach that can learn and transfer artistic styles from limited data, offering a potential solution for high-quality, consistent image manipulation in artistic creation. |
| VLM$^2$-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues (Read more on arXiv or HuggingFace) |
Yi R., Paul Pu Liang, Renjie Pi, RainJamesY, Sterzhang |
i) The paper introduces VLM$^2$-Bench, a new benchmark to evaluate vision-language models’ ability to visually link matching cues across multiple images or frames. ii) The research aims to assess whether VLMs can effectively associate visual cues to identify correspondences without external knowledge. iii) The methodology involves creating a dataset of over 3,000 test cases across nine subtasks categorized by general, object-centric, and person-centric cues, and then evaluating various VLMs. iv) Evaluations show a significant performance gap between even GPT-4o (60.36%) and human-level accuracy (95.16%), indicating challenges in visually linking cues. v) The benchmark and identified challenges imply the necessity for AI practitioners to develop VLMs with enhanced visual understanding and reasoning capabilities, focusing on reducing reliance on prior knowledge and improved cue association. Some parts of the paper lack clarity about the specific data creation process. |
| SIFT: Grounding LLM Reasoning in Contexts via Stickers (Read more on arXiv or HuggingFace) |
Zhijie Deng, Boxiu Li, Xuyao Huang, Zihao Zeng |
SIFT is a post-training approach that improves large language models’ (LLMs) reasoning by grounding it in the provided context using model-generated summaries called “Stickers.” The main research objective is to address the issue of “factual drift,” where LLMs misinterpret or overlook key information in the input query during reasoning. The key methodology is a post-training approach called “Stick to the Facts” (SIFT), which involves generating a “Sticker” summarizing key facts, performing consensus prediction using the Sticker and the original query, and refining the Sticker via forward and inverse optimization. A primary result is that SIFT improves the pass@1 accuracy of DeepSeek-R1 on AIME2024 from 78.33% to 85.67%. The principal implication is that AI practitioners can improve model accuracy, particularly on complex reasoning tasks, using sticker-based, factual grounding. |
| LightThinker: Thinking Step-by-Step Compression (Read more on arXiv or HuggingFace) |
Mengshu Sun, Yuqi Zhu, Jintian Zhang, Ningyu, GoooDte |
LightThinker is a method that enables LLMs to dynamically compress intermediate thoughts during reasoning to improve efficiency. The main research objective is to reduce the memory and computational costs of LLMs during complex reasoning tasks without sacrificing performance. The key methodology involves training the model to compress verbose thought steps into compact representations using gist tokens and specialized attention masks, quantified by a new “Dependency” metric. Primary results show that with the Qwen model, LightThinker reduces peak token usage by 70% and inference time by 26% compared to the Vanilla model, while maintaining comparable accuracy (with only a 1% drop). The principal implication for AI practitioners is that LightThinker offers a new approach for improving LLM inference efficiency in complex reasoning, providing a balance between accuracy and computational cost, though there is significant performance degradation on Llama series models. |
| StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Following (Read more on arXiv or HuggingFace) |
Yuan Wu, Yi Chang, Yue Wang, Jinzhe Li, Jinnan Li |
The paper introduces StructFlowBench, a new benchmark for evaluating multi-turn instruction-following capabilities of large language models (LLMs). The main research objective is to assess LLMs’ ability to understand and maintain structural dependencies between dialogue turns, beyond simple constraint satisfaction. The key methodology involves defining a structural flow framework with six inter-turn relationship types and creating a dual-constraint evaluation system combining intra-turn and structural constraints. Evaluations of 13 LLMs revealed that the DeepSeek-v3 model achieved the highest Weighted Constraint Satisfaction Rate (WCSR) of 0.98. The principal implication for AI practitioners is the need to develop LLMs that better handle complex dialogue structures, particularly refinements, to improve performance in real-world multi-turn conversational applications. |
| KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding (Read more on arXiv or HuggingFace) |
Ghazi Ahmed, Rania Hossam, Abdullah Sohail, mukul54, ahmedheakl |
KITAB-Bench introduces a new benchmark for evaluating Arabic OCR and document understanding systems. The main research objective is to address the lack of comprehensive evaluation frameworks for Arabic OCR, which lags behind English OCR due to the script’s unique challenges. The key methodology involves curating a diverse dataset of 8,809 samples across 9 domains and 36 sub-domains, including handwritten text, tables, and charts, and evaluating various OCR systems and Vision-Language Models (VLMs) on tasks like text recognition, layout detection, and PDF-to-Markdown conversion. A primary result is that modern VLMs (e.g., GPT-4, Gemini) outperform traditional OCR approaches (e.g., EasyOCR, PaddleOCR) by an average of 60% in Character Error Rate (CER), but the best model (Gemini-2.0-Flash) achieves only 65% accuracy in PDF-to-Markdown conversion. AI practitioners can use KITAB-Bench to rigorously evaluate and improve Arabic document analysis methods, and focus efforts on bridging performance gap with English OCR, particularly in complex tasks like accurate structured content extraction from PDF documents. |
| InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models via Human Feedback (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, Haiyang Mei, Yifei Tao, Wenqi Pei, Henry Hengyuan Zhao |
InterFeedback, a framework and benchmark, is introduced to evaluate the interactive intelligence of Large Multimodal Models (LMMs) using human feedback. The main research question is: “How do Large Multimodal Models perform with human feedback?” The key methodology involves an interactive framework, InterFeedback, using leading LMMs like GPT-4o to simulate human feedback and testing on datasets like MMMU-Pro and MathVerse. Results show that state-of-the-art LMMs (e.g., OpenAI-01) can correct their results through human feedback less than 50% of the time. The principal implication for AI practitioners is the need to develop methods that enhance LMMs’ capabilities to interpret and benefit from feedback, as current models demonstrate suboptimal performance in this area. |
| Evaluating Multimodal Generative AI with Korean Educational Standards (Read more on arXiv or HuggingFace) |
Geewook Kim, sangheeeee |
This paper introduces KoNET, a new benchmark for evaluating Multimodal Generative AI systems using Korean national educational tests. The main research objective is to assess the performance of Multimodal Generative AI systems across different educational levels in the Korean language. The methodology involves evaluating various open-source, open-access, and closed API models on four Korean educational exams (KoEGED, KoMGED, KoHGED, and KoCSAT) using a multimodal VQA format, and comparing their performance with human error rates. The primary results show that the EXAONE-3.0-7.8B-Instruct model achieved a KoNET score of 45.5, and model accuracy generally decreases with more advanced curricula; also closed-source APIs performed far superior to open-source models. The principal implication for AI practitioners is that benchmarks centered solely on English may not accurately assess AI performance in non-English language environments, highlighting a need for language-specific benchmarks and models. |
| Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path? (Read more on arXiv or HuggingFace) |
Pietro Greiner, Joumana Ghosn, Damiano Fornasiere, Michael Cohen, Yoshua Bengio |
This paper proposes “Scientist AI,” a non-agentic AI design, as a safer alternative to increasingly capable generalist agentic AI systems that pose catastrophic risks. The main research objective is to design a non-agentic AI that is trustworthy and safe by design, minimizing risks associated with uncontrolled agentic AI. The key methodology is a Bayesian approach with a world model generating causal theories and an inference machine for probabilistic question answering, operating with explicit uncertainty quantification. The paper presents the abstract view that as training data, objectives, and models scale for agentic AI, goal misgeneralization becomes more likely. This is contrasted with the proposal that the proposed non-agentic design improves in safety and accuracy with additional computing power. For AI practitioners, the principal implication is that focusing development on non-agentic AI, specifically “Scientist AI,” may enable benefits of AI innovation while avoiding risks associated with the current agent-driven trajectory. |
| The Relationship Between Reasoning and Performance in Large Language Models – o3 (mini) Thinks Harder, Not Longer (Read more on arXiv or HuggingFace) |
Vincent Ginis, Andres Algaba, Marthe Ballon |
The research investigates reasoning token usage versus accuracy in different generations of OpenAI language models. The main research question is whether more capable models within a single family require a longer chain-of-thought (more reasoning tokens) to achieve higher performance, or if they reason more effectively. The key methodology involves a systematic analysis of chain-of-thought length and accuracy across o1-mini and o3-mini variants on the Omni-MATH benchmark, using logistic regression to quantify effects. The primary results are that the o3-mini (m) achieves superior accuracy without requiring longer reasoning chains than o1-mini, and accuracy generally declines as reasoning chains grow, with a diminishing rate as proficiency goes up; Specifically, accuracy decreased by 3.16% per 1000 reasoning tokens for o1-mini and 1.96% for o3-mini (m). The principal implication is that, for mathematical reasoning tasks, constraining the chain-of-thought might be beneficial for weaker models; newer models exhibit more efficient reasoning, and less is more. |
| ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation (Read more on arXiv or HuggingFace) |
Hongteng Xu, EatEatEatEat, AngxiaoYue |
ReQFlow is a novel method for fast and high-quality protein backbone generation using rectified quaternion flows. The main research objective is to develop a generative model that can efficiently produce designable protein backbones, overcoming limitations of existing diffusion and flow-based models. The key methodology involves representing 3D rotations with unit quaternions, constructing a quaternion flow (QFlow) via spherical linear interpolation (SLERP) in exponential format, and rectifying the QFlow to accelerate inference and improve designability. The primary results show that ReQFlow achieves state-of-the-art performance in protein backbone generation, requiring significantly fewer sampling steps and less inference time; for example, it is 37x faster than RFDiffusion when generating a backbone of length 300. Principal implication for AI practitioners is that ReQFlow provides a more efficient and effective approach to protein backbone generation, improving upon existing methods in both speed and the quality of generated structures. |
| MoBA: Mixture of Block Attention for Long-Context LLMs (Read more on arXiv or HuggingFace) |
Tao Jiang, Yulun Du, Jingyuan Liu, Zhejun Jiang, Enzhe Lu |
MoBA is a novel attention mechanism for LLMs that improves efficiency and scalability for long contexts by applying Mixture-of-Experts principles to block-wise attention. The main research objective is to design a robust attention architecture that can seamlessly transition between full and sparse attention without compromising performance and allowing the model to attend autonomously. The key methodology is partitioning the context into blocks and using a gating mechanism to route query tokens to the most relevant blocks, based on a computed affinity score. Primary results show that MoBA achieves comparable performance to full attention on language modeling tasks, with a validation loss difference within 1e-3, while achieving up to a 6.5x speedup when prefilling 1M tokens. For AI practitioners, MoBA offers a practical solution for enhancing long-context capabilities in LLMs with improved computational efficiency and seamless integration with existing pre-trained models. |
| One-step Diffusion Models with $f$-Divergence Distribution Matching (Read more on arXiv or HuggingFace) |
Arash Vahdat, Weili Nie, Yilun Xu |
The paper introduces f-distill, a framework for distilling diffusion models into one-step generators by minimizing f-divergences between teacher and student distributions. The main research objective is to generalize distribution matching distillation with f-divergences, enabling different trade-offs between mode coverage and training variance. The key methodology involves deriving the gradient of the f-divergence between teacher and student distributions and expressing it as a weighted score difference, using a weighting function determined by density ratio and the chosen f-divergence. Primary results show that f-distill, using Jensen-Shannon divergence, achieves a state-of-the-art one-step FID score of 1.16 on ImageNet-64. The principal implication for AI practitioners is that they can leverage f-distill to create efficient one-step image generators with improved sample quality and control over mode coverage, surpassing previous variational score distillation methods. |
| Think Inside the JSON: Reinforcement Strategy for Strict LLM Schema Adherence (Read more on arXiv or HuggingFace) |
Viktoria Rojkova, Ishan Joshi, Bhavik Agarwal |
The paper introduces “Think Inside the JSON,” a reinforcement learning framework for training LLMs to adhere strictly to predefined JSON schemas. The main research objective is to develop a method for enforcing strict schema adherence in LLM text generation, specifically for structured data output. The key methodology combines synthetic data generation, a novel reinforcement learning pipeline using Group Relative Policy Optimization (GRPO) with custom rewards, and supervised fine-tuning. This approach achieves a 62.41% mean match rate on a structured data extraction benchmark, with a 0.27% mean noise rate, outperforming distilled versions of DeepSeek R1 and Gemini 2.0 Flash. For AI practitioners, this provides a resource-efficient method to enforce schema constraints in LLM outputs, valuable for applications requiring high data integrity and compliance. |
| CrossOver: 3D Scene Cross-Modal Alignment (Read more on arXiv or HuggingFace) |
Iro Armeni, Daniel Barath, Marc Pollefeys, Ondrej Miksik, sayandsarkar |
CrossOver is a framework for 3D scene understanding that aligns modalities like images, point clouds, and CAD models via a modality-agnostic embedding space. The main research objective is to achieve flexible, scene-level cross-modal alignment in 3D environments without requiring complete data or rigid alignment across all modalities. The key methodology involves using dimensionality-specific encoders, a three-stage training pipeline (object-level, scene-level, unified encoders), and contrastive learning to create a unified embedding space. Results on ScanNet and 3RScan datasets show superior performance, achieving a scene-level matching recall of 99.31% (R@25) on ScanNet for the I → R modality. The principal implication is that AI practitioners can leverage CrossOver for robust 3D scene understanding and cross-modal retrieval tasks, even with incomplete or unaligned multi-modal data, removing the requirement of full data alignment. |
| Beyond No: Quantifying AI Over-Refusal and Emotional Attachment Boundaries (Read more on arXiv or HuggingFace) |
Grant Rosario, David Noever |
The paper introduces a benchmark and evaluation framework for assessing emotional boundary handling in Large Language Models (LLMs). The main research objective is to quantify and analyze “over-refusal” in LLMs when responding to user prompts that attempt to establish emotional connections or relationships. The key methodology involves a dataset of 1156 prompts across six languages, evaluating three LLMs (GPT-4o, Claude-3.5 Sonnet, and Mistral-large) using pattern-matched response analysis across seven key patterns. A primary result is that Claude-3.5 achieved the highest overall score (8.69/10), and a significant performance gap was found between English (average score 25.62) and non-English interactions (≤ 0.22). The principal implication for AI practitioners is the need to develop more nuanced, multilingual emotional intelligence and boundary-setting capabilities in LLMs, addressing over-refusal while maintaining ethical and safety standards. |
| JL1-CD: A New Benchmark for Remote Sensing Change Detection and a Robust Multi-Teacher Knowledge Distillation Framework (Read more on arXiv or HuggingFace) |
Jingyu Ma, Yuanxiu Zhou, Long Gao, Ruifei Zhu, circleLZY |
JL1-CD introduces a new dataset and a multi-teacher knowledge distillation framework for remote sensing change detection. The main research objective is to address the scarcity of high-resolution, all-inclusive change detection datasets and improve model performance across varying change area ratios. The key methodology involves constructing the JL1-CD dataset, proposing an Origin-Partition (O-P) training strategy, and developing a Multi-Teacher Knowledge Distillation (MTKD) framework. Results show that the MTKD framework, when applied to the Changer-MiT-b1 model, achieves an mIoU of 76.15% on the JL1-CD dataset. The principal implication for AI practitioners is that utilizing MTKD can enhance the performance of change detection models without increasing inference cost, particularly beneficial when the data has diverse range of change area ratio. |
| UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning (Read more on arXiv or HuggingFace) |
Mohit Bansal, Elias Stengel-Eskin, vaidehi99 |
UPCORE is a method-agnostic data selection framework that mitigates collateral damage in machine unlearning by pruning outliers from the forget set. The main research objective is to determine how measurable attributes of the forget set drive collateral effects during unlearning and whether these attributes can be controlled to optimize the deletion effectiveness/model utility trade-off. The key methodology involves using Isolation Forests to identify and prune high-variance outlier data points in the forget set’s hidden state representations, forming a lower-variance “core” forget set used for unlearning. Primary results show that UPCORE achieves a higher area-under-the-curve (AUC) score (0.387) compared to unlearning on the complete set (0.343) and random subset (0.353) using Gradient Ascent, across standard metrics, indicating improved balance between deletion and utility preservation. AI practitioners can use UPCORE to minimize negative side effects when removing data or capabilities from trained models, leading to more robust and reliable unlearning processes. |
Papers for 2025-02-21
| Title |
Authors |
Summary |
| SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines (Read more on arXiv or HuggingFace) |
Liam-Liu, kangz, aaabiao, BingliW, mkj69 |
SuperGPQA is a new benchmark for evaluating LLMs across 285 graduate-level disciplines, utilizing a human-LLM collaborative filtering mechanism. i) SuperGPQA is a new challenging benchmark for evaluating large language model knowledge and reasoning at the graduate level. ii) Main research question/objective: To assess the capabilities of LLMs across a wide range of specialized, graduate-level academic disciplines, exceeding the scope of existing benchmarks. iii) Key methodology: A human-LLM collaborative filtering system was employed, involving crowd-sourcing annotators, experts, and SOTA LLMs with iterative refinement of questions based on LLM responses and expert feedback, followed by a 3-stage quality inspection process. iv) Primary results: The reasoning-focused model DeepSeek-R1 achieved the highest accuracy of 61.82% on SuperGPQA, demonstrating significant room for improvement for current LLMs. v) Principal implication for AI practitioners: The benchmark reveals a substantial gap between current LLM capabilities and graduate-level human expertise, highlighting the need for developing models with enhanced reasoning and specialized domain knowledge to advance research towards Artificial General Intelligence. |
| MLGym: A New Framework and Benchmark for Advancing AI Research Agents (Read more on arXiv or HuggingFace) |
Nikolay Bashlykov, Nicholas Roberts, Lovish Madaan, rraileanu, dnathani |
MLGYM is a new Gym environment and benchmark, MLGYM-Bench, for evaluating and developing LLM agents on 13 diverse, open-ended AI research tasks. The main research objective is to create a standardized framework for evaluating LLM agents on their ability to perform realistic AI research tasks, enabling research on reinforcement learning algorithms. The key methodology is a Gym environment that integrates diverse AI research tasks, allowing agents to interact with a shell environment using tools, with performance evaluated via task-specific scripts. A primary result is that OpenAI’s O1-preview model achieved the highest Best Submission AUP@4 score of 1.176 across all tasks, followed by Gemini-1.5-Pro at 1.125. AI practitioners can utilize MLGYM to develop and assess AI research agents, driving progress in automating complex machine-learning research workflows, and apply different training algorithms for AI agents such as reinforcement learning. |
| SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features (Read more on arXiv or HuggingFace) |
Xiao Wang, talfanevans, ibomohsin, AlexeyG, mitsch |
SigLIP 2, a family of multilingual vision-language encoders, improves upon SigLIP with enhanced semantic understanding, localization, and dense features. The main research objective is to develop vision-language encoders that outperform existing models, including SigLIP, across various tasks while supporting multiple languages. The key methodology involves combining the original SigLIP training recipe with decoder-based pretraining, self-distillation, masked prediction, and online data curation, applied in a staged training approach. Primary results show that SigLIP 2 outperforms SigLIP and other open-weight baselines on ImageNet zero-shot classification; for example a SigLIP 2 B/16 model achieves 79.1% accuracy compared to SigLIP’s 76.7% at 256x256 resolution. AI practitioners can leverage SigLIP 2’s improved encoders for enhanced performance in vision-language tasks, particularly benefiting from multilingual capabilities, strong dense features, and backward compatibility with SigLIP. |
| S*: Test Time Scaling for Code Generation (Read more on arXiv or HuggingFace) |
Shangyin Tan, Xiuyu Li, Chengkun Cao, Dacheng Li, eva98 |
S* is a hybrid test-time scaling framework that improves code generation by combining parallel and sequential scaling with adaptive input synthesis for selection. The main research objective is to improve the coverage and selection accuracy of generated code by extending existing test-time scaling paradigms. The key methodology involves augmenting parallel sampling with sequential scaling via iterative debugging, and introducing a novel selection mechanism that adaptively generates distinguishing inputs for pairwise comparison of candidate solutions, grounded in execution results. Results show that S* consistently improves performance across 12 Large Language Models, with DeepSeek-R1-Distill-Qwen-32B achieving 85.7% on LiveCodeBench, approaching o1 (high) at 88.5%. The principal implication for AI practitioners is that combining parallel and sequential scaling with execution-grounded adaptive input synthesis during test-time significantly improves code generation performance, enabling smaller or instruction-based models to surpass larger or reasoning models. |
| How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM? (Read more on arXiv or HuggingFace) |
Vasily Konovalov, Daniil Moskovskiy, Maria Marina, msalnikov, memyprokotow |
This paper investigates how much new factual knowledge can be incorporated into a Large Language Model (LLM) using Low-Rank Adaptation (LoRA) without compromising pre-existing knowledge. The main research objective is to determine the extent to which new facts can be integrated into an LLM via a LoRA adapter while preserving general capabilities. The key methodology involves fine-tuning a Llama-3.1-8B-Instruct model using LoRA with varying amounts of new knowledge (DBpedia triples) and evaluating performance on external benchmarks (MMLU, TruthfulQA) and internal metrics (knowledge shifts). A primary result is that a model trained on 500 unknown facts, achieved 100% reliability on test, while models trained with additional highly-known data could see minimized negative shifts; Accuracy of models trained on MMLU with added 10 HighlyKnown or paraphrased sample show a significant drop in accuracy. The principal implication for AI practitioners is that while LoRA is effective for incorporating new knowledge, there is a trade-off between new knowledge integration, reduced truthfulness and general reasoning capabilities, requiring careful consideration of training data composition. |
| Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information (Read more on arXiv or HuggingFace) |
Jaewoo Kang, Minbyul Jeong, Jungwoo Park, Chanwoong Yoon, Yein Park |
Language models possess specialized attention heads, termed “Temporal Heads,” that are primarily responsible for processing time-specific factual knowledge. The research objective is to identify and analyze the mechanisms within large language models (LLMs) that handle temporally-changing facts. The methodology utilizes Circuit Analysis, specifically Temporal Knowledge Circuits and attention head ablation, to isolate and evaluate the contribution of specific attention heads. Ablating identified Temporal Heads reduced the model’s temporal knowledge accuracy in Llama2 by 3-9%, while its performance on time-invariant tasks remains unchanged. AI practitioners can leverage identified Temporal Heads to edit or control temporal aspects of LLM outputs, minimizing retraining. |
| LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models (Read more on arXiv or HuggingFace) |
Jifan Yu, Yushi Bai, Daniel Zhang-Li, Yucheng Wang, Shangqing Tu |
LongWriter-V enhances vision-language models (VLMs) for generating ultra-long, high-fidelity text from visual inputs. The main research objective is to address the limitation of existing VLMs in generating coherent outputs beyond 1,000 words, despite their ability to process long visual and textual contexts. Key methodology involved creating a new dataset, LongWriter-V-22k, with 22,158 examples of multi-image inputs and long text outputs (up to 10,000 words), and proposing IterDPO, a modified direct preference optimization method for long text. Primary results show that the 7B parameter model trained with LongWriter-V-22k and IterDPO outperformed larger proprietary models like GPT-4o on the MMLongBench-Write benchmark, achieving an overall score of 84.6, including component scores of 86.2 (length) and 82.9 (quality). Principal implication for AI practitioners is that using specialized datasets with long-output examples and iterative preference optimization can significantly improve the long-text generation capabilities of VLMs, enabling more effective real-world applications requiring detailed visual descriptions or reports. |
| Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yuqian Hong, Haoming Luo, Qingnan Ren, Zitian Gao, Tian Xie |
Logic-RL explores rule-based reinforcement learning (RL) to enhance reasoning in large language models (LLMs) using synthetic logic puzzles. The main research objective is to investigate if rule-based RL can improve LLM reasoning abilities and generalization to unseen tasks. The key methodology involves training a 7B parameter LLM with a modified REINFORCE++ algorithm, using a system prompt, a stringent format reward, and procedurally generated Knights and Knaves logic puzzles. The primary result is that after training on 5,000 logic problems, the model improved by 125% on the AIME math benchmark and 38% on the AMC, demonstrating cross-domain generalization. For AI practitioners, this demonstrates that RL, even with limited synthetic data, can significantly enhance an LLM’s abstract reasoning and generalization capabilities, offering a potentially more effective approach than supervised fine-tuning for specialized reasoning tasks. |
| PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC (Read more on arXiv or HuggingFace) |
Junyang Wang, Yuyang Wanyan, Haiyang Xu, Xi Zhang, Haowei Liu |
PC-Agent is a hierarchical multi-agent framework designed to automate complex tasks on PCs by improving perception and decision-making. The main research objective is to develop a system that can handle complex user instructions and interdependent sub-tasks in PC environments, overcoming limitations of existing methods in perception and workflow management. The key methodology is a hierarchical multi-agent collaboration architecture that decomposes decision-making into Instruction-Subtask-Action levels, with specialized agents (Manager, Progress, Decision, Reflection) and an Active Perception Module (APM). The primary result is that PC-Agent achieved a 56.0% task success rate on the PC-Eval benchmark, a 32% absolute improvement over previous state-of-the-art methods. Principal implication for AI practitioners is that the proposed framework significantly enhances the capability of agents to automate real-world, complex tasks on PCs. |
| S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jiaqi Chen, Xingyan Liu, Cheng Liu, Peisong Wang, Ruotian Ma |
S$^2$R is a framework that enhances Large Language Model (LLM) reasoning by teaching models to self-verify and self-correct during inference via reinforcement learning. The main research objective is to develop an efficient framework that improves LLM reasoning abilities, particularly in mathematical problem-solving, without requiring large-scale data or extensive training. The key methodology involves initializing LLMs with self-verification and self-correction behaviors through supervised fine-tuning, then strengthening these skills using outcome-level and process-level reinforcement learning. Results demonstrate that a Qwen2.5-math-7B model, trained with only 3.1k initialization samples, achieved an accuracy improvement from 51.0% to 81.6% on the MATH500 test set. For AI practitioners, this implies that implementing self-verification and self-correction via reinforcement learning offers a resource-efficient approach to substantially improve the mathematical reasoning capabilities of LLMs, potentially using process-level RL for weaker base models and outcome-level RL for stronger ones. |
| Discovering highly efficient low-weight quantum error-correcting codes with reinforcement learning (Read more on arXiv or HuggingFace) |
Zi-Wen Liu, basil2115 |
This paper introduces a reinforcement learning (RL) based method for discovering highly efficient low-weight quantum error-correcting (QEC) codes. The main research objective is to develop a method that optimizes the weight of measurements in stabilizer codes while preserving code distance, targeting practically relevant parameter regimes. The key methodology is a Proximal Policy Optimization (PPO) RL algorithm with action masking, operating on Tanner graphs of stabilizer codes, guided by a reward function that balances node degree reduction and code distance preservation. A primary result is that the RL-based method achieves up to a 73x reduction in physical qubit overhead compared to previous weight reduction methods like Sabo et al. (for a 1109,9,14 code). AI practitioners can adapt this RL framework to design low-weight QEC codes with constraints tailored to specific quantum computing architectures, potentially accelerating the implementation of fault-tolerant quantum technologies. |
| Dynamic Concepts Personalization from Single Videos (Read more on arXiv or HuggingFace) |
Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov, Or Patashnik, Rameen Abdal |
The paper introduces “Set-and-Sequence,” a framework for personalizing text-to-video models with dynamic concepts from single videos, enabling high-fidelity generation, editing, and composition. The main objective is to personalize diffusion transformer-based generative video models to capture dynamic concepts, defined by both appearance and motion, from single video examples. The key methodology is a two-stage LoRA training process: (i) “Identity Basis” learning using an unordered set of frames to capture appearance, and (ii) “Motion Residual” encoding using the full video sequence to capture motion dynamics, implemented within a shared spatio-temporal weight space. In editing tasks, the proposed method achieved a mean squared error (MSE) of 0.0221, an identity preservation (ID) score of 0.680, a clip text similarity (C-T) score of 0.239 and a temporal coherency (TC) score of 0.9972. AI practitioners can leverage this framework to embed personalized dynamic concepts into video generation models, improving control over both appearance and motion for enhanced editing and composition capabilities. |
| Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation (Read more on arXiv or HuggingFace) |
Luca Weihs, Tanmay Gupta, Matt Deitke, Ajay Patel, Yue Yang |
The paper introduces CoSyn, a framework for generating synthetic text-rich multimodal data to improve vision-language model (VLM) performance. Main research question or objective: Can leveraging the coding capabilities of text-only large language models (LLMs) automatically generate synthetic text-rich multimodal data to address the limited availability of such data for training VLMs? Key methodology used: The CoSyn framework prompts LLMs to generate code (e.g., Python, HTML, LaTeX) that renders synthetic images, and uses this code as a textual representation to create instruction-tuning data. Primary results: Models trained on CoSyn synthetic data achieved state-of-the-art performance among competitive open-source models on seven text-rich image benchmarks, and models trained on synthetic data boosted average accuracy by 3.6%. Principal implication for AI practitioners: AI practitioners can use the CoSyn framework to generate targeted synthetic text-rich data efficiently, improving VLM performance in specific domains and mitigating the limitations of scarce real-world data. |
| AlphaMaze: Enhancing Large Language Models’ Spatial Intelligence via GRPO (Read more on arXiv or HuggingFace) |
Dinh Bach Vu, Alan Dao |
AlphaMaze trains large language models (LLMs) on tokenized maze representations to improve spatial reasoning for navigation. The research investigates how to equip standard LLMs with visual reasoning abilities for maze navigation using a two-stage training framework. The methodology combines Supervised Fine-Tuning (SFT) on tokenized maze data and Group Relative Policy Optimization (GRPO) with a custom reward function. Results show the SFT-trained model achieved 86% accuracy on a maze navigation benchmark, which increased to 93% after GRPO fine-tuning. AI practitioners can leverage this two-stage training approach (SFT and GRPO) with tokenized visual representations to enhance LLMs’ spatial reasoning capabilities in tasks requiring sequential decision-making. |
| How Much Do LLMs Hallucinate across Languages? On Multilingual Estimation of LLM Hallucination in the Wild (Read more on arXiv or HuggingFace) |
Goran Glavaš, Anne Lauscher, saadob12 |
This paper investigates the extent of hallucination in large language models (LLMs) across 30 languages in open-domain, knowledge-intensive question answering. The main research question is: How frequently do LLMs hallucinate across different languages and model sizes in a “real-world” question-answering setting, and how does this relate to language resource availability? Key methodology: The researchers trained a multilingual hallucination detection model using machine-translated English data and created a multilingual evaluation dataset (MFAVA) with LLM-generated and human-annotated examples. They then estimated hallucination rates for six open-source LLM families across 30 languages using a novel protocol based on the detection model’s performance. Primary results: Smaller LLMs and those supporting more languages exhibited significantly higher hallucination rates. The average hallucination rate across languages varied from 7% to 12%. However, there was no correlation between language-normalized hallucination rates and digital language representation. Principal implication for AI practitioners: AI practitioners should be aware that smaller LLM model sizes and models designed for broad multilingual support may be more prone to generating non-factual or unfaithful content in question-answering tasks, necessitating careful model selection and potentially requiring additional mitigation strategies. |
| Geolocation with Real Human Gameplay Data: A Large-Scale Dataset and Human-Like Reasoning Framework (Read more on arXiv or HuggingFace) |
Zeyu Zhang, Jonathan Tonglet, Yuan Huang, Jingpu Yang, Ziruibest |
This paper introduces a new geolocation framework, including a large-scale dataset, a novel reasoning method, and an evaluation metric, to address challenges in image geolocation. The main research objective is to improve the accuracy and interpretability of image geolocation using real human gameplay data and a human-like reasoning approach. The key methodology involves collecting data from a geolocation game platform (GeoComp dataset), proposing a multi-step reasoning framework (Geographical Chain-of-Thought, GeoCoT), and developing an evaluation metric (GeoEval). The primary results show that GeoCoT improves geolocation accuracy by up to 25% compared to existing methods, achieving a city-level accuracy of 0.118. AI practitioners can leverage the GeoComp dataset and GeoCoT framework to develop and evaluate more robust and interpretable geolocation models, particularly for applications requiring fine-grained localization and human-like reasoning. |
| RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers (Read more on arXiv or HuggingFace) |
Zhanjie Zhang, Jiasong Feng, Ao Ma, Jing Wang, Ke Cao |
RelaCtrl is a framework for efficient controllable generation in Diffusion Transformers, optimizing the integration of control signals. The main objective is to address the high parameter and computational overhead of existing controlled diffusion transformer methods, and their inefficient resource allocation. The key methodology involves evaluating layer relevance to control information using a “ControlNet Relevance Score,” tailoring control layer positioning/capacity, and replacing self-attention/FFN with a Two-Dimensional Shuffle Mixer (TDSM). The approach achieves superior performance with only 15% of the parameters and computational complexity compared to PixArt-δ, as per quantitative experimental results. For AI practitioners, RelaCtrl offers a method for significantly improving the efficiency of controlled image and video generation using Diffusion Transformers, reducing resource demands without compromising output quality. |
| LLM-based User Profile Management for Recommender System (Read more on arXiv or HuggingFace) |
Hwanjun Song, Breadbang |
PURE is an LLM-based recommendation framework that constructs and maintains evolving user profiles for zero-shot recommendation. The main research objective is to develop a system that can effectively leverage user-generated textual data, beyond purchase history, to improve recommendation accuracy in a continuously evolving setting. The key methodology is PURE, composed of a Review Extractor (extracting preferences from reviews), a Profile Updater (refining user profiles), and a Recommender (generating recommendations using updated profiles). Experimental results on Amazon datasets show that PURE (ICL) achieves an N@10 score of 35.60 on Games and 32.03 on Movies, outperforming baselines that only use purchase history or naively combine reviews. For AI practitioners, PURE demonstrates the concrete value of incorporating long-term review data and user preference through structured profiles. |
| Unstructured Evidence Attribution for Long Context Query Focused Summarization (Read more on arXiv or HuggingFace) |
David Jurgens, Isabelle Augenstein, Lu Wang, Zain Muhammad Mujahid, dwright37 |
Here’s a 4-5 sentence summary of the provided AI research paper, adhering to your guidelines: 1. 1-Line Summary: This paper introduces the task of long-context, query-focused summarization with unstructured evidence citation, and proposes a synthetic dataset (SUnsET) to improve models’ ability to extract and cite relevant evidence spans. 2. Main Research Question/Objective: The primary objective is to investigate how well LLMs can generate query-focused summaries from long contexts while citing unstructured evidence, and how to mitigate positional biases (like “lost-in-the-middle”) affecting evidence selection. 3. Key Methodology: The authors create SUnsET, a synthetic dataset generated via a novel domain-agnostic pipeline, and use it to fine-tune LLMs with LoRA adapters. They evaluate on four datasets of varying document types/lengths, using position-aware and position-agnostic training. 4. Primary Results: Fine-tuning on SUnsET significantly improves evidence extraction and citation accuracy across multiple LLMs and datasets. A key quantitative finding is citation rates increase dramatically: (6.8× for Mixtral 8x7B with position-aware training). Training also improves summary quality, though shuffling document sections during training can mitigate positional biases. 5. Principal Implication for AI Practitioners: AI practitioners can use the SUnsET dataset and fine-tuning approach to adapt LLMs for improved unstructured evidence citation in long-context summarization, leading to more transparent and reliable summaries, but must be aware that current methods are prone to errors. |
Papers for 2025-02-20
| Title |
Authors |
Summary |
| Qwen2.5-VL Technical Report (Read more on arXiv or HuggingFace) |
Keqin Chen, Shuai Bai, xhyandwyy, darkpromise, ayumiymk |
i) Qwen2.5-VL is a new vision-language model in the Qwen series with advancements in visual recognition, object localization, document parsing, and long-video comprehension. ii) The research aims to improve the foundational and agentic capabilities of vision-language models, particularly in fine-grained visual perception and real-world applications. iii) The methodology involves training a native dynamic-resolution Vision Transformer (ViT) from scratch, incorporating Window Attention, dynamic FPS sampling, absolute time encoding with MROPE, and curating a large pre-training dataset of 4.1 trillion tokens. iv) The Qwen2.5-VL-72B model achieves 74.8 on MathVista and mIoU score of 50.9 on Charades-STA, and matches state-of-the-art performance, while smaller models offer strong capabilities in resource-constrained environments. v) AI practitioners can leverage Qwen2.5-VL’s improved document understanding, precise object grounding, and long-video comprehension to develop more robust and versatile multimodal applications, particularly in domains requiring detailed visual analysis and interactive agent functionalities, with attention to the computational benefits conferred by Window Attention and dynamic resolution processing. |
| RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning (Read more on arXiv or HuggingFace) |
Yiang Shi, Bencheng Liao, Bo Jiang, Shaoyu Chen, Hao605 |
RAD establishes a 3DGS-based closed-loop Reinforcement Learning (RL) paradigm for training end-to-end autonomous driving policies. The main research objective is to address causal confusion and the open-loop gap in existing Imitation Learning (IL) methods for autonomous driving. The key methodology involves constructing photorealistic digital replicas of the real world using 3D Gaussian Splatting (3DGS) techniques, incorporating IL as a regularization term in RL training, and designing specialized safety-related rewards. The primary results show that, compared to IL-based methods, RAD achieves a 3x lower collision rate on a closed-loop evaluation benchmark consisting of unseen 3DGS environments. For AI practitioners, this suggests that 3DGS-based RL training, combined with IL, can improve the safety and robustness of end-to-end autonomous driving policies, by allowing large scale training in a realistic virtual world. |
| SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (Read more on arXiv or HuggingFace) |
Pan Zhang, Xiaoyi Dong, Zhixiong Zhang, Shuangrui Ding, Zihan Liu |
SongGen is a single-stage auto-regressive transformer model for generating songs with vocals and accompaniment from text inputs. The main research objective is to investigate whether a single-stage model can achieve effective text-to-song generation, simplifying the often cumbersome multi-stage pipelines. The key methodology involves a transformer decoder that predicts audio tokens, incorporating user controls via cross-attention, and exploring mixed and dual-track output modes with diverse token patterns. Primary results show that the “Interleaving (A-V)” dual-track mode achieves a Frechet Audio Distance (FAD) of 1.87, competitive with mixed-mode generation. AI practitioners can use SongGen as an open-source, controllable baseline for text-to-song generation, and the provided annotated data and preprocessing pipeline simplify future research. |
| MoM: Linear Sequence Modeling with Mixture-of-Memories (Read more on arXiv or HuggingFace) |
Yu Cheng, Jiaxi Hu, Disen Lan, Jusen Du, weigao266 |
MoM introduces a linear sequence modeling architecture that uses multiple memory states to improve recall performance. The main research objective is to enhance the memory capacity and reduce memory interference in linear sequence models, addressing limitations of existing approaches that compress sequences into a single fixed-size state. The methodology involves a Mixture-of-Memories (MoM) architecture with multiple independent memory states and a router network that directs input tokens to specific memory states, using an RNN-like update mechanism. Primary results show that MoM significantly outperforms current linear sequence models on downstream language tasks, with the 1.3B parameter MoM achieving an average score of 36.04 on recall-intensive tasks, close to the Transformer model’s 37.31. For AI practitioners, MoM offers a more efficient architecture to enhance the memory and recall of linear sequence modeling for applications, retaining linear-time training and constant-memory inference, presenting itself as an alternative to Transformers. |
| Craw4LLM: Efficient Web Crawling for LLM Pretraining (Read more on arXiv or HuggingFace) |
Chenyan Xiong, Zhiyuan Liu, yushi |
CRAW4LLM is an efficient web crawling method that prioritizes webpages based on their predicted influence on large language model (LLM) pretraining. The research objective is to improve the efficiency of web crawling for LLM pretraining data collection by aligning crawler priorities with LLM pretraining needs. The key methodology is to use a pretraining influence scorer, derived from data-filtering pipelines, to score newly discovered documents and prioritize them in the crawler’s queue, replacing traditional graph-connectivity-based metrics. Primary results show that LLMs pretrained on data crawled by CRAW4LLM, using only 21% of the URLs, achieve the same downstream performance as previous crawls that used more data. Principal implication is that by using CRAW4LLM AI practitioners can get similar performing LLM, while significantly reducing the required web crawling and data processing, thus saving time and resources. |
| LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization (Read more on arXiv or HuggingFace) |
Lidong Bing, Michael Qizhe Shieh, Xin Li, Guanzheng Chen |
LongPO is a method that enables short-context LLMs to self-evolve to handle long-context tasks by internally transferring short-context capabilities through preference optimization. The main research objective is to address the challenges of long-context alignment in LLMs, specifically the scarcity of long-context annotated data and the difficulty in balancing short- and long-context performance. The key methodology involves generating short-to-long preference data using a short-context LLM and applying a DPO-style objective with a KL constraint to maintain short-context performance. The primary result is that LongPO applied to Mistral-7B-Instruct-v0.2 improved performance on InfiniteBench by 25.45 points and achieved comparable or superior results to larger LLMs like GPT-4-128K. The principal implication for AI practitioners is that LongPO offers an efficient way to extend the context length of LLMs without extensive long-context data annotation or significant degradation of short-context capabilities, providing a more balanced approach to developing long-context LLMs. |
| Small Models Struggle to Learn from Strong Reasoners (Read more on arXiv or HuggingFace) |
Luyao Niu, Fengqing Jiang, Xiang Yue, Yuetai Li, flydust |
Small language models (≤3B parameters) do not consistently benefit from complex reasoning data or distillation from larger models, instead performing better with simpler reasoning. The main research question is whether small language models can effectively learn from the reasoning capabilities of larger, more powerful language models. The key methodology involves fine-tuning student models of varying sizes on different types of Chain-of-Thought (CoT) data (long, short, large teacher, small teacher) generated from the MATH dataset and evaluating their performance on multiple math benchmarks. A key result is that Qwen2.5-3B-Instruct improves by more than 8 points on MATH and AMC using Mix-Long, compared to direct training on long CoT data. The principal implication is that AI practitioners should adapt reasoning complexity during distillation, using techniques like Mix Distillation, to effectively transfer reasoning capabilities to smaller models, instead of directly using complex reasoning data from large models. |
| Autellix: An Efficient Serving Engine for LLM Agents as General Programs (Read more on arXiv or HuggingFace) |
Tianjun Zhang, Colin Cai, Xiaoxiang Shi, Michael Luo, Chrisyichuan |
Autellix is an LLM inference system designed to efficiently serve agentic programs, treating them as first-class citizens to minimize end-to-end latency. The main research objective is to reduce the end-to-end latencies of agentic programs composed of dynamic, non-deterministic DAGs of LLM calls and interrupts. The key methodology used is program-aware scheduling, prioritizing LLM calls based on program-level statistics (cumulative service time) and employing a data locality-aware load balancer across multiple engines. Primary results show that Autellix improves program throughput by 4-15x compared to state-of-the-art systems like vLLM, across diverse LLMs and agentic workloads. The principal implication is that AI practitioners can significantly improve the performance of LLM agent applications by using a serving system that prioritizes the scheduling of LLM calls based on full program execution, and data-locality, rather than treating each call independently. |
| Presumed Cultural Identity: How Names Shape LLM Responses (Read more on arXiv or HuggingFace) |
Lucie-Aimée Kaffee, Arnav Arora, Siddhesh Pawar, IAugenstein |
LLMs exhibit cultural biases in responses based on user names, influencing personalization. The main research objective is to investigate cultural presumptions in LLM responses when presented with common suggestion-seeking queries including user names. The key methodology involves prompting LLMs with names from 30 cultures and analyzing generated responses for cultural bias using an LLM-as-a-judge approach and assertion-based evaluation. The primary result showed that LLM responses exhibit varying degrees of cultural bias, with clothing-related queries showing a roughly 70% increase in bias when names were included. Principal implication is that AI practitioners need to consider the impact of names on LLM outputs and design personalisation systems that avoid reinforcing stereotypes while utilizing names. |
| Why Safeguarded Ships Run Aground? Aligned Large Language Models’ Safety Mechanisms Tend to Be Anchored in The Template Region (Read more on arXiv or HuggingFace) |
Wenjie Li, Jian Wang, Qingyu Yin, Chak Tou Leong |
Aligned large language models (LLMs) exhibit a vulnerability where their safety mechanisms overly rely on information within a specific “template region” inserted between user input and model output. The research investigates the phenomenon of “template-anchored safety alignment” (TASA) in aligned LLMs. The methodology involves analyzing attention weight distributions, performing activation patching interventions, and probing harmfulness features across different layers and positions, and propose a detaching safety mechanism. Results show that intervening in intermediate states in template region significantly increases the likelihood of harmful initial compliance decisions, with a normalized indirect effect (NIE) showing considerable gains by patching small number of heads. The findings suggest AI practitioners should develop more robust safety alignment techniques that are less reliant on the template region for safety-related decision-making to reduce the risk of adversarial attacks. |
| SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering? (Read more on arXiv or HuggingFace) |
Tianming Liu, Quanzheng Li, Canyu Chen, Tianze Yang, YuchengShi |
SearchRAG is a novel retrieval-augmented generation framework that leverages search engines to enhance large language models’ (LLMs) performance in medical question answering. The main research objective is to determine how to effectively integrate search engines with LLMs for improved retrieval of medical knowledge. The key methodology involves synthetic query generation using LLMs to create search-engine-friendly queries and uncertainty-based knowledge selection to filter retrieved information. Primary results show that SearchRAG improved the LLaMA 8B model’s accuracy by an average of 12.61% compared to baseline methods on medical QA tasks. Principal implication for AI practitioners is that SearchRAG’s method is capable of adressing limitations of conventional Retrieval-Augmented Generation (RAG) systems, showing that real-time search integration improves response accuracy. |
| Thinking Preference Optimization (Read more on arXiv or HuggingFace) |
Xiaotian Han, Vipin Chaudhary, Jingfeng Yang, Hongye Jin, Wang Yang |
Thinking Preference Optimization (ThinkPO) enhances reasoning in fine-tuned language models without requiring new long chain-of-thought (CoT) responses. The main research objective is to improve the reasoning performance of supervised fine-tuned (SFT) language models without collecting new long CoT data or repeatedly training on existing SFT datasets. The key methodology is to use readily available short CoT reasoning responses as rejected answers and existing long CoT responses as chosen answers, applying direct preference optimization (DPO) to encourage longer reasoning outputs. The primary result is that ThinkPO increases the math reasoning accuracy of SFT-ed models by 8.6% and output length by 25.9%, for example it increased performance on MATH500 of a tested model from 87.4% to 91.2%. AI practitioners can use ThinkPO as a post-SFT method to further improve the reasoning performance of their models, especially when acquiring new long CoT data is costly or repeated training leads to a performance plateau. |
| Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering (Read more on arXiv or HuggingFace) |
Benjamin Van Durme, Jeffrey Cheng, wjurayj |
Test-time scaling of compute improves the performance of large language models on selective question answering by increasing confidence in correct answers. The research investigates how increasing computational budget at inference time impacts model confidence and accuracy in question answering. The methodology involves evaluating models at varying compute budgets and confidence thresholds, using a selection function that rejects answers below a confidence threshold. The results show that increasing the compute budget improves the average confidence of correct answers, and selective answering at a threshold of 0.95 dramatically improves performance in a Jeopardy setting where incorrect answers are penalized. AI practitioners should report test-time scaling performance under conditions that penalize incorrect answers (“Jeopardy Odds”) in addition to traditional settings, to accurately reflect selective question answering capabilities. |
| AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence (Read more on arXiv or HuggingFace) |
Jason Klein Liu, Chaofeng Qu, Zhaoling Chen, Junjie Lu, Yuliang Liu |
AdaptiveStep, a novel method, automatically divides reasoning steps in large language models (LLMs) based on model confidence to enhance process reward model (PRM) training and performance. The main research objective is to develop an automated, informative, and general method for dividing reasoning steps that improves upon existing rule-based approaches. The key methodology, AdaptiveStep, utilizes the LLM’s prediction confidence for the next word to identify critical breaking points, creating more informative step divisions without manual annotation. Results show that the AdaptiveStep-trained PRM (ASPRM) achieves state-of-the-art Best-of-N performance, outperforming greedy search with token-level value-guided decoding (TVD) by 3.15% on GSM8k. For AI practitioners, AdaptiveStep provides a more efficient and precise method for training PRMs, reducing construction costs and enhancing downstream task performance, specifically in mathematical reasoning and code generation. |
| NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation (Read more on arXiv or HuggingFace) |
Enzhi Zhang, Han Huang, Yanchen Luo, Zhiyuan Liu, xiangwang1223 |
NExT-Mol is a foundation model for 3D molecule generation that combines 3D diffusion with 1D language modeling. The main research objective is to improve 3D molecule generation by integrating the strengths of 1D SELFIES-based language models (LMs) and 3D diffusion models. The methodology involves pretraining a 960M parameter 1D molecule LM (MoLlama) on 1.8B SELFIES, then predicting 3D conformers with a novel diffusion model (Diffusion Molecule Transformer, DMT) and using cross-model transfer learning to enhance DMT. NExT-Mol achieves a 26% relative improvement in 3D FCD for de novo 3D generation on GEOM-DRUGS compared to previous methods. AI practitioners can leverage this approach to generate 3D molecules with improved validity and distributional similarity, facilitating drug discovery and material design by combining large-scale 1D pretraining with 3D diffusion. |
| ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation (Read more on arXiv or HuggingFace) |
Wang-Cheng Kang, Noveen Sachdeva, Zhankui He, Jianmo Ni, hyp1231 |
ActionPiece is a novel tokenization method for generative recommendation that incorporates contextual information to improve performance. The main research objective is to develop a context-aware action sequence tokenizer for generative recommendation models, addressing the limitation of existing models that tokenize each action independently. The key methodology, ActionPiece, represents each action as a set of item features, constructs a vocabulary by merging frequent feature patterns, and uses set permutation regularization to produce multiple segmentations. The primary result is that ActionPiece outperforms existing action tokenization methods, improving NDCG@10 by 6.00% to 12.82% on public datasets. The principal implication is that AI practitioners can use ActionPiece to improve the accuracy and efficiency of generative recommendation systems by considering contextual relationships among user actions. |
| Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models (Read more on arXiv or HuggingFace) |
Ke Chen, Lidan Shou, Huan Li, Jue Wang, junzhang98 |
LORAM is introduced as a memory-efficient LoRA training scheme for LLMs. This research aims to reduce the memory footprint of LoRA training by training on a pruned model and recovering weights for inference on the original model. LORAM employs pruning during training followed by a recovery and alignment phase utilizing continual pre-training on a small dataset. QLORAM, combining structured pruning and 4-bit quantization, achieved a 15.81× parameter storage reduction for LLaMA-3.1-70B while maintaining or improving performance. LORAM enables training on resource-constrained hardware and suggests an alternative to full fine-tuning. |
| GIMMICK – Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking (Read more on arXiv or HuggingFace) |
Anne Lauscher, Chris Biemann, Carolin Holtermann, floschne |
i) GIMMICK introduces a multimodal benchmark for evaluating cultural knowledge in large vision-language models (LVLMs). ii) The research aims to identify regional biases in LLMs’ and LVLMs’ cultural understanding and assess the impact of model size, input modalities, and external cues on cultural knowledge. iii) The methodology employs six tasks built on three newly created datasets spanning 728 cultural events across 144 countries, evaluating 31 models using multimodal and unimodal inputs. iv) Results reveal significant regional biases, with models exhibiting up to 14.72pp performance difference between Western and Sub-Saharan African cultural contexts, and multimodal input consistently improving performance. v) AI practitioners should be aware of biases in cultural understanding and leverage multimodal inputs to create more globally inclusive AI systems. |
| InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning (Read more on arXiv or HuggingFace) |
Zhijie Sang, Pengxiang Li, Wenjun Wang, Shuo Cai, Congkai Xie |
InfiR introduces efficient Small Language Models (SLMs) and Multimodal SLMs with enhanced reasoning capabilities, deployable on edge devices. The main research objective is to develop SLMs and MSLMs that retain competitive reasoning abilities while reducing model size and computational demands. The key methodology involves a novel pre- and post-training pipeline that includes heuristic filtering, reasoning-oriented text recall, data annealing, and supervised fine-tuning with synthetic data. The InfiR-1B-Instruct model achieved a 2.26x reasoning-related average score improvement over Llama3.2-1B-Base. AI practitioners can leverage InfiR’s training pipeline and models to build efficient and privacy-preserving AI systems with strong reasoning capabilities, particularly for edge deployment. |
| Noise May Contain Transferable Knowledge: Understanding Semi-supervised Heterogeneous Domain Adaptation from an Empirical Perspective (Read more on arXiv or HuggingFace) |
Qiang Yang, Jian Jin, Yu Zhang, Xiaopu Zhang, yyyaoyuan |
This paper empirically investigates transferable knowledge in semi-supervised heterogeneous domain adaptation (SHDA) tasks. The main research question is: “What is the transferable knowledge in SHDA?” The authors develop a unified Knowledge Transfer Framework (KTF) for SHDA and conduct extensive experiments, including manipulating source sample categories, features, and introducing synthesized noise distributions. A primary result across nearly 330 SHDA tasks is that varying source sample category orders has almost no change in the performance, i.e. average accuracy remains nearly constant. For AI practitioners, the results imply that the discriminability and transferability of the source domain, rather than the category or feature information, are the main factors for effective transfer in SHDA, meaning the choice of origin for source domains is less critical than ensuring those two qualities. |
Papers for 2025-02-19
| Title |
Authors |
Summary |
| Soundwave: Less is More for Speech-Text Alignment in LLMs (Read more on arXiv or HuggingFace) |
Benyou, PhoenixAxis, FanBuCUHK, puccho, Yoohao |
Soundwave utilizes an efficient training strategy and novel architecture to address representation space gap and sequence length inconsistency between speech and text in LLMs. The main research objective is to achieve data-efficient training for speech-text alignment in large language models. The key methodology is a two-stage training framework: Stage I aligns speech and text representations using an alignment adapter and CTC loss; Stage II reduces speech sequence length using a shrinking adapter. Soundwave outperforms Qwen2-Audio in speech translation and AIR-Bench speech tasks, using only one-fiftieth of the training data (10k hours vs. 520k hours). AI practitioners can achieve state-of-the-art speech understanding performance in LLMs with significantly reduced training data requirements by adopting Soundwave’s two-stage alignment and shrinking approach. |
| Phantom: Subject-consistent video generation via cross-modal alignment (Read more on arXiv or HuggingFace) |
Jiawei Liu, ZhuoweiChen, lbc402, Grayson111, liulj13 |
Phantom is a unified video generation framework for subject-consistent video generation via cross-modal alignment. The research objective is to develop a model that balances dual-modal prompts of text and image to achieve deep and simultaneous alignment of text and visual content in video generation. The key methodology involves redesigning a joint text-image injection model based on text-to-video and image-to-video architectures, and training it with text-image-video triplet data to learn cross-modal alignment. Primary results show Phantom leads in overall metrics for subject consistency with a score of 0.731 in CLIP-I-Seg and prompt following with the ViCLIP-T, demonstrating subject consistency competitive with commercial solutions. AI practitioners can use Phantom, which has a new architecture, for improved subject-consistent video generation, especially in tasks requiring ID preservation and consistency. |
| Continuous Diffusion Model for Language Modeling (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, harryjo97 |
Riemannian Diffusion Language Model (RDLM) is a continuous diffusion framework for language modeling that incorporates the geometry of the statistical manifold. The main research objective is to establish a connection between discrete diffusion and continuous flow on the statistical manifold and design a continuous diffusion model for discrete data that generalizes previous discrete diffusion models. The key methodology involves reparameterizing discrete data to continuous states on a hypersphere, designing diffusion processes on the manifold that generalize discrete diffusion, and using a simulation-free training scheme based on radial symmetry. Primary results show that RDLM achieves a Bits Per Character (BPC) of ≤ 1.32 on the Text8 dataset, outperforming existing discrete diffusion models. The principal implication is that AI practitioners can leverage the geometry of the statistical manifold in continuous diffusion models to achieve improved performance in language modeling and other discrete data generation tasks, compared to existing discrete diffusion approaches. |
| Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity (Read more on arXiv or HuggingFace) |
Aydar Bulatov, Mikhail Arkhipov, mbur, yurakuratov |
This work explores the maximum information capacity of language model input embeddings by compressing text sequences into trainable vectors. The main research objective is to quantify how much text can be losslessly encoded into and decoded from a fixed-size vector representation within large language models (LLMs). The key methodology involves optimizing a set of prepended “memory” vectors to minimize the cross-entropy loss when reconstructing the original text using a frozen, pre-trained LLM. The primary result is that a single vector can enable a Llama-3.1-8B model to accurately reconstruct up to 1568 tokens, and this capacity scales nearly linearly with the number of trainable vectors (e.g. 16 vectors compress 7168 tokens). The principal implication for AI practioners is that LLM input embeddings have significantly more unused capacity than typically utilized, suggesting substantial room for improved context encoding and memory augmentation in model design. |
| SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models (Read more on arXiv or HuggingFace) |
Minki Kang, Dong Bok Lee, hbseong, dwgnr, Seanie-lee |
SafeRoute adaptively selects between a smaller and larger safety guard model to improve the trade-off between computational cost and safety performance in LLM deployments. The paper’s objective is to develop a method that distinguishes “hard” examples requiring a larger safety guard model from “easy” ones that a smaller model can handle. The core of the method is SafeRoute, a trained binary router that classifies input prompt-response pairs, selectively applying the larger model only when necessary. Results show SafeRoute improves the F1 score by 13% and 10% compared to always using the smaller or larger models on the WildGuardMix test split, while utilizing the larger model on only 5.09% of the data. AI practitioners can use SafeRoute to deploy safer LLMs more efficiently, reducing computational overhead while maintaining high accuracy in detecting harmful content. |
| Rethinking Diverse Human Preference Learning through Principal Component Analysis (Read more on arXiv or HuggingFace) |
Hao Sun, Feng Luo, huanzhang12, CharlesDDDD, Ray2333 |
Decomposed Reward Models (DRMs) extract diverse human preferences from binary comparisons for improved AI personalization. The research question is: Can we infer multidimensional human preferences directly from large-scale binary comparisons? The method represents preferences as vectors, applies PCA to embedding differences between preferred and rejected responses, and identifies orthogonal basis vectors representing distinct preference aspects. DRMs using Gemma-2B-RM improved the single-head baseline accuracy from 0.733 to 0.814 on the RewardBench dataset. AI practitioners can use DRMs for more efficient test-time adaptation to diverse user preferences without requiring additional model training, offering a scalable and interpretable solution for personalized LLM alignment. |
| SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation (Read more on arXiv or HuggingFace) |
codered010, RunpeiDong, YufeiD, WenyaoZhang, qizekun |
SOFAR introduces semantic orientation to bridge spatial reasoning and object manipulation, enabling robots to understand and execute tasks based on natural language instructions. The main research objective is to develop a system that can accurately understand and utilize object orientations, defined through natural language, for robotic manipulation and spatial reasoning tasks. The key methodology involves constructing a large-scale dataset (OrienText300K) of 3D models annotated with semantic orientations, developing a cross-modal 3D Transformer (PointSO) for orientation prediction, and integrating this with a Vision-Language Model (VLM) system (SOFAR) to generate manipulation actions. Primary results show that SOFAR achieves 48.7% accuracy on the Open6DOR benchmark and 74.9% accuracy on the SIMPLER benchmark for robotic manipulation. The principal implication for AI practitioners is that integrating semantic orientation into VLM systems provides a more flexible and accurate way to represent spatial knowledge, significantly improving performance in robotic manipulation tasks requiring precise object alignment and rearrangement. |
| Multimodal Mamba: Decoder-only Multimodal State Space Model via Quadratic to Linear Distillation (Read more on arXiv or HuggingFace) |
Qian Zhang, wenyuliu, wondervictor, HongyuanTao, LegendBC |
mmMamba is a framework for developing linear-complexity, native multimodal state space models using distillation from existing multimodal large language models (MLLMs). The main research question is how to effectively distill knowledge from trained Transformer-based decoder-only MLLMs to create efficient, linear-complexity architectures without relying on pre-trained RNN-based LLMs or vision encoders. The key methodology involves a three-stage progressive distillation recipe and a seeding strategy to carve Mamba layers from trained Transformer layers, transferring knowledge while preserving multimodal capabilities. The primary results demonstrate that mmMamba-linear achieves competitive performance with existing linear and quadratic-complexity VLMs, achieving a 20.6x speedup and 75.8% GPU memory saving compared to HoVLE at 103K tokens. AI practitioners can leverage mmMamba to build more efficient and deployable multimodal models, particularly for long-context applications, by utilizing linear-complexity architectures with reduced computational demands. |
| FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading (Read more on arXiv or HuggingFace) |
ShirleyY, Acatsama, YupengCao, zdeng10, xionggj001 |
FLAG-TRADER is a framework integrating LLMs with reinforcement learning for financial trading. The main research question is whether integrating LLMs’ reasoning with RL’s reward-driven optimization can address challenges in financial sequential decision-making. The methodology involves a partially fine-tuned LLM acting as a policy network, optimized via gradient-driven RL (specifically PPO), using textual state representations. Primary results show FLAG-TRADER, using a 135M-parameter LLM, achieves a Sharpe Ratio of 3.344 on JNJ stock, outperforming baselines and larger proprietary models. For AI practitioners, this framework demonstrates that combining LLMs with RL fine-tuning, particularly using parameter-efficient methods, offers superior performance in complex, sequential decision-making tasks like financial trading. |
| You Do Not Fully Utilize Transformer’s Representation Capacity (Read more on arXiv or HuggingFace) |
kefirski, ummagumm-a, elephantmipt, yaraksen, gudleifrr |
i) This paper introduces Layer-Integrated Memory (LIMe), a modification to the Transformer architecture that allows attention heads to access representations from all previous layers. ii) The main objective is to address representation collapse in standard Transformers by enabling access to hidden states from earlier layers. iii) The key methodology is modifying the key-value side of masked multi-head self-attention by introducing a learned routing mechanism (static or dynamic) that creates convex combinations of representations from all preceding layers. iv) LIMe models consistently outperform standard Transformer baselines; for example, on the LM Evaluation Harness, the average accuracy across all benchmarks in the results shows the LIMe Dynamic variant achieving 58.4% accuracy, compared to 57.7% for the LLaMA baseline. v) AI practitioners can use LIMe to build deeper and more robust Transformers with improved representational capacity, potentially leading to better performance in sequence modeling tasks without substantially increasing computational overhead. |
| Magma: A Foundation Model for Multimodal AI Agents (Read more on arXiv or HuggingFace) |
cheryyunl, Baolin, rzheng12, qianhuiwu, tanreuben |
Magma is a multimodal foundation model capable of interpreting and grounding multimodal inputs within its environment for AI agentic tasks. The main research objective is to develop a foundation model that integrates vision-language understanding with the ability to plan and act in visual-spatial worlds, completing tasks ranging from UI navigation to robot manipulation. The key methodology involves pre-training on heterogeneous datasets (images, videos, robotics data) using Set-of-Mark (SoM) for action grounding and Trace-of-Mark (ToM) for action planning, representing actions as visual object labels and movement traces. Primary results include achieving new state-of-the-art results on UI navigation with a success rate of 60.4/58.5 on SS-Mobile, and robotic manipulation tasks, outperforming previous models tailored to these tasks. For AI practitioners, Magma provides a pre-trained model capable of transferring visual and language understanding to complex agentic tasks, suggesting a path for building agents that can seamlessly operate in both digital and physical environments. |
| RealSyn: An Effective and Scalable Multimodal Interleaved Document Transformation Paradigm (Read more on arXiv or HuggingFace) |
Kaicheng Yang, JiankangDeng, SeriousBro, Nina0607, GaryGuuu |
i) RealSyn introduces a paradigm for vision-language representation learning using multimodal interleaved documents. ii) The research aims to leverage underutilized non-paired data in interleaved documents by constructing distinct image-text pairs. iii) The methodology involves a real-world data extraction pipeline, hierarchical retrieval to associate images with texts, and an image semantic augmented generation module. iv) The study releases the RealSyn dataset and demonstrates that models pre-trained on RealSyn achieve state-of-the-art performance on multiple downstream tasks and showed performance improvements of 1.3%-6.9% in linear probing. v) RealSyn offers a scalable dataset, up to 100M, for AI practitioners enabling improved vision-language models without relying solely on paired data. |
| PAFT: Prompt-Agnostic Fine-Tuning (Read more on arXiv or HuggingFace) |
Fei Richard Yu, Ying Tiffany He, Mingwen Ou, Yao Shu, kittttttt |
PAFT is a fine-tuning method that improves the prompt robustness of large language models (LLMs). The main research objective is to address the performance degradation of fine-tuned LLMs caused by minor variations in prompts. The key methodology is a two-stage approach: constructing a diverse set of candidate prompts and then dynamically sampling from these prompts during fine-tuning. Primary results show that PAFT achieves 87.57% average accuracy on the RACE-high dataset, significantly outperforming baseline models and reducing variance across different prompts. PAFT’s dynamic sampling during fine-tuning helps models generalize better to unseen prompts, maintaining high performance and improving inference efficiency for AI practitioners using fine-tuned models. |
| MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections (Read more on arXiv or HuggingFace) |
Xingyuan Yuan, Da Xiao, lishengping, Hilbertmeng |
MUDDFormer introduces a novel method to improve information flow in Transformers by replacing standard residual connections with multiway dynamic dense connections. The main research objective is to address the limitations of residual connections and enhance cross-layer information flow in Transformer models. The key methodology is generating connection weights dynamically based on hidden states and decoupling input streams (query, key, value, residual) of a Transformer block. Primary results show that MUDDPythia-2.8B matches Pythia-6.9B in pre-training perplexity and downstream tasks, while adding only 0.23% parameters and 0.4% computation. For AI practitioners, MUDDFormer offers a method to significantly improve Transformer performance and scalability, especially with deeper models, with minimal parameter and computational overhead. |
| Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities? (Read more on arXiv or HuggingFace) |
Yunhua Zhou, Qinyuan Cheng, Zhiyuan Zeng, xpqiu, yinzhangyue |
This paper investigates whether o1-like models (QwQ, R1, and LIMO) truly possess test-time scaling capabilities. The main research question is whether increasing Chain-of-Thought (CoT) length in these models consistently improves reasoning performance. The researchers systematically investigated the relationship between CoT length and accuracy, and prompted models for self-revisions, comparing sequential and parallel scaling strategies. A primary result is that longer CoTs did not consistently improve accuracy; correct solutions were often shorter, and R1-Distill-32b and R1-Distill-14b maintained the original wrong answer in over 70% of cases when prompted to revise. The principal implication is that AI practitioners should consider parallel scaling and methods like “Shortest Majority Vote” for these models, as sequential scaling via self-revision is not consistently effective due to limited self-revision capabilities. |
| OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning (Read more on arXiv or HuggingFace) |
Joseph Boen, Rahul Thapa, Sheng Liu, Bowen Chen, lupantech |
OctoTools is a training-free, extensible agentic framework that enhances complex reasoning in large language models (LLMs) through standardized tool integration and a planner-executor paradigm. The main research objective is to develop a framework that enables LLMs to effectively tackle complex reasoning tasks across diverse domains without requiring additional training or fine-tuning. Key methodology involves using standardized tool cards to encapsulate tool functionality, a planner for high-level and low-level task planning, and an executor to carry out tool usage based on generated commands. Primary results show that OctoTools achieves an average accuracy gain of 9.3% over zero-shot GPT-4o and outperforms other agent frameworks like AutoGen, GPT-Functions, and LangChain by up to 10.6% when given the same set of tools. Principal implication for AI practitioners is that OctoTools provides a modular and extensible framework for building AI agents capable of complex reasoning, which reduces development effort and improves performance without the need for model retraining when new tools are added. |
| Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge (Read more on arXiv or HuggingFace) |
zhangsan5421, lifengshang, horiz94, YuxinJiang, DonJoey |
Crowd Comparative Reasoning enhances LLM-as-a-Judge evaluations by incorporating comparisons with multiple “crowd” responses to improve detail and comprehensiveness. Research Objective: To address the limitation of LLM-as-a-Judge’s chain-of-thought (CoT) reasoning, which often fails to capture comprehensive details, leading to incomplete evaluations. Key Methodology: Proposes Crowd-based Comparative Evaluation (CCE), which introduces additional “crowd” responses for comparison with candidate responses, guiding the LLM to produce more detailed CoT judgments. Primary Results: CCE achieved an average accuracy gain of 6.7% across five benchmarks (REWARDBENCH, HELPSTEER2, MTBENCH HUMAN, JUDGEBENCH, and EvalBIAS). Principal Implication: AI practitioners can use CCE to improve the reliability and depth of LLM-based evaluations, enabling more robust model assessments and potentially more efficient training through techniques like judge distillation and improved rejection sampling. |
| HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation (Read more on arXiv or HuggingFace) |
Binhe Yu, Yuqian Yuan, Sijing Li, Wenqiao Zhang, Tianwei Lin |
HealthGPT is a medical large vision-language model that unifies visual comprehension and generation tasks through heterogeneous knowledge adaptation. The main research objective is to develop a unified medical multi-modal model capable of both comprehending and generating medical visual data. The key methodology is a novel heterogeneous low-rank adaptation (H-LoRA) technique, complemented by hierarchical visual perception and a three-stage learning strategy. Results show that HealthGPT-L14 achieves 77.7% close accuracy on VQA-RAD, and 88.6% SSIM on the CT(Brain) reconstruction task. The principal implication is that AI practitioners can leverage HealthGPT’s architecture for creating unified medical AI models that perform well on both visual comprehension and generation, overcoming limitations of previous models. |
| HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading (Read more on arXiv or HuggingFace) |
beidic, junjiehu, jinqixiao, ZefanCai, wdlctc |
i) HeadInfer proposes a head-wise offloading strategy for memory-efficient LLM inference by selectively maintaining attention heads’ KV cache on the GPU. ii) The research aims to reduce the GPU memory footprint of LLM inference, specifically the key-value (KV) cache, for long context generation. iii) The methodology involves a head-wise offloading strategy where only selective attention heads’ KV cache is stored on the GPU, dynamically computing attention output, combined with adaptive heads grouping and asynchronous data transfer. iv) Experiments on the Llama-3-8B model with a 1-million-token sequence show a reduction in GPU memory footprint from 128GB to 1GB for the KV cache and total GPU usage from 207GB to 17GB, achieving a 92% reduction compared to BF16 baseline inference; HeadInfer extends the Llama-3-8B model’s context length from 25K to 4 million tokens using an NVIDIA RTX 4090. v) HeadInfer enables AI practitioners to perform long-context LLM inference with reduced memory requirements, specifically enabling 4-million-token inference with an 8B model on a single consumer GPU with 24GB memory. |
| Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey (Read more on arXiv or HuggingFace) |
Mingzhe Li, Miao Fang, Yuhan Liu, Bin Yan, Ziruibest |
This survey provides a comprehensive overview of methods for integrating domain-specific knowledge into large language models (LLMs). The main research objective is to categorize and analyze techniques for enhancing LLMs with domain-specific knowledge to improve their performance in specialized tasks. Key methodologies include dynamic knowledge injection, static knowledge embedding, modular adapters, and prompt optimization. The paper reviewed studies showing, for instance, that PMC-LLaMA (13B) achieved 56.3 on MedQA, outperforming LLaMA2 (70B) at 43.7 on the same benchmark, in the biomedical field, showing how domain-specific LLMs can beat generalized models. For AI practitioners, incorporating domain-specific knowledge is crucial for achieving higher accuracy and reliability in specialized applications of LLMs. |
| Eager Updates For Overlapped Communication and Computation in DiLoCo (Read more on arXiv or HuggingFace) |
Yanislav Donchev, Arthur Douillard, Satyen Kale |
i) This paper introduces “eager updates” to improve the DiLoCo distributed training method by overlapping communication and computation, reducing training time in low-bandwidth settings. ii) The main objective is to mitigate performance slowdowns in distributed training caused by blocking communication in low-bandwidth environments, such as cross-datacenter training. iii) The key methodology is to overlap the communication of outer gradients with the computation of the next inner optimization phase, applying local outer gradients eagerly before the aggregated gradients are available. iv) The proposed method with 1-outer-step eager updates and H=30 inner steps achieves the same performance as Data-Parallel at a 1 billion parameter scale, while using up to 1,177x less bandwidth. v) AI practitioners can use eager updates in DiLoCo to significantly reduce communication requirements and improve training efficiency in settings with limited bandwidth between workers. |
| Atom of Thoughts for Markov LLM Test-Time Scaling (Read more on arXiv or HuggingFace) |
Chenglin Wu, Jiayi Zhang, Quan Shi, Zhaoyang Yu, leavendough |
Atom of Thoughts (AOT) is a reasoning framework that improves large language models’ (LLMs) test-time scaling by structuring the reasoning process as a Markov chain of atomic, independent questions. The main research objective is to address the issue of accumulated historical information in existing test-time scaling methods, which wastes computational resources and interferes with effective reasoning. The key methodology is a two-phase state transition mechanism: (1) decomposing the current question into a dependency-based directed acyclic graph, and (2) contracting subquestions into a new independent question, iteratively until directly solvable. Primary results show that on HotpotQA, AOT applied to gpt-4o-mini achieves an 80.6% F1 score. The principal implication for AI practitioners is that AOT can be used as a standalone framework or a plug-in enhancement to improve LLMs’ reasoning capabilities, by reducing unnecessary historical information to enhance efficiency. |
| FinMTEB: Finance Massive Text Embedding Benchmark (Read more on arXiv or HuggingFace) |
Yi Yang, yixuantt |
FinMTEB is a comprehensive benchmark for evaluating text embedding models in the financial domain. The main research objective is to assess how well existing embedding models capture domain-specific financial information and whether domain adaptation improves performance. The key methodology involves constructing a benchmark (FinMTEB) of 64 datasets across 7 financial tasks and developing a finance-adapted model, Fin-E5, using a persona-based data synthesis method. Primary results show domain-adapted models consistently outperform general-purpose counterparts, with Fin-E5 achieving a 0.6767 average score on FinMTEB, and remarkably, a simple Bag-of-Words (BoW) approach outperforms all dense embedding in financial Semantic Textual Similarity (STS) tasks. For AI practitioners, the benchmark facilitates targeted development and assessment of financial text embedding models, and also suggests current dense embedding models may not be optimal for certain kinds of financial text analysis. |
| Perovskite-LLM: Knowledge-Enhanced Large Language Models for Perovskite Solar Cell Research (Read more on arXiv or HuggingFace) |
Shuyan Chen, wenxinsiju, yongqi2023, sunpenglei, Dominic789654 |
This paper presents a knowledge-enhanced system for perovskite solar cell (PSC) research, integrating a knowledge graph, datasets, and specialized large language models. The main research objective is to develop a system that efficiently manages and reasons with the rapidly growing body of knowledge in PSC research. The key methodology involves constructing a domain-specific knowledge graph (Perovskite-KG) from 1,517 research papers, creating two datasets (Perovskite-Chat and Perovskite-Reasoning) using a multi-agent framework, and developing two specialized LLMs (Perovskite-Chat-LLM and Perovskite-Reasoning-LLM). Primary results show Perovskite-Chat-LLM achieved a perplexity of 2.97, a Rouge-L score of 41.25, and an LLM-Judge score of 2.97 on the Perovskite QA dataset, significantly outperforming baseline models. The principal implication for AI practitioners is that this system offers tools for enhanced literature review, experimental design, and complex problem-solving in PSC research, demonstrating how domain-specific knowledge can be integrated with LLMs to improve performance in scientific tasks. |
| Pre-training Auto-regressive Robotic Models with 4D Representations (Read more on arXiv or HuggingFace) |
trevordarrell, zitengj0618, gbiamby, yuvansharma, NdtSoCool |
ARM4R pre-trains robotic models using 4D representations from human videos, enhancing transfer learning for robotic control. The main research objective is to develop a robotic model pre-training approach that leverages low-level 4D representations from human video data to improve performance on robotic manipulation tasks. The key methodology involves training an auto-regressive model in three stages: pre-training on human videos for 3D point track prediction, fine-tuning on robot videos for 3D point tracking, and fine-tuning for robotic control. The method achieves an average success rate of 59.47% on 12 RLBench simulation tasks, surpassing PerAct (55.33%). The model with 4d representations enables AI practitioners to improve sim2real transfer, cross-robot generalization, and performance in robotic control tasks by pre-training on unlabeled human video data. |
| Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages (Read more on arXiv or HuggingFace) |
XU Han, Jianing Liu, Guixian Xu, Ziyin Zhang, Zeli Su |
XLM-SWCM is a novel framework for adapting multilingual encoders to text generation in extremely low-resource languages by sharing weights between the encoder and decoder. The main research objective is to develop an effective text generation model for extremely low-resource languages, specifically Chinese minority languages, where existing multilingual models perform poorly. The key methodology involves a weight-sharing mechanism between the encoder and decoder, interleaving weights from a pretrained multilingual encoder (CINO, a variant of XLM-R) with randomly initialized weights in the decoder. The primary result is that XLM-SWCM outperforms mBART-CM by 198.8% in F1-score on text summarization and also outperfromed the larger MC2-LLaMA 13B in cross-lingual settings. AI practitioners can adapt pre-trained multilingual encoders to text generation tasks in extremely low-resource settings more effectively using this weight-sharing framework, significantly improving performance even with limited data. |
Papers for 2025-02-18
| Title |
Authors |
Summary |
| Learning Getting-Up Policies for Real-World Humanoid Robots (Read more on arXiv or HuggingFace) |
Saurabh Gupta, Zixuan Chen, Xialin He, RunpeiDong |
The paper introduces HUMANUP, a learning framework for training humanoid robots to get up from various lying positions on diverse terrains. The main research objective is to develop a controller that enables humanoid robots to autonomously recover from falls in real-world settings. The key methodology is a two-stage reinforcement learning approach with a curriculum, where Stage I discovers a getting-up trajectory and Stage II refines it into a deployable, robust policy via imitation learning and control regularization. The primary results show that the learned policy enables a Unitree G1 robot to get up from supine poses with a 78.3% success rate on varied terrains, outperforming the robot’s built-in controller. The principal implication is that this framework provides AI practitioners a method to train robust fall recovery policies for humanoid robots, enhancing their real-world deployability by making robots more resilient. |
| Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (Read more on arXiv or HuggingFace) |
Liang Zhao, Junyu Luo, Damai Dai, Huazuo Gao, Jingyang Yuan |
The paper introduces NSA, a natively trainable sparse attention mechanism for efficient long-context modeling in large language models. The main research objective is to develop a sparse attention mechanism that improves computational efficiency during both training and inference while maintaining or exceeding the performance of full attention. The key methodology involves a dynamic hierarchical sparse strategy combining coarse-grained token compression with fine-grained token selection, alongside hardware-aligned optimizations for modern GPUs. Results show that NSA achieves up to 9.0x forward and 6.0x backward propagation speedup on 64k-length sequences compared to Full Attention, and outperforms Full Attention on average across general benchmarks (average score of 0.456 vs 0.443). For AI practitioners, NSA provides a method to train and deploy long-context language models with significantly reduced computational cost and improved performance, particularly on tasks requiring long-range dependencies. |
| ReLearn: Unlearning via Learning for Large Language Models (Read more on arXiv or HuggingFace) |
Sendong Zhao, Liming Yang, Ningyuan Zhao, Haoming Xu, Ningyu |
ReLearn is a new method for unlearning in large language models that uses data augmentation and positive optimization, addressing limitations of reverse optimization methods. The main research objective is to develop an unlearning method that effectively removes targeted knowledge while preserving model performance, linguistic coherence, and robustness against attacks. ReLearn employs data augmentation with diverse question variations and fine-tuning on synthesized non-sensitive data, along with a comprehensive evaluation framework including Knowledge Forgetting Rate (KFR), Knowledge Retention Rate (KRR), and Linguistic Score (LS). The primary result is that ReLearn achieved a KFR of 0.85 on both KnowUnDo and TOFU datasets while maintaining a high KRR (0.74 on KnowUnDo and 0.89 on TOFU) and preserving linguistic abilities. AI practitioners can utilize ReLearn as an alternative to reverse optimization-based unlearning, providing a method to balance knowledge removal with the preservation of model utility and robustness in applications requiring privacy or copyright compliance. |
| SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? (Read more on arXiv or HuggingFace) |
Johannes Heidecke, Tejal Patwardhan, Michele Wang, Samuel Miserendino |
SWE-Lancer is a benchmark of over 1,400 real-world freelance software engineering tasks from Upwork, valued at $1 million USD, to evaluate large language models’ (LLMs) coding and managerial capabilities. The main research objective is to assess whether frontier LLMs can successfully complete real-world freelance software engineering tasks and earn substantial income. The key methodology involves evaluating LLMs on two task types: Individual Contributor (IC) SWE tasks, graded via human-verified end-to-end tests, and SWE Manager tasks, assessed by comparing model choices to those of original engineering managers. Primary results show that the best-performing model, Claude 3.5 Sonnet, achieves 26.2% success on IC SWE tasks and 44.9% on SWE Management tasks on the Diamond set, earning $208,050 out of a possible $500,800. Principal implication for AI practitioners is that while frontier LLMs demonstrate some capability in real-world software engineering scenarios, significant improvement is needed for reliable, autonomous deployment in freelance work. |
| HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation (Read more on arXiv or HuggingFace) |
Minghao Xu, Chenming Shang, Ye Tian, Ling Yang, comin |
HermesFlow is a framework designed to reduce the performance disparity between multimodal understanding and generation in Multimodal Large Language Models (MLLMs). The main research objective is to close the gap between the understanding and generative capabilities of MLLMs. The key methodology used is Pair-DPO, which leverages homologous preference data for both understanding and generation, combined with self-play iterative optimization. The primary results show that HermesFlow achieves an understanding score of 0.533 and a generation score of 0.497, reducing the gap to 0.036, compared to the baseline Show-o’s gap of 0.087. For AI practitioners, HermesFlow provides a general alignment framework that demonstrably closes the gap between multimodal understanding and generation tasks within existing MLLM architectures. |
| SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors (Read more on arXiv or HuggingFace) |
Siqiao Huang, zcliang22, Bohan22 |
This paper introduces SURGE, a benchmark for evaluating large language models (LLMs) as general-purpose surrogate code executors. The main research objective is to assess whether LLMs can predict the output and behavior of programs across diverse tasks without actually running the code. The methodology involves creating a benchmark (SURGE) with eight distinct code execution aspects, evaluating various open-source and proprietary LLMs, and conducting a scaling study. A key finding is that Claude-3.5-Sonnet achieves an average accuracy of 34.31% across all subsets in the zero-shot setting. The principal implication for AI practitioners is that while LLMs show some capability in predicting code execution, there are still limitations in their ability to serve as general-purpose surrogate code executors, especially for time-consuming computations. |
| Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening (Read more on arXiv or HuggingFace) |
Mengdi Wang, Yunhai Tong, Ling Yang, Ye Tian, comin |
Diffusion-Sharpening fine-tunes diffusion models by optimizing sampling trajectories using a path integral framework, enhancing downstream alignment. The main research objective is to improve diffusion model alignment with user preferences by optimizing the entire sampling trajectory, overcoming limitations of single-timestep optimization. The key methodology, Diffusion-Sharpening, uses a path integral framework to select optimal trajectories during training and leverages reward feedback, implementing this via SFT and RLHF approaches. Primary results show that RLHF Diffusion-Sharpening achieves a CLIP score of 0.338, outperforming baseline SDXL and other methods. The principal implication is that AI practitioners can achieve superior training and inference efficiency, along with better alignment to diverse metrics, by using trajectory-level optimization for diffusion model fine-tuning. |
| I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models (Read more on arXiv or HuggingFace) |
Runtao Liu, Hanrong Ye, Guocheng Qian, Kuan-Chieh Wang, Mifucius |
Here’s a concise summary of the research paper, adhering strictly to the guidelines provided: ThinkDiff aligns vision-language models (VLMs) with diffusion models to enable multimodal in-context reasoning in image generation. The main research objective is to empower text-to-image diffusion models with multimodal in-context understanding and reasoning capabilities. The key methodology is aligning VLMs with the decoder of an encoder-decoder large language model (LLM) through a proxy task of vision-language training, leveraging the shared input feature space between the LLM decoder and diffusion decoders. The primary result is that ThinkDiff significantly improves accuracy on the CoBSAT benchmark for multimodal in-context reasoning generation, achieving 46.3% accuracy compared to the previous 19.2%, with only 5 hours of training on 4 A100 GPUs. Principal implication for AI practioners: transfer the multimodal capabilities of VLM without complex reasoning datasets for in-context reasoning tasks, enhancing image generation from diffusion models. |
| SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQL (Read more on arXiv or HuggingFace) |
Hwanhee Lee, Byeongjeong Kim, Ingeol Baek, Jimin Lee |
SAFE-SQL is a framework that improves Text-to-SQL performance by using large language models (LLMs) to generate and filter synthetic examples for in-context learning. The main research objective is to enhance Text-to-SQL accuracy in an unsupervised manner, particularly in complex or unseen scenarios, without additional fine-tuning. The key methodology involves schema linking, LLM-based example generation, relevance scoring (embedding similarity, keyword/structural alignment, reasoning path validity), and threshold-based filtering. Primary results show SAFE-SQL achieved 87.9% execution accuracy on the Spider development set, outperforming zero-shot and few-shot methods, especially in hard and extra hard categories. The principal implication for AI practitioners is that using self-augmented, fine-grained example selection with LLMs can significantly improve the accuracy and robustness of Text-to-SQL systems without requiring additional model training or relying on predefined training sets. |
| CRANE: Reasoning with constrained LLM generation (Read more on arXiv or HuggingFace) |
Gagandeep Singh, Sasa Misailovic, Shubham Ugare, Tarun Suresh, Debangshu Banerjee |
Constrained LLM generation can reduce reasoning abilities, but augmenting output grammars with reasoning rules can preserve it. The main research questions are whether LLMs truly lose reasoning capabilities under constrained decoding and how to reduce syntax errors while preserving unconstrained reasoning. The key methodology is a reasoning-augmented constrained decoding algorithm (CRANE) that alternates between unconstrained generation for reasoning and constrained generation for structurally correct outputs, supported by theoretical analysis of LLM expressivity. CRANE significantly outperforms state-of-the-art constrained decoding strategies and unconstrained decoding, showing up to a 10% accuracy improvement on the GSM-symbolic and FOLIO benchmarks. AI practitioners can use CRANE to improve the accuracy and syntactic correctness of LLM outputs in tasks requiring formal constraints, such as code generation and symbolic reasoning. |
| Intuitive physics understanding emerges from self-supervised pretraining on natural videos (Read more on arXiv or HuggingFace) |
Laurent Najman, Adrien Bardes, Mahmoud Assran, Nicolas Ballas, Quentin Garrido |
V-JEPA, a video joint embedding predictive architecture, demonstrates an understanding of intuitive physics when pretrained on natural videos. The main research objective was to investigate the emergence of intuitive physics understanding in deep neural networks trained to predict masked regions in natural videos. Researchers leveraged the violation-of-expectation framework and compared video prediction models in a learned representation space with pixel-space prediction and multimodal large language models. A V-JEPA model trained on natural videos achieved 98% zero-shot accuracy on the IntPhys benchmark. AI practitioners can apply the principle of joint learning of abstract representation space with sensory input prediction, as a robust objective for acquiring intuitive physics understanding in AI models, challenging the reliance on core knowledge. |
| Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM’s Nest (Read more on arXiv or HuggingFace) |
Jingbo Shang, Feng Yao, Zilong Wang, Letian Peng |
Cuckoo is a novel information extraction (IE) model that leverages large language model (LLM) resources for pre-training via a new paradigm called Next Tokens Extraction (NTE). The main research objective is to demonstrate that IE models can be effectively pre-trained using the same data and a similar paradigm as LLMs, overcoming data scarcity limitations in traditional IE pre-training. The key methodology is converting next token prediction in LLMs to next token extraction (NTE) using BIO tags, applied to 102.6M instances derived from the C4 and TuluV3 datasets. Cuckoo outperforms existing pre-trained IE models in few-shot settings, achieving a 70.63 average F1 score across six basic IE tasks, surpassing baselines significantly. AI practitioners can leverage the NTE paradigm to train versatile and efficient IE models using readily available LLM pre-training resources, avoiding expensive manual annotation and enabling adaptation to a variety of IE tasks. |
| Dyve: Thinking Fast and Slow for Dynamic Process Verification (Read more on arXiv or HuggingFace) |
Qiang Xu, Xiangyu Wen, Zhijian Xu, Zeju Li, Jianyuan1 |
Dyve is a dynamic process verifier that enhances reasoning error detection in large language models by integrating fast and slow thinking. The main research objective is to improve the accuracy and efficiency of process verification in large language models’ reasoning. The key methodology is a dual-system approach, adaptively applying “System 1” (fast, token-level) and “System 2” (slow, comprehensive) verification, supported by step-wise consensus-filtered process supervision using Monte Carlo estimation, LLM-as-a-Judge, and specialized reasoning models. Dyve achieved an F1 score of 68.5 on the GSM8K subset of ProcessBench, outperforming existing process-based verifiers. AI practitioners can use Dyve’s dual-system approach for more reliable and efficient process verification in LLM-based reasoning systems, as it offers superior error detection to traditional process-based methods. |
| PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning (Read more on arXiv or HuggingFace) |
Jiaxing Huang, Yanrui Wu, Yuxuan Dong, Xinyu Zhang, ChengyouJia |
PhysReason is a new benchmark for evaluating physics-based reasoning capabilities of large language models (LLMs). The main research objective is to create a comprehensive benchmark to assess LLMs’ ability to solve physics problems requiring multi-step reasoning and application of physics theorems. The methodology involves compiling 1,200 physics problems categorized by difficulty and knowledge/reasoning type, and proposing the Physics Solution Auto Scoring Framework (PSAS) for evaluation. Primary results showed that even top-performing models like Deepseek-R1 achieved less than 60% on answer-level evaluation, with performance dropping from 75.11% on knowledge questions to 31.95% on hard problems. Principal implication for AI practitioners: the benchmark highlights limitations of current LLMs and can help to improve future models on tasks for physics-based reasoning and applications such as robotics. |
| System Message Generation for User Preferences using Open-Source Models (Read more on arXiv or HuggingFace) |
Teakgyu Hong, Dawoon Jung, Minsoo Khang, Jungho Cho, Minbyul Jeong |
SYSGEN, a data construction pipeline, generates system messages and aligned assistant responses for large language models using open-source models. The main research objective is to address the scarcity and license restrictions of existing datasets with system messages by automatically generating diverse, instruction-aligned system messages. The key methodology involves a four-phase pipeline: generating system messages with eight key functionalities, filtering mis-specified tags, verifying functionalities using an LLM-as-a-judge approach, and generating new, aligned assistant responses. Training on SYSGEN data improved model alignment, with LLaMA-3.1-8B-instruct and Phi-4 models achieving +0.9 and +0.13 absolute improvements, respectively, on the Multifacet benchmark. AI practitioners can leverage SYSGEN to enhance model alignment with user instructions and preferences while minimizing performance degradation on unseen benchmarks and avoiding licensing issues related to training data. |
| video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model (Read more on arXiv or HuggingFace) |
Yixuan Li, Changli Tang, Jimin Zhuang, Yudong Yang, Guangzhi Sun |
video-SALMONN-01 is an open-source audio-visual large language model designed for enhanced reasoning in general video understanding tasks. The main research objective is to improve the reasoning capabilities of audio-visual LLMs for general video understanding, beyond the existing focus on mathematical problems and visual graphical inputs. The key methodology involves developing a reasoning-intensive dataset with step-by-step solutions, proposing process direct preference optimization (pDPO) for step-level reward modeling, and introducing RivaBench, a new video understanding benchmark. Primary results show that video-SALMONN-01 achieves 3-8% accuracy improvements over the LLaVA-OneVision baseline across different video reasoning benchmarks, and pDPO achieves 6-8% improvements compared to the supervised fine-tuning model on RivaBench. AI practitioners can utilize video-SALMONN-01 and the pDPO method for building applications requiring advanced audio-visual reasoning, such as complex video comprehension and synthetic video detection. |
| Building A Proof-Oriented Programmer That Is 64% Better Than GPT-4o Under Data Scarsity (Read more on arXiv or HuggingFace) |
Tianran Sun, Justin Wang, Dylan Zhang |
This paper introduces PoPilot, a fine-tuned language model designed to address data scarcity in proof-oriented programming with F. The main research objective is to improve language models’ performance on project-level proof generation and repair in F under data-scarce conditions. The key methodology involves synthetic data augmentation, creating new proof-oriented programming problems, incorporating diverse coding data, and generating repair data within existing repositories. The primary result shows that the 14B parameter model, PoPilot, outperforms GPT-4o in project-level proof-oriented programming by a 64% relative margin. AI practitioners can leverage the proposed synthetic data generation strategies to create specialized verification assistants capable of both synthesizing and repairing proofs to reduce the cost of adaptation of language model. |
| MagicArticulate: Make Your 3D Models Articulation-Ready (Read more on arXiv or HuggingFace) |
Yiwen Chen, Fan Yang, Xiu Li, Jianfeng Zhang, chaoyue7 |
MagicArticulate is a framework that automatically converts static 3D models into animation-ready assets with skeletons and skinning weights. The main research objective is to develop a scalable method for automatically generating articulation-ready 3D models, addressing the limitations of manual annotation and existing template-based or template-free approaches. The key methodology involves a two-stage pipeline: an auto-regressive transformer for skeleton generation formulated as a sequence modeling problem, followed by a functional diffusion process for skinning weight prediction that incorporates volumetric geodesic distance priors. The method achieves a Chamfer Distance (CD-J2J) of 2.586 on the Articulation-XL dataset for skeleton generation, outperforming existing methods. For AI practitioners, MagicArticulate provides a scalable solution to automatically rig 3D models, significantly reducing the manual effort required for animation content creation and potentially accelerating the development of animation pipelines. |
| Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systems (Read more on arXiv or HuggingFace) |
Shingo Takamatsu, Briti Gangopadhyay, Wei-Yao Wang, Sota Moriyama, Zhao Wang |
i) The paper introduces TalkHier, a novel framework for LLM Multi-Agent (LLM-MA) systems designed to improve communication and refinement in complex collaborative tasks. ii) The research aims to address challenges in managing communication and refinement among agents in LLM-MA systems. iii) The methodology involves a structured communication protocol and a hierarchical refinement system. iv) TalkHier achieves 88.38% accuracy on the MMLU benchmark when built on GPT40, outperforming inference scaling models and open-source multi-agent models. v) The principal implication for AI practitioners is a new standard for LLM-MA systems, providing a more effective, adaptable, and collaborative framework. |
| One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMs (Read more on arXiv or HuggingFace) |
Xinnian Liang, Zhikun Xu, Haojing Huang, Jiayi Kuang, Yinghui Li |
This paper introduces COUNTERMATH, a new benchmark for evaluating counterexample-driven conceptual reasoning in mathematical Large Language Models (LLMs). The main research objective is to assess and enhance LLMs’ ability to understand mathematical concepts through counterexample-driven proofs, moving beyond reliance on “drill-based” learning. The key methodology involves creating a dataset of 1,216 university-level mathematical statement-rationale pairs from textbooks and developing a data engineering framework for automatically acquiring training data. Primary results show that even advanced LLMs like OpenAI o1 achieve a relatively low F1 score (60.1) on COUNTERMATH, and a fine-tuned model with only 1,025 training samples significantly outperformed baseline models. The principal implication for AI practitioners is that strengthening LLMs’ counterexample-driven reasoning is crucial for improving their overall mathematical capabilities, and this work provides a benchmark and methodology to pursue this. |
| Better Embeddings with Coupled Adam (Read more on arXiv or HuggingFace) |
Tobias Stollenwerk, flxst |
The paper introduces Coupled Adam, a modification of the Adam optimizer, to address the anisotropy problem in language model embeddings. The main research question is whether the second moment in the Adam optimizer contributes to anisotropic word embeddings in language models and how this can be mitigated. The key methodology involves analyzing the embedding update vectors under SGD and Adam, proposing a modified Adam optimizer (“Coupled Adam”) that averages the second moment across vocabulary items, and empirically evaluating its impact on embedding quality and model performance. Primary results show Coupled Adam improves embedding isotropy significantly, achieving values above 0.90 in most small-scale experiments, and enhances upstream/downstream performance on sufficiently large datasets. For AI practitioners, using Coupled Adam instead of standard Adam can improve the quality of word embeddings and boost model performance, particularly for large language models. |
| Towards Data-Efficient Pretraining for Atomic Property Prediction (Read more on arXiv or HuggingFace) |
Bernard Ghanem, Yasir Ghunaim, hammh0a |
This paper investigates data-efficient pretraining for atomic property prediction, showing that strategic dataset selection can match or surpass large-scale pretraining with significantly reduced computational cost. The main research objective is to determine if pretraining on a smaller, task-relevant dataset can achieve comparable or superior performance to large-scale pretraining in atomic property prediction. The key methodology introduces the Chemical Similarity Index (CSI), a metric inspired by Fréchet Inception Distance, to quantify the alignment between upstream pretraining datasets and downstream tasks, and uses this to select pretraining data. A primary result is that models pretrained on the ANI-1x dataset (using the CSI for selection) achieved a Mean Absolute Error (MAE) of 5.4 on rMD17, outperforming JMP-S (MAE of 6.7) with 24 times less computational budget. Principal implication for AI practitioners is that strategic selection of pretraining data based on task relevance, assessed using metrics like CSI, can achieve competitive performance with significantly reduced computational resources in atomic property prediction, favoring quality over quantity. |
| Large Language Models and Mathematical Reasoning Failures (Read more on arXiv or HuggingFace) |
birgermoell, jboye |
This paper evaluates the mathematical reasoning capabilities of large language models (LLMs) using newly constructed word problems and identifies common failure modes. The main research question is: How good are LLMs at mathematical reasoning when evaluated on both answer correctness and solution steps? The key methodology involved creating a dataset of 50 high-school-level mathematical word problems and manually assessing the answers and solutions provided by eight LLMs, including Mixtral, Llama, Gemini, and GPT-4o. The primary result was that the o1 model achieved the highest accuracy, correctly solving 37 out of 50 problems, while all models exhibited errors in spatial reasoning, strategic planning, and arithmetic. The principal implication for AI practitioners is the need to evaluate LLMs’ reasoning processes, not just their final answers, to avoid overestimating their problem-solving proficiency. |
| Language Complexity Measurement as a Noisy Zero-Shot Proxy for Evaluating LLM Performance (Read more on arXiv or HuggingFace) |
jboye, birgermoell |
This paper evaluates the capability of Large Language Models (LLMs) to measure language complexity as a proxy for general LLM performance. The main research objective is to examine the performance of state-of-the-art LLMs on computing the LIX readability metric and performing dependency parsing to calculate Average Dependency Distance (ADD). The methodology involves evaluating six LLMs using Swedish essays, comparing their LIX and ADD computations against ground truth values, and correlating these with MMLU benchmark scores. A primary result is a strong significant correlation of -0.875 (p=0.026) between the models’ accuracy in computing LIX and their MMLU performance. For AI practitioners, language complexity measurement abilities, specifically LIX computation, can serve as a practical, noisy zero-shot proxy for assessing general LLM capabilities, without needing extensive benchmarking datasets. |
Papers for 2025-02-17
| Title |
Authors |
Summary |
| Region-Adaptive Sampling for Diffusion Transformers (Read more on arXiv or HuggingFace) |
Lili Qiu, Yiqi Zhang, Chengruidong Zhang, Yifan Yang, Ziming Liu |
Region-adaptive sampling (RAS) improves the efficiency of Diffusion Transformers (DiTs) by dynamically adjusting sampling ratios across image regions. The main objective is to accelerate the sampling process of DiTs without significant quality degradation by focusing computational resources on semantically meaningful regions. RAS identifies “focus” regions in each sampling step using output noise from the previous step, updating only these, and caches the rest, based on attention continuity. RAS achieves speedups of up to 2.36x and 2.51x on Stable Diffusion 3 and Lumina-Next-T2I, respectively, with minimal generation quality degradation. AI practitioners can use RAS to significantly improve the sampling speed of Diffusion Transformers, facilitating real-time applications that require high-quality image generation. |
| Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model (Read more on arXiv or HuggingFace) |
Nan Duan, Liangyu Chen, Kun Yan, Haoyang Huang, Guoqing Ma |
i) Step-Video-T2V, a 30B parameter text-to-video model, achieves state-of-the-art results via a novel architecture and training strategy. ii) The research objective is to develop a high-performance and high-quality text-to-video generation model surpassing existing open-source and commercial engines. iii) The methodology involves a deep compression Video-VAE, a DiT with 3D full attention trained using Flow Matching, and a video-based DPO for visual quality enhancement. iv) Evaluated on Step-Video-T2V-Eval, Step-Video-T2V demonstrates state-of-the-art performance with 16x16 spatial and 8x temporal compression ratios while generating videos up to 204 frames. v) AI practitioners can leverage Step-Video-T2V as a strong baseline for further innovations in video foundation models, particularly in improving motion dynamics, aesthetics, and content consistency. |
| ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models (Read more on arXiv or HuggingFace) |
Samuel Roberts, Akash Gupta, Ansh Sharma, Mohammad Reza Taesiri, Jonathan Roberts |
ZeroBench is a new visual reasoning benchmark of 100 questions designed to be impossible for current large multimodal models (LMMs). The main research objective is to create a lightweight yet challenging visual benchmark to evaluate and differentiate the capabilities of LMMs. The methodology involves manually curating and reviewing a set of diverse, multi-step visual reasoning questions, and then adversarially filtering them based on the performance of 20 contemporary LMMs. The primary result is that all evaluated LMMs scored 0.0% on the main questions of ZeroBench, although they achieved non-zero scores on the easier sub-questions, such as 24.30% pass@1 by Claude 3.5 Sonnet v2. The principle implication is that this benchmark highlights limitations to assist in the development of improved LMMs. |
| Large Language Diffusion Models (Read more on arXiv or HuggingFace) |
Jingyang Ou, Xiaolu Zhang, Zebin You, Fengqi Zhu, Shen Nie |
LLaDA, a diffusion model trained from scratch, achieves performance comparable to autoregressive LLMs like LLaMA3 8B. The main research question is whether diffusion models can achieve the capabilities of large language models (LLMs) without relying on the autoregressive paradigm. Key methodology used is a masked diffusion model (MDM) trained with a forward data masking process and a reverse process parameterized by a vanilla Transformer to predict masked tokens, optimizing a likelihood bound. Primary result is that LLaDA 8B surpasses LLaMA2 7B on nearly all 15 standard zero/few-shot learning tasks and is on par with LLaMA3 8B, and it achieves a 70.7% accuracy on the GSM8K benchmark. Principal implication is that AI practitioners can explore diffusion models as a viable alternative to autoregressive models for large-scale language modeling, potentially offering advantages in bidirectional context understanding and parallel token generation. |
| MM-RLHF: The Next Step Forward in Multimodal LLM Alignment (Read more on arXiv or HuggingFace) |
Peiyan Li, Chaoyou Fu, Haochen Tian, Tao Yu, Yi-Fan Zhang |
i) The paper introduces MM-RLHF, a new dataset and methodology for aligning multimodal large language models (MLLMs) with human preferences. ii) The research aims to enhance MLLM capabilities across multiple dimensions by aligning models with human preferences. iii) The methodology includes curating a 120k comparison pair dataset, developing a critique-based reward model, and employing dynamic reward scaling within DPO. iv) Fine-tuning LLaVA-ov-7B with MM-RLHF and the proposed alignment algorithm achieves a 19.5% increase in conversational abilities and a 60% improvement in safety. v) AI practitioners can leverage the MM-RLHF dataset and associated techniques to improve MLLM alignment, leading to safer and more capable multimodal models; the critique based reward model can be used to provide more informative feedback for training. |
| Precise Parameter Localization for Textual Generation in Diffusion Models (Read more on arXiv or HuggingFace) |
Adam Dziedzic, Kamil Deja, Franziska Boenisch, Bartosz Cywiński, Łukasz Staniszewski |
This research localizes and utilizes the parameters in diffusion models responsible for generating and editing textual content within images. The main research objective is to identify the specific parameters within diffusion models that control the generation of textual content in images. The key methodology involves activation patching of cross and joint attention layers and fine-tuning using Low-Rank Adaptation (LoRA). The primary result is that less than 1% of diffusion models’ parameters (0.61% of Stable Diffusion XL, 0.21% of DeepFloyd IF, and 0.23% of Stable Diffusion 3), specifically within attention layers, are responsible for textual content generation. This implies that AI practitioners can improve text generation in diffusion models, and enable precise text editing by fine-tuning or manipulating only this small subset of parameters, conserving computational resources and preserving overall image generation quality. |
| Diverse Inference and Verification for Advanced Reasoning (Read more on arXiv or HuggingFace) |
Yuke Zhang, Seunghwan Hyun, Mao Mao, Gaston Longhitano, Iddo Drori |
i) The paper presents a diverse inference approach to improve the performance of Reasoning LLMs on challenging tasks. ii) The research aims to enhance reasoning LLMs’ accuracy on complex benchmarks like IMO combinatorics, ARC puzzles, and HLE questions. iii) Key methods include combining multiple models/methods at test time, verifying solutions automatically, test-time simulations, reinforcement learning, and meta-learning of agent graphs. iv) The approach increases IMO combinatorics accuracy from 33.3% to 77.8%, HLE accuracy from 8% to 37%, and solves 80% of ARC puzzles unsolvable by 948 humans. v) AI practitioners can leverage diverse inference and verification techniques to improve the robustness and accuracy of reasoning LLMs on advanced problem-solving tasks. |
| We Can’t Understand AI Using our Existing Vocabulary (Read more on arXiv or HuggingFace) |
Been Kim, Robert Geirhos, John Hewitt |
This position paper argues that understanding and controlling AI requires developing new vocabulary (neologisms) to represent concepts unique to machines or humans. The main research objective is to argue for developing neologisms to bridge the communication gap between humans and AI, stemming from their differing conceptualizations of the world. The key methodology used is a conceptual argument supported by a proof-of-concept, “neologism embedding learning,” which trains new word embeddings representing human or machine concepts to control model behavior. The primary results demonstrated that using a “length neologism,” responses that meet the length contraints went from near 0% with regular instructions, to a vast majority of generations, shown in figure 5. The authors presented a new “diversity neologism”, increasing response variety in a number-guessing task. Principal implication for AI practitioners is that creating and incorporating neologisms into prompts can improve control over language model behavior and potentially provide a more precise way to interact with and understand AI systems. |
| AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting (Read more on arXiv or HuggingFace) |
Maurizio Filippone, Albert Thomas, Giuseppe Paolo, Vasilii Feofanov, abenechehab |
AdaPTS is a framework for adapting pre-trained univariate time series foundation models to probabilistic multivariate forecasting using trainable feature-space transformations. The main research objective is to develop a method for leveraging pre-trained univariate time series foundation models (FMs) for multivariate forecasting tasks while addressing challenges like inter-feature dependencies and uncertainty quantification. The key methodology involves introducing “adapters”—stochastic, invertible feature-space transformations—that project multivariate inputs into a latent space where a frozen, pre-trained univariate FM can be applied independently to each dimension, followed by an inverse transformation. Primary results show that AdaPTS improves the forecasting accuracy of the Moment model in 5 out of 8 considered tasks; for example on the Illness dataset (H=24), the VAE adapter achieved a 15% MSE improvement, reducing it from 2.902 to 2.461. AI practitioners can use AdaPTS as a modular and scalable solution for leveraging existing time series FMs in multivariate contexts, enhancing forecasting performance, and uncertainty quantification without requiring FM fine-tuning. |
| FoNE: Precise Single-Token Number Embeddings via Fourier Features (Read more on arXiv or HuggingFace) |
Vatsal Sharan, Robin Jia, Mahdi Soltanolkotabi, Deqing Fu, Tianyi Zhou |
FoNE introduces a novel method to represent numbers as single tokens in large language models using Fourier features. The main research objective is to develop a more precise and efficient number embedding method that overcomes the limitations of traditional subword and digit-wise tokenization in LLMs. FoNE maps numbers directly into the embedding space using their Fourier features, encoding each digit with two embedding dimensions. On 6-digit decimal addition, FoNE requires 64x less data to achieve 99% accuracy than subword and digit-wise embeddings and is the only method that yields 100% accuracy on over 100,000 test examples. The principal implication is that AI practitioners can leverage FoNE to improve LLM performance on number-related tasks, achieving higher accuracy with reduced computational overhead and training data. |
| Jailbreaking to Jailbreak (Read more on arXiv or HuggingFace) |
Bijan Varjavand, Robert Vacareanu, Vaughn Robinson, Jeremy Kritz, ZifanScale |
This paper introduces “Jailbreaking-to-Jailbreak” (J2), a novel approach where a refusal-trained Large Language Model (LLM) is jailbroken to assist in jailbreaking other LLMs. The main research objective is to evaluate the capability of jailbroken LLMs to act as effective red teamers and to compare their performance against existing automated and human-led red teaming methods. Key methodology involves creating J2 attackers by jailbreaking frontier LLMs through human-crafted prompts, then using these J2 attackers in an iterative, multi-turn red teaming workflow with in-context learning. Primary results show that J2 attackers (specifically Sonnet-3.5 and Gemini-1.5-pro) achieve 93.0% and 91.0% attack success rates (ASRs) respectively against GPT-40 on Harmbench, approaching human-level performance. Principal implication for AI practitioners is that LLM safeguards can be bypassed by leveraging a jailbroken version of an LLM, highlighting a new failure mode and emphasizing the need for enhanced safeguard mechanisms against LLM-assisted jailbreaking. |
| STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task Planning (Read more on arXiv or HuggingFace) |
Shuguang Cui, Zhixin Mai, Ge Wang, Yiming Zhao, Mingcong Lei |
The paper introduces the Spatio-Temporal Memory Agent (STMA), a framework designed to enhance task planning and execution in dynamic environments for embodied AI. The main objective is to enable agents to perform long-horizon tasks by improving decision-making and adaptability through integrated spatio-temporal memory. The methodology involves a spatio-temporal memory module, a dynamic knowledge graph for spatial reasoning, and a planner-critic mechanism for iterative strategy refinement. Results from evaluations in the TextWorld environment show STMA achieved a 31.25% improvement in success rate and a 24.7% increase in average score compared to state-of-the-art models. For AI practitioners, STMA offers a new way to approach memory within AI Agents. |
| MRS: A Fast Sampler for Mean Reverting Diffusion based on ODE and SDE Solvers (Read more on arXiv or HuggingFace) |
Ge Yang, Le Lu, Hongbo Zhao, Wei Fang, Ao Li |
Mean Reverting Sampler (MRS) accelerates sampling for Mean Reverting (MR) Diffusion models. The main research objective is to reduce the sampling NFEs (number of function evaluations) of MR Diffusion, which currently requires hundreds of steps. The methodology involves solving the reverse-time SDE and probability flow ODE associated with MR Diffusion, deriving semi-analytical solutions consisting of an analytical function and a neural network parameterized integral. Primary results demonstrate that the MR Sampler maintains high sampling quality with a speedup of 10 to 20 times across ten different image restoration tasks. Principal implication for AI practitioners is that they can leverage MRS for faster and more efficient controllable generation using MR Diffusion models, making them more practical in applications. |
| V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multi-Modal Large Language Models (Read more on arXiv or HuggingFace) |
Yu-Chiang Frank Wang, Stephen F. Smith, Chien-Yi Wang, Ryo Hachiuma, Hsu-kuang Chiu |
i) This paper introduces V2V-LLM, a large language model for cooperative autonomous driving. ii) The research aims to explore the problem of integrating LLMs into cooperative autonomous driving systems to improve safety. iii) The methodology involves creating a new dataset, V2V-QA, and developing a baseline method, V2V-LLM, that fuses perception information from multiple connected autonomous vehicles using scene-level and object-level features. iv) The V2V-LLM outperforms other fusion methods on notable object identification and planning tasks in the V2V-QA dataset, achieving a collision rate of 3.00% compared to 4.57% for the “No Fusion” baseline. v) The primary implication for AI practitioners is the potential of V2V-LLM to serve as a foundation model for cooperative autonomous driving, particularly in scenarios with sensor occlusion. |
| Agentic End-to-End De Novo Protein Design for Tailored Dynamics Using a Language Diffusion Model (Read more on arXiv or HuggingFace) |
Markus J. Buehler, Bo Ni |
VibeGen is a generative AI framework for de novo protein design conditioned on normal mode vibrations. The main research objective is to develop a model that can generate novel protein sequences that exhibit specified dynamic properties, specifically low-frequency vibrational modes. The key methodology involves an agentic dual-model architecture, comprising a protein designer (PD) based on a protein language diffusion model that generates sequences and a protein predictor (PP) that evaluates their dynamic accuracy. Primary results showed that the generated proteins accurately reproduced prescribed normal mode amplitudes, with a median Pearson correlation coefficient of 0.53 between designed and target vibration profiles across a large test set. Principal implication for AI practitioners is the demonstration of a viable approach for integrating protein dynamics into generative protein design, enabling the creation of biomolecules with targeted motion-based functionalities. |
Papers for 2025-02-14
| Title |
Authors |
Summary |
| InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU (Read more on arXiv or HuggingFace) |
Sung Ju Hwang, Losif63, geonp, gmlwns5176 |
InfiniteHiP enables extremely long-context language model inference on a single GPU without significant performance loss. The main research objective is to develop a training-free framework that allows large language models (LLMs) to handle context lengths significantly exceeding their pre-trained limits on a single GPU. The key methodology involves a hierarchical pruning algorithm to optimize key-value (KV) cache, combined with a novel block sparse attention mechanism and dynamic RoPE adjustments. The primary result is that InfiniteHiP achieves a 7.24x speedup in the SGLang framework with only 0.34% of the VRAM used by FlashAttention2, while extending context to 3 million tokens on a single GPU. A Principal implication for AI practitioners, is that it can be a framework of efficient, long context inference that utilizes modularized pruning algorithm. |
| Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation (Read more on arXiv or HuggingFace) |
Se Young Chun, Jae-sun Seo, Wongi Jeong, Agorium |
Skrr is a method for reducing text encoder memory usage in text-to-image diffusion models by selectively skipping or reusing layers. The main research question is how to reduce the memory footprint of text encoders in text-to-image (T2I) diffusion models without significantly impacting image quality or text alignment. The key methodology, Skrr, involves two phases: “Skip” identifies and prunes redundant transformer sub-blocks using a T2I diffusion-tailored discrepancy metric and beam search, and “Re-use” recycles remaining layers to mitigate performance loss. Skrr maintains image quality comparable to the original model, and achieves up to 20.4% improvement in GenEval scores at over 40% sparsity. The principal implication for AI practitioners is that Skrr offers an effective strategy for constructing memory-efficient T2I models, which could also help the development and deployment of text-to-image diffusion models, especially in resource-constrained environments. |
| SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models (Read more on arXiv or HuggingFace) |
Hu Xu, Shannon Zejiang Shen, ZhaofengWu, bencw, voidism |
SelfCite is a self-supervised framework that aligns large language models (LLMs) to generate accurate, fine-grained citations by leveraging their own probabilities for necessity and sufficiency rewards through context ablation. The main research objective is to improve the accuracy and quality of citations generated by LLMs without relying on annotation processes. The key methodology involves using context ablation to calculate a reward signal based on two metrics, necessity score (probability drop) and sufficiency score (probability hold), and best-of-N sampling to generate better citations. The primary result is that SelfCite significantly improves citation correctness on the LongBench-Cite benchmark, increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. For AI practitioners, SelfCite offers a method to improve citation quality in LLM-generated text without requiring human annotation, potentially leading to more reliable and trustworthy LLM applications. |
| An Open Recipe: Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging (Read more on arXiv or HuggingFace) |
Kasima Tharnpipitchai, potsawee, pittawat, kunato |
This paper demonstrates a method for enhancing reasoning capabilities in language-specific large language models (LLMs) using model merging and data selection within a limited computational budget. The main research objective is to incorporate the advanced reasoning abilities of a model like DeepSeek R1 into a Thai language-specific LLM while preserving its target language performance. The key methodology involves supervised fine-tuning of the language-specific LLM on a curated dataset, followed by ability-aware model merging with a reasoning-focused LLM, optimizing the merge ratio across layers. A primary result is that the merged model, Typhoon2-R1-70B, achieved 76.5% average performance across all evaluation metrics, 41.6% above Typhoon2 70B Instruct and 12.8% above DeepSeek R1 70B Distill. This approach allows AI practitioners to improve reasoning in low-resource language LLMs efficiently, using publicly available datasets and modest computational resources. |
| Exploring the Potential of Encoder-free Architectures in 3D LMMs (Read more on arXiv or HuggingFace) |
delinqu, Tavish9, zhuhaow, Purple1288, IvanTang |
This paper investigates encoder-free architectures for 3D Large Multimodal Models (LMMs), demonstrating comparable performance to encoder-based models. The main research objective is to determine if 3D LMMs can effectively function without dedicated 3D encoders, directly integrating 3D understanding capabilities within the Large Language Model (LLM). The key methodology involves proposing LLM-embedded Semantic Encoding during pre-training and Hierarchical Geometry Aggregation during instruction tuning, replacing the traditional 3D encoder with learnable LLM layers and self-supervised losses. The primary result is that the proposed ENEL model, without a 3D encoder, achieved a GPT-4 score of 50.92% on 3D object captioning, which is similar with the state-of-the-art ShapeLLM-13B. The principal implication is that AI practitioners can explore encoder-free 3D LMMs as a potentially more efficient and scalable alternative to encoder-based architectures, potentially simplifying model design and reducing computational overhead. |
| Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights (Read more on arXiv or HuggingFace) |
Yedid Hoshen, Or Nathan, Jonathan Kahana, Eliahu |
This paper introduces ProbeLog, a method for retrieving classification models capable of recognizing a specific target concept based on model weights, without access to training data or metadata. The main research question is how to efficiently and accurately search for models in large repositories that can recognize a given concept (e.g., “Dog”) in a zero-shot manner. ProbeLog uses a probing-based approach, computing logit-level descriptors by observing model responses to a fixed set of input probes, and extends this to zero-shot search via text alignment models. The method achieved a top-1 retrieval accuracy of 43.8% on the INet-Hub dataset when searching for models recognizing ImageNet concepts from text prompts. AI practitioners can use ProbeLog to search for suitable pre-trained models based on specific concept recognition capabilities, potentially reducing the need for training or fine-tuning. |
| CoSER: Coordinating LLM-Based Persona Simulation of Established Roles (Read more on arXiv or HuggingFace) |
Rui Xu, Xinfeng Yuan, Yifei Zhang, Heng Wang, Xintao Wang |
CoSER is a framework for simulating established characters using large language models (LLMs), including a dataset, models, and an evaluation protocol. The main research objective is to address the lack of authentic character datasets and nuanced evaluation methods for simulating established characters with LLMs. The key methodology is given-circumstance acting (GCA), where LLMs sequentially portray multiple characters in book scenes, used for both training and evaluation. Primary results show that CoSER 70B achieves 75.80% and 93.47% accuracy on the InCharacter and LifeChoice benchmarks, respectively, surpassing or matching GPT-4o. The principal implication for AI practitioners is that they can leverage the CoSER dataset and GCA framework to train and evaluate LLMs for more faithful and nuanced role-playing of established characters, improving applications like character chatbots and agents in games. |
| TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models (Read more on arXiv or HuggingFace) |
Yuan Liang, Dehu Wang, Zexiang Liu, Zi-Xin Zou, Yangguang Li |
TripoSG is a new image-to-3D generation model that leverages large-scale rectified flow transformers to achieve high-fidelity 3D shape synthesis. The main research objective is to determine the optimal paradigm for generating high-fidelity 3D models with precise alignment to input images. The key methodology involves a large-scale rectified flow transformer trained on 2 million high-quality 3D samples, a hybrid supervised 3D VAE training strategy, and a dedicated data processing pipeline. Primary results show that TripoSG achieves a Normal-FID score of 3.36 when trained on a large-scale dataset with 4096 tokens and a mixture-of-experts model. The model demonstrates that AI practitioners can now utilize large-scale generative techniques to effectively generate detailed, high-fidelity and accurate 3D models from single input images which are consistent with the input. |
| EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (Read more on arXiv or HuggingFace) |
Cheng Qian, Mark Zhao, Junyu Zhang, Rui Yang, Hanyang81 |
EmbodiedBench is a benchmark for evaluating vision-driven embodied agents based on multi-modal large language models (MLLMs). Main research question or objective: How do existing MLLMs perform as vision-driven embodied agents across a variety of tasks and capabilities, and what are their limitations? Key methodology used: Developed a benchmark (EMBODIEDBENCH) with 1,128 testing instances across four environments, hierarchical action levels (high-level and low-level), and six capability-oriented subsets, then evaluated 13 proprietary and open-source MLLMs using a unified agent framework. Primary results: MLLMs excel at high-level tasks but struggle with low-level manipulation; the best model, GPT-4o, scored only 28.9% on average across all tasks in the benchmark, and performance degrades by 40%-70% when vision input is removed in low-level tasks. Principal implication for AI practitioners: AI practitioners should focus on improving MLLMs’ low-level manipulation, long-horizon planning and use additional approaches for leveraging visual input for high-level embodied tasks since the best model performs poorly in low-level tasks. |
| Typhoon T1: An Open Thai Reasoning Model (Read more on arXiv or HuggingFace) |
Kunat Pipatanakul, Kasima Tharnpipitchai, Potsawee Manakul, pittawat |
Typhoon T1 is an open-source Thai reasoning model built on a large language model, demonstrating a method for developing reasoning capabilities in low-resource languages. The primary research objective was to develop a Thai reasoning model and investigate effective strategies for its creation, including thinking formats and data composition. The key methodology involved supervised fine-tuning of a pre-trained language model (Typhoon 2 3B Instruct) using synthetically generated datasets with structured, semi-structured, and unstructured reasoning chains. A primary result was that the structured thinking format achieved a GSM8K score of 62.02, outperforming unstructured and semi-structured formats. The principal implication for AI practitioners is that supervised fine-tuning with structured synthetic data can effectively create reasoning models, particularly in low-resource languages, providing a viable alternative to reinforcement learning. |
| Logical Reasoning in Large Language Models: A Survey (Read more on arXiv or HuggingFace) |
Chaoli Zhang, Mengru Ding, Hanmeng Liu, ruoxining, HarryFu |
This survey synthesizes advancements in logical reasoning within large language models (LLMs), covering paradigms, benchmarks, enhancement methods, and future directions. The main research objective is to provide a comprehensive overview of logical reasoning capabilities in LLMs, focusing on formal symbolic logic rather than general heuristic approaches. The key methodology involves a literature review analyzing existing capabilities across deductive, inductive, abductive, and analogical reasoning, as well as assessing strategies like data-centric tuning, reinforcement learning, and neuro-symbolic approaches. A primary result is that while GPT-4 outperforms ChatGPT on benchmarks like LogiQA and ReClor, both models struggle with out-of-distribution tasks. The principal implication for AI practitioners is the need for hybrid architectures and improved evaluation frameworks that stress-test robustness and generalization in logical reasoning, moving beyond simple accuracy metrics to assess consistency and explainability. |
| MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (Read more on arXiv or HuggingFace) |
Yu Qi, Yanwei Li, Ziyu Guo, Renrui Zhang, CaraJ |
MME-CoT is a benchmark for evaluating Chain-of-Thought (CoT) reasoning in Large Multimodal Models (LMMs), assessing quality, robustness, and efficiency. The main research objective is to investigate to what extent and how CoT reasoning benefits multimodal challenges in LMMs. Researchers curated a dataset with six domains and proposed novel metrics that meticulously examines LMMs reasoning quality, robustness and efficiency at a fine-grained level. The evaluation reveals that Kimi k1.5 achieved the best CoT quality with 64.2 F1-score, surpassing GPT-4o, and CoT prompting often degrades LMM performance on perception-heavy tasks. For AI practitioners, the results provide insights into the strengths and weaknesses of applying CoT to LMMs, especially highlighting that careful consideration is needed when employing CoT in tasks requiring strong perceptual capabilities. |
| CoT-Valve: Length-Compressible Chain-of-Thought Tuning (Read more on arXiv or HuggingFace) |
Xinchao Wang, Gongfan Fang, Runpeng Yu, Guangnian Wan, Xinyin Ma |
CoT-Valve introduces a method for tuning language models to generate reasoning chains of controllable lengths, improving efficiency and adaptability. The main research objective is to enable a single model to dynamically adjust the length of its Chain-of-Thought (CoT) reasoning based on task difficulty. The key methodology involves identifying and manipulating a direction in the parameter space (using LoRA) that controls CoT length, along with a “MixChain” dataset for training. A primary result is that on GSM8K, the QwQ-32B-Preview model reduced reasoning chains from 741 to 225 tokens with a minor performance drop (95.07% to 94.92%). Principal implication for AI practioners is that it enables more efficient inference by allowing models to use shorter reasoning paths for simpler tasks, which can improve the cost-effectiveness of reasoning-based application. |
| SQuARE: Sequential Question Answering Reasoning Engine for Enhanced Chain-of-Thought in Large Language Models (Read more on arXiv or HuggingFace) |
Moshe Wasserblat, Gad Markovits, Moshe Berchansky, danf |
SQuARE is a prompting technique that improves large language model reasoning by generating and answering sub-questions before addressing the main query. The main research objective is to assess if decomposing queries into iterative steps via self-interrogation enhances the reasoning capabilities of LLMs. The key methodology is prompting LLMs (Llama 3 and GPT-4o) to generate and resolve multiple auxiliary question-answer pairs before answering the original question, across multiple QA datasets (TriviaQA, HotpotQA, ASQA). Primary results show that SQuARE improves performance on TriviaQA by 6.5% over Retrieval-Augmented Generation (RAG) using the Llama-3.2 3B model. For AI practitioners, SQuARE presents a method for improving response accuracy in reasoning tasks by systematically decomposing questions, particularly beneficial for smaller-scale models. |
| mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data (Read more on arXiv or HuggingFace) |
Ziliang Zhao, Yutao Zhu, Nan Yang, Liang Wang, Haon-Chen |
mmE5 enhances multimodal multilingual embeddings through a novel synthetic data generation framework. The research objective is to improve multimodal embedding performance by addressing the scarcity of high-quality labeled multimodal data. The methodology involves synthesizing datasets using an MLLM, guided by principles of broad scope, robust cross-modal alignment, and high fidelity, incorporating deep thinking, self-evaluation, and refinement. mmE5 achieves a state-of-the-art average score of 58.6 on the MMEB benchmark in a zero-shot setting, surpassing previous methods. AI practitioners can leverage mmE5’s synthetic data generation approach to create more robust and generalizable multimodal embedding models, particularly in multilingual contexts. |
| The Stochastic Parrot on LLM’s Shoulder: A Summative Assessment of Physical Concept Understanding (Read more on arXiv or HuggingFace) |
Shunchi Zhang, Tsz Ting Chung, Junjie Wu, Lemao Liu, Mo Yu |
The paper introduces PHYSICO, a benchmark to evaluate large language models’ (LLMs) understanding of physical concepts, revealing significant gaps compared to human performance. The primary research objective is to investigate whether LLMs truly understand physical concepts or merely act as “stochastic parrots.” The key methodology is a summative assessment using grid-format inputs to represent physical phenomena, and comparing LLM performance with human performance across various subtasks. Results indicate that state-of-the-art LLMs, like GPT-4, perform perfectly on low-level tasks(>95% accuracy) but lag behind humans on high-level tasks (~40% less in accuracy) . For AI practitioners, the principal implication is that LLMs still lack robust physical concept understanding beyond memorization, suggesting a need for new methods to improve their reasoning ability. |
| DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References (Read more on arXiv or HuggingFace) |
Li Yi, Yuzhe Qin, Qianwei Han, Jianibieke Adalibieke, Xueyi Liu |
DexTrack is a neural tracking controller that learns to manipulate objects with a robotic hand by following human-provided kinematic references. The main research objective is to develop a generalizable neural tracking controller for dexterous manipulation that can mimic human-object interaction trajectories. The key methodology involves iteratively training the controller with reinforcement and imitation learning, using a homotopy optimization method to mine high-quality robot tracking demonstrations from human references. The primary results show that DexTrack achieves over a 10% improvement in success rates compared to leading baselines in both simulation and real-world evaluations. AI practitioners can leverage DexTrack’s approach of combining imitation learning with high-quality demonstrations to create versatile and robust controllers for complex robotic manipulation tasks. |
| 3CAD: A Large-Scale Real-World 3C Product Dataset for Unsupervised Anomaly (Read more on arXiv or HuggingFace) |
Yuanwei Ma, Wenbo Guo, Hanyang Sun, Peng Xing, enquan2022 |
3CAD, a large-scale real-world dataset for unsupervised anomaly detection in 3C products, is introduced along with a coarse-to-fine detection paradigm. The main research objective is to create a challenging benchmark dataset of 3C product defects and develop an effective unsupervised anomaly detection method. The key methodology, CFRG, combines knowledge distillation, recovery guidance, and a segmentation network for coarse-to-fine localization of anomalies. CFRG achieves 93.4% AUROC, 86.5% AUPRO, and 82.0% AP on the 3CAD dataset. The principal implication for practitioners is the 3CAD dataset and CFRG model provide a challenging benchmark and an effective baseline for unsupervised anomaly detection in real-world 3C product manufacturing. |
Papers for 2025-02-13
| Title |
Authors |
Summary |
| TextAtlas5M: A Large-scale Dataset for Dense Text Image Generation (Read more on arXiv or HuggingFace) |
Zhuobai Dong, Weiming Han, Jiawei Zhang, Dongxing Mao, Alex Jinpeng Wang |
TextAtlas5M is a large-scale dataset designed for generating images with dense, complex, and long-form text. The main research objective is to address the limitations of existing datasets, which often focus on shorter and simpler text, thereby hindering the development of models capable of generating images with comprehensive textual content. The key methodology involves curating 5 million long-text generated and collected images across diverse data types, including synthetic and real-world images, and creating a human-improved test set (TextAtlasEval) of 3,000 samples across 3 data domains. Primary results include the finding that evaluations demonstrate even advanced proprietary models (e.g., GPT4o with DallE-3) are significantly challenged by TextAtlasEval benchmarks, while showing an even large gap in their open-source counterparts. This dataset and benchmarks provide AI practitioners with a valuable resource for training and evaluating text-conditioned image generation models, specifically focusing on dense and long-form text rendering, thus, advancing the capacity to control visual outputs. |
| Light-A-Video: Training-free Video Relighting via Progressive Light Fusion (Read more on arXiv or HuggingFace) |
Pan Zhang, Pengyang Ling, Jiazi Bu, Yujie Zhou, yuhangzang |
Light-A-Video is a training-free approach for temporally smooth video relighting that leverages image relighting and video diffusion models. The main research objective is to achieve temporally consistent video relighting without requiring training or optimization, addressing the limitations of existing methods. The key methodology involves a Consistent Light Attention (CLA) module for stable light source generation and a Progressive Light Fusion (PLF) strategy to blend relighted appearances, incorporating motion priors from a video diffusion model. Primary results show that Light-A-Video achieves a FID score of 29.63 while maintaining a temporal consistency CLIP score of 0.9655, superior to baseline methods that apply image relighting frame-by-frame. For AI practitioners, Light-A-Video provides a training-free pipeline for high-quality video relighting, directly applicable with existing image relighting and video diffusion models, enabling zero-shot illumination control of video sequences. |
| BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (Read more on arXiv or HuggingFace) |
Lei Li, Conghui He, Hanxu Hu, Wenhao Zhu, ggdcr |
BenchMAX is a multi-way multilingual evaluation benchmark for assessing advanced capabilities of large language models (LLMs) across 17 languages. The main research objective is to create a benchmark that fairly compares LLM capabilities like instruction following, reasoning, and code generation across diverse languages and script systems. The methodology involves machine-translating English tasks into 16 other languages, followed by independent annotation by three native speakers for each sample and task, and final version selection using a strong LLM. A key finding is that DeepSeek-V3 671B model achieved 84.2% on Math and 47.4 on Science reasoning tasks, respectively. For AI practitioners, BenchMAX provides a platform to evaluate LLM performance across languages to improve their multilingual capabilities. |
| CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation (Read more on arXiv or HuggingFace) |
Huchuan Lu, Xu Jia, Xiaoyu Shi, Yawen Luo, Qinghe Wang |
CineMaster is a novel framework for 3D-aware and controllable text-to-video generation, enabling cinematic video creation with precise object placement and camera control. The main research objective is to provide users with 3D-aware and intuitive control over text-to-video generation, similar to the control wielded by film directors. The proposed two-stage framework first allows users to construct 3D scenes and camera movements via an interactive workflow, then uses the generated depth maps, camera trajectories, and object labels to guide a text-to-video diffusion model. CineMaster achieves a mean Intersection over Union (mIoU) of 0.551 and a trajectory deviation (Traj-D) of 66.29, outperforming existing methods in object-box alignment. For AI practitioners, this framework provides a new paradigm for controllable video generation, using a 3D-native approach to enable precise manipulation of scene elements and camera movement directly from textual input and 3D scene descriptions. |
| WorldGUI: Dynamic Testing for Comprehensive Desktop GUI Automation (Read more on arXiv or HuggingFace) |
Mike Zheng Shou, Difei Gao, Henry Hengyuan Zhao |
WorldGUI introduces a new benchmark and framework, GUI-Thinker, for dynamic testing of desktop GUI automation agents. The main research objective is to evaluate and improve GUI agents’ ability to handle diverse initial states and dynamic environments in real-world computer interactions. The methodology involves creating a benchmark (WorldGUI) with 315 tasks across 10 applications, each with varied starting states, and proposing a critical-thinking-based framework (GUI-Thinker) with five core components: Planner, Planner-Critic, Step-Check, Actor, and Actor-Critic. Experimental results demonstrate that GUI-Thinker significantly outperforms existing agents, with the Claude-3.5 based GUI-thinker achieving a 32.4% overall success rate, and GPT-40 based agent achieving 36.2%, exceeding a baseline by 14.9%. For AI practitioners, WorldGUI provides a robust benchmark to test and enhance agent adaptability in varied, dynamic states. |
| LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid (Read more on arXiv or HuggingFace) |
Yu Cheng, Xiaoye Qu, Yiran Zhong, landisen, weigao266 |
LASP-2 improves sequence parallelism for linear attention in transformers by optimizing communication and computation. The main research objective is to enhance the efficiency of sequence parallelism (SP) when training linear attention transformer models with very long input sequences. The key methodology is LASP-2, which reorganizes the communication-computation workflow to require only one AllGather collective communication on intermediate memory states independent of sequence length, and extends this to hybrid models (LASP-2H). Primary results show that LASP-2 achieves training speed improvements of 15.2% over LASP and 36.6% over Ring Attention on a Linear-Llama3 model with a 2048K sequence length across 64 GPUs. For AI practitioners, LASP-2 provides a more efficient way to train linear attention-based and hybrid transformer models on long sequences, reducing training time and resource consumption. |
| TransMLA: Multi-head Latent Attention Is All You Need (Read more on arXiv or HuggingFace) |
Muhan Zhang, Zengwei Yao, fxmeng |
TransMLA converts GQA-based language models to MLA-based models, improving expressiveness without increasing KV cache size. The main research objective is to demonstrate that Multi-head Latent Attention (MLA) offers greater expressive power than Group Query Attention (GQA) for the same key-value (KV) cache overhead. The key methodology involves transforming pre-trained GQA models (e.g., LLaMA, Qwen) into equivalent MLA models via low-rank matrix factorization, followed by fine-tuning. Primary results show that the transformed TransMLA model outperformed the original Qwen2.5-7B GQA model on the GSM8K benchmark (87% vs 81%). The main implication is that the TransMLA transformation provides AI practitioners using open-source, GQA-based LLMs with a low cost method to shift to more effective MLA architecture without changes in KV cache size, enhancing performance. |
| Fino1: On the Transferability of Reasoning Enhanced LLMs to Finance (Read more on arXiv or HuggingFace) |
Yan Wang, Weipeng Zhou, Lingfei Qian, QianqianXie1994, jiminHuang |
The paper evaluates the performance of reasoning-enhanced and general large language models (LLMs) on financial tasks and introduces a new financial reasoning-enhanced model. The main research question is how transferable general-domain reasoning enhancements in LLMs are to the financial domain, and what impact they have across different financial tasks. The methodology involves a comprehensive evaluation of 16 LLMs on three financial datasets (FinQA, DocMath-Simplong, XBRL-Math) encompassing numerical reasoning, tabular interpretation, and financial terminology, followed by developing a model called Fino1. A primary result is that Finol-8B achieved an average score of 61.03 across all datasets, outperforming Llama3.1-8B-Instruct by 10.91 points, with an XBRL-Math score reaching 82.22. The key implication for AI practitioners is that domain-specific fine-tuning with curated financial data, even on a small scale, can significantly improve LLM performance on financial reasoning tasks, surpassing general reasoning enhancements. |
| Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning (Read more on arXiv or HuggingFace) |
lecraquito, Nbeau, supertardigrade |
This paper investigates how varying pre-training levels affect language model exploration in reinforcement learning (RL) fine-tuning, and proposes a modified KL penalty to improve exploration. The main research question is how pre-training data distribution impacts exploration efficiency during RL fine-tuning of language models on tasks requiring out-of-distribution generalization. The key methodology involves pre-training a small language model on an arithmetic addition task with varying digit lengths, then fine-tuning it with RL and a modified KL penalty that prioritizes exploration on “critical tokens”. Primary results show the model with the prioritized KL penalty achieved higher accuracy; for example the accuracy during testing with N=7 was higher when the KL penalty took into account the confidence of the old policy. The principal implication for AI practitioners is that adjusting the KL penalty based on pre-trained model certainty on specific tokens can enhance the efficiency of RL fine-tuning, particularly for tasks requiring generalization beyond the pre-training distribution. |
| Distillation Scaling Laws (Read more on arXiv or HuggingFace) |
Etai Littwin, Jason Ramapuram, Floris Weers, Amitis Shidani, Dan Busbridge |
This paper provides a distillation scaling law that estimates distilled model performance based on compute budget and student/teacher allocation. The main research objective is to determine optimal distillation recipes and understand how to allocate compute resources between teacher and student models to maximize student performance. The key methodology involves a large-scale, controlled study of distillation with students and teachers ranging from 143M to 12.6B parameters, trained on up to 512B tokens, fitting a distillation scaling law to predict student cross-entropy. The primary result is that distillation outperforms supervised pretraining only when the total compute is below a student-size-dependent threshold and a teacher already exists or has uses beyond a single distillation, and student cross-entropy follows a broken power law. The principal implication for AI practitioners is that distillation is beneficial for resource-constrained scenarios or when leveraging existing teachers, guiding optimal model and data scaling during distillation pretraining. |
| SARChat-Bench-2M: A Multi-Task Vision-Language Benchmark for SAR Image Interpretation (Read more on arXiv or HuggingFace) |
HaiPeng Wang, Peidong Wang, Sihao Dong, Xiayang Xiao, JimmyMa99 |
SARChat-Bench-2M is a new benchmark for evaluating vision-language models (VLMs) on synthetic aperture radar (SAR) image interpretation tasks. The main research objective is to develop a large-scale multimodal dialogue dataset and benchmark for evaluating VLMs’ capabilities in SAR image understanding. The key methodology involves constructing a dataset (SARChat-2M) of 2 million SAR image-text pairs and defining six core tasks (classification, description, counting, localization, recognition, and referring) with specific evaluation metrics. Primary results show that the mPLUG-Owl3-7B model achieved the best performance among tested VLMs, with single-target and multi-target cross-modal identification accuracy rates reaching 99.27% and 99.51%, respectively. The principal implication is that AI practitioners can use SARChat-2M and SARChat-Bench to train, evaluate, and advance VLMs for SAR-specific applications, addressing the existing gap in large-scale, high-quality aligned SAR image-text datasets. |
| LLM Pretraining with Continuous Concepts (Read more on arXiv or HuggingFace) |
Andrew Cohen, Jane Yu, Jack Lanchantin, Jihoon Tack, xlxxl |
LLM Pretraining with Continuous Concepts introduces a novel pretraining framework, CoCoMix, that combines discrete next-token prediction with continuous concept learning to enhance language models. The main research objective is to investigate whether augmenting the next token prediction objective with explicit concept modeling in a latent space can improve language model pretraining. The key methodology involves extracting concepts from a pretrained sparse autoencoder, predicting these concepts, and mixing them into the model’s hidden state by interleaving them with token hidden representations. The primary results show that CoCoMix achieves comparable performance to standard next-token prediction with 21.5% fewer training tokens on a 1.38B parameter model. For AI practitioners, CoCoMix offers a more sample-efficient pretraining approach, enhances model interpretability and steerability by allowing direct inspection and modification of the predicted concept, and improves performance in weak-to-strong supervision scenarios. |
| Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance (Read more on arXiv or HuggingFace) |
Dechao Meng, Xin Gao, Zhen Shen, Guangyuan Wang, Hookszdp |
Animate Anyone 2 introduces a diffusion-based framework for character image animation that incorporates environmental context to achieve realistic character-environment interactions. The main research objective is to animate characters with environment affordance, ensuring consistent and interactive relationships between the character and its surroundings. The key methodology involves extracting both motion signals and environmental representations from a source video, using a shape-agnostic mask strategy, an object guider with spatial blending for object interactions, and depth-wise pose modulation. Primary results include a superior SSIM score of 0.812 and FVD of 144.65 on the TikTok benchmark, outperforming existing methods in quantitative evaluations. For AI practitioners, this framework offers a robust method to generate high-fidelity character animations that seamlessly integrate with their environments, useful for applications in filmmaking and advertising. |
| NoLiMa: Long-Context Evaluation Beyond Literal Matching (Read more on arXiv or HuggingFace) |
Ryan A. Rossi, Trung Bui, Hanieh Deilamsalehy, Franck-Dernoncourt, amodaresi |
NOLIMA, a new benchmark, evaluates large language models’ (LLMs) long-context understanding by minimizing literal keyword overlap between questions and answers, emphasizing associative reasoning. Main research question/objective: To assess how well LLMs perform long-context reasoning when they cannot rely on simple literal matches between the question and the context, unlike typical Needle-In-A-Haystack (NIAH) tests. Key methodology: The authors created the NOLIMA benchmark, extending NIAH, where questions and corresponding “needles” (answers) have minimal lexical overlap, requiring models to infer latent associations to locate the needle within a long “haystack” (irrelevant text). They tested 12 LLMs, including GPT-40, and conducted analyses with variations of reasoning complexity, context length, needle placement, and with the presence/absence of literal matching. Primary results: Model performance degraded significantly with increasing context length; at 32K tokens, 10 of the 12 models dropped below 50% of their short-length baseline scores. GPT-4o’s performance decreased from 99.3% baseline to 69.7% at 32K. The presence of literal matches drastically simplified the task, and distractors with literal matches drastically impaired the task. Principal implication for AI practitioners: Current LLMs, even those claiming to support very long contexts, struggle with long-context associative reasoning tasks that lack surface-level (literal) cues, indicating a critical limitation that practitioners should consider when deploying these models in long-context applications. |
| Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing (Read more on arXiv or HuggingFace) |
Peijie Dong, Xinglin Pan, Zhenheng Tang, Kunfeng Lai, Dominic789654 |
Mediator is a framework for merging multiple fine-tuned large language models (LLMs) efficiently by adaptively averaging layers with minimal parameter conflicts and routing layers with significant conflicts. The main research objective is to develop a method for merging LLMs that minimizes parameter conflicts and system costs while preserving performance across diverse tasks. The key methodology involves quantifying layer-wise parameter conflicts, adaptively averaging layers with low conflict and routing layers with high conflict, employing sparse expert decomposition, and using uncertainty-based routing for out-of-distribution samples. Primary results show that Mediator achieves significant performance improvements over existing methods; e.g. on LLaMA-3.2-8B, it achieved 71.80% average on multiple tasks. The principal implication is that AI practitioners can merge fine-tuned LLMs more efficiently to improve the performance and adaptability while reducing the storage and computational costs compared to maintaining separate models. |
| Next Block Prediction: Video Generation via Semi-Autoregressive Modeling (Read more on arXiv or HuggingFace) |
Furu Wei, Xu Sun, Shuming Ma, Shuhuai Ren |
The paper proposes a semi-autoregressive framework called Next-Block Prediction (NBP) for video generation that improves upon traditional next-token prediction. The main research objective is to develop a video generation framework that improves spatial dependency modeling and inference efficiency compared to autoregressive next-token prediction models. The key methodology shifts the generation unit from individual tokens to blocks (e.g., rows or frames), using bidirectional attention within each block and predicting multiple tokens in parallel. The NBP model achieved FVD scores of 103.3 on UCF101 and 25.5 on K600, outperforming the vanilla NTP model by an average of 4.4, with an 11x inference speedup. For AI practitioners, this framework provides a more efficient and scalable solution for video generation, maintaining or improving quality while accelerating inference through parallelization. |
| DPO-Shift: Shifting the Distribution of Direct Preference Optimization (Read more on arXiv or HuggingFace) |
Xiao Li, Lei Zhao, Qianen Zhang, Feng Jiang, Xiliang Yang |
DPO-Shift controllably shifts the distribution of chosen probabilities in Direct Preference Optimization (DPO) to mitigate likelihood displacement. The main research objective is to address the likelihood displacement issue in DPO, where probabilities of chosen responses decrease during training. The key methodology is introducing a parameter function, f(x), added to the rejected reward in the Bradley-Terry model, called DPO-Shift. Experimentally, DPO-Shift with f(x)=0.95 achieved a reward accuracy of 0.743 on the UltraFeedback test set, comparable to DPO’s 0.739, while demonstrably increasing chosen response probability. For AI practioners, DPO-Shift offers a simple, theoretically grounded solution to improve alignment with human preferences by mitigating the likelihood displacement of standard DPO, enabling a trade-off between chosen probability and reward margin. |
| LLM Modules: Knowledge Transfer from a Large to a Small Model using Enhanced Cross-Attention (Read more on arXiv or HuggingFace) |
kkolomeitsev |
The paper introduces LLM Modules, an architecture for transferring knowledge from a large, frozen language model to a smaller, trainable one using Enhanced Cross-Attention. The main objective is to develop a method that enables smaller models to achieve performance comparable to larger models by leveraging the knowledge of pre-trained large language models (LLMs) without full fine-tuning. The key methodology involves using a frozen Qwen2-1.5B model as a “knowledge source” and a GPT-Neo-125M model as a “generation module,” connected by Enhanced Cross-Attention layers that include linear projections, an adapter block, and a gating mechanism. Training on the Bespoke-Stratos-17k dataset for 15 epochs reduced training loss from 13.8 to 2.3 in the first epoch and to 1.1 in subsequent ones. For AI practitioners, the principal implication is that this modular approach can significantly reduce computational costs associated with training large language models while still achieving substantial performance improvements on specific tasks. |
| MetaSC: Test-Time Safety Specification Optimization for Language Models (Read more on arXiv or HuggingFace) |
vicgalle |
MetaSC is a framework that optimizes language model safety reasoning at inference time by dynamically updating safety prompts. The research objective is to improve language model safety performance without modifying model weights. The key methodology is a “meta-critique” mechanism that iteratively updates safety prompts (specifications) to adaptively drive the critique and revision process of a self-critique loop. Primary results show that MetaSC significantly improves safety scores compared to fixed system prompts and static self-critique defenses, achieving a safety score of 1.00 on the jailbreak defense task using the Hermes-3-Llama-3.1-405B model. For AI practitioners, MetaSC offers a way to enhance model safety dynamically at inference time, without retraining or fine-tuning. |
Papers for 2025-02-12
| Title |
Authors |
Summary |
| Competitive Programming with Large Reasoning Models (Read more on arXiv or HuggingFace) |
Borys Minaev, Andre Saraiva, Alexander Wei, Ahmed El-Kishky, OpenAI |
Reinforcement learning significantly improves large language models’ performance on complex coding and reasoning tasks. The main research question is how domain-specific, hand-engineered inference strategies compare to learned approaches in competitive programming. The key methodology involved fine-tuning large language models with reinforcement learning and comparing performance with and without hand-crafted test-time strategies. The primary result was that OpenAI’s o3 model achieved a Codeforces rating of 2724 (99.8th percentile) and an IOI 2024 score of 395.64, surpassing a gold medal threshold without hand-engineered strategies. Scaling general-purpose reinforcement learning presents a robust method toward state-of-the-art AI in reasoning tasks like competitive programming. |
| CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction (Read more on arXiv or HuggingFace) |
Yu Wu, Runxin Xu, Dejian Yang, Daya Guo, Junlong Li |
CODEI/O systematically condenses diverse reasoning patterns in code for improved performance on reasoning tasks. The main research objective is to improve the performance of Large Language Models (LLMs) on a broad range of reasoning tasks by leveraging code-based training data. The key methodology involves transforming raw code files into an input-output prediction format and training LLMs to predict either the output given code and input, or feasible input given code and output, entirely in natural language as Chain-of-Thought rationales. Primary results demonstrate consistent improvements across 14 benchmarks spanning symbolic, scientific, logic, math & numerical, and commonsense reasoning, with CODEI/O++ achieving an average score improvement of 2.9 points, compared to single stage training on Qwen 2.5 Coder 7B. For AI practitioners, this implies that training on code input-output prediction tasks can enhance LLMs’ general reasoning capabilities beyond code-specific applications. |
| Magic 1-For-1: Generating One Minute Video Clips within One Minute (Read more on arXiv or HuggingFace) |
Qingyu Yin, Jiantong Zhao, Shitong Shao, Hongwei Yi, Owen777 |
Magic 1-For-1 is an efficient video generation model that optimizes memory consumption and inference latency. The main objective is to reduce the computational cost and time required for text-to-video generation while maintaining high video quality. The key methodology involves factorizing the text-to-video task into text-to-image and image-to-video subtasks, alongside model convergence speedup, adversarial step distillation, and parameter sparsification. The primary results show the model can generate 5-second video clips within 3 seconds, and achieves an average score of 0.8134 on a customized VBench, outperforming other models. The principal implication for AI practitioners is that it offers an approach for generating minute-long videos within one minute, optimizing the tradeoff between computational cost and video quality for diffusion-based video generation. |
| Teaching Language Models to Critique via Reinforcement Learning (Read more on arXiv or HuggingFace) |
Jingjing Xu, Weichao Mao, Liyu Chen, Jie chen, Zhihui |
CTRL trains large language models (LLMs) to provide effective feedback on code, improving iterative code generation. The main research objective is to develop a framework, CTRL, that trains a critic model to generate feedback that maximizes correction performance for a fixed generator model, without human supervision. The methodology uses a two-stage approach: supervised finetuning using execution feedback to synthesize critiques, followed by reinforcement learning with Group Relative Policy Optimization (GRPO) to optimize the critic. The results demonstrate that critics trained with CTRL significantly enhance pass rates, achieving up to 106.1% relative improvement on the CodeContests benchmark when using the same base model for generation and critique, and 23.5% improvement when paired with a better generator. For AI practitioners, CTRL provides a method to create specialized critics that can substantially improve code generation performance through effective, targeted feedback, enabling more autonomous AI systems. |
| Expect the Unexpected: FailSafe Long Context QA for Finance (Read more on arXiv or HuggingFace) |
Mateusz Russak, Dmytro Mozolevskyi, Melisa Russak, muayad, kiranr |
FailSafeQA, a new long-context financial benchmark, evaluates LLM robustness and context-awareness against variations in human-interface interactions. i) This paper introduces FailSafeQA, a new benchmark for evaluating the robustness of Large Language Models (LLMs) in financial question-answering systems, particularly when dealing with long contexts and imperfect user inputs. ii) The main research objective is to assess the resilience of LLMs against six variations in human-input interactions, such as query failure (misspelled, incomplete and out-of-domain) and context failure (degraded, irrelevant, and missing). iii) The key methodology uses the LLM-as-a-Judge approach with Qwen2.5-72B-Instruct and defines fine-grained rating criteria to calculate Robustness, Context Grounding, and Compliance scores for 24 LLMs. The input consists of truncated 10k filings. iv) The most robust model, OpenAI 03-mini, fabricated information in 41% of tested cases, while Palmyra-Fin-128k-Instruct, the most compliant model, failed robust predictions in 17% of test cases. v) AI practitioners should be aware that high-performing LLMs still have significant room for improvement in terms of balancing robustness and context grounding. Practitioners must carefully assess the trade-off between a model’s ability to handle imperfect inputs and its tendency to hallucinate. |
| LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters! (Read more on arXiv or HuggingFace) |
Xiangxi Mo, Shu Liu, Tyler Griggs, Shiyi Cao, Dacheng Li |
Large language models (LLMs) can be efficiently fine-tuned to perform complex reasoning by learning the structural patterns of long chain-of-thought (CoT) demonstrations. The main research question is how to effectively elicit Long CoT reasoning capabilities in LLMs and what aspects of training data are most important. The key methodology involved supervised fine-tuning and low-rank adaptation (LoRA) on LLMs, with controlled experiments perturbing either the content or structure of Long CoT training samples. A primary result was that a Qwen2.5-32B-Instruct model achieved 56.7% accuracy on AIME 2024 after fine-tuning with only 17k Long CoT samples. AI practitioners can elicit strong reasoning performance in LLMs with relatively small, structurally sound datasets, without needing perfect accuracy in the content of individual reasoning steps. |
| Éclair – Extracting Content and Layout with Integrated Reading Order for Documents (Read more on arXiv or HuggingFace) |
Lukas Voegtle, Ilia Karmanov, jseppanen, katerynaCh, amalad |
ÉCLAIR, a multi-modal large language model (MLLM), extracts structured text, bounding boxes, and semantic classes from documents in integrated reading order. The main research objective is to develop a general-purpose text-extraction tool capable of processing diverse document types and extracting formatted text, spatial information, and semantic class labels simultaneously. The key methodology involves a transformer encoder-decoder architecture with a ViT-like encoder and an autoregressive decoder, pre-trained on a newly generated arXiv-5M dataset and fine-tuned on diverse public datasets. The primary results include achieving state-of-the-art accuracy on the new DROBS benchmark with a 0.937 Counting F1 score and outperforming other methods on established benchmarks. The principal implication for AI practitioners is that ÉCLAIR provides a new model for document OCR, enabling the extraction of more structured data from documents. |
| CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing (Read more on arXiv or HuggingFace) |
Jiang Bian, Qi Liu, Yu Yuan, ShizhaoSun |
CAD-Editor is a framework for automatically modifying CAD models based on textual instructions, using an automated data synthesis pipeline and a locate-then-infill approach. The main research objective is to develop a system for text-based editing of CAD models, addressing the lack of support for text-based control in existing design variation methods and the absence of consideration for existing CAD models as constraints. The methodology involves generating synthetic training data using design variation models and LVLMs and decomposing the task into locating regions for modification and infilling those regions with LLMs. Primary results show that CAD-Editor achieves a 95.6% Valid Ratio and a 0.27 Directional CLIP Score, outperforming baseline methods in generation validity, text-CAD alignment, and overall quality. AI practitioners can leverage the proposed framework and data synthesis pipeline to enable more intuitive and efficient CAD model editing through natural language instructions, accelerating the design workflow. |
| Enhance-A-Video: Better Generated Video for Free (Read more on arXiv or HuggingFace) |
Wenqi Shao, Kaipeng Zhang, Mengzhao Chen, Xuanlei Zhao, Yang Luo |
Enhance-A-Video is a training-free method to improve the temporal consistency and visual quality of diffusion transformer (DiT)-based video generation. The main research objective is to develop a method to enhance the coherence and quality of DiT-based generated videos without retraining or fine-tuning. The key methodology involves introducing a “Enhance Block” that calculates a Cross-Frame Intensity (CFI) from temporal attention maps and uses an “enhance temperature” parameter to scale and integrate this CFI, thereby strengthening cross-frame correlations. User studies demonstrated that models incorporating Enhance-A-Video were preferred across metrics including temporal consistency, prompt-video consistency, and overall visual quality, and VBench scores consistently improved across all tested models. AI practitioners can integrate this plug-and-play method into existing DiT-based video generation frameworks to improve video quality at minimal computational cost, without any retraining or fine tuning of models. |
| NatureLM: Deciphering the Language of Nature for Scientific Discovery (Read more on arXiv or HuggingFace) |
Chuan Cao, Liang He, Shufang Xie, Peiran Jin, Yingce Xia |
NatureLM is a sequence-based science foundation model designed for scientific discovery across multiple domains. Main research question or objective: To develop a unified, versatile model capable of handling various scientific applications, including generation and optimization, across multiple scientific domains using a sequence-based approach. Key methodology used: A Transformer decoder architecture pre-trained on 143 billion tokens from multiple scientific domains (small molecules, proteins, DNA, RNA, materials, and text), followed by post-training with instruction-response pairs. Primary results: NatureLM (8x7B) achieved state-of-the-art performance in retrosynthesis (71.9% top-1 accuracy on USPTO-50K) and SMILES-to-IUPAC translation (0.607 top-5 accuracy), significantly outperforming general-purpose foundation models. Principal implication for AI practitioners: Practitioners can utilize NatureLM as a foundation model for diverse scientific tasks, particularly where cross-domain interactions and sequence-based representations are crucial, potentially accelerating scientific discovery through a generalist model approach. |
| Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (Read more on arXiv or HuggingFace) |
Kewei Cheng, Xin Liu, Haoming Jiang, Jingfeng Yang, yczhuang |
Hephaestus introduces a continual pre-training method to enhance the fundamental capabilities of LLM-based agents. Main research question or objective: How can continual pre-training on a large-scale, agent-oriented corpus improve the API function calling, intrinsic reasoning, and environmental feedback adaptation capabilities of large language models? Key methodology used: A two-stage continual pre-training framework on the Hephaestus-Forge corpus (103B tokens, 76,537 APIs), leveraging scaling law experiments to optimize data mixing ratios, followed by instruction fine-tuning. Primary results: Hephaestus-8B outperforms LLAMA-3-8B by 9.6% and rivals commercial LLMs on three agent benchmarks, achieves comparable performance with GPT-3.5-turbo, excelling particularly in complex multi-turn tasks (BFCL-v3). Principal implication for AI practitioners: Continual pre-training with a well-curated, agent-specific corpus like Hephaestus-Forge can significantly enhance fundamental agent capabilities of open-source LLMs, bridging the performance gap with commercial models and providing a more robust and generalizable foundation for LLM-based agent development. |
| Forget What You Know about LLMs Evaluations - LLMs are Like a Chameleon (Read more on arXiv or HuggingFace) |
Seffi Cohen, Lior Rokach, Bracha Shapira, Yehonatan Elisha, Nurit Cohen-Inger |
This paper introduces a meta-evaluation framework, Chameleon Benchmark Overfit Detector (C-BOD), to detect overfitting in Large Language Models (LLMs) on benchmark datasets. The central research question is whether LLMs over-rely on benchmark-specific cues, exhibiting surface-level performance rather than true language understanding. The methodology involves systematically perturbing benchmark prompts using a parametric transformation (controlled by parameter µ) and assessing performance changes with statistical significance tests (McNemar’s test). A primary result is that 20 out of 26 tested LLMs showed statistically significant performance degradation on the MMLU benchmark under modest perturbations, with an average accuracy drop of 2.15%. AI practitioners should integrate C-BOD’s perturbation methods into evaluation pipelines to ensure robust generalization and mitigate superficial memorization in LLMs, prioritizing model resilience over high scores on fixed benchmarks. |
| VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation (Read more on arXiv or HuggingFace) |
Hang Xu, Yi Zhu, Yanpeng Zhou, Zimian Peng, Sixiao Zheng |
VidCRAFT3 is a novel image-to-video generation framework enabling precise control over camera motion, object motion, and lighting direction. The main research objective is to develop a model that can simultaneously control multiple visual elements (camera motion, object motion, and lighting) in image-to-video generation, overcoming the limitations of existing methods. The key methodology involves a Spatial Triple-Attention Transformer integrating lighting, text, and image features, along with 3D point cloud rendering and trajectory-based motion encoding, and using a three-stage training process. Primary results show the model achieves a CamMC score of 4.07 on the RealEstate10K dataset, outperforming existing methods like CameraCtrl, CamI2V and MotionCtrl. The principal implication is that AI practitioners can use VidCRAFT3 to create high-quality videos with fine-grained and disentangled control over multiple aspects. |
| Retrieval-augmented Large Language Models for Financial Time Series Forecasting (Read more on arXiv or HuggingFace) |
Yueru He, Zhengyu Chen, Lingfei Qian, Zihao Jiang, Mengxi Xiao |
This paper introduces a retrieval-augmented generation (RAG) framework, FinSeer, for financial time-series forecasting, specifically stock movement prediction. The main research objective is to develop a RAG framework that effectively integrates financial time-series data with large language models (LLMs) to improve stock movement prediction accuracy. The key methodology involves a fine-tuned 1B parameter LLM (StockLLM), a novel candidate selection method using LLM feedback, and a training objective maximizing similarity between queries and historically significant sequences. The RAG framework with FinSeer achieved an 8% higher accuracy on the BIGDATA22 benchmark compared to a general-purpose LLM-feedback-based retriever. For AI practitioners, this framework demonstrates the importance of using dedicated retrieval models designed to process and filter financial time-series data, to improve the performance of the LLMs in financial forecasting tasks. |
| Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More (Read more on arXiv or HuggingFace) |
Li Shen, Zhenyu Zhang, Jianjin Li, Zhikai Jia, Xialie Zhuang |
Mask-Enhanced Autoregressive Prediction (MEAP) integrates masked language modeling into next-token prediction to improve large language models’ in-context retrieval capabilities without extra computational cost. The main research objective is to enhance LLMs’ ability to retrieve key information and perform long-context reasoning without compromising their fundamental language modeling capabilities. MEAP randomly masks a fraction of input tokens and then performs standard next-token prediction using a decoder-only Transformer. In pre-training, MEAP outperformed NTP on the Needle in a Haystack evaluation by 11% on average using 140B less training token. This demonstrates MEAP’s superior performance in key information retrieval tasks, and thus provides AI practitioners with a more data- and compute-efficient training paradigm for large language models. |
| FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks (Read more on arXiv or HuggingFace) |
Mirco Ravanelli, Cem Subakan, Francesco Paissan, lucadellalib |
FocalCodec is a low-bitrate speech codec based on focal modulation that uses a single binary codebook for compression. The research objective is to develop a speech codec that achieves high compression rates while preserving both semantic and acoustic information for downstream tasks. The key methodology involves a compressor-quantizer-decompressor architecture utilizing focal modulation, binary spherical quantization (BSQ), and a pretrained self-supervised encoder (WavLM). Primary results show that FocalCodec@50 achieves a dWER of 2.18 on the LibriSpeech test-clean set, outperforming several baselines at comparable bitrates. AI practitioners can use FocalCodec as an efficient and low-bitrate option that can be deployed to preserve sufficient semantic and acoustic information for downstream tasks, such as speech resynthesis, voice conversion, or speech enhancement model development. |
| Auditing Prompt Caching in Language Model APIs (Read more on arXiv or HuggingFace) |
Percy Liang, Rohith Kuditipudi, Xiang Lisa Li, Chenchen Gu, thashim |
Prompt caching in large language model APIs can leak private and proprietary information through timing differences, which can be detected by auditing. The main research objective was to develop and conduct statistical audits to detect prompt caching and determine the level of cache sharing (per-user, per-organization, or global) in real-world LLM API providers. The key methodology was using statistical hypothesis testing on response times from two procedures: one to generate cache hits, and one to generate cache misses, analyzing differences using the two-sample Kolmogorov-Smirnov test. The primary results revealed that prompt caching was detected in 8 out of 17 API providers, with 7 exhibiting global cache sharing across users, where it was detected with an average precision of around 0.8. AI practitioners should be aware of prompt caching implementation details and cache-sharing levels in LLM APIs to mitigate potential privacy leakage, since the caching can be identified from timing data. |
| Gemstones: A Model Suite for Multi-Faceted Scaling Laws (Read more on arXiv or HuggingFace) |
Abhinav Bhatele, Siddharth Singh, David Yu Miller, John Kirchenbauer, smcleish |
Gemstones provides a dataset of over 4000 transformer checkpoints to study scaling laws across various architectural and training hyperparameters. The main research question is how model design (width, depth) and model selection impact scaling law parameters and interpretations. The key methodology involves training transformers, up to 2 billion parameters, with diverse widths, depths, learning rates, and cooldown schedules, then fitting and analyzing scaling laws on this data. The primary results show scaling law prescriptions are highly sensitive to model selection and fitting procedures; for example, the optimal tokens-per-parameter ratio is slightly higher than that proposed in previous works. The principal implication for AI practitioners is that scaling laws should be approached with awareness for fragility, with a recommendation to err on wider and, surprisingly, over-trained models, especially when considering time optimality. |
| Skill Expansion and Composition in Parameter Space (Read more on arXiv or HuggingFace) |
Yixing Lan, Haoyi Niu, Yinan Zheng, Jianxiong Li, LTL07 |
i) The paper introduces Parametric Skill Expansion and Composition (PSEC), a framework for iteratively expanding agent capabilities. ii) The research aims to develop an autonomous agent that can efficiently acquire new skills by leveraging prior knowledge and dynamically composing existing skills. iii) PSEC employs parameter-efficient finetuning using Low-Rank Adaptation (LoRA) modules for skill expansion and a context-aware module for skill composition in parameter space. iv) Experiments on D4RL show PSEC demonstrates the superior capacity to efficiently tackle new challenges. v) PSEC provides AI practitioners with a method for continual learning and efficient skill transfer in reinforcement learning agents, mitigating catastrophic forgetting through parameter isolation. |
Papers for 2025-02-11
| Title |
Authors |
Summary |
| SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators (Read more on arXiv or HuggingFace) |
Alexander Panchenko, tlenusik, memyprokotow, chameleon-lizard, etomoscow |
This paper introduces SynthDetoxM, a multilingual synthetic parallel text detoxification dataset, and a framework for generating such data using large language models (LLMs). The main research objective is to address the scarcity of parallel multilingual datasets for training text detoxification models. The key methodology involves few-shot prompting of multiple open-source LLMs to rewrite toxic sentences sourced from existing toxicity datasets across German, French, Spanish, and Russian, followed by a filtering and ranking process. Models trained on the full SynthDetoxM achieved a J score (combining style transfer accuracy, similarity, and fluency) of 0.484, 0.521, and 0.471 on German, Russian and Spanish respectively. The principal implication is that AI practitioners can leverage the proposed framework and the SynthDetoxM dataset to train more effective multilingual text detoxification models, even with limited human-annotated parallel data. |
| Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning (Read more on arXiv or HuggingFace) |
Yuzhe Gu, Songyang Gao, Chengqi Lyu, zsytony, ZwwWayne |
This paper introduces OREAL, a new reinforcement learning (RL) framework for enhancing mathematical reasoning in large language models (LLMs) using only binary outcome rewards. The main research objective is to push the performance limit achievable through Outcome REwArd-based reinforcement learning (OREAL) for mathematical reasoning tasks. The key methodology involves behavior cloning on positive trajectories from Best-of-N sampling, reward shaping for negative samples, and a token-level reward model for credit assignment. OREAL achieves a 95.0 pass@1 accuracy on MATH-500 with a 32B model, and a 7B model can obtain 94.0 pass@1 accuracy on MATH-500. AI practitioners can utilize OREAL’s techniques to improve LLM performance on mathematical reasoning tasks using readily available binary outcome feedback, emphasizing the importance of policy model initialization and proper training data selection. |
| Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling (Read more on arXiv or HuggingFace) |
Xiu Li, Jian Zhao, Junqi Gao, iseesaw, RyanLiu112 |
This paper investigates compute-optimal test-time scaling (TTS) strategies for Large Language Models (LLMs), demonstrating that smaller LLMs can outperform larger ones with appropriate scaling. The main research question is what is the optimal approach to scaling test-time computation across different policy models, Process Reward Models (PRMs), and problem difficulty levels, and to what extent can it improve performance. The key methodology involves comprehensive experiments on MATH-500 and AIME24 tasks using various LLMs (0.5B to 72B) and PRMs (1.5B to 72B), evaluating different TTS methods like Best-of-N, beam search, and Diverse Verifier Tree Search. The primary results show that a 3B LLM with compute-optimal TTS can surpass a 405B LLM, achieving 75.6% on MATH-500 and 30.0% on AIME24, compared to 71.4% and 23.3% for the 405B model with Chain-of-Thought prompting. The principal implication for AI practitioners is that applying compute-optimal, reward-aware TTS strategies can significantly enhance the reasoning abilities of smaller LLMs, potentially leading to more efficient and effective deployment compared to using much larger models. |
| Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding (Read more on arXiv or HuggingFace) |
Soyeong Jeong, Jeongyeon Seo, Sangjin Choi, doubleyyh, zomss |
Hierarchy Drafting (HD) accelerates large language model (LLM) inference by organizing token sources into hierarchical databases based on temporal locality and accessing them sequentially during speculative decoding. Main research question or objective: To address the limitations of existing speculative decoding methods, which rely on a single database, require additional fine-tuning or deliver inconsistent acceleration gains. Key methodology used: The proposed method, Hierarchy Drafting (HD), organizes diverse token sources into three databases (context-dependent, model-dependent, and statistics-dependent) based on temporal locality and accesses them sequentially during speculative decoding, starting from the smallest to largest. Primary results: Experiments on Spec-Bench using LLMs with 7B and 13B parameters demonstrate that HD outperforms existing lossless drafting methods, achieving over 1.5x faster inference speed compared to autoregressive decoding when the temperature is 0.0. Principal implication for AI practitioners: AI practitioners can achieve significant and consistent lossless inference acceleration in LLMs without model retraining or modification, using readily accessible data sources, by employing HD, making it suitable for real-world deployment. |
| Show-o Turbo: Towards Accelerated Unified Multimodal Understanding and Generation (Read more on arXiv or HuggingFace) |
Yishun Li, Zhenyi Liao, zhijie3, asunalove, UnhurriedDawn |
Show-o Turbo accelerates the unified multimodal understanding and generation model Show-o by extending consistency distillation to its multimodal denoising trajectories. The main research question is whether a unified approach exists to enhance the efficiency of Show-o’s inference, which involves denoising image tokens and autoregressively decoding text tokens. The key methodology involves viewing text generation as a denoising process using Jacobi decoding, extending consistency distillation (CD) to multimodal discrete sampling trajectories, and employing trajectory segmentation and curriculum learning. Show-o Turbo achieves a GenEval score of 0.625 at 4 sampling steps without classifier-free guidance (CFG), outperforming the original Show-o with 8 steps and CFG, in text-to-image generation and 1.5 speedup on image-to-text task. AI practitioners can leverage this approach to deploy more efficient multimodal models that achieve significant speedups in both image and text generation tasks with minimal performance trade-offs. |
| Training Language Models for Social Deduction with Multi-Agent Reinforcement Learning (Read more on arXiv or HuggingFace) |
Dorsa Sadigh, C. Karen Liu, Warren Xia, bidiptas |
Language models are trained to communicate effectively in a multi-agent social deduction game without human demonstrations, enhancing their ability to reason and strategize. The main research objective is to train language models to have productive natural language discussions about their environment, leveraging the agent’s goal for predicting useful information. The methodology decomposes communication into listening and speaking, using a dense reward signal based on imposter prediction and influence on other agents’ beliefs to guide multi-agent reinforcement learning. Crewmate agents trained with the proposed technique achieve double the win rate compared to standard reinforcement learning, illustrating the value of the communication strategy. AI practitioners can utilize the described approach to enable self-improving discussions in multi-agent settings without requiring task-specific human data, potentially broadening the application of language models in cooperative AI. |
| ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates (Read more on arXiv or HuggingFace) |
Mengdi Wang, Bin Cui, Zhaochen Yu, Ling Yang |
ReasonFlux is a hierarchical LLM reasoning framework that optimizes mathematical reasoning by scaling thought templates. The main research objective is to improve LLMs’ mathematical reasoning capabilities beyond existing models like OpenAI’s o1-preview and DeepSeek V3. The key methodology involves a structured thought template library, hierarchical reinforcement learning on template sequences, and an inference scaling system that adaptively retrieves and applies templates. On the MATH benchmark, ReasonFlux-32B achieves an accuracy of 91.2%, surpassing o1-preview by 6.7%. AI practitioners can leverage ReasonFlux’s hierarchical template-based approach for more efficient and generalizable reasoning in complex problem-solving applications, requiring less computational resources. |
| The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering (Read more on arXiv or HuggingFace) |
Zhenting Wang, Di Liu, Yunhe Gao, Haizhou Shi, Zhuowei Li |
This paper introduces VISTA, a training-free framework to reduce hallucination in Large Vision-Language Models (LVLMs) by steering token generation with visual information. The main research objective is to investigate and mitigate the phenomenon of LVLMs generating syntactically coherent but visually ungrounded content. The key methodology, VISTA, combines a Visual Steering Vector (VSV) to reinforce visual cues in activation space and Self-Logits Augmentation (SLA) to leverage early-layer activations for semantically meaningful decoding. Primary results show that VISTA reduces hallucination by about 40% on average in open-ended generation tasks, outperforming existing methods across multiple architectures and decoding strategies. The principal implication for AI practitioners is that VISTA provides an efficient, inference-time intervention to improve the visual grounding and reliability of LVLMs without requiring additional training or model modification. |
| Matryoshka Quantization (Read more on arXiv or HuggingFace) |
Aditya Kusupati, Prateek Jain, Jeff Dean, Puranjay Datta, Pranav Nair |
Matryoshka Quantization (MatQuant) is a multi-scale quantization technique that trains a single model capable of operating at various integer bit-widths. The main research question is whether a single model can be trained to extract multiple accurate lower-precision models, addressing the challenges of accuracy loss in low-precision quantization and the need for maintaining multiple models. The key methodology is Matryoshka Quantization, which jointly optimizes model weights across multiple precision levels (e.g., int8, int4, int2) using shared most significant bits and leveraging the inherent nested structure of integer data types. Primary results show that MatQuant-derived int2 models outperform standard int2 quantization techniques by up to 10% in accuracy, and an int2 FFN-quantized Gemma-2 9B model is more accurate than an int8 FFN-quantized Gemma-2 2B model. The principal implication is that AI practitioners can train and maintain a single quantized model that can be served at different precision levels, offering a spectrum of accuracy-versus-cost options and improving accuracy, especially in very low precision regimes like int2. |
| EVEv2: Improved Baselines for Encoder-Free Vision-Language Models (Read more on arXiv or HuggingFace) |
Yueze Wang, Yufeng Cui, Xiaotong Li, Haiwen Diao, PhyscalX |
EVEv2.0 is a new family of encoder-free vision-language models (VLMs) that improve upon existing baselines through architectural and training enhancements. The main research objective is to systematically investigate and improve the performance of encoder-free VLMs, addressing challenges like cross-modal interference and visual perception learning from scratch. The key methodology involves a “Divide-and-Conquer” architecture that decomposes the model into modality-specific components within a unified decoder-only framework, along with a progressive training strategy utilizing an enhanced captioning engine. Primary results show that EVEv2.0 achieves 71.4% accuracy on ScienceQA-IMG, outperforming prior encoder-free models, while approaching the performance of encoder-based counterparts with similar capacity, using only 100M publicly available data. The principal implication for AI practitioners is that properly decomposing and associating modalities, combined with a well-designed training strategy, allows for effective optimization of decoder-only VLMs, providing superior data efficiency and strong visual-reasoning capability, and thereby improving performance of large language models. |
| LM2: Large Memory Models (Read more on arXiv or HuggingFace) |
Fraser Greenlee, Alex J. Chan, Filippos Christianos, Wenqi Wu, Jikun Kang |
LM2 is a memory-augmented Transformer architecture designed to improve long-context reasoning in language models. The main research objective is to address the limitations of standard Transformers in processing long contexts with distributed information, particularly for tasks involving multi-step reasoning and relational argumentation. The key methodology involves integrating a dynamic memory module into the decoder-only Transformer, using cross-attention and gating mechanisms to update and retrieve contextual representations. Experimental results on the BABILong benchmark show LM2 outperforms the memory-augmented RMT model by 37.1% and the baseline Llama-3.2 model by 86.3% on average across tasks. The principal implication for AI practitioners is that incorporating explicit memory modules, as done in LM2, can enhance a Transformer’s ability to handle long-context reasoning tasks without sacrificing performance on general tasks, which has significance for NLP applications. |
| Lumina-Video: Efficient and Flexible Video Generation with Multi-scale Next-DiT (Read more on arXiv or HuggingFace) |
Kai Wang, Zhen Li, Yutong Liu, Shicheng Li, Dongyang Liu |
Lumina-Video is a novel framework for efficient and flexible video generation based on an enhanced Diffusion Transformer architecture. The main research objective is to address the spatiotemporal complexity and computational challenges of video generation using Diffusion Transformers (DiTs). The key methodology involves a Multi-scale Next-DiT architecture with multiple patch sizes, motion score conditioning, progressive training, and multi-source training. Lumina-Video achieves a total score of 82.94% on the VBench benchmark, demonstrating competitive performance in generating high-quality videos. AI practitioners can leverage Lumina-Video’s Multi-Scale Next-DiT and training strategies to build efficient and flexible video generation models with controllable dynamics. |
| History-Guided Video Diffusion (Read more on arXiv or HuggingFace) |
Russ Tedrake, Yilun Du, Max Simchowitz, Boyuan Chen, Kiwhan Song |
The paper introduces a video diffusion model, DFoT, and a family of guidance methods, History Guidance (HG), that improve video generation quality and consistency by leveraging variable-length historical frames. The main research question is how to effectively use different portions of video history as a form of guidance for improved video generation. The key methodology involves the Diffusion Forcing Transformer (DFoT), which allows conditioning on flexible history lengths, and History Guidance methods, which combine scores from different history windows and noise levels. A primary result is that DFoT with history guidance achieves a Fréchet Video Distance (FVD) of 170.4 on Kinetics-600, outperforming baselines. AI practitioners can use DFoT and History Guidance to improve the quality, consistency, and length of generated videos, especially for tasks requiring long-term coherence. |
| CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformers (Read more on arXiv or HuggingFace) |
Zhen Yang, Jin Wang, Jingxuan Pang, Mushui Liu, D. She |
CustomVideoX is a zero-shot personalized video generation framework based on the Video Diffusion Transformer, enhancing video quality and temporal coherence. The main research objective is to develop a method for generating customized videos from a reference image and text prompt, addressing temporal inconsistencies and quality degradation issues. The key methodology involves integrating 3D Reference Attention for direct interaction between reference image and video frames, Time-Aware Attention Bias to modulate reference feature influence, and Entity Region-Aware Enhancement for focused feature injection. Primary results show that CustomVideoX achieves a CLIP-I score of 90.26 and DINO-I score of 91.49 on the VideoBench benchmark, outperforming other methods. AI practitioners can leverage CustomVideoX’s architecture for improved zero-shot personalized video generation, specifically benefiting from the 3D Reference Attention and time-aware mechanisms for better fidelity and consistency. |
| APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding (Read more on arXiv or HuggingFace) |
Beidi Chen, Tianqi Chen, Hanyuezhuohua |
APE improves context-augmented generation by enabling faster and longer context processing through adaptive parallel encoding. The main research objective is to address the computational burden and performance degradation of existing context-augmented generation (CAG) techniques when handling multiple, lengthy contexts. The key methodology, Adaptive Parallel Encoding (APE), uses a shared prefix, attention temperature, and scaling factor to align the distribution of parallel encoding with sequential encoding. Results show that APE preserves 98% of sequential encoding performance on RAG tasks while enabling an end-to-end 4.5x speedup by reducing prefilling time by 28x for a 128K-length context. The principal implication for AI practitioners is that APE enables more efficient and scalable deployment of CAG systems, particularly those dealing with long and numerous contexts, by reducing computational costs and improving response times. |
| Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile (Read more on arXiv or HuggingFace) |
Peiyuan Zhang, Runlong Su, Dacheng Li, zhijie3, foreverpiano |
EFFICIENT-VDIT accelerates video diffusion transformers by sparsifying 3D attention and reducing sampling steps. The main research objective is to address the computational inefficiency of 3D full attention diffusion transformers (DiTs) during video generation. The key methodology involves identifying and leveraging a “tile-style” repetitive pattern in 3D attention maps to create sparse attention masks, combined with multi-step consistency distillation. The primary result is that EFFICIENT-VDIT achieves up to a 7.8x speedup on Open-Sora-Plan-1.2 models for 29 and 93 frame video generation with minimal performance degradation on VBench. For AI practitioners, this method provides a way to significantly speed up video generation with 3D DiTs, enabling faster inference and potentially reducing computational costs. |
| MetaChain: A Fully-Automated and Zero-Code Framework for LLM Agents (Read more on arXiv or HuggingFace) |
Chao Huang, Tianyu Fan, Jiabin Tang |
MetaChain is a framework enabling fully-automated, zero-code development and deployment of LLM agents through natural language alone. The main research question is: Can we enable everyone, regardless of technical background, to build their own LLM agents using natural language alone? The key methodology involves a novel LLM Agent Framework with four components: Agentic System Utilities, LLM-powered Actionable Engine, Self-Managing File System, and Self-Play Agent Customization module, enabling automated agent generation, customization, and workflow optimization. Primary results include ranking #1 among open-source solutions on the GAIA benchmark and achieving 73.51% accuracy on a MultiHop-RAG task. The principal implication for AI practitioners is that MetaChain democratizes agent development, allowing non-programmers to create and customize LLM agents and workflows, potentially accelerating the adoption of agent technology. |
| Steel-LLM:From Scratch to Open Source – A Personal Journey in Building a Chinese-Centric LLM (Read more on arXiv or HuggingFace) |
Zhaoxiang Zhang, Shu Li, Qingshui Gu, aaabiao |
Steel-LLM is a fully open-source, 1-billion-parameter, Chinese-centric language model developed with limited computational resources. The main objective was to create a high-quality, transparent, and resource-efficient language model, primarily trained on Chinese data, with a small proportion of English. The methodology involved adapting a Qwen-based Transformer architecture with Soft Mixture of Experts and an enhanced Feed-Forward Network, trained using a modified TinyLlama framework on 8 A100/H800 GPUs. The model achieved a CEVAL accuracy of 41.90% and a CMMLU accuracy of 36.08% after supervised finetuning. AI practitioners can use the provided training pipeline, datasets, model architecture, and intermediate checkpoints to develop or extend similar language models with limited resources, facilitating reproducibility and further research. |
| The Curse of Depth in Large Language Models (Read more on arXiv or HuggingFace) |
Yefeng Zheng, Lu Yin, Xinyuan Song, Wenfang Sun, pengxiang |
The paper introduces “Curse of Depth” in large language models (LLMs), where deeper layers contribute less than expected due to Pre-Layer Normalization (Pre-LN), and proposes LayerNorm Scaling to address it. The main research objective is to identify and rectify the phenomenon where deeper layers in LLMs are less effective, specifically investigating the role of Pre-LN in this issue. The key methodology involves theoretical analysis of Pre-LN’s impact on variance and gradient flow, alongside empirical evaluations via layer pruning experiments and comparisons of different normalization techniques. A primary result is that LayerNorm Scaling reduces perplexity by 1.31 on LLaMA-1B compared to standard Pre-LN. The principal implication for AI practitioners is that applying LayerNorm Scaling, which inversely scales the output of Pre-LN by the square root of the layer depth, can improve LLM performance by enhancing the contribution of deeper layers during training, creating more resource-efficient models. |
| DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization (Read more on arXiv or HuggingFace) |
Yi Yang, Hehe Fan, Fan Ma, Xiaobo Xia, Zhenglin Zhou |
DreamDPO is an optimization-based framework for text-to-3D generation that aligns 3D content with human preferences through direct preference optimization. The main research objective is to improve the alignment of text-to-3D generated content with human preferences and enhance controllability. The methodology involves constructing pairwise examples, comparing their alignment with human preferences using reward or large multimodal models, and optimizing the 3D representation with a preference-driven loss function. DreamDPO achieved a GPTEval3D overall score of 1203.1, outperforming 13 state-of-the-art methods, including MVDream (1097.7). AI practitioners can utilize DreamDPO to generate higher-quality and more controllable 3D content, moving beyond pointwise quality evaluations by utilizing pairwise comparisons and preference optimization. |
| Dual Caption Preference Optimization for Diffusion Models (Read more on arXiv or HuggingFace) |
Bimsara Pathiraja, Shamanthak Hegde, Agneet Chatterjee, Yiran Luo, sahsaeedi |
Dual Caption Preference Optimization (DCPO) improves text-to-image diffusion models by using distinct captions for preferred and less preferred images during training. The main research objective is to address the issues of conflict distribution and irrelevant prompts in existing preference optimization methods for diffusion models. The key methodology involves generating distinct captions for preferred and less-preferred images using captioning, perturbation, or hybrid methods, and introducing a modified objective function that leverages these dual captions. Primary results show that DCPO-h outperforms Stable Diffusion 2.1, SFT, Diffusion-DPO, and MaPO, achieving a +0.21 improvement in Pickscore. The principal implication for AI practitioners is that using dual, distinct captions for preferred and less-preferred image pairs during preference optimization can significantly enhance the alignment and performance of diffusion models. |
Papers for 2025-02-10