Copertina

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Meng, Haoming, Huang, Kexin, Wei, Shaohang, Ma, Chiyu, Yang, Shuo, Wang, Xue, Wang, Guoyin, Ding, Bolin, Zhou, Jingren
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
di: Huang, Kexin, et al.
Pubblicazione: (2026)

FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization
di: Ma, Chiyu, et al.
Pubblicazione: (2026)

One-Way Policy Optimization for Self-Evolving LLMs
di: Yang, Shuo, et al.
Pubblicazione: (2026)

Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
di: Lu, Jinda, et al.
Pubblicazione: (2026)

On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting
di: Zhang, Wenhao, et al.
Pubblicazione: (2025)

Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model
di: Wang, Xue, et al.
Pubblicazione: (2025)

EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
di: Pan, Xuchen, et al.
Pubblicazione: (2024)

SMARTFEAT: Efficient Feature Construction through Feature-Level Foundation Model Interactions
di: Lin, Yin, et al.
Pubblicazione: (2023)

Exploring Multi-Temperature Strategies for Token- and Rollout-Level Control in RLVR
di: Zhuang, Haomin, et al.
Pubblicazione: (2025)

Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
di: Chen, Yanxi, et al.
Pubblicazione: (2024)

TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
di: Ma, Dabiao, et al.
Pubblicazione: (2025)

Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
di: Ye, Junjie, et al.
Pubblicazione: (2025)

Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR
di: Wang, Jiakang, et al.
Pubblicazione: (2025)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
di: Wang, Shuo, et al.
Pubblicazione: (2026)

Exploring Selective Layer Fine-Tuning in Federated Learning
di: Sun, Yuchang, et al.
Pubblicazione: (2024)

BridgeScope: A Universal Toolkit for Bridging Large Language Models and Databases
di: Weng, Lianggui, et al.
Pubblicazione: (2025)

EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
di: Chen, Yanxi, et al.
Pubblicazione: (2023)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
di: Chen, Yanxi, et al.
Pubblicazione: (2024)

Tree-based Models for Vertical Federated Learning: A Survey
di: Qian, Bingchen, et al.
Pubblicazione: (2025)

Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
di: Song, Chiyu, et al.
Pubblicazione: (2023)

Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR
di: Lu, Jinda, et al.
Pubblicazione: (2026)

Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning
di: Feng, Weitao, et al.
Pubblicazione: (2025)

Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
di: Wu, Junkang, et al.
Pubblicazione: (2025)

AmbiSQL: Interactive Ambiguity Detection and Resolution for Text-to-SQL
di: Ding, Zhongjun, et al.
Pubblicazione: (2025)

Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
di: Wang, Tao, et al.
Pubblicazione: (2026)

Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding
di: Song, Feifan, et al.
Pubblicazione: (2025)

EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
di: Guo, Song, et al.
Pubblicazione: (2024)

Large Language Model-Enhanced Relational Operators: Taxonomy, Benchmark, and Analysis
di: Su, Yunxiang, et al.
Pubblicazione: (2026)

Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR
di: Min, Zijun, et al.
Pubblicazione: (2026)

GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
di: Zhou, Sifan, et al.
Pubblicazione: (2025)

A Level Set Method with Secant Iterations for the Least-Squares Constrained Nuclear Norm Minimization
di: Ma, Chiyu, et al.
Pubblicazione: (2026)

Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution
di: Xiao, Qiao, et al.
Pubblicazione: (2025)

Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models
di: Pan, Xuchen, et al.
Pubblicazione: (2025)

Fine-grained Analysis of Non-parametric Estimation for Pairwise Learning
di: Zhou, Junyu, et al.
Pubblicazione: (2023)

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
di: Wang, Xu, et al.
Pubblicazione: (2025)

Generalization of RLVR Using Causal Reasoning as a Testbed
di: Lu, Brian, et al.
Pubblicazione: (2025)

Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development
di: Chen, Daoyuan, et al.
Pubblicazione: (2024)

Distributional Alignment Games for Answer-Level Fine-Tuning
di: Mohri, Mehryar, et al.
Pubblicazione: (2026)

LoRAFusion: Efficient LoRA Fine-Tuning for LLMs
di: Zhu, Zhanda, et al.
Pubblicazione: (2025)

Sparse Layer Sharpness-Aware Minimization for Efficient Fine-Tuning
di: Cheng, Yifei, et al.
Pubblicazione: (2026)