Image de couverture de livre

Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Gong, Xue, Yi, Qi, Nan, Ziyuan, Huang, Guanhua, Li, Kejiao, Jiang, Yuhao, Xiong, Ruibin, Xu, Zenan, Guo, Jiaming, Peng, Shaohui, Zhou, Bo
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward
par: Huang, Guanhua, et autres
Publié: (2025)

Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework
par: Xu, Zenan, et autres
Publié: (2025)

Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs
par: Li, Junbo, et autres
Publié: (2025)

ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
par: Zhang, Chenchen, et autres
Publié: (2025)

SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks
par: Wang, Tianyi, et autres
Publié: (2026)

Reinforcement Learning on Pre-Training Data
par: Li, Siheng, et autres
Publié: (2025)

AM-PPO: (Advantage) Alpha-Modulation with Proximal Policy Optimization
par: Sane, Soham
Publié: (2025)

Seismic Performance Parameter Analysis of Prefabricated Segmental Hybrid‐Connected Double‐Column Piers
par: Zuoqiao You, et autres
Publié: (2025)

Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
par: Gong, Shijin, et autres
Publié: (2026)

Ex3: Automatic Novel Writing by Extracting, Excelsior and Expanding
par: Huang, Lei, et autres
Publié: (2024)

Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping
par: Peng, Miao, et autres
Publié: (2026)

StructCoh: Structured Contrastive Learning for Context-Aware Text Semantic Matching
par: Xue, Chao, et autres
Publié: (2025)

Enhancing Uncertainty Estimation in LLMs with Expectation of Aggregated Internal Belief
par: Xiao, Zeguan, et autres
Publié: (2025)

Advantage of utilizing nonlocal magic resource in Haar-random circuits
par: Huang, Xiao, et autres
Publié: (2025)

Seismic Performance Study of Prefabricated Segmental Assembled Hybrid Connected Double‐Column Bridge Piers
par: Zuoqiao You, et autres
Publié: (2025)

Learning from the Right Rollouts: Data Attribution for PPO-based LLM Post-Training
par: Shu, Dong, et autres
Publié: (2026)

Beyond Length: Quantifying Long-Range Information for Long-Context LLM Pretraining Data
par: Deng, Haoran, et autres
Publié: (2025)

Training-Inference Consistent Segmented Execution for Long-Context LLMs
par: Shang, Xianpeng, et autres
Publié: (2026)

BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
par: Gong, Shijin, et autres
Publié: (2026)

Mode-Dependent Rectification for Stable PPO Training
par: Mohamad, Mohamad, et autres
Publié: (2026)

Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study
par: Xu, Shusheng, et autres
Publié: (2024)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
par: Tang, Jiaming, et autres
Publié: (2024)

Efficient Long-Context LLM Inference via KV Cache Clustering
par: Hu, Jie, et autres
Publié: (2025)

Long Context Pre-Training with Lighthouse Attention
par: Peng, Bowen, et autres
Publié: (2026)

3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding
par: Ma, Xindian, et autres
Publié: (2024)

VLA Model Post-Training via Action-Chunked PPO and Self Behavior Cloning
par: Wang, Si-Cheng, et autres
Publié: (2025)

A Self-Training Approach for Whisper to Enhance Long Dysarthric Speech Recognition
par: Wang, Shiyao, et autres
Publié: (2025)

Structured Packing in LLM Training Improves Long Context Utilization
par: Staniszewski, Konrad, et autres
Publié: (2023)

Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
par: Wu, Junkang, et autres
Publié: (2025)

Confidence Interval Construction for Causally Generalized Estimates With Target Sample Summary Information
par: Yi Chen, et autres
Publié: (2026)

Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
par: Huang, Yuxiang, et autres
Publié: (2024)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
par: Gu, Diandian, et autres
Publié: (2024)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
par: Xiao, Chaojun, et autres
Publié: (2024)

Enhancing PPO with Trajectory-Aware Hybrid Policies
par: Liu, Qisai, et autres
Publié: (2025)

Adaptive Alpha Weighting with PPO: Enhancing Prompt-Based LLM-Generated Alphas in Quant Trading
par: Chen, Qizhao, et autres
Publié: (2025)

A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts
par: Ge, Suyu, et autres
Publié: (2024)

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
par: Xiao, Guangxuan, et autres
Publié: (2024)

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
par: Lai, Peng, et autres
Publié: (2025)

Training-Free Long-Context Scaling of Large Language Models
par: An, Chenxin, et autres
Publié: (2024)

VinePPO: Refining Credit Assignment in RL Training of LLMs
par: Kazemnejad, Amirhossein, et autres
Publié: (2024)