Cover Image

EntropyLong: Effective Long-Context Training via Predictive Uncertainty

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jia, Junlong, Chen, Ziyang, Wu, Xing, Gao, Chaochen, Lin, Zijia, Zhang, Debing, Hu, Songlin, Guo, Binghui
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

NExtLong: Toward Effective Long-Context Training without Long Documents
by: Gao, Chaochen, et al.
Published: (2025)

LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
by: Chen, Ziyang, et al.
Published: (2026)

LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
by: Gao, Chaochen, et al.
Published: (2025)

LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
by: Jia, Junlong, et al.
Published: (2025)

PolicyLong: Towards On-Policy Context Extension
by: Jia, Junlong, et al.
Published: (2026)

Quest: Query-centric Data Synthesis Approach for Long-context Scaling of Large Language Model
by: Gao, Chaochen, et al.
Published: (2024)

Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval
by: Ma, Guangyuan, et al.
Published: (2024)

How to Train Long-Context Language Models (Effectively)
by: Gao, Tianyu, et al.
Published: (2024)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
by: Su, Zhenpeng, et al.
Published: (2023)

CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning
by: Yu, Huimu, et al.
Published: (2024)

Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
by: Tian, Junfeng, et al.
Published: (2024)

Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning
by: Zhu, Wenhao, et al.
Published: (2025)

ACC: Compiling Agent Trajectories for Long-Context Training
by: Su, Qisheng, et al.
Published: (2026)

A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts
by: Ge, Suyu, et al.
Published: (2024)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
by: You, Zeng, et al.
Published: (2025)

LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion
by: Ling, Zhan, et al.
Published: (2025)

GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
by: Lv, Minxuan, et al.
Published: (2026)

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models
by: Lian, Haoran, et al.
Published: (2024)

LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs
by: Wu, Yuhao, et al.
Published: (2024)

Long-Context Language Modeling with Parallel Context Encoding
by: Yen, Howard, et al.
Published: (2024)

Long-Short Alignment for Effective Long-Context Modeling in LLMs
by: Du, Tianqi, et al.
Published: (2025)

LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement
by: Ye, Zhifan, et al.
Published: (2025)

Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration
by: Xiao, Sibo, et al.
Published: (2025)

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
by: Ji, Mengmeng, et al.
Published: (2026)

Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents
by: Wang, Jiawei, et al.
Published: (2025)

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
by: Ye, Xi, et al.
Published: (2025)

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
by: Yen, Howard, et al.
Published: (2024)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

LongAttn: Selecting Long-context Training Data via Token-level Attention
by: Wu, Longyun, et al.
Published: (2025)

Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing
by: Wu, Chen, et al.
Published: (2025)

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
by: Jiang, Huiqiang, et al.
Published: (2023)

Long Context Pre-Training with Lighthouse Attention
by: Peng, Bowen, et al.
Published: (2026)

UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation
by: Li, Zixuan, et al.
Published: (2024)

A Controlled Study on Long Context Extension and Generalization in LLMs
by: Lu, Yi, et al.
Published: (2024)

Coding Agents are Effective Long-Context Processors
by: Cao, Weili, et al.
Published: (2026)

WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents
by: Liu, Junteng, et al.
Published: (2025)

NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
by: Chen, Hong, et al.
Published: (2026)

A MapReduce Approach to Effectively Utilize Long Context Information in Retrieval Augmented Language Models
by: Zhang, Gongbo, et al.
Published: (2024)

LLoCO: Learning Long Contexts Offline
by: Tan, Sijun, et al.
Published: (2024)