Cover Image

Training Long-Context LLMs Efficiently via Chunk-wise Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Wenhao, Zhang, Yuxin, Luo, Gen, Yu, Daohai, Ji, Rongrong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Soft-NBCE: Entropy-Weighted Chunk Fusion for Long-Context
by: Ji, Shihao, et al.
Published: (2026)

Cognitive Chunking for Soft Prompts: Accelerating Compressor Learning via Block-wise Causal Masking
by: Liu, Guojie, et al.
Published: (2026)

LycheeCluster: Efficient Long-Context Inference with Structure-Aware Chunking and Hierarchical KV Indexing
by: Li, Dongfang, et al.
Published: (2026)

StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs
by: Luo, Qijun, et al.
Published: (2025)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by: Xiao, Chaojun, et al.
Published: (2024)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
by: Liu, Yuxi, et al.
Published: (2025)

Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
by: Lin, Luxi, et al.
Published: (2026)

SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
by: Li, Zhouyang, et al.
Published: (2025)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
by: Lai, Xin, et al.
Published: (2024)

An Efficient Training Algorithm for Models with Block-wise Sparsity
by: Zhu, Ding, et al.
Published: (2025)

Adaptive Action Chunking via Multi-Chunk Q Value Estimation
by: Shin, Yongjae, et al.
Published: (2026)

Two-Stage Grid Optimization for Group-wise Quantization of LLMs
by: Kim, Junhan, et al.
Published: (2026)

FoldAct: Efficient and Stable Context Folding for Long-Horizon Search Agents
by: Shao, Jiaqi, et al.
Published: (2025)

MeLeMaD: Adaptive Malware Detection via Chunk-wise Feature Selection and Meta-Learning
by: K, Ajvad Haneef, et al.
Published: (2025)

CAFE-GB: Scalable and Stable Feature Selection for Malware Detection via Chunk-wise Aggregated Gradient Boosting
by: K, Ajvad Haneef, et al.
Published: (2026)

CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
by: Wang, Luning, et al.
Published: (2024)

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
by: Ji, Xiaotong, et al.
Published: (2026)

The $\mathbf{Y}$-Combinator for LLMs: Solving Long-Context Rot with $λ$-Calculus
by: Roy, Amartya, et al.
Published: (2026)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
by: Li, Ziniu, et al.
Published: (2025)

IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning
by: Luo, Haohao, et al.
Published: (2026)

Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation
by: Xie, Jingjing, et al.
Published: (2024)

Efficient Graph Understanding with LLMs via Structured Context Injection
by: Waghmare, Govind, et al.
Published: (2025)

LongSafety: Enhance Safety for Long-Context LLMs
by: Huang, Mianqiu, et al.
Published: (2024)

Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
by: Ni, Wentao, et al.
Published: (2026)

Decoupled Q-Chunking
by: Li, Qiyang, et al.
Published: (2025)

MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning
by: Liu, Dong, et al.
Published: (2026)

Overflow Prevention Enhances Long-Context Recurrent LLMs
by: Ben-Kish, Assaf, et al.
Published: (2025)

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
by: Song, Woomin, et al.
Published: (2024)

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
by: Liu, Ziyue, et al.
Published: (2025)

DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation
by: Shing, Makoto, et al.
Published: (2025)

Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts
by: Li, Wenhao, et al.
Published: (2026)

Efficient Knowledge Deletion from Trained Models through Layer-wise Partial Machine Unlearning
by: Gogineni, Vinay Chakravarthi, et al.
Published: (2024)

DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization
by: Jiang, Yanfeng, et al.
Published: (2024)

MoBA: Mixture of Block Attention for Long-Context LLMs
by: Lu, Enzhe, et al.
Published: (2025)

Reinforcement Learning with Action Chunking
by: Li, Qiyang, et al.
Published: (2025)

SmartChunk Retrieval: Query-Aware Chunk Compression with Planning for Efficient Document RAG
by: Zhang, Xuechen, et al.
Published: (2025)

Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training
by: Gong, Xue, et al.
Published: (2026)

GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning
by: Murata, Naoki, et al.
Published: (2026)

Towards Monotonic Improvement in In-Context Reinforcement Learning
by: Zhang, Wenhao, et al.
Published: (2025)

Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
by: Yu, Zhiyin, et al.
Published: (2026)