Image de couverture de livre

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lian, Haoran, Chen, Junmin, Huang, Wei, Xiong, Yizhe, Hu, Wenping, Ding, Guiguang, Chen, Hui, Niu, Jianwei, Lin, Zijia, Zhang, Fuzheng, Zhang, Di
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LBPE: Long-token-first Tokenization to Improve Large Language Models
par: Lian, Haoran, et autres
Publié: (2024)

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
par: Lian, Haoran, et autres
Publié: (2024)

Temporal Scaling Law for Large Language Models
par: Xiong, Yizhe, et autres
Publié: (2024)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
par: Xiong, Yizhe, et autres
Publié: (2025)

Fast Quiet-STaR: Thinking Without Thought Tokens
par: Huang, Wei, et autres
Publié: (2025)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
par: Su, Zhenpeng, et autres
Publié: (2024)

PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
par: Xiong, Yizhe, et autres
Publié: (2024)

Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation
par: Xiong, Yizhe, et autres
Publié: (2025)

Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations
par: Liang, Yiwen, et autres
Publié: (2025)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
par: Su, Zhenpeng, et autres
Publié: (2024)

RepViT-SAM: Towards Real-Time Segmenting Anything
par: Wang, Ao, et autres
Publié: (2023)

LSNet: See Large, Focus Small
par: Wang, Ao, et autres
Publié: (2025)

RepViT: Revisiting Mobile CNN From ViT Perspective
par: Wang, Ao, et autres
Publié: (2023)

Context Enhancement with Reconstruction as Sequence for Unified Unsupervised Anomaly Detection
par: Yang, Hui-Yue, et autres
Publié: (2024)

Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
par: Pan, Leiyu, et autres
Publié: (2025)

CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs
par: Wang, Ao, et autres
Publié: (2023)

[CLS] Token Tells Everything Needed for Training-free Efficient MLLMs
par: Wang, Ao, et autres
Publié: (2024)

YOLOE: Real-Time Seeing Anything
par: Wang, Ao, et autres
Publié: (2025)

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
par: Lv, Minxuan, et autres
Publié: (2025)

YOLOv10: Real-Time End-to-End Object Detection
par: Wang, Ao, et autres
Publié: (2024)

Long N-step Surrogate Stage Reward to Reduce Variances of Deep Reinforcement Learning in Complex Problems
par: Zhong, Junmin, et autres
Publié: (2022)

Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence
par: Lyu, Mengyao, et autres
Publié: (2024)

Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding
par: Wang, Xiao, et autres
Publié: (2024)

CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels
par: Wei, Lingxiao, et autres
Publié: (2024)

PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation
par: Wang, Ao, et autres
Publié: (2024)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
par: Su, Zhenpeng, et autres
Publié: (2023)

NExtLong: Toward Effective Long-Context Training without Long Documents
par: Gao, Chaochen, et autres
Publié: (2025)

LLMI3D: MLLM-based 3D Perception from a Single 2D Image
par: Yang, Fan, et autres
Publié: (2024)

DSPC: Dual-Stage Progressive Compression Framework for Efficient Long-Context Reasoning
par: Gao, Yaxin, et autres
Publié: (2025)

Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
par: Sun, Chenxi, et autres
Publié: (2024)

PolicyLong: Towards On-Policy Context Extension
par: Jia, Junlong, et autres
Publié: (2026)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
par: Zhong, Meizhi, et autres
Publié: (2024)

From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning
par: Niu, Zihan, et autres
Publié: (2026)

C−X (X=C, O, S, N, B, P) Bond Deuteration and Late‐Stage Applications
par: Hui Xu, et autres
Publié: (2024)

PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
par: McDanel, Bradley, et autres
Publié: (2025)

EntropyLong: Effective Long-Context Training via Predictive Uncertainty
par: Jia, Junlong, et autres
Publié: (2025)

Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning
par: Yang, Hui-Yue, et autres
Publié: (2024)

Stage-specific impacts of droughts on crop yields
par: nabi, sakiba, et autres
Publié: (2026)

Unlocking Innovation Through ESG : How Development Stages Shape the Impact
par: Wenjing Xu, et autres
Publié: (2026)

EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
par: Tian, Han, et autres
Publié: (2026)