Copertina

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lian, Haoran, Chen, Junmin, Huang, Wei, Xiong, Yizhe, Hu, Wenping, Ding, Guiguang, Chen, Hui, Niu, Jianwei, Lin, Zijia, Zhang, Fuzheng, Zhang, Di
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LBPE: Long-token-first Tokenization to Improve Large Language Models
di: Lian, Haoran, et al.
Pubblicazione: (2024)

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
di: Lian, Haoran, et al.
Pubblicazione: (2024)

Temporal Scaling Law for Large Language Models
di: Xiong, Yizhe, et al.
Pubblicazione: (2024)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
di: Xiong, Yizhe, et al.
Pubblicazione: (2025)

Fast Quiet-STaR: Thinking Without Thought Tokens
di: Huang, Wei, et al.
Pubblicazione: (2025)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
di: Su, Zhenpeng, et al.
Pubblicazione: (2024)

PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
di: Xiong, Yizhe, et al.
Pubblicazione: (2024)

Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation
di: Xiong, Yizhe, et al.
Pubblicazione: (2025)

Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations
di: Liang, Yiwen, et al.
Pubblicazione: (2025)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
di: Su, Zhenpeng, et al.
Pubblicazione: (2024)

RepViT-SAM: Towards Real-Time Segmenting Anything
di: Wang, Ao, et al.
Pubblicazione: (2023)

LSNet: See Large, Focus Small
di: Wang, Ao, et al.
Pubblicazione: (2025)

RepViT: Revisiting Mobile CNN From ViT Perspective
di: Wang, Ao, et al.
Pubblicazione: (2023)

Context Enhancement with Reconstruction as Sequence for Unified Unsupervised Anomaly Detection
di: Yang, Hui-Yue, et al.
Pubblicazione: (2024)

Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
di: Pan, Leiyu, et al.
Pubblicazione: (2025)

CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs
di: Wang, Ao, et al.
Pubblicazione: (2023)

[CLS] Token Tells Everything Needed for Training-free Efficient MLLMs
di: Wang, Ao, et al.
Pubblicazione: (2024)

YOLOE: Real-Time Seeing Anything
di: Wang, Ao, et al.
Pubblicazione: (2025)

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
di: Lv, Minxuan, et al.
Pubblicazione: (2025)

YOLOv10: Real-Time End-to-End Object Detection
di: Wang, Ao, et al.
Pubblicazione: (2024)

Long N-step Surrogate Stage Reward to Reduce Variances of Deep Reinforcement Learning in Complex Problems
di: Zhong, Junmin, et al.
Pubblicazione: (2022)

Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence
di: Lyu, Mengyao, et al.
Pubblicazione: (2024)

Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding
di: Wang, Xiao, et al.
Pubblicazione: (2024)

CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels
di: Wei, Lingxiao, et al.
Pubblicazione: (2024)

PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation
di: Wang, Ao, et al.
Pubblicazione: (2024)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
di: Su, Zhenpeng, et al.
Pubblicazione: (2023)

NExtLong: Toward Effective Long-Context Training without Long Documents
di: Gao, Chaochen, et al.
Pubblicazione: (2025)

LLMI3D: MLLM-based 3D Perception from a Single 2D Image
di: Yang, Fan, et al.
Pubblicazione: (2024)

DSPC: Dual-Stage Progressive Compression Framework for Efficient Long-Context Reasoning
di: Gao, Yaxin, et al.
Pubblicazione: (2025)

Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
di: Sun, Chenxi, et al.
Pubblicazione: (2024)

PolicyLong: Towards On-Policy Context Extension
di: Jia, Junlong, et al.
Pubblicazione: (2026)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
di: Zhong, Meizhi, et al.
Pubblicazione: (2024)

From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning
di: Niu, Zihan, et al.
Pubblicazione: (2026)

C−X (X=C, O, S, N, B, P) Bond Deuteration and Late‐Stage Applications
di: Hui Xu, et al.
Pubblicazione: (2024)

PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
di: McDanel, Bradley, et al.
Pubblicazione: (2025)

EntropyLong: Effective Long-Context Training via Predictive Uncertainty
di: Jia, Junlong, et al.
Pubblicazione: (2025)

Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning
di: Yang, Hui-Yue, et al.
Pubblicazione: (2024)

Stage-specific impacts of droughts on crop yields
di: nabi, sakiba, et al.
Pubblicazione: (2026)

Unlocking Innovation Through ESG : How Development Stages Shape the Impact
di: Wenjing Xu, et al.
Pubblicazione: (2026)

EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
di: Tian, Han, et al.
Pubblicazione: (2026)