Copertina

An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wu, Tong, Zhao, Yanpeng, Zheng, Zilong
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
di: Hu, Zhiyuan, et al.
Pubblicazione: (2024)

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
di: Zhang, Zhenyu, et al.
Pubblicazione: (2024)

EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
di: Tian, Han, et al.
Pubblicazione: (2026)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
di: Wang, Zhenghua, et al.
Pubblicazione: (2025)

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
di: Zhao, Liang, et al.
Pubblicazione: (2024)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
di: Li, Jiaqi, et al.
Pubblicazione: (2023)

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
di: Zhu, Dawei, et al.
Pubblicazione: (2023)

Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs
di: Liu, Xiaoran, et al.
Pubblicazione: (2025)

TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
di: Wu, Tong, et al.
Pubblicazione: (2025)

Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
di: Long, Lingkun, et al.
Pubblicazione: (2026)

LongAlign: A Recipe for Long Context Alignment of Large Language Models
di: Bai, Yushi, et al.
Pubblicazione: (2024)

Long-Context Language Modeling with Parallel Context Encoding
di: Yen, Howard, et al.
Pubblicazione: (2024)

LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuning
di: Mao, Yansheng, et al.
Pubblicazione: (2025)

Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models
di: Zhao, Runsong, et al.
Pubblicazione: (2024)

QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management
di: Shen, Weizhou, et al.
Pubblicazione: (2025)

3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding
di: Ma, Xindian, et al.
Pubblicazione: (2024)

Reducing Distraction in Long-Context Language Models by Focused Learning
di: Wu, Zijun, et al.
Pubblicazione: (2024)

Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs
di: Bai, Jun, et al.
Pubblicazione: (2025)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
di: Hsieh, Cheng-Yu, et al.
Pubblicazione: (2024)

A Controlled Study on Long Context Extension and Generalization in LLMs
di: Lu, Yi, et al.
Pubblicazione: (2024)

Understanding Synthetic Context Extension via Retrieval Heads
di: Zhao, Xinyu, et al.
Pubblicazione: (2024)

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
di: Mao, Yansheng, et al.
Pubblicazione: (2024)

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training
di: He, Junqing, et al.
Pubblicazione: (2023)

Long Context Alignment with Short Instructions and Synthesized Positions
di: Wu, Wenhao, et al.
Pubblicazione: (2024)

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding
di: Zhao, Tiancheng, et al.
Pubblicazione: (2024)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
di: Lou, Chao, et al.
Pubblicazione: (2024)

Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling
di: Hu, Xiang, et al.
Pubblicazione: (2024)

Wavelet-based Positional Representation for Long Context
di: Oka, Yui, et al.
Pubblicazione: (2025)

FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
di: Ouyang, Mingyu, et al.
Pubblicazione: (2026)

AllMem: A Memory-centric Recipe for Efficient Long-context Modeling
di: Wang, Ziming, et al.
Pubblicazione: (2026)

LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding
di: Tong, Junlong, et al.
Pubblicazione: (2025)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
di: Zhong, Meizhi, et al.
Pubblicazione: (2024)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
di: You, Zeng, et al.
Pubblicazione: (2025)

Positive Focusing is Directly Useful
di: Accattoli, Beniamino, et al.
Pubblicazione: (2024)

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
di: Chen, Yuhan, et al.
Pubblicazione: (2024)

MuDAF: Long-Context Multi-Document Attention Focusing through Contrastive Learning on Attention Heads
di: Liu, Weihao, et al.
Pubblicazione: (2025)

Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking
di: Zhang, Wuwei, et al.
Pubblicazione: (2025)

FocusLLM: Precise Understanding of Long Context by Dynamic Condensing
di: Li, Zhenyu, et al.
Pubblicazione: (2024)

Unstructured Evidence Attribution for Long Context Query Focused Summarization
di: Wright, Dustin, et al.
Pubblicazione: (2025)

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
di: Zheng, Chuanyang, et al.
Pubblicazione: (2024)