Cover Image

Extending Context Window of Large Language Models from a Distributional Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Yingsheng, Gu, Yuxuan, Feng, Xiaocheng, Zhong, Weihong, Xu, Dongliang, Yang, Qing, Liu, Hongtao, Qin, Bing
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding
by: Zhao, Liang, et al.
Published: (2023)

Advancing Large Language Model Attribution through Self-Improving
by: Huang, Lei, et al.
Published: (2024)

Can Large Language Models Simulate Human Cognition Beyond Behavioral Imitation?
by: Gu, Yuxuan, et al.
Published: (2026)

Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models
by: Zhong, Weihong, et al.
Published: (2024)

Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis
by: Ma, Weitao, et al.
Published: (2024)

Learning Fine-Grained Grounded Citations for Attributed Large Language Models
by: Huang, Lei, et al.
Published: (2024)

GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization
by: Ye, Yangfan, et al.
Published: (2024)

Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization
by: Huang, Lei, et al.
Published: (2025)

Discrete Modeling via Boundary Conditional Diffusion Processes
by: Gu, Yuxuan, et al.
Published: (2024)

Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance
by: Xiong, Kai, et al.
Published: (2024)

Context-Aware Hierarchical Taxonomy Generation for Scientific Papers via LLM-Guided Multi-Aspect Clustering
by: Zhu, Kun, et al.
Published: (2025)

Length Controlled Generation for Black-box LLMs
by: Gu, Yuxuan, et al.
Published: (2024)

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
by: Huang, Lei, et al.
Published: (2023)

PSC: Extending Context Window of Large Language Models via Phase Shift Calibration
by: Zhu, Wenqiao, et al.
Published: (2025)

Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration
by: Huang, Yichong, et al.
Published: (2024)

Aligning Translation-Specific Understanding to General Understanding in Large Language Models
by: Huang, Yichong, et al.
Published: (2024)

How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers
by: Lu, Xin, et al.
Published: (2024)

SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models
by: Zhao, Weixiang, et al.
Published: (2024)

Bootstrapping Exploration with Group-Level Natural Language Feedback in Reinforcement Learning
by: Huang, Lei, et al.
Published: (2026)

Extending LLMs' Context Window with 100 Samples
by: Zhang, Yikai, et al.
Published: (2024)

Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation
by: Yuan, Zekun, et al.
Published: (2026)

An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation
by: Zhu, Kun, et al.
Published: (2024)

Review-LLM: Harnessing Large Language Models for Personalized Review Generation
by: Peng, Qiyao, et al.
Published: (2024)

Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective
by: Yu, Xinmiao, et al.
Published: (2024)

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
by: Ding, Yiran, et al.
Published: (2024)

FroM: Frobenius Norm-Based Data-Free Adaptive Model Merging
by: Li, Zijian, et al.
Published: (2025)

SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
by: Wang, Zekun, et al.
Published: (2023)

Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications
by: Feng, Zhangyin, et al.
Published: (2023)

Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code Large Language Models
by: Luo, Xianzhen, et al.
Published: (2024)

Relay Decoding: Concatenating Large Language Models for Machine Translation
by: Fu, Chengpeng, et al.
Published: (2024)

Beyond Fixed Length: Bucket Pre-training is All You Need
by: Yang, Qing, et al.
Published: (2024)

Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Model
by: Lv, Kangtao, et al.
Published: (2026)

Thinking with Nothinking Calibration: A New In-Context Learning Paradigm in Reasoning Large Language Models
by: Wu, Haotian, et al.
Published: (2025)

PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra
by: Feng, Xiachong, et al.
Published: (2026)

Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
by: Luo, Xianzhen, et al.
Published: (2024)

Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarization
by: Feng, Xiachong, et al.
Published: (2023)

Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models
by: Liu, Wenhan, et al.
Published: (2024)

YaRN: Efficient Context Window Extension of Large Language Models
by: Peng, Bowen, et al.
Published: (2023)

Exploring Context Window of Large Language Models via Decomposed Positional Vectors
by: Dong, Zican, et al.
Published: (2024)

Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows
by: Dickson, Billy, et al.
Published: (2025)