Image de couverture de livre

Extending LLMs' Context Window with 100 Samples

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Yikai, Li, Junlong, Liu, Pengfei
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Dissecting Human and LLM Preferences
par: Li, Junlong, et autres
Publié: (2024)

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
par: Ding, Yiran, et autres
Publié: (2024)

Extending Context Window of Large Language Models from a Distributional Perspective
par: Wu, Yingsheng, et autres
Publié: (2024)

Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration
par: Liu, Zijun, et autres
Publié: (2025)

Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows
par: Dickson, Billy, et autres
Publié: (2025)

$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
par: Zhang, Xinrong, et autres
Publié: (2024)

PSC: Extending Context Window of Large Language Models via Phase Shift Calibration
par: Zhu, Wenqiao, et autres
Publié: (2025)

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
par: Jin, Hongye, et autres
Publié: (2024)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
par: Zhu, Shiyi, et autres
Publié: (2023)

LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
par: Rando, Stefano, et autres
Publié: (2025)

Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings
par: Gelberg, Yoav, et autres
Publié: (2025)

Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agents
par: Pollertlam, Natchanon, et autres
Publié: (2026)

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
par: Zhu, Dawei, et autres
Publié: (2023)

LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
par: Jia, Junlong, et autres
Publié: (2025)

Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
par: Wang, Minzheng, et autres
Publié: (2024)

LongRoPE2: Near-Lossless LLM Context Window Scaling
par: Shang, Ning, et autres
Publié: (2025)

Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
par: Zhou, Fan, et autres
Publié: (2024)

Finding Words Associated with DIF: Predicting Differential Item Functioning using LLMs and Explainable AI
par: Maeda, Hotaka, et autres
Publié: (2025)

Rethinking the Role of LLMs in Time Series Forecasting
par: Qiu, Xin, et autres
Publié: (2026)

The Critique of Critique
par: Sun, Shichao, et autres
Publié: (2024)

Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs
par: Paulsen, Norman
Publié: (2025)

Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window
par: Tang, Qiaoyu, et autres
Publié: (2025)

Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs
par: Zhang, Yifang, et autres
Publié: (2025)

Extending Llama-3's Context Ten-Fold Overnight
par: Zhang, Peitian, et autres
Publié: (2024)

SkyLadder: Better and Faster Pretraining via Context Window Scheduling
par: Zhu, Tongyao, et autres
Publié: (2025)

Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
par: Huang, Chensen, et autres
Publié: (2024)

Positional Biases Shift as Inputs Approach Context Window Limits
par: Veseli, Blerta, et autres
Publié: (2025)

Glyph: Scaling Context Windows via Visual-Text Compression
par: Cheng, Jiale, et autres
Publié: (2025)

Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach
par: Zeng, Shenglai, et autres
Publié: (2025)

Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars
par: Zhang, Yitong, et autres
Publié: (2026)

Can LLMs Learn to Map the World from Local Descriptions?
par: Xia, Sirui, et autres
Publié: (2025)

Towards Efficient LLMs Annealing with Principled Sample Selection
par: Xu, Yuanjian, et autres
Publié: (2026)

UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs
par: Li, Wenhao, et autres
Publié: (2024)

More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering
par: Yao, Bingsheng, et autres
Publié: (2023)

Recall, Retrieve and Reason: Towards Better In-Context Relation Extraction
par: Li, Guozheng, et autres
Publié: (2024)

Efficient Solutions For An Intriguing Failure of LLMs: Long Context Window Does Not Mean LLMs Can Analyze Long Sequences Flawlessly
par: Hosseini, Peyman, et autres
Publié: (2024)

GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models
par: Li, Mukai, et autres
Publié: (2024)

Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors
par: Li, Guozheng, et autres
Publié: (2024)

WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
par: Zuo, Youhui, et autres
Publié: (2025)

SWAA: Sliding Window Attention Adaptation for Efficient and Quality Preserving Long Context Processing
par: Yu, Yijiong, et autres
Publié: (2025)