Cover Image

InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Heejun, Park, Geon, Suh, Jaduk, Hwang, Sung Ju
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention
by: Lee, Heejun, et al.
Published: (2024)

Training-Free Exponential Context Extension via Cascading KV Cache
by: Willette, Jeffrey, et al.
Published: (2024)

SEA: Sparse Linear Attention with Estimated Attention Mask
by: Lee, Heejun, et al.
Published: (2023)

GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer
by: Cho, Junmo, et al.
Published: (2026)

Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction
by: Willette, Jeffrey, et al.
Published: (2025)

OmniRetrieval: Unified Retrieval across Heterogeneous Knowledge Sources
by: Baek, Jinheon, et al.
Published: (2026)

TableRAG: Million-Token Table Understanding with Language Models
by: Chen, Si-An, et al.
Published: (2024)

Efficient Real-time Refinement of Language Model Text Generation
by: Ko, Joonho, et al.
Published: (2025)

SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
by: Lee, Chanuk, et al.
Published: (2026)

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
by: Ding, Yiran, et al.
Published: (2024)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
by: Lee, Philip Heejun
Published: (2025)

Folded Context Condensation in Path Integral Formalism for Infinite Context Transformers
by: Paeng, Won-Gi, et al.
Published: (2024)

ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
by: Baek, Jinheon, et al.
Published: (2024)

When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
by: Jeong, Soyeong, et al.
Published: (2025)

Automatically Interpreting Millions of Features in Large Language Models
by: Paulo, Gonçalo, et al.
Published: (2024)

HiGPT: Heterogeneous Graph Language Model
by: Tang, Jiabin, et al.
Published: (2024)

Chain of Retrieval: Multi-Aspect Iterative Search Expansion and Post-Order Search Aggregation for Full Paper Retrieval
by: Park, Sangwoo, et al.
Published: (2025)

Integrating Pre-trained Language Model into Neural Machine Translation
by: Hwang, Soon-Jae, et al.
Published: (2023)

LLiMba: Sardinian on a Single GPU -- Adapting a 3B Language Model to a Vanishing Romance Language
by: Ballore, Luca
Published: (2026)

Extending Input Contexts of Language Models through Training on Segmented Sequences
by: Karypis, Petros, et al.
Published: (2023)

Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agents
by: Kim, Suji, et al.
Published: (2026)

System Prompt Optimization with Meta-Learning
by: Choi, Yumin, et al.
Published: (2025)

Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs
by: Synk, Ryan, et al.
Published: (2025)

Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey
by: Lee, Kunil, et al.
Published: (2026)

Continuous Diffusion Model for Language Modeling
by: Jo, Jaehyeong, et al.
Published: (2025)

Context Parallelism for Scalable Million-Token Inference
by: Yang, Amy, et al.
Published: (2024)

HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models
by: Lee, Seanie, et al.
Published: (2024)

Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching
by: Aytes, Simon A., et al.
Published: (2025)

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
by: Wang, Xindi, et al.
Published: (2024)

EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens
by: Hwang, Sunil, et al.
Published: (2022)

AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML
by: Trirat, Patara, et al.
Published: (2024)

ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
by: Zheng, Kangjie, et al.
Published: (2025)

Sequence-Level Leakage Risk of Training Data in Large Language Models
by: Tiwari, Trishita, et al.
Published: (2024)

Tokenization Tradeoffs in Structured EHR Foundation Models
by: Guo, Lin Lawrence, et al.
Published: (2026)

Language over Content: Tracing Cultural Understanding in Multilingual Large Language Models
by: Cho, Seungho, et al.
Published: (2025)

Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models
by: Kang, Minki, et al.
Published: (2024)

Luna-2: Scalable Single-Token Evaluation with Small Language Models
by: Goel, Vatsal, et al.
Published: (2026)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
by: Zeng, Xiangyu, et al.
Published: (2026)

HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents
by: Yeo, Woongyeng, et al.
Published: (2026)

Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks
by: Seo, Minju, et al.
Published: (2024)