Imagen de Portada

Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Synk, Ryan, Hoover, Monte, Kirchenbauer, John, Jain, Neel, Stein, Alex, Shu, Manli, Sanchez, Josue Melendez, Duraiswami, Ramani, Goldstein, Tom
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

FAST: Factorizable Attention for Speeding up Transformers
por: Gerami, Armin, et al.
Publicado: (2024)

GenQA: Generating Millions of Instructions from a Handful of Prompts
por: Chen, Jiuhai, et al.
Publicado: (2024)

Context Parallelism for Scalable Million-Token Inference
por: Yang, Amy, et al.
Publicado: (2024)

GATES: Self-Distillation under Privileged Context with Consensus Gating
por: Stein, Alex, et al.
Publicado: (2026)

Coercing LLMs to do and reveal (almost) anything
por: Geiping, Jonas, et al.
Publicado: (2024)

Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputers
por: Singh, Siddharth, et al.
Publicado: (2025)

Transformer Based Linear Attention with Optimized GPU Kernel Implementation
por: Gerami, Armin, et al.
Publicado: (2025)

Auditing Algorithmic Bias in Transformer-Based Trading
por: Gerami, Armin, et al.
Publicado: (2025)

Room Impulse Response Synthesis via Differentiable Feedback Delay Networks for Efficient Spatial Audio Rendering
por: Gerami, Armin, et al.
Publicado: (2025)

ViscoReg: Neural Signed Distance Functions via Viscosity Solutions
por: Krishnan, Meenakshi, et al.
Publicado: (2025)

Multi-Token Prediction via Self-Distillation
por: Kirchenbauer, John, et al.
Publicado: (2026)

Applying Automatic Differentiation to Optimize Differential Microphone Array Designs
por: Galougah, Siminfar Samakoush, et al.
Publicado: (2024)

DynaGuard: A Dynamic Guardian Model With User-Defined Policies
por: Hoover, Monte, et al.
Publicado: (2025)

Transformers Can Do Arithmetic with the Right Embeddings
por: McLeish, Sean, et al.
Publicado: (2024)

Inference Time Context Sparsity: Illusion or Opportunity?
por: Joshi, Sahil, et al.
Publicado: (2026)

Quantifying Document Impact in RAG-LLMs
por: Gerami, Armin, et al.
Publicado: (2025)

On The Application of Linear Attention in Multimodal Transformers
por: Gerami, Armin, et al.
Publicado: (2026)

3D Gaussian Splatting with Normal Information for Mesh Extraction and Improved Rendering
por: Krishnan, Meenakshi, et al.
Publicado: (2025)

On the Reliability of Watermarks for Large Language Models
por: Kirchenbauer, John, et al.
Publicado: (2023)

When Can You Get Away with Low Memory Adam?
por: Kalra, Dayal Singh, et al.
Publicado: (2025)

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
por: Huang, Yuxiang, et al.
Publicado: (2025)

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
por: Geiping, Jonas, et al.
Publicado: (2025)

Analytical Exploration of Spatial Audio Cues: A Differentiable Multi-Sphere Scattering Model
por: Galougah, Siminfar Samakoush, et al.
Publicado: (2026)

SparseAccelerate: Efficient Long-Context Inference for Mid-Range GPUs
por: Vo, James
Publicado: (2024)

FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition
por: Kirchenbauer, John, et al.
Publicado: (2025)

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
por: Ding, Yiran, et al.
Publicado: (2024)

Tokenism in Context: Unpacking Token Women Directors in the Corporate Boardroom
por: Tanusree Jain, et al.
Publicado: (2026)

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs
por: Chen, Aodong, et al.
Publicado: (2023)

TSPE: Task-Specific Prompt Ensemble for Improved Zero-Shot Audio Classification
por: Anand, Nishit, et al.
Publicado: (2024)

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
por: Hans, Abhimanyu, et al.
Publicado: (2024)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
por: Tang, Jiaming, et al.
Publicado: (2024)

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
por: He, Linda, et al.
Publicado: (2025)

Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models
por: Jain, Neel, et al.
Publicado: (2024)

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
por: Hooper, Coleman, et al.
Publicado: (2024)

In-Context Positive-Unlabeled Learning
por: Liu, Siyan, et al.
Publicado: (2026)

MGen: Millions of Naturally Occurring Generics in Context
por: Cilleruelo, Gustavo, et al.
Publicado: (2025)

Variational Inference of Structured Line Spectra Exploiting Group-Sparsity
por: Möderl, Jakob, et al.
Publicado: (2023)

Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts
por: Li, Wenhao, et al.
Publicado: (2026)

A Watermark for Large Language Models
por: Kirchenbauer, John, et al.
Publicado: (2023)

Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU
por: Sun, He, et al.
Publicado: (2025)