Cover Image

BitDecoding: Unlocking Tensor Cores for Long-Context LLMs with Low-Bit KV Cache

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Du, Dayou, Cao, Shijie, Cheng, Jianyi, Mai, Luo, Cao, Ting, Yang, Mao
Format:	Preprint
Published:	2025
Subjects:	Hardware Architecture Artificial Intelligence Computation and Language Performance
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits
by: Zhou, Zikai, et al.
Published: (2025)

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes
by: Hendria, Willy Fitra
Published: (2026)

HiKonv: Maximizing the Throughput of Quantized Convolution With Novel Bit-wise Management and Computation
by: Chen, Yao, et al.
Published: (2022)

Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
by: Fang, Yunhua, et al.
Published: (2025)

ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
by: Lei, Jianlong, et al.
Published: (2026)

Adaptive Cache Pollution Control for Large Language Model Inference Workloads Using Temporal CNN-Based Prediction and Priority-Aware Replacement
by: Liu, Songze, et al.
Published: (2025)

T-MAN: Enabling End-to-End Low-Bit LLM Inference on NPUs via Unified Table Lookup
by: Wei, Jianyu, et al.
Published: (2025)

Enhancing Instruction Prefetching via Cache and TLB Management
by: Jamet, Alexandre Valentin, et al.
Published: (2026)

The Bicameral Cache: a split cache for vector architectures
by: Rebolledo, Susana, et al.
Published: (2024)

WaveTune: Wave-aware Bilinear Modeling for Efficient GPU Kernel Auto-tuning
by: Zhang, Kaixuan, et al.
Published: (2026)

PipeWeave: Synergizing Analytical and Learning Models for Unified GPU Performance Prediction
by: Zhang, Kaixuan, et al.
Published: (2026)

CXL-Interference: Analysis and Characterization in Modern Computer Systems
by: Mao, Shunyu, et al.
Published: (2024)

Implementation and Optimization of HQC Decoding on NPU-Integrated Devices
by: Chau, Vu Minh, et al.
Published: (2026)

Dissecting the NVIDIA Hopper Architecture through Microbenchmarking and Multiple Level Analysis
by: Luo, Weile, et al.
Published: (2025)

Examem: Low-Overhead Memory Instrumentation for Intelligent Memory Systems
by: Poduval, Ashwin, et al.
Published: (2024)

Accelerating Transistor-Level Simulation of Integrated Circuits via Equivalence of RC Long-Chain Structures
by: Tang, Ruibai, et al.
Published: (2025)

DGEMM without FP64 Arithmetic - Using FP64 Emulation and FP8 Tensor Cores with Ozaki Scheme
by: Mukunoki, Daichi
Published: (2025)

GeneTEK: Low-power, high-performance and scalable FPGA architecture for exact unit-cost edit distance
by: Espinosa, Elena, et al.
Published: (2025)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
by: Zhang, Hang, et al.
Published: (2025)

LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference
by: Mo, Zhiwen, et al.
Published: (2024)

Cleaning up the Mess: Re-Evaluating the Real-System Modeling Accuracy of Ramulator 2.0
by: Bostanci, F. Nisa, et al.
Published: (2025)

ACALSim: A Scalable Parallel Simulation Framework for High-Performance System Design Space Exploration
by: Lin, Wei-Fen, et al.
Published: (2026)

3RSeT: Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison
by: Cheshmikhani, Elham, et al.
Published: (2025)

Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems
by: Yamamoto, Yuji, et al.
Published: (2026)

Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs
by: Aggarwal, Shivam, et al.
Published: (2023)

Putting the Context back into Memory
by: Roberts, David A.
Published: (2025)

Prefill vs. Decode Bottlenecks: SRAM-Frequency Tradeoffs and the Memory-Bandwidth Ceiling
by: Atmer, Hannah, et al.
Published: (2025)

Heterogeneous Memory Benchmarking Toolkit
by: Ghaemi, Golsana, et al.
Published: (2025)

Simulation-Driven Evaluation of Chiplet-Based Architectures Using VisualSim
by: Ali, Wajid, et al.
Published: (2025)

ETM2: Empowering Traditional Memory Bandwidth Regulation using ETM
by: Zuepke, Alexander, et al.
Published: (2026)

Towards CPU Performance Prediction: New Challenge Benchmark Dataset and Novel Approach
by: Liu, Xiaoman
Published: (2024)

Recurrent CircuitSAT Sampling for Sequential Circuits
by: Ardakani, Arash, et al.
Published: (2025)

Introducing the Arm-membench Throughput Benchmark
by: Burth, Cyrill, et al.
Published: (2025)

Enhancing software-hardware co-design for HEP by low-overhead profiling of single- and multi-threaded programs on diverse architectures with Adaptyst
by: Graczyk, Maksymilian, et al.
Published: (2025)

SPEC CPU2026: Characterization, Representativeness, and Cross-Suite Comparison
by: Li, Ruihao, et al.
Published: (2026)

Single 32-bit Sub-Channel DDR5 DIMMs: Architecture, Performance Bounds, and Standardisation
by: Ke, Chih-Hua
Published: (2026)

Regular-Dead on Arrival: Characterizing and Protecting Against Dead-Entry TLB Misses in GPU Microarchitectures
by: Anik, Shafayat Mowla, et al.
Published: (2026)

Makinote: An FPGA-Based HW/SW Platform for Pre-Silicon Emulation of RISC-V Designs
by: Perdomo, Elias, et al.
Published: (2024)

AI Load Dynamics--A Power Electronics Perspective
by: Li, Yuzhuo, et al.
Published: (2025)

SAHM: State-Aware Heterogeneous Multicore for Single-Thread Performance
by: Wadle, Shayne, et al.
Published: (2025)