Image de couverture de livre

ODMA: On-Demand Memory Allocation Strategy for LLM Serving on LPDDR-Class Accelerators

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zou, Guoqiang, Wang, Wanyu, Zheng, Hao, Yin, Longxiang, Han, Yinhe
Format:	Preprint
Publié:	2025
Sujets:	Hardware Architecture Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LP5X-PIM Sim: A High-Fidelity HW/SW Integrated Simulator for LPDDR5X-PIM
par: Cha, SangHoon, et autres
Publié: (2026)

ROMA: a Read-Only-Memory-based Accelerator for QLoRA-based On-Device LLM
par: Wang, Wenqiang, et autres
Publié: (2025)

Wrong Code, Right Structure: Learning Netlist Representations from Imperfect LLM-Generated RTL
par: Cai, Siyang, et autres
Publié: (2026)

LLM-DSE: Searching Accelerator Parameters with LLM Agents
par: Wang, Hanyu, et autres
Publié: (2025)

PIMSIM-NN: An ISA-based Simulation Framework for Processing-in-Memory Accelerators
par: Wang, Xinyu, et autres
Publié: (2024)

From Buffers to Registers: Unlocking Fine-Grained FlashAttention with Hybrid-Bonded 3D NPU Co-Design
par: Yu, Jinxin, et autres
Publié: (2026)

HALO: Memory-Centric Heterogeneous Accelerator with 2.5D Integration for Low-Batch LLM Inference
par: Negi, Shubham, et autres
Publié: (2025)

PIMCOMP: An End-to-End DNN Compiler for Processing-In-Memory Accelerators
par: Sun, Xiaotian, et autres
Publié: (2024)

Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
par: Fang, Yunhua, et autres
Publié: (2025)

Kelle: Co-design KV Caching and eDRAM for Efficient LLM Serving in Edge Computing
par: Xia, Tianhua, et autres
Publié: (2025)

MICSim: A Modular Simulator for Mixed-signal Compute-in-Memory based AI Accelerator
par: Wang, Cong, et autres
Publié: (2024)

A Configurable and Efficient Memory Hierarchy for Neural Network Hardware Accelerator
par: Bause, Oliver, et autres
Publié: (2024)

LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
par: He, Zifan, et autres
Publié: (2025)

BF-IMNA: A Bit Fluid In-Memory Neural Architecture for Neural Network Acceleration
par: Rakka, Mariam, et autres
Publié: (2024)

Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation
par: Chang, Kaiyan, et autres
Publié: (2024)

CD-PIM: A High-Bandwidth and Compute-Efficient LPDDR5-Based PIM for Low-Batch LLM Acceleration on Edge-Device
par: Lin, Ye, et autres
Publié: (2026)

Be CIM or Be Memory: A Dual-mode-aware DNN Compiler for CIM Accelerators
par: Zhao, Shixin, et autres
Publié: (2025)

Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator
par: Ramachandran, Akshat, et autres
Publié: (2025)

Sangam: Chiplet-Based DRAM-PIM Accelerator with CXL Integration for LLM Inferencing
par: Kiyawat, Khyati, et autres
Publié: (2025)

Comparative Characterization of KV Cache Management Strategies for LLM Inference
par: Mamo, Oteo, et autres
Publié: (2026)

Accelerating Post-Quantum Cryptography via LLM-Driven Hardware-Software Co-Design
par: Liao, Yuchao, et autres
Publié: (2026)

NVR: Vector Runahead on NPUs for Sparse Memory Access
par: Wang, Hui, et autres
Publié: (2025)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
par: Liu, Lian, et autres
Publié: (2026)

OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling
par: Yi, Xiaoling, et autres
Publié: (2024)

Comprehensive Design Space Exploration for Tensorized Neural Network Hardware Accelerators
par: Zhang, Jinsong, et autres
Publié: (2025)

Pushing the Limits of BFP on Narrow Precision LLM Inference
par: Wang, Hui, et autres
Publié: (2025)

ChipGPT: How far are we from natural language hardware design
par: Chang, Kaiyan, et autres
Publié: (2023)

Enhanced LPDDR4X PHY in 12 nm FinFET
par: Feldmann, Johannes, et autres
Publié: (2025)

LP-Spec: Leveraging LPDDR PIM for Efficient LLM Mobile Speculative Inference with Architecture-Dataflow Co-Optimization
par: He, Siyuan, et autres
Publié: (2025)

LLM-Driven Design Space Exploration of FPGA-based Accelerators
par: Sharma, Vinamra, et autres
Publié: (2026)

BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge
par: Ji, Yuhao, et autres
Publié: (2024)

Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference
par: Chen, Chun-Ting, et autres
Publié: (2025)

Scalable Processing-Near-Memory for 1M-Token LLM Inference: CXL-Enabled KV-Cache Management Beyond GPU Limits
par: Kim, Dowon, et autres
Publié: (2025)

Optimizing Neural Networks with Learnable Non-Linear Activation Functions via Lookup-Based FPGA Acceleration
par: Yin, Mengyuan, et autres
Publié: (2025)

APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
par: Ma, Shaobo, et autres
Publié: (2025)

FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization
par: Wang, Aotao, et autres
Publié: (2025)

PIMSYN: Synthesizing Processing-in-memory CNN Accelerators
par: Li, Wanqian, et autres
Publié: (2024)

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
par: Xia, Haojun, et autres
Publié: (2024)

M$^2$-ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantization
par: Liang, Yanbiao, et autres
Publié: (2024)

T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET
par: Moon, Seunghyun, et autres
Publié: (2025)