Cover Image

From Buffers to Registers: Unlocking Fine-Grained FlashAttention with Hybrid-Bonded 3D NPU Co-Design

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yu, Jinxin, Pan, Yudong, Wang, Mengdi, Li, Huawei, Han, Yinhe, Li, Xiaowei, Wang, Ying
Format:	Preprint
Published:	2026
Subjects:	Hardware Architecture Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sim-FA: A GPGPU Simulator Framework for Fine-Grained FlashAttention Pipeline Analysis
by: Zhou, Zhongchun, et al.
Published: (2026)

H-FA: A Hybrid Floating-Point and Logarithmic Approach to Hardware Accelerated FlashAttention
by: Alexandridis, Kosmas, et al.
Published: (2025)

Ouroboros: Wafer-Scale SRAM CIM with Token-Grained Pipelining for Large Language Model Inference
by: Liu, Yiqi, et al.
Published: (2026)

SystolicAttention: Fusing FlashAttention within a Single Systolic Array
by: Lin, Jiawei, et al.
Published: (2025)

COMET: Towards Partical W4A4KV4 LLMs Serving
by: Liu, Lian, et al.
Published: (2024)

Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
by: Alexandridis, Kosmas, et al.
Published: (2025)

FLASH-D: FlashAttention with Hidden Softmax Division
by: Alexandridis, Kosmas, et al.
Published: (2025)

ChipGPT: How far are we from natural language hardware design
by: Chang, Kaiyan, et al.
Published: (2023)

Be CIM or Be Memory: A Dual-mode-aware DNN Compiler for CIM Accelerators
by: Zhao, Shixin, et al.
Published: (2025)

DeepAssert: An LLM-Aided Verification Framework with Fine-Grained Assertion Generation for Modules with Extracted Module Specifications
by: Wang, Yonghao, et al.
Published: (2025)

From Characterization to Microarchitecture: Designing an Elegant and Reliable BFP-Based NPU
by: Zhang, Jie, et al.
Published: (2026)

CLASS: A Controller-Centric Layout Synthesizer for Dynamic Quantum Circuits
by: Chen, Yu, et al.
Published: (2025)

Data is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation framework
by: Chang, Kaiyan, et al.
Published: (2024)

Make LLM Inference Affordable to Everyone: Augmenting GPU Memory with NDP-DIMM
by: Liu, Lian, et al.
Published: (2025)

Strix: Re-thinking NPU Reliability from a System Perspective
by: Guan, Jiapeng, et al.
Published: (2026)

Sequence-Aware Split Heuristic to Mitigate SM Underutilization in FlashAttention-3 Low-Head-Count Decoding
by: Font, Martí Llopart, et al.
Published: (2026)

HLSPilot: LLM-based High-Level Synthesis
by: Xiong, Chenwei, et al.
Published: (2024)

Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation
by: Chang, Kaiyan, et al.
Published: (2024)

ChipSeek: Optimizing Verilog Generation via EDA-Integrated Reinforcement Learning
by: Chen, Zhirong, et al.
Published: (2025)

A System Architecture for Low Latency Multiprogramming Quantum Computing
by: Zhao, Yilun, et al.
Published: (2026)

Lifecycle Cost-Effectiveness Modeling for Redundancy-Enhanced Multi-Chiplet Architectures
by: Liu, Zizhen, et al.
Published: (2026)

MCU-MixQ: A HW/SW Co-optimized Mixed-precision Neural Network Design Framework for MCUs
by: Gong, Junfeng, et al.
Published: (2024)

Rethinking Compute Substrates for 3D-Stacked Near-Memory LLM Decoding: Microarchitecture-Scheduling Co-Design
by: Ai, Chenyang, et al.
Published: (2026)

ApproxPilot: A GNN-based Accelerator Approximation Framework
by: Zhang, Qing, et al.
Published: (2024)

PIMCOMP: An End-to-End DNN Compiler for Processing-In-Memory Accelerators
by: Sun, Xiaotian, et al.
Published: (2024)

LLMulator: Generalizable Cost Modeling for Dataflow Accelerators with Input-Adaptive Control Flow
by: Chang, Kaiyan, et al.
Published: (2025)

EONSim: An NPU Simulator for On-Chip Memory and Embedding Vector Operations
by: Choi, Sangun, et al.
Published: (2025)

TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
by: Pan, Yudong, et al.
Published: (2026)

PIMSYN: Synthesizing Processing-in-memory CNN Accelerators
by: Li, Wanqian, et al.
Published: (2024)

IICPilot: An Intelligent Integrated Circuit Backend Design Framework Using Open EDA
by: Jiang, Zesong, et al.
Published: (2024)

NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing
by: Heo, Guseul, et al.
Published: (2024)

IANUS: Integrated Accelerator based on NPU-PIM Unified Memory System
by: Seo, Minseok, et al.
Published: (2024)

PIMSIM-NN: An ISA-based Simulation Framework for Processing-in-Memory Accelerators
by: Wang, Xinyu, et al.
Published: (2024)

FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs
by: Xie, Xilong, et al.
Published: (2025)

A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs
by: Dang, Peng, et al.
Published: (2024)

Graphitron: A Domain Specific Language for FPGA-based Graph Processing Accelerator Generation
by: Zhang, Xinmiao, et al.
Published: (2024)

TriGen: NPU Architecture for End-to-End Acceleration of Large Language Models based on SW-HW Co-Design
by: Lee, Jonghun, et al.
Published: (2026)

Network Design for Wafer-Scale Systems with Wafer-on-Wafer Hybrid Bonding
by: Iff, Patrick, et al.
Published: (2026)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
by: Liu, Lian, et al.
Published: (2026)

Wrong Code, Right Structure: Learning Netlist Representations from Imperfect LLM-Generated RTL
by: Cai, Siyang, et al.
Published: (2026)