Cover Image

FG-Attn: Leveraging Fine-Grained Sparsity In Diffusion Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Durvasula, Sankeerth, Sreedhar, Kavya, Moustafa, Zain, Kothawade, Suraj, Gondimalla, Ashish, Subramanian, Suvinay, Shahidi, Narges, Vijaykumar, Nandita
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Hardware Architecture
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ACS: Concurrent Kernel Execution on Irregular, Input-Dependent Computational Graphs
by: Durvasula, Sankeerth, et al.
Published: (2024)

Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
by: Bambhaniya, Abhimanyu Rajeshkumar, et al.
Published: (2024)

Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures
by: Vishwanathan, Manoj, et al.
Published: (2026)

Kratos: An FPGA Benchmark for Unrolled DNNs with Fine-Grained Sparsity and Mixed Precision
by: Dai, Xilai, et al.
Published: (2024)

Vision Transformer Computation and Resilience for Dynamic Inference
by: Sreedhar, Kavya, et al.
Published: (2022)

MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness
by: Wang, Huizheng, et al.
Published: (2025)

DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architectures
by: Yang, Peiming, et al.
Published: (2025)

Efficient Sparse Processing-in-Memory Architecture (ESPIM) for Machine Learning Inference
by: He, Mingxuan, et al.
Published: (2024)

FireFly-T: High-Throughput Sparsity Exploitation for Spiking Transformer Acceleration with Dual-Engine Overlay Architecture
by: Li, Tenglong, et al.
Published: (2025)

QED: Scalable Verification of Hardware Memory Consistency
by: Ravi, Gokulan, et al.
Published: (2024)

Piccolo: Large-Scale Graph Processing with Fine-Grained In-Memory Scatter-Gather
by: Shin, Changmin, et al.
Published: (2025)

Choreographer: A Full-System Framework for Fine-Grained Tasks in Cache Hierarchies
by: Nguyen, Hoa, et al.
Published: (2025)

SATA: Sparsity-Aware Scheduling for Selective Token Attention
by: Fan, Zhenkun, et al.
Published: (2026)

Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks
by: Wei, Chiyue, et al.
Published: (2025)

Fine-Grained Fusion: The Missing Piece in Area-Efficient State Space Model Acceleration
by: Geens, Robin, et al.
Published: (2025)

VolTune: A Fine-Grained Runtime Voltage Control Architecture for FPGA Systems
by: Ahmed, Akram Ben, et al.
Published: (2026)

Multilayer Dataflow: Orchestrate Butterfly Sparsity to Accelerate Attention Computation
by: Wu, Haibin, et al.
Published: (2024)

Prosperity: Accelerating Spiking Neural Networks via Product Sparsity
by: Wei, Chiyue, et al.
Published: (2025)

Hardware-Software Co-Design for Accelerating Transformer Inference Leveraging Compute-in-Memory
by: Kim, Dong Eun, et al.
Published: (2025)

Annotating Slack Directly on Your Verilog: Fine-Grained RTL Timing Evaluation for Early Optimization
by: Fang, Wenji, et al.
Published: (2024)

A Prototype-Based Framework to Design Scalable Heterogeneous SoCs with Fine-Grained DFS
by: Montanaro, Gabriele, et al.
Published: (2024)

Sectored DRAM: A Practical Energy-Efficient and High-Performance Fine-Grained DRAM Architecture
by: Olgun, Ataberk, et al.
Published: (2022)

Sim-FA: A GPGPU Simulator Framework for Fine-Grained FlashAttention Pipeline Analysis
by: Zhou, Zhongchun, et al.
Published: (2026)

Squire: A General-Purpose Accelerator to Exploit Fine-Grain Parallelism on Dependency-Bound Kernels
by: Langarita, Rubén, et al.
Published: (2025)

MXFormer: A Microscaling Floating-Point Charge-Trap Transistor Compute-in-Memory Transformer Accelerator
by: Karfakis, George, et al.
Published: (2026)

LEAP: LLM Inference on Scalable PIM-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism
by: Wang, Yimin, et al.
Published: (2025)

FLICKER: A Fine-Grained Contribution-Aware Accelerator for Real-Time 3D Gaussian Splatting
by: Ou, Wenhui, et al.
Published: (2026)

ReDas: A Lightweight Architecture for Supporting Fine-Grained Reshaping and Multiple Dataflows on Systolic Array
by: Han, Meng, et al.
Published: (2023)

PULSE: Parametric Hardware Units for Low-power Sparsity-Aware Convolution Engine
by: Aliyev, Ilkin, et al.
Published: (2024)

VIKIN: A Reconfigurable Accelerator for KANs and MLPs with Two-Stage Sparsity Support
by: Ou, Wenhui, et al.
Published: (2026)

DeepAssert: An LLM-Aided Verification Framework with Fine-Grained Assertion Generation for Modules with Extracted Module Specifications
by: Wang, Yonghao, et al.
Published: (2025)

PowerFlow-DNN: Compiler-Directed Fine-Grained Power Orchestration for End-to-End Edge AI Inference
by: Chen, Paul, et al.
Published: (2026)

Sparsity-Aware Streaming SNN Accelerator with Output-Channel Dataflow for Automatic Modulation Classification
by: Yang, Kuilian, et al.
Published: (2026)

Exploring the Sparsity-Quantization Interplay on a Novel Hybrid SNN Event-Driven Architecture
by: Aliyev, Ilkin, et al.
Published: (2024)

LogicSparse: Enabling Engine-Free Unstructured Sparsity for Quantised Deep-learning Accelerators
by: Li, Changhong, et al.
Published: (2025)

Commercial Evaluation of Zero-Skipping MAC Design for Bit Sparsity Exploitation in DL Inference
by: Nair, Harideep, et al.
Published: (2024)

Towards Efficient SRAM-PIM Architecture Design by Exploiting Unstructured Bit-Level Sparsity
by: Duan, Cenlin, et al.
Published: (2024)

TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge
by: Huang, Zhirui, et al.
Published: (2025)

DeMM: A Decoupled Matrix Multiplication Engine Supporting Relaxed Structured Sparsity
by: Peltekis, Christodoulos, et al.
Published: (2024)

DreamRAM: A Fine-Grained Configurable Design Space Modeling Tool for Custom 3D Die-Stacked DRAM
by: Cai, Victor, et al.
Published: (2025)