Copertina

H-FA: A Hybrid Floating-Point and Logarithmic Approach to Hardware Accelerated FlashAttention

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Alexandridis, Kosmas, Dimitrakopoulos, Giorgos
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Hardware Architecture
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
di: Alexandridis, Kosmas, et al.
Pubblicazione: (2025)

FLASH-D: FlashAttention with Hidden Softmax Division
di: Alexandridis, Kosmas, et al.
Pubblicazione: (2025)

Online Alignment and Addition in Multi-Term Floating-Point Adders
di: Alexandridis, Kosmas, et al.
Pubblicazione: (2024)

Floating-Point Multiply-Add with Approximate Normalization for Low-Cost Matrix Engines
di: Alexandridis, Kosmas, et al.
Pubblicazione: (2024)

Reusing Softmax Hardware Unit for GELU Computation in Transformers
di: Peltekis, Christodoulos, et al.
Pubblicazione: (2024)

Custom Algorithm-based Fault Tolerance for Attention Layers in Transformers
di: Titopoulos, Vasileios, et al.
Pubblicazione: (2025)

Register Dispersion: Reducing the Footprint of the Vector Register File in Vector Engines of Low-Cost RISC-V CPUs
di: Titopoulos, Vasileios, et al.
Pubblicazione: (2025)

Optimizing Structured-Sparse Matrix Multiplication in RISC-V Vector Processors
di: Titopoulos, Vasileios, et al.
Pubblicazione: (2025)

Sim-FA: A GPGPU Simulator Framework for Fine-Grained FlashAttention Pipeline Analysis
di: Zhou, Zhongchun, et al.
Pubblicazione: (2026)

Vectorized FlashAttention with Low-cost Exponential Computation in RISC-V Vector Processors
di: Titopoulos, Vasileios, et al.
Pubblicazione: (2025)

High-Performance Pipelined NTT Accelerators with Homogeneous Digit-Serial Modulo Arithmetic
di: Alexakis, George, et al.
Pubblicazione: (2025)

SystolicAttention: Fusing FlashAttention within a Single Systolic Array
di: Lin, Jiawei, et al.
Pubblicazione: (2025)

From Buffers to Registers: Unlocking Fine-Grained FlashAttention with Hybrid-Bonded 3D NPU Co-Design
di: Yu, Jinxin, et al.
Pubblicazione: (2026)

Error Checking for Sparse Systolic Tensor Arrays
di: Peltekis, Christodoulos, et al.
Pubblicazione: (2024)

GCN-ABFT: Low-Cost Online Error Checking for Graph Convolutional Networks
di: Peltekis, Christodoulos, et al.
Pubblicazione: (2024)

DeMM: A Decoupled Matrix Multiplication Engine Supporting Relaxed Structured Sparsity
di: Peltekis, Christodoulos, et al.
Pubblicazione: (2024)

Efficient Implementation of RISC-V Vector Permutation Instructions
di: Titopoulos, Vasileios, et al.
Pubblicazione: (2025)

Periodic Online Testing for Sparse Systolic Tensor Arrays
di: Peltekis, Christodoulos, et al.
Pubblicazione: (2025)

A Hybrid-Domain Floating-Point Compute-in-Memory Architecture for Efficient Acceleration of High-Precision Deep Neural Networks
di: Yi, Zhiqiang, et al.
Pubblicazione: (2025)

Sequence-Aware Split Heuristic to Mitigate SM Underutilization in FlashAttention-3 Low-Head-Count Decoding
di: Font, Martí Llopart, et al.
Pubblicazione: (2026)

Closing the Gap Between Float and Posit Hardware Efficiency
di: Jonnalagadda, Aditya Anirudh, et al.
Pubblicazione: (2026)

BBAL: A Bidirectional Block Floating Point-Based Quantisation Accelerator for Large Language Models
di: Han, Xiaomeng, et al.
Pubblicazione: (2025)

MXFormer: A Microscaling Floating-Point Charge-Trap Transistor Compute-in-Memory Transformer Accelerator
di: Karfakis, George, et al.
Pubblicazione: (2026)

TimeFloats: Train-in-Memory with Time-Domain Floating-Point Scalar Products
di: Hashem, Maeesha Binte, et al.
Pubblicazione: (2024)

Xpikeformer: Hybrid Analog-Digital Hardware Acceleration for Spiking Transformers
di: Song, Zihang, et al.
Pubblicazione: (2024)

SafeCiM: Investigating Resilience of Hybrid Floating-Point Compute-in-Memory Deep Learning Accelerators
di: Bhattacharya, Swastik, et al.
Pubblicazione: (2025)

From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing
di: Zhao, Yushu, et al.
Pubblicazione: (2025)

The AetherFloat Family: Block-Scale-Free Quad-Radix Floating-Point Architectures for AI Accelerators
di: Morisaki, Keita
Pubblicazione: (2026)

Hybrid Photonic-digital Accelerator for Attention Mechanism
di: Li, Huize, et al.
Pubblicazione: (2025)

Enabling Efficient Hardware Acceleration of Hybrid Vision Transformer (ViT) Networks at the Edge
di: Dumoulin, Joren, et al.
Pubblicazione: (2025)

Towards the Certification of Hybrid Architectures: Analysing Interference on Hardware Accelerators through PML
di: Lesage, Benjamin, et al.
Pubblicazione: (2024)

Generalized Methodology for Determining Numerical Features of Hardware Floating-Point Matrix Multipliers: Part I
di: Khattak, Faizan A, et al.
Pubblicazione: (2025)

Fast Generation of Custom Floating-Point Spatial Filters on FPGAs
di: Campos, Nelson, et al.
Pubblicazione: (2024)

Floating Point HUB Adder for RISC-V Sargantana Processor
di: Bandera, Gerardo, et al.
Pubblicazione: (2024)

MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration
di: Armeniakos, Giorgos, et al.
Pubblicazione: (2025)

Hardware-Aware Neural Network Compilation with Learned Optimization: A RISC-V Accelerator Approach
di: Ganti, Ravindra, et al.
Pubblicazione: (2025)

A Stochastic Rounding-Enabled Low-Precision Floating-Point MAC for DNN Training
di: Ali, Sami Ben, et al.
Pubblicazione: (2024)

Accelerator-assisted Floating-point ASIP for Communication and Positioning in Massive MIMO Systems
di: Attari, Mohammad, et al.
Pubblicazione: (2025)

Converting Binary Floating-Point Numbers to Shortest Decimal Strings: An Experimental Review
di: Gareau, Jaël Champagne, et al.
Pubblicazione: (2026)

Presto: Hardware Acceleration of Ciphers for Hybrid Homomorphic Encryption
di: Jeon, Yeonsoo, et al.
Pubblicazione: (2025)