Cover Image

Custom Algorithm-based Fault Tolerance for Attention Layers in Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Titopoulos, Vasileios, Alexandridis, Kosmas, Dimitrakopoulos, Giorgos
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Hardware Architecture
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
by: Alexandridis, Kosmas, et al.
Published: (2025)

FLASH-D: FlashAttention with Hidden Softmax Division
by: Alexandridis, Kosmas, et al.
Published: (2025)

Reusing Softmax Hardware Unit for GELU Computation in Transformers
by: Peltekis, Christodoulos, et al.
Published: (2024)

H-FA: A Hybrid Floating-Point and Logarithmic Approach to Hardware Accelerated FlashAttention
by: Alexandridis, Kosmas, et al.
Published: (2025)

Register Dispersion: Reducing the Footprint of the Vector Register File in Vector Engines of Low-Cost RISC-V CPUs
by: Titopoulos, Vasileios, et al.
Published: (2025)

Optimizing Structured-Sparse Matrix Multiplication in RISC-V Vector Processors
by: Titopoulos, Vasileios, et al.
Published: (2025)

Online Alignment and Addition in Multi-Term Floating-Point Adders
by: Alexandridis, Kosmas, et al.
Published: (2024)

Floating-Point Multiply-Add with Approximate Normalization for Low-Cost Matrix Engines
by: Alexandridis, Kosmas, et al.
Published: (2024)

Vectorized FlashAttention with Low-cost Exponential Computation in RISC-V Vector Processors
by: Titopoulos, Vasileios, et al.
Published: (2025)

Efficient Implementation of RISC-V Vector Permutation Instructions
by: Titopoulos, Vasileios, et al.
Published: (2025)

DeMM: A Decoupled Matrix Multiplication Engine Supporting Relaxed Structured Sparsity
by: Peltekis, Christodoulos, et al.
Published: (2024)

GCN-ABFT: Low-Cost Online Error Checking for Graph Convolutional Networks
by: Peltekis, Christodoulos, et al.
Published: (2024)

Periodic Online Testing for Sparse Systolic Tensor Arrays
by: Peltekis, Christodoulos, et al.
Published: (2025)

FORTALESA: Fault-Tolerant Reconfigurable Systolic Array for DNN Inference
by: Cherezova, Natalia, et al.
Published: (2025)

Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM
by: Li, Bingbing, et al.
Published: (2024)

MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration
by: Armeniakos, Giorgos, et al.
Published: (2025)

High-Performance Pipelined NTT Accelerators with Homogeneous Digit-Serial Modulo Arithmetic
by: Alexakis, George, et al.
Published: (2025)

Error Checking for Sparse Systolic Tensor Arrays
by: Peltekis, Christodoulos, et al.
Published: (2024)

Algorithmic Strategies for Sustainable Reuse of Neural Network Accelerators with Permanent Faults
by: Alama, Youssef A. Ait, et al.
Published: (2024)

An Analog and Digital Hybrid Attention Accelerator for Transformers with Charge-based In-memory Computing
by: Moradifirouzabadi, Ashkan, et al.
Published: (2024)

ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers
by: İslamoğlu, Gamze, et al.
Published: (2023)

AdAM: Adaptive Fault-Tolerant Approximate Multiplier for Edge DNN Accelerators
by: Taheri, Mahdi, et al.
Published: (2024)

SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference
by: Wang, Wenxun, et al.
Published: (2025)

COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inference
by: Qiao, Ye, et al.
Published: (2025)

VeriBug: An Attention-based Framework for Bug-Localization in Hardware Designs
by: Stracquadanio, Giuseppe, et al.
Published: (2024)

MAx-DNN: Multi-Level Arithmetic Approximation for Energy-Efficient DNN Hardware Accelerators
by: Leon, Vasileios, et al.
Published: (2025)

Exploration of Activation Fault Reliability in Quantized Systolic Array-Based DNN Accelerators
by: Taheri, Mahdi, et al.
Published: (2024)

Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow
by: Wiese, Philip, et al.
Published: (2024)

An ECC-based Fault Tolerance Approach for DNNs
by: Raji, Mohsen, et al.
Published: (2025)

MCEL: Margin-Based Cross-Entropy Loss for Error-Tolerant Quantized Neural Networks
by: Yayla, Mikail, et al.
Published: (2026)

Bit-Flip Fault Attack: Crushing Graph Neural Networks via Gradual Bit Search
by: Abharian, Sanaz Kazemi, et al.
Published: (2025)

Combining Fault Tolerance Techniques and COTS SoC Accelerators for Payload Processing in Space
by: Leon, Vasileios, et al.
Published: (2025)

PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Models
by: Cho, Eunyeong, et al.
Published: (2026)

Evaluating Four FPGA-accelerated Space Use Cases based on Neural Network Algorithms for On-board Inference
by: Antunes, Pedro, et al.
Published: (2026)

Circuit Transformer: A Transformer That Preserves Logical Equivalence
by: Li, Xihan, et al.
Published: (2024)

SCATTER: Algorithm-Circuit Co-Sparse Photonic Accelerator with Thermal-Tolerant, Power-Efficient In-situ Light Redistribution
by: Yin, Ziang, et al.
Published: (2024)

Layer-wise Weight Selection for Power-Efficient Neural Network Acceleration
by: Fang, Jiaxun, et al.
Published: (2025)

Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
by: Choi, Dawon, et al.
Published: (2026)

Cross-Layer Co-Optimized LSTM Accelerator for Real-Time Gait Analysis
by: Ahmadilivani, Mohammad Hasan, et al.
Published: (2026)

Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits
by: Deng, Chenhui, et al.
Published: (2024)