Cover Image

Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gabetni, Firas, Curci, Giuseppe, Pilzer, Andrea, Roy, Subhankar, Ricci, Elisa, Franchi, Gianni
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

From Local Geometry to Global Pseudo Labeling for Robust Positive Unlabeled Learning under Covariate Shift
by: Gabetni, Firas, et al.
Published: (2026)

Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation
by: Franchi, Gianni, et al.
Published: (2024)

LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups
by: Aminbeidokhti, Masih, et al.
Published: (2025)

Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification
by: Lafage, Adrien, et al.
Published: (2025)

Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery
by: Liu, Mingxuan, et al.
Published: (2023)

Organizing Unstructured Image Collections using Natural Language
by: Liu, Mingxuan, et al.
Published: (2024)

How (Mis)calibrated is Your Federated CLIP and What To Do About It?
by: Singha, Mainak, et al.
Published: (2025)

Clean-GS: Semantic Mask-Guided Pruning for 3D Gaussian Splatting
by: Mishra, Subhankar
Published: (2026)

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference
by: Pathak, Surendra, et al.
Published: (2026)

COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
by: Humblot-Renaux, Galadrielle, et al.
Published: (2025)

Weighted Ensemble Models Are Strong Continual Learners
by: Marouf, Imad Eddine, et al.
Published: (2023)

A Geometric Unification of Concept Learning with Concept Cones
by: Rocchi--Henry, Alexandre, et al.
Published: (2025)

Hierarchical Light Transformer Ensembles for Multimodal Trajectory Forecasting
by: Lafage, Adrien, et al.
Published: (2024)

FakeParts: a New Family of AI-Generated DeepFakes
by: Liu, Ziyi, et al.
Published: (2025)

Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs
by: Chorna, Sofiia, et al.
Published: (2025)

Bayesian Autoencoder for Medical Anomaly Detection: Uncertainty-Aware Approach for Brain 2 MRI Analysis
by: Roy, Dip
Published: (2025)

Rényi Attention Entropy for Patch Pruning
by: Aizawa, Hiroaki, et al.
Published: (2026)

Uncertainty-Aware Token Importance Estimation in Spiking Transformers
by: Liu, Wenxuan, et al.
Published: (2026)

Large-scale Dataset Pruning with Dynamic Uncertainty
by: He, Muyang, et al.
Published: (2023)

Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It
by: Xia, Guoxuan, et al.
Published: (2024)

Efficient Image Generation with Variadic Attention Heads
by: Walton, Steven, et al.
Published: (2022)

Head Pursuit: Probing Attention Specialization in Multimodal Transformers
by: Basile, Lorenzo, et al.
Published: (2025)

Robust Few-Shot Ensemble Learning with Focal Diversity-Based Pruning
by: Tekin, Selim Furkan, et al.
Published: (2024)

Uncertainty-Aware Vision-Language Segmentation for Medical Imaging
by: Das, Aryan, et al.
Published: (2026)

CAPA: Contribution-Aware Pruning and FFN Approximation for Efficient Large Vision-Language Models
by: Jha, Samyak, et al.
Published: (2026)

Learning to Generate Training Datasets for Robust Semantic Segmentation
by: Hariat, Marwane, et al.
Published: (2023)

Detecting Brain Tumors through Multimodal Neural Networks
by: Curci, Antonio, et al.
Published: (2024)

GUESS: Generative Uncertainty Ensemble for Self Supervision
by: Mohamadi, Salman, et al.
Published: (2024)

Steering Sparse Autoencoder Latents to Control Dynamic Head Pruning in Vision Transformers (Student Abstract)
by: Lee, Yousung, et al.
Published: (2026)

EDiT: Efficient Diffusion Transformers with Linear Compressed Attention
by: Becker, Philipp, et al.
Published: (2025)

MoH: Multi-Head Attention as Mixture-of-Head Attention
by: Jin, Peng, et al.
Published: (2024)

Equivariant-Aware Structured Pruning for Efficient Edge Deployment: A Comprehensive Framework with Adaptive Fine-Tuning
by: Alnemari, Mohammed
Published: (2025)

ICE-Pruning: An Iterative Cost-Efficient Pruning Pipeline for Deep Neural Networks
by: Hu, Wenhao, et al.
Published: (2025)

PruneFuse: Efficient Data Selection via Weight Pruning and Network Fusion
by: Kousar, Humaira, et al.
Published: (2026)

Uncertainty-Aware Dual-Student Knowledge Distillation for Efficient Image Classification
by: Gore, Aakash, et al.
Published: (2025)

NECO: NEural Collapse Based Out-of-distribution detection
by: Ammar, Mouïn Ben, et al.
Published: (2023)

AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer
by: Shan, Jiquan, et al.
Published: (2025)

Adaptive Sharpness-Aware Pruning for Robust Sparse Networks
by: Bair, Anna, et al.
Published: (2023)

Attention-ResUNet for Automated Fetal Head Segmentation
by: Bhilwarawala, Ammar, et al.
Published: (2026)

Boosting Adversarial Transferability via Ensemble Non-Attention
by: Zou, Yipeng, et al.
Published: (2025)