Image de couverture de livre

On the Surprising Effectiveness of Attention Transfer for Vision Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Alexander C., Tian, Yuandong, Chen, Beidi, Pathak, Deepak, Chen, Xinlei
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Neural and Evolutionary Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Generative Classifiers Avoid Shortcut Solutions
par: Li, Alexander C., et autres
Publié: (2025)

Understanding Transformer-based Vision Models through Inversion
par: Rathjens, Jan, et autres
Publié: (2024)

DS2TA: Denoising Spiking Transformer with Attenuated Spatiotemporal Attention
par: Xu, Boxun, et autres
Publié: (2024)

DRiVE: Dynamic Recognition in VEhicles using snnTorch
par: Vora, Heerak, et autres
Publié: (2025)

TPC-ViT: Token Propagation Controller for Efficient Vision Transformer
par: Zhu, Wentao
Publié: (2024)

SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
par: Zhang, Jintao, et autres
Publié: (2024)

Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization
par: Bhati, Deepshikha, et autres
Publié: (2024)

Kolmogorov-Arnold Transformer
par: Yang, Xingyi, et autres
Publié: (2024)

HyperGALE: ASD Classification via Hypergraph Gated Attention with Learnable Hyperedges
par: Arora, Mehul, et autres
Publié: (2024)

Training Frozen Feature Pyramid DINOv2 for Eyelid Measurements with Infinite Encoding and Orthogonal Regularization
par: Chen, Chun-Hung
Publié: (2025)

STAS: Spatio-Temporal Adaptive Computation Time for Spiking Transformers
par: Kang, Donghwa, et autres
Publié: (2025)

QKFormer: Hierarchical Spiking Transformer using Q-K Attention
par: Zhou, Chenlin, et autres
Publié: (2024)

Neural Sentinel: Unified Vision Language Model (VLM) for License Plate Recognition with Human-in-the-Loop Continual Learning
par: Sivakoti, Karthik
Publié: (2026)

SMART: Scalable Mesh-free Aerodynamic Simulations from Raw Geometries using a Transformer-based Surrogate Model
par: Hagnberger, Jan, et autres
Publié: (2026)

Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networks
par: Chen, Shuo, et autres
Publié: (2023)

Fast and Efficient Local Search for Genetic Programming Based Loss Function Learning
par: Raymond, Christian, et autres
Publié: (2024)

Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models!
par: Kotyan, Shashank, et autres
Publié: (2024)

STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction
par: Wu, Dennis, et autres
Publié: (2023)

Attention via Synaptic Plasticity is All You Need: A Biologically Inspired Spiking Neuromorphic Transformer
par: Mondal, Kallol, et autres
Publié: (2025)

TDFormer: A Top-Down Attention-Controlled Spiking Transformer
par: Zhu, Zizheng, et autres
Publié: (2025)

Making Images from Images: Interleaving Denoising and Transformation
par: Baluja, Shumeet, et autres
Publié: (2024)

Meta-Evolve: Continuous Robot Evolution for One-to-many Policy Transfer
par: Liu, Xingyu, et autres
Publié: (2024)

NeuRN: Neuro-inspired Domain Generalization for Image Classification
par: Jalil, Hamd, et autres
Publié: (2025)

Mice to Machines: Neural Representations from Visual Cortex for Domain Generalization
par: Qazi, Ahmed, et autres
Publié: (2025)

Concept Probing: Where to Find Human-Defined Concepts (Extended Version)
par: Ribeiro, Manuel de Sousa, et autres
Publié: (2025)

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
par: Kumar, Ravin
Publié: (2025)

The Platonic Representation Hypothesis
par: Huh, Minyoung, et autres
Publié: (2024)

Neuromimetic metaplasticity for adaptive continual learning
par: Cho, Suhee, et autres
Publié: (2024)

CoNO: Complex Neural Operator for Continous Dynamical Physical Systems
par: Tiwari, Karn, et autres
Publié: (2024)

Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models
par: Kynkäänniemi, Tuomas, et autres
Publié: (2024)

Benchmarking Spiking Neural Network Learning Methods with Varying Locality
par: Lin, Jiaqi, et autres
Publié: (2024)

Winning the Lottery by Preserving Network Training Dynamics with Concrete Ticket Search
par: Arora, Tanay, et autres
Publié: (2025)

Learning Evolution via Optimization Knowledge Adaptation
par: Wang, Chao, et autres
Publié: (2025)

When Person Re-Identification Meets Event Camera: A Benchmark Dataset and An Attribute-guided Re-Identification Framework
par: Wang, Xiao, et autres
Publié: (2025)

Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness
par: Du, Kangrui, et autres
Publié: (2025)

An extended asymmetric sigmoid with Perceptron (SIGTRON) for imbalanced linear classification
par: Woo, Hyenkyun
Publié: (2023)

APTx: better activation function than MISH, SWISH, and ReLU's variants used in deep learning
par: Kumar, Ravin
Publié: (2022)

Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methods
par: Mumuni, Alhassan, et autres
Publié: (2024)

It's Not a Lottery, It's a Race: Understanding How Gradient Descent Adapts the Network's Capacity to the Task
par: Pinson, Hannah
Publié: (2026)

Fooling Contrastive Language-Image Pre-trained Models with CLIPMasterPrints
par: Freiberger, Matthias, et autres
Publié: (2023)