Copertina

Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context Transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Horton, Mark, Molom-Ochir, Tergel, Liu, Peter, Gopal, Bhavna, Wei, Chiyue, Guo, Cong, Taylor, Brady, Fan, Deliang, Wang, Shan X., Li, Hai, Chen, Yiran
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Image and Video Processing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MonoSparse-CAM: Efficient Tree Model Processing via Monotonicity and Sparsity in CAMs
di: Molom-Ochir, Tergel, et al.
Pubblicazione: (2024)

CAMformer: Associative Memory is All You Need
di: Molom-Ochir, Tergel, et al.
Pubblicazione: (2025)

End-to-End Transformer Acceleration Through Processing-in-Memory Architectures
di: Yang, Xiaoxuan, et al.
Pubblicazione: (2025)

SAFER: Sharpness Aware layer-selective Finetuning for Enhanced Robustness in vision transformers
di: Gopal, Bhavna, et al.
Pubblicazione: (2025)

Binarized Low-light Raw Video Enhancement
di: Zhang, Gengchen, et al.
Pubblicazione: (2024)

Criticality Leveraged Adversarial Training (CLAT) for Boosted Performance via Parameter Efficiency
di: Gopal, Bhavna, et al.
Pubblicazione: (2024)

Context-Aware Asymmetric Ensembling for Interpretable Retinopathy of Prematurity Screening via Active Query and Vascular Attention
di: Hassan, Md. Mehedi, et al.
Pubblicazione: (2026)

HippoMM: Hippocampal-inspired Multimodal Memory for Long Audiovisual Event Understanding
di: Lin, Yueqian, et al.
Pubblicazione: (2025)

Parallel Context Modeling for Sliding Window Attention in Neural Video Coding
di: Kopte, Alexander, et al.
Pubblicazione: (2026)

ELFATT: Efficient Linear Fast Attention for Vision Transformers
di: Wu, Chong, et al.
Pubblicazione: (2025)

PupiNet: Seamless OCT-OCTA Interconversion Through Wavelet-Driven and Multi-Scale Attention Mechanisms
di: Tian, Renzhi, et al.
Pubblicazione: (2025)

Transformer based Endmember Fusion with Spatial Context for Hyperspectral Unmixing
di: Ratnayake, R. M. K. L., et al.
Pubblicazione: (2024)

Comparative Analysis of Binarization Methods For Medical Image Hashing On Odir Dataset
di: Muzoglu, Nedim
Pubblicazione: (2026)

Building Lightweight Semantic Segmentation Models for Aerial Images Using Dual Relation Distillation
di: Li, Minglong, et al.
Pubblicazione: (2025)

OSLO-IC: On-the-Sphere Learned Omnidirectional Image Compression with Attention Modules and Spatial Context
di: Wawerek-López, Paul, et al.
Pubblicazione: (2025)

Global and Local Attention-Based Transformer for Hyperspectral Image Change Detection
di: Wang, Ziyi, et al.
Pubblicazione: (2024)

Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression
di: Koyuncu, A. Burakhan, et al.
Pubblicazione: (2022)

KDPhys: An Attention Guided 3D to 2D Knowledge Distillation for Real-time Video-Based Physiological Measurement
di: Sahoo, Nicky Nirlipta, et al.
Pubblicazione: (2026)

4D-ACFNet: A 4D Attention Mechanism-Based Prognostic Framework for Colorectal Cancer Liver Metastasis Integrating Multimodal Spatiotemporal Features
di: Li, Zesheng, et al.
Pubblicazione: (2025)

ATFusion: An Alternate Cross-Attention Transformer Network for Infrared and Visible Image Fusion
di: Yan, Han, et al.
Pubblicazione: (2024)

FedWSIDD: Federated Whole Slide Image Classification via Dataset Distillation
di: Jin, Haolong, et al.
Pubblicazione: (2025)

A Graph-Augmented knowledge Distillation based Dual-Stream Vision Transformer with Region-Aware Attention for Gastrointestinal Disease Classification with Explainable AI
di: Assaduzzaman, Md, et al.
Pubblicazione: (2025)

Efficient Star Distillation Attention Network for Lightweight Image Super-Resolution
di: Hao, Fangwei, et al.
Pubblicazione: (2025)

Prompted Contextual Transformer for Incomplete-View CT Reconstruction
di: Ma, Chenglong, et al.
Pubblicazione: (2023)

Adaptive Online Learning of Separable Path Graph Transforms for Intra-prediction
di: Lu, Wen-Yang, et al.
Pubblicazione: (2024)

Synthetic Volumetric Data Generation Enables Zero-Shot Generalization of Foundation Models in 3D Medical Image Segmentation
di: Chakrabarty, Satrajit, et al.
Pubblicazione: (2026)

Multi class activity classification in videos using Motion History Image generation
di: Gopal, Senthilkumar
Pubblicazione: (2024)

CSAKD: Knowledge Distillation with Cross Self-Attention for Hyperspectral and Multispectral Image Fusion
di: Hsu, Chih-Chung, et al.
Pubblicazione: (2024)

S2LIC: Learned Image Compression with the SwinV2 Block, Adaptive Channel-wise and Global-inter Attention Context
di: Wang, Yongqiang, et al.
Pubblicazione: (2024)

VOLT: Volumetric Wide-Field Microscopy via 3D-Native Probabilistic Transport
di: He, Yetao, et al.
Pubblicazione: (2026)

Key-Graph Transformer for Image Restoration
di: Ren, Bin, et al.
Pubblicazione: (2024)

Designing Parameter and Compute Efficient Diffusion Transformers using Distillation
di: Sundaresha, Vignesh
Pubblicazione: (2025)

Region Attention Transformer for Medical Image Restoration
di: Yang, Zhiwen, et al.
Pubblicazione: (2024)

Prognostic Model for Idiopathic Pulmonary Fibrosis Using Context-Aware Sequential-Parallel Hybrid Transformer and Enriched Clinical Information
di: Dolatabadi, Mahdie, et al.
Pubblicazione: (2025)

Adaptive-avg-pooling based Attention Vision Transformer for Face Anti-spoofing
di: Yang, Jichen, et al.
Pubblicazione: (2024)

Beyond Subspace Isolation: Many-to-Many Transformer for Light Field Image Super-resolution
di: Hu, Zeke Zexi, et al.
Pubblicazione: (2024)

S$^3$Attention: Improving Long Sequence Attention with Smoothed Skeleton Sketching
di: Wang, Xue, et al.
Pubblicazione: (2024)

Window-based Channel Attention for Wavelet-enhanced Learned Image Compression
di: Xu, Heng, et al.
Pubblicazione: (2024)

KDAS: Knowledge Distillation via Attention Supervision Framework for Polyp Segmentation
di: Trinh, Quoc-Huy, et al.
Pubblicazione: (2023)

SGSR: Structure-Guided Multi-Contrast MRI Super-Resolution via Spatio-Frequency Co-Query Attention
di: Zheng, Shaoming, et al.
Pubblicazione: (2024)