Cover Image

Towards Better Multi-head Attention via Channel-wise Sample Permutation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yuan, Shen, Xu, Hongteng
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation
by: Yuan, Shen, et al.
Published: (2024)

Multi-Task Model Merging via Adaptive Weight Disentanglement
by: Xiong, Feng, et al.
Published: (2024)

Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization
by: Li, Xintong, et al.
Published: (2025)

Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift
by: Xu, Qinwu
Published: (2026)

CSA-Net: Channel-wise Spatially Autocorrelated Attention Networks
by: Nikzad, Nick, et al.
Published: (2024)

Exploring Attention Mechanisms in Integration of Multi-Modal Information for Sign Language Recognition and Translation
by: Hakim, Zaber Ibn Abdul, et al.
Published: (2023)

Reinforced Attention Learning
by: Li, Bangzheng, et al.
Published: (2026)

Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition
by: Tang, Minxue, et al.
Published: (2026)

Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
by: Kim, Jinhee, et al.
Published: (2025)

Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models
by: Rajabi, Navid, et al.
Published: (2023)

TechING: Towards Real World Technical Image Understanding via VLMs
by: Nadeem, Tafazzul, et al.
Published: (2026)

Rethinking Training Dynamics in Scale-wise Autoregressive Generation
by: Zhou, Gengze, et al.
Published: (2025)

RadZero: Similarity-Based Cross-Attention for Explainable Vision-Language Alignment in Chest X-ray with Zero-Shot Multi-Task Capability
by: Park, Jonggwon, et al.
Published: (2025)

Interleaving Reasoning for Better Text-to-Image Generation
by: Huang, Wenxuan, et al.
Published: (2025)

Listen Then See: Video Alignment with Speaker Attention
by: Agrawal, Aviral, et al.
Published: (2024)

Transformer with Controlled Attention for Synchronous Motion Captioning
by: Radouane, Karim, et al.
Published: (2024)

MHA2MLA-VLM: Enabling DeepSeek's Economical Multi-Head Latent Attention across Vision-Language Models
by: Fan, Xiaoran, et al.
Published: (2026)

Towards Class-wise Robustness Analysis
by: Medi, Tejaswini, et al.
Published: (2024)

Head Pursuit: Probing Attention Specialization in Multimodal Transformers
by: Basile, Lorenzo, et al.
Published: (2025)

CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers
by: Yamada, Yoshihiro
Published: (2025)

MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention
by: Li, Yucheng, et al.
Published: (2025)

VONet: Unsupervised Video Object Learning With Parallel U-Net Attention and Object-wise Sequential VAE
by: Yu, Haonan, et al.
Published: (2024)

URRL-IMVC: Unified and Robust Representation Learning for Incomplete Multi-View Clustering
by: Teng, Ge, et al.
Published: (2024)

The ART of Composition: Attention-Regularized Training for Compositional Visual Grounding
by: Luo, Jiayun, et al.
Published: (2024)

R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization
by: Zhang, Jingyi, et al.
Published: (2025)

LaMI: Augmenting Large Language Models via Late Multi-Image Fusion
by: Yariv, Guy, et al.
Published: (2024)

Enhancing Multi-view Open-set Learning via Ambiguity Uncertainty Calibration and View-wise Debiasing
by: Fang, Zihan, et al.
Published: (2025)

ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement
by: Salamatian, Ali, et al.
Published: (2025)

Exploring Multi-Grained Concept Annotations for Multimodal Large Language Models
by: Xu, Xiao, et al.
Published: (2024)

A Survey on Data Augmentation in Large Model Era
by: Zhou, Yue, et al.
Published: (2024)

Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
by: Madaan, Divyam, et al.
Published: (2025)

ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities
by: Ghosh, Adhiraj, et al.
Published: (2024)

Is Pre-training Truly Better Than Meta-Learning?
by: Miranda, Brando, et al.
Published: (2023)

Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Tasks
by: Luo, Yaxin, et al.
Published: (2026)

MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning
by: Liang, Yiqing, et al.
Published: (2025)

Towards Statistical Factuality Guarantee for Large Vision-Language Models
by: Li, Zhuohang, et al.
Published: (2025)

Sparser Block-Sparse Attention via Token Permutation
by: Wang, Xinghao, et al.
Published: (2025)

VisionZip: Longer is Better but Not Necessary in Vision Language Models
by: Yang, Senqiao, et al.
Published: (2024)

Towards Better Understanding Attribution Methods
by: Rao, Sukrut, et al.
Published: (2022)

GenSim: Generating Robotic Simulation Tasks via Large Language Models
by: Wang, Lirui, et al.
Published: (2023)