Cover Image

Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Teo, Rachel S. Y., Nguyen, Tan M.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling
by: Teo, Rachel S. Y., et al.
Published: (2025)

MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
by: Teo, Rachel S. Y., et al.
Published: (2024)

Elliptical Attention
by: Nielsen, Stefan K., et al.
Published: (2024)

A Reproduction Study: The Kernel PCA Interpretation of Self-Attention Fails Under Scrutiny
by: Sarıtaş, Karahan, et al.
Published: (2025)

Direction-Flipped Influence Audits Reveal Hidden Structure in Moral Choices of LLMs
by: Blandfort, Phil, et al.
Published: (2026)

A Primal-Dual Framework for Transformers and Neural Networks
by: Nguyen, Tan M., et al.
Published: (2024)

Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes
by: Chen, Yingyi, et al.
Published: (2024)

Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis
by: Nagar, Aishik, et al.
Published: (2024)

A Novel Framework for Automated Explain Vision Model Using Vision-Language Models
by: Nguyen, Phu-Vinh, et al.
Published: (2025)

Continual Learning Using a Kernel-Based Method Over Foundation Models
by: Momeni, Saleh, et al.
Published: (2024)

Vision-Language Models Can Self-Improve Reasoning via Reflection
by: Cheng, Kanzhi, et al.
Published: (2024)

Why Stop at Words? Unveiling the Bigger Picture through Line-Level OCR
by: Vempati, Shashank, et al.
Published: (2025)

BiCLIP: Domain Canonicalization via Structured Geometric Transformation
by: Mantini, Pranav, et al.
Published: (2026)

Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models
by: Chen, Zijun, et al.
Published: (2024)

Mitigating Object and Action Hallucinations in Multimodal LLMs via Self-Augmented Contrastive Alignment
by: Chang, Kai-Po, et al.
Published: (2025)

Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start
by: Chen, Kun, et al.
Published: (2025)

Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement
by: Wang, Xiyao, et al.
Published: (2024)

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
by: Yuan, Qianhao, et al.
Published: (2026)

LatentLLM: Attention-Aware Joint Tensor Compression
by: Koike-Akino, Toshiaki, et al.
Published: (2025)

SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
by: Wu, Chang-Hsun, et al.
Published: (2025)

Divide & Bind Your Attention for Improved Generative Semantic Nursing
by: Li, Yumeng, et al.
Published: (2023)

Out-of-Distribution Detection with Attention Head Masking for Multimodal Document Classification
by: Constantinou, Christos, et al.
Published: (2024)

AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
by: Achtibat, Reduan, et al.
Published: (2024)

Voila-A: Aligning Vision-Language Models with User's Gaze Attention
by: Yan, Kun, et al.
Published: (2023)

Federated Document Visual Question Answering: A Pilot Study
by: Nguyen, Khanh, et al.
Published: (2024)

Unveiling the Pitfalls of Knowledge Editing for Large Language Models
by: Li, Zhoubo, et al.
Published: (2023)

OSCaR: Object State Captioning and State Change Representation
by: Nguyen, Nguyen, et al.
Published: (2024)

Self-Supervised Visual Preference Alignment
by: Zhu, Ke, et al.
Published: (2024)

CHASM: Unveiling Covert Advertisements on Chinese Social Media
by: Zheng, Jingyi, et al.
Published: (2026)

Diving into Self-Evolving Training for Multimodal Reasoning
by: Liu, Wei, et al.
Published: (2024)

Contextual Experience Replay for Self-Improvement of Language Agents
by: Liu, Yitao, et al.
Published: (2025)

RewardHarness: Self-Evolving Agentic Post-Training
by: Zhang, Yuxuan, et al.
Published: (2026)

The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition
by: Tan, Yuwen, et al.
Published: (2025)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
by: Liu, Yang, et al.
Published: (2026)

Translution: Unifying Self-attention and Convolution for Adaptive and Relative Modeling
by: Fan, Hehe, et al.
Published: (2025)

Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs
by: Pan, Zhiyu, et al.
Published: (2026)

MHA2MLA-VLM: Enabling DeepSeek's Economical Multi-Head Latent Attention across Vision-Language Models
by: Fan, Xiaoran, et al.
Published: (2026)

Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels
by: Babaiee, Zahra, et al.
Published: (2024)

VisPlay: Self-Evolving Vision-Language Models from Images
by: He, Yicheng, et al.
Published: (2025)

SQ-LLaVA: Self-Questioning for Large Vision-Language Assistant
by: Sun, Guohao, et al.
Published: (2024)