Copertina

Quaternion Self-Attention with Shared Scores

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yamauchi, Shogo, Nitta, Tohru, Tamori, Hideaki
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Learning Characteristics of Reverse Quaternion Neural Network
di: Yamauchi, Shogo, et al.
Pubblicazione: (2024)

CSAttention: Centroid-Scoring Attention for Accelerating LLM Inference
di: Song, Chuxu, et al.
Pubblicazione: (2026)

UMoE: Unifying Attention and FFN with Shared Experts
di: Yang, Yuanhang, et al.
Pubblicazione: (2025)

Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
di: Swain, Kabir, et al.
Pubblicazione: (2026)

Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
di: Athiwaratkun, Ben, et al.
Pubblicazione: (2024)

Improving Semi-Supervised Contrastive Learning via Entropy-Weighted Confidence Integration of Anchor-Positive Pairs
di: Nakayama, Shogo, et al.
Pubblicazione: (2026)

Integrating Distribution Matching into Semi-Supervised Contrastive Learning for Labeled and Unlabeled Data
di: Nakayama, Shogo, et al.
Pubblicazione: (2026)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
di: Li, Cheng, et al.
Pubblicazione: (2025)

Robustness of Graph Self-Supervised Learning to Real-World Noise: A Case Study on Text-Driven Biomedical Graphs
di: Kabal, Othmane, et al.
Pubblicazione: (2026)

Technical Report: Small Language Model for Japanese Clinical and Medicine
di: Watanabe, Shogo
Pubblicazione: (2024)

DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs
di: Jin, Haolin, et al.
Pubblicazione: (2025)

Robust Filter Attention: Self-Attention as Precision-Weighted State Estimation
di: Racioppo, Peter
Pubblicazione: (2025)

Modeling Choice via Self-Attention
di: Ko, Joohwan, et al.
Pubblicazione: (2023)

Share Your Attention: Transformer Weight Sharing via Matrix-based Dictionary Learning
di: Zhussip, Magauiya, et al.
Pubblicazione: (2025)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
di: Yan, Fanqi, et al.
Pubblicazione: (2025)

ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation
di: Yue, Angxiao, et al.
Pubblicazione: (2025)

Are Self-Attentions Effective for Time Series Forecasting?
di: Kim, Dongbin, et al.
Pubblicazione: (2024)

Graph Convolutions Enrich the Self-Attention in Transformers!
di: Choi, Jeongwhan, et al.
Pubblicazione: (2023)

Beyond KV Caching: Shared Attention for Efficient LLMs
di: Liao, Bingli, et al.
Pubblicazione: (2024)

CASA: CNN Autoencoder-based Score Attention for Efficient Multivariate Long-term Time-series Forecasting
di: Lee, Minhyuk, et al.
Pubblicazione: (2025)

Data-Free Pruning of Self-Attention Layers in LLMs
di: Saikumar, Dhananjay, et al.
Pubblicazione: (2025)

Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow
di: Delestre, Cyrile, et al.
Pubblicazione: (2024)

CSAI: Conditional Self-Attention Imputation for Healthcare Time-series
di: Qian, Linglong, et al.
Pubblicazione: (2023)

Asymptotically Stable Quaternion-valued Hopfield-structured Neural Network with Periodic Projection-based Supervised Learning Rules
di: Wang, Tianwei, et al.
Pubblicazione: (2025)

From SHAP Scores to Feature Importance Scores
di: Letoffe, Olivier, et al.
Pubblicazione: (2024)

Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys
di: Yang, Xu, et al.
Pubblicazione: (2026)

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain
di: Wi, Hyowon, et al.
Pubblicazione: (2025)

From MLP to NeoMLP: Leveraging Self-Attention for Neural Fields
di: Kofinas, Miltiadis, et al.
Pubblicazione: (2024)

Smart Sampling: Self-Attention and Bootstrapping for Improved Ensembled Q-Learning
di: Khan, Muhammad Junaid, et al.
Pubblicazione: (2024)

Regularizing Attention Scores with Bootstrapping
di: Chung, Neo Christopher, et al.
Pubblicazione: (2026)

Learning to Score
di: Kriger, Yogev, et al.
Pubblicazione: (2025)

Self-Tuning Sparse Attention: Multi-Fidelity Hyperparameter Optimization for Transformer Acceleration
di: Dev, Arundhathi, et al.
Pubblicazione: (2026)

Cross-Temporal Attention Fusion (CTAF) for Multimodal Physiological Signals in Self-Supervised Learning
di: Khorasani, Arian, et al.
Pubblicazione: (2026)

An Explainable Market Integrity Monitoring System with Multi-Source Attention Signals and Transparent Scoring
di: Neela, Sandeep
Pubblicazione: (2026)

A Self-Supervised Transformer for Unusable Shared Bike Detection
di: Huang, Yin, et al.
Pubblicazione: (2025)

Multi-scale Quaternion CNN and BiGRU with Cross Self-attention Feature Fusion for Fault Diagnosis of Bearing
di: Liu, Huanbai, et al.
Pubblicazione: (2024)

ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification
di: Gazeran, Mahsa, et al.
Pubblicazione: (2026)

Mitigating Degree Bias in Graph Representation Learning with Learnable Structural Augmentation and Structural Self-Attention
di: Hoang, Van Thuy, et al.
Pubblicazione: (2025)

Stock Market Price Prediction: A Hybrid LSTM and Sequential Self-Attention based Approach
di: Pardeshi, Karan, et al.
Pubblicazione: (2023)

A Semi-supervised Fake News Detection using Sentiment Encoding and LSTM with Self-Attention
di: Shaeri, Pouya, et al.
Pubblicazione: (2024)