Imagen de Portada

Muon in Vision Transformers: Optimizer-Recipe Interactions and Gradient Spectra

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Southworth, Ben S., Jiang, Shuai, McBride, Daniel, Cyr, Eric C., Thomas, Stephen
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computer Vision and Pattern Recognition
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

HoneyBee: Data Recipes for Vision-Language Reasoners
por: Bansal, Hritik, et al.
Publicado: (2025)

On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime
por: Jiang, Shuai, et al.
Publicado: (2026)

Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training
por: Southworth, Ben S., et al.
Publicado: (2026)

On Vision Transformers for Classification Tasks in Side-Scan Sonar Imagery
por: Sheffield, BW, et al.
Publicado: (2024)

RecipeGen: A Benchmark for Real-World Recipe Image Generation
por: Zhang, Ruoxuan, et al.
Publicado: (2025)

SelaFD:Seamless Adaptation of Vision Transformer Fine-tuning for Radar-based Human Activity Recognition
por: Wang, Yijun, et al.
Publicado: (2025)

RecipeGen: A Step-Aligned Multimodal Benchmark for Real-World Recipe Generation
por: Zhang, Ruoxuan, et al.
Publicado: (2025)

PRUE: A Practical Recipe for Field Boundary Segmentation at Scale
por: Muhawenayo, Gedeon, et al.
Publicado: (2026)

Tarsier: Recipes for Training and Evaluating Large Video Description Models
por: Wang, Jiawei, et al.
Publicado: (2024)

Deep Image-to-Recipe Translation
por: Ma, Jiangqin, et al.
Publicado: (2024)

HEAL-SWIN: A Vision Transformer On The Sphere
por: Carlsson, Oscar, et al.
Publicado: (2023)

LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributions
por: Mehri, Faridoun, et al.
Publicado: (2024)

Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization
por: Jiang, Jiarui, et al.
Publicado: (2024)

Efficient Training of Diffusion Mixture-of-Experts Models: A Practical Recipe
por: Liu, Yahui, et al.
Publicado: (2025)

Mixture-of-Experts Models in Vision: Routing, Optimization, and Generalization
por: Rokah, Adam, et al.
Publicado: (2026)

Native Segmentation Vision Transformers
por: Brasó, Guillem, et al.
Publicado: (2025)

A Deep Learning Approach to Estimate Canopy Height and Uncertainty by Integrating Seasonal Optical, SAR and Limited GEDI LiDAR Data over Northern Forests
por: Castro, Jose B., et al.
Publicado: (2024)

A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning
por: Almuzairee, Abdulaziz, et al.
Publicado: (2024)

Iwin Transformer: Hierarchical Vision Transformer using Interleaved Windows
por: Huo, Simin, et al.
Publicado: (2025)

Feedback Alignment Meets Low-Rank Manifolds: A Structured Recipe for Local Learning
por: Roy, Arani, et al.
Publicado: (2025)

Slicing Vision Transformer for Flexible Inference
por: Zhang, Yitian, et al.
Publicado: (2024)

RAViT: Resolution-Adaptive Vision Transformer
por: Guidez, Martial, et al.
Publicado: (2026)

Rotary Position Embedding for Vision Transformer
por: Heo, Byeongho, et al.
Publicado: (2024)

Decorrelation Speeds Up Vision Transformers
por: Carrigg, Kieran, et al.
Publicado: (2025)

Hybrid Convolution and Vision Transformer NAS Search Space for TinyML Image Classification
por: Djajapermana, Mikhael, et al.
Publicado: (2025)

ZENITH: Automated Gradient Norm Informed Stochastic Optimization
por: Saha, Dhrubo
Publicado: (2026)

Retrieval Augmented Recipe Generation
por: Liu, Guoshan, et al.
Publicado: (2024)

CLIP-UP: A Simple and Efficient Mixture-of-Experts CLIP Training Recipe with Sparse Upcycling
por: Wang, Xinze, et al.
Publicado: (2025)

Block-Recurrent Dynamics in Vision Transformers
por: Jacobs, Mozes, et al.
Publicado: (2025)

A Comparative Survey of Vision Transformers for Feature Extraction in Texture Analysis
por: Scabini, Leonardo, et al.
Publicado: (2024)

SPoT: Subpixel Placement of Tokens in Vision Transformers
por: Hjelkrem-Tan, Martine, et al.
Publicado: (2025)

Instance-Aware Group Quantization for Vision Transformers
por: Moon, Jaehyeon, et al.
Publicado: (2024)

Vision Transformer-based Adversarial Domain Adaptation
por: Li, Yahan, et al.
Publicado: (2024)

Elastic Attention Cores for Scalable Vision Transformers
por: Song, Alan Z., et al.
Publicado: (2026)

Compact Vision Transformer by Reduction of Kernel Complexity
por: Wang, Yancheng, et al.
Publicado: (2025)

Attention Transfer Is Not Universally Effective for Vision Transformers
por: Qin, Huaiyuan, et al.
Publicado: (2026)

Split Adaptation for Pre-trained Vision Transformers
por: Wang, Lixu, et al.
Publicado: (2025)

DASViT: Differentiable Architecture Search for Vision Transformer
por: Wu, Pengjin, et al.
Publicado: (2025)

Stable Vision Concept Transformers for Medical Diagnosis
por: Hu, Lijie, et al.
Publicado: (2025)

Self-Supervised Vision Transformers for Writer Retrieval
por: Raven, Tim, et al.
Publicado: (2024)