Imagen de Portada

Attention-Only Transformers via Unrolled Subspace Denoising

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Peng, Lu, Yifu, Yu, Yaodong, Pai, Druv, Qu, Qing, Ma, Yi
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction
por: Wu, Ziyang, et al.
Publicado: (2024)

Masked Completion via Structured Diffusion with White-Box Transformers
por: Pai, Druv, et al.
Publicado: (2024)

A Global Geometric Analysis of Maximal Coding Rate Reduction
por: Wang, Peng, et al.
Publicado: (2024)

On the Edge of Memorization in Diffusion Models
por: Buchanan, Sam, et al.
Publicado: (2025)

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
por: Yu, Yaodong, et al.
Publicado: (2023)

Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing
por: Chen, Siyi, et al.
Publicado: (2024)

Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs
por: Guo, Tianyu, et al.
Publicado: (2024)

Diffusion Models Learn Low-Dimensional Distributions via Subspace Clustering
por: Wang, Peng, et al.
Publicado: (2024)

Lightweight Transformer for EEG Classification via Balanced Signed Graph Algorithm Unrolling
por: Yao, Junyi, et al.
Publicado: (2025)

Multi-Omics Analysis for Cancer Subtype Inference via Unrolling Graph Smoothness Priors
por: Lu, Jielong, et al.
Publicado: (2025)

A Constrained Optimization Perspective of Unrolled Transformers
por: Porras-Valenzuela, Javier, et al.
Publicado: (2026)

Contextual Text Denoising with Masked Language Models
por: Sun, Yifu, et al.
Publicado: (2019)

Scaling White-Box Transformers for Vision
por: Yang, Jinrui, et al.
Publicado: (2024)

Attention-Refined Unrolling for Sparse Sequential micro-Doppler Reconstruction
por: Mazzieri, Riccardo, et al.
Publicado: (2023)

Cross-Validated Cross-Channel Self-Attention and Denoising for Automatic Modulation Classification
por: Suman, Prakash, et al.
Publicado: (2026)

Transformers as Unrolled Inference in Probabilistic Laplacian Eigenmaps: An Interpretation and Potential Improvements
por: Ravuri, Aditya, et al.
Publicado: (2025)

Training Data Attribution via Approximate Unrolled Differentiation
por: Bae, Juhan, et al.
Publicado: (2024)

Lightweight and Interpretable Transformer via Mixed Graph Algorithm Unrolling for Traffic Forecast
por: Qi, Ji, et al.
Publicado: (2025)

The Emergence of Reproducibility and Generalizability in Diffusion Models
por: Zhang, Huijie, et al.
Publicado: (2023)

Learning Expressive Random Feature Models via Parametrized Activations
por: Ma, Zailin, et al.
Publicado: (2024)

Precipitation Nowcasting Using Diffusion Transformer with Causal Attention
por: Li, ChaoRong, et al.
Publicado: (2024)

Understanding the Curse of Unrolling
por: Mehmood, Sheheryar, et al.
Publicado: (2026)

Continual Learning with Query-Only Attention
por: Bekal, Gautham, et al.
Publicado: (2025)

Unrolled Neural Networks for Constrained Optimization
por: Hadou, Samar, et al.
Publicado: (2026)

An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling
por: Yang, Linxin, et al.
Publicado: (2024)

Efficient Resource-Constrained Training of Transformers via Subspace Optimization
por: Nguyen, Le-Trung, et al.
Publicado: (2025)

Independent and Decentralized Learning in Markov Potential Games
por: Maheshwari, Chinmay, et al.
Publicado: (2022)

Stochastic Unrolled Federated Learning
por: Hadou, Samar, et al.
Publicado: (2023)

EEGDnet: Fusing Non-Local and Local Self-Similarity for 1-D EEG Signal Denoising with 2-D Transformer
por: Yi, Peng, et al.
Publicado: (2021)

Interpretable Lightweight Transformer via Unrolling of Learned Graph Smoothness Priors
por: Do, Tam Thuc, et al.
Publicado: (2024)

Unrolled Graph Neural Networks for Constrained Optimization
por: Hadou, Samar, et al.
Publicado: (2025)

DS2TA: Denoising Spiking Transformer with Attenuated Spatiotemporal Attention
por: Xu, Boxun, et al.
Publicado: (2024)

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers
por: Adhikari, Rabin
Publicado: (2025)

Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
por: Huang, Yancheng, et al.
Publicado: (2026)

Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective
por: Kwon, Soo Min, et al.
Publicado: (2025)

PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
por: Li, Bingheng, et al.
Publicado: (2024)

Shallow Diffuse: Robust and Invisible Watermarking through Low-Dimensional Subspaces in Diffusion Models
por: Li, Wenda, et al.
Publicado: (2024)

Robust Stochastically-Descending Unrolled Networks
por: Hadou, Samar, et al.
Publicado: (2023)

Recurrent Transformer U-Net Surrogate for Flow Modeling and Data Assimilation in Subsurface Formations with Faults
por: Han, Yifu, et al.
Publicado: (2025)

Joint Data Inpainting and Graph Learning via Unrolled Neural Networks
por: Batreddy, Subbareddy, et al.
Publicado: (2024)