Imagen de Portada

Subcritical Signal Propagation at Initialization in Normalization-Free Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Alekseev, Sergey
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective
por: Wang, Senmiao, et al.
Publicado: (2025)

Geometric Dynamics of Signal Propagation Predict Trainability of Transformers
por: Cowsik, Aditya, et al.
Publicado: (2024)

DyTTP: Trajectory Prediction with Normalization-Free Transformers
por: Zhu, JianLin, et al.
Publicado: (2025)

Stronger Normalization-Free Transformers
por: Chen, Mingzhi, et al.
Publicado: (2025)

Beyond Oversquashing: Understanding Signal Propagation in GNNs Via Observables
por: Nagar, Eden, et al.
Publicado: (2026)

Convolutional Signal Propagation: A Simple Scalable Algorithm for Hypergraphs
por: Procházka, Pavel, et al.
Publicado: (2024)

Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
por: Lee, Hyunwoo, et al.
Publicado: (2025)

Normalize Then Propagate: Efficient Homophilous Regularization for Few-shot Semi-Supervised Node Classification
por: Zhang, Baoming, et al.
Publicado: (2025)

No Free Prune: Information-Theoretic Barriers to Pruning at Initialization
por: Kumar, Tanishq, et al.
Publicado: (2024)

FlashNorm: Fast Normalization for Transformers
por: Graef, Nils, et al.
Publicado: (2024)

Uncertainty Propagation in the Fast Fourier Transform
por: Schmid, Luca, et al.
Publicado: (2025)

ART: Artifact Removal Transformer for Reconstructing Noise-Free Multichannel Electroencephalographic Signals
por: Chuang, Chun-Hsiang, et al.
Publicado: (2024)

Learning in Compact Spaces with Approximately Normalized Transformer
por: Franke, Jörg K. H., et al.
Publicado: (2025)

Conditional Pseudo-Reversible Normalizing Flow for Surrogate Modeling in Quantifying Uncertainty Propagation
por: Yang, Minglei, et al.
Publicado: (2024)

The Free Transformer
por: Fleuret, François
Publicado: (2025)

Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers
por: Saada, Thiziri Nait, et al.
Publicado: (2024)

Observable Propagation: Uncovering Feature Vectors in Transformers
por: Dunefsky, Jacob, et al.
Publicado: (2023)

Graph Propagation Transformer for Graph Representation Learning
por: Chen, Zhe, et al.
Publicado: (2023)

Universal Learning of Stochastic Dynamics for Exact Belief Propagation using Bernstein Normalizing Flows
por: Amorese, Peter, et al.
Publicado: (2025)

UnitNorm: Rethinking Normalization for Transformers in Time Series
por: Huang, Nan, et al.
Publicado: (2024)

Nash Initialization for Recurrent Depth Transformers: Stable Signal Propagation at Initialization Without Layer Normalization
por: Bigeard, Nicolas
Publicado: (2026)

Normalized Matching Transformer
por: Pourhadi, Abtin, et al.
Publicado: (2025)

Real-time Prediction of Urban Sound Propagation with Conditioned Normalizing Flows
por: Eckerle, Achim, et al.
Publicado: (2025)

Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks
por: Zhao, Yequan, et al.
Publicado: (2025)

Differentially Private Clustered Federated Learning with Privacy-Preserving Initialization and Normality-Driven Aggregation
por: Xu, Jie, et al.
Publicado: (2026)

How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences
por: Seleznova, Mariia
Publicado: (2026)

Learning Rate Transfer in Normalized Transformers
por: Shigida, Boris, et al.
Publicado: (2026)

Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks
por: Kinderman, Edan, et al.
Publicado: (2024)

Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing
por: Zhang, Zhongwang, et al.
Publicado: (2024)

The Radio-Frequency Transformer for Signal Separation
por: Lifar, Egor, et al.
Publicado: (2026)

CONTRA: Conformal Prediction Region via Normalizing Flow Transformation
por: Fang, Zhenhan, et al.
Publicado: (2026)

Looped Transformers with Layer Normalization Provably Learn the Power Method
por: Wu, Lyumin, et al.
Publicado: (2026)

Free-form Flows: Make Any Architecture a Normalizing Flow
por: Draxler, Felix, et al.
Publicado: (2023)

Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs
por: Lin, Jiaqi, et al.
Publicado: (2025)

Local to Global: Learning Dynamics and Effect of Initialization for Transformers
por: Makkuva, Ashok Vardhan, et al.
Publicado: (2024)

Transformers Are Born Biased: Structural Inductive Biases at Random Initialization and Their Practical Consequences
por: Li, Siquan, et al.
Publicado: (2026)

SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights
por: Müller, Lorenz K., et al.
Publicado: (2025)

T2I-Diff: fMRI Signal Generation via Time-Frequency Image Transform and Classifier-Free Denoising Diffusion Models
por: Tew, Hwa Hui, et al.
Publicado: (2025)

PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction
por: Jiang, Jiawei, et al.
Publicado: (2023)

Neural Click Models for Recommender Systems
por: Shirokikh, Mikhail, et al.
Publicado: (2024)