Cover Image

Understanding and Minimising Outlier Features in Neural Network Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Bobby, Noci, Lorenzo, Paliotta, Daniele, Schlag, Imanol, Hofmann, Thomas
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Super Consistency of Neural Network Landscapes and Learning Rate Transfer
by: Noci, Lorenzo, et al.
Published: (2024)

Navigating Scaling Laws: Compute Optimality in Adaptive Model Training
by: Anagnostidis, Sotiris, et al.
Published: (2023)

Towards Fully FP8 GEMM LLM Training at Scale
by: Hernández-Cano, Alejandro, et al.
Published: (2025)

Deriving Activation Functions Using Integration
by: Huang, Allen Hao, et al.
Published: (2024)

On the Effect of (Near) Duplicate Subwords in Language Modelling
by: Schäfer, Anton, et al.
Published: (2024)

How Good is a Single Basin?
by: Lion, Kai, et al.
Published: (2024)

Asymptotic Characterisation of Robust Empirical Risk Minimisation Performance in the Presence of Outliers
by: Vilucchio, Matteo, et al.
Published: (2023)

The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments
by: Schäfer, Anton, et al.
Published: (2024)

Simplifying Transformer Blocks
by: He, Bobby, et al.
Published: (2023)

Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy
by: Singh, Sidak Pal, et al.
Published: (2024)

Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers
by: Belloni, Annalisa, et al.
Published: (2026)

The Importance of Being Lazy: Scaling Limits of Continual Learning
by: Graldi, Jacopo, et al.
Published: (2025)

Thinking into the Future: Latent Lookahead Training for Transformers
by: Noci, Lorenzo, et al.
Published: (2026)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
by: Limozin, Alexis, et al.
Published: (2026)

Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
by: Anagnostidis, Sotiris, et al.
Published: (2023)

The Mamba in the Llama: Distilling and Accelerating Hybrid Models
by: Wang, Junxiong, et al.
Published: (2024)

Leveraging the true depth of LLMs
by: González, Ramón Calvo, et al.
Published: (2025)

Understanding Pooling in Graph Neural Networks
by: Grattarola, Daniele, et al.
Published: (2021)

Minimising Willmore Energy via Neural Flow
by: Hirst, Edward, et al.
Published: (2026)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
by: Dong, Yihe, et al.
Published: (2025)

M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
by: Wang, Junxiong, et al.
Published: (2025)

Toward Understanding the Disagreement Problem in Neural Network Feature Attribution
by: Koenen, Niklas, et al.
Published: (2024)

Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs
by: Fan, Dongyang, et al.
Published: (2025)

Recurrent Distance Filtering for Graph Representation Learning
by: Ding, Yuhui, et al.
Published: (2023)

Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features
by: Wu, Yulun, et al.
Published: (2025)

Minimising changes to audit when updating decision trees
by: Simmons, Anj, et al.
Published: (2024)

Global Outlier Detection in a Federated Learning Setting with Isolation Forest
by: Malpetti, Daniele, et al.
Published: (2024)

On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders
by: Simon, Elana, et al.
Published: (2026)

Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
by: Lanzillotta, Giulia, et al.
Published: (2025)

Training-free Graph Neural Networks and the Power of Labels as Features
by: Sato, Ryoma
Published: (2024)

Training Neural Networks at Any Scale
by: Pethick, Thomas, et al.
Published: (2025)

Graph Neural Networks for Surgical Scene Segmentation
by: Li, Yihan, et al.
Published: (2025)

PreNeT: Leveraging Computational Features to Predict Deep Neural Network Training Time
by: Pourali, Alireza, et al.
Published: (2024)

Hierarchical Representation Learning in Graph Neural Networks with Node Decimation Pooling
by: Bianchi, Filippo Maria, et al.
Published: (2019)

Understanding the Benefits of SimCLR Pre-Training in Two-Layer Convolutional Neural Networks
by: Zhang, Han, et al.
Published: (2024)

Three Revisits to Node-Level Graph Anomaly Detection: Outliers, Message Passing and Hyperbolic Neural Networks
by: Gu, Jing, et al.
Published: (2024)

Towards the Training of Deeper Predictive Coding Neural Networks
by: Qi, Chang, et al.
Published: (2025)

Diffusion Alignment Beyond KL: Variance Minimisation as Effective Policy Optimiser
by: Ou, Zijing, et al.
Published: (2026)

Efficient Generation of Hidden Outliers for Improved Outlier Detection
by: Cribeiro-Ramallo, Jose, et al.
Published: (2024)

On the Neural Feature Ansatz for Deep Neural Networks
by: Tansley, Edward, et al.
Published: (2025)