Copertina

Understanding and Minimising Outlier Features in Neural Network Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	He, Bobby, Noci, Lorenzo, Paliotta, Daniele, Schlag, Imanol, Hofmann, Thomas
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Super Consistency of Neural Network Landscapes and Learning Rate Transfer
di: Noci, Lorenzo, et al.
Pubblicazione: (2024)

Navigating Scaling Laws: Compute Optimality in Adaptive Model Training
di: Anagnostidis, Sotiris, et al.
Pubblicazione: (2023)

Towards Fully FP8 GEMM LLM Training at Scale
di: Hernández-Cano, Alejandro, et al.
Pubblicazione: (2025)

Deriving Activation Functions Using Integration
di: Huang, Allen Hao, et al.
Pubblicazione: (2024)

On the Effect of (Near) Duplicate Subwords in Language Modelling
di: Schäfer, Anton, et al.
Pubblicazione: (2024)

How Good is a Single Basin?
di: Lion, Kai, et al.
Pubblicazione: (2024)

Asymptotic Characterisation of Robust Empirical Risk Minimisation Performance in the Presence of Outliers
di: Vilucchio, Matteo, et al.
Pubblicazione: (2023)

The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments
di: Schäfer, Anton, et al.
Pubblicazione: (2024)

Simplifying Transformer Blocks
di: He, Bobby, et al.
Pubblicazione: (2023)

Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy
di: Singh, Sidak Pal, et al.
Pubblicazione: (2024)

Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers
di: Belloni, Annalisa, et al.
Pubblicazione: (2026)

The Importance of Being Lazy: Scaling Limits of Continual Learning
di: Graldi, Jacopo, et al.
Pubblicazione: (2025)

Thinking into the Future: Latent Lookahead Training for Transformers
di: Noci, Lorenzo, et al.
Pubblicazione: (2026)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
di: Limozin, Alexis, et al.
Pubblicazione: (2026)

Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
di: Anagnostidis, Sotiris, et al.
Pubblicazione: (2023)

The Mamba in the Llama: Distilling and Accelerating Hybrid Models
di: Wang, Junxiong, et al.
Pubblicazione: (2024)

Leveraging the true depth of LLMs
di: González, Ramón Calvo, et al.
Pubblicazione: (2025)

Understanding Pooling in Graph Neural Networks
di: Grattarola, Daniele, et al.
Pubblicazione: (2021)

Minimising Willmore Energy via Neural Flow
di: Hirst, Edward, et al.
Pubblicazione: (2026)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
di: Dong, Yihe, et al.
Pubblicazione: (2025)

M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
di: Wang, Junxiong, et al.
Pubblicazione: (2025)

Toward Understanding the Disagreement Problem in Neural Network Feature Attribution
di: Koenen, Niklas, et al.
Pubblicazione: (2024)

Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs
di: Fan, Dongyang, et al.
Pubblicazione: (2025)

Recurrent Distance Filtering for Graph Representation Learning
di: Ding, Yuhui, et al.
Pubblicazione: (2023)

Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features
di: Wu, Yulun, et al.
Pubblicazione: (2025)

Minimising changes to audit when updating decision trees
di: Simmons, Anj, et al.
Pubblicazione: (2024)

Global Outlier Detection in a Federated Learning Setting with Isolation Forest
di: Malpetti, Daniele, et al.
Pubblicazione: (2024)

On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders
di: Simon, Elana, et al.
Pubblicazione: (2026)

Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
di: Lanzillotta, Giulia, et al.
Pubblicazione: (2025)

Training-free Graph Neural Networks and the Power of Labels as Features
di: Sato, Ryoma
Pubblicazione: (2024)

Training Neural Networks at Any Scale
di: Pethick, Thomas, et al.
Pubblicazione: (2025)

Graph Neural Networks for Surgical Scene Segmentation
di: Li, Yihan, et al.
Pubblicazione: (2025)

PreNeT: Leveraging Computational Features to Predict Deep Neural Network Training Time
di: Pourali, Alireza, et al.
Pubblicazione: (2024)

Hierarchical Representation Learning in Graph Neural Networks with Node Decimation Pooling
di: Bianchi, Filippo Maria, et al.
Pubblicazione: (2019)

Understanding the Benefits of SimCLR Pre-Training in Two-Layer Convolutional Neural Networks
di: Zhang, Han, et al.
Pubblicazione: (2024)

Three Revisits to Node-Level Graph Anomaly Detection: Outliers, Message Passing and Hyperbolic Neural Networks
di: Gu, Jing, et al.
Pubblicazione: (2024)

Towards the Training of Deeper Predictive Coding Neural Networks
di: Qi, Chang, et al.
Pubblicazione: (2025)

Diffusion Alignment Beyond KL: Variance Minimisation as Effective Policy Optimiser
di: Ou, Zijing, et al.
Pubblicazione: (2026)

Efficient Generation of Hidden Outliers for Improved Outlier Detection
di: Cribeiro-Ramallo, Jose, et al.
Pubblicazione: (2024)

On the Neural Feature Ansatz for Deep Neural Networks
di: Tansley, Edward, et al.
Pubblicazione: (2025)