Cover Image

Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Singh, Sidak Pal, He, Bobby, Hofmann, Thomas, Schölkopf, Bernhard
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
by: Bozic, Vukasin, et al.
Published: (2023)

Accelerating Neural Network Training Along Sharp and Flat Directions
by: Zakarin, Daniyar, et al.
Published: (2025)

Avoiding spurious sharpness minimization broadens applicability of SAM
by: Singh, Sidak Pal, et al.
Published: (2025)

Some Fundamental Aspects about Lipschitz Continuity of Neural Networks
by: Khromov, Grigory, et al.
Published: (2023)

Landscaping Linear Mode Connectivity
by: Singh, Sidak Pal, et al.
Published: (2024)

Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks
by: Zhao, Jim, et al.
Published: (2024)

Identifying Intervenable and Interpretable Features via Orthogonality Regularization
by: Miller, Moritz, et al.
Published: (2026)

On the Emergence and Test-Time Use of Structural Information in Large Language Models
by: Chen, Michelle Chao, et al.
Published: (2026)

Generalized Interpolating Discrete Diffusion
by: von Rütte, Dimitri, et al.
Published: (2025)

Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
by: Song, Yifan, et al.
Published: (2024)

Improving Large Language Model Safety with Contrastive Representation Learning
by: Simko, Samuel, et al.
Published: (2025)

Counterfactual reasoning: an analysis of in-context emergence
by: Miller, Moritz, et al.
Published: (2025)

Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
by: Xiao, Tim Z., et al.
Published: (2025)

Verbalized Machine Learning: Revisiting Machine Learning with Language Models
by: Xiao, Tim Z., et al.
Published: (2024)

MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs
by: Opedal, Andreas, et al.
Published: (2024)

Hyperbolic Busemann Neural Networks
by: Chen, Ziheng, et al.
Published: (2026)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
by: Huang, Yangyi, et al.
Published: (2026)

Orthogonal Finetuning Made Scalable
by: Qiu, Zeju, et al.
Published: (2025)

Towards Meta-Pruning via Optimal Transport
by: Theus, Alexander, et al.
Published: (2024)

Limits of Transformer Language Models on Learning to Compose Algorithms
by: Thomm, Jonathan, et al.
Published: (2024)

What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis
by: Ormaniec, Weronika, et al.
Published: (2024)

Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
by: Guo, Siyuan, et al.
Published: (2024)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
by: Qiu, Zeju, et al.
Published: (2025)

Learning to Reason Efficiently with A* Post-Training
by: Opedal, Andreas, et al.
Published: (2026)

Local vs Global continual learning
by: Lanzillotta, Giulia, et al.
Published: (2024)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
by: Song, Jiwon, et al.
Published: (2024)

Understanding Reference Policies in Direct Preference Optimization
by: Liu, Yixin, et al.
Published: (2024)

Accelerating Direct Preference Optimization with Prefix Sharing
by: Wang, Franklin, et al.
Published: (2024)

Efficient Second-Order Neural Network Optimization via Adaptive Trust Region Methods
by: Vo, James
Published: (2024)

GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression
by: Liu, Kainan, et al.
Published: (2024)

RAD: Redundancy-Aware Distillation for Hybrid Models via Self-Speculative Decoding
by: Hoshino, Yuichiro, et al.
Published: (2025)

GNN: Graph Neural Network and Large Language Model for Data Discovery
by: Hoang, Thomas
Published: (2024)

RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping
by: Chen, Yuzong, et al.
Published: (2025)

Direct Multi-Turn Preference Optimization for Language Agents
by: Shi, Wentao, et al.
Published: (2024)

The Crucial Role of Samplers in Online Direct Preference Optimization
by: Shi, Ruizhe, et al.
Published: (2024)

Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)

Analyzing the Role of Semantic Representations in the Era of Large Language Models
by: Jin, Zhijing, et al.
Published: (2024)

Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?
by: Opedal, Andreas, et al.
Published: (2024)

Can Large Language Models Infer Causation from Correlation?
by: Jin, Zhijing, et al.
Published: (2023)