Imagen de Portada

Aligning Distributionally Robust Optimization with Practical Deep Learning Needs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Feoktistov, Dmitrii, Ignashin, Igor, Veprikov, Andrey, Borovko, Nikita, Bogdanov, Alexander, Chezhegov, Savelii, Beznosikov, Aleksandr
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling
por: Feoktistov, Dmitrii, et al.
Publicado: (2026)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
por: Chezhegov, Savelii, et al.
Publicado: (2025)

Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling
por: Medyakov, Daniil, et al.
Publicado: (2025)

Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
por: Bylinkin, Dmitry, et al.
Publicado: (2025)

Local SGD for Near-Quadratic Problems: Improving Convergence under Unconstrained Noise Conditions
por: Sadchikov, Andrey, et al.
Publicado: (2024)

WeightLoRA: Keep Only Necessary Adapters
por: Veprikov, Andrey, et al.
Publicado: (2025)

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
por: Veprikov, Andrey, et al.
Publicado: (2025)

Incorporating Preconditioning into Accelerated Approaches: Theoretical Guarantees and Practical Improvement
por: Trifonov, Stepan, et al.
Publicado: (2025)

Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
por: Riabinin, Artem, et al.
Publicado: (2026)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
por: Chezhegov, Savelii, et al.
Publicado: (2024)

Benchmarking Optimizers for MLPs in Tabular Deep Learning
por: Gorishniy, Yury, et al.
Publicado: (2026)

LionMuon: Alternating Spectral and Sign Descent for Efficient Training
por: Bolatov, Arman, et al.
Publicado: (2026)

Zero-Order Optimization for LLM Fine-Tuning via Learnable Direction Sampling
por: Parfenov, Valery, et al.
Publicado: (2026)

New Aspects of Black Box Conditional Gradient: Variance Reduction and One Point Feedback
por: Veprikov, Andrey, et al.
Publicado: (2024)

Local Methods with Adaptivity via Scaling
por: Chezhegov, Savelii, et al.
Publicado: (2024)

Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order
por: Petrov, Egor, et al.
Publicado: (2025)

Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update
por: Almansoori, Abdulla Jasem, et al.
Publicado: (2025)

Beyond SGD, Without SVD: Proximal Subspace Iteration LoRA with Diagonal Fractional K-FAC
por: Almansoori, Abdulla Jasem, et al.
Publicado: (2026)

Accelerated Stochastic Gradient Method with Applications to Consensus Problem in Markov-Varying Networks
por: Solodkin, Vladimir, et al.
Publicado: (2024)

Optimal Data Splitting in Distributed Optimization for Machine Learning
por: Medyakov, Daniil, et al.
Publicado: (2024)

Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks
por: Kovalev, Dmitry, et al.
Publicado: (2024)

Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
por: Beznosikov, Aleksandr, et al.
Publicado: (2020)

Decentralized Finite-Sum Optimization over Time-Varying Networks
por: Metelev, Dmitry, et al.
Publicado: (2024)

Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similarity
por: Bylinkin, Dmitry, et al.
Publicado: (2024)

Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features
por: Beznosikov, Aleksandr, et al.
Publicado: (2023)

A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems
por: Veprikov, Andrey, et al.
Publicado: (2024)

Hierarchical Mixture-of-Experts with Two-Stage Optimization
por: Molodtsov, Gleb, et al.
Publicado: (2026)

Ito Diffusion Approximation of Universal Ito Chains for Sampling, Optimization and Boosting
por: Ustimenko, Aleksei, et al.
Publicado: (2023)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
por: Khah, Saleh Vatan, et al.
Publicado: (2025)

Accelerated Stochastic ExtraGradient: Mixing Hessian and Gradient Similarity to Reduce Communication in Distributed and Federated Learning
por: Bylinkin, Dmitry, et al.
Publicado: (2024)

Methods for Optimization Problems with Markovian Stochasticity and Non-Euclidean Geometry
por: Solodkin, Vladimir, et al.
Publicado: (2024)

DyKAF: Dynamical Kronecker Approximation of the Fisher Information Matrix for Gradient Preconditioning
por: Yudin, Nikolay, et al.
Publicado: (2025)

On Biased Compression for Distributed Learning
por: Beznosikov, Aleksandr, et al.
Publicado: (2020)

Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities
por: Beznosikov, Aleksandr, et al.
Publicado: (2023)

Random-reshuffled SARAH does not need a full gradient computations
por: Beznosikov, Aleksandr, et al.
Publicado: (2021)

FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Training
por: Zmushko, Philip, et al.
Publicado: (2024)

Curvature-Aligned Probing for Local Loss-Landscape Stabilization
por: Kiselev, Nikita, et al.
Publicado: (2026)

Gradient-Free Approaches is a Key to an Efficient Interaction with Markovian Stochasticity
por: Prokhorov, Boris, et al.
Publicado: (2026)

HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
por: Zagitov, Artur, et al.
Publicado: (2026)

Just a Simple Transformation is Enough for Data Protection in Vertical Federated Learning
por: Semenov, Andrei, et al.
Publicado: (2024)