Imagen de Portada

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Pan, Rui, Zhang, Dylan, Zhang, Hanning, Pan, Xingyuan, Xu, Minrui, Zhang, Jipeng, Pi, Renjie, Wang, Xiaoyu, Zhang, Tong
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Effective Bilevel Optimization via Minimax Reformulation
por: Wang, Xiaoyu, et al.
Publicado: (2023)

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
por: Pan, Rui, et al.
Publicado: (2024)

Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise
por: Pan, Rui, et al.
Publicado: (2023)

Second-Order Optimality Conditions for Nonsmooth Constrained Optimization with Applications to Bilevel Programming
por: Liu, Xiang, et al.
Publicado: (2025)

Reducing Contextual Stochastic Bilevel Optimization via Structured Function Approximation
por: Bouscary, Maxime, et al.
Publicado: (2025)

Augmenting Iterative Trajectory for Bilevel Optimization: Methodology, Analysis and Extensions
por: Liu, Risheng, et al.
Publicado: (2023)

AdaGrad under Anisotropic Smoothness
por: Liu, Yuxing, et al.
Publicado: (2024)

Fully First-Order Methods for Decentralized Bilevel Optimization
por: Wang, Xiaoyu, et al.
Publicado: (2024)

On the Nature of Regularity Assumptions in Bilevel Optimization with Constrained Lower-level Problem
por: Jiang, Xiaotian, et al.
Publicado: (2026)

Hessian-Free Distributed Bilevel Optimization via Penalization with Time-Scale Separation
por: Niu, Youcheng, et al.
Publicado: (2024)

A Single-Loop Gradient Algorithm for Pessimistic Bilevel Optimization via Smooth Approximation
por: Cao, Qichao, et al.
Publicado: (2025)

A Doubly Stochastically Perturbed Algorithm for Linearly Constrained Bilevel Optimization
por: Khanduri, Prashant, et al.
Publicado: (2025)

On the Condition Number Dependency in Bilevel Optimization
por: Chen, Lesi, et al.
Publicado: (2025)

Problem-Parameter-Free Decentralized Bilevel Optimization
por: Zhai, Zhiwei, et al.
Publicado: (2025)

Reweighted Quasi Norm Regularized Low-Rank Factorization for Matrix Robust PCA
por: Qin, Zhenzhi, et al.
Publicado: (2024)

SUN-DSBO: A Structured Unified Framework for Nonconvex Decentralized Stochastic Bilevel Optimization
por: Ma, Yaoshuai, et al.
Publicado: (2026)

A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization
por: Chu, Tianshu, et al.
Publicado: (2025)

An Adaptive Algorithm for Bilevel Optimization on Riemannian Manifolds
por: Shi, Xu, et al.
Publicado: (2025)

Functionally Constrained Algorithm Solves Convex Simple Bilevel Problems
por: Zhang, Huaqing, et al.
Publicado: (2024)

Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
por: Dong, Youran, et al.
Publicado: (2025)

Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions
por: Yao, Wei, et al.
Publicado: (2024)

First-Order Methods for Linearly Constrained Bilevel Optimization
por: Kornowski, Guy, et al.
Publicado: (2024)

ASGO: Adaptive Structured Gradient Optimization
por: An, Kang, et al.
Publicado: (2025)

qNBO: quasi-Newton Meets Bilevel Optimization
por: Fang, Sheng, et al.
Publicado: (2025)

Neur2BiLO: Neural Bilevel Optimization
por: Dumouchelle, Justin, et al.
Publicado: (2024)

Faster Gradient Methods for Highly-Smooth Stochastic Bilevel Optimization
por: Chen, Lesi, et al.
Publicado: (2025)

On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis
por: Chen, Lesi, et al.
Publicado: (2023)

Scalable Derivative-Free Optimization Algorithms with Low-Dimensional Subspace Techniques
por: Zhang, Zaikun
Publicado: (2025)

Efficient Computation of the Non-convex Quasi-norm Ball Projection with Iterative Reweighted Approach
por: An, Qi, et al.
Publicado: (2024)

Distributed Stochastic Bilevel Optimization: Improved Complexity and Heterogeneity Analysis
por: Niu, Youcheng, et al.
Publicado: (2023)

Adaptive Algorithms for Nonconvex Bilevel Optimization under PŁ Conditions
por: Shi, Xu, et al.
Publicado: (2025)

Distributed Bilevel Optimization with Communication Compression
por: He, Yutong, et al.
Publicado: (2024)

Optimal Zeroth-Order Bilevel Optimization
por: Aghasi, Alireza, et al.
Publicado: (2025)

Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence
por: Liu, Yuxing, et al.
Publicado: (2025)

Scalable Bi-causal Optimal Transport via KL Relaxation and Policy Gradients
por: Cao, Haoyang, et al.
Publicado: (2026)

FAB: A First-Order AB-based Gradient Algorithm for Distributed Bilevel Optimization over Time-Varying Directed Graphs
por: Ma, Yaoshuai, et al.
Publicado: (2026)

Enhanced Barrier-Smoothing Technique for Bilevel Optimization with Nonsmooth Mappings
por: Xu, Mengwei, et al.
Publicado: (2024)

Near-Optimal Convex Simple Bilevel Optimization with a Bisection Method
por: Wang, Jiulin, et al.
Publicado: (2024)

Bilevel Optimization and Heuristic Algorithms for Integrating Latent Demand into the Design of Large-Scale Transit Systems
por: Guan, Hongzhao, et al.
Publicado: (2022)

A Bilevel Optimization Framework for Imbalanced Data Classification
por: Medlin, Karen, et al.
Publicado: (2024)