Cover Image

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Jing, Cai, Kaitong, Fan, Yijia, Yang, Yufeng, Wang, Keze
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Causal Invariance and Counterfactual Learning Driven Cooperative Game for Multi-Label Classification
by: Fan, Yijia, et al.
Published: (2025)

MAT-Agent: Adaptive Multi-Agent Training Optimization
by: Zhang, Jusheng, et al.
Published: (2025)

CF-VLM:CounterFactual Vision-Language Fine-tuning
by: Zhang, Jusheng, et al.
Published: (2025)

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
by: Liu, Ningyuan, et al.
Published: (2025)

Kolmogorov-Arnold Fourier Networks
by: Zhang, Jusheng, et al.
Published: (2025)

Guardian: Decoupling Exploration from Safety in Reinforcement Learning
by: Cai, Kaitong, et al.
Published: (2025)

Learning Dynamics of VLM Finetuning
by: Zhang, Jusheng, et al.
Published: (2025)

Spectral Gating Networks
by: Zhang, Jusheng, et al.
Published: (2026)

Why Keep Your Doubts to Yourself? Trading Visual Uncertainties in Multi-Agent Bandit Systems
by: Zhang, Jusheng, et al.
Published: (2026)

RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability
by: Cai, Kaitong, et al.
Published: (2025)

A Scalable Curiosity-Driven Game-Theoretic Framework for Long-Tail Multi-Label Learning in Data Mining
by: Yang, Jing, et al.
Published: (2026)

Cost-Effective Communication: An Auction-based Method for Language Agent Interaction
by: Fan, Yijia, et al.
Published: (2025)

PTTA: A Pure Text-to-Animation Framework for High-Quality Creation
by: Chen, Ruiqi, et al.
Published: (2025)

OSC: Cognitive Orchestration through Dynamic Knowledge Alignment in Multi-Agent LLM Collaboration
by: Zhang, Jusheng, et al.
Published: (2025)

Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization
by: Shen, Qianli, et al.
Published: (2024)

Thinking Forward and Backward: Effective Backward Planning with Large Language Models
by: Ren, Allen Z., et al.
Published: (2024)

Approximated Orthogonal Projection Unit: Stabilizing Regression Network Training Using Natural Gradient
by: Wang, Shaoqi, et al.
Published: (2024)

Using Forwards-Backwards Models to Approximate MDP Homomorphisms
by: Mavor-Parker, Augustine N., et al.
Published: (2022)

Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
by: Katz, Shahar, et al.
Published: (2024)

Scaling and Transferability of Annealing Strategies in Large Language Model Training
by: Wang, Siqi, et al.
Published: (2025)

Agent-GSPO: Communication-Efficient Multi-Agent Systems via Group Sequence Policy Optimization
by: Fan, Yijia, et al.
Published: (2025)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

ORACLE: Optimizing Reasoning Abilities of Large Language Models via Constraint-Led Synthetic Data Elicitation
by: Yang, Zhuojie, et al.
Published: (2026)

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
by: Zhang, Songming, et al.
Published: (2026)

Rational ANOVA Networks
by: Zhang, Jusheng, et al.
Published: (2026)

Training Large Language Models to Reason via EM Policy Gradient
by: Xu, Tianbing
Published: (2025)

GVPO: Group Variance Policy Optimization for Large Language Model Post-Training
by: Zhang, Kaichen, et al.
Published: (2025)

Entropy-Gated Selective Policy Optimization:Token-Level Gradient Allocation for Hybrid Training of Large Language Models
by: Hu, Yuelin, et al.
Published: (2026)

Can Large Language Models Reason and Optimize Under Constraints?
by: Bernier, Fabien, et al.
Published: (2026)

Memory Determines Learning Direction: A Theory of Gradient-Based Optimization in State Space Models
by: Guan, JingChuan, et al.
Published: (2025)

Learning to Solve Combinatorial Optimization under Positive Linear Constraints via Non-Autoregressive Neural Networks
by: Wang, Runzhong, et al.
Published: (2024)

HybridToken-VLM: Hybrid Token Compression for Vision-Language Models
by: Zhang, Jusheng, et al.
Published: (2025)

Large Language Models as Discounted Bayesian Filters
by: Zhang, Jensen, et al.
Published: (2025)

Forward-Backward Reasoning in Large Language Models for Mathematical Verification
by: Jiang, Weisen, et al.
Published: (2023)

GWT: Scalable Optimizer State Compression for Large Language Model Training
by: Wen, Ziqing, et al.
Published: (2025)

LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
by: Lee, Jung Hyun, et al.
Published: (2024)

PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
by: Xiao, He, et al.
Published: (2025)

Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints
by: Markovic-Voronov, Jelena, et al.
Published: (2026)

Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey
by: Gu, Yang, et al.
Published: (2024)

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation
by: Yang, Yuchen, et al.
Published: (2024)