Imagen de Portada

Efficient Backpropagation with Variance-Controlled Adaptive Sampling

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Ziteng, Chen, Jianfei, Zhu, Jun
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
por: Wang, Ziteng, et al.
Publicado: (2024)

S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training
por: Hu, Yuezhou, et al.
Publicado: (2024)

Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variance
por: Zeng, Dun, et al.
Publicado: (2023)

Efficient Deep Learning with Decorrelated Backpropagation
por: Dalm, Sander, et al.
Publicado: (2024)

LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models
por: Zhu, Fengqi, et al.
Publicado: (2025)

Efficient Hyperparameter Tuning via Trajectory Invariance Principle
por: Li, Bingrui, et al.
Publicado: (2025)

SparseDM: Toward Sparse Efficient Diffusion Models
por: Wang, Kafeng, et al.
Publicado: (2024)

Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency
por: Zhang, Yibo Jacky, et al.
Publicado: (2026)

Exploring Variance Reduction in Importance Sampling for Efficient DNN Training
por: Kutsuna, Takuro
Publicado: (2025)

Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization
por: Xi, Haocheng, et al.
Publicado: (2024)

Optimal Control Theoretic Neural Optimizer: From Backpropagation to Dynamic Programming
por: Liu, Guan-Horng, et al.
Publicado: (2025)

Unbiased Approximate Vector-Jacobian Products for Efficient Backpropagation
por: Bakong, Killian, et al.
Publicado: (2026)

C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
por: Luo, Tianjiao, et al.
Publicado: (2024)

Backpropagation Neural Tree
por: Ojha, Varun, et al.
Publicado: (2022)

The Cost of Avoiding Backpropagation
por: Panchal, Kunjal, et al.
Publicado: (2025)

Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs
por: Zheng, Kaiwen, et al.
Publicado: (2023)

SAL: Selective Adaptive Learning for Backpropagation-Free Training with Sparsification
por: Liu, Fanping, et al.
Publicado: (2026)

Dynamic Spectral Backpropagation for Efficient Neural Network Training
por: Muthuraman, Mannmohan
Publicado: (2025)

Efficient and Unbiased Sampling from Boltzmann Distributions via Variance-Tuned Diffusion Models
por: Zhang, Fengzhe, et al.
Publicado: (2025)

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
por: Lu, Cheng, et al.
Publicado: (2022)

Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation
por: Gong, Qingchun, et al.
Publicado: (2025)

Towards Scalable Backpropagation-Free Gradient Estimation
por: Wang, Daniel, et al.
Publicado: (2025)

Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation
por: Kang, Qiyu, et al.
Publicado: (2025)

Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices
por: Song, Congzheng, et al.
Publicado: (2025)

Variance-Adaptive Muon: Accelerating LLM Pretraining with NSR-Modulated and Variance-Scaled Momentum
por: Li, Jingru, et al.
Publicado: (2026)

Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning
por: Park, Juneyoung, et al.
Publicado: (2026)

Stochastic Layer-wise Learning: Scalable and Efficient Alternative to Backpropagation
por: Yin, Bojian, et al.
Publicado: (2025)

Practical Boolean Backpropagation
por: Golbert, Simon
Publicado: (2025)

CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
por: Huang, Weiyu, et al.
Publicado: (2025)

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation
por: Yang, Yuchen, et al.
Publicado: (2024)

Accelerating Transformer Pre-training with 2:4 Sparsity
por: Hu, Yuezhou, et al.
Publicado: (2024)

Diffusion Bridge Implicit Models
por: Zheng, Kaiwen, et al.
Publicado: (2024)

Accurate INT8 Training Through Dynamic Block-Level Fallback
por: Zhang, Pengle, et al.
Publicado: (2025)

Sampling with Adaptive Variance for Multimodal Distributions
por: Engquist, Björn, et al.
Publicado: (2024)

Aligning Diffusion Behaviors with Q-functions for Efficient Continuous Control
por: Chen, Huayu, et al.
Publicado: (2024)

Efficient and Adaptive Posterior Sampling Algorithms for Bandits
por: Hu, Bingshan, et al.
Publicado: (2024)

Succeed or Learn Slowly: Sample Efficient Off-Policy Reinforcement Learning for Mobile App Control
por: Papoudakis, Georgios, et al.
Publicado: (2025)

Exact Finite-Sample Variance Decomposition of Subagging: A Spectral Filtering Perspective
por: Su, Ye, et al.
Publicado: (2026)

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
por: Huang, Kai, et al.
Publicado: (2023)

KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels
por: Wang, Han, et al.
Publicado: (2026)