Imagen de Portada

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yang, Yuchen, Shi, Yingdong, Wang, Cheems, Zhen, Xiantong, Shi, Yuxuan, Xu, Jun
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
por: Shi, Jiang-Xin, et al.
Publicado: (2025)

Study of Training Dynamics for Memory-Constrained Fine-Tuning
por: Quélennec, Aël, et al.
Publicado: (2025)

StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs
por: Luo, Qijun, et al.
Publicado: (2025)

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
por: Liu, Ningyuan, et al.
Publicado: (2025)

FedKRSO: Communication and Memory Efficient Federated Fine-Tuning of Large Language Models
por: Yang, Guohao, et al.
Publicado: (2026)

Parameter Efficiency Is Not Memory Efficiency: Rethinking Fine-Tuning for On-Device LLM Adaptation
por: Tenison, Irene, et al.
Publicado: (2026)

MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
por: Raje, Arian, et al.
Publicado: (2026)

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
por: Yang, Yifan, et al.
Publicado: (2024)

GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
por: Zhou, Sifan, et al.
Publicado: (2025)

Understanding Transformer from the Perspective of Associative Memory
por: Zhong, Shu, et al.
Publicado: (2025)

Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps
por: Yang, Ningyuan, et al.
Publicado: (2025)

MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning
por: Chen, Jinhao, et al.
Publicado: (2025)

RPO:Reinforcement Fine-Tuning with Partial Reasoning Optimization
por: Yi, Hongzhu, et al.
Publicado: (2026)

Gains: Fine-grained Federated Domain Adaptation in Open Set
por: Zhong, Zhengyi, et al.
Publicado: (2025)

LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning
por: Chen, Junyu, et al.
Publicado: (2025)

LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs
por: Kim, Taeho, et al.
Publicado: (2024)

AutoMixQ: Self-Adjusting Quantization for High Performance Memory-Efficient Fine-Tuning
por: Zhou, Changhai, et al.
Publicado: (2024)

Neuroscience-Inspired Memory Replay for Continual Learning: A Comparative Study of Predictive Coding and Backpropagation-Based Strategies
por: Nalagatla, Goutham, et al.
Publicado: (2025)

Identity Bridge: Enabling Implicit Reasoning via Shared Latent Memory
por: Lin, Pengxiao, et al.
Publicado: (2025)

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models
por: Arteaga, Gabriel Y., et al.
Publicado: (2024)

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
por: Lu, Yiyang, et al.
Publicado: (2026)

NASH: Neural Architecture and Accelerator Search for Multiplication-Reduced Hybrid Models
por: Xu, Yang, et al.
Publicado: (2024)

Variational Task Vector Composition
por: Zhang, Boyuan, et al.
Publicado: (2025)

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
por: Yang, Jing, et al.
Publicado: (2025)

CERSA: Cumulative Energy-Retaining Subspace Adaptation for Memory-Efficient Fine-Tuning
por: Ge, Jingze, et al.
Publicado: (2026)

ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning
por: Sun, Zhishen, et al.
Publicado: (2026)

TS-Memory: Plug-and-Play Memory for Time Series Foundation Models
por: Lyu, Sisuo, et al.
Publicado: (2026)

Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models
por: Son, Hyegang, et al.
Publicado: (2024)

Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning
por: Park, Juneyoung, et al.
Publicado: (2026)

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
por: Wang, Yibin, et al.
Publicado: (2024)

Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
por: Dong, Songwei, et al.
Publicado: (2026)

Experience-Evolving Multi-Turn Tool-Use Agent with Hybrid Episodic-Procedural Memory
por: Li, Sijia, et al.
Publicado: (2025)

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
por: Huang, Kai, et al.
Publicado: (2023)

Internalizing Meta-Experience into Memory for Guided Reinforcement Learning in Large Language Models
por: Huang, Shiting, et al.
Publicado: (2026)

FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition
por: Yan, Yuxuan, et al.
Publicado: (2024)

Practical Boolean Backpropagation
por: Golbert, Simon
Publicado: (2025)

Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments
por: Qu, Yun, et al.
Publicado: (2025)

RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
por: Li, Yuduo, et al.
Publicado: (2026)

MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models
por: Zhang, Zhen, et al.
Publicado: (2025)

Memory Caching: RNNs with Growing Memory
por: Behrouz, Ali, et al.
Publicado: (2026)