Cover Image

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Ningyuan, Yang, Jing, Cai, Kaitong, Wang, Keze
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
by: Yang, Jing, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
by: Zhang, Buze, et al.
Published: (2026)

Guardian: Decoupling Exploration from Safety in Reinforcement Learning
by: Cai, Kaitong, et al.
Published: (2025)

Learning Dynamics of VLM Finetuning
by: Zhang, Jusheng, et al.
Published: (2025)

MAT-Agent: Adaptive Multi-Agent Training Optimization
by: Zhang, Jusheng, et al.
Published: (2025)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
by: Liu, Yilun, et al.
Published: (2024)

CF-VLM:CounterFactual Vision-Language Fine-tuning
by: Zhang, Jusheng, et al.
Published: (2025)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
by: Liu, Yilun, et al.
Published: (2025)

UMoE: Unifying Attention and FFN with Shared Experts
by: Yang, Yuanhang, et al.
Published: (2025)

Causal Invariance and Counterfactual Learning Driven Cooperative Game for Multi-Label Classification
by: Fan, Yijia, et al.
Published: (2025)

MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
by: Raje, Arian, et al.
Published: (2026)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
by: Kunwar, Pradip, et al.
Published: (2025)

Rational ANOVA Networks
by: Zhang, Jusheng, et al.
Published: (2026)

Kolmogorov-Arnold Fourier Networks
by: Zhang, Jusheng, et al.
Published: (2025)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
by: Cong, Peizhuang, et al.
Published: (2025)

From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices
by: Slamanig, Georg, et al.
Published: (2025)

Parameter-Efficient Continual Fine-Tuning: A Survey
by: Coleman, Eric Nuertey, et al.
Published: (2025)

Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs
by: Zhang, Zheng, et al.
Published: (2024)

Spectral Gating Networks
by: Zhang, Jusheng, et al.
Published: (2026)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
by: He, Yifei, et al.
Published: (2025)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
by: Sun, Mengyang, et al.
Published: (2025)

A Scalable Curiosity-Driven Game-Theoretic Framework for Long-Tail Multi-Label Learning in Data Mining
by: Yang, Jing, et al.
Published: (2026)

Why Keep Your Doubts to Yourself? Trading Visual Uncertainties in Multi-Agent Bandit Systems
by: Zhang, Jusheng, et al.
Published: (2026)

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies
by: Wang, Luping, et al.
Published: (2024)

TuckA: Hierarchical Compact Tensor Experts for Efficient Fine-Tuning
by: Lei, Qifeng, et al.
Published: (2025)

Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
by: Taraghi, Mina, et al.
Published: (2025)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
by: Guo, Yongxin, et al.
Published: (2024)

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models
by: Arteaga, Gabriel Y., et al.
Published: (2024)

Long Exposure: Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsity
by: Wang, Tuowei, et al.
Published: (2025)

A Bayesian Hybrid Parameter-Efficient Fine-Tuning Method for Large Language Models
by: Chai, Yidong, et al.
Published: (2025)

EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
by: Guo, Song, et al.
Published: (2024)

Parameter Efficiency Is Not Memory Efficiency: Rethinking Fine-Tuning for On-Device LLM Adaptation
by: Tenison, Irene, et al.
Published: (2026)

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
by: Hui, Tingfeng, et al.
Published: (2024)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
by: Yan, Fanqi, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
by: Gao, Ziqi, et al.
Published: (2024)

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs
by: Wang, Ruoyu, et al.
Published: (2024)

ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning
by: Sun, Zhishen, et al.
Published: (2026)

Parameter-Efficient Fine-Tuning for Foundation Models
by: Zhang, Dan, et al.
Published: (2025)

Mixtures of Experts Unlock Parameter Scaling for Deep RL
by: Obando-Ceron, Johan, et al.
Published: (2024)

PiCa: Parameter-Efficient Fine-Tuning with Column Space Projection
by: Hwang, Junseo, et al.
Published: (2025)