Cover Image

MemFine: Memory-Aware Fine-Grained Scheduling for MoE Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Lu, Shi, Rong, Zhang, Shaoqing, Chen, Yueqiang, He, Baoguo, Sun, Hongfeng, Yin, Ziqing, Su, Shangchao, Cui, Zhiyan, Dong, Liang, Li, Xiyuan, Wang, Lingbin, He, Jianwei, Ma, Jiesong, Huang, Weikang, Tong, Jianglei, Gao, Dongdong, Zhang, Jian, Tian, Hong, Shen, Hui, Luo, Zongtai, Sun, Zhaoqun, Niu, Hongxing, Sun, Yue
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MoFa: A Unified Performance Modeling Framework for LLM Pretraining
by: Zhao, Lu, et al.
Published: (2025)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
by: Chen, Xing, et al.
Published: (2025)

Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management
by: Ma, Weitao, et al.
Published: (2026)

On uniformly quasiconformal Anosov diffeomorphisms with two dimensional distributions
by: Zhang, Jiesong
Published: (2023)

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
by: Pan, Xinglin, et al.
Published: (2025)

Towards Fine-Grained Webpage Fingerprinting at Scale
by: Zhao, Xiyuan, et al.
Published: (2024)

Antioxidant Carbon Dot of Selenomethionine Alleviates Oxidative Stress in Intervertebral Disc Degeneration
by: Qingzheng Zhang, et al.
Published: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
by: Muzio, Alexandre, et al.
Published: (2024)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
by: Sun, Yifan, et al.
Published: (2026)

Remarks on Linear Growth of Vorticity Gradients and Support Diameters for 2D Euler Flow in Half-Plane
by: Chen, Shaoqing, et al.
Published: (2026)

Additional Cover: Cover Image, Volume 24, Issue 4
by: Luyao Zhao, et al.
Published: (2025)

From Industry 4.0 to 5.0: Exploring the Opportunity of Biodegradable Freshness Indicator Packaging
by: Luyao Zhao, et al.
Published: (2025)

Supervised Fine-Tuning as Inverse Reinforcement Learning
by: Sun, Hao
Published: (2024)

Is One-Shot In-Context Learning Helpful for Data Selection in Task-Specific Fine-Tuning of Multimodal LLMs?
by: An, Xiao, et al.
Published: (2026)

Invariant distributions of partially hyperbolic systems: fractal graphs, excessive regularity, and rigidity
by: Xu, Disheng, et al.
Published: (2024)

On holomorphic partially hyperbolic systems
by: Xu, Disheng, et al.
Published: (2024)

FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema
by: Lu, Junru, et al.
Published: (2024)

S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain
by: Zhang, Baoquan, et al.
Published: (2026)

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
by: Hou, Zhiyan, et al.
Published: (2026)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
by: He, Xixiang, et al.
Published: (2025)

MASIM: An Efficient Multi-Array Scheduler for In-Memory SIMD Computation
by: Qian, Xingyue, et al.
Published: (2024)

Integrating Weather Foundation Model and Satellite to Enable Fine-Grained Solar Irradiance Forecasting
by: Ma, Ziqing, et al.
Published: (2026)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
by: Manzoni, Andrea
Published: (2026)

CooperLLM: Cloud-Edge-End Cooperative Federated Fine-tuning for LLMs via ZOO-based Gradient Correction
by: Sun, He, et al.
Published: (2026)

APPT: Boosting Automated Patch Correctness Prediction via Fine-tuning Pre-trained Models
by: Zhang, Quanjun, et al.
Published: (2023)

Concentrating solutions of the fractional $(p,q)$-Choquard equation with exponential growth
by: Song, Yueqiang, et al.
Published: (2025)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
by: Zhang, Kairun, et al.
Published: (2025)

Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling
by: Liao, Ning, et al.
Published: (2025)

OmniSparse: Training-Aware Fine-Grained Sparse Attention for Long-Video MLLMs
by: Chen, Feng, et al.
Published: (2025)

Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!
by: Zhang, Zhexin, et al.
Published: (2025)

MemX: A Local-First Long-Term Memory System for AI Assistants
by: Sun, Lizheng
Published: (2026)

Fine-tuning vs Prompting, Can Language Models Understand Human Values?
by: Sun, Pingwei
Published: (2024)

Fine-grained MoE Load Balancing with Linear Programming
by: Zhao, Chenqi, et al.
Published: (2025)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
by: Chen, Yupeng, et al.
Published: (2024)

Retinexmamba: Retinex-based Mamba for Low-light Image Enhancement
by: Bai, Jiesong, et al.
Published: (2024)

NTTSuite: Number Theoretic Transform Benchmarks for Accelerating Encrypted Computation
by: Ding, Juran, et al.
Published: (2024)

HyMem: Hybrid Memory Architecture with Dynamic Retrieval Scheduling
by: Zhao, Xiaochen, et al.
Published: (2026)

On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models
by: Sun, Qian, et al.
Published: (2024)

High-Quality Iterative Logic Compiler for In-Memory SIMD Computation with Tight Coupling of Synthesis and Scheduling
by: Qian, Xingyue, et al.
Published: (2024)