Cover Image

Model Balancing Helps Low-data Training and Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Zihang, Hu, Yuanzhe, Pang, Tianyu, Zhou, Yefan, Ren, Pu, Yang, Yaoqing
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search
by: Xiong, Jinglong, et al.
Published: (2026)

A Three-regime Model of Network Pruning
by: Zhou, Yefan, et al.
Published: (2023)

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
by: Hsiung, Lei, et al.
Published: (2025)

LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
by: Liu, Zihang, et al.
Published: (2025)

Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
by: Pang, Tianyu, et al.
Published: (2026)

RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization
by: Deng, Shenyang, et al.
Published: (2026)

MD tree: a model-diagnostic tree grown on loss landscape
by: Zhou, Yefan, et al.
Published: (2024)

A Model Zoo on Phase Transitions in Neural Networks
by: Schürholt, Konstantin, et al.
Published: (2025)

Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
by: Lu, Haiquan, et al.
Published: (2024)

AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
by: Lu, Haiquan, et al.
Published: (2024)

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction
by: Pang, Tianyu, et al.
Published: (2026)

Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
by: Deng, Shenyang, et al.
Published: (2026)

Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias
by: Hu, Yuanzhe, et al.
Published: (2025)

Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
by: Wang, Yuxin, et al.
Published: (2026)

Depth, Not Data: An Analysis of Hessian Spectral Bifurcation
by: Deng, Shenyang, et al.
Published: (2026)

From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
by: Kothapalli, Vignesh, et al.
Published: (2024)

Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation
by: Liu, Xiaotian, et al.
Published: (2026)

Learning to Discover Iterative Spectral Algorithms
by: Liu, Zihang, et al.
Published: (2026)

TuneComp: Joint Fine-tuning and Compression for Large Foundation Models
by: Chen, Xiangyu, et al.
Published: (2025)

Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions
by: Wang, Jingtan, et al.
Published: (2024)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
by: Feng, Jinyuan, et al.
Published: (2025)

Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
by: Wang, Yezhen, et al.
Published: (2025)

Balancing Continual Learning and Fine-tuning for Human Activity Recognition
by: Tang, Chi Ian, et al.
Published: (2024)

AFLoRA: Adaptive Federated Fine-Tuning of Large Language Models with Resource-Aware Low-Rank Adaption
by: Zhou, Yajie, et al.
Published: (2025)

EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning
by: Pang, Jing-Cheng, et al.
Published: (2026)

Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures
by: Chen, Yiming, et al.
Published: (2024)

Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
by: Zhou, Huichi, et al.
Published: (2025)

Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
by: Yao, Yongqiang, et al.
Published: (2025)

KCES: Training-Free Defense for Robust Graph Neural Networks via Kernel Complexity
by: Jia, Yaning, et al.
Published: (2025)

Fine-tuning Flow Matching Generative Models with Intermediate Feedback
by: Fan, Jiajun, et al.
Published: (2025)

VAR: Visual Analysis for Rashomon Set of Machine Learning Models' Performance
by: Jin, Yuanzhe
Published: (2025)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
by: Liu, Xuyuan, et al.
Published: (2025)

Pre-Trained Model Recommendation for Downstream Fine-tuning
by: Bai, Jiameng, et al.
Published: (2024)

DP-MicroAdam: Private and Frugal Algorithm for Training and Fine-tuning
by: Hudişteanu, Mihaela, et al.
Published: (2025)

Aggregating Low Rank Adapters in Federated Fine-tuning
by: Trautmann, Evelyn, et al.
Published: (2025)

Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack
by: Huang, Tiansheng, et al.
Published: (2024)

UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
by: Tran, Van-Tuan, et al.
Published: (2026)

Enhancing Multi-modal Models with Heterogeneous MoE Adapters for Fine-tuning
by: Zhou, Sashuai, et al.
Published: (2025)