Cover Image

Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Tao, Jiang, Weisen, Liu, Fanghui, Huang, Xiaolin, Kwok, James T.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Effective Structured Prompting by Meta-Learning and Representative Verbalizer
by: Jiang, Weisen, et al.
Published: (2023)

RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models
by: Chen, Shuhao, et al.
Published: (2024)

End-to-end Kernel Learning via Generative Random Fourier Features
by: Fang, Kun, et al.
Published: (2020)

Forward-Backward Reasoning in Large Language Models for Mathematical Verification
by: Jiang, Weisen, et al.
Published: (2023)

BYOM: Building Your Own Multi-Task Model For Free
by: Jiang, Weisen, et al.
Published: (2023)

Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning
by: Chen, Minghui, et al.
Published: (2024)

Self-Soupervision: Cooking Model Soups without Labels
by: Fuller, Anthony, et al.
Published: (2026)

Efficient Pareto Manifold Learning with Low-Rank Structure
by: Chen, Weiyu, et al.
Published: (2024)

SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training
by: Rajabi, Sahar, et al.
Published: (2025)

OASIS: Online Activation Subspace Learning for Memory-Efficient Training
by: Choudhary, Sakshi, et al.
Published: (2026)

RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation
by: Chen, Shuhao, et al.
Published: (2026)

Enhancing Sharpness-Aware Minimization by Learning Perturbation Radius
by: Wang, Xuehao, et al.
Published: (2024)

Randomized Gradient Subspaces for Efficient Large Language Model Training
by: Rajabi, Sahar, et al.
Published: (2025)

Dual-Balancing for Multi-Task Learning
by: Lin, Baijiong, et al.
Published: (2023)

Bi-LoRA: Efficient Sharpness-Aware Minimization for Fine-Tuning Large-Scale Models
by: Liu, Yuhang, et al.
Published: (2025)

LoRA-One: One-Step Full Gradient Could Suffice for Fine-Tuning Large Language Models, Provably and Efficiently
by: Zhang, Yuanhe, et al.
Published: (2025)

A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
by: Chen, Yiming, et al.
Published: (2025)

LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups
by: Aminbeidokhti, Masih, et al.
Published: (2025)

Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks
by: Liu, Fanghui, et al.
Published: (2024)

SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection
by: Chen, Shuhao, et al.
Published: (2026)

Memory-Efficient LLM Training with Online Subspace Descent
by: Liang, Kaizhao, et al.
Published: (2024)

Scalable Hyperparameter-Divergent Ensemble Training with Automatic Learning Rate Exploration for Large Models
by: Cheng, Hailing, et al.
Published: (2026)

PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
by: Li, Yanyi, et al.
Published: (2026)

Subspace Optimization for Efficient Federated Learning under Heterogeneous Data
by: Zhu, Shuchen, et al.
Published: (2026)

SALSA: Soup-based Alignment Learning for Stronger Adaptation in RLHF
by: Chegini, Atoosa, et al.
Published: (2024)

Trainable Weight Averaging: Accelerating Training and Improving Generalization
by: Li, Tao, et al.
Published: (2022)

Efficient Resource-Constrained Training of Transformers via Subspace Optimization
by: Nguyen, Le-Trung, et al.
Published: (2025)

Self-Regularized Learning Methods
by: Schölpple, Max, et al.
Published: (2026)

A Novel Spatiotemporal Coupling Graph Convolutional Network
by: Bi, Fanghui
Published: (2024)

Enhanced Soups for Graph Neural Networks
by: Zuber, Joseph, et al.
Published: (2025)

Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation
by: Xie, Guofu, et al.
Published: (2025)

NGDB-Zoo: Towards Efficient and Scalable Neural Graph Databases Training
by: Xie, Zhongwei, et al.
Published: (2026)

Spectral Souping: A Unified Framework for Online Preference Alignment
by: Chow, Yinlam, et al.
Published: (2026)

Improving Robustness of Foundation Models in Domain Adaptation with Soup-Adapters
by: Roschkowski, Marco
Published: (2025)

Fast and Scalable Semi-Supervised Learning for Multi-View Subspace Clustering
by: Ling, Huaming, et al.
Published: (2024)

State Soup: In-Context Skill Learning, Retrieval and Mixing
by: Pióro, Maciej, et al.
Published: (2024)

MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation
by: Jiang, Weisen, et al.
Published: (2025)

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
by: Zheng, Rui, et al.
Published: (2024)

RADIN: Souping on a Budget
by: Menes, Thibaut, et al.
Published: (2024)

A Survey on Time-Series Pre-Trained Models
by: Ma, Qianli, et al.
Published: (2023)