Cover Image

S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Xinyu, Leng, Jixuan, Guo, Geyang, Zhao, Jiawei, Nakada, Ryumei, Zhang, Linjun, Yao, Huaxiu, Chen, Beidi
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Residual Feature Integration is Sufficient to Prevent Negative Transfer
by: Xu, Yichen, et al.
Published: (2025)

Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference
by: Zhang, Zhe, et al.
Published: (2024)

PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers
by: Zhong, Yibo, et al.
Published: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
by: Lu, Taiming, et al.
Published: (2024)

Synthetic Oversampling: Theory and A Practical Approach Using LLMs to Address Data Imbalance
by: Nakada, Ryumei, et al.
Published: (2024)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
by: Guo, Wentao, et al.
Published: (2024)

Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation
by: Dong, Harry, et al.
Published: (2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
by: Zheng, Haizhong, et al.
Published: (2024)

A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
by: Nakada, Ryumei, et al.
Published: (2025)

Contrastive Network Representation Learning
by: Dong, Zihan, et al.
Published: (2025)

Contrastive Learning on Multimodal Analysis of Electronic Health Records
by: Cai, Tianxi, et al.
Published: (2024)

Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training
by: Huang, Alyssa, et al.
Published: (2023)

Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?
by: Zheng, Haizhong, et al.
Published: (2025)

APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding
by: Yang, Xinyu, et al.
Published: (2025)

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
by: Dong, Harry, et al.
Published: (2024)

Scalable LLM Reasoning Acceleration with Low-rank Distillation
by: Dong, Harry, et al.
Published: (2025)

Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
by: Zheng, Haizhong, et al.
Published: (2025)

FineFT: Efficient and Risk-Aware Ensemble Reinforcement Learning for Futures Trading
by: Qin, Molei, et al.
Published: (2025)

FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents
by: Li, Qizheng, et al.
Published: (2026)

Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment
by: Wang, Haozhong, et al.
Published: (2026)

StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignment
by: Kim, Younghyun, et al.
Published: (2025)

Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
by: Shrestha, Susav, et al.
Published: (2025)

RobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Response
by: Luo, Junyu, et al.
Published: (2024)

DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
by: Yao, Jinwei, et al.
Published: (2024)

HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading
by: Luo, Cheng, et al.
Published: (2025)

LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
by: Roh, Yuji, et al.
Published: (2024)

FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees
by: Nie, Fan, et al.
Published: (2024)

Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
by: Yao, Yongqiang, et al.
Published: (2025)

Sirius: Contextual Sparsity with Correction for Efficient LLMs
by: Zhou, Yang, et al.
Published: (2024)

One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments
by: Yi, Ke, et al.
Published: (2024)

Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails
by: Han, Siwei, et al.
Published: (2025)

Efficient Test-Time Scaling via Self-Calibration
by: Huang, Chengsong, et al.
Published: (2025)

Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning
by: Luo, Yu, et al.
Published: (2026)

Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction
by: Liu, Zhexiong, et al.
Published: (2025)

A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI
by: Gao, Fang, et al.
Published: (2023)

SimpleMem: Efficient Lifelong Memory for LLM Agents
by: Liu, Jiaqi, et al.
Published: (2026)

MedVerse: Efficient and Reliable Medical Reasoning via DAG-Structured Parallel Execution
by: Chen, Jianwen, et al.
Published: (2026)

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
by: Ran, Yide, et al.
Published: (2025)

Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning
by: Sehanobish, Arijit, et al.
Published: (2024)

EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agents
by: Liu, Jiaqi, et al.
Published: (2026)