Cover Image

SlimPack: Fine-Grained Asymmetric Packing for Balanced and Efficient Variable-Length LLM Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yuliang, Wu, Guohao, Zhang, Shenglong, Zhang, Wei, Zhu, Qianchao, Li, Zhouyang, Wang, Chenyu
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
by: Li, Zhouyang, et al.
Published: (2025)

Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
by: Yao, Yongqiang, et al.
Published: (2025)

C-Pack: Packed Resources For General Chinese Embeddings
by: Xiao, Shitao, et al.
Published: (2023)

OPA-Pack: Object-Property-Aware Robotic Bin Packing
by: Pan, Jia-Hui, et al.
Published: (2025)

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
by: Wang, Shuhe, et al.
Published: (2024)

SlimGPT: Layer-wise Structured Pruning for Large Language Models
by: Ling, Gui, et al.
Published: (2024)

Enhancing Training Efficiency Using Packing with Flash Attention
by: Kundu, Achintya, et al.
Published: (2024)

Pack-PTQ: Advancing Post-training Quantization of Neural Networks by Pack-wise Reconstruction
by: Li, Changjun, et al.
Published: (2025)

SlimPajama-DC: Understanding Data Combinations for LLM Training
by: Shen, Zhiqiang, et al.
Published: (2023)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
by: Wang, Shiju, et al.
Published: (2025)

SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flow
by: Wang, Kaidi, et al.
Published: (2025)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

Model-Based and Sample-Efficient AI-Assisted Math Discovery in Sphere Packing
by: Tutunov, Rasul, et al.
Published: (2025)

An In-depth Study of LLM Contributions to the Bin Packing Problem
by: Herrmann, Julien, et al.
Published: (2025)

Contrastive Self-Supervised Learning As Neural Manifold Packing
by: Zhang, Guanming, et al.
Published: (2025)

GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention
by: Xue, Tianyang, et al.
Published: (2024)

ColPackAgent: Agent-Skill-Guided Hard-Particle Monte Carlo Workflows for Colloidal Packing
by: Ding, Lijie, et al.
Published: (2026)

SABER: Switchable and Balanced Training for Efficient LLM Reasoning
by: Zhao, Kai, et al.
Published: (2025)

RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing
by: Ai, Bo, et al.
Published: (2024)

Pack and Force Your Memory: Long-form and Consistent Video Generation
by: Wu, Xiaofei, et al.
Published: (2025)

PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference
by: Mao, Xiaofeng, et al.
Published: (2026)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
by: Pang, Jinlong, et al.
Published: (2025)

Re-evaluating LLM-based Heuristic Search: A Case Study on the 3D Packing Problem
by: Quan, Guorui, et al.
Published: (2025)

A Beam Search Based Parallel Algorithm for the Two-Dimensional Strip Packing Problem
by: Wen, Yajie, et al.
Published: (2025)

A Block-Based Heuristic Algorithm for the Three-Dimensional Nuclear Waste Packing Problem
by: Wen, Yajie, et al.
Published: (2025)

Kotlin ML Pack: Technical Report
by: Titov, Sergey, et al.
Published: (2024)

FastFHE: Packing-Scalable and Depthwise-Separable CNN Inference Over FHE
by: Song, Wenbo, et al.
Published: (2025)

TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning
by: Sui, Yuan, et al.
Published: (2023)

PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
by: Jiang, Bo, et al.
Published: (2025)

OctoPack: Instruction Tuning Code Large Language Models
by: Muennighoff, Niklas, et al.
Published: (2023)

PackMamba: Efficient Processing of Variable-Length Sequences in Mamba training
by: Xu, Haoran, et al.
Published: (2024)

Knowledge Fusion of Large Language Models Via Modular SkillPacks
by: Du, Guodong, et al.
Published: (2025)

The Packing Chromatic Number of the Infinite Square Grid is 15
by: Subercaseaux, Bernardo, et al.
Published: (2023)

FedKRSO: Communication and Memory Efficient Federated Fine-Tuning of Large Language Models
by: Yang, Guohao, et al.
Published: (2026)

Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLM
by: Zhang, Biyao, et al.
Published: (2025)

Fints: Efficient Inference-Time Personalization for LLMs with Fine-Grained Instance-Tailored Steering
by: Du, Kounianhua, et al.
Published: (2025)

Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search
by: Fang, Jiangyi, et al.
Published: (2026)

Efficient Dynamic Ensembling for Multiple LLM Experts
by: Hu, Jinwu, et al.
Published: (2024)

Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Prediction
by: Zhang, Zhicheng, et al.
Published: (2026)

E^2-LLM: Efficient and Extreme Length Extension of Large Language Models
by: Liu, Jiaheng, et al.
Published: (2024)