Copertina

Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yao, Yongqiang, Tan, Jingru, Liang, Kaihuan, Zhang, Feizhao, Hu, Jiahao, Wu, Shuo, Niu, Yazhe, Gong, Ruihao, Lin, Dahua, Xu, Ningyi
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

OmniBal: Towards Fast Instruction-Tuning for Vision-Language Models via Omniverse Computation Balance
di: Yao, Yongqiang, et al.
Pubblicazione: (2024)

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
di: Wang, Shuhe, et al.
Pubblicazione: (2024)

ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning
di: Du, Yiming, et al.
Pubblicazione: (2025)

SlimPack: Fine-Grained Asymmetric Packing for Balanced and Efficient Variable-Length LLM Training
di: Liu, Yuliang, et al.
Pubblicazione: (2025)

Efficient Long Context Fine-tuning with Chunk Flow
di: Yuan, Xiulong, et al.
Pubblicazione: (2025)

Skrull: Towards Efficient Long Context Fine-tuning through Dynamic Data Scheduling
di: Xu, Hongtao, et al.
Pubblicazione: (2025)

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
di: Yao, Feiyu, et al.
Pubblicazione: (2026)

Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
di: Long, Lingkun, et al.
Pubblicazione: (2026)

Fine-Tuned In-Context Learners for Efficient Adaptation
di: Bornschein, Jorg, et al.
Pubblicazione: (2025)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
di: Chen, Yukang, et al.
Pubblicazione: (2023)

Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning
di: Luo, Yu, et al.
Pubblicazione: (2026)

HH-Codec: High Compression High-fidelity Discrete Neural Codec for Spoken Language Modeling
di: Xue, Rongkun, et al.
Pubblicazione: (2025)

Structured Packing in LLM Training Improves Long Context Utilization
di: Staniszewski, Konrad, et al.
Pubblicazione: (2023)

Analysis and Optimized CXL-Attached Memory Allocation for Long-Context LLM Fine-Tuning
di: Liaw, Yong-Cheng, et al.
Pubblicazione: (2025)

On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training
di: Niu, Xueyan, et al.
Pubblicazione: (2026)

DynSplit-KV: Dynamic Semantic Splitting for KVCache Compression in Efficient Long-Context LLM Inference
di: Ye, Jiancai, et al.
Pubblicazione: (2026)

Let Androids Dream of Electric Sheep: A Human-Inspired Image Implication Understanding and Reasoning Framework
di: Zhang, Chenhao, et al.
Pubblicazione: (2025)

TreeTensor: Boost AI System on Nested Data with Constrained Tree-Like Tensor
di: Zhang, Shaoang, et al.
Pubblicazione: (2026)

CoTZero: Annotation-Free Human-Like Vision Reasoning via Hierarchical Synthetic CoT
di: Du, Chengyi, et al.
Pubblicazione: (2026)

LoFT: Parameter-Efficient Fine-Tuning for Long-tailed Semi-Supervised Learning in Open-World Scenarios
di: Huang, Zhiyuan, et al.
Pubblicazione: (2025)

CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference
di: Fei, Chao, et al.
Pubblicazione: (2026)

When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
di: Zheng, Yingming, et al.
Pubblicazione: (2025)

Data-efficient Fine-tuning for LLM-based Recommendation
di: Lin, Xinyu, et al.
Pubblicazione: (2024)

The Detector Teaches Itself: Lightweight Self-Supervised Adaptation for Open-Vocabulary Object Detection
di: Wan, Yazhe, et al.
Pubblicazione: (2026)

LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K
di: Yuan, Tao, et al.
Pubblicazione: (2024)

Listen, Pause, and Reason: Toward Perception-Grounded Hybrid Reasoning for Audio Understanding
di: Wang, Jieyi, et al.
Pubblicazione: (2026)

Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs
di: Dong, Jiancheng, et al.
Pubblicazione: (2024)

S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
di: Yang, Xinyu, et al.
Pubblicazione: (2024)

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
di: Zou, Heming, et al.
Pubblicazione: (2025)

Where Does Long-Context Supervision Actually Go? Effective-Context Exposure Balancing
di: Zhu, Jinchang, et al.
Pubblicazione: (2026)

Balancing Fine-tuning and RAG: A Hybrid Strategy for Dynamic LLM Recommendation Updates
di: Meng, Changping, et al.
Pubblicazione: (2025)

Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
di: Zhou, Huichi, et al.
Pubblicazione: (2025)

The Devil is in Fine-tuning and Long-tailed Problems:A New Benchmark for Scene Text Detection
di: Cao, Tianjiao, et al.
Pubblicazione: (2025)

Efficient Long-Context LLM Inference via KV Cache Clustering
di: Hu, Jie, et al.
Pubblicazione: (2025)

Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning
di: Tan, Qitao, et al.
Pubblicazione: (2025)

Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations
di: Mi, Zhendong, et al.
Pubblicazione: (2025)

Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach
di: Zeng, Shenglai, et al.
Pubblicazione: (2025)

Transformers for Supervised Online Continual Learning
di: Bornschein, Jorg, et al.
Pubblicazione: (2024)

Towards Safeguarding LLM Fine-tuning APIs against Cipher Attacks
di: Youstra, Jack, et al.
Pubblicazione: (2025)

MetaphorStar: Image Metaphor Understanding and Reasoning with End-to-End Visual Reinforcement Learning
di: Zhang, Chenhao, et al.
Pubblicazione: (2026)