Cover Image

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Liang, Wei, Tianwen, Zeng, Liang, Cheng, Cheng, Yang, Liu, Cheng, Peng, Wang, Lijie, Li, Chenxia, Wu, Xuejie, Zhu, Bo, Gan, Yimeng, Hu, Rui, Yan, Shuicheng, Fang, Han, Zhou, Yahui
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
by: Zeng, Liang, et al.
Published: (2024)

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
by: Wei, Tianwen, et al.
Published: (2024)

Skywork Open Reasoner 1 Technical Report
by: He, Jujie, et al.
Published: (2025)

Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
by: Liu, Chris Yuhao, et al.
Published: (2024)

Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs
by: Zeng, Liang, et al.
Published: (2025)

Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation
by: Wang, Peiyu, et al.
Published: (2025)

LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
by: Hu, Zhiyuan, et al.
Published: (2024)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
by: Liu, Chris Yuhao, et al.
Published: (2025)

Skywork-R1V3 Technical Report
by: Shen, Wei, et al.
Published: (2025)

Efficient Training of Diffusion Mixture-of-Experts Models: A Practical Recipe
by: Liu, Yahui, et al.
Published: (2025)

ContextPilot: Fast Long-Context Inference via Context Reuse
by: Jiang, Yinsicheng, et al.
Published: (2025)

KV Admission: Learning What to Write for Efficient Long-Context Inference
by: Huang, Yen-Chieh, et al.
Published: (2025)

QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management
by: Shen, Weizhou, et al.
Published: (2025)

Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
by: Tian, Junfeng, et al.
Published: (2024)

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding
by: Zhao, Tiancheng, et al.
Published: (2024)

Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoning
by: Wang, Xiaokun, et al.
Published: (2025)

Heterogeneous Learning Rate Scheduling for Neural Architecture Search on Long-Tailed Datasets
by: Tang, Chenxia
Published: (2024)

Plug-and-Play Context Feature Reuse for Efficient Masked Generation
by: Liu, Xuejie, et al.
Published: (2025)

LongAlign: A Recipe for Long Context Alignment of Large Language Models
by: Bai, Yushi, et al.
Published: (2024)

Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch
by: Zhang, Yifan, et al.
Published: (2025)

Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought
by: Peng, Yi, et al.
Published: (2025)

Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning
by: Wang, Peiyu, et al.
Published: (2025)

Optimization Hyper-parameter Laws for Large Language Models
by: Xie, Xingyu, et al.
Published: (2024)

An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding
by: Wu, Tong, et al.
Published: (2024)

Full-State Prescribed Performance-Based Consensus of Double-Integrator Multi-Agent Systems with Jointly Connected Topologies
by: Hou, Yahui, et al.
Published: (2024)

Skywork UniPic 3.0: Unified Multi-Image Composition via Sequence Modeling
by: Wei, Hongyang, et al.
Published: (2026)

Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Model
by: Wei, Hongyang, et al.
Published: (2025)

From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models
by: Xu, Chejian, et al.
Published: (2025)

LongEmbed: Extending Embedding Models for Long Context Retrieval
by: Zhu, Dawei, et al.
Published: (2024)

New Recipe for Semi-supervised Community Detection: Clique Annealing under Crystallization Kinetics
by: Cheng, Ling, et al.
Published: (2025)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
by: Wang, Shiju, et al.
Published: (2025)

Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration
by: Wu, Kangxi, et al.
Published: (2024)

On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
by: Kim, Sunghwan, et al.
Published: (2026)

Universal Speed Limit in a Far-from-Equilibrium Bose Gas: Symmetry and Dynamical Decoherence
by: Liang, Jun-Cheng, et al.
Published: (2026)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
by: You, Zeng, et al.
Published: (2025)

FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
by: Zhu, Zeyu, et al.
Published: (2024)

On local rigidity theorems with respect to the scalar curvature
by: Cheng, Liang
Published: (2023)

On locally conformally flat manifolds with positive pinched Ricci curvature
by: Cheng, Liang
Published: (2023)

The power series expansions of logarithmic Sobolev, $\mathcal{W}$- functionals and scalar curvature rigidity
by: Cheng, Liang
Published: (2024)

On the rigidity of manifolds with respect to Gagliardo-Nirenberg inequalities
by: Cheng, Liang
Published: (2025)