Image de couverture de livre

Improving Continual Pre-training Through Seamless Data Packing

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yin, Ruicheng, Gao, Xuan, Lv, Changze, Wang, Xiaohua, Zheng, Xiaoqing, Huang, Xuanjing
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Decoding Continuous Character-based Language from Non-invasive Brain Recordings
par: Zhang, Cenyuan, et autres
Publié: (2024)

Explainable Synthetic Image Detection through Diffusion Timestep Ensembling
par: Wu, Yixin, et autres
Publié: (2025)

Searching for Best Practices in Retrieval-Augmented Generation
par: Wang, Xiaohua, et autres
Publié: (2024)

UPLex: Fine-Grained Personality Control in Large Language Models via Unsupervised Lexical Modulation
par: Li, Tianlong, et autres
Publié: (2023)

Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective
par: Li, Tianlong, et autres
Publié: (2024)

Spiking Convolutional Neural Networks for Text Classification
par: Lv, Changze, et autres
Publié: (2024)

Aligning Large Language Models with Human Preferences through Representation Engineering
par: Liu, Wenhao, et autres
Publié: (2023)

Promoting Data and Model Privacy in Federated Learning through Quantized LoRA
par: Zhu, JianHao, et autres
Publié: (2024)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
par: Wang, Zhenghua, et autres
Publié: (2025)

Advancing Parameter Efficiency in Fine-tuning via Representation Editing
par: Wu, Muling, et autres
Publié: (2024)

SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation
par: Lv, Changze, et autres
Publié: (2023)

Beyond Single Labels: Improving Conversational Recommendation through LLM-Powered Data Augmentation
par: Xu, Haozhe, et autres
Publié: (2025)

VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck
par: Zhang, Feiran, et autres
Publié: (2026)

Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning
par: Wu, Muling, et autres
Publié: (2025)

SpikeCLIP: A Contrastive Language-Image Pretrained Spiking Neural Network
par: Lv, Changze, et autres
Publié: (2023)

Enhancing the Capability and Robustness of Large Language Models through Reinforcement Learning-Driven Query Refinement
par: Wang, Xiaohua, et autres
Publié: (2024)

Benchmark^2: Systematic Evaluation of LLM Benchmarks
par: Qian, Qi, et autres
Publié: (2026)

Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation
par: Lv, Changze, et autres
Publié: (2026)

Advancing Spiking Neural Networks for Sequential Modeling with Central Pattern Generators
par: Lv, Changze, et autres
Publié: (2024)

Efficient and Effective Time-Series Forecasting with Spiking Neural Networks
par: Lv, Changze, et autres
Publié: (2024)

Scaling Agents via Continual Pre-training
par: Su, Liangcai, et autres
Publié: (2025)

CSSG: Measuring Code Similarity with Semantic Graphs
par: Lu, Yiyang, et autres
Publié: (2026)

Revealing the Learning Dynamics of Long-Context Continual Pre-training
par: Liang, Yupu, et autres
Publié: (2026)

Toward Relative Positional Encoding in Spiking Transformers
par: Lv, Changze, et autres
Publié: (2025)

SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading
par: Shen, Yuanzhe, et autres
Publié: (2025)

TripTailor: A Real-World Benchmark for Personalized Travel Planning
par: Shen, Yuanzhe, et autres
Publié: (2025)

Adaptive Layer-skipping in Pre-trained LLMs
par: Luo, Xuan, et autres
Publié: (2025)

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
par: Lv, Changze, et autres
Publié: (2025)

Biologically Plausible Learning via Bidirectional Spike-Based Distillation
par: Lv, Changze, et autres
Publié: (2025)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
par: Guo, Xu, et autres
Publié: (2026)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
par: Wang, Shaobo, et autres
Publié: (2026)

Efficient Continual Pre-training by Mitigating the Stability Gap
par: Guo, Yiduo, et autres
Publié: (2024)

Bag of Lies: Robustness in Continuous Pre-training BERT
par: Gevers, Ine, et autres
Publié: (2024)

CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs
par: Wang, Liang, et autres
Publié: (2026)

Design as Desired: Utilizing Visual Question Answering for Multimodal Pre-training
par: Su, Tongkun, et autres
Publié: (2024)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
par: Liu, Lei, et autres
Publié: (2025)

AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages
par: Yu, Hao, et autres
Publié: (2026)

From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Models
par: Zhang, Ruiqi, et autres
Publié: (2026)

Thinking Augmented Pre-training
par: Wang, Liang, et autres
Publié: (2025)

Enhancing Question Answering on Charts Through Effective Pre-training Tasks
par: Gupta, Ashim, et autres
Publié: (2024)