Copertina

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Luo, Zheheng, Zhang, Xin, Liu, Xiao, Li, Haoling, Gong, Yeyun, Qi, Chen, Cheng, Peng
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
di: Yang, Kailai, et al.
Pubblicazione: (2025)

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
di: Li, Haoling, et al.
Pubblicazione: (2024)

Process-based Self-Rewarding Language Models
di: Zhang, Shimao, et al.
Pubblicazione: (2025)

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training
di: He, Nan, et al.
Pubblicazione: (2024)

PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization
di: Ma, Xinbei, et al.
Pubblicazione: (2023)

Improving Data and Reward Design for Scientific Reasoning in Large Language Models
di: Chen, Zijie, et al.
Pubblicazione: (2026)

DeepThink: Aligning Language Models with Domain-Specific User Intents
di: Li, Yang, et al.
Pubblicazione: (2025)

Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
di: Cheng, Runxi, et al.
Pubblicazione: (2026)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
di: Shin, Haebin, et al.
Pubblicazione: (2025)

Revealing the Learning Dynamics of Long-Context Continual Pre-training
di: Liang, Yupu, et al.
Pubblicazione: (2026)

ToReMi: Topic-Aware Data Reweighting for Dynamic Pre-Training Data Selection
di: Zhu, Xiaoxuan, et al.
Pubblicazione: (2025)

HiCaM: A Hierarchical-Causal Modification Framework for Long-Form Text Modification
di: Shi, Yuntao, et al.
Pubblicazione: (2025)

Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning
di: Park, Sungjin, et al.
Pubblicazione: (2024)

Factual consistency evaluation of summarization in the Era of large language models
di: Luo, Zheheng, et al.
Pubblicazione: (2024)

The Lay Person's Guide to Biomedicine: Orchestrating Large Language Models
di: Luo, Zheheng, et al.
Pubblicazione: (2024)

Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation
di: Zhu, Hongyin, et al.
Pubblicazione: (2021)

G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks
di: Wan, Zhongwei, et al.
Pubblicazione: (2022)

Pre-trained Large Language Models for Financial Sentiment Analysis
di: Luo, Wei, et al.
Pubblicazione: (2024)

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
di: Shin, Haebin, et al.
Pubblicazione: (2025)

Scaling Agents via Continual Pre-training
di: Su, Liangcai, et al.
Pubblicazione: (2025)

Task Oriented In-Domain Data Augmentation
di: Liang, Xiao, et al.
Pubblicazione: (2024)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
di: Xie, Yong, et al.
Pubblicazione: (2023)

Domain Pre-training Impact on Representations
di: Gonzalez-Gutierrez, Cesar, et al.
Pubblicazione: (2025)

MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics
di: Wang, Jiapeng, et al.
Pubblicazione: (2025)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
di: Li, Chen-An, et al.
Pubblicazione: (2024)

Are Large Language Models True Healthcare Jacks-of-All-Trades? Benchmarking Across Health Professions Beyond Physician Exams
di: Luo, Zheheng, et al.
Pubblicazione: (2024)

Learning from the Best, Differently: A Diversity-Driven Rethinking on Data Selection
di: He, Hongyi, et al.
Pubblicazione: (2025)

Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain
di: Uğur, Özgür, et al.
Pubblicazione: (2026)

Continual Pre-Training is (not) What You Need in Domain Adaption
di: Chen, Pin-Er, et al.
Pubblicazione: (2025)

Efficient Continual Pre-training by Mitigating the Stability Gap
di: Guo, Yiduo, et al.
Pubblicazione: (2024)

Optimizing Large Language Model Training Using FP4 Quantization
di: Wang, Ruizhe, et al.
Pubblicazione: (2025)

InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions
di: Wang, Yifan, et al.
Pubblicazione: (2024)

VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervision
di: Gong, Xuan, et al.
Pubblicazione: (2025)

Effectiveness of Pre-training for Few-shot Intent Classification
di: Zhang, Haode, et al.
Pubblicazione: (2021)

DoGE: Domain Reweighting with Generalization Estimation
di: Fan, Simin, et al.
Pubblicazione: (2023)

Can Pre-trained Language Models Understand Chinese Humor?
di: Chen, Yuyan, et al.
Pubblicazione: (2024)

Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training
di: Hirano, Masanori, et al.
Pubblicazione: (2024)

New Intent Discovery with Pre-training and Contrastive Learning
di: Zhang, Yuwei, et al.
Pubblicazione: (2022)

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
di: Xiao, Chaojun, et al.
Pubblicazione: (2023)

Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
di: Chen, Yuyan, et al.
Pubblicazione: (2024)