Imagen de Portada

Efficient Continual Pre-training by Mitigating the Stability Gap

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Guo, Yiduo, Fu, Jie, Zhang, Huishuai, Zhao, Dongyan, Shen, Yikang
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Evidence-Enhanced Triplet Generation Framework for Hallucination Alleviation in Generative Question Answering
por: Du, Haowei, et al.
Publicado: (2024)

Synthetic Data RL: Task Definition Is All You Need
por: Guo, Yiduo, et al.
Publicado: (2025)

PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion
por: Zhang, Zekai, et al.
Publicado: (2024)

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
por: Gong, Zhuocheng, et al.
Publicado: (2025)

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training
por: Du, Wenyu, et al.
Publicado: (2024)

ReasVQA: Advancing VideoQA with Imperfect Reasoning Process
por: Liang, Jianxin, et al.
Publicado: (2025)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
por: Yuan, Danlong, et al.
Publicado: (2024)

VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Format
por: Wang, Yueqian, et al.
Publicado: (2024)

Beyond Isolated Facts: Synthesizing Narrative and Grounded Supervision for VideoQA
por: Liang, Jianxin, et al.
Publicado: (2025)

MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning
por: Wang, Yueqian, et al.
Publicado: (2025)

AIDBench: A benchmark for evaluating the authorship identification capability of large language models
por: Wen, Zichen, et al.
Publicado: (2024)

Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models
por: Agarwal, Mayank, et al.
Publicado: (2024)

Scaling Agents via Continual Pre-training
por: Su, Liangcai, et al.
Publicado: (2025)

Efficient Continual Pre-training of LLMs for Low-resource Languages
por: Nag, Arijit, et al.
Publicado: (2024)

Shorten After You're Right: Lazy Length Penalties for Reasoning RL
por: Yuan, Danlong, et al.
Publicado: (2025)

Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules
por: Gong, Zhuocheng, et al.
Publicado: (2024)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
por: Xie, Yong, et al.
Publicado: (2023)

Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences
por: Yu, Liu, et al.
Publicado: (2025)

Towards Effective and Efficient Continual Pre-training of Large Language Models
por: Chen, Jie, et al.
Publicado: (2024)

Can Continual Pre-training Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain?
por: Doll, Niclas, et al.
Publicado: (2026)

The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasing
por: Kaneko, Masahiro, et al.
Publicado: (2024)

Revealing the Learning Dynamics of Long-Context Continual Pre-training
por: Liang, Yupu, et al.
Publicado: (2026)

De-Anonymization at Scale via Tournament-Style Attribution
por: Zhang, Lirui, et al.
Publicado: (2026)

Projective Methods for Mitigating Gender Bias in Pre-trained Language Models
por: Dawkins, Hillary, et al.
Publicado: (2024)

xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
por: Cheng, Xin, et al.
Publicado: (2024)

Bag of Lies: Robustness in Continuous Pre-training BERT
por: Gevers, Ine, et al.
Publicado: (2024)

GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language
por: Kim, Jinwoong, et al.
Publicado: (2026)

Internal and External Knowledge Interactive Refinement Framework for Knowledge-Intensive Question Answering
por: Du, Haowei, et al.
Publicado: (2024)

In-Context Learning with Reinforcement Learning for Incomplete Utterance Rewriting
por: Du, Haowei, et al.
Publicado: (2024)

Text to Band Gap: Pre-trained Language Models as Encoders for Semiconductor Band Gap Prediction
por: Yeh, Ying-Ting, et al.
Publicado: (2025)

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
por: Liu, Yan, et al.
Publicado: (2024)

Improving Continual Pre-training Through Seamless Data Packing
por: Yin, Ruicheng, et al.
Publicado: (2025)

Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction
por: Zhang, Chong, et al.
Publicado: (2024)

Evaluating Discourse Cohesion in Pre-trained Language Models
por: He, Jie, et al.
Publicado: (2025)

Octo-planner: On-device Language Model for Planner-Action Agents
por: Chen, Wei, et al.
Publicado: (2024)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
por: Ma, Shengjie, et al.
Publicado: (2025)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
por: Yan, Junbing, et al.
Publicado: (2024)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
por: Li, Chen-An, et al.
Publicado: (2024)

SongSage: A Large Musical Language Model with Lyric Generative Pre-training
por: Guo, Jiani, et al.
Publicado: (2026)

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
por: Luo, Zheheng, et al.
Publicado: (2024)