Imagen de Portada

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yan, Junbing, Wang, Chengyu, Zhang, Taolin, He, Xiaofeng, Huang, Jun, Huang, Longtao, Xue, Hui, Zhang, Wei
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning
por: Li, Dongyang, et al.
Publicado: (2024)

On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models
por: Li, Dongyang, et al.
Publicado: (2024)

DAFNet: Dynamic Auxiliary Fusion for Sequential Model Editing in Large Language Models
por: Zhang, Taolin, et al.
Publicado: (2024)

R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models
por: Zhang, Taolin, et al.
Publicado: (2024)

Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning
por: Chen, Qizhou, et al.
Publicado: (2024)

UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding
por: Li, Dongyang, et al.
Publicado: (2024)

Do Large Language Models Understand Logic or Just Mimick Context?
por: Yan, Junbing, et al.
Publicado: (2024)

An Information-Theoretic Framework for Robust Large Language Model Editing
por: Chen, Qizhou, et al.
Publicado: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
por: Wang, Chengyu, et al.
Publicado: (2025)

DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models
por: Wang, Chengyu, et al.
Publicado: (2025)

UniEdit: A Unified Knowledge Editing Benchmark for Large Language Models
por: Chen, Qizhou, et al.
Publicado: (2025)

Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts
por: Chen, Qizhou, et al.
Publicado: (2024)

Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit
por: Chen, Qizhou, et al.
Publicado: (2024)

A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directions
por: Wang, Chengyu, et al.
Publicado: (2025)

Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment
por: Cai, Wenrui, et al.
Publicado: (2025)

Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series
por: Cai, Wenrui, et al.
Publicado: (2025)

BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering
por: Zhang, Taolin, et al.
Publicado: (2025)

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
por: Lyu, Yuanjie, et al.
Publicado: (2026)

Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations
por: Cai, Wenrui, et al.
Publicado: (2025)

AMATA: Adaptive Multi-Agent Trajectory Alignment for Knowledge-Intensive Question Answering
por: Zhang, Taolin, et al.
Publicado: (2026)

QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs
por: Zhang, Taolin, et al.
Publicado: (2025)

Towards Versatile and Efficient Visual Knowledge Integration into Pre-trained Language Models with Cross-Modal Adapters
por: Zhang, Xinyun, et al.
Publicado: (2023)

Learning Transferable Topology Priors for Multi-Agent LLM Collaboration Across Domains
por: Zhang, Taolin, et al.
Publicado: (2026)

Towards Effective and Efficient Continual Pre-training of Large Language Models
por: Chen, Jie, et al.
Publicado: (2024)

S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models
por: Yuan, Xiaohan, et al.
Publicado: (2024)

Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text
por: Huang, Chengyu, et al.
Publicado: (2026)

BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillation
por: Wang, Chen, et al.
Publicado: (2024)

Taming "Zombie'' Agents: A Markov State-Aware Framework for Resilient Multi-Agent Evolution
por: Zhang, Taolin, et al.
Publicado: (2026)

QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration
por: Ren, Shaola, et al.
Publicado: (2025)

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
por: Liang, Juhao, et al.
Publicado: (2024)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
por: Wang, Shaobo, et al.
Publicado: (2026)

STEP: Staged Parameter-Efficient Pre-training for Large Language Models
por: Yano, Kazuki, et al.
Publicado: (2025)

Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
por: Geng, Yuxia, et al.
Publicado: (2023)

Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning
por: Yue, Yuanhao, et al.
Publicado: (2024)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
por: Sharma, Kartik, et al.
Publicado: (2025)

Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training
por: Wu, Linjuan, et al.
Publicado: (2025)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
por: Zhang, Ying, et al.
Publicado: (2024)

Open World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training
por: Wang, Haoran, et al.
Publicado: (2026)

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training
por: He, Nan, et al.
Publicado: (2024)

DocMamba: Efficient Document Pre-training with State Space Model
por: Hu, Pengfei, et al.
Publicado: (2024)