Copertina

KoCo: Conditioning Language Model Pre-training on Knowledge Coordinates

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Yudong, Cai, Jiawei, Shen, Linlin
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

KoGEC : Korean Grammatical Error Correction with Pre-trained Translation Models
di: Kim, Taeeun, et al.
Pubblicazione: (2025)

Metadata Conditioning Accelerates Language Model Pre-training
di: Gao, Tianyu, et al.
Pubblicazione: (2025)

KoLA: Carefully Benchmarking World Knowledge of Large Language Models
di: Yu, Jifan, et al.
Pubblicazione: (2023)

KoRe: Compact Knowledge Representations for Large Language Models
di: Cavicchini, Davide, et al.
Pubblicazione: (2026)

DEPT: Decoupled Embeddings for Pre-training Language Models
di: Iacob, Alex, et al.
Pubblicazione: (2024)

Knowledge-augmented Pre-trained Language Models for Biomedical Relation Extraction
di: Sänger, Mario, et al.
Pubblicazione: (2025)

Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
di: Cheng, Runxi, et al.
Pubblicazione: (2026)

SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity
di: Xi, Xiangyu, et al.
Pubblicazione: (2025)

Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
di: Geng, Yuxia, et al.
Pubblicazione: (2023)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
di: Yan, Junbing, et al.
Pubblicazione: (2024)

Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation
di: Zhu, Hongyin, et al.
Pubblicazione: (2021)

CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training
di: Brandfonbrener, David, et al.
Pubblicazione: (2024)

Pre-training and Diagnosing Knowledge Base Completion Models
di: Kocijan, Vid, et al.
Pubblicazione: (2024)

Investigating Data Contamination for Pre-training Language Models
di: Jiang, Minhao, et al.
Pubblicazione: (2024)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
di: Sharma, Kartik, et al.
Pubblicazione: (2025)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
di: Zhang, Ying, et al.
Pubblicazione: (2024)

Model Merging in Pre-training of Large Language Models
di: Li, Yunshui, et al.
Pubblicazione: (2025)

GPTFace: Generative Pre-training of Facial-Linguistic Transformer by Span Masking and Weakly Correlated Text-image Data
di: Li, Yudong, et al.
Pubblicazione: (2025)

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
di: Gu, Jiawei, et al.
Pubblicazione: (2024)

Pre-training Limited Memory Language Models with Internal and External Knowledge
di: Zhao, Linxi, et al.
Pubblicazione: (2025)

Towards Versatile and Efficient Visual Knowledge Integration into Pre-trained Language Models with Cross-Modal Adapters
di: Zhang, Xinyun, et al.
Pubblicazione: (2023)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
di: Li, Chen-An, et al.
Pubblicazione: (2024)

Evaluating Discourse Cohesion in Pre-trained Language Models
di: He, Jie, et al.
Pubblicazione: (2025)

Development of Cognitive Intelligence in Pre-trained Language Models
di: Shah, Raj Sanjay, et al.
Pubblicazione: (2024)

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark
di: Park, Chanjun, et al.
Pubblicazione: (2024)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
di: Zhang, Haode, et al.
Pubblicazione: (2022)

SparseLLM: Towards Global Pruning for Pre-trained Language Models
di: Bai, Guangji, et al.
Pubblicazione: (2024)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
di: Ma, Shengjie, et al.
Pubblicazione: (2025)

Masked Structural Growth for 2x Faster Language Model Pre-training
di: Yao, Yiqun, et al.
Pubblicazione: (2023)

Cross-layer Attention Sharing for Pre-trained Large Language Models
di: Mu, Yongyu, et al.
Pubblicazione: (2024)

Tracr-Injection: Distilling Algorithms into Pre-trained Language Models
di: Vergara-Browne, Tomás, et al.
Pubblicazione: (2025)

Stable Language Model Pre-training by Reducing Embedding Variability
di: Chung, Woojin, et al.
Pubblicazione: (2024)

Establishing Knowledge Preference in Language Models
di: Zhou, Sizhe, et al.
Pubblicazione: (2024)

Anatomical Structure-Guided Medical Vision-Language Pre-training
di: Li, Qingqiu, et al.
Pubblicazione: (2024)

SongSage: A Large Musical Language Model with Lyric Generative Pre-training
di: Guo, Jiani, et al.
Pubblicazione: (2026)

KoCoNovel: Annotated Dataset of Character Coreference in Korean Novels
di: Kim, Kyuhee, et al.
Pubblicazione: (2024)

Can Pre-trained Language Models Understand Chinese Humor?
di: Chen, Yuyan, et al.
Pubblicazione: (2024)

Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models
di: Zhang, Ying, et al.
Pubblicazione: (2024)

Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
di: Tu, Zhijun, et al.
Pubblicazione: (2025)

Decoupled DiLoCo for Resilient Distributed Pre-training
di: Douillard, Arthur, et al.
Pubblicazione: (2026)