Copertina

Stable Language Model Pre-training by Reducing Embedding Variability

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chung, Woojin, Hong, Jiwoo, An, Na Min, Thorne, James, Yun, Se-Young
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training
di: Chung, Woojin, et al.
Pubblicazione: (2025)

On the Robustness of Reward Models for Language Model Alignment
di: Hong, Jiwoo, et al.
Pubblicazione: (2025)

ORPO: Monolithic Preference Optimization without Reference Model
di: Hong, Jiwoo, et al.
Pubblicazione: (2024)

Evaluating the Consistency of LLM Evaluators
di: Lee, Noah, et al.
Pubblicazione: (2024)

Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning
di: Son, Guijin, et al.
Pubblicazione: (2025)

Cross-lingual Transfer of Reward Models in Multilingual Alignment
di: Hong, Jiwoo, et al.
Pubblicazione: (2024)

On Initializing Transformers with Pre-trained Embeddings
di: Kim, Ha Young, et al.
Pubblicazione: (2024)

DEPT: Decoupled Embeddings for Pre-training Language Models
di: Iacob, Alex, et al.
Pubblicazione: (2024)

Epistemology of Language Models: Do Language Models Have Holistic Knowledge?
di: Kim, Minsu, et al.
Pubblicazione: (2024)

Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
di: Geng, Yuxia, et al.
Pubblicazione: (2023)

Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions
di: Kim, Taehyeon, et al.
Pubblicazione: (2023)

Block Transformer: Global-to-Local Language Modeling for Fast Inference
di: Ho, Namgyu, et al.
Pubblicazione: (2024)

Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models
di: Zhang, Ying, et al.
Pubblicazione: (2024)

Benchmarks Are Not That Out of Distribution: Word Overlap Predicts Performance
di: Chung, Woojin, et al.
Pubblicazione: (2026)

Guiding Reasoning in Small Language Models with LLM Assistance
di: Kim, Yujin, et al.
Pubblicazione: (2025)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
di: Sharma, Kartik, et al.
Pubblicazione: (2025)

Vi-Mistral-X: Building a Vietnamese Language Model with Advanced Continual Pre-training
di: Vo, James
Pubblicazione: (2024)

DistiLLM: Towards Streamlined Distillation for Large Language Models
di: Ko, Jongwoo, et al.
Pubblicazione: (2024)

Teaching Language Models to Think in Code
di: Hwang, Hyeon, et al.
Pubblicazione: (2026)

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
di: Liu, Yan, et al.
Pubblicazione: (2024)

Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training
di: Zhang, Longhui, et al.
Pubblicazione: (2024)

ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
di: Chen, Tong, et al.
Pubblicazione: (2025)

CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model
di: Chiang, Shang-Hsuan, et al.
Pubblicazione: (2024)

Multi-Drafter Speculative Decoding with Alignment Feedback
di: Kim, Taehyeon, et al.
Pubblicazione: (2026)

Development of Cognitive Intelligence in Pre-trained Language Models
di: Shah, Raj Sanjay, et al.
Pubblicazione: (2024)

Metadata Conditioning Accelerates Language Model Pre-training
di: Gao, Tianyu, et al.
Pubblicazione: (2025)

Evaluating Discourse Cohesion in Pre-trained Language Models
di: He, Jie, et al.
Pubblicazione: (2025)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
di: Zhang, Haode, et al.
Pubblicazione: (2022)

RegMix: Data Mixture as Regression for Language Model Pre-training
di: Liu, Qian, et al.
Pubblicazione: (2024)

Text Embeddings by Weakly-Supervised Contrastive Pre-training
di: Wang, Liang, et al.
Pubblicazione: (2022)

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts
di: Zang, Yuan, et al.
Pubblicazione: (2024)

Tracr-Injection: Distilling Algorithms into Pre-trained Language Models
di: Vergara-Browne, Tomás, et al.
Pubblicazione: (2025)

Bayesian Preference Learning for Test-Time Steerable Reward Models
di: Hong, Jiwoo, et al.
Pubblicazione: (2026)

Model Merging in Pre-training of Large Language Models
di: Li, Yunshui, et al.
Pubblicazione: (2025)

Context Filtering with Reward Modeling in Question Answering
di: Kim, Sangryul, et al.
Pubblicazione: (2024)

Knowledge Graphs and Pre-trained Language Models enhanced Representation Learning for Conversational Recommender Systems
di: Qiu, Zhangchi, et al.
Pubblicazione: (2023)

Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
di: Kim, Woojin, et al.
Pubblicazione: (2025)

When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts
di: Kim, Jun Seong, et al.
Pubblicazione: (2025)

Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer
di: Lee, Seungyoon, et al.
Pubblicazione: (2025)

Probing Language Models for Pre-training Data Detection
di: Liu, Zhenhua, et al.
Pubblicazione: (2024)