Image de couverture de livre

Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Ying, Li, Dongyuan, Okumura, Manabu
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders
par: Lin, Ailiang, et autres
Publié: (2026)

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
par: Lin, Ailiang, et autres
Publié: (2025)

Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition
par: Li, Dongyuan, et autres
Publié: (2024)

Inside the Black Box: Detecting Data Leakage in Pre-trained Language Encoders
par: Xin, Yuan, et autres
Publié: (2024)

On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generation
par: Wu, Di, et autres
Publié: (2024)

Text to Band Gap: Pre-trained Language Models as Encoders for Semiconductor Band Gap Prediction
par: Yeh, Ying-Ting, et autres
Publié: (2025)

Stable Language Model Pre-training by Reducing Embedding Variability
par: Chung, Woojin, et autres
Publié: (2024)

DEPT: Decoupled Embeddings for Pre-training Language Models
par: Iacob, Alex, et autres
Publié: (2024)

Can we obtain significant success in RST discourse parsing by using Large Language Models?
par: Maekawa, Aru, et autres
Publié: (2024)

Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
par: Geng, Yuxia, et autres
Publié: (2023)

Length Representations in Large Language Models
par: Moon, Sangjun, et autres
Publié: (2025)

Understanding Fact Recall in Language Models: Why Two-Stage Training Encourages Memorization but Mixed Training Teaches Knowledge
par: Zhang, Ying, et autres
Publié: (2025)

InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models
par: Juseon-Do, et autres
Publié: (2024)

DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation
par: Maekawa, Aru, et autres
Publié: (2024)

NITP: Next Implicit Token Prediction for LLM Pre-training
par: Zhang, Xiangdong, et autres
Publié: (2026)

Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pre-trained Models
par: Purason, Taido, et autres
Publié: (2025)

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification
par: Calbucura, Nicolas, et autres
Publié: (2025)

Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens
par: Zhang, Anqi, et autres
Publié: (2024)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
par: Zhang, Ying, et autres
Publié: (2024)

Automatic Answerability Evaluation for Question Generation
par: Wang, Zifan, et autres
Publié: (2023)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
par: Sharma, Kartik, et autres
Publié: (2025)

Fuzzy Fingerprinting Encoder Pre-trained Language Models for Emotion Recognition in Conversations: Human Assessment and Validity Study
par: Pereira, Patrícia, et autres
Publié: (2026)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
par: Zhang, Haode, et autres
Publié: (2022)

QoSBERT: An Uncertainty-Aware Approach based on Pre-trained Language Models for Service Quality Prediction
par: Wang, Ziliang, et autres
Publié: (2025)

Concept Tokens: Learning Behavioral Embeddings Through Concept Definitions
par: Sastre, Ignacio, et autres
Publié: (2026)

Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation
par: Lyu, Boxuan, et autres
Publié: (2024)

Masked Structural Growth for 2x Faster Language Model Pre-training
par: Yao, Yiqun, et autres
Publié: (2023)

Data Augmentation Method Utilizing Template Sentences for Variable Definition Extraction
par: Nagayama, Kotaro, et autres
Publié: (2024)

On Initializing Transformers with Pre-trained Embeddings
par: Kim, Ha Young, et autres
Publié: (2024)

Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
par: Jin, Yang, et autres
Publié: (2024)

Are Transformers in Pre-trained LM A Good ASR Encoder? An Empirical Study
par: An, Keyu, et autres
Publié: (2024)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
par: Li, Chen-An, et autres
Publié: (2024)

KoCo: Conditioning Language Model Pre-training on Knowledge Coordinates
par: Li, Yudong, et autres
Publié: (2026)

Model Merging in Pre-training of Large Language Models
par: Li, Yunshui, et autres
Publié: (2025)

Development of Cognitive Intelligence in Pre-trained Language Models
par: Shah, Raj Sanjay, et autres
Publié: (2024)

Metadata Conditioning Accelerates Language Model Pre-training
par: Gao, Tianyu, et autres
Publié: (2025)

Evaluating Discourse Cohesion in Pre-trained Language Models
par: He, Jie, et autres
Publié: (2025)

Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
par: Zhuang, Xinlin, et autres
Publié: (2025)

SparseLLM: Towards Global Pruning for Pre-trained Language Models
par: Bai, Guangji, et autres
Publié: (2024)

On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons
par: Kojima, Takeshi, et autres
Publié: (2024)