Imagen de Portada

On Initializing Transformers with Pre-trained Embeddings

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kim, Ha Young, Balasubramanian, Niranjan, Kang, Byungkon
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language I.2.7
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Pre-training data selection for biomedical domain adaptation using journal impact metrics
por: Laï-king, Mathieu, et al.
Publicado: (2024)

Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing
por: Fernández-González, Daniel, et al.
Publicado: (2026)

Arabic Hate Speech Identification and Masking in Social Media using Deep Learning Models and Pre-trained Models Fine-tuning
por: Doghmash, Salam Thabet, et al.
Publicado: (2025)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
por: Ashuach, Tomer, et al.
Publicado: (2025)

A Multi-Pass Large Language Model Framework for Precise and Efficient Radiology Report Error Detection
por: Kim, Songsoo, et al.
Publicado: (2025)

Influence-driven Curriculum Learning for Pre-training on Limited Data
por: Schoenegger, Loris, et al.
Publicado: (2025)

Revisiting Word Embeddings in the LLM Era
por: Mahajan, Yash, et al.
Publicado: (2024)

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
por: Özeren, Enes, et al.
Publicado: (2025)

Breaking Free Transformer Models: Task-specific Context Attribution Promises Improved Generalizability Without Fine-tuning Pre-trained LLMs
por: Tytarenko, Stepan, et al.
Publicado: (2024)

Towards Effective and Efficient Continual Pre-training of Large Language Models
por: Chen, Jie, et al.
Publicado: (2024)

Benchmarking the Performance of Pre-trained LLMs across Urdu NLP Tasks
por: Tahir, Munief Hassan, et al.
Publicado: (2024)

Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion
por: Xu, Wenjie, et al.
Publicado: (2023)

SynSym: A Synthetic Data Generation Framework for Psychiatric Symptom Identification
por: Kang, Migyeong, et al.
Publicado: (2026)

Zero-Shot Spam Email Classification Using Pre-trained Large Language Models
por: Rojas-Galeano, Sergio
Publicado: (2024)

LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output
por: Karinshak, Elise, et al.
Publicado: (2024)

Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
por: He, Yanjin, et al.
Publicado: (2025)

WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
por: Tian, Changxin, et al.
Publicado: (2025)

Layer-Aware Embedding Fusion for LLMs in Text Classifications
por: Gwak, Jiho, et al.
Publicado: (2025)

idT5: Indonesian Version of Multilingual T5 Transformer
por: Fuadi, Mukhlish, et al.
Publicado: (2023)

Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweets
por: Bilehsavar, Meysam Shirdel, et al.
Publicado: (2025)

myNER: Contextualized Burmese Named Entity Recognition with Bidirectional LSTM and fastText Embeddings via Joint Training with POS Tagging
por: Thant, Kaung Lwin, et al.
Publicado: (2025)

Combining Denoising Autoencoders with Contrastive Learning to fine-tune Transformer Models
por: Lopez-Avila, Alejo, et al.
Publicado: (2024)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
por: Saji, Alan, et al.
Publicado: (2025)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
por: Peters, Sydney, et al.
Publicado: (2025)

Linguistic Interpretability of Transformer-based Language Models: a systematic review
por: López-Otal, Miguel, et al.
Publicado: (2025)

A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models
por: Balasubramanian, Sriram, et al.
Publicado: (2025)

Comparing Complex Concepts with Transformers: Matching Patent Claims Against Natural Language Text
por: Blume, Matthias, et al.
Publicado: (2024)

Heidelberg-Boston @ SIGTYP 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers
por: Riemenschneider, Frederick, et al.
Publicado: (2024)

Please Make it Sound like Human: Encoder-Decoder vs. Decoder-Only Transformers for AI-to-Human Text Style Transfer
por: Paneru, Utsav
Publicado: (2026)

COA-GPT: Generative Pre-trained Transformers for Accelerated Course of Action Development in Military Operations
por: Goecks, Vinicius G., et al.
Publicado: (2024)

Efficient Toxicity Detection in Gaming Chats: A Comparative Study of Embeddings, Fine-Tuned Transformers and LLMs
por: Tereshchenko, Yehor, et al.
Publicado: (2025)

Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
por: Collado-Montañez, Jaime, et al.
Publicado: (2025)

SeLeRoSa: Sentence-Level Romanian Satire Detection Dataset
por: Smădu, Răzvan-Alexandru, et al.
Publicado: (2025)

SentiCSE: A Sentiment-aware Contrastive Sentence Embedding Framework with Sentiment-guided Textual Similarity
por: Kim, Jaemin, et al.
Publicado: (2024)

Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models
por: Lee, Hwiyeong, et al.
Publicado: (2025)

ML-Promise: A Multilingual Dataset for Corporate Promise Verification
por: Seki, Yohei, et al.
Publicado: (2024)

Character-Level Transformer for Tajik-Persian Transliteration with a Parallel Lexical Corpus
por: Arabov, Mullosharaf K.
Publicado: (2026)

HumanLLM: Benchmarking and Improving LLM Anthropomorphism via Human Cognitive Patterns
por: Wang, Xintao, et al.
Publicado: (2026)

Enhancing Emotion Prediction in News Headlines: Insights from ChatGPT and Seq2Seq Models for Free-Text Generation
por: Gao, Ge, et al.
Publicado: (2024)

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity
por: Pan, Xinghan
Publicado: (2025)