Copertina

On Initializing Transformers with Pre-trained Embeddings

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kim, Ha Young, Balasubramanian, Niranjan, Kang, Byungkon
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language I.2.7
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Pre-training data selection for biomedical domain adaptation using journal impact metrics
di: Laï-king, Mathieu, et al.
Pubblicazione: (2024)

Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing
di: Fernández-González, Daniel, et al.
Pubblicazione: (2026)

Arabic Hate Speech Identification and Masking in Social Media using Deep Learning Models and Pre-trained Models Fine-tuning
di: Doghmash, Salam Thabet, et al.
Pubblicazione: (2025)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
di: Ashuach, Tomer, et al.
Pubblicazione: (2025)

A Multi-Pass Large Language Model Framework for Precise and Efficient Radiology Report Error Detection
di: Kim, Songsoo, et al.
Pubblicazione: (2025)

Influence-driven Curriculum Learning for Pre-training on Limited Data
di: Schoenegger, Loris, et al.
Pubblicazione: (2025)

Revisiting Word Embeddings in the LLM Era
di: Mahajan, Yash, et al.
Pubblicazione: (2024)

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
di: Özeren, Enes, et al.
Pubblicazione: (2025)

Breaking Free Transformer Models: Task-specific Context Attribution Promises Improved Generalizability Without Fine-tuning Pre-trained LLMs
di: Tytarenko, Stepan, et al.
Pubblicazione: (2024)

Towards Effective and Efficient Continual Pre-training of Large Language Models
di: Chen, Jie, et al.
Pubblicazione: (2024)

Benchmarking the Performance of Pre-trained LLMs across Urdu NLP Tasks
di: Tahir, Munief Hassan, et al.
Pubblicazione: (2024)

Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion
di: Xu, Wenjie, et al.
Pubblicazione: (2023)

SynSym: A Synthetic Data Generation Framework for Psychiatric Symptom Identification
di: Kang, Migyeong, et al.
Pubblicazione: (2026)

Zero-Shot Spam Email Classification Using Pre-trained Large Language Models
di: Rojas-Galeano, Sergio
Pubblicazione: (2024)

LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output
di: Karinshak, Elise, et al.
Pubblicazione: (2024)

Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
di: He, Yanjin, et al.
Pubblicazione: (2025)

WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
di: Tian, Changxin, et al.
Pubblicazione: (2025)

Layer-Aware Embedding Fusion for LLMs in Text Classifications
di: Gwak, Jiho, et al.
Pubblicazione: (2025)

idT5: Indonesian Version of Multilingual T5 Transformer
di: Fuadi, Mukhlish, et al.
Pubblicazione: (2023)

Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweets
di: Bilehsavar, Meysam Shirdel, et al.
Pubblicazione: (2025)

myNER: Contextualized Burmese Named Entity Recognition with Bidirectional LSTM and fastText Embeddings via Joint Training with POS Tagging
di: Thant, Kaung Lwin, et al.
Pubblicazione: (2025)

Combining Denoising Autoencoders with Contrastive Learning to fine-tune Transformer Models
di: Lopez-Avila, Alejo, et al.
Pubblicazione: (2024)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
di: Saji, Alan, et al.
Pubblicazione: (2025)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
di: Peters, Sydney, et al.
Pubblicazione: (2025)

Linguistic Interpretability of Transformer-based Language Models: a systematic review
di: López-Otal, Miguel, et al.
Pubblicazione: (2025)

A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models
di: Balasubramanian, Sriram, et al.
Pubblicazione: (2025)

Comparing Complex Concepts with Transformers: Matching Patent Claims Against Natural Language Text
di: Blume, Matthias, et al.
Pubblicazione: (2024)

Heidelberg-Boston @ SIGTYP 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers
di: Riemenschneider, Frederick, et al.
Pubblicazione: (2024)

Please Make it Sound like Human: Encoder-Decoder vs. Decoder-Only Transformers for AI-to-Human Text Style Transfer
di: Paneru, Utsav
Pubblicazione: (2026)

COA-GPT: Generative Pre-trained Transformers for Accelerated Course of Action Development in Military Operations
di: Goecks, Vinicius G., et al.
Pubblicazione: (2024)

Efficient Toxicity Detection in Gaming Chats: A Comparative Study of Embeddings, Fine-Tuned Transformers and LLMs
di: Tereshchenko, Yehor, et al.
Pubblicazione: (2025)

Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
di: Collado-Montañez, Jaime, et al.
Pubblicazione: (2025)

SeLeRoSa: Sentence-Level Romanian Satire Detection Dataset
di: Smădu, Răzvan-Alexandru, et al.
Pubblicazione: (2025)

SentiCSE: A Sentiment-aware Contrastive Sentence Embedding Framework with Sentiment-guided Textual Similarity
di: Kim, Jaemin, et al.
Pubblicazione: (2024)

Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models
di: Lee, Hwiyeong, et al.
Pubblicazione: (2025)

ML-Promise: A Multilingual Dataset for Corporate Promise Verification
di: Seki, Yohei, et al.
Pubblicazione: (2024)

Character-Level Transformer for Tajik-Persian Transliteration with a Parallel Lexical Corpus
di: Arabov, Mullosharaf K.
Pubblicazione: (2026)

HumanLLM: Benchmarking and Improving LLM Anthropomorphism via Human Cognitive Patterns
di: Wang, Xintao, et al.
Pubblicazione: (2026)

Enhancing Emotion Prediction in News Headlines: Insights from ChatGPT and Seq2Seq Models for Free-Text Generation
di: Gao, Ge, et al.
Pubblicazione: (2024)

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity
di: Pan, Xinghan
Pubblicazione: (2025)