Imagen de Portada

New Encoders for German Trained from Scratch: Comparing ModernGBERT with Converted LLM2Vec Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wunderle, Julia, Ehrmanntraut, Anton, Pfister, Jan, Jannidis, Fotis, Hotho, Andreas
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch
por: Pfister, Jan, et al.
Publicado: (2024)

Historical German Text Normalization Using Type- and Token-Based Language Modeling
por: Ehrmanntraut, Anton
Publicado: (2024)

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
por: BehnamGhader, Parishad, et al.
Publicado: (2024)

Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings
por: Li, Shiyu, et al.
Publicado: (2025)

A Comparative Analysis of Bilingual and Trilingual Wav2Vec Models for Automatic Speech Recognition in Multilingual Oral History Archives
por: Lehečka, Jan, et al.
Publicado: (2024)

SozKZ: Training Efficient Small Language Models for Kazakh from Scratch
por: Tukenov, Saken
Publicado: (2026)

AraModernBERT: Transtokenized Initialization and Long-Context Encoder Modeling for Arabic
por: Elshehy, Omar, et al.
Publicado: (2026)

From Scratch to Fine-Tuned: A Comparative Study of Transformer Training Strategies for Legal Machine Translation
por: Barman, Amit, et al.
Publicado: (2025)

Higher-Order DeepTrails: Unified Approach to *Trails
por: Koopmann, Tobias, et al.
Publicado: (2023)

VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks
por: Jiang, Ziyan, et al.
Publicado: (2024)

Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratch
por: Ding, Yuyang, et al.
Publicado: (2024)

Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training
por: Zhang, Mozhi, et al.
Publicado: (2025)

Steel-LLM:From Scratch to Open Source -- A Personal Journey in Building a Chinese-Centric LLM
por: Gu, Qingshui, et al.
Publicado: (2025)

How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapters
por: Oji, Romina, et al.
Publicado: (2025)

Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance
por: Djuhera, Aladin, et al.
Publicado: (2025)

Adaptability of ASR Models on Low-Resource Language: A Comparative Study of Whisper and Wav2Vec-BERT on Bangla
por: Ridoy, Md Sazzadul Islam, et al.
Publicado: (2025)

LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators
por: Lango, Mateusz, et al.
Publicado: (2025)

Spoken Word2Vec: Learning Skipgram Embeddings from Speech
por: Sayeed, Mohammad Amaan, et al.
Publicado: (2023)

PolicyBank: Evolving Policy Understanding for LLM Agents
por: Choi, Jihye, et al.
Publicado: (2026)

Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
por: Shao, Yijia, et al.
Publicado: (2024)

MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation
por: Tamayo, Daniel, et al.
Publicado: (2026)

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
por: Lin, Ailiang, et al.
Publicado: (2025)

Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks
por: Nielsen, Dan Saattrup, et al.
Publicado: (2024)

BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP
por: Sounack, Thomas, et al.
Publicado: (2025)

Chunking German Legal Code
por: Prior, Max, et al.
Publicado: (2026)

FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
por: Ma, Liqun, et al.
Publicado: (2024)

STICKERCONV: Generating Multimodal Empathetic Responses from Scratch
por: Zhang, Yiqun, et al.
Publicado: (2024)

Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection
por: Wang, Haoming, et al.
Publicado: (2025)

Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations
por: Ma, Ziqiao, et al.
Publicado: (2024)

Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch
por: Wen, Xueru, et al.
Publicado: (2025)

LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification
por: Schlee, Michael, et al.
Publicado: (2025)

PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving
por: Parmar, Mihir, et al.
Publicado: (2025)

Backtranslation and paraphrasing in the LLM era? Comparing data augmentation methods for emotion classification
por: Radliński, Łukasz, et al.
Publicado: (2025)

BibSonomy Meets ChatLLMs for Publication Management: From Chat to Publication Management: Organizing your related work using BibSonomy & LLMs
por: Völker, Tom, et al.
Publicado: (2024)

Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch
por: Kim, Hyunwoo, et al.
Publicado: (2026)

Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
por: Warner, Benjamin, et al.
Publicado: (2024)

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
por: Li, Kenneth, et al.
Publicado: (2022)

A Fuzzy Evaluation of Sentence Encoders on Grooming Risk Classification
por: Bihani, Geetanjali, et al.
Publicado: (2025)

Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training
por: Chen, Maximillian, et al.
Publicado: (2024)

GT2Vec: Large Language Models as Multi-Modal Encoders for Text and Graph-Structured Data
por: Lin, Jiacheng, et al.
Publicado: (2024)