Cover Image

m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yaoxiang, Zuo, Simiao, Hu, Qingguo, Ding, Yucheng, Gong, Yeyun, Jiao, Jian, Su, Jinsong
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization
by: Wang, Yaoxiang, et al.
Published: (2025)

TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation
by: Wang, Yaoxiang, et al.
Published: (2024)

Task Oriented In-Domain Data Augmentation
by: Liang, Xiao, et al.
Published: (2024)

VeriAgent: A Tool-Integrated Multi-Agent System with Evolving Memory for PPA-Aware RTL Code Generation
by: Wang, Yaoxiang, et al.
Published: (2026)

Cross-lingual Matryoshka Representation Learning across Speech and Text
by: Sy, Yaya, et al.
Published: (2026)

Mixture of Neuron Experts
by: Cheng, Runxi, et al.
Published: (2025)

RexBERT: Context Specialized Bidirectional Encoders for E-commerce
by: Bajaj, Rahul, et al.
Published: (2026)

MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining
by: Portes, Jacob, et al.
Published: (2023)

Sigma-MoE-Tiny Technical Report
by: Hu, Qingguo, et al.
Published: (2025)

CLM-Bench: Benchmarking and Analyzing Cross-lingual Misalignment of LLMs in Knowledge Editing
by: Hu, Yucheng, et al.
Published: (2026)

AraModernBERT: Transtokenized Initialization and Long-Context Encoder Modeling for Arabic
by: Elshehy, Omar, et al.
Published: (2026)

ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance
by: Antoun, Wissam, et al.
Published: (2025)

mmBERT: A Modern Multilingual Encoder with Annealed Language Learning
by: Marone, Marc, et al.
Published: (2025)

MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation
by: Tamayo, Daniel, et al.
Published: (2026)

BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP
by: Sounack, Thomas, et al.
Published: (2025)

Geometric Patterns of Meaning: A PHATE Manifold Analysis of Multi-lingual Embeddings
by: Gong, Wen G
Published: (2025)

Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation
by: Xiao, Meiman, et al.
Published: (2026)

Feeding LLM Annotations to BERT Classifiers at Your Own Risk
by: Lu, Yucheng, et al.
Published: (2025)

DeepThink: Aligning Language Models with Domain-Specific User Intents
by: Li, Yang, et al.
Published: (2025)

Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Models
by: Elshabrawy, Ahmed, et al.
Published: (2025)

Multilingual BERT language model for medical tasks: Evaluation on domain-specific adaptation and cross-linguality
by: Luo, Yinghao, et al.
Published: (2025)

APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning
by: Sun, Jiashuo, et al.
Published: (2022)

Chinese ModernBERT with Whole-Word Masking
by: Zhao, Zeyu, et al.
Published: (2025)

FinBERT2: A Specialized Bidirectional Encoder for Bridging the Gap in Finance-Specific Deployment of Large Language Models
by: Xu, Xuan, et al.
Published: (2025)

EuroBERT: Scaling Multilingual Encoders for European Languages
by: Boizard, Nicolas, et al.
Published: (2025)

Selected Languages are All You Need for Cross-lingual Truthfulness Transfer
by: Liu, Weihao, et al.
Published: (2024)

Fine-tuning the SwissBERT Encoder Model for Embedding Sentences and Documents
by: Grosjean, Juri, et al.
Published: (2024)

Pretraining and Benchmarking Modern Encoders for Latvian
by: Znotins, Arturs
Published: (2026)

Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
by: Warner, Benjamin, et al.
Published: (2024)

DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks
by: He, Jianing, et al.
Published: (2024)

Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Prompts
by: Wang, Teng, et al.
Published: (2024)

TabiBERT: A Large-Scale ModernBERT Foundation Model and A Unified Benchmark for Turkish
by: Türker, Melikşah, et al.
Published: (2025)

Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs?
by: Zhang, Junyan, et al.
Published: (2025)

$μ$KE: Matryoshka Unstructured Knowledge Editing of Large Language Models
by: Su, Zian, et al.
Published: (2025)

An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
by: Nguyen, Ba-Quang
Published: (2025)

2D Matryoshka Training for Information Retrieval
by: Wang, Shuai, et al.
Published: (2024)

Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning
by: Qin, Chengwei, et al.
Published: (2023)

ModernBERT is More Efficient than Conventional BERT for Chest CT Findings Classification in Japanese Radiology Reports
by: Yamagishi, Yosuke, et al.
Published: (2025)

HiCaM: A Hierarchical-Causal Modification Framework for Long-Form Text Modification
by: Shi, Yuntao, et al.
Published: (2025)

Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning
by: Park, Sungjin, et al.
Published: (2024)