Cover Image

Chinese ModernBERT with Whole-Word Masking

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Zeyu, Wang, Ningtao, Fu, Xing, Cheng, Yu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Patent Language Model Pretraining with ModernBERT
by: Yousefiramandi, Amirhossein, et al.
Published: (2025)

Clinical ModernBERT: An efficient and long context encoder for biomedical text
by: Lee, Simon A., et al.
Published: (2025)

NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus
by: Silva, Enzo S. N., et al.
Published: (2026)

BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP
by: Sounack, Thomas, et al.
Published: (2025)

Improved Personalized Headline Generation via Denoising Fake Interests from Implicit Feedback
by: Liu, Kejin, et al.
Published: (2025)

Mask-guided BERT for Few Shot Text Classification
by: Liao, Wenxiong, et al.
Published: (2023)

ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models
by: Chen, Hao, et al.
Published: (2025)

TabiBERT: A Large-Scale ModernBERT Foundation Model and A Unified Benchmark for Turkish
by: Türker, Melikşah, et al.
Published: (2025)

BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining
by: Liang, Wen, et al.
Published: (2024)

CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling
by: Fang, Zheng, et al.
Published: (2023)

ModernBERT + ColBERT: Enhancing biomedical RAG through an advanced re-ranking retriever
by: Rivera, Eduardo Martínez, et al.
Published: (2025)

AraModernBERT: Transtokenized Initialization and Long-Context Encoder Modeling for Arabic
by: Elshehy, Omar, et al.
Published: (2026)

ModernBERT is More Efficient than Conventional BERT for Chest CT Findings Classification in Japanese Radiology Reports
by: Yamagishi, Yosuke, et al.
Published: (2025)

Table as a Modality for Large Language Models
by: Li, Liyao, et al.
Published: (2025)

Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry
by: Wang, Shanshan, et al.
Published: (2025)

Pretraining Finnish ModernBERTs
by: Reunamo, Akseli, et al.
Published: (2025)

Can ChatGPT Really Understand Modern Chinese Poetry?
by: Wang, Shanshan, et al.
Published: (2026)

Incremental and Data-Efficient Concept Formation to Support Masked Word Prediction
by: Lian, Xin, et al.
Published: (2024)

It's All in The [MASK]: Simple Instruction-Tuning Enables BERT-like Masked Language Models As Generative Classifiers
by: Clavié, Benjamin, et al.
Published: (2025)

A Diversity Diet for a Healthier Model: A Case Study of French ModernBERT
by: Estève, Louis, et al.
Published: (2026)

NeoBERT: A Next-Generation BERT
by: Breton, Lola Le, et al.
Published: (2025)

Selecting Between BERT and GPT for Text Classification in Political Science Research
by: Wang, Yu, et al.
Published: (2024)

Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale
by: Javis AI Team, et al.
Published: (2025)

MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation
by: Tamayo, Daniel, et al.
Published: (2026)

MMBERT: Scaled Mixture-of-Experts Multimodal BERT for Robust Chinese Hate Speech Detection under Cloaking Perturbations
by: Xue, Qiyao, et al.
Published: (2025)

IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method
by: Kim, Mihyeon, et al.
Published: (2025)

KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts
by: Autenried, Christian, et al.
Published: (2026)

StableMask: Refining Causal Masking in Decoder-only Transformer
by: Yin, Qingyu, et al.
Published: (2024)

A Comparative Analysis of Word Segmentation, Part-of-Speech Tagging, and Named Entity Recognition for Historical Chinese Sources, 1900-1950
by: Fang, Zhao, et al.
Published: (2025)

KuBERT: Central Kurdish BERT Model and Its Application for Sentiment Analysis
by: Awlla, Kozhin muhealddin, et al.
Published: (2025)

Analyzing Multi-Head Attention on Trojan BERT Models
by: Wang, Jingwei
Published: (2024)

LB-KBQA: Large-language-model and BERT based Knowledge-Based Question and Answering System
by: Zhao, Yan, et al.
Published: (2024)

ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance
by: Antoun, Wissam, et al.
Published: (2025)

Ensemble BERT: A student social network text sentiment classification model based on ensemble learning and BERT architecture
by: Jiang, Kai, et al.
Published: (2024)

RAG-based Crowdsourcing Task Decomposition via Masked Contrastive Learning with Prompts
by: Yang, Jing, et al.
Published: (2024)

Semantic Similarity Matching for Patent Documents Using Ensemble BERT-related Model and Novel Text Processing Method
by: Yu, Liqiang, et al.
Published: (2024)

Word Order and World Knowledge
by: Zhao, Qinghua, et al.
Published: (2024)

Unitary Multi-Margin BERT for Robust Natural Language Processing
by: Chang, Hao-Yuan, et al.
Published: (2024)

Optimized Biomedical Question-Answering Services with LLM and Multi-BERT Integration
by: Qian, Cheng, et al.
Published: (2024)

CNsum:Automatic Summarization for Chinese News Text
by: Zhao, Yu, et al.
Published: (2025)