Cover Image

Clinical ModernBERT: An efficient and long context encoder for biomedical text

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Simon A., Wu, Anthony, Chiang, Jeffrey N.
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Patent Language Model Pretraining with ModernBERT
by: Yousefiramandi, Amirhossein, et al.
Published: (2025)

Chinese ModernBERT with Whole-Word Masking
by: Zhao, Zeyu, et al.
Published: (2025)

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
by: Reddy, K. Sahit, et al.
Published: (2025)

Advancing Chinese biomedical text mining with community challenges
by: Zong, Hui, et al.
Published: (2024)

BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP
by: Sounack, Thomas, et al.
Published: (2025)

Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representation
by: Lee, Simon A., et al.
Published: (2024)

NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus
by: Silva, Enzo S. N., et al.
Published: (2026)

MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation
by: Tamayo, Daniel, et al.
Published: (2026)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
by: Yang, Yu, et al.
Published: (2024)

ModernBERT + ColBERT: Enhancing biomedical RAG through an advanced re-ranking retriever
by: Rivera, Eduardo Martínez, et al.
Published: (2025)

A Fusion of context-aware based BanglaBERT and Two-Layer Stacked LSTM Framework for Multi-Label Cyberbullying Detection
by: Raquib, Mirza, et al.
Published: (2026)

Injecting linguistic knowledge into BERT for Dialogue State Tracking
by: Feng, Xiaohan, et al.
Published: (2023)

Sub-SA: Strengthen In-context Learning via Submodular Selective Annotation
by: Qian, Jian, et al.
Published: (2024)

Adjoint sharding for very long context training of state space models
by: Xu, Xingzi, et al.
Published: (2025)

Cartridges: Lightweight and general-purpose long context representations via self-study
by: Eyuboglu, Sabri, et al.
Published: (2025)

BERT-LSH: Reducing Absolute Compute For Attention
by: Li, Zezheng, et al.
Published: (2024)

Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models
by: Wu, Da, et al.
Published: (2023)

Implicit In-context Learning
by: Li, Zhuowei, et al.
Published: (2024)

BERT Learns (and Teaches) Chemistry
by: Payne, Josh, et al.
Published: (2020)

BERT-JEPA: Reorganizing CLS Embeddings for Language-Invariant Semantics
by: Gillin, Taj, et al.
Published: (2026)

Feature Structure Distillation with Centered Kernel Alignment in BERT Transferring
by: Jung, Hee-Jun, et al.
Published: (2022)

A Comprehensive Approach to Misspelling Correction with BERT and Levenshtein Distance
by: Naziri, Amirreza, et al.
Published: (2024)

Scaling BERT Models for Turkish Automatic Punctuation and Capitalization Correction
by: Saoud, Abdulkader, et al.
Published: (2024)

BERTCaps: BERT Capsule for Persian Multi-Domain Sentiment Analysis
by: Memari, Mohammadali, et al.
Published: (2024)

2-Tier SimCSE: Elevating BERT for Robust Sentence Embeddings
by: Wang, Yumeng, et al.
Published: (2025)

Seeing Through VisualBERT: A Causal Adventure on Memetic Landscapes
by: Bandyopadhyay, Dibyanayan, et al.
Published: (2024)

Toward Understanding BERT-Like Pre-Training for DNA Foundation Models
by: Liang, Chaoqi, et al.
Published: (2023)

Improving VTE Identification through Language Models from Radiology Reports: A Comparative Study of Mamba, Phi-3 Mini, and BERT
by: Deng, Jamie, et al.
Published: (2024)

Single layer tiny Co$^4$ outpaces GPT-2 and GPT-BERT
by: Zain, Noor Ul, et al.
Published: (2025)

BERT-ASC: Auxiliary-Sentence Construction for Implicit Aspect Learning in Sentiment Analysis
by: Ahmed, Murtadha, et al.
Published: (2022)

AI-Generated Text Detection and Classification Based on BERT Deep Learning Algorithm
by: Wang, Hao, et al.
Published: (2024)

Fresh in memory: Training-order recency is linearly encoded in language model activations
by: Krasheninnikov, Dmitrii, et al.
Published: (2025)

Generalized knowledge-enhanced framework for biomedical entity and relation extraction
by: Nguyen, Minh, et al.
Published: (2024)

In-context Time Series Predictor
by: Lu, Jiecheng, et al.
Published: (2024)

Stylometry recognizes human and LLM-generated texts in short samples
by: Przystalski, Karol, et al.
Published: (2025)

Effects of term weighting approach with and without stop words removing on Arabic text classification
by: Alhenawi, Esra'a, et al.
Published: (2024)

Harnessing Large Language Models: Fine-tuned BERT for Detecting Charismatic Leadership Tactics in Natural Language
by: Saeid, Yasser, et al.
Published: (2024)

Cancer Diagnosis Categorization in Electronic Health Records Using Large Language Models and BioBERT: Model Performance Evaluation Study
by: Hashtarkhani, Soheil, et al.
Published: (2025)

CreditAudit: 2$^\text{nd}$ Dimension for LLM Evaluation and Selection
by: Song, Yiliang, et al.
Published: (2026)

Mechanistic Fine-tuning for In-context Learning
by: Cho, Hakaze, et al.
Published: (2025)