Imagen de Portada

NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Czinczoll, Tamara, Hönes, Christoph, Schall, Maximilian, de Melo, Gerard
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

CommitBench: A Benchmark for Commit Message Generation
por: Schall, Maximilian, et al.
Publicado: (2024)

Query-Level Uncertainty in Large Language Models
por: Chen, Lihu, et al.
Publicado: (2025)

Learning to Predict Usage Options of Product Reviews with LLM-Generated Labels
por: Kohlenberg, Leo, et al.
Publicado: (2024)

NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning
por: Yi, Xin, et al.
Publicado: (2024)

ChuLo: Chunk-Level Key Information Representation for Long Document Understanding
por: Li, Yan, et al.
Publicado: (2024)

TaxonRL: Reinforcement Learning with Intermediate Rewards for Interpretable Fine-Grained Visual Reasoning
por: von Klinski, Maximilian, et al.
Publicado: (2026)

MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations
por: Yadav, Hemant, et al.
Publicado: (2024)

NeoBERT: A Next-Generation BERT
por: Breton, Lola Le, et al.
Publicado: (2025)

BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining
por: Liang, Wen, et al.
Publicado: (2024)

Adapting Large Language Models for Document-Level Machine Translation
por: Wu, Minghao, et al.
Publicado: (2024)

GraphLSS: Integrating Lexical, Structural, and Semantic Features for Long Document Extractive Summarization
por: Bugueño, Margarita, et al.
Publicado: (2024)

Rethinking Graph-Based Document Classification: Learning Data-Driven Structures Beyond Heuristic Approaches
por: Bugueño, Margarita, et al.
Publicado: (2025)

Are Expert-Level Language Models Expert-Level Annotators?
por: Tseng, Yu-Min, et al.
Publicado: (2024)

Multilingual Contextualization of Large Language Models for Document-Level Machine Translation
por: Ramos, Miguel Moura, et al.
Publicado: (2025)

Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?
por: Bugueño, Margarita, et al.
Publicado: (2023)

SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT
por: Cho, Cheol Jun, et al.
Publicado: (2023)

An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
por: Nguyen, Ba-Quang
Publicado: (2025)

UniBERT: Adversarial Training for Language-Universal Representations
por: Avram, Andrei-Marius, et al.
Publicado: (2025)

DOREMI: Optimizing Long Tail Predictions in Document-Level Relation Extraction
por: Menotti, Laura, et al.
Publicado: (2026)

General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level
por: Shi, Bingkang, et al.
Publicado: (2023)

Representation Deficiency in Masked Language Modeling
por: Meng, Yu, et al.
Publicado: (2023)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
por: Shao, Chenze, et al.
Publicado: (2024)

ANGO: A Next-Level Evaluation Benchmark For Generation-Oriented Language Models In Chinese Domain
por: Wang, Bingchao
Publicado: (2024)

Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learning
por: Cui, Menglong, et al.
Publicado: (2024)

AutoRE: Document-Level Relation Extraction with Large Language Models
por: Xue, Lilong, et al.
Publicado: (2024)

Chinese ModernBERT with Whole-Word Masking
por: Zhao, Zeyu, et al.
Publicado: (2025)

ExpertLongBench: Benchmarking Language Models on Expert-Level Long-Form Generation Tasks with Structured Checklists
por: Ruan, Jie, et al.
Publicado: (2025)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
por: Su, Zhenpeng, et al.
Publicado: (2024)

It's All in The [MASK]: Simple Instruction-Tuning Enables BERT-like Masked Language Models As Generative Classifiers
por: Clavié, Benjamin, et al.
Publicado: (2025)

FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models
por: Dobler, Konstantin, et al.
Publicado: (2023)

K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning
por: Zhang, Yadong, et al.
Publicado: (2024)

Within-Document Event Coreference with BERT-Based Contextualized Representations
por: Ahmed, Shafiuddin Rehan, et al.
Publicado: (2021)

MaBERT:A Padding Safe Interleaved Transformer Mamba Hybrid Encoder for Efficient Extended Context Masked Language Modeling
por: Kim, Jinwoong, et al.
Publicado: (2026)

Enhancing Next-Generation Language Models with Knowledge Graphs: Extending Claude, Mistral IA, and GPT-4 via KG-BERT
por: Chaabene, Nour El Houda Ben, et al.
Publicado: (2025)

Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
por: Dhillon, Paramveer S., et al.
Publicado: (2024)

Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating Multi-Dialect Phoneme-Level BERT
por: Yamauchi, Kazuki, et al.
Publicado: (2024)

Mask-guided BERT for Few Shot Text Classification
por: Liao, Wenxiong, et al.
Publicado: (2023)

SwissBERT: The Multilingual Language Model for Switzerland
por: Vamvas, Jannis, et al.
Publicado: (2023)

A Representation Level Analysis of NMT Model Robustness to Grammatical Errors
por: Issam, Abderrahmane, et al.
Publicado: (2025)

Docopilot: Improving Multimodal Models for Document-Level Understanding
por: Duan, Yuchen, et al.
Publicado: (2025)