Imagen de Portada

Bag of Lies: Robustness in Continuous Pre-training BERT

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Gevers, Ine, Daelemans, Walter
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Do You Get the Hint? Benchmarking LLMs on the Board Game Concept
por: Gevers, Ine, et al.
Publicado: (2025)

WinoWhat: A Parallel Corpus of Paraphrased WinoGrande Sentences with Common Sense Categorization
por: Gevers, Ine, et al.
Publicado: (2025)

ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models
por: Chaffin, Antoine, et al.
Publicado: (2026)

FaBERT: Pre-training BERT on Persian Blogs
por: Masumi, Mostafa, et al.
Publicado: (2024)

Pre-training technique to localize medical BERT and enhance biomedical BERT
por: Wada, Shoya, et al.
Publicado: (2020)

BEIR-NL: Zero-shot Information Retrieval Benchmark for the Dutch Language
por: Banar, Nikolay, et al.
Publicado: (2024)

TempCharBERT: Keystroke Dynamics for Continuous Access Control Based on Pre-trained Language Models
por: Simão, Matheus, et al.
Publicado: (2024)

CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data
por: Touchent, Rian, et al.
Publicado: (2023)

Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Task
por: Zhao, Jinxu, et al.
Publicado: (2024)

Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval
por: Ma, Guangyuan, et al.
Publicado: (2024)

PersonalityChat: Conversation Distillation for Personalized Dialog Modeling with Facts and Traits
por: Lotfi, Ehsan, et al.
Publicado: (2024)

Bilingual BSARD: Extending Statutory Article Retrieval to Dutch
por: Lotfi, Ehsan, et al.
Publicado: (2024)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
por: Zhang, Ying, et al.
Publicado: (2024)

MaiBERT: A Pre-training Corpus and Language Model for Low-Resourced Maithili Language
por: Yadav, Sumit, et al.
Publicado: (2025)

Scaling Agents via Continual Pre-training
por: Su, Liangcai, et al.
Publicado: (2025)

One Size Does Not Fit All: Exploring Variable Thresholds for Distance-Based Multi-Label Text Classification
por: Van Nooten, Jens, et al.
Publicado: (2025)

Applying Pre-trained Multilingual BERT in Embeddings for Improved Malicious Prompt Injection Attacks Detection
por: Rahman, Md Abdur, et al.
Publicado: (2024)

Efficient Continual Pre-training by Mitigating the Stability Gap
por: Guo, Yiduo, et al.
Publicado: (2024)

MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations
por: Yadav, Hemant, et al.
Publicado: (2024)

DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures
por: Uthayasooriyar, Benno, et al.
Publicado: (2025)

An Agentic AI Framework for Training General Practitioner Student Skills
por: De Marez, Victor, et al.
Publicado: (2025)

MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch
por: Banar, Nikolay, et al.
Publicado: (2025)

Revealing the Learning Dynamics of Long-Context Continual Pre-training
por: Liang, Yupu, et al.
Publicado: (2026)

Improving Continual Pre-training Through Seamless Data Packing
por: Yin, Ruicheng, et al.
Publicado: (2025)

Chinese MentalBERT: Domain-Adaptive Pre-training on Social Media for Chinese Mental Health Text Analysis
por: Zhai, Wei, et al.
Publicado: (2024)

LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library
por: Yu, Tianhao, et al.
Publicado: (2024)

ManufactuBERT: Efficient Continual Pretraining for Manufacturing
por: Armingaud, Robin, et al.
Publicado: (2025)

DomURLs_BERT: Pre-trained BERT-based Model for Malicious Domains and URLs Detection and Classification
por: Mahdaouy, Abdelkader El, et al.
Publicado: (2024)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
por: Li, Chen-An, et al.
Publicado: (2024)

IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method
por: Kim, Mihyeon, et al.
Publicado: (2025)

From Robustness to Improved Generalization and Calibration in Pre-trained Language Models
por: Jukić, Josip, et al.
Publicado: (2024)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
por: Xie, Yong, et al.
Publicado: (2023)

Efficient Continual Pre-training of LLMs for Low-resource Languages
por: Nag, Arijit, et al.
Publicado: (2024)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
por: Yan, Junbing, et al.
Publicado: (2024)

Efficient Training of Robust Traditional Chinese LLaMA-1B on a Single Consumer GPU: Continual Pre-training, SFT, and DPO
por: Chih, Yu-Cheng, et al.
Publicado: (2025)

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
por: Luo, Zheheng, et al.
Publicado: (2024)

A Comparative Study of Pre-training and Self-training
por: Wang, Yiheng, et al.
Publicado: (2024)

On Predicting the Post-training Potential of Pre-trained LLMs
por: Li, Xiaoyuan, et al.
Publicado: (2026)

Vi-Mistral-X: Building a Vietnamese Language Model with Advanced Continual Pre-training
por: Vo, James
Publicado: (2024)

Bagging-Based Model Merging for Robust General Text Embeddings
por: Zhang, Hengran, et al.
Publicado: (2026)