Imagen de Portada

ManufactuBERT: Efficient Continual Pretraining for Manufacturing

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Armingaud, Robin, Besançon, Romaric
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

GLiDRE: Generalist Lightweight model for Document-level Relation Extraction
por: Armingaud, Robin, et al.
Publicado: (2025)

From LIMA to DeepLIMA: following a new path of interoperability
por: Bocharov, Victor, et al.
Publicado: (2024)

Patent Language Model Pretraining with ModernBERT
por: Yousefiramandi, Amirhossein, et al.
Publicado: (2025)

EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corpora
por: Qarah, Faisal
Publicado: (2024)

MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining
por: Portes, Jacob, et al.
Publicado: (2023)

Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining
por: Vielzeuf, Valentin
Publicado: (2024)

Efficient Domain-adaptive Continual Pretraining for the Process Industry in the German Language
por: Zhukova, Anastasia, et al.
Publicado: (2025)

RedWhale: An Adapted Korean LLM Through Efficient Continual Pretraining
por: Vo, Anh-Dung, et al.
Publicado: (2024)

ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMs
por: Yoo, HanGyeol, et al.
Publicado: (2026)

AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis
por: Qarah, Faisal
Publicado: (2024)

SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corpora
por: Qarah, Faisal
Publicado: (2024)

Bag of Lies: Robustness in Continuous Pre-training BERT
por: Gevers, Ine, et al.
Publicado: (2024)

PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords
por: Sriwirote, Panyut, et al.
Publicado: (2023)

Explaining How Visual, Textual and Multimodal Encoders Share Concepts
por: Cornet, Clément, et al.
Publicado: (2025)

ModernBERT is More Efficient than Conventional BERT for Chest CT Findings Classification in Japanese Radiology Reports
por: Yamagishi, Yosuke, et al.
Publicado: (2025)

Efficient Pretraining Length Scaling
por: Wu, Bohong, et al.
Publicado: (2025)

LLM Pretraining with Continuous Concepts
por: Tack, Jihoon, et al.
Publicado: (2025)

Positional Attention for Efficient BERT-Based Named Entity Recognition
por: Sun, Mo, et al.
Publicado: (2025)

FaBERT: Pre-training BERT on Persian Blogs
por: Masumi, Mostafa, et al.
Publicado: (2024)

NusaBERT: Teaching IndoBERT to be Multilingual and Multicultural
por: Wongso, Wilson, et al.
Publicado: (2024)

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
por: Liu, Yihong, et al.
Publicado: (2023)

HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition
por: Yoon, Ji Won, et al.
Publicado: (2022)

NeoDictaBERT: Pushing the Frontier of BERT models for Hebrew
por: Shmidman, Shaltiel, et al.
Publicado: (2025)

Symmetric Dot-Product Attention for Efficient Training of BERT Language Models
por: Courtois, Martin, et al.
Publicado: (2024)

QiBERT -- Classifying Online Conversations Messages with BERT as a Feature
por: Ferreira-Saraiva, Bruno D., et al.
Publicado: (2024)

Pre-training technique to localize medical BERT and enhance biomedical BERT
por: Wada, Shoya, et al.
Publicado: (2020)

HRM-Text: Efficient Pretraining Beyond Scaling
por: Wang, Guan, et al.
Publicado: (2026)

NeoBERT: A Next-Generation BERT
por: Breton, Lola Le, et al.
Publicado: (2025)

Investigating Continual Pretraining in Large Language Models: Insights and Implications
por: Yıldız, Çağatay, et al.
Publicado: (2024)

SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation
por: Lv, Changze, et al.
Publicado: (2023)

Leveraging IndoBERT and DistilBERT for Indonesian Emotion Classification in E-Commerce Reviews
por: Christian, William, et al.
Publicado: (2025)

ACL: Aligned Contrastive Learning Improves BERT and Multi-exit BERT Fine-tuning
por: Li, Liz, et al.
Publicado: (2026)

InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline
por: Kugler, Kai, et al.
Publicado: (2021)

PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
por: Zeng, Boyi, et al.
Publicado: (2025)

MuCPT: Music-related Natural Language Model Continued Pretraining
por: Tian, Kai, et al.
Publicado: (2025)

Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMs
por: Christophe, Clément, et al.
Publicado: (2024)

BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM
por: Shen, Zhewen, et al.
Publicado: (2024)

Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models
por: Parmar, Jupinder, et al.
Publicado: (2024)

Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0
por: Huo, Robin, et al.
Publicado: (2025)

An investigation of structures responsible for gender bias in BERT and DistilBERT
por: Leteno, Thibaud, et al.
Publicado: (2024)