Copertina

CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Touchent, Rian, Romary, Laurent, de la Clergerie, Eric
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

CamemBERT 2.0: A Smarter French Language Model Aged to Perfection
di: Antoun, Wissam, et al.
Pubblicazione: (2024)

A Causal Language Modeling Detour Improves Encoder Continued Pretraining
di: Touchent, Rian, et al.
Pubblicazione: (2026)

Gaperon: A Peppered English-French Generative Language Model Suite
di: Godey, Nathan, et al.
Pubblicazione: (2025)

Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content
di: Touchent, Rian, et al.
Pubblicazione: (2025)

On the Scaling Laws of Geographical Representation in Language Models
di: Godey, Nathan, et al.
Pubblicazione: (2024)

Multi-objective Representation for Numbers in Clinical Narratives: A CamemBERT-Bio-Based Alternative to Large-Scale LLMs
di: Lompo, Boammani Aser, et al.
Pubblicazione: (2024)

Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression
di: Godey, Nathan, et al.
Pubblicazione: (2025)

Knowledge-Instruct: Effective Continual Pre-training from Limited Data using Instructions
di: Ovadia, Oded, et al.
Pubblicazione: (2025)

Topic Over Source: The Key to Effective Data Mixing for Language Models Pre-training
di: Peng, Jiahui, et al.
Pubblicazione: (2025)

Is Crowdsourcing Breaking Your Bank? Cost-Effective Fine-Tuning of Pre-trained Language Models with Proximal Policy Optimization
di: Yang, Shuo, et al.
Pubblicazione: (2024)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
di: Ma, Shengjie, et al.
Pubblicazione: (2025)

PatentEval: Understanding Errors in Patent Generation
di: Zuo, You, et al.
Pubblicazione: (2024)

Probing Language Models for Pre-training Data Detection
di: Liu, Zhenhua, et al.
Pubblicazione: (2024)

Optimized Biomedical Question-Answering Services with LLM and Multi-BERT Integration
di: Qian, Cheng, et al.
Pubblicazione: (2024)

DataMan: Data Manager for Pre-training Large Language Models
di: Peng, Ru, et al.
Pubblicazione: (2025)

EQ-5D Classification Using Biomedical Entity-Enriched Pre-trained Language Models and Multiple Instance Learning
di: Rostam, Zhyar Rzgar K, et al.
Pubblicazione: (2026)

BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP
di: Sounack, Thomas, et al.
Pubblicazione: (2025)

Using Machine Mental Imagery for Representing Common Ground in Situated Dialogue
di: Mohapatra, Biswesh, et al.
Pubblicazione: (2026)

Adaptation of Biomedical and Clinical Pretrained Models to French Long Documents: A Comparative Study
di: Bazoge, Adrien, et al.
Pubblicazione: (2024)

CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling
di: Fang, Zheng, et al.
Pubblicazione: (2023)

Frame of Reference: Addressing the Challenges of Common Ground Representation in Situational Dialogs
di: Mohapatra, Biswesh, et al.
Pubblicazione: (2026)

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science
di: Kim, Junho, et al.
Pubblicazione: (2024)

Making Pre-trained Language Models Better Continual Few-Shot Relation Extractors
di: Ma, Shengkun, et al.
Pubblicazione: (2024)

Understanding Data Temporality Impact on Large Language Models Pre-training
di: Pilchen, Hippolyte, et al.
Pubblicazione: (2026)

RegMix: Data Mixture as Regression for Language Model Pre-training
di: Liu, Qian, et al.
Pubblicazione: (2024)

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
di: Yang, Kailai, et al.
Pubblicazione: (2025)

Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
di: Fan, Zhiyuan, et al.
Pubblicazione: (2023)

Investigating Data Contamination for Pre-training Language Models
di: Jiang, Minhao, et al.
Pubblicazione: (2024)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
di: Ibrahim, Adam, et al.
Pubblicazione: (2024)

FRASIMED: a Clinical French Annotated Resource Produced through Crosslingual BERT-Based Annotation Projection
di: Zaghir, Jamil, et al.
Pubblicazione: (2023)

Toward Understanding BERT-Like Pre-Training for DNA Foundation Models
di: Liang, Chaoqi, et al.
Pubblicazione: (2023)

CA-BERT: Leveraging Context Awareness for Enhanced Multi-Turn Chat Interaction
di: Liu, Minghao, et al.
Pubblicazione: (2024)

Data Darwinism Part I: Unlocking the Value of Scientific Data for Pre-training
di: Qin, Yiwei, et al.
Pubblicazione: (2026)

KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts
di: Autenried, Christian, et al.
Pubblicazione: (2026)

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training
di: He, Nan, et al.
Pubblicazione: (2024)

Enhancing Text Classification with a Novel Multi-Agent Collaboration Framework Leveraging BERT
di: Baban, Hediyeh, et al.
Pubblicazione: (2025)

Multi-BERT: Leveraging Adapters and Prompt Tuning for Low-Resource Multi-Domain Adaptation
di: Azad, Parham Abed, et al.
Pubblicazione: (2024)

KuBERT: Central Kurdish BERT Model and Its Application for Sentiment Analysis
di: Awlla, Kozhin muhealddin, et al.
Pubblicazione: (2025)

PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
di: Ranaldi, Leonardo, et al.
Pubblicazione: (2023)

ORBIT: Cost-Effective Dataset Curation for Large Language Model Domain Adaptation with an Astronomy Case Study
di: Modesitt, Eric, et al.
Pubblicazione: (2024)