PhysBERT: A Text Embedding Model for Physics Scientific Literature

Fuente: arXiv
Enregistré dans:
Détails bibliographiques
Auteurs principaux: Hellert, Thorsten, Montenegro, João, Pollastro, Andrea
Format: Preprint
Publié: 2024
Sujets:
Accès en ligne:
Tags: Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
_version_ 1866911993631342592
author Hellert, Thorsten
Montenegro, João
Pollastro, Andrea
author_facet Hellert, Thorsten
Montenegro, João
Pollastro, Andrea
contents The specialized language and complex concepts in physics pose significant challenges for information extraction through Natural Language Processing (NLP). Central to effective NLP applications is the text embedding model, which converts text into dense vector representations for efficient information retrieval and semantic analysis. In this work, we introduce PhysBERT, the first physics-specific text embedding model. Pre-trained on a curated corpus of 1.2 million arXiv physics papers and fine-tuned with supervised data, PhysBERT outperforms leading general-purpose models on physics-specific tasks including the effectiveness in fine-tuning for specific physics subdomains.
format Preprint
id arxiv_https___arxiv_org_abs_2408_09574
institution arXiv
publishDate 2024
record_format arxiv
spellingShingle PhysBERT: A Text Embedding Model for Physics Scientific Literature
Hellert, Thorsten
Montenegro, João
Pollastro, Andrea
Computational Physics
Computation and Language
The specialized language and complex concepts in physics pose significant challenges for information extraction through Natural Language Processing (NLP). Central to effective NLP applications is the text embedding model, which converts text into dense vector representations for efficient information retrieval and semantic analysis. In this work, we introduce PhysBERT, the first physics-specific text embedding model. Pre-trained on a curated corpus of 1.2 million arXiv physics papers and fine-tuned with supervised data, PhysBERT outperforms leading general-purpose models on physics-specific tasks including the effectiveness in fine-tuning for specific physics subdomains.
title PhysBERT: A Text Embedding Model for Physics Scientific Literature
topic Computational Physics
Computation and Language
url https://arxiv.org/abs/2408.09574