Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918202185875456 |
|---|---|
| author | Ghosh, Satanu Brodnik, Neal R. Frey, Carolina Holgate, Collin Pollock, Tresa M. Daly, Samantha Carton, Samuel |
| author_facet | Ghosh, Satanu Brodnik, Neal R. Frey, Carolina Holgate, Collin Pollock, Tresa M. Daly, Samantha Carton, Samuel |
| contents | We explore the ability of GPT-4 to perform ad-hoc schema based information extraction from scientific literature. We assess specifically whether it can, with a basic prompting approach, replicate two existing material science datasets, given the manuscripts from which they were originally manually extracted. We employ materials scientists to perform a detailed manual error analysis to assess where the model struggles to faithfully extract the desired information, and draw on their insights to suggest research directions to address this broadly important task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_05348 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets Ghosh, Satanu Brodnik, Neal R. Frey, Carolina Holgate, Collin Pollock, Tresa M. Daly, Samantha Carton, Samuel Computation and Language Materials Science Artificial Intelligence Information Retrieval I.2; I.2.7; H.4; H.5 We explore the ability of GPT-4 to perform ad-hoc schema based information extraction from scientific literature. We assess specifically whether it can, with a basic prompting approach, replicate two existing material science datasets, given the manuscripts from which they were originally manually extracted. We employ materials scientists to perform a detailed manual error analysis to assess where the model struggles to faithfully extract the desired information, and draw on their insights to suggest research directions to address this broadly important task. |
| title | Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets |
| topic | Computation and Language Materials Science Artificial Intelligence Information Retrieval I.2; I.2.7; H.4; H.5 |
| url | https://arxiv.org/abs/2406.05348 |