Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.25857 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912984215846912 |
|---|---|
| author | Busch, Matthias Tacke, Marius Lamaka, Sviatlana V. Zheludkevich, Mikhail L. Cyron, Christian J. Feiler, Christian Aydin, Roland C. |
| author_facet | Busch, Matthias Tacke, Marius Lamaka, Sviatlana V. Zheludkevich, Mikhail L. Cyron, Christian J. Feiler, Christian Aydin, Roland C. |
| contents | The capabilities of large language models (LLMs) have expanded beyond natural language processing to scientific prediction tasks, including molecular property prediction. However, their effectiveness in in-context learning remains ambiguous, particularly given the potential for training data contamination in widely used benchmarks. This paper investigates whether LLMs perform genuine in-context regression on molecular properties or rely primarily on memorized values. Furthermore, we analyze the interplay between pre-trained knowledge and in-context information through a series of progressively blinded experiments. We evaluate nine LLM variants across three families (GPT-4.1, GPT-5, Gemini 2.5) on three MoleculeNet datasets (Delaney solubility, Lipophilicity, QM7 atomization energy) using a systematic blinding approach that iteratively reduces available information. Complementing this, we utilize varying in-context sample sizes (0-, 60-, and 1000-shot) as an additional control for information access. This work provides a principled framework for evaluating molecular property prediction under controlled information access, addressing concerns regarding memorization and exposing conflicts between pre-trained knowledge and in-context information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_25857 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts Busch, Matthias Tacke, Marius Lamaka, Sviatlana V. Zheludkevich, Mikhail L. Cyron, Christian J. Feiler, Christian Aydin, Roland C. Machine Learning The capabilities of large language models (LLMs) have expanded beyond natural language processing to scientific prediction tasks, including molecular property prediction. However, their effectiveness in in-context learning remains ambiguous, particularly given the potential for training data contamination in widely used benchmarks. This paper investigates whether LLMs perform genuine in-context regression on molecular properties or rely primarily on memorized values. Furthermore, we analyze the interplay between pre-trained knowledge and in-context information through a series of progressively blinded experiments. We evaluate nine LLM variants across three families (GPT-4.1, GPT-5, Gemini 2.5) on three MoleculeNet datasets (Delaney solubility, Lipophilicity, QM7 atomization energy) using a systematic blinding approach that iteratively reduces available information. Complementing this, we utilize varying in-context sample sizes (0-, 60-, and 1000-shot) as an additional control for information access. This work provides a principled framework for evaluating molecular property prediction under controlled information access, addressing concerns regarding memorization and exposing conflicts between pre-trained knowledge and in-context information. |
| title | In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2603.25857 |