Can LLMs Produce Original Astronomy Research in a Semester? A Graduate Class Experiment
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912984371036160 |
|---|---|
| author | Zabludoff, Ann Chuang, Chen-Yu Johnson, Parker Thomas Liu, Yichen Martinez, Brina Bianca Shah, Neev Steffes, Lucille Weible, Gabriel Glen |
| author_facet | Zabludoff, Ann Chuang, Chen-Yu Johnson, Parker Thomas Liu, Yichen Martinez, Brina Bianca Shah, Neev Steffes, Lucille Weible, Gabriel Glen |
| contents | We discuss the results of using large language models (LLMs) to conduct original scientific research in an unfamiliar subject area during the Fall 2025 semester. Students in a graduate astronomy and astrophysics course were asked to test whether LLMs could help them complete research tasks faster and at a level of detail and accuracy required for scientific publication. Most students employed LLMs for a total of 5-10 hours. While all students completed a draft paper on an unsolved problem related to galaxies by semester's end, their impressions of the models' value varied. About half thought that the models saved them time. Many noted that LLMs failed to provide appropriately detailed insights or steps to addressing open, niche questions over a several-month timeframe. The LLMs also frequently (about 20% of the time) returned false citations, links, or summaries of papers. The models struggled with generating complex functional code, accessing online packages or Application Programming Interfaces (APIs), and retrieving astronomical datasets from existing archives. In writing code and in chats, the LLMs made implicit, overly simplifying assumptions and often doubled down even after being corrected. Given the rapid pace of LLM development, new models may soon address at least some of these issues and thus significantly enhance research productivity. Yet students expressed concerns about how LLM use might dampen creativity and reflection during the research process. To improve learning experiences in future semesters, the class will first discuss LLM best practices and limitations. Students will be encouraged to explore free online resources for tips for generative model applications and will decide for themselves whether to use LLMs for their research project. This white paper was not written using LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_25984 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Can LLMs Produce Original Astronomy Research in a Semester? A Graduate Class Experiment Zabludoff, Ann Chuang, Chen-Yu Johnson, Parker Thomas Liu, Yichen Martinez, Brina Bianca Shah, Neev Steffes, Lucille Weible, Gabriel Glen Instrumentation and Methods for Astrophysics Astrophysics of Galaxies Physics Education We discuss the results of using large language models (LLMs) to conduct original scientific research in an unfamiliar subject area during the Fall 2025 semester. Students in a graduate astronomy and astrophysics course were asked to test whether LLMs could help them complete research tasks faster and at a level of detail and accuracy required for scientific publication. Most students employed LLMs for a total of 5-10 hours. While all students completed a draft paper on an unsolved problem related to galaxies by semester's end, their impressions of the models' value varied. About half thought that the models saved them time. Many noted that LLMs failed to provide appropriately detailed insights or steps to addressing open, niche questions over a several-month timeframe. The LLMs also frequently (about 20% of the time) returned false citations, links, or summaries of papers. The models struggled with generating complex functional code, accessing online packages or Application Programming Interfaces (APIs), and retrieving astronomical datasets from existing archives. In writing code and in chats, the LLMs made implicit, overly simplifying assumptions and often doubled down even after being corrected. Given the rapid pace of LLM development, new models may soon address at least some of these issues and thus significantly enhance research productivity. Yet students expressed concerns about how LLM use might dampen creativity and reflection during the research process. To improve learning experiences in future semesters, the class will first discuss LLM best practices and limitations. Students will be encouraged to explore free online resources for tips for generative model applications and will decide for themselves whether to use LLMs for their research project. This white paper was not written using LLMs. |
| title | Can LLMs Produce Original Astronomy Research in a Semester? A Graduate Class Experiment |
| topic | Instrumentation and Methods for Astrophysics Astrophysics of Galaxies Physics Education |
| url | https://arxiv.org/abs/2603.25984 |