Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911472220635136 |
|---|---|
| author | Haddad, Dany Bareket, Dan Chang, Joseph Chee DeYoung, Jay Hwang, Jena D. Katz, Uri Polak, Mark Suh, Sangho Surana, Harshit Tiktinsky, Aryeh Atmakuri, Shriya Bragg, Jonathan D'Arcy, Mike Feldman, Sergey Hassan-Ali, Amal Lozano, Rubén Majumder, Bodhisattwa Prasad McGrady, Charles Singh, Amanpreet Vlahos, Brooke Goldberg, Yoav Downey, Doug |
| author_facet | Haddad, Dany Bareket, Dan Chang, Joseph Chee DeYoung, Jay Hwang, Jena D. Katz, Uri Polak, Mark Suh, Sangho Surana, Harshit Tiktinsky, Aryeh Atmakuri, Shriya Bragg, Jonathan D'Arcy, Mike Feldman, Sergey Hassan-Ali, Amal Lozano, Rubén Majumder, Bodhisattwa Prasad McGrady, Charles Singh, Amanpreet Vlahos, Brooke Goldberg, Yoav Downey, Doug |
| contents | AI-powered scientific research tools are rapidly being integrated into research workflows, yet the field lacks a clear lens into how researchers use these systems in real-world settings. We present and analyze the Asta Interaction Dataset, a large-scale resource comprising over 200,000 user queries and interaction logs from two deployed tools (a literature discovery interface and a scientific question-answering interface) within an LLM-powered retrieval-augmented generation platform. Using this dataset, we characterize query patterns, engagement behaviors, and how usage evolves with experience. We find that users submit longer and more complex queries than in traditional search, and treat the system as a collaborative research partner, delegating tasks such as drafting content and identifying research gaps. Users treat generated responses as persistent artifacts, revisiting and navigating among outputs and cited evidence in non-linear ways. With experience, users issue more targeted queries and engage more deeply with supporting citations, although keyword-style queries persist even among experienced users. We release the anonymized dataset and analysis with a new query intent taxonomy to inform future designs of real-world AI research assistants and to support realistic evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23335 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset Haddad, Dany Bareket, Dan Chang, Joseph Chee DeYoung, Jay Hwang, Jena D. Katz, Uri Polak, Mark Suh, Sangho Surana, Harshit Tiktinsky, Aryeh Atmakuri, Shriya Bragg, Jonathan D'Arcy, Mike Feldman, Sergey Hassan-Ali, Amal Lozano, Rubén Majumder, Bodhisattwa Prasad McGrady, Charles Singh, Amanpreet Vlahos, Brooke Goldberg, Yoav Downey, Doug Human-Computer Interaction Artificial Intelligence Information Retrieval AI-powered scientific research tools are rapidly being integrated into research workflows, yet the field lacks a clear lens into how researchers use these systems in real-world settings. We present and analyze the Asta Interaction Dataset, a large-scale resource comprising over 200,000 user queries and interaction logs from two deployed tools (a literature discovery interface and a scientific question-answering interface) within an LLM-powered retrieval-augmented generation platform. Using this dataset, we characterize query patterns, engagement behaviors, and how usage evolves with experience. We find that users submit longer and more complex queries than in traditional search, and treat the system as a collaborative research partner, delegating tasks such as drafting content and identifying research gaps. Users treat generated responses as persistent artifacts, revisiting and navigating among outputs and cited evidence in non-linear ways. With experience, users issue more targeted queries and engage more deeply with supporting citations, although keyword-style queries persist even among experienced users. We release the anonymized dataset and analysis with a new query intent taxonomy to inform future designs of real-world AI research assistants and to support realistic evaluation. |
| title | Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset |
| topic | Human-Computer Interaction Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2602.23335 |