Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.15294 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912909480689664 |
|---|---|
| author | Du, Ming Luo, Yanqi Banerjee, Srutarshi Wojcik, Michael Popovic, Jelena Cherukara, Mathew J. |
| author_facet | Du, Ming Luo, Yanqi Banerjee, Srutarshi Wojcik, Michael Popovic, Jelena Cherukara, Mathew J. |
| contents | We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and optional long-term memory to support both autonomous procedures and interactive user-guided measurements. Built on a flexible task-manager architecture, the system enables workflows ranging from fully agent-driven automation to logic-defined routines that embed localized LLM queries. EAA further provides a modern tool ecosystem with two-way compatibility for Model Context Protocol (MCP), allowing instrument-control tools to be consumed or served across applications. We demonstrate EAA at an imaging beamline at the Advanced Photon Source, including automated zone plate focusing, natural language-described feature search, and interactive data acquisition. These results illustrate how vision-capable agents can enhance beamline efficiency, reduce operational burden, and lower the expertise barrier for users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15294 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EAA: Automating materials characterization with vision language model agents Du, Ming Luo, Yanqi Banerjee, Srutarshi Wojcik, Michael Popovic, Jelena Cherukara, Mathew J. Artificial Intelligence 68T42 I.2.1 We present Experiment Automation Agents (EAA), a vision-language-model-driven agentic system designed to automate complex experimental microscopy workflows. EAA integrates multimodal reasoning, tool-augmented action, and optional long-term memory to support both autonomous procedures and interactive user-guided measurements. Built on a flexible task-manager architecture, the system enables workflows ranging from fully agent-driven automation to logic-defined routines that embed localized LLM queries. EAA further provides a modern tool ecosystem with two-way compatibility for Model Context Protocol (MCP), allowing instrument-control tools to be consumed or served across applications. We demonstrate EAA at an imaging beamline at the Advanced Photon Source, including automated zone plate focusing, natural language-described feature search, and interactive data acquisition. These results illustrate how vision-capable agents can enhance beamline efficiency, reduce operational burden, and lower the expertise barrier for users. |
| title | EAA: Automating materials characterization with vision language model agents |
| topic | Artificial Intelligence 68T42 I.2.1 |
| url | https://arxiv.org/abs/2602.15294 |