AMSbench: A Comprehensive Benchmark for Evaluating MLLM Capabilities in AMS Circuits
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909838913568768 |
|---|---|
| author | Shi, Yichen Zhang, Ze Wang, Hongyang Tao, Zhuofu Li, Zhongyi Chen, Bingyu Wang, Yaxin huang, Zhen Liu, Xuhua Chen, Quan Yu, Zhiping Lin, Ting-Jung He, Lei |
| author_facet | Shi, Yichen Zhang, Ze Wang, Hongyang Tao, Zhuofu Li, Zhongyi Chen, Bingyu Wang, Yaxin huang, Zhen Liu, Xuhua Chen, Quan Yu, Zhiping Lin, Ting-Jung He, Lei |
| contents | Analog/Mixed-Signal (AMS) circuits play a critical role in the integrated circuit (IC) industry. However, automating Analog/Mixed-Signal (AMS) circuit design has remained a longstanding challenge due to its difficulty and complexity. Although recent advances in Multi-modal Large Language Models (MLLMs) offer promising potential for supporting AMS circuit analysis and design, current research typically evaluates MLLMs on isolated tasks within the domain, lacking a comprehensive benchmark that systematically assesses model capabilities across diverse AMS-related challenges. To address this gap, we introduce AMSbench, a benchmark suite designed to evaluate MLLM performance across critical tasks including circuit schematic perception, circuit analysis, and circuit design. AMSbench comprises approximately 8000 test questions spanning multiple difficulty levels and assesses eight prominent models, encompassing both open-source and proprietary solutions such as Qwen 2.5-VL and Gemini 2.5 Pro. Our evaluation highlights significant limitations in current MLLMs, particularly in complex multi-modal reasoning and sophisticated circuit design tasks. These results underscore the necessity of advancing MLLMs' understanding and effective application of circuit-specific knowledge, thereby narrowing the existing performance gap relative to human expertise and moving toward fully automated AMS circuit design workflows. Our data is released at this URL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24138 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AMSbench: A Comprehensive Benchmark for Evaluating MLLM Capabilities in AMS Circuits Shi, Yichen Zhang, Ze Wang, Hongyang Tao, Zhuofu Li, Zhongyi Chen, Bingyu Wang, Yaxin huang, Zhen Liu, Xuhua Chen, Quan Yu, Zhiping Lin, Ting-Jung He, Lei Machine Learning Artificial Intelligence Analog/Mixed-Signal (AMS) circuits play a critical role in the integrated circuit (IC) industry. However, automating Analog/Mixed-Signal (AMS) circuit design has remained a longstanding challenge due to its difficulty and complexity. Although recent advances in Multi-modal Large Language Models (MLLMs) offer promising potential for supporting AMS circuit analysis and design, current research typically evaluates MLLMs on isolated tasks within the domain, lacking a comprehensive benchmark that systematically assesses model capabilities across diverse AMS-related challenges. To address this gap, we introduce AMSbench, a benchmark suite designed to evaluate MLLM performance across critical tasks including circuit schematic perception, circuit analysis, and circuit design. AMSbench comprises approximately 8000 test questions spanning multiple difficulty levels and assesses eight prominent models, encompassing both open-source and proprietary solutions such as Qwen 2.5-VL and Gemini 2.5 Pro. Our evaluation highlights significant limitations in current MLLMs, particularly in complex multi-modal reasoning and sophisticated circuit design tasks. These results underscore the necessity of advancing MLLMs' understanding and effective application of circuit-specific knowledge, thereby narrowing the existing performance gap relative to human expertise and moving toward fully automated AMS circuit design workflows. Our data is released at this URL. |
| title | AMSbench: A Comprehensive Benchmark for Evaluating MLLM Capabilities in AMS Circuits |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.24138 |