Extremal Testing for Network Software using LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916845459603456 |
|---|---|
| author | Singha, Rathin Qian, Harry Saikrishnan, Srinath Zhao, Tracy Beckett, Ryan Kakarla, Siva Kesava Reddy Varghese, George |
| author_facet | Singha, Rathin Qian, Harry Saikrishnan, Srinath Zhao, Tracy Beckett, Ryan Kakarla, Siva Kesava Reddy Varghese, George |
| contents | Physicists often manually consider extreme cases when testing a theory. In this paper, we show how to automate extremal testing of network software using LLMs in two steps: first, ask the LLM to generate input constraints (e.g., DNS name length limits); then ask the LLM to generate tests that violate the constraints. We demonstrate how easy this process is by generating extremal tests for HTTP, BGP and DNS implementations, each of which uncovered new bugs. We show how this methodology extends to centralized network software such as shortest path algorithms, and how LLMs can generate filtering code to reject extremal input. We propose using agentic AI to further automate extremal testing. LLM-generated extremal testing goes beyond an old technique in software testing called Boundary Value Analysis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_11898 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Extremal Testing for Network Software using LLMs Singha, Rathin Qian, Harry Saikrishnan, Srinath Zhao, Tracy Beckett, Ryan Kakarla, Siva Kesava Reddy Varghese, George Software Engineering Networking and Internet Architecture Physicists often manually consider extreme cases when testing a theory. In this paper, we show how to automate extremal testing of network software using LLMs in two steps: first, ask the LLM to generate input constraints (e.g., DNS name length limits); then ask the LLM to generate tests that violate the constraints. We demonstrate how easy this process is by generating extremal tests for HTTP, BGP and DNS implementations, each of which uncovered new bugs. We show how this methodology extends to centralized network software such as shortest path algorithms, and how LLMs can generate filtering code to reject extremal input. We propose using agentic AI to further automate extremal testing. LLM-generated extremal testing goes beyond an old technique in software testing called Boundary Value Analysis. |
| title | Extremal Testing for Network Software using LLMs |
| topic | Software Engineering Networking and Internet Architecture |
| url | https://arxiv.org/abs/2507.11898 |