Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.24517 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912982299049984 |
|---|---|
| author | Chen, Terry Ye, Zhifan Xu, Bing Ye, Zihao Liu, Timmy Hassani, Ali Chen, Tianqi Kerr, Andrew Wu, Haicheng Xu, Yang Chen, Yu-Jung Chen, Hanfeng Kane, Aditya Krashinsky, Ronny Liu, Ming-Yu Grover, Vinod Ceze, Luis Bringmann, Roger Tran, John Liu, Wei Xie, Fung Lightstone, Michael Shi, Humphrey |
| author_facet | Chen, Terry Ye, Zhifan Xu, Bing Ye, Zihao Liu, Timmy Hassani, Ali Chen, Tianqi Kerr, Andrew Wu, Haicheng Xu, Yang Chen, Yu-Jung Chen, Hanfeng Kane, Aditya Krashinsky, Ronny Liu, Ming-Yu Grover, Vinod Ceze, Luis Bringmann, Roger Tran, John Liu, Wei Xie, Fung Lightstone, Michael Shi, Humphrey |
| contents | Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_24517 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AVO: Agentic Variation Operators for Autonomous Evolutionary Search Chen, Terry Ye, Zhifan Xu, Bing Ye, Zihao Liu, Timmy Hassani, Ali Chen, Tianqi Kerr, Andrew Wu, Haicheng Xu, Yang Chen, Yu-Jung Chen, Hanfeng Kane, Aditya Krashinsky, Ronny Liu, Ming-Yu Grover, Vinod Ceze, Luis Bringmann, Roger Tran, John Liu, Wei Xie, Fung Lightstone, Michael Shi, Humphrey Machine Learning Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware. |
| title | AVO: Agentic Variation Operators for Autonomous Evolutionary Search |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2603.24517 |