Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.14116 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915948277006336 |
|---|---|
| author | Ma, Zerun Wang, Guoqiang Xie, Xinchen Chen, Yicheng Du, He Li, Bowen Sun, Yanan Liu, Wenran Chen, Kai Li, Yining |
| author_facet | Ma, Zerun Wang, Guoqiang Xie, Xinchen Chen, Yicheng Du, He Li, Bowen Sun, Yanan Liu, Wenran Chen, Kai Li, Yining |
| contents | While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_14116 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration Ma, Zerun Wang, Guoqiang Xie, Xinchen Chen, Yicheng Du, He Li, Bowen Sun, Yanan Liu, Wenran Chen, Kai Li, Yining Artificial Intelligence Computation and Language While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks. |
| title | TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2604.14116 |