Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916975476736000 |
|---|---|
| author | Pan, Xuchen Chen, Yanxi Chen, Yushuo Sun, Yuchang Chen, Daoyuan Zhang, Wenhao Xie, Yuexiang Huang, Yilun Zhang, Yilei Gao, Dawei Shi, Weijie Li, Yaliang Ding, Bolin Zhou, Jingren |
| author_facet | Pan, Xuchen Chen, Yanxi Chen, Yushuo Sun, Yuchang Chen, Daoyuan Zhang, Wenhao Xie, Yuexiang Huang, Yilun Zhang, Yilei Gao, Dawei Shi, Weijie Li, Yaliang Ding, Bolin Zhou, Jingren |
| contents | Trinity-RFT is a general-purpose, unified and easy-to-use framework designed for reinforcement fine-tuning (RFT) of large language models. It is built with a modular and decoupled design, consisting of (1) an RFT-core that unifies and generalizes synchronous/asynchronous, on-policy/off-policy, and online/offline modes of RFT; (2) seamless integration for agent-environment interaction with high efficiency and robustness; and (3) systematic data pipelines optimized for RFT. Trinity-RFT can be easily adapted for diverse application scenarios, and serves as a unified platform for development and research of advanced reinforcement learning paradigms at both macroscopic and microscopic levels. This technical report outlines the vision, features, design and implementations of Trinity-RFT, accompanied by extensive examples, applications and experiments that demonstrate its functionalities and user-friendliness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17826 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models Pan, Xuchen Chen, Yanxi Chen, Yushuo Sun, Yuchang Chen, Daoyuan Zhang, Wenhao Xie, Yuexiang Huang, Yilun Zhang, Yilei Gao, Dawei Shi, Weijie Li, Yaliang Ding, Bolin Zhou, Jingren Machine Learning Computation and Language Distributed, Parallel, and Cluster Computing Trinity-RFT is a general-purpose, unified and easy-to-use framework designed for reinforcement fine-tuning (RFT) of large language models. It is built with a modular and decoupled design, consisting of (1) an RFT-core that unifies and generalizes synchronous/asynchronous, on-policy/off-policy, and online/offline modes of RFT; (2) seamless integration for agent-environment interaction with high efficiency and robustness; and (3) systematic data pipelines optimized for RFT. Trinity-RFT can be easily adapted for diverse application scenarios, and serves as a unified platform for development and research of advanced reinforcement learning paradigms at both macroscopic and microscopic levels. This technical report outlines the vision, features, design and implementations of Trinity-RFT, accompanied by extensive examples, applications and experiments that demonstrate its functionalities and user-friendliness. |
| title | Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models |
| topic | Machine Learning Computation and Language Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2505.17826 |