Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.17158 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917137732337664 |
|---|---|
| author | Froger, Romain Andrews, Pierre Bettini, Matteo Budhiraja, Amar Cabral, Ricardo Silveira Do, Virginie Garreau, Emilien Gaya, Jean-Baptiste Laurençon, Hugo Lecanu, Maxime Malkan, Kunal Mekala, Dheeraj Ménard, Pierre Bertran, Gerard Moreno-Torres Piterbarg, Ulyana Plekhanov, Mikhail Rita, Mathieu Rusakov, Andrey Vorotilov, Vladislav Wang, Mengjue Yu, Ian Benhalloum, Amine Mialon, Grégoire Scialom, Thomas |
| author_facet | Froger, Romain Andrews, Pierre Bettini, Matteo Budhiraja, Amar Cabral, Ricardo Silveira Do, Virginie Garreau, Emilien Gaya, Jean-Baptiste Laurençon, Hugo Lecanu, Maxime Malkan, Kunal Mekala, Dheeraj Ménard, Pierre Bertran, Gerard Moreno-Torres Piterbarg, Ulyana Plekhanov, Mikhail Rita, Mathieu Rusakov, Andrey Vorotilov, Vladislav Wang, Mengjue Yu, Ian Benhalloum, Amine Mialon, Grégoire Scialom, Thomas |
| contents | We introduce Meta Agents Research Environments (ARE), a research platform for scalable creation of environments, integration of synthetic or real applications, and execution of agentic orchestrations. ARE provides simple abstractions to build complex and diverse environments, each with their own rules, tools, content, and verifiers, helping to bridge the gap between model development and real-world deployment. We also propose Gaia2, a benchmark built in ARE and designed to measure general agent capabilities. Beyond search and execution, Gaia2 requires agents to handle ambiguities and noise, adapt to dynamic environments, collaborate with other agents, and operate under temporal constraints. Unlike prior benchmarks, Gaia2 runs asynchronously, surfacing new failure modes that are invisible in static settings. Our experiments show that no system dominates across the intelligence spectrum: stronger reasoning often comes at the cost of efficiency, and budget scaling curves plateau, highlighting the need for new architectures and adaptive compute strategies. Perhaps more importantly, ARE abstractions enable continuous extension of Gaia2 to other environments, empowering the community to rapidly create new benchmarks tailored to their domains. In AI's second half, progress increasingly depends on defining meaningful tasks and robust evaluations to drive frontier capabilities forward. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17158 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ARE: Scaling Up Agent Environments and Evaluations Froger, Romain Andrews, Pierre Bettini, Matteo Budhiraja, Amar Cabral, Ricardo Silveira Do, Virginie Garreau, Emilien Gaya, Jean-Baptiste Laurençon, Hugo Lecanu, Maxime Malkan, Kunal Mekala, Dheeraj Ménard, Pierre Bertran, Gerard Moreno-Torres Piterbarg, Ulyana Plekhanov, Mikhail Rita, Mathieu Rusakov, Andrey Vorotilov, Vladislav Wang, Mengjue Yu, Ian Benhalloum, Amine Mialon, Grégoire Scialom, Thomas Artificial Intelligence Computation and Language We introduce Meta Agents Research Environments (ARE), a research platform for scalable creation of environments, integration of synthetic or real applications, and execution of agentic orchestrations. ARE provides simple abstractions to build complex and diverse environments, each with their own rules, tools, content, and verifiers, helping to bridge the gap between model development and real-world deployment. We also propose Gaia2, a benchmark built in ARE and designed to measure general agent capabilities. Beyond search and execution, Gaia2 requires agents to handle ambiguities and noise, adapt to dynamic environments, collaborate with other agents, and operate under temporal constraints. Unlike prior benchmarks, Gaia2 runs asynchronously, surfacing new failure modes that are invisible in static settings. Our experiments show that no system dominates across the intelligence spectrum: stronger reasoning often comes at the cost of efficiency, and budget scaling curves plateau, highlighting the need for new architectures and adaptive compute strategies. Perhaps more importantly, ARE abstractions enable continuous extension of Gaia2 to other environments, empowering the community to rapidly create new benchmarks tailored to their domains. In AI's second half, progress increasingly depends on defining meaningful tasks and robust evaluations to drive frontier capabilities forward. |
| title | ARE: Scaling Up Agent Environments and Evaluations |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2509.17158 |