Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916652151472128 |
|---|---|
| author | Zhang, Zhejun Karkus, Peter Igl, Maximilian Ding, Wenhao Chen, Yuxiao Ivanovic, Boris Pavone, Marco |
| author_facet | Zhang, Zhejun Karkus, Peter Igl, Maximilian Ding, Wenhao Chen, Yuxiao Ivanovic, Boris Pavone, Marco |
| contents | Traffic simulation aims to learn a policy for traffic agents that, when unrolled in closed-loop, faithfully recovers the joint distribution of trajectories observed in the real world. Inspired by large language models, tokenized multi-agent policies have recently become the state-of-the-art in traffic simulation. However, they are typically trained through open-loop behavior cloning, and thus suffer from covariate shift when executed in closed-loop during simulation. In this work, we present Closest Among Top-K (CAT-K) rollouts, a simple yet effective closed-loop fine-tuning strategy to mitigate covariate shift. CAT-K fine-tuning only requires existing trajectory data, without reinforcement learning or generative adversarial imitation. Concretely, CAT-K fine-tuning enables a small 7M-parameter tokenized traffic simulation policy to outperform a 102M-parameter model from the same model family, achieving the top spot on the Waymo Sim Agent Challenge leaderboard at the time of submission. The code is available at https://github.com/NVlabs/catk. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05334 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models Zhang, Zhejun Karkus, Peter Igl, Maximilian Ding, Wenhao Chen, Yuxiao Ivanovic, Boris Pavone, Marco Machine Learning Traffic simulation aims to learn a policy for traffic agents that, when unrolled in closed-loop, faithfully recovers the joint distribution of trajectories observed in the real world. Inspired by large language models, tokenized multi-agent policies have recently become the state-of-the-art in traffic simulation. However, they are typically trained through open-loop behavior cloning, and thus suffer from covariate shift when executed in closed-loop during simulation. In this work, we present Closest Among Top-K (CAT-K) rollouts, a simple yet effective closed-loop fine-tuning strategy to mitigate covariate shift. CAT-K fine-tuning only requires existing trajectory data, without reinforcement learning or generative adversarial imitation. Concretely, CAT-K fine-tuning enables a small 7M-parameter tokenized traffic simulation policy to outperform a 102M-parameter model from the same model family, achieving the top spot on the Waymo Sim Agent Challenge leaderboard at the time of submission. The code is available at https://github.com/NVlabs/catk. |
| title | Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.05334 |