THFM: A Unified Video Foundation Model for 4D Human Perception and Beyond
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908916202340352 |
|---|---|
| author | Wang, Letian Zanfir, Andrei Bazavan, Eduard Gabriel Andriluka, Misha Sminchisescu, Cristian |
| author_facet | Wang, Letian Zanfir, Andrei Bazavan, Eduard Gabriel Andriluka, Misha Sminchisescu, Cristian |
| contents | We present THFM, a unified video foundation model for human-centric perception that jointly addresses dense tasks (depth, normals, segmentation, dense pose) and sparse tasks (2d/3d keypoint estimation) within a single architecture. THFM is derived from a pretrained text-to-video diffusion model, repurposed as a single-forward-pass perception model and augmented with learnable tokens for sparse predictions. Modulated by the text prompt, our single unified model is capable of performing various perception tasks. Crucially, our model is on-par or surpassing state-of-the-art specialized models on a variety of benchmarks despite being trained exclusively on synthetic data (i.e.~without training on real-world or benchmark specific data). We further highlight intriguing emergent properties of our model, which we attribute to the underlying diffusion-based video representation. For example, our model trained on videos with a single human in the scene generalizes to multiple humans and other object classes such as anthropomorphic characters and animals -- a capability that hasn't been demonstrated in the past. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_25892 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | THFM: A Unified Video Foundation Model for 4D Human Perception and Beyond Wang, Letian Zanfir, Andrei Bazavan, Eduard Gabriel Andriluka, Misha Sminchisescu, Cristian Computer Vision and Pattern Recognition We present THFM, a unified video foundation model for human-centric perception that jointly addresses dense tasks (depth, normals, segmentation, dense pose) and sparse tasks (2d/3d keypoint estimation) within a single architecture. THFM is derived from a pretrained text-to-video diffusion model, repurposed as a single-forward-pass perception model and augmented with learnable tokens for sparse predictions. Modulated by the text prompt, our single unified model is capable of performing various perception tasks. Crucially, our model is on-par or surpassing state-of-the-art specialized models on a variety of benchmarks despite being trained exclusively on synthetic data (i.e.~without training on real-world or benchmark specific data). We further highlight intriguing emergent properties of our model, which we attribute to the underlying diffusion-based video representation. For example, our model trained on videos with a single human in the scene generalizes to multiple humans and other object classes such as anthropomorphic characters and animals -- a capability that hasn't been demonstrated in the past. |
| title | THFM: A Unified Video Foundation Model for 4D Human Perception and Beyond |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.25892 |