PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908812571574272 |
|---|---|
| author | Lin, Xiaopeng Lian, Shijie Yu, Bin Yang, Ruoqi Shen, Zhaolong Wu, Changti Miao, Yuzhuo Jin, Yurun Shi, Yukun He, Jiyan Huang, Cong Cheng, Bojun Chen, Kai |
| author_facet | Lin, Xiaopeng Lian, Shijie Yu, Bin Yang, Ruoqi Shen, Zhaolong Wu, Changti Miao, Yuzhuo Jin, Yurun Shi, Yukun He, Jiyan Huang, Cong Cheng, Bojun Chen, Kai |
| contents | Robotic generalization relies on physical intelligence: the ability to reason about state changes, contact-rich interactions, and long-horizon planning under egocentric perception and action. Vision Language Models (VLMs) are essential to Vision-Language-Action (VLA) systems, but the reliance on third-person training data creates a viewpoint gap for humanoid robots. Collecting massive robot-centric data is an ideal but impractical solution due to cost and diversity constraints. Conversely, human egocentric videos offer a highly scalable data source with rich interaction context, yet the embodiment mismatch prevents the direct application. To bridge this gap, we propose an Egocentric2Embodiment Translation Pipeline that transforms raw human egocentric videos into multi-level, schema-driven embodiment supervision with enforced evidence grounding and temporal consistency, enabling the construction of the Egocentric2Embodiment dataset (E2E-3M) at scale. An egocentric-aware embodied brain, termed PhysBrain, is obtained by training on the E2E-3M dataset. PhysBrain exhibits substantially improved egocentric understanding, particularly for planning. It provides an egocentric-aware initialization that enables more sample-efficient VLA fine-tuning and higher success rates, demonstrating effective transfer from human egocentric supervision to downstream robot control. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16793 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence Lin, Xiaopeng Lian, Shijie Yu, Bin Yang, Ruoqi Shen, Zhaolong Wu, Changti Miao, Yuzhuo Jin, Yurun Shi, Yukun He, Jiyan Huang, Cong Cheng, Bojun Chen, Kai Robotics Robotic generalization relies on physical intelligence: the ability to reason about state changes, contact-rich interactions, and long-horizon planning under egocentric perception and action. Vision Language Models (VLMs) are essential to Vision-Language-Action (VLA) systems, but the reliance on third-person training data creates a viewpoint gap for humanoid robots. Collecting massive robot-centric data is an ideal but impractical solution due to cost and diversity constraints. Conversely, human egocentric videos offer a highly scalable data source with rich interaction context, yet the embodiment mismatch prevents the direct application. To bridge this gap, we propose an Egocentric2Embodiment Translation Pipeline that transforms raw human egocentric videos into multi-level, schema-driven embodiment supervision with enforced evidence grounding and temporal consistency, enabling the construction of the Egocentric2Embodiment dataset (E2E-3M) at scale. An egocentric-aware embodied brain, termed PhysBrain, is obtained by training on the E2E-3M dataset. PhysBrain exhibits substantially improved egocentric understanding, particularly for planning. It provides an egocentric-aware initialization that enables more sample-efficient VLA fine-tuning and higher success rates, demonstrating effective transfer from human egocentric supervision to downstream robot control. |
| title | PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence |
| topic | Robotics |
| url | https://arxiv.org/abs/2512.16793 |