Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2604.07823 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913032066564096 |
|---|---|
| author | Zeng, Ailing Yang, Casper Ge, Chauncey Zhang, Eddie Xu, Garvey Lin, Gavin Gu, Gilbert Pi, Jeremy Li, Leo Shi, Mingyi Wang, Shawn Bi, Sheng Tang, Steven Hang, Thorn Guo, Tobey Li, Vincent Tong, Xin Li, Yikang Sun, Yuchen Zhao, Yue Lu, Yuhan Li, Yuwei Zhang, Zane Yang, Zeshi Ye, Zi |
| author_facet | Zeng, Ailing Yang, Casper Ge, Chauncey Zhang, Eddie Xu, Garvey Lin, Gavin Gu, Gilbert Pi, Jeremy Li, Leo Shi, Mingyi Wang, Shawn Bi, Sheng Tang, Steven Hang, Thorn Guo, Tobey Li, Vincent Tong, Xin Li, Yikang Sun, Yuchen Zhao, Yue Lu, Yuhan Li, Yuwei Zhang, Zane Yang, Zeshi Ye, Zi |
| contents | Performance, the externalization of intent, emotion, and personality through visual, vocal, and temporal behavior, is what makes a character alive. Learning such performance from video is a promising alternative to traditional 3D pipelines. However, existing video models struggle to jointly achieve high expressiveness, real-time inference, and long-horizon identity stability, a tension we call the performance trilemma. Conversation is the most comprehensive performance scenario, as characters simultaneously speak, listen, react, and emote while maintaining identity over time. To address this, we present LPM 1.0 (Large Performance Model), focusing on single-person full-duplex audio-visual conversational performance. Concretely, we build a multimodal human-centric dataset through strict filtering, speaking-listening audio-video pairing, performance understanding, and identity-aware multi-reference extraction; train a 17B-parameter Diffusion Transformer (Base LPM) for highly controllable, identity-consistent performance through multimodal conditioning; and distill it into a causal streaming generator (Online LPM) for low-latency, infinite-length interaction. At inference, given a character image with identity-aware references, LPM 1.0 generates listening videos from user audio and speaking videos from synthesized audio, with text prompts for motion control, all at real-time speed with identity-stable, infinite-length generation. LPM 1.0 thus serves as a visual engine for conversational agents, live streaming characters, and game NPCs. To systematically evaluate this setting, we propose LPM-Bench, the first benchmark for interactive character performance. LPM 1.0 achieves state-of-the-art results across all evaluated dimensions while maintaining real-time inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_07823 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LPM 1.0: Video-based Character Performance Model Zeng, Ailing Yang, Casper Ge, Chauncey Zhang, Eddie Xu, Garvey Lin, Gavin Gu, Gilbert Pi, Jeremy Li, Leo Shi, Mingyi Wang, Shawn Bi, Sheng Tang, Steven Hang, Thorn Guo, Tobey Li, Vincent Tong, Xin Li, Yikang Sun, Yuchen Zhao, Yue Lu, Yuhan Li, Yuwei Zhang, Zane Yang, Zeshi Ye, Zi Computer Vision and Pattern Recognition Artificial Intelligence Multimedia Performance, the externalization of intent, emotion, and personality through visual, vocal, and temporal behavior, is what makes a character alive. Learning such performance from video is a promising alternative to traditional 3D pipelines. However, existing video models struggle to jointly achieve high expressiveness, real-time inference, and long-horizon identity stability, a tension we call the performance trilemma. Conversation is the most comprehensive performance scenario, as characters simultaneously speak, listen, react, and emote while maintaining identity over time. To address this, we present LPM 1.0 (Large Performance Model), focusing on single-person full-duplex audio-visual conversational performance. Concretely, we build a multimodal human-centric dataset through strict filtering, speaking-listening audio-video pairing, performance understanding, and identity-aware multi-reference extraction; train a 17B-parameter Diffusion Transformer (Base LPM) for highly controllable, identity-consistent performance through multimodal conditioning; and distill it into a causal streaming generator (Online LPM) for low-latency, infinite-length interaction. At inference, given a character image with identity-aware references, LPM 1.0 generates listening videos from user audio and speaking videos from synthesized audio, with text prompts for motion control, all at real-time speed with identity-stable, infinite-length generation. LPM 1.0 thus serves as a visual engine for conversational agents, live streaming characters, and game NPCs. To systematically evaluate this setting, we propose LPM-Bench, the first benchmark for interactive character performance. LPM 1.0 achieves state-of-the-art results across all evaluated dimensions while maintaining real-time inference. |
| title | LPM 1.0: Video-based Character Performance Model |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Multimedia |
| url | https://arxiv.org/abs/2604.07823 |