Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2602.05638 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908972628312064 |
|---|---|
| author | Wu, Jinlin Holm, Felix Chen, Chuxi Wang, An Hu, Yaxin Ye, Xiaofan Zang, Zelin Xu, Miao Zhou, Lihua Liao, Huai Chan, Danny T. M. Feng, Ming Poon, Wai S. Ren, Hongliang Yi, Dong Navab, Nassir Meng, Gaofeng Luo, Jiebo Liu, Hongbin Lei, Zhen |
| author_facet | Wu, Jinlin Holm, Felix Chen, Chuxi Wang, An Hu, Yaxin Ye, Xiaofan Zang, Zelin Xu, Miao Zhou, Lihua Liao, Huai Chan, Danny T. M. Feng, Ming Poon, Wai S. Ren, Hongliang Yi, Dong Navab, Nassir Meng, Gaofeng Luo, Jiebo Liu, Hongbin Lei, Zhen |
| contents | While foundation models have advanced surgical video analysis, current approaches rely predominantly on pixel-level reconstruction objectives that waste model capacity on low-level visual details, such as smoke, specular reflections, and fluid motion, rather than semantic structures essential for surgical understanding. We present SurgMotion, a video-native foundation model that shifts the learning paradigm from pixel-level reconstruction to latent motion prediction. Built on the Video Joint Embedding Predictive Architecture (V-JEPA), SurgMotion introduces three key technical innovations tailored to surgical videos: (1) motion-guided latent masked prediction to prioritize semantically meaningful regions, (2) spatiotemporal affinity self-distillation to enforce relational consistency, and (3) spatiotemporal feature diversity regularization (SFDR) to prevent representation collapse in texture-sparse surgical scenes. To enable large-scale pretraining, we curate SurgMotion-15M, the largest surgical video dataset to date, comprising 3,658 hours of video from 50 sources across 13 anatomical regions. Extensive experiments across 17 benchmarks demonstrate that SurgMotion significantly outperforms state-of-the-art methods on surgical workflow recognition, achieving 14.6 percent improvement in F1 score on EgoSurgery and 10.3 percent on PitVis; on action triplet recognition with 39.54 percent mAP-IVT on CholecT50; as well as on skill assessment, polyp segmentation, and depth estimation. These results establish SurgMotion as a new standard for universal, motion-oriented surgical video understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05638 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos Wu, Jinlin Holm, Felix Chen, Chuxi Wang, An Hu, Yaxin Ye, Xiaofan Zang, Zelin Xu, Miao Zhou, Lihua Liao, Huai Chan, Danny T. M. Feng, Ming Poon, Wai S. Ren, Hongliang Yi, Dong Navab, Nassir Meng, Gaofeng Luo, Jiebo Liu, Hongbin Lei, Zhen Computer Vision and Pattern Recognition While foundation models have advanced surgical video analysis, current approaches rely predominantly on pixel-level reconstruction objectives that waste model capacity on low-level visual details, such as smoke, specular reflections, and fluid motion, rather than semantic structures essential for surgical understanding. We present SurgMotion, a video-native foundation model that shifts the learning paradigm from pixel-level reconstruction to latent motion prediction. Built on the Video Joint Embedding Predictive Architecture (V-JEPA), SurgMotion introduces three key technical innovations tailored to surgical videos: (1) motion-guided latent masked prediction to prioritize semantically meaningful regions, (2) spatiotemporal affinity self-distillation to enforce relational consistency, and (3) spatiotemporal feature diversity regularization (SFDR) to prevent representation collapse in texture-sparse surgical scenes. To enable large-scale pretraining, we curate SurgMotion-15M, the largest surgical video dataset to date, comprising 3,658 hours of video from 50 sources across 13 anatomical regions. Extensive experiments across 17 benchmarks demonstrate that SurgMotion significantly outperforms state-of-the-art methods on surgical workflow recognition, achieving 14.6 percent improvement in F1 score on EgoSurgery and 10.3 percent on PitVis; on action triplet recognition with 39.54 percent mAP-IVT on CholecT50; as well as on skill assessment, polyp segmentation, and depth estimation. These results establish SurgMotion as a new standard for universal, motion-oriented surgical video understanding. |
| title | SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.05638 |