EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918452788199424 |
|---|---|
| author | Ju, Xuan Wang, Tianyu Zhou, Yuqian Zhang, He Liu, Qing Zhao, Nanxuan Zhang, Zhifei Li, Yijun Cai, Yuanhao Liu, Shaoteng Pakhomov, Daniil Lin, Zhe Kim, Soo Ye Xu, Qiang |
| author_facet | Ju, Xuan Wang, Tianyu Zhou, Yuqian Zhang, He Liu, Qing Zhao, Nanxuan Zhang, Zhifei Li, Yijun Cai, Yuanhao Liu, Shaoteng Pakhomov, Daniil Lin, Zhe Kim, Soo Ye Xu, Qiang |
| contents | Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While image generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due to architectural limitations and data scarcity. In this work, we introduce EditVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and outputs with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while exhibiting emergent editing and generation abilities across modalities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20360 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning Ju, Xuan Wang, Tianyu Zhou, Yuqian Zhang, He Liu, Qing Zhao, Nanxuan Zhang, Zhifei Li, Yijun Cai, Yuanhao Liu, Shaoteng Pakhomov, Daniil Lin, Zhe Kim, Soo Ye Xu, Qiang Computer Vision and Pattern Recognition Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While image generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due to architectural limitations and data scarcity. In this work, we introduce EditVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and outputs with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while exhibiting emergent editing and generation abilities across modalities. |
| title | EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.20360 |