Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912969075458048 |
|---|---|
| author | Ishii, Masato Hayakawa, Akio Shibuya, Takashi Mitsufuji, Yuki |
| author_facet | Ishii, Masato Hayakawa, Akio Shibuya, Takashi Mitsufuji, Yuki |
| contents | We introduce a novel pipeline for joint audio-visual editing that enhances the coherence between edited video and its accompanying audio. Our approach first applies state-of-the-art video editing techniques to produce the target video, then performs audio editing to align with the visual changes. To achieve this, we present a new video-to-audio generation model that conditions on the source audio, target video, and a text prompt. We extend the model architecture to incorporate conditional audio input and propose a data augmentation strategy that improves training efficiency. Furthermore, our model dynamically adjusts the influence of the source audio based on the complexity of the edits, preserving the original audio structure where possible. Experimental results demonstrate that our method outperforms existing approaches in maintaining audio-visual alignment and content integrity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_07209 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits Ishii, Masato Hayakawa, Akio Shibuya, Takashi Mitsufuji, Yuki Multimedia Machine Learning Sound We introduce a novel pipeline for joint audio-visual editing that enhances the coherence between edited video and its accompanying audio. Our approach first applies state-of-the-art video editing techniques to produce the target video, then performs audio editing to align with the visual changes. To achieve this, we present a new video-to-audio generation model that conditions on the source audio, target video, and a text prompt. We extend the model architecture to incorporate conditional audio input and propose a data augmentation strategy that improves training efficiency. Furthermore, our model dynamically adjusts the influence of the source audio based on the complexity of the edits, preserving the original audio structure where possible. Experimental results demonstrate that our method outperforms existing approaches in maintaining audio-visual alignment and content integrity. |
| title | Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits |
| topic | Multimedia Machine Learning Sound |
| url | https://arxiv.org/abs/2512.07209 |