Cover Image

FolAI: Synchronized Foley Sound Generation with Semantic and Temporal Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gramaccioni, Riccardo Fosco, Marinoni, Christian, Postolache, Emilian, Comunità, Marco, Cosmo, Luca, Reiss, Joshua D., Comminiello, Danilo
Format:	Preprint
Published:	2024
Subjects:	Sound Computer Vision and Pattern Recognition Machine Learning Multimedia Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FoleyGRAM: Video-to-Audio Generation with GRAM-Aligned Multimodal Encoders
by: Gramaccioni, Riccardo Fosco, et al.
Published: (2025)

StereoSync: Spatially-Aware Stereo Audio Generation from Video
by: Marinoni, Christian, et al.
Published: (2025)

Rhythmic Foley: A Framework For Seamless Audio-Visual Alignment In Video-to-Audio Synthesis
by: Huang, Zhiqi, et al.
Published: (2024)

StereoFoley: Object-Aware Stereo Audio Generation from Video
by: Karchkhadze, Tornike, et al.
Published: (2025)

STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment
by: Ren, Yong, et al.
Published: (2024)

An automatic mixing speech enhancement system for multi-track audio
by: Liu, Xiaojing, et al.
Published: (2024)

Visual-based spatial audio generation system for multi-speaker environments
by: Liu, Xiaojing, et al.
Published: (2025)

Video-Foley: Two-Stage Video-To-Sound Generation via Temporal Event Condition For Foley Sound
by: Lee, Junwon, et al.
Published: (2024)

Video-Guided Foley Sound Generation with Multimodal Controls
by: Chen, Ziyang, et al.
Published: (2024)

Representation Learning for Semantic Alignment of Language, Audio, and Visual Modalities
by: Sudarsanam, Parthasaarathy, et al.
Published: (2025)

Generalized Multi-Source Inference for Text Conditioned Music Diffusion Models
by: Postolache, Emilian, et al.
Published: (2024)

Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
by: Lei, Ke, et al.
Published: (2026)

MoMu-Diffusion: On Learning Long-Term Motion-Music Synchronization and Correspondence
by: You, Fuming, et al.
Published: (2024)

Multi-Source Diffusion Models for Simultaneous Music Generation and Separation
by: Mariani, Giorgio, et al.
Published: (2023)

SonicVisionLM: Playing Sound with Vision Language Models
by: Xie, Zhifeng, et al.
Published: (2024)

Sound-VECaps: Improving Audio Generation with Visual Enhanced Captions
by: Yuan, Yi, et al.
Published: (2024)

PerformSinger: Multimodal Singing Voice Synthesis Leveraging Synchronized Lip Cues from Singing Performance Videos
by: Gu, Ke, et al.
Published: (2025)

MuVi: Video-to-Music Generation with Semantic Alignment and Rhythmic Synchronization
by: Li, Ruiqi, et al.
Published: (2024)

Overview of the L3DAS23 Challenge on Audio-Visual Extended Reality
by: Marinoni, Christian, et al.
Published: (2024)

Differentiable Black-box and Gray-box Modeling of Nonlinear Audio Effects
by: Comunità, Marco, et al.
Published: (2025)

pyAMPACT: A Score-Audio Alignment Toolkit for Performance Data Estimation and Multi-modal Processing
by: Devaney, Johanna, et al.
Published: (2024)

Semantically consistent Video-to-Audio Generation using Multimodal Language Large Model
by: Chen, Gehui, et al.
Published: (2024)

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text
by: Liu, Haohe, et al.
Published: (2024)

Robust LLM-based Audio-Visual Speech Recognition with Sparse Modality Alignment and Visual Unit-Guided Refinement
by: Su, Fei, et al.
Published: (2026)

MusicSem: A Semantically Rich Language--Audio Dataset of Natural Music Descriptions
by: Salganik, Rebecca, et al.
Published: (2026)

COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representations
by: Ciranni, Ruben, et al.
Published: (2024)

Naturalistic Music Decoding from EEG Data via Latent Diffusion Models
by: Postolache, Emilian, et al.
Published: (2024)

MambaFoley: Foley Sound Generation using Selective State-Space Models
by: Colombo, Marco Furio, et al.
Published: (2024)

Zero-Shot Duet Singing Voices Separation with Diffusion Models
by: Yu, Chin-Yun, et al.
Published: (2023)

Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment
by: Senocak, Arda, et al.
Published: (2024)

Bimodal Connection Attention Fusion for Speech Emotion Recognition
by: Luo, Jiachen, et al.
Published: (2025)

DanceAnyWay: Synthesizing Beat-Guided 3D Dances with Randomized Temporal Contrastive Learning
by: Bhattacharya, Aneesh, et al.
Published: (2023)

Can Sound Replace Vision in LLaVA With Token Substitution?
by: Vosoughi, Ali, et al.
Published: (2025)

From Sound to Sight: Towards AI-authored Music Videos
by: Vitasovic, Leo, et al.
Published: (2025)

Microphone Conversion: Mitigating Device Variability in Sound Event Classification
by: Ryu, Myeonghoon, et al.
Published: (2024)

Embedding Alignment in Code Generation for Audio
by: Kouteili, Sam, et al.
Published: (2025)

Audio-Visual Speaker Tracking: Progress, Challenges, and Future Directions
by: Zhao, Jinzheng, et al.
Published: (2023)

Video-Guided Text-to-Music Generation Using Public Domain Movie Collections
by: Kim, Haven, et al.
Published: (2025)

Efficient Speech Watermarking for Speech Synthesis via Progressive Knowledge Distillation
by: Cui, Yang, et al.
Published: (2025)

ARECHO: Autoregressive Evaluation via Chain-Based Hypothesis Optimization for Speech Multi-Metric Estimation
by: Shi, Jiatong, et al.
Published: (2025)