Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.10509 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915800727683072 |
|---|---|
| author | Zhang, Zihan Cheng, Xize Jiang, Zhennan Fu, Dongjie Chen, Jingyuan Zhao, Zhou Jin, Tao |
| author_facet | Zhang, Zihan Cheng, Xize Jiang, Zhennan Fu, Dongjie Chen, Jingyuan Zhao, Zhou Jin, Tao |
| contents | Universal sound separation faces a fundamental misalignment: models optimized for low-level signal metrics often produce semantically contaminated outputs, failing to suppress perceptually salient interference from acoustically similar sources. We introduce a preference alignment perspective, analogous to aligning LLMs with human intent. To address this, we introduce MARS-Sep, a reinforcement learning framework that reformulates separation as decision making. Instead of simply regressing ground-truth masks, MARS-Sep learns a factorized Beta mask policy that is steered by a preference reward model and optimized by a stable, clipped trust-region surrogate. The reward, derived from a progressively-aligned audio-text-vision encoder, directly incentivizes semantic consistency with query prompts. Extensive experiments on multiple benchmarks demonstrate consistent gains in Text-, Audio-, and Image-Queried separation, with notable improvements in signal metrics and semantic quality. Our code is available at https://github.com/mars-sep/MARS-Sep. Sound separation samples are available at https://mars-sep.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_10509 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MARS-Sep: Multimodal-Aligned Reinforced Sound Separation Zhang, Zihan Cheng, Xize Jiang, Zhennan Fu, Dongjie Chen, Jingyuan Zhao, Zhou Jin, Tao Sound Artificial Intelligence Universal sound separation faces a fundamental misalignment: models optimized for low-level signal metrics often produce semantically contaminated outputs, failing to suppress perceptually salient interference from acoustically similar sources. We introduce a preference alignment perspective, analogous to aligning LLMs with human intent. To address this, we introduce MARS-Sep, a reinforcement learning framework that reformulates separation as decision making. Instead of simply regressing ground-truth masks, MARS-Sep learns a factorized Beta mask policy that is steered by a preference reward model and optimized by a stable, clipped trust-region surrogate. The reward, derived from a progressively-aligned audio-text-vision encoder, directly incentivizes semantic consistency with query prompts. Extensive experiments on multiple benchmarks demonstrate consistent gains in Text-, Audio-, and Image-Queried separation, with notable improvements in signal metrics and semantic quality. Our code is available at https://github.com/mars-sep/MARS-Sep. Sound separation samples are available at https://mars-sep.github.io/. |
| title | MARS-Sep: Multimodal-Aligned Reinforced Sound Separation |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2510.10509 |