DCER: Dual-Stage Compression and Energy-Based Reconstruction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910012015640576 |
|---|---|
| author | Wang, Yiwen Qin, Jiahao |
| author_facet | Wang, Yiwen Qin, Jiahao |
| contents | Multimodal fusion faces two robustness challenges: noisy inputs degrade representation quality, and missing modalities cause prediction failures. We propose DCER, a
unified framework addressing both challenges through dual-stage compression and energy-based reconstruction. The compression stage operates at two levels:
within-modality frequency transforms (wavelet for audio, DCT for video) remove noise while preserving task-relevant patterns, and cross-modality bottleneck tokens
force genuine integration rather than modality-specific shortcuts. For missing modalities, energy-based reconstruction recovers representations via gradient descent
on a learned energy function, with the final energy providing intrinsic uncertainty quantification (\r{ho} > 0.72 correlation with prediction error). Experiments on
CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate state-of-the-art performance across all benchmarks, with a U-shaped robustness pattern favoring multimodal fusion at
both complete and high-missing conditions. The code will be available on Github. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_04904 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DCER: Dual-Stage Compression and Energy-Based Reconstruction Wang, Yiwen Qin, Jiahao Machine Learning Artificial Intelligence Multimedia Image and Video Processing 68T05, 68T10 I.2.6; I.5.4; I.2.7 Multimodal fusion faces two robustness challenges: noisy inputs degrade representation quality, and missing modalities cause prediction failures. We propose DCER, a unified framework addressing both challenges through dual-stage compression and energy-based reconstruction. The compression stage operates at two levels: within-modality frequency transforms (wavelet for audio, DCT for video) remove noise while preserving task-relevant patterns, and cross-modality bottleneck tokens force genuine integration rather than modality-specific shortcuts. For missing modalities, energy-based reconstruction recovers representations via gradient descent on a learned energy function, with the final energy providing intrinsic uncertainty quantification (\r{ho} > 0.72 correlation with prediction error). Experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate state-of-the-art performance across all benchmarks, with a U-shaped robustness pattern favoring multimodal fusion at both complete and high-missing conditions. The code will be available on Github. |
| title | DCER: Dual-Stage Compression and Energy-Based Reconstruction |
| topic | Machine Learning Artificial Intelligence Multimedia Image and Video Processing 68T05, 68T10 I.2.6; I.5.4; I.2.7 |
| url | https://arxiv.org/abs/2602.04904 |