Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.15417 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908373378662400 |
|---|---|
| author | Chlon, Leon Chlon, Maggie Awada, MarcAntonio M. |
| author_facet | Chlon, Leon Chlon, Maggie Awada, MarcAntonio M. |
| contents | Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15417 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Robust Multimodal Learning via Entropy-Gated Contrastive Fusion Chlon, Leon Chlon, Maggie Awada, MarcAntonio M. Machine Learning Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference. |
| title | Robust Multimodal Learning via Entropy-Gated Contrastive Fusion |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.15417 |