Lightweight Cross-Modal Representation Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910593120731136 |
|---|---|
| author | Faye, Bilal Azzag, Hanane Lebbah, Mustapha Bouchaffra, Djamel |
| author_facet | Faye, Bilal Azzag, Hanane Lebbah, Mustapha Bouchaffra, Djamel |
| contents | Low-cost cross-modal representation learning is crucial for deriving semantic representations across diverse modalities such as text, audio, images, and video. Traditional approaches typically depend on large specialized models trained from scratch, requiring extensive datasets and resulting in high resource and time costs. To overcome these challenges, we introduce a novel approach named Lightweight Cross-Modal Representation Learning (LightCRL). This method uses a single neural network titled Deep Fusion Encoder (DFE), which projects data from multiple modalities into a shared latent representation space. This reduces the overall parameter count while still delivering robust performance comparable to more complex systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04650 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lightweight Cross-Modal Representation Learning Faye, Bilal Azzag, Hanane Lebbah, Mustapha Bouchaffra, Djamel Machine Learning Artificial Intelligence Low-cost cross-modal representation learning is crucial for deriving semantic representations across diverse modalities such as text, audio, images, and video. Traditional approaches typically depend on large specialized models trained from scratch, requiring extensive datasets and resulting in high resource and time costs. To overcome these challenges, we introduce a novel approach named Lightweight Cross-Modal Representation Learning (LightCRL). This method uses a single neural network titled Deep Fusion Encoder (DFE), which projects data from multiple modalities into a shared latent representation space. This reduces the overall parameter count while still delivering robust performance comparable to more complex systems. |
| title | Lightweight Cross-Modal Representation Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2403.04650 |