LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915628217008128 |
|---|---|
| author | Wang, Qing Ngo, Chong-Wah Lim, Ee-Peng Sun, Qianru |
| author_facet | Wang, Qing Ngo, Chong-Wah Lim, Ee-Peng Sun, Qianru |
| contents | Training a model for food recognition is challenging because the training samples, which are typically crawled from the Internet, are visually different from the pictures captured by users in the free-living environment. In addition to this domain-shift problem, the real-world food datasets tend to be long-tailed distributed and some dishes of different categories exhibit subtle variations that are difficult to distinguish visually. In this paper, we present a framework empowered with large language models (LLMs) to address these challenges in food recognition. We first leverage LLMs to parse food images to generate food titles and ingredients. Then, we project the generated texts and food images from different domains to a shared embedding space to maximize the pair similarities. Finally, we take the aligned features of both modalities for recognition. With this simple framework, we show that our proposed approach can outperform the existing approaches tailored for long-tailed data distribution, domain adaptation, and fine-grained classification, respectively, on two food datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_16037 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets Wang, Qing Ngo, Chong-Wah Lim, Ee-Peng Sun, Qianru Computer Vision and Pattern Recognition Training a model for food recognition is challenging because the training samples, which are typically crawled from the Internet, are visually different from the pictures captured by users in the free-living environment. In addition to this domain-shift problem, the real-world food datasets tend to be long-tailed distributed and some dishes of different categories exhibit subtle variations that are difficult to distinguish visually. In this paper, we present a framework empowered with large language models (LLMs) to address these challenges in food recognition. We first leverage LLMs to parse food images to generate food titles and ingredients. Then, we project the generated texts and food images from different domains to a shared embedding space to maximize the pair similarities. Finally, we take the aligned features of both modalities for recognition. With this simple framework, we show that our proposed approach can outperform the existing approaches tailored for long-tailed data distribution, domain adaptation, and fine-grained classification, respectively, on two food datasets. |
| title | LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.16037 |