Nutrition Estimation for Dietary Management: A Transformer Approach with Depth Sensing
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911902174543872 |
|---|---|
| author | Kwan, Zhengyi Zhang, Wei Wang, Zhengkui Ng, Aik Beng See, Simon |
| author_facet | Kwan, Zhengyi Zhang, Wei Wang, Zhengkui Ng, Aik Beng See, Simon |
| contents | Nutrition estimation is crucial for effective dietary management and overall health and well-being. Existing methods often struggle with sub-optimal accuracy and can be time-consuming. In this paper, we propose NuNet, a transformer-based network designed for nutrition estimation that utilizes both RGB and depth information from food images. We have designed and implemented a multi-scale encoder and decoder, along with two types of feature fusion modules, specialized for estimating five nutritional factors. These modules effectively balance the efficiency and effectiveness of feature extraction with flexible usage of our customized attention mechanisms and fusion strategies. Our experimental study shows that NuNet outperforms its variants and existing solutions significantly for nutrition estimation. It achieves an error rate of 15.65%, the lowest known to us, largely due to our multi-scale architecture and fusion modules. This research holds practical values for dietary management with huge potential for transnational research and deployment and could inspire other applications involving multiple data types with varying degrees of importance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01938 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nutrition Estimation for Dietary Management: A Transformer Approach with Depth Sensing Kwan, Zhengyi Zhang, Wei Wang, Zhengkui Ng, Aik Beng See, Simon Computer Vision and Pattern Recognition Multimedia Nutrition estimation is crucial for effective dietary management and overall health and well-being. Existing methods often struggle with sub-optimal accuracy and can be time-consuming. In this paper, we propose NuNet, a transformer-based network designed for nutrition estimation that utilizes both RGB and depth information from food images. We have designed and implemented a multi-scale encoder and decoder, along with two types of feature fusion modules, specialized for estimating five nutritional factors. These modules effectively balance the efficiency and effectiveness of feature extraction with flexible usage of our customized attention mechanisms and fusion strategies. Our experimental study shows that NuNet outperforms its variants and existing solutions significantly for nutrition estimation. It achieves an error rate of 15.65%, the lowest known to us, largely due to our multi-scale architecture and fusion modules. This research holds practical values for dietary management with huge potential for transnational research and deployment and could inspire other applications involving multiple data types with varying degrees of importance. |
| title | Nutrition Estimation for Dietary Management: A Transformer Approach with Depth Sensing |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2406.01938 |