New York Smells: A Large Multimodal Dataset for Olfaction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918217991061504 |
|---|---|
| author | Ozguroglu, Ege Liang, Junbang Liu, Ruoshi Chiquier, Mia DeTienne, Michael Qian, Wesley Wei Horowitz, Alexandra Owens, Andrew Vondrick, Carl |
| author_facet | Ozguroglu, Ege Liang, Junbang Liu, Ruoshi Chiquier, Mia DeTienne, Michael Qian, Wesley Wei Horowitz, Alexandra Owens, Andrew Vondrick, Carl |
| contents | While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key bottleneck is the lack of diverse, multimodal olfactory training data collected in natural settings. We present New York Smells, a large dataset of paired image and olfactory signals captured ``in the wild.'' Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments, with approximately 70$\times$ more objects than existing olfactory datasets. Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species. Through experiments on our dataset, we find that visual data enables cross-modal olfactory representation learning, and that our learned olfactory representations outperform widely-used hand-crafted features. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20544 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | New York Smells: A Large Multimodal Dataset for Olfaction Ozguroglu, Ege Liang, Junbang Liu, Ruoshi Chiquier, Mia DeTienne, Michael Qian, Wesley Wei Horowitz, Alexandra Owens, Andrew Vondrick, Carl Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key bottleneck is the lack of diverse, multimodal olfactory training data collected in natural settings. We present New York Smells, a large dataset of paired image and olfactory signals captured ``in the wild.'' Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments, with approximately 70$\times$ more objects than existing olfactory datasets. Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species. Through experiments on our dataset, we find that visual data enables cross-modal olfactory representation learning, and that our learned olfactory representations outperform widely-used hand-crafted features. |
| title | New York Smells: A Large Multimodal Dataset for Olfaction |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2511.20544 |