Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.22787 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915664443211776 |
|---|---|
| author | Kim, Eunsu Park, Junyeong An, Na Min Kim, Junseong Patel, Hitesh Laxmichand Jin, Jiho Kruk, Julia Agarwal, Amit Panda, Srikant Ilasariya, Fenal Ashokbhai Shim, Hyunjung Oh, Alice |
| author_facet | Kim, Eunsu Park, Junyeong An, Na Min Kim, Junseong Patel, Hitesh Laxmichand Jin, Jiho Kruk, Julia Agarwal, Amit Panda, Srikant Ilasariya, Fenal Ashokbhai Shim, Hyunjung Oh, Alice |
| contents | In a globalized world, cultural elements from diverse origins frequently appear together within a single visual scene. We refer to these as culture mixing scenarios, yet how Large Vision-Language Models (LVLMs) perceive them remains underexplored. We investigate culture mixing as a critical challenge for LVLMs and examine how current models behave when cultural items from multiple regions appear together. To systematically analyze these behaviors, we construct CultureMix, a food Visual Question Answering (VQA) benchmark with 23k diffusion-generated, human-verified culture mixing images across four subtasks: (1) food-only, (2) food+food, (3) food+background, and (4) food+food+background. Evaluating 10 LVLMs, we find consistent failures to preserve individual cultural identities in mixed settings. Models show strong background reliance, with accuracy dropping 14% when cultural backgrounds are added to food-only baselines, and they produce inconsistent predictions for identical foods across different contexts. To address these limitations, we explore three robustness strategies. We find supervised fine-tuning using a diverse culture mixing dataset substantially improve model consistency and reduce background sensitivity. We call for increased attention to culture mixing scenarios as a critical step toward developing LVLMs capable of operating reliably in culturally diverse real-world environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_22787 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models Kim, Eunsu Park, Junyeong An, Na Min Kim, Junseong Patel, Hitesh Laxmichand Jin, Jiho Kruk, Julia Agarwal, Amit Panda, Srikant Ilasariya, Fenal Ashokbhai Shim, Hyunjung Oh, Alice Computer Vision and Pattern Recognition In a globalized world, cultural elements from diverse origins frequently appear together within a single visual scene. We refer to these as culture mixing scenarios, yet how Large Vision-Language Models (LVLMs) perceive them remains underexplored. We investigate culture mixing as a critical challenge for LVLMs and examine how current models behave when cultural items from multiple regions appear together. To systematically analyze these behaviors, we construct CultureMix, a food Visual Question Answering (VQA) benchmark with 23k diffusion-generated, human-verified culture mixing images across four subtasks: (1) food-only, (2) food+food, (3) food+background, and (4) food+food+background. Evaluating 10 LVLMs, we find consistent failures to preserve individual cultural identities in mixed settings. Models show strong background reliance, with accuracy dropping 14% when cultural backgrounds are added to food-only baselines, and they produce inconsistent predictions for identical foods across different contexts. To address these limitations, we explore three robustness strategies. We find supervised fine-tuning using a diverse culture mixing dataset substantially improve model consistency and reduce background sensitivity. We call for increased attention to culture mixing scenarios as a critical step toward developing LVLMs capable of operating reliably in culturally diverse real-world environments. |
| title | World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.22787 |