OCC-MLLM:Empowering Multimodal Large Language Model For the Understanding of Occluded Objects
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916420646862848 |
|---|---|
| author | Qiu, Wenmo Di, Xinhan |
| author_facet | Qiu, Wenmo Di, Xinhan |
| contents | There is a gap in the understanding of occluded objects in existing large-scale visual language multi-modal models. Current state-of-the-art multimodal models fail to provide satisfactory results in describing occluded objects for visual-language multimodal models through universal visual encoders. Another challenge is the limited number of datasets containing image-text pairs with a large number of occluded objects. Therefore, we introduce a novel multimodal model that applies a newly designed visual encoder to understand occluded objects in RGB images. We also introduce a large-scale visual-language pair dataset for training large-scale visual-language multimodal models and understanding occluded objects. We start our experiments comparing with the state-of-the-art models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01261 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OCC-MLLM:Empowering Multimodal Large Language Model For the Understanding of Occluded Objects Qiu, Wenmo Di, Xinhan Computer Vision and Pattern Recognition There is a gap in the understanding of occluded objects in existing large-scale visual language multi-modal models. Current state-of-the-art multimodal models fail to provide satisfactory results in describing occluded objects for visual-language multimodal models through universal visual encoders. Another challenge is the limited number of datasets containing image-text pairs with a large number of occluded objects. Therefore, we introduce a novel multimodal model that applies a newly designed visual encoder to understand occluded objects in RGB images. We also introduce a large-scale visual-language pair dataset for training large-scale visual-language multimodal models and understanding occluded objects. We start our experiments comparing with the state-of-the-art models. |
| title | OCC-MLLM:Empowering Multimodal Large Language Model For the Understanding of Occluded Objects |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.01261 |