EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911838884593664 |
|---|---|
| author | Zou, Henry Peng Yu, Gavin Heqing Fan, Ziwei Bu, Dan Liu, Han Dai, Peng Jia, Dongmei Caragea, Cornelia |
| author_facet | Zou, Henry Peng Yu, Gavin Heqing Fan, Ziwei Bu, Dan Liu, Han Dai, Peng Jia, Dongmei Caragea, Cornelia |
| contents | In e-commerce, accurately extracting product attribute values from multimodal data is crucial for improving user experience and operational efficiency of retailers. However, previous approaches to multimodal attribute value extraction often struggle with implicit attribute values embedded in images or text, rely heavily on extensive labeled data, and can easily confuse similar attribute values. To address these issues, we introduce EIVEN, a data- and parameter-efficient generative framework that pioneers the use of multimodal LLM for implicit attribute value extraction. EIVEN leverages the rich inherent knowledge of a pre-trained LLM and vision encoder to reduce reliance on labeled data. We also introduce a novel Learning-by-Comparison technique to reduce model confusion by enforcing attribute value comparison and difference identification. Additionally, we construct initial open-source datasets for multimodal implicit attribute value extraction. Our extensive experiments reveal that EIVEN significantly outperforms existing methods in extracting implicit attribute values while requiring less labeled data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_08886 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM Zou, Henry Peng Yu, Gavin Heqing Fan, Ziwei Bu, Dan Liu, Han Dai, Peng Jia, Dongmei Caragea, Cornelia Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Information Retrieval Machine Learning In e-commerce, accurately extracting product attribute values from multimodal data is crucial for improving user experience and operational efficiency of retailers. However, previous approaches to multimodal attribute value extraction often struggle with implicit attribute values embedded in images or text, rely heavily on extensive labeled data, and can easily confuse similar attribute values. To address these issues, we introduce EIVEN, a data- and parameter-efficient generative framework that pioneers the use of multimodal LLM for implicit attribute value extraction. EIVEN leverages the rich inherent knowledge of a pre-trained LLM and vision encoder to reduce reliance on labeled data. We also introduce a novel Learning-by-Comparison technique to reduce model confusion by enforcing attribute value comparison and difference identification. Additionally, we construct initial open-source datasets for multimodal implicit attribute value extraction. Our extensive experiments reveal that EIVEN significantly outperforms existing methods in extracting implicit attribute values while requiring less labeled data. |
| title | EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2404.08886 |