Reply with Sticker: New Dataset and Model for Sticker Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908543023579136 |
|---|---|
| author | Liang, Bin Wang, Bingbing Bai, Zhixin Lang, Qiwei Sun, Mingwei Hou, Kaiheng Zhou, Lanjun Xu, Ruifeng Wong, Kam-Fai |
| author_facet | Liang, Bin Wang, Bingbing Bai, Zhixin Lang, Qiwei Sun, Mingwei Hou, Kaiheng Zhou, Lanjun Xu, Ruifeng Wong, Kam-Fai |
| contents | Using stickers in online chatting is very prevalent on social media platforms, where the stickers used in the conversation can express someone's intention/emotion/attitude in a vivid, tactful, and intuitive way. Existing sticker retrieval research typically retrieves stickers based on context and the current utterance delivered by the user. That is, the stickers serve as a supplement to the current utterance. However, in the real-world scenario, using stickers to express what we want to say rather than as a supplement to our words only is also important. Therefore, in this paper, we create a new dataset for sticker retrieval in conversation, called \textbf{StickerInt}, where stickers are used to reply to previous conversations or supplement our words. Based on the created dataset, we present a simple yet effective framework for sticker retrieval in conversation based on the learning of intention and the cross-modal relationships between conversation context and stickers, coined as \textbf{Int-RA}. Specifically, we first devise a knowledge-enhanced intention predictor to introduce the intention information into the conversation representations. Subsequently, a relation-aware sticker selector is devised to retrieve the response sticker via cross-modal relationships. Extensive experiments on two datasets show that the proposed model achieves state-of-the-art performance and generalization capability in sticker retrieval. The dataset and source code of this work are released at https://github.com/HITSZ-HLT/Int-RA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_05427 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Reply with Sticker: New Dataset and Model for Sticker Retrieval Liang, Bin Wang, Bingbing Bai, Zhixin Lang, Qiwei Sun, Mingwei Hou, Kaiheng Zhou, Lanjun Xu, Ruifeng Wong, Kam-Fai Multimedia Using stickers in online chatting is very prevalent on social media platforms, where the stickers used in the conversation can express someone's intention/emotion/attitude in a vivid, tactful, and intuitive way. Existing sticker retrieval research typically retrieves stickers based on context and the current utterance delivered by the user. That is, the stickers serve as a supplement to the current utterance. However, in the real-world scenario, using stickers to express what we want to say rather than as a supplement to our words only is also important. Therefore, in this paper, we create a new dataset for sticker retrieval in conversation, called \textbf{StickerInt}, where stickers are used to reply to previous conversations or supplement our words. Based on the created dataset, we present a simple yet effective framework for sticker retrieval in conversation based on the learning of intention and the cross-modal relationships between conversation context and stickers, coined as \textbf{Int-RA}. Specifically, we first devise a knowledge-enhanced intention predictor to introduce the intention information into the conversation representations. Subsequently, a relation-aware sticker selector is devised to retrieve the response sticker via cross-modal relationships. Extensive experiments on two datasets show that the proposed model achieves state-of-the-art performance and generalization capability in sticker retrieval. The dataset and source code of this work are released at https://github.com/HITSZ-HLT/Int-RA. |
| title | Reply with Sticker: New Dataset and Model for Sticker Retrieval |
| topic | Multimedia |
| url | https://arxiv.org/abs/2403.05427 |