Understanding Modality Preferences in Search Clarification

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tavakoli, Leila, Castiglia, Giovanni, Calo, Federica, Deldjoo, Yashar, Zamani, Hamed, Trippas, Johanne R.
Format:	Preprint
Published:	2024
Subjects:	Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866911943742193664
author	Tavakoli, Leila Castiglia, Giovanni Calo, Federica Deldjoo, Yashar Zamani, Hamed Trippas, Johanne R.
author_facet	Tavakoli, Leila Castiglia, Giovanni Calo, Federica Deldjoo, Yashar Zamani, Hamed Trippas, Johanne R.
contents	This study is the first attempt to explore the impact of clarification question modality on user preference in search engines. We introduce the multi-modal search clarification dataset, MIMICS-MM, containing clarification questions with associated expert-collected and model-generated images. We analyse user preferences over different clarification modes of text, image, and combination of both through crowdsourcing by taking into account image and text quality, clarity, and relevance. Our findings demonstrate that users generally prefer multi-modal clarification over uni-modal approaches. We explore the use of automated image generation techniques and compare the quality, relevance, and user preference of model-generated images with human-collected ones. The study reveals that text-to-image generation models, such as Stable Diffusion, can effectively generate multi-modal clarification questions. By investigating multi-modal clarification, this research establishes a foundation for future advancements in search systems.
format	Preprint
id	arxiv_https___arxiv_org_abs_2406_19546
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	Understanding Modality Preferences in Search Clarification Tavakoli, Leila Castiglia, Giovanni Calo, Federica Deldjoo, Yashar Zamani, Hamed Trippas, Johanne R. Human-Computer Interaction This study is the first attempt to explore the impact of clarification question modality on user preference in search engines. We introduce the multi-modal search clarification dataset, MIMICS-MM, containing clarification questions with associated expert-collected and model-generated images. We analyse user preferences over different clarification modes of text, image, and combination of both through crowdsourcing by taking into account image and text quality, clarity, and relevance. Our findings demonstrate that users generally prefer multi-modal clarification over uni-modal approaches. We explore the use of automated image generation techniques and compare the quality, relevance, and user preference of model-generated images with human-collected ones. The study reveals that text-to-image generation models, such as Stable Diffusion, can effectively generate multi-modal clarification questions. By investigating multi-modal clarification, this research establishes a foundation for future advancements in search systems.
title	Understanding Modality Preferences in Search Clarification
topic	Human-Computer Interaction
url	https://arxiv.org/abs/2406.19546