Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.14322 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909082948993024 |
|---|---|
| author | Srinivasan, Hansa Schumann, Candice Sinha, Aradhana Madras, David Olanubi, Gbolahan Oluwafemi Beutel, Alex Ricco, Susanna Chen, Jilin |
| author_facet | Srinivasan, Hansa Schumann, Candice Sinha, Aradhana Madras, David Olanubi, Gbolahan Oluwafemi Beutel, Alex Ricco, Susanna Chen, Jilin |
| contents | Capturing the diversity of people in images is challenging: recent literature tends to focus on diversifying one or two attributes, requiring expensive attribute labels or building classifiers. We introduce a diverse people image ranking method which more flexibly aligns with human notions of people diversity in a less prescriptive, label-free manner. The Perception-Aligned Text-derived Human representation Space (PATHS) aims to capture all or many relevant features of people-related diversity, and, when used as the representation space in the standard Maximal Marginal Relevance (MMR) ranking algorithm, is better able to surface a range of types of people-related diversity (e.g. disability, cultural attire). PATHS is created in two stages. First, a text-guided approach is used to extract a person-diversity representation from a pre-trained image-text model. Then this representation is fine-tuned on perception judgments from human annotators so that it captures the aspects of people-related similarity that humans find most salient. Empirical results show that the PATHS method achieves diversity better than baseline methods, according to side-by-side ratings from human annotators. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14322 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images Srinivasan, Hansa Schumann, Candice Sinha, Aradhana Madras, David Olanubi, Gbolahan Oluwafemi Beutel, Alex Ricco, Susanna Chen, Jilin Computer Vision and Pattern Recognition Computers and Society Capturing the diversity of people in images is challenging: recent literature tends to focus on diversifying one or two attributes, requiring expensive attribute labels or building classifiers. We introduce a diverse people image ranking method which more flexibly aligns with human notions of people diversity in a less prescriptive, label-free manner. The Perception-Aligned Text-derived Human representation Space (PATHS) aims to capture all or many relevant features of people-related diversity, and, when used as the representation space in the standard Maximal Marginal Relevance (MMR) ranking algorithm, is better able to surface a range of types of people-related diversity (e.g. disability, cultural attire). PATHS is created in two stages. First, a text-guided approach is used to extract a person-diversity representation from a pre-trained image-text model. Then this representation is fine-tuned on perception judgments from human annotators so that it captures the aspects of people-related similarity that humans find most salient. Empirical results show that the PATHS method achieves diversity better than baseline methods, according to side-by-side ratings from human annotators. |
| title | Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images |
| topic | Computer Vision and Pattern Recognition Computers and Society |
| url | https://arxiv.org/abs/2401.14322 |