Omni-ID: Holistic Identity Representation Designed for Generative Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912380464660480 |
|---|---|
| author | Qian, Guocheng Wang, Kuan-Chieh Patashnik, Or Heravi, Negin Ostashev, Daniil Tulyakov, Sergey Cohen-Or, Daniel Aberman, Kfir |
| author_facet | Qian, Guocheng Wang, Kuan-Chieh Patashnik, Or Heravi, Negin Ostashev, Daniil Tulyakov, Sergey Cohen-Or, Daniel Aberman, Kfir |
| contents | We introduce Omni-ID, a novel facial representation designed specifically for generative tasks. Omni-ID encodes holistic information about an individual's appearance across diverse expressions and poses within a fixed-size representation. It consolidates information from a varied number of unstructured input images into a structured representation, where each entry represents certain global or local identity features. Our approach uses a few-to-many identity reconstruction training paradigm, where a limited set of input images is used to reconstruct multiple target images of the same individual in various poses and expressions. A multi-decoder framework is further employed to leverage the complementary strengths of diverse decoders during training. Unlike conventional representations, such as CLIP and ArcFace, which are typically learned through discriminative or contrastive objectives, Omni-ID is optimized with a generative objective, resulting in a more comprehensive and nuanced identity capture for generative tasks. Trained on our MFHQ dataset -- a multi-view facial image collection, Omni-ID demonstrates substantial improvements over conventional representations across various generative tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_09694 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Omni-ID: Holistic Identity Representation Designed for Generative Tasks Qian, Guocheng Wang, Kuan-Chieh Patashnik, Or Heravi, Negin Ostashev, Daniil Tulyakov, Sergey Cohen-Or, Daniel Aberman, Kfir Computer Vision and Pattern Recognition We introduce Omni-ID, a novel facial representation designed specifically for generative tasks. Omni-ID encodes holistic information about an individual's appearance across diverse expressions and poses within a fixed-size representation. It consolidates information from a varied number of unstructured input images into a structured representation, where each entry represents certain global or local identity features. Our approach uses a few-to-many identity reconstruction training paradigm, where a limited set of input images is used to reconstruct multiple target images of the same individual in various poses and expressions. A multi-decoder framework is further employed to leverage the complementary strengths of diverse decoders during training. Unlike conventional representations, such as CLIP and ArcFace, which are typically learned through discriminative or contrastive objectives, Omni-ID is optimized with a generative objective, resulting in a more comprehensive and nuanced identity capture for generative tasks. Trained on our MFHQ dataset -- a multi-view facial image collection, Omni-ID demonstrates substantial improvements over conventional representations across various generative tasks. |
| title | Omni-ID: Holistic Identity Representation Designed for Generative Tasks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.09694 |