Learning Gaze-aware Compositional GAN
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914817603796992 |
|---|---|
| author | Aranjuelo, Nerea Huang, Siyu Arganda-Carreras, Ignacio Unzueta, Luis Otaegui, Oihana Pfister, Hanspeter Wei, Donglai |
| author_facet | Aranjuelo, Nerea Huang, Siyu Arganda-Carreras, Ignacio Unzueta, Luis Otaegui, Oihana Pfister, Hanspeter Wei, Donglai |
| contents | Gaze-annotated facial data is crucial for training deep neural networks (DNNs) for gaze estimation. However, obtaining these data is labor-intensive and requires specialized equipment due to the challenge of accurately annotating the gaze direction of a subject. In this work, we present a generative framework to create annotated gaze data by leveraging the benefits of labeled and unlabeled data sources. We propose a Gaze-aware Compositional GAN that learns to generate annotated facial images from a limited labeled dataset. Then we transfer this model to an unlabeled data domain to take advantage of the diversity it provides. Experiments demonstrate our approach's effectiveness in generating within-domain image augmentations in the ETH-XGaze dataset and cross-domain augmentations in the CelebAMask-HQ dataset domain for gaze estimation DNN training. We also show additional applications of our work, which include facial image editing and gaze redirection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_20643 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Gaze-aware Compositional GAN Aranjuelo, Nerea Huang, Siyu Arganda-Carreras, Ignacio Unzueta, Luis Otaegui, Oihana Pfister, Hanspeter Wei, Donglai Computer Vision and Pattern Recognition Artificial Intelligence Gaze-annotated facial data is crucial for training deep neural networks (DNNs) for gaze estimation. However, obtaining these data is labor-intensive and requires specialized equipment due to the challenge of accurately annotating the gaze direction of a subject. In this work, we present a generative framework to create annotated gaze data by leveraging the benefits of labeled and unlabeled data sources. We propose a Gaze-aware Compositional GAN that learns to generate annotated facial images from a limited labeled dataset. Then we transfer this model to an unlabeled data domain to take advantage of the diversity it provides. Experiments demonstrate our approach's effectiveness in generating within-domain image augmentations in the ETH-XGaze dataset and cross-domain augmentations in the CelebAMask-HQ dataset domain for gaze estimation DNN training. We also show additional applications of our work, which include facial image editing and gaze redirection. |
| title | Learning Gaze-aware Compositional GAN |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2405.20643 |