GRPose: Learning Graph Relations for Human Image Generation with Pose Priors
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916543026167808 |
|---|---|
| author | Yin, Xiangchen Di, Donglin Fan, Lei Li, Hao Chen, Wei Gou, Xiaofei Song, Yang Sun, Xiao Yang, Xun |
| author_facet | Yin, Xiangchen Di, Donglin Fan, Lei Li, Hao Chen, Wei Gou, Xiaofei Song, Yang Sun, Xiao Yang, Xun |
| contents | Recent methods using diffusion models have made significant progress in human image generation with various control signals such as pose priors. However, existing efforts are still struggling to generate high-quality images with consistent pose alignment, resulting in unsatisfactory output. In this paper, we propose a framework that delves into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models to capture the intrinsic associations between different pose parts. A Progressive Graph Integrator (PGI) is designed to learn the spatial relationships of the pose priors with the graph structure, adopting a hierarchical strategy within an Adapter to gradually propagate information across different pose parts. Besides, a pose perception loss is introduced based on a pretrained pose estimation network to minimize the pose differences. Extensive qualitative and quantitative experiments conducted on the Human-Art and LAION-Human datasets clearly demonstrate that our model can achieve significant performance improvement over the latest benchmark models. The code is available at \url{https://xiangchenyin.github.io/GRPose/}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_16540 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GRPose: Learning Graph Relations for Human Image Generation with Pose Priors Yin, Xiangchen Di, Donglin Fan, Lei Li, Hao Chen, Wei Gou, Xiaofei Song, Yang Sun, Xiao Yang, Xun Computer Vision and Pattern Recognition Recent methods using diffusion models have made significant progress in human image generation with various control signals such as pose priors. However, existing efforts are still struggling to generate high-quality images with consistent pose alignment, resulting in unsatisfactory output. In this paper, we propose a framework that delves into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models to capture the intrinsic associations between different pose parts. A Progressive Graph Integrator (PGI) is designed to learn the spatial relationships of the pose priors with the graph structure, adopting a hierarchical strategy within an Adapter to gradually propagate information across different pose parts. Besides, a pose perception loss is introduced based on a pretrained pose estimation network to minimize the pose differences. Extensive qualitative and quantitative experiments conducted on the Human-Art and LAION-Human datasets clearly demonstrate that our model can achieve significant performance improvement over the latest benchmark models. The code is available at \url{https://xiangchenyin.github.io/GRPose/}. |
| title | GRPose: Learning Graph Relations for Human Image Generation with Pose Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.16540 |