PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929669898502144 |
|---|---|
| author | Hu, Xinting Wang, Haoran Lenssen, Jan Eric Schiele, Bernt |
| author_facet | Hu, Xinting Wang, Haoran Lenssen, Jan Eric Schiele, Bernt |
| contents | We introduce PersonaHOI, a training- and tuning-free framework that fuses a general StableDiffusion model with a personalized face diffusion (PFD) model to generate identity-consistent human-object interaction (HOI) images. While existing PFD models have advanced significantly, they often overemphasize facial features at the expense of full-body coherence, PersonaHOI introduces an additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By incorporating cross-attention constraints in the PFD branch and spatial merging at both latent and residual levels, PersonaHOI preserves personalized facial details while ensuring interactive non-facial regions. Experiments, validated by a novel interaction alignment metric, demonstrate the superior realism and scalability of PersonaHOI, establishing a new standard for practical personalized face with HOI generation. Our code will be available at https://github.com/JoyHuYY1412/PersonaHOI |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05823 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation Hu, Xinting Wang, Haoran Lenssen, Jan Eric Schiele, Bernt Computer Vision and Pattern Recognition We introduce PersonaHOI, a training- and tuning-free framework that fuses a general StableDiffusion model with a personalized face diffusion (PFD) model to generate identity-consistent human-object interaction (HOI) images. While existing PFD models have advanced significantly, they often overemphasize facial features at the expense of full-body coherence, PersonaHOI introduces an additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By incorporating cross-attention constraints in the PFD branch and spatial merging at both latent and residual levels, PersonaHOI preserves personalized facial details while ensuring interactive non-facial regions. Experiments, validated by a novel interaction alignment metric, demonstrate the superior realism and scalability of PersonaHOI, establishing a new standard for practical personalized face with HOI generation. Our code will be available at https://github.com/JoyHuYY1412/PersonaHOI |
| title | PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.05823 |