Learned Single-Pass Multitasking Perceptual Graphics for Immersive Displays
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908481179615232 |
|---|---|
| author | Yılmaz, Doğa Wang, He Takikawa, Towaki Ceylan, Duygu Akşit, Kaan |
| author_facet | Yılmaz, Doğa Wang, He Takikawa, Towaki Ceylan, Duygu Akşit, Kaan |
| contents | Emerging immersive display technologies efficiently utilize resources with perceptual graphics methods such as foveated rendering and denoising. Running multiple perceptual graphics methods challenges devices with limited power and computational resources. We propose a computationally-lightweight learned multitasking perceptual graphics model. Given RGB images and text-prompts, our model performs text-described perceptual tasks in a single inference step. Simply daisy-chaining multiple models or training dedicated models can lead to model management issues and exhaust computational resources. In contrast, our flexible method unlocks consistent high quality perceptual effects with reasonable compute, supporting various permutations at varied intensities using adjectives in text prompts (e.g. mildly, lightly). Text-guidance provides ease of use for dynamic requirements such as creative processes. To train our model, we propose a dataset containing source and perceptually enhanced images with corresponding text prompts. We evaluate our model on desktop and embedded platforms and validate perceptual quality through a user study. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_07836 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learned Single-Pass Multitasking Perceptual Graphics for Immersive Displays Yılmaz, Doğa Wang, He Takikawa, Towaki Ceylan, Duygu Akşit, Kaan Computer Vision and Pattern Recognition Graphics Image and Video Processing Emerging immersive display technologies efficiently utilize resources with perceptual graphics methods such as foveated rendering and denoising. Running multiple perceptual graphics methods challenges devices with limited power and computational resources. We propose a computationally-lightweight learned multitasking perceptual graphics model. Given RGB images and text-prompts, our model performs text-described perceptual tasks in a single inference step. Simply daisy-chaining multiple models or training dedicated models can lead to model management issues and exhaust computational resources. In contrast, our flexible method unlocks consistent high quality perceptual effects with reasonable compute, supporting various permutations at varied intensities using adjectives in text prompts (e.g. mildly, lightly). Text-guidance provides ease of use for dynamic requirements such as creative processes. To train our model, we propose a dataset containing source and perceptually enhanced images with corresponding text prompts. We evaluate our model on desktop and embedded platforms and validate perceptual quality through a user study. |
| title | Learned Single-Pass Multitasking Perceptual Graphics for Immersive Displays |
| topic | Computer Vision and Pattern Recognition Graphics Image and Video Processing |
| url | https://arxiv.org/abs/2408.07836 |