CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913021894328320 |
|---|---|
| author | Kasaei, Seyed Amir Aghayari, Ali Marioriyad, Arash Sepasian, Niki Nejad, Shayan Baghayi Fazli, MohammadAmin Baghshah, Mahdieh Soleymani Rohban, Mohammad Hossein |
| author_facet | Kasaei, Seyed Amir Aghayari, Ali Marioriyad, Arash Sepasian, Niki Nejad, Shayan Baghayi Fazli, MohammadAmin Baghshah, Mahdieh Soleymani Rohban, Mohammad Hossein |
| contents | Text-to-image diffusion models, such as Stable Diffusion, can produce high-quality and diverse images but often fail to achieve compositional alignment, particularly when prompts describe complex object relationships, attributes, or spatial arrangements. Recent inference-time approaches address this by optimizing or exploring the initial noise under the guidance of reward functions that score text-image alignment without requiring model fine-tuning. While promising, each strategy has intrinsic limitations when used alone: optimization can stall due to poor initialization or unfavorable search trajectories, whereas exploration may require a prohibitively large number of samples to locate a satisfactory output. Our analysis further shows that neither single reward metrics nor ad-hoc combinations reliably capture all aspects of compositionality, leading to weak or inconsistent guidance. To overcome these challenges, we present Category-Aware Reward-based Initial Noise Optimization and Exploration (CARINOX), a unified framework that combines noise optimization and exploration with a principled reward selection procedure grounded in correlation with human judgments. Evaluations on two complementary benchmarks covering diverse compositional challenges show that CARINOX raises average alignment scores by +16% on T2I-CompBench++ and +11% on the HRS benchmark, consistently outperforming state-of-the-art optimization and exploration-based methods across all major categories, while preserving image quality and diversity. The project page is available at https://amirkasaei.com/carinox/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17458 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration Kasaei, Seyed Amir Aghayari, Ali Marioriyad, Arash Sepasian, Niki Nejad, Shayan Baghayi Fazli, MohammadAmin Baghshah, Mahdieh Soleymani Rohban, Mohammad Hossein Computer Vision and Pattern Recognition Computation and Language Text-to-image diffusion models, such as Stable Diffusion, can produce high-quality and diverse images but often fail to achieve compositional alignment, particularly when prompts describe complex object relationships, attributes, or spatial arrangements. Recent inference-time approaches address this by optimizing or exploring the initial noise under the guidance of reward functions that score text-image alignment without requiring model fine-tuning. While promising, each strategy has intrinsic limitations when used alone: optimization can stall due to poor initialization or unfavorable search trajectories, whereas exploration may require a prohibitively large number of samples to locate a satisfactory output. Our analysis further shows that neither single reward metrics nor ad-hoc combinations reliably capture all aspects of compositionality, leading to weak or inconsistent guidance. To overcome these challenges, we present Category-Aware Reward-based Initial Noise Optimization and Exploration (CARINOX), a unified framework that combines noise optimization and exploration with a principled reward selection procedure grounded in correlation with human judgments. Evaluations on two complementary benchmarks covering diverse compositional challenges show that CARINOX raises average alignment scores by +16% on T2I-CompBench++ and +11% on the HRS benchmark, consistently outperforming state-of-the-art optimization and exploration-based methods across all major categories, while preserving image quality and diversity. The project page is available at https://amirkasaei.com/carinox/. |
| title | CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2509.17458 |