Efficient Personalization of Generative Models via Optimal Experimental Design
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915800970952704 |
|---|---|
| author | Schacht, Guy Sheebaelhamd, Ziyad De Santi, Riccardo Mutný, Mojmír Krause, Andreas |
| author_facet | Schacht, Guy Sheebaelhamd, Ziyad De Santi, Riccardo Mutný, Mojmír Krause, Andreas |
| contents | Preference learning from human feedback has the ability to align generative models with the needs of end-users. Human feedback is costly and time-consuming to obtain, which creates demand for data-efficient query selection methods. This work presents a novel approach that leverages optimal experimental design to ask humans the most informative preference queries, from which we can elucidate the latent reward function modeling user preferences efficiently. We formulate the problem of preference query selection as the one that maximizes the information about the underlying latent preference model. We show that this problem has a convex optimization formulation, and introduce a statistically and computationally efficient algorithm ED-PBRL that is supported by theoretical guarantees and can efficiently construct structured queries such as images or text. We empirically present the proposed framework by personalizing a text-to-image generative model to user-specific styles, showing that it requires less preference queries compared to random query selection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_19057 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Personalization of Generative Models via Optimal Experimental Design Schacht, Guy Sheebaelhamd, Ziyad De Santi, Riccardo Mutný, Mojmír Krause, Andreas Machine Learning Information Theory Preference learning from human feedback has the ability to align generative models with the needs of end-users. Human feedback is costly and time-consuming to obtain, which creates demand for data-efficient query selection methods. This work presents a novel approach that leverages optimal experimental design to ask humans the most informative preference queries, from which we can elucidate the latent reward function modeling user preferences efficiently. We formulate the problem of preference query selection as the one that maximizes the information about the underlying latent preference model. We show that this problem has a convex optimization formulation, and introduce a statistically and computationally efficient algorithm ED-PBRL that is supported by theoretical guarantees and can efficiently construct structured queries such as images or text. We empirically present the proposed framework by personalizing a text-to-image generative model to user-specific styles, showing that it requires less preference queries compared to random query selection. |
| title | Efficient Personalization of Generative Models via Optimal Experimental Design |
| topic | Machine Learning Information Theory |
| url | https://arxiv.org/abs/2512.19057 |