Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912459484299264 |
|---|---|
| author | Bendada, Walid Salha-Galvan, Guillaume Hennequin, Romain Bontempelli, Théo Bouabça, Thomas Cazenave, Tristan |
| author_facet | Bendada, Walid Salha-Galvan, Guillaume Hennequin, Romain Bontempelli, Théo Bouabça, Thomas Cazenave, Tristan |
| contents | This paper introduces von Mises-Fisher exploration (vMF-exp), a scalable method for exploring large action sets in reinforcement learning problems where hyperspherical embedding vectors represent these actions. vMF-exp involves initially sampling a state embedding representation using a von Mises-Fisher distribution, then exploring this representation's nearest neighbors, which scales to virtually unlimited numbers of candidate actions. We show that, under theoretical assumptions, vMF-exp asymptotically maintains the same probability of exploring each action as Boltzmann Exploration (B-exp), a popular alternative that, nonetheless, suffers from scalability issues as it requires computing softmax values for each action. Consequently, vMF-exp serves as a scalable alternative to B-exp for exploring large action sets with hyperspherical embeddings. Experiments on simulated data, real-world public data, and the successful large-scale deployment of vMF-exp on the recommender system of a global music streaming service empirically validate the key properties of the proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_00518 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling Bendada, Walid Salha-Galvan, Guillaume Hennequin, Romain Bontempelli, Théo Bouabça, Thomas Cazenave, Tristan Machine Learning Information Retrieval This paper introduces von Mises-Fisher exploration (vMF-exp), a scalable method for exploring large action sets in reinforcement learning problems where hyperspherical embedding vectors represent these actions. vMF-exp involves initially sampling a state embedding representation using a von Mises-Fisher distribution, then exploring this representation's nearest neighbors, which scales to virtually unlimited numbers of candidate actions. We show that, under theoretical assumptions, vMF-exp asymptotically maintains the same probability of exploring each action as Boltzmann Exploration (B-exp), a popular alternative that, nonetheless, suffers from scalability issues as it requires computing softmax values for each action. Consequently, vMF-exp serves as a scalable alternative to B-exp for exploring large action sets with hyperspherical embeddings. Experiments on simulated data, real-world public data, and the successful large-scale deployment of vMF-exp on the recommender system of a global music streaming service empirically validate the key properties of the proposed method. |
| title | Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling |
| topic | Machine Learning Information Retrieval |
| url | https://arxiv.org/abs/2507.00518 |