Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.21862 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913875947945984 |
|---|---|
| author | Bilancia, Massimo Magro, Samuele |
| author_facet | Bilancia, Massimo Magro, Samuele |
| contents | This paper presents a variant of the Multinomial mixture model tailored to the unsupervised classification of short text data. While the Multinomial probability vector is traditionally assigned a Dirichlet prior distribution, this work explores an alternative formulation based on the Beta-Liouville distribution, which offers a more flexible correlation structure than the Dirichlet. We examine the theoretical properties of the Beta-Liouville distribution, with particular focus on its conjugacy with the Multinomial likelihood. This property enables the derivation of update equations for a CAVI (Coordinate Ascent Variational Inference) algorithm, facilitating approximate posterior inference of the model parameters. In addition, we introduce a stochastic variant of the CAVI algorithm to enhance scalability. The paper concludes with empirical examples demonstrating effective strategies for selecting the Beta-Liouville hyperparameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_21862 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hierarchical mixtures of Unigram models for short text clustering: The role of Beta-Liouville priors Bilancia, Massimo Magro, Samuele Machine Learning Computation This paper presents a variant of the Multinomial mixture model tailored to the unsupervised classification of short text data. While the Multinomial probability vector is traditionally assigned a Dirichlet prior distribution, this work explores an alternative formulation based on the Beta-Liouville distribution, which offers a more flexible correlation structure than the Dirichlet. We examine the theoretical properties of the Beta-Liouville distribution, with particular focus on its conjugacy with the Multinomial likelihood. This property enables the derivation of update equations for a CAVI (Coordinate Ascent Variational Inference) algorithm, facilitating approximate posterior inference of the model parameters. In addition, we introduce a stochastic variant of the CAVI algorithm to enhance scalability. The paper concludes with empirical examples demonstrating effective strategies for selecting the Beta-Liouville hyperparameters. |
| title | Hierarchical mixtures of Unigram models for short text clustering: The role of Beta-Liouville priors |
| topic | Machine Learning Computation |
| url | https://arxiv.org/abs/2410.21862 |