On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914662098927616 |
|---|---|
| author | Wu, Xiaobao Pan, Fengjun Nguyen, Thong Feng, Yichao Liu, Chaoqun Nguyen, Cong-Duy Luu, Anh Tuan |
| author_facet | Wu, Xiaobao Pan, Fengjun Nguyen, Thong Feng, Yichao Liu, Chaoqun Nguyen, Cong-Duy Luu, Anh Tuan |
| contents | Hierarchical topic modeling aims to discover latent topics from a corpus and organize them into a hierarchy to understand documents with desirable semantic granularity. However, existing work struggles with producing topic hierarchies of low affinity, rationality, and diversity, which hampers document understanding. To overcome these challenges, we in this paper propose Transport Plan and Context-aware Hierarchical Topic Model (TraCo). Instead of early simple topic dependencies, we propose a transport plan dependency method. It constrains dependencies to ensure their sparsity and balance, and also regularizes topic hierarchy building with them. This improves affinity and diversity of hierarchies. We further propose a context-aware disentangled decoder. Rather than previously entangled decoding, it distributes different semantic granularity to topics at different levels by disentangled decoding. This facilitates the rationality of hierarchies. Experiments on benchmark datasets demonstrate that our method surpasses state-of-the-art baselines, effectively improving the affinity, rationality, and diversity of hierarchical topic modeling with better performance on downstream tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14113 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling Wu, Xiaobao Pan, Fengjun Nguyen, Thong Feng, Yichao Liu, Chaoqun Nguyen, Cong-Duy Luu, Anh Tuan Computation and Language Hierarchical topic modeling aims to discover latent topics from a corpus and organize them into a hierarchy to understand documents with desirable semantic granularity. However, existing work struggles with producing topic hierarchies of low affinity, rationality, and diversity, which hampers document understanding. To overcome these challenges, we in this paper propose Transport Plan and Context-aware Hierarchical Topic Model (TraCo). Instead of early simple topic dependencies, we propose a transport plan dependency method. It constrains dependencies to ensure their sparsity and balance, and also regularizes topic hierarchy building with them. This improves affinity and diversity of hierarchies. We further propose a context-aware disentangled decoder. Rather than previously entangled decoding, it distributes different semantic granularity to topics at different levels by disentangled decoding. This facilitates the rationality of hierarchies. Experiments on benchmark datasets demonstrate that our method surpasses state-of-the-art baselines, effectively improving the affinity, rationality, and diversity of hierarchical topic modeling with better performance on downstream tasks. |
| title | On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.14113 |