GloCOM: A Short Text Neural Topic Model via Global Clustering Context
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917899970543616 |
|---|---|
| author | Nguyen, Quang Duc Nguyen, Tung Nguyen, Duc Anh Van, Linh Ngo Dinh, Sang Nguyen, Thien Huu |
| author_facet | Nguyen, Quang Duc Nguyen, Tung Nguyen, Duc Anh Van, Linh Ngo Dinh, Sang Nguyen, Thien Huu |
| contents | Uncovering hidden topics from short texts is challenging for traditional and neural models due to data sparsity, which limits word co-occurrence patterns, and label sparsity, stemming from incomplete reconstruction targets. Although data aggregation offers a potential solution, existing neural topic models often overlook it due to time complexity, poor aggregation quality, and difficulty in inferring topic proportions for individual documents. In this paper, we propose a novel model, GloCOM (Global Clustering COntexts for Topic Models), which addresses these challenges by constructing aggregated global clustering contexts for short documents, leveraging text embeddings from pre-trained language models. GloCOM can infer both global topic distributions for clustering contexts and local distributions for individual short texts. Additionally, the model incorporates these global contexts to augment the reconstruction loss, effectively handling the label sparsity issue. Extensive experiments on short text datasets show that our approach outperforms other state-of-the-art models in both topic quality and document representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00525 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GloCOM: A Short Text Neural Topic Model via Global Clustering Context Nguyen, Quang Duc Nguyen, Tung Nguyen, Duc Anh Van, Linh Ngo Dinh, Sang Nguyen, Thien Huu Computation and Language Uncovering hidden topics from short texts is challenging for traditional and neural models due to data sparsity, which limits word co-occurrence patterns, and label sparsity, stemming from incomplete reconstruction targets. Although data aggregation offers a potential solution, existing neural topic models often overlook it due to time complexity, poor aggregation quality, and difficulty in inferring topic proportions for individual documents. In this paper, we propose a novel model, GloCOM (Global Clustering COntexts for Topic Models), which addresses these challenges by constructing aggregated global clustering contexts for short documents, leveraging text embeddings from pre-trained language models. GloCOM can infer both global topic distributions for clustering contexts and local distributions for individual short texts. Additionally, the model incorporates these global contexts to augment the reconstruction loss, effectively handling the label sparsity issue. Extensive experiments on short text datasets show that our approach outperforms other state-of-the-art models in both topic quality and document representations. |
| title | GloCOM: A Short Text Neural Topic Model via Global Clustering Context |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.00525 |