InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929290043457536 |
|---|---|
| author | Wu, Xiaobao Dong, Xinshuai Nguyen, Thong Liu, Chaoqun Pan, Liangming Luu, Anh Tuan |
| author_facet | Wu, Xiaobao Dong, Xinshuai Nguyen, Thong Liu, Chaoqun Pan, Liangming Luu, Anh Tuan |
| contents | Cross-lingual topic models have been prevalent for cross-lingual text analysis by revealing aligned latent topics. However, most existing methods suffer from producing repetitive topics that hinder further analysis and performance decline caused by low-coverage dictionaries. In this paper, we propose the Cross-lingual Topic Modeling with Mutual Information (InfoCTM). Instead of the direct alignment in previous work, we propose a topic alignment with mutual information method. This works as a regularization to properly align topics and prevent degenerate topic representations of words, which mitigates the repetitive topic issue. To address the low-coverage dictionary issue, we further propose a cross-lingual vocabulary linking method that finds more linked cross-lingual words for topic alignment beyond the translations of a given dictionary. Extensive experiments on English, Chinese, and Japanese datasets demonstrate that our method outperforms state-of-the-art baselines, producing more coherent, diverse, and well-aligned topics and showing better transferability for cross-lingual classification tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_03544 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling Wu, Xiaobao Dong, Xinshuai Nguyen, Thong Liu, Chaoqun Pan, Liangming Luu, Anh Tuan Computation and Language Cross-lingual topic models have been prevalent for cross-lingual text analysis by revealing aligned latent topics. However, most existing methods suffer from producing repetitive topics that hinder further analysis and performance decline caused by low-coverage dictionaries. In this paper, we propose the Cross-lingual Topic Modeling with Mutual Information (InfoCTM). Instead of the direct alignment in previous work, we propose a topic alignment with mutual information method. This works as a regularization to properly align topics and prevent degenerate topic representations of words, which mitigates the repetitive topic issue. To address the low-coverage dictionary issue, we further propose a cross-lingual vocabulary linking method that finds more linked cross-lingual words for topic alignment beyond the translations of a given dictionary. Extensive experiments on English, Chinese, and Japanese datasets demonstrate that our method outperforms state-of-the-art baselines, producing more coherent, diverse, and well-aligned topics and showing better transferability for cross-lingual classification tasks. |
| title | InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2304.03544 |