Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.09556 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915292160983040 |
|---|---|
| author | Kardos, Márton Kostkan, Jan Vermillet, Arnault-Quentin Nielbo, Kristoffer Enevoldsen, Kenneth Rocca, Roberta |
| author_facet | Kardos, Márton Kostkan, Jan Vermillet, Arnault-Quentin Nielbo, Kristoffer Enevoldsen, Kenneth Rocca, Roberta |
| contents | Topic models are useful tools for discovering latent semantic structures in large textual corpora. Recent efforts have been oriented at incorporating contextual representations in topic modeling and have been shown to outperform classical topic models. These approaches are typically slow, volatile, and require heavy preprocessing for optimal results. We present Semantic Signal Separation ($S^3$), a theory-driven topic modeling approach in neural embedding spaces. $S^3$ conceptualizes topics as independent axes of semantic space and uncovers these by decomposing contextualized document embeddings using Independent Component Analysis. Our approach provides diverse and highly coherent topics, requires no preprocessing, and is demonstrated to be the fastest contextual topic model, being, on average, 4.5x faster than the runner-up BERTopic. We offer an implementation of $S^3$, and all contextual baselines, in the Turftopic Python package. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09556 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | $S^3$ -- Semantic Signal Separation Kardos, Márton Kostkan, Jan Vermillet, Arnault-Quentin Nielbo, Kristoffer Enevoldsen, Kenneth Rocca, Roberta Machine Learning Computation and Language I.2.7 Topic models are useful tools for discovering latent semantic structures in large textual corpora. Recent efforts have been oriented at incorporating contextual representations in topic modeling and have been shown to outperform classical topic models. These approaches are typically slow, volatile, and require heavy preprocessing for optimal results. We present Semantic Signal Separation ($S^3$), a theory-driven topic modeling approach in neural embedding spaces. $S^3$ conceptualizes topics as independent axes of semantic space and uncovers these by decomposing contextualized document embeddings using Independent Component Analysis. Our approach provides diverse and highly coherent topics, requires no preprocessing, and is demonstrated to be the fastest contextual topic model, being, on average, 4.5x faster than the runner-up BERTopic. We offer an implementation of $S^3$, and all contextual baselines, in the Turftopic Python package. |
| title | $S^3$ -- Semantic Signal Separation |
| topic | Machine Learning Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2406.09556 |