Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913615008759808 |
|---|---|
| author | Yamagiwa, Hiroaki Oyama, Momose Shimodaira, Hidetoshi |
| author_facet | Yamagiwa, Hiroaki Oyama, Momose Shimodaira, Hidetoshi |
| contents | Cosine similarity is widely used to measure the similarity between two embeddings, while interpretations based on angle and correlation coefficient are common. In this study, we focus on the interpretable axes of embeddings transformed by Independent Component Analysis (ICA), and propose a novel interpretation of cosine similarity as the sum of semantic similarities over axes. The normalized ICA-transformed embeddings exhibit sparsity, enhancing the interpretability of each axis, and the semantic similarity defined by the product of the components represents the shared meaning between the two embeddings along each axis. The effectiveness of this approach is demonstrated through intuitive numerical examples and thorough numerical experiments. By deriving the probability distributions that govern each component and the product of components, we propose a method for selecting statistically significant axes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10984 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings Yamagiwa, Hiroaki Oyama, Momose Shimodaira, Hidetoshi Computation and Language Cosine similarity is widely used to measure the similarity between two embeddings, while interpretations based on angle and correlation coefficient are common. In this study, we focus on the interpretable axes of embeddings transformed by Independent Component Analysis (ICA), and propose a novel interpretation of cosine similarity as the sum of semantic similarities over axes. The normalized ICA-transformed embeddings exhibit sparsity, enhancing the interpretability of each axis, and the semantic similarity defined by the product of the components represents the shared meaning between the two embeddings along each axis. The effectiveness of this approach is demonstrated through intuitive numerical examples and thorough numerical experiments. By deriving the probability distributions that govern each component and the product of components, we propose a method for selecting statistically significant axes. |
| title | Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.10984 |