Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.20083 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908964480876544 |
|---|---|
| author | Zhang, Yingji Carvalho, Danilo S. Freitas, André |
| author_facet | Zhang, Yingji Carvalho, Danilo S. Freitas, André |
| contents | Integrating compositional and symbolic properties into current distributional semantic spaces can enhance the interpretability, controllability, compositionality, and generalisation capabilities of Transformer-based auto-regressive language models (LMs). In this survey, we offer a novel perspective on latent space geometry through the lens of compositional semantics, a direction we refer to as \textit{semantic representation learning}. This direction enables a bridge between symbolic and distributional semantics, helping to mitigate the gap between them. We review and compare three mainstream autoencoder architectures-Variational AutoEncoder (VAE), Vector Quantised VAE (VQVAE), and Sparse AutoEncoder (SAE)-and examine the distinctive latent geometries they induce in relation to semantic structure and interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20083 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder Zhang, Yingji Carvalho, Danilo S. Freitas, André Computation and Language Integrating compositional and symbolic properties into current distributional semantic spaces can enhance the interpretability, controllability, compositionality, and generalisation capabilities of Transformer-based auto-regressive language models (LMs). In this survey, we offer a novel perspective on latent space geometry through the lens of compositional semantics, a direction we refer to as \textit{semantic representation learning}. This direction enables a bridge between symbolic and distributional semantics, helping to mitigate the gap between them. We review and compare three mainstream autoencoder architectures-Variational AutoEncoder (VAE), Vector Quantised VAE (VQVAE), and Sparse AutoEncoder (SAE)-and examine the distinctive latent geometries they induce in relation to semantic structure and interpretability. |
| title | Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.20083 |