Multi-Aspect Dense Retrieval for Large JSON Files
Fuente:
Zenodo
Enregistré dans:
| Auteur principal: | |
|---|---|
| Format: | Recurso digital |
| Langue: | anglais |
| Publié: |
Zenodo
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866901621827436544 |
|---|---|
| author | Konratt, Justin |
| author_facet | Konratt, Justin |
| contents | <p>Multi-Aspect Dense Retrieval is an interesting field, where a bi-encoder architecture is used to encode queries and documents independently and calculate their similarity score based on a similarity function like cosine similarity. To compute the query and document encoding respectively, the output embeddings of a transformer encoder, like BERT, is used to extract aspect embedding and fuse them using a weighted sum. The purpose of this paper is to analyze its applicability on large JSON files. Unfortunately, despite all the promising interim results, all models had a very poor performance. This shows the weaknesses of a too small dataset. Multiple design decisions around multi-aspect dense retrieval models, like a special loss to handle continuous aspect values and a chunk fusion network, are discussed. This paper also presents some insights into the model architecture and its interpretability before giving further ideas for improvements.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_15462622 |
| institution | Zenodo |
| language | eng |
| publishDate | 2024 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Multi-Aspect Dense Retrieval for Large JSON Files Konratt, Justin Dense Retrieval Embeddings BERT Chunking Multi-Aspect <p>Multi-Aspect Dense Retrieval is an interesting field, where a bi-encoder architecture is used to encode queries and documents independently and calculate their similarity score based on a similarity function like cosine similarity. To compute the query and document encoding respectively, the output embeddings of a transformer encoder, like BERT, is used to extract aspect embedding and fuse them using a weighted sum. The purpose of this paper is to analyze its applicability on large JSON files. Unfortunately, despite all the promising interim results, all models had a very poor performance. This shows the weaknesses of a too small dataset. Multiple design decisions around multi-aspect dense retrieval models, like a special loss to handle continuous aspect values and a chunk fusion network, are discussed. This paper also presents some insights into the model architecture and its interpretability before giving further ideas for improvements.</p> |
| title | Multi-Aspect Dense Retrieval for Large JSON Files |
| topic | Dense Retrieval Embeddings BERT Chunking Multi-Aspect |
| url | https://doi.org/10.5281/zenodo.15462622 |