Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911849857941504 |
|---|---|
| author | Zeng, Hansi Luo, Chen Zamani, Hamed |
| author_facet | Zeng, Hansi Luo, Chen Zamani, Hamed |
| contents | This paper introduces PAG-a novel optimization and decoding approach that guides autoregressive generation of document identifiers in generative retrieval models through simultaneous decoding. To this aim, PAG constructs a set-based and sequential identifier for each document. Motivated by the bag-of-words assumption in information retrieval, the set-based identifier is built on lexical tokens. The sequential identifier, on the other hand, is obtained via quantizing relevance-based representations of documents. Extensive experiments on MSMARCO and TREC Deep Learning Track data reveal that PAG outperforms the state-of-the-art generative retrieval model by a large margin (e.g., 15.6% MRR improvements on MS MARCO), while achieving 22x speed up in terms of query latency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14600 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding Zeng, Hansi Luo, Chen Zamani, Hamed Information Retrieval Computation and Language This paper introduces PAG-a novel optimization and decoding approach that guides autoregressive generation of document identifiers in generative retrieval models through simultaneous decoding. To this aim, PAG constructs a set-based and sequential identifier for each document. Motivated by the bag-of-words assumption in information retrieval, the set-based identifier is built on lexical tokens. The sequential identifier, on the other hand, is obtained via quantizing relevance-based representations of documents. Extensive experiments on MSMARCO and TREC Deep Learning Track data reveal that PAG outperforms the state-of-the-art generative retrieval model by a large margin (e.g., 15.6% MRR improvements on MS MARCO), while achieving 22x speed up in terms of query latency. |
| title | Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2404.14600 |