Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.01753 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910855793213440 |
|---|---|
| author | Mai, Quan Gauch, Susan Adams, Douglas |
| author_facet | Mai, Quan Gauch, Susan Adams, Douglas |
| contents | We present Boolean-aware attention, a novel attention mechanism that dynamically adjusts token focus based on Boolean operators (e.g., and, or, not). Our model employs specialized Boolean experts, each tailored to amplify or suppress attention for operator-specific contexts. A predefined gating mechanism activates the corresponding experts based on the detected Boolean type. Experiments on Boolean retrieval datasets demonstrate that integrating BoolAttn with BERT greatly enhances the model's capability to process Boolean queries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01753 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Boolean-aware Attention for Dense Retrieval Mai, Quan Gauch, Susan Adams, Douglas Computation and Language We present Boolean-aware attention, a novel attention mechanism that dynamically adjusts token focus based on Boolean operators (e.g., and, or, not). Our model employs specialized Boolean experts, each tailored to amplify or suppress attention for operator-specific contexts. A predefined gating mechanism activates the corresponding experts based on the detected Boolean type. Experiments on Boolean retrieval datasets demonstrate that integrating BoolAttn with BERT greatly enhances the model's capability to process Boolean queries. |
| title | Boolean-aware Attention for Dense Retrieval |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.01753 |