Structural Deep Encoding for Table Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912256680263680 |
|---|---|
| author | Mouravieff, Raphaël Piwowarski, Benjamin Lamprier, Sylvain |
| author_facet | Mouravieff, Raphaël Piwowarski, Benjamin Lamprier, Sylvain |
| contents | Although Transformers-based architectures excel at processing textual information, their naive adaptation for tabular data often involves flattening the table structure. This simplification can lead to the loss of essential inter-dependencies between rows, columns, and cells, while also posing scalability challenges for large tables. To address these issues, prior works have explored special tokens, structured embeddings, and sparse attention patterns. In this paper, we conduct a comprehensive analysis of tabular encoding techniques, which highlights the crucial role of attention sparsity in preserving structural information of tables. We also introduce a set of novel sparse attention mask designs for tabular data, that not only enhance computational efficiency but also preserve structural integrity, leading to better overall performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01457 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Structural Deep Encoding for Table Question Answering Mouravieff, Raphaël Piwowarski, Benjamin Lamprier, Sylvain Computation and Language Artificial Intelligence Although Transformers-based architectures excel at processing textual information, their naive adaptation for tabular data often involves flattening the table structure. This simplification can lead to the loss of essential inter-dependencies between rows, columns, and cells, while also posing scalability challenges for large tables. To address these issues, prior works have explored special tokens, structured embeddings, and sparse attention patterns. In this paper, we conduct a comprehensive analysis of tabular encoding techniques, which highlights the crucial role of attention sparsity in preserving structural information of tables. We also introduce a set of novel sparse attention mask designs for tabular data, that not only enhance computational efficiency but also preserve structural integrity, leading to better overall performance. |
| title | Structural Deep Encoding for Table Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.01457 |