Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.08606 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918299186495488 |
|---|---|
| author | Uthayasooriyar, Benno Ly, Antoine Vermet, Franck Corro, Caio |
| author_facet | Uthayasooriyar, Benno Ly, Antoine Vermet, Franck Corro, Caio |
| contents | We introduce DocPolarBERT, a layout-aware BERT model for document understanding that eliminates the need for absolute 2D positional embeddings. We extend self-attention to take into account text block positions in relative polar coordinate system rather than the Cartesian one. Despite being pre-trained on a dataset more than six times smaller than the widely used IIT-CDIP corpus, DocPolarBERT achieves state-of-the-art results. These results demonstrate that a carefully designed attention mechanism can compensate for reduced pre-training data, offering an efficient and effective alternative for document understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_08606 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures Uthayasooriyar, Benno Ly, Antoine Vermet, Franck Corro, Caio Computation and Language We introduce DocPolarBERT, a layout-aware BERT model for document understanding that eliminates the need for absolute 2D positional embeddings. We extend self-attention to take into account text block positions in relative polar coordinate system rather than the Cartesian one. Despite being pre-trained on a dataset more than six times smaller than the widely used IIT-CDIP corpus, DocPolarBERT achieves state-of-the-art results. These results demonstrate that a carefully designed attention mechanism can compensate for reduced pre-training data, offering an efficient and effective alternative for document understanding. |
| title | DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.08606 |