Self-Supervised Pre-Training for Table Structure Recognition Transformer
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913243789787136 |
|---|---|
| author | Peng, ShengYun Lee, Seongmin Wang, Xiaojing Balasubramaniyan, Rajarajeswari Chau, Duen Horng |
| author_facet | Peng, ShengYun Lee, Seongmin Wang, Xiaojing Balasubramaniyan, Rajarajeswari Chau, Duen Horng |
| contents | Table structure recognition (TSR) aims to convert tabular images into a machine-readable format. Although hybrid convolutional neural network (CNN)-transformer architecture is widely used in existing approaches, linear projection transformer has outperformed the hybrid architecture in numerous vision tasks due to its simplicity and efficiency. However, existing research has demonstrated that a direct replacement of CNN backbone with linear projection leads to a marked performance drop. In this work, we resolve the issue by proposing a self-supervised pre-training (SSP) method for TSR transformers. We discover that the performance gap between the linear projection transformer and the hybrid CNN-transformer can be mitigated by SSP of the visual encoder in the TSR model. We conducted reproducible ablation studies and open-sourced our code at https://github.com/poloclub/unitable to enhance transparency, inspire innovations, and facilitate fair comparisons in our domain as tables are a promising modality for representation learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15578 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Self-Supervised Pre-Training for Table Structure Recognition Transformer Peng, ShengYun Lee, Seongmin Wang, Xiaojing Balasubramaniyan, Rajarajeswari Chau, Duen Horng Computer Vision and Pattern Recognition Table structure recognition (TSR) aims to convert tabular images into a machine-readable format. Although hybrid convolutional neural network (CNN)-transformer architecture is widely used in existing approaches, linear projection transformer has outperformed the hybrid architecture in numerous vision tasks due to its simplicity and efficiency. However, existing research has demonstrated that a direct replacement of CNN backbone with linear projection leads to a marked performance drop. In this work, we resolve the issue by proposing a self-supervised pre-training (SSP) method for TSR transformers. We discover that the performance gap between the linear projection transformer and the hybrid CNN-transformer can be mitigated by SSP of the visual encoder in the TSR model. We conducted reproducible ablation studies and open-sourced our code at https://github.com/poloclub/unitable to enhance transparency, inspire innovations, and facilitate fair comparisons in our domain as tables are a promising modality for representation learning. |
| title | Self-Supervised Pre-Training for Table Structure Recognition Transformer |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.15578 |