Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Ranking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911119521611776 |
|---|---|
| author | Kokate, Prathamesh Sarnaik, Mitali Khopade, Manavi Takalikar, Mukta Joshi, Raviraj |
| author_facet | Kokate, Prathamesh Sarnaik, Mitali Khopade, Manavi Takalikar, Mukta Joshi, Raviraj |
| contents | Transformer-based models like BERT excel at short text classification but struggle with long document classification (LDC) due to input length limitations and computational inefficiencies. In this work, we propose an efficient, zero-shot approach to LDC that leverages sentence ranking to reduce input context without altering the model architecture. Our method enables the adaptation of models trained on short texts, such as headlines, to long-form documents by selecting the most informative sentences using a TF-IDF-based ranking strategy. Using the MahaNews dataset of long Marathi news articles, we evaluate three context reduction strategies that prioritize essential content while preserving classification accuracy. Our results show that retaining only the top 50\% ranked sentences maintains performance comparable to full-document inference while reducing inference time by up to 35\%. This demonstrates that sentence ranking is a simple yet effective technique for scalable and efficient zero-shot LDC. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17490 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Ranking Kokate, Prathamesh Sarnaik, Mitali Khopade, Manavi Takalikar, Mukta Joshi, Raviraj Computation and Language Machine Learning Transformer-based models like BERT excel at short text classification but struggle with long document classification (LDC) due to input length limitations and computational inefficiencies. In this work, we propose an efficient, zero-shot approach to LDC that leverages sentence ranking to reduce input context without altering the model architecture. Our method enables the adaptation of models trained on short texts, such as headlines, to long-form documents by selecting the most informative sentences using a TF-IDF-based ranking strategy. Using the MahaNews dataset of long Marathi news articles, we evaluate three context reduction strategies that prioritize essential content while preserving classification accuracy. Our results show that retaining only the top 50\% ranked sentences maintains performance comparable to full-document inference while reducing inference time by up to 35\%. This demonstrates that sentence ranking is a simple yet effective technique for scalable and efficient zero-shot LDC. |
| title | Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Ranking |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2508.17490 |