Scaling Laws for Cross-Encoder Reranking
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914484573962240 |
|---|---|
| author | Seetharaman, Rahul Bansal, Aman Zamani, Hamed Dhole, Kaustubh |
| author_facet | Seetharaman, Rahul Bansal, Aman Zamani, Hamed Dhole, Kaustubh |
| contents | Scaling laws are well studied for language models and first-stage retrieval, but not for reranking. We present the first systematic study of scaling laws for cross-encoder rerankers across pointwise, pairwise, and listwise objectives. Across model size and training exposure, ranking quality follows predictable power laws, enabling larger rerankers to be forecast from smaller runs. Using models up to 150M parameters, we forecast 400M and 1B rerankers on MSMARCO-dev and TREC DL. Beyond forecasting, we derive compute-allocation rules from the fitted joint scaling law and compare them with equal-compute checkpoints, showing that retrieval metrics often favor data-heavy scaling, though the recommendation depends on the training objective. The forecasts are accurate and typically conservative, making them useful for planning expensive large-model training. These results provide practical scaling principles for industrial reranking systems, and we will release code and evaluation protocols. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_04816 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Scaling Laws for Cross-Encoder Reranking Seetharaman, Rahul Bansal, Aman Zamani, Hamed Dhole, Kaustubh Information Retrieval Scaling laws are well studied for language models and first-stage retrieval, but not for reranking. We present the first systematic study of scaling laws for cross-encoder rerankers across pointwise, pairwise, and listwise objectives. Across model size and training exposure, ranking quality follows predictable power laws, enabling larger rerankers to be forecast from smaller runs. Using models up to 150M parameters, we forecast 400M and 1B rerankers on MSMARCO-dev and TREC DL. Beyond forecasting, we derive compute-allocation rules from the fitted joint scaling law and compare them with equal-compute checkpoints, showing that retrieval metrics often favor data-heavy scaling, though the recommendation depends on the training objective. The forecasts are accurate and typically conservative, making them useful for planning expensive large-model training. These results provide practical scaling principles for industrial reranking systems, and we will release code and evaluation protocols. |
| title | Scaling Laws for Cross-Encoder Reranking |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2603.04816 |