Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.23656 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910248162295808 |
|---|---|
| author | Henry, Maxim Deliège, Adrien Piérard, Sébastien Van Droogenbroeck, Marc |
| author_facet | Henry, Maxim Deliège, Adrien Piérard, Sébastien Van Droogenbroeck, Marc |
| contents | Training high-capacity vision models from scratch requires substantial computational resources. To improve training efficiency of a wide target model, existing growth methods often assume the availability of narrower models, obscuring the true computational cost of the entire pipeline. We propose an efficient training protocol, RBDC, that builds wide models by coupling in a parameter-free block-diagonal way narrower, independently trained models in a recursive way. This allows a flexible allocation of the training budget available across all the models involved. Evaluated with vision transformers (DeiT) and convolutional networks (ResNet) on ImageNet, our RBDC training protocol shows a much better efficiency than models trained from scratch with the standard protocol, yielding 30% FLOPs reduction at similar test accuracies. It also achieves higher performances at same training FLOPs than training protocols from the model growth literature. Finally, we show that our models can serve as better backbones than their original counterparts for downstream object detection and instance segmentation tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_23656 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models Henry, Maxim Deliège, Adrien Piérard, Sébastien Van Droogenbroeck, Marc Computer Vision and Pattern Recognition Training high-capacity vision models from scratch requires substantial computational resources. To improve training efficiency of a wide target model, existing growth methods often assume the availability of narrower models, obscuring the true computational cost of the entire pipeline. We propose an efficient training protocol, RBDC, that builds wide models by coupling in a parameter-free block-diagonal way narrower, independently trained models in a recursive way. This allows a flexible allocation of the training budget available across all the models involved. Evaluated with vision transformers (DeiT) and convolutional networks (ResNet) on ImageNet, our RBDC training protocol shows a much better efficiency than models trained from scratch with the standard protocol, yielding 30% FLOPs reduction at similar test accuracies. It also achieves higher performances at same training FLOPs than training protocols from the model growth literature. Finally, we show that our models can serve as better backbones than their original counterparts for downstream object detection and instance segmentation tasks. |
| title | Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.23656 |