Sparad:
| Huvudupphovsmän: | , |
|---|---|
| Materialtyp: | Preprint |
| Publicerad: |
2026
|
| Ämnen: | |
| Länkar: | https://arxiv.org/abs/2602.09492 |
| Taggar: |
Lägg till en tagg
Inga taggar, Lägg till första taggen!
|
Innehållsförteckning:
- Low-rank adaptation (LoRA) is a standard approach for fine-tuning large language models, yet its many variants report conflicting empirical gains, often on the same benchmarks. We show that these contradictions arise from a single overlooked factor: the batch size. When properly tuned, vanilla LoRA often matches the performance of more complex variants. We further propose a proxy-based, cost-efficient strategy for batch size tuning, revealing the impact of rank, dataset size, and model capacity on the optimal batch size. Our findings elevate batch size from a minor implementation detail to a first-order design parameter, reconciling prior inconsistencies and enabling more reliable evaluations of LoRA variants.