Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.06239 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914183025524736 |
|---|---|
| author | Sundararaman, Dhanasekar Li, Keying Xiong, Wayne Garg, Aashna |
| author_facet | Sundararaman, Dhanasekar Li, Keying Xiong, Wayne Garg, Aashna |
| contents | We present LOCUS (LOw-cost Customization for Universal Specialization), a pipeline that consumes few-shot data to streamline the construction and training of NLP models through targeted retrieval, synthetic data generation, and parameter-efficient tuning. With only a small number of labeled examples, LOCUS discovers pertinent data in a broad repository, synthesizes additional training samples via in-context data generation, and fine-tunes models using either full or low-rank (LoRA) parameter adaptation. Our approach targets named entity recognition (NER) and text classification (TC) benchmarks, consistently outperforming strong baselines (including GPT-4o) while substantially lowering costs and model sizes. Our resultant memory-optimized models retain 99% of fully fine-tuned accuracy while using barely 5% of the memory footprint, also beating GPT-4o on several benchmarks with less than 1% of its parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_06239 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LOCUS: A System and Method for Low-Cost Customization for Universal Specialization Sundararaman, Dhanasekar Li, Keying Xiong, Wayne Garg, Aashna Computation and Language We present LOCUS (LOw-cost Customization for Universal Specialization), a pipeline that consumes few-shot data to streamline the construction and training of NLP models through targeted retrieval, synthetic data generation, and parameter-efficient tuning. With only a small number of labeled examples, LOCUS discovers pertinent data in a broad repository, synthesizes additional training samples via in-context data generation, and fine-tunes models using either full or low-rank (LoRA) parameter adaptation. Our approach targets named entity recognition (NER) and text classification (TC) benchmarks, consistently outperforming strong baselines (including GPT-4o) while substantially lowering costs and model sizes. Our resultant memory-optimized models retain 99% of fully fine-tuned accuracy while using barely 5% of the memory footprint, also beating GPT-4o on several benchmarks with less than 1% of its parameters. |
| title | LOCUS: A System and Method for Low-Cost Customization for Universal Specialization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2512.06239 |