Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910900196212736 |
|---|---|
| author | Hoang, Cong Duy Vu Tangari, Gioacchino Lanfranchi, Clemence Guo, Dalu Cayet, Paul Siu, Steve Dharmasiri, Don Li, Yuan-Fang Duong, Long Hilloulin, Damien Patra, Rhicheek Hong, Sungpack Chafi, Hassan |
| author_facet | Hoang, Cong Duy Vu Tangari, Gioacchino Lanfranchi, Clemence Guo, Dalu Cayet, Paul Siu, Steve Dharmasiri, Don Li, Yuan-Fang Duong, Long Hilloulin, Damien Patra, Rhicheek Hong, Sungpack Chafi, Hassan |
| contents | The growing adoption of large language models (LLMs) in business applications has amplified interest in Natural Language to SQL (NL2SQL) solutions, in which there is competing demand for high performance and efficiency. Domain- and customer-specific requirements further complicate the problem. To address this conundrum, we introduce Distill-C, a distilled customization framework tailored for NL2SQL tasks. Distill-C utilizes large teacher LLMs to produce high-quality synthetic data through a robust and scalable pipeline. Finetuning smaller and open-source LLMs on this synthesized data enables them to rival or outperform teacher models an order of magnitude larger. Evaluated on multiple challenging benchmarks, Distill-C achieves an average improvement of 36% in execution accuracy compared to the base models from three distinct LLM families. Additionally, on three internal customer benchmarks, Distill-C demonstrates a 22.6% performance improvement over the base models. Our results demonstrate that Distill-C is an effective, high-performing and generalizable approach for deploying lightweight yet powerful NL2SQL models, delivering exceptional accuracies while maintaining low computational cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_00048 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs Hoang, Cong Duy Vu Tangari, Gioacchino Lanfranchi, Clemence Guo, Dalu Cayet, Paul Siu, Steve Dharmasiri, Don Li, Yuan-Fang Duong, Long Hilloulin, Damien Patra, Rhicheek Hong, Sungpack Chafi, Hassan Computation and Language Artificial Intelligence The growing adoption of large language models (LLMs) in business applications has amplified interest in Natural Language to SQL (NL2SQL) solutions, in which there is competing demand for high performance and efficiency. Domain- and customer-specific requirements further complicate the problem. To address this conundrum, we introduce Distill-C, a distilled customization framework tailored for NL2SQL tasks. Distill-C utilizes large teacher LLMs to produce high-quality synthetic data through a robust and scalable pipeline. Finetuning smaller and open-source LLMs on this synthesized data enables them to rival or outperform teacher models an order of magnitude larger. Evaluated on multiple challenging benchmarks, Distill-C achieves an average improvement of 36% in execution accuracy compared to the base models from three distinct LLM families. Additionally, on three internal customer benchmarks, Distill-C demonstrates a 22.6% performance improvement over the base models. Our results demonstrate that Distill-C is an effective, high-performing and generalizable approach for deploying lightweight yet powerful NL2SQL models, delivering exceptional accuracies while maintaining low computational cost. |
| title | Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.00048 |