Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909769945579520 |
|---|---|
| author | Zhong, Renyi Li, Yichen Yu, Guangba Gu, Wenwei Kuang, Jinxi Huo, Yintong Lyu, Michael R. |
| author_facet | Zhong, Renyi Li, Yichen Yu, Guangba Gu, Wenwei Kuang, Jinxi Huo, Yintong Lyu, Michael R. |
| contents | Developers use logging statements to create logs that document system behavior and aid in software maintenance. As such, high-quality logging is essential for effective maintenance; however, manual logging often leads to errors and inconsistency. Recent methods emphasize using large language models (LLMs) for automated logging statement generation, but these present privacy and resource issues, hindering their suitability for enterprise use. This paper presents the first large-scale empirical study evaluating small open-source language models (SOLMs) for automated logging statement generation. We evaluate four prominent SOLMs using various prompt strategies and parameter-efficient fine-tuning techniques, such as Low-Rank Adaptation (LoRA) and Retrieval-Augmented Generation (RAG). Our results show that fine-tuned SOLMs with LoRA and RAG prompts, particularly Qwen2.5-coder-14B, outperform existing tools and LLM baselines in predicting logging locations and generating high-quality statements, with robust generalization across diverse repositories. These findings highlight SOLMs as a privacy-preserving, efficient alternative for automated logging. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16590 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generation Zhong, Renyi Li, Yichen Yu, Guangba Gu, Wenwei Kuang, Jinxi Huo, Yintong Lyu, Michael R. Software Engineering Developers use logging statements to create logs that document system behavior and aid in software maintenance. As such, high-quality logging is essential for effective maintenance; however, manual logging often leads to errors and inconsistency. Recent methods emphasize using large language models (LLMs) for automated logging statement generation, but these present privacy and resource issues, hindering their suitability for enterprise use. This paper presents the first large-scale empirical study evaluating small open-source language models (SOLMs) for automated logging statement generation. We evaluate four prominent SOLMs using various prompt strategies and parameter-efficient fine-tuning techniques, such as Low-Rank Adaptation (LoRA) and Retrieval-Augmented Generation (RAG). Our results show that fine-tuned SOLMs with LoRA and RAG prompts, particularly Qwen2.5-coder-14B, outperform existing tools and LLM baselines in predicting logging locations and generating high-quality statements, with robust generalization across diverse repositories. These findings highlight SOLMs as a privacy-preserving, efficient alternative for automated logging. |
| title | Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generation |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2505.16590 |