NanoNet: Parameter-Efficient Learning with Label-Scarce Supervision for Lightweight Text Mining Model
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914308679532544 |
|---|---|
| author | Mao, Qianren Luo, Yashuo Qin, Ziqi Liu, Junnan Jiang, Weifeng Chen, Zhijun Li, Zhuoran Xiao, Likang Xu, Chuou Zhang, Qili Hao, Hanwen Li, Jingzheng Lin, Chunghua Li, Jianxin Yu, Philip S. |
| author_facet | Mao, Qianren Luo, Yashuo Qin, Ziqi Liu, Junnan Jiang, Weifeng Chen, Zhijun Li, Zhuoran Xiao, Likang Xu, Chuou Zhang, Qili Hao, Hanwen Li, Jingzheng Lin, Chunghua Li, Jianxin Yu, Philip S. |
| contents | The lightweight semi-supervised learning (LSL) strategy provides an effective approach of conserving labeled samples and minimizing model inference costs. Prior research has effectively applied knowledge transfer learning and co-training regularization from large to small models in LSL. However, such training strategies are computationally intensive and prone to local optima, thereby increasing the difficulty of finding the optimal solution. This has prompted us to investigate the feasibility of integrating three low-cost scenarios for text mining tasks: limited labeled supervision, lightweight fine-tuning, and rapid-inference small models. We propose NanoNet, a novel framework for lightweight text mining that implements parameter-efficient learning with limited supervision. It employs online knowledge distillation to generate multiple small models and enhances their performance through mutual learning regularization. The entire process leverages parameter-efficient learning, reducing training costs and minimizing supervision requirements, ultimately yielding a lightweight model for downstream inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_06093 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | NanoNet: Parameter-Efficient Learning with Label-Scarce Supervision for Lightweight Text Mining Model Mao, Qianren Luo, Yashuo Qin, Ziqi Liu, Junnan Jiang, Weifeng Chen, Zhijun Li, Zhuoran Xiao, Likang Xu, Chuou Zhang, Qili Hao, Hanwen Li, Jingzheng Lin, Chunghua Li, Jianxin Yu, Philip S. Machine Learning Artificial Intelligence The lightweight semi-supervised learning (LSL) strategy provides an effective approach of conserving labeled samples and minimizing model inference costs. Prior research has effectively applied knowledge transfer learning and co-training regularization from large to small models in LSL. However, such training strategies are computationally intensive and prone to local optima, thereby increasing the difficulty of finding the optimal solution. This has prompted us to investigate the feasibility of integrating three low-cost scenarios for text mining tasks: limited labeled supervision, lightweight fine-tuning, and rapid-inference small models. We propose NanoNet, a novel framework for lightweight text mining that implements parameter-efficient learning with limited supervision. It employs online knowledge distillation to generate multiple small models and enhances their performance through mutual learning regularization. The entire process leverages parameter-efficient learning, reducing training costs and minimizing supervision requirements, ultimately yielding a lightweight model for downstream inference. |
| title | NanoNet: Parameter-Efficient Learning with Label-Scarce Supervision for Lightweight Text Mining Model |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.06093 |