Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | En |
| Published: |
Zenodo
2026
|
| Subjects: | |
| Online Access: | https://doi.org/10.57967/hf/7712 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866901705355952128 |
|---|---|
| author | Victor Manuel López González |
| author_facet | Victor Manuel López González |
| contents | <p>We present Yuuki-best, an experimental demonstration that large language model fine-tuning is technically feasible on consumer mobile hardware without GPU acceleration or cloud resources. Using a Redmi 12 smartphone (Snapdragon 685 CPU, 6GB RAM) with zero computational budget, we fine-tuned GPT-2 (124M parameters) on 75,000 code samples over 50+ continuous hours. </p> <p> </p> <p>Our investigation reveals three key findings: (1) Loss-quality divergence: checkpoint 2700 achieved 19% lower training loss than checkpoint 2000 but scored 12% worse in code generation quality, challenging the assumption that lower loss guarantees better performance; (2) Dataset ordering effects: alphabetically-early languages (Agda: 55/100 quality) substantially outperformed later languages (Python: 8/100) at identical training progress, demonstrating unintended curriculum learning; (3) Non-monotonic quality patterns: optimal checkpoints occurred mid-training rather than at convergence, with checkpoint 2000 outperforming all subsequent checkpoints through step 2700. </p> <p> </p> <p>While not production-ready (average quality: 24.6/100), this work demonstrates that meaningful ML experimentation is accessible without traditional computational infrastructure, provides empirical evidence of loss-quality divergence in heterogeneous code datasets, and quantifies the impact of dataset ordering on multilingual model training.</p> <p> </p> <p>Model available at: https://huggingface.co/OpceanAI/Yuuki-best</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_57967_hf_7712 |
| institution | Zenodo |
| language | enc |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Yuuki-paper Victor Manuel López González Llm AI <p>We present Yuuki-best, an experimental demonstration that large language model fine-tuning is technically feasible on consumer mobile hardware without GPU acceleration or cloud resources. Using a Redmi 12 smartphone (Snapdragon 685 CPU, 6GB RAM) with zero computational budget, we fine-tuned GPT-2 (124M parameters) on 75,000 code samples over 50+ continuous hours. </p> <p> </p> <p>Our investigation reveals three key findings: (1) Loss-quality divergence: checkpoint 2700 achieved 19% lower training loss than checkpoint 2000 but scored 12% worse in code generation quality, challenging the assumption that lower loss guarantees better performance; (2) Dataset ordering effects: alphabetically-early languages (Agda: 55/100 quality) substantially outperformed later languages (Python: 8/100) at identical training progress, demonstrating unintended curriculum learning; (3) Non-monotonic quality patterns: optimal checkpoints occurred mid-training rather than at convergence, with checkpoint 2000 outperforming all subsequent checkpoints through step 2700. </p> <p> </p> <p>While not production-ready (average quality: 24.6/100), this work demonstrates that meaningful ML experimentation is accessible without traditional computational infrastructure, provides empirical evidence of loss-quality divergence in heterogeneous code datasets, and quantifies the impact of dataset ordering on multilingual model training.</p> <p> </p> <p>Model available at: https://huggingface.co/OpceanAI/Yuuki-best</p> |
| title | Yuuki-paper |
| topic | Llm AI |
| url | https://doi.org/10.57967/hf/7712 |