Cover Image

A Scaling Law for Token Efficiency in LLM Fine-Tuning Under Fixed Compute Budgets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lagasse, Ryan, Kierans, Aidan, Ghosh, Avijit, Dori-Hacohen, Shiri
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment
by: Kierans, Aidan, et al.
Published: (2024)

SynDy: Synthetic Dynamic Dataset Generation Framework for Misinformation Tasks
by: Shliselberg, Michael, et al.
Published: (2024)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
by: Pang, Jinlong, et al.
Published: (2025)

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
by: Miyamoto, Sora, et al.
Published: (2026)

AI-Powered Detection of Inappropriate Language in Medical School Curricula
by: Salavati, Chiman, et al.
Published: (2025)

SD-E$^2$: Semantic Exploration for Reasoning Under Token Budgets
by: Mishra, Kshitij, et al.
Published: (2026)

Token-Budget-Aware LLM Reasoning
by: Han, Tingxu, et al.
Published: (2024)

Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
by: Shi, Xiaofeng, et al.
Published: (2025)

SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning
by: Li, Zheng, et al.
Published: (2025)

JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency
by: Cai, Aichen, et al.
Published: (2026)

Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale
by: Soni, Chinmay, et al.
Published: (2026)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
by: Zeng, Runjia, et al.
Published: (2026)

Extending Token Computation for LLM Reasoning
by: Liao, Bingli, et al.
Published: (2024)

KunlunBaize: LLM with Multi-Scale Convolution and Multi-Token Prediction Under TransformerX Framework
by: Li, Cheng, et al.
Published: (2025)

TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
by: Ma, Dabiao, et al.
Published: (2025)

A Fixed-Budget, Cluster-Aware Standard for LLM-as-a-Judge Evaluation: A Multi-Hop RAG Stress Test
by: Sartori, Camilo Chacón, et al.
Published: (2026)

Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models
by: Wang, Xinyi, et al.
Published: (2025)

The Scaling Laws of Skills in LLM Agent Systems
by: Chen, Charles, et al.
Published: (2026)

Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
by: Ye, Junjie, et al.
Published: (2025)

CARE: A QLoRA-Fine Tuned Multi-Domain Chatbot With Fast Learning On Minimal Hardware
by: Dutta, Ankit, et al.
Published: (2025)

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
by: Yang, Yuchen, et al.
Published: (2026)

Why Fine-Tuning Encourages Hallucinations and How to Fix It
by: Kaplan, Guy, et al.
Published: (2026)

NeurIPS 2023 LLM Efficiency Fine-tuning Competition
by: Saroufim, Mark, et al.
Published: (2025)

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
by: Shen, Zhanming, et al.
Published: (2026)

Scaling Laws for Fine-Grained Mixture of Experts
by: Krajewski, Jakub, et al.
Published: (2024)

Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law
by: Bashir, Ali Hamza, et al.
Published: (2026)

Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
by: Song, Feifan, et al.
Published: (2024)

Steering LLM Thinking with Budget Guidance
by: Li, Junyan, et al.
Published: (2025)

HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation
by: Zhang, Geyuan, et al.
Published: (2024)

From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning
by: Li, Yafu, et al.
Published: (2025)

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
by: He, Linda, et al.
Published: (2025)

Establishing Task Scaling Laws via Compute-Efficient Model Ladders
by: Bhagia, Akshita, et al.
Published: (2024)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
by: Chang, Aofei, et al.
Published: (2024)

Token-level Data Selection for Safe LLM Fine-tuning
by: Li, Yanping, et al.
Published: (2026)

LEMUR: A Corpus for Robust Fine-Tuning of Multilingual Law Embedding Models for Retrieval
by: Ahmadi, Narges Baba, et al.
Published: (2026)

Parameter-Efficient Fine-Tuning for Medical Text Summarization: A Comparative Study of Lora, Prompt Tuning, and Full Fine-Tuning
by: Shernazarov, Ulugbek, et al.
Published: (2026)

Parameter Efficient Fine Tuning Llama 3.1 for Answering Arabic Legal Questions: A Case Study on Jordanian Laws
by: Fasha, Mohammed, et al.
Published: (2026)

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
by: Meng, Haoming, et al.
Published: (2026)

Scaling Sparse Fine-Tuning to Large Language Models
by: Ansell, Alan, et al.
Published: (2024)

Understanding the Performance and Estimating the Cost of LLM Fine-Tuning
by: Xia, Yuchen, et al.
Published: (2024)