Image de couverture de livre

Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kopiczko, Dawid J., Vaze, Sagar, Blankevoort, Tijmen, Asano, Yuki M.
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Bitune: Leveraging Bidirectional Attention to Improve Decoder-Only LLMs
par: Kopiczko, Dawid J., et autres
Publié: (2024)

VeRA: Vector-based Random Matrix Adaptation
par: Kopiczko, Dawid J., et autres
Publié: (2023)

What Layers When: Learning to Skip Compute in LLMs with Residual Gates
par: Laitenberger, Filipe, et autres
Publié: (2025)

Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
par: Bergner, Benjamin, et autres
Publié: (2024)

The LLM Surgeon
par: van der Ouderaa, Tycho F. A., et autres
Publié: (2023)

Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning
par: Yang, Cehao, et autres
Publié: (2025)

KV Cache Steering for Controlling Frozen LLMs
par: Belitsky, Max, et autres
Publié: (2025)

Rethinking Data Selection for Supervised Fine-Tuning
par: Shen, Ming
Publié: (2024)

Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
par: Puerto, Haritz, et autres
Publié: (2024)

Stealth Fine-Tuning: Efficiently Breaking Alignment in RVLMs Using Self-Generated CoT
par: Yu, Le, et autres
Publié: (2025)

RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?
par: Xu, Haotian, et autres
Publié: (2025)

Little Data, Big Impact: Privacy-Aware Visual Language Models via Minimal Tuning
par: Samson, Laurens, et autres
Publié: (2024)

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning
par: Zhao, Hao, et autres
Publié: (2024)

Elastic ViTs from Pretrained Models without Retraining
par: Simoncini, Walter, et autres
Publié: (2025)

SIM-CoT: Supervised Implicit Chain-of-Thought
par: Wei, Xilin, et autres
Publié: (2025)

D-SCoRE: Document-Centric Segmentation and CoT Reasoning with Structured Export for QA-CoT Data Generation
par: Zhou, Weibo, et autres
Publié: (2025)

Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
par: Le, Chenqian, et autres
Publié: (2025)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
par: Pang, Jinlong, et autres
Publié: (2025)

CoT-Valve: Length-Compressible Chain-of-Thought Tuning
par: Ma, Xinyin, et autres
Publié: (2025)

Beyond Isolated Capabilities: Bridging Long CoT Reasoning and Long-Context Understanding
par: Wang, Yifei
Publié: (2025)

From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
par: Deng, Yuntian, et autres
Publié: (2024)

Efficient Long CoT Reasoning in Small Language Models
par: Wang, Zhaoyang, et autres
Publié: (2025)

Mind the Gap: Data Rewriting for Stable Off-Policy Supervised Fine-Tuning
par: Zhao, Shiwan, et autres
Publié: (2025)

Fine-Tuning LLMs for Report Summarization: Analysis on Supervised and Unsupervised Data
par: Rallapalli, Swati, et autres
Publié: (2025)

Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning
par: Yan, Shaotian, et autres
Publié: (2026)

Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning
par: Hu, Tianxiang, et autres
Publié: (2024)

Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning
par: Jin, Senjie, et autres
Publié: (2025)

ToxiFrench: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection
par: Delaval, Axel, et autres
Publié: (2025)

Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
par: Song, Feifan, et autres
Publié: (2024)

Mol-R1: Towards Explicit Long-CoT Reasoning in Molecule Discovery
par: Li, Jiatong, et autres
Publié: (2025)

Long or short CoT? Investigating Instance-level Switch of Large Reasoning Models
par: Zhang, Ruiqi, et autres
Publié: (2025)

Through the Valley: Path to Effective Long CoT Training for Small Language Models
par: Luo, Renjie, et autres
Publié: (2025)

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning
par: Wu, Haodong, et autres
Publié: (2026)

Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs
par: Lu, Yuxiao, et autres
Publié: (2024)

Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions
par: Suh, Joseph, et autres
Publié: (2025)

On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning
par: Li, Zhaoyi, et autres
Publié: (2026)

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
par: Cook, Jack, et autres
Publié: (2025)

Beyond In-Distribution Success: Scaling Curves of CoT Granularity for Language Model Generalization
par: Wang, Ru, et autres
Publié: (2025)

Advancing Speech Language Models by Scaling Supervised Fine-Tuning with Over 60,000 Hours of Synthetic Speech Dialogue Data
par: Zhao, Shuaijiang, et autres
Publié: (2024)

Anchored Supervised Fine-Tuning
par: Zhu, He, et autres
Publié: (2025)