Copertina

SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Alrashed, Sultan
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data
di: Alrashed, Sultan, et al.
Pubblicazione: (2025)

CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs
di: Kim, Jin Young, et al.
Pubblicazione: (2025)

Fineweb-Edu-Ar: Machine-translated Corpus to Support Arabic Small Language Models
di: Alrashed, Sultan, et al.
Pubblicazione: (2024)

DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
di: Shao, Rulin, et al.
Pubblicazione: (2025)

Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study
di: Ning, Xuefei, et al.
Pubblicazione: (2024)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
di: Shen, Yikang, et al.
Pubblicazione: (2024)

Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
di: Padarha, Shreyansh
Pubblicazione: (2025)

ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following
di: Yang, Yuancheng, et al.
Pubblicazione: (2026)

Benchmarking LLMs and SLMs for patient reported outcomes
di: Marengo, Matteo, et al.
Pubblicazione: (2024)

Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3
di: Yoon, Junsang, et al.
Pubblicazione: (2024)

G-Boost: Boosting Private SLMs with General LLMs
di: Fan, Yijiang, et al.
Pubblicazione: (2025)

Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness
di: Bi, Baolong, et al.
Pubblicazione: (2024)

Orca-Math: Unlocking the potential of SLMs in Grade School Math
di: Mitra, Arindam, et al.
Pubblicazione: (2024)

Trained Miniatures: Low cost, High Efficacy SLMs for Sales & Marketing
di: Bhola, Ishaan, et al.
Pubblicazione: (2025)

Learning From Mistakes Makes LLM Better Reasoner
di: An, Shengnan, et al.
Pubblicazione: (2023)

Return of the Encoder: Maximizing Parameter Efficiency for SLMs
di: Elfeki, Mohamed, et al.
Pubblicazione: (2025)

Learning to Self-Verify Makes Language Models Better Reasoners
di: Chen, Yuxin, et al.
Pubblicazione: (2026)

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought
di: Lee, Jooyoung, et al.
Pubblicazione: (2024)

RSQ: Learning from Important Tokens Leads to Better Quantized LLMs
di: Sung, Yi-Lin, et al.
Pubblicazione: (2025)

Reasoning Models Better Express Their Confidence
di: Yoon, Dongkeun, et al.
Pubblicazione: (2025)

LLMs Can Teach Themselves to Better Predict the Future
di: Turtel, Benjamin, et al.
Pubblicazione: (2025)

Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
di: Liang, Zhuowen, et al.
Pubblicazione: (2026)

Debating for Better Reasoning: An Unsupervised Multimodal Approach
di: Adhikari, Ashutosh, et al.
Pubblicazione: (2025)

Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning
di: Zhang, Kongcheng, et al.
Pubblicazione: (2025)

Does Biomedical Training Lead to Better Medical Performance?
di: Dada, Amin, et al.
Pubblicazione: (2024)

Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs
di: Marco, Guillermo, et al.
Pubblicazione: (2024)

Timo: Towards Better Temporal Reasoning for Language Models
di: Su, Zhaochen, et al.
Pubblicazione: (2024)

Abstraction-of-Thought Makes Language Models Better Reasoners
di: Hong, Ruixin, et al.
Pubblicazione: (2024)

The Cognitive Penalty: Ablating System 1 and System 2 Reasoning in Edge-Native SLMs for Decentralized Consensus
di: Rizvi, Syed Muhammad Aqdas
Pubblicazione: (2026)

AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment
di: Chen, Nuo, et al.
Pubblicazione: (2024)

When Does Multimodality Lead to Better Time Series Forecasting?
di: Zhang, Xiyuan, et al.
Pubblicazione: (2025)

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
di: Corbeil, Jean-Philippe, et al.
Pubblicazione: (2025)

CausalEval: Towards Better Causal Reasoning in Language Models
di: Yu, Longxuan, et al.
Pubblicazione: (2024)

Recall, Retrieve and Reason: Towards Better In-Context Relation Extraction
di: Li, Guozheng, et al.
Pubblicazione: (2024)

Mixed Distillation Helps Smaller Language Model Better Reasoning
di: Li, Chenglin, et al.
Pubblicazione: (2023)

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size
di: Hayou, Soufiane, et al.
Pubblicazione: (2025)

CEGI: Measuring the trade-off between efficiency and carbon emissions for SLMs and VLMs
di: Kumar, Abhas, et al.
Pubblicazione: (2024)

Can One-sided Arguments Lead to Response Change in Large Language Models?
di: Cisneros-Velarde, Pedro
Pubblicazione: (2026)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
di: Bergsma, Shane, et al.
Pubblicazione: (2025)

Eliciting Better Multilingual Structured Reasoning from LLMs through Code
di: Li, Bryan, et al.
Pubblicazione: (2024)