Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Dongge, Xia, Menglin, Diaz, Daniel Madrigal, Kessler, Samuel, Mallick, Ankur, Zhang, Xuchao, Garcia, Mirian Del Carmen Hipolito, Xu, Jin, Rühle, Victor, Rajmohan, Saravan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866910998823174144
author	Han, Dongge Xia, Menglin Diaz, Daniel Madrigal Kessler, Samuel Mallick, Ankur Zhang, Xuchao Garcia, Mirian Del Carmen Hipolito Xu, Jin Rühle, Victor Rajmohan, Saravan
author_facet	Han, Dongge Xia, Menglin Diaz, Daniel Madrigal Kessler, Samuel Mallick, Ankur Zhang, Xuchao Garcia, Mirian Del Carmen Hipolito Xu, Jin Rühle, Victor Rajmohan, Saravan
contents	Small language models (SLMs) offer promising and efficient alternatives to large language models (LLMs). However, SLMs' limited capacity restricts their reasoning capabilities and makes them sensitive to prompt variations. To address these challenges, we propose a novel framework that enhances SLM reasoning capabilities through LLM generated blueprints. The blueprints provide structured, high-level reasoning guides that help SLMs systematically tackle related problems. Furthermore, our framework integrates a prompt template search mechanism to mitigate the SLMs' sensitivity to prompt variations. Our framework demonstrates improved SLM performance across various tasks, including math (GSM8K), coding (MBPP), and logic reasoning (BBH). Our approach improves the reasoning capabilities of SLMs without increasing model size or requiring additional training, offering a lightweight and deployment-friendly solution for on-device or resource-constrained environments.
format	Preprint
id	arxiv_https___arxiv_org_abs_2506_08669
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search Han, Dongge Xia, Menglin Diaz, Daniel Madrigal Kessler, Samuel Mallick, Ankur Zhang, Xuchao Garcia, Mirian Del Carmen Hipolito Xu, Jin Rühle, Victor Rajmohan, Saravan Machine Learning Artificial Intelligence Small language models (SLMs) offer promising and efficient alternatives to large language models (LLMs). However, SLMs' limited capacity restricts their reasoning capabilities and makes them sensitive to prompt variations. To address these challenges, we propose a novel framework that enhances SLM reasoning capabilities through LLM generated blueprints. The blueprints provide structured, high-level reasoning guides that help SLMs systematically tackle related problems. Furthermore, our framework integrates a prompt template search mechanism to mitigate the SLMs' sensitivity to prompt variations. Our framework demonstrates improved SLM performance across various tasks, including math (GSM8K), coding (MBPP), and logic reasoning (BBH). Our approach improves the reasoning capabilities of SLMs without increasing model size or requiring additional training, offering a lightweight and deployment-friendly solution for on-device or resource-constrained environments.
title	Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
topic	Machine Learning Artificial Intelligence
url	https://arxiv.org/abs/2506.08669