Cover Image

Small Language Models: Survey, Measurements, and Insights

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Zhenyan, Li, Xiang, Cai, Dongqi, Yi, Rongjie, Liu, Fangming, Zhang, Xiwen, Lane, Nicholas D., Xu, Mengwei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
by: Yi, Rongjie, et al.
Published: (2024)

MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs
by: Lu, Zhenyan, et al.
Published: (2025)

EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices
by: Yi, Rongjie, et al.
Published: (2023)

ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer
by: Liu, Dongqi, et al.
Published: (2023)

FwdLLM: Efficient FedLLM using Forward Gradient
by: Xu, Mengwei, et al.
Published: (2023)

A Survey of Resource-efficient LLM and Multimodal Foundation Models
by: Xu, Mengwei, et al.
Published: (2024)

RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization
by: Liu, Dongqi, et al.
Published: (2024)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
by: Li, Yanan, et al.
Published: (2025)

ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models
by: Zheng, Dongqi
Published: (2025)

Recall: Empowering Multimodal Embedding for Edge Devices
by: Cai, Dongqi, et al.
Published: (2024)

Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
by: Liu, Dongqi, et al.
Published: (2023)

PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models
by: Zhao, Chenzhuo, et al.
Published: (2025)

PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning
by: Brahman, Faeze, et al.
Published: (2023)

Small Language Models for Application Interactions: A Case Study
by: Li, Beibin, et al.
Published: (2024)

Training-free Ultra Small Model for Universal Sparse Reconstruction in Compressed Sensing
by: Tang, Chaoqing, et al.
Published: (2025)

Squat: Quant Small Language Models on the Edge
by: Shen, Xuan, et al.
Published: (2024)

SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generation
by: Liu, Dongqi, et al.
Published: (2024)

Explanatory Summarization with Discourse-Driven Planning
by: Liu, Dongqi, et al.
Published: (2025)

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
by: Zhou, Zhanhui, et al.
Published: (2024)

A Systematic Survey on Large Language Models for Algorithm Design
by: Liu, Fei, et al.
Published: (2024)

Stacking Small Language Models for Generalizability
by: Liang, Laurence
Published: (2024)

All Language Models Large and Small
by: Chen, Zhixun, et al.
Published: (2024)

A Survey on LoRA of Large Language Models
by: Mao, Yuren, et al.
Published: (2024)

Jailbreak Attacks and Defenses Against Large Language Models: A Survey
by: Yi, Sibo, et al.
Published: (2024)

Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees
by: Chen, Sijia, et al.
Published: (2024)

Mixture of Heterogeneous Grouped Experts for Language Modeling
by: Ma, Zhicheng, et al.
Published: (2026)

A Survey on Collaborative Mechanisms Between Large and Small Language Models
by: Chen, Yi, et al.
Published: (2025)

Fox-1: Open Small Language Model for Cloud and Edge
by: Hu, Zijian, et al.
Published: (2024)

Knowledge Fusion of Large Language Models Via Modular SkillPacks
by: Du, Guodong, et al.
Published: (2025)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Curriculum Learning for Small Code Language Models
by: Naïr, Marwa, et al.
Published: (2024)

Towards Reasoning Ability of Small Language Models
by: Srivastava, Gaurav, et al.
Published: (2025)

$T^2$ of Thoughts: Temperature Tree Elicits Reasoning in Large Language Models
by: Cai, Chengkun, et al.
Published: (2024)

Are Small Language Models Ready to Compete with Large Language Models for Practical Applications?
by: Sinha, Neelabh, et al.
Published: (2024)

Model Compression and Efficient Inference for Large Language Models: A Survey
by: Wang, Wenxiao, et al.
Published: (2024)

A Survey on Diffusion Language Models
by: Li, Tianyi, et al.
Published: (2025)

Continual Learning of Large Language Models: A Comprehensive Survey
by: Shi, Haizhou, et al.
Published: (2024)

Purifying Large Language Models by Ensembling a Small Language Model
by: Li, Tianlin, et al.
Published: (2024)

Hymba: A Hybrid-head Architecture for Small Language Models
by: Dong, Xin, et al.
Published: (2024)

Measuring and Controlling Instruction (In)Stability in Language Model Dialogs
by: Li, Kenneth, et al.
Published: (2024)