Cover Image

Architectural Trade-offs in Small Language Models Under Compute Constraints

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Bhatti, Shivraj Singh
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Optimizing Humor Generation in Large Language Models: Temperature Configurations and Architectural Trade-offs
by: Evstafev, Evgenii
Published: (2025)

Emissions and Performance Trade-off Between Small and Large Language Models
by: Garg, Anandita, et al.
Published: (2025)

Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models
by: Moar, Chakshu, et al.
Published: (2024)

Understanding the Quality-Diversity Trade-off in Diffusion Language Models
by: Buzzard, Zak
Published: (2025)

Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
by: Yang, Seongjun, et al.
Published: (2023)

Exploring Accuracy-Fairness Trade-off in Large Language Models
by: Zhang, Qingquan, et al.
Published: (2024)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
by: Chen, Pin-Yu, et al.
Published: (2025)

Hymba: A Hybrid-head Architecture for Small Language Models
by: Dong, Xin, et al.
Published: (2024)

Downstream Trade-offs of a Family of Text Watermarks
by: Ajith, Anirudh, et al.
Published: (2023)

GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
by: Wang, Yue, et al.
Published: (2025)

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
by: Nawrot, Piotr, et al.
Published: (2025)

Conformal Linguistic Calibration: Trading-off between Factuality and Specificity
by: Jiang, Zhengping, et al.
Published: (2025)

TweakLLM: A Routing Architecture for Dynamic Tailoring of Cached Responses
by: Cheema, Muhammad Taha, et al.
Published: (2025)

Scaling Laws for Mixture Pretraining Under Data Constraints
by: Sedova, Anastasiia, et al.
Published: (2026)

MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
by: Zhao, Yang, et al.
Published: (2026)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
by: Singh, Joykirat, et al.
Published: (2025)

Text to Trust: Evaluating Fine-Tuning and LoRA Trade-offs in Language Models for Unfair Terms of Service Detection
by: Juttu, Noshitha Padma Pratyusha, et al.
Published: (2025)

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
by: Fonseca, Joao, et al.
Published: (2025)

$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
by: Khandelwal, Apoorv, et al.
Published: (2024)

The Structure-Content Trade-off in Knowledge Graph Retrieval
by: Six, Valentin, et al.
Published: (2025)

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
by: Li, Zichao, et al.
Published: (2026)

Self-Taught Self-Correction for Small Language Models
by: Moskvoretskii, Viktor, et al.
Published: (2025)

An Analysis for Reasoning Bias of Language Models with Small Initialization
by: Yao, Junjie, et al.
Published: (2025)

Small Language Models Improve Giants by Rewriting Their Outputs
by: Vernikos, Giorgos, et al.
Published: (2023)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
by: Cao, Jinghan, et al.
Published: (2026)

Small Vision-Language Models: A Survey on Compact Architectures and Techniques
by: Patnaik, Nitesh, et al.
Published: (2025)

From Small to Large Language Models: Revisiting the Federalist Papers
by: Jeong, So Won, et al.
Published: (2025)

Domain-Adaptive Continued Pre-Training of Small Language Models
by: Faroz, Salman
Published: (2025)

Need a Small Specialized Language Model? Plan Early!
by: Grangier, David, et al.
Published: (2024)

Why Linear Interpretability Works: Invariant Subspaces as a Result of Architectural Constraints
by: Saurez, Andres, et al.
Published: (2026)

Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
by: Hu, Tiancheng, et al.
Published: (2025)

Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
by: Lou, Xingzhou, et al.
Published: (2024)

What Happens When Small Is Made Smaller? Exploring the Impact of Compression on Small Data Pretrained Language Models
by: Awobade, Busayo, et al.
Published: (2024)

Life Cycle-Aware Evaluation of Knowledge Distillation for Machine Translation: Environmental Impact and Translation Quality Trade-offs
by: Attieh, Joseph, et al.
Published: (2026)

Domain-Adaptive Small Language Models for Structured Tax Code Prediction
by: Nath, Souvik, et al.
Published: (2025)

Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
by: Scaria, Nicy, et al.
Published: (2024)

Kanana: Compute-efficient Bilingual Language Models
by: Kanana LLM Team, et al.
Published: (2025)

From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
by: Wang, Fei, et al.
Published: (2024)

Efficient Context Propagating Perceiver Architectures for Auto-Regressive Language Modeling
by: Mahmood, Kaleel, et al.
Published: (2024)

No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices
by: Pang, Qi, et al.
Published: (2024)