Image de couverture de livre

Architectural Trade-offs in Small Language Models Under Compute Constraints

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Bhatti, Shivraj Singh
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Optimizing Humor Generation in Large Language Models: Temperature Configurations and Architectural Trade-offs
par: Evstafev, Evgenii
Publié: (2025)

Emissions and Performance Trade-off Between Small and Large Language Models
par: Garg, Anandita, et autres
Publié: (2025)

Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models
par: Moar, Chakshu, et autres
Publié: (2024)

Understanding the Quality-Diversity Trade-off in Diffusion Language Models
par: Buzzard, Zak
Publié: (2025)

Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
par: Yang, Seongjun, et autres
Publié: (2023)

Exploring Accuracy-Fairness Trade-off in Large Language Models
par: Zhang, Qingquan, et autres
Publié: (2024)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
par: Chen, Pin-Yu, et autres
Publié: (2025)

Hymba: A Hybrid-head Architecture for Small Language Models
par: Dong, Xin, et autres
Publié: (2024)

Downstream Trade-offs of a Family of Text Watermarks
par: Ajith, Anirudh, et autres
Publié: (2023)

GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
par: Wang, Yue, et autres
Publié: (2025)

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
par: Nawrot, Piotr, et autres
Publié: (2025)

Conformal Linguistic Calibration: Trading-off between Factuality and Specificity
par: Jiang, Zhengping, et autres
Publié: (2025)

TweakLLM: A Routing Architecture for Dynamic Tailoring of Cached Responses
par: Cheema, Muhammad Taha, et autres
Publié: (2025)

Scaling Laws for Mixture Pretraining Under Data Constraints
par: Sedova, Anastasiia, et autres
Publié: (2026)

MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
par: Zhao, Yang, et autres
Publié: (2026)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
par: Singh, Joykirat, et autres
Publié: (2025)

Text to Trust: Evaluating Fine-Tuning and LoRA Trade-offs in Language Models for Unfair Terms of Service Detection
par: Juttu, Noshitha Padma Pratyusha, et autres
Publié: (2025)

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
par: Fonseca, Joao, et autres
Publié: (2025)

$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
par: Khandelwal, Apoorv, et autres
Publié: (2024)

The Structure-Content Trade-off in Knowledge Graph Retrieval
par: Six, Valentin, et autres
Publié: (2025)

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
par: Li, Zichao, et autres
Publié: (2026)

Self-Taught Self-Correction for Small Language Models
par: Moskvoretskii, Viktor, et autres
Publié: (2025)

An Analysis for Reasoning Bias of Language Models with Small Initialization
par: Yao, Junjie, et autres
Publié: (2025)

Small Language Models Improve Giants by Rewriting Their Outputs
par: Vernikos, Giorgos, et autres
Publié: (2023)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
par: Cao, Jinghan, et autres
Publié: (2026)

Small Vision-Language Models: A Survey on Compact Architectures and Techniques
par: Patnaik, Nitesh, et autres
Publié: (2025)

From Small to Large Language Models: Revisiting the Federalist Papers
par: Jeong, So Won, et autres
Publié: (2025)

Domain-Adaptive Continued Pre-Training of Small Language Models
par: Faroz, Salman
Publié: (2025)

Need a Small Specialized Language Model? Plan Early!
par: Grangier, David, et autres
Publié: (2024)

Why Linear Interpretability Works: Invariant Subspaces as a Result of Architectural Constraints
par: Saurez, Andres, et autres
Publié: (2026)

Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
par: Hu, Tiancheng, et autres
Publié: (2025)

Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
par: Lou, Xingzhou, et autres
Publié: (2024)

What Happens When Small Is Made Smaller? Exploring the Impact of Compression on Small Data Pretrained Language Models
par: Awobade, Busayo, et autres
Publié: (2024)

Life Cycle-Aware Evaluation of Knowledge Distillation for Machine Translation: Environmental Impact and Translation Quality Trade-offs
par: Attieh, Joseph, et autres
Publié: (2026)

Domain-Adaptive Small Language Models for Structured Tax Code Prediction
par: Nath, Souvik, et autres
Publié: (2025)

Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
par: Scaria, Nicy, et autres
Publié: (2024)

Kanana: Compute-efficient Bilingual Language Models
par: Kanana LLM Team, et autres
Publié: (2025)

From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
par: Wang, Fei, et autres
Publié: (2024)

Efficient Context Propagating Perceiver Architectures for Auto-Regressive Language Modeling
par: Mahmood, Kaleel, et autres
Publié: (2024)

No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices
par: Pang, Qi, et autres
Publié: (2024)