Cover Image

Small LLMs with Expert Blocks Are Good Enough for Hyperparamter Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Naphade, Om, Bansal, Saksham, Pareek, Parikshit
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Enabling Approximate Joint Sampling in Diffusion LMs
by: Bansal, Parikshit, et al.
Published: (2025)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
by: Zhuang, Haomin, et al.
Published: (2024)

EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
by: Guo, Song, et al.
Published: (2024)

On the Utility of Domain-Adjacent Fine-Tuned Model Ensembles for Few-shot Problems
by: Alam, Md Ibrahim Ibne, et al.
Published: (2024)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
by: He, Haoze, et al.
Published: (2026)

Reward Is Enough: LLMs Are In-Context Reinforcement Learners
by: Song, Kefan, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
by: Zhang, Buze, et al.
Published: (2026)

Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent
by: He, Zeyu, et al.
Published: (2025)

Enough Coin Flips Can Make LLMs Act Bayesian
by: Gupta, Ritwik, et al.
Published: (2025)

Revisiting the Robustness of Watermarking to Paraphrasing Attacks
by: Rastogi, Saksham, et al.
Published: (2024)

Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization
by: Thulke, David, et al.
Published: (2024)

Glider: Global and Local Instruction-Driven Expert Router
by: Li, Pingzhi, et al.
Published: (2024)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
by: Barazandeh, Babak, et al.
Published: (2025)

What Level of Automation is "Good Enough"? A Benchmark of Large Language Models for Meta-Analysis Data Extraction
by: Li, Lingbo, et al.
Published: (2025)

Are LLMs Good Cryptic Crossword Solvers?
by: Sadallah, Abdelrahman, et al.
Published: (2024)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
by: Huang, Wei, et al.
Published: (2024)

One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts
by: Wang, Ruochen, et al.
Published: (2024)

Rational Synthesizers or Heuristic Followers? Analyzing LLMs in RAG-based Question-Answering
by: Naphade, Atharv
Published: (2026)

SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
by: Song, Jiwon, et al.
Published: (2024)

QEFT: Quantization for Efficient Fine-Tuning of LLMs
by: Lee, Changhun, et al.
Published: (2024)

Steering MoE LLMs via Expert (De)Activation
by: Fayyaz, Mohsen, et al.
Published: (2025)

Screening Is Enough
by: Nakanishi, Ken M.
Published: (2026)

BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
by: Song, Chenyang, et al.
Published: (2025)

Learning to Route LLMs with Confidence Tokens
by: Chuang, Yu-Neng, et al.
Published: (2024)

Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs
by: Zhou, Kuan Lok, et al.
Published: (2025)

LLMs Are Already Good Tutors: Training-Free Prompt Optimization for Pedagogical Math Tutoring
by: Lee, Unggi, et al.
Published: (2026)

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
by: Sun, Chung-En, et al.
Published: (2024)

Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts
by: Martin, Liu O., et al.
Published: (2026)

Can Small Language Models be Good Reasoners for Sequential Recommendation?
by: Wang, Yuling, et al.
Published: (2024)

STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings
by: Rastogi, Saksham, et al.
Published: (2025)

Evaluation of LLMs for Process Model Analysis and Optimization
by: Kumar, Akhil, et al.
Published: (2025)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
by: Deng, Jianing, et al.
Published: (2026)

QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources
by: Li, Zhikai, et al.
Published: (2023)

Does Alignment Tuning Really Break LLMs' Internal Confidence?
by: Oh, Hongseok, et al.
Published: (2024)

PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs
by: Peng, Dan, et al.
Published: (2024)

Confidence-Credibility Aware Weighted Ensembles of Small LLMs Outperform Large LLMs in Emotion Detection
by: Elgabry, Menna, et al.
Published: (2025)

Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time
by: Han, Yixuan, et al.
Published: (2025)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
by: Zhao, Ziyu, et al.
Published: (2026)

Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design
by: Cai, Ruisi, et al.
Published: (2024)

SAFT: Structure-Aware Fine-Tuning of LLMs for AMR-to-Text Generation
by: Kamel, Rafiq, et al.
Published: (2025)