Cover Image

Hyperparameter Optimization for Large Language Model Instruction-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tribes, Christophe, Benarroch-Lelong, Sacha, Lu, Peng, Kobyzev, Ivan
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation
by: Ma, Zeyuan, et al.
Published: (2024)

Efficient search strategies for constrained multiobjective blackbox optimization
by: Digabel, Sébastien Le, et al.
Published: (2025)

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
by: Pan, Rui, et al.
Published: (2024)

OptiChat: Bridging Optimization Models and Practitioners with Large Language Models
by: Chen, Hao, et al.
Published: (2025)

Solving General Natural-Language-Description Optimization Problems with Large Language Models
by: Zhang, Jihai, et al.
Published: (2024)

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
by: Liu, Hong, et al.
Published: (2023)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
by: Gautam, Tanmay, et al.
Published: (2024)

A penalty-interior point method combined with MADS for equality and inequality constrained optimization
by: Audet, Charles, et al.
Published: (2026)

DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization
by: Cummins, Michael, et al.
Published: (2024)

Bayesian Optimization for Hyperparameters Tuning in Neural Networks
by: Onorato, Gabriele
Published: (2024)

Reward Collapse in Aligning Large Language Models
by: Song, Ziang, et al.
Published: (2023)

Distributional Surgery for Language Model Activations
by: Nguyen, Bao, et al.
Published: (2025)

Leveraging Large Language Models for Solving Rare MIP Challenges
by: Wang, Teng, et al.
Published: (2024)

COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework
by: Ren, Yinuo, et al.
Published: (2024)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

One-Shot Safety Alignment for Large Language Models via Optimal Dualization
by: Huang, Xinmeng, et al.
Published: (2024)

Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
by: Kunstner, Frederik, et al.
Published: (2024)

Large Language Model for Discrete Optimization Problems: Evaluation and Step-by-step Reasoning
by: Qian, Tianhao, et al.
Published: (2026)

ACING: Actor-Critic for Instruction Learning in Black-Box LLMs
by: Kharrat, Salma, et al.
Published: (2024)

Implicit Differentiation for Hyperparameter Tuning the Weighted Graphical Lasso
by: Pouliquen, Can, et al.
Published: (2023)

Sign-Based Optimizers Are Effective Under Heavy-Tailed Noise
by: Yu, Dingzhi, et al.
Published: (2026)

Foundation Models to the Rescue: Deadlock Resolution in Connected Multi-Robot Systems
by: Garg, Kunal, et al.
Published: (2024)

A Formal Perspective on Byte-Pair Encoding
by: Zouhar, Vilém, et al.
Published: (2023)

Towards automatic generation of Piping and Instrumentation Diagrams (P&IDs) with Artificial Intelligence
by: Hirtreiter, Edwin, et al.
Published: (2022)

Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices
by: Zhao, Pengxiang, et al.
Published: (2024)

Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback
by: Wang, Yikai, et al.
Published: (2026)

City-LEO: Toward Transparent City Management Using LLM with End-to-End Optimization
by: Jiao, Zihao, et al.
Published: (2024)

solar: A solar thermal power plant simulator for blackbox optimization benchmarking
by: Andrés-Thió, Nicolau, et al.
Published: (2024)

Work Smarter...Not Harder: Efficient Minimization of Dependency Length in SOV Languages
by: Ranjan, Sidharth, et al.
Published: (2024)

Resonance RoPE: Improving Context Length Generalization of Large Language Models
by: Wang, Suyuchen, et al.
Published: (2024)

Convolutional optimization with convex kernel and power lift
by: Lu, Zhipeng
Published: (2025)

Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models
by: Zhang, Yihao, et al.
Published: (2024)

Lower-level Duality Based Reformulation and Majorization Minimization Algorithm for Hyperparameter Optimization
by: Chen, He, et al.
Published: (2024)

A Martingale approach to continuous Portfolio Optimization under CVaR like constraints
by: Lelong, Jérôme, et al.
Published: (2025)

Learning Algorithm Hyperparameters for Fast Parametric Convex Optimization
by: Sambharya, Rajiv, et al.
Published: (2024)

Benchmarking of Quantum and Classical Computing in Large-Scale Dynamic Portfolio Optimization Under Market Frictions
by: Chen, Ying, et al.
Published: (2025)

Variational Learning is Effective for Large Deep Networks
by: Shen, Yuesong, et al.
Published: (2024)

Optimizing Optimizations: Case Study on Detecting Specific Types of Mathematical Optimization Constraints with E-Graphs in JijModeling
by: Ishii, Hiromi, et al.
Published: (2025)

PowerStep: Memory-Efficient Adaptive Optimization via $\ell_p$-Norm Steepest Descent
by: Lu, Yao, et al.
Published: (2026)

On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfer
by: Xu, Ruihan, et al.
Published: (2026)