Cover Image

LLMSYS-HPOBench: Hyperparameter Optimization Benchmark Suite for Real-World LLM Systems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Siyu, Ye, Yulong, Xiang, Zezhen, Chen, Pengzhou, Xiong, Gangda, Chen, Tao
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language Performance Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG
by: Chen, Pengzhou, et al.
Published: (2026)

Stencil-Lifting: Hierarchical Recursive Lifting System for Extracting Summary of Stencil Kernel in Legacy Codes
by: Li, Mingyi, et al.
Published: (2025)

MapReplay: Trace-Driven Benchmark Generation for Java HashMap
by: Schiavio, Filippo, et al.
Published: (2026)

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
by: Ma, Jeffrey Jian, et al.
Published: (2025)

CoTune: Co-evolutionary Configuration Tuning
by: Xiong, Gangda, et al.
Published: (2025)

AI-NativeBench: An Open-Source White-Box Agentic Benchmark Suite for AI-Native Systems
by: Wang, Zirui, et al.
Published: (2026)

SysLLMatic: Large Language Models are Software System Optimizers
by: Peng, Huiyun, et al.
Published: (2025)

An Empirical Study on the Performance and Energy Usage of Compiled Python Code
by: Stoico, Vincenzo, et al.
Published: (2025)

Input-Gen: Guided Generation of Stateful Inputs for Testing, Tuning, and Training
by: Ivanov, Ivan R., et al.
Published: (2024)

Runtime Verification on Abstract Finite State Models
by: Jevitha, KP, et al.
Published: (2024)

Testing the Unknown: A Framework for OpenMP Testing via Random Program Generation
by: Laguna, Ignacio, et al.
Published: (2024)

Do AI Models Dream of Faster Code? An Empirical Study on LLM-Proposed Performance Improvements in Real-World Software
by: Yi, Lirong, et al.
Published: (2025)

Investigating Execution-Aware Language Models for Code Optimization
by: Di Menna, Federico, et al.
Published: (2025)

Efficiently Ranking Software Variants with Minimal Benchmarks
by: Matricon, Théo, et al.
Published: (2025)

PerfBench: Can Agents Resolve Real-World Performance Bugs?
by: Garg, Spandan, et al.
Published: (2025)

Tracing Optimization for Performance Modeling and Regression Detection
by: Shahedi, Kaveh, et al.
Published: (2024)

Predicting Software Performance with Divide-and-Learn
by: Gong, Jingzhi, et al.
Published: (2023)

Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
by: Su, Qisheng, et al.
Published: (2026)

On the Role of Search Budgets in Model-Based Software Refactoring Optimization
by: Diaz-Pace, J. Andres, et al.
Published: (2023)

Optimas: An Intelligent Analytics-Informed Generative AI Framework for Performance Optimization
by: Zaeed, Mohammad, et al.
Published: (2026)

FlipFlop: A Static Analysis-based Energy Optimization Framework for GPU Kernels
by: Rajput, Saurabhsingh, et al.
Published: (2026)

Formal Analysis of Metastable Failures in Software Systems
by: Alvaro, Peter, et al.
Published: (2025)

Estimating the Energy Footprint of Software Systems: a Primer
by: Castor, Fernando
Published: (2024)

Employing Software Diversity in Cloud Microservices to Engineer Reliable and Performant Systems
by: Akhtarian, Nazanin, et al.
Published: (2024)

A Model-driven Approach for Continuous Performance Engineering in Microservice-based Systems
by: Cortellessa, Vittorio, et al.
Published: (2023)

Real-time Event Joining in Practice With Kafka and Flink
by: Saket, Srijan, et al.
Published: (2024)

Who Wins the Race? (R Vs Python) - An Exploratory Study on Energy Consumption of Machine Learning Algorithms
by: Chattaraj, Rajrupa, et al.
Published: (2025)

Library Liberation: Competitive Performance Matmul Through Compiler-composed Nanokernels
by: Thangamani, Arun, et al.
Published: (2025)

MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
by: Huang, Zhengxiang, et al.
Published: (2025)

Dually Hierarchical Drift Adaptation for Online Configuration Performance Learning
by: Xiang, Zezhen, et al.
Published: (2025)

PromiseTune: Unveiling Causally Promising and Explainable Configuration Tuning
by: Chen, Pengzhou, et al.
Published: (2025)

SuperCoder: Assembly Program Superoptimization with Large Language Models
by: Wei, Anjiang, et al.
Published: (2025)

QOPS: A Compiler Framework for Quantum Circuit Simulation Acceleration with Profile Guided Optimizations
by: Wu, Yu-Tsung, et al.
Published: (2024)

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
by: Zhang, Zehua, et al.
Published: (2025)

FaaSter Troubleshooting -- Evaluating Distributed Tracing Approaches for Serverless Applications
by: Borges, Maria C., et al.
Published: (2021)

gigiProfiler: Diagnosing Performance Issues by Uncovering Application Resource Bottlenecks
by: Hu, Yigong, et al.
Published: (2025)

A Magnified View into Heterogeneous-ISA Thread Migration Performance without State Transformation
by: Mavrogeorgis, Nikolaos, et al.
Published: (2025)

An Empirical Study on Method-Level Performance Evolution in Open-Source Java Projects
by: Shahedi, Kaveh, et al.
Published: (2025)

What Is the Cost of Energy Monitoring? An Empirical Study on the Overhead of RAPL-Based Tools
by: Diamond, Jeremy, et al.
Published: (2026)

Performance of Genetic Algorithms in the Context of Software Model Refactoring
by: Cortellessa, Vittorio, et al.
Published: (2023)