Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards

Fuente: arXiv
Saved in:
Bibliographic Details
Main Authors: Wu, Shirley, Sarthi, Parth, Zhao, Shiyu, Lee, Aaron, Shandilya, Herumb, Grobelnik, Adrian Mladenic, Choudhary, Nurendra, Huang, Eddie, Subbian, Karthik, Zhang, Linjun, Yang, Diyi, Zou, James, Leskovec, Jure
Format: Preprint
Published: 2025
Subjects:
Online Access:
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866917258321723392
author Wu, Shirley
Sarthi, Parth
Zhao, Shiyu
Lee, Aaron
Shandilya, Herumb
Grobelnik, Adrian Mladenic
Choudhary, Nurendra
Huang, Eddie
Subbian, Karthik
Zhang, Linjun
Yang, Diyi
Zou, James
Leskovec, Jure
author_facet Wu, Shirley
Sarthi, Parth
Zhao, Shiyu
Lee, Aaron
Shandilya, Herumb
Grobelnik, Adrian Mladenic
Choudhary, Nurendra
Huang, Eddie
Subbian, Karthik
Zhang, Linjun
Yang, Diyi
Zou, James
Leskovec, Jure
contents Compound AI systems integrating multiple components, such as Large Language Models, specialized tools, and traditional machine learning models, are increasingly deployed to solve complex real-world tasks. However, optimizing compound systems remains challenging due to their non-differentiable structures and diverse configuration types across components, including prompts, hyperparameters, and model parameters. To address this challenge, we propose Optimas, a unified framework for effective optimization of compound systems. The core idea of Optimas is to maintain one Local Reward Function (LRF) per component, each satisfying a local-global alignment property, i.e., each component's local reward correlates with the global system performance. In each iteration, Optimas efficiently adapts the LRFs to maintain this property while simultaneously maximizing each component's local reward. This approach enables independent updates of heterogeneous configurations using the designated optimization method, while ensuring that local improvements consistently lead to performance gains. We present extensive evaluations across five real-world compound systems to demonstrate that Optimas outperforms strong baselines by an average improvement of 11.92%, offering a general and effective approach for improving compound systems. Our website is at https://optimas.stanford.edu.
format Preprint
id arxiv_https___arxiv_org_abs_2507_03041
institution arXiv
publishDate 2025
record_format arxiv
spellingShingle Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards
Wu, Shirley
Sarthi, Parth
Zhao, Shiyu
Lee, Aaron
Shandilya, Herumb
Grobelnik, Adrian Mladenic
Choudhary, Nurendra
Huang, Eddie
Subbian, Karthik
Zhang, Linjun
Yang, Diyi
Zou, James
Leskovec, Jure
Machine Learning
Artificial Intelligence
Compound AI systems integrating multiple components, such as Large Language Models, specialized tools, and traditional machine learning models, are increasingly deployed to solve complex real-world tasks. However, optimizing compound systems remains challenging due to their non-differentiable structures and diverse configuration types across components, including prompts, hyperparameters, and model parameters. To address this challenge, we propose Optimas, a unified framework for effective optimization of compound systems. The core idea of Optimas is to maintain one Local Reward Function (LRF) per component, each satisfying a local-global alignment property, i.e., each component's local reward correlates with the global system performance. In each iteration, Optimas efficiently adapts the LRFs to maintain this property while simultaneously maximizing each component's local reward. This approach enables independent updates of heterogeneous configurations using the designated optimization method, while ensuring that local improvements consistently lead to performance gains. We present extensive evaluations across five real-world compound systems to demonstrate that Optimas outperforms strong baselines by an average improvement of 11.92%, offering a general and effective approach for improving compound systems. Our website is at https://optimas.stanford.edu.
title Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards
topic Machine Learning
Artificial Intelligence
url https://arxiv.org/abs/2507.03041