A two-stage search framework for constrained multi-gradient descent
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910911490424832 |
|---|---|
| author | Lei, Yuan-Zheng Gong, Yaobang Yang, Xianfeng Terry |
| author_facet | Lei, Yuan-Zheng Gong, Yaobang Yang, Xianfeng Terry |
| contents | The multi-gradient descent algorithm (MGDA) finds a common descent direction that can improve all objectives by identifying the minimum-norm point in the convex hull of the objective gradients. This method has become a foundational tool in large-scale multi-objective optimization, particularly in multi-task learning. However, MGDA may struggle with constrained problems, whether constraints are incorporated into the gradient hull or handled via projection onto the feasible region. To address this limitation, we propose a two-stage search algorithm for constrained multi-objective optimization. The first stage formulates a min-max problem that minimizes the upper bound of directional derivatives under constraints, yielding a weakly Pareto stationary solution with balanced progress across objectives. The second stage refines this solution by minimizing the lower bound of directional derivatives to achieve full Pareto stationarity. We evaluate the proposed method on three numerical examples. In a simple case with a known analytical Pareto front, our algorithm converges rapidly. In more complex real-world problems, it consistently outperforms the evolutionary baselines NSGA-II and NSGA-III. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14104 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A two-stage search framework for constrained multi-gradient descent Lei, Yuan-Zheng Gong, Yaobang Yang, Xianfeng Terry Optimization and Control The multi-gradient descent algorithm (MGDA) finds a common descent direction that can improve all objectives by identifying the minimum-norm point in the convex hull of the objective gradients. This method has become a foundational tool in large-scale multi-objective optimization, particularly in multi-task learning. However, MGDA may struggle with constrained problems, whether constraints are incorporated into the gradient hull or handled via projection onto the feasible region. To address this limitation, we propose a two-stage search algorithm for constrained multi-objective optimization. The first stage formulates a min-max problem that minimizes the upper bound of directional derivatives under constraints, yielding a weakly Pareto stationary solution with balanced progress across objectives. The second stage refines this solution by minimizing the lower bound of directional derivatives to achieve full Pareto stationarity. We evaluate the proposed method on three numerical examples. In a simple case with a known analytical Pareto front, our algorithm converges rapidly. In more complex real-world problems, it consistently outperforms the evolutionary baselines NSGA-II and NSGA-III. |
| title | A two-stage search framework for constrained multi-gradient descent |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2502.14104 |