A Gradient Method for Multilevel Optimization
Although application examples of multilevel optimization have already been discussed since the 1990s, the development of solution methods was almost limited to bilevel cases due to the difficulty of the problem. In recent years, in machine learning, Franceschi et al. have proposed a method for solvi...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Although application examples of multilevel optimization have already been
discussed since the 1990s, the development of solution methods was almost
limited to bilevel cases due to the difficulty of the problem. In recent years,
in machine learning, Franceschi et al. have proposed a method for solving
bilevel optimization problems by replacing their lower-level problems with the
$T$ steepest descent update equations with some prechosen iteration number $T$.
In this paper, we have developed a gradient-based algorithm for multilevel
optimization with $n$ levels based on their idea and proved that our
reformulation asymptotically converges to the original multilevel problem. As
far as we know, this is one of the first algorithms with some theoretical
guarantee for multilevel optimization. Numerical experiments show that a
trilevel hyperparameter learning model considering data poisoning produces more
stable prediction results than an existing bilevel hyperparameter learning
model in noisy data settings. |
---|---|
DOI: | 10.48550/arxiv.2105.13954 |