Moderate Adaptive Linear Units (MoLU)
We propose a new high-performance activation function, Moderate Adaptive Linear Units (MoLU), for the deep neural network. The MoLU is a simple, beautiful and powerful activation function that can be a good main activation function among hundreds of activation functions. Because the MoLU is made up...
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Zusammenfassung: | We propose a new high-performance activation function, Moderate Adaptive
Linear Units (MoLU), for the deep neural network. The MoLU is a simple,
beautiful and powerful activation function that can be a good main activation
function among hundreds of activation functions. Because the MoLU is made up of
the elementary functions, not only it is a diffeomorphism (i.e. analytic over
whole domains), but also it reduces the training time. |
|---|---|
| DOI: | 10.48550/arxiv.2302.13696 |