iNALU: Improved Neural Arithmetic Logic Unit
Zitieren Sie bitte immer diese URN: urn:nbn:de:bvb:20-opus-212301
- Neural networks have to capture mathematical relationships in order to learn various tasks. They approximate these relations implicitly and therefore often do not generalize well. The recently proposed Neural Arithmetic Logic Unit (NALU) is a novel neural architecture which is able to explicitly represent the mathematical relationships by the units of the network to learn operations such as summation, subtraction or multiplication. Although NALUs have been shown to perform well on various downstream tasks, an in-depth analysis reveals practicalNeural networks have to capture mathematical relationships in order to learn various tasks. They approximate these relations implicitly and therefore often do not generalize well. The recently proposed Neural Arithmetic Logic Unit (NALU) is a novel neural architecture which is able to explicitly represent the mathematical relationships by the units of the network to learn operations such as summation, subtraction or multiplication. Although NALUs have been shown to perform well on various downstream tasks, an in-depth analysis reveals practical shortcomings by design, such as the inability to multiply or divide negative input values or training stability issues for deeper networks. We address these issues and propose an improved model architecture. We evaluate our model empirically in various settings from learning basic arithmetic operations to more complex functions. Our experiments indicate that our model solves stability issues and outperforms the original NALU model in means of arithmetic precision and convergence.…
Autor(en): | Daniel Schlör, Markus Ring, Andreas Hotho |
---|---|
URN: | urn:nbn:de:bvb:20-opus-212301 |
Dokumentart: | Artikel / Aufsatz in einer Zeitschrift |
Institute der Universität: | Fakultät für Mathematik und Informatik / Institut für Informatik |
Sprache der Veröffentlichung: | Englisch |
Titel des übergeordneten Werkes / der Zeitschrift (Englisch): | Frontiers in Artificial Intelligence |
ISSN: | 2624-8212 |
Erscheinungsjahr: | 2020 |
Band / Jahrgang: | 3 |
Aufsatznummer: | 71 |
Originalveröffentlichung / Quelle: | Frontiers in Artificial Intelligence 2020, 3:71. doi: 10.3389/frai.2020.00071 |
DOI: | https://doi.org/10.3389/frai.2020.00071 |
Allgemeine fachliche Zuordnung (DDC-Klassifikation): | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
Freie Schlagwort(e): | arithmetic calculations; experimental evaluation; machine learning; neural architecture; neural networks |
Datum der Freischaltung: | 10.03.2021 |
Datum der Erstveröffentlichung: | 29.09.2020 |
Open-Access-Publikationsfonds / Förderzeitraum 2020 | |
Lizenz (Deutsch): | CC BY: Creative-Commons-Lizenz: Namensnennung 4.0 International |