TY  - JOUR
A1  - Schlör, Daniel
A1  - Ring, Markus
A1  - Hotho, Andreas
T1  - iNALU: Improved Neural Arithmetic Logic Unit
JF  - Frontiers in Artificial Intelligence
N2  - Neural networks have to capture mathematical relationships in order to learn various tasks. They approximate these relations implicitly and therefore often do not generalize well. The recently proposed Neural Arithmetic Logic Unit (NALU) is a novel neural architecture which is able to explicitly represent the mathematical relationships by the units of the network to learn operations such as summation, subtraction or multiplication. Although NALUs have been shown to perform well on various downstream tasks, an in-depth analysis reveals practical shortcomings by design, such as the inability to multiply or divide negative input values or training stability issues for deeper networks. We address these issues and propose an improved model architecture. We evaluate our model empirically in various settings from learning basic arithmetic operations to more complex functions. Our experiments indicate that our model solves stability issues and outperforms the original NALU model in means of arithmetic precision and convergence.
KW  - neural networks
KW  - machine learning
KW  - arithmetic calculations
KW  - neural architecture
KW  - experimental evaluation
Y1  - 2020
U6  - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-212301
SN  - 2624-8212
VL  - 3
ER  - 
TY  - JOUR
A1  - Steininger, Michael
A1  - Abel, Daniel
A1  - Ziegler, Katrin
A1  - Krause, Anna
A1  - Paeth, Heiko
A1  - Hotho, Andreas
T1  - ConvMOS: climate model output statistics with deep learning
JF  - Data Mining and Knowledge Discovery
N2  - Climate models are the tool of choice for scientists researching climate change. Like all models they suffer from errors, particularly systematic and location-specific representation errors. One way to reduce these errors is model output statistics (MOS) where the model output is fitted to observational data with machine learning. In this work, we assess the use of convolutional Deep Learning climate MOS approaches and present the ConvMOS architecture which is specifically designed based on the observation that there are systematic and location-specific errors in the precipitation estimates of climate models. We apply ConvMOS models to the simulated precipitation of the regional climate model REMO, showing that a combination of per-location model parameters for reducing location-specific errors and global model parameters for reducing systematic errors is indeed beneficial for MOS performance. We find that ConvMOS models can reduce errors considerably and perform significantly better than three commonly used MOS approaches and plain ResNet and U-Net models in most cases. Our results show that non-linear MOS models underestimate the number of extreme precipitation events, which we alleviate by training models specialized towards extreme precipitation events with the imbalanced regression method DenseLoss. While we consider climate MOS, we argue that aspects of ConvMOS may also be beneficial in other domains with geospatial data, such as air pollution modeling or weather forecasts.
KW  - Klima
KW  - Modell
KW  - Deep learning
KW  - Neuronales Netz
KW  - climate
KW  - neural networks
KW  - model output statistics
Y1  - 2023
U6  - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-324213
SN  - 1384-5810
VL  - 37
IS  - 1
ER  -