Density-based weighting for imbalanced regression
Please always quote using this URN: urn:nbn:de:bvb:20-opus-269177
- In many real world settings, imbalanced data impedes model performance of learning algorithms, like neural networks, mostly for rare cases. This is especially problematic for tasks focusing on these rare occurrences. For example, when estimating precipitation, extreme rainfall events are scarce but important considering their potential consequences. While there are numerous well studied solutions for classification settings, most of them cannot be applied to regression easily. Of the few solutions for regression tasks, barely any have exploredIn many real world settings, imbalanced data impedes model performance of learning algorithms, like neural networks, mostly for rare cases. This is especially problematic for tasks focusing on these rare occurrences. For example, when estimating precipitation, extreme rainfall events are scarce but important considering their potential consequences. While there are numerous well studied solutions for classification settings, most of them cannot be applied to regression easily. Of the few solutions for regression tasks, barely any have explored cost-sensitive learning which is known to have advantages compared to sampling-based methods in classification tasks. In this work, we propose a sample weighting approach for imbalanced regression datasets called DenseWeight and a cost-sensitive learning approach for neural network regression with imbalanced data called DenseLoss based on our weighting scheme. DenseWeight weights data points according to their target value rarities through kernel density estimation (KDE). DenseLoss adjusts each data point’s influence on the loss according to DenseWeight, giving rare data points more influence on model training compared to common data points. We show on multiple differently distributed datasets that DenseLoss significantly improves model performance for rare data points through its density-based weighting scheme. Additionally, we compare DenseLoss to the state-of-the-art method SMOGN, finding that our method mostly yields better performance. Our approach provides more control over model training as it enables us to actively decide on the trade-off between focusing on common or rare cases through a single hyperparameter, allowing the training of better models for rare data points.…
Author: | Michael Steininger, Konstantin Kobs, Padraig Davidson, Anna Krause, Andreas Hotho |
---|---|
URN: | urn:nbn:de:bvb:20-opus-269177 |
Document Type: | Journal article |
Faculties: | Fakultät für Mathematik und Informatik / Institut für Informatik |
Language: | English |
Parent Title (English): | Machine Learning |
ISSN: | 1573-0565 |
Year of Completion: | 2021 |
Volume: | 110 |
Issue: | 8 |
Pagenumber: | 2187–2211 |
Source: | Machine Learning 2021, 110(8):2187–2211. DOI: 10.1007/s10994-021-06023-5 |
DOI: | https://doi.org/10.1007/s10994-021-06023-5 |
Dewey Decimal Classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
Tag: | Kerneldensity estimation; cost-sensitive learning; imbalanced regression; sample weighting; supervised learning |
Release Date: | 2022/06/13 |
Licence (German): | CC BY: Creative-Commons-Lizenz: Namensnennung 4.0 International |