TY - JOUR A1 - Baumhoer, Celia A. A1 - Dietz, Andreas J. A1 - Kneisel, C. A1 - Kuenzer, C. T1 - Automated Extraction of Antarctic Glacier and Ice Shelf Fronts from Sentinel-1 Imagery Using Deep Learning JF - Remote Sensing N2 - Sea level rise contribution from the Antarctic ice sheet is influenced by changes in glacier and ice shelf front position. Still, little is known about seasonal glacier and ice shelf front fluctuations as the manual delineation of calving fronts from remote sensing imagery is very time-consuming. The major challenge of automatic calving front extraction is the low contrast between floating glacier and ice shelf fronts and the surrounding sea ice. Additionally, in previous decades, remote sensing imagery over the often cloud-covered Antarctic coastline was limited. Nowadays, an abundance of Sentinel-1 imagery over the Antarctic coastline exists and could be used for tracking glacier and ice shelf front movement. To exploit the available Sentinel-1 data, we developed a processing chain allowing automatic extraction of the Antarctic coastline from Seninel-1 imagery and the creation of dense time series to assess calving front change. The core of the proposed workflow is a modified version of the deep learning architecture U-Net. This convolutional neural network (CNN) performs a semantic segmentation on dual-pol Sentinel-1 data and the Antarctic TanDEM-X digital elevation model (DEM). The proposed method is tested for four training and test areas along the Antarctic coastline. The automatically extracted fronts deviate on average 78 m in training and 108 m test areas. Spatial and temporal transferability is demonstrated on an automatically extracted 15-month time series along the Getz Ice Shelf. Between May 2017 and July 2018, the fronts along the Getz Ice Shelf show mostly an advancing tendency with the fastest moving front of DeVicq Glacier with 726 ± 20 m/yr. KW - Antarctica KW - coastline KW - deep learning KW - semantic segmentation KW - Getz Ice Shelf KW - calving front KW - glacier front KW - U-Net KW - convolutional neural network KW - glacier terminus Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-193150 SN - 2072-4292 VL - 11 IS - 21 ER - TY - JOUR A1 - Hoeser, Thorsten A1 - Bachofer, Felix A1 - Kuenzer, Claudia T1 - Object detection and image segmentation with deep learning on Earth Observation data: a review — part II: applications JF - Remote Sensing N2 - In Earth observation (EO), large-scale land-surface dynamics are traditionally analyzed by investigating aggregated classes. The increase in data with a very high spatial resolution enables investigations on a fine-grained feature level which can help us to better understand the dynamics of land surfaces by taking object dynamics into account. To extract fine-grained features and objects, the most popular deep-learning model for image analysis is commonly used: the convolutional neural network (CNN). In this review, we provide a comprehensive overview of the impact of deep learning on EO applications by reviewing 429 studies on image segmentation and object detection with CNNs. We extensively examine the spatial distribution of study sites, employed sensors, used datasets and CNN architectures, and give a thorough overview of applications in EO which used CNNs. Our main finding is that CNNs are in an advanced transition phase from computer vision to EO. Upon this, we argue that in the near future, investigations which analyze object dynamics with CNNs will have a significant impact on EO research. With a focus on EO applications in this Part II, we complete the methodological review provided in Part I. KW - artificial intelligence KW - AI KW - machine learning KW - deep learning KW - neural networks KW - convolutional neural networks KW - CNN KW - image segmentation KW - object detection KW - earth observation Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-213152 SN - 2072-4292 VL - 12 IS - 18 ER - TY - JOUR A1 - Hoeser, Thorsten A1 - Kuenzer, Claudia T1 - Object detection and image segmentation with deep learning on Earth observation data: a review-part I: evolution and recent trends JF - Remote Sensing N2 - Deep learning (DL) has great influence on large parts of science and increasingly established itself as an adaptive method for new challenges in the field of Earth observation (EO). Nevertheless, the entry barriers for EO researchers are high due to the dense and rapidly developing field mainly driven by advances in computer vision (CV). To lower the barriers for researchers in EO, this review gives an overview of the evolution of DL with a focus on image segmentation and object detection in convolutional neural networks (CNN). The survey starts in 2012, when a CNN set new standards in image recognition, and lasts until late 2019. Thereby, we highlight the connections between the most important CNN architectures and cornerstones coming from CV in order to alleviate the evaluation of modern DL models. Furthermore, we briefly outline the evolution of the most popular DL frameworks and provide a summary of datasets in EO. By discussing well performing DL architectures on these datasets as well as reflecting on advances made in CV and their impact on future research in EO, we narrow the gap between the reviewed, theoretical concepts from CV and practical application in EO. KW - artificial intelligence KW - AI KW - machine learning KW - deep learning KW - neural networks KW - convolutional neural networks KW - CNN KW - image segmentation KW - object detection KW - Earth observation Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-205918 SN - 2072-4292 VL - 12 IS - 10 ER - TY - JOUR A1 - Dirscherl, Mariel A1 - Dietz, Andreas J. A1 - Kneisel, Christof A1 - Kuenzer, Claudia T1 - A novel method for automated supraglacial lake mapping in Antarctica using Sentinel-1 SAR imagery and deep learning JF - Remote Sensing N2 - Supraglacial meltwater accumulation on ice sheets can be a main driver for accelerated ice discharge, mass loss, and global sea-level-rise. With further increasing surface air temperatures, meltwater-induced hydrofracturing, basal sliding, or surface thinning will cumulate and most likely trigger unprecedented ice mass loss on the Greenland and Antarctic ice sheets. While the Greenland surface hydrological network as well as its impacts on ice dynamics and mass balance has been studied in much detail, Antarctic supraglacial lakes remain understudied with a circum-Antarctic record of their spatio-temporal development entirely lacking. This study provides the first automated supraglacial lake extent mapping method using Sentinel-1 synthetic aperture radar (SAR) imagery over Antarctica and complements the developed optical Sentinel-2 supraglacial lake detection algorithm presented in our companion paper. In detail, we propose the use of a modified U-Net for semantic segmentation of supraglacial lakes in single-polarized Sentinel-1 imagery. The convolutional neural network (CNN) is implemented with residual connections for optimized performance as well as an Atrous Spatial Pyramid Pooling (ASPP) module for multiscale feature extraction. The algorithm is trained on 21,200 Sentinel-1 image patches and evaluated in ten spatially or temporally independent test acquisitions. In addition, George VI Ice Shelf is analyzed for intra-annual lake dynamics throughout austral summer 2019/2020 and a decision-level fused Sentinel-1 and Sentinel-2 maximum lake extent mapping product is presented for January 2020 revealing a more complete supraglacial lake coverage (~770 km\(^2\)) than the individual single-sensor products. Classification results confirm the reliability of the proposed workflow with an average Kappa coefficient of 0.925 and a F\(_1\)-score of 93.0% for the supraglacial water class across all test regions. Furthermore, the algorithm is applied in an additional test region covering supraglacial lakes on the Greenland ice sheet which further highlights the potential for spatio-temporal transferability. Future work involves the integration of more training data as well as intra-annual analyses of supraglacial lake occurrence across the whole continent and with focus on supraglacial lake development throughout a summer melt season and into Antarctic winter. KW - Antarctica KW - Antarctic ice sheet KW - supraglacial lakes KW - ice sheet hydrology KW - Sentinel-1 KW - remote sensing KW - machine learning KW - deep learning KW - semantic segmentation KW - convolutional neural network Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-222998 SN - 2072-4292 VL - 13 IS - 2 ER - TY - THES A1 - Höser, Thorsten T1 - Global Dynamics of the Offshore Wind Energy Sector Derived from Earth Observation Data - Deep Learning Based Object Detection Optimised with Synthetic Training Data for Offshore Wind Energy Infrastructure Extraction from Sentinel-1 Imagery T1 - Globale Dynamik des Offshore-Windenergiesektors abgeleitet aus Erdbeobachtungsdaten - Deep Learning-basierte Objekterkennung, optimiert mit synthetischen Trainingsdaten für die Extraktion von Offshore-Windenergieinfrastrukturen aus Sentinel-1 Bildern N2 - The expansion of renewable energies is being driven by the gradual phaseout of fossil fuels in order to reduce greenhouse gas emissions, the steadily increasing demand for energy and, more recently, by geopolitical events. The offshore wind energy sector is on the verge of a massive expansion in Europe, the United Kingdom, China, but also in the USA, South Korea and Vietnam. Accordingly, the largest marine infrastructure projects to date will be carried out in the upcoming decades, with thousands of offshore wind turbines being installed. In order to accompany this process globally and to provide a database for research, development and monitoring, this dissertation presents a deep learning-based approach for object detection that enables the derivation of spatiotemporal developments of offshore wind energy infrastructures from satellite-based radar data of the Sentinel-1 mission. For training the deep learning models for offshore wind energy infrastructure detection, an approach is presented that makes it possible to synthetically generate remote sensing data and the necessary annotation for the supervised deep learning process. In this synthetic data generation process, expert knowledge about image content and sensor acquisition techniques is made machine-readable. Finally, extensive and highly variable training data sets are generated from this knowledge representation, with which deep learning models can learn to detect objects in real-world satellite data. The method for the synthetic generation of training data based on expert knowledge offers great potential for deep learning in Earth observation. Applications of deep learning based methods can be developed and tested faster with this procedure. Furthermore, the synthetically generated and thus controllable training data offer the possibility to interpret the learning process of the optimised deep learning models. The method developed in this dissertation to create synthetic remote sensing training data was finally used to optimise deep learning models for the global detection of offshore wind energy infrastructure. For this purpose, images of the entire global coastline from ESA's Sentinel-1 radar mission were evaluated. The derived data set includes over 9,941 objects, which distinguish offshore wind turbines, transformer stations and offshore wind energy infrastructures under construction from each other. In addition to this spatial detection, a quarterly time series from July 2016 to June 2021 was derived for all objects. This time series reveals the start of construction, the construction phase and the time of completion with subsequent operation for each object. The derived offshore wind energy infrastructure data set provides the basis for an analysis of the development of the offshore wind energy sector from July 2016 to June 2021. For this analysis, further attributes of the detected offshore wind turbines were derived. The most important of these are the height and installed capacity of a turbine. The turbine height was calculated by a radargrammetric analysis of the previously detected Sentinel-1 signal and then used to statistically model the installed capacity. The results show that in June 2021, 8,885 offshore wind turbines with a total capacity of 40.6 GW were installed worldwide. The largest installed capacities are in the EU (15.2 GW), China (14.1 GW) and the United Kingdom (10.7 GW). From July 2016 to June 2021, China has expanded 13 GW of offshore wind energy infrastructure. The EU has installed 8 GW and the UK 5.8 GW of offshore wind energy infrastructure in the same period. This temporal analysis shows that China was the main driver of the expansion of the offshore wind energy sector in the period under investigation. The derived data set for the description of the offshore wind energy sector was made publicly available. It is thus freely accessible to all decision-makers and stakeholders involved in the development of offshore wind energy projects. Especially in the scientific context, it serves as a database that enables a wide range of investigations. Research questions regarding offshore wind turbines themselves as well as the influence of the expansion in the coming decades can be investigated. This supports the imminent and urgently needed expansion of offshore wind energy in order to promote sustainable expansion in addition to the expansion targets that have been set. N2 - Der Ausbau erneuerbarer Energien wird durch den sukzessiven Verzicht auf fossile Energieträger zur Reduktion der Treibhausgasemissionen, dem stetig steigenden Energiebedarf sowie, in jüngster Zeit, von geopolitischen Ereignissen stark vorangetrieben. Der offshore Windenergiesektor steht in Europa, dem Vereinigten Königreich, China, aber auch den USA, Süd-Korea und Vietnam vor einer massiven Expansion. In den nächsten Dekaden werden die bislang größten marinen Infrastrukturprojekte mit tausenden neu installierten offshore Windturbinen realisiert. Um diesen Prozess global zu begleiten und eine Datengrundlage für die Forschung, für Entscheidungsträger und für ein kontinuierliches Monitoring bereit zu stellen, präsentiert diese Dissertation einen Deep Learning basierten Ansatz zur Detektion von offshore Windkraftanalagen aus satellitengestützten Radardaten der Sentinel-1 Mission. Für das überwachte Training der verwendeten Deep Learning Modelle zur Objektdetektion wird ein Ansatz vorgestellt, der es ermöglicht, Fernerkundungsdaten und die notwendigen Label synthetisch zu generieren. Hierbei wird Expertenwissen über die Bildinhalte, wie offshore Windkraftanlagen aber auch ihre natürliche Umgebung, wie Küsten oder andere Infrastruktur, gemeinsam mit Informationen über den Sensor strukturiert und maschinenlesbar gemacht. Aus dieser Wissensrepräsentation werden schließlich umfangreiche und höchst variable Trainingsdaten erzeugt, womit Deep Learning Modelle die Detektion von Objekten in Satellitendaten erlernen können. Das Verfahren zur synthetischen Erzeugung von Trainingsdaten basierend auf Expertenwissen bietet großes Potential für Deep Learning in der Erdbeobachtung. Deep Learning Ansätze können hierdurch schneller entwickelt und getestet werden. Darüber hinaus bieten die synthetisch generierten und somit kontrollierbaren Trainingsdaten die Möglichkeit, den Lernprozess der optimierten Deep Learning Modelle zu interpretieren. Das in dieser Dissertation für Fernerkundungsdaten entwickelte Verfahren zur Erstellung synthetischer Trainingsdaten wurde schließlich zur Optimierung von Deep Learning Modellen für die globale Detektion von offshore Windenergieanlagen eingesetzt. Hierfür wurden Aufnahmen der gesamten globalen Küstenlinie der Sentinel-1 Mission der ESA ausgewertet. Der abgeleitete Datensatz, welcher 9.941 Objekte umfasst, unterscheidet offshore Windturbinen, Trafostationen und im Bau befindliche offshore Windenergieinfrastrukturen voneinander. Zusätzlich zu dieser räumlichen Detektion wurde eine vierteljährliche Zeitreihe von Juli 2016 bis Juni 2021 für alle Objekte generiert. Diese Zeitreihe zeigt den Start des Baubeginns, die Bauphase und den Zeitpunkt der Fertigstellung mit anschließendem Betrieb für jedes Objekt. Der gewonnene Datensatz dient weiterhin als Grundlage für eine Analyse der Entwicklung des offshore Windenergiesektors von Juli 2016 bis Juni 2021. Für diese Analyse wurden weitere Attribute der Turbinen abgeleitet. In einem radargrammetrischen Verfahren wurde die Turbinenhöhe berechnet und anschließend verwendet, um die installierte Leistung statistisch zu modellieren. Die Ergebnisse hierzu zeigen, dass im Juni 2021 weltweit 8.885 offshore Windturbinen mit insgesamt 40,6 GW Leistung installiert waren. Die größten installierten Leistungen stellen dabei die EU (15,2 GW), China (14,1 GW) und das Vereinigte Königreich (10,7 GW). Von Juli 2016 bis Juni 2021 hat China 13 GW installierte Leistung ausgebaut. Die EU hat im selben Zeitraum 8 GW und das Vereinigte Königreich 5,8 GW offshore Windenergieinfrastruktur installiert. Diese zeitliche Analyse verdeutlicht, dass China der maßgebliche Treiber in der Expansion des offshore Windenergiesektors im untersuchten Zeitraum war. Der abgeleitete Datensatz zur Beschreibung des offshore Windenergiesektors wurde öffentlich zugänglich gemacht. Somit steht er allen Entscheidungsträgern und Stakeholdern, die am Ausbau von offshore Windenergieanlagen beteiligt sind, frei zur Verfügung. Vor allem im wissenschaftlichen Kontext dient er als Datenbasis, welche unterschiedlichste Untersuchungen ermöglicht. Hierbei können sowohl Forschungsfragen bezüglich der offshore Windenergieanlagen selbst, als auch der Einfluss des Ausbaus der kommenden Dekaden untersucht werden. Somit wird der bevorstehende und dringend notwendige Ausbau der offshore Windenergie unterstützt, um neben den gesteckten Zielen auch einen nachhaltigen Ausbau zu fördern. KW - deep learning KW - offshore wind energy KW - artificial intelligence KW - earth observation KW - remote sensing Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-292857 ER - TY - JOUR A1 - Philipp, Marius A1 - Dietz, Andreas A1 - Ullmann, Tobias A1 - Kuenzer, Claudia T1 - Automated extraction of annual erosion rates for Arctic permafrost coasts using Sentinel-1, Deep Learning, and Change Vector Analysis JF - Remote Sensing N2 - Arctic permafrost coasts become increasingly vulnerable due to environmental drivers such as the reduced sea-ice extent and duration as well as the thawing of permafrost itself. A continuous quantification of the erosion process on large to circum-Arctic scales is required to fully assess the extent and understand the consequences of eroding permafrost coastlines. This study presents a novel approach to quantify annual Arctic coastal erosion and build-up rates based on Sentinel-1 (S1) Synthetic Aperture RADAR (SAR) backscatter data, in combination with Deep Learning (DL) and Change Vector Analysis (CVA). The methodology includes the generation of a high-quality Arctic coastline product via DL, which acted as a reference for quantifying coastal erosion and build-up rates from annual median and standard deviation (sd) backscatter images via CVA. The analysis was applied on ten test sites distributed across the Arctic and covering about 1038 km of coastline. Results revealed maximum erosion rates of up to 160 m for some areas and an average erosion rate of 4.37 m across all test sites within a three-year temporal window from 2017 to 2020. The observed erosion rates within the framework of this study agree with findings published in the previous literature. The proposed methods and data can be applied on large scales and, prospectively, even for the entire Arctic. The generated products may be used for quantifying the loss of frozen ground, estimating the release of stored organic material, and can act as a basis for further related studies in Arctic coastal environments. KW - permafrost KW - coastal erosion KW - deep learning KW - change vector analysis KW - Google Earth Engine KW - synthetic aperture RADAR Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-281956 SN - 2072-4292 VL - 14 IS - 15 ER - TY - JOUR A1 - Philipp, Marius A1 - Dietz, Andreas A1 - Ullmann, Tobias A1 - Kuenzer, Claudia T1 - A circum-Arctic monitoring framework for quantifying annual erosion rates of permafrost coasts JF - Remote Sensing N2 - This study demonstrates a circum-Arctic monitoring framework for quantifying annual change of permafrost-affected coasts at a spatial resolution of 10 m. Frequent cloud coverage and challenging lighting conditions, including polar night, limit the usability of optical data in Arctic regions. For this reason, Synthetic Aperture RADAR (SAR) data in the form of annual median and standard deviation (sd) Sentinel-1 (S1) backscatter images covering the months June–September for the years 2017–2021 were computed. Annual composites for the year 2020 were hereby utilized as input for the generation of a high-quality coastline product via a Deep Learning (DL) workflow, covering 161,600 km of the Arctic coastline. The previously computed annual S1 composites for the years 2017 and 2021 were employed as input data for the Change Vector Analysis (CVA)-based coastal change investigation. The generated DL coastline product served hereby as a reference. Maximum erosion rates of up to 67 m per year could be observed based on 400 m coastline segments. Overall highest average annual erosion can be reported for the United States (Alaska) with 0.75 m per year, followed by Russia with 0.62 m per year. Out of all seas covered in this study, the Beaufort Sea featured the overall strongest average annual coastal erosion of 1.12 m. Several quality layers are provided for both the DL coastline product and the CVA-based coastal change analysis to assess the applicability and accuracy of the output products. The predicted coastal change rates show good agreement with findings published in previous literature. The proposed methods and data may act as a valuable tool for future analysis of permafrost loss and carbon emissions in Arctic coastal environments. KW - permafrost KW - coastal erosion KW - circum-Arctic KW - deep learning KW - change vector analysis KW - Google Earth Engine KW - synthetic aperture RADAR Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-304447 SN - 2072-4292 VL - 15 IS - 3 ER -