TY - JOUR A1 - Reinhard, Sebastian A1 - Helmerich, Dominic A. A1 - Boras, Dominik A1 - Sauer, Markus A1 - Kollmannsberger, Philip T1 - ReCSAI: recursive compressed sensing artificial intelligence for confocal lifetime localization microscopy JF - BMC Bioinformatics N2 - Background Localization-based super-resolution microscopy resolves macromolecular structures down to a few nanometers by computationally reconstructing fluorescent emitter coordinates from diffraction-limited spots. The most commonly used algorithms are based on fitting parametric models of the point spread function (PSF) to a measured photon distribution. These algorithms make assumptions about the symmetry of the PSF and thus, do not work well with irregular, non-linear PSFs that occur for example in confocal lifetime imaging, where a laser is scanned across the sample. An alternative method for reconstructing sparse emitter sets from noisy, diffraction-limited images is compressed sensing, but due to its high computational cost it has not yet been widely adopted. Deep neural network fitters have recently emerged as a new competitive method for localization microscopy. They can learn to fit arbitrary PSFs, but require extensive simulated training data and do not generalize well. A method to efficiently fit the irregular PSFs from confocal lifetime localization microscopy combining the advantages of deep learning and compressed sensing would greatly improve the acquisition speed and throughput of this method. Results Here we introduce ReCSAI, a compressed sensing neural network to reconstruct localizations for confocal dSTORM, together with a simulation tool to generate training data. We implemented and compared different artificial network architectures, aiming to combine the advantages of compressed sensing and deep learning. We found that a U-Net with a recursive structure inspired by iterative compressed sensing showed the best results on realistic simulated datasets with noise, as well as on real experimentally measured confocal lifetime scanning data. Adding a trainable wavelet denoising layer as prior step further improved the reconstruction quality. Conclusions Our deep learning approach can reach a similar reconstruction accuracy for confocal dSTORM as frame binning with traditional fitting without requiring the acquisition of multiple frames. In addition, our work offers generic insights on the reconstruction of sparse measurements from noisy experimental data by combining compressed sensing and deep learning. We provide the trained networks, the code for network training and inference as well as the simulation tool as python code and Jupyter notebooks for easy reproducibility. KW - compressed sensing KW - AI KW - SMLM KW - FLIMbee KW - dSTORM Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-299768 VL - 23 IS - 1 ER - TY - JOUR A1 - Hoeser, Thorsten A1 - Kuenzer, Claudia T1 - Object detection and image segmentation with deep learning on Earth observation data: a review-part I: evolution and recent trends JF - Remote Sensing N2 - Deep learning (DL) has great influence on large parts of science and increasingly established itself as an adaptive method for new challenges in the field of Earth observation (EO). Nevertheless, the entry barriers for EO researchers are high due to the dense and rapidly developing field mainly driven by advances in computer vision (CV). To lower the barriers for researchers in EO, this review gives an overview of the evolution of DL with a focus on image segmentation and object detection in convolutional neural networks (CNN). The survey starts in 2012, when a CNN set new standards in image recognition, and lasts until late 2019. Thereby, we highlight the connections between the most important CNN architectures and cornerstones coming from CV in order to alleviate the evaluation of modern DL models. Furthermore, we briefly outline the evolution of the most popular DL frameworks and provide a summary of datasets in EO. By discussing well performing DL architectures on these datasets as well as reflecting on advances made in CV and their impact on future research in EO, we narrow the gap between the reviewed, theoretical concepts from CV and practical application in EO. KW - artificial intelligence KW - AI KW - machine learning KW - deep learning KW - neural networks KW - convolutional neural networks KW - CNN KW - image segmentation KW - object detection KW - Earth observation Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-205918 SN - 2072-4292 VL - 12 IS - 10 ER - TY - JOUR A1 - Hoeser, Thorsten A1 - Bachofer, Felix A1 - Kuenzer, Claudia T1 - Object detection and image segmentation with deep learning on Earth Observation data: a review — part II: applications JF - Remote Sensing N2 - In Earth observation (EO), large-scale land-surface dynamics are traditionally analyzed by investigating aggregated classes. The increase in data with a very high spatial resolution enables investigations on a fine-grained feature level which can help us to better understand the dynamics of land surfaces by taking object dynamics into account. To extract fine-grained features and objects, the most popular deep-learning model for image analysis is commonly used: the convolutional neural network (CNN). In this review, we provide a comprehensive overview of the impact of deep learning on EO applications by reviewing 429 studies on image segmentation and object detection with CNNs. We extensively examine the spatial distribution of study sites, employed sensors, used datasets and CNN architectures, and give a thorough overview of applications in EO which used CNNs. Our main finding is that CNNs are in an advanced transition phase from computer vision to EO. Upon this, we argue that in the near future, investigations which analyze object dynamics with CNNs will have a significant impact on EO research. With a focus on EO applications in this Part II, we complete the methodological review provided in Part I. KW - artificial intelligence KW - AI KW - machine learning KW - deep learning KW - neural networks KW - convolutional neural networks KW - CNN KW - image segmentation KW - object detection KW - earth observation Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-213152 SN - 2072-4292 VL - 12 IS - 18 ER - TY - JOUR A1 - Kazuhino, Koshino A1 - Werner, Rudolf A. A1 - Toriumi, Fuijo A1 - Javadi, Mehrbod S. A1 - Pomper, Martin G. A1 - Solnes, Lilja B. A1 - Verde, Franco A1 - Higuchi, Takahiro A1 - Rowe, Steven P. T1 - Generative Adversarial Networks for the Creation of Realistic Artificial Brain Magnetic Resonance Images JF - Tomography N2 - Even as medical data sets become more publicly accessible, most are restricted to specific medical conditions. Thus, data collection for machine learning approaches remains challenging, and synthetic data augmentation, such as generative adversarial networks (GAN), may overcome this hurdle. In the present quality control study, deep convolutional GAN (DCGAN)-based human brain magnetic resonance (MR) images were validated by blinded radiologists. In total, 96 T1-weighted brain images from 30 healthy individuals and 33 patients with cerebrovascular accident were included. A training data set was generated from the T1-weighted images and DCGAN was applied to generate additional artificial brain images. The likelihood that images were DCGAN-created versus acquired was evaluated by 5 radiologists (2 neuroradiologists [NRs], vs 3 non-neuroradiologists [NNRs]) in a binary fashion to identify real vs created images. Images were selected randomly from the data set (variation of created images, 40%-60%). None of the investigated images was rated as unknown. Of the created images, the NRs rated 45% and 71% as real magnetic resonance imaging images (NNRs, 24%, 40%, and 44%). In contradistinction, 44% and 70% of the real images were rated as generated images by NRs (NNRs, 10%, 17%, and 27%). The accuracy for the NRs was 0.55 and 0.30 (NNRs, 0.83, 0.72, and 0.64). DCGAN-created brain MR images are similar enough to acquired MR images so as to be indistinguishable in some cases. Such an artificial intelligence algorithm may contribute to synthetic data augmentation for "data-hungry" technologies, such as supervised machine learning approaches, in various clinical applications. KW - AI KW - Magnetresonanztomografie KW - artificial intelligence KW - magnetic resonance imaging KW - MRI KW - DCGAN KW - GAN KW - stroke KW - machine learning Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-172185 VL - 4 IS - 4 ER - TY - THES A1 - Lohr, David T1 - Functional and Structural Characterization of the Myocardium T1 - Funktionelle und Strukturelle Charakterisierung des Myokardiums N2 - Clinical practice in CMR with respect to cardiovascular disease is currently focused on tissue characterization, and cardiac function, in particular. In recent years MRI based diffusion tensor imaging (DTI) has been shown to enable the assessment of microstructure based on the analysis of Brownian motion of water molecules in anisotropic tissue, such as the myocardium. With respect to both functional and structural imaging, 7T MRI may increase SNR, providing access to information beyond the reach of clinically applied field strengths. To date, cardiac 7T MRI is still a research modality that is only starting to develop towards clinical application. In this thesis we primarily aimed to advance methods of ultrahigh field CMR using the latest 7T technology and its application towards the functional and structural characterization of the myocardium. Regarding the assessment of myocardial microstructure at 7T, feasibility of ex vivo DTI of large animal hearts was demonstrated. In such hearts a custom sequence implemented for in vivo DTI was evaluated and fixation induced alterations of derived diffusion metrics and tissue properties were assessed. Results enable comparison of prior and future ex vivo DTI studies and provide information on measurement parameters at 7T. Translating developed methodology to preclinical studies of mouse hearts, ex vivo DTI provided highly sensitive surrogates for microstructural remodeling in response to subendocardial damage. In such cases echocardiography measurements revealed mild diastolic dysfunction and impaired longitudinal deformation, linking disease induced structural and functional alterations. Complementary DTI and echocardiography data also improved our understanding of structure-function interactions in cases of loss of contractile myofiber tracts, replacement fibrosis, and LV systolic failure. Regarding the functional characterization of the myocardium at 7T, sequence protocols were expanded towards a dedicated 7T routine protocol, encompassing accurate cardiac planning and the assessment of cardiac function via cine imaging in humans. This assessment requires segmentation of myocardial contours. For that, artificial intelligence (AI) was developed and trained, enabling rapid automatic generation of cardiac segmentation in clinical data. Using transfer learning, AI models were adapted to cine data acquired using the latest generation 7T system. Methodology for AI based segmentation was translated to cardiac pathology, where automatic segmentation of scar tissue, edema and healthy myocardium was achieved. Developed radiofrequency hardware facilitates translational studies at 7T, providing controlled conditions for future method development towards cardiac 7T MRI in humans. In this thesis the latest 7T technology, cardiac DTI, and AI were used to advance methods of ultrahigh field CMR. In the long run, obtained results contribute to diagnostic methods that may facilitate early detection and risk stratification in cardiovascular disease. N2 - Bei kardiovaskulären Erkrankungen konzentriert sich die kardiale MRT aktuell auf die Gewebecharakterisierung und insbesondere die Herzfunktion. In den letzten Jahren hat sich gezeigt, dass MRT-basierte Diffusions-Tensor-Bildgebung (DTI) die Beurteilung der Mikrostruktur anhand der Analyse der Brownschen Bewegung von Wassermolekülen in anisotropem Gewebe, wie dem Myokardium, ermöglicht. In Bezug auf sowohl die funktionelle als auch die strukturelle Bildgebung kann 7T MRT SNR verbessern und Information messbar machen, die außerhalb der Reichweite von klinisch angewendeten Feldstärken liegt. Heute ist kardiale 7T MRT noch eine Forschungsmodalität, die sich Richtung klinischer Anwendung entwickelt. Hauptziel dieser Dissertation war die Weiterentwicklung von Methoden der kardialen Ultrahochfeld-Bildgebung mittels der neuesten 7T-Technologie und dessen Anwendung für die funktionelle und strukturelle Charakterisierung des Myokardiums. Für die Mikrostrukturcharakterisierung des Myokardiums bei 7T wurde die Durchführbarkeit von ex vivo DTI Messungen von Großtierherzen demonstriert. In solchen Herzen wurde eine Sequenz evaluiert, die für in vivo DTI etabliert wurde. Zudem wurden fixationsbedinge Veränderungen von Diffusionsparametern und Gewebeeigenschaften ermittelt. Die Ergebnisse erlauben den Vergleich von bestehenden und zukünftigen ex vivo Studien und geben Informationen zu Messparametern bei 7T. Der Transfer von etablierten Methoden zu präklinischen Studien in Mäuseherzen demonstrierte, dass ex vivo DTI sensitive Marker für Mikrostruktur-Remodeling nach Subendokard-Schäden liefern kann. In solchen Fällen zeigte Echokardiographie eine leichte diastolische Dysfunktion und eingeschränkte Longitudinalverformung. Komplementäre DTI und Echokardiographie-Daten erweiterten zudem unser Verständnis von Struktur-Funktions-Interaktionen in Fällen von Verlust von kontraktilen Faserbündeln, Fibrose und linksventrikulärem, systolischem Versagen. Für die funktionelle Charakterisierung des Myokardiums bei 7T wurde ein dediziertes 7T-Humanprotokoll erarbeitet, welches akkurate Schichtplanung und die Bestimmung der Herzfunktion mittels Cine-Bildgebung umfasst. Die Herzfunktionsbestimmung erfordert die Segmentierung des Myokards. Hierfür wurde künstliche Intelligenz (KI) entwickelt, die eine schnelle, automatische Herzsegmentierung in klinischen Daten ermöglicht. Mittels Lerntransfer wurden KI-Modelle für Bilder angepasst, die mit der neuesten 7T-Technologie aufgenommen wurden. Methoden für die KI-basierte Segmentierung wurden zudem für die Bestimmung und Segmentierung von Narbengewebe, Ödemen und gesundem Myokard erweitert. Entwickelte Radiofrequenz-Komponenten ermöglichen translationale 7T-Studien, welche kontrollierte Bedingungen für die Methodenentwicklung von kardialen 7T-Anwendungen für den Humanbereich liefern. In dieser Arbeit werden die neueste 7T-Technologie, DTI am Herzen und AI genutzt, um Methoden der kardialen Ultrahochfeld-Bildgebung weiterzuentwickeln. Langfristig erweitern die erzielten Ergebnisse diagnostische Methoden, die Früherkennung und Risikoabschätzung in kardiovaskulären Erkrankungen ermöglichen können. KW - Diffusionsgewichtete Magnetresonanztomografie KW - Künstliche Intelligenz KW - 7T KW - DTI KW - AI KW - Cardiac Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-234486 ER -