TY - THES A1 - Hofmann, Jan T1 - Deep Reinforcement Learning for Configuration of Time-Sensitive-Networking T1 - Deep Reinforcement Learning zur Konfiguration von Time-Sensitive-Networking N2 - Reliable, deterministic real-time communication is fundamental to most industrial systems today. In many other domains Ethernet has become the most common platform for communication networks, but has been unsuitable to satisfy the requirements of industrial networks for a long time. This has changed with the introduction of Time-Sensitive-Networking (TSN), a set of standards utilizing Ethernet to implement deterministic real-time networks. This makes Ethernet a viable alternative to the expensive fieldbus systems commonly used in industrial environments. However, TSN is not a silver bullet. Industrial networks are a complex and highly dynamic environment and the configuration of TSN, especially with respect to latency, is a challenging but crucial task. Various approaches have been pursued for the configuration of TSN in dynamic industrial environments. Optimization techniques like Linear Programming (LP) are able to determine an optimal configuration for a given network, but the time consumption exponentially increases with the complexity of the environment. Machine Learning (ML) has become widely popular in the last years and is able to approximate a near-optimal TSN configuration for networks of different complexity. Yet, ML models are usually trained in a supervised manner which requires large amounts of data that have to be generated for the specific environment. Therefore, supervised methods are not scalable and do not adapt to changing dynamics of the network environment. To address these issues, this work proposes a Deep Reinforcement Learning (DRL) approach to the configuration of TSN in industrial networks. DRL combines two different disciplines, Deep Learning (DL) and Reinforcement Learning (RL), and has gained considerable traction in the last years due to breakthroughs in various domains. RL is supposed to autonomously learn a challenging task like the configuration of TSN without requiring any training data. The addition of DL allows to apply well-studied RL methods to a complex environment such as dynamic industrial networks. There are two major contributions made in this work. In the first step, an interactive environment is proposed which allows for the simulation and configuration of industrial networks using basic TSN mechanisms. The environment provides an interface that allows to apply various DRL methods to the problem of TSN configuration. The second contribution of this work is an in-depth study on the application of two fundamentally different DRL methods to the proposed environment. Both methods are evaluated on networks of different complexity and the results are compared to the ground truth and to the results of two supervised ML approaches. Ultimately, this work investigates if DRL can adapt to changing dynamics of the environment in a more scalable manner than supervised methods. N2 - Zuverlässige Echtzeitnetzwerke spielen eine zentrale Rolle im heutigen industriellen Umfeld. Während sich in anderen Anwendungsbereichen Ethernet als Technik für Kommunikationsnetze durchsetzen konnte, basiert industrielle Kommunikation bis heute häufig noch auf teuren Feldbus-Systemen. Mit der Einführung von Time-Sensitive-Networking (TSN) wurde Ethernet schließlich um eine Reihe von Standards erweitert, die die hohen Anforderungen an Echtzeitkommunikation erfüllen und Ethernet damit auch im industriellen Umfeld etablieren sollen. Doch für eine zuverlässige Kommunikation, besonders im Hinblick auf die Übertragungsverzögerung von Datenpaketen (Latenz), ist die richtige Konfiguration von TSN entscheidend. Dynamische Netzwerke zu konfigurieren ist ein Optimierungsproblem, das verschiedene Herausforderungen birgt. Verfahren wie die lineare Optimierung liefern zwar optimale Ergebnisse, jedoch steigt der Zeitaufwand exponentiell mit der Größe der Netzwerke. Moderne Lösungsansätze wie Machine Learning (ML) können sich einer optimalen Lösung annähern, benötigen jedoch üblicherweise große Datenmengen, auf denen sie trainiert werden (Supervised Learning). Diese Arbeit untersucht die Anwendung von Deep Reinforcement Learning (DRL) zur Konfiguration von TSN. DRL kombiniert Reinforcement Learning (RL), also das selbstständige Lernen ausschließlich durch Interaktion, mit dem Deep Learning (DL), dem Lernen mittels tiefer neuronaler Netze. Die Arbeit beschreibt, wie sich eine Umgebung für DRL zur Simulation und Konfiguration von industriellen Netzwerken implementieren lässt, und untersucht die Anwendung zweier unterschiedlicher Ansätze von DRL auf das Problem der TSN-Konfiguration. Beide Methoden wurden anhand von zwei unterschiedlich komplexen Datensätzen ausgewertet und die Ergebnisse sowohl mit den zeitaufwändig generierten Optimallösungen als auch mit den Ergebnissen zweier Supervised Learning-Ansätze verglichen. Es konnte gezeigt werden, dass DRL optimale Ergebnisse auf kleinen Netzwerken erzielen kann und insgesamt in der Lage ist, Supervised Learning bei der Konfiguration von TSN zu übertreffen. Weiterhin konnte in der Arbeit demonstriert werden, dass sich DRL schnell an fundamentale Veränderungen der Umgebung anpassen kann, was mit Supervised Learning nur durch deutlichen Mehraufwand möglich ist. KW - Reinforcement Learning KW - Time-Sensitive Networking KW - Deep Reinforcement Learning KW - Time-Sensitive-Networking KW - Real-Time-Networks KW - Bestärkendes Lernen KW - Echtzeit-Netzwerke Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-215953 ER - TY - JOUR A1 - Traub, Jan A1 - Grondey, Katja A1 - Gassenmaier, Tobias A1 - Schmitt, Dominik A1 - Fette, Georg A1 - Frantz, Stefan A1 - Boivin-Jahns, Valérie A1 - Jahns, Roland A1 - Störk, Stefan A1 - Stoll, Guido A1 - Reiter, Theresa A1 - Hofmann, Ulrich A1 - Weber, Martin S. A1 - Frey, Anna T1 - Sustained increase in serum glial fibrillary acidic protein after first ST-elevation myocardial infarction JF - International Journal of Molecular Sciences N2 - Acute ischemic cardiac injury predisposes one to cognitive impairment, dementia, and depression. Pathophysiologically, recent positron emission tomography data suggest astroglial activation after experimental myocardial infarction (MI). We analyzed peripheral surrogate markers of glial (and neuronal) damage serially within 12 months after the first ST-elevation MI (STEMI). Serum levels of glial fibrillary acidic protein (GFAP) and neurofilament light chain (NfL) were quantified using ultra-sensitive molecular immunoassays. Sufficient biomaterial was available from 45 STEMI patients (aged 28 to 78 years, median 56 years, 11% female). The median (quartiles) of GFAP was 63.8 (47.0, 89.9) pg/mL and of NfL 10.6 (7.2, 14.8) pg/mL at study entry 0–4 days after STEMI. GFAP after STEMI increased in the first 3 months, with a median change of +7.8 (0.4, 19.4) pg/mL (p = 0.007). It remained elevated without further relevant increases after 6 months (+11.7 (0.6, 23.5) pg/mL; p = 0.015), and 12 months (+10.3 (1.5, 22.7) pg/mL; p = 0.010) compared to the baseline. Larger relative infarction size was associated with a higher increase in GFAP (ρ = 0.41; p = 0.009). In contrast, NfL remained unaltered in the course of one year. Our findings support the idea of central nervous system involvement after MI, with GFAP as a potential peripheral biomarker of chronic glial damage as one pathophysiologic pathway. KW - myocardial infarction KW - STEMI KW - glial fibrillary acidic protein KW - GFAP KW - neurofilament light chain KW - NfL KW - glial damage KW - cardiac magnetic resonance imaging KW - MRI KW - infarction size Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-288261 SN - 1422-0067 VL - 23 IS - 18 ER -