TY - JOUR A1 - Krenzer, Adrian A1 - Makowski, Kevin A1 - Hekalo, Amar A1 - Fitting, Daniel A1 - Troya, Joel A1 - Zoller, Wolfram G. A1 - Hann, Alexander A1 - Puppe, Frank T1 - Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists JF - BioMedical Engineering OnLine N2 - Background Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between. Methods In our framework, an expert reviews the video and annotates a few video frames to verify the object’s annotations for the non-expert. In a second step, a non-expert has visual confirmation of the given object and can annotate all following and preceding frames with AI assistance. After the expert has finished, relevant frames will be selected and passed on to an AI model. This information allows the AI model to detect and mark the desired object on all following and preceding frames with an annotation. Therefore, the non-expert can adjust and modify the AI predictions and export the results, which can then be used to train the AI model. Results Using this framework, we were able to reduce workload of domain experts on average by a factor of 20 on our data. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated AI model enhances the annotation speed further. Through a prospective study with 10 participants, we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool. Conclusion In summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source. KW - object detection KW - machine learning KW - deep learning KW - annotation KW - endoscopy KW - gastroenterology KW - automation Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-300231 VL - 21 IS - 1 ER - TY - JOUR A1 - Hartelt, Alexander A1 - Puppe, Frank T1 - Optical Medieval Music Recognition using background knowledge JF - Algorithms N2 - This paper deals with the effect of exploiting background knowledge for improving an OMR (Optical Music Recognition) deep learning pipeline for transcribing medieval, monophonic, handwritten music from the 12th–14th century, whose usage has been neglected in the literature. Various types of background knowledge about overlapping notes and text, clefs, graphical connections (neumes) and their implications on the position in staff of the notes were used and evaluated. Moreover, the effect of different encoder/decoder architectures and of different datasets for training a mixed model and for document-specific fine-tuning based on an extended OMR pipeline with an additional post-processing step were evaluated. The use of background models improves all metrics and in particular the melody accuracy rate (mAR), which is based on the insert, delete and replace operations necessary to convert the generated melody into the correct melody. When using a mixed model and evaluating on a different dataset, our best model achieves without fine-tuning and without post-processing a mAR of 90.4%, which is raised by nearly 30% to 93.2% mAR using background knowledge. With additional fine-tuning, the contribution of post-processing is even greater: the basic mAR of 90.5% is raised by more than 50% to 95.8% mAR. KW - Optical Music Recognition KW - historical document analysis KW - medieval manuscripts KW - neume notation KW - fully convolutional neural networks KW - background knowledge Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-278756 SN - 1999-4893 VL - 15 IS - 7 ER - TY - JOUR A1 - Puppe, Frank T1 - Gesellschaftliche Perspektiven einer fachspezifischen KI für automatisierte Entscheidungen JF - Informatik Spektrum N2 - Die künstliche Intelligenz (KI) entwickelt sich rasant und hat bereits eindrucksvolle Erfolge zu verzeichnen, darunter übermenschliche Kompetenz in den meisten Spielen und vielen Quizshows, intelligente Suchmaschinen, individualisierte Werbung, Spracherkennung, -ausgabe und -übersetzung auf sehr hohem Niveau und hervorragende Leistungen bei der Bildverarbeitung, u. a. in der Medizin, der optischen Zeichenerkennung, beim autonomen Fahren, aber auch beim Erkennen von Menschen auf Bildern und Videos oder bei Deep Fakes für Fotos und Videos. Es ist zu erwarten, dass die KI auch in der Entscheidungsfindung Menschen übertreffen wird; ein alter Traum der Expertensysteme, der durch Lernverfahren, Big Data und Zugang zu dem gesammelten Wissen im Web in greifbare Nähe rückt. Gegenstand dieses Beitrags sind aber weniger die technischen Entwicklungen, sondern mögliche gesellschaftliche Auswirkungen einer spezialisierten, kompetenten KI für verschiedene Bereiche der autonomen, d. h. nicht nur unterstützenden Entscheidungsfindung: als Fußballschiedsrichter, in der Medizin, für richterliche Entscheidungen und sehr spekulativ auch im politischen Bereich. Dabei werden Vor- und Nachteile dieser Szenarien aus gesellschaftlicher Sicht diskutiert. KW - Künstliche Intelligenz KW - Ethik KW - Entscheidungsfindung Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-324197 SN - 0170-6012 VL - 45 IS - 2 ER - TY - JOUR A1 - Dakroub, Mohamad A1 - Verma-Fuehring, Raoul A1 - Agorastou, Vaia A1 - Schön, Julian A1 - Hillenkamp, Jost A1 - Puppe, Frank A1 - Loewen, Nils A. T1 - Inter-eye correlation analysis of 24-h IOPs and glaucoma progression JF - Graefe’s Archive for Clinical and Experimental Ophthalmology N2 - Purpose To determine whether 24-h IOP monitoring can be a predictor for glaucoma progression and to analyze the inter-eye relationship of IOP, perfusion, and progression parameters. Methods We extracted data from manually drawn IOP curves with HIOP-Reader, a software suite we developed. The relationship between measured IOPs and mean ocular perfusion pressures (MOPP) to retinal nerve fiber layer (RNFL) thickness was analyzed. We determined the ROC curves for peak IOP (T\(_{max}\)), average IOP(T\(_{avg}\)), IOP variation (IOP\(_{var}\)), and historical IOP cut-off levels to detect glaucoma progression (rate of RNFL loss). Bivariate analysis was also conducted to check for various inter-eye relationships. Results Two hundred seventeen eyes were included. The average IOP was 14.8 ± 3.5 mmHg, with a 24-h variation of 5.2 ± 2.9 mmHg. A total of 52% of eyes with RNFL progression data showed disease progression. There was no significant difference in T\(_{max}\), T\(_{avg}\), and IOP\(_{var}\) between progressors and non-progressors (all p > 0.05). Except for T\(_{avg}\) and the temporal RNFL, there was no correlation between disease progression in any quadrant and T\(_{max}\), T\(_{avg}\), and IOP\(_{var}\). Twenty-four-hour and outpatient IOP variables had poor sensitivities and specificities in detecting disease progression. The correlation of inter-eye parameters was moderate; correlation with disease progression was weak. Conclusion In line with our previous study, IOP data obtained during a single visit (outpatient or inpatient monitoring) make for a poor diagnostic tool, no matter the method deployed. Glaucoma progression and perfusion pressure in left and right eyes correlated weakly to moderately with each other. Key messages What is known: ● Our prior study showed that manually obtained 24-hour inpatient IOP measurements in right eyes are poor predictors for glaucoma progression. The inter-eye relationship of 24-hour IOP parameters and disease progression on optical coherence tomography (OCT) has not been examined. What we found: ● 24-hour IOP profiles of left eyes from the same study were a poor diagnostic tool to detect worsening glaucoma. ● Significant inter-eye correlations of various strengths were found for all tested parameters KW - glaucoma progression KW - nycthemeral intraocular pressure KW - right-left comparison KW - laterality Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-323831 VL - 260 IS - 10 ER - TY - JOUR A1 - Lux, Thomas J. A1 - Banck, Michael A1 - Saßmannshausen, Zita A1 - Troya, Joel A1 - Krenzer, Adrian A1 - Fitting, Daniel A1 - Sudarevic, Boban A1 - Zoller, Wolfram G. A1 - Puppe, Frank A1 - Meining, Alexander A1 - Hann, Alexander T1 - Pilot study of a new freely available computer-aided polyp detection system in clinical practice JF - International Journal of Colorectal Disease N2 - Purpose Computer-aided polyp detection (CADe) systems for colonoscopy are already presented to increase adenoma detection rate (ADR) in randomized clinical trials. Those commercially available closed systems often do not allow for data collection and algorithm optimization, for example regarding the usage of different endoscopy processors. Here, we present the first clinical experiences of a, for research purposes publicly available, CADe system. Methods We developed an end-to-end data acquisition and polyp detection system named EndoMind. Examiners of four centers utilizing four different endoscopy processors used EndoMind during their clinical routine. Detected polyps, ADR, time to first detection of a polyp (TFD), and system usability were evaluated (NCT05006092). Results During 41 colonoscopies, EndoMind detected 29 of 29 adenomas in 66 of 66 polyps resulting in an ADR of 41.5%. Median TFD was 130 ms (95%-CI, 80–200 ms) while maintaining a median false positive rate of 2.2% (95%-CI, 1.7–2.8%). The four participating centers rated the system using the System Usability Scale with a median of 96.3 (95%-CI, 70–100). Conclusion EndoMind’s ability to acquire data, detect polyps in real-time, and high usability score indicate substantial practical value for research and clinical practice. Still, clinical benefit, measured by ADR, has to be determined in a prospective randomized controlled trial. KW - colonoscopy KW - polyp KW - artificial intelligence KW - deep learning KW - CADe Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-324459 VL - 37 IS - 6 ER -