TY - JOUR A1 - Zimmerer, Chris A1 - Fischbach, Martin A1 - Latoschik, Marc Erich T1 - Semantic Fusion for Natural Multimodal Interfaces using Concurrent Augmented Transition Networks JF - Multimodal Technologies and Interaction N2 - Semantic fusion is a central requirement of many multimodal interfaces. Procedural methods like finite-state transducers and augmented transition networks have proven to be beneficial to implement semantic fusion. They are compliant with rapid development cycles that are common for the development of user interfaces, in contrast to machine-learning approaches that require time-costly training and optimization. We identify seven fundamental requirements for the implementation of semantic fusion: Action derivation, continuous feedback, context-sensitivity, temporal relation support, access to the interaction context, as well as the support of chronologically unsorted and probabilistic input. A subsequent analysis reveals, however, that there is currently no solution for fulfilling the latter two requirements. As the main contribution of this article, we thus present the Concurrent Cursor concept to compensate these shortcomings. In addition, we showcase a reference implementation, the Concurrent Augmented Transition Network (cATN), that validates the concept’s feasibility in a series of proof of concept demonstrations as well as through a comparative benchmark. The cATN fulfills all identified requirements and fills the lack amongst previous solutions. It supports the rapid prototyping of multimodal interfaces by means of five concrete traits: Its declarative nature, the recursiveness of the underlying transition network, the network abstraction constructs of its description language, the utilized semantic queries, and an abstraction layer for lexical information. Our reference implementation was and is used in various student projects, theses, as well as master-level courses. It is openly available and showcases that non-experts can effectively implement multimodal interfaces, even for non-trivial applications in mixed and virtual reality. KW - multimodal fusion KW - multimodal interface KW - semantic fusion KW - procedural fusion methods KW - natural interfaces KW - human-computer interaction Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-197573 SN - 2414-4088 VL - 2 IS - 4 ER - TY - JOUR A1 - Wolf, Erik A1 - Döllinger, Nina A1 - Mal, David A1 - Wenninger, Stephan A1 - Bartl, Andrea A1 - Botsch, Mario A1 - Latoschik, Marc Erich A1 - Wienrich, Carolin T1 - Does distance matter? Embodiment and perception of personalized avatars in relation to the self-observation distance in virtual reality JF - Frontiers in Virtual Reality N2 - Virtual reality applications employing avatar embodiment typically use virtual mirrors to allow users to perceive their digital selves not only from a first-person but also from a holistic third-person perspective. However, due to distance-related biases such as the distance compression effect or a reduced relative rendering resolution, the self-observation distance (SOD) between the user and the virtual mirror might influence how users perceive their embodied avatar. Our article systematically investigates the effects of a short (1 m), middle (2.5 m), and far (4 m) SOD between users and mirror on the perception of their personalized and self-embodied avatars. The avatars were photorealistic reconstructed using state-of-the-art photogrammetric methods. Thirty participants repeatedly faced their real-time animated self-embodied avatars in each of the three SOD conditions, where they were repeatedly altered in their body weight, and participants rated the 1) sense of embodiment, 2) body weight perception, and 3) affective appraisal towards their avatar. We found that the different SODs are unlikely to influence any of our measures except for the perceived body weight estimation difficulty. Here, the participants perceived the difficulty significantly higher for the farthest SOD. We further found that the participants’ self-esteem significantly impacted their ability to modify their avatar’s body weight to their current body weight and that it positively correlated with the perceived attractiveness of the avatar. Additionally, the participants’ concerns about their body shape affected how eerie they perceived their avatars. The participants’ self-esteem and concerns about their body shape influenced the perceived body weight estimation difficulty. We conclude that the virtual mirror in embodiment scenarios can be freely placed and varied at a distance of one to four meters from the user without expecting major effects on the perception of the avatar. KW - virtual human KW - virtual body ownership KW - agency KW - body image distortion KW - body weight perception KW - body weight modification KW - affective appraisal KW - distance compression Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-299415 SN - 2673-4192 VL - 3 ER - TY - JOUR A1 - Winter, Carla A1 - Kern, Florian A1 - Gall, Dominik A1 - Latoschik, Marc Erich A1 - Pauli, Paul A1 - Käthner, Ivo T1 - Immersive virtual reality during gait rehabilitation increases walking speed and motivation: a usability evaluation with healthy participants and individuals with multiple sclerosis and stroke JF - Journal of Neuroengineering and Rehabilitation N2 - Background: The rehabilitation of gait disorders in patients with multiple sclerosis (MS) and stroke is often based on conventional treadmill training. Virtual reality (VR)-based treadmill training can increase motivation and improve therapy outcomes. The present study evaluated an immersive virtual reality application (using a head-mounted display, HMD) for gait rehabilitation with patients to (1) demonstrate its feasibility and acceptance and to (2) compare its short-term effects to a semi-immersive presentation (using a monitor) and a conventional treadmill training without VR to assess the usability of both systems and estimate the effects on walking speed and motivation. Methods: In a within-subjects study design, 36 healthy participants and 14 persons with MS or stroke participated in each of the three experimental conditions (VR via HMD, VR via monitor, treadmill training without VR). Results: For both groups, the walking speed in the HMD condition was higher than in treadmill training without VR and in the monitor condition. Healthy participants reported a higher motivation after the HMD condition as compared with the other conditions. Importantly, no side effects in the sense of simulator sickness occurred and usability ratings were high. No increases in heart rate were observed following the VR conditions. Presence ratings were higher for the HMD condition compared with the monitor condition for both user groups. Most of the healthy study participants (89%) and patients (71%) preferred the HMD-based training among the three conditions and most patients could imagine using it more frequently. Conclusions For the first time, the present study evaluated the usability of an immersive VR system for gait rehabilitation in a direct comparison with a semi-immersive system and a conventional training without VR with healthy participants and patients. The study demonstrated the feasibility of combining a treadmill training with immersive VR. Due to its high usability and low side effects, it might be particularly suited for patients to improve training motivation and training outcome e. g. the walking speed compared with treadmill training using no or only semi-immersive VR. Immersive VR systems still require specific technical setup procedures. This should be taken into account for specific clinical use-cases during a cost-benefit assessment. KW - rehabilitation KW - gait disorder KW - virtual reality KW - multiple sclerosis KW - stroke KW - head-mounted display KW - motivation Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-258698 SN - 1743-0003 VL - 18 IS - 1 ER - TY - JOUR A1 - Wienrich, Carolin A1 - Latoschik, Marc Erich T1 - eXtended Artificial Intelligence: New Prospects of Human-AI Interaction Research JF - Frontiers in Virtual Reality N2 - Artificial Intelligence (AI) covers a broad spectrum of computational problems and use cases. Many of those implicate profound and sometimes intricate questions of how humans interact or should interact with AIs. Moreover, many users or future users do have abstract ideas of what AI is, significantly depending on the specific embodiment of AI applications. Human-centered-design approaches would suggest evaluating the impact of different embodiments on human perception of and interaction with AI. An approach that is difficult to realize due to the sheer complexity of application fields and embodiments in reality. However, here XR opens new possibilities to research human-AI interactions. The article’s contribution is twofold: First, it provides a theoretical treatment and model of human-AI interaction based on an XR-AI continuum as a framework for and a perspective of different approaches of XR-AI combinations. It motivates XR-AI combinations as a method to learn about the effects of prospective human-AI interfaces and shows why the combination of XR and AI fruitfully contributes to a valid and systematic investigation of human-AI interactions and interfaces. Second, the article provides two exemplary experiments investigating the aforementioned approach for two distinct AI-systems. The first experiment reveals an interesting gender effect in human-robot interaction, while the second experiment reveals an Eliza effect of a recommender system. Here the article introduces two paradigmatic implementations of the proposed XR testbed for human-AI interactions and interfaces and shows how a valid and systematic investigation can be conducted. In sum, the article opens new perspectives on how XR benefits human-centered AI design and development. KW - human-artificial intelligence interface KW - human-artificial intelligence interaction KW - XR-artificial intelligence continuum KW - XR-artificial intelligence combination KW - research methods KW - human-centered, human-robot KW - recommender system Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-260296 VL - 2 ER - TY - JOUR A1 - Unruh, Fabian A1 - Landeck, Maximilian A1 - Oberdörfer, Sebastian A1 - Lugrin, Jean-Luc A1 - Latoschik, Marc Erich T1 - The Influence of Avatar Embodiment on Time Perception - Towards VR for Time-Based Therapy JF - Frontiers in Virtual Reality N2 - Psycho-pathological conditions, such as depression or schizophrenia, are often accompanied by a distorted perception of time. People suffering from this conditions often report that the passage of time slows down considerably and that they are “stuck in time.” Virtual Reality (VR) could potentially help to diagnose and maybe treat such mental conditions. However, the conditions in which a VR simulation could correctly diagnose a time perception deviation are still unknown. In this paper, we present an experiment investigating the difference in time experience with and without a virtual body in VR, also known as avatar. The process of substituting a person’s body with a virtual body is called avatar embodiment. Numerous studies demonstrated interesting perceptual, emotional, behavioral, and psychological effects caused by avatar embodiment. However, the relations between time perception and avatar embodiment are still unclear. Whether or not the presence or absence of an avatar is already influencing time perception is still open to question. Therefore, we conducted a between-subjects design with and without avatar embodiment as well as a real condition (avatar vs. no-avatar vs. real). A group of 105 healthy subjects had to wait for seven and a half minutes in a room without any distractors (e.g., no window, magazine, people, decoration) or time indicators (e.g., clocks, sunlight). The virtual environment replicates the real physical environment. Participants were unaware that they will be asked to estimate their waiting time duration as well as describing their experience of the passage of time at a later stage. Our main finding shows that the presence of an avatar is leading to a significantly faster perceived passage of time. It seems to be promising to integrate avatar embodiment in future VR time-based therapy applications as they potentially could modulate a user’s perception of the passage of time. We also found no significant difference in time perception between the real and the VR conditions (avatar, no-avatar), but further research is needed to better understand this outcome. KW - virtual reality KW - time perception KW - avatar embodiment KW - immersion KW - human computer interaction (HCI) Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-259076 VL - 2 ER - TY - JOUR A1 - Steinhaeusser, Sophia C. A1 - Oberdörfer, Sebastian A1 - von Mammen, Sebastian A1 - Latoschik, Marc Erich A1 - Lugrin, Birgit T1 - Joyful adventures and frightening places – designing emotion-inducing virtual environments JF - Frontiers in Virtual Reality N2 - Virtual environments (VEs) can evoke and support emotions, as experienced when playing emotionally arousing games. We theoretically approach the design of fear and joy evoking VEs based on a literature review of empirical studies on virtual and real environments as well as video games’ reviews and content analyses. We define the design space and identify central design elements that evoke specific positive and negative emotions. Based on that, we derive and present guidelines for emotion-inducing VE design with respect to design themes, colors and textures, and lighting configurations. To validate our guidelines in two user studies, we 1) expose participants to 360° videos of VEs designed following the individual guidelines and 2) immerse them in a neutral, positive and negative emotion-inducing VEs combining all respective guidelines in Virtual Reality. The results support our theoretically derived guidelines by revealing significant differences in terms of fear and joy induction. KW - virtual reality KW - virtual environments KW - immersion KW - emotions KW - design Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-284831 SN - 2673-4192 VL - 3 ER - TY - JOUR A1 - Stauffert, Jan-Philipp A1 - Niebling, Florian A1 - Latoschik, Marc Erich T1 - Latency and Cybersickness: Impact, Causes, and Measures. A Review JF - Frontiers in Virtual Reality N2 - Latency is a key characteristic inherent to any computer system. Motion-to-Photon (MTP) latency describes the time between the movement of a tracked object and its corresponding movement rendered and depicted by computer-generated images on a graphical output screen. High MTP latency can cause a loss of performance in interactive graphics applications and, even worse, can provoke cybersickness in Virtual Reality (VR) applications. Here, cybersickness can degrade VR experiences or may render the experiences completely unusable. It can confound research findings of an otherwise sound experiment. Latency as a contributing factor to cybersickness needs to be properly understood. Its effects need to be analyzed, its sources need to be identified, good measurement methods need to be developed, and proper counter measures need to be developed in order to reduce potentially harmful impacts of latency on the usability and safety of VR systems. Research shows that latency can exhibit intricate timing patterns with various spiking and periodic behavior. These timing behaviors may vary, yet most are found to provoke cybersickness. Overall, latency can differ drastically between different systems interfering with generalization of measurement results. This review article describes the causes and effects of latency with regard to cybersickness. We report on different existing approaches to measure and report latency. Hence, the article provides readers with the knowledge to understand and report latency for their own applications, evaluations, and experiments. It should also help to measure, identify, and finally control and counteract latency and hence gain confidence into the soundness of empirical data collected by VR exposures. Low latency increases the usability and safety of VR systems. KW - virtual reality KW - latency KW - cybersickness KW - jitter KW - simulator sickness Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-236133 VL - 1 ER - TY - JOUR A1 - Rack, Christian A1 - Fernando, Tamara A1 - Yalcin, Murat A1 - Hotho, Andreas A1 - Latoschik, Marc Erich T1 - Who is Alyx? A new behavioral biometric dataset for user identification in XR JF - Frontiers in Virtual Reality N2 - Introduction: This paper addresses the need for reliable user identification in Extended Reality (XR), focusing on the scarcity of public datasets in this area. Methods: We present a new dataset collected from 71 users who played the game “Half-Life: Alyx” on an HTC Vive Pro for 45 min across two separate sessions. The dataset includes motion and eye-tracking data, along with physiological data from a subset of 31 users. Benchmark performance is established using two state-of-the-art deep learning architectures, Convolutional Neural Networks (CNN) and Gated Recurrent Units (GRU). Results: The best model achieved a mean accuracy of 95% for user identification within 2 min when trained on the first session and tested on the second. Discussion: The dataset is freely available and serves as a resource for future research in XR user identification, thereby addressing a significant gap in the field. Its release aims to facilitate advancements in user identification methods and promote reproducibility in XR research. KW - dataset KW - behaviometric KW - deep learning KW - physiological dataset KW - user identification Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-353979 SN - 2673-4192 VL - 4 ER - TY - JOUR A1 - Palmisano, Chiara A1 - Kullmann, Peter A1 - Hanafi, Ibrahem A1 - Verrecchia, Marta A1 - Latoschik, Marc Erich A1 - Canessa, Andrea A1 - Fischbach, Martin A1 - Isaias, Ioannis Ugo T1 - A fully-immersive virtual reality setup to study gait modulation JF - Frontiers in Human Neuroscience N2 - Objective: Gait adaptation to environmental challenges is fundamental for independent and safe community ambulation. The possibility of precisely studying gait modulation using standardized protocols of gait analysis closely resembling everyday life scenarios is still an unmet need. Methods: We have developed a fully-immersive virtual reality (VR) environment where subjects have to adjust their walking pattern to avoid collision with a virtual agent (VA) crossing their gait trajectory. We collected kinematic data of 12 healthy young subjects walking in real world (RW) and in the VR environment, both with (VR/A+) and without (VR/A-) the VA perturbation. The VR environment closely resembled the RW scenario of the gait laboratory. To ensure standardization of the obstacle presentation the starting time speed and trajectory of the VA were defined using the kinematics of the participant as detected online during each walking trial. Results: We did not observe kinematic differences between walking in RW and VR/A-, suggesting that our VR environment per se might not induce significant changes in the locomotor pattern. When facing the VA all subjects consistently reduced stride length and velocity while increasing stride duration. Trunk inclination and mediolateral trajectory deviation also facilitated avoidance of the obstacle. Conclusions: This proof-of-concept study shows that our VR/A+ paradigm effectively induced a timely gait modulation in a standardized immersive and realistic scenario. This protocol could be a powerful research tool to study gait modulation and its derangements in relation to aging and clinical conditions. KW - gait modulation KW - virtual reality KW - obstacle avoidance KW - gait analysis KW - kinematics Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-267099 SN - 1662-5161 VL - 16 ER - TY - JOUR A1 - Oberdörfer, Sebastian A1 - Schraudt, David A1 - Latoschik, Marc Erich T1 - Embodied gambling — investigating the influence of level of embodiment, avatar appearance, and virtual environment design on an online VR slot machine JF - Frontiers in Virtual Reality N2 - Slot machines are one of the most played games by players suffering from gambling disorder. New technologies like immersive Virtual Reality (VR) offer more possibilities to exploit erroneous beliefs in the context of gambling. Recent research indicates a higher risk potential when playing a slot machine in VR than on desktop. To continue this investigation, we evaluate the effects of providing different degrees of embodiment, i.e., minimal and full embodiment. The avatars used for the full embodiment further differ in their appearance, i.e., they elicit a high or a low socio-economic status. The virtual environment (VE) design can cause a potential influence on the overall gambling behavior. Thus, we also embed the slot machine in two different VEs that differ in their emotional design: a colorful underwater playground environment and a virtual counterpart of our lab. These design considerations resulted in four different versions of the same VR slot machine: 1) full embodiment with high socio-economic status, 2) full embodiment with low socio-economic status, 3) minimal embodiment playground VE, and 4) minimal embodiment laboratory VE. Both full embodiment versions also used the playground VE. We determine the risk potential by logging gambling frequency as well as stake size, and measuring harm-inducing factors, i.e., dissociation, urge to gamble, dark flow, and illusion of control, using questionnaires. Following a between groups experimental design, 82 participants played for 20 game rounds one of the four versions. We recruited our sample from the students enrolled at the University of Würzburg. Our safety protocol ensured that only participants without any recent gambling activity took part in the experiment. In this comparative user study, we found no effect of the embodiment nor VE design on neither the gambling frequency, stake sizes, nor risk potential. However, our results provide further support for the hypothesis of the higher visual angle on gambling stimuli and hence the increased emotional response being the true cause for the higher risk potential. KW - virtual reality KW - virtual environments KW - immersion KW - gambling KW - risks KW - embodiment KW - avatars Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-284662 SN - 2673-4192 VL - 3 ER -