Refine
Has Fulltext
- yes (159) (remove)
Is part of the Bibliography
- yes (159)
Year of publication
Document Type
- Doctoral Thesis (159) (remove)
Keywords
- Kognition (14)
- Aufmerksamkeit (11)
- EEG (10)
- Psychologie (10)
- Virtuelle Realität (10)
- Motivation (9)
- Angst (8)
- Gefühl (7)
- Informationsverarbeitung (7)
- Verkehrspsychologie (7)
Institute
- Institut für Psychologie (159) (remove)
Sonstige beteiligte Institutionen
- Adam Opel AG (1)
- BMBF (1)
- Evangelisches Studienwerk e.V. (1)
- Forschungsverbund ForChange des Bayrischen Kultusministeriums (1)
- IFT Institut für Therapieforschung München (1)
- Klinik für Psychiatrie und Psychotherapie, Universität Würzburg (1)
- Opel Automobile GmbH (1)
- Technische Universität Dresden (1)
- Würzburger Institut für Verkehrswissenschaften (WIVW GmbH) (1)
The “active self” approach suggests that any object we manipulate voluntarily and foreseeably becomes part of our “self” in the sense that we feel control over this object (sense of agency) and experience it as belonging to our own body (sense of ownership). While there is considerable evidence that we can indeed experience both a sense of agency and a sense of ownership over a broad variety of objects when we control these through our actions, the approach has also been criticized for exaggerating the flexibility of the human self. In this thesis, I investigate the influence that the relationship between the body movements controlling an object and the movements of the object itself has on the process of integrating an object into the self. I demonstrate that fully controlling an object is not sufficient for it to be integrated into the self since both explicit and implicit measures of the sense of agency and the sense of ownership indicate less or no integration when body movements are transformed into inverted object movements. Furthermore, I show that such inversions lead to the downregulation of sensory signals either from the body or from the controlled object in order to deal with the conflicting multisensory information when performing such actions. I argue that this downregulation is the underlying factor behind the diminished or eliminated integration of inverted body and object movements and I discuss further pathways for possible future studies building up on these findings.
Anxiety research is one of the major psychological research domains and looks back on decades of research activity. Traditionally, novel theories and approaches are tested utilizing animal models. One way to study inherent anxiety in rodents is the elevated plus-maze (EPM). The EPM is a plus-shaped platform with two closed, i.e., walled, arms and two open unwalled arms. If given the opportunity to freely explore the apparatus, rodents instinctively avoid the open arms to protect themselves from predators. Hence, they spent less time on open and more time on closed arms, which is behaviorally associated with general anxiety. In the course of the pharmacological validation, it was found that this exploratory pattern can be reversed by anxiolytic substances, e.g., benzodiazepines, or potentiated by anxiogenics. One of the significant advantages of the EPM is that no prior training session is required in contrast to conditioning studies, thus allowing to observe natural behavior. Therefore, together with the economic and uncomplicated setup, the EPM has become a standard preclinical rodent anxiety test over the decades. In order to validate these rodent anxiety tests, there have recently been attempts to retranslate them to humans. A paramount of cross-species validation is not only the simple transferability of these animal tests but also the observation of anxiety behaviors that are evolutionarily conserved across species. Accordingly, it could be possible to conclude various factors associated with the etiology and maintenance of anxiety disorders in humans. So far, convincing translations of the EPM to humans are still lacking. For that reason, the primary aim of this dissertation is to retranslate the EPM throughout three studies and to evaluate cross-species validity critically. Secondly, the undertaken studies are set out to observe ambulatory activity equivalent to rodent EPM behavior, i.e., open arm avoidance. Thirdly, the undertaken studies aimed to assess the extent to which trait anxiety influences human exploratory activity on the platform to associate it with the assumption that rodent EPM-behavior is a reflection of general anxiety. Finally, virtual reality (VR) was the method of choice to maintain the economic advantage and adjust the EPM size to humans. Study 1 (N = 30) was set up to directly transfer the rodent EPM regarding test design and experimental procedure using a Computer Automatic Virtual Environment (CAVE). The results revealed that humans unlike rodents display a general open arms approach during free exploration. However, open arm avoidance was associated with high trait anxiety and acrophobia (fear of height), which was initially assessed as a control variable due to the virtual platform height. Regression analyses and subjective anxiety ratings hinted at a more significant influence of acrophobia on open arm avoidance. In addition, it was assumed that the open arms approach might have resulted from claustrophobic tendencies experienced in the closed arms due to the high walls. Study 2 (N = 61) sought to differentiate the influence of trait anxiety and acrophobia and adapt the virtual EPM to humans. Therefore, parts of the platform held a semi-transparent grid-floor texture, and the wall height on the closed arms was reduced to standard handrail level. Moreover, participants were priorly screened to exclude clinically significant levels of acrophobia, claustrophobia, and agoraphobia. The data on general exploratory activity showed no arm preference. Regression analyses confirmed that acrophobia is related to open arm avoidance, corroborating the finding of Study 1. Surprisingly, for trait anxiety, the result of Study 1 could not be replicated. Instead, for trait anxiety, no significant effect was found indicating that predominantly fear of heights shapes human EPM behavior even on a subclinical stage. In Study 3 (N = 57), the EPM was embedded into a city setting to 1) create a more natural human environment and 2) eliminate height. Furthermore, a head-mounted display was utilized for VR presentation, and arousal ratings were introduced. Participants were screened for high and low levels of trait anxiety and agoraphobia, and claustrophobia. Replicating the findings of Study 2, no difference in open and closed arm activity was observed, and no effect was found in relationship with trait anxiety. The data on anxiety ratings and claustrophobia suggest a positive correlation indicating that in this city EPM, claustrophobic tendencies might play a role in closed arm avoidance. In summary, this thesis added valuable insights into the retranslation of a well-established standard anxiety test used in rodents. However, it also majorly challenges current findings on the cross-species validity of the EPM. Various explanatory models for the results are critically discussed and associated with clinical implications concerning future research.
Die Rehabilitation von Gangstörungen bei Patienten mit MS und Schlaganfall erfolgt häufig mithilfe eines konventionellen Laufbandtrainings. Einige Studien haben bereits gezeigt, dass durch eine Erweiterung dieses Trainings um eine virtuelle Realität die Motivation der Patienten gesteigert und die Therapieergebnisse verbessert werden können.
In der vorliegenden Studie wurde eine immersive VR-Anwendung (unter Verwendung eines HMD) für die Gangrehabilitation von Patienten evaluiert. Hierbei wurden ihre Anwendbarkeit und Akzeptanz geprüft sowie ihre Kurzzeiteffekte mit einer semi-immersiven Präsentation (unter Verwendung eines Monitors) und mit einem konventionellen Laufbandtraining ohne VR verglichen. Der Fokus lag insbesondere auf der Untersuchung der Anwendbarkeit beider Systeme und der Auswirkungen auf die Laufgeschwindigkeit und Motivation der Benutzer.
Im Rahmen einer Studie mit Innersubjekt-Design nahmen zunächst 36 gesunde Teilnehmer und anschließend 14 Patienten mit MS oder Schlaganfall an drei experimentellen Bedingungen (VR über HMD, VR über Monitor, Laufbandtraining ohne VR) teil.
Sowohl in der Studie mit gesunden Teilnehmern als auch in der Patientenstudie zeigte sich in der HMD-Bedingung eine höhere Laufgeschwindigkeit als beim Laufbandtraining ohne VR und in der Monitor-Bedingung. Die gesunden Studienteilnehmer berichteten über eine höhere Motivation nach der HMD-Bedingung als nach den anderen Bedingungen. Es traten in beiden Gruppen keine Nebenwirkungen im Sinne einer Simulator Sickness auf und es wurden auch keine Erhöhungen der Herzfrequenzen nach den VR-Bedingungen detektiert. Die Bewertungen des Präsenzerlebens waren in beiden Gruppen in der HMD-Bedingung höher als in der Monitor-Bedingung. Beide VR-Bedingungen erhielten hohe Bewertungen für die Benutzerfreundlichkeit. Die meisten der gesunden Teilnehmer (89 %) und Patienten (71 %) präferierten das HMD-basierte Laufbandtraining unter den drei Trainingsformen und die meisten Patienten könnten sich vorstellen, es häufiger zu nutzen.
Mit der vorliegenden Studie wurde eine strukturierte Evaluation der Anwendbarkeit eines immersiven VR-Systems für die Gangrehabilitation geprüft und dieses erstmals in den direkten Vergleich zu einem semi-immersiven System und einem konventionellen Training ohne VR gesetzt. Die Studie bestätigte die Praktikabilität der Kombination eines Laufbandtrainings mit immersiver VR. Aufgrund ihrer hohen Benutzerfreundlichkeit und der geringen Nebenwirkungen scheint diese Trainingsform besonders für Patienten geeignet zu sein, um deren Trainingsmotivation und Trainingserfolge, wie z. B. die Laufgeschwindigkeit, zu steigern. Da immersive VR-Systeme allerdings nach wie vor spezifische technische Installationsprozeduren erfordern, sollte für die spezifische klinische Anwendung eine Kosten-Nutzen-Bewertung erfolgen.
During natural behavior, cognitive processes constantly coincide with body movements such as head or eye movements or blinks. However, during experimental investigations of cognitive processes, movements are often highly restricted which is rather unnatural. In order to improve our understanding of natural behavior, this thesis investigates the interaction between cognition and movements by focusing on spontaneous blinks, which naturally interact with other body movements.
Spontaneous blinks are inevitably connected to vision as they shut out incoming visual information. Both sensory-based and cognitive factors, for example, stimulus occurrence and evaluation, were reported to influence blink behavior. Our first study investigated if such influences are comparable for visual and non-visual input. The chosen experimental design allowed dissociating sensory-driven and cognitive influences, which then could be compared between the visual and auditory domain. Our results show that blinks are more strongly modulated during passive observation of visual input compared to auditory input. This modulation is however enhanced for both input modalities by an increased attentional demand. In addition, the cognitively defined meaning of a stimulus changes blink latency independent of the sensory domain. Overall, our findings show that spontaneous blinks and cognitive processes are linked beyond vision. Moreover, the underlying cognitive processes that influence blinks are largely the same across different sensory input indicating that blinks are profoundly integrated into our system.
When investigating natural behavior, it is important to consider that movements rarely occur in isolation, but are executed side by side. As these movements interact and have a link to cognitive processes, the complexity of our system increases. In order to take this complexity into account, the second part of the experimental research focused on movement interactions, more specifically on the interactions between blinks, pupil size and speaking. Our results reveal that speech-related motor activity increases blink rate and pupil size as well as modulates blink timing. This is in line with previous research that described a relation between different body and eye movements. Importantly, each bodily-induced change in eye movements affects visual information intake. Therefore, different movements can be tightly linked to perceptual processes through complex interactions.
Altogether, the work of this thesis provides rich evidence that movements and cognitive processes are deeply intertwined. Therefore, movements should be seen as an integral part of our system. Taking the relevance of movements and their interactions into account during experimental investigations is necessary in order to reveal a more realistic and complete picture of human natural behavior.
Gazes are of central relevance for people. They are crucial for navigating the world and communicating with others. Nevertheless, research in recent years shows that many findings from experimental research on gaze behavior cannot be transferred from the laboratory to everyday behavior. For example, the frequency with which conspecifics are looked at is considerably higher in experimental contexts than what can be observed in daily behavior. In short: findings from laboratories cannot be generalized into general statements. This thesis is dedicated to this matter. The dissertation describes and documents the current state of research on social attention through a literature review, including a meta-analysis on the /gaze cueing/ paradigm and an empirical study on the robustness of gaze following behavior. In addition, virtual reality was used in one of the first studies in this research field. Virtual reality has the potential to significantly improve the transferability of experimental laboratory studies to everyday behavior. This is because the technology enables a high degree of experimental control in naturalistic research designs. As such, it has the potential to transform empirical research in the same way that the introduction of computers to psychological research did some 50 years ago. The general literature review on social attention is extended to the classic /gaze cueing/ paradigm through a systematic review of publications and a meta-analytic evaluation (Study 1). The cumulative evidence supported the findings of primary studies: Covert spatial attention is directed by faces. However, the experimental factors included do not explain the surprisingly large variance in the published results. Thus, there seem to be further, not well-understood variables influencing these social processes. Moreover, classic /gaze cueing/ studies have limited ecological validity. This is discussed as a central reason for the lack of generalisability. Ecological validity describes the correspondence between experimental factors and realistic situations. A stimulus or an experimental design can have high and low ecological validity on different dimensions and have different influences on behavior. Empirical research on gaze following behavior showed that the /gaze cueing/ effect also occurs with contextually embedded stimuli (Study 2). The contextual integration of the directional cue contrasted classical /gaze cueing/ studies, which usually show heads in isolation. The research results can thus be transferred /within/ laboratory studies to higher ecologically valid research paradigms. However, research shows that the lack of ecological validity in experimental designs significantly limits the transferability of experimental findings to complex situations /outside/ the laboratory. This seems to be particularly the case when social interactions and norms are investigated. However, ecological validity is also often limited in these studies for other factors, such as contextual embedding /of participants/, free exploration behavior (and, thus, attentional control), or multimodality. In a first study, such high ecological validity was achieved for these factors with virtual reality, which could not be achieved in the laboratory so far (Study 3). Notably, the observed fixation patterns showed differences even under /most similar/ conditions in the laboratory and natural environments. Interestingly, these were similar to findings also derived from comparisons of eye movement in the laboratory and field investigations. These findings, which previously came from hardly comparable groups, were thus confirmed by the present Study 3 (which did not have this limitation). Overall, /virtual reality/ is a new technical approach to contemporary social attention research that pushes the boundaries of previous experimental research. The traditional trade-off between ecological validity and experimental control thus becomes obsolete, and laboratory studies can closely inherit an excellent approximation of reality. Finally, the present work describes and discusses the possibilities of this technology and its practical implementation. Within this context, the extent to which this development can still guarantee a constructive classification of different laboratory tests in the future is examined.
This dissertation focuses on the construct and criterion validity of integrity tests and aims to enhance both. To accomplish this goal, three approaches were adopted: First, an overview and systematic comparison of integrity tests was conducted with reference to the construction and application of the tests. Second, the nomological network of integrity tests was expanded with reference to honesty-humility and organizational citizenship behavior at their factor and facet level. Third, two promising methods to reduce faking on integrity tests were tested: the double rating method (Hui, 2001) and the indirect questioning technique. In line with previous research, the results of the overview and comparison of integrity measures confirmed that integrity tests are multidimensional and heterogenous. A clear definition of integrity is urgently needed. The personality trait of honesty-humility and its facets of fairness, and modesty revealed the most significant relationships to integrity. Moreover, organizational citizenship behavior and its facets of altruism, conscientiousness, and sportsmanship were found to significantly relate to integrity. Furthermore, integrity tests were able not only to predict organizational citizenship behavior but also to incrementally predict job performance and organizational citizenship behavior beyond the factor and facet level of the personality traits of conscientiousness and honesty-humility. In contrast to the indirect questioning technique, the double rating method, which includes an other rating and a self rating, was shown to be able to significantly reduce faking on integrity tests in an anonymous survey setting. This dissertation makes an important contribution to better explain the construct and nomological network of integrity, provide a more detailed view on integrity tests and their protection against faking, and expand the predictive and incremental validity of these tests. The implications for future research and practice are further discussed.
Experimental investigation of the effect of distal stress induction on threat conditioning in humans
(2022)
Stress constitutes a major risk factor for the development of psychiatric disorders, such as PTSD and anxiety disorders, by shifting the brain into a state of sensitization and makes it more vulnerable when being exposed to further aversive events. This was experimentally in-vestigated in rodents by examining the effect of a distal stress induction on threat conditioning, where stress impaired extinction learning and caused spontaneous recovery. However, this effect has never been experimentally investigated in humans, so far. Thus, the aim of this dissertation was to investigate the effect of distal stress on threat conditioning in humans.
Therefore, two subsequent studies were conducted. For both studies, the threat conditioning paradigm comprised threat acquisition, extinction learning, and re-extinction. In the threat acquisition phase, two geometrical shapes were used as conditioned stimulus (CS), from which one (CS+) was paired with a painful electric stimulus (unconditioned stimulus, US), but not the other one (CS-). During extinction learning 24 h later and re-extinction seventeen days later, CSs were again presented but without any US delivery.
In Study 1, 69 participants underwent either a stress (socially evaluated cold pressor test; SECPT) or sham protocol 10 days prior to threat conditioning. Furthermore, context effects were examined by placing the stress protocol in the same context (context-A stress, and sham group) or a different context (context-B stress group) than conditioning. Results revealed that the context-A, but not context-B, stress group displayed impaired safety learning (i.e. potenti-ation towards CS-) for startle response during threat acquisition. Moreover, the same stress group showed impaired threat extinction, evident in sustained CS discrimination in valence and arousal ratings during extinction learning, and memory recall. In sum, distal stress on the one hand impaired safety learning during threat conditioning on a level of startle response. On the other hand, stress impaired threat extinction on a level of ratings. Noteworthy, the effect of distal stress was only found when the stressor was placed in the same context as later threat learning. Hence, suggesting that the combination of stressor and stressor-associated context exerted the effect on threat extinction.
In Study 2, it was examined if distal stress induction could also have an impact on threat and extinction processes without the necessity of context association. Therefore, the same stress (n = 45) or sham protocol (n = 44) as in Study 1 was conducted in a different context than and 24 h prior to a threat conditioning paradigm. Similar to Study 1, weakened extinction learning was found in fear ratings for the stress (vs. sham) group, which was indicated by persistent CS+/CS- differentiation after the first block of extinction trials. Alterations in safety learning towards the CS- during threat acquisition were only supported by significant correlations between stress measures on the stress day and conditioned startle response of the CS- during acquisition.
Taken together, in two subsequent studies this dissertation provided first evidence of impaired threat extinction after distal stress induction in humans. Furthermore, impairments in safety learning, as can be observed in PTSD, were additionally demonstrated. Interestingly, the effects were boosted and more profound when associating the stressor to the later learning context. These results have clinical implications as they can be translated to the notion that prior stress exposure makes an individual more vulnerable for later aversive events.
The incidence of cancer cases is rising steadily, while improved early detection and new cancer-specific therapies are reducing the mortality rate. In addition to curing cancer or prolonging life, increasing the quality of life is thus an important goal of oncology, which is why the burdens of cancer and treatment are becoming more important. A common side effect of cancer and its therapy is cancer-related fatigue, a tiredness that manifests itself on physical, emotional and cognitive levels and is not in proportion to previous physical efforts. Since the etiology of fatigue has not yet been fully clarified, symptom-oriented therapy is preferable to cause-specific therapy. In addition to activity management, sleep hygiene, and cognitive behavioral therapy, mind-body interventions such as yoga are recommended for reducing fatigue.
Previous studies with small sample sizes were able to examine the efficacy of yoga regarding fatigue predominantly in patients with breast cancer. Long-term effects of yoga have rarely been studied and there have been no attempts to increase long-term effects through interventions such as reminder e-mails. This dissertation takes a closer look at these mentioned aspects of the study sample and long-term effects. An 8-week randomized controlled yoga intervention was conducted, including patients with different cancer types reporting mild to severe fatigue. Following the 8-week yoga therapy, a randomized group of participants received weekly reminder e-mails for 6 months for regular yoga practice, whereas the control group did not receive reminder e-mails.
The first paper is a protocol article, which addresses the design and planned implementation of the research project this dissertation is based upon. This serves to ensure better replicability and comparability with other yoga studies. Due to a very low consent rate of patients in the pilot phase, it was necessary to deviate from the protocol article in the actual implementation and the planned inclusion criterion of fatigue >5 was reduced to fatigue >1.
The second paper examines the efficacy of the eight-week yoga intervention. Patients in the intervention group who participated in the yoga classes seven times or more showed a significantly greater reduction in general and physical fatigue than those who participated less often. The efficacy of yoga was related to the number of attended yoga sessions. Women with breast cancer who participated in yoga reported greater reductions in fatigue than women with other cancer types. There was also an improvement for depression and quality of life after eight weeks of yoga therapy compared to no yoga therapy. These results imply that yoga is helpful in reducing depression and cancer-related fatigue, especially in terms of physical aspects and improving quality of life.
The third paper focuses on the efficacy of reminder e-mails in terms of fatigue and practice frequency. Patients who received reminder e-mails reported greater reductions in general and emotional fatigue, as well as significant increases in practice frequency, compared to patients who did not receive reminder e-mails. Compared to fatigue scores before yoga, significantly lower fatigue and depression scores and higher quality of life were reported after yoga therapy and at follow-up six months later. Weekly e-mail reminders after yoga therapy may have positive effects on general and emotional fatigue and help cancer patients with fatigue establish a regular yoga practice at home. However, higher practice frequency did not lead to higher improvement in physical fatigue as found in Paper 2. This may indicate other factors that influence the efficacy of yoga practice on physical fatigue, such as mindfulness or side effects of therapy.
This research project provides insight into the efficacy of yoga therapy for oncology patients with fatigue. It is important that such interventions be offered early, while fatigue symptoms are not too severe. Regular guided yoga practice can reduce physical fatigue, but subsequent yoga practice at home does not further reduce physical fatigue. Reminder emails after completed yoga therapy could only reduce patients' emotional fatigue. It may be that physical fatigue was reduced as much as possible by the previous yoga therapy and that there was a floor effect, or it may be that reminder emails are not suitable as an intervention to reduce physical fatigue at all. Further research is needed to examine the mechanisms of the different interventions in more detail and to find appropriate interventions that reduce all levels of fatigue equally.
Humans use their eyes not only as visual input devices to perceive the environment, but also as an action tool in order to generate intended effects in their environment. For instance, glances are used to direct someone else's attention to a place of interest, indicating that gaze control is an important part of social communication. Previous research on gaze control in a social context mainly focused on the gaze recipient by asking how humans respond to perceived gaze (gaze cueing). So far, this perspective has hardly considered the actor’s point of view by neglecting to investigate what mental processes are involved when actors decide to perform an eye movement to trigger a gaze response in another person. Furthermore, eye movements are also used to affect the non-social environment, for instance when unlocking the smartphone with the help of the eyes. This and other observations demonstrate the necessity to consider gaze control in contexts other than social communication whilst at the same time focusing on commonalities and differences inherent to the nature of a social (vs. non-social) action context. Thus, the present work explores the cognitive mechanisms that control such goal-oriented eye movements in both social and non-social contexts.
The experiments presented throughout this work are built on pre-established paradigms from both the oculomotor research domain and from basic cognitive psychology. These paradigms are based on the principle of ideomotor action control, which provides an explanatory framework for understanding how goal-oriented, intentional actions come into being. The ideomotor idea suggests that humans acquire associations between their actions and the resulting effects, which can be accessed in a bi-directional manner: Actions can trigger anticipations of their effects, but the anticipated resulting effects can also trigger the associated actions. According to ideomotor theory, action generation involves the mental anticipation of the intended effect (i.e., the action goal) to activate the associated motor pattern. The present experiments involve situations where participants control the gaze of a virtual face via their eye movements. The triggered gaze responses of the virtual face are consistent to the participant’s eye movements, representing visual action effects. Experimental situations are varied with respect to determinants of action-effect learning (e.g., contingency, contiguity, action mode during acquisition) in order to unravel the underlying dynamics of oculomotor control in these situations. In addition to faces, conditions involving changes in non-social objects were included to address the question of whether mechanisms underlying gaze control differ for social versus non-social context situations.
The results of the present work can be summarized into three major findings. 1. My data suggest that humans indeed acquire bi-directional associations between their eye movements and the subsequently perceived gaze response of another person, which in turn affect oculomotor action control via the anticipation of the intended effects. The observed results show for the first time that eye movements in a gaze-interaction scenario are represented in terms of their gaze response in others. This observation is in line with the ideomotor theory of action control. 2. The present series of experiments confirms and extends pioneering results of Huestegge and Kreutzfeldt (2012) with respect to the significant influence of action effects in gaze control. I have shown that the results of Huestegge and Kreutzfeldt (2012) can be replicated across different contexts with different stimulus material given that the perceived action effects were sufficiently salient. 3. Furthermore, I could show that mechanisms of gaze control in a social gaze-interaction context do not appear to be qualitatively different from those in a non-social context.
All in all, the results support recent theoretical claims emphasizing the role of anticipation-based action control in social interaction. Moreover, my results suggest that anticipation-based gaze control in a social context is based on the same general psychological mechanisms as ideomotor gaze control, and thus should be considered as an integral part rather than as a special form of ideomotor gaze control.
This doctoral thesis is part of a research project on the development of the cognitive compre-hension of film at Würzburg University that was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft) between 2013 and 2019 and awarded to Gerhild Nied-ing. That project examined children’s comprehension of narrative text and its development in illustrated versus non-illustrated formats. For this purpose, van Dijk and Kintsch’s (1983) tri-partite model was used, according to which text recipients form text surface and textbase rep-resentations and construct a situation model. In particular, predictions referring to the influ-ence of illustrations on these three levels of text representation were derived from the inte-grated model of text and picture comprehension (ITPC; Schnotz, 2014), which holds that text-picture units are processed on both text-based (descriptive) and picture-based (depictive) paths. Accordingly, illustrations support the construction of a situation model. Moreover, in line with the embodied cognition account (e.g., Barsalou, 1999), it was assumed that the situa-tion model is grounded in perception and action; text recipients mentally simulate the situation addressed in the text through their neural systems related to perception (perceptual simulation) and action (motor resonance). Therefore, the thesis also examines whether perceptual simula-tion takes place during story reception, whether it improves the comprehension of illustrated stories, and whether motor resonance is related to the comprehension of text accompanied by dynamic illustrations. Finally, predictions concerning the development of comprehending illus-trated text were made in line with Springer’s (2001) hypotheses according to which younger children, compared with older children and adults, focus more on illustrations during text comprehension (perceptual boundedness) and use illustrations for the development of cogni-tive skills (perceptual support).
The first research question sought to validate the tripartite model in the context of children’s comprehension of narrative text, so Hypothesis 1 predicted that children yield representations of the text surface, the textbase, and the situation model during text reception. The second research question comprised the assumptions regarding the impact of illustrations on text comprehension. Accordingly, it was expected that illustrations improve the situation model (Hypothesis 2a), especially when they are processed before their corresponding text passages (Hypothesis 2b). Both hypotheses were derived from the ITPC and the assumption that per-ceptual simulation supports the situation model. It was further predicted that dynamic illustra-tions evoke more accurate situation models than static ones (Hypothesis 2c); this followed from the assumption that motor resonance supports the situation model. In line with the ITPC, it was assumed that illustrations impair the textbase (Hypothesis 2d), especially when they are presented after their corresponding text passages (Hypothesis 2e). In accordance with earlier results, it was posited that illustrations have a beneficial effect for the text surface (Hypothesis 2f). The third research question addressed the embodied approach to the situation model. Here, it was assumed that perceptual simulation takes place during text reception (Hypothesis 3a) and that it is more pronounced in illustrated than in non-illustrated text (Hypothesis 3b); the latter hypothesis was related to a necessary premise of the assumption that perceptual sim-ulation improves the comprehension of illustrated text. The fourth research question was relat-ed to perceptual boundedness and perceptual support and predicted age-related differences; younger children were expected to benefit more from illustrations regarding the situation model (Hypothesis 4a) and to simulate vertical object movements in a more pronounced fash-ion (Hypothesis 4b) than older children. In addition, Hypothesis 4c held that perceptual simu-lation is more pronounced in younger children particularly when illustrations are present.
Three experiments were conducted to investigate these hypotheses. Experiment 1 (Seger, Wannagat, & Nieding, submitted).compared the tripartite representations of written text without illustrations, with illustrations presented first, and with illustrations presented after their corresponding sentences. Students between 7 and 13 years old (N = 146) took part. Ex-periment 2 (Seger, Wannagat, & Nieding, 2019) investigated the tripartite representations of auditory text, audiovisual text with static illustrations, and audiovisual text with dynamic il-lustrations among children in the same age range (N = 108). In both experiments, a sentence recognition method similar to that introduced by Schmalhofer and Glavanov (1986) was em-ployed. This method enables the simultaneous measurement of all three text representations. Experiment 3 (Seger, Hauf, & Nieding, 2020) determined the perceptual simulation of vertical object movements during the reception of auditory and audiovisual narrative text among chil-dren between 5 and 11 years old and among adults (N = 190). For this experiment, a picture verification task based on Stanfield and Zwaan’s (2001) paradigm and adapted from Hauf (2016) was used.
The first two experiments confirmed Hypothesis 1, indicating that the tripartite model is appli-cable to the comprehension of auditory and written narrative text among children. A benefi-cial effect of illustrations to the situation model was observed when they were presented syn-chronously with auditory text (Hypotheses 2a), but not when presented asynchronously with written text (Hypothesis 2b), so the ITPC is partly supported on this point. Hypothesis 2c was rejected, indicating that motor resonance does not make an additional contribution to the comprehension of narrative text with dynamic illustrations. Regarding the textbase, a general negative effect of illustrations was not observed (Hypothesis 2d), but a specific negative effect of illustrations that follow their corresponding text passages was seen (Hypothesis 2e); the latter result is also in line with the ITPC. The text surface (Hypothesis 2f) appears to benefit from illustrations in auditory but not written text. The results obtained in Experiment 3 sug-gest that children and adults perceptually simulate vertical object movements (Hypothesis 3a), but there appears to be no difference between auditory and audiovisual text (Hypothesis 3b), so there is no support for a functional relationship between perceptual simulation and the situ-ation model in illustrated text. Hypotheses 4a–4c were investigated in all three experiments and did not receive support in any of them, which indicates that representations of illustrated and non-illustrated narrative text remain stable within the age range examined here.