Refine
Has Fulltext
- yes (5)
Is part of the Bibliography
- yes (5)
Document Type
- Journal article (3)
- Doctoral Thesis (2)
Language
- English (5)
Keywords
- text comprehension (5) (remove)
Institute
In three studies, we investigated, if and how different modes of presentation - written, auditory, audiovisual (auditory combined with pictures) - affect comprehension of semantically identical materials. Children, beginning from the age of 7, and adults were included into the studies. A vast amount of studies have shown that pictures can facilitate text comprehension (e.g. Carney & Levin, 2002).
Other than the majority of these previous studies, we assessed text comprehension with methods that we assume to allow more differentiated insights into the cognitive processes that - according to current theories - underlie text comprehension. Text comprehension involves at least three levels of mental representations (see Kintsch, 1998). Moreover, text comprehension means constructing a locally and globally coherent mental representation of the text content.
Using a sentence recognition task (see Schmalhofer & Glavanov, 1986), we examined whether the memory of the text surface, the text base, and the situation model differs between written, auditory, and audiovisual text presentation in a sample of 103 8- and 10-year-olds and adults (Study I), and between auditory and audiovisual text presentation in a sample of 106 7-, 9-, and 11-year-olds (Study II). Furthermore, we examined with 155 9- and 11-year-olds, whether the ability to draw inferences to establish local and global coherence differs between written, auditory, and audiovisual text presentation. These inferences were indicated by reaction times to words associated with a protagonist's super- (global) or subordinate (local) goal.
Overall, the results of these three studies taken together, indicate that children up to age 11 do not only have better memory of not only the text surface, but also of the situation model when pictures are added to an auditory text. This effect became apparent in comparison with both auditory and written texts. For the adults, in contrast, we did not find an effect of the presentation mode. Furthermore, both 9- and 11-year-olds were better at establishing global coherence at audiovisual compared to auditory text presentation. Written presentation turned out to be superior to auditory presentation in terms of both local and global coherence.
Validation of text information as a general mechanism for detecting inconsistent or false information is an integral part of text comprehension. This study examined how the credibility of the information source affects validation processes. Two experiments investigated combined effects of source credibility and plausibility of information during validation with explicit (ratings) and implicit (reading times) measurements. Participants read short stories with a high-credible versus low-credible person that stated a consistent or inconsistent assertion with general world knowledge. Ratings of plausibility and ratings of source credibility were lower when a credible source stated a world-knowledge inconsistent assertion compared with a low-credible source. Reading times on target sentences and on spillover sentences were slower when a credible source stated an assertion inconsistent with world knowledge compared with a low-credible source, suggesting that source information modulated the validation of implausible information. These results show that source credibility modulates validation and suggest a bidirectional relationship of perceived plausibility and source credibility in the reading process.
In an experiment with 114 children aged 9–12 years, we compared the ability to establish local and global coherence of narrative texts between auditory and audiovisual (auditory text and pictures) presentation. The participants listened to a series of short narrative texts, in each of which a protagonist pursued a goal. Following each text, we collected the response time to a query word that was either associated with a near or a distant causal antecedent of the final sentence. Analysis of these response times indicated that audiovisual presentation has advantages over auditory presentation for accessing information relevant for establishing both local and global coherence, but there are indications that this effect may be slightly more pronounced for global coherence.
According to the tripartite model of text representation (van Dijk & Kintsch, 1983), readers form representations of the text surface and textbase, and construct a situation model. In this study, an experiment was conducted to investigate whether these levels of representation would be affected by adding illustrations to narrative text and whether the order of text and illustrations would make a difference. Students aged between 7 and 13 years (N = 146) read 12 narrative texts, 4 of them with illustrations presented before their corresponding sentences, 4 with illustrations presented after, and 4 without any illustration. A sentence recognition task was used to assess the accuracy for text surface, textbase, and situation model. For the text surface and situation model, neither the presence of illustrations nor the order of text and illustrations influenced accuracy. However, the textbase was negatively affected by illustrations when they followed their corresponding sentences. We suggest that illustrations can initiate model inspection after situation model construction (Schnotz, 2014), a process that can make substantial changes to the textbase representation.
This doctoral thesis is part of a research project on the development of the cognitive compre-hension of film at Würzburg University that was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft) between 2013 and 2019 and awarded to Gerhild Nied-ing. That project examined children’s comprehension of narrative text and its development in illustrated versus non-illustrated formats. For this purpose, van Dijk and Kintsch’s (1983) tri-partite model was used, according to which text recipients form text surface and textbase rep-resentations and construct a situation model. In particular, predictions referring to the influ-ence of illustrations on these three levels of text representation were derived from the inte-grated model of text and picture comprehension (ITPC; Schnotz, 2014), which holds that text-picture units are processed on both text-based (descriptive) and picture-based (depictive) paths. Accordingly, illustrations support the construction of a situation model. Moreover, in line with the embodied cognition account (e.g., Barsalou, 1999), it was assumed that the situa-tion model is grounded in perception and action; text recipients mentally simulate the situation addressed in the text through their neural systems related to perception (perceptual simulation) and action (motor resonance). Therefore, the thesis also examines whether perceptual simula-tion takes place during story reception, whether it improves the comprehension of illustrated stories, and whether motor resonance is related to the comprehension of text accompanied by dynamic illustrations. Finally, predictions concerning the development of comprehending illus-trated text were made in line with Springer’s (2001) hypotheses according to which younger children, compared with older children and adults, focus more on illustrations during text comprehension (perceptual boundedness) and use illustrations for the development of cogni-tive skills (perceptual support).
The first research question sought to validate the tripartite model in the context of children’s comprehension of narrative text, so Hypothesis 1 predicted that children yield representations of the text surface, the textbase, and the situation model during text reception. The second research question comprised the assumptions regarding the impact of illustrations on text comprehension. Accordingly, it was expected that illustrations improve the situation model (Hypothesis 2a), especially when they are processed before their corresponding text passages (Hypothesis 2b). Both hypotheses were derived from the ITPC and the assumption that per-ceptual simulation supports the situation model. It was further predicted that dynamic illustra-tions evoke more accurate situation models than static ones (Hypothesis 2c); this followed from the assumption that motor resonance supports the situation model. In line with the ITPC, it was assumed that illustrations impair the textbase (Hypothesis 2d), especially when they are presented after their corresponding text passages (Hypothesis 2e). In accordance with earlier results, it was posited that illustrations have a beneficial effect for the text surface (Hypothesis 2f). The third research question addressed the embodied approach to the situation model. Here, it was assumed that perceptual simulation takes place during text reception (Hypothesis 3a) and that it is more pronounced in illustrated than in non-illustrated text (Hypothesis 3b); the latter hypothesis was related to a necessary premise of the assumption that perceptual sim-ulation improves the comprehension of illustrated text. The fourth research question was relat-ed to perceptual boundedness and perceptual support and predicted age-related differences; younger children were expected to benefit more from illustrations regarding the situation model (Hypothesis 4a) and to simulate vertical object movements in a more pronounced fash-ion (Hypothesis 4b) than older children. In addition, Hypothesis 4c held that perceptual simu-lation is more pronounced in younger children particularly when illustrations are present.
Three experiments were conducted to investigate these hypotheses. Experiment 1 (Seger, Wannagat, & Nieding, submitted).compared the tripartite representations of written text without illustrations, with illustrations presented first, and with illustrations presented after their corresponding sentences. Students between 7 and 13 years old (N = 146) took part. Ex-periment 2 (Seger, Wannagat, & Nieding, 2019) investigated the tripartite representations of auditory text, audiovisual text with static illustrations, and audiovisual text with dynamic il-lustrations among children in the same age range (N = 108). In both experiments, a sentence recognition method similar to that introduced by Schmalhofer and Glavanov (1986) was em-ployed. This method enables the simultaneous measurement of all three text representations. Experiment 3 (Seger, Hauf, & Nieding, 2020) determined the perceptual simulation of vertical object movements during the reception of auditory and audiovisual narrative text among chil-dren between 5 and 11 years old and among adults (N = 190). For this experiment, a picture verification task based on Stanfield and Zwaan’s (2001) paradigm and adapted from Hauf (2016) was used.
The first two experiments confirmed Hypothesis 1, indicating that the tripartite model is appli-cable to the comprehension of auditory and written narrative text among children. A benefi-cial effect of illustrations to the situation model was observed when they were presented syn-chronously with auditory text (Hypotheses 2a), but not when presented asynchronously with written text (Hypothesis 2b), so the ITPC is partly supported on this point. Hypothesis 2c was rejected, indicating that motor resonance does not make an additional contribution to the comprehension of narrative text with dynamic illustrations. Regarding the textbase, a general negative effect of illustrations was not observed (Hypothesis 2d), but a specific negative effect of illustrations that follow their corresponding text passages was seen (Hypothesis 2e); the latter result is also in line with the ITPC. The text surface (Hypothesis 2f) appears to benefit from illustrations in auditory but not written text. The results obtained in Experiment 3 sug-gest that children and adults perceptually simulate vertical object movements (Hypothesis 3a), but there appears to be no difference between auditory and audiovisual text (Hypothesis 3b), so there is no support for a functional relationship between perceptual simulation and the situ-ation model in illustrated text. Hypotheses 4a–4c were investigated in all three experiments and did not receive support in any of them, which indicates that representations of illustrated and non-illustrated narrative text remain stable within the age range examined here.