Multimodal features in picture books: Challenges for SFL analysis

Abstract

Ideational meaning has been extensively discussed in SFL in terms of the resources available for construing experiential and logical meaning in verbal language. Kress and Van Leeuwen (1996, 2006, 2021) set the guidelines in their visual social grammar for interpreting and analysing visual representational meaning as well. However, the interpretation of how the visual and the verbal modes intertwine and contribute to multimodal meaning making has given way to a variety of approaches to multimodal analysis, sometimes blurring the crucial distinctions of Halliday’s (1985, 2004, 2014) ideational, interpersonal and textual metafunctions, or their visual counterparts in Kress and Van Leeuwen’s model (representational, interactional and compositional metafunctions respectively). In this talk I address the multimodal construal of ideational meaning as it is realized in the depiction of characters in multimodal narratives. I focus on the analysis of visual and verbal transitivity in picture books, as a text type where multimodality is exploited to create narrative tensions, irony and other effects on the reader (Nikolajeva & Scott, 2006). I will draw on Painter, Martin and Unsworth’s (2013) model of multimodal analysis of picture books, as well as other relevant sources to discuss challenging issues of transitivity in character representation. This invited talk also focuses on some aspects of the multimodal construal of characters in multimodal narratives from the perspective of Halliday’s (1973) ‘sociological semantics’, in terms of semantic potential and patterns of behaviour. I illustrate the discussion with examples taken from recent research on multimodal analysis of non-normative male characters (Elorza, 2022; 2023) and the ongoing MIAMUL research project on migrants’ narratives in picture books.