ABSTRACT: The increasingly integrative use of images with language in many different types of texts in electronic and paper media has created an urgent need to go beyond logocentric accounts of literacy and literacy pedagogy. Correspondingly there is a need to augment the genre, grammar and discourse descriptions of verbal text as resourcesfor literacy pedagogy to include descriptions of the meaning-making resources of images. Some augmentation along these lines has involved the articulation of Hallidayan systemic functional descriptions of language, mainly focussed on verbal grammar, with the social semiotic descriptions of the meaning-making resources of images described in a grammar of visual design proposed by Kress and van Leeuwen. However, current research indicates that articulating discrete visual and verbal grammars is not sitfficient to account for meanings made at the intersection of language and image. This paper adopts a systemic functional semiotic perspective in outlining a range of different types of such meanings in different kinds of texts, suggesting the significance of such meanings in comprehending and composing contemporary multimodal texts, and the importance of developing an appropriate metalanguage to enable explicit discussion of these meaning- making resources by teachers and students.
Literacy cannot be seen as only a linguistic accomplishment anymore and there is no longer the close association between print and learning Multimodality can be seen as an eclectic approach modal affordances: what is possible to express easily. The metalanguage of multimodalities must be taught and understood, as choice of mode can affect pedagogic design and interpretation: the teacher's choice of mode shapes the knowledge or even interpretation. New possibilities.
Relevant for MS teachers. Notion of intellectually challenging EAL students. Transformation of information impacts register. Scaffolding to encourage thinking.
In this paper
A focus on the reshaping of the entity „character‟ in the transformation of the novel Of
Mice and Men (Steinbeck, 1937), to CD-ROM (1996)- interesting but not as relevant to primary
Jewtii describes approaches to using multimodality , specifically cdrom in a classroom. Description of approach, genre and mode . technical terms ot analyse the images: inclusion of various types of multimodal images. Transformation of original text.
R. Utescher, and S. Zarrieß. Proceedings of the Third Workshop on Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN), page 53--60. Kyiv, Ukraine, Association for Computational Linguistics, (April 2021)