Genres are not what they used to be. They are both more and less. More in the sense that today many genres of interest are increasingly multimodal, making their meanings through the co- deployment of resources from both language and other semiotic systems. Less in the sense that as people cross institutional and genre boundaries on shorter and shorter timescales (surfing across television channels from genre to genre, across websites from institution to institution, and living their lives between as well as within multiple jobs, tasks, and institutions), we in- creasingly not only hybridize formerly insulated genres, but we now also make meaning along our traversals across traditional genres. Genres are becoming units, raw material, for flexible trans-generic constructions: resources for meaning in a new, externally-oriented sense. Looking at genre from these contemporary viewpoints provides insights into the phenomenon of genre from new functional perspectives,
ABSTRACT: The increasingly integrative use of images with language in many different types of texts in electronic and paper media has created an urgent need to go beyond logocentric accounts of literacy and literacy pedagogy. Correspondingly there is a need to augment the genre, grammar and discourse descriptions of verbal text as resourcesfor literacy pedagogy to include descriptions of the meaning-making resources of images. Some augmentation along these lines has involved the articulation of Hallidayan systemic functional descriptions of language, mainly focussed on verbal grammar, with the social semiotic descriptions of the meaning-making resources of images described in a grammar of visual design proposed by Kress and van Leeuwen. However, current research indicates that articulating discrete visual and verbal grammars is not sitfficient to account for meanings made at the intersection of language and image. This paper adopts a systemic functional semiotic perspective in outlining a range of different types of such meanings in different kinds of texts, suggesting the significance of such meanings in comprehending and composing contemporary multimodal texts, and the importance of developing an appropriate metalanguage to enable explicit discussion of these meaning- making resources by teachers and students.
Literacy cannot be seen as only a linguistic accomplishment anymore and there is no longer the close association between print and learning Multimodality can be seen as an eclectic approach modal affordances: what is possible to express easily. The metalanguage of multimodalities must be taught and understood, as choice of mode can affect pedagogic design and interpretation: the teacher's choice of mode shapes the knowledge or even interpretation. New possibilities.
New types of texts should encourage new approaches, and the digital multi-modal tools need to be used appropriately, new affordances, new opportunities for negotiation for meaning.
Jewtii describes approaches to using multimodality , specifically cdrom in a classroom. Description of approach, genre and mode . technical terms ot analyse the images: inclusion of various types of multimodal images. Transformation of original text.
Description of some of the semiotic resources students use in decoding multimodal texts, in this case the graphic novel. Students seem unaware of the but uses a range of resources (colour, angle, panels, perspective text) in decoding the texts
Bezemer,J. and Kress.G.(2008) consider the changing roles of image and writing in the representation of knowledge in secondary school English, Science and Mathematics texts published between 1930 and 2005.
Jewitt, C. (2008) ‘Multimodality and Literacy in school classrooms’, AERA Review of Research in Education, vol. 32, pp. 241–67.
In this article, Jewitt reviews research into multimodality and literacy in the classroom, and asks what these changes mean for being literate in contemporary society, where digital media are embedded in everyday literacy practices. Jewitt argues that the time for associating learning primarily with language and print literacy is over.
R. Utescher, и S. Zarrieß. Proceedings of the Third Workshop on Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN), стр. 53--60. Kyiv, Ukraine, Association for Computational Linguistics, (апреля 2021)