This empirical article investigates multimodality in English as a foreign language, both as seen in the use of multimodal texts as artefacts and pedagogical texts for learning, and through an analysis of the multimodal learning designs. We present observations from a year 10 classroom in Norway that worked with the novel The Absolutely True Diary of a Part-Time Indian (Alexie, 2007). We explore a four-week teaching sequence, asking how different modes were involved when the educator designed literacy events around the novel, and how multimodality is present in the students’ meaning making. Our aim is to make explicit and discuss some of the silent literacy practices in English teaching at lower secondary level in Norwegian schools.
This article takes its starting point in the joint understanding, of multimodal social semiotics and design-oriented didactics, that learning can be understood as a social, meaning-making process. This entails modes other than written and spoken language playing important roles in students’ learning in school, even in language learning. The ‘multimodal turn’, in which attention is focused on the interplay between modes, opens up new ways of understanding the designs of classroom activity (Kress, 2003; Kress and van Leeuwen, 2006; Mills, 2010). These phenomena are not new, but our ways of thinking about them are changing or ‘turning’ (Jewitt, 2014a, pp. 3–4) towards giving attention to modes beyond verbal language.
English as a school subject has a tradition of using visual modes in both first language teaching (Jewitt, 2014a) and second language teaching (Jakobsen, 2015; Skjelbred et al., 2017). Previous research has shown an increase in the use of images in textbooks (Bezemer and Kress, 2009). Furthermore, over recent decades, the written page has developed from a verbal to a visual unit (Baldry and Thibault, 2006; Bezemer and Kress, 2010). The visually organized one-spread layout in textbooks demands an active reader to create coherence and reading paths (Bezemer and Kress, 2009). English taught as a foreign language (EFL) or second language (L2/ESL) in Norway (where the two terms tend to be used interchangeably (see e.g. Røkenes, 2016)), has a long tradition of using multimodal resources and activities for learning, ranging from textbooks to film, music and drama (Maagerø and Simonsen, 2006; Scott and Ytreberg, 1990; Simensen, 2007). Multimodality is thus inherent in the English subject in Norway, though not an explicit part of the English subject curriculum.
Over the past decade, multimodality as a concept has been gradually introduced into curricula in several countries, most notably in Australia (Unsworth, 2014; Walsh, 2010), the UK (Matthewman, Blight, and Davies, 2004), and the Scandinavian countries (Christensen, 2016; Løvland, 2006; Tønnessen, 2010), and, importantly, in the mother tongue or first language subject (L1). Norway was the first Scandinavian country to introduce multimodal texts into school curricula in 2006, with Sweden and Denmark following in 2011 and 2014, respectively, according to Christensen (2016).
Much of the research on English and multimodality in school takes place in environments where English is the majority language (e.g. Jewitt, 2006; Kress et al., 2005; Rowsell & Walsh, 2011). A call for research into reading and the production of multimodal texts in the English subject in Norway was made in 2012 by Skulstad (Skulstad, 2012). As far as we know, hardly anybody has explicitly answered her call. Maagerø and Tønnessen (2014) have devoted one chapter to multimodality in language learning in their book on multimodal literacy and include examples from English. Birketveit (2015) and Birketveit and Rimmereide (2017) have researched the use of picture books for ESL, and Ørevik (2015) has researched the use of remediation from book to screen. Lund (2016) and Waallann Brown and Habegger-Conti (2017) have examined the way indigenous cultures are visually presented in English textbooks in Norway. Otherwise, we need to turn our gaze outside Norway, to find research on multimodality in English classrooms.
In order to find out more about the role of multimodality in the EFL classroom’s literacy practices, this case study looks at literacy events and asks the following questions: In what way does multimodality come into play, firstly, in the teacher’s designs for learning, including her choice of learning materials, and, secondly, in the students’ design in learning? Our aim is to make use of multimodal design-oriented theory to examine and discuss some of the silent literacy practices in EFL at the lower secondary level of Norwegian schooling1.
This section gives an overview of our use of the concepts of multimodality, literacy and design, which are perspectives that we find to be particularly relevant for discussing language learning in modern classrooms. In foreign language learning, verbal language (listening, speaking, reading and writing) is both the means and the objective of learning. Consequently, skeptics might ask whether semiotic resources other than verbal ones are relevant. However, learning a language is also about gaining communicative competence and learning about cultures, history and texts. This is apparent in the way language studies are usually structured in three dimensions: linguistics, literature and culture (Brøgger, 1986; Kramsch, 1995; Rindal, 2014; Udir, 2013). Theoretically, this connection between language and culture can be underpinned by a basic understanding of language as functional, as we find it in social semiotics, where Halliday states: ‘Every actual instance of linguistic interaction has meaning not only in particular but also in general, as an expression of the social system’ (1975, p. 80). It follows from this that linguistic resources for meaning making will always be understood as part of a context, and that other modes of communication, such as images (still or live), music and sound, may provide a supporting or even defining context to the learning of verbal language. Furthermore, modes other than spoken and written language can convey curriculum content knowledge. In the following, there will be examples of how an English teacher uses photos and the multiple modes of video to show a Spokane Indian pow-wow and the North-West Washington landscape. The process and outcome of language education thus includes a lot more than mastery of oral and written language.
Multimodality involves the use of several semiotic modes in communication (Kress & van Leeuwen, 2001, p. 20). We understand mode as defined by Gunther Kress: ‘Mode is a socially shaped and culturally given resource for making meaning’ (Kress, 2014, p. 60). This entails that modes are used with a certain regularity, developed within a community, and that modes are characterized by their affordances, that is their ‘potentials and constraints for making meaning’ (Bezemer & Kress, 2016, p. 23). The affordances of modes enable semiotic work (Kress, 2014, p. 62), drawing on the materiality of the mode as well as the cultural shaping and reshaping of the mode. The semiotic work of making meaning is closely connected to learning, which involves engaging with the world through resources for making meaning (Bezemer & Kress 2016). Modes appear in ensembles, and the way we combine modes into ensembles, e.g. of words, images and layout in school textbooks, is in itself shaped through social practices. In a mediatized world, one example would be the media that organize multimodal communication, such as film/video.
Our understanding of multimodality is based within the theoretical framework of social semiotics. Jewitt has outlined four assumptions that are common to the many different approaches to multimodality. These are, firstly, that ‘language is part of a multimodal ensemble’ and ‘that representation and communication always draw on a multiplicity of modes …’ (Jewitt, 2014b, p. 15). Secondly, in such an ensemble, every single mode communicates in a distinct way ‘shaped through their cultural, historical and social uses’ (2014b, p. 16). The third assumption is particularly important for this article: ‘people orchestrate meaning through their selection and configuration of modes. Thus the interaction between modes is significant for meaning-making’ (2014b, p. 16). Jewitt’s fourth point is the social, which means that the meanings of signs are ‘shaped by the norms and rules operating at the moment of sign-making, influenced by the interest and motivation of [the] sign-maker in a specific social context’ (p. 17). All in all, multimodal social semiotics gives us tools to speak about the classroom context and the interests and practices of teachers and students, and to take seriously the range of modes used in communication in school (Bezemer and Kress, 2016).
The concept of literacy, which has traditionally been understood as the ability to read and write, has been expanded in New Literacy Studies to involve other modes of communication (New London Group, 1996; Jewitt & Kress, 2003; Rowsell et al., 2013). In line with the functional view on language in social semiotics, New Literacy Studies understand literacy as a social practice, situated in specific situations and in a cultural context. In recent discussions about multimodal literacy, one might differentiate between bottom-up perspectives that focus on what it takes to master a mode, analyzing one mode at a time (Kress, 2003, p. 23), and top-down perspectives focusing on what it takes to interpret the multimodal ensemble as a whole (Danielsson and Selander, 2016; Walsh, 2010; Yi, 2014). In our study of classroom practice, we find it fruitful to conceive of literacy in line with Jewitt’s understanding that the reality of meaning making involves taking the multimodal design as a whole into consideration. Jewitt states that:
The static notion of literacy as the acquisition of sets of competencies can be replaced with a notion of literacy as a dynamic process through which students use and transform the multimodal signs and design new meanings. (Jewitt, 2006, p. 135)
Literacy practices are realized in literacy events, which can be observed as concrete phenomena. The term literacy event underlines an understanding of literacy as situated in a specific space and time, and that the meaning making that is taking place must be understood as part of this situation: ‘literacy is best understood as a set of social practices; these are observable in events which are mediated by written texts’ (Barton & Hamilton, 2000, p. 9).2 In classroom practices, literacy events are mainly staged by the teacher, and they tend to come in sequences, in which the outcome of one event may initiate the next. In the English classroom, literacy events are designed to create situations that entail listening and speaking, reading and writing English, and often at the same time learning about cultures where English is spoken. One type of literacy event that is well known to the English classroom is reading literature. In this article we examine literacy events in which a work of fiction is at the center. The literary text provides a world of its own, which the reader can enter, explore and engage in, a totality that provides a shared context for engaging in language. The literary text is a text designed for aesthetic experience (Sørbø, 2003). When it is taken into the classroom, a didactic dimension is added. At the same time, the aesthetic dimension may contribute meaning in terms of emotional engagement and motivation (Tørnby, 2013).
The New London Group called for a broadened understanding of literacy to cater for ‘the increasing multiplicity and integration of significant modes of meaning-making’ and ‘as a way to focus on the realities of increasing local diversity and global connectedness’ (New London Group, 1996, p. 64). Design is a central concept in this pedagogy of multiliteracies. It is used for semiotic activity in the process of producing and consuming texts. The point is to emphasize that ‘meaning making is an active and dynamic process, and not something governed by static rules’. The process includes three elements: ‘Available designs, Designing and the Redesigned’ (1996, p. 74). From its starting point in ‘available design’, the designing activity results in an outcome, ‘the redesigned’, representing a new meaning: ‘The Redesigned is founded on historically and culturally received patterns of meaning. At the same time, it is a unique product of human agency: a transformed meaning’ (1996, p. 76). The manifesto also points out that teachers are designers of learning processes (1996, p. 73).
This understanding has been taken one step further by Selander and Kress (2010, p. 24), who distinguish between design for learning and design in learning. Design for learning happens on many levels, framed by national curricula, local planning and infrastructure in schools. One could say that this represents the available design for the teacher when planning the lesson in the classroom. This design is met by the students’ design in learning, that is how the individual student realizes his/her interests within the cultural setting (Selander and Kress, 2010, p. 97). In our context, this is seen in the way the students choose to carry out the tasks given by the teacher. In this article, we have chosen to focus on one of these tasks, to highlight the relations between designs for and in learning.
Selander and Kress include a third dimension to their model of learning designs, and that is assessment. This will only be touched upon indirectly in this article, since some of the student tasks are part of the formal assessment, and others are not.
Finally, a design perspective on the activities going on in the classroom includes, on the one hand, a socio-cultural framing that teacher and students may have more or less in common and, on the other hand, the previous experiences and personal interests of teachers as well as students. Design and redesign happen on several levels in the classroom investigated in our study: in the choice of literary text, in the staging of reading, interpreting and discussing the text, and in the cultural norms and practices surrounding school learning. These levels may be understood as different timescales or activity scales, where exchanges on a higher level form the context for exchanges on a more detailed level. Together, the system of scales constitutes a cultural pattern or social semiotic formation (Lemke, 2000, p. 276). In our observations, the choice of literary text to work with represents one scale, a realization of aims and objectives in the curriculum on the scale above, and, at the same time, always relating to the activities included in the didactic design on the scale below, as will be explained as a model of interacting cogwheels below.
This study adopted an ethnographic approach, and what we present here is a single-case study (Creswell, 2013). A case study is ‘a study that investigates a contemporary phenomenon in depth and within its real-life context’ (Yin, 2014, p. 237). Selection of this school and this class was based on purposeful sampling (Creswell, 2013), in the sense that one of the researchers had previously performed a research project at the school and was familiar with the setting. It was stressed that the teacher was not to change her plans or make adaptions for the researcher, who would take the role as nonparticipant observer. Chance would have it that the researcher had donated a class set of a novel as a gift of thanks to the school for research participation. It turned out that this same novel, now five years later, was on the plan for the period the researcher came to make observations. As researchers, this makes us more closely associated with the literacy practice we study. We, nevertheless, take an observer’s perspective.
Data was collected over four weeks in one class during English lessons that concentrated on this novel. It was a class of 14 students, aged 15 to 16 years. All the students and the teacher gave informed consent to observation, and 11 students agreed to share their written assignment. All names used are pseudonyms. The study has been approved by the Norwegian Centre for Research Data.
This small-scale qualitative study looks at one teacher’s design for work with a full novel and one particular form of response from the students. This gives the opportunity to study in depth the interplay of modes in certain literacy events, although it gives limited information about more general multimodal literacy practices in English classes here and elsewhere in Norway. Despite these limitations, by approaching this class, we can begin to understand literary reading in lower secondary school English from a multimodal design-oriented perspective. One area in which this school differs from many others in Norway is the fact that it includes extensive reading of literature in both the Norwegian subject and the English subject; a study by Penne (2013) has shown that a large number of Norwegian students leave lower secondary school having read only excerpts from novels.
Observation and field notes, together with collected printed material and student products, form the basis for analysis in this article. By looking at the literacy events as they unfolded, in addition to the printed static texts that form the basis for literacy events, we gain a better idea of the overall literacy practice of the class. At the end of the observation period, interviews were conducted with three pairs of students and, finally, with their teacher. The audio-recorded interviews add important perspectives to the analysis concerning the motivation behind choices made in designs for and in learning.
The analysis has been structured according to three activity scales (Lemke, 2000): The novel is fundamental to the lesson sequence and, hence, is presented first in the analysis section. The sequence of lessons is analyzed at the next level of activities, designed by the teacher (Diana) to inform, motivate and activate the students’ interpretations of the literary text. Finally, at a more detailed level, we analyze one particular case of students’ responses to the written assignment, paying attention to the use and function of images.
Sherman Alexie’s young adult novel The Absolutely True Diary of a Part-time Indian (2007) is a distinctively multimodal text that describes Native Americans in a non-sentimental and nuanced way. In addition to troubles any teen can recognize, the novel deals with sensitive issues like identity, racism, poverty, death, and alcoholism, all told with disarming humor. Fourteen-year-old Arnold portrays his experiences of growing up on the Spokane Indian reservation in Washington. Seeing little future on ‘the rez’, he changes schools to attend an all-white school. Excerpts from the novel made up part of the 2014 national year 10 examination in English in Norway, although the novel has been on the list of banned books in libraries in the US.
In terms of genre and form, the novel comes close to the students’ out-of-school textual world. This is potentially important both in light of motivation for reading and in the analysis of how the students design their multimodal texts. The extensive exposure to English through popular media for Norwegian students was documented in a comparison of the role of out-of-school exposure to English for upper secondary students in Norway and Poland, by Aniol (2011). The novel combines doodle-styled images and first-person narrative in writing. It was published in the same year as the first of the Diary of a Wimpy Kid series by Jeff Kinney, the novel that marks the beginning of a wave of graphic or cartoon-like fictional diaries of young adults, which has surged over us this past decade and is familiar to students. In contrast to these, The Absolutely True Diary of a Part-time Indian is a stand-alone novel and not a series, and the book comes close to being an autobiography, which amplifies the impact of the social and cultural portrait. The novel’s form and content mean it is a text that can potentially ‘provide this imaginative leap that will enable learners to imagine cultures different from their own’ (Kramsch, 1995, p. 85).
Graphic novels make use of several modes, especially verbal and visual, and are multimodal texts, in which image and writing interact. Whether this novel really is a graphic novel depends upon the width of definition. The primary narrative in The Absolutely True Diary is carried by the words, making illustrated novel a fair label. However, the 60 or so illustrations in this 230-page novel do more than illustrate. The images vary according to the mood of the narrator. When he has time to draw with detail, and probably feels calm, the images are soft pencil drawings. Then, when he is emotional, this is reflected in bolder (perhaps felt pen) lines and scribbled drawings. The fictional first-person narrator tells us they are his own drawings and that they are significant as a means of expression for him:
I draw because words are too unpredictable. I draw because words are too limited… I draw because I feel like it might be my only real chance to escape the reservation. I think the world is a series of broken dams and floods, and my cartoons are tiny little lifeboats. (Alexie, 2007, pp. 5–6)
Verbal text thus reinforces the images’ legitimacy as a mode to be, in the words of the young narrator, ‘taken seriously’ (Alexie, 2007, p. 95). The drawings convey meaning by themselves, as well as in the multimodal ensemble of the book.
While cartoons and illustrated novels were previously looked (down) upon as stepping stones to more canonical or classic text-based literature (Krashen, 2004), graphic novels, comics, and other multimodal texts are now gaining ground in Norwegian literature, as well as in education.3 In recent Norwegian textbooks for pre-service English teachers, graphic novels are promoted (e.g. Wiland, 2016, pp. 153–158). They are depicted as giving the opportunity to teach decoding skills for images and verbal expressions together as ‘a necessary literacy skill’ (Rimmereide, 2013, p. 131). We will show, however, that, rather than focusing on the ability to read the multimodal ensemble, Diana states other reasons for using this multimodal text.
Diana’s design for working with the novel The Absolutely True Diary of a Part-Time Indian is expressed in a lesson plan that includes specific activities for each lesson, curricular goals, and directions for how to reach them in the form of assessment criteria. Each student received the lesson plan and a copy of the novel before their Christmas holiday, making it possible to read and study during their holidays. The lesson plan extended over seven hour-long lessons and included several bouts of silent individual reading and one traditional lecture by the teacher (including a PowerPoint presentation with video embedded); students also had to keep an individual written reading log and devise questions and answers for a Kahoot quiz played in class; the plan ended with one written and one oral product for assessment. Time for reading and writing was given in class, as well as for homework. Writing was done primarily on individual laptops provided by the school, but some students wrote their logs by hand.
Oral and written products form the basis for grade-giving assessment in the teacher’s design. For the oral activity, the teacher changed her design from individual presentations with PowerPoint to a shared discussion, called a subject conversation. This is a dialogue about the novel, led by the teacher, commonly organized with everyone in a circle of chairs. It is a teaching method often used in the subject Norwegian to talk about literature. Diana said in the interview that she changed over to this strategy in order to prepare the students for their final examination in English.
From the students’ perspective, extensive reading as part of English was not new. In each of the three years of lower secondary school, they read a full novel in both Norwegian and English. Students expressed by their body postures, and confirmed in interviews, that they found it demanding to read a whole novel, though the reading was more enjoyable for them than writing a log. The students were very positive about the oral activities in the form of a Kahoot quiz and the subject conversation, the latter being a learning activity they had only made use of in Norwegian classes. All the students handed in a written assignment; as researchers, we were given permission to read and keep copies of 11 of them, before the teacher wrote her feed forward and graded them.
Out of this sample of 11 papers, three students chose a task that specifically asked them to discuss the images in the novel. The remaining eight papers were evenly distributed between two other tasks, which asked them to write a text about the reading experience or a book report, respectively.
The students wrote texts of varying length, from half a page to just over three full pages. Two typewritten pages were required. Seven out of 11 students used images in their texts, four of them in response to tasks designed by the teacher for traditional written texts.
The teacher’s design for learning pivots around a multimodal text, and the design of the teaching sequence itself can also be considered a multimodal text (Boeriis and Nørgaard, 2015). In the model in Figure 1, the cogwheels represent the main literacy events designed by the teacher to create a good learning environment for the shared reading of the novel. The cogwheels’ shapes and arrows are, for us, an apt representation of the complex interaction happening in the classroom on the medium level of activities, with each part shaping and continuously reshaping the students’ learning. The teacher’s design aims to direct the students’ interest and aid their comprehension of both language and content in the novel as a whole. In their design in learning, each student will turn each cogwheel differently; that is, they will respond differently to each literacy event, which affects their understandings of other cogwheels, even the past ones, and thus the sequence as a whole. New turnings of the cogwheels will, in other words, affect the foregone understanding built up by other cogwheels.
We have chosen to highlight the following artefacts and associated activities in our model: the novel (1), along with the lesson plan with the concomitant assessment criteria (2) symbolically on top and representing a timescale running across the whole period. These two are also placed as cogwheels inside the second timescale, with the teacher’s lecture (3), the individual student reading logs (4), the written assignment (5), the teacher when she interacts and helps students while they are writing on their laptops (6), and the subject conversation (7).
Diana’s design for learning is closely linked to student motivation. Extensive reading is firmly recognized as an effective pedagogy for second language learning (Elley, 1991; Mason and Krashen, 1997) and is a recommended strategy in Norwegian EFL didactics (Hellekjær, 2008). Motivation for extensive reading is a different matter. Knowing her class well, it is not surprising that the teacher is concerned with motivation and with creating conditions for mastery. Her design, which we interpret as a multimodal design for learning, is directed at these challenges:
I use both visual and auditory approaches so that [the students] see it in various ways, and for me it is about motivation and different learning styles. I do use mostly printed text, but I often support it, almost always, I would say, with pictures or sound. (Diana)
In other words, Diana’s multimodal design for learning is not primarily aimed at developing multimodal literacy skills. Rather, she endeavors to motivate, and she uses a range of modes to accommodate a variety of what she calls learning styles. Her principal aims are shaped by curricular aims, namely mastering verbal language and content knowledge.
A respect for each individual student’s opinion and interpretation of literature is at the backbone of Diana’s classes. In her teaching and in interviews, she expresses unambiguously that each student is offered the right to have his or her own response to the novel. This is her way of navigating the tension between her own wish and curricular obligation to foster a joy of reading (Udir, 2013, pp. 2–3) and her desire to simultaneously give agency to her students, including those who have a negative attitude to reading literature or to the subject.
Spatial organization in the classroom is a design little used by this teacher, making the exception all the more visible when she organizes the subject conversation as a circle in the middle of the room. Normally, and across subjects, the students are seated in rows, with individual desks facing the teacher’s desk at the front. Diana specifically uses spatial organization to help overcome some of the affective filters of the students and to create a more engaging atmosphere:
I believe that when we sit so closely together, not too close, but close and without anything between us, so it feels safer, more intimate, safer, so they dare to open up, it becomes, um…, they forget in a way that it is an assessment situation. And then they forget to think about pronunciation and whether they pronounce correctly, because they really want to contribute. (Diana)
Diana’s statement shows the duality of form and content that makes the English subject so complex (Rindal, 2014, p. 2). The conversation is a learning activity, as well as an assessment situation, and it is better to contribute something in imperfect English than to remain silent. Diana is very conscious of working to prepare her students for the final examinations; she balances wide curricular learning goals with examination demands:
… we need to master the subject conversation form; for that conversation, it is the exam we work towards. But also, if you put away the pressures of grades and exams and concentrate on learning, I find that the subject conversation means they help each other grow. The communication between them makes them think of other aspects, and it becomes more of a natural setting. (Diana)
During their subject conversation, the students were encouraged to talk about the same topics as those they had written about in their assignments, as Diana thinks this will make it easier for them to speak. In this sense, the conversation was more of a prepared talk than a spontaneous discussion. Still, the learning potential, not just for the speaker but also for the listener, is evident, and the conversation makes for co-construction of meaning (Kress and Burn, 2005). Student Martin writes in his assignment:
Almost everyone [in] the world think it is boring to read for a test, homework or whatever that have something whit school to do. … it is something you need to read and learn to get a good grade. [In our] School you learn more about the book and the [writer]. The reason is because they have “fagsamtale” [subject conversation] where everyone can listen what they know about the book and the writher. Also they that have not read the book. I recommend this kind of studying. you doesn’t learned only some “boring” tings” [original spelling]. (Martin)
In the lesson sequence as a whole, the multimodal interplay can be seen partly within each cogwheel (activity) and partly between the cogwheels. The teacher’s presentation (3) may serve as an example of modal density (Norris 2014, p. 90), with its combination of printed text, image and film clips, and her voice, gestures, and words; all modes combine into a unity for understanding the novel in a broader perspective. In other cases, the modal interplay comes in sequences, for instance when the students talk about the novel in the subject conversation (7) in response to reading the words and studying the images. The semiotic work of articulating their thoughts is seen as an expression of learning (Bezemer & Kress, 2016). The individual task of reading a novel is scaffolded by whole-class activities, as well as individual tasks activating different modes.
In the students’ design in learning, time was largely devoted to the written text assignment and preparation for the subject conversation, which were both graded. This reminds us that designs for and in learning are inextricably connected by the third element of pedagogical design: assessment (Selander & Kress, 2010). In the following, we will look more closely into how the students make meaning from the novel in their written assignments. Our focus will be on how their texts may be inspired by the available design, and we look specifically at the role of images in texts produced within the literacy practices of language learning.
The design for learning has a multimodal text at its core, and one of the writing tasks poses questions about the role of the images in relation to the written narrative. In the design in learning this has probably had an effect on what the students see as useful modes in their own texts. Half of the students who did not write about images still used images in their texts. In our analysis we will focus specifically on the assignments responding to the task in which students are asked to express in words what they read from the images. As mentioned (see Table 1), three students wrote about the images in The Absolutely True Diary of a Part-Time Indian. They had the following three questions to guide them: ‘What can we see? What does it reveal about the character(s) in the novel? How does it influence your reading of the book?’
|Teacher’s design||Students’ design|
|Tasks||Number of students||Verbal response||Verbal and image response|
One of the three papers about the images in the novel does not include the drawings it discusses. Sander has added an image of the novel’s front page but otherwise refers to the page numbers where his chosen images are. His text is mainly descriptive, but he concludes by relating the first image that interested him to the overall theme of the novel: ‘I think the author drew this picture because good grades in the school was [Arnold’s] ticket out of the rez’. Moreover, Sander comments on how the novel’s direct style and pictures appealed to him: ‘I chose this picture because I have thoughts like [Arnold’s]. I too have big dreams…’
The other two students have included the images they discuss, even though the assignment does not require this. In fact, Ida has even included three more, as an illustration of the drawing styles she finds. Figure 2 shows a facsimile, to give an overview of the layout of her analysis.
These pages show that Ida is very attentive to layout and image use. She has placed the three examples of different drawing styles at the bottom of her first page and uses them to support her written text. She connects the drawing style to the main character’s mood and purpose with each drawing and with the topic in the narrative. She has found information on the drawing styles in an interview with the artist Ellen Forney, appended to the novel, but she finds her own examples of different styles. When Ida goes on to discuss two other images in greater depth, she places image and printed text next to each other. She shows a keen sense of the image-word interaction in the novel when she writes: ‘In this chapter, he had already used so many words to tell so little, so I think it was perfect with a simple drawing like this … it does not take the attention away from the words or the story.’ Ida understands that the novel and its drawings are a work of art and says: ‘Ellen [Forney] did a great job getting into the head of Arnold.’ Ida has read the full multimodal ensemble of image and printed text, and she is able to take a meta-perspective.
Henrik used his phone to take pictures of the images and then placed the images he discusses on the right-hand side of the page (Figure 3). In other words, Henrik did not let available images on the Internet decide his choice of image, as some of the others did. The first image is a drawing of the main character divided into two, combined with handwritten tags that specify the difference when he compares ‘White’ and ‘Indian.’ Henrik points out how meaning is differently made in each mode and in the ensemble: ‘when you read you start thinking about how the characters look, and it is really fun when the picture match[es] your imagination.’ He then points out how the drawings make ‘it feel like the book you are reading is more personal.’
One student who did not choose the task about images, but used images nonetheless, is Julie (Figure 4).
As may be seen from Figure 4, Julie has only written half a page, when the requirement was to write two. Perhaps she has included the large image of the novel’s front page simply to fill her two pages? When we examined her second page, however, we realized that, though her text is short and has errors and slips in grammar, lexis, and spelling, she shows understanding of the main contents of the novel and communicates quite effectively in this ensemble. Her plot summary is concise, and she ends by leaving the reader with a cliffhanger: ‘Wild they ever be friends again?/Or wild Rowdy hate Arnold forever?’ [original spelling]. Her ending echoes the logbook task given by the teacher, in which there was a question about whether the students would recommend the novel to others. Julie has placed her enticing questions next to an image of Rowdy that aptly specifies his aggressive nature, which she describes in words: ‘Rowdy becomes so angry that he punches Arnold in the face! He shouts that he would never see him again.’ The drawing’s close-up frontal view of Rowdy demands the viewer’s emotional involvement (Kress and van Leeuwen, 2006). Julie’s final effort in persuading others to read the novel is an image of the author, also looking directly at the viewer. Sherman Alexie, however, looks pleasantly relaxed in contrast to Rowdy. This echoes the book, in which Rowdy serves as a foil to Alexie’s alter ego, Arnold. In sum, Julie has created an engaging multimodal design of her own. We include her design, because it shows qualities that are easily overlooked if written text alone is assessed.
In our analysis, we find that the teacher includes a whole range of resources for making meaning throughout the processes she designs, but these are more varied at the beginning. First, Diana has chosen a multimodal text as the core text for the whole process, and, in her introduction of the author, the topic, and the novel, Diana uses a range of modes with her PowerPoint and video. Research on English (L1) in Britain shows a similar use of image, PowerPoint and YouTube in introductory classes (Jewitt, 2011). Furthermore, we have shown that the alternation between individual and collective activities, including reading, writing, talking about the novel, and teacher to student assistance in creating a written assignment, all in all creates a learning sequence that in itself forms a multimodal ensemble.
The overall design for learning is met by the design in learning; knowing precisely how students ‘turn the cogwheels’ is impossible, but signs of it can be discerned. For example, Diana makes the autobiographical aspect of Alexie’s narrative known and shows a video of the author reading. Student Daniel comments in his assignment that, when he saw the video, this interfered with the mental image he had formed during reading: ‘the picture mind became blury’ [original spelling]. Daniel has taken the exaggerations of the novel’s images and words at face value and has to modify his interpretation in light of the video. The teacher, thus, has created an environment for seeing the novel in a broader perspective, and Daniel adjusts his understanding of the written mode, in light of the video.
The modes and design options explicitly afforded to the students for the final stage of their design in learning are traditional, that is writing and speaking. Thus, it seems that the visual resources are primarily seen as support for what the learning process is really about: learning written and oral English language, culture, and skills in literary analysis. This rich use of modes in the pre-reading stage, followed by an increasingly verbal textual orientation, is in accordance with other findings in research on multimodality in Norway (Smidt, Tønnessen, & Aamotsbakken, 2011) and internationally (Jewitt, 2011; Kress et al. 2005).
Diana seems more willing to include visuals to increase the aesthetic engagement and motivation and to adjust to the perceived learning styles of her students, than to recognize the diversity of modes as a resource for making meaning and literacy development. Her motivation seems to be that other modes may support written and spoken language. On the one hand, this is one of the advantages of using visuals for language learning. Picture book specialist Nikolajeva points out: ‘A visual image can potentially evoke a wider range of emotions circumventing the relative precision of words’ (2014, p. 96). This is perhaps even more important in foreign and second language learning, in which the words’ ‘relative precision’ is made even more relative by their being new or vague to the student. On the other hand, by using images mainly as scaffolding for learning, the design for learning risks missing a possibility to develop multimodal literacy. The exception to this is one out of three written assignment options, in which Diana asks for explicit attention to the images. Here, the role of images in the story as a whole, and an appreciation of the aesthetic experience, are observable in the design for learning.
In their designs in learning, some students seem to have a slightly different take on the multimodal ensemble of verbal language and images. In the examples we have discussed, we have seen that the students produce texts in which words and images are as closely knit together as in the novel they are commenting on. In some of the student assignments, the total meaning would not come through if words and images were separated. Especially, Julie’s and Ida’s assignments are examples of visually led texts, in which each page makes a visual unit (Bezemer & Kress, 2010). The affordances of the word-image ensemble allow Ida and Julie to dig deeper into the meaning of the novel than they would have been able to do through words alone.
What emerges from our analysis is that the teacher and the students adhere to different cultural patterns in their social semiotic formations (Lemke, 2000). The teacher’s school practice is deeply rooted in teaching traditions, framed by curricula and plans, and to some extent renewed through new text forms and digital technology. This meets with the students’ literacy practices that stem partly from their schooling experience and partly from leisure time activity, in which they are used to reading and producing multimodal texts tightly interlaced in word-image cohesion, in social media, to mention but one example. This distance between designs for and in learning is not overwhelming. Depending on how the students’ response is met by the teacher, it may be seen as a fruitful tension, creating a space for further development – or, alternatively, as a space for miscommunication and frustration.
Likewise, while multimodal texts are gaining ground in Norwegian education, the dominance of writing (Kress, 2003) still lurks just beneath the surface, as even the pre-service teacher textbook mentioned states:
In terms of language learning, the visuals may support the understanding of the story and may help the reader to fill the gaps that are not easily accessible through the verbal text. This way of reading is spatial and is useful and highly relevant in today’s multimodal society. (Rimmereide, 2013, p. 134).
On the one hand, this statement recognizes multimodality as part of literacy in modern language learning, but it simultaneously relegates images to the role of scaffolding the seemingly more important verbal elements. There is an ambiguity that is understandable, in terms of both the curriculum and the nature of the language subject.
In our analysis, we found that the teacher includes a wide range of resources for making meaning throughout the processes she designs in response to the multimodal literary text she has chosen to bring into the didactic work in the classroom. Both the interplay of words and images and the autobiographical nature of the text are reflected in her multimodal designs. However, it seems that the visual and other resources are primarily seen as support for learning written and oral English language and culture, and skills in literary analysis. The teacher’s design for learning moves from a rich multimodal literacy practice towards traditional assessment based on verbal language products, though images form an option as a basis for this language production.
The students (working within the teacher’s design for learning) are given a more limited range of modes in what they are asked to produce for assessment than the range they get offered for use during the learning process. We also observe, however, that, in their design in learning, many students include images as a mode without being asked to. Visual modes are easily afforded to them by the digital word processing software. Furthermore, we have claimed that the students have acted as interpreters of the parts of the teacher’s design that interested them (Bezemer and Kress, 2016; Jewitt, 2014a; Selander and Kress, 2010) and that the teacher’s attention to image has paved the way for students’ inclusion of images in their assignments. Some of the students produce texts in which we discern a slightly different take on the multimodal ensemble of verbal language and images, seen in their production of texts in which the total meaning would not emerge if words and images were separated.
English teaching and learning, as seen in this paper, has multimodal qualities that are largely silent and untapped in relation to literacy development. Literacy is now a part of all subjects in Norwegian schooling (Blikstad-Balas 2016), and this paper, though based on one case, shows that, without the training of teachers to pay attention to modes beyond the verbal in their teaching and assessment, an important part of reading and production of texts is potentially lost as a means of learning. We hereby call for a ‘multimodal turn’ beyond the L1 subject. Today’s curriculum for English does not include the production of multimodal texts, and it seems high time that students were qualified to do more than consume: to also produce multimodal texts in order to realize their full potential for meaning making in the classroom.
3Images in picture books make for steady interest in the Nordic countries, exemplified by the topic in the Nordic Children’s Book Conference (Nordisk barnebokkonferanse) which, in 2017, had the title “The Visual Turn”: http://stavanger-kulturhus.no/Arrangementer/Nordisk-barnebokkonferanse. See also Lene Ask (2016).
We would like to thank the students and teacher in the school where we collected the data. Also, we appreciate the advice and feedback offered by NAFOL’s Professor Kari Smith on an earlier version of this article.
The authors have no competing interests to declare.
Aniol, M. (2011). New media and new literacies: Mapping extracurricular English language competences of Polish and Norwegian adolescents. In: Kaczmarek M. (ed.), Health and well-being in adolescence: Part two, Media 101–123. Poznan: Bogucki Wydawnictowo Naukowe.
Ask, L. (2016). Vår manglende kompetanse. Retrieved from: http://www.periskop.no/var-manglende-kompetanse/.
Bezemer, J., & Kress, G. (2009). Visualizing English: A social semiotic history of a school subject. Visual Communication, 8(3), 247–262. DOI: https://doi.org/10.1177/1470357209106467
Bezemer, J., & Kress, G. (2010). Changing text: A social semiotic analysis of textbooks. Designs for Learning, 3(1–2), 10–29. DOI: https://doi.org/10.16993/dfl.26
Birketveit, A. (2015). Picture books in EFL; vehicles of visual and verbal literacy. Nordic Journal of Modern Language Methodology, 3(1), 1–27. Retrieved from http://journal.uia.no/index.php/NJMLM/article/view/108.
Birketveit, A., & Rimmereide, H. E. (2017). Using authentic picture books and illustrated books to improve L2 writing among 11-year-olds. The Language Learning Journal, 45(1), 100–116. DOI: https://doi.org/10.1080/09571736.2013.833280
Boeriis, M., & Nørgaard, N. (2015). “Multimodal text”. Key Terms in Multimodality: Definitions, Issues, Discussions. Retrieved from https://multimodalkeyterms.wordpress.com.
Christensen, V. (2016). Elevers produktion af multimodale tekster; hvad ved vi og hvad mangler vi? Acta Didactica Norge, 10(3), 1–19. DOI: https://doi.org/10.5617/adno.2841
Danielsson, K., & Selander, S. (2016). Reading multimodal texts for learning – A model for cultivating multimodal literacy. Designs for Learning, 8(1), 25–36. DOI: https://doi.org/10.16993/dfl.72
Elley, W. B. (1991). Acquiring literacy in a second language: The effect of book-based programs. Language Learning, 41(3), 375–411. DOI: https://doi.org/10.1111/j.1467-1770.1991.tb00611.x
Halliday, M. A. K. (1975). Learning how to mean: Explorations in the development of language. London: Edward Arnold. DOI: https://doi.org/10.1016/B978-0-12-443701-2.50025-1
Jakobsen, I. K. (2015). Multimodal literacy in English as a school subject in Norway. Paper presented at the conference Multimodality and Cultural Change, University of Agder, Campus Kristiansand, Norway.
Jewitt, C. (2011). The changing pedagogic landscape of subject English in UK classrooms. In: O’Halloran, K. L. and Smith, B. A. (eds.), Multimodal studies: Exploring issues and domains 184–201. New York: Routledge.
Kramsch, C. (1995). The cultural component of language teaching. Language, Culture and Curriculum, 8(2), 83–92. DOI: https://doi.org/10.1080/07908319509525192
Kress, G., Jewitt, C., Bourne, J., Franks, A., Hardcastle, J., Jones, K., & Reid, E. (2005). English in urban classrooms: A multimodal perspective on teaching and learning. London: Routledge. DOI: https://doi.org/10.4324/9780203397305
Lemke, J. L. (2000). Across the scales of time: Artifacts, activities, and meanings in ecosocial systems. Mind, Culture, and Activity, 7(4), 273–290. DOI: https://doi.org/10.1207/S15327884MCA0704_03
Lund, R. (2016). Searching for the Indigenous: Urfolk i engelskverket Searching. In: Askeland, N. & Aamotsbakken, B. (eds.), Folk uten land? Å gi stemme og status til urfolk og nasjonale minoriteter 15–33. Kristiansand: Portal.
Mason, B., & Krashen, S. (1997). Extensive reading in English as a foreign language. System, 25(1), 91–102. DOI: https://doi.org/10.1016/S0346-251X(96)00063-2
Matthewman, S., Blight, A., & Davies, C. (2004). What does multimodality mean for English? Creative tensions in teaching new texts and new literacies. Education, Communication & Information, 4(1), 153–176. DOI: https://doi.org/10.1080/1463631042000210944
Mills, K. A. (2010). A review of the “Digital Turn” in the New Literacy Studies. Review of Educational Research, 80(2), 246–271. DOI: https://doi.org/10.3102/0034654310364401
New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92. DOI: https://doi.org/10.17763/haer.66.1.17370n67v22j160u
Nikolajeva, M. (2014). Reading for learning: Cognitive approaches to children’s literature. Amsterdam: John Benjamins Publishing Company. DOI: https://doi.org/10.1075/clcc.3
Rindal, U. (2014). What is English? Acta Didactica Norge, 8(2), 1–17. DOI: https://doi.org/10.5617/adno.1137
Røkenes, F. M. (2016). Preparing future teachers to teach with ICT: An investigation of digital competence development in ESL student teachers in a Norwegian teacher education program. (Doctoral dissertation, NTNU, Trondheim). http://hdl.handle.net/11250/2395012.
Rowsell, J., Kress, G., Pahl, K., & Street, B. (2013). The social practice of multimodal reading: A new literacy studies- multimodal perspective on reading. In: Unrau, N. J. & Ruddell, R. B. (Eds.), Theoretical models and processes of reading (6th ed.), 1182–1207. Newark: International Reading Association. DOI: https://doi.org/10.1598/0710.43
Rowsell, J., & Walsh, M. (2011). Rethinking literacy education in new times: Multimodality, multiliteracies, & new literacies. Brock Education: A Journal of Educational Research and Practice, 21(1), 53–62. DOI: https://doi.org/10.26522/brocked.v21i1.236
Udir. [Ministry of Education and Research]. (2013). English Subject Curriculum ENG-103 [translated to English]. Oslo. Retrieved from http://data.udir.no/kl06/ENG1-03.pdf?lang%20=%20eng.
Unsworth, L. (2014). Multimodal reading comprehension: Curriculum expectations and large-scale literacy testing practices. Pedagogies: An International Journal, 9(1), 26–44. DOI: https://doi.org/10.1080/1554480X.2014.878968
Yi, Y. (2014). Possibilities and challenges of multimodal literacy practices in teaching and learning English as an additional language. Language and Linguistics Compass, 8(4), 158–169. DOI: https://doi.org/10.1111/lnc3.12076