In Denmark, primary and lower-secondary schools are generally focusing a great deal of attention on integrating digital technologies in their teaching. One feature of such new technologies is that they enable the creation of digital representations. As such, the current digitization in progress at these schools is allowing teachers to present their material in new ways and, similarly, students to produce new representations of academic concepts and contexts. In science teaching, however, the use of such technologies remains limited, as a gap currently exists between teachers’ understanding of how school students learn science and the digital options that teachers have and are expected to use in their teaching (Søndergaard & Hasse, 2012). What is more, according to an Australian study, science teachers tend to choose representational modalities primarily with a view to accommodating their students’ differing learning styles rather than as a way of conveying science content in itself (Prain & Waldrip, 2006). Striving to close the gap between the ways students learn science and the digital options available for them to produce digital multimodal representations, this study proposes three initial design principles for teachers to apply.
As a field of discourse, science relies on a mix of multimodal forms of representation (Tytler et al., 2007). For example, scientific concepts are represented through verbal language, formulae, models, graphs, diagrams and other visual representations (Prain & Waldrip, 2006). One can therefore argue that working with multimodal representations in science classes plays an essential role in involving students in scientific culture, and that such representations are thus an especially apt way of working with the subject itself (Murcia, 2010).
Over the past fifteen years the research on student engagement with representational modes and on the development of representational competencies has primarily been conducted from three perspectives. The first perspective involves a research interest in analysing and clarifying what parameters might affect students’ learning when they interact with the various representations used in science subjects, as well as in how the fact that the vast majority of these representations are digital might impact their learning (Waldrip & Prain, 2012). The second perspective, which has mostly been studied in tertiary level education, focuses on discipline knowledge and its representation. For example, researchers have studied the discipline-related affordances of representations, the role such affordances play in the representation of discipline knowledge and the pedagogical affordances of representations, as well as explored the role these affordances play in the teaching and learning of discipline knowledge (Airey & Linder, 2017). The third perspective has focused on student-generated representations as a means of increasing scientific literacy, as there is a distinction between learning from representations and learning with them.
These three research perspectives are interrelated, as students’ scientific literacy depends on their ability both to analyse and conceptualize scientific representations and to produce such representations themselves (Waldrip, & Prain, 2012). However, less research has been done from the third perspective concerning student-generated representations (Waldrip & Prain, 2012), even though the possibilities of student-generated digital representations continue to grow in step with ever-developing digital technologies. For this reason, research concerning student engagement with and production of representations in science subjects is a rapidly expanding field of science education research – especially as regards digital representations and their potential. As Murci (2010) put it: “greater understanding of the impact of digital tools on learning and teaching is required as they have the potential to change the way knowledge is represented and re-represented.”
Research also offers a basis for considering a multimodal avenue that embraces digital technologies as holding a learning potential for students: “This exponential growth in personal digital technologies coincides with a growing body of research which suggests that getting students to create a multimodal representation of a science concept is a good way to enhance learning” (Hoban & Nielsen, 2010). As such, the literature on multimodal representations becomes a useful lens through which to examine how digital representations born of the growing prevalence of digital technologies can work to support learning. This line of research could provide guidelines for designing such teaching at the macro level, but a framework to describe learner-constructed representations as a way of meaning-making in science has yet to be fully developed (Tippett, 2016). According to Tippett (2016), more research is required into how students construct representations in a variety of contexts and what is possible in real-life learning environments. Thus, many questions remain unanswered at the micro-level (Prain & Hand, 2016). Consequently, this research project aimed to team up with teachers to develop designs for learning that raise student awareness of the various affordances provided by different representations in general but that also specifically support students in generating digital representations of scientific content as an expression of their learning and of science culture. The didactic design developed for this project is informed by existing research and has been further developed in the context of concrete practices, for which reason this article is intended to discuss the potentials and challenges experienced with relation to implementing this design.
The theoretical framework of this study is a social-semiotic view of representation and learning. In terms of learning theory, social semiotics sees learning as “a sign-generating activity” of meaning-making that takes place within the framework of a didactic design (Kress & Van Leeuwen 2006). Representation can be defined as a type of information obtained through the conceptualization or visualization of an item in a certain mode. In science, representations are integral to its language (Tang et al., 2014) and are thus devices used to symbolize a scientific idea or concept. These representations have an array of modalities, for example, taking the form of text, a mathematical formula, a diagram, a graph or a simulation (Lemke 1990), and multimodal representations thus involve integrating more than one of these modalities in order to communicate a given idea or concept (Airey & Linder 2009). In social-semiotic thinking, a representation of any form of meaning is criteria-based (Kress & Van Leeuwen 2006). Consequently, a number of “authorized” representations have been developed for communication (and thus learning) in the individual science disciplines, as students need to understand the modal representations of scientific concepts and be able to transfer them in order to think and behave scientifically. According to Jens Dolin (2005):
“Mastering a subject area requires the ability to express it in all its representational forms and to switch freely between them – this is a great demand to put on students and something only learned if it is the explicit objective of the teaching.”
In other words, to conceptualize scientific concepts or content, one has to have a solid understanding of the semiotic systems used to perform and transform science (Lemke, 1998). Thus, science teaching must focus on developing students’ representational competencies. According to diSessa (2004) and Kozma and Russel (2005), learners have representational competency when they understand the appropriate types and uses of multiple modes of representations, the transformation or transduction between representations and the creation of new representations.
For this study, Vaughan Prain’s and Bruce Waldrip’s comprehensive work in the field of scientific representation has informed our work (Prain & Waldrip, 2012). In essence, they argue that students’ transformations and transductions across different representations as well as their construction of multimodal representations can give them a deeper understanding of scientific concepts and contexts. Furthermore, student work with multimodal representations in science learning can be considered as an expanded writing-to-learn activity, (Wallace et al., 2004). Writing allows students to formulate meaning and re-represent science-based ideas, and since multimodal representation is an integrated part of science, science teaching can include multimodal representation in its writing-to-learn activities. What is more the re-representations produced by students also give the teacher insight into their understanding of the content.
According to Prain and Tytler (2012), representational construction affordances (RCA) can be used to explain how work with representations strengthens learning. The RCA model that focuses on learning with representations includes three dimensions of meaning-making. These are:
- knowing about the functions of various symbolic representations,
- knowing when and how to use a range of symbolic representations that are specific to science,
- understanding that knowing and reasoning can be enhanced by the construction and interpretation of representations.
This is in line with Hubber et al. (2010), who emphasize that representations must be introduced and used as thinking tools and not merely as self-contained units to be learned. The work with representations must be continually coupled to practical activities and objects, and sufficient class time must be allocated to exploring the explicit meaning of the various representations. This coupling of practical work and the specific modes of representation in science has been a source of influence for this study.
Carolan et al., (2008) have developed another framework related to classroom use of representations – the Identify-Focus-Sequence-Ongoing Assessment (IF-SO). According to this framework, the teacher must identify key concepts in the planning phase and focus on the form and function of relevant representations in the teaching phase. Through a sequence of activities, students must work to represent the concepts presented, refine their own representations and evaluate these themselves. This process of identifying key concepts and associated representations and students’ work of re-representing and evaluating their own representations have also inspired this study.
In Hand, MacDermott and Prain (2016), contributors agree that students learn multimodal representation by participating in guided meaning-making practices. Learning increases when students:
- are motivated to represent and justify causal claims about topics;
- have multiple opportunities to re-represent, translate, justify and refine understandings through processes of experimentation, collaborative peer learning, consultation and teacher-guided consensus around representational adequacy;
- come to understand the form and function of different visual, verbal and mathematical scientific representations; and
- can integrate these modes to interpret and create convincing textual claims in the subject of science.
Science education researchers have classified representations in a wide array of ways, (Gunel & Yesildag-Hasancebi, 2016), with Wu and Puntambekar’s (2012) classification demonstrating the true breadth of external representations in science, as they distinguish between verbal-textual, symbolic-mathematical, visual-graphical and actional-operational. These are the categories used in this study.
This research project is a qualitative study that takes a design-based approach, as the project both tries to understand and improve practice. The following describes the research phases, the timescale, participants, data sources, procedure and data analysis.
The study was conducted in four phases. In the first phase the researchers identified problems posed by the existing research as well as some problems occurring in current practice, for example, that school science teachers appear to have very limited awareness of representational modes. This brought us to the following research question: How can designs for learning in science teaching be constructed to help students become more aware of the conceptual affordances inherent in representational modes by generating digital representations as an expression of learning and science culture.
In phase two, researchers involved the participating teachers in validating the problems identified and developed proposals for solutions based on the theoretical framework. The designs for learning proposed in this study thus partly build on existing research on representations in science education, but also on ideas that came up in the joint work of the researchers and teachers to create a design for learning. In phase three, the design was tested and re-tested in practice in three iterations, and the researchers observed the associated teaching sessions in order to determine the viability of the design. The design principles were applied in all three iterations, but gradually modified based on the results from the previous iteration. In the fourth phase, the researchers reflected on the robustness of the empirically founded design principles in terms of their ability to transcend the local context.
The overall research period ran from 2014-2016/17 and involved teachers and students from two different schools in Copenhagen, Denmark, Table 1.
|School||Number of students (n)||Grade||Teachers involved||Subject and period|
|Trial (2015)||School 1||N = 60||8th||2||Microbiology
|Trial (2016)||School 2||N = 27||7th||1||Photosynthesis and respiration
|Trial (2016)||School 2||N = 26||7th||1 (same as second trial)||Photosynthesis and respiration
School and teachers
The study was conducted in two different schools in Copenhagen, Denmark, with students from similar socio-economic backgrounds. Three biology teachers were selected based on their willingness to participate in the project. The first iteration of the design involved two biology teachers at the first school, whereas the second and third iterations were conducted with a single biology teacher at the second school. All three teachers had four to five years’ experience teaching science subjects. However, none had any specific previous experience with teaching about digital multimodal representations in biology.
Sixty eighth-grade students participated in the first iteration of the design, 27 seventh-grade students in the second iteration, and 26 seventh-grade students in the third. In Denmark students start having biology from the beginning of the seventh grade. The average age of the students was 14-15.
The first iteration was tested at the first school during a biology course based on a previous unit concerning protein synthesis and basic genetics. This led to another course on microbiology, which entailed some related laboratory work during which students studied microbial growth and hygiene and designed their own experiments. The students used various digital products to represent their results and the process leading to them, and were also tasked with justifying their choices of representational forms. The transition between the two courses as well as the subsequent practical laboratory work and generation of digital multimodal products was planned in collaboration between teachers and researchers and followed preliminary design principles inspired by the theoretical framework. The second and third iterations focused on basic concepts of photosynthesis and respiration, including a micro-container experiment and graphing as well as the photographing of chloroplasts and stomata openings on plant leaves by means of microscopes and mobile-phone cameras. The setup was planned in collaboration between the teacher and the researchers and based on the experience gained from the previous iterations. Thus, the content of all three iterations was based on the already planned curriculum, while the actual design and implementation took place in dialogue between teachers and researchers.
For this study, data were collected during classroom observations and subsequent interviews with students and teachers. The researchers were participant observers throughout the research period, which amounted to a total of 64 hours divided over 16 days. The classes were video recorded, and individual students were audio-recorded during classwork as a basis for subsequent analysis. After each iteration, students’ digital multimodal representations were collected for analysis.
The following data were used:
- Field notes were made for every single classroom visit. These included descriptive notes about the activities in the classroom and reflective notes about the design and special contextual factors. Video recordings and sound recordings were used as backup and to further elaborate field notes.
- Formal and informal interviews were conducted with students and teachers. After each teaching session, the researchers interviewed the teachers about the design’s viability and legitimacy. After the completion of the education process, eight students from each of the three classes were interviewed. These 24 interviews followed predesigned protocols and were based on artefacts used in teaching as well as the specific representations that the students had engaged with.
- Student products and student digital multimodal representations were collected.
Data analysis focused on the correlation between contextual factors and the use of the concrete design. During the three iterations, the analysis aimed to generate knowledge that could help reduce the variations between the intended, implemented and realized design. Therefore, the design was tested in practice in every iteration, then evaluated, analysed and redesigned. This enabled us to see what did and did not work, what could be improved and whether the design could be applied in other contexts.
The interviews with the students were transcribed and coded in relation to their understanding of the scientific concepts they had encountered, their awareness of the affordances of the different representations and their reflections on the use of digital multimodal representations.
Ultimately, the students’ digital products were analysed and evaluated with regard to the efficacy of the design. This analysis was based on diSessa’s assessment criteria for representations in science: adequacy, relevance, comprehensibility, clear connection between the various components of the representations and link to overall concepts and accepted convention (diSessa, 2004).
Results and discussion
The findings of this study are three design principles that appear to enhance student awareness of the affordances of different representations. The design principles are aimed to apply beyond the local context of this study as a part of a natural science culture. The study additionally uncovered some challenges associated with the design principles, which will also be discussed.
The three overall design principles are shown in Figure 1 below. The next sections describe how the design principles were tested and modified through the three iterations, and the design principles are discussed in relation to the theoretical framework.
First design principle
The first design principle is to organize student activities and dialogues that will enhance awareness of the affordances provided by the different forms of representations. Thus, the first activity conducted focused on what the different forms of representation have to offer.
For all the iterations, various representational forms regarding the given topic were distributed to groups of students. In the part of the first iteration concerning protein synthesis the representational forms were a text, a visual representation and an animation about protein synthesis. During this teaching session, the class was divided into eight groups, with each group being given only one specific representational form. Representatives from the individual groups then met with representatives from the other groups and collectively attempted to describe how their representational mode contributed to their understanding of the biological process and made an effort to combine all the representations into a coherent hole. This resembles the critical constellations approach suggested by Airey and Linder (2009). At the end of this activity, all participants in the session followed up with a plenary meta-discussion about the affordances inherent in the various representations, with the students’ personal reflections and recognitions forming the basis for the talk.
The result of this initial exercise is supported by data from the first iteration, where the subject on which this activity was based was protein synthesis. In the collective dialogue, students commented on and compared the possibilities and limitations of the various representations as follows:
If you had the video, it might be a little hard to see why something really is happening, but you just see that something happens in the video. But if you have this text, it may be that it justifies some of the differences…
The text can describe things specifically and give details, provide some academic concepts, describe precisely what happens.
However, a student also said:
It was a little difficult to understand everything that was written, because sometimes it can be difficult to visualize … That’s maybe the disadvantage of the text.
With regard to the visual representations, the students expressed the following sentiments, among others:
Illustrations can do things with colours, sequences and what is highlighted and stuff like that.
The disadvantage is that if I hadn’t known it was about protein synthesis, I wouldn’t have known what it depicted other than something about DNA.”
As for the animations, the students noted, for instance:
The animation is easy to understand; it showed exactly what was happening and when.
However, another student said:
The animation is good if you are very visual, that is, if you need to see the things visually … I think the animation really needed some text, because if it hadn’t said that the ‘Pac-Man’ there was an enzyme, I certainly wouldn’t have grasped that that was what was happening.
In the subsequent plenary discussion, the students and teachers concluded that it was precisely the combination of the various representational modes and their respective affordances that generated a true understanding of the academic topic – protein synthesis – in the best possible way. To a high degree, the exercise proved to heighten the students’ awareness of the affordances provided by the individual representational modes. Moreover, each representation was categorized and the affordances were discussed on the basis of our classroom observations as well as our individual interviews with students. Here we were interested in how the discussion itself could raise awareness of the affordance of each representation. However, some of the students experienced this activity conducted under the first design principle as rather complex, and despite gaining a greater understanding of the differences between the affordances of the chosen representations, the students had a less firm grasp of the scientific content. Furthermore, the activity connected with the microbiology unit failed to guide the students into the new content area, and they paid only limited attention to the complexity of each representation in their final products.
Since the above-described activity helped to sharpen the students’ awareness of the affordances of the various forms of representations, the activity remained part of the design for the second and third iterations, where the scientific content was photosynthesis – the topic on which the representations presented in the introductory activity were based. At the same time, however, a greater focus was put on mathematical representations since these were absent during the first iteration. Thus, the activity of building understanding from multiple forms of representation gave students an understanding of the affordances of the various modalities, but proved a difficult way into the scientific content.
Second design principle
The second design principle consolidated through the study is that students learn to use representations that show data for meaning making, such as graphs and charts, through their own thorough practical exercises and dialogues. This activity was included in all three iterations, although with an increasing focus on scaffolding the students’ practical work and their choices of representations.
The analysis of the student products from the first iteration working with microbiology revealed that the students almost exclusively used visual representations, text and film clips in their digital representations. Only a single group out of 14 used a chart to document their work, see Figure 2.
Although using schemes and diagrams to represent data from practical work might seem obvious, only a single group did this. Typical scientific representations like graphs and charts were thus absent and therefore not among the students’ representational competencies. This could be because graphs and charts were not included in the introductory activity of the first iteration, so the students did not have these representations fresh in their memories and had not explicitly expressed their personal strengths and weaknesses. Consequently, a graph was included in the initial activity of the second and third iterations. During these iterations, students measured carbon-dioxide levels produced and absorbed by a plant in a closed container exposed to various light intensities, and then used the measured data to produce their own representations of the data. However, only one group included a graph among other representations in the second iteration as shown in Figure 3, while the other 6 groups only used, verbal, text and a few pictures to support their work.
The graphs show how exposure to high light intensities lowers the carbon-dioxide content and increases the oxygen content of the container due to the photosynthetic activity of the plant.
Because only one group in the second iteration had included a graph, the work with the graph was scaffolded to a greater extent in the third iteration. In the initial dialogue about the affordance of the graph, clear parallels were drawn to students’ knowledge of graphs from their maths instruction. During these plenum discussions, students were able to explain what a graph was and to state, e.g.:
A graph can show how things develop over time.
However, they found it difficult to transfer this knowledge from maths and use it in the field of science, for which reason the teacher was in close dialogue with the students during the practical work. Nevertheless, only two groups out of eight used the graph as a digital representation in their digital representation in the third iteration.
During the interviews after each iteration, students clearly stated that it was difficult to comprehend and use the graphs in their final digital representations. All the different representations were available for the students to look at during the interviews, and they were challenged to explain the meaning of each representation in a sequence of their choice. They tended to choose the least complex representations first, i.e., those that were relatively concrete, like the visual representation, and only when encouraged did they make an effort to explain the more abstract ones, such as the formula and the graph. Only one student pointed at the graph and explained:
Initially, it increases [the carbon-dioxide level], so the plants are doing more respiration than photosynthesis, then it begins to decrease rapidly, because, it takes it in, and then begins to produce more oxygen.
In summary, it requires very close scaffolding and repetitive exercises for students to learn to represent data from practical research in science by using representations like graphs and charts. This confirms the findings of Airey and Linder (2009), who have described how students need to first become fluent in using each of the representations of a critical constellation before they can represent scientific meanings across a suitable range of modes.
Third design principle
The third design principle is that students produce digital multimodal representations as expressions of their learning and provide reasoning for and evaluate such representations by using known assessment criteria for multimodal representations in science. Thus, after preparing the digital multimodal representations, the students presented their products to each other and reflected on their choices of representations. During the first iteration, students’ considerations about and reasons for their choices were very sparse. In the second iteration, the students became more aware of the different representations used in science, as they were presented with di Sessa’s assessment criteria for scientific representations before beginning to develop their own digital multimodal representations. These criteria, which include adequacy, relevance, comprehensibility, clear connection between the various components of the representations and a link to overall concepts and accepted convention, were carefully described to the students and exemplified. The criteria have since functioned as analysis parameters for assessing students’ digital multimodal representations as expressions of learning and thus for determining the validity of the design. Moreover, the students subsequently used the assessment criteria as a basis for feedback and self-assessment in relation to their products. This led to a few more explicit considerations, but the reflections remained consistently quite limited.
Based on the results from the second iteration, a storyboard was developed to increase the scaffolding for the students and to enable the students to plan their digital multimodal representations and then reflect on and justify their choice of representations, see Figure 4 below.
The results described below refer to students’ digital multimodal representations from the third iteration, which include eight products, all of which are screen recordings. In terms of diSessa’s first assessment criterion, “adequacy”, overall students’ digital multimodal representations show a sufficient understanding of the basic concepts of photosynthesis and, to some extent, respiration when compared to curriculum goals; see Figure 5. Thus, the students’ expression of learning legitimizes the design, and is also triangulated by the subsequent student interviews, where the majority of the students were able to account for the processes taking place by using the concepts at a reasonable level. The student interviews also revealed that the students themselves had learned from the representations:
I feel like you’re getting a better understanding of what happened during respiration and photosynthesis than if you only read about it. When you had to explain yourself with a project.
The content of students’ digital multimodal representations is generally relevant, and most of it is explained and represented comprehensively, as can be seen in the example below, where students use relevant representations, such as photos, a model, a formula and a graph. The students explain the content with relevant concepts.
In summary, on the basis of the assessment of digital multimodal representations and of the student interviews, it was determined that the students received a reasonable academic benefit with regard to the photosynthesis and respiration requirements set out in their curriculum and to their ability to correctly use the concepts.
As regards the students’ representational competency, all groups had a clear connection between their speech and other representations in their screencasts. In the above example the students make red marks during the relevant parts of the speech in reference to the screencast. However, the logical coherence when it came to how the students ordered the screencast representations varied, as did the cohesion of the the overall concepts of photosynthesis and respiration presented. In the above example, the students succeed in creating an overall coherent whole by asking an initial question that they then tried to explain and answer during the screencast. From 5 B-5 D the students provide a coherent speech based on the photosynthesis and respiration content. However, the picture in 5 F is out of place, as the uptake and excretion of CO2 and O2 take place in figure 5 B. The cohesion from 5 D to 5 E is supported by the use the word “therefore”.
Apart from a single group, all students worked within accepted conventions with authorized models and concepts in their digital multimodal representations as expressions of science content. When presenting the multimodal digital products in the third iteration, the students were able to justify their choices of digital multimodal representations. Their reasoning concerned not only their choices of representations but also how they were mediated. Representation relates to the sign-maker’s intention, but communication concerns the sign recipient’s interest. Despite focusing on the representation of content during the course, many students also motivated their choices with reference to the recipients. For example, recipients find visual representations easier to understand.
The fact that the students had few reflections on their choice of representations could also be interpreted as indicating that they were experiencing a cognitive overload. Photosynthesis and respiration as content are difficult and abstract in themselves, which perhaps also made it more difficult to meta-communicate about representations. The subsequent student interviews conducted two months after the third iteration substantiate the difficulty of reflecting on different representations. However, some students did become aware of representations:
So, I think that when you work with these different representations, you begin to think more about what they could do, these forms of representation, why we should choose this one specifically… it’s so much about what it can do.
In summary, when one analyses how correctly students use concepts in their digital multimodal products, as well as the meaning they make of them, the activity of producing digital multimodal re-representation related to biological processes and concepts appears to strengthen the students’ learning. In addition, highlighting specific assessment criteria and exemplifying them helped students to understand and complete their work. The communication perspective must also be taken into account when one assesses student reflections on their choice of representation. Finally, representational competency is not achieved during six weeks.
Research in using representations for teaching and learning in science reveals that transforming representations and producing multimodal representations can strengthen students’ potential for learning concepts. Digital media provide new and easily accessible options for re-representing content. However, work with such media is only sporadic in Danish primary and lower-secondary schools. Against this background this study has sought to develop a didactic design that can introduce and start to increase students’ awareness of the conceptual affordances provided by representations and thus enhance their production of digital multimodal representations as an expression of learning.
Throughout the project, a sketch for such a didactic design was tested in three iterations and further developed into three design principles:
- The first design principle is to organize activities and dialogues among the students that will enhance awareness of the affordances provided by the different modes of representations.
- The second design principle consolidated through the study is that students, through their own thorough practical exercises and dialogues, learn to use representations that show data, e.g. graphs and charts.
- The third design principle is that students produce digital multimodal representations as expressions of their learning and reflect on and evaluate these on the basis of known assessment criteria for multimodal representations in science.
Regardless of context, the initial activity strengthened the students’ awareness of the affordances of representations. However, the activity constituted an abstract approach to the content, for which reason more easily accessible content should be considered as an alternative when the design is tested in new contexts. The design could also facilitate the students in acquiring scientific knowledge, while also enabling them to meta-reflect on representations. Students that are not used to representing data from their own practical work need thorough scaffolding and gradual training in working with representations like graphs and charts.
The study has provided three design principles for introducing multimodal representation as part of science culture. However, for students to develop their representational competency, they must work continuously with representations in science. To this end, repetition can serve as a means of developing both fluency in the individual representations and in transforming and transducting information between representations (Airey & Linder 2009, 2017).
The authors have no competing interests to declare.