In recent years, the digital development has been very strong within schools (OECD, 2015) encompassing for example the use of digital teaching materials. Such materials provide new potentials in teaching, for example by the use of animations and interactive features. The new technology has shown a large impact on learning situations and teachers work, for example because digital teaching material can support new forms of individualised learning and interaction with new types of resources (Utterberg, Tallvid, Lundin, & Lindström, 2019; Pepin, Choppin, Ruthven, & Sinclair, 2017). Based on a comparison between mathematics text in digital environments and texts in print, Usiskin (2018) concludes that the digital environment is well suited for the display of several different representations, especially because of the dynamic space allowed on screen. Digital teaching materials enable opportunities to include new multimodal resources and to organise mathematical information in new ways, by linking to explanations, definitions, examples and tasks that can be shown or hidden (O’Halloran, Beezer, & Farmer, 2018). These possibilities place new demands on the students, something that must be addressed in teaching.
A deepened understanding of the gains and losses related to the use of digital teaching materials is crucial since knowledge about design and interaction with such texts^{1} in mathematics do not keep pace with the digital development. The new features of digital teaching materials mean that analysis models intended for analyses of printed material are not applicable without adjustments and entirely new models are needed. One model for describing the new opportunities provided by digital materials was developed by Pohl and Schacht (2017) who describe a broad variety of dynamic features unique to exercises in digital teaching materials and analyse students’ reactions to different options in the material. The opportunities of the digital environment and how students interact with the material are further analysed by O’Halloran, Beezer, and Farmer (2018). Their method makes it possible to characterize the structure of the material including chapters with sections, subsections and their elements and to relate it to students’ interaction with the material. In comparison to these previous frameworks, our model takes another approach by combining a description of the mathematical meaning offered by different textual resources and the use of the dynamic functions provided by the digital media. The textual features, highlighted in our model, are chosen based on research in mathematics education. These features, which are described in the theory section below, provide different means to express mathematical content to the students.
Consequently, the aim of this article is to describe an analytical model and its application in analyses of how different features of multimodal resources are used in digital teaching material in mathematics. Especially, the model enables an interpretation of how various design features may alter the meaning offered by the material.
The model rests on a social semiotic perspective (see Kress & van Leeuwen, 2006; O’Halloran, 2005). An important point within this perspective is that the chosen multimodal expressions direct the realised meaning. By studying the textual features expressing a content, it is possible to gain understanding of the meaning offered to the reader. From the social semiotic perspective, all instances of communication are constituted by three metafunctions: ideational, interpersonal and textual function (Halliday & Matthiessen, 2014). The textual function concerns how the content is organized in a text. The interpersonal metafunction captures the relation between the reader and the text. The ideational function concerns how the text is logically built up and how the essential ideas are expressed.
Based on social semiotic theory and the metafunctions, we have developed four analytical layers, constituting a coherent whole (see Figure 1). The textual features included in each of the four layers in the model, are chosen because they are highlighted in research in mathematics education as essential for the realization of the mathematical content. These features and their functions for mathematics education are further described below.
The analytical layers are presented separately, but since the metafunctions are closely intertwined and interact in expressing the content, all layers need to be considered in combination, as visualised by the arrow in Figure 1.
The analysis question addressed in the base layer (textual metafunction) is: Which semiotic resources, modes, and dynamic functions are utilized to express the central aspects of the targeted mathematical concept?
The base layer is the foundation in the model by a characterisation of different constituents and their roles in the text. Each element (a coherent part, distinguished e.g., by empty space) in the text is categorised in terms of semiotic resource, mode, and dynamic function (Figure 2). In this model, semiotic resources are defined as natural language, mathematical notation and images in accordance with O’Halloran (2005). The use of different semiotic resources is essential in mathematics since they can express different types of mathematical meaning. For example, natural language is a very poor resource for formulating quantity, continuous covariation, and gradation (Lemke, 1998). Besides the variation in semiotic resources, the dynamic environment provides opportunities to use these different semiotic resources in various modes. Mode is the channel used to offer meaning, words can for example be communicated via sound or writing. In the digital media the dynamic function, a variety of opportunities to interact and of change in relation to time, is still another resource possible to utilize to convey a particular message. Accordingly, these options mean there is a huge variation in how meaning can be offered. Digital teaching materials also enable opportunities to organize mathematical information in new ways, by linking to explanations, definitions, examples and tasks that can be shown or hidden (O’Halloran et al., 2018). In our model, an emphasis is laid on the dynamic opportunities offered in the material (the five element types, Figure 2).
The analysis question addressed in the relational layer (interpersonal metafunction) is: How is the relation between the reader and the text realised, regarding to what extent the students are invited to act as active participants in creating the mathematics?
In this layer, the model captures the relation between the reader and the text, based on acts of communication and coding orientation (defined after Table 1). The focus is laid on the central aspects of the mathematical content identified in the base layer, and in what way these aspects have been expressed, and thus made available for students.
Analytical focus  Textual focus  Textual features 

Relation between reader and central aspects in the text  Acts of communication  Claims

Coding orientation  Naturalistic


Originally, four speech acts: claims, questions, offers and prompts, were described by Halliday and Matthiessen (2014). These speech acts have been developed to acts of communication, for embracing analysis of multimodal texts where different semiotic resources are combined (Björkvall, 2009). In digital teaching material, the potential to engage in questions and prompts is expanded due to dynamic features, something that is captured in our model when the layers of the model are combined. This part of the model provides a structure for an overview of (central aspects in) communicative acts in each element.
Included in the relational layer is also coding orientation (Kress & van Leeuwen, 2006). Coding orientation captures the range between naturalistic and technical sense. Based on the coding orientation, different types of truth claims are made. In scientific contexts, for example, an abstract and decontextualized image is attributed higher reliability compared to an image expressing a naturalistic representation of a situation. Coding orientation has mainly been used as an analytical approach when it comes to images, but in this model, we extend the concept to embrace also natural language and mathematical notation. Different types of coding orientation with varying degrees of abstraction, such as technically oriented diagrams or naturalistic images providing a context, are used in teaching materials in mathematics and are of utmost importance since they have different functions to express the content (Bergvall, 2016). A context can for example describe situations familiar to the student indicating that mathematics is something that we interact with in everyday life.
The analysis question addressed in the process layer and logical layer (ideational metafunction) is: How is the mathematics ‘brought out’ to the reader in terms of what the text suggests mathematics is about experientally, through the types of processes that are present, and logically through the logical networks of information offered within and between elements and parts of the text.
The process layer illuminates how the nature of mathematics is presented to the students. The model provides a categorisation of the central aspects as either part of operational processes mirroring a view of mathematics as constructed by doing, or as part of relational processes mirroring mathematics as a system of relationships between objects. In mathematics education, there has been a great emphasis on the importance of understanding both operational and relational processes (Sfard, 1991). In the development of conceptual understanding, operational conceptions most often precede relational conceptions, and are likely to be less highly valued than the relational ones as those conceptions may be seen as representing an earlier stage of thinking. These two approaches are however prerequisites for each other and it is often necessary to switch between these two ways of perceiving mathematics, an often difficult step for the student (Sfard, 1991).
Operational and relational processes can be expressed in different ways with different semiotic resources (see Table 2).
Operational process  Relational process  

Indicates that^{a}  Math is constructed by doing  Math is a system of relationships between objects 
Natural language^{b}  Describes how something is done, or how something happens. Process verbs such as

Describes how something is, or properties something has. Relational verbs such as

lmages^{c}  Narrative temporal order

Conceptual tree structures taxonomy

Mathematical notation  expressing a process: 3 + 8 =_  expressing a relation: 3x = 15 
The logical layer captures the logical networks of information around the central aspects, in and between static and dynamic elements (see Table 3). These networks of information are constituted by an expression of the targeted concept, introducing a theme relevant throughout the section, and various types of expansions of the concept. Mathematics text is characterized by structures of logical relations that are realised by means of the logical metafunction (e.g., see O’Halloran, 2005) and the digital media enable new types of connections to be made. These types of meaning relations are essential to include in analyses of mathematics text because the ability to correctly decode such relations is part of a mathematical competence (e.g., see Dyrvold, 2020).
Analytical focus  Text focus  Textual features 

Logic in the text  Expansion  Logical

Expansions often mean that new central aspects of the concept are presented. The expansions can be logical, additional, or constituted as an extension of the concept (see also van Leeuwen, 2005).
This section presents an analysis based on the model. The analysis of the introduction of the concept proportionality in a digital teaching material for grade 9, highlights how variations in use of resources affect the offered network of information. Two theory parts from the same digital teaching material are included; one page with no dynamic elements (referred to as ‘static page’) and a film (‘film’). In the digital teaching material, the two parts are simultaneously available and the students can choose to read the ‘static page’, or to watch the ‘film’, or both. The static page is an example of the first element type using solely static functions whereas the film is an example of the third element type, and the content is dynamically presented over time. Both parts explain proportionality, presenting a relation between weight and cost using a table with coordinates, and a corresponding graph.
The analysis at the base layer highlights dynamic aspects that are unique for the ‘film’: sequentially occurring images, colour marks highlighting important details, words and voice over (also typed). Figure 3 visualizes a simplified representation of the elements in the ‘static page’ and the ‘film’.
The analysis at the relational layer reveals a difference between the parts regarding coding orientation; the ‘static page’ expresses the concept by a technical coding orientation whereas also instances with naturalistic coding orientation occur in the ‘film’. A naturalistic coding orientation involves expressions relating to the students’ reality and is often used in mathematics textbooks to guide the student from operational processes to an understanding of more advanced relational processes.
The analysis at the process layer reveals that the theory as ‘static page’ (identified having mostly technical coding orientation) utilizes mostly relational processes. Relational processes are realisations of structures and relations in mathematics, indicating a higher level of abstraction, which according to mathematics education research, often is preceded by and developed from an understanding of mathematics as operational processes (Sfard, 1991). In contrast, an operational process is utilized by a voice expressing the concept in the ‘film’.
It is striking that the dynamic features of the ‘film’ add value because the operational guidance given by the voice allow the reader to strictly focus on the technical content given visually, while instant support is given audible. The analysis at the logical layer, (Table 3) further highlights the significance of the voice because it expands all central aspects in the theory by means of adding, logical expansion and extension.
A combination of the analyses at the separate layers reveals that the voice in the ‘film’ provides an operational and more naturalistic explanation in addition to the visual content in the ‘film’, which gives the opportunity to purify the relational and technical in the visual content. By this means, the ‘film’ guides the students from the accessible operational process and naturalistic coding orientation, to an understanding of the relational process and technical content. This guiding is not available in the ‘static page’, and an active choice to watch the film is a prerequisite for receiving this support. The option to choose between the ‘film’ and the ‘static page’ may seem trivial because the mathematical content appears the same, but the analysis reveals differences in the offered meaning, due to utilization of digital resources. These differences may affect the accessibility of the mathematics to the reader.
In this article, we describe a model for multimodal analysis of digital teaching materials in mathematics, developed from social semiotic theory. The model is suggested as a tool to analyse the mathematical meaning offered by digital teaching material, and how students, by static and dynamic elements in these materials, are invited to engage with the content. An analysis where two parts from the same digital teaching materials are contrasted, a static page and a film about the same theory with sequentially occurring images, demonstrate the potential of the model. The example analysis shows the extent to which student are allowed to create their own text by means of the choices possible in the dynamic environment. This possibility for the reader to choose reading paths is an asset since such choices adapt the text to the students’ various needs and previous knowledge. The model also enables an investigation of whether there is a risk that the student by one sided choices in the reading only get to engage in text that might be very naturalistic or very technical, or that very deliberate choices are needed to engage in a technical discourse. Analyses based on the model have the potential to reveal gains and losses in how the dynamic function in the teaching material adds or hides interpersonal or logical relations that can support learning. In conclusion, the model has an important role to play since there is a great need for increased knowledge about digital teaching materials in mathematics, not least considering the increasing use of distance learning, and the model has potential for comparisons and evaluation of the meaning offered to the students by these teaching materials.
^{1}A comprehensive definition of text including multisemiotic (e.g. also images) and multimodal (e.g. also sound) resources is used.
The authors have no competing interests to declare.
This work was supported by the Swedish Research Council [Grant no. 201905005].
Alshwaikh, J. (2011). Geometrical diagrams as representation and communication: a functional analytic framework. Institute of Education, University of London.
Bergvall, I. (2016). Bokstavligt, bildligt och symboliskt i skolans matematik – en studie om ämnesspråk i TIMSS. (PhD dissertation). Uppsala.
Björkvall, A. (2009). Den visuella texten: Multimodal analys i praktiken. Stockholm: Hallgren & Fallgren.
Dyrvold, A. (2020). Relations between semiotic resources in mathematics tasks: A source of students’ difficulties. Research in Mathematics Education, 22(3), 265–283. DOI: https://doi.org/10.1080/14794802.2019.1689160
Halliday, M. A. K., & Matthiessen, C. M. (2014). An introduction to functional grammar (4 ed.). London: Arnold. DOI: https://doi.org/10.4324/9780203431269
Holmberg, P., & Karlsson, A. (2006). Grammatik med betydelse: En introduktion till funktionell grammatik. Uppsala: Hallgren & Fallgren.
Kress, G. R., & Van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). London: Routledge. DOI: https://doi.org/10.4324/9780203619728
Lemke, J. L. (1998). Multiplying Meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading Science (pp. 87–113). London: Routledge.
Morgan, C. (1998). Writing mathematically: The discourse of investigation. Bristol, PA: Farmer Press.
OECD. (2015). Students, Computers and Learning: Making the Connection. PISA, OECD Publishing. DOI: https://doi.org/10.1787/9789264239555en
O’Halloran, K. L. (2005). Mathematical discourse: Language, symbolism and visual images. London: Bloomsbury UK.
O’Halloran, K. L., Beezer, R. A., & Farmer, D. W. (2018). A new generation of mathematics textbook research and development. ZDM, 50(5), 863–879. DOI: https://doi.org/10.1007/s1185801809598
Pepin, B., Choppin, J., Ruthven, K., & Sinclair, N. (2017). Digital curriculum resources in mathematics education: foundations for change. ZDM, 49(5), 645–661. DOI: https://doi.org/10.1007/s118580170879z
Pohl, M., & Schacht, F. (2017). Digital mathematics textbooks: Analyzing structure of student uses. In G. Aldon & J. Trgalová (Eds.), Proceedings of the 13th International Conference on Technology in Mathematics Teaching/ICTMT 2017 3rd to 6th July, 2017 (pp. 453–456). Lyon, France.
Sfard, A. (1991). On the dual nature of mathematical conceptions: reflections on processes and objects as different sides of the same coin. Educational Studies in Mathematics, 22, 1–36. DOI: https://doi.org/10.1007/BF00302715
Usiskin, Z. (2018). Electronic vs. paper textbook presentations of the various aspects of mathematics. ZDM, 50(5), 849–861. DOI: https://doi.org/10.1007/s1185801809362
Utterberg, M., Tallvid, M., Lundin, J., & Lindström, B. (2019). Challenges in Mathematics Teachers’ Introduction to a Digital Textbook: Analyzing Contradictions. Journal of Computers in Mathematics & Science Teaching, 38(4), 337–359.
Van Leeuwen, T. (2005). Introducing social semiotics. London: Routledge. DOI: https://doi.org/10.4324/9780203647028