Start Submission Become a Reviewer

Reading: Transduction and Science Learning: Multimodality in the Physics Laboratory


A- A+
Alt. Display


Transduction and Science Learning: Multimodality in the Physics Laboratory


Trevor Stanton Volkwyn ,

Uppsala University, SE; University of the Western Cape, ZA
X close

John Airey,

Stockholm University; Uppsala University, SE
X close

Bor Gregorcic,

Uppsala University, SE
X close

Filip Heijkenskjöld

Uppsala University, SE
X close


In this paper we discuss the role of transduction in the teaching and learning of science. We video-filmed pairs of upper-secondary physics students working with a laboratory task designed to encourage transduction (Bezemer & Kress, 2008). The students were simply instructed to use a hand-held electronic measurement device (IOLab) to find the direction of the Earth’s magnetic field and mark its direction using a paper arrow.

A full multimodal transcription of the student interaction was made. In our analysis of this transcription we identify three separate transductions of meaning. In particular, we observed that student transduction of meaning to the paper arrow allowed it to function as both a persistent placeholder for all the meaning making that had occurred up until that point and as a coordinating hub for further meaning making.

Our findings lead us to recommend that teachers interrogate the set of resources necessary for appropriate disciplinary knowledge construction in the tasks they present to students. Here, teachers should think carefully about whether the introduction of a persistent placeholder would be useful and in that case what this placeholder could be. We also suggest that teachers should think about what persistent resource may function as a coordinating hub for the students.

Finally, we suggest that teachers should be on the lookout for student transductions to new semiotic resources in their classrooms as a sign that learning is taking place. We claim that the constraining and complementary nature of transduction offers a good opportunity for teachers to check student understanding, since disciplinary meanings need to be coherent across semiotic systems (modes).

How to Cite: Volkwyn, T. S., Airey, J., Gregorcic, B., & Heijkenskjöld, F. (2019). Transduction and Science Learning: Multimodality in the Physics Laboratory. Designs for Learning, 11(1), 16–29. DOI:
  Published on 26 Mar 2019
 Accepted on 08 Feb 2019            Submitted on 28 Jun 2018


In this paper we discuss the role of transduction in the teaching and learning of science. We start out by discussing the function of physics devices and the role transduction plays in their operation. We then go on to illustrate the role that transduction can play in the teaching and learning of physics, both with and without the use of physics devices. Following Bezemer and Kress (2008), we define transduction as the movement of semiotic material from one mode (or semiotic system) to another. When semiotic material is moved from one semiotic system to another in this way, a number of changes occur. These changes radically alter the meaning potential of the transducted material. For an everyday example, consider the simple sentence “The man moved out of the way”. Here, we do not know anything about the direction that the man moved. However, when we transduct the meaning of this sentence to a diagram such ambiguity is no longer possible—a decision needs to be made about how to represent the movement of the man and this will necessarily entail a choice between the man moving left, right, backwards, forwards, ducking down, leaning to one side, etc. In this paper we will illustrate the ways in which physicists tacitly use the changes in meaning potential that transduction entails to both do physics and teach physics. The data consists of video recordings of pairs of high school students working with an open-ended laboratory task. Analysis of a multimodal transcription of this data led to us identifying three ways in which transduction was leveraged to make physics meanings. We argue that in moving from a tacit to an explicit understanding of the role of transduction, teachers can better understand the processes at work in their classrooms and will thus be better equipped to both assess and affect their students’ learning. The article is an elaborated version of our presentation given at the 8icom conference in Cape Town in 2016 (Volkwyn, et al., 2016).

Transduction in Physics—the role of the device

The ultimate goal of the discipline of physics is to describe the behaviour of the universe and its contents. Physicists do this by interpreting the semiotic material around them—in effect, treating the environment as the origin of signs about the world. Historically, physicists were limited to direct observation—that is, they could only work with the input from the environment that was directly available to their senses. Today however, most experimental physics is carried out by means of mediated observation, that is, by using apparatus of some sort. Physicists have at their disposal a vast array of devices specifically developed to help them interpret environmental input. At the most basic level these physics devices can fulfil three functions—they can intensify, filter and transduct the meaning potential in the environment.


The first function a physics device can perform is to intensify a signal in order to make it available to our senses. An optical telescope is a classic example of a tool that intensifies a signal; for example, enabling physicists to see the rings of Saturn that are not visible to the naked eye.


The second function a physics device can perform is to select input of interest. This process can be used to separate out certain input from other unwanted information. One everyday example is the way that polaroid sunglasses allow us to see the bottom of a swimming pool by suppressing the light reflected from the surface of the pool.


The third function a physics device can perform is to transduct. This is the focus of our paper. As we mentioned earlier, transduction is defined as the movement of semiotic material from one system to another (Bezemer & Kress, 2008). Devices that transduct have been designed to receive environmental input in a form not available to our senses and change it to one which is. One well-known example of this function is the Geiger counter for detecting radioactivity—it transducts information about invisible radiation to an audible click—the more frequent the clicks, the higher the level of radiation.

The majority of physics devices actually perform a mixture of these three functions, often in sequences or chains.

Interpreting the environment

In order to better understand the transduction carried out by a physics device it is useful to consider how humans make meaning from their environment without a device and then in turn communicate that meaning to others. In Figure 1, an individual sees marks on the ground. This individual already has a clear idea about what a deer is, the way it looks, its habits, etc. so when this person sees the marks, these are interpreted as deer tracks—signs that a deer is present. From a semiotic viewpoint, these tracks are interesting because although they are quite clearly a sign of a deer, there is no intentionality in this sign making—the deer was not attempting to communicate anything through its tracks. Meaning is created solely by the interpreter.

Figure 1 

An individual making meaning from environmental input.

Clearly this is always the case with signs—strictly speaking, meaning does not inhabit the sign itself, but rather is always assigned ‘on the fly’ in the process of interpretation.

In semiotics we have become accustomed to dealing with communication between individuals. Here, it is quite usual to talk about the interests of the sign maker and this is clearly particularly pertinent in educational settings. In a given social setting, an individual makes a sign by first deciding which aspects of an ‘object’ are relevant to communicate. The individual then chooses between a range of available resources within the social setting, selecting a resource deemed apt to represent some of the pertinent aspects of the object. Note that this selection process is often tacit, however analysis of the resources used still has the potential to reveal the interests of the sign maker, even though the sign makers themselves may be unaware of the choices that they have made (see for example Reading Images, Kress & Van Leeuwen, 2006).

In Figure 2 we have a system of meaning making between two individuals employing a single sign—the word ‘deer’. As in Figure 1, the environmental input is interpreted by the first individual, but now it is transducted to the word ‘deer’. This spoken word is then interpreted by the second individual who has not seen the original tracks in the environment. Notice that there is always ambiguity and incompleteness in both the transduction and in the interpretation of the transducted sign. For example, we can imagine that the original environmental input may well have been interpreted by the first individual as a sign of a range of aspects, such as the kind of deer, size, direction of movement, time since the tracks were made, etc. This meaning is not transducted into the word ‘deer’. Similarly, the simple word ‘deer’ itself is ambiguous and can be interpreted in a number of ways. This has been denoted in Figure 2 by the different kinds of deer envisaged by the two individuals.

Figure 2 

Transduction of visual environmental input to a spoken word, which is then interpreted by a second individual.

Transduction to one sign alone can never mediate a full understanding of the original environmental input, rather a multimodal ensemble of signs is usually needed that leverages the generic affordances of different semiotic systems (modes) in order to approximate to the original environmental input. Meaning making is more likely to be successful (and require fewer signs) if the sign maker and the interpreter have shared experiences and are from the same social group. In such cases there will probably be a shared understanding of the particular interests of the group and the provenance (Mavers & Oliver, n.d.) of the signs produced, i.e. what they have been used to represent in the past (cf. Airey, 2014). In this respect, the interests of the sign maker are not totally unknown, the provenance of the signs and the interests of the social group mean that sign making and interpretation is a far from arbitrary process. However, the sign maker can never be truly certain that the intended meaning has been accurately interpreted, nor can the interpreter be fully confident that the understood meaning was indeed that intended by the sign maker (cf. the notion of language games in Wittgenstein, Anscombe, & Wittgenstein, 1963). This is because the meaning of a sign is not fixed, but rather can be thought of as a flexible resource for meaning making—meaning subtly shifts each time a sign is (re)produced (Van Leeuwen, 2005).

In this article we would like to point out that in physics (and science in general) we have an interesting, specialised form of the meaning making system described in Figures 1 and 2. In physics, devices have been purposely designed to generate specific signs from environmental input. Here a decision has already been made about which aspect or aspects of a phenomenon are of interest. Thereafter, a device has been purposefully created in order to detect these aspects and intensify, filter and/or transduct them. In Figure 3, for example, environmental input that is not available to the human senses (in this case an x-ray source from space) is first transducted by an orbiting telescope to a graphical readout. This graphical readout is then interpreted in terms of two stars rotating around each other in a binary system. The physics community has not only decided which aspect or aspects of the phenomenon are important, it has also decided how the signs created by the device should be interpreted (cf. our discussion of the provenance of signs in the previous paragraph). This means that compared with meaning making between two individuals, the possibilities for meaning making with devices within the discipline of physics are extremely constrained. However, it is this very restriction of possible meanings which allows physicists to make such powerful knowledge claims (see Ainsworth, 2006 for a discussion of the constraining and complementary roles of resources). For example, in Figure 3 the input from the environment has a myriad of possible meanings; however, the sign that is produced by the device is narrowly defined and carries a specific meaning for the discipline that is relatively unambiguous. Notice that it is the device that creates the sign from environmental input, and therefore semiotically the design of the device itself tells us in a much less ambiguous way about the interests of the device maker, that is, the interests of the physics community.

Figure 3 

Transduction with a device in physics.

Of course not all physics devices have been designed for the express purpose of producing a sign by intensifying, filtering and/or transducting a pre-existing signal from the environment. Clearly, many physics devices also produce signals that are sent out into the environment to generate a response. Nonetheless, the response signals received will still be intensified, filtered and/or transducted by a device and the signs so produced will still be interpreted following the praxis developed by the physics community.

So, transduction of environmental input represents one of the three main ways in which meaning is made using a device in physics. Clearly, however, for this kind of meaning making to be successful in the teaching and learning of science, students will need to come to understand two things: the interests of the scientific community with respect to the phenomenon at hand, and the particular ways in which the community has decided that the signs generated by the device should be interpreted i.e. the disciplinary affordance of the device (Airey, 2015; Airey & Linder, 2017).

Disciplinary and pedagogical affordance

In this paper we adopt a social semiotic approach to our data analysis. Social semiotics has been defined as “the study of the development and reproduction of specialised systems of meaning making in particular sections of society” (Airey & Linder, 2017: 95). Following Fredlund, Airey, and Linder (2012), Airey (2015) defines disciplinary affordance as “the agreed meaning making functions that a semiotic resource fulfils for a particular disciplinary community”. Airey (2015) introduces a further term, pedagogical affordance, which he defines as “the aptness of a semiotic resource for teaching some educational content”. The disciplinary affordance of any given resource is agreed within the physics community, but the pedagogical affordance of a resource will of necessity always be dependent on the individual learner.

In this paper we are primarily interested in leveraging these terms to describe the role of transduction in the teaching and learning of physics, in particular with respect to understanding the phenomenon of magnetic field.

The compass—a transduction device for magnetic field

Although migratory birds can sense the direction of the Earth’s magnetic field and use it for navigation, the same cannot be said of humans. As far as we know, humans cannot sense magnetic field without some sort of transduction device. Historically, the effects of magnetic field on naturally occurring magnetite—or lodestones as they were called—had been known for thousands of years. The first compass-like objects are thought to have been constructed as a device for divination by Chinese geomancers in the second century BC, by fashioning a spoon-like object from a lodestone (Needham, 1962). The ‘handle of the spoon’ always mysteriously pointed south. The modern magnetic compass is simply a development of this idea and is in essence a transduction device for magnetic field, however it is important that we remember the interests of the device maker here. Clearly, modern compasses have been designed for navigation across the surface of the Earth. This means that compasses do not actually show us the direction of the Earth’s magnetic field, rather they show us the direction of the component of the magnetic field along the surface of the Earth. The actual direction of the magnetic field depends on where we are on the surface of the Earth. The Earth’s magnetic field can be modelled in terms of a large imaginary bar magnet within the Earth approximately aligned from pole to pole (Figure 4).

Figure 4 

The Earth’s magnetic field modelled in terms of a large bar magnet within the Earth. Notice the compass needles are aligned with the imaginary field lines.

As can be seen from the imaginary field lines in the diagram (Figure 4), at the equator, the magnetic field does indeed point along the surface of the Earth, but as we move north, the direction of the magnetic field points more and more steeply into the Earth. Similarly, as we move south from the equator, the magnetic field points more and more steeply out of the Earth. Because of this, compasses are often balanced with small weights when they are manufactured so that they point along the surface of the Earth. Compasses are therefore often designed to function at a particular latitude. In our terms, compasses manufactured in this way perform two of the functions of physics devices we described earlier; they transduct magnetic field to a compass needle that we can see and they also filter out the vertical component of the magnetic field so that the needle points along the surface of the Earth. In physics, however, we are usually interested in the actual magnetic field in three dimensions. Thus, whilst the compass may have high disciplinary affordance for geographers it actually has low disciplinary affordance for physicists, because it does not show the true direction of the magnetic field. In our case we introduce a new device, the IOLab with the potential for high disciplinary and pedagogical affordances in the area of physics (see Airey and Eriksson, in review, and Airey and Linder, 2017, for a discussion of the interrelated terms of disciplinary and pedagogical affordance).

The IOLab—a pedagogical transduction device

IOLab stands for Interactive Online Laboratory. The IOLab is a generic pedagogical device containing a range of physical sensors such as an accelerometer, magnetometer, gyroscope, light intensity sensor, atmospheric pressure sensor, temperature sensor, etc. (see Figure 5). Using this device, students can investigate a wide range of physics phenomena. Since the device has generic pedagogical and disciplinary affordances, these need to be leveraged for teaching through the creation of a concrete learning task. (See Selen, 2013 for a discussion of the development of the device and a range of possible pedagogical uses). In our terms, the IOLab is an example of a multi-purpose, disciplinary-focused, transduction device where the interests of the device maker are pedagogical—that is the device has been made with the intention of teaching physics.

Figure 5 

The IOLab device. Note the printed sets of axes on the top and bottom surfaces. The USB stick is for wireless data transmission between the IOLab device and a computer.

As mentioned above, the IOLab contains a sensor (magnetometer) that is used to measure the strength of magnetic field in three dimensions. The IOLab-computer interface transducts this information to a graphical display in real-time producing three colour-coded plots (see Figure 6).

Figure 6 

A screenshot of the transducted, three-component-plot of the magnetic field on the computer screen as the IOLab is moved. In the learning sequence documented for this study our intention was for students to learn something about the Earth’s magnetic field by using this device.

Research questions

  1. How do pairs of students leverage the pedagogical affordances of a physics device (IOLab) when working with an open-ended task to determine the direction of the Earth’s magnetic field?
  2. What stages can be identified in this process in multimodal terms?
  3. What does a multimodal analysis of the students’ coordination of semiotic resources suggest about the role that transduction can play in the teaching and learning of physics?


In this study we gave pairs of high school students an open-ended task—to find the direction of the Earth’s magnetic field. The usefulness of such tasks in laboratory settings has been well documented (see for example Roychoudhury and Roth, 1996, and Etkina, 2015). Each student pair was provided with an IOLab that had been set up to display the readings of the inbuilt magnetometer on a laptop computer screen (see Figures 6 and 7). Three facilitators were also present in the laboratory to provide prompts and Socratic questions as needed. As part of the task, students were provided with a red paper arrow and asked to fix this to some surface in the room to show their findings. A total of six pairs were video recorded using fixed cameras and microphones on the tables. After reviewing the six recordings one pair was selected for further analysis and a full multimodal transcription of the engagement was created (Baldry and Thibault, 2006)—see the Analysis section for further details.

Figure 7 

Diagram of the classroom setup.


The study followed Swedish Research Council (2017) guidelines for research involving human subjects. Each student received an information sheet detailing the purpose of the study and an ethical consent form that was signed before the session commenced. Students were also provided with a copy of the form they had signed and contact details for the first author should they decide to withdraw their permission at any time. Ethical protocol information was also provided verbally. As stated in the previous section, the activity formed a part of a normal physics learning laboratory for senior high school students, so all student groups had equal access to the team of facilitators. Some groups were not comfortable with participating in the study, so we positioned the tables and cameras in the room to exclude these students from the video recordings.


Analysis started while the data collection was occurring. At that time, one of the facilitators directed our attention to one particular student pair where interesting meaning making events and learning appeared to be taking place. When we reviewed the video material from all six pairs the initial observations of the facilitator were confirmed—it was apparent that not only did learning take place in this particular pair, the students also articulated their learning at several stages. The interaction of this pair was therefore chosen for further investigation. This particular interaction consisted of approximately 80 minutes of video data, of which nearly 40 minutes was transcribed and analysed for this study.

The transcription process followed a multimodal approach to discourse analysis (MDA). MDA is now a well-established analytical tool which has its origins in systemic functional linguistics, discourse analysis and multimodality (an exemplary text setting out the various issues and aspects is Jewitt, Bezemer, and O’Halloran, 2016). The multimodal transcription carried out for this study draws on the work of Baldry and Thibault (2006), and Bezemer and Mavers (2011). In our case this multimodal transcription entailed carefully recording the relationships between the graphical output on the screen, manipulation and orientation of the IOLab box and the students’ and facilitator’s use of speech, gesture, and gaze.

The first author produced the transcript of the full sequence, and in the process made notes of what appeared to be key events. After this, all the authors individually viewed the video recording together with the transcript to identify what they thought were key events in the learning sequence. We then met to discuss which key events we had noticed and their possible interpretations. Systematically, in iterative cycles of critiquing and refining, the entire research group (including those not directly involved with the data collection and analysis) helped us to focus and refine our understandings of what was taking place.

In the next section, excerpts of the transcript (given in the Appendix) accompany diagrams as empirical illustrations of our conclusions about the multimodal sequence that occurred during the task. Naturally, we are not claiming that the other five student pairs followed an identical multimodal sequence. Rather, we are interested in using the interaction of this particular pair to illustrate the role that transduction can play in the teaching and learning of science.

Results and discussion

The IOLab, transduction and the leveraging of pedagogical affordances

Given the open-ended nature of the task design, the students in all six pairs were at first uncertain about what to make of the collection of resources in front of them. In the Appendix, the first transcript extract typifies this initial stage of the learning sequence.

However, within a minute or less of engaging with the IOLab, the students started making connections between the sets of resources. In Figure 7 the boy tests various positions and orientations of the IOLab box with reference to the graphical display on the computer screen. In doing this he is leveraging his natural proprioception (that is knowledge of the position of his own hand without the need to look at it), and also exploiting one of the key pedagogical affordances (Airey, 2015; Airey & Linder, 2017) of the IOLab device—the real-time link between changes in the orientation of the device and the immediate representation of the device’s measurements on the graphical display (three coloured lines on the graph representing the components of the magnetic field in three dimensions).

By designing the IOLab device to be hand-held and equipping it with sensors that give access to physics phenomena, the physics community has afforded students the opportunity to directly engage with physics using their own body. The IOLab’s aptness for teaching physics—that is, its pedagogical affordance (Airey, 2015; Airey & Linder, 2017)—is contained in the fact that the system facilitates seamless shifts between disciplinary resources (in this case a graph of a three-dimensional field) and everyday resources such as the senses (in this case proprioception). However, the IOLab also has high disciplinary affordance—the device can be used to actually do physics. Thus the transductive nature of the IOLab device made it possible for students to start to ‘experience’ the Earth’s magnetic field—an otherwise invisible and unchanging field in the room— by simultaneously observing the changes in screen output as they ‘felt’ the changes in orientation of the IOLab in their hand.

As the learning sequence progressed, the student pair devised a strategy for obtaining the direction of the magnetic field by first making one and then two of the graphical readouts show zero on the screen. At this point ‘all’ of the magnetic field is shown by the third readout and thus the IOLab at this stage must be in such an orientation that the third axis is aligned with the magnetic field. The students quickly learned to ‘feel’ their way to this result by observing the real-time readout as they moved their hand.

Facilitator use of transduction

Understanding our open-ended laboratory task involved a great deal of disciplinary knowledge that is not immediately accessible to the novice. Students were learning about an invisible phenomenon (magnetic field), by using a physics device they had never met before (IOLab) to transduct meaning to a resource (the graphical display on the screen) which involves an understanding of the orientation of an invented, imaginary coordinate system (the three axes at right angles to each other that are printed on the IOLab – see Figure 7). Despite the students having now devised an appropriate strategy to determine the direction of the magnetic field, it was clear at this stage that they had not been able to grasp all the disciplinary meanings and transductions that had occurred in their coordination of the resources (extract 2 in the Appendix illustrates this). The students had simply implemented a trial-and-error approach in which they manipulated the IOLab device until they had only one non-zero component on the screen. Empirically, they had not yet coordinated all the resources to make physics meanings—for example they had not referred to or used the printed axes on the box in any direct sense-making way. Our data shows that the facilitators all spontaneously leveraged their bodies and hands to illustrate the transductions of meaning between these various systems of resources. In Figure 8 we see a facilitator using his outstretched arms to help students make the link between a zero-component on the screen and the printed axes on the IOLab.

Figure 8 

Transduction made by a facilitator—purposeful bodily gesture accompanied by questions.

Transduction to a persistent semiotic resource

At this point one of the facilitators proceeded to ask the student pair a series of exploratory questions in a Socratic dialogue. When asked to explain their strategy, the students had difficulties at first—see extract 3 in the Appendix. The facilitator helped the students fix the cut-out arrow to show the direction of the magnetic field that the students had found. After about thirty minutes of working with the IOLab and the associated systems of resources, most of the students in the laboratory had been able to determine the direction of the Earth’s magnetic field and had fixed the provided cut-out red paper arrow to a vertical surface to denote this direction. Most strategies were along the lines described above. Figure 9 powerfully illustrates the general agreement and alignment of the red arrows pasted individually by the groups of students.

Figure 9 

The red arrow serves as a persistent placeholder for previously transducted meanings.

As a persistent semiotic resource, the red arrow now ‘became’ the magnetic field for the student groups that is it served as a placeholder for all the multimodal meaning making that had gone on up until this point. The importance of transduction of a range of temporal coordinations of semiotic resources to a single, persistent placeholder cannot be over emphasised. Having found the direction of the magnetic field, further meaning making was facilitated by having a permanent visual representation of the earlier coordinations of resources—the students did not need continue to hold the IOLab device in the orientation they had discovered, but could interrogate the arrow instead. This tangible, visual resource had been deliberately chosen by the research team as the visual site of display for disciplinary knowledge about what physicists know about the Earth’s magnetic field at specific locations—the arrows pasted by all the groups created a visual map of the imaginary magnetic field lines in the room. In this way, students could observe that, even though each group may have used different strategies and made choices for themselves (about IOLab orientation, etc.), the physics (i.e. the direction of the Earth’s magnetic field in the laboratory) was not dependent on these individual choices.

The cut-out arrow now became a persistent representation of the whole chain of transductions which had occurred thus far in the sequence. Going forward, the arrow now also functioned as a coordinating hub (Fredlund, Airey, & Linder, 2012) for further meaning making. Coordinating hubs have been described by Fredlund (2015). They are (usually persistent) representations that appear to be central to a given critical constellation of semiotic resources. Coordinating hubs function as a central resource around which meaning making with other persistent and non-persistent semiotic resources can be arranged.

By holding the IOLab box in a particular orientation in relation to the cut-out arrow, the facilitator could ask the students to reflect on the box’s position and the relation between the arrow, the axes printed on the box and the graphical readout on the display. Extract 4 in the Appendix provides one instance demonstrating the coordinating function that the persistent semiotic resource (the arrow) filled in the learning sequence.

Transduction to new resources

Earlier in this section, we reported on the students’ initial incomplete appreciation of the meanings underlying the information displayed by the ensemble of resources they had used. However, in the sequence after the fixing of the cut-out arrow, the students started spontaneously using gestures that they had not used until this point. These new gestures were made in direct relation to the arrow. The arrow now appeared to function as a coordinating hub for bringing together the orientation of the IOLab and the readout on the graphical display. Students now made expressions of understanding together with their new gestures. (See Figure 10 and extract 4 in the Appendix). Their demonstrated understanding was now tested by the facilitator who asked them to try different orientations of the IOLab box, eliciting explanations for the information displayed on the graph with reference to the axes and the arrow.

Figure 10 

Introduction of a new resource (gesture). The red arrow functions as a coordinating hub.

Demonstrating an understanding of magnetic field

We set students a task to locate the direction of the invisible Earth’s magnetic field. However, it was clear that with the red arrow now as a persistent representation of the direction of the field, the students had no problem in directly making reference to issues related to the magnetic field, and magnetism in general. Students asked questions related to what affects the magnetic field, how it ‘looks’, and possible uses of magnetometer devices. It therefore seems that the students’ newly gained understanding or appreciation of the phenomenon through the learning activity stimulated disciplinary appropriate and relevant questions. As a follow-up exercise, students leveraged their new understandings of magnetic field by using the IOLab to locate the steel beams in the concrete of the laboratory building.


We started this article with three research questions. Our first question asked how pairs of students leverage the pedagogical affordances of a physics device (IOLab) when working with an open-ended task to determine the direction of the Earth’s magnetic field. Here we saw that the students were immediately able to leverage their own hand movements and proprioception to start making connections between the IOLab device orientations and the graphical output. Note that the students did not need to understand any of the physics involved to start to engage with the invisible magnetic field. They quickly found the direction of the Earth’s magnetic field by implementing a trial-and-error strategy based on their physical experiences and interaction with the IOLab system. Here we note that for this task the IOLab has both high disciplinary affordance and high pedagogical affordance. In this respect the IOLab is quite special. In his description of these two affordances, Airey (2015) points out that they are often in functional opposition—that is an increase in pedagogical affordance often lowers the disciplinary affordance of a semiotic resource and vice versa (see Airey and Linder, 2017 for a fuller description of this relationship).

Our second research question addressed our interest in describing the stages of the learning process in multimodal terms. Here, we identify the following steps. As mentioned above the students quickly coordinated their talk, their proprioception of the position of the IOLab and the real-time changes in the graph to find the direction of the Earth’s magnetic field. At this point students were encouraged to use a persistent semiotic resource (the arrow) to indicate this direction. The arrow now functioned as a persistent placeholder for all of the meaning making that had occurred up until that point. In effect, the arrow now became the magnetic field for the students and was used as such in their continued meaning making. In terms of disciplinary learning, Airey and Linder (2009) have argued that there is a critical constellation of semiotic resources that is required for appropriate construction of disciplinary knowledge. In this regard, Fredlund, Airey and Linder (2012) noticed that persistent semiotic resources could function as a coordinating hub around which critical constellations of resources could be coordinated. In this case, the arrow could be seen to function as a coordinating hub, where the critical constellation of semiotic resources necessary for an understanding of the Earth’s magnetic field appeared to consist of the arrow, the graph and the printed axes on the IOLab and its orientation. At this point, the students introduced new semiotic resources (gestures) to help to explain their understanding. These gestures were also made in coordination with the arrow and its related resources. We suggest that this introduction of new semiotic resources was a sign that learning had taken place.

In our final research question, we wondered about what our analysis of the interaction could tell us about the role that transduction can play in the teaching and learning of physics. First, as we described in the introduction, transduction is central to doing physics and is an inherent property of many physics devices. In this paper, we have shown how transduction is also central to learning physics. We identify three distinct shifts in this learning sequence. First, the IOLab transducts the meaning potential in the room (magnetic field) to a visual resource (graph). Next, the students transduct all previously made meanings to a persistent resource (arrow). Finally, students summarize their understanding by transducting meaning to new semiotic resources (gestures). Using the lens of transduction, we argue that there are a number of recommendations that can be made for teachers about the use of placeholders and coordinating hubs, and the way in which transduction can be seen as a sign that learning has taken place.

Suggestions for teachers

We have two types of recommendation based on our results for when teachers plan and teach lessons.

Planning a lesson

When planning a lesson, we suggest that teachers should consider the set of resources that students will need in order to construct the desired disciplinary meanings. Here, the range and type of resources are important. Too many resources will be difficult to coordinate, particularly if these resources are non-persistent. Can the coordinations of persistent and non-persistent resources be substituted by a persistent placeholder? Here, we suggest that teachers should spend time thinking about what this persistent placeholder might be and when it should be introduced. Teachers should also think about the role that this placeholder will play in the continued meaning making. Is there a need for a hub around which the resources can be coordinated, and if so, what might this coordinating hub be? (Note – in our case, the placeholder and the coordinating hub are one and the same—the red paper arrow—but this is not necessarily always the case.)

Teaching a lesson

At the outset, teachers should know which resources are necessary for appropriate constitution of disciplinary knowledge (the critical constellation). During the lesson, teachers should be looking for student use of these particular resources. If a placeholder is needed, then the teacher should make sure that it is used as intended. Similarly, if a coordinating hub is necessary, then teachers should be looking for this (cf. Fredlund, Airey and Linder, 2012).

Our analysis suggests that teachers should not expect students to understand disciplinary meanings directly. Even though students may have coordinated the correct semiotic resources in a disciplinary manner, this does not mean they now understand the physics involved. Students need time to interrogate the resources they have used and the coordinations they have made. Airey and Linder (2009) have described this process in terms of becoming fluent in a critical constellation of semiotic resources. When this fluency has not yet been achieved, they claim that students imitate disciplinary discourse, that is they use semiotic resources appropriately, but without an appropriate understanding of the disciplinary meanings they represent. From a multimodal perspective, we suggest that transduction between semiotic resources is both the means by which students and teachers can notice when discourse imitation is occurring and the way in which students ‘discover’ disciplinary meanings for themselves. Here the role of the instructor is key, either to encourage and confirm correct transductions of disciplinary meanings, or to ask questions that help students notice that they may still not have grasped key issues in a disciplinary manner.

Thus, we suggest that teachers should be looking for student introduction of new semiotic resources. Here transduction is a sign that learning is taking place. The transduction to new semiotic resources fills two important functions: first, it allows students to demonstrate their learning, and second, and perhaps more importantly, it allows teachers to check this learning. This is because of the complementary and constraining functions that transduction entails. As we discussed in the introduction to this article, when meaning is transducted from one semiotic resource to another, information can be added or taken away. Thus transduction serves as a useful check of student understanding since disciplinary meaning must be coherent across all transductions.

Transduction devices in science teaching

Finally, we would like to make the following observation about the use of devices in the teaching and learning of science. Whilst the disciplinary affordance of transduction devices is clearly understood, (i.e. we know very precisely what function a particular device plays in science), we believe that we are only beginning to scratch the surface when it comes to the pedagogical affordances of devices in the teaching and learning of science. We suggest that future work should explicitly examine the pedagogical affordances of devices. What is it that makes a device suitable for teaching a particular kind of content? Should we demand that devices have both high pedagogical and disciplinary affordance, (as in the case of the IOLab) or is it enough in some cases that devices have pedagogical affordance?

One of the main pedagogical affordances of the IOLab is that it was possible to manually manipulate the device whilst simultaneously following a real time readout on a screen. This allowed students to ‘feel’ their way to the magnetic field direction. The potential for devices to allow students to ‘feel’ other physics phenomena is something that we suggest is worthy of further investigation.

Appendix – Transcript Summary (selected sections)


The transcript summary and full extracts of selected sections are provided in a textual form only, i.e. text formatting is used to distinguish different features of the multimodal engagement. These are listed in the following points:

  1. Time stamps provide an indication of the placement of the selected excerpt within the entire sequence.
  2. Actual speech by participants is given in italics.
  3. The male student in this sequence is given the label B, and the female student, G. Facilitators are given the codes F1, F2 and F3.
  4. Actions of participants are given within square brackets, transcriber additions or comments about talk and actions are added in normal parentheses, and other synchronous and asynchronous multimodal activity, e.g. the position of the IOLab device and axes orientation, are recorded inside curled brackets. Actions and multimodal activity given below speech lines indicate coordination with that speech.
  5. Three dots before and after speech lines indicate pauses and/or simultaneity with preceding or subsequent speech.

Introductory sequence

Time:- 0:00–08:35

Setting up; calibration of IOLab; task introduction by F2.

Sequence Extract 1

Time [minutes:seconds]→ 09:30–10:18

G: We should be showing the direction of this…
[G grabs cut-out red arrow]
B: I don’t know, how do we read that, how do we know that?
[B looks at screen]
G: (reading instruction sheet)“…and to fix the arrow that represents the direction”
[G chuckling and looking at screen]
{holds and dangles red arrow in front of screen}
B: Ohh, I guess, I don’t know, mmm, what happens if we move it. Oh S#!t, look at that …
[B grabs IOLab and rotates IOLab while looking at screen, …both look at screen]
G: chuckles (hah, hah, hah)
B: Holy crap, holy crap … Ummh, yeah, moving it changes it of course, so I guess we could figure out by the amount…
G: No, I guess…
[G takes hold of IOLab and rotates it]
B: You see the blue line went up to the top
[G is still rotating IOLab]
[B points at graph on screen]
G: Yeaah, ok.
B: Yeah, I don’t really know what this tells us.

Sequence Extract 2

Time→ 22:22 – //– 24:20

G: You want z- to be zero?
B: Yah, I’m trying to get both of them zero at the same time.
[B now takes hold of device on top of box and looks at screen; G peers over outstretched arm of B at screen.]
B: Isn’t that z-? I don’t understand, that should be z-.
[B moves device towards him slightly and looks at screen]
{The +z-axis on the device is in a line away from B, the y-axis is still pointing up, and the x-axis points to his left}
G: That should be z?
B: It’s not moving.
// a short while later:
G: … but if, if … if we tilt it this way, then z- changes, see …
[G swivels the device about the x-axis]
{x-axis pointing perpendicular to field which gives a zero reading for the x-component on the graph}
B: Yeah
G: If I put it on the horizontal… way; see now it gets, x, no, y is zero.
{At end of G’s manipulation, the device is held so that the y-axis is pointing almost perpendicular to the magnetic field, i.e. almost horizontal}
B: But how?
G: But why is it, why is y- zero?
[G looks around class (for assistance?), smiles, then looks directly at camera]
{G is still holding device in position where y-value is zero}
B: I don’t know… by my logic it shouldn’t be working like that, or I don’t know …
G: {laughs}…could we get help, …see… he explains …OK.
[G points to a facilitator across the room]
{G looks around class (for assistance?), smiles, then looks directly at camera, still holding device in position so that y-value is zero.}

Sequence Extract 3

Time→ 26:26–26:57

F1: OK, … good, now that you have the blue line; so what’s the blue line? It’s the y- right?
[F1 points at screen, then comes closer to screen]
B: Yeah
G: Yes.
F1: So the y one, No, actually, no… the red one is zero right now, I’m sorry.
[F1 corrects himself by pointing at screen]
B: Yeah, … or close to zero it is.
F1: …close to zero, yes. So what does that mean for the field; in which direction is it not pointing?
B: Umm…
F1: You’ve eliminated one, one family of directions … which family would that be?
[F1 uses hands in encircling motion (“family of directions”)]
G: MM-mm … the horizontal…
F1: All the horizontal …or just …
B: Wouldn’t it be this way?
[B moves hand back and forth in line of x-direction with index finger pointing in +x-dir’n]
F1: Yes, exactly…
G: …yeah, yeah, true.

Sequence Extract 4

Time→ 31:59–32:20

F1: So, if you want to align (emphasis) this vector to this vector, what should you do?
[F1 points in order at axes labels (printed on IOLab) in direction of +z, then at red arrow]
G: Ah-hah! This way … ah, OK,
[G grabs IOLab (smiles broadly) and holds device with {+z pointing up at an angle directly opposite to dir’n of red arrow}]
[B extends hand towards device (but G grabs it first), then withdraws hand to mouth and watches screen]
F1: Yahh…
[F1 looks at B (to check if B also gets it)]
G: Yeah, ah-hah, [indistinct] its a negative [indistinct] because it’s pointing this way and …
[G looks at screen to look at value for Bz; explains negative screen value for z- with hand and finger first gesturing along dir’n of red arrow (down at angle), then opposite]
B: Yeah
[B nods head up and down]
F1: What if you flipped it around? It’s positive?
[F1 points at IOLab, and then makes a flipping gesture with right hand and two fingers in V-shape]
[G flips device about y-axis so that +z now points in direction of red arrow, looks at screen and smiles]
B: Yeah, it’s positive 50.
[B’s right hand first over mouth (closed, loose fist); looks at screen and confirms z-value, right hand now first strokes hair over right ear lightly, then rests fingers against neck and chin, and starts pinching at cheek gently, touches ear etc.]
F1: Makes sense?
[F1 looking at B intently]
B: …Yeah.
G: Mm-mm … That’s pretty cool. I mean that is cool.
B: Yeah, ah yeah.


The authors would like to thank Cedric Linder and the physics education research group at Uppsala University for fruitful discussions and comments on our initial data analysis and earlier drafts of this paper. Two anonymous reviewers have also helped refine the message of this paper. Funding from the Swedish Research Council (project number VR 2016-04113) is gratefully acknowledged.

Competing Interests

The authors have no competing interests to declare.


  1. Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198. DOI: 

  2. Airey, J. (2014). Representations in Undergraduate Physics. Retrieved June 7 2018 from: 

  3. Airey, J. (2015). Social Semiotics in Higher Education: Examples from teaching and learning in undergraduate physics. In: SACF Singapore-Sweden Excellence Seminars, 103. Swedish Foundation for International Cooperation in Research in Higher Education (STINT). Available from: 

  4. Airey, J., & Eriksson, U. (in review). Unpacking the Hertzsprung-Russell Diagram. Designs for Learning. 

  5. Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. Journal of Research in Science Teaching, 46(1), 27–49. DOI: 

  6. Airey, J., & Linder, C. (2017). Multiple Representations in Physics Education. In: Treagust, D. F. Duit, R., & Fischer, H. E. (eds.), Multiple Representations in Physics Education, 10, 95–122. Cham: Springer International Publishing. DOI: 

  7. Baldry, A., & Thibault, P. J. (2006). Multimodal Transcription and Text Analysis: A Multimodal Toolkit and Coursebook with Associated on-line Course. London: Equinox Pub. 

  8. Bezemer, J., & Kress, G. (2008). Writing in multimodal texts: A social semiotic account of designs for learning. Written Communication, 25(2), 166–195. DOI: 

  9. Bezemer, J., & Mavers, D. (2011). Multimodal transcription as academic practice: A social semiotic perspective. International Journal of Social Research Methodology, 14(3), 191–206. DOI: 

  10. Etkina, E. (2015). Millikan award lecture: Students of physics—Listeners, observers, or collaborative participants in physics scientific practices? American Journal of Physics, 83(8), 669–679. DOI: 

  11. Fredlund, T. (2015). Using a Social Semiotic Perspective to Inform the Teaching and Learning of Physics. Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology. Uppsala: Acta Universitatis Upsaliensis. 

  12. Fredlund, T., Airey, J., & Linder, C. (2012). Exploring the role of physics representations: An illustrative example from students sharing knowledge about refraction. European Journal of Physics, 33(3), 657–666. DOI: 

  13. Jewitt, C., Bezemer, J. J., & O’Halloran, K. L. (2016). Introducing multimodality. London: Routledge. 

  14. Kress, G. R., & Van Leeuwen, T. (2006). Reading images: The grammar of visual design (Second). London and New York: Routledge. DOI: 

  15. Mavers, D., & Oliver, M. (n.d.). Affordance|Glossary of multimodal terms. Retrieved June 7 2018 from: 

  16. Needham, J. (1962). Science and Civilisation in China: Volume 4, Physics and Physical Technology, Part I: Physics. Cambridge University Press. 

  17. Roychoudhury, A., & Roth, W.-M. (1996). Interactions in an open-inquiry physics laboratory. International Journal of Science Education, 18(4), 423–445. DOI: 

  18. Selen, M. (2013). Pedagogy meets Technology: Optimizing Labs in Large Enrollment Introductory Courses. Retrieved May 19 2017 from: 

  19. Swedish Research Council. (2017). Good Research Practice. Stockholm: Swedish Research Council. Retrieved from: 

  20. Van Leeuwen, T. (2005). Introducing social semiotics. London: Routledge. 

  21. Volkwyn, T., Airey, J., Gregorcic, B., & Heijkenskjöld, F. (2016). Multimodal transduction in secondary school physics. Presentation given at the 8th International Conference on Multimodality, Cape Town, South Africa. Retrieved from: 

  22. Wittgenstein, L., Anscombe, G. E. M., & Wittgenstein, L. (1963). Philosophical Investigations… Translated by GEM Anscombe. [A Reprint of the English Translation Contained in the Polyglot Edition of 1958.]. Oxford: Blackwell. 

comments powered by Disqus