Hacking (1983, p. 186) quotes the poem In commendation of the microscope by 17th century writer Henry Powers, in arguing for the need to describe how technology widens the scope of what can be observed: ‘We ought to have some understanding of those astounding physical systems “by whose augmenting power we now see more/than all the world has ever done before”.’
This paper is a contribution to the emerging research field of visualization technology in science education (e.g. Linn & Eylon, 2011), in particular we deal with infrared (IR) cameras in learning thermodynamics (science concerning transfer and transformation of energy, fundamental for physics, chemistry and engineering).
In everyday life, we interact with thermal phenomena through the sense of touch. In the science practice, our senses are complemented by tools like thermometers, which give access to the phenomena in an indirect way (Hacking, 1983). Now, the introduction of IR cameras offers new ways of measuring and visualizing thermal phenomena (see Figure 1). The image on the camera is displayed through different colors or nuances to show the temperature variation of the surfaces at which the IR camera is aimed. The temperature is indicated as a number in the corner of the display and colors and temperature are combined as a vertical scale, on the right side of the screen.
The idea of using IR cameras for educational purposes emerged by the turn of the millennium (Vollmer, Möllmann, Pinno, & Karstädt, 2001). In recent years, a number of IR-camera experiments for educational purposes have been suggested (Möllmann & Vollmer, 2007; Xie, 2011), and empirical studies have been conducted of the use of the tool to promote the learning of thermodynamics (Haglund, Jeppsson, Melander, Pendrill, & Xie, 2016). In order to be able to interpret thermal images, students need some level of conceptual understanding of the involved phenomena (Meiringer, 2013; Schönborn, Haglund, & Xie, 2014).
The present study is based on a multimodal approach to learning by building on a social semiotic framework. In this way, the communicative means of the IR camera, such as displaying colors and numbers, can be regarded as novel semiotic resources in the investigation of thermal phenomena.
By doing science, students practice coordination and translation of meaning between different communicative means: this includes verbal action such as statements and questions, and non-verbal action such as shifts in gaze and posture but also through the use of physical tools (Airey & Linder, 2009) such as interpreting the colors of a litmus paper indicating the pH value of a substance. In this vein, within the tradition of social semiotics, Lemke (1990) emphasizes the need for science students to learn to ‘talk science’. Mercer (1995), in turn, describes different types of talk in the classroom, in terms of cumulative, disputational and exploratory talk. These analytical categories have been used to analyze the talk made during science learning through laboratory practice (Andersson & Enghag, 2017) and will be used in this paper.
The study aims at exploring students’ and instructors’ first use of IR cameras to investigate a thermal phenomenon (e.g. what the cameras afford in terms of discipline and pedagogy) and what potential discoveries and conclusions can be made with the technology through peer discussion (e.g. how the talk of the participants relates to what the cameras afford). We are interested in how IR cameras can support learners’ meaning making and communication in a chemistry lab setting, or, in other words, what affordances (Gibson, 1979) the technology brings (Bernhard, 2018).
The two cases chosen for this study represent a scientific community at two different levels of expertise: first-year university students, participating in their second lab at the university level, and PhD students acting as instructors at the lab. The students come to attend to thermal aspects of what happens when a salt (sodium hydroxide) is exposed to air, with and without the use of IR cameras. The instructors, in turn, get to investigate this phenomenon, which is known to them, with IR cameras as a novel technology. The choice of these two cases has been made to accommodate for the idea that the affordance of a semiotic resource, within a discipline, can be two-fold: disciplinary and pedagogical (Airey & Linder, 2017).
In particular the following research questions are addressed:
A key assumption in this paper is that ‘learning and development cannot be understood without taking account of the intrinsically social and communicative nature of human life’ (Mercer, 2004, p. 139).
Learning typically involves multiple participants that combine their intellectual resources through the communicative means that are available to them in the learning situation. We will call such means semiotic resources (Lemke, 1990), as they are resources with which we signify meaning. Semiotic resources may be of any format that the participants find understandable within the situation in which they interact. Going beyond written and spoken language, such semiotic resources can be e.g. different colors (Kress & van Leeuwen, 2002), gestures and posture (Goodwin, 2003).
The concept of affordance was introduced by Gibson (1979): An object or situation invites, or affords, a particular interpretation or action. For example, the form of a tool that includes a handle affords grasping. Gibson explained that ‘what the object affords us is what we normally pay attention to’ (Gibson, 1979, p. 134). From the point of view of meaning making, the affordances reflect the ways in which semiotic resources are combined and used in a particular context. In van Leeuwen’s (2005) interpretation of affordance there are many possible uses of semiotic resources depending on the relationship between the individual and the environment. The perceiver would then be able to notice a part of the full range of these affordances depending on his or her needs. In this way, the affordances of semiotic resources are likely to change over time due to the familiarity and experience gained with the contextualized semiotic resource, or when coordinated with other semiotic resources.
Some semiotic resources have agreed upon meaning only within a particular discipline. This indicates that affordances can be viewed as a relation between a semiotic resource and a community of people (rather than an individual organism in Gibson’s sense). To shift focus to the community level of a discipline and acknowledge the difference between people entering the community and experienced members of the community, Airey and Linder (2017) suggest a differentiation between pedagogical and disciplinary affordance. Disciplinary affordance can be defined as: ‘the agreed meaning making functions that a semiotic resource fulfills for a particular disciplinary community’ (Airey, 2015, p. 120). Natural sciences use semiotic resources with high disciplinary affordance, such as specialized vocabulary, technology and mathematical formalism for efficient communication. However, without the experience of formal education within the discipline, such resources are opaque to the novice. Therefore, for educational purposes semiotic resources need pedagogical affordance, characterized as ‘the aptness of a semiotic resource for the teaching and learning of some particular educational content’ (Airey & Linder, 2017, p. 107). By unpacking a semiotic resource with high disciplinary affordance and low pedagogical affordance, the former is decreased and the latter increased (Fredlund, Linder, Airey, & Linder, 2014).
Even though other semiotic resources have been introduced in multimodal studies, spoken language has from the beginning been an important part of semiotics. By participating in a discussion, many participants can contribute to joint meaning making. For example, a pair of students that are working on a task, can, through the talk that emerges during their combined effort, assess one another’s knowledge in the construction of a shared framework of understanding. The talk can also have a metacognitive function in that each peer’s individual thoughts are contrasted with those of the others and the shared framework. This could involve talking about the task’s goals and how to achieve them, thus constituting ‘a powerful problem-solving tool’ (Mercer, 2013, p. 151).
In order to analyze different types of talk in groups, Mercer (1995) proposes a typology with three categories: Disputational talk, Cumulative talk and Exploratory talk:
The mutually shared knowledge that emerges through the negotiation between learners, when working with a problem or exploring a phenomenon, reflects the affordances of the semiotic resources involved in the task (Fredlund, Airey, & Linder, 2012; Fredlund et al. 2014). As such, what the participants say and do can be used as a source of information on the set of affordances of the semiotic resources for the community to which the participants belong. Goodwin (2003), in turn, emphasizes that we should not study the use of different sign systems, such as talk, gestures or posture, in isolation, but consider how they work in conjunction, with different communicative functions, in forming an ecology of signs. While gestures are usually coordinated around the object of talk, sustained actions such as postural configurations set up boundaries for where the mutually shared attention should be directed. By adding or removing entire sign systems, the conditions of communication change. In our case, comparing two similar situations, with and without the IR cameras, the affordance of the set of semiotic resources of IR cameras could be discerned. Thus, the new semiotic resources could be attributed the affordances that emerge through the communication of the participants.
However, the comparison would need to acknowledge the range of multimodal means with which the participants communicate and interpret the talk informed by these. It is impossible to know whether the sentence ‘Well, okey’ brings the meaning of confirmation, challenge or dispute without hearing how it is said or communicated through other modes of communication such as facial expression. The range of modes helps us understand how the students frame (i.e. interpret) their activity. This is important as their framing will inform their decisions on what to pay attention to and what actions to take (Scherr & Hammer, 2009), linking the framing directly to the affordances of the available semiotic resources. Andersson and Enghag (2017) have suggested that Mercer’s (1995) typology of talk can be used to analyze how students in physics lab practice frame their activity, and thus the affordances of the available semiotic resources.
The study was conducted during two laboratory sessions on calorimetry, the study of heat in relation to chemical reactions, in an introductory chemistry course. Prior to our intervention, the students had worked with their instructed lab tasks, which involved dissolution reactions in solution: When the salt sodium hydroxide is added to water, the temperature increases, a so-called exothermic reaction. In practice, the exothermic reaction between sodium hydroxide and water occurs already when sodium hydroxide for some time is exposed to air, from which water vapor is absorbed. The sodium hydroxide gets sticky and the temperature increases; a process called deliquescence has started. However, many students fail to notice that the exothermic reaction has started already before the salt is added to the bulk water. The deliquescence was not explicitly brought up in the lab instructions but could affect the results of the lab’s main experiment and was therefore possible to be noted as a source of error for the result.
The laboratory session on calorimetry was chosen for the study as we believed that there was an opportunity for students to gain a richer understanding of the involved phenomena through visualization technology. Therefore, after the students had conducted their laboratory exercises in pairs, they were invited to study deliquescence, but this time with the help of IR cameras (Figure 1). In addition, after the students had finished their class, the instructors were invited to perform the same task as their students had performed with the tools.
This is a study with an empirical focus on interaction between learners, the phenomenon and the technology used for investigation. Visual semiotic resources like gesture and action are part of the data. Therefore, video recording was chosen as a method for data collection.
We as researchers acted through reactive observation, which is a type of naturalistic observation (Angrosino, 2012). We circulated in the laboratory during the students’ instructed lab work and occasionally recorded clips, with a handheld video camera, of students’ activities, approaching them with open-ended questions like ‘how is it going?’ to make no assumptions on their progress and let them own the discussion. The participants’ discussions and actions were also recorded during the intervention after the instructed lab work. In total, about 3 hours of video data were recorded in the two sessions, which comprises the basis for analysis.
All students were informed on how the data would be used, that they would be able to withdraw from the study at any time and that no personal information would be disclosed to anyone other than the researchers. The students who volunteered for participation signed consent forms with information on the study.
The interactions of one pair of students and one pair of instructors (two PhD students in chemistry, who acted as teachers for the laboratory class) were selected for analysis. Clips that involve the deliquescence phenomenon were chosen for analysis. The students chosen were the only students that noticed the deliquescence spontaneously already during their instructed lab work while not using IR cameras. These students were video recorded both during the time they notified us about their discovery and during the subsequent intervention when using the IR cameras. The instructors were video recorded during their use of the IR cameras. The resulting video data were then iteratively transcribed and analyzed in Swedish, the language of use in the dialogues, based on the routine of multimodal conversation analysis: focus on selected clips and watching the clips iteratively in data sessions (Jewitt, Bezemer, & O’Halloran, 2016). The communicative means included in the final transcription were speech, posture, gaze and gestures, presented as verbal and non-verbal action.
We used a combination of multimodal conversation analysis and the typology of talk introduced by Mercer (1995). Conversation analysis (CA) was originally proposed as a way to study social interaction through talk (Jewitt et al. 2016), but has later been expanded to include other semiotic resources like gaze and gesture (Goodwin, 2003). The set of methods in multimodal CA usually involves video recording of conversations from short interactions, transcription of those recordings, including features such as pauses and prosody, and iterative analysis through organized data sessions within the research group (Jewitt et al. 2016). In addition, CA typically focuses on naturally occurring dialogue, where the researcher assumes a passive role. In line with Angrosino (2012), we however consider reactive observation data relevant for CA. Previous research, using this or other multimodal frameworks, has put some effort into exploring different ways in which we make meaning, in our interpretation, through coordination of different semiotic resources, providing ‘an arena for mutual orientation, shared attention to a common environment and collaborative action’ (Goodwin, 2003, p. 32). This includes talk, gestures and posture (Goodwin, 2003), colors (Goodwin, 1997; Kress & van Leeuwen, 2002), and design (Norman, 2002). In addition, the notion of semiotic resource has found its place in science education research (Airey, 2009; Airey & Linder, 2009). To learn a discipline in science one needs to learn how to coordinate several semiotic resources and through this develop discursive fluency (Airey & Linder, 2009). An indication of this learning process or the already achieved fluency is the ability to transform the meaning across semiotic resources (Volkwyn, Airey, Gregorcic, & Heijkenskjöld, 2016). With a specific focus on spoken language, Mercer’s (1995) typology of different types of talk was used to analyze the participants’ interaction.
An outline of all the conversational speech was done in the first iterations, followed by adding the different sounds, like laughter, that could be heard in the recording, in accordance with conversation analysis (Jewitt et al. 2016). The next iteration of transcription attended to gaze, posture, and actions. In the presented transcript excerpts, speech is presented under the category Verbal action, and all other actions and means of communication are included under the category Non-verbal action. In this way the two types of data are presented as they chronologically unfolded. Speech was put under its own category to acknowledge the conventional structure which is often missing from other communicative means (Goodwin, 2003) and to faciliatate the application of Mercer’s (1995) typology of different types of talk, which was used to analyze the participants’ interaction. Finally, transcripts were translated to English.
The two sequences of talk between the students, where they explore the phenomenon in focus in this paper, and the talk between the instructors where they investigate the same phenomenon, are presented here.
During the instructed lab work, two of the students notice something unusual with their material and want to make the researcher (CRS) pay attention to their finding. They do this by picking up the container with the salt from the bench and point at what caught their attention, the wet areas around and on the salt (Figure 2).
This serves as the starting point for a dialogue between the students and the researcher:
|Line||Interlocutor||Verbal Action||Non-Verbal Action|
|1||Researcher||How is it going?|
|2||Student 2||Fine, we have weighed the salt now and ehm… though, I am a bit interested by it since after a while, …||Picks up container with salt from the bench.|
|I don’t know if it is visible it is kind of melting.||Points around the salt.|
|It’s sticky.||Rubs fingers.|
Apparently, the phenomenon observed by the students piqued their interest. Student 2 picks up the container and highlights the aspects that differ by pointing at the wet areas around and on the salt (Figure 3a), describing the salt as ‘melting’ and ‘sticky’. The use of the word ‘melting’ indicates a first hypothesis of what has caused the salt to become moist.
By telling the researcher about their findings and picking up the object of interest, the students succeed at forming a shared point of attention that potentially could lead to a deeper understanding of the phenomenon on a disciplinary level as they discern disciplinarily relevant aspects of the phenomenon. As the focus is established, Student 2 puts the container back on the bench (Figure 3b).
|Line||Interlocutor||Verbal Action||Non-Verbal Action|
|3||Researcher||Mhm, and what do you think is the cause of that?|
|4||Student 2||I’m not really sure since it is…Ah!||Displays the container.|
|Okey!||Puts the container back on the bench.|
|If it hasn’t got to do with, what do you call it, the temperature, then maybe it has to do with the light exposure perhaps, but…|
|5||Student 1||Don’t you think it is residual water dissolving?|
|6||Student 2||Yeah, maybe it is!||Looks at the container on the bench and exclaims surprisingly.|
|But…||Looks towards preparation bench and moves body towards the working bench.|
|…I thought it was supposed to be dry over there [the preparation bench] but yeah, perhaps…|
|7||Student 1||Ah…||Moves towards the bench while looking at the container|
|8||Student 2||…yeah if it isn’t…||Student 1 crosses his arms and looks at Student 2|
|9||Student 1||…it could…it could’ve attracted wat…||Student 2 looks at Student 1|
|10||Student 2||Exactly.||Student 1 points at Student 2|
|11||Student 1||…it may also have been that it attracted…||Looks at the container|
|…water…||Student 2 looks at Student 1|
|…from the air.|
|12||Student 2||Yeah, water vapor.||Looks at the container|
A second hypothesis, that the salt is moist due to light exposure, is suggested after the researcher asks for an explanation of the phenomenon, which Student 2 supposedly interprets as dismissing the initial explanation as due to ‘melting’, something to do with temperature. A suggestive question from Student 1, seen in line 5, leads them on to a third potential explanation, that the content of the container is contaminated with water. Finally, they suggest a fourth hypothesis in lines 9–12, that the salt has attracted water from the air. The sequence of talk then shifts into one with an aim to regain focus on the task at hand (Figure 3c), thus the initial attempt to explain the phenomenon comes to an end when the students do not find any way to test and confirm their hypotheses.
|Line||Interlocutor||Verbal Action||Non-Verbal Action|
|13||Student 1||…that it’s been stored…||Moves towards the bench and looks at the container.|
|…well-ventilated…ehm but did we get a thermometer into that or what?|
|14||Student 2||We’re here going to…||Looks at the equipment on the bench and picks up the thermometer.|
|15||Researcher||How do you think it will affect the experiment later on?||Student 1 looks at Researcher.|
|16||Student 2||This? That it’s already been dissolved? Yes, if we s…||Looks at the container.|
|17||Student 1||There will be a marginally larger amount of water.||Student 2 looks at Student 1|
|18||Student 2||Yes, some, maybe…||Continues with instructed lab work|
|…heat, has already been used up, when it’s dissolved already, so to speak.|
|19||Student 1||Because…because it’s already a bit…ah…should already be a bit…|
|20||Student 2||But I highly doubt that it is noticeable, otherwise we just have to do this experiment again. It’s just fun.|
After another question from the researcher, in Line 15, attempting at making them aware of potential consequences of the phenomenon for their instructed work, the students conclude that it would include more water in the experiment and that some heat would have been ‘used up’. It is not clear whether the last consequence should be interpreted as referring to heat transfer to the surroundings or as heat being ‘used up’ to melt the salt.
Regarding the researcher’s request for explanations of the phenomenon, four hypotheses are proposed during the pre-IR camera talk:
None of these hypotheses is really confirmed by the students during this part of the activity and, as seen in lines 13–14, the students try to move focus back to the instructed lab work after being drained out of more ideas in the conversation.
In their reasoning, the students make their information accessible to all participants in the discussion by sharing their thoughts, as in the dialogue in lines 4–12. The response made by Student 1 is challenging Student 2’s proposal, but gives an alternative explanation, which immediately is considered by Student 1 who then challenges it with a justification. This pendulum of reasoning and generation of new ideas is a characteristic of the exploratory talk of Mercer’s (1995) typology, where the students generate, justify and challenge hypotheses. This chain of exploratory talk ends when the participants shift focus back to the instructed lab work, where the talk is of a more cumulative type.
The range of embodied semiotic resources that the students use in the sequence frames the exploratory talk. The gaze and posture of the students change according to whether they discuss something together, as in lines 8–11, or aim at confirming a property related to the phenomenon, as in lines 6, 11 and 12. When jointly discussing an explanation, the students engage socially with each other by turning to and looking at each other and the researcher. During this discussion, they just briefly look at the phenomenon and the equipment as a shared point of reference. After some time spent on discussing the findings, the students try to break out of the shared talk to go back to the instructed experiment again. This is shown in lines 13 and 14 when the students move away, both physically and verbally, from the joint discussion on the uninstructed phenomenon: Both students are turned towards the bench. After Student 1 asks the other student: ‘but did we get a thermometer into that or what?’, Student 2 moves over to the bench and kneels down in front of it to continue working on the instructed set up while still trying to answer the questions posed by the researcher.
At a later occasion, after they have finished the instructed lab practice, we reintroduce them to the phenomenon that they had observed on their own earlier that day. This time though, they are provided with IR cameras and then informed on the basic functions of the cameras, like the information in the display. A new sequence of talk is recorded:
|Line||Interlocutor||Verbal Action||Non-Verbal Action|
|1||Researcher||So if you try to aim it at these two for example.||Moves two containers with salt towards the students.|
|Here you have some sodium hydroxide and it has been here for a while and you remember that you talked about how it was a bit wet.||Student 2 laughs in confirmation when seeing the salt through the IR camera (see Figure 4).|
|What do you now see when observing it with IR cameras?|
|2||Student 2||It gets very red and…|
|3||Student 1||Yeah, I have a whi… it is even white.|
|4||Student 2||Yes, a bit.|
|5||Student 1||In other words, it is warm.|
|6||Researcher||Yes, what do you think is the cause of that?|
|7||Student 1||That it has attracted heat…or what am I saying…water from the air and that the ehm exothermic dissolution reaction has started.||Student 2 looks up from the IR camera and back again.|
When observing the sticky salt with the IR camera, Student 2 laughs in a way that we interpret as indicating that what he sees should have been obvious also without an IR camera. What then unfolds is a confirmatory chain of talk, which reflects a transformation, from one semiotic resource to another, starting in color observations from the display (line 2–4), which is translated into a property of the phenomenon, it is warm (line 5), and finally ends in a disciplinary verbal formulation and explanation of what phenomenon they are observing (line 7).
The students’ immediate attention is on the red color, as seen in line 2, which usually is associated with warmth or energy (Kress & van Leeuwen, 2002). The attention then shifts to the white color in the camera image, which also can be associated with something hot, as in ‘white hot’. In accordance with this association, the students interpret the colors as the object being warm, a property that typically otherwise is perceived through the sense of touch. Finally, the students connect the warmness to an attraction of water. As opposed to when they studied the phenomenon without the use of IR cameras, the sticky salt is now unequivocally experienced as a thermal phenomenon, in fact it is ‘the exothermic reaction’ that they have worked with throughout the lab that has started before they pour the salt into water.
During this chain of talk, the students have a focused gaze on the display of the IR camera (Figure 4) throughout the sequence until just at the end of line 7 where Student 2 looks up for a second from it. In addition, the students have a static posture centered around the cameras during most of the sequence. This can be contrasted with their more dynamic interaction when they studied the phenomenon without the IR cameras.
In the talk, the students describe what they see and then confirm and elaborate the last hypothesis stated in Sequence 1, that the salt has absorbed water from the air. The talk used by the students in this sequence involves statements that add to one another, but lacks challenges or considerations of alternatives. Thereby we regard it as cumulative, according to Mercer’s (1995) typology. There is no attempt at discussing any of the other hypotheses proposed in Sequence 1 and no consideration of the constraints and possibilities of the technology itself.
For comparison, we introduce IR cameras to two instructors for the laboratory activity. We then ask them to conduct the same task as the students.
During the task, the instructors start out by stating what they see without the IR cameras and then move on to observing the phenomenon with the provided IR cameras (Figure 5a), changing the distance of the cameras, relative to the phenomenon, to vary the observation (Figure 2b). Initially they note that the sodium hydroxide is difficult to scoop, and want to replace it with new salt that has not yet reacted with water vapor. The researcher sees this as a starting point for discussing the nature of the phenomenon.
|Line||Interlocutor||Verbal Action||Non-Verbal Action|
|1||Instructor 1||But you can see that…||Instructor 1 aims his IR camera, close, at the salt and then moves his gaze from his camera to the salt and then the camera screen of Instructor 2|
|…it has attracted a lot of water and…||Instructor 1 moves away his camera from the salt.|
|…see||Instructor 2 leans in closer to the salt with his IR camera, but with his posture directed toward Researcher.|
|…take…that it is very hygroscopic and…||Instructor 1 moves his camera back close to the salt again.|
|…you clearly see the exothermic reaction.||Instructor 2 moves the camera away and prepares to interact physically with the salt.|
|Do you have anything to add [name of Instructor 2]?|
|2||Instructor 2||It is also visible where they have been placed, I am thinking of when you remove them from…||Touches one of the containers.|
|4||Instructor 1||Yes. Yes, it is also clearly visible.||Instructor 2 moves one of the containers across the bench and Instructor 1 moves the other container.|
|They do not even have to be there for a long time and it is still visible.||Both instructors aim their IR cameras at the locations on the bench where the containers originally were, showing a trail indicating transfer of heat.|
After having noted that the salt is difficult to scoop, the instructors begin explaining what they observe. They use disciplinary language, such as ‘hygroscopic’ and ‘exothermic reaction’ in their investigation of the phenomenon. In line 2, they start to interact with the phenomenon by touching and moving around the container (Figure 5c). As a result, a heat trail is visible through the IR camera. In this introductory dialogue, cumulative talk is used to construct a basis around which the instructors can share their thoughts and engage in a more critical and constructive discussion, thus shifting over to exploratory talk.
Throughout the descriptions and study of the phenomenon, the participants change their positions and postures by alternating the distance to the salt, thus varying the image provided by the IR camera. Their postures, as a semiotic resource, are coordinated together with the descriptions of their observations. They also vary the focus of their gazes, shifting between the display of the IR cameras and the salt itself.
|Line||Interlocutor||Verbal Action||Non-Verbal Action|
|5||Instructor 2||You see that already when it dissolves water from the air, so it is enough to give indication. Does that tell us something about the reaction or the cameras? I don’t know.|
|6||Instructor 1||The reaction I think.|
|7||Researcher||Yes, talk a bit about the reaction. What is happening?|
|8||Instructor 2||So my throat is hurting, it is your turn to talk.||Instructor 2 looks up at Instructor 1, who then looks back.|
|They turn their gazes back towards the salts.|
|9||Instructor 1||Sodium hydroxide dissolves in water.||Looks up at the researcher.|
|10||Instructor 2||Ehm, yeah, I would not call it that in this case.|
|11||Instructor 1||But it is rather…||Instructor 1 looks at Instructor 2 and does a pinching gesture with one hand.|
|…attracting water from the air but…||Instructor 2 looks at Instructor 1, who then looks back.|
|…you will still get a…||Gesture swirls with both hands.|
|Both instructors continue looking at each other.|
|12||Instructor 2||Is it water dissolving in sodium hydroxide…||Instructor 2 looks at Researcher while Instructor 1 looks down for a moment.|
|…in…||The instructors look at each other again.|
|…when it is here?||Both instructors look down on the salt, still holding their IR cameras aimed at the salt.|
|But it is rather going into…there is more sodium hydroxide…||Both instructors look at each other again.|
|…than the water you got, so it is…||The gazes of the instructors switch back to the salt.|
|…rather than hydra…it goes into the crystal structure and hydrates.|
|13||Instructor 1||Hang on…||Instructor 1 leans over close to the salt, still aiming his IR camera at the salt. Instructor 2 does the same.|
|…we do not have…||Instructor 1 looks at Instructor 2’s container with his IR camera.|
|…that much left of…||Shakes his container.|
|…the crystal structure on the surface, so there is no hydration in the crystal structure.||Looks at Instructor 2.|
|14||Instructor 2||Nonono.||Instructor 2 shakes his head and both instructors laugh.|
|Not the structure. Sorry, …||Looks at Instructor 1.|
|…structural chemist, I do not work with that kind of trash, …||Instructor 1 looks up at Instructor 2 and then back down again.|
|I have batteries.||Instructor 2 looks back down on the salt.|
|15||Instructor 1||Ehem, inorganic chemist, I work with structures. Yeahyeahyeah…|
When the instructors mention the reaction, the researcher asks them to elaborate on what is happening in regards to that. An attempt at explaining the reaction is made by one of the instructors: ‘Sodium hydroxide dissolves in water’. This then triggers a discussion in lines 9–14 (Figure 5d), which ends in statements regarding their disciplinary identities in lines 14 and 15. In lines 8–14, the participants start to look at each other, a shift in gaze that is coordinated with exploratory talk, where they expect responses to what they say. In this way, the gaze may be interpreted as a communicative tool and thereby a semiotic resource in its own right.
The discussion now alternates between the two participants. One instructor gives an explanation that the other immediately challenges, which pushes the first one to justify his stance more deeply. When not sufficiently satisfied by the account, the other instructor gives an alternative, more specific explanation, which builds on the dialogue up until that point. In particular, the dialogue relates to the ratios of the two compounds that react, and shifts the framing from a macroscopic to a (sub-)microscopic perspective by talking about the salt at an atomic level. This then gives a new, shared framework for the two instructors to use as a basis for discussion. In line 13, Instructor 1 adopts the microscopic perspective, but challenges the specific interpretation of the reaction by differentiating what happens on the surface of the salt from its inner structure. No proper conclusion is reached, but the reasons for the disagreement are agreed upon by pointing out their respective sub-disciplinary identity within the discipline of chemistry. The initial observation made by the instructors is accompanied by cumulative talk, but as soon as they start explaining the observation the discussion has the characteristics of exploratory talk where the ‘knowledge is made more publicly accountable’ (Mercer, 1995, p. 104) as the discussion goes on. It seems that their gazes indicate a genuineness regarding the aim of the discussion; to come to some sort of agreement. This strengthens the shared framework used to explore the phenomenon.
The students managed to explain deliquescence by use of the IR camera. However, we claim that they failed to take full opportunity of the technology in their interaction with the phenomenon. During Sequence 1, before being introduced to the IR cameras, the students showed their capability to discuss the phenomenon of deliquescence in a dynamic, multimodal way, where they coordinated exploratory talk with swift changes of posture and dynamic use of gestures. By picking up the container, they interacted directly with the phenomenon, and used hand gestures to establish shared attention and enable collaborative action (Goodwin, 2003). However, when they looked at the phenomenon with the help of IR cameras during Sequence 2, this shifted into passive observation from one perspective only, where they confirmed what they saw through cumulative talk. In contrast, the instructors engaged in exploratory talk in conjunction with dynamic changes of posture and use of gestures, also when observing with IR cameras. Why was the students’ interaction with the phenomenon more limited when they were provided with IR cameras, and thereby offered additional semiotic resources, compared to when not using the IR cameras? Why did the students and instructors differ in the use of the tools?
Firstly, both the students and the instructors were given limited time for getting to know the technology and were given detailed instructions by the researchers on what to focus on. As the students were novices both to the technology and to the phenomenon, they likely found it challenging to coordinate all the varied and novel semiotic resources at once. This may have limited the students in taking advantage of the full range of semiotic resources. Nevertheless, in this specific case, the affordances that were available to them were enough to facilitate an understanding of the phenomenon in relation to their earlier work; they could quickly confirm one of their four hypothesis of the phenomenon.
The instructors, on the other hand, easily took advantage of the movability affordance of the IR cameras’ form. By moving the IR camera and the container, they noticed heat transfer between the container and the surroundings. Thus, one could argue that the instructors were actually displaying their ability to move between various facets of ways of knowing the phenomenon fluently to transform meaning across semiotic resources. This ability is part of what Airey and Linder (2009) call discursive fluency. In this case, the discursive fluency was expressed in the instructors’ success in exploring the phenomenon with IR cameras at the same time as they explained and reflected on what they saw together, both verbally and through embodied means, switching between and coordinating semiotic resources without hesitation. The instructors had a quite rigorous knowledge of the phenomenon beforehand, with which they could compensate any inexperience with the technology, and thus still use the semiotic resources that were available to them through novel tools.
Secondly, the students’ reluctance to interact with the phenomenon in an explorative fashion may have been due to them accepting the phenomenon as part of their instructed lab work. In fact, they concluded that the moist salt involved the same exothermic reaction that was part of the regular lab. In this lab work setting, students are expected to be careful and focused in their practice and not intervene with the phenomenon. With a more active engagement, there could have been a risk of jeopardizing the accuracy of the measurements.
However, as pointed out by Andersson and Enghag (2017), students’ observations during science laboratory exercises are often accompanied by cumulative talk, a process of mutual attention where they get to agree on what to do and what they see. Such phases of observation lay the ground for subsequent explanation of their shared experiences through exploratory talk. In the present study, the students’ switch to focused observation and cumulative talk in Sequence 2 served the function to confirm one of the four hypotheses from Sequence 1. The students’ use of the IR cameras was therefore an important part of establishing an explanation of the phenomenon, and probably more valuable for their learning process than the more dynamic interaction was for the instructors, who already understood the phenomenon.
In the light of the presented results, we now revisit the research questions of the study:
Without access to an IR camera, the students came up with four possible alternative explanations of why the salt is moist, but did not get very far in their evaluation of the hypotheses. In contrast, when introduced to the IR cameras, the students quickly focused on one of the explanations.
The affordance of the IR camera is really the affordance of the many semiotic resources coordinated in the use of the tool, for example, the colors that give the image displayed on the screen a meaning that is relevant to the studied content. In this study, we chose a color scheme, which seems to have had pedagogical affordance and contributed to the intuitive power of the IR camera: ranging from white and red (high temperature) to blue (low temperature). In the data (see Sequence 2), we saw that the students noticed the colors, red and white, that are immediately associated with warmness (Kress & van Leeuwen, 2002). This then led them on to referring to heat and explaining the studied phenomenon in disciplinary terms, such as an exothermic dissolution reaction.
The students’ intuitive interpretation of the IR images shows that color in this context has pedagogical affordance in the investigation of heat. This does not exclude that the IR cameras also have disciplinary affordance, however, as the instructors were comfortable using them with a disciplinary language to explain the phenomenon. Thus, in contrast to a suggested inverse relationship between the disciplinary affordance and pedagogical affordance of individual semiotic resources (Airey & Linder, 2017), the IR camera thereby aspires to be a type of tool where high disciplinary affordance, within the science community, comes hand in hand with high pedagogical affordance in science education.
From the point of view of the chemistry discipline, the students settled for the hypothesis of absorption of water from the air, which causes an exothermic reaction and thereby an increase in temperature, which was clearly visible with the IR camera. In fact, they pointed out that this reaction is identical to the one they had studied throughout the lab, dissolution of salts in water. A possible explanation for their success with this is the additional affordance of the IR camera and its semiotic resources. Alternatively, their more advanced line of reasoning by the end of the exercise may have been a result of the experience they gained with the phenomenon throughout the lab work. Nevertheless, their laugh and expression of amazement when they saw the IR images indicate that they realized something that they thought should have been obvious all along. The IR camera, through its semiotic resources, thus affords a thermal framing of the phenomenon. The laugh implies that an abstract idea turned into an intuitive understanding, which suggests a pedagogical affordance of the tool in line with what Xie (2011, p. 885) has stated in his research:
‘From a pedagogical point of view, IR imaging has great potential to enable novel inquiry designs to transform students’ learning experience with chemistry. It significantly lowers the technical barrier of experimental skills that would have been needed to reach the same depth of science and the same level of inquiry.’
An even clearer example of the use of IR cameras for inquiry was provided by the instructors’ more active engagement with the technology. By moving the container with salt and looking at it from different angles with the IR cameras, they took advantage of the movability affordance of the tool.
The results show that the students shifted in talk, from exploratory talk, without access to IR cameras (Sequence 1), to cumulative talk, with access to IR cameras (Sequence 2). In contrast, the instructors mostly maintained exploratory talk during their investigation of the phenomenon with IR cameras, after a brief episode of cumulative talk.
Educational research has emphasized the importance of exploratory talk for learning (Mercer, 1995). Andersson and Enghag (2017) have shown that cumulative talk also is central in the learning process, particularly during data collection in laboratory exercises, in order to establish a shared point of reference. In our study, the instructors started with cumulative talk to agree on what phenomenon to attend to, before turning to explaining the phenomenon through exploratory talk. They ended their investigation by agreeing that they have different disciplinary perspectives that influence their explanations. We think that their sustained focus on exploratory talk is a sign of their expertise in relation to the studied phenomenon.
We suggest that a shift between different types of talk is important for learning, and may thereby serve as an indicator for learning taking place, which in our study is seen in the students’ interaction. When the students move from exploratory talk to cumulative talk, they go from generating hypotheses to confirming one of them. Introducing new semiotic resources may thus lead to the students discerning the disciplinary relevant aspects of a phenomenon if the resources have pedagogical affordance. This shared point of reference could be a platform for establishing an explanation of the phenomenon and for further discussions. However, for the students to reach these further discussions, another shift back to exploratory talk is needed. The shift from exploratory talk to cumulative talk is stabilizing the mutually shared knowledge. In contrast, the opposite shift from cumulative talk to exploratory talk serves to destabilize what is taken for granted and coming further in the thought process. In this study, we have shown that the IR cameras can help stabilize the knowledge, while previous research (Haglund, Jeppsson, Hedberg, & Schönborn, 2015) has shown the destabilizing process of noticing new phenomena, which contributes to the pedagogical affordance of semiotic resources of the technology.
In line with Andersson and Enghag (2017), our study suggests that cumulative talk is used to keep students on track with instruction. In our case, most students (including the ones in focus in the present analysis) described and stated what they were doing without really discussing why they were doing it. There may not have been a need for reasoning as the written instructions function as a basis for what to attend to. Thus, the exploratory talk used by the students in Sequence 1 could demonstrate that they perceived the phenomenon as something off topic, a phenomenon not related to the experiment at hand. As they were off the instructed track they needed to find a common basis for the new encounter and thereby shifted into exploratory talk. However, when they were given the IR cameras, the shift into cumulative talk suggests that they accepted the phenomenon (and the tools) as part of their instructed work and managed to relate it to the previous experiment.
The two undergraduate students quickly managed to confirm one of four generated hypotheses for why sodium hydroxide gets sticky when exposed to air when they were given an IR camera. This shows that the technology has pedagogical affordance in the area of thermal phenomena. Despite the positive impact on the disciplinary understanding of the phenomenon, however, the technology seemed to inhibit the kinds of actions and talk that characterize inquiry, which support learning and contributes to deeper understanding. These kinds of actions and talk may push the discussion into a more advanced level of understanding, in the way the instructors managed to take advantage of the full range of affordances the technology brought. This may require that the participants either have knowledge of the technology or the studied phenomenon (as the instructors had), which may be especially important if the students lack extensive experience of lab work in general. Therefore, the previously mentioned shifts, between exploratory and cumulative talk, are extra important for the students. To encourage such shifts, one could potentially add another unexpected or peculiar phenomenon, similar to the one already studied.
In the light of this, we recommend giving learners some time to familiarize with the technology of IR cameras if they are to fully appreciate and take advantage of the affordances of the semiotic resources involved in the tools. Also, it is important to reflect on how to integrate IR cameras into the laboratory practice for specific labs. The tools are useful in open-ended, inquiry-based approaches to laboratory practice (Xie & Hazzard, 2011), and the cameras could therefore contribute to such an approach by not centering the tasks around the technology. The cameras would then be used as a complement to the regular laboratory equipment. Instead of putting the cameras in the hands of the students at certain moments, they could be readily available in the lab in the same way as any other kind of lab equipment. From a methodological point of view, students’ spontaneous use of IR cameras in such circumstances would enable collection of more spontaneously occurring interactional data, adhering more closely with the tradition of conversation analysis.
We would like to thank Cedric Linder and the researchers at the Physics Education Research group of Uppsala University for valuable discussions and comments on the paper. We gratefully acknowledge the funding from Vetenskapsrådet (project VR 2016-04113). This work was also supported by the Centre for Discipline-Based Education Research in Mathematics, Engineering, Science, and technology at Uppsala University.
The authors have no competing interests to declare.
Airey, J. (2015). Social semiotics in higher education: Examples from teaching and learning in undergraduate physics. In: SACF Singapore-Sweden Excellence Seminars. Swedish Foundation for International Cooperation in Research in Higher Education (STINT).
Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. Journal of Research in Science Teaching, 46(1), 27–49. DOI: https://doi.org/10.1002/tea.20265
Airey, J., & Linder, C. (2017). Social semiotics in university physics education. In: Treagust, D. F., Duit, R., & Fischer, H. E. (eds.), Multiple Representations in Physics Education, 95–122. Cham: Springer. DOI: https://doi.org/10.1007/978-3-319-58914-5_5
Andersson, J., & Enghag, M. (2017). The relation between students’ communicative moves during laboratory work in physics and outcomes of their actions. International Journal of Science Education, 39(2), 158–180. DOI: https://doi.org/10.1080/09500693.2016.1270478
Bernhard, J. (2018). What matters for students’ learning in the laboratory? Do not neglect the role of experimental equipment! Instructional Science, (0123456789). DOI: https://doi.org/10.1007/s11251-018-9469-x
Fredlund, T., Airey, J., & Linder, C. (2012). Exploring the role of physics representations: An illustrative example from students sharing knowledge about refraction. European Journal of Physics, 33(3), 657–666. DOI: https://doi.org/10.1088/0143-0807/33/3/657
Fredlund, T., Linder, C., Airey, J., & Linder, A. (2014). Unpacking physics representations: Towards an appreciation of disciplinary affordance. Physical Review Special Topics – Physics Education Research, 10(2), 1–13. DOI: https://doi.org/10.1103/PhysRevSTPER.10.020129
Goodwin, C. (1997). The blackness of black: Color categories as situated practice. In: Resnick, L. B., Säljö, R., Pontecorvo, C., & Burge, B. (eds.), Discourse, Tools and Reasoning: Essays on Situated Cognition, 111–140. Berlin, Heidelberg, New York: Springer-Verlag. DOI: https://doi.org/10.1007/978-3-662-03362-3
Goodwin, C. (2003). The body in action. In: Coupland, J., & Gwyn, R. (eds.), Discourse, the Body, and Identity, 19–42. Hampshire: Palgrave Mcmillan. DOI: https://doi.org/10.1057/9781403918543_2
Hacking, I. (1983). Representing and Intervening: Introductory Topics in the Philosophy of Natural Science. Cambridge, UK: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511814563
Haglund, J., Jeppsson, F., Hedberg, D., & Schönborn, K. J. (2015). Students’ framing of laboratory exercises using infrared cameras. Physical Review Special Topics – Physics Education Research, 11(2), 1–22. DOI: https://doi.org/10.1103/PhysRevSTPER.11.020127
Haglund, J., Jeppsson, F., Melander, E., Pendrill, A.-M., & Xie, C. (2016). Infrared cameras in science education. Infrared Physics & Technology, 75(March), 150–152. DOI: https://doi.org/10.1016/j.infrared.2015.12.009
Kress, G., & van Leeuwen, T. (2002). Colour as a semiotic mode: Notes for a grammar of colour. Visual Communication, 1(3), 343–368. DOI: https://doi.org/10.1177/147035720200100306
Linn, M. C., & Eylon, B.-S. (2011). Science learning and instruction: taking advantage of technology to promote knowledge integration. New York, NY: Routledge. DOI: https://doi.org/10.4324/9780203806524
Mercer, N. (2004). Sociocultural discourse analysis: Analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137–168. DOI: https://doi.org/10.1558/japl.2004.1.2.137
Mercer, N. (2013). The social brain, language, and goal-directed collective thinking: A social conception of cognition and its implications for understanding how we think, teach, and learn. Educational Psychologist, 48(3), 148–168. DOI: https://doi.org/10.1080/00461520.2013.804394
Möllmann, K.-P., & Vollmer, M. (2007). Infrared thermal imaging as a tool in university physics education. European Journal of Physics, 28(3), S37–S50. DOI: https://doi.org/10.1088/0143-0807/28/3/S04
Scherr, R. E., & Hammer, D. (2009). Student behavior and epistemological framing: Examples from collaborative active-learning activities in physics. Cognition and Instruction, 27(2), 147–174. DOI: https://doi.org/10.1080/07370000902797379
Vollmer, M., Möllmann, K.-P., Pinno, F., & Karstädt, D. (2001). There is more to see than eyes can detect. The Physics Teacher, 39, 371–376. DOI: https://doi.org/10.1119/1.1407135
Xie, C. (2011). Visualizing chemistry with infrared imaging. Journal of Chemical Education, 88(7), 881–885. DOI: https://doi.org/10.1021/ed1009656