This paper investigates the potential aspects of visual literacy that might appear when visual analytics and students interact in social science secondary classrooms. Interacting with visual technology likely demands new forms of literacy as various dimensions of complexity emerge in such learning activities where reading imposes order and relevance on what is displayed. However, only a few studies have evaluated how these visual processes emerge. Applying a socio-material semiotic approach, this paper examines the interactions between teachers, students and a visual analytics application, clarifying what strengthens or weakens the socio-material relations at work in emerging visual literacy. Methodologically, a design-based research approach is chosen. Notably, it is the early stages of the designed-based research cycle that are applied. Interventions were designed and conducted in five classes in three secondary schools in Sweden (97 students). The visual analytics application introduced was Statistics eXplorer. For each class, two to three lessons were video recorded to capture how the students interacted with the application. The socio-material analyses show that the interactions between the visual analytics and the students were both strengthened and weakened by different social as well as material forces. The actions were directed by visual properties such as movement, highlighting, and color, properties that often resulted in quick vision or locked vision. This paper argues that there needs to be a close didactic alignment and deeper knowledge of how visual interfaces attract students’ attention and how students’ visual literacy emerges in that relationship.
In contemporary culture, interpreting information from any source, especially visual sources, is a crucial skill that requires developing visual literacy (Avgerinou & Pettersson, 2011). For Glazer (2011, p. 184), the growing use of information and technology requires using visual data in the form of graphs and tables. Although students often are both consumers and producers of digital visual media, they are not always skilled at interpreting visual texts, diagrams and graphs and communicating their understandings. Moreover, developing these skills is difficult. These skills require a certain level of analytical reasoning, critical thinking, and reflection. Although not new, these higher level thinking skills and learning skills have become more important (Glazer, 2011). These skills require discussion and practice (Roberts & Philip, 2006). However, teachers often lack the knowledge and resources to integrate visual literacy into their classrooms. Furthermore, teachers often assume students already know how to interpret visual data (Bowen & Roth, 2005). For example, teachers may assume that students immediately can grasp the meaning of diagrams and graphs even though reading a diagram is a learned skill (Glazer, 2011). This complex activity often results in misinterpretations of a graph’s characteristics, a graph’s content and a viewer’s prior knowledge (Bowen & Roth, 2005; Glazer, 2011; Janvier, 1998; Stenliden, 2014).
In addition, visual media such as visual analytics (VA) has transformed how data are handled and analyzed. VA offers interactive diagrams and graphs that move and change as they are displayed on an interactive digital screen. As the data change, the graphs change. These graphs provide immediate information. Although these graphs help process information efficiently, they often increase the importance of reading or translating visual information. Hence, reading interactive graphs is different from reading printed graphs. So, an examination of interactive graphs requires a consideration of how readers design their reading of these graphs (Ho, Lundblad, Åström, & Jern, 2011; Kress, 2010; Lundblad & Jern, 2012; Roberts & Philip, 2006; Stenliden, 2014).
VA tools support student learning (Lundblad & Jern, 2012; Stenliden, 2015) as these tools trigger actions in which the task, students, visualization, data types and so on interact to support proper learning activities. But the interactions between a viewer/reader and the visuals are not always straight forward and some features can be confusing. The students might shift their focus from meaning making or rather translating the data when transforming it to knowledge (Ainley, Nardi, & Pratt, 2000; Bamford, 2003; Glazer, 2011; Stenliden, 2015). Together, these aspects raise questions of how teachers can help students grasp facts from visual data, learn from visual presentation of information and communicate their findings and knowledge. The availability of VA and the growing use of computer-based graphs could change the way students learn to read graphs. Visual properties and their visual language renegotiate how reading and learning can be viewed.
If pedagogy and technology were better integrated, visual media like VA could improve the way students analyze visualized data (Stenliden, 2018). To develop this closer alignment, deeper knowledge is needed about how the visual interface captures students’ attention and how their visual literacy abilities emerge.
This paper asks how visual literacy abilities, most often referred to as the ability to understand the composition and meaning of a visual property through interpretation and analysis, are enacted in an educational setting. What are the barriers and enablers for developing visual literacy in schools? Visual literacy (VL) requires retrieving information (e.g., from graphs) and expressing the developed knowledge. VL involves critical viewing, visual reasoning, visual discrimination, visual thinking, visual association, and visual reconstruction as well as the construction and reconstruction of knowledge (Avgerinou & Pettersson, 2011). In other studies, these skills might be called ‘graph interpretation competence’ or ‘graph reading competence’ (Glazer, 2011). We use VL to include the double aspect of the skill (both retrieving information and expressing knowledge). ‘Meaning making’ is often used to describe the learning process that occurs when viewing a graph. Rather than using ‘meaning making’, we use ‘translation’ when referring to knowledge production. Latour (1987) uses ‘translation’ to describe how knowledge appears and develops, emphasizing that when actors (e.g., social viewers) and the material connect, they change one another and form links. When this happens, as in translation, knowledge progresses. The connections between both the social and the material not only work together but also work upon, or translate, each other, coordinating actions. In this process of translation, knowledge emerges. This approach emphasizes that action is shaped by relational materiality and performativity. Therefore, what can be studied is how social and material actions together constitute a particular enactment. So far, we do not know much about the performativity of a VA –i.e., we do not know how visual properties and students connect with each other and what VL might emerge. Our view is different from empirical investigations that focus on individuals’ interactions with each other and with artefacts. These studies focus on the social aspect of technological practices where the user’s application of technology is central, where artefacts (technology) are seen as tools to be used or applied according to the user’s will.
This study examines emerging socio-material relations when visual analytics is introduced to students as new equipment in social science secondary classrooms. The aim is to deliberate what potential aspects of VL that might appear when a VA and students interact in social science secondary classrooms. Three questions are investigated:
Five classrooms were video recorded to examine how students and a VA interacted. New types of literacy are probably required to effectively interact with visual media, interpret graphs and diagrams, draw conclusions from the visual properties and communicate assumptions. This paper generates information about interactive graphs and their visuals in relation to students. It also reveals new questions about how to promote and support VL processes. To meet the challenges highlighted in the introduction, educators need to understand how to better align pedagogy and technology to develop their students’ visual skills. In turn, this will help future students develop their literacy skills.
Previous research has identified several advantages for developing children’s VL skills. These skills, for example, have been linked to oral, reading and writing literacy (Stewig, 2010), to foreign language proficiency (Tomaseviae-Daneeviae, 1999), to metalinguistic skills (Callow, 2008) and to improvements in critical thinking (Williams, 2007). However, little attention has focused on how VL skills are developed, especially by young students (Lopatovska, 2016; Stenliden, Nissen, & Bodén, 2017). Moreover, few studies have examined how VL emerges when VA applications are introduced in schools (Lundblad, 2013).
VA, an interdisciplinary scientific field, is based on information visualization and cognitive and perceptual sciences. It addresses how to design and arrange interactive visual interfaces that facilitate analytical reasoning (Andrienko, Andrienko, Keim, MacEachren, & Wrobel, 2011). Two examples of interactive data visualization tools are Gapminder (Rosling, Rönnlund, & Rosling, 2007) and the Statistics eXplorer application (Ho et al., 2011). The visualization of data enhances the ability of analytical reasoning. The visual properties on the screen (e.g., semiotics such as composition, color, and images) form a multimodal connection for visual communication (Kress, 2010). Through VA in a visual exploration process, the interactions between the viewer/reader/analyst (the student) and the data visualization create relations between the viewer and the data (Andrienko et al., 2011; Tomaszewski & MacEachren, 2012). In particular, the graphical user interface requires reading and writing skills to develop knowledge from what is being communicated (Ainsworth, 2006; Bowen & Roth, 2005; Glazer, 2011; Stokes, 2002).
In a literature review, Glazer (2011) notes several major challenges with reading graphs and diagrams: confusing the slope for the height, confusing an interval for a point, conceiving a graph as constructed of discrete points, focusing on x-y trends, unclear format or inappropriate choice of visual features and teacher’s expertise (or lack of). Also, some graphs are difficult to understand because they provide too much information: ‘Instead of using the right graph it is also possible to use the wrong graph’ (Glazer, 2011, p. 197). Research also shows that graph characteristics affect how viewers interpret graphs. If the display (e.g., line graphs and bar graphs) is clearly shown and the scale of the graph (i.e., absolute or percentage) do not require additional calculations, the graph will be easier to understand (Shah, Freedman, & Vekiri, 2005). In addition to the graph type, visual properties such as color, size, scale and legend affect comprehension. Color for example, may often represent either quantitative information or categorize variables (e.g., blue for dogs and red for cats). Tufte (1983) emphasizes that the most important use of color is to distinguish one element from another. Although much research has examined the visual processing of color, depth, texture and luminance, only a few studies have examined empirical studies of diagram comprehension (Purchase, 2014). Even fewer of them focus on the literacy aspects and how these might be developed.
However, two major approaches may develop visual literacy skills (Heinich et al., 1999). One approach helps learners read or decode visuals by practicing analysis techniques. Decoding involves interpreting and creating knowledge from visual properties. For example, students need to understand (1) how information is encoded in each visual property (representation), (2) how each visual property is related to the physical world, and (3) how information in one visual property can be related to or transformed into information in another visual property (Ainsworth, 1999, 2006; Treagust, Duit, & Fischer, 2017). The other approach helps learners write or encode visuals as a tool for communication. Knowledge production often requires students to develop their visual abilities through the use of visualizations and the creation of visual messages (Eppler, 2007; Sabol, Kienreich, Muhr, Klieber, & Granitzer, 2009; Segel & Heer, 2010; Tversky & Suwa, 2009).
Unfortunately, developing VL skills challenges traditional schooling (Bamford, 2003; Johansson Svensson, Rustand, Sofkova Hashemi, & Steffensen, 2013; Stafford & Terpak, 2001; Stenliden, 2015; Åkerfeldt, 2014). Metros (2008) highlights the educator’s role in preparing visually literate learners in K–12 schools and in higher education. Stenliden (2018) argues that the didactic design of classroom activities must be adjusted when students interact with VA such as Statistics eXplorer to accommodate their VL abilities (see also Sofkova Hashemi, Petersen, & Bunting, 2013; Reynolds & Vinterek, 2013; Johansson Svensson et al., 2013). These researchers call for models that enhance not only students’ interactions with technology but also their VL, including student knowledge and particularly modes of expression that embrace visuals and a visual vocabulary. In other words, emphasis is put on the main components of VL and models that address them adequately (Avgerinou, 2007). This paper–an interdisciplinary study that includes VA, graphs, and diagrams as well as VL and education–may offer new understandings of how VL emerges when VA and students interact.
A socio-material semiotic approach (within the tradition of actor network theory) guides the examination of emerging socio-material relations because it underlines that action is formed by relational materiality and its performativity (Law, 2007). Relational materiality makes it possible to study how social and material actions perform and transform when they come together. This approach emphasizes that social and material actors are intermeshed and interdependent (Hayles, 1999) and that these are to be treated as equals (Callon, 1986). In other words, it focuses on how actions or rather distinct enactments are established. Thus, when students and VA technology are studied, working with relational materiality helps to examine not only how humans connect in new ways through technology but also how technology (digital and other) plays an active part in shaping the enactments (the practice). Hence, by employing the concept relational materiality the expectation is to clarify how actions are shaped by the socio-material actors’ performativity (VA application, the students, the teachers and other actors).
The relations or enactments between the social and material actors create networks, which can be seen as an assemblage of actors forming links through interaction (Law, 2007). Therefore, it is possible to follow and describe the kind of socio-material relations that are enacted. Here, we illustrate how actions affect, perform and translate networks.
What happens when actors connect, changing each other and forming links, is defined by the concept translation. This means that actors do not simply work with another actor; they also transform and work upon each other. This dual relationship affects and coordinates things and actions, transforming actors (Callon, 1986). These translations can vary, and may at times weaken and/or strengthen the ties among the actors (Callon, 1986). These relations may be difficult to sustain because trials of strength by actors inside or outside a network may challenge the actions of actors trying to establish links or alliances. To strengthen the relations, different lines of force may be added by the actors to further stabilize or mobilize the network (Law, 1987). Accordingly, exploring the enactments in the social science classroom by employing the concepts trials of strength and lines of force, we can examine what enables the enactments or what creates barriers for both social and material actors trying to connect. By following the actors’ actions, we track socio-material relations and discern whether and how enactments produce VL in the classrooms when lines of force and trials of strength affect enactments and translations.
Our approach connects with insights following what can be called a socio-material turn in educational studies generally and in language and literacy studies specifically (Martín-Bylund, 2017; Masny & Cole, 2009; Olsson, 2012, 2013; K. Roy, 2005; K. Roy, 2008; Stenliden, 2014; Sørensen, 2011).
Since this study focuses on socio-material relations when visual analytics is introduced to students as new technological equipment, it is necessary to design pedagogical interventions together with teachers. Therefore the first stages of a designed-based research (DBR) cycle was applied. A DBR study includes: a) situated in a real educational context, b) collaboration partnership between researchers and practitioners, c) focus on the design and an examination of a significant intervention, d) mixed methods, e) multiple iterations, f) development of design principles, and g) a practical impact on practice (Anderson & Shattuck, 2012, p. 16–18). Engaging in stages a – d in the DBR cycle enabled us, within an educational context, to set up a collaboration between researchers and teachers to design interventions. As this study is part of a larger study the later stages, e – g, will be addressed in future studies. They will focus on design principles, and examine the practical impact by also performing multiple iterations of various (enhanced) interventions (Easterday, Rees Lewis, & Gerber, 2017). To set up the collaboration, teachers and researchers formed a Teacher Research Team (TRT). The TRT consisted of 15 social science teachers and three researchers. The researchers provided theory and earlier research findings related to developing educational models aligned to VA and VL theories (Tomaszewski & MacEachren, 2012; Avgerinou & Pettersson, 2011). The teachers shared their classroom experiences, well-tried methods and didactic skills. The team members discussed their views and knowledge during four occasions (half or full day). Specific ways of using the VA, instructions, assignments and assessment methods were produced in form of lesson plans, that is, the designed interventions. Using the VA application Statistics eXplorer (Lundblad, 2013), (Figure 1), three vislets (interactive data visualizations) were also produced. More information of Statistics eXplorer is available at National Center for Visual Analytics (2017).
The vislets demonstrate official statistics of the world in an interactive and visual analytic manner that makes it possible for students to interact with the visual information, analyze it and draw conclusions. Statistics eXplorer and other VA applications intend to facilitate analytical reasoning and provide a deeper understanding that will enhance knowledge building for both students and teachers (Jern & Stenliden, 2011; Lundblad & Jern, 2012; Rosling et al., 2007). The vislets encompass demographic development and relations between indicators such as expenditure on education, health, general net income (GNI), trade and life expectancy. The developed lesson plans, the interventions, included how to implement the vislets in the classrooms.
The lesson plans were put to work in five social science classes in three secondary schools, comprising 97 K12 students aged 13–15 (grades 7, 8, and 9). After the teachers’ introduction of the vislets, the students completed the assignments in pairs. Except for the instructions, the teachers only observed and assisted the students, when requested, during the lessons.
The interventions were tested during two to three lessons at each school. Each lesson was followed by two researchers who took field notes of the emerging enactments. The enactments were also followed and captured by web cam recordings (TechSmith, 2010). The recordings captured the faces, voices and gestures of the students as well as the activities on the screens. The empirical material contains, in addition to field notes, 82 hours of web cam recordings. Before the study started, all the students (97) and their parents received information about the study, that participation was voluntary and that involvement could be terminated at any time. Written consent to participate were collected. Pseudonyms are used to protect student anonymity.
The paper has applied what Law (2004) calls ‘method assemblages’ in an effort to examine, comprehend and amplify patterns of socio-material relations found in the data. This is a tentative, and at most, partial unfolding of the data, understood as an enactment of crafting (Law, 2004). This way of working with data can help focus what will eventually be discovered as well as what may be overlooked. The pieces selected for analysis from the video captures have therefore been picked according to the concept of intensification (Stafford & Terpak, 2001) or moments of wonder (MacLure, 2013) rather than categorization and coding. This approach is a multisensory experience where perception, cognition and affect come together in an open-ended practice of sense-making (Massumi, 2002). It is also important to spell out that in this process of selecting data some relations are picked and others not. This is always a component when applying a socio-material semiotic approach and is by Strathern (1996) explained as a necessary aspect of ‘cutting a network’.
To the selected pieces of video data, the field notes added further and deepened information. From the analyzed data, we have produced two events following Callon’s three principles (Callon, 1986). These guide an open mindset when engaging with the data, the video transcriptions and the field notes. The first principle, a matter of style, means not selectively censoring the entities (the students, the screens, the teachers, the visual properties, etc.) when they enact or connect (i.e., produce relations as networks). Consequently, no pre-defined analytical classes have been used for the montage of data. The second principle–generalized symmetry–means that the same vocabulary is used to describe social and material objects (entities). The terminology has been carefully considered when the analyses are presented. The third principle–free association–abandons all a priori distinctions between material and social events, so the goal has been not to make disjunctures between different entities and their actions.
This section analyzes two events in the classrooms that depict how socio-material relations are enacted: how visuals steer the vision of the students and how VL emerges among the students.
The actions are shaped by close, intense relations among the students, teachers, the VA application and other socio-material actors. The visuals’ strong performativity direct the students’ vision. Distinct enactments are established by both the visuals and the visions working upon and transforming each other. The following events illustrate such enactments.
The event ‘Now come the jeans’ takes place in a classroom of 20 students (Grade 8) and their teacher. It focuses on two students, Ava and Kim. Their assignment is to explore where various goods and services are produced and consumed in the world. With support from the vislet, they try to draw conclusions using GNI, export and import data of different countries. The students investigate various connections. Ava and Kim focus on Bangladesh as they earlier had learned about the economic development of Bangladesh and want to find more information.
After scrutinizing the GNI and export of goods and services of the USA, Sweden and Sudan, with great astonishment and excitement, they turn their interest toward Bangladesh.
Ava: Where is Bangladesh?
At the same time, the cursor moves around the map. Whenever it passes a country, a small, black and white rectangular box appears. Inside the box, in tiny letters, the name of the country, the continent and the indicator/s chosen, are presented followed by the year and exact value (in this case the sum of exports of goods and services). The cursor moves rather haphazardly. At first, it moves around the African continent. Then it slowly turns toward Asia. Ava, positioned rather closely to the screen, repeats her question. As the cursor passes India, the letters of Bangladesh are visible for an instant. The cursor continues its movement and the names of other countries appear immediately. Then the cursor stops.
Ava: There, I found it! There it is!
The cursor clicks on the area at the map and the national boarders are highlighted in dark black. Because of the dynalink function, (Figure 2), simultaneously the bubble of Bangladesh in the scatterplot and the bar of Bangladesh in the bar chart are highlighted.
Ava: Ok, now I have highlighted it.
The cursor moves toward the bar chart that displays values of life expectancy.
Ava: Here somewhere.
The cursor moves the fisheye function of the bar chart and seconds later the bar of Bangladesh is in sight. Within milliseconds, Ava’s gaze finds the highlighted bar of Bangladesh.
Ava: There! … And then play!
The cursor clicks the start button on the timeline and the screen starts to display the development of exports of the countries from 1960 until 2014. Simultaneously, the colors of the map, the bubbles of the scatterplot and the bars move according to the values of each year. The cursor moves close to the highlighted bar of Bangladesh and starts to follow the movements of the bar.
During the display of the timeline, the movement is reinforced as the cursor follows the bar. The girls’ visions and the cursor follow the bar of Bangladesh closely. The girls move closer to the screen and make loud expressions when following the economic development of Bangladesh. The girls comment on the economic decline, suddenly the bar moves to the left, signalling an economic turning point.
Kim: Now! Now come the jeans! Now come the jeans!
Simultaneously, Ava calls out, Wow, wow, wow! Wow! It just ssscchhh …
Ava makes a swishing noise with her mouth and at the same time she lifts her arm, shaped like an arrow, it moves past the screen in the same direction as the bar.
The girls talk and laugh about their expression ‘Now come the jeans’ and watch the timeline demonstrating the development again.
At the beginning of the event, Ava and the cursor try to find Bangladesh. The cursor circulates over the world map, passing over many countries. This enactment causes a small text box with tiny letters to appear. Working together; the cursor, the map, the box and Ava’s gaze make it possible to find the borders of the country. This shows how socio-material relations emerge between these actors.
These enactments show how the human and non-human actors are intermeshed and interdependent. The enactments also show how the actions are shaped by the tight and intense relations between the actors involved. These relations are evident, for example, in the emotional and bodily expressions as well as in the girls’ lively exclamations when they discover the economic development of Bangladesh by the moving, highlighted bar of Bangladesh. In this relational materiality, the actors’ actions can be understood as performative, where the visuals and the vision shape each other. The performativity, for example, is demonstrated when the cursor moves around the map while Ava is searching for Bangladesh. The tiny letters in the box signifying Bangladesh are visible for just an instant. When the box disappears, Ava and the cursor continue the movement beyond the box but can make the textbox visible again when returning to the area. The performativity of the text box when it first ‘flashes’ influences where Ava looks and where the cursor moves. As these actors connect, they affect the actions, as in translation, and they gain new opportunities to form links.
The translations of visual properties ‘movement’ and ‘highlighting’ shape the students’ vision. For example, later in the event, when the actors highlight Bangladesh and the movement of the timeline starts, the actors change and form new links. That is, when the cursor moves toward the bar chart and Ava says ‘here somewhere’, indicating she is close to finding what she is looking for. When Ava and the cursor make the fisheye function move, changing the timeline, Ava finds the highlighted bar of Bangladesh. What appears when Ava connects to the visuals produces a ‘quick vision’. Her vision is affected, she instantly spots what she is looking for. Her gaze rapidly connects to that area on the map. The translation that emerges is supported by several actors. The properties of these visuals are characterized by movement: when the cursor or timeline move and when Ava connects with the highlighted bar that displayed Bangladesh through quick vision. As the timeline starts to run and the colors of the map, the bubbles of the scatterplot and the bars on the bar chart move, the cursor follows the highlighted bar for Bangladesh, steadily following the movements of the bar on the screen.
Another kind of translation also emerges as the students’ vision and the properties of the visuals enact/connect. The highlighted bar and the movement of the cursor seem to lock the vision of the students. When the cursor follows the highlighted bar, the visuals’ performativity seems to direct the students’ vision. The visuals produce a ‘locked vision’. The performativity holds the gaze in place and locks their vision on a specific area. A locked vision enables the students to follow the bar, comment and make conclusions about the economic development of exports. Throughout the display of the timeline, the cursor follows the highlighted bar and the girls’ vision is focused, i.e., their vision is locked.
Quick vision and locked vision together with visual properties such as highlighting and movement seem to enhance the translations in the network. However, when something is spotted by quick vision or focused by locked vision, other things might not be seen. That is, performativity might affect the student’s ability to have an overview of the screen. Such translation occurs in the following interaction.
Ava: Ok, let’s see here.
The cursor moves around in the vislets’ textbox and stops at the black text articulating ‘Investigate GNI compared to the import of countries’. The word import is an orange link. Right away, the cursor clicks on the link. At the top of the screen, the bar chart immediately displays the values of imports. The girls do not notice this change; they are concentrated on the orange link. Above the bars, in black text, the indicator list demonstrates ‘import of goods and services’. The girls’ vision continues to focus on the orange link.
Kim: Press it again.
The cursor clicks several times on the orange link. The girls lean toward the screen and still look at the link, unable to continue with the assignment.
Not until they call for help from the teacher and she makes them aware that the right values (imports) already are demonstrated in the vislet, the girls continue to work on the assignment again.
When Ava spots the orange link, the visual property of ‘color’, she interacts with the cursor and the appearance of the vislet changes a little: the values of ‘import’ appears at the top of the screen. The orange color first produces quick vision and then Ava’s gaze is held firmly, becoming locked vision, making the changes at the top of the screen less apparent, almost invisible to both Ava and Kim.
In this case, the trial of strength, the orange link producing locked vision, interrupts the enactment between the students and the interface of the visual analytics. Hence, the students are unable to continue with the assignment. The girls gradually lose interest, so the network is weakened. They call for help from their teacher as a line of force. The teacher starts to read the text of the orange link with emphasis on the word ‘import’ and makes the girls aware of the changed indicator expressing ‘import of goods and services’. This line of force makes the girls connect to the network again as they now notice the text. The explanation and demonstration by the teacher enable the girls to continue their relations with the vislet. The trial of strength is overcome.
All together, these enactments can be seen as examples of how actors do not simply work with another actor, they also transform and work upon each other.
The performativity of the visual properties shown above, such as movement, highlighting and color may produce both quick and/or locked vision. When quick and locked vision are produced, the enactments between the students and the visual interface may at times weaken the possibilities to build connections/enact translations, but at other times the performativity may strengthen those possibilities. Movement, highlighting and color offer the possibilities for students to succeed as these visual properties engage the students’ eye movements and they influence each other. As explained earlier, when the eye is locked on specific visual properties, there is the possibility that other objects or information may be ignored. As the actors influence each other, the translations in the classrooms are characterized by being varied in force and durability and the process of holding together the socio-material relations between the actors is insecure and precarious.
Not only does the visuals’ performativity cause trials of strength, but also it is significant how assignments are constructed and formulated. This will be demonstrated in the next event, ‘Check the red one’. This event takes place in a class of 18 students (Grade 7). The vislet is about demographic composition and one assignment is to examine how population is stratified by age (Figure 3).
Cate: How is the population divided? I don’t know.
The cursor moves upwards from the text box, where the questions about population are written, to the map and the scatterplot. Jim shakes his head and moves closer to the screen, Cate leans back from it.
Jim: Ehh, but we can check that out here.
The cursor moves back to the text with three orange links displaying the words “children,” “adults,” and “elderly.” The cursor clicks on the link “children,” and the map immediately switches from a mix of colors to blue over almost all the country (Figure 4).
This indicates a low percentage of children in the municipalities. The scatterplot also switches into demonstrating the percentage of children. The red color indicates a high percentage. The map consists of blue-colored municipalities almost everywhere. Exceptions are one light yellow and four small orange areas. One tiny municipality is red. When the students look at the map, the light yellow municipality first draws their attention and the cursor starts to move there, but within a millisecond Cate spots the tiny red area. Both Cate and Jim move close to the screen.
Cate: Check that out! Check the red one!
Jim: Oh, there live …
The cursor moves to the red municipality. The tooltip function makes a black and white box appear with the text ‘Knivsta’ and the percentage of children (Figure 5).
She then looks at the scatterplot, points with her pen, and instructs Jim to move the cursor to the red bubble high in the right corner.
Cate: Click on that! Is it Knivsta?
The cursor moves to the red bubble, tool tips the highest of all bubbles, and the text “Knivsta” becomes visible (Figure 4). The students read “23.9%.”
As Cate reads the assignment’s question, she immediately stops reading after the question mark to say that she does not know the answer or what to do. Her interactions in the network cease as she leans back from the screen. Here the trial of strength is probably how the question is formulated and placed within the text box. The question – “How is the population divided?” – can be interpreted in several ways. For example, a population can be divided in relation to nationality, income, or sex. In this case, it is implied that the question refers to demographic composition, how different ages of people are distributed in different areas of the country. Cate seams not to connect with the textual explanation of the assignment that follows the question mark nor does she connect to the orange links that also are placed after the question. This might have been different if the question were formulated more precisely and if it were placed in the text box after the explaining text. As is, the question demands an action that she is unable to connect with or respond to. What at first may look as a lack of interest (i.e., Cate leaning back from the screen) may instead be interpreted as actors’ relations in the network being dissolved.
Jim also hesitates at first, but then he explains where they can check it out. Jim becomes a line of force to Cate who thereby is able to connect to the network again. Within a short time, Cate reconnects with the visuals as the cursor clicks on the orange link ‘children’ and the interface of the vislet changes as the map turns almost entirely blue. This shift in colors seems to attract Cate’s vision. The switch, the changed view, evokes her attention. The lines of force – the boy suggesting where the answer might be, the cursor clicking the orange link and the interface’s change of colors, strengthen the translations, encouraging Cate and Jim to move closer to the screen to explore the map and the scatterplot. The trial of strength is overcome and a successful translation is possible.
Several aspects of VL emerged simultaneously as other actions were enacted in the networks. One of the abilities is to discern differences between visuals, visual discrimination (Avgerinou, 2007). Visual discrimination was put to work when the visuals stood out, attracting the students’ vision. But as the students’ vision was drawn to visual contrast, other visual information was obscured. If students fail to encounter the visuals productively, visual discrimination might misdirect the student’s focus, diminishing their chances to finish an assignment as in the example of the orange link. However, if directed effectively in relation to the visuals, students are able to interpret information, draw conclusions and successfully finish assignments. Another VL ability, visual association, was put to work when the visual properties were associated with other visuals at the interface or associated with earlier insights (Avgerinou, 2007). For example, Cate and Jim tried to confirm their insights by connecting with the visuals and the dynalink function of the map and the scatterplot. They associated these visuals, the red spot on the map (Knivsta), with the red bubble placed highest in the scatterplot. This visual association allowed them to compare these visuals to earlier insights. When the visual properties made it possible for the students to understand a visual message and to connect the message to other information, they were constructing meaning. This ability emerged in the event of the jeans when the timeline displayed the annual economic development of Bangladesh. Here, Kim and Ava concluded from the visuals that Bangladesh’s economic development was related to the production of jeans. As Kim stated, ‘Now come the jeans’, Ava reinforced this meaning by shaping her hand like an arrow, moving it in the same direction as the bar, and even making a swishing noise. Previously acquired knowledge of Bangladesh allowed them to construct meaning. The visual message was connected to other information.
For most of the classroom enactments, the students reasoned together with the visual properties. This kind of literacy – visual reasoning–was difficult to differentiate from other aspects of VL. All through the lessons, visuals on the screen in general but specifically properties in form of highlighting, movement, and color, connected to the students’ vision, generating discussions that illustrated the students’ logical thinking. The visuals made it possible for the students to read graphs, reflect, communicate their understandings, and reason in coherent ways.
Our point of departure is that the enactments between the social and material actors not only work together, but also work upon, or translate, each other and coordinate things and actions. Altogether, the results in this paper point to the importance of being aware of different aspects of relational materiality and the performativity in the information process, the simultaneous emerging VL aspects and educational design that supports the students’ visual information process and models their development of VL abilities.
The paper shows how VA and students interact in social science secondary classrooms and how VL emerges as the socio-material relations connect and construct networks. These networks produce several aspects of VL as the actors (the VA, the students, the visual properties and the gazes) interact. This study shows how the actors, when reading diagrams and graphs, are interlinked, working upon and influencing each other. The actions in the network are often intense; there are quick changes of the interface, emotional outbursts, diagram switching, reinforcing gestures, bars moving rapidly, concentrated gazes, bodies leaning closer to the screen, text boxes appearing and cursor movements. The networks produced by the socio-material relations are highly complex, a view in line with Glazer’s (2011) belief that graph reading as a complex activity. In this paper, the complexity of reading a graph is also explained by several aspects such as confusing the slope for the height and confusing an interval for a point (Glazer, 2011) or aspects that might confuse, for example color, size, scale, and legend (Shah et. al., 2005) or encode the visual (Ainsworth, 1999, 2006). This paper shows how network actions, related to diagram reading, are enacted and shaped by the actors’ performativity. Visual properties in the VA application such as highlighting, movement, and color affect the reader’s gaze, producing quick vision or locked vision. There are multiple possibilities that connect visuals and vision. This multiplicity adds an additional dimension to the complexity defined in earlier studies. In this aggregated complexity, the mutual performativity by the socio-material actors affect the possible relations in the networks. Clearly, the visual properties and the two kinds of vision might sometimes act as either trials of strength or lines of force in the networks. Therefore, the performativity of the visuals and/or the visions and how they connect might weaken, dissolve, or strengthen the networks. Graph reading is designed in close collaboration between the visual properties and the vision that might be produced. The performativity of the visuals easily engaged the students’ vision. Other aspects that feature the possible translations could include how assignments are presented as well as how text and visuals are placed in relation to each other. Pressure might be put on the network if the assignments lack precision or information is placed without concern.
Our findings about how VL emerges are in line with Bamford (2003) who showed that visuals help students gather information and that it can be difficult to interpret visuals as they also may confuse the intended message. Developing VL abilities seems essential for embracing visual discrimination, visual association, constructing meaning, visual reasoning and reading graphs. This is in line also with Stenliden (2017), Sofkova Hashemi et al. (2013), Reynolds and Vinterek (2013), Johansson Svensson et al. (2013) and Glazer (2011).
To help learners read or decode interactive diagrams and graphs in a complex network, it seems vital to practice analysis techniques (c.f Heinich et al., 1999). Like Ainsworth (2006), we suggest that it is vital for students to understand how information is encoded, but we would like to add that students might be supported by helping them see how each visual property might attract their eyes. We also put forward, as argued by Treagust, Duit, and Fischer (2017), that it is important for students to understand how each visual property is related to the physical world. In addition, students may benefit from understanding how visual properties might produce attention towards phenomena that are not in focus for the investigation and how the students might end up with information in relation to the physical world of which they had been previously unaware.
It is important to understand that these skills need to be learned and constantly practiced (Heinich et al., 1999). This reinforces the importance of teachers and students building an awareness of the strong impact of the visuals on vision. Therefore, teachers and students should understand the importance of recognizing the aggregated complexity highlighted in this paper. Teachers and students might have to consider the strong performativity of visuals and how they attract attention and vision more or less completely. If students are to create meaning through visual interpretation, to be visually literate, then teachers need to create a didactic design that carefully supports their students’ visual information processes and models how they develop VL abilities. This includes augmented knowledge of what attracts students’ attention and how to take advantage of it in the process of their becoming visually literate.
In all, this means that future research plans include taking into account the problems of the aggregated complexity and how to best support students emerging visual literacy abilities. By applying the later stages of a full DBR cycle, enhanced designs of new interventions with the TRT will be possible. Hopefully, the development of design principles will contribute to valuable and practical impact on practice.
The authors have no competing interests to declare.
Ainley, J., Nardi, E., & Pratt, D. (2000). The construction fo meanings for trend in active graphing. International Journal of Computers for Mathematical Learning, 5(2), 85–114. DOI: https://doi.org/10.1023/A:1009854103737
Ainsworth, S. (1999). The functions of multiple representations. Compututers & Education, 33(2–3), 131–152. DOI: https://doi.org/10.1016/S0360-1315(99)00029-9
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198. DOI: https://doi.org/10.1016/j.learninstruc.2006.03.001
Anderson, T., & Shattuck, J. (2012). Design-Based Research. Educational Researcher, 41(1), 16–25. DOI: https://doi.org/10.3102/0013189X11428813
Andrienko, G., Andrienko, N., Keim, D., MacEachren, A. M., & Wrobel, S. (2011). Challenging problems of geospatial visual analytics. Part Special Issue on Challenging Problems in Geovisual Analytics, 22(4), 251–256. DOI: https://doi.org/10.1016/j.jvlc.2011.04.001
Avgerinou, M. D. (2007). Towards a Visual Literacy Index. Journal of Visual Literacy, 27(1), 29–46. DOI: https://doi.org/10.1080/23796529.2007.11674644
Avgerinou, M. D., & Pettersson, R. (2011). Toward a cohesive theory of visual literacy. Journal of Visual Literacy, 30(2), 1–19. DOI: https://doi.org/10.1080/23796529.2011.11674687
Bowen, G. M., & Roth, W. M. (2005). Data and graph interpretation practices among preservice science teachers. Journal of Research in Science Teaching, 42(10), 1063–1088. DOI: https://doi.org/10.1002/tea.20086
Callon, M. (1986). Some elements of a sociology of translation: domestication of the scallops and the fishermen of St Brieuc Bay. In: Law, J. (Ed.), Power, action, and belief: A new sociology of knowledge, 196–233. London: Routledge & Kegan Paul.
Callow, J. (2008). Show Me: Principles for Assessing Students’ Visual Literacy. The Reading Teacher, 61(8), 616–626. DOI: https://doi.org/10.1598/RT.61.8.3
Easterday, M. W., Rees Lewis, D. G., & Gerber, E. M. (2017). The logic of design research. Learning: Research and Practice, 1–30. DOI: https://doi.org/10.1080/23735082.2017.1286367
Eppler, M. (2007). Toward a visual turn in collaboration analysis? Building Research & Information, 35(5). DOI: https://doi.org/10.1080/09613210701355724
Glazer, N. (2011). Challenges with graph interpretation: A review of the literature. Studies in Science Education, 47(2), 183–210. DOI: https://doi.org/10.1080/03057267.2011.605307
Hayles, N. K. (1999). How we became posthuman: Virtual bodies in cybernetics, literature, and informatics. Chicago, Ill.: Univ. of Chicago Press. DOI: https://doi.org/10.7208/chicago/9780226321394.001.0001
Ho, Q., Lundblad, P., Åström, T., & Jern, M. (2011). A Web-Enabled Visualization Toolkit for Geovisual Analytics Visualisation and Data Analysis. Paper presented at the A Web-Enabled Visualization Toolkit for Geovisual Analytics Visualisation and Data Analysis, San Francisco.
Janvier, C. (1998). The notion of chronicle as an epistemological obstacle to the concept of function. Journal of Mathematical Behavior, 17(1): 79–103. DOI: https://doi.org/10.1016/S0732-3123(99)80062-5
Johansson Svensson, A., Rustand, K. A., Sofkova Hashemi, S., & Steffensen, T. (2013). Students’ Use of Semiotic Structures in Synchronous Computer-Mediated Communication – An Inter-Scandinavian Study. Paper presented at the ECTC-2013, The European Conference on Technology in the Classroom: “The Impact of Innovation: Technology and You”. Brighton, UK.
Law, J. (1987). Technology and heterogeneous engineering: The case of Portuguese expansion. In: Bijker, W. E., Hughes, T. P., & Pinch, T. J. (Eds.), The social construction of technological systems: New directions in the sociology and history of technology, 111–134. Cambridge, Mass.: MIT Press.
Law, J. (2007). Actor Network Theory and Material Semiotics. version of 25th April 2007. Available at: http://www.heterogeneities.net/publications/Law2007ANTandMaterialSemiotics.pdf (downloaded on 4th of January, 2018).
Lopatovska, I. (2016). Engaging young children in visual literacy instruction. Proceedings of the Association for Information Science and Technology, 53(1), 1–5. DOI: https://doi.org/10.1002/pra2.2016.14505301101
Lundblad, P., & Jern, M. (2012). Visual Storytelling in Education Applied to Spatial-Temporal Multivariate Statistics Data. In: Dill, J., Earnshaw, R., Kasik, D., Vince, J., & Wong, P. C. (Eds.), Expanding the Frontiers of Visual Analytics and Visualization, 175–198. Springer.
Martín-Bylund, A. (2017). Towards a minor bilingualism: Exploring variations of language and literacy in early childhood education. (33 Doctoral thesis, comprehensive summary), Linköping: Linköping University Electronic Press. Retrieved from: http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-139853 DiVA database. DOI: https://doi.org/10.3384/diss.diva-139853
Massumi, B. (2002). Parables for the virtual: Movement, affect, sensation. Durham, N.C.: Duke University Press. DOI: https://doi.org/10.1215/9780822383574
National Center for Visual Analytics. (2017). Statistics eXplorer. Retrieved 2019-02-06 from: https://ncva.itn.liu.se/explorer?sc=true&l=en.
Olsson, L. M. (2013). Taking Children’s Questions Seriously: The need for creative thought. Global Studies of Childhood, 4(2), 230–253. DOI: https://doi.org/10.2304/gsch.2013.3.3.230
Purchase, H. C. (2014). Twelve years of diagrams research. Journal of Visual Languages & Computing, 25(2), 57–75. DOI: https://doi.org/10.1016/j.jvlc.2013.11.004
Roberts, S., & Philip, R. (2006). The grammar of visual design, 22(2). DOI: https://doi.org/10.14742/ajet.1299
Roy, K. (2005). On Sense and Nonsense: Looking Beyond the Literacy Wars. Journal of Philosophy of Education, 39(1), 99–111. DOI: https://doi.org/10.1111/j.0309-8249.2005.00422.x
Sabol, V., Kienreich, W., Muhr, M., Klieber, W., & Granitzer, M. (2009). Visual Knowledge Discovery in Dynamic Enterprise Text Repositories. Paper presented at the 13th International Conference on Information Visualisation, IV Barcelona, Spain. DOI: https://doi.org/10.1109/IV.2009.35
Segel, E., & Heer, J. (2010). Narrative Visualization: Telling Stories with Data. IEEE Transactions on Visualization and Computer Graphics, 16(6), 1139–1148. DOI: https://doi.org/10.1109/TVCG.2010.179
Shah, P., Freedman, E. G., & Vekiri, I. (2005). The Comprehension of Quantitative Information in Graphical Displays. In: Miyake, A., & Shah, P. (Eds.), The Cambridge Handbook of Visuospatial Thinking, 426–476. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511610448.012
Sofkova Hashemi, S., Petersen, A-L., & Bunting, L. (2013). En elev en dator i grundskolans tidigare år: En analys av didaktiska förhållningssätt utifrån perspektiv på lärarens ledarskap, texter och textpraktiker, samt språklärande. Trollhättan: Högskolan Väst.
Stafford, B., & Terpak, F. (2001). Revealing Technologies/Magical Domains’ in a Box to Images on a Screenh. In: Stafford, B., & Terpak, F. (Eds.), Devices of Wonder: From the World in a Box to Images on a Screen, 1–109. Los angeles: Getty Research Institute.
Stenliden, L. (2014). Visual Storytelling Interacting in School Learning Conditions in the Social Science Classroom. (20), Linköping: Linköping University. Retrieved from: http://liu.diva-portal.org/smash/record.jsf?parentRecord=diva2%3A410571&pid=diva2%3A719298&dswid=9340. DOI: https://doi.org/10.3384/diss.diva-106885
Stenliden, L. (2015). Visual Analytics in K12 Education – Emerging Dimensions of Complexity. International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 9(2), 663–671.
Stenliden, L. (2018). Geovisual Analytics in School: Challenges for the Didactic Design of the Classroom. International Journal of Information and Education Technology, 8(3), 178–185. DOI: https://doi.org/10.18178/ijiet.2018.8.3.1030
Stenliden, L., Nissen, J., & Bodén, U. (2017). Innovative didactic designs: Visual analytics and visual literacy in school. Journal of Visual Literacy, 1–18. DOI: https://doi.org/10.1080/1051144X.2017.1404800
Stewig, J. W. (2010). First Graders Talk About Paintings. The Journal of Educational Research, 87(5), 309–316. DOI: https://doi.org/10.1080/00220671.1994.9941259
Strathern, M. (1996). Cutting the network. Journal of the Royal Anthropological Institute, 2(3), 517–535. DOI: https://doi.org/10.2307/3034901
Tomaszewski, B., & MacEachren, A. M. (2012). Geovisual analytics to support crisis management: Information foraging for geo-historical context. Information Visualization, 11(4), 339–359. DOI: https://doi.org/10.1177/1473871612456122
Treagust, D. F., Duit, R., & Fischer, H. E. (2017). Multiple Representations in Physics Education [Elektronisk resurs]. Cham: Springer. DOI: https://doi.org/10.1007/978-3-319-58914-5
Tversky, B., & Suwa, M. (2009). Thinking with sketches: Tools for Innovation. In: Markman, A. B. A. W. K. L. (Ed.), The Science Behind the Practical Methods that Drive Innovation, 75–85. Oxford: Oxford University Press.
Williams, T. L. (2007). “Reading” the Painting: Exploring Visual Literacy in the Primary Grades. The Reading Teacher, 60(7), 636–642. DOI: https://doi.org/10.1598/RT.60.7.4