Start Submission Become a Reviewer

Reading: The Multimodal Meaning-Making Process in Educational Design Team Meetings

Download

A- A+
Alt. Display

Research

The Multimodal Meaning-Making Process in Educational Design Team Meetings

Author:

Dewa Wardak

The University of Sydney, AU
X close

Abstract

The aim of this study is to contribute to a better understanding of the nuances of multimodal communication through which educational designers construct shared meanings and ideas. Educational design is a broad and multi-faceted area. Meaning-making in face-to-face educational design meetings is a complex process but is yet to gain attention as a topic of research. Most educational design projects involve various stakeholders, including professional designers and people who offer distinct expertise according to the needs of the project. Effective communication is imperative in such environments. We need a thorough understanding of how educational designers work together before we theorise their practice and attempt to create prescriptive tools and process models. The analysis of the meaning-making process reported in this paper shows that educational design is accomplished through intricate verbal and non-verbal interaction with various tools, resources, and representations. In particular, the results show how educational designers reconstructed pre-created visual design artefacts through drawing inscriptions over them – blending digital and non-digital artefacts. In addition, this study shows that gestures are indispensable to the meaning-making process in educational design team meetings. It is important to build a body of knowledge in this area that supports the practice and research in educational design. This paper would be of primary interest to researchers who study the way in which educational design is accomplished. Beyond this, those who work as educational designers or those who work with educational designers, might benefit from an awareness of the different interactional tools at their disposal.
How to Cite: Wardak, D. (2020). The Multimodal Meaning-Making Process in Educational Design Team Meetings. Designs for Learning, 12(1), 56–70. DOI: http://doi.org/10.16993/dfl.117
140
Views
24
Downloads
6
Twitter
  Published on 26 Jun 2020
 Accepted on 24 Apr 2020            Submitted on 13 Jun 2018

Introduction

Borrowed from cognitive semiotics, meaning-making in design is concerned with the construction of meaning – that is, ‘a thought induced in the receiver, which is originated by the contact with a design’ (Kazmierczak, 2003: 47). It is an interpretive process which is the result of interaction between people and design artefacts and shapes how humans act toward designs (Ma, 2015) or representations of designs.

The process of meaning-making in educational design team meetings is yet to become a topic of research. It is important to build a body of knowledge in this area that supports the practice, research and teaching of educational design. A thorough understanding of how educational designers work is essential because the educational design ‘field is replete with prescriptive theories and systemic process models that attempt to theorize practice, often without a rich and detailed understanding of that practice’ (Gray et al., 2015: 43).

Effective communication is considered imperative for educational designers to succeed in their roles (Ritzhaupt & Kumar, 2015). This communication is often multimodal (e.g. talk, gestures, posture, gaze, and drawing). The various communicative modes complement and supplement each other by working in concert (Tversky, Heiser, Lee, & Daniel, 2009). While some modes are lasting others are fleeting. Gestures are fleeting, and to some extent, words too can easily escape our attention. Inscriptions on the other hand leave a visible, more persistent and tangible record of interaction. Each communicative mode has a role to play in the meaning-making process. Gestures in particular are an important part of multimodal interaction with the environment, including drawings, which can enrich our understanding of how people communicate and collaborate (Davidsen & Christiansen, 2014; Frølunde, 2009).

We invited a team of thirteen professionals who were working on the design of an educational game to meet in the Educational Design Research Studio (EDRS or Design Studio for short). This design meeting was a brainstorming session. It was an opportunity to bring all the project contributors and stakeholders together and accumulate a list of ideas for further development of the game. This design session was part of a real-world project to create a game to teach school-age children about environmental issues such as the sustainable use of resources. The focus of this paper is not related to how the children learn through the game, but on how the educational designers jointly constructed meaning during the team meeting while working with various design representations including a logo, a diagram and preliminary interface designs for the game.

The paper begins by defining what is meant by educational design and how gestures can communicate meaning in face-to-face interaction. This is followed by presenting the relevant research on inscriptions and gestures in design. The methods section describes how meaning-making can be employed as an analytical framework and explains the process of analysis in this study. The results of this study are presented in three sections that describe the meaning-making process through the creation of hybrid inscriptions, the reconstruction of meaning through the globe inscription, and meaning-making while enacting the globe gesture. This is followed by the discussion and conclusion.

Defining educational design

Educational design is a broad and multi-faceted area. In general, educational design can be defined as ‘a series of approaches, with the intent of producing new theories, artefacts, and practices that account for and potentially impact learning and teaching in naturalistic settings’ (Barab & Squire, 2004: 2). Educational designers are people who are involved in designing things that help other people learn (Goodyear & Dimitriadis, 2013). They work with different kinds of solutions including educational products, programs, processes and policies (McKenney & Reeves, 2019). They often work on solutions to educational problems together with various stakeholders. Educational design research can be characterised as interventionist, which focus on understanding and improving interventions that are evaluated in part by their practicality for users (Van den Akker, Gravemeijer, McKenney, & Nieveen, 2006).

Educational design in this study is defined as ‘the set of practices involved in constructing representations of how to support learning in particular cases’ (Goodyear, 2005: 82). This definition is more in line with the focus of this paper on meaning-making by educational designers while working with various design representations.

Gestures

Gestures are an important part of multimodal interaction. Gestures are movements of the hands and arms during which the hands represent something other than themselves (McNeill, 1992). They are a visible form of interaction and can be as important for effective communication as language and inscriptions are.

Gestures synchronise with speech at certain points where both embody shared underlying meanings (McNeill, 2005). This can happen at two levels. The first is when gesture is sub-ordinate to spoken words (Norris, 2011). In this situation gesture builds up one system together with spoken words, but if the gesture is omitted it does not render the spoken words meaningless. For example, if a person is looking at a paining while stating that she likes ‘this’ painting, it does not render the spoken words meaningless if she does not point to the painting as well. The rest of the context often clarifies the intentions of the speaker. The second is when gesture and spoken words are on equal level (Norris, 2011). The omission of the gesture in this situation may affect communication. An example is gesturing movement or trajectory with hands. Gesture can pass through five phases. These may include preparation, pre-stroke hold, the stroke itself, post-stroke hold, and retraction (McNeill, 1992). Except for the stroke itself, which is the most visible part of the gesture, the other phases are optional.

When the same gesture is repeated, it is called a cohesive gesture (McNeill, 1992). The repetition of the gesture illustrates the continuation of a theme. It is said that the repeated cohesive gesture can become a ‘local semiotic resource’ for talking about certain concepts developed during face-to-face interaction (Koschmann & LeBaron, 2002). These kinds of gestures have been observed in laboratory settings (Becvar, Hollan, & Hutchins, 2008) and in pilot training situations (Nomura & Hutchins, 2007).

Research on inscriptions and gestures in educational design

This paper reports the process of meaning-making by educational designers while they were working with visual representations and hand drawings.

Drawing is an important activity in most design fields. Drawing and sketching in design is an act of exploration (Hokanson, 2008), discovery (Berger, 2007), seeing new relations among elements, promoting new ideas and refining current ones (Suwa & Tversky, 2002), and an aid to a designer’s thought process (Buxton, 2007). Goldschmidt (2003) states that designers often do not need to draw perfectly, rather, they should be able to use drawing to reason with on the fly.

In this study the term inscription is used to refer to the drawings created by the educational designers. This is because educational designers often draw abstract ideas rather than concrete entities (Wardak, 2016b). The term inscription is borrowed from science and technology studies (Latour, 1987; Roth & McGinn, 1998) and refers to all types of visual marks created through writing, sketching, drawing, and engraving onto a surface. The inscriptions reported in this study are produced during the conceptual phase of design. Bilda, Gero and Purcell (2006) state that the conceptual phase of design is at an early stage of the process; this is when designers are coming up with new ideas and proposed schemes. The drawings created during this phase are tentative and exploratory, functioning as aids in the decision making process.

Apart from a few exploratory studies (Stubbs, 2006; Wardak, 2016b), we know very little about the process of drawing in educational design team meetings. We know even less about how educational designers use gestures in face-to-face design sessions (Wardak, 2016a).

In the general domain of design, the study of gesture has become a topic of research relatively recently (Bekker, Olson, & Olson, 1995; Donovan, Heinemann, Matthews, & Buur, 2011; Herold & Stahovich, 2011; Luck, 2010, 2014; van den Hoven & Mazalek, 2011; Visser, 2009, 2010; Visser & Maher, 2011). Many gestures in design team meetings are enacted in relation to design artefacts, such as drawings (Artman, House, & Hultén, 2014; Bekker et al., 1995). Visser and Maher state, ‘The use of gestures in the construction of representations of design objects is fundamental’ (Visser & Maher, 2011: 216). However, the majority of studies in design still seem to overlook the importance of gestures in face-to-face interaction.

Methods

Multimodal meaning-making as an analytical tool in design

If we consider design as a trigger or an interface for meaning-making, we can analyse ‘what actually happens when the receiver infers meaning from the design’ (Kazmierczak, 2003: 47). In other words, we can investigate how the receiver responds to the designed entity. According to this perspective, design is considered as communication. There are various perspectives on the communicative potential of designs (Crilly, Good, Matravers, & Clarkson, 2008). One perspective is that designed entities themselves have the potential to communicate meaning and so the main goal of the designer is to successfully transfer the intended meaning to the receiver (Waltersdorfer, Gericke, Desmet, & Blessing, 2017).

The perspective taken in this paper is that meaning is not entirely located in the design artefact but is actively constructed through the receiver’s contact with the design (Kazmierczak, 2003). Therefore, by investigating how receivers respond to the design we can gain some insight into how they comprehend the design. In other words, since we cannot see how people think, we can make inferences from their observable patterns of behaviour. Hutchins states, ‘in some activity settings, acting in the world is thinking’ (Hutchins, 2006: 391, emphasis in original).

Acting and interacting in the world is always multimodal (Norris, 2002, 2004). Accordingly, the analysis in this study included the designers’ speech as well as gestures, drawings, gaze, posture, and object-handling. Particular attention was given to how the designers responded to the design representations or artefacts.

The process of analysis

The aim of this study was to acquire a better understanding of the meaning-making process in educational design team meetings. The analysis therefore searched for instances when the designers reacted to the design artefacts while communicating with each other. For this purpose, a micro and meso-level analysis was conducted. At the micro-level, the verbal and non-verbal interaction of each designer was analysed separately. At this level the gestures of each participant was analysed in relation to the inscriptions, the shape of the gesture and its peak, and its relation to the spoken words. For example, the Globe gesture was at times at a subordinate level with spoken words but increasingly became equal with spoken words until it became autonomous and could make sense on its own without specific verbal descriptions accompanying it. At the meso-level, group interaction was analysed. At this stage, the Globe gesture was classified as a cohesive gesture as it was repeated by several members of the team.

The recording of the design session was streamed into a purpose-built tool named ‘D3 Data Viewing Software’. The tool facilitated the simultaneous viewing of synchronised recordings from all the cameras in the design studio. Due to the complexity of the analysis, a particular form of transcription was developed which was inspired by Multimodal Interaction Analysis (Norris, 2004), Conversation Analysis (Sacks, Schegloff, & Jefferson, 1974), Critical Discourse Analysis (Fairclough, 1995), and Interaction Analysis (Jordan & Henderson, 1995). See Table 1 for the transcription conventions used in this study.

Table 1

The analytical modalities and transcription conventions used.

Mode Definition Transcription conventions used

Talk This includes statements uttered by the participants in the interaction. [
=
(.)
(1.5)

:
point of speech overlap onset
a latched utterance
short pause
timed pause
utterance trailing off
elongation of the preceding sound
Gesture This describes how the participants moved their hands to point to and/or interact with the drawings/or in relation to the drawings and other resources. {
}
point of gesture onset
point of gesture termination
Gaze This is referred to observable gaze. Fleeting gazes might not be transcribable due to their brevity. ^text^ These symbols are used to enclose parts of the utterance when the participant looks at something while speaking. Not all instances were marked in the text.
Posture This mode illustrates how people positioned their bodies in relation to each other, the drawings, and the resources in the environment, in ways that had a communicative meaning. Posture and use of resources are described in the column next to the utterances. They are not signalled in the utterance transcription.

The analysis followed a three-stage iterative process; each stage delving deeper into the nuances of the participants’ interaction. The first stage involved basic cataloguing of the activities of the designers in the accompanying transcripts. Descriptive marks were added to the data identifying the types of events that occurred such as the start and ending of a drawing, how and when it was used for different purposes, and when and how resources in the environment were used. Some interesting gestures in relation to the drawings were also noted in this stage for later analysis. Stage two involved a deeper analysis of some of these design events and activities. In addition, further details were added to the transcripts of the events (see Table 1).

Only certain communicative modes were transcribed, as they were deemed important for meaning making at the micro level. For example, gaze was only noted in the transcription if it was important for the meaning-making process. Gestures were only transcribed if they were enacted in relation to the inscriptions and the previously designed visual representations. The general focus was more on the actions rather than speech intonations.

In stage three, a deeper analysis revealed how the designers jointly constructed meaning through gestures and interaction with the design artefacts. This was when the relation of the globe gesture with the globe logo and inscription (described below) was discovered and further analysed through an iterative process.

The Context of this Research

For a better understanding of the context of this research study, it is useful to clarify who is doing the design, why is the design work being done, what is designed, and most importantly – how is the design work undertaken (Goodyear & Dimitriadis, 2013).

Who is doing the design?

The educational design team in this study comprised of thirteen members of a project that worked on the development of an educational game. Their meeting in the Design Studio lasted 167 minutes. The team separated into three groups half way through the meeting. This paper is mostly discussing the design work by Group 2, which consisted of four male and one female members. It is important to note that only a few of the members of this team were actual professional educational designers. Due to the complexity of this project, it was necessary to bring together professionals from various domains. Among the members of Group 2, only Jarrod, the creative director, could be considered to have had experience as an educational designer. As for the rest of the members, Galileo was composition and music designer, Gino and Shima were graphic designers, and Bart was user interface designer.

Pseudonyms are used for all participants. Photographs of the designers are de-identified to preserve their privacy. This research was carried out in accordance with the regulations and ethical codes of conduct prescribed by the concerned university. All participants signed consent forms giving permission for the researcher to use the data for research and publication purposes.

In the context of this meeting, the participants in this study are both designers and receivers of the designs. They are receivers because during the meeting they were working with various visual representations that were designed previously including several interfaces for the game, a main logo or home page, as well as a diagram that represented the game navigation structure. Therefore, as the receivers, the participants had to respond to the designed representations and jointly construct meaning while working on further design decision for the game.

Why is the design work being done and what is designed?

The team was working on the design of an educational game. It was intended for phone and tablet experience targeting children between the ages of seven to eleven years. The players in the game need to take care of a family of polar bears living in a glacier environment. The players interact with the bears and keep them healthy by cleaning and feeding them. The bears also need their environment to be maintained clean and healthy. This means that if the ice melts the bears are not happy. In order to maintain the environment at the healthy level the players have an opportunity to play mini-games, which are separate activities but accessed from inside the main game. These activities allow the players to engage in real-life tasks that would keep the environment healthy, such as switching off light bulbs in their homes, planting trees and more. The players are able to add these real-world activities to their game profile and earn points. The game also has a social component to it. The players are able to connect with other players and see their progress in the game.

How is the design work undertaken?

The design session was conducted in the Design Studio, which is a purposefully created space equipped with an interactive whiteboard, two projectors, and two writable walls. The designers working in the studio have access to laptops and iPads which can be wirelessly connected to the projectors for display on the writable walls. For research purposes, the Design Studio is also equipped with audio and video recording facilities that can capture designers’ verbal utterances as well as non-verbal design discourse such as gestures, facial expressions, and drawing activities.

During the meeting, members of Group 2 gathered around the coffee table and couch area in front of one of the writable walls. They used PowerPoint slides on a laptop as the backdrop to their inscriptions on the wall. The slides included several visual representations including screen designs, a main logo, and a navigation diagram.

The remaining of this paper will further describe how the design work was undertaken. In particular, the joint meaning-making process will be elucidated through a description of how the designers restructured design artefacts and how they created and repeatedly re-enacted a gesture.

Results

The results describe the meaning-making process during one educational design team meeting. They tell a story of how the designers interacted with various digital and non-digital design artefacts and how they created a cohesive gesture as part of the meaning-making process that facilitated the communication of complex concepts and ideas.

Meaning-making with multilayered hybrid inscriptions

The globe inscription was created by Group 2 as part of the multilayered hybrid inscriptions. I called these inscriptions hybrid, which means that these inscriptions were created with a marker pen on the writable wall over the projection of a number of PowerPoint slides. They were a mixture of both digital and non-digital visual representations.

When Group 2 started their discussion, the image of the navigation diagram was projected on the wall. See Figure 1A. The group assembled in front of this projection. However, for approximately 12 minutes, the group members did not talk about the diagram nor did they explicitly pay attention to it or gestured towards it. The diagram only became the focus of attention when Gino started drawing the first inscription over it. This was as if the inscription made the diagram relevant. From this point forward, the projected slides and the inscriptions over them became the focus of discussion for the group, and this continued for the remainder of the group meeting.

Figure 1 

The first three slides projected on the wall.

The meaning-making process here involved a reconstruction of the artefact (Kazmierczak, 2003). The new design artefacts were hybrid representations which comprised of the PowerPoint slide projections and the hand-drawn inscriptions over them.

Throughout the rest of the group session more inscriptions were added over several other slides. The inscriptions made earlier over the different slides were still visible over the subsequent slides. See Figure 2. However, since the meaning of each inscription was attached to a specific slide, it did not hinder the group’s design process. The reconstruction of these artefacts was through a joint meaning-making process, which will be further described in the next section.

Figure 2 

Further inscriptions created over the PowerPoint slides.

Before the design session ended, Group 2 members took photographs of the wall with their phones. These photographs were the new design artefacts that would be used in future design sessions. These were layered artefacts, which comprised of the background slides and the new hand-drawn inscriptions over them.

The blending of digital and non-digital working space

The projected images in the PowerPoint slides dictated where the inscriptions were drawn on the wall. For example, the house was the base for the drawing of the panels on the roof, the window on the side of the house and the bear next to the house (see Figure 2A).

The projection area on the wall was seen as the space where ideas were shared and recorded as part of the group’s discussion. Anything outside of this area was treated as extra resources and not the focus of the group’s discussion or output. At one point Galileo created some inscriptions on the outside of the illuminated area but then explicitly connected his inscriptions using lines and arrows to the inscriptions on the illuminated area of the wall, so that they would be understood as part of Group 2’s ideas for the game (see Figure 2B). This is because our perception links together elements based on their proximity (Kazmierczak, 2003). Therefore, Galileo had to explicitly make his inscriptions part of the group’s meaning-making process by linking them to the working space.

The illuminated area was a blended digital and non-digital working space. This created an interesting situation. While the hand-drawn inscriptions helped reconstruct the meaning associated with the design artefacts in the background slides, they could not make sense on their own. Figure 4 shows an example of when the projection of the slides was temporarily turned off during the meeting. The inscriptions remained on the wall scattered and without any structure or meaning. The projection of the slides provided a background for the inscriptions against which the inscriptions became meaningful.

Although the constant changes in the slides did not obstruct the process of drawing the inscriptions or the design task itself, the illuminated area was still important for understanding what was relevant to the Group 2’s design discussion.

Meaning-making through the creation of the globe inscription

It is now important to have a closer look at how the navigation diagram was reconstructed through a joint meaning-making process.

After the first three slides were projected on the wall (Figure 1) the globe inscription was created. At this stage the group was discussing how the player would transition from the home screen to the bear habitat. Gino suggested they could use the globe logo to give the user ‘a sense of being on the globe’ by zooming into an activity or a game and zooming back out when selecting another activity.

During this discussion, the diagram slide was projected on the wall. Originally, the globe logo was designed to represent the home page for the game. However, Gino questioned this function of the globe logo (see the except below).

Gino: Look does it have to be the globe do you think for home screen?

Jarrod: Yeah, it could be. That was the original idea but look it doesn’t have to be.

Gino: Maybe that is the load screen or something?

Jarrod: Yeah sure say it is for now.

Gino thought that it would be better for the globe logo to be a loading screen rather than the home page. When Jarrod expressed agreement, Gino drew a small globe in the middle of the navigation diagram. He then drew arrows linking the four major components of the game which included the Home Screen, Player Home, Habitat Shop, and the Glacier Park (see Figure 3).

Figure 3 

Creation of the globe inscription.

Figure 4 

Inscriptions without the background PowerPoint slides.

By seeing the globe inscription in the middle of the diagram, Jarrod began scrolling through the PowerPoint slides to bring back the globe logo slide again stating ‘where has my thing gone?’. While searching for the logo slide Jarrod stated, ‘you’re saying that we could go (.) we pop out to that sort of world’, and once he found the logo slide, he said ‘pop out to that?’. Gino quickly responded saying ‘and then zoom back in’. Gino then further explained his idea for the zoom feature by walking to wall touching and each section of the globe projection.

During meaning-making process, designers often use metaphors that link to either our real-world experiences or parallel models of a similar nature (Kazmierczak, 2003). The idea for the zoom function was most likely influenced by the designers’ experience with other games or map-based tools where the user moves through the map and then lands on a location. Perhaps similar to Google Maps. The challenge for designers is to use metaphors that are shared with others which can then ensure the effectiveness of the communication in the meaning-making process.

The navigation diagram was reconstructed with the help of the hybrid globe inscription. The next version of the navigation diagram will have a loading screen in the middle that then connects to other game components. Most importantly, as a result of this meaning-making process, the function of the globe logo changed from representing the home page to representing the loading screen.

Meaning-making through the enactment of the globe gesture

During this educational design meeting, the members enacted a distinctive gesture that I have labelled the globe gesture. The physical form of the globe gesture was holding both hands facing each other to make a ball shape. This gesture often closely followed a pointing gesture. The pointing gesture was used to draw attention to the topic of conversation such as pointing to the diagram or the inscriptions layered over. It created a connection between the diagram or inscription and the globe gesture that followed.

The globe gesture was first enacted by Jarrod during the initial whole team discussion, before the team was divided into groups. See Table 2 for the except.

Table 2

The first enactment of globe gesture by Jarrod before the team separated into groups.

Line Time Participant Talk Non-verbal interaction

179 1:02:34 Kassidy I’d love to see them (.) and I don’t know how we’ve done it, but it’s been a bit boring in the past and taking away from the bear, but give them the opportunity in the past to add to their footprint. You obviously can’t ask them 50 questions (.) but you can ask them questions as they go you know (.) keep them engaged with that whole thing along the way.
180 1:02:52 Jarrod But how do you feel about the fun angle as well?
1. You know, the bear having to do some of the things in in (.) we’re already {into stat document} getting into non-bear behavior.
2. Ahh I just thought that was a way to reinforce that or to make (1.5) make it a bit more fun, {just totally}
3. {internally within the world} with a view that you’re setting up these concepts
Pointing gesture 1: with his right hand pointing quickly towards the writable wall where the PowerPoint slides were projected.
Globe gesture 2: holding hands to make a ball shape with the left hand on top (Figure 5A).
Globe gesture 3: flips the previous gesture – holding hands to make a ball shape with the right hand on top now (Figure 5B).

In this excerpt, Jarrod was trying to communicate his idea about integrating more engaging and ‘fun’ activities into the game (see line 180 of Table 2 for the utterance). He held his hands like the shape of a ball, initially the left hand over the right hand (Figure 5A) and then flipping the hands and bringing the right hand over the left hand to make the ball shape again (Figure 5B).

Figure 5 

Jarrod enacting the globe gesture for the first time.

Here Jarrod was responding to Kassidy who was discussing how the game would collect information from the players that would be used as statistics to determine the player behaviour changes. Thus, Jarrod’s pointing gesture-1 referred to this ‘stat document’, which was included in the PowerPoint. Since the mini games were located outside the main game (and these were the most fun parts), Jarrod wanted to incorporate more fun activities inside the main game as well.

According to Kazmierczak (2003), at the first stage of the meaning-making process, the intended meaning is encoded in the design itself. This meaning is defined by the designers of the artefact. In the context of this study, the designers of the globe logo intended a certain function to be attributed to the logo. The globe logo was designed to be the home page for the game. Consequently, Jarrod’s gesture referred to the globe logo’s initial purpose; as the home page for the game. This was before the globe inscription changed the function of the globe logo to represent the loading screen (described in the previous section).

The globe gesture here was at a sub-ordinate level to spoken words (Norris, 2011). The team would still be able to understand the meaning of Jarrod’s words without the globe gesture.

The second occurrence of the globe gesture was when the team was divided into groups and Gino in Group 2 suggested that the globe logo could be used for the loading screen rather than the home page. See Table 3 for the except.

Table 3

First enactment of the globe gesture in Group 2 during the creation of the globe inscription.

Line Time Participant Talk Non-verbal interaction

47 1:57:28 Gino 4. Look does it have to be {the globe}
5. do you think for {home screen}
Posture: standing to the left of the projected screen, turns around to face the group
Gaze: looks at Jarrod
Globe gesture 4: holds both hands to make a ball shape (Figure 6).
Pointing gesture 5: pointing at the diagram projected on the wall.
48 1:57:31 Jarrod Yeah, it could be. That was the original idea but look it doesn’t have to be Posture: sitting, facing the wall.
Gaze: the diagram on the wall.
49 1:57:42 Gino 6. Maybe that is the {load screen} or something? Gaze: looks at the diagram, then at Jarrod
Gesture 6: makes a rolling motion with both hands (for loading screen).
50 1:57:50 Jarrod Yeah::: sure say it is for now Gaze: looks back at the other groups in the studio, then looks at the wall.

Before drawing the globe inscription over the diagram, Gino turned around and asked Jarrod ‘Look does it have to be the globe’, while holding his hands in the shape of a ball (line 47, Table 3). See Figure 6 for the gesture.

Figure 6 

First enactment of the globe gesture in Group 2 by Gino.

When Gino changed the function of the globe logo (while drawing the globe inscription) from representing the home page to representing the loading screen, all other features of the game were now accessible through the globe logo screen, including the home page. The globe logo thus became a single point of entry into the game. When Gino asked ‘does it have to be the globe’, he communicated two things to Jarrod; one, that he was talking about the loading screen as a single entry into the game; and two, that he was also adding the visual property of the loading screen to the discussion: the shape of the globe.

Although the globe gesture here is still at a sub-ordinate level to the spoken words since Gino utters the words ‘home screen’, when also referring to the shape of the globe, the gesture takes more prominence and becomes almost as important as the words. Gino was hesitant to have the home page in the shape of a globe.

According to Kazmierczak (2003), the second stage of the meaning-making process is when the design triggers a thought in the receiver while the third stage is when the receiver reconstructs the meaning of the design. Here when Jarrod mentioned the intended function of the globe logo as the home screen, it triggered a thought for Gino. He suggested a new function for the globe logo as the loading screen and exhibited this with the rolling motion of both hands (see Gesture 6, Table 3).

In general, people may exhibit three levels of awareness when approaching designed products (Crilly, 2011). In the first level they might be aware that the product has been designed. In the next level they would understand that the designers are at a liberty to make choices. In the third level, people might infer that the designers have used certain tactics and that those require them to respond to the designed product in a certain way. Here, Gino perhaps understood that the globe logo is a designed artefact and as such, its shape and function is the result of the designers’ choices that they made during the creation of the logo. The designers were certainly at liberty to create a different shape logo and to assign a different function to it. According to this, the function of the logo as the home screen is not fixed and can be altered.

Gino then moved on to the third stage of the meaning-making process by drawing the globe inscription in the middle of the navigation diagram and reconstructing the function of the globe logo from home page to loading screen.

The next occurrence of the globe gesture was observed in three consecutive turns by Shima; once in each of her utterances, as recorded in lines 102, 105, and 107 of Table 4.

Table 4

Shima’s three enactments of the globe gesture.

Line Time Participant Talk Non-verbal interaction

101 2:06:18 Jarrod Sorry Shima, you were saying? Gaze: slightly turning towards Shima.
102 2:06:20 Shima 7. I was saying {do the mini games}
8. have their own {environments} as well?
Gaze: looking at the diagram on the wall, then briefly looking at Galileo and then Jarrod.
Pointing gesture 7: pointing at the diagram with index finger extended.
Globe gesture 8: briefly holding both hands in front of her with palms facing each other (Figure 7A).
103 2:06:25 Gino [Yeah (.) the mini games… Gaze: looks at Shima, then looks at Bart.
104 2:06:25 Bart [they’re not incorporated into the 3D world Gaze: looking at Shima and then Jarrod.
105 2:06:30 Shima 9. If we can access Player Home {through the 3D world}
10. then we can access {those environments} separately
Gaze: looking at the wall and then Bart.
Globe gesture 9: holds both hands to make a ball shape (Figure 7B).
Pointing gesture 10: swiping her left hand from left to right with the index finger extended towards the diagram.
106 2:06:40 Bart It’s whatever you guys want (laughs) Gaze: Jarrod, then Shima and Gino
107 2:06:44 Shima 11. Will {they be very different?}
12. Like the mini game environments. Will they be very different to what’s on {the globe}
Gaze: looking at Jarrod, then briefly Bart and back at Jarrod.
Pointing gesture 11: pointing at the diagram with left hand then towards the end rolling finger in a circular motion.
Globe gesture 12: holds both hands to make a ball shape (Figure 7C).
108 2:06:50 Jarrod I don’t know and I think it’s kind of up to us to decide Gaze: looking at the diagram.

In this excerpt, Shima asked whether the mini games would have their own environments (line 102). The mini games were represented by small circles in the diagram (see Figure 3 for the diagram). She held her hands in a shape similar to the globe gesture (see Figure 7A for the gesture). However, her hands did not fully display the fullness of the globe at this time; the hands were straighter, rather than curved as in previous versions of this gesture. The stroke of the gesture coincided with the word ‘environments’, which indicates that Shima was trying to find out whether the mini games would be self contained and have a single entry of their own, just like the globe screen was an entry to a self-contained environment – the game. Shima took the gesture and all the meanings associated with it and applied it to assess the status of the mini games. Bart responded that ‘they’re not incorporated into the 3D world’. Here although Bart did not display the globe gesture, Shima understood the reference to the 3D world which indicated the single-entry globe loading screen because they discussed earlier that the globe loading screen would be in 3D.

Figure 7 

The three enactments of the globe gesture by Shima.

According to Kazmierczak (2003), changes in the organisation of signs in visual representations may trigger different interpretation by the receivers of the designs and thus create different responses. Since the navigation diagram was reconstructed through the inclusion of the globe inscription as the new loading screen, the designers took this into account in their subsequent responses to the game navigation. The globe inscription was a visual reminder that the globe logo is no longer the home page and is now the single-entry loading screen. This was a visible form of meaning-making.

This was the first time when the globe gesture was not at a sub-ordinate level to the spoken words. Here, the gesture was needed to communicate what Shima meant by ‘their own environments’. The Globe gesture became equal with spoken words.

In her next turn, Shima stated ‘If we can access Player Home {through the 3D world} then we can access those environments separately’ (line 105, Table 4). See Figure 7B for the gesture. By ‘those environments’ she meant the mini games displayed in the diagram with small circles. Here Shima enacted the gesture alongside the terms that Bart used earlier to refer to the single-entry loading screen, as in ‘3D world’. The reason for enacting the gesture for the second time may be that Shima wanted to confirm her understanding of Bart’s response in that the 3D world still referred to the globe loading screen. In this turn, Shima took the gesture and its associated meanings and applied it back to the globe loading screen as a single point of entry into the game. So, after applying the gesture to assess the status of the mini games in her first utterance, Shima brought the gesture back and applied it to the globe loading screen in her second utterance. By bringing the gesture back and applying it to the globe loading screen, Shima also separated the two different game environments in her comment. The globe gesture now accompanied ‘the 3D world’ and a pointing gesture towards the small circles in the diagram accompanied ‘those environments’. Thus, the ‘3D world’ referred to the globe loading screen and ‘those environments’ referred to the mini games.

The second globe gesture by Shima was also at an equal level with the spoken words. It was a vital mode of communication and its omission would have caused confusion among the designers.

Shima enacted the globe gesture one more time. See line 107 of Table 4 for the utterance and Figure 7C for the gesture. Although the stroke of this gesture accompanied ‘the globe’, which referred to the globe loading screen, it was used again as a way to assess the status of the mini games. Since Shima’s first gesture determined that the mini games would not be incorporated into the single-entry loading screen and that the loading screen would be displayed in 3D, at this point, Shima wanted to know whether the mini games would also be in 3D similar to the globe loading screen.

Here one might think that the globe gesture is at a sub-ordinate level to the spoken words since the stroke coincided with ‘the globe’ utterance. However, the gesture was needed because the entire conversation up to this point was packaged into this single gesture and its omission at this stage might have resulted in confusion.

The globe gesture was enacted one more time by Gino. See the excerpt in Table 5.

Table 5

Last enactment of the globe gesture by Gino in Group 2.

Line Time Participant Talk Non-verbal interaction

133 2:12:42 Gino 13. If we have a game that’s turning off lights in an unrelated house, we now have to design an unrelated house. If we have a pinball machine xyz, then we have to design all those things. So {keeping it within the world}
14. like {in the habitat} (.)
15. {the Glacier environment} (.) then that’s great because it simplifies it in terms of design tasks for us
Gaze: looking at Jarrod and very briefly at others in the group.
Pointing gesture 13: pointing with right hand holding marker pen towards the diagram without looking at it.
Pointing gesture 14: the left hand joining the right hand, holding open hands towards the diagram.
Globe gesture 15: curves both hands slightly to make a ball shape (Figure 8).

The last occurrence of the globe gesture observed was when Jarrod suggested they focus more on the bear and everything should come back to the bear and what it does in its environment and in its own house and igloo. Gino agreed and stated that from a technical point of view it would be easier to design fewer game assets than many. In the last part of this utterance (line 133, Table 5), Gino enacted the globe gesture. See Figure 8 for the gesture.

Figure 8 

The last enactment of the globe gesture by Gino.

The stroke the globe gesture here coincided with the utterance ‘the Glacier environment’. The two other gestures in this utterance were pointing gestures towards the projection of the house in the glacier environment on the wall. Although the globe gesture is the focus here, the other two pointing gestures are also important in order to understand the context. In both of those gestures, Gino was pointing first with one hand and then both hands towards the wall. After a short pause he then used the globe gesture as well. The reason for using the globe gesture in combination with the other two pointing gestures may be that Gino felt his two pointing gestures did not fully communicate his idea and thus he needed to clarify it with the globe gesture. It was a way to communicate that he envisioned the bear habitat and the glacier environment where the house was located, to be all encompassed within the single-entry into the game.

Once again, the globe gesture was at an equal level with the spoken words. It increasingly attained importance to a point when the designers felt that their utterance, even in combination with the hybrid visual representations on the wall, did not fully communicate their ideas and so they needed to enact the globe gesture. In other words, the globe gesture became a local semiotic resource, which enabled the designers to communicate complex ideas and assign new meanings to design artefacts.

Discussion

Meaning-making in face-to-face educational design meetings is a complex process but is yet to gain attention as a topic of research. This study contributes to a better understanding of the nuances of multimodal communication through which educational designers construct shared meanings and ideas. In addition, this paper proposes meaning-making as an analytical tool that can elucidate how designers respond to artefacts and visual representations which might give us some insights into how they comprehend and receive the artefacts.

Educational design is seldom a solitary activity. Most projects involve educational designers working with various stakeholders, including people who specialise in certain skills and offer distinct expertise. Effective communication is imperative in such environments (Ritzhaupt & Kumar, 2015). Since face-to-face communication is always multimodal (Norris, 2002, 2004), it is essential to include the analyses of non-verbal modes of communication in the meaning-making process.

Drawing as a form of communication and an aid in the thinking process, has been investigated extensively in design research (Berger, 2007; Goldschmidt, 1991, 2003; Hokanson, 2008; Suwa & Tversky, 2002), though still limited in educational design (Stubbs, 2006; Wardak, 2016b). Gestures on the other hand have only recently came to the attention of designers. Interestingly, most gestures in design are observed to be enacted in relation to design artefacts and inscriptions (Artman, House, & Hultén, 2014; Bekker et al., 1995). This is in line with the findings of this study.

In this study, both inscriptions and gestures were part of the analysis in order to elucidate how the educational designers responded to design artefacts and how they jointly reconstructed the meanings associated with them. The globe gesture in particular was instrumental in the meaning-making process. The gesture initially represented the home screen the for game, keeping with the original function of the globe logo, but soon took on new meaning when it came to represent the single point of entry into the game. The gesture came to mean keeping things within a self-contained environment. In this way, the game was given a boundary, with the globe loading screen as the entry and everything within it represented by the globe gesture. Research on how educational design is accomplished would benefit from this kind of micro-analysis of face-to-face interaction which can elucidate the intricacies of design work.

The globe gesture was a cohesive gesture, in that its repetition showed the continuation of the same theme. There was no single term or verbal description associated with the globe gesture, which indicates that the gesture attained considerable strength and was not reliant on specific accompanying terms in order to make sense. There were also several variations of the globe gesture; hands facing each other vertically or horizontally and fingers curved fully or partially. This indicates that the gesture did not need to follow a strict physical form in order to make sense to the audience. At each turn, the globe gesture gained more prominence.

The globe gesture was used as a local semiotic resource (Koschmann & LeBaron, 2002) to communicate ideas that would otherwise require a lot of explanations. It allowed the designers to discuss more complex ideas. While initially the gesture was at a sub-ordinate level to the spoken words, it soon became indispensable to the meaning-making process. Research on gestures in design would benefit from further exploration of this finding in other design contexts.

Conclusion

The meaning-making process reported in this paper shows that educational design is accomplished through intricate verbal and non-verbal interaction with design artefacts and various tools, resources, and representations. Theories and process models aimed at abstracting and simplifying the educational design process need to take these findings into account.

The educational designers in this study modified the design artefacts and tools provided to them by adapting them to suit the way they were working and collaborating. They created hybrid design artefacts by sketching over static digital images. They then took photos of these hybrid artefacts, creating new layered design artefacts for use in future meetings. When these hybrid artefacts failed to support the complex intricate communication, the designers created a cohesive gesture that worked in tandem with the other design artefacts and supported the communication of complex concepts becoming a local semiotic resource.

This paper would be of primary interest to researchers who study the way in which educational design is accomplished. Beyond this, those who work as educational designers or those who work with educational designers, might benefit from an awareness of the different interactional tools at their disposal.

Competing Interests

The author has no competing interests to declare.

References

  1. Artman, H., House, D., & Hultén, M. (2014). Designed by Engineers: An analysis of interactionaries with engineering students. Designs for Learning, 7(2), 27–57. DOI: https://doi.org/10.2478/dfl-2014-0062 

  2. Barab, S., & Squire, K. (2004). Design-Based Research: Putting a Stake in the Ground. Journal of the Learning Sciences, 13(1), 1–14. DOI: https://doi.org/10.1207/s15327809jls1301_1 

  3. Becvar, A., Hollan, J., & Hutchins, E. (2008). Representational Gestures as Cognitive Artifacts for Developing Theories in a Scientific Laboratory. In M. S. Ackerman, C. A. Halverson, T. Erickson & W. A. Kellogg (Eds.), Resources, Co-Evolution and Artifacts: Theory in CSCW (pp. 117–143). London: Springer. DOI: https://doi.org/10.1007/978-1-84628-901-9_5 

  4. Bekker, M. M., Olson, J. S., & Olson, G. M. (1995). Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design. Paper presented at the Proceedings of the 1st conference on Designing interactive systems: processes, practices, methods, & techniques. DOI: https://doi.org/10.1145/225434.225452 

  5. Berger, J. (2007). Berger on drawing (2 ed.). Occasional Press. 

  6. Bilda, Z., Gero, J. S., & Purcell, T. (2006). To sketch or not to sketch? That is the question. Design Studies, 27(5), 587–613. DOI: https://doi.org/10.1016/j.destud.2006.02.002 

  7. Buxton, B. (2007). Sketching User Experiences: Getting the Design Right and the Right Design. Redmond, Washington: Microsoft Research. 

  8. Crilly, N. (2011). Do users know what designers are up to? Product experience and the inference of persuasive intentions. International Journal of Design, 5(3), 1–15. 

  9. Crilly, N., Good, D., Matravers, D., & Clarkson, P. J. (2008). Design as communication: exploring the validity and utility of relating intention to interpretation. Design Studies, 29(5), 425–457. DOI: https://doi.org/10.1016/j.destud.2008.05.002 

  10. Davidsen, J., & Christiansen, E. (2014). Mind the hand: A study on children’s embodied and multimodal collaborative learning around touchscreens. Designs for Learning, 7(1), 34–53. DOI: https://doi.org/10.2478/dfl-2014-0010 

  11. Donovan, J., Heinemann, T., Matthews, B., & Buur, J. (2011). Getting the point: The role of gesture in managing intersubjectivity in a design activity. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 25(Special Issue 03), 221–235. DOI: https://doi.org/10.1017/S0890060411000059 

  12. Fairclough, N. (1995). Critical discourse analysis: the critical study of language. London; New York: Longman. 

  13. Frølunde, L. (2009). Storyboarding an animated film: A case study of multimodal learning processes in a Danish upper secondary school. Designs for Learning, 2(1), 8–21. DOI: https://doi.org/10.16993/dfl.16 

  14. Goldschmidt, G. (1991). The dialectics of sketching. Creativity Research Journal, 4(2), 123–143. DOI: https://doi.org/10.1080/10400419109534381 

  15. Goldschmidt, G. (2003). The Backtalk of Self-Generated Sketches. Design Issues, 19(1), 72–88. DOI: https://doi.org/10.1162/074793603762667728 

  16. Goodyear, P. (2005). Educational design and networked learning: patterns, pattern languages and design practice Australasian Journal of Educational Technology 21(1), 82–101. DOI: https://doi.org/10.14742/ajet.1344 

  17. Goodyear, P., & Dimitriadis, Y. (2013). In medias res: Reframing design for learning. Research in Learning Technology, 21(1). DOI: https://doi.org/10.3402/rlt.v21i0.19909 

  18. Gray, C. M., Dagli, C., Demiral-Uzan, M., Ergulec, F., Tan, V., Altuwaijri, A. A., … Boling, E. (2015). Judgment and Instructional Design: How ID Practitioners Work In Practice. Performance Improvement Quarterly, 28(3), 25–49. DOI: https://doi.org/10.1002/piq.21198 

  19. Herold, J., & Stahovich, T. F. (2011). Using speech to identify gesture pen strokes in collaborative, multimodal device descriptions. AI EDAM, 25(Special Issue 03), 237–254. DOI: https://doi.org/10.1017/S0890060411000060 

  20. Hokanson, B. (2008). The Virtue of Paper: Drawing as a means to innovation in instructional design. In L. Botturi & T. Stubbs (Eds.), Handbook of Visual Languages for Instructional Design: Theories and Practices (pp. 76–89). New York: Information Science Reference. DOI: https://doi.org/10.4018/978-1-59904-729-4.ch005 

  21. Hutchins, E. (2006). The distributed cognition perspective on human interaction. In N. Enfield & S. C. Levinson (Eds.), Roots of human sociality: culture, cognition and interaction (pp. 375–398). Oxford, UK: Berg Publishers. 

  22. Jordan, B., & Henderson, A. (1995). Interaction Analysis: Foundations and Practice. The Journal of the Learning Sciences, 4(1), 39–103. DOI: https://doi.org/10.1207/s15327809jls0401_2 

  23. Kazmierczak, E. T. (2003). Design as meaning making: from making things to the design of thinking. Design Issues, 19(2), 45–59. DOI: https://doi.org/10.1162/074793603765201406 

  24. Koschmann, T., & LeBaron, C. (2002). Learner articulation as interactional achievement: Studying the conversation of gesture. Cognition and Instruction, 20(2), 249–282. DOI: https://doi.org/10.1207/S1532690XCI2002_4 

  25. Latour, B. (1987). Science in action: how to follow scientists and engineers through society. Cambridge, Mass: Harvard University Press. 

  26. Luck, R. (2010). Using objects to coordinate design activity in interaction. Construction Management and Economics, 28(6), 641–655. DOI: https://doi.org/10.1080/01446193.2010.489924 

  27. Luck, R. (2014). Seeing architecture in action: Designing, evoking and depicting space and form in embodied interaction. International Journal of Design Creativity and Innovation, 2(3), 165–181. DOI: https://doi.org/10.1080/21650349.2013.875488 

  28. Ma, J. (2015). When Human-Centered Design Meets Social Innovation: The Idea of Meaning Making Revisited. Paper presented at the 17th International Conference on Human-Computer Interaction, Los Angeles, CA, USA. DOI: https://doi.org/10.1007/978-3-319-20907-4_32 

  29. McKenney, S., & Reeves, T. C. (2019). Conducting Educational Design Research. London: Routledge. DOI: https://doi.org/10.4324/9781315105642 

  30. McNeill, D. (1992). Hand and Mind: What Gestures Reveal about Thought. University Of Chicago Press. 

  31. McNeill, D. (2005). Gesture and Thought. University Of Chicago Press. DOI: https://doi.org/10.7208/chicago/9780226514642.001.0001 

  32. Nomura, S., & Hutchins, E. (2007). The Multimodal Production of Common Ground Understandings in Intercultural Flight Training. Paper presented at the Proceedings of the 14th International Symposium on Aviation Psychology. 

  33. Norris, S. (2002). The implication of visual research for discourse analysis: transcription beyond language. Visual Communication, 1(1), 97–121. DOI: https://doi.org/10.1177/147035720200100108 

  34. Norris, S. (2004). Analyzing multimodal interaction: A methodological framework. New York: Routledge. DOI: https://doi.org/10.4324/9780203379493 

  35. Norris, S. (2011). Three hierarchical positions of deictic gesture in relation to spoken language: a multimodal interaction analysis. Visual Communication, 10(2), 129–147. DOI: https://doi.org/10.1177/1470357211398439 

  36. Ritzhaupt, A. D., & Kumar, S. (2015). Knowledge and Skills Needed by Instructional Designers in Higher Education. Performance Improvement Quarterly, 28(3), 51–69. DOI: https://doi.org/10.1002/piq.21196 

  37. Roth, W.-M., & McGinn, M. K. (1998). Inscriptions: Toward a Theory of Representing as Social Practice. Review of Educational Research, 68(1), 35–59. DOI: https://doi.org/10.3102/00346543068001035 

  38. Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A simplest systematics for the organization of turn-taking for conversation. Language, 50, 696–735. DOI: https://doi.org/10.1353/lan.1974.0010 

  39. Stubbs, T. (2006). Design Drawing in Instructional Design at Brigham Young University’s Center for Instructional Design: A Case Study. PhD, Brigham Young University. 

  40. Suwa, M., & Tversky, B. (2002). External Representations Contribute to the Dynamic Construction of Ideas. Paper presented at the Proceedings of the Second International Conference on Diagrammatic Representation and Inference. DOI: https://doi.org/10.1007/3-540-46037-3_33 

  41. Tversky, B., Heiser, J., Lee, P. U., & Daniel, M. P. (2009). Explanations in gesture, diagram, and word In K. R. Coventry, T. Tenbrink & J. Bateman (Eds.), Spatial Language and dialogue (pp. 119–131). Oxford: Oxford University Press. DOI: https://doi.org/10.1093/acprof:oso/9780199554201.003.0009 

  42. Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.). (2006). Educational design research. Routledge. DOI: https://doi.org/10.4324/9780203088364 

  43. van den Hoven, E., & Mazalek, A. (2011). Grasping gestures: Gesturing with physical artifacts. AI EDAM, 25(Special Issue 03), 255–271. DOI: https://doi.org/10.1017/S0890060411000072 

  44. Visser, W. (2009). The function of gesture in an architectural design meeting. In J. McDonnell, P. Lloyd, F. Reid, R. Luck & N. Cross (Eds.), About: Designing. Analysing Design Meetings (pp. 269–284). London: Taylor & Francis. 

  45. Visser, W. (2010). Function and Form of Gestures in a Collaborative Design Meeting. In S. Kopp & I. Wachsmuth (Eds.), Gesture in Embodied Communication and Human-Computer Interaction, 5934, 61–72. Berlin, Heidelberg: Springer. DOI: https://doi.org/10.1007/978-3-642-12553-9_6 

  46. Visser, W., & Maher, M. L. (2011). The role of gesture in designing. AI EDAM, 25(Special Issue 03), 213–220. DOI: https://doi.org/10.1017/S0890060411000047 

  47. Waltersdorfer, G., Gericke, K., Desmet, P., & Blessing, L. (2017). An Engineering Approach to Mapping Meanings in Products and Services. Paper presented at the 21st International Conference on Engineering Design, ICED17, The University Of British Columbia, Vancouver, Canada. 

  48. Wardak, D. (2016a). Gestures orchestrating the multimodal development of ideas in educational design team meetings. Design Studies, 47, 1–22. DOI: https://doi.org/10.1016/j.destud.2016.08.001 

  49. Wardak, D. (2016b). Inscriptions in Educational Design Team Meetings. Performance Improvement Quarterly, 29(3), 257–285. DOI: https://doi.org/10.1002/piq.21224 

comments powered by Disqus