Resources

Unit 4: Social Semiotics

Chapter overview

Learning objectives

This study unit will help you to:

  • Engage critically with a social semiotic approach to multimodality
  • Familiarize yourself with the key principles and concepts of social semiotics
  • Try out some basic social semiotic analytical methods

Overview of Chapter 4

Topics

  • The origins and early development of social semiotics and how it has been developed to account for multimodality
  • Key social semiotic principles and concepts
  • Methods and analysis
  • Fields of application, limitations, potentialities and challenges

Summary

In Chapter 4 we discussed the origins of social semiotics in three main strands of influence: systemic functional linguistics, semiotics, and critical discourse analysis (CDA).

We showed how these three strands came together through the work of the Newtown Semiotic Circle, and the collaborations of Kress with Hodge and then van Leeuwen. In the 1980s, the focus was on the visual; in the late 1990s attention was broadened to include a range of modes, and the notion of multimodality was introduced.

We showed how a social semiotic approach highlights the social dimensions of meaning, its production, interpretation and circulation, and its implications. It sets out to reveal how processes of meaning-making (i.e. signification and interpretation or what is called semiosis) shape individuals and societies. The focus of social semiotics is on understanding the affordances of modes, the semiotic choices available to a person in a specific context, how people choose from these to make meaning and what motivates their choices, as well as the social effects of these choices. We also noted the interest of social semiotics in how modes change over time. We introduced the following social semiotic concepts:

  • Semiotic resource refers to the meaning potential of material resources, which developed and accumulated over time through their use in a particular community and in response to certain social requirements of that community.
  • Mode refers to a socially organized set of semiotic resources for making meaning. Examples of modes include image, writing, layout, and speech among others. For something to count as a mode it needs have a set of semiotic resources and organizing principles that are recognized within a community as realizing meaning.
  • Modal affordance refers to the idea that different modes offer different potentials for making meaning. Modal affordances affect the kinds of semiotic work a mode can be used for, the ease with which it can be done, and the different ways in which modes can be used to achieve broadly similar semiotic work. Modal affordances are connected both to a mode’s material and social histories, that is, the social purposes that it has been used for in a specific context.
  • Motivated sign is a term used to reference that meaning (‘signified’) and form (‘signifier’) are brought together in a relation motivated by the aptness of fit between the interest of the sign maker and the affordances of a semiotic resource.
  • Interest is a term used to refer to a momentary condensation of all the social experiences that have shaped an individual’s subjectivity – a condensation prompted by the social environment (of which the available modes are a significant element) that a sign is made within. A person’s interest connects their choice of one resource over another with the social context of sign production.
  • Design is used to refer to the situated process in which a sign maker chooses and arranges semiotic resources to realize a particular social function or purpose. Design is conceived of as a starting point for making meaning and both the original producer of a sign and its interpreter are understood as making meaning. It is always socially located and regulated, for example by the types of resources that are made available and to whom, as well as the regimes that regulate and shape how these resources are used to create various norms and expectations.

The chapter then described the methodology of a social semiotic approach, its focus on the motivated character of sign making and how it is highly responsive to and shaped by the artefacts and interaction that it works with. We showed how in a social semiotic approach the analyst shifts between processes of deduction and induction. We showed how observational methods are used to recognize and theorize the social meaning and effect of signs. We also outlined the typical research questions and applications that social semiotics is used to address, and discussed the limitations of the approach.

Study questions

Read Chapter 4 and think about how you would answer the questions below. Make notes of your responses and review them when you have completed the study guide.

  1. How is the multimodal concept ‘mode’ used within a social semiotic approach?
  2. In your view, what does the concept of ‘motivated sign’ mean for multimodality?
  3. Reflecting on the two boxed examples presented at the end of Chapter 4, why in your view would it be important to understand what modes and semiotic resources are made available, to whom, and the different ways these are brought together?
  4. What can you say (and what not) about the ‘interest’ of a sign maker by analysing the artefacts that they make?

Exercises

Exercise 4.1: Operationalizing social semiotic concepts

  1. Select a paper from one of the suggested readings for Chapter 4.
  2. Which of the theoretical concepts that we have discussed in Chapter 4 are used in the paper (i.e. semiotic resource, mode, modal affordance, motivated sign, interest, design)?
  3. How does the author of the paper connect these theoretical concepts with the materials that they analyse?
  4. How do they warrant their analytical claims?
  5. In your view, is the analysis convincing? If yes, why and if not, why not?

Tip: You will need to show whether and how the theoretical concepts that you use connect with the materials you use in your study.

Exercise 4.2: Exploring the motivated sign

The concept of the motivated sign is intimately connected to the interest of a sign maker and the social-cultural context of their sign making. This exercise explores how these three interconnected concepts can shape a social semiotic analysis.

  1. Select a context where you can observe someone making a multimodal artefact (such as, writing a text message on a phone, or in Word, or posting something on Facebook, or watching a child making a drawing).
  2. If possible, video record them while they are making the artefact.
  3. Map the range of modes and semiotic resources that were available to them.
  4. Describe the modes and resources that they used to make the artefact.
  5. How did the social and cultural setting shape the sign maker’s choice of resources and the artefact they made?
  6. How did their ‘interest’ (including their skills and abilities) shape the process of making the artefact?

Exercise 4.3: Exploring the layout of an artefact through commutation

In Chapter 4 we introduced ‘commutation’ as an analytical process used in social semiotics to explore the design of multimodal texts, notably layout as a mode and the relationships between image and writing. In this exercise we invite you to try it out:

  1. Select a text that you are interested in analysing.
  2. Identify the layout-structure of the text. This might be in the form of boxed areas, columns, strips or grid-like structures on the page or screen (you could use a pen or annotation tool to highlight these).
  3. Rearrange the elements of the text, using drawing software (i.e. Paint for Windows, or x for Mac) to ‘cut and paste’ (or work manually with print outs and scissors), to create three variations of the artefact.
  4. Explore how the re-configuration of the modal relationships impacts on the relations between the modes and the meaning of the text.

Tip: The commutation test is a way to explore the choices available to a sign maker by changing a signifier – e.g. substituting one lexical item for another, or one spatial arrangement for another – and considering its effect on the signified.

Suggested resources

Key reading

Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.

Further reading

Bezemer, J. & Kress, G. (2016). Multimodality, Learning and Communication: A social semiotic frame. London: Routledge.

Adami, E. (2009). ‘We/YouTube’: exploring sign-making in video-interaction, Visual Communication 8(4), 379–399.

Kress, G. & van Leeuwen, T. (2006). Reading Images: A Visual Grammar of Design. London: Routledge.

Mavers, D. (2011). Children's Drawing and Writing: The Remarkable in the Unremarkable. London: Routledge.

Van Leeuwen, T. (2005). Introducing Social Semiotics. London: Routledge.

Online resources

Interview with Arlene Archer on social semiotics, social justice and multimodal pedagogy

Podcast of Diane Mavers’ keynote presentation analysing children’s drawing and writing as ‘semiotic work’