Multimodality

This video presents the term multimodality and how this aspect of our perceptual system has implications for how we perceive music.

Can Body Movement Teach us Something about Music Perception

As we have learned previously, the theory of embodied cognition tells us that our involvement with the world is based on the fact that we are humans with physical, moving bodies.

The same holds true for our involvement with music. The paradigm of embodied music cognition suggests that our bodies are integral to how we experience music. As such, we can learn a lot about how we perceive music by studying how we move to music.

Studying music cognition

How would you design an experiment to study people's perception of music? There are many possible answers to this question. Let us consider two alternative methods.

Using qualitative interviews is one method to study how people experience. This makes people reflect on their own behaviour, and will therefore give valuable subjective data. By asking people about how they move to music, they are forced to articulate and express how they perceive the music, but only indirectly. Our experience is that people answer quite differently whether they are asked: "How did you move to this song?" rather than "What do you think about this song?". Asking both questions may also be a way to reveal more about their musical experience than they would have otherwise shared.

However, talking about one's own behaviour may be challenging. Few people have the vocabulary to express the nuances of their own bodily behaviour and perception of music. Furthermore, if the interview occurs after a concert, the person's long-term memory becomes an important factor in shaping their answer.

A more quantitative approach is to ask people to use a device with a button or slider to indicate their excitement with the music. This removes the language barrier and can be carried out in real time. However, such an apparatus may not be so easy to use in most natural music environments, such as a concert hall. Furthermore, by asking people to indicate their excitement, some preconditions are put into the experiment by the researcher and many nuances of how people perceive the music are not considered at all.

Yet another method, and the one that we will focus most on in Music Moves, is that of "motion capture". Observing people's bodily response to music is "easy" for most people, since they do not have to put their experience into words nor do any conscious activity other than just moving regularly. However, the close connection between perceiving and moving (as discussed in the video on perception: affordances, mirror neurons, etc.) suggests that movement to music may be a bodily expression of how you perceive the music. Marc Leman calls this corporeal articulations, which he discusses thoroughly in the book Embodied music cognition and mediation technology.

Suggested reading