Is there a common thread that connects different symbols of an underlying state of mind, asks S.Ananthanarayanan.

Raised voices and flailing arms usually accompany fury. Graceful movement and melodious tones suggest peace and tranquility. Are there parallels in vocal and visual symbols, colours or forms of expression? Are these symbols universal? Or, are they culture specific?

Beau Sievers, Caitlyn Lee William Haslett and Thalia Wheatley, from Harvard University and Dartmouth College and the Geisel School of Medicine, Hanover, New Hampshire, USA subject these questions to scientific analysis and present their findings in the journal, Proceedings of the Royal Society, published from London. The authors find that emotional arousal is conveyed and perceived with the help of a stimulus feature that is the same with the different senses, which leads to symbols that are used being shared between cultures and even between species.

The authors refer to earlier work that suggests that music and movement, as in dance or a military march, share features which tally with the kind of music that is associated with kinds of movement. There is also evidence, they say, that basic, low-level physical features of shape, sound, colour, motion, are shared when emotions are expressed or interpreted. The authors were hence motivated to enquire to into the mechanism that brings this about.

The authors develop an index, or unit of measurement, that can characterise a sound, or a vocal signal, and they extend the unit to visual or other forms. In the case of vocal signals, any complex sound can be considered as many simple tones sounded together. In the pure tones of the flute or the violin, for instance, the vibration of the air is at the same rate as the named pitch of the tone. In complex tones, as of a trumpet, however, the character comes from the mixing of overtones of the basic tone, as shown in the figure. We can see that the shape of the outline, when the two tones are put together, is bumpier than with pure tones.

We can also imagine that the energy in a complex sound is distributed over the frequencies that the sound consists of. While the energy at some frequencies may be larger than at others, a central frequency that could be considered to account for all the energy would be somewhere near the most energetic frequencies. The authors work out this frequency, as the 'spectral centroid' (SC), a kind of 'centre of gravity', but in the terms of pitch.

The authors also refer to studies that have related shapes and sounds, with sharp sounds or soft sounds being related to spiky shapes or rounded shapes, and devise a measure of the 'spectral centroid' of shapes. As shown in the picture, rounded shapes have low SC, while shapes with many corners have high SC.

The experiment was then to test the proposition that high SC sounds and high SC shapes were related to high arousal emotions, like being angry or excited, while low SC sounds and shapes related to low arousal emotions, like being sad or peaceful.

The first trial was to see whether the SC of a stimulus was correlated with emotional arousal. Pairs of shapes were created, with the general outline matching, but with low or high SC. A set of sounds was also created, using non-phonetic pairs, 1.5 seconds long, with low or high SC. The sound to match the spiky shape consisted of four 54-millisecond bursts, sounded out of beat. The sound to match the smooth shape was a low pitch, pure tone, varying in loudness with a rhythm that matched the bursts in the other sound.

In the trial, participants, on seeing a shape, had to choose emotions, either 'angry' or 'sad'. Other participants had to see shapes and choose between 'peaceful' or 'excited'. And with sounds, again, a group of participants made choices, on hearing sounds of high or low SC. "Across all matching tasks, participants associated high mean SC sounds (noise burst) and shapes (spiky) with high-arousal emotions (angry, excited), and low mean SC sounds (sine wave) and shapes (rounded) with low-arousal emotions (sad, peaceful)" the paper says.

The second trial was to see how people might choose shapes to express emotional arousal. Participants were first shown a shape, which they had to classify as 'angry' or 'sad'. Next, they had to draw a shape that stood for the emotion they did NOT choose. And then answer questions – what was different between the shapes? And why did you draw what you did? A second phase of the trial was carried out in the same way, but without a prompting image. The participants, of course were chosen so that they had no idea what the trial was about.

In the first part of the trail, angry shapes were found to have a mean of 23.3 corners while sad shapes had 6.6. In the second part, where there were both negative and positive emotions, angry shapes had 24.2 corners, sad shapes had 8.8, excited shapes had 17.1 corners and peaceful shapes had 6.8.

The third trial was to test the hypothesis that shapes and sounds shared a relationship with emotional arousal. This was done by creating a large and varied sample of shapes and sounds and getting a theoretical model to predict what the response of participants may be. The result was that the most elaborate model, which evaluated SC, counted corners and all other features, got it right 78% of the time. With only SC and corners considered, the result was 77%, but other models seemed to flounder. This shows that SC is reliable indicator of what emotion the symbol would evoke, the paper says.

The next trial was to see if SC could predict judgements of extracts from databases of professional actors portraying emotions and reading emotionally neutral sentences. With SC for the vocal part and a suitable SC derived for movements, it was found that SC could correctly classify 'angry' and 'sad' expressions 86% and 90% of the time. The last trial was again drawn from the data bases, but portraying a range of emotional arousal categories. While the samples were in German, the participants selected knew no German. Here again, SC was found to be a reliable predictor of the emotional arousal a sound or form would evoke.

A valid question that can be raised is that the code of SC, by which sounds and shapes are found to convey emotion, could be learnt by all of us during our early years. The trials may hence not really prove that the code is universal. As the answer to this, the paper cites a report of similar trials which were carried out in L’ak, a remote Kreung village in northeastern Cambodia. What is special about this village is its geographical and cultural isolation. The results of the study at L'ak, which were replicated in the US, showed that common features relating different modes, like sound or shape, are independent of the cultural setting. Going further, the paper cites studies that show that humans are able to recognise emotional arousal from the vocalisation of animals.

While the paper notes that "understanding how the brain extracts low-level, cross-modal features to determine meaning" would advance our knowledge of communication, we can rest easy in the thought that there is still no algorithm that can create sounds or shapes to arouse emotions like a Mozart or a Van Gogh!

------------------------------------------------------------------------------------------ Do respond to : response@simplescience.in