In this blog as a whole, I have been exploring the ways that Theatre and Affective Neuroscience might be in fact each other’s reflections, since both work closely with emotions. Both theatre (acting in particular) and psychology analyze emotional production and perception, as well as the variety of possible sources of emotions and their manifestations. The representation, or rather the performance of human emotions relies strongly on verbal expression. Many researchers suggest that the language is what makes people so exquisitely developed in comparison to any other mammals. Whereas the language ‘belongs’ to the cognitive domain, there are emotional elements in it. One of the essential parts of language is prosody. In the following blog I attempt to summarize the latest research on emotional prosody. I also present a famous and useful exercise from the world of performing arts – the gibberish language, – hoping that it may help us to facilitate the understanding of the production and perception of emotions.
One of the most favorite and challenging exercises used in the Into to Acting classes is speaking so-called “gibberish” language. It is used in rehearsals and even performances. After experiencing speaking gibberish, most students admit how hard it is to express a thought without using an actual language. However, a rather contradictory revelation usually follows: students realize that expressing emotions does not require the use of meaningful words. They notice how the non-verbal aspects of vocal expression can be used to achieve the same result as words.
For those who want to hear what gibberish sounds like, this is an example:
/gibberish starts after 03:05/
Gibberish, by Josh Hutcherson (Late Night with Jimmy Fallon, 11/14/2014)
Gibberish represents the speech excluding its semantic component. The closest analogy would be the pre-linguistic vocalizations created and used by little children in order to communicate with their parents or simply persons other than themselves. Grammelot (or gibberish) is imaginary, it has no existence outside of the temporal frame of the performance; it is a communication invented for a given group which is gathered for a limited time (Jaffe-Berg 2001). It is claimed to have a freeing or liberating effect by its “ability to create a space for communication which relies little on accepted communicative linguistic codes.” (Jaffe-Berg, 2001, pp. 8) Gibberish is not a stranger to anyone who was a child once (so, literally everyone). The use of sounds in gibberish communicates the meaning to the audience, similarly to the vocalizations of infants communicating the meanings, or desires, or requests and reactions to the adults or fellow babies. From the artistic perspective, infantile gibberish is the purest example of free and unique creativity. From the neurological perspective, gibberish is the purest form of human emotional prosody.
Research shows that the most important function of infant-directed (ID) speech prosody is to help create and maintain an emotional bond between caregiver and pre-linguistic infant, which is crucial to survival. (Trainor, 2000) Trainor suggests that “social conventions often dictate the restraint of prosodic expressions of emotion, perhaps in order to allow more cognitive, reflective reaction to prevail over immediate emotional actions.” (pp. 188) Infants are unable to understand the language per se; instead it is the prosody that manages to transmit the important information to them. In terms of the form and structure, the language of kids is often referred to as gibberish, since it lacks linguistic meaning. Trainor concludes that since the auditory system matures earlier than the visual system and because language is primarily based in the auditory modality, the vocal expression of emotion likely plays a major role in early emotional development. Indeed, it seems that the perception of prosody by the ‘language-free’ infants relies solely on emotional prosody of the vocal expression, which may explain the universality of ID speech production across cultures. (Trainor, 2000)
Overall, researchers agree that prosody can be sub-divided into two categories: emotional and linguistic. When synthetic speech is created, the lack of emotional prosody leads to uninvolved and unnaturally sounding speech (Mozziconacci 2002). Fruhholz (2011) defines emotional prosody as affective cues and emotional states decoded and inferred from suprasegmental vocal modulations. Mitchell (2003) provides a nuanced distinction between different manifestations of prosody: non-linguistic, or emotional functions of prosody usually are represented by gradient features such as pitch height and range, as well as rhythmicity. (pp.1410) Aubergé (2002) suggests that as an integrative agent of the main functions of the communicative system, prosody becomes the frame of the direct emotional expression, of the coding system of the attitudes and expressive strategies for the acoustic material.
Various studies of neural circuits involved in the processing of emotional prosody showed that the amygdala is inevitably involved in the perception of emotional vocalizations (Morris, 1998; Sander, 2001; Adolphs, 2002; Mitchell, 2003; Fecteau, 2007). Based on the fMRI results, it has been suggested that the amygdala “exhibits a similarly enhanced response pattern to various emotions expressed by prosody”. (Wiethoff et al., 2009, pp. 1359) According to one of the latest fMRI experiments, Pichon and Kell (2013) found support for two models of emotional prosody: during the preparation, they recorded strong bilateral involvement of basal ganglia, limbic regions, temporal pole and anterior insula; during the actual production, they observed right-lateralized auditory feedback-related processing. (p. 1647)
From this rather brief overview, one may conclude that the emotional prosody is a language in itself. In the context of acting, it impacts the audience as effectively as the power of words. Such acting tool as Gibberish language provides a direct opportunity for the emotional prosody to be manifested. It is a ‘shortcut’ through the cognitive aspect of language, and it is connected with emotions directly. Gibberish bypasses the cognitive brain activity during the acquisition and/or production of emotions. The author has not found any studies that analyze brain activity and neural circuits of emotional prosody experienced during performances in gibberish language. Maybe it is a potential future research in which emotional prosody could be explored in its purest form.
References:
Adolphs, R., Damasio, H., & Tranel, D. (2002). Neural systems for recognition of emotional prosody: a 3-D lesion study. Emotion, 2(1), 23.
Aubergé, V. (2002). Prosodie et émotion. Actes des deuxiemes assises nationales du GdR I, 3, 263-273.
Fecteau, S., Belin, P., Joanette, Y., & Armony, J. L. (2007). Amygdala responses to nonlinguistic emotional vocalizations. Neuroimage, 36(2), 480-487.
Jaffe-Berg, E. (2001). Forays into Grammelot: The Language of Nonsense. Journal of Dramatic Theory and Criticism, (2), 3-16.
Morris, J. S., Scott, S. K., & Dolan, R. J. (1999). Saying it with feeling: neural responses to emotional vocalizations. Neuropsychologia, 37(10), 1155-1163.
Mozziconacci, S. (2002). Prosody and emotions. In Speech Prosody 2002, International Conference.
Mitchell, R. L., Elliott, R., Barry, M., Cruttenden, A., & Woodruff, P. W. (2003). The neural response to emotional prosody, as revealed by functional magnetic resonance imaging. Neuropsychologia, 41(10), 1410-1421.
Pichon, S., & Kell, C. A. (2013). Affective and sensorimotor components of emotional prosody generation. The Journal of Neuroscience, 33(4), 1640-1650.
Sander, K., & Scheich, H. (2001). Auditory perception of laughing and crying activates human amygdala regardless of attentional state. Cognitive Brain Research, 12(2), 181-198.
Trainor, L. J., Austin, C. M., & Desjardins, R. N. (2000). Is infant-directed speech prosody a result of the vocal expression of emotion? Psychological science, 11(3), 188-195.
Wiethoff, S., Wildgruber, D., Grodd, W., & Ethofer, T. (2009). Response and habituation of the amygdala during processing of emotional prosody. Neuroreport, 20(15), 1356-1360.
Wildgruber, D., Ackermann, H., Kreifelts, B., & Ethofer, T. (2006). Cerebral processing of linguistic and emotional prosody: fMRI studies. Progress in brain research, 156, 249-268.
Wildgruber, D. (2009). A cerebral network model of speech prosody comprehension. International Journal of Speech-Language Pathology, 11(4), 277-281. doi:10.1080/17549500902943043
Below is some additional more detailed information about the neural processes of emotional prosody… I didn’t feel like it had a place in this blog post, but it’s very interesting:
Adolphs and colleagues (2006) suggest that early and later perceptual processing of the auditory features of a stimulus involve primary and high level auditory cortices; the amygdala and the oPFC link the perceptual auditory representations to structures that can enact emotional responses or that can reconstruct associated knowledge; motor and premotor structures play a role in simulating components of the emotional body state normally associated with producing the emotion (the right premotor cortices, left frontal operculum, and the basal ganglia); and finally, somatosensory structures in the right hemisphere represent the emotional body states simulated by the motor structures.
Wildgruber suggests a three-step process to describe the perception of emotional prosody:
The first step, extraction of supra-segmental acoustic information, is associated with activation of predominantly right hemispheric primary and secondary acoustic regions. The second step, representation of meaningful supra-segmental acoustic sequences, is linked to posterior aspects of the right superior temporal sulcus. The third step, emotional judgment, is linked to the bilateral inferior-frontal cortex (IFC). (p. 278)
And for those who are interested in empathy, here is a research that links it with gibberish language:
Besides being an essential component of human social and emotional development, gibberish is linked to empathy. In 1995, Ellen Junn and colleagues conducted an experiment where 61 young college students were asked to interact fully in gibberish (a minimal script and some directions were provided). The topic of their conversations was “Multicultural awareness.” As a result, the majority of students admitted that after participating in the exercise, they felt “more sensitized, appreciative, and empathic to the challenges that face nonnative English speaking students.”
Junn, E. N., Morton, K. R., & Yee, I. (1995). The” Gibberish” exercise: Facilitating empathetic multicultural awareness. Journal of Instructional Psychology.
I just want to add from a computational perspective the importance of prosody in understanding emotions. Most computational speech recognizers of emotions take advantage of various prosodic cues, and the effectiveness of using just prosody is far greater than most models that use the semantics of what is being spoken. However, this becomes problematic if the speech lacks prosody. My lab is currently working on a project with Parkinson’s patients. One symptom they have is a significant lack of prosody in their speech. This makes it difficult for people (never mind computers) to assess the affective content of what is being said.