Fill in this form to receive a download link:

Name *

Email *

In this paper, I compare three different assumptions about the relationship between speech, thought and gesture. These assumptions have profound conse- quences for theories about the representations and processing involved in ges- ture and speech production. I associate these assumptions with three simplified processing architectures. In the Window Architecture, gesture provides us with
a ‘window into the mind’. In the Language Architecture, properties of language have an influence on gesture. In the Postcard Architecture, gesture and speech are planned by a single process to become one multimodal message. The popular Window Architecture is based on the assumption that gestures come, as it were, straight out of the mind. I argue that during the creation of overt imagistic gestures, many processes, especially those related to (a) recipient design, and
(b) effects of language structure, cause an observable gesture to be very different from the original thought that it expresses. The Language Architecture and the Postcard Architecture differ from the Window Architecture in that they both incorporate a central component which plans gesture and speech together, how- ever they differ from each other in the way they align gesture and speech. The Postcard Architecture assumes that the process creating a multimodal message involving both gesture and speech has access to the concepts that are available in speech, while the Language Architecture relies on interprocess communication to resolve potential conflicts between the content of gesture and speech.

Fill in this form to receive a download link:

Name *

Email *

Human communication is often thought about in terms of transmitted messages in a conventional code like a language. But communication requires a specialized interactive intelligence. Senders have to be able to perform recipient design, while receivers need to be able to do intention recognition, knowing that recipient design has taken place. To study this interactive intelligence in the lab, we developed a new task that taps directly into the underlying abilities to communicate in the absence of a conventional code. We show that subjects are remarkably successful communicators under these conditions, especially when senders get feedback from receivers. Signaling is accomplished by the manner in which an instrumental action is performed, such that instrumentally dysfunctional components of an action are used to convey communicative intentions. The findings have important implications for the nature of the human communicative infrastructure, and the task opens up a line of experimentation on human communication.

Fill in this form to receive a download link:

Name *

Email *

Any model of motor control or sensorimotor transformations starts from an intention to trigger a cascade of neural computations, yet how intentions themselves are generated remains a mystery. Part of the difficulty in dealing with this mystery might be related to the received wisdom of studying sensorimotor processes and intentions in individual agents. Here we explore the use of an alternative approach, focused on understanding how we induce intentions in other people. Under the assumption that generating intentions in a third person relies on similar mechanisms to those involved in generating first- person intentions, this alternative approach might shed light on the origin of our own intentions. Therefore, we focus on the cognitive and cerebral operations supporting the generation of communicative actions, i.e. actions designed (by a Sender) to trigger (in a Receiver) the recognition of a given communicative intention. We present empirical findings indicating that communication requires the Sender to select his behavior on the basis of a prediction of how the Receiver will interpret this behavior; and that there is spatial overlap between the neural structures supporting the generation of communicative actions and the generation of first-person intentions. These results support the hypothesis that the generation of intentions might be a particular instance of our ability to induce and attribute mental states to an agent. We suggest that motor intentions are retrodictive with respect to the neurophysiological mechanisms that generate a given action, while being predictive with respect to the potential intention attribution evoked by a given action in other agents.

Fill in this form to receive a download link:

Name *

Email *

In this paper, we outline a cognitive architecture for communicators, called the BIC model. The model consist of three main components. First, a (B)iological component, in which the genetic or built-in capacities of the communicator are specified. Second, an (I)nteraction Engine which uses neo- Gricean mutual simulation to attribute communicative intentions to signals, and to create signals to convey communicative intentions to the I-system of other agents. The third component of the BIC model is a content addressable database of (C)onventions which is used to store form/meaning mappings that have been successfully computed by the I-system. These stored form/meaning mappings are indexed by types of communicative context, so they can be retrieved by the I-system to save computational resources. The model can be used both as a computational architecture for a communication module in an artificial agent and as a conceptual model of the human communicator.

Fill in this form to receive a download link:

Name *

Email *

Structural priming is a sufficient, but not a necessary condition for proving the existence of representations. Absence of evidence is not evidence of absence. Cognitive science relies on the legitimacy of positing representations and processes without “proving” every component. Also, psycholinguistics relies on other methods, including acceptability judgments, to find the materials for priming experiments in the first place.

Fill in this form to receive a download link:

Name *

Email *

Background: The background to this study is that aphasiologists have increasingly started to pay attention not only to the speech that people with aphasia produce, but also to their gestures. As there are a number of competing models about the production of gesture and speech in healthy subjects, it is important to evaluate whether, and if so how, these models could be used to guide the research into gesture and speech, and the relationship between these, in speakers with aphasia.

Aims: The aims and objectives of this study are to see how existing models of gesture and speech are able to accommodate the findings regarding the gesture and speech behaviour of speakers with aphasia, in the hope that (1) these models could shed light on the use of gesture in aphasic speakers, and potentially suggest new approaches to therapy for people with aphasia and (2) the aphasia gesture data might help fundamental psycholinguistics to evaluate the adequacy of existing gesture and speech models.

Methods & Procedures: The methodology here was theoretical. Four models of gesture and speech interaction were critiqued and we reviewed their ability to explain some of the central empirical findings in the area of gesture and speech in aphasia.
Outcomes & Results: The outcomes and results of this theoretical analysis were that, with respect to the relationship between gesture and speech in aphasia, (1) the four models under investigation could be reduced to two models, because three of the investigated models were based on the same core assumptions and (2) both of these models adequately explain these findings, but the Growth Point/Sketch/Interface Model is more satisfactory than the Lexical Access Model, because of the better fit with the experimental results on the use of gesture for facilitating word finding, and because it is more compatible with the finding that gestures are also used to enhance communicative efficiency by replacing speech.

Fill in this form to receive a download link:

Name *

Email *

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Speci!cally, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others’ actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-de!ned levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscienti!c data on language processing and discuss some of the implications of our proposal.

Fill in this form to receive a download link:

Name *

Email *

The rapid and fluent nature of human communicative interactions strongly suggests the existence of an online mechanism for intention recognition. We motivate and outline a mathematical model that addresses these requirements. Our model provides a way of integrating knowledge about the relationship between linguistic expressions and communicative intentions, through a rapid process of Bayesian update. It enables us to frame predictions about the processes of intention recognition, utterance planning and other-repair mechanisms, and contributes towards a broader theory of communication.

Fill in this form to receive a download link:

Name *

Email *

The tradeoff hypothesis in the speech–gesture relationship claims that (a) when gesturing gets harder, speakers will rely relatively more on speech, and (b) when speaking gets harder, speakers will rely relatively more on gestures. We tested the second part of this hypothesis in an experimental col- laborative referring paradigm where pairs of participants (directors and matchers) identified targets to each other from an array visible to both of them. We manipulated two factors known to affect the difficulty of speaking to assess their effects on the gesture rate per 100 words. The first factor, coda- bility, is the ease with which targets can be described. The second factor, repetition, is whether the targets are old or new (having been already described once or twice). We also manipulated a third factor, mutual visibility, because it is known to affect the rate and type of gesture produced. None of the manipulations systematically affected the gesture rate. Our data are thus mostly inconsistent with the tradeoff hypothesis. However, the gesture rate was sensitive to concurrent features of referring expressions, suggesting that gesture parallels aspects of speech. We argue that the redundancy between speech and gesture is communicatively motivated.

Fill in this form to receive a download link:

Name *

Email *

The tradeoff hypothesis in the speech–gesture relationship claims that (a) when gesturing gets harder, speakers will rely relatively more on speech, and (b) when speaking gets harder, speakers will rely relatively more on gestures. We tested the second part of this hypothesis in an experimental col- laborative referring paradigm where pairs of participants (directors and matchers) identified targets to each other from an array visible to both of them. We manipulated two factors known to affect the difficulty of speaking to assess their effects on the gesture rate per 100 words. The first factor, coda- bility, is the ease with which targets can be described. The second factor, repetition, is whether the targets are old or new (having been already described once or twice). We also manipulated a third factor, mutual visibility, because it is known to affect the rate and type of gesture produced. None of the manipulations systematically affected the gesture rate. Our data are thus mostly inconsistent with the tradeoff hypothesis. However, the gesture rate was sensitive to concurrent features of referring expressions, suggesting that gesture parallels aspects of speech. We argue that the redundancy between speech and gesture is communicatively motivated.