At the Amsterdam Gesture Center, based at the Vrije Universiteit/VU, we study the interplay of audible and visible bodily actions in communicative settings. Spoken language is not only ‘audio’ in nature but can also have visual components – that is: it is multimodal (audio-visual) to varying degrees. While our focus is on manual gesture with speech, we also take other bodily movements into consideration, such as bodily position, eye gaze, etc.

Our data come from different natural settings of language use, such as conversations between friends, interviews, speeches, and sales pitches, and so include topics and domains of discourse such as politics, entrepreneurship, leadership, environmental issues, and shared memories. Theoretical frameworks we draw on include cognitive linguistics (e.g., conceptual metaphor theory, conceptual integration or blending theory, fictive interaction), cognitive and functional theories of grammar, memory studies, and critical discourse analysis. Our methods usually involve special attention to qualitative micro-analysis of spoken language and gesture use in real time, but our approaches include various types of linguistic and discourse analysis, such as frame analysis, the analysis of behavioral alignment, conversation analysis, and predictive modeling.

We examine speech and co-verbal bahavior as dynamic and variably multimodal – in terms of individuals’ formulation and expression of ideas and also in how this plays out in interaction with others. Our approaches often integrate the psychological with the social, allowing for insight into processes of idea development in real time in various contexts of human interaction.

This work relates to two of the research domains profiled within the VU’s strategic plan, namely Connected World and Professional Services. For example, the research on behavioral alignment (e.g., mirroring in the use of linguistic structures, gestures, eye gaze) has important potential for work on human-computer interaction, an integral component of the theme Connected World and therefore of the interfaculty Network Institute at the VU. Furthermore, applied research in the AGC, e.g. in relation to persuasive communication in institutional settings, also fits the theme of Professional Services in the VU strategic plan.

Though the study of gesture dates back at least to the time of Roman orators, research in the modern era only gained momentum in the 1990s. The journal Gesture has been in publication since 2001 and the International Society for Gesture Studies was founded in 2002. Gesture studies is thus an area of research that is old as a basic notion, but novel in terms of the theoretical and methodological approaches currently being employed.