Gestures in spoken and signed discourse

When producing discourse, humans use conventional language but also gestures.

For spoken language, gestures are defined as visible actions, mostly of the hands and arms, which hearing speakers use communicatively while they talk. For sign language, it is assumed that deaf signers gesture too, but the evidence for gestural behavior is limited.

Defining gestures in sign language is challenging, mainly since deaf signers use the same articulators (hands and arms, but also body and face) to encode words, grammar, and discourse.

Therefore, the first aim of the project is to examine whether deaf signers use gestures as well, in combination with sign, in order to communicate.

More importantly, we aim to carry out a truly novel crosslinguistic study with a cross-modal element, comparing a spoken (German) to a signed language (German Sign Language, DGS), in order to examine the relationship between gestures and information structure in discourse.

We will qualitatively and quantitatively analyze an already existing corpus of video-taped narrative retellings by German speakers and DGS signers, focusing on the incidence and formal features of gestures.

By identifying the details of when and how gestures are used during the production of spoken/signed discourse, the study will shed new light onto the functions that gestures can fulfill in relation to what is said/signed, and will thus strengthen our understanding of the tight relationship between language and gesture use in communication.