Catchments, prosody and discourse Academic Article uri icon

abstract

  • We present our work on the temporal integration of hierarchies of communicative actions: kinesic, prosodic and discursive. We use the device of the catchment as the locus around which this integration proceeds. We present a detailed case study of a gesture and speech elicitation experiment in which a subject describes her living space to an interlocutor. First, we process the video data to obtain the motion traces of both of the subjects hands using the vector coherence mapping algorithm. We code the gestures to identify the catchments. We recover discourse purposes utilizing a system of guided questions. Finally, we define prosody in terms of the ToBI system. The results of these analyses are compared against the computed motion traces to identify the cues accessible in the gestural and audio data that correlate well with the psycholinguistic analyses. The results show that motion, prosody and discourse structure are integrated at each moment of speaking. The electronic edition of this article includes audio-visual data.

published proceedings

  • Gesture

altmetric score

  • 1

author list (cited authors)

  • McNeill, D., Quek, F., McCullough, K., Duncan, S. D., Furuyama, N., Bryll, R., Furuyama, N., & Ansari, R.

citation count

  • 61

complete list of authors

  • McNeill, David||Quek, Francis||McCullough, Karl-Erik||Duncan, Susan D||Furuyama, Nobuhiro||Bryll, Robert||Furuyama, Nobuhiro||Ansari, Rashid

publication date

  • December 2001