Gestural trajectory symmetries and discourse segmentation
Conference Paper
Overview
Additional Document Info
View All
Overview
abstract
Our approach is motivated by the conviction that gesture and speech are coexpressive of the underlying dynamic ideation that drives human communication. As such, transitions and cohesions is gestural behavior would inform us as to the discourse conceptualization. In this paper, we examine the role of motion symmetries of twohanded gestures in the structuring of speech. We employ a set of hand motion traces extracted from video and compute the correlation of these traces. The signs and magnitudes of the correlation coefficients computed in the cardinal directions of the subject's torso (lateral and vertical in this work) characterize the symmetries. We employ a windowed computation approach that permits a balance between temporal resolution and robustness to noise. The resulting correlation profiles are merged according to a temporal proximity rule. We apply this analysis to two conversational video sequences. A detailed analysis of the first sequence reveals the persistence of gestural imagery between semantically-similar discourse pieces. A symmetry transition analysis is applied to the second dataset and compared against a manually generated discourse segmentation to demonstrate the potential of cross-modal discourse segmentation.
