Gesture, Speech, and Gaze in Discourse Segmentation and the NSF KDI program, Grant No. BCS-9980054, Cross-Modal Analysis of Signal and Sense: Multimedia Corpora and Tools for Gesture, Speech, and Gaze Research Hand Gesture Symmetric Behavior Detection and Analysis in Natural Conversation* *This research has been supported by the U.S. National Science Foundation STIMULATE program Grant No. IRI-9618887

2002 IEEE. We present an experimental investigation into the phenomenon of gestural symmetry for two-handed gestures accompanying speech. We describe an approach to compute hand motion symmetries based on the correlation computations. Local symmetries are detected using a windowing operation. We demonstrate that the selection of a smaller window size results in better sensitivity to local symmetries at the expense of noise in the form of spurious symmetries and 'symmetry dropoffs'. Our algorithm applies a 'hole filling' post process to address these detection problems. We examine the role of the detected motion symmetries of two-handed gestures in the structuring of speech. We compared discourse segments corresponding to extracted symmetries in two natural conversations against a discourse analysis by expert psycholinguistic coders. These comparisons illustrate the effectiveness of the symmetry feature for the understanding of underlying discourse structure. We believe that this basic characteristic of two-handed gestures accompanying speech must be incorporated in any multimodal interaction system involving two-handed gestures and speech.

Overview