Structural and temporal inference search (STIS): pattern identification in multimodal data Conference Paper uri icon

abstract

  • There are a multitude of annotated behavior corpora (manual and automatic annotations) available as research expands in multimodal analysis of human behavior. Despite the rich representations within these datasets, search strategies are limited with respect to the advanced representations and complex structures describing human interaction sequences. The relationships amongst human interactions are structural in nature. Hence, we present Structural and Temporal Inference Search (STIS) to support search for relevant patterns within a multimodal corpus based on the structural and temporal nature of human interactions. The user defines the structure of a behavior of interest driving a search focused on the characteristics of the structure. Occurrences of the structure are returned. We compare against two pattern mining algorithms purposed for pattern identification amongst sequences of symbolic data (e.g., sequence of events such as behavior interactions). The results are promising as STIS performs well with several datasets. Copyright 2012 ACM.

name of conference

  • Proceedings of the 14th ACM international conference on Multimodal interaction

published proceedings

  • Proceedings of the 14th ACM international conference on Multimodal interaction

author list (cited authors)

  • Miller, C., Morency, L., & Quek, F.

citation count

  • 6

complete list of authors

  • Miller, Chreston||Morency, Louis-Philippe||Quek, Francis

publication date

  • October 2012