Using Vision Based Tracking to Support Real-Time Graphical Instruction for Students Who Have Visual Impairments Conference Paper uri icon

abstract

  • In this paper, we describe a real-time vision-based tracking system to help students who are blind or visually impaired (SBVI) to follow instructional discourse that employs graphical illustrations. The vision system employs a color model based tracking for both the instructor's pointing behavior and the SBVI's reading behavior, and maps the pointing positions into the same coordinates. Our Haptic Deictic System -HDS system also employs a haptic glove to provide SBVI access to the situated pointing behavior of instructor that is performed in conjunction with speech, and provides the instructor real-time visual feedback on the SBVI's reading actions. Thus, our system supports two-way situated multimodal communication. In this paper, we first introduce our system, and then we discuss studies that show the efficacy of our approach. 2010 IEEE.

name of conference

  • 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops

published proceedings

  • 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops

altmetric score

  • 3

author list (cited authors)

  • Fang, B., Oliveira, F., & Quek, F.

citation count

  • 2

complete list of authors

  • Fang, Bing||Oliveira, Francisco||Quek, Francis

publication date

  • June 2010