Read What You Touch with Intelligent Audio System for Non-Visual Interaction Academic Article uri icon

abstract

  • Slate-type devices allow Individuals with Blindness or Severe Visual Impairment (IBSVI) to read in place with the touch of their fingertip by audio-rendering the words they touch. Such technologies are helpful for spatial cognition while reading. However, users have to move their fingers slowly, or they may lose their place on screen. Also, IBSVI may wander between lines without realizing they did. We addressed these two interaction problems by introducing a dynamic speech-touch interaction model and an intelligent reading support system. With this model, the speed of the speech will dynamically change with the users finger speed. The proposed model is composed of (1) an Audio Dynamics Model and (2) an Off-line Speech Synthesis Technique. The intelligent reading support system predicts the direction of reading, corrects the reading word if the user drifts, and notifies the user using a sonic gutter to help him/her from straying off the reading line. We tested the new audio dynamics model, the sonic gutter, and the reading support model in two user studies. The participants feedback helped us fine-tune the parameters of the two models. A decomposition study was conducted to evaluate the main components of the system. The results showed that both intelligent reading support with tactile feedback are required to achieve the best performance in terms of efficiency and effectiveness. Finally, we ran an evaluation study where the reading support system is compared to other VoiceOver technologies. The results showed preponderance to the reading support system with its audio dynamics and intelligent reading support components.

published proceedings

  • ACM Transactions on Interactive Intelligent Systems

altmetric score

  • 0.75

author list (cited authors)

  • El-Glaly, Y. N., & Quek, F.

citation count

  • 3

complete list of authors

  • El-Glaly, Yasmine N||Quek, Francis

publication date

  • October 2016