Paulson, Brandon C. (2010-05). Rethinking Pen Input Interaction: Enabling Freehand Sketching Through Improved Primitive Recognition. Doctoral Dissertation.
Thesis
Overview
abstract
-
Online sketch recognition uses machine learning and artificial intelligence techniques
to interpret markings made by users via an electronic stylus or pen. The
goal of sketch recognition is to understand the intention and meaning of a particular
user's drawing. Diagramming applications have been the primary beneficiaries
of sketch recognition technology, as it is commonplace for the users of these tools to
rst create a rough sketch of a diagram on paper before translating it into a machine
understandable model, using computer-aided design tools, which can then be used to
perform simulations or other meaningful tasks.
Traditional methods for performing sketch recognition can be broken down into
three distinct categories: appearance-based, gesture-based, and geometric-based. Although
each approach has its advantages and disadvantages, geometric-based methods
have proven to be the most generalizable for multi-domain recognition. Tools, such as
the LADDER symbol description language, have shown to be capable of recognizing
sketches from over 30 different domains using generalizable, geometric techniques.
The LADDER system is limited, however, in the fact that it uses a low-level recognizer
that supports only a few primitive shapes, the building blocks for describing
higher-level symbols. Systems which support a larger number of primitive shapes have
been shown to have questionable accuracies as the number of primitives increase, or
they place constraints on how users must input shapes (e.g. circles can only be drawn
in a clockwise motion; rectangles must be drawn starting at the top-left corner).
This dissertation allows for a significant growth in the possibility of free-sketch
recognition systems, those which place little to no drawing constraints on users. In
this dissertation, we describe multiple techniques to recognize upwards of 18 primitive
shapes while maintaining high accuracy. We also provide methods for producing
confidence values and generating multiple interpretations, and explore the difficulties
of recognizing multi-stroke primitives. In addition, we show the need for a standardized
data repository for sketch recognition algorithm testing and propose SOUSA
(sketch-based online user study application), our online system for performing and
sharing user study sketch data. Finally, we will show how the principles we have
learned through our work extend to other domains, including activity recognition
using trained hand posture cues.
ETD Chair
publication date
