CAREER: A Multimodal Mixed-Initiative Research Notebook for Information Discovery
Grant
Overview
Affiliation
Other
View All
Overview
abstract
While search engines address the problem of helping people find particular information, they are not directly built to support people developing creative ideas. This research supports people engaged in sustained information discovery tasks, such as invention and thesis development, in which the goal is to create new knowledge by forming ideas while finding and assembling information. New modalities of human computer interaction will be developed to support people engaged in assembling their building blocks of invention, utilizing information semantics, dynamic organization, and visualization. The approach is based on the form of composition, which visually and conceptually integrates elements to represent a collection. The research notebook will support sustained innovation, in which people need to connect and integrate key findings in the collections they create while conducting research and invention over time. Physically-based techniques, such as mass-spring and flocking models, will be applied and extended to facilitate human manipulation of collections as compositions. New modalities of interaction, such as pen-based, will improve sensory feedback and engagement by enabling participants to literally get their hands into digital information. The physically-based techniques and new interaction modalities will be integrated with the combinFormation mixed-initiative composition platform to provide a basis for augmenting the research notebook beyond the desktop metaphor. Our principal hypothesis is that mixed-initiative composition and physically-based multimodal interaction will improve the emergence of new ideas during individual and team-based information discovery tasks. Using the research notebook on assignments by over 1,000 undergraduate students developing new inventions each year in The Design Process course will catalyze innovative education, while also generating evaluation data on information discovery. The proposed research is innovative in its interdisciplinary approach, expected to yield significant outcomes: (1) new techniques for physically-based interaction with information; (2) new multimodal techniques for interaction with information; (3) new integrated semantic, visual, and interactive methods for supporting human discovery of connections among ideas; (4) a research notebook infrastructure for supporting sustained interaction with digital information by students and researchers; (5) new data about how students and researchers develop ideas over time in practice; (6) new understanding of creative processes that make up research and invention; and (7) new methods for supporting research and invention in education. Advanced learning technologies developed by this proposal will be implemented and distributed through the Research Notebook and the combinFormation platform. Software technologies will be made available to the students and the public through the web. A multimodal interaction lab will make integrated software/hardware systems available to some of the students. The Research Notebook methods, techniques, and cyberinfrastructure can amplify the productivity of scientists. The new interactive techniques for discovery and semantics can transform how students, researchers, knowledge workers, and the public experience the internet and digital repositories. The integration into education of research issues, designs, and implementations, with a learner-centered approach, will contribute to the development of a new generation of creativity-oriented human-centered computing researchers and practitioners.
