Measuring Human Learning in Design Space Exploration to Assess Effectiveness of Knowledge Discovery Tools

Abstract While there exist various knowledge discovery tools to support designers learning during design space exploration, there is no established definition of what is expected to be learned and how it should be measured. Measuring learning is important, as it enables assessing and comparing different knowledge discovery methods. In this paper, we review the major categories of learning goals that are introduced in the field of education. Then, 7 different measures are developed to target specific learning goals relevant to design space exploration. Different learning goals are targeted by modifying the domain of the knowledge that is tested by each measure, and the specific task that the user is asked to perform. A human-subject experiment is conducted to measure how these metrics are related. Specifically, the consistency and correlations between different combinations of the measures are examined. Based on the observations made in this study, we discuss the implications and issues for future usage.

Measuring Human Learning in Design Space Exploration to Assess Effectiveness of Knowledge Discovery Tools Conference Paper