Using a multisensory interface system, we examined how peoples emotional experiences change as their tactile sense (touching a plant) was augmented with visual sense (seeing their touch). Our system (the Interactive Plant system) senses the electrical capacitance of the human body and visualizes users tactile information on a flat screen (when the touch is gentle, the program draws small and thin roots around the pot; when the touch is more harsh or abrupt, big and thick roots are displayed). We contrasted this multimodal combination (touch + vision) with a unimodal interface (touch only or watch only) and measured the impact of the multimodal interaction on participants emotion. We found significant emotional gains in the multimodal interaction. Participants self-reported positive affect, joviality, attentiveness and self-assurance increased dramatically in multimodal interaction relative to unimodal interaction; participants electrodermal activity (EDA) increased in the multimodal condition, suggesting that our plant-based multisensory visual-tactile interaction raised arousal. We suggest that plant-based tactile interfaces are advantageous for emotion generation because haptic perception is by nature embodied and emotional.
