Exploring Siamese Neural Network Architectures for Preserving Speaker Identity in Speech Emotion Classification Conference Paper uri icon

abstract

  • 2018 Association for Computing Machinery. Voice-enabled communication is increasingly being used in real-world applications, such as the ones involving conversational bots or chatbots. Chatbots can spark and sustain user engagement by effectively recognizing their emotions and acting upon them. However, the majority of emotion recognition systems rely on rich spectrotemporal acoustic features. Beyond the emotion-related information, such systems tend to preserve information relevant to the identity of the speaker, therefore raising major privacy concerns from the users. This paper introduces two hybrid architectures for privacy-preserving emotion recognition from speech. These architectures rely on a Siamese neural network, whose input and intermediate layers are transformed using various privacy-performing operations in order to retain emotion-dependent content and suppress information related to the identity of a speaker. The proposed approach is evaluated through emotion classification and speaker identification performance metrics. Results indicate that the proposed framework can achieve up to 67.4% for classifying between happy, sad, frustrated, anger, neutral and other emotions, obtained from the publicly available Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset. At the same time, the proposed approach reduces speaker identification accuracy to 50%, compared to 81%, the latter being achieved through a feedforward neural network solely trained on the speaker identification task using the same input features.

name of conference

  • Proceedings of the 4th International Workshop on Multimodal Analyses Enabling Artificial Agents in Human-Machine Interaction

published proceedings

  • PROCEEDINGS OF THE 4TH INTERNATIONAL WORKSHOP ON MULTIMODAL ANALYSES ENABLING ARTIFICIAL AGENTS IN HUMAN-MACHINE INTERACTION (MA3HMI 2018)

author list (cited authors)

  • Arora, P., & Chaspari, T.

citation count

  • 7

complete list of authors

  • Arora, Priya||Chaspari, Theodora

publication date

  • October 2015