TY - GEN

T1 - Remote Facial Emotion Recognition System

AU - Хазанкин, Григорий Романович

AU - Шмаков, Иван Сергеевич

AU - Малинин, Алексей Николаевич

N1 - Conference code: V

PY - 2019/10

Y1 - 2019/10

N2 - This paper describes a method for collecting information of user's emotional state based on its facial expressions and linking the data acquired to the user's actions in a time. Methods and Algorithms: At client side, we use reactive programming model implemented by RxJS library to manage data streams. Our client library includes logic to combining events from different sources such as MediaStream APIs, classic DOM event API. We have developed our own neural network architecture that allows us to map the facial expressions of people into a continuous space of arousal/valence. The same architecture for classification of emotions was used. We used inception and residual blocks and batch normalization layers to achieve higher accuracy. In addition, we augment train data by adding pose variations using 3D face reconstruction. Results: We have developed a system that allows us to collect various data such as video stream and events of user interaction with the webpage. We also trained a neural network that shows high accuracy. One part of the system is a client library that does not require the user to install any special software and can be easily integrated into the website. The other part is a server application that enables the site owner to analyze the data collected.

AB - This paper describes a method for collecting information of user's emotional state based on its facial expressions and linking the data acquired to the user's actions in a time. Methods and Algorithms: At client side, we use reactive programming model implemented by RxJS library to manage data streams. Our client library includes logic to combining events from different sources such as MediaStream APIs, classic DOM event API. We have developed our own neural network architecture that allows us to map the facial expressions of people into a continuous space of arousal/valence. The same architecture for classification of emotions was used. We used inception and residual blocks and batch normalization layers to achieve higher accuracy. In addition, we augment train data by adding pose variations using 3D face reconstruction. Results: We have developed a system that allows us to collect various data such as video stream and events of user interaction with the webpage. We also trained a neural network that shows high accuracy. One part of the system is a client library that does not require the user to install any special software and can be easily integrated into the website. The other part is a server application that enables the site owner to analyze the data collected.

KW - Biometrics , Behaviometrics , Network Traffic Analysis , Identification , Signature-based Approach , Inference-Based approach , Hybrid approach , YAF , GMMs

KW - facial expression

KW - data streams

KW - valence

KW - machine learning

KW - remote diagnostics

KW - psycho-physiology

KW - arousal

UR - http://www.scopus.com/inward/record.url?scp=85079058904&partnerID=8YFLogxK

U2 - 10.1109/SIBIRCON48586.2019.8958047

DO - 10.1109/SIBIRCON48586.2019.8958047

M3 - Conference contribution

SN - 978-1-7281-4402-3

T3 - SIBIRCON 2019 - International Multi-Conference on Engineering, Computer and Information Sciences, Proceedings

SP - 975

EP - 979

BT - SIBIRCON 2019 - International Multi-Conference on Engineering, Computer and Information Sciences, Proceedings

PB - IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

T2 - 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON)

Y2 - 21 October 2019 through 27 October 2019

ER -