A CNN-Based Approach for Facial Emotion Recognition Using Hybrid Feature Fusion

Abstract

Facial Expression Recognition is a machine learning problem which aims to improve human computer interaction, pain estimation, healthcare monitoring and emotion detection. While many methods have been implemented for this problem, the overall performance is still unsatisfactory. In this work, a combination of Histogram of Gradients (HOG), Facial Landmarks, HOG features from a sliding window, and Convolutional Neural Network (CNN) features are used. HOG and Sliding Window HOG are used as handcrafted feature extraction techniques, while CNN is used as an automated feature extraction technique. At the end, handcrafted and automated features were concatenated in the fully connected layer of the CNN for emotion detection. The combination of handcrafted and automated features represents the latest evolution in image classification. A comparative analysis is conducted on FER2013 dataset for four different combinations of the above stated techniques. The comparative discussion of implementing these methods highlights that as the number of features extraction techniques increases, accuracy improves.