Micro-expression detection and analysis
Collaboration with Machine Learning team of Lab. Hubert Curien and Lab. LIBM / MPR CHU Saint-Etienne
The research topic involves capturing, detecting, analyzing and later modeling and classifying human facial expressions, specifically micro-expressions. Micro-expressions (MEs) are very subtle facial expressions, lasting only a fraction of a second. This work can be used in many application domains such as in human-machine interfaces, behavioral analysis, pain assessment for non-communicative patients, affective state identification for autistic patients and video security systems.
In a video, the micro-expressions are sporadic events and have low amplitudes with respect expressions. The spotting of the three parts of a micro-expression (onset, apex and offset) is still a challenge, like their classifications with respect to the valence (positive, negative and neutral). Color and thermal videos can be used. But how to properly manage the complementary information remains an opened question. Any complementary information could also be used (cardiac rate, breathing rate, ...), so a fusion step (or how combining the different data) comes a specific challenging problem.
On the basis of results about motion amplification for color videos, the monogenic signal obtained from a Riesz pyramid allows us to obtain the local phase and direction of the regions of the face where the micro-expressions occur. To this aim, we are also using a robust estimation for facial landmarks. From these data, the objective is to propose a detection and recognition scheme and we explore various methodologies like Gaussian processes to describe the spatio-temporal evolution of a micro-expressions.
In order to test our ME spotting method we used two databases containing spontaneously elicited MEs recorded by high-speed cameras: CASME II [10] and SMIC-HS [11]. We have obtained similar results to the state of the art and even surpass them in the case of CASME II.
We are currently working with the CHU of Saint-Etienne for the acquisition and the processing of gray scale and thermal videos of patient with disorders of consciousness. This project has obtained a grant from the Fondation of Jean Monnet University.
Framework for the detection and analysis of micro-expressions