An effective methodology for dynamic 3D facial expression retrieval

Action unit detection in 3D facial videos with application in facial expression retrieval and recognition

“…Thus, we are capable of evaluating our scheme in terms of valid AU detections per 3D mesh of the sequence. Similar results can be drawn using the method of [14]. Table 5 illustrates the detection results on the BP 4D − Spontaneous data set for the per frame case.…”

“…'N/A' stands for not available information. The average detection rate of the proposed 24 AU s detection method is 66.23% compared to 65.50% for [14]. The proposed 13 AU s detection method also performs slightly better than the proposed 24 AU s detection method in the per mesh case for the reason explained above.…”

“…The proposed 24 AU detection method detects seventeen more AU s (AU 6, AU 7, AU 9, AU 10, AU 11, AU 12, AU 13, AU 14, AU 16, AU 17, AU 18, AU 19, AU 22, AU 23, AU 24, AU 28 and AU 30) than the method of [69], twelve more AU s (AU 5, AU 9, AU 11, AU 13, AU 16, AU 18, AU 19, AU 20, AU 22, AU 27, AU 28 and AU 30) than the method of [50], thirteen more AU s (AU 5, AU 9, AU 11, AU 13, AU 16, AU 18, AU 19, AU 20, AU 22, AU 24, AU 27, AU 28 and AU 30) than the method of [81], and ten more (AU 2, AU 10, AU 11, AU 13, AU 18, AU 19, AU 22, AU 24, AU 28 and AU 30) than [14]. It achieves an average per sequence detection rate of 80.45% for these 24 AU s, compared to 63.51% for the 12 AU s of [50] and 77.85% for the 14 AU s of [14]. In most cases, the proposed 13 AU detection method performs slightly better than the proposed 24 AU detection method.…”

“…Although there are plenty of methods using 2D images, 2D videos and 3D images for facial expression recognition [26,12,90,96,92,74,43,97,99,72,67,39,40,49,56,18,48], we are focused on the 3D video modality techniques [7,94,51,79,85,71,33,55,14,105]. Only the technique presented in [14] is tested on the BP 4D − Spontaneous data set and, thus, it can be reliable compared to the proposed method. Table 9 presents the results of 4D facial expression recognition for all the expressions of the BP 4D − Spontaneous data set.…”

“…However, when using a plain geometry-based feature, such as curvature, the information provided by the facial topology is not exploited. Method [14] presents a descriptor capturing the topological and geometric information of a 3D facial mesh sequence. Although the aforementioned descriptor was designed for facial expression retrieval purposes, its topological part is completely AU -based, which makes it possible to use it for AU detection.…”