Low-rank sparse coding and region of interest pooling for dynamic 3D facial expression recognition

Zarbakhsh, Payam; Demirel, Hasan

doi:10.1007/s11760-018-1318-5

Cited by 14 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, a 1 minute background sound with video were set up before the formal experiment to avoid the effect of visual factors. After the background sound, the main recording then plays after a 2-second transition, for 2 minutes (Zarbakhsh and Demirel, 2018).…”

Section: Stimulimentioning

confidence: 99%

“…As a environment evaluation tool, FaceReader (Noldus, 2014), a software based on facial expression recognition (FER), has been applied in psychological evaluations (Zarbakhsh and Demirel, 2018;Bartlett et al, 2005;Amor et al, 2014). Video cameras have been the predominant method of measuring facial expressions in this context (Oliver et al, 2000).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the effectiveness of facial expression recognition for evaluation of urban sound perception

Meng

Kang

et al. 2020

Science of The Total Environment

View full text Add to dashboard Cite

Sound perception studies mostly depend on questionnaires with fixed indicators. Therefore, it is desirable to explore methods with dynamic outputs. The present study aims to explore the effects of sound perception in the urban environment on facial expressions using software named FaceReader based on facial expression recognition (FER). The experiment involved three typical urban sound recordings, namely, traffic noise, natural sound, and community sound. A questionnaire on the evaluation of sound perception was also used, for comparison. The results show that, first, FER is an effective tool for sound perception research, since it is capable of detecting differences in participants' reactions to different sounds and how their facial expressions change over time in response to those sounds, with mean difference of valence between recordings from 0.019 to 0.059 (p<0.05or p<0.01). In a natural sound environment, for example, facial expression increased by 0.04 in the first 15 seconds and then went down steadily at 0.004 every 20 s. Second, the expression indices, namely, happy, sad, and surprised, change significantly under the effect of sound perception. In the traffic sound environment, for example, happy decreased by 0.012, sad increased by 0.032, and surprised decreased by 0.018. Furthermore, social characteristics such as distance from living place to natural environment (r=0.313), inclination to communicate (r=0.253), and preference for crowd (r=0.296) have effects on facial expression. Finally, the comparison of FER and questionnaire survey results showed that in the traffic noise recording, valence in the first 20 s best represents acoustic comfort and eventfulness; for natural sound, valence in the first 40 s best represents pleasantness; and for community sound, valence in the first 20 s of the recording best represents acoustic comfort, subjective loudness, and calmness.

show abstract

Section: Stimulimentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

On the effectiveness of facial expression recognition for evaluation of urban sound perception

Meng

Kang

et al. 2020

Science of The Total Environment

View full text Add to dashboard Cite

show abstract

“…Psychologists generally believe that expressions are a quantitative form of changes in emotions. As a tool for evaluating emotions, the software FaceReader, based on facial expression recognition (FER), has been applied in psychological evaluation (Bartlett et al, 2005;Amor et al, 2014;Zarbakhsh and Demirel, 2018). The effectiveness of FER has been proven in many previous studies, and it can measure emotions with more than 87% efficacy (Terzis et al, 2010).…”

Section: Introductionmentioning

confidence: 99%

The Effectiveness of Facial Expression Recognition in Detecting Emotional Responses to Sound Interventions in Older Adults With Dementia

Liu

Wang

2021

Front. Psychol.

View full text Add to dashboard Cite

This research uses facial expression recognition software (FaceReader) to explore the influence of different sound interventions on the emotions of older people with dementia. The field experiment was carried out in the public activity space of an older adult care facility. Three intervention sound sources were used, namely, music, stream, and birdsong. Data collected through the Self-Assessment Manikin Scale (SAM) were compared with facial expression recognition (FER) data. FaceReader identified differences in the emotional responses of older people with dementia to different sound interventions and revealed changes in facial expressions over time. The facial expression of the participants had significantly higher valence for all three sound interventions than in the intervention without sound (p < 0.01). The indices of sadness, fear, and disgust differed significantly between the different sound interventions. For example, before the start of the birdsong intervention, the disgust index initially increased by 0.06 from 0 s to about 20 s, followed by a linear downward trend, with an average reduction of 0.03 per 20 s. In addition, valence and arousal were significantly lower when the sound intervention began before, rather than concurrently with, the start of the activity (p < 0.01). Moreover, in the birdsong and stream interventions, there were significant differences between intervention days (p < 0.05 or p < 0.01). Furthermore, facial expression valence significantly differed by age and gender. Finally, a comparison of the SAM and FER results showed that, in the music intervention, the valence in the first 80 s helps to predict dominance (r = 0.600) and acoustic comfort (r = 0.545); in the stream sound intervention, the first 40 s helps to predict pleasure (r = 0.770) and acoustic comfort (r = 0.766); for the birdsong intervention, the first 20 s helps to predict dominance (r = 0.824) and arousal (r = 0.891).

show abstract

“…In equation (3), Φ shows the feature quantity, and ∇Φ stands for the gradient of Φ. The matching items corresponding to the local template are represented by E LBF ; the sampling components corresponding to edge pixels are represented by E RGB , and the fusion template of the visual image region can be obtained by sparse linear segmentation (Zarbakhsh and Demirel, 2018 ). The function Data of the fusion template is expressed as:…”

Section: Methodsmentioning

confidence: 99%

The interior environment design for entrepreneurship education under the virtual reality and artificial intelligence-based learning environment

Zhang

et al. 2022

Front. Psychol.

View full text Add to dashboard Cite

Nowadays, with the rapid growth of artificial intelligence (AI), entrepreneurship education has attracted more and more attention from society. To this end, it is necessary to gradually transform the traditional teaching mode into a new type of teaching that is more innovative, practical, and inclusive and in line with entrepreneurship education. The focus of the teaching mode change is on the optimization of the teaching environment. For this purpose, a method derived from distributed virtual reality (DVR) technology is specially designed. It refers to the fact that multiple users can join together through a computer network and participate in a virtual space at the same time to experience the virtual experience together. Based on this, the distributed 3D interior design is innovatively proposed. The innovation is mainly reflected in the application of VR technology, which is different from traditional software design. According to the functions and needs of the entrepreneurship teaching environment, first, the distributed feature information is collected, and second, the corresponding color image model is constructed by the fusion method, and edge contour detection and corresponding feature data extraction are carried out for the distributed image. Using a Red, Green, and Blue (RGB) color decomposition method, the pixel feature decomposition of spatially distributed image color is performed. And the feature reorganization of the 3D point cloud is combined to optimize the color space and color features of the combined design. On this basis, the distributed 3D interior design system is designed with VR and visual simulation technology. Finally, the Three-Dimensional Studio Max (3ds MAX) is used to establish 3D modeling, and the modeling software Multigen Creator is adopted to carry out the hierarchical structural design. The test results manifest that the Normalized Root Mean Square Error (RMSE) and information saturation of the distributed 3D interior design are reduced by 0.2 compared with the traditional design, the time overhead is shortened to one-sixth of the original, and the effect is more in line with the design requirements. It is hoped that this design method can provide new ideas and new perspectives for the optimization of the entrepreneurship teaching environment.

show abstract

Low-rank sparse coding and region of interest pooling for dynamic 3D facial expression recognition

Cited by 14 publications

References 33 publications

On the effectiveness of facial expression recognition for evaluation of urban sound perception

On the effectiveness of facial expression recognition for evaluation of urban sound perception

The Effectiveness of Facial Expression Recognition in Detecting Emotional Responses to Sound Interventions in Older Adults With Dementia

The interior environment design for entrepreneurship education under the virtual reality and artificial intelligence-based learning environment

Contact Info

Product

Resources

About