The rapid evolution of large language models necessitates effective benchmarks for evaluating their role knowledge, which is essential for establishing connections with the real world and providing more immersive interactions. This paper introduces RoleEval, a bilingual benchmark designed to assess the memorization, utilization, and reasoning capabilities of role knowledge. RoleEval comprises RoleEval-Global (including internationally recognized characters) and RoleEval-Chinese (including characters popular in China), with 6,000 Chinese-English parallel multiple-choice questions focusing on 300 influential people and fictional characters drawn from a variety of domains including celebrities, anime, comics, movies, TV series, games, and fictions. These questions cover basic knowledge and multi-hop reasoning abilities, aiming to systematically probe various aspects such as personal information, relationships, abilities, and experiences of the characters. To maintain high standards, we perform a hybrid quality check process combining both automatic and human verification, ensuring that the questions are diverse, challenging, and discriminative.
Aberrant affective neural processing and negative emotional bias are trait-marks of major depression disorders (MDDs). However, most research on biased emotional perception in depression has only focused on unimodal experimental stimuli, the neural basis of potentially biased emotional processing of multimodal inputs remains unclear. Here, we addressed this issue by implementing an audiovisual emotional task during functional MRI scanning sessions with 37 patients with MDD and 37 gender-, age-and education-matched healthy controls. Participants were asked to distinguish laughing and crying sounds while being exposed to faces with different emotional valences as background. We combined general linear model and psychophysiological interaction analyses to identify abnormal local functional activity and integrative processes during audiovisual emotional processing in MDD patients. At the local neural level, MDD patients showed increased bias activity in the ventromedial prefrontal cortex (vmPFC) while listening to negative auditory stimuli and concurrently processing visual facial expressions, along with decreased dorsolateral prefrontal cortex (dlPFC) activity in both the positive and negative visual facial conditions. At the network level, MDD exhibited significantly decreased connectivity in areas involved in automatic emotional processes and voluntary control systems during perception of negative stimuli, including the vmPFC, dlPFC, insula, as well as the subcortical regions of posterior cingulate cortex and striatum. These findings support a multimodal emotion dysregulation hypothesis for MDD by demonstrating that negative bias effects may be facilitated by the excessive ventral bottom-up negative emotional influences along with incapability in dorsal prefrontal top-down control system.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.