Methods for Determination of Psychophysiological Condition of User Within Smart Environment Based on Complex Analysis of Heterogeneous Data

Levonevskii, Dmitrii; Shumskaya, Olga; Velichko, Alena; Uzdiaev, Mikhail; Malov, Dmitrii

doi:10.1007/978-981-13-9267-2_42

Cited by 10 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Однако, данный подход имеет свои ограничения: методы извлечения признаков, используемых для классификации, представляют собой сложные и не эффективные с вычислительной точки зрения процедуры, задаваемые вручную. В работах [21,22,23] были предложены концептуальные модели многомодальной обработки данных в задаче распознавания агрессии в социокиберфзических пространствах, однако в данных работах не представлены результаты экспериментальных исследований.…”

Section: известные подходы многомодального анализа при решении задачи...unclassified

Нейросетевая Модель Многомодального Распознавания Человеческой Агрессии

Uzdiaev¹

2020

Вестник КРАУНЦ. Физико-Математические Науки

Self Cite

View full text Add to dashboard Cite

Увеличение количества пользователей социокиберфизических систем, умных пространств, систем интернета вещей актуализирует проблему выявления деструктивных действий пользователей, таких как агрессия. При этом, деструктивные действия пользователей могут быть представлены в различных модальностях: двигательная активность тела, сопутствующее выражение лица, невербальное речевое поведение, вербальное речевое поведение. В статье рассматривается нейросетевая модель многомодального распознавания человеческой агрессии, основанная на построении промежуточного признакового пространства, инвариантного виду обрабатываемой модальности. Предлагаемая модель позволяет распознавать с высокой точностью агрессию в условиях отсутствия или недостатка информации какой-либо модальности. Экспериментальное исследование показало 81:8% верных распознаваний на наборе данных IEMOCAP. Также приводятся результаты экспериментов распознавания агрессии на наборе данных IEMOCAP для 15 различных сочетаний обозначенных выше модальностей. Growing user base of socio-cyberphysical systems, smart environments, IoT (Internet of Things) systems actualizes the problem of revealing of destructive user actions, such as various acts of aggression. Thereby destructive user actions can be represented in different modalities: locomotion, facial expression, associated with it, non-verbal speech behavior, verbal speech behavior. This paper considers a neural network model of multi-modal recognition of human aggression, based on the establishment of an intermediate feature space, invariant to the actual modality, being processed. The proposed model ensures high-fidelity aggression recognition in the cases when data on certain modality are scarce or lacking. Experimental research showed 81.8% correct recognition instances on the IEMOCAP dataset. Also, experimental results are given concerning aggression recognition on the IEMOCAP dataset for 15 different combinations of the modalities, outlined above.

show abstract

Section: известные подходы многомодального анализа при решении задачи...unclassified

Нейросетевая Модель Многомодального Распознавания Человеческой Агрессии

Uzdiaev¹

2020

Вестник КРАУНЦ. Физико-Математические Науки

Self Cite

View full text Add to dashboard Cite

show abstract

“…Absolutely all investigated platforms for electronic communication represent web-oriented systems [13,14]. "Parsers" are used for the automated gathering and structuring of the information from similar resources [15][16][17][18][19][20].…”

Section: Problem Of the Data Parsing Automationmentioning

confidence: 99%

Problems of Automation of the Aggression Analysis in Socio-Cyberphysical Environment

Kulagina¹,

Iskhakov²

2020

Proceedings of the 8th Scientific Conference on Information Technologies for Intelligent Decision Making Support (ITIDS 2020)

View full text Add to dashboard Cite

The article presents the results of the carried out interdisciplinary research that allowed creating algorithmic software to determine the structural composition of investigated cases of electronic communication that could be interpreted as aggressive. Such interpretation is offered to be carried out by means of the system analysis of the hypertext according to the following metrics: the intensity of discussion, density and reference. The research allowed us to define sociometric models of electronic communication accompanied by aggression; the primary detection of these models is the basis for carrying out the procedure of a detailed linguistic analysis of the text of the communication. Such an approach promotes the reduction of expenses for the management of aggression processes in the socio-cyberphysical environment. The received results prove the possibility to automate the analysis of information content and to raise the interpretability of phenomena in the socio-cyberphysical environment. We examined the aspects of crawling and parsing of modern webplatforms, focused data gathering, automatic and automated extraction of comments with the application of various technologies. The architecture of a system of multithreaded data gathering is offered.

show abstract

“…To solve this complex, multifaceted and ambiguous problem, it is necessary to consider a number of smaller private tasks, among which it is important to monitor the content of the virtual environment, create automatic means of recognizing and detecting possible destructive effect on the user and respond to such content in a timely manner. An example of a text material analysis is proposed in the work [1][2][3], the processing of audio materials (or audio parts of it) to detect perception anomalies is presented according to [4,5]. In this paper the authors give the results of research of graphic images, as an integral part of Internet content, which very often occurs on web resources of any orientation and can also negatively affect the condition of a person, change his mood, strengthen the impression of accompanying material of another type, and so on.…”

Section: Introductionmentioning

confidence: 99%

Detecting Images That Have a Destructive Impact on Users on the Internet

Iskhakova¹,

Meshcherykov²

2020

Proceedings of the 8th Scientific Conference on Information Technologies for Intelligent Decision Making Support (ITIDS 2020)

View full text Add to dashboard Cite

This article deals with the current problem of automated detection of facts of destructive influence of images on the Internet on the person. It should be noted that detecting aggressive content in images is more difficult than detecting an object and defining its category. The reason for this is that aggressive content has no specific color or object parameters, there are no common features for classification. In this case the use of convolutional neural networks with pre-learning is a successful solution. Due to increase in aggressive content on the Internet, the problem of identifying such objects in order to minimize its impact on users is acute. The paper presents the developed architecture of the neural network for solving the problem of image with aggressive content recognition. Experiments during 180 epochs showed that at the number of epochs ~ 95-100 during training and ~ during testing it is possible to achieve the classification accuracy 91.8% taking into account the top-5 results.

show abstract

Methods for Determination of Psychophysiological Condition of User Within Smart Environment Based on Complex Analysis of Heterogeneous Data

Cited by 10 publications

References 21 publications

Нейросетевая Модель Многомодального Распознавания Человеческой Агрессии

Нейросетевая Модель Многомодального Распознавания Человеческой Агрессии

Problems of Automation of the Aggression Analysis in Socio-Cyberphysical Environment

Detecting Images That Have a Destructive Impact on Users on the Internet

Contact Info

Product

Resources

About