Multisource deep learning for situation awareness

Blasch, Erik; Liu, Zheng; Zheng, Yefeng; Majumder, Uttam; Aved, Alexander; Zulch, Peter

doi:10.1117/12.2519236

Cited by 3 publications

(1 citation statement)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Measures of performance have been proposed for deep learning (DL) approaches that use multimodal or multisource information along with contextual reasoning [9]. In order to intelligently fuse multiple observations, weights must be optimally selected to provide the most relevant or informative observations for enhanced situational awareness [10].…”

Section: Multidomain Image Data Augmentation and Processingmentioning

confidence: 99%

Application of multidomain sensor image fusion and training data augmentation for enhanced CNN image classification

Arnous

Narayanan

2022

J. Electron. Imag.

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) provide the sensing and detection community with a discriminative approach for classifying images. However, one of the largest limitations for deep CNN image classifiers is the need for extensive training datasets containing a variety of image representations. While current methods such as GAN data augmentation, additions of noise, rotations, and translations can allow CNNs to better associate new images and their feature representations to ones of a learned image class, many fail to provide new contexts of ground truth feature information. To expand the association of critical class features within CNN image training datasets, an image pairing and training dataset augmentation paradigm via a multi-sensor domain image data fusion algorithm is proposed. This algorithm uses a mutual information and merit-based feature selection subroutine to pair highly correlated cross domain images from multiple sensor domain image datasets. It then re-augments the corresponding cross domain image pairs into the opposite sensor domain's feature set via a highest mutual information, cross sensor domain, image concatenation function. This augmented image set then acts to retrain the CNN to recognize greater generalizations of image class features via cross-domain, mixed representations. Experimental results indicated an increased ability of CNNs to generalize and discriminate between image classes during testing of class images from SAR vehicle, solar cell device reliability screening and lung cancer detection image datasets.

show abstract

Section: Multidomain Image Data Augmentation and Processingmentioning

confidence: 99%

Application of multidomain sensor image fusion and training data augmentation for enhanced CNN image classification

Arnous

Narayanan

2022

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

Self-improving situation awareness for human–robot-collaboration using intelligent Digital Twin

et al. 2023

View full text Add to dashboard Cite

The situation awareness, especially for collaborative robots, plays a crucial role when humans and machines work together in a human-centered, dynamic environment. Only when the humans understands how well the robot is aware of its environment can they build trust and delegate tasks that the robot can complete successfully. However, the state of situation awareness has not yet been described for collaborative robots. Furthermore, the improvement of situation awareness is now only described for humans but not for robots. In this paper, the authors propose a metric to measure the state of situation awareness. Furthermore, the models are adapted to the collaborative robot domain to systematically improve the situation awareness. The proposed metric and the improvement process of the situation awareness are evaluated using the mobile robot platform Robotino. The authors conduct extensive experiments and present the results in this paper to evaluate the effectiveness of the proposed approach. The results are compared with the existing research on the situation awareness, highlighting the advantages of our approach. Therefore, the approach is expected to significantly improve the performance of cobots in human–robot collaboration and enhance the communication and understanding between humans and machines.

show abstract

Sentiment Analysis on Movie Reviews Dataset Using Support Vector Machines and Ensemble Learning

Jaithunbi

Harikrishnan

et al. 2022

International Journal of Information Technology and Web Engineering

View full text Add to dashboard Cite

The internet makes it easier for people to connect to each other and has become a platform to express ideas and share information with the world. The growth of the internet has indirectly led to the development of social networking sites. The reviews posted by people on these sites implies their opinion, and analysis over reviews is required to understand their intent. In this paper, natural language processing technique and machine learning algorithms are applied to classify the text data. The contributions of the proposed approach are three-fold: 1) chi square selector is applied to select the k-best features, 2) support vector machines is executed to classify the reviews (hyperparameters of the SVM classifier are tuned using GridSearch approach), and 3) bagging algorithm is applied with the base classifier over the newly built SVM classifier. The number of base classifiers of the bagging algorithm is varied accordingly. The results of the proposed approach are compared to the similar existing work, and hence, it is found to achieve better results as compared to the existing systems.

show abstract

Multisource deep learning for situation awareness

Cited by 3 publications

References 36 publications

Application of multidomain sensor image fusion and training data augmentation for enhanced CNN image classification

Application of multidomain sensor image fusion and training data augmentation for enhanced CNN image classification

Self-improving situation awareness for human–robot-collaboration using intelligent Digital Twin

Sentiment Analysis on Movie Reviews Dataset Using Support Vector Machines and Ensemble Learning

Contact Info

Product

Resources

About