In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with selfupdated attention. Our VideoQA model firstly generates the global context-aware visual and textual features respectively by interacting current inputs with memory contents. After that, it makes the attentional fusion of the multimodal visual and textual representations to infer the correct answer. Multiple cycles of reasoning can be made to iteratively refine attention weights of the multimodal data and improve the final representation of the QA pair. Experimental results demonstrate our approach achieves state-of-theart performance on four VideoQA benchmark datasets.

Abstract-During the past two decades, a variety of multiobjective evolutionary algorithms (MOEAs) have been proposed in the literature. As pointed out in some recent studies, however, the performance of an MOEA can strongly depend on the Pareto front shape of the problem to be solved, whereas most existing MOEAs show poor versatility on problems with different shapes of Pareto fronts. To address this issue, we propose an MOEA based on an enhanced inverted generational distance indicator, in which an adaptation method is suggested to adjust a set of reference points based on the indicator contributions of candidate solutions in an external archive. Our experimental results demonstrate that the proposed algorithm is versatile for solving problems with various types of Pareto fronts, outperforming several state-of-the-art evolutionary algorithms for multi-objective and many-objective optimization.

Recently, non-linear statistical measures such as multi-scale entropy (MSE) have been introduced as indices of the complexity of electrophysiology and fMRI time-series across multiple time scales. In this work, we investigated the neurophysiological underpinnings of complexity (MSE) of electrophysiology and fMRI signals and their relations to functional connectivity (FC). MSE and FC analyses were performed on simulated data using neural mass model based brain network model with the Brain Dynamics Toolbox, on animal models with concurrent recording of fMRI and electrophysiology in conjunction with pharmacological manipulations, and on resting-state fMRI data from the Human Connectome Project. Our results show that the complexity of regional electrophysiology and fMRI signals is positively correlated with network FC. The associations between MSE and FC are dependent on the temporal scales or frequencies, with higher associations between MSE and FC at lower temporal frequencies. Our results from theoretical modeling, animal experiment and human fMRI indicate that (1) Regional neural complexity and network FC may be two related aspects of brain's information processing: the more complex regional neural activity, the higher FC this region has with other brain regions; (2) MSE at high and low frequencies may represent local and distributed information processing across brain regions. Based on literature and our data, we propose that the complexity of regional neural signals may serve as an index of the brain's capacity of information processing—increased complexity may indicate greater transition or exploration between different states of brain networks, thereby a greater propensity for information processing.

No abstract

No abstract

Hyperspectral imaging technique was employed to determine spatial distributions of chlorophyll (Chl), and carotenoid (Car) contents in cucumber leaves in response to angular leaf spot (ALS). Altogether, 196 hyperspectral images of cucumber leaves with five infection severities of ALS were captured by a hyperspectral imaging system in the range of 380–1,030 nm covering 512 wavebands. Mean spectrum were extracted from regions of interest (ROIs) in the hyperspectral images. Partial least square regression (PLSR) models were used to develop quantitative analysis between the spectra and the pigment contents measured by biochemical analyses. In addition, regression coefficients (RCs) in PLSR models were employed to select important wavelengths (IWs) for modelling. It was found that the PLSR models developed by the IWs provided the optimal measurement results with correlation coefficient (R) of prediction of 0.871 and 0.876 for Chl and Car contents, respectively. Finally, Chl and Car distributions in cucumber leaves with the ALS infection were mapped by applying the optimal models pixel-wise to the hyperspectral images. The results proved the feasibility of hyperspectral imaging for visualizing the pigment distributions in cucumber leaves in response to ALS.

We consider scenarios in which we wish to perform joint scene understanding, object tracking, activity recognition, and other tasks in environments in which multiple people are wearing body-worn cameras while a third-person static camera also captures the scene. To do this, we need to establish person-level correspondences across first-and thirdperson videos, which is challenging because the camera wearer is not visible from his/her own egocentric video, preventing the use of direct feature matching. In this paper, we propose a new semi-Siamese Convolutional Neural Network architecture to address this novel challenge. We formulate the problem as learning a joint embedding space for first-and third-person videos that considers both spatial-and motion-domain cues. A new triplet loss function is designed to minimize the distance between correct first-and third-person matches while maximizing the distance between incorrect ones. This end-to-end approach performs significantly better than several baselines, in part by learning the first-and third-person features optimized for matching jointly with the distance measure itself.

No abstract