Representation Learning and Nature Encoded Fusion for Heterogeneous Sensor Networks

Abstract

Target detection based on heterogeneous sensor networks is considered in this paper. Fusion problem is investigated to fully take advantage of the information of multi-modal data. The sensing data may not be compatible with each other due to heterogeneous sensing modalities, and the joint PDF of the sensors is not easily available. A two-stage fusion method is proposed to solve the heterogeneous data fusion problem. First, the multi-modality data is transformed into the same representation form by a certain linear or nonlinear transformation. Since there is a model mismatch among the different modalities, each modality is trained by an individual statistical model. In this way, the information of different modalities is preserved. Then, the representation is used as the input of the probabilistic fusion. The probabilistic framework allows data from different modalities to be processed in a unified information fusion space. The inherent inter-sensor relationship is exploited to encode the original sensor data on a graph. Iterative belief propagation is used to fuse the local sensing belief. The more general correlation case is also considered, in which the relation between two sensors is characterized by the correlation factor. The numerical results are provided to validate the effectiveness of the proposed method in heterogeneous sensor network fusion. INDEX TERMS Heterogeneous sensor networks, multi-modal data fusion, representation learning, nature encoded fusion, belief propagation.