No abstract

Activation functions facilitate deep neural networks by introducing non-linearity to the learning process. The non-linearity feature gives the neural network the ability to learn complex patterns. Recently, the most widely used activation function is the Rectified Linear Unit (ReLU). Though, other various existing activation including hand-designed alternatives to ReLU have been proposed. However, none has succeeded in replacing ReLU due to their existing inconsistencies. In this work, activation function called ReLU-Memristor-like Activation Function (RMAF) is proposed to leverage benefits of negative values in neural networks. RMAF introduces a constant parameter (α) and a threshold parameter (p) making the function smooth, non-monotonous, and introduces non-linearity in the network. Our experiments show that, the RMAF works better than ReLU and other activation functions on deeper models and across number of challenging datasets. Firstly, experiments are performed by training and classifying on multi-layer perceptron (MLP) over benchmark data such as the Wisconsin breast cancer, MNIST, Iris and Car evaluation. RMAF achieves high performance of 98.74%, 99.67%, 98.81% and 99.42% respectively, compared to Sigmoid, Tanh and ReLU. Secondly, experiments were performed on convolution neural network (ResNet) over MNIST, CIFAR-10 and CIFAR-100 data and observed the proposed activation function achieves higher performance accuracy of 99.73%, 98.77% and 79.82% respectively than Tanh, ReLU and Swish. Additionally, we experimented our work on deep networks i.e. squeeze network (SqueezeNet), Dense connected neural network (DenseNet121) and ImageNet dataset, which RMAF produced the best performance. We note that, the RMAF converges faster than the other functions and can replace ReLU in any neural network due to the efficiency, scalability and its similarity to both ReLU and Swish.

Affine matrix rank minimization problem is a fundamental problem with a lot of important applications in many fields. It is well known that this problem is combinatorial and NP-hard in general. In this paper, a continuous promoting low rank non-convex fraction function is studied to replace the rank function in this NP-hard problem. Inspired by our former work in compressed sensing, an iterative singular value thresholding algorithm is proposed to solve the regularization transformed affine matrix rank minimization problem. For different a > 0, we could get a much better result by adjusting the different value of a, which is one of the advantages for the iterative singular value thresholding algorithm compared with some state-of-art methods. Some convergence results are established and numerical experiments show that this thresholding algorithm is feasible for solving the regularization transformed affine matrix rank minimization problem. Moreover, we proved that the value of the regularization parameter λ > 0 can not be chosen too large. Indeed, there exists λ > 0 such that the optimal solution of the regularization transformed affine matrix rank minimization problem is equal to zero for any λ >λ. Numerical experiments on matrix completion problems show that our method performs powerful in finding a low-rank matrix and the numerical experiments about image inpainting problems show that our algorithm has better performances than some state-of-art methods.

No abstract

No abstract

No abstract

Using a two-dimensional (2D) correlation analysis technique to determine the sequential order of physical or chemical events has received keen interests in the past ten years. However, our continuous work demonstrates that the sequential order of events determined by the "sequential order" rules of this technique may lead to ambiguous or even wrong conclusions, because the physical significance of the sequential order in generalized 2D correlation analysis is neither well-defined nor meaningful in general situations, and the word "occur" used in the "sequential order" rules may easily give rise to ambiguity. In contrast to the integrated sequential order derived from periodic changes as in mechanical perturbation based 2D correlation infrared spectroscopy, there is a local/chronological sequential order for nonperiodic changes in general situations. The current work shows that the integrated sequential order in 2D correlation analysis is a reflection of the sequential order of the phases, i.e., phase sequence/difference. The integrated sequential order may indicate the relative state of two events (one event occurs/exists before or after the other one) according to a specific reference, only if both are obtained under the same frequency for periodic changes or even speeds for nonperiodic changes in general situations. The integrated sequential order may not always be able to reveal whether one event occurs/happens before or after another one for nonperiodic changes in terms of timings of happenings. For nonperiodic changes, the integrated sequential order is not so meaningful and must be replaced by the local/chronological sequential order. To judge whether one event occurs/happens before or after another one for two nonperiodic changes in general situations, the original spectral intensity changes must be verified to determine if a chronological/local sequential order exists between two events.

Recent years have witnessed tremendous progress of intelligent robots brought about by mimicking human intelligence. However, current robots are still far from being able to handle multiple tasks in a dynamic environment as efficiently as humans. To cope with complexity and variability, further progress toward scalability and adaptability are essential for intelligent robots. Here, we report a brain-inspired robotic platform implemented by an unmanned bicycle that exhibits scalability of network scale, quantity and diversity to handle the changing needs of different scenarios. The platform adopts rich coding schemes and a trainable and scalable neural state machine, enabling flexible cooperation of hybrid networks. In addition, an embedded system is developed using a cross-paradigm neuromorphic chip to facilitate the implementation of diverse neural networks in spike or non-spike form. The platform achieved various real-time tasks concurrently in different real-world scenarios, providing a new pathway to enhance robots’ intelligence.