Recent work has shown how to train Convolutional Neural Networks (CNNs) rapidly on large image datasets [1], then transfer the knowledge gained from these models to a variety of tasks [2]. Following [3], in this work, we demonstrate similar scalability and transfer for Recurrent Neural Networks (RNNs) for Natural Language tasks.By utilizing mixed precision arithmetic and a 32k batch size distributed across 128 NVIDIA Tesla V100 GPUs, we are able to train a character-level 4096-dimension multiplicative LSTM (mLSTM) [4] for unsupervised text reconstruction over 3 epochs of the 40 GB Amazon Reviews dataset [5] in four hours. This runtime compares favorably with previous work taking one month to train the same size and configuration for one epoch over the same dataset [3].Converging large batch RNN models can be challenging. Recent work has suggested scaling the learning rate as a function of batch size, but we find that simply scaling the learning rate as a function of batch size leads either to significantly worse convergence or immediate divergence for this problem. We provide a learning rate schedule that allows our model to converge with a 32k batch size.Since our model converges over the Amazon Reviews dataset in hours, and our compute requirement of 128 Tesla V100 GPUs, while substantial, is commercially available, this work opens up large scale unsupervised NLP training to most commercial applications and deep learning researchers 1 . A model can be trained over most public or private text datasets overnight.
The problems of designing a hexapod control system - a mechanism with parallel kinematics, designed for guidance and positioning of instruments and antennas of orbiting satellite platforms are considered. Based on the solution of the extended kinematics problem, the algorithm for controlling linear drives with a kinematic pair of screw-nut and two two-axis hinges is specified. The hexapod control scheme with the spatial load position sensor is given, the feasibility of positional control algorithms is estimated on the basis of the modern domestic element base. The estimation is made by the method of mathematical modeling. An algorithm for adaptive neural network control of a hexapod is proposed. An artificial neural network has been developed, which together with a nonlinear controller regulates the force acting on linear actuators by control error. To assess the quality of hexapod control, a dynamic model of the hexapod control system was created in the simulation package SimMechanics of the MATLAB Simulink system. A description is given of the hardware part of the digital control system-the hexapod control unit).
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.