Abstract:Recurrent neural networks (RNNs) are known to be difficult to train due to the gradient vanishing and exploding problems and thus difficult to learn long-term patterns and construct deep networks. To address these problems, this paper proposes a new type of RNNs with the recurrent connection formulated as Hadamard product, referred to as independently recurrent neural network (IndRNN), where neurons in the same layer are independent of each other and connected across layers. The gradient vanishing and explodin… Show more
“…We aim to obtain z t+1 , which can generate the flow information at the next time step, by advancing z t in a low dimension. Here, IndyLSTM [34,35], a kind of recurrent neural networks (RNNs), is used to update z t to z t+1 . A similar approach that applies RNNs to the time advancement in a low dimension was used for the prediction of unsteady flows [36,37,14].…”
Section: Rnn-gan For Time-varying Flow Generationmentioning
A B S T R A C TA realistic inflow boundary condition is essential for successful simulation of the developing turbulent boundary layer or channel flows. Recent advances in artificial intelligence (AI) have enabled the development of an inflow generator that performs better than the synthetic methods based on intuitions. In the present work, we applied generative adversarial networks (GANs), a representative of unsupervised learning, to generate an inlet boundary condition of turbulent channel flow. Upon learning the two-dimensional spatial structure of turbulence using data obtained from direct numerical simulation (DNS) of turbulent channel flow, the GAN could generate instantaneous flow fields that are statistically similar to those of DNS. Surprisingly, the GAN could produce fields at various Reynolds numbers without any additional simulation based on the trained data of only three Reynolds numbers. This indicates that the GAN could learn the universal nature of Reynolds number effect and might reflect other simulation conditions. Eventually, through a combination of the GAN and a recurrent neural network (RNN), we developed a novel model (RNN-GAN) that could generate time-varying fully developed flow for a long time. The spatiotemporal correlations of the generated flow are in good agreement with those of the DNS. This proves the usefulness of unsupervised learning in the generation of synthetic turbulence fields.
“…We aim to obtain z t+1 , which can generate the flow information at the next time step, by advancing z t in a low dimension. Here, IndyLSTM [34,35], a kind of recurrent neural networks (RNNs), is used to update z t to z t+1 . A similar approach that applies RNNs to the time advancement in a low dimension was used for the prediction of unsteady flows [36,37,14].…”
Section: Rnn-gan For Time-varying Flow Generationmentioning
A B S T R A C TA realistic inflow boundary condition is essential for successful simulation of the developing turbulent boundary layer or channel flows. Recent advances in artificial intelligence (AI) have enabled the development of an inflow generator that performs better than the synthetic methods based on intuitions. In the present work, we applied generative adversarial networks (GANs), a representative of unsupervised learning, to generate an inlet boundary condition of turbulent channel flow. Upon learning the two-dimensional spatial structure of turbulence using data obtained from direct numerical simulation (DNS) of turbulent channel flow, the GAN could generate instantaneous flow fields that are statistically similar to those of DNS. Surprisingly, the GAN could produce fields at various Reynolds numbers without any additional simulation based on the trained data of only three Reynolds numbers. This indicates that the GAN could learn the universal nature of Reynolds number effect and might reflect other simulation conditions. Eventually, through a combination of the GAN and a recurrent neural network (RNN), we developed a novel model (RNN-GAN) that could generate time-varying fully developed flow for a long time. The spatiotemporal correlations of the generated flow are in good agreement with those of the DNS. This proves the usefulness of unsupervised learning in the generation of synthetic turbulence fields.
“…For protein representation, we have chosen SSE as the resolution for interpretability due to the known sequence-size limitation of RNN models (Li et al, 2018). One can easily increase the resolution to residuelevel by simply feeding to our models amino-acid sequences (preferentially of length below 1,000) instead of SPS sequences, but needs to be aware of the much increased computational burden and much worse convergence when training RNNs.…”
Section: Resultsmentioning
confidence: 99%
“…All these are achieved with a much smaller alphabet of size 76 , which leads to around 100-times more compact representation of a protein sequence than the baseline. In addition, the SPS sequences are much shorter than aminoacid sequences and prevents convergence issues when training RNN and LSTM for sequences longer than 1,000 (Li et al, 2018).…”
Motivation: Drug discovery demands rapid quantification of compound-protein interaction (CPI). However, there is a lack of methods that can predict compound-protein affinity from sequences alone with high applicability, accuracy, and interpretability. Results: We present a seamless integration of domain knowledges and learning-based approaches. Under novel representations of structurally-annotated protein sequences, a semi-supervised deep learning model that unifies recurrent and convolutional neural networks has been proposed to exploit both unlabeled and labeled data, for jointly encoding molecular representations and predicting affinities. Our representations and models outperform conventional options in achieving relative error in IC50 within 5-fold for test cases and 20-fold for protein classes not included for training. Performances for new protein classes with few labeled data are further improved by transfer learning. Furthermore, separate and joint attention mechanisms are developed and embedded to our model to add to its interpretability, as illustrated in case studies for predicting and explaining selective drug-target interactions. Lastly, alternative representations using protein sequences or compound graphs and a unified RNN/GCNN-CNN model using graph CNN (GCNN) are also explored to reveal algorithmic challenges ahead. Availability: Data and source codes are available at https://github.com/Shen-Lab/DeepAffinity
“…However, this naïve method may suffer from long-term memory loss. Li et al [27] showed that LSTM models could only memorize less than 1,000 steps. Our experiments shown in Sec.…”
Section: Pooling On Featurementioning
confidence: 99%
“…4) were calculated on 1.7M mixtures synthesized with the setup described in Sec.3.3. Memory retaining Li et al [27] showed that LSTM can only keep a mid-range memory (about 500-1,000 time steps). To check if LSTM models have a similar memory forgetting issue on AEC, we can look at the red curves of 'LastFrame' in Fig.…”
Acoustic event classification (AEC) and acoustic event detection (AED) refer to the task of detecting whether specific target events occur in audios. As long short-term memory (LSTM) leads to stateof-the-art results in various speech related tasks, it is employed as a popular solution for AEC as well. This paper focuses on investigating the dynamics of LSTM model on AEC tasks. It includes a detailed analysis on LSTM memory retaining, and a benchmarking of nine different pooling methods on LSTM models using 1.7M generated mixture clips of multiple events with different signal-tonoise ratios. This paper focuses on understanding: 1) utterance-level classification accuracy; 2) sensitivity to event position within an utterance. The analysis is done on the dataset for the detection of rare sound events from DCASE 2017 Challenge. We find max pooling on the prediction level to perform the best among the nine pooling approaches in terms of classification accuracy and insensitivity to event position within an utterance. To authors' best knowledge, this is the first kind of such work focused on LSTM dynamics for AEC tasks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.