Recurrent neural networks (RNNs) are known to be difficult to train due to the gradient vanishing and exploding problems and thus difficult to learn long-term patterns and construct deep networks. To address these problems, this paper proposes a new type of RNNs with the recurrent connection formulated as Hadamard product, referred to as independently recurrent neural network (IndRNN), where neurons in the same layer are independent of each other and connected across layers. The gradient vanishing and exploding problems are solved in IndRNN, and thus long-term dependencies can be learned. Moreover, an IndRNN can work with non-saturated activation functions such as ReLU (rectified linear unit) and be still trained robustly. Different deeper IndRNN architectures, including the basic stacked IndRNN, residual IndRNN and densely connected IndRNN, have been investigated, all of which can be much deeper than the existing RNNs. Furthermore, IndRNN reduces the computation at each time step and can be over 10 times faster than the commonly used Long short-term memory (LSTM). Experimental results have shown that the proposed IndRNN is able to process very long sequences and construct very deep networks. Better performances have been achieved on various tasks with IndRNNs compared with the traditional RNN and LSTM.unit (GRU) [15] have been proposed to address the gradient problems. However, the use of the hyperbolic tangent and the sigmoid functions as the activation function in these variants results in gradient decay over layers. Consequently, construction and training of a deep LSTM or GRU based RNN network is practically difficult.On the other hand, the existing RNN models share the same component σ(Wx t + Uh t−1 + b) in (1), where the recurrent connection connects all the neurons through time. This makes it hard to interpret and understand the roles of each individual neuron (e.g., what patterns each neuron responds to) without considering the others. Moreover, with the recurrent connections, matrix product is performed at each time step and the computation cannot be easily paralleled, leading to a very time-consuming process when dealing with long sequences.In this paper, we propose a new type of RNN, referred to as independently recurrent neural network (IndRNN). In the proposed IndRNN, the recurrent inputs are processed with the Hadamard product as h t = σ(Wx t +u h t−1 +b). This provides a number of advantages over the traditional RNNs including:• Able to process longer sequences: the gradient vanishing and exploding problem is solved by regulating the recurrent weights, and long-term memory can be kept in order to process long sequences. Experiments have demonstrated that an IndRNN can well process sequences over 5000 steps. • Able to construct deeper networks: multiple layers of IndRNNs can be efficiently stacked, especially with skip-connection and dense connection, to increase the depth of the network. An example of 21-layer residual IndRNN and deep densely connected In-dRNN are demonstrated in the experiments. ...
Scheimpflug photographs of the zero-diopter-accommodated anterior segments of 100 human subjects, aged 18 to 70 yr and evenly spaced over this range, were digitized and analyzed to characterize lens and lens nucleus shape as a function of age by the Hough transform and other image analysis methods. Anterior and posterior lens surface curves exhibit a decrease in radius of curvature with increasing age, in qualitative but not quantitative agreement with the earlier observations of Brown [Exp. Eye Res. 19, 175 (1974)]. In contrast, the shape of the lens nuclear boundaries changes little with age. Overall lens volume at zero diopters increases with age, but the volume of the lens nucleus remains unchanged. The lens center of mass moves anteriorly with increasing age, as does the central clear region of the lens. Although these data sets were found to be more variable than those of Brown, the complementary variability of other factors, such as anterior chamber depth, for each subject leads to a very high statistical correlation between lens shape and lens location relative to the cornea. This supports the finding of previous work that image formation on the retina for a given individual results from the multifactorial balancing of related factors.
Accommodation in the human eye occurs through controlled changes in crystalline lens shape, thickness, and refractive surface placement relative to the cornea. The changes in lens curvatures, whether surface or internal, have been characterized as a function of accommodation and subject age by use of quantitative analysis of Scheimpflug slit-lamp photographic images. Radii of curvature of the major lens refractive surfaces--the external and nuclear boundaries--decrease linearly with increasing accommodation in all eyes that are capable of accommodation. The rates at which they change with accommodation are age dependent, decreasing steadily toward zero with increased age. For the curves visible in each lens half, arising from boundaries between adjacent zones of discontinuity, radius of curvature and location are linearly related over the entire accommodative range for a given lens and over the age range for the population. The slope of this relationship changes with both accommodation and age, decreasing linearly in both cases. The relationship between these geometric changes and the loss of accommodative amplitude is discussed.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.