No abstract

No abstract

We explicitly analyze the trajectories of learning near singularities in hierarchical networks, such as multilayer perceptrons and radial basis function networks, which include permutation symmetry of hidden nodes, and show their general properties. Such symmetry induces singularities in their parameter space, where the Fisher information matrix degenerates and odd learning behaviors, especially the existence of plateaus in gradient descent learning, arise due to the geometric structure of singularity. We plot dynamic vector fields to demonstrate the universal trajectories of learning near singularities. The singularity induces two types of plateaus, the on-singularity plateau and the near-singularity plateau, depending on the stability of the singularity and the initial parameters of learning. The results presented in this letter are universally applicable to a wide class of hierarchical models. Detailed stability analysis of the dynamics of learning in radial basis function networks and multilayer perceptrons will be presented in separate work.

No abstract

No abstract

No abstract

No abstract

A nonlinear method has been developed to estimate climate feedbacks based on the Neural Network (NN) taking advantage of its self‐learning skills. The NN model developed here is trained using a reanalysis data set and predicts radiation flux globally from atmospheric and surface variables. The radiative feedbacks of temperature, water vapor, surface albedo, and cloud in the interannual climate variations estimated from the NN method are in agreement with those from a broadly used kernel method. However, the NN method demonstrates significant advantages: (1) it withdraws the linearity assumption of the kernel method and accounts for the nonlinear effects of the feedbacks. In the case of large climate perturbations, such as that in the Arctic caused by sea ice melt, the NN method achieves better radiation closure. (2) The method can directly calculate the radiative feedback of cloud and its components. We find that the high, middle, and low cloud feedback components analyzed from the NN method are linearly additive in the interannual climate variations, although there is a considerable nonlinear effect arising from the interactions between cloud and noncloud variables.