Constructive Approximation of Discontinuous Functions by Neural Networks

Llanas, B.; Lantarón, Sagrario; Sáinz, Francisco Javier

doi:10.1007/s11063-007-9070-9

Cited by 46 publications

(24 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that results for continuous functions indicate that functions that have better smoothness properties can have lower approximation error for a particular number of neurons [32,10,33]. For discontinuous functions, Llanas et al [24] showed that a real and piecewise continuous function can be approximated in an almost uniform way. However, Llanas et al [24] also noted that piecewise continuous functions when trained with gradient descent methods require many neurons and training iterations, and yet do not give very good results.…”

Section: Related Workmentioning

confidence: 78%

“…For discontinuous functions, Llanas et al [24] showed that a real and piecewise continuous function can be approximated in an almost uniform way. However, Llanas et al [24] also noted that piecewise continuous functions when trained with gradient descent methods require many neurons and training iterations, and yet do not give very good results. These results suggest that continuous rotation representations might perform better in practice.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On the Continuity of Rotation Representations in Neural Networks

Zhou

Barnes

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

763

534

View full text Add to dashboard Cite

In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.

show abstract

Section: Related Workmentioning

confidence: 78%

Section: Related Workmentioning

confidence: 99%

On the Continuity of Rotation Representations in Neural Networks

Zhou

Barnes

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

763

534

View full text Add to dashboard Cite

show abstract

“…Recent studies proved that the universal approximation property of a multi-layer feedforward neural network holds for unbounded activation function such as the rectifier function we used in this paper [41]. Studies also proved that a neural network can approximate a discontinuous function [42], and in practice the distinction between continuous and discontinuous functions is not an important factor for the approximation capacity of a neural network [43]. The approximation theorem states that, for any > 0, we are able to construct a multi-layer feed-forward neural network F such that:…”

Section: B Solution Overviewmentioning

confidence: 77%

dTrust: A Simple Deep Learning Approach for Social Recommendation

Dang

Ignat

2017

2017 IEEE 3rd International Conference on Collaboration and Internet Computing (CIC)

View full text Add to dashboard Cite

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. dTrust: a simple deep learning approach for social recommendationQuang-Vinh Dang, Claudia-Lavinia IgnatTo cite this version: Abstract-Rating prediction is a key task of e-commerce recommendation mechanisms. Recent studies in social recommendation enhance the performance of rating predictors by taking advantage of user relationships. However, these prediction approaches mostly rely on user personal information which is a privacy threat.In this paper, we present dTrust, a simple social recommendation approach that avoids using user personal information. It relies uniquely on the topology of an anonymized trust-useritem network that combines user trust relations with user rating scores. This topology is fed into a deep feed-forward neural network. Experiments on real-world data sets showed that dTrust outperforms state-of-the-art in terms of Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) scores for both warmstart and cold-start problems.

show abstract

“…However, most of the works prescribed need the assumption that the function to be approximated by neural network is continuous. In this work, with resorting to the techniques described in [29,23], we extend the discussion to approximation and compensation for piecewise discontinuous function within the proposed control framework. Another challenge of control of hydraulic actuated exoskeleton is how to compensate for the input saturation caused by the hydraulic actuator, which in principle might lead to poor tracking performance, including longer period of transient, unacceptable overshoot, even larger tracking error.…”

Section: Introductionmentioning

confidence: 99%

A hierarchical Lyapunov-based cascade adaptive control scheme for lower-limb exoskeleton

Zhang

Jiang

Li³

et al. 2019

European Journal of Control

View full text Add to dashboard Cite

This paper proposes a hierarchical Lyapunov-based adaptive cascade control scheme for a lower-limb exoskeleton with control saturation. The proposed approach is composed by two control levels with cascade structure. At the higher layer of the structure, a Lyapunov-based back-stepping regulator including adaptive estimation of uncertain parameters and friction force is designed for the leg dynamics, to minimize the deviation of the joint position and its reference value. At the lower layer, a Lyapunov-based neural network adaptive controller is in charge of computing control action for the hydraulic servo system, to follow the force reference computed at the high level, also to compensate for model uncertainty, nonlinearity, and control saturation. The proposed approach shows to be capable in minimizing the interaction torque between machine and human, and suitable for possible imprecise models. The robustness of the closed-loop system is discussed under input constraint. Simulation experiments are reported, which shows that the proposed scheme is effective in imposing smaller interaction torque with respect to PD controller, and in control of models with uncertainty and nonlinearity.

show abstract

Constructive Approximation of Discontinuous Functions by Neural Networks

Cited by 46 publications

References 32 publications

On the Continuity of Rotation Representations in Neural Networks

On the Continuity of Rotation Representations in Neural Networks

dTrust: A Simple Deep Learning Approach for Social Recommendation

A hierarchical Lyapunov-based cascade adaptive control scheme for lower-limb exoskeleton

Contact Info

Product

Resources

About