Efficient training of interval Neural Networks for imprecise training data

Sadeghi, Jonathan; Angelis, Marco de; Patelli, Edoardo

doi:10.1016/j.neunet.2019.07.005

Cited by 24 publications

(15 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The number of terms in (43) increases with each iteration because the updating rules for momentum include the functions S and D from previous iterations unlike the standard gradient descent. However, expression (43) depends on the previously computed ∂W/∂Y and ∂b/∂Y and has same neighboring derivatives as in formula (22) which should be computed once. In addition, we use the relationships between the derivatives…”

Section: Gradient Descent With Momentummentioning

confidence: 99%

“…The main feature of these proposed methods is that they exploit separate expressions for the lower and upper limits of the interval outputs in the learning algorithm. In [43] Sadeghi et al presented an alternative way to efficiently train interval neural networks for imprecise data using the framework of interval predictor models [44,45]. Unlike standard regression techniques, these models try to bound an envelope of the data using a conceptualization of fit different from the traditional least squares criterion, which produce interval estimates for the regression even from point-valued input data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural network model for imprecise regression with interval dependent variables

Tretiak¹,

Schollmeyer²,

Ferson³

2022

Preprint

View full text Add to dashboard Cite

We propose a new iterative method using machine learning algorithms to fit an imprecise regression model to data that consist of intervals rather than point values. The method is based on a single-layer interval neural network which can be trained to produce an interval prediction. It seeks parameters for the optimal model that minimize the mean squared error between the actual and predicted interval values of the dependent variable using a first-order gradient-based optimization and interval analysis computations to model the measurement imprecision of the data. The method captures the relationship between the explanatory variables and a dependent variable by fitting an imprecise regression model, which is linear with respect to unknown interval parameters even the regression model is nonlinear. We consider the explanatory variables to be precise point values, but the measured dependent values are characterized by interval bounds without any probabilistic information. Thus, the imprecision is modeled non-probabilistically even while the scatter of dependent values is modeled probabilistically by homoscedastic Gaussian distributions. The proposed iterative method estimates the lower and upper bounds of the expectation region, which is an envelope of all possible precise regression lines obtained by ordinary regression analysis based on any configuration of real-valued points from the respective intervals and their x-values.

show abstract

Section: Gradient Descent With Momentummentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Neural network model for imprecise regression with interval dependent variables

Tretiak¹,

Schollmeyer²,

Ferson³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It is particularly popular when undertaking risk or reliability analyses when data is not perfectly known [24,6,5]. PBA objects and methods can also be used within machine learning techniques [15,25,26].…”

Section: Introductionmentioning

confidence: 99%

Probability bounds analysis for Python

et al. 2022

Self Cite

View full text Add to dashboard Cite

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

show abstract

“…Quantile regression itself is a well-studied field with the first results from [22], see also [23,46]. Quantile regression in deep learning has been also recently considered as a general statistical modeling tool [38,53,40,50,45]. Bayesian quantile regression has also been studied [26,51].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Tubes for Tube MPC

Fan¹,

Agha–mohammadi²,

Theodorou³

2020

Preprint

View full text Add to dashboard Cite

Learning-based control aims to construct models of a system to use for planning or trajectory optimization, e.g. in model-based reinforcement learning. In order to obtain guarantees of safety in this context, uncertainty must be accurately quantified. This uncertainty may come from errors in learning (due to a lack of data, for example), or may be inherent to the system. Propagating uncertainty in learned dynamics models is a difficult problem. Common approaches rely on restrictive assumptions of how distributions are parameterized or propagated in time. In contrast, in this work we propose using deep learning to obtain expressive and flexible models of how these distributions behave, which we then use for nonlinear Model Predictive Control (MPC). First, we introduce a deep quantile regression framework for control which enforces probabilistic quantile bounds and quantifies epistemic uncertainty. Next, using our method we explore three different approaches for learning tubes which contain the possible trajectories of the system, and demonstrate how to use each of them in a Tube MPC scheme. Furthermore, we prove these schemes are recursively feasible and satisfy constraints with a desired margin of probability. Finally, we present experiments in simulation on a nonlinear quadrotor system, demonstrating the practical efficacy of these ideas.

show abstract

Efficient training of interval Neural Networks for imprecise training data

Cited by 24 publications

References 29 publications

Neural network model for imprecise regression with interval dependent variables

Neural network model for imprecise regression with interval dependent variables

Probability bounds analysis for Python

Deep Learning Tubes for Tube MPC

Contact Info

Product

Resources

About