Learning solutions to partial differential equations using LS-SVM

Mehrkanoon, Siamak; Suykens, Johan A. K.

doi:10.1016/j.neucom.2015.02.013

Cited by 58 publications

(42 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section compares the methodologies described in the previous sections on four problems given in references [12] and [27]. Problem #1 is a first order linear ODE, problem #2 is a first order nonlinear ODE, problem #3 is a second order linear ODE, and problem #4 is a second order linear PDE.…”

Section: Numerical Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

Leake

Johnston

Smith

et al. 2019

MAKE

View full text Add to dashboard Cite

Differential equations (DEs) are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. While there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Functional Connections (TFC) with one based on least-squares support vector machines (LS-SVM). The TFC method uses a constrained expression, an expression that always satisfies the DE constraints, which transforms the process of solving a DE into solving an unconstrained optimization problem that is ultimately solved via least-squares (LS). In addition to individual analysis, the two methods are merged into a new methodology, called constrained SVMs (CSVM), by incorporating the LS-SVM method into the TFC framework to solve unconstrained problems. Numerical tests are conducted on four sample problems: One first order linear ordinary differential equation (ODE), one first order nonlinear ODE, one second order linear ODE, and one two-dimensional linear partial differential equation (PDE). Using the LS-SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean squared error on the training and test sets. In general, TFC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) than the LS-SVM and CSVM approaches. IntroductionDifferential equations (DE) and their solutions are important topics in science and engineering. The solutions drive the design of predictive systems models and optimization tools. Currently, these equations are solved by a variety of existing approaches with the most popular based on the Runge-Kutta family [1]. Other methods include those which leverage low-order Taylor expansions, namely Gauss-Jackson [2] and Chebyshev-Picard iteration [3,4,5], which have proven to be highly effective. More recently developed techniques are based on spectral collocation methods [6]. This approach discretizes the domain about collocation points, and the solution of the DE is expressed by a sum of "basis" functions with unknown coefficients that are approximated in order to satisfy the DE as closely as possible. Yet, in order to incorporate boundary conditions, one or more equations must be added to enforce the constraints.The Theory of Functional Connections (TFC) is a new technique that analytically derives a constrained expression which satisfies the problem's constraints exactly while maintaining a function that can be freely chosen [7]. This theory, initially called "Theory of Connections", has been renamed for two reasons. First, the "Theory of Connections" already identifies a specific theory in differential geometry, and second, what this theory is actually doing is "functional interpolation", as it provides all functions satisfying a set of constraints in terms of a function ...

show abstract

Section: Numerical Resultsmentioning

confidence: 99%

“…The steps for solving linear PDEs using LS-SVM are the same as when solving linear ODEs, and are shown in detail in reference [27]. The first step is to write out the optimization problem to be solved.…”

Section: Linear Pdesmentioning

confidence: 99%

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

Leake

Johnston

Smith

et al. 2019

MAKE

View full text Add to dashboard Cite

show abstract

“…According to the Lagrange multipliers method [50], the optimization problem with constraints is transformed into the Lagrangian function which is composed of the LS-SVM cost function and constraints that the approximate solution y = ω T φ(x) + b satisfies the given first-order linear ordinary differential equation and the initial condition at the collocation points. The described methodology is applicable for solving other types of differential equations including second-order boundary value problems, partial differential equations, and descriptor systems [42][43][44].…”

Section: Least Squares Support Vector Machinesmentioning

confidence: 99%

“…Therefore, LS-SVM algorithms have various applications in the area of pattern recognition [38], fault diagnosis [39], and time-series prediction [40,41]. In addition, LS-SVM algorithms have been successfully applied for solving differential equations [42,43], differential algebraic equations [44,45], and integral equations [46].…”

Section: Introductionmentioning

confidence: 99%

The LS-SVM algorithms for boundary value problems of high-order ordinary differential equations

Yin

et al. 2019

Adv Differ Equ

View full text Add to dashboard Cite

This paper introduces the improved LS-SVM algorithms for solving two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations. To demonstrate the reliability and powerfulness of the improved LS-SVM algorithms, some numerical experiments for third-order, fourth-order linear and nonlinear ordinary differential equations with two-point and multi-point boundary conditions are performed. The idea can be extended to other complicated ordinary differential equations.

show abstract

“…It has been successfully used in many real world pattern recognition problems, such as disease diagnosis [2], fault detection [3], image classification [4], partial differential equations solving [5] and visual tracking [6]. LSSVM tries to minimize least squares errors on the training samples.…”

Section: Introductionmentioning

confidence: 99%

Sparse algorithm for robust LSSVM in primal space

Chen

Zhou

2018

Neurocomputing

View full text Add to dashboard Cite

As enjoying the closed form solution, least squares support vector machine (LSSVM) has been widely used for classification and regression problems having the comparable performance with other types of SVMs. However, LSSVM has two drawbacks: sensitive to outliers and lacking sparseness. Robust LSSVM (R-LSSVM) overcomes the first partly via nonconvex truncated loss function, but the current algorithms for R-LSSVM with the dense solution are faced with the second drawback and are inefficient for training large-scale problems. In this paper, we interpret the robustness of R-LSSVM from a re-weighted viewpoint and give a primal R-LSSVM by the representer theorem.The new model may have sparse solution if the corresponding kernel matrix has low rank. Then approximating the kernel matrix by a low-rank matrix and smoothing the loss function by entropy penalty function, we propose a convergent sparse R-LSSVM (SR-LSSVM) algorithm to achieve the sparse solution of primal R-LSSVM, which overcomes two drawbacks of LSSVM simultaneously. The proposed algorithm has lower complexity than the existing algorithms and is very efficient for training large-scale problems. Many experimental results illustrate that SR-LSSVM can achieve better or comparable performance with less training time than related algorithms, especially for training large scale problems.

show abstract

Learning solutions to partial differential equations using LS-SVM

Cited by 58 publications

References 26 publications

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

The LS-SVM algorithms for boundary value problems of high-order ordinary differential equations

Sparse algorithm for robust LSSVM in primal space

Contact Info

Product

Resources

About