We compare a first-order stochastic swarm intelligence model called consensus-based optimization (CBO), which may be used for the global optimization of a function in multiple dimensions, to other particle swarm algorithms for global optimization. CBO allows for passage to the mean-field limit resulting in a nonlocal, degenerate, parabolic PDE. Exploiting tools from PDE analysis, it is possible to rigorously prove convergence results for the algorithm (see [3]). In the present article we discuss numerical results obtained with the Particle Swarm Optimization (PSO) [4], Wind-Driven Optimization (WDO) [6] and CBO and show that CBO leads to very competitive results.
This work presents neural network based minimal entropy closures for the moment system of the Boltzmann equation, that preserve the inherent structure of the system of partial differential equations, such as entropy dissipation and hyperbolicity. The described method embeds convexity of the moment to entropy map in the neural network approximation to preserve the structure of the minimal entropy closure. Two techniques are used to implement the methods. The first approach approximates the map between moments and the minimal entropy of the moment system and is convex by design. The second approach approximates the map between moments and Lagrange multipliers of the dual of the minimal entropy optimization problem, which present the gradients of the entropy with respect to the moments, and is enforced to be monotonic by introduction of a penalty function. We derive an error bound for the generalization gap of convex neural networks which are trained in Sobolev norm and use the results to construct data sampling methods for neural network training. Numerical experiments are conducted, which show that neural network-based entropy closures provide a significant speedup for kinetic solvers while maintaining a sufficient level of accuracy. The code for the described implementations can be found in the Github repositories [1,2].
Direct simulation of physical processes on a kinetic level is prohibitively expensive in aerospace applications due to the extremely high dimension of the solution spaces. In this paper, we consider the moment system of the Boltzmann equation, which projects the kinetic physics onto the hydrodynamic scale. The unclosed moment system can be solved in conjunction with the entropy closure strategy. Using an entropy closure provides structural benefits to the physical system of partial differential equations. Usually computing such closure of the system spends the majority of the total computational cost, since one needs to solve an ill-conditioned constrained optimization problem. Therefore, we build a neural network surrogate model to close the moment system, which preserves the structural properties of the system by design, but reduces the computational cost significantly. Numerical experiments are conducted to illustrate the performance of the current method in comparison to the traditional closure.
Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. In this work, we propose a novel algorithm to find efficient low-rank subnetworks. Remarkably, these subnetworks are determined and adapted already during the training phase and the overall time and memory resources required by both training and evaluating them is significantly reduced. The main idea is to restrict the weight matrices to a low-rank manifold and to update the low-rank factors rather than the full matrix during training. To derive training updates that are restricted to the prescribed manifold, we employ techniques from dynamic model order reduction for matrix differential equations. Moreover, our method automatically and dynamically adapts the ranks during training to achieve a desired approximation accuracy. The efficiency of the proposed method is demonstrated through a variety of numerical experiments on fully-connected and convolutional networks.
Direct simulation of physical processes on a kinetic level is prohibitively expensive in aerospace applications due to the extremely high dimension of the solution spaces. In this paper, we consider the moment system of the Boltzmann equation, which projects the kinetic physics onto the hydrodynamic scale. The unclosed moment system can be solved in conjunction with the entropy closure strategy. Using an entropy closure provides structural benefits to the physical system of partial differential equations. Usually computing such closure of the system spends the majority of the total computational cost, since one needs to solve an ill-conditioned constrained optimization problem. Therefore, we build a neural network surrogate model to close the moment system, which preserves the structural properties of the system by design, but reduces the computational cost significantly. Numerical experiments are conducted to illustrate the performance of the current method in comparison to the traditional closure.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.