ReLU networks as surrogate models in mixed-integer linear programs

Grimstad, Bjarne; Andersson, Henrik

doi:10.1016/j.compchemeng.2019.106580

Cited by 109 publications

(71 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ReLU is a non-linear activation function. It is used in multi-layer neural networks or deep neural networks, the output of ReLU is the maximum value between zero and the input value, which effectively removes negative values from an activation map by setting them to zero [ 40 ].…”

Section: Materials and Methodologymentioning

confidence: 99%

Analyzing Malaria Disease Using Effective Deep Learning Approach

2020

View full text Add to dashboard Cite

Medical tools used to bolster decision-making by medical specialists who offer malaria treatment include image processing equipment and a computer-aided diagnostic system. Malaria images can be employed to identify and detect malaria using these methods, in order to monitor the symptoms of malaria patients, although there may be atypical cases that need more time for an assessment. This research used 7000 images of Xception, Inception-V3, ResNet-50, NasNetMobile, VGG-16 and AlexNet models for verification and analysis. These are prevalent models that classify the image precision and use a rotational method to improve the performance of validation and the training dataset with convolutional neural network models. Xception, using the state of the art activation function (Mish) and optimizer (Nadam), improved the effectiveness, as found by the outcomes of the convolutional neural model evaluation of these models for classifying the malaria disease from thin blood smear images. In terms of the performance, recall, accuracy, precision, and F1 measure, a combined score of 99.28% was achieved. Consequently, 10% of all non-dataset training and testing images were evaluated utilizing this pattern. Notable aspects for the improvement of a computer-aided diagnostic to produce an optimum malaria detection approach have been found, supported by a 98.86% accuracy level.

show abstract

Section: Materials and Methodologymentioning

confidence: 99%

Analyzing Malaria Disease Using Effective Deep Learning Approach

2020

View full text Add to dashboard Cite

show abstract

“…Modelling ReLU neural networks as MIPs is considered in the literature for other application domains, too. Grimstad and Andersson [15] investigate the usage of ReLU neural networks as surrogate models in MIPs and study various bound tightening techniques. Serra et al [28] apply a MIP formulation of a ReLU neural network to enable a lossless pruning method.…”

Section: Related Workmentioning

confidence: 99%

Advances in verification of ReLU neural networks

Rössig

Petković

2020

J Glob Optim

View full text Add to dashboard Cite

We consider the problem of verifying linear properties of neural networks. Despite their success in many classification and prediction tasks, neural networks may return unexpected results for certain inputs. This is highly problematic with respect to the application of neural networks for safety-critical tasks, e.g. in autonomous driving. We provide an overview of algorithmic approaches that aim to provide formal guarantees on the behaviour of neural networks. Moreover, we present new theoretical results with respect to the approximation of ReLU neural networks. On the other hand, we implement a solver for verification of ReLU neural networks which combines mixed integer programming with specialized branching and approximation techniques. To evaluate its performance, we conduct an extensive computational study. For that we use test instances based on the ACAS Xu system and the MNIST handwritten digit data set. The results indicate that our approach is very competitive with others, i.e. it outperforms the solvers of Bunel et al. (in: Bengio, Wallach, Larochelle, Grauman, Cesa-Bianchi, Garnett (eds) Advances in neural information processing systems (NIPS 2018), 2018) and Reluplex (Katz et al. in: Computer aided verification—29th international conference, CAV 2017, Heidelberg, Germany, July 24–28, 2017, Proceedings, 2017). In comparison to the solvers ReluVal (Wang et al. in: 27th USENIX security symposium (USENIX Security 18), USENIX Association, Baltimore, 2018a) and Neurify (Wang et al. in: 32nd Conference on neural information processing systems (NIPS), Montreal, 2018b), the number of necessary branchings is much smaller. Our solver is publicly available and able to solve the verification problem for instances which do not have independent bounds for each input neuron.

show abstract

“…Furthermore, over recent decades, novel modeling techniques have been developed which can substantially aid the optimization of process systems. For instance, surrogate models such as Kriging [39][40][41][42][43][44], radial basis functions [45][46][47][48][49][50], artificial neural networks [51][52][53][54][55][56], splines [57,58], among others were shown to accurately represent complex physical systems while aiding optimal search algorithms. No literature exists which explores the application of such techniques to advance the study of CHP dispatch.…”

Section: Optimal Combined Heat and Power Dispatchmentioning

confidence: 99%

“…Rectified neural networks are known to be continuous piecewise-linear functions with the universal approximation ability [99]. As a result, they are a subject of interest for capturing complex nonlinear physics in MILPs [51,100]. Neural networks are popular because of their ability to model large high-dimensional data-sets; however they also possess drawbacks when considering their application in modeling for use in an MILP.…”

Section: Combined Unit Commitment and Economic Dispatchmentioning

confidence: 99%

A Critical Review of the Modeling and Optimization of Combined Heat and Power Dispatch

Kazda¹,

Li²

2020

Processes

View full text Add to dashboard Cite

Combined heat and power (CHP) systems are attracting increasing attention for their ability to improve the economics and sustainability of the electricity system. Determining how to best operate these systems is difficult because they can consist of many generating units whose operation is governed by complex nonlinear physics. Mathematical programming is a useful tool to support the operation of CHP systems, and has been the subject of substantial research attention since the early 1990s. This paper critically reviews the modeling and optimization work that has been done on the CHP economic dispatch problem, and the CHP economic and emission dispatch problem. A summary of the common models used for these problems is provided, along with comments on future modeling work that would beneficial to the field. The majority of optimization approaches studied for CHP system operation are metaheuristic algorithms. A discussion of the limitations and benefits of metaheuristic algorithms is given. Finally, a case study optimizing five classic CHP system test instances demonstrates the advantages of the using deterministic global search algorithms over metaheuristic search algorithms.

show abstract

ReLU networks as surrogate models in mixed-integer linear programs

Cited by 109 publications

References 56 publications

Analyzing Malaria Disease Using Effective Deep Learning Approach

Analyzing Malaria Disease Using Effective Deep Learning Approach

Advances in verification of ReLU neural networks

A Critical Review of the Modeling and Optimization of Combined Heat and Power Dispatch

Contact Info

Product

Resources

About