Methods to find the optimization solution are fundamental and extremely crucial for scientists to program computational software to solve optimization problems efficiently and for practitioners to use it efficiently. Thus, it is very essential to know about the idea, origin, and usage of these methods. Although the methods have been used for very long time and the theory has been developed too long, most, if not all, of the authors who develop the theory are unknown and the theory has not been stated clearly and systematically. To bridge the gap in the literature in this area and provide academics and practitioners with an overview of the methods, this paper reviews and discusses the four most commonly used methods to find the optimization solution including the bisection, gradient, Newton-Raphson, and secant methods. We first introduce the origin and idea of the methods and develop all the necessary theorems to prove the existence and convergence of the estimate for each method.We then give two examples to illustrate the approaches. Thereafter, we review the literature of the applications of getting the optimization solutions in some important issues and discuss the advantages and disadvantages of each method. We note that all the theorems developed in our paper could be well-known but, so far, we have not seen any book or paper that discusses all the theorems stated in our paper in detail. Thus, we believe the theorems developed in our paper could still have some contributions to the literature. Our review is useful for academics and practitioners in finding the optimization solutions in their studies.

No abstract