Despite the growing interest in the interplay of machine learning and optimization, existing contributions remain scattered across the research board, and a comprehensive overview on such reciprocity still lacks at this stage. In this context, this paper visits one particular direction of interplay between learningdriven solutions and optimization, and further explicates the subject matter with a clear background and summarized theory. For instance, machine learning and its offsprings are trending because of their enhanced capabilities in automating analytical modeling. In this realm, learning-based techniques (supervised, unsupervised, and reinforcement) have grown to complement many of the optimization problems in testing and training. This paper overviews how machine learning-based techniques, namely deep neural networks, echo-state networks, reinforcement learning, and federated learning, can be used to solve complex and analytically intractable optimization problems, for which specific cases are examined in this paper. The paper particularly overviews when learning-based algorithms are useful at solving particular optimizing problems, especially those of random, dynamic, and mathematically complex nature. The paper then illustrates such applications by presenting particular use-cases in communications and signal processing including wireless scheduling, wireless offloading and resource management, power control, aerial base station placement, virtual reality, and vehicular networks. Lastly, the paper sheds light on some future research directions, where the dynamicity and randomness of the underlying optimization problems make deep learning-driven techniques a necessity, namely in sensing at the terahertz (THz) bands, cellular vehicleto-everything, 6G communication networks, underwater optical networks, distributed optimization, and applications of emerging learning-based techniques.
