How Meta-heuristic Algorithms Contribute to Deep Learning in the Hype of Big Data Analytics

Abstract

Final accepted version (with author's formatting)This version is available at: http://eprints.mdx.ac.uk/22276/ Copyright:Middlesex University Research Repository makes the University's research available electronically.Copyright and moral rights to this work are retained by the author and/or other copyright owners unless otherwise stated. The work is supplied on the understanding that any use for commercial gain is strictly forbidden. A copy may be downloaded for personal, non-commercial, research or study without prior permission and without charge.Works, including theses and research projects, may not be reproduced in any format or medium, or extensive quotations taken from them, or their content changed in any way, without first obtaining permission in writing from the copyright holder(s). They may not be sold or exploited commercially in any format or medium without the prior written permission of the copyright holder(s).Full bibliographic details must be given when referring to, or quoting from full items including the author's name, the title of the work, publication details where relevant (place, publisher, date), pagination, and for theses or dissertations the awarding institution, the degree type awarded, and the date of the award.If you believe that any material held in the repository infringes copyright law, please contact the Repository Team at Middlesex University via the following email address:eprints@mdx.ac.ukThe item will be removed from the repository while any claim is being investigated. ccfong@umac.mo, suashdeb@gmail.com, x.yang@mdx.ac.uk Abstract. Deep learning (DL) is one of the most emerging type of contemporary machine learning techniques that mimic the cognitive patterns of animal visual cortex to learn the new abstract features automatically by deep and hierarchical layers. DL is believed to be a suitable tool so far for extracting insights from very huge volume of so-called big data. Nevertheless, one of the three "V" or big data is velocity that implies the learning has to be incremental as data are accumulating up rapidly. DL must be fast and accurate. By the technical design of DL, it is extended from feed-forward artificial neural network with many multi-hidden layers of neurons called deep neural network (DNN). In the training process of DNN, it has certain inefficiency due to very long training time required. Obtaining the most accurate DNN within a reasonable run-time is a challenge, given there are potentially many parameters in the DNN model configuration, and high-dimensionality of the feature space in the training dataset. Meta-heuristic has a history of optimizing machine learning models successfully. How well meta-heuristic could be used to optimize DL in the context of big data analytics is a thematic topic which we pondered on in this paper. As a position paper, we review the recent advances of applying metaheuristics on DL, discuss about their pros and cons, and point out some feasible research directions for bridging the gaps between meta-heuristics and DL.Keyword...

A review on the self and dual interactions between machine learning and optimisation

“…For example, the approach in [59] encoded the kernel parameter and regularisation parameter of a SVM into a candidate solution (chromosome) of an EA, and a generalisation performance measure was chosen as the fitness function. Other than SVM, several previous studies provided an overview of the approaches that used metaheuristics to tune the hyper-parameters of NNs [56,125,193].…”

“…Even though these methods are usually more time consuming, they may achieve better performance. For example, in [56,125,193], several overviews of approaches that optimise the weights of NNs based on various metaheuristics were provided. For some ML algorithms, some of the model parameters are discrete, and therefore gradient-based optimisation algorithms do not work on training these models, instead metaheuristics can be used.…”

“…In the past few years, a number of review papers have been published about the dual interactions [18,33,39], individual interactions [5,24,84,97,177] and the interactions that focused on a specific ML or optimisation algorithm [16,56,58,79,117,118,125,189,193,197] between ML and optimisation. Differently, our study provides a global overview of the interaction between ML and optimisation as well as those within themselves as shown in Fig.…”

Location-Aware Energy-Efficient Workload Allocation in Geo Distributed Cloud Environment

“…However, in a Large set experiment, the SLA violation using LAEE algorithm is less than 0.5% compared to actual time constraint and less 80% compared to other algorithms. This is due to the nature of meta-heuristics GA algorithms that operates in a high performance using large search space to find a near-optimal solution (Fong et al, 2018).…”

Introduction to special issue on ‘intelligent computing and adaptive systems’

“…The proceedings highlight two invited articles: 'How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics' by Fong et al [2] and 'Using games to solve challenging multimedia problems' by Marques [4]. The conference showcased technical talks by Prof. Robert Bestak, Czech Technical University, Czech Republic; Prof. Amitava Chatterjee (FIETE, FIE(I), SMIEEE), Jadavpur University, Kolkata, India; and Prof. Sankhayan Choudhury, Calcutta University, Kolkata, India.…”