Deep learning in the context of nano-photonics is mostly discussed in terms of its potential for inverse design of photonic devices or nanostructures. Many of the recent works on machine-learning inverse design are highly specific, and the drawbacks of the respective approaches are often not immediately clear. In this review we want therefore to provide a critical review on the capabilities of deep learning for inverse design and the progress which has been made so far. We classify the different deep learning-based inverse design approaches at a higher level as well as by the context of their respective applications and critically discuss their strengths and weaknesses. While a significant part of the community's attention lies on nano-photonic inverse design, deep learning has evolved as a tool for a large variety of applications. The second part of the review will focus therefore on machine learning research in nano-photonics "beyond inverse design". This spans from physics informed neural networks for tremendous acceleration of photonics simulations, over sparse data reconstruction, imaging and "knowledge discovery" to experimental applications. http://dx.doi.org/XX.XXXX/XX.XX.XXXXXX CONTENTS C Deep learning for interpretation of photonics experiments 11 4 Conclusions and perspectives 13

Deep artificial neural networks are powerful tools with many possible applications in nanophotonics. Here, we demonstrate how a deep neural network can be used as a fast, general purpose predictor of the full near-field and far-field response of plasmonic and dielectric nanostructures. A trained neural network is shown to infer the internal fields of arbitrary three-dimensional nanostructures many orders of magnitude faster compared to conventional numerical simulations. Secondary physical quantities are derived from the deep learning predictions and faithfully reproduce a wide variety of physical effects without requiring specific training. We discuss the strengths and limitations of the neural network approach using a number of model studies of single particles and their near-field interactions. Our approach paves the way for fast, yet universal methods for design and analysis of nanophotonic systems.

The rational design of photonic nanostructures consists in anticipating their optical response from simple models and their systematic variations. This strategy, however, has limited success when multiple objectives are simultaneously targeted because it requires demanding computational schemes. To this end, evolutionary algorithms can drive the morphology of a nano-object towards an optimum through several cycles of selection, mutation and cross-over, mimicking the process of natural selection. Here, we present a numerical technique to design photonic nanostructures with optical properties optimised along several arbitrary objectives. We combine evolutionary multiobjective algorithms with frequency-domain electro-dynamical simulations to optimise the design of colour pixels based on silicon nanostructures that resonate at two user-defined, polarisationdependent wavelengths. The scattering spectra of optimised pixels fabricated by electron beam lithography show excellent agreement with the targeted objectives. The method is self-adaptive to arbitrary constraints, and therefore particularly apt for the design of complex structures within predefined technological limits.Over the last decade, the field of nanophotonics or nano-optics has been rapidly increasing, mainly driven by plasmonics, since noble metal nanoparticles allow to spectrally tune plasmon resonances 1 and tailor several optical properties like directional scattering, 2 polarisation conversion, 3 optical chirality 4 or nonlinear effects 5 . Recently, high-index dielectric nanostructures have gained increasing interest thanks to their ability to provide exceptionally strong electric 6,7 and magnetic [8][9][10] resonances, tunable from the UV to the near IR. [11][12][13] In analogy to plasmonics, it is possible to design functionalities like transmissive metasurfaces, 14 enhanced nonlinear effects 15,16 or directional scattering. 17 When designing photonic nanostructures, a particular geometry is usually selected from qualitative considerations and its properties are subsequently studied systematically. As it comes to applications, a more convenient approach is to define the requested properties and design a nanostructure that optimally exhibits the desired features. For the latter approach, a structure model has to be developed, which, based on a certain set of parameters, can describe a large variety of particle geometries. However, this leads to huge parameter spaces which usually cannot be explored systematically. Also trial-and-error is not an efficient search strategy. More promising techniques are evolutionary optimisation strategies which, by mimicking natural selection, are able to find fittest parameter sets to a complex non-analytical problem. 18 In the field of nanophotonics, evolutionary algorithms have been applied to the maximisation * e-mail : peter. These studies were limited to the maximisation of one target property at a specific wavelength and polarisation. Such single-objective scenarii represent the simplest case of an optimisatio...