Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

Okunev, A. G.; Mashukov, Mikhail Yu.; Nartova, Anna V.; Матвеев, А. В.

doi:10.3390/nano10071285

Cited by 59 publications

(42 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It has been tested in various fields such as medical image analysis (brain tumor detection [19], nucleus detection in microscopy [20], detection of lung nodule [21]), satellite images analysis [22], very high spatial resolution aerial imagery [23] or astronomy [24] to name but a few examples. This algorithm showed also promising results on STM images of nanoparticles [25].…”

Section: Deep Learning Based Instance Segmentationmentioning

confidence: 74%

Deep Learning Based Instance Segmentation of Titanium Dioxide Particles in the Form of Agglomerates in Scanning Electron Microscopy

et al. 2021

View full text Add to dashboard Cite

The size characterization of particles present in the form of agglomerates in images measured by scanning electron microscopy (SEM) requires a powerful image segmentation tool in order to properly define the boundaries of each particle. In this work, we propose to use an algorithm from the deep statistical learning community, the Mask-RCNN, coupled with transfer learning to overcome the problem of generalization of the commonly used image processing methods such as watershed or active contour. Indeed, the adjustment of the parameters of these algorithms is almost systematically necessary and slows down the automation of the processing chain. The Mask-RCNN is adapted here to the case study and we present results obtained on titanium dioxide samples (non-spherical particles) with a level of performance evaluated by different metrics such as the DICE coefficient, which reaches an average value of 0.95 on the test images.

show abstract

Section: Deep Learning Based Instance Segmentationmentioning

confidence: 74%

Deep Learning Based Instance Segmentation of Titanium Dioxide Particles in the Form of Agglomerates in Scanning Electron Microscopy

et al. 2021

View full text Add to dashboard Cite

show abstract

“…When trained on such big data sets, CNNs are able to achieve task-relevant object detection performances that are comparable or even superior to the capabilities of humans. [31,32] In recent years, new methods have been proposed for the analysis of nanostructures in SEM and TEM images that rely on advanced machine and deep learning techniques [33][34][35][36][37][38][39][40][41][42][43][44][45] which allow for accurate and high-throughput image analysis. However, as most of these methods use a supervised learning approach, significant human effort is needed to prepare the training data.…”

Section: Introductionmentioning

confidence: 99%

“…In recent years, new methods have been proposed for the analysis of nanostructures in SEM and TEM images that rely on advanced machine and deep learning techniques [ 33–45 ] which allow for accurate and high‐throughput image analysis. However, as most of these methods use a supervised learning approach, significant human effort is needed to prepare the training data.…”

Section: Introductionmentioning

confidence: 99%

Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation

Mill

Wolff²,

Gerrits

et al. 2021

Small Methods

View full text Add to dashboard Cite

and are important in various technological fields such as energy, electronics, medicine, and many more. [1][2][3][4][5] However, as a consequence of industrial processes and man-made pollution, unwanted nanoparticle size distributions and concentrations [6] give rise to concerns with respect to human health and environmental pollution. While the nanoparticles' physicochemical properties (size, shape, surface chemistry, etc.) determine the quality of products, [7,8] such characteristics are also important in order to evaluate the biological impact of nanoparticles at a molecular, cellular, and systemic level for any risk assessment for environmental and human health. [9] Characterizing nanoparticles in a dynamic context and on a case-by-case basis, microscopic imaging techniques including those that use focused electron or ion beams in scanning electron microscopes (SEMs) or helium ion microscopes [10] (HIMs) to generate nanometer scale spatial resolution are frequently applied in the scientific community. Given the substantial information content of digital images, these techniques often benefit from, or require, automated high-throughput data analysis that enables the accurate identification of large numbers of particles in a robust way.Nanoparticles occur in various environments as a consequence of man-made processes, which raises concerns about their impact on the environment and human health. To allow for proper risk assessment, a precise and statistically relevant analysis of particle characteristics (such as size, shape, and composition) is required that would greatly benefit from automated image analysis procedures. While deep learning shows impressive results in object detection tasks, its applicability is limited by the amount of representative, experimentally collected and manually annotated training data. Here, an elegant, flexible, and versatile method to bypass this costly and tedious data acquisition process is presented. It shows that using a rendering software allows to generate realistic, synthetic training data to train a state-of-the art deep neural network. Using this approach, a segmentation accuracy can be derived that is comparable to man-made annotations for toxicologically relevant metal-oxide nanoparticle ensembles which were chosen as examples. The presented study paves the way toward the use of deep learning for automated, highthroughput particle detection in a variety of imaging techniques such as in microscopies and spectroscopies, for a wide range of applications, including the detection of micro-and nanoplastic particles in water and tissue samples.

show abstract

“…35 Nanoparticle recognition was accelerated in scanning probe microscopy. 36 In the present study, we utilized the concept for visualization of defects using metal nanoparticles as contrast agents, and we developed a machine learning approach for automated analysis of both parameters-the number and type of defects. Two machine learning tasks were solved, and the corresponding results and algorithms were analyzed, yielding unique features for the quality assessment of layered carbon materials.…”

Section: Introductionmentioning

confidence: 99%

Deep neural network analysis of nanoparticle ordering to identify defects in layered carbon materials

et al. 2021

View full text Add to dashboard Cite

show abstract

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

Cited by 59 publications

References 32 publications

Deep Learning Based Instance Segmentation of Titanium Dioxide Particles in the Form of Agglomerates in Scanning Electron Microscopy

Deep Learning Based Instance Segmentation of Titanium Dioxide Particles in the Form of Agglomerates in Scanning Electron Microscopy

Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation

Deep neural network analysis of nanoparticle ordering to identify defects in layered carbon materials

Contact Info

Product

Resources

About