Deep learning detection of nanoparticles and multiple object tracking of their dynamic evolution during in situ ETEM studies

Abstract

In situ transmission electron microscopy (TEM) studies of dynamic events produce large quantities of data especially under the form of images. In the important case of heterogeneous catalysis, environmental TEM (ETEM) under gas and temperature allows to follow a large population of supported nanoparticles (NPs) evolving under reactive conditions. Interpreting properly large image sequences gives precious information on the catalytic properties of the active phase by identifying causes for its deactivation. To perform a quantitative, objective and robust treatment, we propose an automatic procedure to track nanoparticles observed in Scanning ETEM (STEM in ETEM). Our approach involves deep learning and computer vision developments in multiple object tracking. At first, a registration step corrects the image displacements and misalignment inherent to the in situ acquisition. Then, a deep learning approach detects the nanoparticles on all frames of video sequences. Finally, an iterative tracking algorithm reconstructs their trajectories. This treatment allows to deduce quantitative and statistical features about their evolution or motion, such as a Brownian behavior and merging or crossing events. We treat the case of in situ calcination of palladium (oxide) / delta-alumina, where the present approach allows a discussion of operating processes such as Ostwald ripening or NP aggregative coalescence.

Analysis and implementation of dynamic evolution of paleoclimate dynamics simulation based on temporal GIS

“…The model can realize various uncertainties in the evolution of the paleoclimate cycle. The model can provide users with real-time dynamic historical data, and can effectively process the original geomorphological information, so that it can play a better role in the process of paleoclimate evolution and provide reference for people's research, thus promoting the development of the theory and technology of dynamic evolution of paleoclimate environment [10] .…”

A Review of Advanced Multifunctional Magnetic Nanostructures for Cancer Diagnosis and Therapy Integrated into an Artificial Intelligence Approach

“…For instance, AI-enabled nanoparticle synthesis platforms and nanoparticle delivery systems (AI-assisted algorithms) have been adopted to optimize the synthesis and delivery of magnetic nanoparticles for anticancer drug delivery, respectively. It uses machine learning, deep learning, and computer vision techniques to predict the optimal parameters for nanoparticle synthesis that accurately target the nanoparticles to the desired cells [ 258 , 259 ]. Furthermore, AI-assisted algorithms have also been employed to track the magnetic nanoparticles and monitor the toxicity of the drug during delivery to provide insights into cancer progression and identify potential therapeutic strategies [ 260 ].…”

Direct Observation of Sample Dynamics in the Gaseous Environment: A Perspective on Current Trends and Future Directions of In Situ Gas Cell Transmission Electron Microscopy

“…NP trajectories and their fusion events were precisely retrieved from the experimental data set enabling the accurate determination of NP behavior. 85…”

“…For example, in an ETEM setup, Faraz et al have reported the calcination process of Pd­(O)@δ-Al 2 O 3 nanoparticles in an oxygen environment, where DL algorithms were utilized for the quantitative analysis of the evolution of NPs under reactive conditions. NP trajectories and their fusion events were precisely retrieved from the experimental data set enabling the accurate determination of NP behavior . Further, several AI-developed programs are already commercially available and can perform in situ TEM experiments with minimal thermal drift, better synchronization between the column, detectors, and camera, and a facile transition from TEM to STEM mode.…”