Digital video microscopy enhanced by deep learning

Abstract

Single particle tracking is essential in many branches of science and technology, from the measurement of biomolecular forces to the study of colloidal crystals. Standard current methods rely on algorithmic approaches: by fine-tuning several user-defined parameters, these methods can be highly successful at tracking a well-defined kind of particle under low-noise conditions with constant and homogenous illumination. Here, we introduce an alternative data-driven approach based on a convolutional neural network, which we name DeepTrack. We show that DeepTrack outperforms algorithmic approaches, especially in the presence of noise and under poor illumination conditions. We use DeepTrack to track an optically trapped particle under very noisy and unsteady illumination conditions, where standard algorithmic approaches fail. We then demonstrate how DeepTrack can also be used to track multiple particles and non-spherical objects such as bacteria, also at very low signal-to-noise ratios. In order to make DeepTrack readily available for other users, we provide a Python software package, which can be easily personalized and optimized for specific applications.

Deep Learning Enabled Laser Speckle Wavemeter with a High Dynamic Range

“…It has been shown in a previous study that the CNN, once trained on a given dataset, learns to overcome a low signal to noise environment. [35] In our study we find analogous results. Through the training process, the CNN learns to reject variations in the speckle patterns which do not correspond to the control parameter, that is, wavelength.…”

Machine learning for active matter

“…For example, ref. 37 demonstrates how a neural-networkbased particle-detection algorithm outperforms conventional particle-tracking methods (Fig. 2a).…”

“…Developing such skills entails a cost, so the corresponding strategies can only emerge through Figure adapted with permission from ref. 37 , OSa (a); and ref. 44 , PNaS (b).…”

“…Data acquisition and analysis. The most important and common use of machine learning in active-matter research is in the analysis and classification of experimental data using supervised learning models (usually neural nets) [33][34][35][36][37] (Box 1). In fact, most active-matter experiments are performed using video microscopy, which provides large, high-quality training datasets that also cover less likely experimental conditions, which are ideally suited to image analysis with supervised machine-learning methods 38 .…”

“…Until now, activematter research has used standard machine-learning models for image processing (for example, convolutional nets for particle tracking [33][34][35][36][37] ) and for time series analysis (recurrent nets for data classification 44,45 ). Going forward, suitable neural-net architectures for analysing more complex, multimodal data will require more targeted, hybrid approaches 76 .…”