Simultaneous Depth Estimation and Localization for Cell Manipulation Based on Deep Learning

Wang, Zengshuo; Gong, Huiying; Yang, Bin; Du, Yue; Liu, Yaowei; Zhao, Xin; Sun, Mingzhu

doi:10.1109/iros47612.2022.9982228

Cited by 2 publications

(1 citation statement)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The plane positioning methods of the pipette include image intensity analysis [12], template matching [13], line intersection detection [14], and so on. Wang et al proposed a convolutional neural network (CNN) method for synchronous depth estimation and object localization in cell manipulation [15]. However, this method requires both the focused and defocused images of the pipette as input, limiting its application in patch-clamp experiments.…”

Section: Introductionmentioning

confidence: 99%

Neuron Contact Detection Based on Pipette Precise Positioning for Robotic Brain-Slice Patch Clamps

Li,

Gong,

Qiu

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

A patch clamp is the “gold standard” method for studying ion-channel biophysics and pharmacology. Due to the complexity of the operation and the heavy reliance on experimenter experience, more and more researchers are focusing on patch-clamp automation. The existing automated patch-clamp system focuses on the process of completing the experiment; the detection method in each step is relatively simple, and the robustness of the complex brain film environment is lacking, which will increase the detection error in the microscopic environment, affecting the success rate of the automated patch clamp. To address these problems, we propose a method that is suitable for the contact between pipette tips and neuronal cells in automated patch-clamp systems. It mainly includes two key steps: precise positioning of pipettes and contact judgment. First, to obtain the precise coordinates of the tip of the pipette, we use the Mixture of Gaussian (MOG) algorithm for motion detection to focus on the tip area under the microscope. We use the object detection model to eliminate the encirclement frame of the pipette tip to reduce the influence of different shaped tips, and then use the sweeping line algorithm to accurately locate the pipette tip. We also use the object detection model to obtain a three-dimensional bounding frame of neuronal cells. When the microscope focuses on the maximum plane of the cell, which is the height in the middle of the enclosing frame, we detect the focus of the tip of the pipette to determine whether the contact between the tip and the cell is successful, because the cell and the pipette will be at the same height at this time. We propose a multitasking network CU-net that can judge the focus of pipette tips in complex contexts. Finally, we design an automated contact sensing process in combination with resistance constraints and apply it to our automated patch-clamp system. The experimental results show that our method can increase the success rate of pipette contact with cells in patch-clamp experiments.

show abstract