Physics-Based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-Guided Breast Biopsy

Abstract

In the context of ultrasound (US) guided breast biopsy, image fusion techniques can be employed to track the position of USinvisible lesions previously identified on a pre-operative image. Such methods have to account for the large anatomical deformations resulting from probe pressure during US scanning within the real-time constraint. Although biomechanical models based on the finite element (FE) method represent the preferred approach to model breast behavior, they cannot achieve real-time performances. In this paper we propose to use deep neural networks to learn large deformations occurring in ultrasoundguided breast biopsy and then to provide accurate prediction of lesion displacement in real-time. We train a U-Net architecture on a relatively small amount of synthetic data generated in an offline phase from FE simulations of probe-induced deformations on the breast anatomy of interest. Overall, both training data generation and network training are performed in less than 5 hours, which is clinically acceptable considering that the biopsy can be performed at most the day after the pre-operative scan. The method is tested both on synthetic and on real data acquired on a realistic breast phantom. Results show that our method correctly learns the deformable behavior modelled via FE simulations and is able to generalize to real data, achieving a target registration error comparable to that of FE models, while being about a hundred times faster.

Real‐time biomechanics using the finite element method and machine learning: Review and perspective

“…Finally, there are cases where the biomechanical simulations are used to predict the individual displacement of elements of a group in space and time. These elements can be, for example, nodes of a mesh that represents the soft tissue of the liver, 6,7 the breast, 20 the brain, 37 the prostate 8 or the stomach, 11 or points along a line that represents the shape of the spine 17 …”

“…Other works focus on the use of feedforward and radial basis function (RBF) NN to simulate the behavior of various anatomical structures. Finally, seven reviewed publications use DL architectures, such as the U-Net, 20,31 to generate simulations.…”

“…For example, in some cases, the hyperparameters of NN, such as the number of layers, number of hidden units, activation functions, learning rate, number of epochs, and others are not tuned by using a validation set given the considerable computational requirement to train a NN. In those cases, common percentages of datasets assigned to the training and testing datasets are 70% and 30%, 32 80% and 20%, 20,31,35,45 or 90% and 10%, 6,49 respectively.…”

“…Some examples in the literature include meshes reconstructed from computed tomography (CT) images of ex vivo discarded livers intended to be used for transplantation 39 or from magnetic resonance (MR) images of brains that presented a tumor 37 . Additionally, meshes were also reconstructed from ultrasound images of physical phantoms that represent the breast 20 and MR images of the prostate 8 . It should be noted that even when the meshes mentioned in this paragraph come from individual subjects or physical phantoms, the phenomena applied to them are simulations and follow the assumptions and simplifications associated to the FEM.…”

“…19 to model stress in atherosclerotic arterial walls, by minimizing the total potential energy in a situation of equilibrium. A different approach is taken by Mendizabal et al 20 , where the elastic behavior of the breast in response to the presence of an ultrasound probe is modeled by considering the Dirichlet and Neumann boundary conditions.…”