High Efficiency Video Coding (HEVC)–Based Surgical Telementoring System Using Shallow Convolutional Neural Network

Abstract

Surgical telementoring systems have gained lots of interest, especially in remote locations. However, bandwidth constraint has been the primary bottleneck for efficient telementoring systems. This study aims to establish an efficient surgical telementoring system, where the qualified surgeon (mentor) provides real-time guidance and technical assistance for surgical procedures to the on-spot physician (surgeon). High Efficiency Video Coding (HEVC/H.265)-based video compression has shown promising results for telementoring applications. However, there is a trade-off between the bandwidth resources required for video transmission and quality of video received by the remote surgeon. In order to efficiently compress and transmit real-time surgical videos, a hybrid lossless-lossy approach is proposed where surgical incision region is coded in high quality whereas the background region is coded in low quality based on distance from the surgical incision region. For surgical incision region extraction, state-of-the-art deep learning (DL) architectures for semantic segmentation can be used. However, the computational complexity of these architectures is high resulting in large training and inference times. For telementoring systems, encoding time is crucial; therefore, very deep architectures are not suitable for surgical incision extraction. In this study, we propose a shallow convolutional neural network (S-CNN)-based segmentation approach that consists of encoder network only for surgical region extraction. The segmentation performance of S-CNN is compared with one of the state-of-the-art image segmentation networks (SegNet), and results demonstrate the effectiveness of the proposed network. The proposed telementoring system is efficient and explicitly considers the physiological nature of the human visual system to encode the video by providing good overall visual impact in the location of surgery. The results of the proposed S-CNN-based segmentation demonstrated a pixel accuracy of 97% and a mean intersection over union accuracy of 79%. Similarly, HEVC experimental results showed that the proposed surgical region-based encoding scheme achieved an average bitrate reduction of 88.8% at high-quality settings in comparison with default full-frame HEVC encoding. The average gain in encoding performance (signal-to-noise) of the proposed algorithm is 11.5 dB in the surgical region. The bitrate saving and visual quality of the proposed optimal bit allocation scheme are compared with the mean shift segmentation-based coding scheme for fair comparison. The results show that the proposed scheme maintains high visual quality in surgical incision region along with achieving good bitrate saving. Based on comparison and results, the proposed encoding algorithm can be considered as an efficient and effective solution for surgical telementoring systems for low-bandwidth networks.

Multi-View Feature Fusion Based Four Views Model for Mammogram Classification Using Convolutional Neural Network

“…However, over the last five years deep learning paradigm has been increasingly employed in computer vision related tasks. Deep learning has been commonly used in activities related to medical imaging [9]- [12]. It is based on the workings of the human brain and it models them in deep architectures to detect abnormalities in images.…”

Multi-Feature View-Based Shallow Convolutional Neural Network for Road Segmentation

“…In this study, the segmentation accuracy is calculated using the most commonly used semantic segmentation metrics [32], [52], such as mean Intersection over Union (IoU), mean accuracy, pixel accuracy, and frequency weighted (f.w.). IoU.…”

“…Hence, such techniques do not perform well in noisy images as well as images with different lighting conditions. Such techniques require pre-processing of images and manual fine-tuning which varies with different types of images under different circumstances [32].…”

“…Many DL based applications have been proposed recently. The applications include image classification [32], [38], [39], image segmentation [40], [41], change detection in images [42], [43] and object tracking [44], [45]. While Deep Convolutional Neural Networks (DCNN) have been useful in the aforementioned DL applications, it also proves helpful in semantic segmentation [29], [46].…”

Implementation and Application of Telemedicine in China: Cross-Sectional Study

“…Telemedicine refers to the remote delivery of health care services with the use of modern communication technology, electronic technology, and multimedia computer technology to realize the remote collection, transmission, processing, storage, and inquiry of medical information and to further provide the examination, surveillance and diagnosis of disease, remote education, and information management [ 1 , 2 ]. The common types of telemedicine services include teleconsultation, telediagnosis, and remote surgery teaching [ 3 - 5 ]. With the rapid development of telemedicine equipment and information communication technology, telemedicine has developed rapidly and is used widely around the world as a new mode of medical service [ 6 , 7 ].…”