Pathology imaging is broadly used for identifying the causes and effects of diseases or injuries. Given a pathology image, being able to answer questions about the clinical findings contained in the image is very important for medical decision making. In this paper, we aim to develop a pathological visual question answering framework to analyze pathology images and answer medical questions related to these images. To build such a framework, we create PathVQA, a pathology VQA dataset with 32,795 questions asked from 4,998 pathology images. We also propose a three-level optimization framework which performs self-supervised pretraining and VQA finetuning end-to-end to learn powerful visual and textual representations jointly and automatically identifies and excludes noisy self-supervised examples from pretraining.We perform experiments on our created PathVQA dataset and the results demonstrate the effectiveness of our proposed methods. The datasets and code are available at https://github.com/UCSD-AI4H/PathVQA
Within the development of the deep convolutional neural network, the great achievements had been made in the single-image super-resolution (SISR) task. However, the higher performance always comes with the deeper layers which also brings larger numbers of network operations and parameters that make it hard to implement in practice. In our work, a lightly super-resolution, named Mobile Share-Source Network (MSSN), is purposed to address these practical issues. In MSSN, a high-efficiency block, the mobile adaptive weighted residual unit, is designed to fulfill the need for the reduction in both parameters and the Mult-Adds while maintaining the performance with importing the deep separable convolution. Moreover, it brings into the Adaptive Weighted Share-Source Skip Connection, getting abundant information from the shallow layer which helps reconstruct better images. The experimental results show that our network has fewer numbers of parameters and operations than the state-of-the-art lightweight network while maintaining high reconstruction quality comparing with many state-of-the-art super-resolution methods in terms of both peak signal-to-noise ratio (PSNR) and structural similarity index metrics (SSIM).
We develop datasets and methods to perform visual question answering on pathology images.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.