Stitched Multipanel Biomedical Figure Separation

Santosh, K. C.; Thoma, George R.

doi:10.1109/cbms.2015.51

Cited by 9 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Santosh et al ’s method [18] (based on their previous work [2,11]) achieves an accuracy of 84.64%, while Taschwer et al [7] report an accuracy of 84.90%. Our method performs significantly better than all other systems with an accuracy of 90.65%.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…In the second set of experiments, we compare the results obtained by our comprehensive method with those of other systems submitted to ImageCLEF’15 Medical, using the 2015 test dataset. Santosh et al ’s method [18] (based on their previous work [2,11]) achieves an accuracy of 84.64%, while Taschwer et al [7] report an accuracy of 84.90%. Our method performs significantly better than all other systems with an accuracy of 90.65%.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Notably, none of the above methods can segment stitched compound images whose panels are not separated by visible gaps. Santosh et al [11] first proposed a method to separate stitched compound images based on straight lines detected in the images. Their method is applicable only to stitched compound images and as such relies on a manual selection step in which such images are identified within the dataset.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Segmenting Compound Biomedical Figures into Their Constituent Panels

Jiang

Kambhamettu

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Many of the figures in biomedical publications are compound figures consisting of multiple panels. Segmenting such figures into constituent panels is an essential first step for harvesting the visual information within the biomedical documents. Current figure separation methods are based primarily on gap-detection and suffer from over- and under-segmentation. In this paper, we propose a new compound figure segmentation scheme based on Connected Component Analysis. To overcome shortcomings typically manifested by existing methods, we develop a quality assessment step for evaluating and modifying segmentations. Two methods are proposed to re-segment the images if the initial segmentations are inaccurate. Experiments and results comparing the performance of our method to that of other top methods demonstrate the effectiveness of our approach.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Segmenting Compound Biomedical Figures into Their Constituent Panels

Jiang

Kambhamettu

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Overall, traditional computer vision methods are usually based on visual information, such as gaps, edges, connected components, and so on. For example, stitched figures are usually segmented by applying edge map analysis, local line segment detection and line vectorization to connect prominent line segments while eliminating insignificant ones (Aafaque & Santosh, ; Santosh, Aafaque, Antani, & Thoma, ; Santosh, Antani, & Thoma, ). FigSplit, a multipanel figure segmentation system developed by Li et al (Li et al, ; Li, Jiang, Kambhamettu, & Shatkay, ), is mainly based on connected component analysis.…”

Section: Related Workmentioning

confidence: 99%

Unified deep neural network for segmentation and labeling of multipanel biomedical figures

Zou

Thoma

Antani

2020

Asso for Info Science & Tech

Self Cite

View full text Add to dashboard Cite

Recent efforts in biomedical visual question answering (VQA) research rely on combined information gathered from the image content and surrounding text supporting the figure. Biomedical journals are a rich source of information for such multimodal content indexing. For multipanel figures in these journals, it is critical to develop automatic figure panel splitting and label recognition algorithms to associate individual panels with text metadata in the figure caption and the body of the article. Challenges in this task include large variations in figure panel layout, label location, size, contrast to background, and so on. In this work, we propose a deep convolutional neural network, which splits the panels and recognizes the panel labels in a single step. Visual features are extracted from several layers at various depths of the backbone neural network and organized to form a feature pyramid. These features are fed into classification and regression networks to generate candidates of panels and their labels. These candidates are merged to create the final panel segmentation result through a beam search algorithm. We evaluated the proposed algorithm on the ImageCLEF data set and achieved better performance than the results reported in the literature. In order to thoroughly investigate the proposed algorithm, we also collected and annotated our own data set of 10,642 figures. The experiments, trained on 9,642 figures and evaluated on the remaining 1,000 figures, show that combining panel splitting and panel label recognition mutually benefit each other. 1 | INTRODUCTION Multipanel figures are figure images in journal articles that contain more than one subfigure. About 40-60% of the figures in the biomedical literature are multipanel (Garcia Seco de Herrera, Kalpathy-Cramer, Demner-Fushman, Antani, & Muller, 2013). In these multipanel figures, panel labels help associate them with the biomedical text associated with the figures, for example, captions, mentions in the body text (You, Apostolova, Antani, Demner-Fushman, & Thoma, 2009, January). Precise image feature extraction and indexing for each panel is critical for many applications such as biomedical visual question answering (VQA), subfigure classification, and multimodal (image and text) search and retrieval (Apostolova et al., 2013; Chhatkuli, Markonis, Foncubierta-Rodriguez, Meriaudeau, & Muller, 2013, February). In these tasks, image features extracted from each image panel need to be associated with the biomedical concepts contained in the surrounding texts. In this † Dr. George Thoma has retired, and this work was done during his tenure with the National Library of Medicine.

show abstract

“…Such extraction is not a simple task due to the complex and diverse layout of scientific publications and the variations in figure structure, texture, and contents. As biomedical figures often comprise multiple image panels, identifying compound figures and their constituent panels has itself been a topic of much research (Chhatkuli et al , 2013; Li et al , 2018; Santosh et al , 2015). These lines of research assume that the figures are already extracted from the publications, and do not focus on the extraction task.…”

Section: Introductionmentioning

confidence: 99%

Figure and caption extraction from biomedical documents

Jiang

Shatkay

2019

Bioinformatics

View full text Add to dashboard Cite

MotivationFigures and captions convey essential information in biomedical documents. As such, there is a growing interest in mining published biomedical figures and in utilizing their respective captions as a source of knowledge. Notably, an essential step underlying such mining is the extraction of figures and captions from publications. While several PDF parsing tools that extract information from such documents are publicly available, they attempt to identify images by analyzing the PDF encoding and structure and the complex graphical objects embedded within. As such, they often incorrectly identify figures and captions in scientific publications, whose structure is often non-trivial. The extraction of figures, captions and figure-caption pairs from biomedical publications is thus neither well-studied nor yet well-addressed.ResultsWe introduce a new and effective system for figure and caption extraction, PDFigCapX. Unlike existing methods, we first separate between text and graphical contents, and then utilize layout information to effectively detect and extract figures and captions. We generate files containing the figures and their associated captions and provide those as output to the end-user.We test our system both over a public dataset of computer science documents previously used by others, and over two newly collected sets of publications focusing on the biomedical domain. Our experiments and results comparing PDFigCapX to other state-of-the-art systems show a significant improvement in performance, and demonstrate the effectiveness and robustness of our approach.Availability and implementationOur system is publicly available for use at: https://www.eecis.udel.edu/~compbio/PDFigCapX. The two new datasets are available at: https://www.eecis.udel.edu/~compbio/PDFigCapX/Downloads

show abstract

Stitched Multipanel Biomedical Figure Separation

Cited by 9 publications

References 20 publications

Segmenting Compound Biomedical Figures into Their Constituent Panels

Segmenting Compound Biomedical Figures into Their Constituent Panels

Unified deep neural network for segmentation and labeling of multipanel biomedical figures

Figure and caption extraction from biomedical documents

Contact Info

Product

Resources

About