A similarity-based data warehousing environment for medical images

Abstract

A core issue of the decision-making process in the medical field is to support the execution of analytical (OLAP) similarity queries over images in data warehousing environments. In this paper, we focus on this issue. We propose imageDWE, a non-conventional data warehousing environment that enables the storage of intrinsic features taken from medical images in a data warehouse and supports OLAP similarity queries over them. To comply with this goal, we introduce the concept of perceptual layer, which is an abstraction used to represent an image dataset according to a given feature descriptor in order to enable similarity search. Based on this concept, we propose the imageDW, an extended data warehouse with dimension tables specifically designed to support one or more perceptual layers. We also detail how to build an imageDW and how to load image data into it. Furthermore, we show how to process OLAP similarity queries composed of a conventional predicate and a similarity search predicate that encompasses the specification of one or more perceptual layers. Moreover, we introduce an index technique to improve the OLAP query processing over images. We carried out performance tests over a data warehouse environment that consolidated medical images from exams of several modalities. The results demonstrated the feasibility and efficiency of our proposed imageDWE to manage images and to process OLAP similarity queries. The results also demonstrated that the use of the proposed index technique guaranteed a great improvement in query processing.

Processamento Eficiente de Consultas Analíticas Estendidas com Predicado de Similaridade em Spark

“…Em [Teixeira et al 2015]é proposto o processamento de consultas analíticas estendidas com predicado de similaridade. Esse trabalho usa a técnica Omni, porém não considera o processamento dessas consultas em ambientes computacionais paralelos e distribuídos.…”

“…Além das tabelas de dimensão convencionais, um DW de imagens também contém outras que armazenam as características intrínsecas das imagens: uma tabela Vetor Características e várias tabelas Camada Perceptual [Teixeira et al 2015]. A tabela Vetor Características contém, para cada camada perceptual de cada imagem, um vetor de características gerado por um extrator de características.…”

“…Um data warehousing de imagens estende um data warehousing convencional para também manipular imagens [Teixeira et al 2015]. O processo ETL (extract, transform, load)é estendido para realizar a extração das características intrínsecas das imagens, as quais são representadas por vetores de características e atributos para pesquisa por similaridade.…”

“…Assim, uma nova gama de consultas analíticas pode ser realizada, enriquecendo a tomada de decisão estratégica. Por exemplo, em uma organização médica voltadaà análise de imagens de exames, pode-se determinar quantas imagens de câncer pulmonar são similares a uma dada imagem, considerando pacientes maiores do que 40 anos no estado de São Paulo nosúltimos 3 anos [Teixeira et al 2015]. No contexto da agricultura, medidas agrícolas referentes ao solo, fertilidade e umidade ao longo dos anos representam tipos de dados convencionais e os vetores de características de imagens armazenadas nesse contexto representam tipos de dados estendidos ].…”

Research and realization of improved extract–transform–load scheduler in China Southern Power Grid

“…Supporting smart grid secure, self-healing, green, strong, and reliable operations are based on the acquisition, transmission, and storage of the panoramic view real-time data, as well as the rapid analysis of huge amount of accumulated and multi-source data. 4 Big data in the smart grid usually show five key characteristics, mainly including volume, velocity, variety, veracity, and value, which also have been widely applied to intelligent transportation, 5 business, 6 medicine, 7 building, 8 and so on. In order to achieve an integrated collection of the subject-oriented big data in the support of making decision, the term data warehouse (DW) is proposed by Inmon 9 in the 1990s.…”