Scalable Architecture for Personalized Healthcare Service Recommendation Using Big Data Lake

Abstract

The personalized health care service utilizes the relational patient data and big data analytics to tailor the medication recommendations. However, most of the health care data are in unstructured form and it consumes a lot of time and effort to pull them into relational form. This study proposes a novel data lake architecture to reduce the data ingestion time and improve the precision of healthcare analytics. It also removes the data silos and enhances the analytics by allowing the connectivity to the third-party data providers (such as clinical lab results, chemist, insurance company,etc.). The data lake architecture uses the Hadoop Distributed File System (HDFS) to provide the storage for both structured and unstructured data. This study uses K-means clustering algorithm to find the patient clusters with similar health conditions. Subsequently, it employs a support vector machine to find the most successful healthcare recommendations for the each cluster. Our experiment results demonstrate the ability of data lake to reduce the time for ingesting data from various data vendors regardless of its format. Moreover, it is evident that the data lake poses the potential to generate clusters of patients more precisely than the existing approaches. It is obvious that the data lake provides an unified storage location for the data in its native format. It can also improve the personalized healthcare medication recommendations by removing the data silos.

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

“…Tableau is a software for interactive data visualization. [6], [9], [31], [47], [55], [68], [76], [82], [85] Apache Flume 7 [1], [6], [27], [52], [61], [70], [83] Apache Sqoop 5 [1], [27], [47], [52], [ [6], [21], [40], [41], [45] MongoDB 6 [16], [33], [35], [41], [43], [ [1], [6], [8], [9], [12], [18], [24], [27], [33], [35], [40], [42], [44], [47], [49], [51], [52], [55], [61], [68], [69], [71],…”

“…Initial Accepted Scopus 108 53 papers: [1]- [3], [5], [9], [10], [13]- [19], [23]- [29], [31]- [33], [37], [40], [45], [49], [50], [57], [60]- [66], [68], [70], [71], [73], [76]- [78], [81]- [84], [88], [90], [91], [93]- [95] Springer 222 20 papers: [4], [6], [12], [21], [30], [36], [38], [39], [41]- [43], [47], [51], [53], [69], [74], [79], [85], [86], [92] Google Scholar 197 6 papers:...…”

From multisource data to clinical decision aids in radiation oncology: The need for a clinical data science community

“…We talk about two types of infrastructure: centralized collaborative big data infrastructures and federated big data infrastructures. In centralized infrastructures, different hospitals and healthcare providers enter a collaboration and upload patient data to a secured centralized repository [31]. Researchers can use data from the repository either train their algorithms and perform analysis.…”

Analysis of Data Warehouse Architectures: Modeling and Classification

“…The Data Lake Architecture: The DL can be leveraged directly as a centralised raw data repository providing information for further analyses (Rangarajan et al, 2015). The DL and DWH can also cooperate together to address the big data issues and achieve data analysis requirements, which are generalised and modelled in figure 1.…”