Automating Data Mart Construction from Semi-structured Data Sources

Scriney, Michael; McCarthy, Suzanne; McCarren, Andrew; Cappellari, Paolo; Roantree, Mark

doi:10.1093/comjnl/bxy064

Cited by 10 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Finally, the two datasets require integration. A previous work (Scriney et al, 2019 ) proposes a methodology for the determination of an integration strategy by examining the common datasets for each source in order to design a common data model. In this study, the common data model is the FHIR JSON schema.…”

Section: Enhancing Semantic Interoperability With the Semantic Enginementioning

confidence: 99%

An Ostensive Information Architecture to Enhance Semantic Interoperability for Healthcare Information Systems

2023

Self Cite

View full text Add to dashboard Cite

Semantic interoperability establishes intercommunications and enables data sharing across disparate systems. In this study, we propose an ostensive information architecture for healthcare information systems to decrease ambiguity caused by using signs in different contexts for different purposes. The ostensive information architecture adopts a consensus-based approach initiated from the perspective of information systems re-design and can be applied to other domains where information exchange is required between heterogeneous systems. Driven by the issues in FHIR (Fast Health Interoperability Resources) implementation, an ostensive approach that supplements the current lexical approach in semantic exchange is proposed. A Semantic Engine with an FHIR knowledge graph as the core is constructed using Neo4j to provide semantic interpretation and examples. The MIMIC III (Medical Information Mart for Intensive Care) datasets and diabetes datasets have been employed to demonstrate the effectiveness of the proposed information architecture. We further discuss the benefits of the separation of semantic interpretation and data storage from the perspective of information system design, and the semantic reasoning towards patient-centric care underpinned by the Semantic Engine.

show abstract

Section: Enhancing Semantic Interoperability With the Semantic Enginementioning

confidence: 99%

An Ostensive Information Architecture to Enhance Semantic Interoperability for Healthcare Information Systems

2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…• Interoperable. Data interoperability means the ability to share and integrate data from different users and sources [16]. This can only happen if a standard (meta) data model is employed to describe data, an important concept which generally requires data engineering skills to deliver.…”

Section: Application Of Fair Metricsmentioning

confidence: 99%

Constructing a Searchable Knowledge Repository for FAIR Climate Data

Roantree¹,

Lalić

Savić

et al. 2023

Preprint

View full text Add to dashboard Cite

The development of a knowledge repository for climate science data is a multidisciplinary effort between the domain experts (climate scientists), data engineers who's skills include design and building a knowledge repository, and machine learning researchers who provide expertise on data preparation tasks such as gap filling and advise on different machine learning models that can exploit this data. One of the main goals of the CA20108 cost action is to develop a knowledge portal that is fully compliant with the FAIR principles for scientific data management. In the first year, a bespoke knowledge portal was developed to capture metadata for FAIR datasets. Its purpose was to provide detailed metadata descriptions for shareable micro-meteorological (micromet) data using the WMO standard. While storing Network, Site and Sensor metadata locally, the system passes the actual data to Zenodo, receives back the DOI and thus, creates a permanent link between the Knowledge Portal and the storage platform Zenodo. While the user searches the Knowledge portal (metadata), results provide both detailed descriptions and links to data on the Zenodo platform. Our adherence to FAIR principles are documented below: <ul> <li>Findable. Machine-readable metadata is required for automatic discovery of datasets and services. A metadata description is supplied by the data owners for all micro-meteorological data shared on the system which subsequently drives the search engine, using keywords or network, site and sensor search terms.</li> <li>Accessible. When suitable datasets have been identified, access details should be provided. Assuming data is freely accessible, Zenodo DOIs and links are provided for direct data access.</li> <li>Interoperable. Data interoperability means the ability to share and integrate data from different users and sources. This can only happen if a standard (meta)data model is employed to describe data, an important concept which generally requires data engineering skills to deliver. In the knowledge portal presented here, the WMO guide provides the design and structure for metadata.&#160;&#160;&#160;&#160;</li> <li>Reusable. To truly deliver reusability, metadata should be expressed in as detailed a manner as possible. In this way, data can be replicated and integrated according to different scientific requirements. While the Knowledge Portal facilitates very detailed metadata descriptions, not all metadata is compulsory as it was accepted that in some cases, the overhead in providing this information can be very costly.&#160;</li> </ul> Simple analytics are in place to monitor the volume and size of networks in the system. Current metrics include: network count; average size of network (number of sites); dates and size of datasets per network/site; numbers and types of sensors in each site, etc. The current Portal is in Beta version meaning that the system is currently functional but open only to members of the Cost Action who are nominated testers. This status is due to change in Q1/2023 when access will be open to the wider climate science community.&#160;&#160; Current plans include new Tools and Services to assess the quality of data, including the level of gaps and in some cases, machine learning tools will be provided to attempt gap filling for datasets meeting certain requirements. &#160;

show abstract

“…However, in many cases, variables that are available are not suited to machine learning algorithms with the identification and preparation Han et al (2011) of the data set seen as a complex process. Another critical task requires the transformation of variables (Components) (King and Jackson, 1999;Westad et al, 2003;Granato et al, 2018;Scriney et al, 2019;Yun et al, 2019) using the appropriate normalization methodology (Milligan and Cooper, 1988). Our research is based on a problem that is common to many retailers: computing CLV values where customer records are separated across different points of sales with no clear method to combine them.…”

Section: Problem Description and Motivationmentioning

confidence: 99%

A methodology for classification and validation of customer datasets

Nie

Cappellari

Roantree

2020

JBIM

Self Cite

View full text Add to dashboard Cite

Purpose The purpose of this paper is to develop a method to classify customers according to their value to an organization. This process is complicated by the disconnected nature of a customer record in an industry such as insurance. With large numbers of customers, it is of significant benefit to managers and company analysts to create a broad classification for all customers. Design/methodology/approach The initial step is to construct a full customer history and extract a feature set suited to customer lifetime value calculations. This feature set must then be validated to determine its ability to classify customers in broad terms. Findings The method successfully classifies customer data sets with an accuracy of 90%. This study also discovered that by examining the average value for key variables in each customer segment, an algorithm can label the group of clusters with an accuracy of 99.3%. Research limitations/implications Working with a real-world data set, it is always the case that some features are unavailable as they were never recorded. This can impair the algorithm’s ability to make good classifications in all cases. Originality/value This study believes that this research makes a novel contribution as it automates the classification of customers but in addition, the approach provides a high-level classification result (recall and precision identify the best cluster configuration) and detailed insights into how each customer is classified by two validation metrics. This supports managers in terms of market spend on new and existing customers.

show abstract

Automating Data Mart Construction from Semi-structured Data Sources

Cited by 10 publications

References 22 publications

An Ostensive Information Architecture to Enhance Semantic Interoperability for Healthcare Information Systems

An Ostensive Information Architecture to Enhance Semantic Interoperability for Healthcare Information Systems

Constructing a Searchable Knowledge Repository for FAIR Climate Data

A methodology for classification and validation of customer datasets

Contact Info

Product

Resources

About