Utility Metrics for Evaluating Synthetic Health Data Generation Methods: Validation Study (Preprint)

Emam, Khaled El; Mosquera, Lucy; Fang, Xi; El‐Hussuna, Alaa

doi:10.2196/preprints.35734

Cited by 27 publications

(37 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The original data is always at risk of being compromised or exposed as long as it is in play, especially in business, where data sharing is severely limited within and outside the company [63]. Therefore, it is vital to investigate methods for generating financial datasets that incorporate the same properties as the "real data" while still respecting the privacy of the parties involved.…”

Section: Businessmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning for Synthetic Data Generation: a Review

Lu¹,

Wang²,

Wei³

2023

Preprint

View full text Add to dashboard Cite

Data plays a crucial role in machine learning. However, in real-world applications, there are several problems with data, e.g., data are of low quality; a limited number of data points lead to under-fitting of the machine learning model; it is hard to access the data due to privacy, safety and regulatory concerns. Synthetic data generation offers a promising new avenue, as it can be shared and used in ways that real-world data cannot. This paper systematically reviews the existing works that leverage machine learning models for synthetic data generation. Specifically, we discuss the synthetic data generation works from several perspectives: (i) applications, including computer vision, speech, natural language, healthcare, and business; (ii) machine learning methods, particularly neural network architectures and deep generative models; (iii) privacy and fairness issue. In addition, we identify the challenges and opportunities in this emerging field and suggest future research directions.

show abstract

Section: Businessmentioning

confidence: 99%

“…Therefore, it is vital to investigate methods for generating financial datasets that incorporate the same properties as the "real data" while still respecting the privacy of the parties involved. [63].…”

Section: Businessmentioning

confidence: 99%

Machine Learning for Synthetic Data Generation: a Review

Lu¹,

Wang²,

Wei³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…A common interpretation is that as long as the real data remains in a secure environment during the generation of synthetic data, there is little to no risk to the original subjects. 103 As a consequence, the use of synthetic data can help prevent researchers from inadvertently using and possibly exposing patients identifiable data. Synthetic data can also lessen the controls imposed by Institutional Review Boards (IRBs) and based on international regulations by ensuring data is never mapped to real individuals.…”

Section: Legal Framework For Sharing Of Synthetic and Real Patient Datamentioning

confidence: 99%

medigan: a Python library of pretrained generative models for medical image synthesis

et al. 2023

View full text Add to dashboard Cite

Purpose: Deep learning has shown great promise as the backbone of clinical decision support systems. Synthetic data generated by generative models can enhance the performance and capabilities of data-hungry deep learning models. However, there is (1) limited availability of (synthetic) datasets and (2) generative models are complex to train, which hinders their adoption in research and clinical applications. To reduce this entry barrier, we explore generative model sharing to allow more researchers to access, generate, and benefit from synthetic data.Approach: We propose medigan, a one-stop shop for pretrained generative models implemented as an open-source framework-agnostic Python library. After gathering end-user requirements, design decisions based on usability, technical feasibility, and scalability are formulated. Subsequently, we implement medigan based on modular components for generative model (i) execution, (ii) visualization, (iii) search & ranking, and (iv) contribution. We integrate pretrained models with applications across modalities such as mammography, endoscopy, x-ray, and MRI. Results:The scalability and design of the library are demonstrated by its growing number of integrated and readily-usable pretrained generative models, which include 21 models utilizing nine different generative adversarial network architectures trained on 11 different datasets. We further analyze three medigan applications, which include (a) enabling community-wide sharing of restricted data, (b) investigating generative model evaluation metrics, and (c) improving clinical downstream tasks. In (b), we extract Fréchet inception distances (FID) demonstrating FID variability based on image normalization and radiology-specific feature extractors.Conclusion: medigan allows researchers and developers to create, increase, and domain-adapt their training data in just a few lines of code. Capable of enriching and accelerating the development of clinical machine learning models, we show medigan's viability as platform for generative model sharing. Our multimodel synthetic data experiments uncover standards for assessing and reporting metrics, such as FID, in image synthesis studies.

show abstract

“…The analysis of the performance of the algorithm requires a reasonable amount of empirical data. One of the best practices is to generate synthetic data to have a sound foundation for the data analytics, in order to verify the model of the data analysis [100]. Therefore, it is a feasible approach to generate a set of data that represents Business Processes that can be categorized as well-formed or with erroneous behavior.…”

Section: Model Checking For Dynamically Modified Business Processesmentioning

confidence: 99%

The Application of Directed Hyper-Graphs for Analysis of Models of Information Systems

Molnár

Benczúr

2022

Mathematics

View full text Add to dashboard Cite

Hyper-graphs offer the opportunity to formulate logical statements about their components, for example, using Horn clauses. Several models of Information Systems can be represented using hyper-graphs as the workflows, i.e., the business processes. During the modeling of Information Systems, many constraints should be maintained during the development process. The models of Information Systems are complex objects, for this reason, the analysis of algorithms and graph structures that can support the consistency and integrity of models is an essential issue. A set of interdependencies between models and components of architecture can be formulated by functional dependencies and can be investigated via algorithmic methods. Information Systems can be perceived as overarching documents that includes data collections; documents to be processed; and representations of business processes, activities, and services. Whe selecting and working out an appropriate method encoding of artifacts in Information Systems, the complex structure can be represented using hyper-graphs. This representation enables the application of various model-checking, verification, and validation tools that are based on formal approaches. This paper describes the proposed representations in different situations using hyper-graphs, moreover, the formal, algorithmic-based model-checking methods that are coupled with the representations. The model-checking methods are realized by algorithms that are grounded in graph-theoretical approaches and tailored to the specificity of hyper-graphs. Finally, the possible applications in a real-life enterprise environment are outlined.

show abstract

Utility Metrics for Evaluating Synthetic Health Data Generation Methods: Validation Study (Preprint)

Cited by 27 publications

References 30 publications

Machine Learning for Synthetic Data Generation: a Review

Machine Learning for Synthetic Data Generation: a Review

medigan: a Python library of pretrained generative models for medical image synthesis

The Application of Directed Hyper-Graphs for Analysis of Models of Information Systems

Contact Info

Product

Resources

About