Clustering and Network Analysis for the Embedding Spaces of Sentences and Sub-Sentences

An, Yuan; Kalinowski, Alexander; Greenberg, Jane

doi:10.1109/idsta53674.2021.9660801

Cited by 2 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gupta et al [48] finds naive clustering of high-dimensional contextual BERT embeddings produces deficient results. An et al [47] reinforces this theory by surveying embedding model's clustering ability using Spatial Histograms, which found high-dimensional dynamic SBERT to be least able to cluster when compared to low-dimensional static GloVe models. We argue that by reducing embedding dimensionality and therefore clustering complexity, an increase in clustering performance can be observed.…”

Section: Dimensionality Reduction Algorithms Selectionmentioning

confidence: 85%

“…Bodrunova et al [46] uses Hierarchal Agglomerative Clustering to group Universal Sentence Encoder embeddings, with the addition of Markov Stopping moment to choose optimal number of clusters. Similarly, An et al [47] uses a range of both static and dynamic Sentence Embeddings, which are clustered with k-means at into a specified number of groups by Spatial Histogram analysis. Gupta et al [48] makes the insight that lowering embedding dimensionality previous to clustering using an Encoder-Decoder Model, improves clustering performance.…”

Section: Semantic Clustering Of Variables Into Domainsmentioning

confidence: 99%

“…An extension of Prim's algorithm is used to construct a Minimum Spanning Tree, given density-based groupings, in order to extract the HDBSCAN hierarchy. An optimisation method is used to extract a globally optimal solution from the hierarchal structure [47][48][49][50][51].…”

Section: Clustering Algorithms Selectionmentioning

confidence: 99%

See 2 more Smart Citations

Pretrained Language Models for Semantics-Aware Data Harmonisation of Observational Clinical Studies in the Era of Big Data

Dylag,

Zlatev,

Boniface

2024

Preprint

View full text Add to dashboard Cite

BackgroundIn clinical research, there is a strong drive to leverage big data from population cohort studies and routine electronic healthcare records to design new interventions, improve health outcomes and increase efficiency of healthcare delivery. Yet, realising these potential demands requires substantial efforts in harmonising source datasets and curating study data, which currently relies on costly and time-consuming manual and labour-intensive methods.ObjectivesWe evaluate the applicability of AI methods for natural language processing (NLP) and unsupervised machine learning (ML) to the challenges of big data semantic harmonisation and curation. Our aim is to establish an efficient and robust technological foundation for the development of automated tools supporting data curation of large clinical datasets.MethodsWe assess NLP and unsupervised ML algorithms and propose two pipelines for automated semantic harmonisation: a pipeline for semantics-aware search for domain relevant variables and a pipeline for clustering of semantically similar variables. We evaluate pipeline performance using 94,037 textual variable descriptions from the English Longitudinal Study of Ageing (ELSA) database.ResultsWe observe high accuracy of our Semantic Search pipeline with an AUC of 0.899 (SD=0.056). Our Semantic Clustering pipeline achieves a V-measure of 0.237 (SD=0.157), which is on par with leading implementations in other relevant domains. Automation can significantly accelerate the process of dataset harmonization. Manual labelling was performed at a speed of 2.1 descriptions per minute, with our automated labelling increasing speed to 245 descriptions per minute.ConclusionsOur study findings underscore the potential of AI technologies, such as NLP and unsupervised ML, in automating the harmonisation and curation of big data for clinical research. By establishing a robust technological foundation, we pave the way for the development of automated tools that streamline the process, enabling health data scientists to leverage big data more efficiently and effectively in their studies, accelerating insights from data for clinical benefit.

show abstract

Section: Dimensionality Reduction Algorithms Selectionmentioning

confidence: 85%

Section: Semantic Clustering Of Variables Into Domainsmentioning

confidence: 99%

See 1 more Smart Citation

Pretrained Language Models for Semantics-Aware Data Harmonisation of Observational Clinical Studies in the Era of Big Data

Dylag,

Zlatev,

Boniface

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…In total, ontology-based applications were observed in 14 of the selected papers. [28][29][30][32][33][34][35]37,38,40,41,[50][51][52] These applications include the reuse, combination, and extension of existing ontologies and semantic resources (Figure 3b,i,ii). An example is given by Li et al [33] demonstrating the extension of two nanotechnology ontologies with new concepts and axioms in a two-step procedure.…”

Section: Ontology Applicationmentioning

confidence: 99%

“…An additional important application is the mapping or alignment of one ontology to another (Figure 3b,iii). An et al [50] illustrated this by introducing the two-component system OTMapOnto. In this process, terms in different ontologies are identified and mapped with each other using ontology embedding and an optimal transport approach.…”

Section: Ontology Applicationmentioning

confidence: 99%

The Intersection Between Semantic Web and Materials Science

Valdestilhas

Bayerlein

Torres

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

The application and benefits of Semantic Web Technologies (SWT) for managing, sharing, and (re‐)using of research data are demonstrated in implementations in the field of Materials Science and Engineering (MSE). However, a compilation and classification are needed to fully recognize the scattered published works with its unique added values. Here, the primary use of SWT at the interface with MSE is identified using specifically created categories. This overview highlights promising opportunities for the application of SWT to MSE, such as enhancing the quality of experimental processes, enriching data with contextual information in knowledge graphs, or using ontologies to perform specific queries on semantically structured data. While interdisciplinary work between the two fields is still in its early stages, a great need is identified to facilitate access for nonexperts and develop and provide user‐friendly tools and workflows. The full potential of SWT can best be achieved in the long term by the broad acceptance and active participation of the MSE community. In perspective, these technological solutions will advance the field of MSE by making data FAIR. Data‐driven approaches will benefit from these data structures and their connections to catalyze knowledge generation in MSE.

show abstract

Clustering and Network Analysis for the Embedding Spaces of Sentences and Sub-Sentences

Cited by 2 publications

References 24 publications

Pretrained Language Models for Semantics-Aware Data Harmonisation of Observational Clinical Studies in the Era of Big Data

Pretrained Language Models for Semantics-Aware Data Harmonisation of Observational Clinical Studies in the Era of Big Data

The Intersection Between Semantic Web and Materials Science

Contact Info

Product

Resources

About