A Multi-layer Approach for Data Cleaning in the Healthcare Domain

Mavrogiorgos, Konstantinos; Kiourtis, Athanasios; Mavrogiorgou, Argyro; Kleftakis, Spyridon; Kyriazis, Dimosthenis

doi:10.1145/3512850.3512856

Cited by 8 publications

(2 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our work, we utilize users' ratings from the MovieLens dataset to construct a recommendation system. Initially, we merge all the rating information for an individual user into the required input format for our transformer model [44]. We then construct a vocabulary for movie IDs and user IDs and create sequences of user interactions in chronological order.…”

Section: Data Processing and Sequence Creationmentioning

confidence: 99%

Enhancing Sequence Movie Recommendation System Using Deep Learning and KMeans

Siet,

Peng,

Ilkhomjon

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

A flood of information has occurred, making it challenging for people to find and filter their favorite items. Recommendation systems (RSs) have emerged as a solution to this problem; however, traditional Appenrecommendation systems, including collaborative filtering, and content-based filtering, face significant challenges such as data scalability, data scarcity, and the cold-start problem, all of which require advanced solutions. Therefore, we propose a ranking and enhancing sequence movie recommendation system that utilizes the combination model of deep learning to resolve the existing issues. To mitigate these challenges, we design an RSs model that utilizes user information (age, gender, occupation) to analyze new users and match them with others who have similar preferences. Initially, we construct sequences of user behavior to effectively predict the potential next target movie of users. We then incorporate user information and movie sequence embeddings as input features to reduce the dimensionality, before feeding them into a transformer architecture and multilayer perceptron (MLP). Our model integrates a transformer layer with positional encoding for user behavior sequences and multi-head attention mechanisms to enhance prediction accuracy. Furthermore, the system applies KMeans clustering to movie genre embeddings, grouping similar movies and integrating this clustering information with predicted ratings to ensure diversity in the personalized recommendations for target users. Evaluating our model on two MovieLens datasets (100 Kand 1 M) demonstrated significant improvements, achieving RMSE, MAE, precision, recall, and F1 scores of 1.0756, 0.8741, 0.5516, 0.3260, and 0.4098 for the 100 K dataset, and 0.9927, 0.8007, 0.5838, 0.4723, and 0.5222 for the 1 M dataset, respectively. This approach not only effectively mitigates cold-start and scalability issues but also surpasses baseline techniques in Top-N item recommendations, highlighting its efficacy in the contemporary environment of abundant data.

show abstract

Section: Data Processing and Sequence Creationmentioning

confidence: 99%

Enhancing Sequence Movie Recommendation System Using Deep Learning and KMeans

Siet,

Peng,

Ilkhomjon

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Data preprocessing plays a pivotal role in the realm of AI applications, acting as the essential foundation upon which accurate and efficient models are built. This crucial step involves cleaning, transforming, and organizing raw data to ensure its quality and relevance, as illustrated in [21,22]. By identifying and rectifying errors, handling missing values, and standardizing formats, data preprocessing enhances the overall integrity of the dataset.…”

Section: Data Preprocessingmentioning

confidence: 99%

Risk Prediction Model for Chronic Kidney Disease in Thailand Using Artificial Intelligence and SHAP

Tsai,

Lojanapiwat,

Chang

et al. 2023

Diagnostics

View full text Add to dashboard Cite

Chronic kidney disease (CKD) is a multifactorial, complex condition that requires proper management to slow its progression. In Thailand, 11.6 million people (17.5%) have CKD, with 5.7 million (8.6%) in the advanced stages and >100,000 requiring hemodialysis (2020 report). This study aimed to develop a risk prediction model for CKD in Thailand. Data from 17,100 patients were collected to screen for 14 independent variables selected as risk factors, using the IBK, Random Tree, Decision Table, J48, and Random Forest models to train the predictive models. In addition, we address the unbalanced category issue using the synthetic minority oversampling technique (SMOTE). The indicators of performance include classification accuracy, sensitivity, specificity, and precision. This study achieved an accuracy rate of 92.1% with the top-performing Random Forest model. Moreover, our empirical findings substantiate previous research through highlighting the significance of serum albumin, blood urea nitrogen, age, direct bilirubin, and glucose. Furthermore, this study used the SHapley Additive exPlanations approach to analyze the attributes of the top six critical factors and then extended the comparison to include dual-attribute factors. Finally, our proposed machine learning technique can be used to evaluate the effectiveness of these risk factors and assist in the development of future personalized treatment.

show abstract