Algorithmic Splitting: A Method for Dataset Preparation

Abstract

Datasets that appear in publications are curated and split into training, testing, and validation sub-datasets by domain experts. Consequently, machine learning models typically perform well on such split-by-hand datasets, whereas preparing real-world datasets into curated splits, i.e., training, testing, and validation sub-datasets, require extensive effort. Usually, random repetitive splitting is carried out, practiced, and evaluated until a better score is reached on the evaluation metrics. In this paper, a novel algorithmic method is proposed for splitting datasets for machine learning models. Algorithmic Splitting utilizes deterministic dimension reduction and density-based clustering techniques for estimating the data distribution. Afterward, splitting the dataset is carried out based on the estimated data distribution. Our objective is to achieve evenly representative splits of a given dataset in a standard and algorithmic way that reduces the perplexity of random splitting using the thorough splitting method. Experiments demonstrate the potential of Algorithmic Splitting through qualitative and quantitative evaluation of MNIST, Fashion MNIST, CIFAR-10, SmallNORB, and Shapes3D.

Averaging Strategy for Interpretable Machine Learning on Small Datasets to Understand Element Uptake after Seed Nanotreatment

“…Therefore, relying on a single dataset split to create a model may pose challenges to the model’s representativeness and reliability. The performance of models established on small datasets would be obviously impacted by dataset size, split ratio, and split strategy, − and machine learning may not capture the full range of features and patterns present in the given training set. This underscores the importance of evaluating model performance and interpreting what a model learned across various splits.…”

Disentangling Shallow Subsidence Sources by Data Assimilation in a Reclaimed Urbanized Coastal Plain, South Flevoland Polder, the Netherlands

“…The practice of dimensionality reduction followed by clustering is common for large input data and has been applied to SAR data sets (Van de Kerkhof et al., 2020), and for a wide range of other data types (Fernández Llamas et al., 2019; R. Harrison et al., 2019; Kahloot & Ekler, 2019). T‐SNE is a dimensionality reduction method that can group similarly behaving time series of height measurements of the different reflection points (Van der Maaten & Hinton, 2008).…”

MLOps critical success factors - A systematic literature review

“…To determine the quality of the trained model, we use the held-out validation datasets. You may learn a lot about the model's performance from its validation [40]. Absolute error and mean absolute error are two metrics used to measure the quality of the model, which allows for testing and comparison of multiple models [41,42].…”