Review of Small Data Learning Methods

Li, Xiali; Deng, Songting; Wang, Song; Lv, Zhengyu; Wu, Licheng

doi:10.1109/compsac.2018.10212

Cited by 6 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…patients with rare disseises), the sample is small comparing to number of features like in genetics or biomarkers detection, sampling, there is a lot of noisy or missing data or measurements are extremely expensive or data imbalanced meaning that the size of one class in a data set has very few objects. [28][29][30][31][32] Using machine learning on small size datasets present a problem, because, in general, the 'power' of machine learning in recognising patterns is proportional to the size of the dataset, the smaller the dataset, less powerful and less accurate are the machine learning algorithms. Despite the commonality of the above problem and various approaches to solve it we didn't found any holistic studies concerned with this important area of the machine learning field.…”

Section: Introductionmentioning

confidence: 99%

Machine learning on small size samples: A synthetic knowledge synthesis

Kokol

Kokol²,

Zagoranski³

2022

Science Progress

119

View full text Add to dashboard Cite

Machine Learning is an increasingly important technology dealing with the growing complexity of the digitalised world. Despite the fact, that we live in a ‘Big data’ world where, almost ‘everything’ is digitally stored, there are many real-world situations, where researchers are still faced with small data samples. The present bibliometric knowledge synthesis study aims to answer the research question ‘What is the small data problem in machine learning and how it is solved?’ The analysis a positive trend in the number of research publications and substantial growth of the research community, indicating that the research field is reaching maturity. Most productive countries are China, United States and United Kingdom. Despite notable international cooperation, the regional concentration of research literature production in economically more developed countries was observed. Thematic analysis identified four research themes. The themes are concerned with to dimension reduction in complex big data analysis, data augmentation techniques in deep learning, data mining and statistical learning on small datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

Machine learning on small size samples: A synthetic knowledge synthesis

Kokol

Kokol²,

Zagoranski³

2022

Science Progress

119

View full text Add to dashboard Cite

show abstract

“…As expected, computer simulation with a small training sample was limited due to the use of the simplest ANNs based on a single-layer perceptron, resulting in the need to synthesize several models [ 64 , 65 ]. Therefore, we needed assessment the adequacy of the obtained models, which was a somewhat subjective task.…”

Section: Discussionmentioning

confidence: 99%

Optimizing Ultrasonic Welding Parameters for Multilayer Lap Joints of PEEK and Carbon Fibers by Neural Network Simulation

2022

View full text Add to dashboard Cite

The aim of this study is to substantiate the use machine learning methods to optimize a combination of ultrasonic welding (USW) parameters for manufacturing of multilayer lap joints consisting of two outer PEEK layers, a middle prepreg of unidirectional carbon fibers (CFs), and two energy directors (EDs) between them. As a result, a mathematical problem associated with determining the optimal combination of technological parameters was formulated for the formation of USW joints possessing improved functional properties. In addition, a methodology was proposed to analyze the mechanical properties of USW joints based on neural network simulation (NNS). Experiments were performed, and threshold values of the optimality conditions for the USW parameters were chosen. Accordingly, NNS was carried out to determine the parameter ranges, showing that the developed optimality condition was insufficient and required correction, taking into account other significant structural characteristics of the formed USW joints. The NNS study enabled specification of an extra area of USW parameters that were not previously considered optimal when designing the experiment. The NNS-predicted USW mode (P = 1.5 atm, t = 800 ms, and τ = 1500 ms) ensured formation of a lap joint with the required mechanical and structural properties (σUTS = 80.5 MPa, ε = 4.2 mm, A = 273 N·m, and Δh = 0.30 mm).

show abstract

“…There has been a huge amount of work done on the use of bootstrap and its theoretical limitations. The interested reader is referred to several general overviews [20][21][22][23][24][25][26][27][28][29][30][31][32], to the discussion on the evaluation of the confidence intervals [26,27,29,33,34], to the discussion on how to remove the iid hypothesis [26,35], and the application and limitations in various fields, from medicine to nuclear physics and geochemistry [5,[26][27][28][29][36][37][38][39][40][41]]. An analysis of the statistical theory on which the bootstrap method is based, goes beyond this tutorial and will not be covered here.…”

Section: Bootstrapmentioning

confidence: 99%

Estimating Neural Network’s Performance with Bootstrap: A Tutorial

Michelucci

Venturini

2021

MAKE

View full text Add to dashboard Cite

Neural networks present characteristics where the results are strongly dependent on the training data, the weight initialisation, and the hyperparameters chosen. The determination of the distribution of a statistical estimator, as the Mean Squared Error (MSE) or the accuracy, is fundamental to evaluate the performance of a neural network model (NNM). For many machine learning models, as linear regression, it is possible to analytically obtain information as variance or confidence intervals on the results. Neural networks present the difficulty of not being analytically tractable due to their complexity. Therefore, it is impossible to easily estimate distributions of statistical estimators. When estimating the global performance of an NNM by estimating the MSE in a regression problem, for example, it is important to know the variance of the MSE. Bootstrap is one of the most important resampling techniques to estimate averages and variances, between other properties, of statistical estimators. In this tutorial, the application of resampling techniques (including bootstrap) to the evaluation of neural networks’ performance is explained from both a theoretical and practical point of view. The pseudo-code of the algorithms is provided to facilitate their implementation. Computational aspects, as the training time, are discussed, since resampling techniques always require simulations to be run many thousands of times and, therefore, are computationally intensive. A specific version of the bootstrap algorithm is presented that allows the estimation of the distribution of a statistical estimator when dealing with an NNM in a computationally effective way. Finally, algorithms are compared on both synthetically generated and real data to demonstrate their performance.

show abstract

Review of Small Data Learning Methods

Cited by 6 publications

References 17 publications

Machine learning on small size samples: A synthetic knowledge synthesis

Machine learning on small size samples: A synthetic knowledge synthesis

Optimizing Ultrasonic Welding Parameters for Multilayer Lap Joints of PEEK and Carbon Fibers by Neural Network Simulation

Estimating Neural Network’s Performance with Bootstrap: A Tutorial

Contact Info

Product

Resources

About