Open-source python module for automated preprocessing of near infrared spectroscopic data

Knee ligaments and tendons play an important role in stabilizing and controlling the motions of the knee. Injuries to the ligaments can lead to abnormal mechanical loading of the other supporting tissues (e.g., cartilage and meniscus) and even osteoarthritis. While the condition of knee ligaments can be examined during arthroscopic repair procedures, the arthroscopic evaluation suffers from subjectivity and poor repeatability. Near infrared spectroscopy (NIRS) is capable of non-destructively quantifying the composition and structure of collagen-rich connective tissues, such as articular cartilage and meniscus. Despite the similarities, NIRS-based evaluation of ligament composition has not been previously attempted. In this study, ligaments and patellar tendon of ten bovine stifle joints were measured with NIRS, followed by chemical and histological reference analysis. The relationship between the reference properties of the tissue and NIR spectra was investigated using partial least squares regression. NIRS was found to be sensitive towards the water (R2CV = .65) and collagen (R2CV = .57) contents, while elastin, proteoglycans, and the internal crimp structure remained undetectable. As collagen largely determines the mechanical response of ligaments, we conclude that NIRS demonstrates potential for quantitative evaluation of knee ligaments.

show abstract

“…Numerous (N = 96) NIRS preprocessing options were tested using the open-source Python module [ 35 ] (see section NIRS Preprocessing for details).…”

Section: Methodsmentioning

confidence: 99%

“…Each sample was considered independent and ligament type was not included in the model. All analysis was performed in Python using [ 36 ], [ 37 ], [ 38 ], and [ 35 ] packages.…”

Section: Methodsmentioning

confidence: 99%

Near infrared spectroscopic evaluation of biochemical and crimp properties of knee joint ligaments and patellar tendon

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…The brand-specific approach maximizes accuracy at the cost of robustness. These models are useful for product authentication ,− or manufacturing process control, ,, but they are not optimal for detection of SFPs because they tend to give false alarms for formulations or brands outside their training set. , The reliance on libraries of authentic products is a major barrier to the SFP use case. ,,, Problems can arise when manufacturers use different excipients, forms of the API (e.g., different crystal polymorphs and particle sizes), and formulation technologies (e.g., coated vs noncoated tablets) in different brands of the same product, or even in different batches of the same brand. These formulation differences can create differences in the NIR spectra that result in identification of a good quality product as an SFP.…”

Section: Introductionmentioning

confidence: 99%

“…39,40 The reliance on libraries of authentic products is a major barrier to the SFP use case. 19,28,33,34 Problems can arise when manufacturers use different excipients, forms of the API (e.g., different crystal polymorphs and particle sizes), and formulation technologies (e.g., coated vs noncoated tablets) in different brands of the same product, or even in different batches of the same brand. These formulation differences can create differences in the NIR spectra that result in identification of a good quality product as an SFP.…”

Section: Introductionmentioning

confidence: 99%

Discrimination of Substandard and Falsified Formulations from Genuine Pharmaceuticals Using NIR Spectra and Machine Learning

et al. 2022

View full text Add to dashboard Cite

Near-infrared (NIR) spectroscopy is a promising technique for field identification of substandard and falsified drugs because it is portable, rapid, nondestructive, and can differentiate many formulated pharmaceutical products. Portable NIR spectrometers rely heavily on chemometric analyses based on libraries of NIR spectra from authentic pharmaceutical samples. However, it is difficult to build comprehensive product libraries in many low- and middle-income countries due to the large numbers of manufacturers who supply these markets, frequent unreported changes in materials sourcing and product formulation by the manufacturers, and general lack of cooperation in providing authentic samples. In this work, we show that a simple library of lab-formulated binary mixtures of an active pharmaceutical ingredient (API) with two diluents gave good performance on field screening tasks, such as discriminating substandard and falsified formulations of the API. Six data analysis models, including principal component analysis and support-vector machine classification and regression methods and convolutional neural networks, were trained on binary mixtures of acetaminophen with either lactose or ascorbic acid. While the models all performed strongly in cross-validation (on formulations similar to their training set), they individually showed poor robustness for formulations outside the training set. However, a predictive algorithm based on the six models, trained only on binary samples, accurately predicts whether the correct amount of acetaminophen is present in ternary mixtures, genuine acetaminophen formulations, adulterated acetaminophen formulations, and falsified formulations containing substitute APIs. This data analytics approach may extend the utility of NIR spectrometers for analysis of pharmaceuticals in low-resource settings.

show abstract

“…These techniques include Smoothing, Scatter Correction, Trimming, Clipping, Resampling, and Derivatives. The order of pre-processing operations applied can affect the performance of the model [14]. Smoothing aims to smooth the spectral and help remove noise.…”

Section: Introductionmentioning

confidence: 99%

Effects of Oversampling Smote and Spectral Transformations in the Classification of Mango Cultivars Using Near-Infrared Spectroscopy

Khumaidi¹,

Raafi'udin²

2022

International Journal on Advanced Science, Engineering and Information Technology

View full text Add to dashboard Cite

Near-Infrared spectroscopy (NIR) is a non-destructive analytical technique that can provide chemical and structural information on samples in a speedy and accurate time. NIR has a wavelength of 750-2500 nm. However, the absorbance bands of the NIR spectrum are often broad, non-specific, and overlapping. NIR spectrum analysis requires a multivariate method which is very subjective to noise arising from instrumentation. There is no standard protocol in modeling for classification and prediction using NIR spectra. Several models have been developed with and without pre-processing techniques. The SMOTE technique can improve the model to predict all class responses accurately. This research contributes to creating a multiclass classification model for grouping mango cultivars by finding the best pre-processing technique and using SMOTE oversampling. The results of the four test scenarios on the model's performance built using the Support Vector Machine (SVM) that the best model is obtained using spectral transformations with LSNV and CLIP operations with 100% accuracy, precision, and recall values. The Decision Tree (DT) has the performance results in 100% model was obtained by using spectral transformation with LSNV, CLIP and SAVGOL operations with parameters {'deriv_order': 0,1, 2, 'filter_win': 11, 13, 'poly_order': 3}. Using SMOTE has better accuracy than without pre-processing, with an accuracy of 92% on SVM and 94% on DT. In comparison, the combination of SMOTE and Spectral Transformation gives classification results for SVM and DT with the same accuracy of 96%, better than using SMOTE only.

show abstract

Open-source python module for automated preprocessing of near infrared spectroscopic data

Cited by 51 publications

References 39 publications

Near infrared spectroscopic evaluation of biochemical and crimp properties of knee joint ligaments and patellar tendon

Near infrared spectroscopic evaluation of biochemical and crimp properties of knee joint ligaments and patellar tendon

Discrimination of Substandard and Falsified Formulations from Genuine Pharmaceuticals Using NIR Spectra and Machine Learning

Effects of Oversampling Smote and Spectral Transformations in the Classification of Mango Cultivars Using Near-Infrared Spectroscopy

Contact Info

Product

Resources

About