Gradient Boosting with Piece-Wise Linear Regression Trees

Shi, Yu; Li, Jian; Li, Zhize

doi:10.24963/ijcai.2019/476

Cited by 26 publications

(15 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For BERT-m7G, we explore six different classifiers in the first place to choose the base classifiers of the stacking ensemble classifier algorithm, including light gradient boosting machine (LightGBM) [ 31 ], support vector machine (SVM) [ 32 ], random forest (RF) [ 33 ], naive Bayes (NB) classifier [ 34 , 35 ], logistic regression (LR) [ 36 , 37 ], and gradient boosting decision tree (GBDT) [ 38 , 39 ]. Subsequently, LightGBM, SVM, and LR are selected as optimum combination of the base classifier because when the optimal feature subset is used as the input features of classifiers, their prediction accuracy values are higher than those of other machine learning classifiers.…”

Section: Methodsmentioning

confidence: 99%

BERT-m7G: A Transformer Architecture Based on BERT and Stacking Ensemble to Identify RNA N7-Methylguanosine Sites from Sequence Information

Zhang

Qin

Liu

et al. 2021

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

As one of the most prevalent posttranscriptional modifications of RNA, N7-methylguanosine (m7G) plays an essential role in the regulation of gene expression. Accurate identification of m7G sites in the transcriptome is invaluable for better revealing their potential functional mechanisms. Although high-throughput experimental methods can locate m7G sites precisely, they are overpriced and time-consuming. Hence, it is imperative to design an efficient computational method that can accurately identify the m7G sites. In this study, we propose a novel method via incorporating BERT-based multilingual model in bioinformatics to represent the information of RNA sequences. Firstly, we treat RNA sequences as natural sentences and then employ bidirectional encoder representations from transformers (BERT) model to transform them into fixed-length numerical matrices. Secondly, a feature selection scheme based on the elastic net method is constructed to eliminate redundant features and retain important features. Finally, the selected feature subset is input into a stacking ensemble classifier to predict m7G sites, and the hyperparameters of the classifier are tuned with tree-structured Parzen estimator (TPE) approach. By 10-fold cross-validation, the performance of BERT-m7G is measured with an ACC of 95.48% and an MCC of 0.9100. The experimental results indicate that the proposed method significantly outperforms state-of-the-art prediction methods in the identification of m7G modifications.

show abstract

Section: Methodsmentioning

confidence: 99%

BERT-m7G: A Transformer Architecture Based on BERT and Stacking Ensemble to Identify RNA N7-Methylguanosine Sites from Sequence Information

Zhang

Qin

Liu

et al. 2021

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

show abstract

“…classifiers, e.g., k-nearest neighbor (KNN) [61], support vector machines (SVM) [62], random forest (RF) [63], gradient boosting decision tree (GBDT) [64], Naï ve Bayes classifier (NB) [65], logistic regression (LR) [66], light gradient boosting machine (LightGBM) [67], extreme gradient boosting (XGBoost) [54], and adaptive boosting (AdaBoost) [68]. Step…”

Section: Stacked Ensemble Classifiermentioning

confidence: 99%

StackPDB: Predicting DNA-binding proteins based on XGB-RFE feature optimization and stacked ensemble classifier

Zhang

Liu

Wang

et al. 2021

Applied Soft Computing

View full text Add to dashboard Cite

DNA binding proteins (DBPs) not only play an important role in all aspects of genetic activities such as DNA replication, recombination, repair, and modification but also are used as key components of antibiotics, steroids, and anticancer drugs in the field of drug discovery. Identifying DBPs becomes one of the most challenging problems in the domain of proteomics research.Considering the high-priced and inefficient of the experimental method, constructing a detailed DBPs prediction model becomes an urgent problem for researchers. In this paper, we propose a stacked ensemble classifier based method for predicting DBPs called StackPDB. Firstly, pseudo amino acid composition (PseAAC), pseudo position-specific scoring matrix (PsePSSM), position-specific scoring matrix-transition probability composition (PSSM-TPC), evolutionary distance transformation (EDT), and residue probing transformation (RPT) are applied to extract protein sequence features. Secondly, extreme gradient boosting-recursive feature elimination (XGB-RFE) is employed to gain an excellent feature subset. Finally, the best features are applied to the stacked ensemble classifier composed of XGBoost, LightGBM, and SVM to construct StackPDB. After applying leave-one-out cross-validation (LOOCV), StackPDB obtains high ACC and MCC on PDB1075，93.44% and 0.8687, respectively. Besides, the ACC of the independent test datasets PDB186 and PDB180 are 84.41% and 90.00%, respectively. The MCC of the independent test datasets PDB186 and PDB180 are 0.6882 and 0.7997, respectively. The results on the training dataset and the independent test dataset show that StackPDB has a great predictive  Corresponding author.

show abstract

“…The best split will be found after sweeping over all features and thresholds. While this method is widely adopted [43], [44], it has prohibitively high computational complexity because we need to solve a large number of LS problems.…”

Section: B Binary-split Alternate Minimization Algorithmmentioning

confidence: 99%

Boosted Model Tree-Based Behavioral Modeling for Digital Predistortion of RF Power Amplifiers

Wang

Pang

et al. 2021

IEEE Trans. Microwave Theory Techn.

View full text Add to dashboard Cite

In this article, we propose a new behavioral modeling approach, called boosted model tree, to characterize and compensate for the complex nonlinear distortions induced by wideband high-efficiency radio frequency power amplifiers. With the proposed model, the input data are classified into different zones by decision trees and each zone is assigned separate submodels. We also employ a model boosting technique to build multiple parallel tree structures that jointly model the desired nonlinear behavior. By designing dedicated optimization procedures, both tree structures and submodel coefficients can be efficiently identified. It is demonstrated that the combination of piecewise and parallel structures provides a powerful and hardware-efficient way to model nonlinear memory effect and cross terms. Based on the experimental results, the proposed method can achieve improved linearization performance with low hardware complexity under challenging wideband predistortion scenarios.

show abstract

Gradient Boosting with Piece-Wise Linear Regression Trees

Cited by 26 publications

References 1 publication

BERT-m7G: A Transformer Architecture Based on BERT and Stacking Ensemble to Identify RNA N7-Methylguanosine Sites from Sequence Information

BERT-m7G: A Transformer Architecture Based on BERT and Stacking Ensemble to Identify RNA N7-Methylguanosine Sites from Sequence Information

StackPDB: Predicting DNA-binding proteins based on XGB-RFE feature optimization and stacked ensemble classifier

Boosted Model Tree-Based Behavioral Modeling for Digital Predistortion of RF Power Amplifiers

Contact Info

Product

Resources

About