A fast Branch-and-Bound algorithm for U-curve feature selection

Atashpaz-Gargari, Esmaeil; Reis, Marcelo S.; Braga-Neto, Ulisses; Barrera, Júnior; Dougherty, Edward R.

doi:10.1016/j.patcog.2017.08.013

Cited by 24 publications

(28 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Choosing the best set of features consists of comparing the minimum of all lattice chains. There are some NP-hard algorithms that find the absolute minimum [ 9 , 10 ]. There are also some heuristics that give approximate solutions such as Sequential Forward Selection (SFS) [ 11 ], which adds features progressively until it finds a local minimum, and Sequential Forward Floating Selection (SFFS) [ 11 ], which, at first, adds features, but after takes some of them out and adds others, trying to improve the first local minimum found.…”

Section: Introductionmentioning

confidence: 99%

Feature Selection based on the Local Lift Dependence Scale

Marcondes

Simonis

Barrera

2018

Entropy

Self Cite

View full text Add to dashboard Cite

This paper uses a classical approach to feature selection: minimization of a cost function applied on estimated joint distributions. However, in this new formulation, the optimization search space is extended. The original search space is the Boolean lattice of features sets (BLFS), while the extended one is a collection of Boolean lattices of ordered pairs (CBLOP), that is (features, associated value), indexed by the elements of the BLFS. In this approach, we may not only select the features that are most related to a variable Y, but also select the values of the features that most influence the variable or that are most prone to have a specific value of Y. A local formulation of Shannon's mutual information, which generalizes Shannon's original definition, is applied on a CBLOP to generate a multiple resolution scale for characterizing variable dependence, the Local Lift Dependence Scale (LLDS). The main contribution of this paper is to define and apply the LLDS to analyse local properties of joint distributions that are neglected by the classical Shannon's global measure in order to select features. This approach is applied to select features based on the dependence between: i-the performance of students on university entrance exams and on courses of their first semester in the university; ii-the congress representative party and his vote on different matters; iii-the cover type of terrains and several terrain properties.

show abstract

Section: Introductionmentioning

confidence: 99%

Feature Selection based on the Local Lift Dependence Scale

Marcondes

Simonis

Barrera

2018

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…As studied in Section 2.3.6, the U -curve problem is a search problem in a complete Boolean lattice (P(S), ⊆) embedded with a cost function c with the U -curve property. Moreover, it is known that there exist proposed optimal solutions for this problem such as [Ris et al, 2010], [Atashpaz-Gargari et al, 2013], [Reis, 2013] and [Atashpaz-Gargari et al, 2018]. Unfortunately, the U -curve problem is NP-hard so these approaches present exponential computational complexity.…”

Section: Methodsmentioning

confidence: 99%

“…In the literature, there are approaches that solve the U -curve problem and variations of this problem such as [Ris et al, 2010], [Atashpaz-Gargari et al, 2013], [Reis, 2013] and [Atashpaz-Gargari et al, 2018]. Furthermore, Reis et al [2017] present a practical framework for feature selection algorithms and cost functions, it was also used in this work as we will see later in Section 4.3.…”

Section: U-curve Phenomenonmentioning

confidence: 99%

“…Furthermore, search strategies can exploit the fact that out-of-sample errors taken from nested models present the U-curve phenomenon as shown earlier in this chapter. Some algorithms that deal exactly with this kind of problem can be seen in [Ris et al, 2010], [Reis, 2013], [Atashpaz-Gargari et al, 2013], [Atashpaz-Gargari et al, 2018].…”

Section: Generic Model Selector and Lattice Search Algorithmsmentioning

confidence: 99%

“…This event is known as peaking phenomenon [Hastie et al, 2009] (also known as U -curve phenomenon [Sima and Dougherty, 2008]). Although, we can find some learning methods in the literature as [Martins et al, 2006], [Ris et al, 2010], [Atashpaz-Gargari et al, 2013], [Reis, 2013], [Atashpaz-Gargari et al, 2018] that exploit the U -curve phenomenon to find appropriate hypothesis according to a training sample. We can improve the understanding on this subject in order to find more applications of U -curve algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Model selection for learning boolean hypothesis

Castro¹

View full text Add to dashboard Cite

show abstract

Feature Selection for Cotton Matter Classification

Zhao

Huang

Zhao

et al. 2019

Computer and Computing Technologies in Agriculture X

View full text Add to dashboard Cite

Feature selection are highly important to improve the classification accuracy of recognition systems for foreign matter in cotton. To address this problem, this paper presents six filter approaches of feature selection for obtaining the good feature combination with high classification accuracy and small size, and make comparisons using support vector machine and k-nearest neighbor classifier. The result shows that filter approach can efficiently find the good feature sets with high classification accuracy and small size, and the selected feature sets can effectively improve the performane of recognition system for foreign matter in cotton. The selected feature combination has smaller size and higher accuracy than original feature combination. It is important for developing the recognition systems for cotton matter using machine vision technology.

show abstract

A fast Branch-and-Bound algorithm for U-curve feature selection

Cited by 24 publications

References 13 publications

Feature Selection based on the Local Lift Dependence Scale

Feature Selection based on the Local Lift Dependence Scale

Model selection for learning boolean hypothesis

Feature Selection for Cotton Matter Classification

Contact Info

Product

Resources

About