A Branch and Bound Algorithm for Feature Subset Selection

Narendra,; Fukunaga,

doi:10.1109/tc.1977.1674939

Cited by 1,068 publications

(506 citation statements)

References 6 publications

Supporting

Mentioning

463

Contrasting

Unclassified

Order By: Relevance

“…Four groups of experiments have been made on SIMPLIcity dataset: one with all the features without selection, the second with features selected with filter methods, such as fisher filter [23] and principal component analysis (PCA) [26], the third with features selected using a wrapper method, such as SFS, and the last with the best features selected by ESFS. Five types of one step global classifiers are tested: Multi-layer Perceptron (Neural Network, marked as MP in the following text), Decision Tree (C4.5), Linear Discriminant Analysis (LDA), K-Nearest Neighbors (K-NN), and multiclass SVM (C-SVC).…”

Section: Resultsmentioning

confidence: 99%

Image Categorization Using ESFS: A New Embedded Feature Selection Method Based on SFS

Fu¹,

Xiao²,

Dellandréa³

et al. 2009

Advanced Concepts for Intelligent Vision Systems

View full text Add to dashboard Cite

Abstract. Feature subset selection is an important subject when training classifiers in Machine Learning (ML) problems. Too many input features in a ML problem may lead to the so-called "curse of dimensionality", which describes the fact that the complexity of the classifier parameters adjustment during training increases exponentially with the number of features. Thus, ML algorithms are known to suffer from important decrease of the prediction accuracy when faced with many features that are not necessary. In this paper, we introduce a novel embedded feature selection method, called ESFS, which is inspired from the wrapper method SFS since it relies on the simple principle to add incrementally most relevant features. Its originality concerns the use of mass functions from the evidence theory that allows to merge elegantly the information carried by features, in an embedded way, and so leading to a lower computational cost than original SFS. This approach has successfully been applied to the domain of image categorization and has shown its effectiveness through the comparison with other feature selection methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Image Categorization Using ESFS: A New Embedded Feature Selection Method Based on SFS

Fu¹,

Xiao²,

Dellandréa³

et al. 2009

Advanced Concepts for Intelligent Vision Systems

View full text Add to dashboard Cite

show abstract

“…However, several other feature selection or extraction algorithms leading to dimensionality reduction could not be evaluated due to long computation times required by these methods for the texture feature vector datasets used in this study. In the future, these algorithms, such as Exhaustive Selection (Jain and Zongker, 1997) and Branch and Bound method (Narendra and Fukunaga, 1977) can be applied to the existing feature vector datasets provided that they can be carried out within limits of computational feasibility.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Automated labelling of cancer textures in colorectal histopathology slides using quasi-supervised learning

Onder¹,

Sarıoğlu²,

Karaçali³

2013

Micron

View full text Add to dashboard Cite

a b s t r a c tQuasi-supervised learning is a statistical learning algorithm that contrasts two datasets by computing estimate for the posterior probability of each sample in either dataset. This method has not been applied to histopathological images before. The purpose of this study is to evaluate the performance of the method to identify colorectal tissues with or without adenocarcinoma. Light microscopic digital images from histopathological sections were obtained from 30 colorectal radical surgery materials including adenocarcinoma and non-neoplastic regions. The texture features were extracted by using local histograms and co-occurrence matrices. The quasi-supervised learning algorithm operates on two datasets, one containing samples of normal tissues labelled only indirectly, and the other containing an unlabeled collection of samples of both normal and cancer tissues. As such, the algorithm eliminates the need for manually labelled samples of normal and cancer tissues for conventional supervised learning and significantly reduces the expert intervention. Several texture feature vector datasets corresponding to different extraction parameters were tested within the proposed framework. The Independent Component Analysis dimensionality reduction approach was also identified as the one improving the labelling performance evaluated in this series. In this series, the proposed method was applied to the dataset of 22,080 vectors with reduced dimensionality 119 from 132. Regions containing cancer tissue could be identified accurately having false and true positive rates up to 19% and 88% respectively without using manually labelled ground-truth datasets in a quasi-supervised strategy. The resulting labelling performances were compared to that of a conventional powerful supervised classifier using manually labelled ground-truth data. The supervised classifier results were calculated as 3.5% and 95% for the same case. The results in this series in comparison with the benchmark classifier, suggest that quasi-supervised image texture labelling may be a useful method in the analysis and classification of pathological slides but further study is required to improve the results.

show abstract

“…These problems include approximate nearest neighbor search [2], Gaussian summation [39], particle smoothing [33], Gaussian process regression [59], clustering [34], feature subset selection [52], and mixture model training [48]. More recently, Gray and Moore proposed using a second tree for problems with large query sets [25], such as all-nearest-neighbors and density estimation [26].…”

Section: Speedups Via Treesmentioning

confidence: 99%

Dual‐tree fast exact max‐kernel search

Curtin

Ram

2014

Statistical Analysis

View full text Add to dashboard Cite

The problem of max-kernel search arises everywhere: given a query point pq, a set of reference objects Sr and some kernel K, find arg max pr ∈Sr K(pq, pr). Max-kernel search is ubiquitous and appears in countless domains of science, thanks to the wide applicability of kernels. A few domains include image matching, information retrieval, bio-informatics, similarity search, and collaborative filtering (to name just a few). However, there are no generalized techniques for efficiently solving max-kernel search. This paper presents a single-tree algorithm called single-tree FastMKS which returns the max-kernel solution for a single query point in provably O(log N ) time (where N is the number of reference objects), and also a dual-tree algorithm (dual-tree FastMKS) which is useful for max-kernel search with many query points. If the set of query points is of size O(N ), this algorithm returns a solution in provably O(N ) time, which is significantly better than the O(N 2 ) linear scan solution; these bounds are dependent on the expansion constant of the data. These algorithms work for abstract objects, as they do not require explicit representation of the points in kernel space. Empirical results for a variety of datasets show up to 5 orders of magnitude speedup in some cases. In addition, we present approximate extensions of the FastMKS algorithms that can achieve further speedups. Max-kernel searchOne particularly ubiquitous problem in computer science is that of max-kernel search: for a given set S r of N objects (the reference set), a similarity function K(·, ·), and a query object p q , find the object p r ∈ R such thatOften, max-kernel search is performed for a large set of query objects S q .The most simple approach to this general problem is a linear scan over all the objects in S r . However, the computational cost of this approach scales linearly with the size of the reference set for a single query, making it computationally prohibitive for large datasets. If |S q | = |S r | = O(N ), then this approach scales as O(N 2 ); thus, the approach quickly becomes infeasible for large N .In our setting we restrict the similarity function K(·, ·) to be a Mercer kernel. As we will see, this is not very restrictive. A Mercer kernel is a positive semidefinite kernel function; these can be expressed as an inner product in some Hilbert space H:

show abstract

A Branch and Bound Algorithm for Feature Subset Selection

Cited by 1,068 publications

References 6 publications

Image Categorization Using ESFS: A New Embedded Feature Selection Method Based on SFS

Image Categorization Using ESFS: A New Embedded Feature Selection Method Based on SFS

Automated labelling of cancer textures in colorectal histopathology slides using quasi-supervised learning

Dual‐tree fast exact max‐kernel search

Contact Info

Product

Resources

About