Nonlinear Deep Kernel Learning for Image Annotation

Jiu, Mingyuan; Sahbi, Hichem

doi:10.1109/tip.2017.2666038

Cited by 70 publications

(62 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the classification ability is quantified by the generalization error, we will attempt to develop a convergence bound of the generalization error of HSIC‐MKL based on the established theory of Rademacher complexities. Besides, the further validation of the use of the proposed algorithm on more real‐world applications, such as computer vision, speech and signal processing, and natural language processing, and expanding the proposed model to extreme learning machine and deep kernel learning are also important issues to be investigated.…”

Section: Resultsmentioning

confidence: 99%

Multiple kernel learning using nonlinear lasso

Wang

2018

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

Multiple kernel learning (MKL) is a principled way for kernel fusion for various learning tasks such as classification, clustering, and dimensionality reduction. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on the linear dependence between input features and output values. In this paper, we develop a novel MKL model based on a nonlinear Lasso, that is, the Hilbert-Schmidt independence criterion (HSIC) Lasso. In the proposed model, we first propose the HSIC Lasso-based MKL formulation, which has a clear statistical interpretation that minimum redundant kernels with maximum dependence on output labels are found and combined, and also that the global optimal solution can be computed efficiently by solving a Lasso optimization problem. After the optimal kernel is obtained, the support vector machine (SVM) is used to select the prediction hypothesis. It is evident that the proposed MKL is a two-stage kernel learning approach. Extensive experiments on real-world datasets from the UCI benchmark repository validate the superiority of the proposed model in terms of prediction accuracy.

show abstract

Section: Resultsmentioning

confidence: 99%

Multiple kernel learning using nonlinear lasso

Wang

2018

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

show abstract

“…Semantic-based image retrieval has become an active research topic in recent times. There were many techniques of image retrieval, which have been implemented aiming to reduce the "semantic gap" by modeling high-level semantics, such as techniques to build a model for mapping between low-level features and high-level semantics [2,21], query techniques based on ontology to accurately describe semantics for images [18,25], techniques for classification data [12,13,17], etc.…”

Section: Related Workmentioning

confidence: 99%

A Self-Balanced Clustering Tree for Semantic-Based Image Retrieval

Nhi

Van

Thạnh

2020

JCC

View full text Add to dashboard Cite

The image retrieval and semantic extraction play an important role in the multimedia systems such as geographic information system, hospital information system, digital library system, etc. Therefore, the research and development of semantic-based image retrieval (SBIR) systems have become extremely important and urgent. Major recent publications are included covering different aspects of the research in this area, including building data models, low-level image feature extraction, and deriving high-level semantic features. However, there is still no general approach for semantic-based image retrieval (SBIR), due to the diversity and complexity of high-level semantics. In order to improve the retrieval accuracy of SBIR systems, our focus research is to build a data structure for finding similar images, from that retrieving its semantic. In this paper, we proposed a data structure which is a self-balanced clustering tree named C-Tree. Firstly, a method of visual semantic analysis relied on visual features and image content is proposed on C-Tree. The building of this structure is created based on a combination of methods including hierarchical clustering and partitional clustering. Secondly, we design ontology for the image dataset and create the SPARQL (SPARQL Protocol and RDF Query Language) query by extracting semantics of image. Finally, the semantic-based image retrieval on C-Tree (SBIR\_CT) model is created hinging on our proposal. The experimental evaluation 20,000 images of ImageCLEF dataset indicates the effectiveness of the proposed method. These results are compared with some of recently published methods on the same dataset and demonstrate that the proposed method improves the retrieval accuracy and efficiency.

show abstract

“…In our experiments (see Table 6), we use four elementary kernels (linear, polynomial, RBF and HI) combined with different features as inputs to the designed DKN and DMN networks: "handcrafted features" including GIST and SIFT and "learned features" taken from ResNet [50] (pretrained on the ImageNet) which is a very deep architecture consisting of 152 layers; the 2048 dimensional features of the last pooling layer are used in our annotation task. Using all these elementary kernels and features, we first train a DKN in a supervised way according to [49], then we design and finetune its associated DMNs with |S| = 700 and |S | = 3000 (as done in Table. 4).…”

Section: Initial and Fine-tuned Dmns Assuming The Weights {Wmentioning

confidence: 99%

Deep representation design from deep kernel networks

Jiu

Sahbi

2019

Pattern Recognition

Self Cite

View full text Add to dashboard Cite

Deep kernel learning aims at designing nonlinear combinations of multiple standard elementary kernels by training deep networks. This scheme has proven to be effective, but intractable when handling large-scale datasets especially when the depth of the trained networks increases; indeed, the complexity of evaluating these networks scales quadratically w.r.t. the size of training data and linearly w.r.t. the depth of the trained networks.In this paper, we address the issue of efficient computation in Deep Kernel Networks (DKNs) by designing effective maps in the underlying Reproducing Kernel Hilbert Spaces. Given a pretrained DKN, our method builds its associated Deep Map Network (DMN) whose inner product approximates the original network while being far more efficient. The design principle of our method is greedy and achieved layer-wise, by finding maps that approximate DKNs at different (input, intermediate and output) layers. This design also considers an extra fine-tuning step based on unsupervised learning, that further enhances the generalization ability of the trained DMNs. When plugged into SVMs, these DMNs turn out to be as accurate as the underlying DKNs while being at least an order of magnitude faster on large-scale datasets, as shown through extensive experiments on the challenging ImageCLEF and COREL5k benchmarks.

show abstract

Nonlinear Deep Kernel Learning for Image Annotation

Cited by 70 publications

References 39 publications

Multiple kernel learning using nonlinear lasso

Multiple kernel learning using nonlinear lasso

A Self-Balanced Clustering Tree for Semantic-Based Image Retrieval

Deep representation design from deep kernel networks

Contact Info

Product

Resources

About